2024-02-16T19:06:58.721 INFO:root:teuthology version: 0.0.1.dev260+gb1dac55 2024-02-16T19:06:58.722 DEBUG:teuthology.run:Teuthology command: teuthology --description fs:upgrade:mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{v18.2.0} 0a 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} --archive /home/teuthworker/archive/pdonnell-2024-02-16_18:26:20-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi/7562690 --owner scheduled_pdonnell@teuthology --name pdonnell-2024-02-16_18:26:20-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi --verbose -- /home/teuthworker/archive/pdonnell-2024-02-16_18:26:20-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi/7562690/orig.config.yaml 2024-02-16T19:06:58.776 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-16T19:07:25.785 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-02-16_18:26:20-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi/7562690 branch: wip-batrick-testing-20240215.160715 description: fs:upgrade:mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{v18.2.0} 0a 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} email: pdonnell@redhat.com first_in_suite: false job_id: '7562690' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi meta: - desc: 'setup ceph/v18.2.0 ' name: pdonnell-2024-02-16_18:26:20-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi no_nested_subset: false nuke-on-error: true os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-batrick-testing-20240215.160715 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: mon pg warn min per osd: 0 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - OSD_DOWN - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: f78a58c0ffd401d1493058a1022c35f011d65275 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: f78a58c0ffd401d1493058a1022c35f011d65275 kclient: syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: i64440 sha1: ed7ab725e1a43615009dfb74a42bf33d80081bc8 owner: scheduled_pdonnell@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - host.a - client.0 - osd.0 - osd.1 - osd.2 - - host.b - client.1 - osd.3 - osd.4 - osd.5 seed: 2634 sha1: f78a58c0ffd401d1493058a1022c35f011d65275 sleep_before_teardown: 0 suite: fs:upgrade:mds_upgrade_sequence suite_branch: i64440 suite_path: /home/teuthworker/src/github.com_batrick_ceph_ed7ab725e1a43615009dfb74a42bf33d80081bc8/qa suite_relpath: qa suite_repo: https://github.com/batrick/ceph.git suite_sha1: ed7ab725e1a43615009dfb74a42bf33d80081bc8 targets: smithi022.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMIZ2DUw6BxcUnqEaKE+IKXnccw9fWwvrdV4p+4inJbsRKm506Qrl49uIpTK0S5Pjer+EmQo/vhpKVtqBwUsjBg= smithi117.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEJxXBiOuUWPDc39YvRbfeVRlYJD1jWheXbOUbxnt1vZaWfcFBGNqi0zuGez3+y8nMaxYncImhSdtedLvRNbCNI= tasks: - install: exclude_packages: - ceph-volume tag: v18.2.0 - print: '**** done install task...' - cephadm: compiled_cephadm_branch: reef conf: osd: osd_class_default_list: '*' osd_class_load_list: '*' image: quay.io/ceph/ceph:v18.2.0 roleless: true - print: '**** done end installing v18.2.0 cephadm ...' - cephadm.shell: host.a: - ceph config set mgr mgr/cephadm/use_repo_digest true --force - print: '**** done cephadm.shell ceph config set mgr...' - cephadm.shell: host.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - full_sequential_finally: - duration: 3600 sleep: null - cephadm.shell: host.a: - ceph fs volume create cephfs --placement=4 - ceph fs dump - cephadm.shell: host.a: - ceph fs set cephfs max_mds 1 - cephadm.shell: host.a: - ceph fs set cephfs allow_standby_replay false - cephadm.shell: host.a: - ceph fs set cephfs inline_data true --yes-i-really-really-mean-it - cephadm.shell: host.a: - ceph fs dump - ceph --format=json fs dump | jq -e ".filesystems | length == 1" - while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done - fs.pre_upgrade_save: null - ceph-fuse: null - print: '**** done client' - parallel: - upgrade-tasks - workload-tasks - cephadm.shell: host.a: - ceph fs dump - fs.post_upgrade_checks: null teuthology: fragments_dropped: [] meta: {} postmerge: [] variables: fail_fs: true teuthology_branch: main teuthology_sha1: b1dac5519c57c269ea98a22fb7729016a1d0e4d2 timestamp: 2024-02-16_18:26:20 tube: smithi upgrade-tasks: sequential: - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mgr - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done - ceph versions | jq -e '.mgr | length == 1' - ceph versions | jq -e '.mgr | keys' | grep $sha1 - ceph versions | jq -e '.overall | length == 2' - ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '.up_to_date | length == 2' - ceph orch ps - cephadm.shell: env: - sha1 host.a: - ceph config set mgr mgr/orchestrator/fail_fs true - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 - cephadm.shell: env: - sha1 host.a: - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done - ceph orch ps - ceph orch upgrade status - ceph health detail - ceph versions - echo "wait for servicemap items w/ changing names to refresh" - sleep 60 - ceph orch ps - ceph versions - ceph versions | jq -e '.overall | length == 1' - ceph versions | jq -e '.overall | keys' | grep $sha1 user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 workload-tasks: sequential: - workunit: clients: all: - suites/fsstress.sh 2024-02-16T19:07:25.786 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_batrick_ceph_ed7ab725e1a43615009dfb74a42bf33d80081bc8/qa; will attempt to use it 2024-02-16T19:07:25.787 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_batrick_ceph_ed7ab725e1a43615009dfb74a42bf33d80081bc8/qa/tasks 2024-02-16T19:07:25.787 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-02-16T19:07:25.788 INFO:teuthology.task.internal:Checking packages... 2024-02-16T19:07:25.813 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'f78a58c0ffd401d1493058a1022c35f011d65275' 2024-02-16T19:07:25.813 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-02-16T19:07:25.813 INFO:teuthology.packaging:ref: None 2024-02-16T19:07:25.813 INFO:teuthology.packaging:tag: None 2024-02-16T19:07:25.813 INFO:teuthology.packaging:branch: wip-batrick-testing-20240215.160715 2024-02-16T19:07:25.813 INFO:teuthology.packaging:sha1: f78a58c0ffd401d1493058a1022c35f011d65275 2024-02-16T19:07:25.813 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-batrick-testing-20240215.160715 2024-02-16T19:07:25.957 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1450.gf78a58c0 2024-02-16T19:07:25.958 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-02-16T19:07:25.966 INFO:teuthology.task.internal:no buildpackages task found 2024-02-16T19:07:25.966 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-02-16T19:07:25.988 INFO:teuthology.task.internal:Saving configuration 2024-02-16T19:07:26.008 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-02-16T19:07:26.014 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-16T19:07:26.042 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi022.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-02-16_18:26:20-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi/7562690', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-16 19:01:56.506535', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMIZ2DUw6BxcUnqEaKE+IKXnccw9fWwvrdV4p+4inJbsRKm506Qrl49uIpTK0S5Pjer+EmQo/vhpKVtqBwUsjBg='} 2024-02-16T19:07:26.064 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi117.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-02-16_18:26:20-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi/7562690', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-16 19:01:56.508386', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEJxXBiOuUWPDc39YvRbfeVRlYJD1jWheXbOUbxnt1vZaWfcFBGNqi0zuGez3+y8nMaxYncImhSdtedLvRNbCNI='} 2024-02-16T19:07:26.065 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-02-16T19:07:26.070 INFO:teuthology.task.internal:roles: ubuntu@smithi022.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2'] 2024-02-16T19:07:26.071 INFO:teuthology.task.internal:roles: ubuntu@smithi117.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5'] 2024-02-16T19:07:26.071 INFO:teuthology.run_tasks:Running task console_log... 2024-02-16T19:07:26.142 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f1700b240d0>, signals=[15]) 2024-02-16T19:07:26.143 INFO:teuthology.run_tasks:Running task internal.connect... 2024-02-16T19:07:26.148 INFO:teuthology.task.internal:Opening connections... 2024-02-16T19:07:26.148 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi022.front.sepia.ceph.com 2024-02-16T19:07:26.151 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T19:07:26.261 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi117.front.sepia.ceph.com 2024-02-16T19:07:26.262 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T19:07:26.355 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-02-16T19:07:26.362 DEBUG:teuthology.orchestra.run.smithi022:> uname -m 2024-02-16T19:07:26.398 INFO:teuthology.orchestra.run.smithi022.stdout:x86_64 2024-02-16T19:07:26.398 DEBUG:teuthology.orchestra.run.smithi022:> cat /etc/os-release 2024-02-16T19:07:26.453 INFO:teuthology.orchestra.run.smithi022.stdout:NAME="CentOS Stream" 2024-02-16T19:07:26.453 INFO:teuthology.orchestra.run.smithi022.stdout:VERSION="9" 2024-02-16T19:07:26.453 INFO:teuthology.orchestra.run.smithi022.stdout:ID="centos" 2024-02-16T19:07:26.453 INFO:teuthology.orchestra.run.smithi022.stdout:ID_LIKE="rhel fedora" 2024-02-16T19:07:26.453 INFO:teuthology.orchestra.run.smithi022.stdout:VERSION_ID="9" 2024-02-16T19:07:26.453 INFO:teuthology.orchestra.run.smithi022.stdout:PLATFORM_ID="platform:el9" 2024-02-16T19:07:26.453 INFO:teuthology.orchestra.run.smithi022.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-16T19:07:26.453 INFO:teuthology.orchestra.run.smithi022.stdout:ANSI_COLOR="0;31" 2024-02-16T19:07:26.453 INFO:teuthology.orchestra.run.smithi022.stdout:LOGO="fedora-logo-icon" 2024-02-16T19:07:26.453 INFO:teuthology.orchestra.run.smithi022.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-16T19:07:26.454 INFO:teuthology.orchestra.run.smithi022.stdout:HOME_URL="https://centos.org/" 2024-02-16T19:07:26.454 INFO:teuthology.orchestra.run.smithi022.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-16T19:07:26.454 INFO:teuthology.orchestra.run.smithi022.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-16T19:07:26.454 INFO:teuthology.orchestra.run.smithi022.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-16T19:07:26.454 INFO:teuthology.lock.ops:Updating smithi022.front.sepia.ceph.com on lock server 2024-02-16T19:07:26.484 DEBUG:teuthology.orchestra.run.smithi117:> uname -m 2024-02-16T19:07:26.498 INFO:teuthology.orchestra.run.smithi117.stdout:x86_64 2024-02-16T19:07:26.499 DEBUG:teuthology.orchestra.run.smithi117:> cat /etc/os-release 2024-02-16T19:07:26.554 INFO:teuthology.orchestra.run.smithi117.stdout:NAME="CentOS Stream" 2024-02-16T19:07:26.554 INFO:teuthology.orchestra.run.smithi117.stdout:VERSION="9" 2024-02-16T19:07:26.554 INFO:teuthology.orchestra.run.smithi117.stdout:ID="centos" 2024-02-16T19:07:26.554 INFO:teuthology.orchestra.run.smithi117.stdout:ID_LIKE="rhel fedora" 2024-02-16T19:07:26.554 INFO:teuthology.orchestra.run.smithi117.stdout:VERSION_ID="9" 2024-02-16T19:07:26.554 INFO:teuthology.orchestra.run.smithi117.stdout:PLATFORM_ID="platform:el9" 2024-02-16T19:07:26.554 INFO:teuthology.orchestra.run.smithi117.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-16T19:07:26.554 INFO:teuthology.orchestra.run.smithi117.stdout:ANSI_COLOR="0;31" 2024-02-16T19:07:26.554 INFO:teuthology.orchestra.run.smithi117.stdout:LOGO="fedora-logo-icon" 2024-02-16T19:07:26.554 INFO:teuthology.orchestra.run.smithi117.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-16T19:07:26.555 INFO:teuthology.orchestra.run.smithi117.stdout:HOME_URL="https://centos.org/" 2024-02-16T19:07:26.555 INFO:teuthology.orchestra.run.smithi117.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-16T19:07:26.555 INFO:teuthology.orchestra.run.smithi117.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-16T19:07:26.555 INFO:teuthology.orchestra.run.smithi117.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-16T19:07:26.555 INFO:teuthology.lock.ops:Updating smithi117.front.sepia.ceph.com on lock server 2024-02-16T19:07:26.584 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-02-16T19:07:26.593 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-02-16T19:07:26.598 INFO:teuthology.task.internal:Checking for old test directory... 2024-02-16T19:07:26.599 DEBUG:teuthology.orchestra.run.smithi022:> test '!' -e /home/ubuntu/cephtest 2024-02-16T19:07:26.601 DEBUG:teuthology.orchestra.run.smithi117:> test '!' -e /home/ubuntu/cephtest 2024-02-16T19:07:26.615 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-02-16T19:07:26.621 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-02-16T19:07:26.621 DEBUG:teuthology.orchestra.run.smithi022:> test -z $(ls -A /var/lib/ceph) 2024-02-16T19:07:26.656 DEBUG:teuthology.orchestra.run.smithi117:> test -z $(ls -A /var/lib/ceph) 2024-02-16T19:07:26.736 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-02-16T19:07:26.791 INFO:teuthology.run_tasks:Running task kernel... 2024-02-16T19:07:26.806 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-02-16T19:07:26.806 INFO:teuthology.task.kernel:config {'host.a': {'kdb': True, 'sha1': 'distro'}, 'host.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-02-16T19:07:26.806 DEBUG:teuthology.orchestra.run.smithi022:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-16T19:07:26.807 DEBUG:teuthology.orchestra.run.smithi117:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-16T19:07:26.821 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-16T19:07:26.821 DEBUG:teuthology.orchestra.run.smithi022:> uname -r 2024-02-16T19:07:26.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-16T19:07:26.822 DEBUG:teuthology.orchestra.run.smithi117:> uname -r 2024-02-16T19:07:26.877 INFO:teuthology.orchestra.run.smithi022.stdout:5.14.0-361.el9.x86_64 2024-02-16T19:07:26.877 INFO:teuthology.task.kernel:Running kernel on smithi022: 5.14.0-361.el9.x86_64 2024-02-16T19:07:26.877 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum install -y kernel 2024-02-16T19:07:26.878 INFO:teuthology.orchestra.run.smithi117.stdout:5.14.0-361.el9.x86_64 2024-02-16T19:07:26.879 INFO:teuthology.task.kernel:Running kernel on smithi117: 5.14.0-361.el9.x86_64 2024-02-16T19:07:26.879 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum install -y kernel 2024-02-16T19:07:28.909 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS-9 - AppStream 16 MB/s | 6.7 MB 00:00 2024-02-16T19:07:28.944 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS-9 - AppStream 15 MB/s | 6.7 MB 00:00 2024-02-16T19:07:30.370 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS-9 - BaseOS 7.3 MB/s | 2.0 MB 00:00 2024-02-16T19:07:30.543 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS-9 - BaseOS 3.9 MB/s | 2.0 MB 00:00 2024-02-16T19:07:31.868 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 9 - BaseOS 6.9 MB/s | 8.0 MB 00:01 2024-02-16T19:07:32.108 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - BaseOS 6.5 MB/s | 8.0 MB 00:01 2024-02-16T19:07:34.384 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - AppStream 16 MB/s | 19 MB 00:01 2024-02-16T19:07:36.219 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 9 - AppStream 5.9 MB/s | 19 MB 00:03 2024-02-16T19:07:38.519 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - CRB 9.5 MB/s | 6.1 MB 00:00 2024-02-16T19:07:39.927 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - Extras packages 53 kB/s | 15 kB 00:00 2024-02-16T19:07:40.779 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 9 - CRB 6.7 MB/s | 6.1 MB 00:00 2024-02-16T19:07:42.262 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 9 - Extras packages 47 kB/s | 15 kB 00:00 2024-02-16T19:07:43.294 INFO:teuthology.orchestra.run.smithi117.stdout:Extra Packages for Enterprise Linux 6.3 MB/s | 20 MB 00:03 2024-02-16T19:07:43.633 INFO:teuthology.orchestra.run.smithi022.stdout:Extra Packages for Enterprise Linux 16 MB/s | 20 MB 00:01 2024-02-16T19:07:48.168 INFO:teuthology.orchestra.run.smithi117.stdout:lab-extras 16 kB/s | 1.7 kB 00:00 2024-02-16T19:07:48.714 INFO:teuthology.orchestra.run.smithi022.stdout:lab-extras 23 kB/s | 1.7 kB 00:00 2024-02-16T19:07:50.475 INFO:teuthology.orchestra.run.smithi117.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-02-16T19:07:50.475 INFO:teuthology.orchestra.run.smithi117.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-02-16T19:07:50.570 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-02-16T19:07:50.581 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-02-16T19:07:50.581 INFO:teuthology.orchestra.run.smithi117.stdout: Package Arch Version Repository Size 2024-02-16T19:07:50.581 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-02-16T19:07:50.581 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2024-02-16T19:07:50.581 INFO:teuthology.orchestra.run.smithi117.stdout: kernel x86_64 5.14.0-421.el9 CentOS-BaseOS 6.1 M 2024-02-16T19:07:50.581 INFO:teuthology.orchestra.run.smithi117.stdout:Installing dependencies: 2024-02-16T19:07:50.581 INFO:teuthology.orchestra.run.smithi117.stdout: kernel-core x86_64 5.14.0-421.el9 CentOS-BaseOS 21 M 2024-02-16T19:07:50.581 INFO:teuthology.orchestra.run.smithi117.stdout: kernel-modules x86_64 5.14.0-421.el9 CentOS-BaseOS 39 M 2024-02-16T19:07:50.582 INFO:teuthology.orchestra.run.smithi117.stdout: kernel-modules-core x86_64 5.14.0-421.el9 CentOS-BaseOS 33 M 2024-02-16T19:07:50.582 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:07:50.582 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-02-16T19:07:50.582 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-02-16T19:07:50.582 INFO:teuthology.orchestra.run.smithi117.stdout:Install 4 Packages 2024-02-16T19:07:50.582 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:07:50.582 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 100 M 2024-02-16T19:07:50.582 INFO:teuthology.orchestra.run.smithi117.stdout:Installed size: 123 M 2024-02-16T19:07:50.583 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2024-02-16T19:07:50.928 INFO:teuthology.orchestra.run.smithi117.stdout:(1/4): kernel-5.14.0-421.el9.x86_64.rpm 18 MB/s | 6.1 MB 00:00 2024-02-16T19:07:51.112 INFO:teuthology.orchestra.run.smithi022.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-02-16T19:07:51.112 INFO:teuthology.orchestra.run.smithi022.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-02-16T19:07:51.214 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-02-16T19:07:51.225 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-02-16T19:07:51.225 INFO:teuthology.orchestra.run.smithi022.stdout: Package Arch Version Repository Size 2024-02-16T19:07:51.225 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-02-16T19:07:51.225 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2024-02-16T19:07:51.226 INFO:teuthology.orchestra.run.smithi022.stdout: kernel x86_64 5.14.0-421.el9 CentOS-BaseOS 6.1 M 2024-02-16T19:07:51.226 INFO:teuthology.orchestra.run.smithi022.stdout:Installing dependencies: 2024-02-16T19:07:51.226 INFO:teuthology.orchestra.run.smithi022.stdout: kernel-core x86_64 5.14.0-421.el9 CentOS-BaseOS 21 M 2024-02-16T19:07:51.226 INFO:teuthology.orchestra.run.smithi022.stdout: kernel-modules x86_64 5.14.0-421.el9 CentOS-BaseOS 39 M 2024-02-16T19:07:51.226 INFO:teuthology.orchestra.run.smithi022.stdout: kernel-modules-core x86_64 5.14.0-421.el9 CentOS-BaseOS 33 M 2024-02-16T19:07:51.226 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:07:51.226 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2024-02-16T19:07:51.226 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-02-16T19:07:51.226 INFO:teuthology.orchestra.run.smithi022.stdout:Install 4 Packages 2024-02-16T19:07:51.226 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:07:51.227 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 100 M 2024-02-16T19:07:51.227 INFO:teuthology.orchestra.run.smithi022.stdout:Installed size: 123 M 2024-02-16T19:07:51.227 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading Packages: 2024-02-16T19:07:51.437 INFO:teuthology.orchestra.run.smithi117.stdout:(2/4): kernel-core-5.14.0-421.el9.x86_64.rpm 24 MB/s | 21 MB 00:00 2024-02-16T19:07:51.732 INFO:teuthology.orchestra.run.smithi022.stdout:(1/4): kernel-5.14.0-421.el9.x86_64.rpm 12 MB/s | 6.1 MB 00:00 2024-02-16T19:07:52.515 INFO:teuthology.orchestra.run.smithi022.stdout:(2/4): kernel-core-5.14.0-421.el9.x86_64.rpm 16 MB/s | 21 MB 00:01 2024-02-16T19:07:52.595 INFO:teuthology.orchestra.run.smithi117.stdout:(3/4): kernel-modules-core-5.14.0-421.el9.x86_6 20 MB/s | 33 MB 00:01 2024-02-16T19:07:53.595 INFO:teuthology.orchestra.run.smithi117.stdout:(4/4): kernel-modules-5.14.0-421.el9.x86_64.rpm 13 MB/s | 39 MB 00:03 2024-02-16T19:07:53.598 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-02-16T19:07:53.598 INFO:teuthology.orchestra.run.smithi117.stdout:Total 33 MB/s | 100 MB 00:03 2024-02-16T19:07:53.599 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-02-16T19:07:53.674 INFO:teuthology.orchestra.run.smithi022.stdout:(3/4): kernel-modules-core-5.14.0-421.el9.x86_6 17 MB/s | 33 MB 00:01 2024-02-16T19:07:53.985 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-02-16T19:07:53.985 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-02-16T19:07:54.232 INFO:teuthology.orchestra.run.smithi022.stdout:(4/4): kernel-modules-5.14.0-421.el9.x86_64.rpm 13 MB/s | 39 MB 00:03 2024-02-16T19:07:54.235 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2024-02-16T19:07:54.235 INFO:teuthology.orchestra.run.smithi022.stdout:Total 33 MB/s | 100 MB 00:03 2024-02-16T19:07:54.236 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2024-02-16T19:07:54.671 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction check succeeded. 2024-02-16T19:07:54.671 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2024-02-16T19:07:55.017 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-02-16T19:07:55.018 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-02-16T19:07:55.726 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded. 2024-02-16T19:07:55.727 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2024-02-16T19:07:57.424 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-02-16T19:07:58.085 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : kernel-modules-core-5.14.0-421.el9.x86_64 1/4 2024-02-16T19:07:58.203 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : kernel-core-5.14.0-421.el9.x86_64 2/4 2024-02-16T19:07:58.282 INFO:teuthology.orchestra.run.smithi022.stdout: Preparing : 1/1 2024-02-16T19:07:58.966 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : kernel-modules-core-5.14.0-421.el9.x86_64 1/4 2024-02-16T19:07:59.100 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : kernel-core-5.14.0-421.el9.x86_64 2/4 2024-02-16T19:08:00.964 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: kernel-core-5.14.0-421.el9.x86_64 2/4 2024-02-16T19:08:01.140 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : kernel-modules-5.14.0-421.el9.x86_64 3/4 2024-02-16T19:08:02.012 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: kernel-core-5.14.0-421.el9.x86_64 2/4 2024-02-16T19:08:02.185 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : kernel-modules-5.14.0-421.el9.x86_64 3/4 2024-02-16T19:08:05.836 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: kernel-modules-5.14.0-421.el9.x86_64 3/4 2024-02-16T19:08:05.972 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : kernel-5.14.0-421.el9.x86_64 4/4 2024-02-16T19:08:07.288 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: kernel-modules-5.14.0-421.el9.x86_64 3/4 2024-02-16T19:08:07.392 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : kernel-5.14.0-421.el9.x86_64 4/4 2024-02-16T19:08:10.729 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: kernel-modules-core-5.14.0-421.el9.x86_64 4/4 2024-02-16T19:08:12.522 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: kernel-modules-core-5.14.0-421.el9.x86_64 4/4 2024-02-16T19:08:39.824 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: kernel-core-5.14.0-421.el9.x86_64 4/4 2024-02-16T19:08:39.824 INFO:teuthology.orchestra.run.smithi117.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-02-16T19:08:39.824 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:08:39.893 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: kernel-modules-5.14.0-421.el9.x86_64 4/4 2024-02-16T19:08:40.284 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: kernel-5.14.0-421.el9.x86_64 4/4 2024-02-16T19:08:40.284 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : kernel-5.14.0-421.el9.x86_64 1/4 2024-02-16T19:08:40.284 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : kernel-core-5.14.0-421.el9.x86_64 2/4 2024-02-16T19:08:40.284 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : kernel-modules-5.14.0-421.el9.x86_64 3/4 2024-02-16T19:08:40.710 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : kernel-modules-core-5.14.0-421.el9.x86_64 4/4 2024-02-16T19:08:40.710 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:08:40.710 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2024-02-16T19:08:40.711 INFO:teuthology.orchestra.run.smithi117.stdout: kernel-5.14.0-421.el9.x86_64 2024-02-16T19:08:40.711 INFO:teuthology.orchestra.run.smithi117.stdout: kernel-core-5.14.0-421.el9.x86_64 2024-02-16T19:08:40.711 INFO:teuthology.orchestra.run.smithi117.stdout: kernel-modules-5.14.0-421.el9.x86_64 2024-02-16T19:08:40.711 INFO:teuthology.orchestra.run.smithi117.stdout: kernel-modules-core-5.14.0-421.el9.x86_64 2024-02-16T19:08:40.711 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:08:40.711 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-02-16T19:08:41.225 DEBUG:teuthology.orchestra.run.smithi117:> rpm -q kernel | sort -rV | head -n 1 2024-02-16T19:08:41.355 INFO:teuthology.orchestra.run.smithi117.stdout:kernel-5.14.0-421.el9.x86_64 2024-02-16T19:08:41.356 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-421.el9.x86_64 2024-02-16T19:08:41.356 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-421.el9.x86_64 2024-02-16T19:08:41.356 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-16T19:08:41.356 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-16T19:08:41.356 INFO:teuthology.task.kernel:Installing distro kernel on host.b... 2024-02-16T19:08:41.356 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi117.front.sepia.ceph.com, path=None, version=distro) 2024-02-16T19:08:41.356 DEBUG:teuthology.orchestra.run.smithi117:> rpm -q kernel | sort -rV | head -n 1 2024-02-16T19:08:41.463 INFO:teuthology.orchestra.run.smithi117.stdout:kernel-5.14.0-421.el9.x86_64 2024-02-16T19:08:41.464 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-421.el9.x86_64 2024-02-16T19:08:41.464 DEBUG:teuthology.orchestra.run.smithi117:> sudo rpm -qi grub2-tools 2024-02-16T19:08:41.498 INFO:teuthology.orchestra.run.smithi117.stdout:Name : grub2-tools 2024-02-16T19:08:41.498 INFO:teuthology.orchestra.run.smithi117.stdout:Epoch : 1 2024-02-16T19:08:41.498 INFO:teuthology.orchestra.run.smithi117.stdout:Version : 2.06 2024-02-16T19:08:41.498 INFO:teuthology.orchestra.run.smithi117.stdout:Release : 68.el9 2024-02-16T19:08:41.498 INFO:teuthology.orchestra.run.smithi117.stdout:Architecture: x86_64 2024-02-16T19:08:41.498 INFO:teuthology.orchestra.run.smithi117.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-02-16T19:08:41.498 INFO:teuthology.orchestra.run.smithi117.stdout:Group : Unspecified 2024-02-16T19:08:41.498 INFO:teuthology.orchestra.run.smithi117.stdout:Size : 8264254 2024-02-16T19:08:41.498 INFO:teuthology.orchestra.run.smithi117.stdout:License : GPLv3+ 2024-02-16T19:08:41.498 INFO:teuthology.orchestra.run.smithi117.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-02-16T19:08:41.499 INFO:teuthology.orchestra.run.smithi117.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-02-16T19:08:41.499 INFO:teuthology.orchestra.run.smithi117.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-02-16T19:08:41.499 INFO:teuthology.orchestra.run.smithi117.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-02-16T19:08:41.499 INFO:teuthology.orchestra.run.smithi117.stdout:Packager : builder@centos.org 2024-02-16T19:08:41.499 INFO:teuthology.orchestra.run.smithi117.stdout:Vendor : CentOS 2024-02-16T19:08:41.499 INFO:teuthology.orchestra.run.smithi117.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-16T19:08:41.499 INFO:teuthology.orchestra.run.smithi117.stdout:Summary : Support tools for GRUB. 2024-02-16T19:08:41.499 INFO:teuthology.orchestra.run.smithi117.stdout:Description : 2024-02-16T19:08:41.499 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:08:41.499 INFO:teuthology.orchestra.run.smithi117.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-16T19:08:41.499 INFO:teuthology.orchestra.run.smithi117.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-16T19:08:41.499 INFO:teuthology.orchestra.run.smithi117.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-16T19:08:41.499 INFO:teuthology.orchestra.run.smithi117.stdout:hardware devices. 2024-02-16T19:08:41.499 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:08:41.499 INFO:teuthology.orchestra.run.smithi117.stdout:This subpackage provides tools for support of all platforms. 2024-02-16T19:08:41.500 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-16T19:08:41.500 INFO:teuthology.task.kernel:Updating grub on smithi117 to boot 5.14.0-421.el9.x86_64 2024-02-16T19:08:41.500 DEBUG:teuthology.orchestra.run.smithi117:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-16T19:08:42.082 INFO:teuthology.orchestra.run.smithi117.stderr:Generating grub configuration file ... 2024-02-16T19:08:43.272 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: kernel-core-5.14.0-421.el9.x86_64 4/4 2024-02-16T19:08:43.272 INFO:teuthology.orchestra.run.smithi022.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-02-16T19:08:43.272 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:08:43.344 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: kernel-modules-5.14.0-421.el9.x86_64 4/4 2024-02-16T19:08:43.473 INFO:teuthology.orchestra.run.smithi117.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-02-16T19:08:43.514 INFO:teuthology.orchestra.run.smithi117.stderr:done 2024-02-16T19:08:43.516 DEBUG:teuthology.orchestra.run.smithi117:> mktemp 2024-02-16T19:08:43.531 INFO:teuthology.orchestra.run.smithi117.stdout:/tmp/tmp.wCKvn0TiDZ 2024-02-16T19:08:43.531 DEBUG:teuthology.orchestra.run.smithi117:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.wCKvn0TiDZ 2024-02-16T19:08:43.608 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 0666 /tmp/tmp.wCKvn0TiDZ 2024-02-16T19:08:43.755 DEBUG:teuthology.orchestra.remote:smithi117:/tmp/tmp.wCKvn0TiDZ is 6KB 2024-02-16T19:08:43.798 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: kernel-5.14.0-421.el9.x86_64 4/4 2024-02-16T19:08:43.798 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : kernel-5.14.0-421.el9.x86_64 1/4 2024-02-16T19:08:43.798 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : kernel-core-5.14.0-421.el9.x86_64 2/4 2024-02-16T19:08:43.798 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : kernel-modules-5.14.0-421.el9.x86_64 3/4 2024-02-16T19:08:43.804 DEBUG:teuthology.orchestra.run.smithi117:> rm -fr /tmp/tmp.wCKvn0TiDZ 2024-02-16T19:08:43.818 DEBUG:teuthology.orchestra.run.smithi117:> sudo /bin/ls /boot/loader/entries || true 2024-02-16T19:08:43.883 INFO:teuthology.orchestra.run.smithi117.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-02-16T19:08:43.883 INFO:teuthology.orchestra.run.smithi117.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-02-16T19:08:43.883 INFO:teuthology.orchestra.run.smithi117.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-02-16T19:08:43.883 INFO:teuthology.orchestra.run.smithi117.stdout:15949e3c88704300882482eddfd7baae-5.14.0-421.el9.x86_64.conf 2024-02-16T19:08:43.884 DEBUG:teuthology.orchestra.run.smithi117:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-421.el9.x86_64 2024-02-16T19:08:44.025 DEBUG:teuthology.orchestra.run.smithi117:> sudo shutdown -r now 2024-02-16T19:08:44.266 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : kernel-modules-core-5.14.0-421.el9.x86_64 4/4 2024-02-16T19:08:44.266 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:08:44.266 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2024-02-16T19:08:44.266 INFO:teuthology.orchestra.run.smithi022.stdout: kernel-5.14.0-421.el9.x86_64 2024-02-16T19:08:44.266 INFO:teuthology.orchestra.run.smithi022.stdout: kernel-core-5.14.0-421.el9.x86_64 2024-02-16T19:08:44.266 INFO:teuthology.orchestra.run.smithi022.stdout: kernel-modules-5.14.0-421.el9.x86_64 2024-02-16T19:08:44.266 INFO:teuthology.orchestra.run.smithi022.stdout: kernel-modules-core-5.14.0-421.el9.x86_64 2024-02-16T19:08:44.266 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:08:44.266 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-02-16T19:08:44.771 DEBUG:teuthology.orchestra.run.smithi022:> rpm -q kernel | sort -rV | head -n 1 2024-02-16T19:08:44.948 INFO:teuthology.orchestra.run.smithi022.stdout:kernel-5.14.0-421.el9.x86_64 2024-02-16T19:08:44.949 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-421.el9.x86_64 2024-02-16T19:08:44.949 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-421.el9.x86_64 2024-02-16T19:08:44.949 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-16T19:08:44.949 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-16T19:08:44.949 INFO:teuthology.task.kernel:Installing distro kernel on host.a... 2024-02-16T19:08:44.949 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi022.front.sepia.ceph.com, path=None, version=distro) 2024-02-16T19:08:44.949 DEBUG:teuthology.orchestra.run.smithi022:> rpm -q kernel | sort -rV | head -n 1 2024-02-16T19:08:45.062 INFO:teuthology.orchestra.run.smithi022.stdout:kernel-5.14.0-421.el9.x86_64 2024-02-16T19:08:45.063 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-421.el9.x86_64 2024-02-16T19:08:45.063 DEBUG:teuthology.orchestra.run.smithi022:> sudo rpm -qi grub2-tools 2024-02-16T19:08:45.111 INFO:teuthology.orchestra.run.smithi022.stdout:Name : grub2-tools 2024-02-16T19:08:45.111 INFO:teuthology.orchestra.run.smithi022.stdout:Epoch : 1 2024-02-16T19:08:45.111 INFO:teuthology.orchestra.run.smithi022.stdout:Version : 2.06 2024-02-16T19:08:45.111 INFO:teuthology.orchestra.run.smithi022.stdout:Release : 68.el9 2024-02-16T19:08:45.111 INFO:teuthology.orchestra.run.smithi022.stdout:Architecture: x86_64 2024-02-16T19:08:45.111 INFO:teuthology.orchestra.run.smithi022.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-02-16T19:08:45.111 INFO:teuthology.orchestra.run.smithi022.stdout:Group : Unspecified 2024-02-16T19:08:45.111 INFO:teuthology.orchestra.run.smithi022.stdout:Size : 8264254 2024-02-16T19:08:45.111 INFO:teuthology.orchestra.run.smithi022.stdout:License : GPLv3+ 2024-02-16T19:08:45.111 INFO:teuthology.orchestra.run.smithi022.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-02-16T19:08:45.111 INFO:teuthology.orchestra.run.smithi022.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-02-16T19:08:45.111 INFO:teuthology.orchestra.run.smithi022.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-02-16T19:08:45.112 INFO:teuthology.orchestra.run.smithi022.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-02-16T19:08:45.112 INFO:teuthology.orchestra.run.smithi022.stdout:Packager : builder@centos.org 2024-02-16T19:08:45.112 INFO:teuthology.orchestra.run.smithi022.stdout:Vendor : CentOS 2024-02-16T19:08:45.112 INFO:teuthology.orchestra.run.smithi022.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-16T19:08:45.112 INFO:teuthology.orchestra.run.smithi022.stdout:Summary : Support tools for GRUB. 2024-02-16T19:08:45.112 INFO:teuthology.orchestra.run.smithi022.stdout:Description : 2024-02-16T19:08:45.112 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:08:45.112 INFO:teuthology.orchestra.run.smithi022.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-16T19:08:45.112 INFO:teuthology.orchestra.run.smithi022.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-16T19:08:45.112 INFO:teuthology.orchestra.run.smithi022.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-16T19:08:45.112 INFO:teuthology.orchestra.run.smithi022.stdout:hardware devices. 2024-02-16T19:08:45.112 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:08:45.112 INFO:teuthology.orchestra.run.smithi022.stdout:This subpackage provides tools for support of all platforms. 2024-02-16T19:08:45.113 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-16T19:08:45.113 INFO:teuthology.task.kernel:Updating grub on smithi022 to boot 5.14.0-421.el9.x86_64 2024-02-16T19:08:45.114 DEBUG:teuthology.orchestra.run.smithi022:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-16T19:08:45.731 INFO:teuthology.orchestra.run.smithi022.stderr:Generating grub configuration file ... 2024-02-16T19:08:47.154 INFO:teuthology.orchestra.run.smithi022.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-02-16T19:08:47.212 INFO:teuthology.orchestra.run.smithi022.stderr:done 2024-02-16T19:08:47.214 DEBUG:teuthology.orchestra.run.smithi022:> mktemp 2024-02-16T19:08:47.271 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/tmp.yLGtkaiUrx 2024-02-16T19:08:47.271 DEBUG:teuthology.orchestra.run.smithi022:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.yLGtkaiUrx 2024-02-16T19:08:47.345 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 0666 /tmp/tmp.yLGtkaiUrx 2024-02-16T19:08:47.496 DEBUG:teuthology.orchestra.remote:smithi022:/tmp/tmp.yLGtkaiUrx is 6KB 2024-02-16T19:08:47.544 DEBUG:teuthology.orchestra.run.smithi022:> rm -fr /tmp/tmp.yLGtkaiUrx 2024-02-16T19:08:47.560 DEBUG:teuthology.orchestra.run.smithi022:> sudo /bin/ls /boot/loader/entries || true 2024-02-16T19:08:47.628 INFO:teuthology.orchestra.run.smithi022.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-02-16T19:08:47.628 INFO:teuthology.orchestra.run.smithi022.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-02-16T19:08:47.628 INFO:teuthology.orchestra.run.smithi022.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-02-16T19:08:47.628 INFO:teuthology.orchestra.run.smithi022.stdout:15949e3c88704300882482eddfd7baae-5.14.0-421.el9.x86_64.conf 2024-02-16T19:08:47.629 DEBUG:teuthology.orchestra.run.smithi022:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-421.el9.x86_64 2024-02-16T19:08:47.810 DEBUG:teuthology.orchestra.run.smithi022:> sudo shutdown -r now 2024-02-16T19:09:14.048 INFO:teuthology.task.kernel:Checking client host.b for new kernel version... 2024-02-16T19:09:14.049 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-02-16T19:09:14.050 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T19:09:17.816 INFO:teuthology.task.kernel:Checking client host.a for new kernel version... 2024-02-16T19:09:17.816 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi022.front.sepia.ceph.com' 2024-02-16T19:09:17.817 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T19:09:32.604 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.117 2024-02-16T19:09:36.188 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.22 2024-02-16T19:09:41.606 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-02-16T19:09:41.607 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T19:09:44.668 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.117 2024-02-16T19:09:45.189 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi022.front.sepia.ceph.com' 2024-02-16T19:09:45.189 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T19:09:56.680 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-02-16T19:09:56.681 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T19:10:28.526 DEBUG:teuthology.orchestra.run.smithi117:> true 2024-02-16T19:10:29.059 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-02-16T19:10:29.060 INFO:teuthology.task.kernel:Checking kernel version of host.b, want "5.14.0-421.el9.x86_64"... 2024-02-16T19:10:29.060 DEBUG:teuthology.orchestra.run.smithi117:> uname -r 2024-02-16T19:10:29.073 INFO:teuthology.orchestra.run.smithi117.stdout:5.14.0-421.el9.x86_64 2024-02-16T19:10:29.074 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-421.el9.x86_64 vs 5.14.0-421.el9.x86_64 2024-02-16T19:10:29.074 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-16T19:10:29.074 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-16T19:10:30.075 INFO:teuthology.task.kernel:Enabling kdb on host.b... 2024-02-16T19:10:30.076 DEBUG:teuthology.orchestra.run.smithi117:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-02-16T19:10:30.171 INFO:teuthology.orchestra.run.smithi117.stdout:ttyS1 2024-02-16T19:10:30.205 DEBUG:teuthology.parallel:result is None 2024-02-16T19:10:45.191 DEBUG:teuthology.orchestra.remote:timed out 2024-02-16T19:10:57.193 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi022.front.sepia.ceph.com' 2024-02-16T19:10:57.194 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T19:10:57.603 DEBUG:teuthology.orchestra.run.smithi022:> true 2024-02-16T19:10:57.994 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi022.front.sepia.ceph.com' 2024-02-16T19:10:57.994 INFO:teuthology.task.kernel:Checking kernel version of host.a, want "5.14.0-421.el9.x86_64"... 2024-02-16T19:10:57.994 DEBUG:teuthology.orchestra.run.smithi022:> uname -r 2024-02-16T19:10:58.009 INFO:teuthology.orchestra.run.smithi022.stdout:5.14.0-421.el9.x86_64 2024-02-16T19:10:58.009 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-421.el9.x86_64 vs 5.14.0-421.el9.x86_64 2024-02-16T19:10:58.009 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-16T19:10:58.009 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-16T19:10:59.010 INFO:teuthology.task.kernel:Enabling kdb on host.a... 2024-02-16T19:10:59.010 DEBUG:teuthology.orchestra.run.smithi022:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-02-16T19:10:59.111 INFO:teuthology.orchestra.run.smithi022.stdout:ttyS1 2024-02-16T19:10:59.153 DEBUG:teuthology.parallel:result is None 2024-02-16T19:10:59.153 INFO:teuthology.run_tasks:Running task internal.base... 2024-02-16T19:10:59.163 INFO:teuthology.task.internal:Creating test directory... 2024-02-16T19:10:59.163 DEBUG:teuthology.orchestra.run.smithi022:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-16T19:10:59.166 DEBUG:teuthology.orchestra.run.smithi117:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-16T19:10:59.184 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-02-16T19:10:59.191 INFO:teuthology.run_tasks:Running task internal.archive... 2024-02-16T19:10:59.199 INFO:teuthology.task.internal:Creating archive directory... 2024-02-16T19:10:59.199 DEBUG:teuthology.orchestra.run.smithi022:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-16T19:10:59.224 DEBUG:teuthology.orchestra.run.smithi117:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-16T19:10:59.374 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-02-16T19:10:59.381 INFO:teuthology.task.internal:Enabling coredump saving... 2024-02-16T19:10:59.381 DEBUG:teuthology.orchestra.run.smithi022:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-16T19:10:59.384 DEBUG:teuthology.orchestra.run.smithi117:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-16T19:10:59.422 INFO:teuthology.orchestra.run.smithi022.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-16T19:10:59.424 INFO:teuthology.orchestra.run.smithi117.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-16T19:10:59.436 INFO:teuthology.orchestra.run.smithi022.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-16T19:10:59.438 INFO:teuthology.orchestra.run.smithi117.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-16T19:10:59.440 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-02-16T19:10:59.448 INFO:teuthology.task.internal:Configuring sudo... 2024-02-16T19:10:59.448 DEBUG:teuthology.orchestra.run.smithi022:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-16T19:10:59.480 DEBUG:teuthology.orchestra.run.smithi117:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-16T19:10:59.510 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-02-16T19:10:59.519 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-02-16T19:10:59.519 DEBUG:teuthology.orchestra.run.smithi022:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-16T19:10:59.551 DEBUG:teuthology.orchestra.run.smithi117:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-16T19:10:59.567 DEBUG:teuthology.orchestra.run.smithi022:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-16T19:10:59.649 DEBUG:teuthology.orchestra.run.smithi022:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-16T19:10:59.697 DEBUG:teuthology.orchestra.run.smithi022:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-16T19:10:59.775 DEBUG:teuthology.orchestra.run.smithi022:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-16T19:10:59.822 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-02-16T19:10:59.822 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-16T19:10:59.889 DEBUG:teuthology.orchestra.run.smithi117:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-16T19:10:59.935 DEBUG:teuthology.orchestra.run.smithi117:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-16T19:11:00.000 DEBUG:teuthology.orchestra.run.smithi117:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-16T19:11:00.040 DEBUG:teuthology.orchestra.run.smithi117:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-16T19:11:00.088 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-02-16T19:11:00.088 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-16T19:11:00.156 DEBUG:teuthology.orchestra.run.smithi022:> sudo service rsyslog restart 2024-02-16T19:11:00.158 DEBUG:teuthology.orchestra.run.smithi117:> sudo service rsyslog restart 2024-02-16T19:11:00.201 INFO:teuthology.orchestra.run.smithi022.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-16T19:11:00.248 INFO:teuthology.orchestra.run.smithi117.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-16T19:11:00.502 INFO:teuthology.run_tasks:Running task internal.timer... 2024-02-16T19:11:00.510 INFO:teuthology.task.internal:Starting timer... 2024-02-16T19:11:00.510 INFO:teuthology.run_tasks:Running task pcp... 2024-02-16T19:11:00.519 INFO:teuthology.run_tasks:Running task selinux... 2024-02-16T19:11:00.527 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-02-16T19:11:00.528 DEBUG:teuthology.orchestra.run.smithi022:> sudo service auditd rotate 2024-02-16T19:11:00.588 INFO:teuthology.orchestra.run.smithi022.stdout:Rotating logs: 2024-02-16T19:11:00.590 DEBUG:teuthology.orchestra.run.smithi117:> sudo service auditd rotate 2024-02-16T19:11:00.661 INFO:teuthology.orchestra.run.smithi117.stdout:Rotating logs: 2024-02-16T19:11:00.662 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-02-16T19:11:00.663 DEBUG:teuthology.orchestra.run.smithi022:> /usr/sbin/getenforce 2024-02-16T19:11:00.690 INFO:teuthology.orchestra.run.smithi022.stdout:Permissive 2024-02-16T19:11:00.690 DEBUG:teuthology.orchestra.run.smithi117:> /usr/sbin/getenforce 2024-02-16T19:11:00.722 INFO:teuthology.orchestra.run.smithi117.stdout:Permissive 2024-02-16T19:11:00.722 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi022.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi117.front.sepia.ceph.com': 'permissive'} 2024-02-16T19:11:00.722 DEBUG:teuthology.orchestra.run.smithi022:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-16T19:11:00.758 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-16T19:11:00.758 DEBUG:teuthology.orchestra.run.smithi117:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-16T19:11:00.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-16T19:11:00.790 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-02-16T19:11:00.790 DEBUG:teuthology.orchestra.run.smithi022:> sudo /usr/sbin/setenforce permissive 2024-02-16T19:11:00.832 DEBUG:teuthology.orchestra.run.smithi117:> sudo /usr/sbin/setenforce permissive 2024-02-16T19:11:00.874 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-02-16T19:11:00.884 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-02-16T19:11:00.889 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-02-16T19:11:01.035 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-02-16T19:11:01.051 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-02-16T19:11:01.055 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi022.front.sepia.ceph.com,smithi117.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-02-16T19:15:11.979 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi022.front.sepia.ceph.com'), Remote(name='ubuntu@smithi117.front.sepia.ceph.com')] 2024-02-16T19:15:11.980 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi022.front.sepia.ceph.com' 2024-02-16T19:15:11.981 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T19:15:12.073 DEBUG:teuthology.orchestra.run.smithi022:> true 2024-02-16T19:15:12.146 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi022.front.sepia.ceph.com' 2024-02-16T19:15:12.147 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-02-16T19:15:12.147 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T19:15:12.235 DEBUG:teuthology.orchestra.run.smithi117:> true 2024-02-16T19:15:12.322 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-02-16T19:15:12.322 INFO:teuthology.run_tasks:Running task clock... 2024-02-16T19:15:12.334 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-02-16T19:15:12.334 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-16T19:15:12.334 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-16T19:15:12.337 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-16T19:15:12.337 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-16T19:15:12.366 INFO:teuthology.orchestra.run.smithi022.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-16T19:15:12.384 INFO:teuthology.orchestra.run.smithi022.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-16T19:15:12.393 INFO:teuthology.orchestra.run.smithi117.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-16T19:15:12.409 INFO:teuthology.orchestra.run.smithi117.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-16T19:15:12.414 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: ntpd: command not found 2024-02-16T19:15:12.428 INFO:teuthology.orchestra.run.smithi022.stdout:506 Cannot talk to daemon 2024-02-16T19:15:12.438 INFO:teuthology.orchestra.run.smithi117.stderr:sudo: ntpd: command not found 2024-02-16T19:15:12.445 INFO:teuthology.orchestra.run.smithi022.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-16T19:15:12.452 INFO:teuthology.orchestra.run.smithi117.stdout:506 Cannot talk to daemon 2024-02-16T19:15:12.461 INFO:teuthology.orchestra.run.smithi022.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-16T19:15:12.468 INFO:teuthology.orchestra.run.smithi117.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-16T19:15:12.484 INFO:teuthology.orchestra.run.smithi117.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-16T19:15:12.511 INFO:teuthology.orchestra.run.smithi022.stderr:bash: line 1: ntpq: command not found 2024-02-16T19:15:12.514 INFO:teuthology.orchestra.run.smithi022.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-16T19:15:12.514 INFO:teuthology.orchestra.run.smithi022.stdout:=============================================================================== 2024-02-16T19:15:12.515 INFO:teuthology.orchestra.run.smithi022.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-16T19:15:12.515 INFO:teuthology.orchestra.run.smithi022.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-16T19:15:12.515 INFO:teuthology.orchestra.run.smithi022.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-16T19:15:12.515 INFO:teuthology.orchestra.run.smithi022.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-16T19:15:12.536 INFO:teuthology.orchestra.run.smithi117.stderr:bash: line 1: ntpq: command not found 2024-02-16T19:15:12.538 INFO:teuthology.orchestra.run.smithi117.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-16T19:15:12.538 INFO:teuthology.orchestra.run.smithi117.stdout:=============================================================================== 2024-02-16T19:15:12.538 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-16T19:15:12.538 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-16T19:15:12.539 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-16T19:15:12.539 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-16T19:15:12.539 INFO:teuthology.run_tasks:Running task install... 2024-02-16T19:15:12.548 DEBUG:teuthology.task.install:project ceph 2024-02-16T19:15:12.548 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'f78a58c0ffd401d1493058a1022c35f011d65275'}} 2024-02-16T19:15:12.548 DEBUG:teuthology.task.install:config {'exclude_packages': ['ceph-volume'], 'tag': 'v18.2.0', 'flavor': 'default', 'sha1': 'f78a58c0ffd401d1493058a1022c35f011d65275'} 2024-02-16T19:15:12.548 INFO:teuthology.task.install:Using flavor: default 2024-02-16T19:15:12.553 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-02-16T19:15:12.553 INFO:teuthology.task.install:extra packages: [] 2024-02-16T19:15:12.554 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'f78a58c0ffd401d1493058a1022c35f011d65275', 'tag': 'v18.2.0', 'wait_for_package': False} 2024-02-16T19:15:12.554 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-02-16T19:15:12.554 INFO:teuthology.packaging:ref: None 2024-02-16T19:15:12.554 INFO:teuthology.packaging:tag: v18.2.0 2024-02-16T19:15:12.554 INFO:teuthology.packaging:branch: None 2024-02-16T19:15:12.554 INFO:teuthology.packaging:sha1: f78a58c0ffd401d1493058a1022c35f011d65275 2024-02-16T19:15:12.888 DEBUG:teuthology.repo_utils:git ls-remote https://git.ceph.com/ceph-ci v18.2.0^{} -> None 2024-02-16T19:15:12.888 INFO:teuthology.packaging:Tag 'v18.2.0' not found in https://git.ceph.com/ceph-ci; will also look in https://git.ceph.com/ceph 2024-02-16T19:15:14.677 DEBUG:teuthology.repo_utils:git ls-remote https://git.ceph.com/ceph v18.2.0^{} -> 5dd24139a1eada541a3bc16b6941c5dde975e26d 2024-02-16T19:15:14.678 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5dd24139a1eada541a3bc16b6941c5dde975e26d 2024-02-16T19:15:14.680 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'f78a58c0ffd401d1493058a1022c35f011d65275', 'tag': 'v18.2.0', 'wait_for_package': False} 2024-02-16T19:15:14.680 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-02-16T19:15:14.680 INFO:teuthology.packaging:ref: None 2024-02-16T19:15:14.680 INFO:teuthology.packaging:tag: v18.2.0 2024-02-16T19:15:14.680 INFO:teuthology.packaging:branch: None 2024-02-16T19:15:14.680 INFO:teuthology.packaging:sha1: f78a58c0ffd401d1493058a1022c35f011d65275 2024-02-16T19:15:14.681 INFO:teuthology.packaging:Tag 'v18.2.0' not found in https://git.ceph.com/ceph-ci; will also look in https://git.ceph.com/ceph 2024-02-16T19:15:14.681 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5dd24139a1eada541a3bc16b6941c5dde975e26d 2024-02-16T19:15:14.842 INFO:teuthology.task.install.rpm:Pulling from https://chacra.ceph.com/r/ceph/reef/5dd24139a1eada541a3bc16b6941c5dde975e26d/centos/9/flavors/default/ 2024-02-16T19:15:14.842 INFO:teuthology.task.install.rpm:Package version is 18.2.0-0 2024-02-16T19:15:14.847 INFO:teuthology.task.install.rpm:Pulling from https://chacra.ceph.com/r/ceph/reef/5dd24139a1eada541a3bc16b6941c5dde975e26d/centos/9/flavors/default/ 2024-02-16T19:15:14.847 INFO:teuthology.task.install.rpm:Package version is 18.2.0-0 2024-02-16T19:15:14.889 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://chacra.ceph.com/r/ceph/reef/5dd24139a1eada541a3bc16b6941c5dde975e26d/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://chacra.ceph.com/r/ceph/reef/5dd24139a1eada541a3bc16b6941c5dde975e26d/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://chacra.ceph.com/r/ceph/reef/5dd24139a1eada541a3bc16b6941c5dde975e26d/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-16T19:15:14.890 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-02-16T19:15:14.890 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-16T19:15:14.893 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://chacra.ceph.com/r/ceph/reef/5dd24139a1eada541a3bc16b6941c5dde975e26d/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://chacra.ceph.com/r/ceph/reef/5dd24139a1eada541a3bc16b6941c5dde975e26d/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://chacra.ceph.com/r/ceph/reef/5dd24139a1eada541a3bc16b6941c5dde975e26d/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-16T19:15:14.894 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-02-16T19:15:14.894 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-16T19:15:14.923 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-02-16T19:15:14.923 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-02-16T19:15:14.924 INFO:teuthology.packaging:ref: None 2024-02-16T19:15:14.924 INFO:teuthology.packaging:tag: v18.2.0 2024-02-16T19:15:14.924 INFO:teuthology.packaging:branch: None 2024-02-16T19:15:14.924 INFO:teuthology.packaging:sha1: f78a58c0ffd401d1493058a1022c35f011d65275 2024-02-16T19:15:14.924 DEBUG:teuthology.orchestra.run.smithi117:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/v18.2.0/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-16T19:15:14.926 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-02-16T19:15:14.926 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-02-16T19:15:14.926 INFO:teuthology.packaging:ref: None 2024-02-16T19:15:14.927 INFO:teuthology.packaging:tag: v18.2.0 2024-02-16T19:15:14.927 INFO:teuthology.packaging:branch: None 2024-02-16T19:15:14.927 INFO:teuthology.packaging:sha1: f78a58c0ffd401d1493058a1022c35f011d65275 2024-02-16T19:15:14.927 DEBUG:teuthology.orchestra.run.smithi022:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/v18.2.0/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-16T19:15:14.995 DEBUG:teuthology.orchestra.run.smithi117:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-16T19:15:14.996 DEBUG:teuthology.orchestra.run.smithi022:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-16T19:15:15.078 DEBUG:teuthology.orchestra.run.smithi022:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-16T19:15:15.079 DEBUG:teuthology.orchestra.run.smithi117:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-16T19:15:15.145 INFO:teuthology.orchestra.run.smithi022.stdout:check_obsoletes = 1 2024-02-16T19:15:15.147 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum clean all 2024-02-16T19:15:15.147 INFO:teuthology.orchestra.run.smithi117.stdout:check_obsoletes = 1 2024-02-16T19:15:15.148 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum clean all 2024-02-16T19:15:15.400 INFO:teuthology.orchestra.run.smithi117.stdout:74 files removed 2024-02-16T19:15:15.401 INFO:teuthology.orchestra.run.smithi022.stdout:74 files removed 2024-02-16T19:15:15.427 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install ceph-radosgw 2024-02-16T19:15:15.428 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install ceph-radosgw 2024-02-16T19:15:15.861 INFO:teuthology.orchestra.run.smithi117.stdout:ceph packages for x86_64 641 kB/s | 76 kB 00:00 2024-02-16T19:15:15.875 INFO:teuthology.orchestra.run.smithi022.stdout:ceph packages for x86_64 613 kB/s | 76 kB 00:00 2024-02-16T19:15:16.028 INFO:teuthology.orchestra.run.smithi117.stdout:ceph noarch packages 78 kB/s | 9.3 kB 00:00 2024-02-16T19:15:16.061 INFO:teuthology.orchestra.run.smithi022.stdout:ceph noarch packages 69 kB/s | 9.3 kB 00:00 2024-02-16T19:15:16.196 INFO:teuthology.orchestra.run.smithi117.stdout:ceph source packages 17 kB/s | 2.2 kB 00:00 2024-02-16T19:15:16.237 INFO:teuthology.orchestra.run.smithi022.stdout:ceph source packages 16 kB/s | 2.2 kB 00:00 2024-02-16T19:15:17.123 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS-9 - AppStream 7.5 MB/s | 6.7 MB 00:00 2024-02-16T19:15:17.189 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS-9 - AppStream 7.3 MB/s | 6.7 MB 00:00 2024-02-16T19:15:18.548 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS-9 - BaseOS 6.4 MB/s | 2.0 MB 00:00 2024-02-16T19:15:18.681 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS-9 - BaseOS 6.8 MB/s | 2.0 MB 00:00 2024-02-16T19:15:20.105 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 9 - BaseOS 7.4 MB/s | 8.0 MB 00:01 2024-02-16T19:15:20.497 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - BaseOS 4.9 MB/s | 8.0 MB 00:01 2024-02-16T19:15:35.754 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 9 - AppStream 1.3 MB/s | 19 MB 00:14 2024-02-16T19:15:40.388 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 9 - CRB 6.5 MB/s | 6.1 MB 00:00 2024-02-16T19:15:41.989 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 9 - Extras packages 37 kB/s | 15 kB 00:00 2024-02-16T19:15:43.940 INFO:teuthology.orchestra.run.smithi022.stdout:Extra Packages for Enterprise Linux 11 MB/s | 20 MB 00:01 2024-02-16T19:15:49.074 INFO:teuthology.orchestra.run.smithi022.stdout:lab-extras 34 kB/s | 1.7 kB 00:00 2024-02-16T19:15:51.858 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-02-16T19:15:51.871 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-02-16T19:15:51.872 INFO:teuthology.orchestra.run.smithi022.stdout: Package Arch Version Repository Size 2024-02-16T19:15:51.872 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-02-16T19:15:51.872 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2024-02-16T19:15:51.872 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-radosgw x86_64 2:18.2.0-0.el9 ceph 7.6 M 2024-02-16T19:15:51.872 INFO:teuthology.orchestra.run.smithi022.stdout:Upgrading: 2024-02-16T19:15:51.872 INFO:teuthology.orchestra.run.smithi022.stdout: librados2 x86_64 2:18.2.0-0.el9 ceph 3.3 M 2024-02-16T19:15:51.872 INFO:teuthology.orchestra.run.smithi022.stdout: librbd1 x86_64 2:18.2.0-0.el9 ceph 3.0 M 2024-02-16T19:15:51.872 INFO:teuthology.orchestra.run.smithi022.stdout:Installing dependencies: 2024-02-16T19:15:51.872 INFO:teuthology.orchestra.run.smithi022.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-02-16T19:15:51.872 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-base x86_64 2:18.2.0-0.el9 ceph 5.2 M 2024-02-16T19:15:51.872 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-common x86_64 2:18.2.0-0.el9 ceph 18 M 2024-02-16T19:15:51.872 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-selinux x86_64 2:18.2.0-0.el9 ceph 24 k 2024-02-16T19:15:51.872 INFO:teuthology.orchestra.run.smithi022.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-02-16T19:15:51.873 INFO:teuthology.orchestra.run.smithi022.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-02-16T19:15:51.873 INFO:teuthology.orchestra.run.smithi022.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-02-16T19:15:51.873 INFO:teuthology.orchestra.run.smithi022.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-02-16T19:15:51.873 INFO:teuthology.orchestra.run.smithi022.stdout: libcephfs2 x86_64 2:18.2.0-0.el9 ceph 653 k 2024-02-16T19:15:51.873 INFO:teuthology.orchestra.run.smithi022.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-02-16T19:15:51.873 INFO:teuthology.orchestra.run.smithi022.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-02-16T19:15:51.873 INFO:teuthology.orchestra.run.smithi022.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-02-16T19:15:51.873 INFO:teuthology.orchestra.run.smithi022.stdout: libradosstriper1 x86_64 2:18.2.0-0.el9 ceph 474 k 2024-02-16T19:15:51.873 INFO:teuthology.orchestra.run.smithi022.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-02-16T19:15:51.873 INFO:teuthology.orchestra.run.smithi022.stdout: librgw2 x86_64 2:18.2.0-0.el9 ceph 4.4 M 2024-02-16T19:15:51.873 INFO:teuthology.orchestra.run.smithi022.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-02-16T19:15:51.873 INFO:teuthology.orchestra.run.smithi022.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-02-16T19:15:51.873 INFO:teuthology.orchestra.run.smithi022.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-02-16T19:15:51.874 INFO:teuthology.orchestra.run.smithi022.stdout: python3-ceph-argparse x86_64 2:18.2.0-0.el9 ceph 45 k 2024-02-16T19:15:51.874 INFO:teuthology.orchestra.run.smithi022.stdout: python3-ceph-common x86_64 2:18.2.0-0.el9 ceph 119 k 2024-02-16T19:15:51.874 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cephfs x86_64 2:18.2.0-0.el9 ceph 155 k 2024-02-16T19:15:51.874 INFO:teuthology.orchestra.run.smithi022.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-02-16T19:15:51.874 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rados x86_64 2:18.2.0-0.el9 ceph 321 k 2024-02-16T19:15:51.874 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rbd x86_64 2:18.2.0-0.el9 ceph 297 k 2024-02-16T19:15:51.874 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rgw x86_64 2:18.2.0-0.el9 ceph 99 k 2024-02-16T19:15:51.874 INFO:teuthology.orchestra.run.smithi022.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-02-16T19:15:51.874 INFO:teuthology.orchestra.run.smithi022.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-02-16T19:15:51.874 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:15:51.874 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2024-02-16T19:15:51.874 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-02-16T19:15:51.874 INFO:teuthology.orchestra.run.smithi022.stdout:Install 28 Packages 2024-02-16T19:15:51.874 INFO:teuthology.orchestra.run.smithi022.stdout:Upgrade 2 Packages 2024-02-16T19:15:51.874 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:15:51.875 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 53 M 2024-02-16T19:15:51.875 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading Packages: 2024-02-16T19:15:52.233 INFO:teuthology.orchestra.run.smithi022.stdout:(1/30): ceph-base-18.2.0-0.el9.x86_64.rpm 30 MB/s | 5.2 MB 00:00 2024-02-16T19:15:52.258 INFO:teuthology.orchestra.run.smithi022.stdout:(2/30): ceph-selinux-18.2.0-0.el9.x86_64.rpm 967 kB/s | 24 kB 00:00 2024-02-16T19:15:52.291 INFO:teuthology.orchestra.run.smithi022.stdout:(3/30): libcephfs2-18.2.0-0.el9.x86_64.rpm 19 MB/s | 653 kB 00:00 2024-02-16T19:15:52.325 INFO:teuthology.orchestra.run.smithi022.stdout:(4/30): libradosstriper1-18.2.0-0.el9.x86_64.rp 14 MB/s | 474 kB 00:00 2024-02-16T19:15:52.425 INFO:teuthology.orchestra.run.smithi022.stdout:(5/30): librgw2-18.2.0-0.el9.x86_64.rpm 44 MB/s | 4.4 MB 00:00 2024-02-16T19:15:52.450 INFO:teuthology.orchestra.run.smithi022.stdout:(6/30): python3-ceph-argparse-18.2.0-0.el9.x86_ 1.8 MB/s | 45 kB 00:00 2024-02-16T19:15:52.476 INFO:teuthology.orchestra.run.smithi022.stdout:(7/30): python3-ceph-common-18.2.0-0.el9.x86_64 4.7 MB/s | 119 kB 00:00 2024-02-16T19:15:52.501 INFO:teuthology.orchestra.run.smithi022.stdout:(8/30): python3-cephfs-18.2.0-0.el9.x86_64.rpm 6.1 MB/s | 155 kB 00:00 2024-02-16T19:15:52.601 INFO:teuthology.orchestra.run.smithi022.stdout:(9/30): ceph-radosgw-18.2.0-0.el9.x86_64.rpm 14 MB/s | 7.6 MB 00:00 2024-02-16T19:15:52.627 INFO:teuthology.orchestra.run.smithi022.stdout:(10/30): python3-rados-18.2.0-0.el9.x86_64.rpm 2.5 MB/s | 321 kB 00:00 2024-02-16T19:15:52.652 INFO:teuthology.orchestra.run.smithi022.stdout:(11/30): python3-rgw-18.2.0-0.el9.x86_64.rpm 3.9 MB/s | 99 kB 00:00 2024-02-16T19:15:52.677 INFO:teuthology.orchestra.run.smithi022.stdout:(12/30): python3-rbd-18.2.0-0.el9.x86_64.rpm 3.9 MB/s | 297 kB 00:00 2024-02-16T19:15:52.944 INFO:teuthology.orchestra.run.smithi022.stdout:(13/30): ceph-common-18.2.0-0.el9.x86_64.rpm 21 MB/s | 18 MB 00:00 2024-02-16T19:15:52.969 INFO:teuthology.orchestra.run.smithi022.stdout:(14/30): boost-program-options-1.75.0-8.el9.x86 334 kB/s | 106 kB 00:00 2024-02-16T19:15:53.003 INFO:teuthology.orchestra.run.smithi022.stdout:(15/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 489 kB/s | 159 kB 00:00 2024-02-16T19:15:53.019 INFO:teuthology.orchestra.run.smithi022.stdout:(16/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 590 kB/s | 44 kB 00:00 2024-02-16T19:15:53.053 INFO:teuthology.orchestra.run.smithi022.stdout:(17/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.5 MB/s | 282 kB 00:00 2024-02-16T19:15:53.095 INFO:teuthology.orchestra.run.smithi022.stdout:(18/30): librdkafka-1.6.1-102.el9.x86_64.rpm 7.0 MB/s | 661 kB 00:00 2024-02-16T19:15:53.120 INFO:teuthology.orchestra.run.smithi022.stdout:(19/30): python3-prettytable-0.7.2-27.el9.noarc 410 kB/s | 41 kB 00:00 2024-02-16T19:15:54.413 INFO:teuthology.orchestra.run.smithi022.stdout:(20/30): fmt-8.1.1-5.el9.x86_64.rpm 81 kB/s | 111 kB 00:01 2024-02-16T19:15:54.696 INFO:teuthology.orchestra.run.smithi022.stdout:(21/30): libarrow-doc-9.0.0-10.el9.noarch.rpm 92 kB/s | 26 kB 00:00 2024-02-16T19:15:54.746 INFO:teuthology.orchestra.run.smithi022.stdout:(22/30): gperftools-libs-2.9.1-2.el9.x86_64.rpm 187 kB/s | 309 kB 00:01 2024-02-16T19:15:54.996 INFO:teuthology.orchestra.run.smithi022.stdout:(23/30): liboath-2.6.7-2.el9.x86_64.rpm 162 kB/s | 49 kB 00:00 2024-02-16T19:15:55.063 INFO:teuthology.orchestra.run.smithi022.stdout:(24/30): libunwind-1.6.2-1.el9.x86_64.rpm 212 kB/s | 67 kB 00:00 2024-02-16T19:15:55.614 INFO:teuthology.orchestra.run.smithi022.stdout:(25/30): libarrow-9.0.0-10.el9.x86_64.rpm 1.8 MB/s | 4.4 MB 00:02 2024-02-16T19:15:55.839 INFO:teuthology.orchestra.run.smithi022.stdout:(26/30): parquet-libs-9.0.0-10.el9.x86_64.rpm 996 kB/s | 838 kB 00:00 2024-02-16T19:15:55.872 INFO:teuthology.orchestra.run.smithi022.stdout:(27/30): re2-20211101-3.el9.x86_64.rpm 237 kB/s | 192 kB 00:00 2024-02-16T19:15:55.990 INFO:teuthology.orchestra.run.smithi022.stdout:(28/30): librados2-18.2.0-0.el9.x86_64.rpm 22 MB/s | 3.3 MB 00:00 2024-02-16T19:15:56.048 INFO:teuthology.orchestra.run.smithi022.stdout:(29/30): librbd1-18.2.0-0.el9.x86_64.rpm 17 MB/s | 3.0 MB 00:00 2024-02-16T19:15:56.207 INFO:teuthology.orchestra.run.smithi022.stdout:(30/30): thrift-0.15.0-2.el9.x86_64.rpm 2.7 MB/s | 1.6 MB 00:00 2024-02-16T19:15:56.214 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2024-02-16T19:15:56.214 INFO:teuthology.orchestra.run.smithi022.stdout:Total 12 MB/s | 53 MB 00:04 2024-02-16T19:15:56.215 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2024-02-16T19:15:56.255 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction check succeeded. 2024-02-16T19:15:56.255 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2024-02-16T19:15:56.721 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded. 2024-02-16T19:15:56.721 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2024-02-16T19:15:58.166 INFO:teuthology.orchestra.run.smithi022.stdout: Preparing : 1/1 2024-02-16T19:15:58.241 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2024-02-16T19:15:58.269 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2024-02-16T19:15:58.327 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2024-02-16T19:15:58.702 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2024-02-16T19:15:58.727 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : librados2-2:18.2.0-0.el9.x86_64 5/32 2024-02-16T19:15:58.802 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: librados2-2:18.2.0-0.el9.x86_64 5/32 2024-02-16T19:15:58.884 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-rados-2:18.2.0-0.el9.x86_64 6/32 2024-02-16T19:15:58.928 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libcephfs2-2:18.2.0-0.el9.x86_64 7/32 2024-02-16T19:15:58.990 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: libcephfs2-2:18.2.0-0.el9.x86_64 7/32 2024-02-16T19:15:59.020 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2024-02-16T19:15:59.067 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-ceph-argparse-2:18.2.0-0.el9.x86_64 9/32 2024-02-16T19:15:59.139 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-cephfs-2:18.2.0-0.el9.x86_64 10/32 2024-02-16T19:15:59.164 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libradosstriper1-2:18.2.0-0.el9.x86_64 11/32 2024-02-16T19:15:59.230 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: libradosstriper1-2:18.2.0-0.el9.x86_64 11/32 2024-02-16T19:15:59.341 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : re2-1:20211101-3.el9.x86_64 12/32 2024-02-16T19:15:59.376 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 13/32 2024-02-16T19:15:59.412 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2024-02-16T19:15:59.452 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/32 2024-02-16T19:15:59.479 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 16/32 2024-02-16T19:15:59.516 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2024-02-16T19:15:59.548 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2024-02-16T19:15:59.871 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2024-02-16T19:15:59.902 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : librbd1-2:18.2.0-0.el9.x86_64 20/32 2024-02-16T19:15:59.985 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: librbd1-2:18.2.0-0.el9.x86_64 20/32 2024-02-16T19:16:00.016 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-rbd-2:18.2.0-0.el9.x86_64 21/32 2024-02-16T19:16:00.059 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2024-02-16T19:16:00.477 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 23/32 2024-02-16T19:16:00.505 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : librgw2-2:18.2.0-0.el9.x86_64 24/32 2024-02-16T19:16:00.593 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: librgw2-2:18.2.0-0.el9.x86_64 24/32 2024-02-16T19:16:00.663 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-rgw-2:18.2.0-0.el9.x86_64 25/32 2024-02-16T19:16:00.691 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-ceph-common-2:18.2.0-0.el9.x86_64 26/32 2024-02-16T19:16:02.795 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-common-2:18.2.0-0.el9.x86_64 27/32 2024-02-16T19:16:02.829 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-common-2:18.2.0-0.el9.x86_64 27/32 2024-02-16T19:16:03.410 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-common-2:18.2.0-0.el9.x86_64 27/32 2024-02-16T19:16:03.448 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-base-2:18.2.0-0.el9.x86_64 28/32 2024-02-16T19:16:03.523 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-base-2:18.2.0-0.el9.x86_64 28/32 2024-02-16T19:16:03.524 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-02-16T19:16:03.524 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-02-16T19:16:03.524 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:16:03.558 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-selinux-2:18.2.0-0.el9.x86_64 29/32 2024-02-16T19:16:13.603 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-selinux-2:18.2.0-0.el9.x86_64 29/32 2024-02-16T19:16:13.603 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /sys 2024-02-16T19:16:13.603 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /proc 2024-02-16T19:16:13.603 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /mnt 2024-02-16T19:16:13.603 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /var/tmp 2024-02-16T19:16:13.603 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /home 2024-02-16T19:16:13.603 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /root 2024-02-16T19:16:13.603 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /tmp 2024-02-16T19:16:13.603 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:16:13.829 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - AppStream 374 kB/s | 19 MB 00:52 2024-02-16T19:16:14.365 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-radosgw-2:18.2.0-0.el9.x86_64 30/32 2024-02-16T19:16:14.389 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-radosgw-2:18.2.0-0.el9.x86_64 30/32 2024-02-16T19:16:14.389 INFO:teuthology.orchestra.run.smithi022.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-16T19:16:14.389 INFO:teuthology.orchestra.run.smithi022.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-02-16T19:16:14.389 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-16T19:16:14.389 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-16T19:16:14.389 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:16:14.391 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-02-16T19:16:14.483 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-02-16T19:16:14.483 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2024-02-16T19:16:15.511 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2024-02-16T19:16:15.511 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-base-2:18.2.0-0.el9.x86_64 1/32 2024-02-16T19:16:15.511 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-common-2:18.2.0-0.el9.x86_64 2/32 2024-02-16T19:16:15.511 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-radosgw-2:18.2.0-0.el9.x86_64 3/32 2024-02-16T19:16:15.511 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-selinux-2:18.2.0-0.el9.x86_64 4/32 2024-02-16T19:16:15.511 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libcephfs2-2:18.2.0-0.el9.x86_64 5/32 2024-02-16T19:16:15.511 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libradosstriper1-2:18.2.0-0.el9.x86_64 6/32 2024-02-16T19:16:15.511 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librgw2-2:18.2.0-0.el9.x86_64 7/32 2024-02-16T19:16:15.511 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-ceph-argparse-2:18.2.0-0.el9.x86_64 8/32 2024-02-16T19:16:15.511 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-ceph-common-2:18.2.0-0.el9.x86_64 9/32 2024-02-16T19:16:15.511 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-cephfs-2:18.2.0-0.el9.x86_64 10/32 2024-02-16T19:16:15.511 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-rados-2:18.2.0-0.el9.x86_64 11/32 2024-02-16T19:16:15.511 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-rbd-2:18.2.0-0.el9.x86_64 12/32 2024-02-16T19:16:15.512 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-rgw-2:18.2.0-0.el9.x86_64 13/32 2024-02-16T19:16:15.512 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2024-02-16T19:16:15.512 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2024-02-16T19:16:15.512 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2024-02-16T19:16:15.512 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2024-02-16T19:16:15.512 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2024-02-16T19:16:15.512 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2024-02-16T19:16:15.512 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2024-02-16T19:16:15.512 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 21/32 2024-02-16T19:16:15.513 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 22/32 2024-02-16T19:16:15.513 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 23/32 2024-02-16T19:16:15.513 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2024-02-16T19:16:15.513 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2024-02-16T19:16:15.513 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 26/32 2024-02-16T19:16:15.513 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : re2-1:20211101-3.el9.x86_64 27/32 2024-02-16T19:16:15.513 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2024-02-16T19:16:15.513 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librados2-2:18.2.0-0.el9.x86_64 29/32 2024-02-16T19:16:15.513 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2024-02-16T19:16:15.513 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librbd1-2:18.2.0-0.el9.x86_64 31/32 2024-02-16T19:16:16.617 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2024-02-16T19:16:16.617 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:16:16.617 INFO:teuthology.orchestra.run.smithi022.stdout:Upgraded: 2024-02-16T19:16:16.617 INFO:teuthology.orchestra.run.smithi022.stdout: librados2-2:18.2.0-0.el9.x86_64 librbd1-2:18.2.0-0.el9.x86_64 2024-02-16T19:16:16.617 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2024-02-16T19:16:16.617 INFO:teuthology.orchestra.run.smithi022.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-02-16T19:16:16.617 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-base-2:18.2.0-0.el9.x86_64 2024-02-16T19:16:16.618 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-common-2:18.2.0-0.el9.x86_64 2024-02-16T19:16:16.618 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-radosgw-2:18.2.0-0.el9.x86_64 2024-02-16T19:16:16.618 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-selinux-2:18.2.0-0.el9.x86_64 2024-02-16T19:16:16.618 INFO:teuthology.orchestra.run.smithi022.stdout: fmt-8.1.1-5.el9.x86_64 2024-02-16T19:16:16.618 INFO:teuthology.orchestra.run.smithi022.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-02-16T19:16:16.618 INFO:teuthology.orchestra.run.smithi022.stdout: libarrow-9.0.0-10.el9.x86_64 2024-02-16T19:16:16.618 INFO:teuthology.orchestra.run.smithi022.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-02-16T19:16:16.618 INFO:teuthology.orchestra.run.smithi022.stdout: libcephfs2-2:18.2.0-0.el9.x86_64 2024-02-16T19:16:16.618 INFO:teuthology.orchestra.run.smithi022.stdout: liboath-2.6.7-2.el9.x86_64 2024-02-16T19:16:16.618 INFO:teuthology.orchestra.run.smithi022.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-02-16T19:16:16.618 INFO:teuthology.orchestra.run.smithi022.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-02-16T19:16:16.618 INFO:teuthology.orchestra.run.smithi022.stdout: libradosstriper1-2:18.2.0-0.el9.x86_64 2024-02-16T19:16:16.618 INFO:teuthology.orchestra.run.smithi022.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-02-16T19:16:16.618 INFO:teuthology.orchestra.run.smithi022.stdout: librgw2-2:18.2.0-0.el9.x86_64 2024-02-16T19:16:16.619 INFO:teuthology.orchestra.run.smithi022.stdout: libunwind-1.6.2-1.el9.x86_64 2024-02-16T19:16:16.619 INFO:teuthology.orchestra.run.smithi022.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-02-16T19:16:16.619 INFO:teuthology.orchestra.run.smithi022.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-02-16T19:16:16.619 INFO:teuthology.orchestra.run.smithi022.stdout: python3-ceph-argparse-2:18.2.0-0.el9.x86_64 2024-02-16T19:16:16.619 INFO:teuthology.orchestra.run.smithi022.stdout: python3-ceph-common-2:18.2.0-0.el9.x86_64 2024-02-16T19:16:16.619 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cephfs-2:18.2.0-0.el9.x86_64 2024-02-16T19:16:16.619 INFO:teuthology.orchestra.run.smithi022.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-02-16T19:16:16.619 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rados-2:18.2.0-0.el9.x86_64 2024-02-16T19:16:16.619 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rbd-2:18.2.0-0.el9.x86_64 2024-02-16T19:16:16.619 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rgw-2:18.2.0-0.el9.x86_64 2024-02-16T19:16:16.619 INFO:teuthology.orchestra.run.smithi022.stdout: re2-1:20211101-3.el9.x86_64 2024-02-16T19:16:16.619 INFO:teuthology.orchestra.run.smithi022.stdout: thrift-0.15.0-2.el9.x86_64 2024-02-16T19:16:16.619 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:16:16.619 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-02-16T19:16:16.962 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install ceph-test 2024-02-16T19:16:17.559 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:00:28 ago on Fri 16 Feb 2024 07:15:49 PM UTC. 2024-02-16T19:16:17.731 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-02-16T19:16:17.731 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-02-16T19:16:17.731 INFO:teuthology.orchestra.run.smithi022.stdout: Package Arch Version Repository Size 2024-02-16T19:16:17.731 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-02-16T19:16:17.732 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2024-02-16T19:16:17.732 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-test x86_64 2:18.2.0-0.el9 ceph 40 M 2024-02-16T19:16:17.732 INFO:teuthology.orchestra.run.smithi022.stdout:Installing dependencies: 2024-02-16T19:16:17.732 INFO:teuthology.orchestra.run.smithi022.stdout: jq x86_64 1.6-16.el9 CentOS-BaseOS 186 k 2024-02-16T19:16:17.732 INFO:teuthology.orchestra.run.smithi022.stdout: libcephsqlite x86_64 2:18.2.0-0.el9 ceph 161 k 2024-02-16T19:16:17.732 INFO:teuthology.orchestra.run.smithi022.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-BaseOS 217 k 2024-02-16T19:16:17.732 INFO:teuthology.orchestra.run.smithi022.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-02-16T19:16:17.732 INFO:teuthology.orchestra.run.smithi022.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-02-16T19:16:17.732 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:16:17.732 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2024-02-16T19:16:17.732 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-02-16T19:16:17.732 INFO:teuthology.orchestra.run.smithi022.stdout:Install 6 Packages 2024-02-16T19:16:17.732 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:16:17.733 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 40 M 2024-02-16T19:16:17.733 INFO:teuthology.orchestra.run.smithi022.stdout:Installed size: 167 M 2024-02-16T19:16:17.733 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading Packages: 2024-02-16T19:16:17.863 INFO:teuthology.orchestra.run.smithi022.stdout:(1/6): libcephsqlite-18.2.0-0.el9.x86_64.rpm 1.2 MB/s | 161 kB 00:00 2024-02-16T19:16:17.947 INFO:teuthology.orchestra.run.smithi022.stdout:(2/6): socat-1.7.4.1-5.el9.x86_64.rpm 1.4 MB/s | 300 kB 00:00 2024-02-16T19:16:17.963 INFO:teuthology.orchestra.run.smithi022.stdout:(3/6): xmlstarlet-1.6.1-20.el9.x86_64.rpm 625 kB/s | 63 kB 00:00 2024-02-16T19:16:18.031 INFO:teuthology.orchestra.run.smithi022.stdout:(4/6): jq-1.6-16.el9.x86_64.rpm 2.2 MB/s | 186 kB 00:00 2024-02-16T19:16:18.072 INFO:teuthology.orchestra.run.smithi022.stdout:(5/6): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.0 MB/s | 217 kB 00:00 2024-02-16T19:16:18.439 INFO:teuthology.orchestra.run.smithi022.stdout:(6/6): ceph-test-18.2.0-0.el9.x86_64.rpm 56 MB/s | 40 MB 00:00 2024-02-16T19:16:18.442 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2024-02-16T19:16:18.443 INFO:teuthology.orchestra.run.smithi022.stdout:Total 57 MB/s | 40 MB 00:00 2024-02-16T19:16:18.443 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2024-02-16T19:16:18.464 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction check succeeded. 2024-02-16T19:16:18.464 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2024-02-16T19:16:18.824 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded. 2024-02-16T19:16:18.824 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2024-02-16T19:16:19.387 INFO:teuthology.orchestra.run.smithi022.stdout: Preparing : 1/1 2024-02-16T19:16:19.454 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 1/6 2024-02-16T19:16:19.517 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : jq-1.6-16.el9.x86_64 2/6 2024-02-16T19:16:19.561 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 3/6 2024-02-16T19:16:19.607 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 4/6 2024-02-16T19:16:19.634 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libcephsqlite-2:18.2.0-0.el9.x86_64 5/6 2024-02-16T19:16:20.196 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - CRB 2.1 MB/s | 6.1 MB 00:02 2024-02-16T19:16:21.855 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - Extras packages 29 kB/s | 15 kB 00:00 2024-02-16T19:16:23.575 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: libcephsqlite-2:18.2.0-0.el9.x86_64 5/6 2024-02-16T19:16:23.661 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-test-2:18.2.0-0.el9.x86_64 6/6 2024-02-16T19:16:25.705 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-test-2:18.2.0-0.el9.x86_64 6/6 2024-02-16T19:16:25.705 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-test-2:18.2.0-0.el9.x86_64 1/6 2024-02-16T19:16:25.705 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libcephsqlite-2:18.2.0-0.el9.x86_64 2/6 2024-02-16T19:16:25.705 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/6 2024-02-16T19:16:25.705 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/6 2024-02-16T19:16:25.705 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : jq-1.6-16.el9.x86_64 5/6 2024-02-16T19:16:26.150 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 6/6 2024-02-16T19:16:26.150 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:16:26.150 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2024-02-16T19:16:26.151 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-test-2:18.2.0-0.el9.x86_64 jq-1.6-16.el9.x86_64 2024-02-16T19:16:26.151 INFO:teuthology.orchestra.run.smithi022.stdout: libcephsqlite-2:18.2.0-0.el9.x86_64 oniguruma-6.9.6-1.el9.5.x86_64 2024-02-16T19:16:26.151 INFO:teuthology.orchestra.run.smithi022.stdout: socat-1.7.4.1-5.el9.x86_64 xmlstarlet-1.6.1-20.el9.x86_64 2024-02-16T19:16:26.151 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:16:26.151 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-02-16T19:16:26.425 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install ceph 2024-02-16T19:16:27.007 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:00:38 ago on Fri 16 Feb 2024 07:15:49 PM UTC. 2024-02-16T19:16:27.187 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-02-16T19:16:27.190 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-02-16T19:16:27.190 INFO:teuthology.orchestra.run.smithi022.stdout: Package Arch Version Repository Size 2024-02-16T19:16:27.190 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-02-16T19:16:27.190 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2024-02-16T19:16:27.190 INFO:teuthology.orchestra.run.smithi022.stdout: ceph x86_64 2:18.2.0-0.el9 ceph 6.4 k 2024-02-16T19:16:27.190 INFO:teuthology.orchestra.run.smithi022.stdout:Installing dependencies: 2024-02-16T19:16:27.190 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mds x86_64 2:18.2.0-0.el9 ceph 2.1 M 2024-02-16T19:16:27.190 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr x86_64 2:18.2.0-0.el9 ceph 1.4 M 2024-02-16T19:16:27.190 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-modules-core noarch 2:18.2.0-0.el9 ceph-noarch 240 k 2024-02-16T19:16:27.190 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mon x86_64 2:18.2.0-0.el9 ceph 4.4 M 2024-02-16T19:16:27.190 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-osd x86_64 2:18.2.0-0.el9 ceph 18 M 2024-02-16T19:16:27.190 INFO:teuthology.orchestra.run.smithi022.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-02-16T19:16:27.191 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-02-16T19:16:27.191 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-02-16T19:16:27.191 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-02-16T19:16:27.191 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-02-16T19:16:27.191 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-02-16T19:16:27.191 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-02-16T19:16:27.191 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-02-16T19:16:27.191 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-02-16T19:16:27.191 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-02-16T19:16:27.191 INFO:teuthology.orchestra.run.smithi022.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-02-16T19:16:27.191 INFO:teuthology.orchestra.run.smithi022.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-02-16T19:16:27.191 INFO:teuthology.orchestra.run.smithi022.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-02-16T19:16:27.191 INFO:teuthology.orchestra.run.smithi022.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-02-16T19:16:27.191 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-02-16T19:16:27.191 INFO:teuthology.orchestra.run.smithi022.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-02-16T19:16:27.192 INFO:teuthology.orchestra.run.smithi022.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-02-16T19:16:27.192 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-02-16T19:16:27.192 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-02-16T19:16:27.192 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-02-16T19:16:27.192 INFO:teuthology.orchestra.run.smithi022.stdout: python3-requests noarch 2.25.1-8.el9 CentOS-BaseOS 113 k 2024-02-16T19:16:27.192 INFO:teuthology.orchestra.run.smithi022.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-02-16T19:16:27.192 INFO:teuthology.orchestra.run.smithi022.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-02-16T19:16:27.192 INFO:teuthology.orchestra.run.smithi022.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-02-16T19:16:27.192 INFO:teuthology.orchestra.run.smithi022.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-02-16T19:16:27.192 INFO:teuthology.orchestra.run.smithi022.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-02-16T19:16:27.192 INFO:teuthology.orchestra.run.smithi022.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-02-16T19:16:27.192 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:16:27.192 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2024-02-16T19:16:27.192 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-02-16T19:16:27.193 INFO:teuthology.orchestra.run.smithi022.stdout:Install 33 Packages 2024-02-16T19:16:27.193 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:16:27.193 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 30 M 2024-02-16T19:16:27.193 INFO:teuthology.orchestra.run.smithi022.stdout:Installed size: 109 M 2024-02-16T19:16:27.193 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading Packages: 2024-02-16T19:16:27.366 INFO:teuthology.orchestra.run.smithi022.stdout:(1/33): ceph-18.2.0-0.el9.x86_64.rpm 63 kB/s | 6.4 kB 00:00 2024-02-16T19:16:27.408 INFO:teuthology.orchestra.run.smithi022.stdout:(2/33): ceph-mgr-18.2.0-0.el9.x86_64.rpm 10 MB/s | 1.4 MB 00:00 2024-02-16T19:16:27.458 INFO:teuthology.orchestra.run.smithi022.stdout:(3/33): ceph-mds-18.2.0-0.el9.x86_64.rpm 11 MB/s | 2.1 MB 00:00 2024-02-16T19:16:27.483 INFO:teuthology.orchestra.run.smithi022.stdout:(4/33): ceph-mgr-modules-core-18.2.0-0.el9.noar 9.4 MB/s | 240 kB 00:00 2024-02-16T19:16:27.584 INFO:teuthology.orchestra.run.smithi022.stdout:(5/33): ceph-mon-18.2.0-0.el9.x86_64.rpm 20 MB/s | 4.4 MB 00:00 2024-02-16T19:16:27.626 INFO:teuthology.orchestra.run.smithi022.stdout:(6/33): python3-markupsafe-1.1.1-12.el9.x86_64. 785 kB/s | 32 kB 00:00 2024-02-16T19:16:27.659 INFO:teuthology.orchestra.run.smithi022.stdout:(7/33): python3-mako-1.1.4-6.el9.noarch.rpm 822 kB/s | 144 kB 00:00 2024-02-16T19:16:27.684 INFO:teuthology.orchestra.run.smithi022.stdout:(8/33): python3-pytz-2021.1-5.el9.noarch.rpm 814 kB/s | 47 kB 00:00 2024-02-16T19:16:27.710 INFO:teuthology.orchestra.run.smithi022.stdout:(9/33): python3-toml-0.10.2-6.el9.noarch.rpm 772 kB/s | 39 kB 00:00 2024-02-16T19:16:27.735 INFO:teuthology.orchestra.run.smithi022.stdout:(10/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-02-16T19:16:27.777 INFO:teuthology.orchestra.run.smithi022.stdout:(11/33): python3-cryptography-36.0.1-4.el9.x86_ 17 MB/s | 1.1 MB 00:00 2024-02-16T19:16:27.802 INFO:teuthology.orchestra.run.smithi022.stdout:(12/33): python3-ply-3.11-14.el9.noarch.rpm 1.5 MB/s | 103 kB 00:00 2024-02-16T19:16:27.827 INFO:teuthology.orchestra.run.smithi022.stdout:(13/33): python3-pycparser-2.20-6.el9.noarch.rp 2.4 MB/s | 124 kB 00:00 2024-02-16T19:16:27.853 INFO:teuthology.orchestra.run.smithi022.stdout:(14/33): python3-requests-2.25.1-8.el9.noarch.r 2.2 MB/s | 113 kB 00:00 2024-02-16T19:16:27.878 INFO:teuthology.orchestra.run.smithi022.stdout:(15/33): python3-urllib3-1.26.5-5.el9.noarch.rp 3.7 MB/s | 187 kB 00:00 2024-02-16T19:16:28.078 INFO:teuthology.orchestra.run.smithi022.stdout:(16/33): ceph-osd-18.2.0-0.el9.x86_64.rpm 26 MB/s | 18 MB 00:00 2024-02-16T19:16:28.095 INFO:teuthology.orchestra.run.smithi022.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 180 kB/s | 43 kB 00:00 2024-02-16T19:16:28.129 INFO:teuthology.orchestra.run.smithi022.stdout:(18/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 320 kB/s | 11 kB 00:00 2024-02-16T19:16:28.154 INFO:teuthology.orchestra.run.smithi022.stdout:(19/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 623 kB/s | 172 kB 00:00 2024-02-16T19:16:28.187 INFO:teuthology.orchestra.run.smithi022.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 302 kB/s | 18 kB 00:00 2024-02-16T19:16:28.229 INFO:teuthology.orchestra.run.smithi022.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 308 kB/s | 23 kB 00:00 2024-02-16T19:16:28.254 INFO:teuthology.orchestra.run.smithi022.stdout:(22/33): python3-jaraco-functools-3.5.0-2.el9.n 291 kB/s | 19 kB 00:00 2024-02-16T19:16:28.280 INFO:teuthology.orchestra.run.smithi022.stdout:(23/33): python3-jaraco-text-3.2.0-6.el9.noarch 393 kB/s | 20 kB 00:00 2024-02-16T19:16:28.305 INFO:teuthology.orchestra.run.smithi022.stdout:(24/33): python3-logutils-0.3.5-21.el9.noarch.r 921 kB/s | 46 kB 00:00 2024-02-16T19:16:28.330 INFO:teuthology.orchestra.run.smithi022.stdout:(25/33): python3-more-itertools-8.12.0-2.el9.no 1.5 MB/s | 79 kB 00:00 2024-02-16T19:16:28.358 INFO:teuthology.orchestra.run.smithi022.stdout:(26/33): python3-cherrypy-18.6.1-2.el9.noarch.r 1.3 MB/s | 358 kB 00:00 2024-02-16T19:16:28.383 INFO:teuthology.orchestra.run.smithi022.stdout:(27/33): python3-portend-3.1.0-2.el9.noarch.rpm 313 kB/s | 16 kB 00:00 2024-02-16T19:16:28.408 INFO:teuthology.orchestra.run.smithi022.stdout:(28/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.8 MB/s | 90 kB 00:00 2024-02-16T19:16:28.434 INFO:teuthology.orchestra.run.smithi022.stdout:(29/33): python3-tempora-5.0.0-2.el9.noarch.rpm 710 kB/s | 36 kB 00:00 2024-02-16T19:16:28.459 INFO:teuthology.orchestra.run.smithi022.stdout:(30/33): python3-pecan-1.4.2-3.el9.noarch.rpm 1.7 MB/s | 272 kB 00:00 2024-02-16T19:16:28.484 INFO:teuthology.orchestra.run.smithi022.stdout:(31/33): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-02-16T19:16:28.510 INFO:teuthology.orchestra.run.smithi022.stdout:(32/33): python3-zc-lockfile-2.0-10.el9.noarch. 398 kB/s | 20 kB 00:00 2024-02-16T19:16:28.535 INFO:teuthology.orchestra.run.smithi022.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.2 MB/s | 427 kB 00:00 2024-02-16T19:16:28.539 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2024-02-16T19:16:28.539 INFO:teuthology.orchestra.run.smithi022.stdout:Total 22 MB/s | 30 MB 00:01 2024-02-16T19:16:28.539 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2024-02-16T19:16:28.582 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction check succeeded. 2024-02-16T19:16:28.582 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2024-02-16T19:16:28.964 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded. 2024-02-16T19:16:28.964 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2024-02-16T19:16:30.205 INFO:teuthology.orchestra.run.smithi022.stdout: Preparing : 1/1 2024-02-16T19:16:30.279 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-02-16T19:16:30.312 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-02-16T19:16:30.345 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-02-16T19:16:30.440 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-02-16T19:16:30.517 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-02-16T19:16:30.564 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-02-16T19:16:30.647 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-02-16T19:16:30.707 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 8/33 2024-02-16T19:16:30.745 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 9/33 2024-02-16T19:16:30.803 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-ply-3.11-14.el9.noarch 10/33 2024-02-16T19:16:30.889 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 11/33 2024-02-16T19:16:31.088 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 12/33 2024-02-16T19:16:31.141 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 13/33 2024-02-16T19:16:31.179 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 14/33 2024-02-16T19:16:31.214 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 15/33 2024-02-16T19:16:31.243 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 16/33 2024-02-16T19:16:31.323 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 17/33 2024-02-16T19:16:31.355 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-02-16T19:16:31.387 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 19/33 2024-02-16T19:16:31.428 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 20/33 2024-02-16T19:16:31.472 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/33 2024-02-16T19:16:31.506 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 22/33 2024-02-16T19:16:31.601 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 23/33 2024-02-16T19:16:31.637 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 24/33 2024-02-16T19:16:31.721 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/33 2024-02-16T19:16:31.862 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 26/33 2024-02-16T19:16:31.992 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-02-16T19:16:32.175 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mgr-modules-core-2:18.2.0-0.el9.noarch 28/33 2024-02-16T19:16:32.199 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mgr-2:18.2.0-0.el9.x86_64 29/33 2024-02-16T19:16:32.224 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mgr-2:18.2.0-0.el9.x86_64 29/33 2024-02-16T19:16:32.224 INFO:teuthology.orchestra.run.smithi022.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-16T19:16:32.224 INFO:teuthology.orchestra.run.smithi022.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-02-16T19:16:32.224 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-16T19:16:32.225 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-16T19:16:32.225 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:16:33.834 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-osd-2:18.2.0-0.el9.x86_64 30/33 2024-02-16T19:16:33.863 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-osd-2:18.2.0-0.el9.x86_64 30/33 2024-02-16T19:16:33.863 INFO:teuthology.orchestra.run.smithi022.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-16T19:16:33.863 INFO:teuthology.orchestra.run.smithi022.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-02-16T19:16:33.863 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-16T19:16:33.863 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-16T19:16:33.863 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:16:34.335 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mon-2:18.2.0-0.el9.x86_64 31/33 2024-02-16T19:16:34.358 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mon-2:18.2.0-0.el9.x86_64 31/33 2024-02-16T19:16:34.358 INFO:teuthology.orchestra.run.smithi022.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-16T19:16:34.358 INFO:teuthology.orchestra.run.smithi022.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-02-16T19:16:34.359 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-16T19:16:34.359 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-16T19:16:34.359 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:16:34.611 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mds-2:18.2.0-0.el9.x86_64 32/33 2024-02-16T19:16:34.634 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mds-2:18.2.0-0.el9.x86_64 32/33 2024-02-16T19:16:34.634 INFO:teuthology.orchestra.run.smithi022.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-16T19:16:34.634 INFO:teuthology.orchestra.run.smithi022.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-02-16T19:16:34.634 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-16T19:16:34.634 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-16T19:16:34.634 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:16:34.701 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-2:18.2.0-0.el9.x86_64 33/33 2024-02-16T19:16:36.426 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-2:18.2.0-0.el9.x86_64 33/33 2024-02-16T19:16:36.426 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-2:18.2.0-0.el9.x86_64 1/33 2024-02-16T19:16:36.426 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mds-2:18.2.0-0.el9.x86_64 2/33 2024-02-16T19:16:36.426 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mgr-2:18.2.0-0.el9.x86_64 3/33 2024-02-16T19:16:36.426 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mon-2:18.2.0-0.el9.x86_64 4/33 2024-02-16T19:16:36.427 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-osd-2:18.2.0-0.el9.x86_64 5/33 2024-02-16T19:16:36.427 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mgr-modules-core-2:18.2.0-0.el9.noarch 6/33 2024-02-16T19:16:36.427 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 7/33 2024-02-16T19:16:36.427 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 8/33 2024-02-16T19:16:36.427 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 9/33 2024-02-16T19:16:36.427 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 10/33 2024-02-16T19:16:36.427 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 11/33 2024-02-16T19:16:36.427 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 12/33 2024-02-16T19:16:36.427 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-ply-3.11-14.el9.noarch 13/33 2024-02-16T19:16:36.427 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 14/33 2024-02-16T19:16:36.427 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 15/33 2024-02-16T19:16:36.427 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 16/33 2024-02-16T19:16:36.427 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-02-16T19:16:36.427 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-02-16T19:16:36.428 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-02-16T19:16:36.428 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-02-16T19:16:36.428 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-02-16T19:16:36.428 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-02-16T19:16:36.428 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-02-16T19:16:36.428 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-02-16T19:16:36.428 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-02-16T19:16:36.428 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-02-16T19:16:36.428 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-02-16T19:16:36.428 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-02-16T19:16:36.428 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-02-16T19:16:36.428 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-02-16T19:16:36.428 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-02-16T19:16:36.428 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-02-16T19:16:37.504 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-02-16T19:16:37.504 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:16:37.504 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2024-02-16T19:16:37.504 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-2:18.2.0-0.el9.x86_64 2024-02-16T19:16:37.504 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mds-2:18.2.0-0.el9.x86_64 2024-02-16T19:16:37.504 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-2:18.2.0-0.el9.x86_64 2024-02-16T19:16:37.504 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-modules-core-2:18.2.0-0.el9.noarch 2024-02-16T19:16:37.504 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mon-2:18.2.0-0.el9.x86_64 2024-02-16T19:16:37.504 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-osd-2:18.2.0-0.el9.x86_64 2024-02-16T19:16:37.505 INFO:teuthology.orchestra.run.smithi022.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-02-16T19:16:37.505 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-02-16T19:16:37.505 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-02-16T19:16:37.505 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-02-16T19:16:37.505 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-02-16T19:16:37.505 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-02-16T19:16:37.505 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-02-16T19:16:37.505 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-02-16T19:16:37.505 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-02-16T19:16:37.505 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-02-16T19:16:37.505 INFO:teuthology.orchestra.run.smithi022.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-02-16T19:16:37.505 INFO:teuthology.orchestra.run.smithi022.stdout: python3-mako-1.1.4-6.el9.noarch 2024-02-16T19:16:37.505 INFO:teuthology.orchestra.run.smithi022.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-02-16T19:16:37.505 INFO:teuthology.orchestra.run.smithi022.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-02-16T19:16:37.505 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-02-16T19:16:37.505 INFO:teuthology.orchestra.run.smithi022.stdout: python3-ply-3.11-14.el9.noarch 2024-02-16T19:16:37.506 INFO:teuthology.orchestra.run.smithi022.stdout: python3-portend-3.1.0-2.el9.noarch 2024-02-16T19:16:37.506 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-02-16T19:16:37.506 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pycparser-2.20-6.el9.noarch 2024-02-16T19:16:37.506 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pytz-2021.1-5.el9.noarch 2024-02-16T19:16:37.506 INFO:teuthology.orchestra.run.smithi022.stdout: python3-requests-2.25.1-8.el9.noarch 2024-02-16T19:16:37.506 INFO:teuthology.orchestra.run.smithi022.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-02-16T19:16:37.506 INFO:teuthology.orchestra.run.smithi022.stdout: python3-toml-0.10.2-6.el9.noarch 2024-02-16T19:16:37.506 INFO:teuthology.orchestra.run.smithi022.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-02-16T19:16:37.506 INFO:teuthology.orchestra.run.smithi022.stdout: python3-webob-1.8.7-6.el9.noarch 2024-02-16T19:16:37.506 INFO:teuthology.orchestra.run.smithi022.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-02-16T19:16:37.506 INFO:teuthology.orchestra.run.smithi022.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-02-16T19:16:37.506 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:16:37.506 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-02-16T19:16:37.838 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install ceph-base 2024-02-16T19:16:38.449 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:00:49 ago on Fri 16 Feb 2024 07:15:49 PM UTC. 2024-02-16T19:16:38.567 INFO:teuthology.orchestra.run.smithi022.stdout:Package ceph-base-2:18.2.0-0.el9.x86_64 is already installed. 2024-02-16T19:16:38.621 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-02-16T19:16:38.622 INFO:teuthology.orchestra.run.smithi022.stdout:Nothing to do. 2024-02-16T19:16:38.622 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-02-16T19:16:38.697 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install cephadm 2024-02-16T19:16:39.286 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:00:50 ago on Fri 16 Feb 2024 07:15:49 PM UTC. 2024-02-16T19:16:39.459 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-02-16T19:16:39.460 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-02-16T19:16:39.460 INFO:teuthology.orchestra.run.smithi022.stdout: Package Architecture Version Repository Size 2024-02-16T19:16:39.460 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-02-16T19:16:39.460 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2024-02-16T19:16:39.460 INFO:teuthology.orchestra.run.smithi022.stdout: cephadm noarch 2:18.2.0-0.el9 ceph-noarch 209 k 2024-02-16T19:16:39.460 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:16:39.460 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2024-02-16T19:16:39.460 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-02-16T19:16:39.460 INFO:teuthology.orchestra.run.smithi022.stdout:Install 1 Package 2024-02-16T19:16:39.460 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:16:39.460 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 209 k 2024-02-16T19:16:39.460 INFO:teuthology.orchestra.run.smithi022.stdout:Installed size: 200 k 2024-02-16T19:16:39.461 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading Packages: 2024-02-16T19:16:39.526 INFO:teuthology.orchestra.run.smithi022.stdout:cephadm-18.2.0-0.el9.noarch.rpm 3.1 MB/s | 209 kB 00:00 2024-02-16T19:16:39.526 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2024-02-16T19:16:39.527 INFO:teuthology.orchestra.run.smithi022.stdout:Total 3.1 MB/s | 209 kB 00:00 2024-02-16T19:16:39.527 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2024-02-16T19:16:39.532 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction check succeeded. 2024-02-16T19:16:39.532 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2024-02-16T19:16:39.535 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded. 2024-02-16T19:16:39.535 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2024-02-16T19:16:39.606 INFO:teuthology.orchestra.run.smithi022.stdout: Preparing : 1/1 2024-02-16T19:16:40.054 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: cephadm-2:18.2.0-0.el9.noarch 1/1 2024-02-16T19:16:40.149 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : cephadm-2:18.2.0-0.el9.noarch 1/1 2024-02-16T19:16:40.525 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: cephadm-2:18.2.0-0.el9.noarch 1/1 2024-02-16T19:16:40.972 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : cephadm-2:18.2.0-0.el9.noarch 1/1 2024-02-16T19:16:40.972 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:16:40.972 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2024-02-16T19:16:40.972 INFO:teuthology.orchestra.run.smithi022.stdout: cephadm-2:18.2.0-0.el9.noarch 2024-02-16T19:16:40.972 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:16:40.972 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-02-16T19:16:41.190 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install ceph-immutable-object-cache 2024-02-16T19:16:41.770 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:00:52 ago on Fri 16 Feb 2024 07:15:49 PM UTC. 2024-02-16T19:16:41.947 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-02-16T19:16:41.948 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-02-16T19:16:41.948 INFO:teuthology.orchestra.run.smithi022.stdout: Package Arch Version Repo Size 2024-02-16T19:16:41.948 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-02-16T19:16:41.948 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2024-02-16T19:16:41.948 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-immutable-object-cache x86_64 2:18.2.0-0.el9 ceph 142 k 2024-02-16T19:16:41.948 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:16:41.948 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2024-02-16T19:16:41.948 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-02-16T19:16:41.948 INFO:teuthology.orchestra.run.smithi022.stdout:Install 1 Package 2024-02-16T19:16:41.948 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:16:41.949 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 142 k 2024-02-16T19:16:41.949 INFO:teuthology.orchestra.run.smithi022.stdout:Installed size: 399 k 2024-02-16T19:16:41.949 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading Packages: 2024-02-16T19:16:42.000 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-immutable-object-cache-18.2.0-0.el9.x86_64 2.6 MB/s | 142 kB 00:00 2024-02-16T19:16:42.000 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2024-02-16T19:16:42.001 INFO:teuthology.orchestra.run.smithi022.stdout:Total 2.5 MB/s | 142 kB 00:00 2024-02-16T19:16:42.001 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2024-02-16T19:16:42.009 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction check succeeded. 2024-02-16T19:16:42.009 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2024-02-16T19:16:42.062 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded. 2024-02-16T19:16:42.062 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2024-02-16T19:16:42.344 INFO:teuthology.orchestra.run.smithi022.stdout: Preparing : 1/1 2024-02-16T19:16:42.418 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-immutable-object-cache-2:18.2.0-0.el9.x86_64 1/1 2024-02-16T19:16:42.441 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.0-0.el9.x86_64 1/1 2024-02-16T19:16:42.441 INFO:teuthology.orchestra.run.smithi022.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-16T19:16:42.441 INFO:teuthology.orchestra.run.smithi022.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-02-16T19:16:42.441 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:16:43.472 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-immutable-object-cache-2:18.2.0-0.el9.x86_64 1/1 2024-02-16T19:16:43.472 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:16:43.472 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2024-02-16T19:16:43.472 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-immutable-object-cache-2:18.2.0-0.el9.x86_64 2024-02-16T19:16:43.472 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:16:43.472 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-02-16T19:16:43.741 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install ceph-mgr 2024-02-16T19:16:44.340 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:00:55 ago on Fri 16 Feb 2024 07:15:49 PM UTC. 2024-02-16T19:16:44.459 INFO:teuthology.orchestra.run.smithi022.stdout:Package ceph-mgr-2:18.2.0-0.el9.x86_64 is already installed. 2024-02-16T19:16:44.515 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-02-16T19:16:44.516 INFO:teuthology.orchestra.run.smithi022.stdout:Nothing to do. 2024-02-16T19:16:44.516 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-02-16T19:16:44.584 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install ceph-mgr-dashboard 2024-02-16T19:16:45.172 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:00:56 ago on Fri 16 Feb 2024 07:15:49 PM UTC. 2024-02-16T19:16:45.344 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-02-16T19:16:45.345 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-02-16T19:16:45.345 INFO:teuthology.orchestra.run.smithi022.stdout: Package Arch Version Repository Size 2024-02-16T19:16:45.345 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-02-16T19:16:45.345 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2024-02-16T19:16:45.345 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-dashboard noarch 2:18.2.0-0.el9 ceph-noarch 1.7 M 2024-02-16T19:16:45.345 INFO:teuthology.orchestra.run.smithi022.stdout:Installing dependencies: 2024-02-16T19:16:45.345 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-grafana-dashboards noarch 2:18.2.0-0.el9 ceph-noarch 23 k 2024-02-16T19:16:45.345 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-prometheus-alerts noarch 2:18.2.0-0.el9 ceph-noarch 15 k 2024-02-16T19:16:45.345 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-02-16T19:16:45.345 INFO:teuthology.orchestra.run.smithi022.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-02-16T19:16:45.345 INFO:teuthology.orchestra.run.smithi022.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-02-16T19:16:45.346 INFO:teuthology.orchestra.run.smithi022.stdout:Installing weak dependencies: 2024-02-16T19:16:45.346 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-02-16T19:16:45.346 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:16:45.346 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2024-02-16T19:16:45.346 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-02-16T19:16:45.346 INFO:teuthology.orchestra.run.smithi022.stdout:Install 7 Packages 2024-02-16T19:16:45.346 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:16:45.346 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 2.0 M 2024-02-16T19:16:45.347 INFO:teuthology.orchestra.run.smithi022.stdout:Installed size: 8.1 M 2024-02-16T19:16:45.347 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading Packages: 2024-02-16T19:16:45.637 INFO:teuthology.orchestra.run.smithi022.stdout:(1/7): ceph-grafana-dashboards-18.2.0-0.el9.noa 233 kB/s | 23 kB 00:00 2024-02-16T19:16:45.663 INFO:teuthology.orchestra.run.smithi022.stdout:(2/7): ceph-prometheus-alerts-18.2.0-0.el9.noar 118 kB/s | 15 kB 00:00 2024-02-16T19:16:45.713 INFO:teuthology.orchestra.run.smithi022.stdout:(3/7): ceph-mgr-dashboard-18.2.0-0.el9.noarch.r 9.7 MB/s | 1.7 MB 00:00 2024-02-16T19:16:45.897 INFO:teuthology.orchestra.run.smithi117.stdout:Extra Packages for Enterprise Linux 871 kB/s | 20 MB 00:23 2024-02-16T19:16:45.955 INFO:teuthology.orchestra.run.smithi022.stdout:(4/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 28 kB/s | 9.0 kB 00:00 2024-02-16T19:16:46.105 INFO:teuthology.orchestra.run.smithi022.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 92 kB/s | 41 kB 00:00 2024-02-16T19:16:46.130 INFO:teuthology.orchestra.run.smithi022.stdout:(6/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 74 kB/s | 31 kB 00:00 2024-02-16T19:16:46.405 INFO:teuthology.orchestra.run.smithi022.stdout:(7/7): python3-routes-2.5.1-5.el9.noarch.rpm 418 kB/s | 188 kB 00:00 2024-02-16T19:16:46.409 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2024-02-16T19:16:46.410 INFO:teuthology.orchestra.run.smithi022.stdout:Total 1.9 MB/s | 2.0 MB 00:01 2024-02-16T19:16:46.410 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2024-02-16T19:16:46.418 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction check succeeded. 2024-02-16T19:16:46.419 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2024-02-16T19:16:46.486 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded. 2024-02-16T19:16:46.486 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2024-02-16T19:16:46.790 INFO:teuthology.orchestra.run.smithi022.stdout: Preparing : 1/1 2024-02-16T19:16:46.857 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-02-16T19:16:46.892 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-02-16T19:16:46.947 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-02-16T19:16:46.977 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-02-16T19:16:47.006 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-prometheus-alerts-2:18.2.0-0.el9.noarch 5/7 2024-02-16T19:16:47.255 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-grafana-dashboards-2:18.2.0-0.el9.noarch 6/7 2024-02-16T19:16:47.284 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mgr-dashboard-2:18.2.0-0.el9.noarch 7/7 2024-02-16T19:16:47.813 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.0-0.el9.noarch 7/7 2024-02-16T19:16:47.813 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-grafana-dashboards-2:18.2.0-0.el9.noarch 1/7 2024-02-16T19:16:47.813 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mgr-dashboard-2:18.2.0-0.el9.noarch 2/7 2024-02-16T19:16:47.813 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-prometheus-alerts-2:18.2.0-0.el9.noarch 3/7 2024-02-16T19:16:47.814 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-02-16T19:16:47.814 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-02-16T19:16:47.814 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-02-16T19:16:48.124 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-02-16T19:16:48.125 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:16:48.125 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2024-02-16T19:16:48.125 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-grafana-dashboards-2:18.2.0-0.el9.noarch 2024-02-16T19:16:48.125 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-dashboard-2:18.2.0-0.el9.noarch 2024-02-16T19:16:48.125 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-prometheus-alerts-2:18.2.0-0.el9.noarch 2024-02-16T19:16:48.125 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-02-16T19:16:48.125 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-02-16T19:16:48.125 INFO:teuthology.orchestra.run.smithi022.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-02-16T19:16:48.125 INFO:teuthology.orchestra.run.smithi022.stdout: python3-routes-2.5.1-5.el9.noarch 2024-02-16T19:16:48.125 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:16:48.125 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-02-16T19:16:48.345 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install ceph-mgr-diskprediction-local 2024-02-16T19:16:48.932 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:00:59 ago on Fri 16 Feb 2024 07:15:49 PM UTC. 2024-02-16T19:16:49.107 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-02-16T19:16:49.108 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-02-16T19:16:49.108 INFO:teuthology.orchestra.run.smithi022.stdout: Package Arch Version Repository Size 2024-02-16T19:16:49.108 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-02-16T19:16:49.108 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2024-02-16T19:16:49.108 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.0-0.el9 ceph-noarch 7.4 M 2024-02-16T19:16:49.108 INFO:teuthology.orchestra.run.smithi022.stdout:Installing dependencies: 2024-02-16T19:16:49.108 INFO:teuthology.orchestra.run.smithi022.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-02-16T19:16:49.108 INFO:teuthology.orchestra.run.smithi022.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-02-16T19:16:49.108 INFO:teuthology.orchestra.run.smithi022.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-02-16T19:16:49.109 INFO:teuthology.orchestra.run.smithi022.stdout: libgfortran x86_64 11.4.1-3.el9 CentOS-BaseOS 802 k 2024-02-16T19:16:49.109 INFO:teuthology.orchestra.run.smithi022.stdout: libquadmath x86_64 11.4.1-3.el9 CentOS-BaseOS 194 k 2024-02-16T19:16:49.109 INFO:teuthology.orchestra.run.smithi022.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-02-16T19:16:49.109 INFO:teuthology.orchestra.run.smithi022.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-02-16T19:16:49.109 INFO:teuthology.orchestra.run.smithi022.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-02-16T19:16:49.109 INFO:teuthology.orchestra.run.smithi022.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-02-16T19:16:49.109 INFO:teuthology.orchestra.run.smithi022.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-02-16T19:16:49.109 INFO:teuthology.orchestra.run.smithi022.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-02-16T19:16:49.109 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:16:49.109 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2024-02-16T19:16:49.109 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-02-16T19:16:49.109 INFO:teuthology.orchestra.run.smithi022.stdout:Install 12 Packages 2024-02-16T19:16:49.109 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:16:49.110 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 37 M 2024-02-16T19:16:49.110 INFO:teuthology.orchestra.run.smithi022.stdout:Installed size: 209 M 2024-02-16T19:16:49.110 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading Packages: 2024-02-16T19:16:49.404 INFO:teuthology.orchestra.run.smithi022.stdout:(1/12): ceph-mgr-diskprediction-local-18.2.0-0. 51 MB/s | 7.4 MB 00:00 2024-02-16T19:16:49.562 INFO:teuthology.orchestra.run.smithi022.stdout:(2/12): flexiblas-3.0.4-8.el9.x86_64.rpm 104 kB/s | 32 kB 00:00 2024-02-16T19:16:49.621 INFO:teuthology.orchestra.run.smithi022.stdout:(3/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 102 kB/s | 17 kB 00:00 2024-02-16T19:16:49.655 INFO:teuthology.orchestra.run.smithi022.stdout:(4/12): openblas-0.3.21-2.el9.x86_64.rpm 376 kB/s | 34 kB 00:00 2024-02-16T19:16:49.955 INFO:teuthology.orchestra.run.smithi022.stdout:(5/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 4.3 MB/s | 3.0 MB 00:00 2024-02-16T19:16:50.180 INFO:teuthology.orchestra.run.smithi022.stdout:(6/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 8.5 MB/s | 4.7 MB 00:00 2024-02-16T19:16:50.239 INFO:teuthology.orchestra.run.smithi022.stdout:(7/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 8.2 MB/s | 4.8 MB 00:00 2024-02-16T19:16:50.264 INFO:teuthology.orchestra.run.smithi022.stdout:(8/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 704 kB/s | 217 kB 00:00 2024-02-16T19:16:50.760 INFO:teuthology.orchestra.run.smithi117.stdout:lab-extras 16 kB/s | 1.7 kB 00:00 2024-02-16T19:16:50.806 INFO:teuthology.orchestra.run.smithi022.stdout:(9/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 25 MB/s | 16 MB 00:00 2024-02-16T19:16:50.831 INFO:teuthology.orchestra.run.smithi022.stdout:(10/12): libquadmath-11.4.1-3.el9.x86_64.rpm 344 kB/s | 194 kB 00:00 2024-02-16T19:16:50.856 INFO:teuthology.orchestra.run.smithi022.stdout:(11/12): libgfortran-11.4.1-3.el9.x86_64.rpm 1.3 MB/s | 802 kB 00:00 2024-02-16T19:16:50.973 INFO:teuthology.orchestra.run.smithi022.stdout:(12/12): python3-devel-3.9.17-2.el9.x86_64.rpm 1.4 MB/s | 245 kB 00:00 2024-02-16T19:16:50.976 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2024-02-16T19:16:50.976 INFO:teuthology.orchestra.run.smithi022.stdout:Total 20 MB/s | 37 MB 00:01 2024-02-16T19:16:50.997 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2024-02-16T19:16:51.023 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction check succeeded. 2024-02-16T19:16:51.023 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2024-02-16T19:16:51.473 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded. 2024-02-16T19:16:51.474 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2024-02-16T19:16:52.187 INFO:teuthology.orchestra.run.smithi022.stdout: Preparing : 1/1 2024-02-16T19:16:52.272 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 1/12 2024-02-16T19:16:52.302 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-02-16T19:16:52.411 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2024-02-16T19:16:52.456 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-02-16T19:16:52.645 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-02-16T19:16:52.680 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-02-16T19:16:52.760 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-02-16T19:16:53.392 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-02-16T19:16:53.409 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-02-16T19:16:53.409 INFO:teuthology.orchestra.run.smithi117.stdout: Package Arch Version Repository Size 2024-02-16T19:16:53.409 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-02-16T19:16:53.409 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2024-02-16T19:16:53.409 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-radosgw x86_64 2:18.2.0-0.el9 ceph 7.6 M 2024-02-16T19:16:53.409 INFO:teuthology.orchestra.run.smithi117.stdout:Upgrading: 2024-02-16T19:16:53.410 INFO:teuthology.orchestra.run.smithi117.stdout: librados2 x86_64 2:18.2.0-0.el9 ceph 3.3 M 2024-02-16T19:16:53.410 INFO:teuthology.orchestra.run.smithi117.stdout: librbd1 x86_64 2:18.2.0-0.el9 ceph 3.0 M 2024-02-16T19:16:53.410 INFO:teuthology.orchestra.run.smithi117.stdout:Installing dependencies: 2024-02-16T19:16:53.410 INFO:teuthology.orchestra.run.smithi117.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-02-16T19:16:53.410 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-base x86_64 2:18.2.0-0.el9 ceph 5.2 M 2024-02-16T19:16:53.410 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-common x86_64 2:18.2.0-0.el9 ceph 18 M 2024-02-16T19:16:53.410 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-selinux x86_64 2:18.2.0-0.el9 ceph 24 k 2024-02-16T19:16:53.410 INFO:teuthology.orchestra.run.smithi117.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-02-16T19:16:53.410 INFO:teuthology.orchestra.run.smithi117.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-02-16T19:16:53.410 INFO:teuthology.orchestra.run.smithi117.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-02-16T19:16:53.410 INFO:teuthology.orchestra.run.smithi117.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-02-16T19:16:53.410 INFO:teuthology.orchestra.run.smithi117.stdout: libcephfs2 x86_64 2:18.2.0-0.el9 ceph 653 k 2024-02-16T19:16:53.410 INFO:teuthology.orchestra.run.smithi117.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-02-16T19:16:53.410 INFO:teuthology.orchestra.run.smithi117.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-02-16T19:16:53.410 INFO:teuthology.orchestra.run.smithi117.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-02-16T19:16:53.411 INFO:teuthology.orchestra.run.smithi117.stdout: libradosstriper1 x86_64 2:18.2.0-0.el9 ceph 474 k 2024-02-16T19:16:53.411 INFO:teuthology.orchestra.run.smithi117.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-02-16T19:16:53.411 INFO:teuthology.orchestra.run.smithi117.stdout: librgw2 x86_64 2:18.2.0-0.el9 ceph 4.4 M 2024-02-16T19:16:53.411 INFO:teuthology.orchestra.run.smithi117.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-02-16T19:16:53.411 INFO:teuthology.orchestra.run.smithi117.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-02-16T19:16:53.411 INFO:teuthology.orchestra.run.smithi117.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-02-16T19:16:53.411 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ceph-argparse x86_64 2:18.2.0-0.el9 ceph 45 k 2024-02-16T19:16:53.411 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ceph-common x86_64 2:18.2.0-0.el9 ceph 119 k 2024-02-16T19:16:53.411 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cephfs x86_64 2:18.2.0-0.el9 ceph 155 k 2024-02-16T19:16:53.411 INFO:teuthology.orchestra.run.smithi117.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-02-16T19:16:53.411 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rados x86_64 2:18.2.0-0.el9 ceph 321 k 2024-02-16T19:16:53.411 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rbd x86_64 2:18.2.0-0.el9 ceph 297 k 2024-02-16T19:16:53.411 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rgw x86_64 2:18.2.0-0.el9 ceph 99 k 2024-02-16T19:16:53.411 INFO:teuthology.orchestra.run.smithi117.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-02-16T19:16:53.411 INFO:teuthology.orchestra.run.smithi117.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-02-16T19:16:53.412 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:16:53.412 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-02-16T19:16:53.412 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-02-16T19:16:53.412 INFO:teuthology.orchestra.run.smithi117.stdout:Install 28 Packages 2024-02-16T19:16:53.412 INFO:teuthology.orchestra.run.smithi117.stdout:Upgrade 2 Packages 2024-02-16T19:16:53.412 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:16:53.412 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 53 M 2024-02-16T19:16:53.413 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2024-02-16T19:16:53.467 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-02-16T19:16:53.580 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-02-16T19:16:53.778 INFO:teuthology.orchestra.run.smithi117.stdout:(1/30): ceph-base-18.2.0-0.el9.x86_64.rpm 29 MB/s | 5.2 MB 00:00 2024-02-16T19:16:53.803 INFO:teuthology.orchestra.run.smithi117.stdout:(2/30): ceph-selinux-18.2.0-0.el9.x86_64.rpm 974 kB/s | 24 kB 00:00 2024-02-16T19:16:53.845 INFO:teuthology.orchestra.run.smithi117.stdout:(3/30): libcephfs2-18.2.0-0.el9.x86_64.rpm 15 MB/s | 653 kB 00:00 2024-02-16T19:16:53.878 INFO:teuthology.orchestra.run.smithi117.stdout:(4/30): libradosstriper1-18.2.0-0.el9.x86_64.rp 14 MB/s | 474 kB 00:00 2024-02-16T19:16:54.004 INFO:teuthology.orchestra.run.smithi117.stdout:(5/30): ceph-radosgw-18.2.0-0.el9.x86_64.rpm 19 MB/s | 7.6 MB 00:00 2024-02-16T19:16:54.029 INFO:teuthology.orchestra.run.smithi117.stdout:(6/30): python3-ceph-argparse-18.2.0-0.el9.x86_ 1.8 MB/s | 45 kB 00:00 2024-02-16T19:16:54.087 INFO:teuthology.orchestra.run.smithi117.stdout:(7/30): librgw2-18.2.0-0.el9.x86_64.rpm 21 MB/s | 4.4 MB 00:00 2024-02-16T19:16:54.113 INFO:teuthology.orchestra.run.smithi117.stdout:(8/30): python3-ceph-common-18.2.0-0.el9.x86_64 1.4 MB/s | 119 kB 00:00 2024-02-16T19:16:54.138 INFO:teuthology.orchestra.run.smithi117.stdout:(9/30): python3-cephfs-18.2.0-0.el9.x86_64.rpm 6.2 MB/s | 155 kB 00:00 2024-02-16T19:16:54.163 INFO:teuthology.orchestra.run.smithi117.stdout:(10/30): python3-rados-18.2.0-0.el9.x86_64.rpm 6.3 MB/s | 321 kB 00:00 2024-02-16T19:16:54.188 INFO:teuthology.orchestra.run.smithi117.stdout:(11/30): python3-rbd-18.2.0-0.el9.x86_64.rpm 5.8 MB/s | 297 kB 00:00 2024-02-16T19:16:54.214 INFO:teuthology.orchestra.run.smithi117.stdout:(12/30): python3-rgw-18.2.0-0.el9.x86_64.rpm 1.9 MB/s | 99 kB 00:00 2024-02-16T19:16:54.531 INFO:teuthology.orchestra.run.smithi117.stdout:(13/30): ceph-common-18.2.0-0.el9.x86_64.rpm 20 MB/s | 18 MB 00:00 2024-02-16T19:16:54.581 INFO:teuthology.orchestra.run.smithi117.stdout:(14/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 916 kB/s | 44 kB 00:00 2024-02-16T19:16:54.606 INFO:teuthology.orchestra.run.smithi117.stdout:(15/30): boost-program-options-1.75.0-8.el9.x86 253 kB/s | 106 kB 00:00 2024-02-16T19:16:54.631 INFO:teuthology.orchestra.run.smithi117.stdout:(16/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 381 kB/s | 159 kB 00:00 2024-02-16T19:16:54.681 INFO:teuthology.orchestra.run.smithi117.stdout:(17/30): librdkafka-1.6.1-102.el9.x86_64.rpm 6.5 MB/s | 661 kB 00:00 2024-02-16T19:16:54.698 INFO:teuthology.orchestra.run.smithi117.stdout:(18/30): python3-prettytable-0.7.2-27.el9.noarc 622 kB/s | 41 kB 00:00 2024-02-16T19:16:54.724 INFO:teuthology.orchestra.run.smithi117.stdout:(19/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.0 MB/s | 282 kB 00:00 2024-02-16T19:16:54.909 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-02-16T19:16:55.049 INFO:teuthology.orchestra.run.smithi117.stdout:(20/30): fmt-8.1.1-5.el9.x86_64.rpm 302 kB/s | 111 kB 00:00 2024-02-16T19:16:55.124 INFO:teuthology.orchestra.run.smithi117.stdout:(21/30): libarrow-doc-9.0.0-10.el9.noarch.rpm 346 kB/s | 26 kB 00:00 2024-02-16T19:16:55.166 INFO:teuthology.orchestra.run.smithi117.stdout:(22/30): gperftools-libs-2.9.1-2.el9.x86_64.rpm 661 kB/s | 309 kB 00:00 2024-02-16T19:16:55.200 INFO:teuthology.orchestra.run.smithi117.stdout:(23/30): liboath-2.6.7-2.el9.x86_64.rpm 646 kB/s | 49 kB 00:00 2024-02-16T19:16:55.250 INFO:teuthology.orchestra.run.smithi117.stdout:(24/30): libunwind-1.6.2-1.el9.x86_64.rpm 806 kB/s | 67 kB 00:00 2024-02-16T19:16:55.350 INFO:teuthology.orchestra.run.smithi117.stdout:(25/30): re2-20211101-3.el9.x86_64.rpm 1.9 MB/s | 192 kB 00:00 2024-02-16T19:16:55.425 INFO:teuthology.orchestra.run.smithi117.stdout:(26/30): libarrow-9.0.0-10.el9.x86_64.rpm 6.3 MB/s | 4.4 MB 00:00 2024-02-16T19:16:55.467 INFO:teuthology.orchestra.run.smithi117.stdout:(27/30): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.1 MB/s | 838 kB 00:00 2024-02-16T19:16:55.543 INFO:teuthology.orchestra.run.smithi117.stdout:(28/30): librbd1-18.2.0-0.el9.x86_64.rpm 40 MB/s | 3.0 MB 00:00 2024-02-16T19:16:55.593 INFO:teuthology.orchestra.run.smithi117.stdout:(29/30): librados2-18.2.0-0.el9.x86_64.rpm 20 MB/s | 3.3 MB 00:00 2024-02-16T19:16:55.635 INFO:teuthology.orchestra.run.smithi117.stdout:(30/30): thrift-0.15.0-2.el9.x86_64.rpm 5.6 MB/s | 1.6 MB 00:00 2024-02-16T19:16:55.642 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-02-16T19:16:55.642 INFO:teuthology.orchestra.run.smithi117.stdout:Total 24 MB/s | 53 MB 00:02 2024-02-16T19:16:55.642 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-02-16T19:16:55.677 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-02-16T19:16:55.677 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-02-16T19:16:56.002 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-02-16T19:16:56.029 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.0-0.el9.noarc 12/12 2024-02-16T19:16:56.134 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-02-16T19:16:56.134 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-02-16T19:16:57.504 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-02-16T19:16:57.585 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2024-02-16T19:16:57.614 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2024-02-16T19:16:57.665 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2024-02-16T19:16:58.027 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2024-02-16T19:16:58.063 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : librados2-2:18.2.0-0.el9.x86_64 5/32 2024-02-16T19:16:58.138 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librados2-2:18.2.0-0.el9.x86_64 5/32 2024-02-16T19:16:58.219 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-rados-2:18.2.0-0.el9.x86_64 6/32 2024-02-16T19:16:58.248 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libcephfs2-2:18.2.0-0.el9.x86_64 7/32 2024-02-16T19:16:58.294 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: libcephfs2-2:18.2.0-0.el9.x86_64 7/32 2024-02-16T19:16:58.322 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2024-02-16T19:16:58.371 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-ceph-argparse-2:18.2.0-0.el9.x86_64 9/32 2024-02-16T19:16:58.440 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cephfs-2:18.2.0-0.el9.x86_64 10/32 2024-02-16T19:16:58.467 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libradosstriper1-2:18.2.0-0.el9.x86_64 11/32 2024-02-16T19:16:58.516 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: libradosstriper1-2:18.2.0-0.el9.x86_64 11/32 2024-02-16T19:16:58.518 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.0-0.el9.noarc 12/12 2024-02-16T19:16:58.518 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.0-0.el9.noarc 1/12 2024-02-16T19:16:58.518 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2024-02-16T19:16:58.518 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2024-02-16T19:16:58.518 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2024-02-16T19:16:58.518 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2024-02-16T19:16:58.518 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-02-16T19:16:58.519 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2024-02-16T19:16:58.519 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2024-02-16T19:16:58.519 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2024-02-16T19:16:58.519 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 10/12 2024-02-16T19:16:58.519 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 11/12 2024-02-16T19:16:58.623 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : re2-1:20211101-3.el9.x86_64 12/32 2024-02-16T19:16:58.668 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 13/32 2024-02-16T19:16:58.707 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2024-02-16T19:16:58.739 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/32 2024-02-16T19:16:58.766 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 16/32 2024-02-16T19:16:58.802 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2024-02-16T19:16:58.834 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2024-02-16T19:16:59.131 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2024-02-16T19:16:59.140 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 12/12 2024-02-16T19:16:59.140 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:16:59.140 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2024-02-16T19:16:59.141 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-diskprediction-local-2:18.2.0-0.el9.noarch 2024-02-16T19:16:59.141 INFO:teuthology.orchestra.run.smithi022.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-02-16T19:16:59.141 INFO:teuthology.orchestra.run.smithi022.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-02-16T19:16:59.141 INFO:teuthology.orchestra.run.smithi022.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-02-16T19:16:59.141 INFO:teuthology.orchestra.run.smithi022.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-02-16T19:16:59.141 INFO:teuthology.orchestra.run.smithi022.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-02-16T19:16:59.141 INFO:teuthology.orchestra.run.smithi022.stdout: openblas-0.3.21-2.el9.x86_64 2024-02-16T19:16:59.141 INFO:teuthology.orchestra.run.smithi022.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-02-16T19:16:59.141 INFO:teuthology.orchestra.run.smithi022.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-02-16T19:16:59.141 INFO:teuthology.orchestra.run.smithi022.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-02-16T19:16:59.141 INFO:teuthology.orchestra.run.smithi022.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-02-16T19:16:59.141 INFO:teuthology.orchestra.run.smithi022.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-02-16T19:16:59.141 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:16:59.141 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-02-16T19:16:59.156 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : librbd1-2:18.2.0-0.el9.x86_64 20/32 2024-02-16T19:16:59.229 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librbd1-2:18.2.0-0.el9.x86_64 20/32 2024-02-16T19:16:59.260 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-rbd-2:18.2.0-0.el9.x86_64 21/32 2024-02-16T19:16:59.302 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2024-02-16T19:16:59.516 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install ceph-mgr-rook 2024-02-16T19:16:59.689 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 23/32 2024-02-16T19:16:59.717 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : librgw2-2:18.2.0-0.el9.x86_64 24/32 2024-02-16T19:16:59.780 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librgw2-2:18.2.0-0.el9.x86_64 24/32 2024-02-16T19:16:59.837 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-rgw-2:18.2.0-0.el9.x86_64 25/32 2024-02-16T19:16:59.861 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-ceph-common-2:18.2.0-0.el9.x86_64 26/32 2024-02-16T19:17:00.109 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:01:11 ago on Fri 16 Feb 2024 07:15:49 PM UTC. 2024-02-16T19:17:00.283 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-02-16T19:17:00.284 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-02-16T19:17:00.284 INFO:teuthology.orchestra.run.smithi022.stdout: Package Arch Version Repository Size 2024-02-16T19:17:00.284 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-02-16T19:17:00.284 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2024-02-16T19:17:00.284 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-rook noarch 2:18.2.0-0.el9 ceph-noarch 47 k 2024-02-16T19:17:00.284 INFO:teuthology.orchestra.run.smithi022.stdout:Installing dependencies: 2024-02-16T19:17:00.284 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-02-16T19:17:00.285 INFO:teuthology.orchestra.run.smithi022.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-02-16T19:17:00.285 INFO:teuthology.orchestra.run.smithi022.stdout: python3-google-auth noarch 1:2.27.0-1.el9 epel 203 k 2024-02-16T19:17:00.285 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-02-16T19:17:00.285 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-02-16T19:17:00.285 INFO:teuthology.orchestra.run.smithi022.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-02-16T19:17:00.285 INFO:teuthology.orchestra.run.smithi022.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-02-16T19:17:00.285 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-02-16T19:17:00.285 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-02-16T19:17:00.285 INFO:teuthology.orchestra.run.smithi022.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-02-16T19:17:00.285 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-02-16T19:17:00.285 INFO:teuthology.orchestra.run.smithi022.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-02-16T19:17:00.285 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:17:00.285 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2024-02-16T19:17:00.285 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-02-16T19:17:00.286 INFO:teuthology.orchestra.run.smithi022.stdout:Install 13 Packages 2024-02-16T19:17:00.286 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:17:00.286 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 2.0 M 2024-02-16T19:17:00.286 INFO:teuthology.orchestra.run.smithi022.stdout:Installed size: 26 M 2024-02-16T19:17:00.286 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading Packages: 2024-02-16T19:17:00.471 INFO:teuthology.orchestra.run.smithi022.stdout:(1/13): ceph-mgr-rook-18.2.0-0.el9.noarch.rpm 417 kB/s | 47 kB 00:00 2024-02-16T19:17:00.495 INFO:teuthology.orchestra.run.smithi022.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 176 kB/s | 24 kB 00:00 2024-02-16T19:17:00.529 INFO:teuthology.orchestra.run.smithi022.stdout:(3/13): python3-jsonpointer-2.0-4.el9.noarch.rp 105 kB/s | 18 kB 00:00 2024-02-16T19:17:00.596 INFO:teuthology.orchestra.run.smithi022.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.4 MB/s | 174 kB 00:00 2024-02-16T19:17:00.621 INFO:teuthology.orchestra.run.smithi022.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.0 MB/s | 132 kB 00:00 2024-02-16T19:17:00.647 INFO:teuthology.orchestra.run.smithi022.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.8 MB/s | 211 kB 00:00 2024-02-16T19:17:00.672 INFO:teuthology.orchestra.run.smithi022.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 567 kB/s | 43 kB 00:00 2024-02-16T19:17:00.897 INFO:teuthology.orchestra.run.smithi022.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 117 kB/s | 32 kB 00:00 2024-02-16T19:17:00.923 INFO:teuthology.orchestra.run.smithi022.stdout:(9/13): python3-certifi-2021.10.8-2.el9.noarch. 54 kB/s | 15 kB 00:00 2024-02-16T19:17:01.056 INFO:teuthology.orchestra.run.smithi022.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 443 kB/s | 59 kB 00:00 2024-02-16T19:17:01.081 INFO:teuthology.orchestra.run.smithi022.stdout:(11/13): python3-google-auth-2.27.0-1.el9.noarc 495 kB/s | 203 kB 00:00 2024-02-16T19:17:01.168 INFO:teuthology.orchestra.run.smithi022.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 806 kB/s | 90 kB 00:00 2024-02-16T19:17:01.515 INFO:teuthology.orchestra.run.smithi022.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 1.7 MB/s | 1.0 MB 00:00 2024-02-16T19:17:01.519 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2024-02-16T19:17:01.519 INFO:teuthology.orchestra.run.smithi022.stdout:Total 1.7 MB/s | 2.0 MB 00:01 2024-02-16T19:17:01.520 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2024-02-16T19:17:01.534 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction check succeeded. 2024-02-16T19:17:01.534 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2024-02-16T19:17:01.639 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded. 2024-02-16T19:17:01.641 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2024-02-16T19:17:01.967 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-common-2:18.2.0-0.el9.x86_64 27/32 2024-02-16T19:17:02.007 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-common-2:18.2.0-0.el9.x86_64 27/32 2024-02-16T19:17:02.165 INFO:teuthology.orchestra.run.smithi022.stdout: Preparing : 1/1 2024-02-16T19:17:02.338 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-02-16T19:17:02.402 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-02-16T19:17:02.466 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-02-16T19:17:02.501 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-02-16T19:17:02.543 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-02-16T19:17:02.572 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-common-2:18.2.0-0.el9.x86_64 27/32 2024-02-16T19:17:02.610 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-base-2:18.2.0-0.el9.x86_64 28/32 2024-02-16T19:17:02.641 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-02-16T19:17:02.705 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-base-2:18.2.0-0.el9.x86_64 28/32 2024-02-16T19:17:02.705 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-02-16T19:17:02.705 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-02-16T19:17:02.706 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:17:02.745 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-selinux-2:18.2.0-0.el9.x86_64 29/32 2024-02-16T19:17:02.750 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-google-auth-1:2.27.0-1.el9.noarch 7/13 2024-02-16T19:17:02.804 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-02-16T19:17:03.574 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-02-16T19:17:03.624 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-02-16T19:17:03.658 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-02-16T19:17:03.696 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-02-16T19:17:03.719 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mgr-rook-2:18.2.0-0.el9.noarch 13/13 2024-02-16T19:17:04.546 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.0-0.el9.noarch 13/13 2024-02-16T19:17:04.546 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mgr-rook-2:18.2.0-0.el9.noarch 1/13 2024-02-16T19:17:04.546 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-02-16T19:17:04.546 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-02-16T19:17:04.547 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-02-16T19:17:04.547 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-02-16T19:17:04.547 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-02-16T19:17:04.547 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-02-16T19:17:04.547 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-02-16T19:17:04.547 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-02-16T19:17:04.547 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-google-auth-1:2.27.0-1.el9.noarch 10/13 2024-02-16T19:17:04.547 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-02-16T19:17:04.547 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-02-16T19:17:04.932 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-02-16T19:17:04.932 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:17:04.932 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2024-02-16T19:17:04.932 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-rook-2:18.2.0-0.el9.noarch 2024-02-16T19:17:04.932 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-02-16T19:17:04.932 INFO:teuthology.orchestra.run.smithi022.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-02-16T19:17:04.932 INFO:teuthology.orchestra.run.smithi022.stdout: python3-google-auth-1:2.27.0-1.el9.noarch 2024-02-16T19:17:04.932 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-02-16T19:17:04.932 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-02-16T19:17:04.932 INFO:teuthology.orchestra.run.smithi022.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-02-16T19:17:04.932 INFO:teuthology.orchestra.run.smithi022.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-02-16T19:17:04.932 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-02-16T19:17:04.932 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-02-16T19:17:04.932 INFO:teuthology.orchestra.run.smithi022.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-02-16T19:17:04.933 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rsa-4.9-2.el9.noarch 2024-02-16T19:17:04.933 INFO:teuthology.orchestra.run.smithi022.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-02-16T19:17:04.933 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:17:04.933 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-02-16T19:17:05.183 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install ceph-mgr-cephadm 2024-02-16T19:17:05.774 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:01:16 ago on Fri 16 Feb 2024 07:15:49 PM UTC. 2024-02-16T19:17:05.949 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-02-16T19:17:05.950 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-02-16T19:17:05.950 INFO:teuthology.orchestra.run.smithi022.stdout: Package Arch Version Repository Size 2024-02-16T19:17:05.950 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-02-16T19:17:05.950 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2024-02-16T19:17:05.951 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-cephadm noarch 2:18.2.0-0.el9 ceph-noarch 127 k 2024-02-16T19:17:05.951 INFO:teuthology.orchestra.run.smithi022.stdout:Installing dependencies: 2024-02-16T19:17:05.951 INFO:teuthology.orchestra.run.smithi022.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2024-02-16T19:17:05.951 INFO:teuthology.orchestra.run.smithi022.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-02-16T19:17:05.951 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jinja2 noarch 2.11.3-5.el9 CentOS-AppStream 227 k 2024-02-16T19:17:05.951 INFO:teuthology.orchestra.run.smithi022.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-02-16T19:17:05.951 INFO:teuthology.orchestra.run.smithi022.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-02-16T19:17:05.951 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:17:05.951 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2024-02-16T19:17:05.951 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-02-16T19:17:05.951 INFO:teuthology.orchestra.run.smithi022.stdout:Install 6 Packages 2024-02-16T19:17:05.951 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:17:05.952 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 6.8 M 2024-02-16T19:17:05.952 INFO:teuthology.orchestra.run.smithi022.stdout:Installed size: 33 M 2024-02-16T19:17:05.952 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading Packages: 2024-02-16T19:17:06.285 INFO:teuthology.orchestra.run.smithi022.stdout:(1/6): ceph-mgr-cephadm-18.2.0-0.el9.noarch.rpm 1.7 MB/s | 127 kB 00:00 2024-02-16T19:17:06.543 INFO:teuthology.orchestra.run.smithi022.stdout:(2/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 688 kB/s | 227 kB 00:00 2024-02-16T19:17:06.568 INFO:teuthology.orchestra.run.smithi022.stdout:(3/6): python3-asyncssh-2.13.2-3.el9.noarch.rpm 2.3 MB/s | 548 kB 00:00 2024-02-16T19:17:06.635 INFO:teuthology.orchestra.run.smithi022.stdout:(4/6): python3-typing-extensions-4.4.0-2.el9.no 775 kB/s | 51 kB 00:00 2024-02-16T19:17:06.669 INFO:teuthology.orchestra.run.smithi022.stdout:(5/6): python3-natsort-7.1.1-5.el9.noarch.rpm 463 kB/s | 58 kB 00:00 2024-02-16T19:17:06.761 INFO:teuthology.orchestra.run.smithi022.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 11 MB/s | 5.8 MB 00:00 2024-02-16T19:17:06.763 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2024-02-16T19:17:06.763 INFO:teuthology.orchestra.run.smithi022.stdout:Total 8.3 MB/s | 6.8 MB 00:00 2024-02-16T19:17:06.763 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2024-02-16T19:17:06.773 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction check succeeded. 2024-02-16T19:17:06.773 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2024-02-16T19:17:06.866 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded. 2024-02-16T19:17:06.866 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2024-02-16T19:17:07.167 INFO:teuthology.orchestra.run.smithi022.stdout: Preparing : 1/1 2024-02-16T19:17:07.350 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-02-16T19:17:07.422 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 2/6 2024-02-16T19:17:07.973 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-02-16T19:17:08.059 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-02-16T19:17:08.126 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-02-16T19:17:08.150 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mgr-cephadm-2:18.2.0-0.el9.noarch 6/6 2024-02-16T19:17:08.969 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.0-0.el9.noarch 6/6 2024-02-16T19:17:08.969 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mgr-cephadm-2:18.2.0-0.el9.noarch 1/6 2024-02-16T19:17:08.969 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-02-16T19:17:08.969 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-02-16T19:17:08.969 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 4/6 2024-02-16T19:17:08.969 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-02-16T19:17:09.286 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-02-16T19:17:09.286 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:17:09.287 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2024-02-16T19:17:09.287 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-cephadm-2:18.2.0-0.el9.noarch 2024-02-16T19:17:09.287 INFO:teuthology.orchestra.run.smithi022.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2024-02-16T19:17:09.287 INFO:teuthology.orchestra.run.smithi022.stdout: python3-babel-2.9.1-2.el9.noarch 2024-02-16T19:17:09.287 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-02-16T19:17:09.287 INFO:teuthology.orchestra.run.smithi022.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-02-16T19:17:09.287 INFO:teuthology.orchestra.run.smithi022.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-02-16T19:17:09.287 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:17:09.287 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-02-16T19:17:09.561 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install ceph-fuse 2024-02-16T19:17:10.150 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:01:21 ago on Fri 16 Feb 2024 07:15:49 PM UTC. 2024-02-16T19:17:10.325 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-02-16T19:17:10.325 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-02-16T19:17:10.325 INFO:teuthology.orchestra.run.smithi022.stdout: Package Architecture Version Repository Size 2024-02-16T19:17:10.325 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-02-16T19:17:10.325 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2024-02-16T19:17:10.325 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-fuse x86_64 2:18.2.0-0.el9 ceph 835 k 2024-02-16T19:17:10.325 INFO:teuthology.orchestra.run.smithi022.stdout:Installing dependencies: 2024-02-16T19:17:10.326 INFO:teuthology.orchestra.run.smithi022.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-02-16T19:17:10.326 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:17:10.326 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2024-02-16T19:17:10.326 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-02-16T19:17:10.326 INFO:teuthology.orchestra.run.smithi022.stdout:Install 2 Packages 2024-02-16T19:17:10.326 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:17:10.326 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 913 k 2024-02-16T19:17:10.326 INFO:teuthology.orchestra.run.smithi022.stdout:Installed size: 2.7 M 2024-02-16T19:17:10.326 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading Packages: 2024-02-16T19:17:10.421 INFO:teuthology.orchestra.run.smithi022.stdout:(1/2): ceph-fuse-18.2.0-0.el9.x86_64.rpm 8.6 MB/s | 835 kB 00:00 2024-02-16T19:17:10.479 INFO:teuthology.orchestra.run.smithi022.stdout:(2/2): fuse-2.9.9-15.el9.x86_64.rpm 514 kB/s | 79 kB 00:00 2024-02-16T19:17:10.481 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2024-02-16T19:17:10.481 INFO:teuthology.orchestra.run.smithi022.stdout:Total 5.8 MB/s | 913 kB 00:00 2024-02-16T19:17:10.481 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2024-02-16T19:17:10.502 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction check succeeded. 2024-02-16T19:17:10.503 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2024-02-16T19:17:10.586 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded. 2024-02-16T19:17:10.586 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2024-02-16T19:17:10.779 INFO:teuthology.orchestra.run.smithi022.stdout: Preparing : 1/1 2024-02-16T19:17:10.906 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-02-16T19:17:10.982 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-fuse-2:18.2.0-0.el9.x86_64 2/2 2024-02-16T19:17:11.672 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-fuse-2:18.2.0-0.el9.x86_64 2/2 2024-02-16T19:17:11.672 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-fuse-2:18.2.0-0.el9.x86_64 1/2 2024-02-16T19:17:11.993 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-02-16T19:17:11.994 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:17:11.994 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2024-02-16T19:17:11.994 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-fuse-2:18.2.0-0.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-02-16T19:17:11.994 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:17:11.994 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-02-16T19:17:12.202 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-selinux-2:18.2.0-0.el9.x86_64 29/32 2024-02-16T19:17:12.202 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /sys 2024-02-16T19:17:12.202 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /proc 2024-02-16T19:17:12.202 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /mnt 2024-02-16T19:17:12.202 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /var/tmp 2024-02-16T19:17:12.202 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /home 2024-02-16T19:17:12.202 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /root 2024-02-16T19:17:12.202 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /tmp 2024-02-16T19:17:12.202 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:17:12.285 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install librados-devel 2024-02-16T19:17:12.881 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:01:23 ago on Fri 16 Feb 2024 07:15:49 PM UTC. 2024-02-16T19:17:12.971 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-radosgw-2:18.2.0-0.el9.x86_64 30/32 2024-02-16T19:17:12.995 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-radosgw-2:18.2.0-0.el9.x86_64 30/32 2024-02-16T19:17:12.995 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-16T19:17:12.995 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-02-16T19:17:12.995 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-16T19:17:12.995 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-16T19:17:12.995 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:17:12.997 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-02-16T19:17:13.055 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-02-16T19:17:13.055 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-02-16T19:17:13.055 INFO:teuthology.orchestra.run.smithi022.stdout: Package Architecture Version Repository Size 2024-02-16T19:17:13.055 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-02-16T19:17:13.055 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2024-02-16T19:17:13.055 INFO:teuthology.orchestra.run.smithi022.stdout: librados-devel x86_64 2:18.2.0-0.el9 ceph 126 k 2024-02-16T19:17:13.056 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:17:13.056 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2024-02-16T19:17:13.056 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-02-16T19:17:13.056 INFO:teuthology.orchestra.run.smithi022.stdout:Install 1 Package 2024-02-16T19:17:13.056 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:17:13.056 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 126 k 2024-02-16T19:17:13.056 INFO:teuthology.orchestra.run.smithi022.stdout:Installed size: 456 k 2024-02-16T19:17:13.056 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading Packages: 2024-02-16T19:17:13.090 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-02-16T19:17:13.091 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2024-02-16T19:17:13.126 INFO:teuthology.orchestra.run.smithi022.stdout:librados-devel-18.2.0-0.el9.x86_64.rpm 1.8 MB/s | 126 kB 00:00 2024-02-16T19:17:13.126 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2024-02-16T19:17:13.127 INFO:teuthology.orchestra.run.smithi022.stdout:Total 1.7 MB/s | 126 kB 00:00 2024-02-16T19:17:13.127 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2024-02-16T19:17:13.132 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction check succeeded. 2024-02-16T19:17:13.132 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2024-02-16T19:17:13.189 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded. 2024-02-16T19:17:13.189 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2024-02-16T19:17:13.343 INFO:teuthology.orchestra.run.smithi022.stdout: Preparing : 1/1 2024-02-16T19:17:13.443 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : librados-devel-2:18.2.0-0.el9.x86_64 1/1 2024-02-16T19:17:13.847 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: librados-devel-2:18.2.0-0.el9.x86_64 1/1 2024-02-16T19:17:14.154 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2024-02-16T19:17:14.154 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-base-2:18.2.0-0.el9.x86_64 1/32 2024-02-16T19:17:14.154 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-common-2:18.2.0-0.el9.x86_64 2/32 2024-02-16T19:17:14.154 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-radosgw-2:18.2.0-0.el9.x86_64 3/32 2024-02-16T19:17:14.154 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-selinux-2:18.2.0-0.el9.x86_64 4/32 2024-02-16T19:17:14.154 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libcephfs2-2:18.2.0-0.el9.x86_64 5/32 2024-02-16T19:17:14.154 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libradosstriper1-2:18.2.0-0.el9.x86_64 6/32 2024-02-16T19:17:14.154 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librgw2-2:18.2.0-0.el9.x86_64 7/32 2024-02-16T19:17:14.155 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-ceph-argparse-2:18.2.0-0.el9.x86_64 8/32 2024-02-16T19:17:14.155 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-ceph-common-2:18.2.0-0.el9.x86_64 9/32 2024-02-16T19:17:14.155 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cephfs-2:18.2.0-0.el9.x86_64 10/32 2024-02-16T19:17:14.155 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-rados-2:18.2.0-0.el9.x86_64 11/32 2024-02-16T19:17:14.155 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-rbd-2:18.2.0-0.el9.x86_64 12/32 2024-02-16T19:17:14.155 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-rgw-2:18.2.0-0.el9.x86_64 13/32 2024-02-16T19:17:14.155 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2024-02-16T19:17:14.155 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2024-02-16T19:17:14.155 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2024-02-16T19:17:14.155 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2024-02-16T19:17:14.155 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2024-02-16T19:17:14.155 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2024-02-16T19:17:14.155 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2024-02-16T19:17:14.155 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 21/32 2024-02-16T19:17:14.155 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 22/32 2024-02-16T19:17:14.156 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 23/32 2024-02-16T19:17:14.156 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2024-02-16T19:17:14.156 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2024-02-16T19:17:14.156 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 26/32 2024-02-16T19:17:14.156 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : re2-1:20211101-3.el9.x86_64 27/32 2024-02-16T19:17:14.156 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2024-02-16T19:17:14.156 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librados2-2:18.2.0-0.el9.x86_64 29/32 2024-02-16T19:17:14.156 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2024-02-16T19:17:14.156 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librbd1-2:18.2.0-0.el9.x86_64 31/32 2024-02-16T19:17:14.175 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librados-devel-2:18.2.0-0.el9.x86_64 1/1 2024-02-16T19:17:14.175 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:17:14.175 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2024-02-16T19:17:14.175 INFO:teuthology.orchestra.run.smithi022.stdout: librados-devel-2:18.2.0-0.el9.x86_64 2024-02-16T19:17:14.175 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:17:14.175 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-02-16T19:17:14.463 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install libcephfs2 2024-02-16T19:17:15.058 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:01:26 ago on Fri 16 Feb 2024 07:15:49 PM UTC. 2024-02-16T19:17:15.179 INFO:teuthology.orchestra.run.smithi022.stdout:Package libcephfs2-2:18.2.0-0.el9.x86_64 is already installed. 2024-02-16T19:17:15.222 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2024-02-16T19:17:15.223 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:17:15.223 INFO:teuthology.orchestra.run.smithi117.stdout:Upgraded: 2024-02-16T19:17:15.223 INFO:teuthology.orchestra.run.smithi117.stdout: librados2-2:18.2.0-0.el9.x86_64 librbd1-2:18.2.0-0.el9.x86_64 2024-02-16T19:17:15.223 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2024-02-16T19:17:15.223 INFO:teuthology.orchestra.run.smithi117.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-02-16T19:17:15.223 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-base-2:18.2.0-0.el9.x86_64 2024-02-16T19:17:15.223 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-common-2:18.2.0-0.el9.x86_64 2024-02-16T19:17:15.223 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-radosgw-2:18.2.0-0.el9.x86_64 2024-02-16T19:17:15.223 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-selinux-2:18.2.0-0.el9.x86_64 2024-02-16T19:17:15.223 INFO:teuthology.orchestra.run.smithi117.stdout: fmt-8.1.1-5.el9.x86_64 2024-02-16T19:17:15.223 INFO:teuthology.orchestra.run.smithi117.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-02-16T19:17:15.223 INFO:teuthology.orchestra.run.smithi117.stdout: libarrow-9.0.0-10.el9.x86_64 2024-02-16T19:17:15.223 INFO:teuthology.orchestra.run.smithi117.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-02-16T19:17:15.223 INFO:teuthology.orchestra.run.smithi117.stdout: libcephfs2-2:18.2.0-0.el9.x86_64 2024-02-16T19:17:15.224 INFO:teuthology.orchestra.run.smithi117.stdout: liboath-2.6.7-2.el9.x86_64 2024-02-16T19:17:15.224 INFO:teuthology.orchestra.run.smithi117.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-02-16T19:17:15.224 INFO:teuthology.orchestra.run.smithi117.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-02-16T19:17:15.224 INFO:teuthology.orchestra.run.smithi117.stdout: libradosstriper1-2:18.2.0-0.el9.x86_64 2024-02-16T19:17:15.224 INFO:teuthology.orchestra.run.smithi117.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-02-16T19:17:15.224 INFO:teuthology.orchestra.run.smithi117.stdout: librgw2-2:18.2.0-0.el9.x86_64 2024-02-16T19:17:15.224 INFO:teuthology.orchestra.run.smithi117.stdout: libunwind-1.6.2-1.el9.x86_64 2024-02-16T19:17:15.224 INFO:teuthology.orchestra.run.smithi117.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-02-16T19:17:15.224 INFO:teuthology.orchestra.run.smithi117.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-02-16T19:17:15.224 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ceph-argparse-2:18.2.0-0.el9.x86_64 2024-02-16T19:17:15.224 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ceph-common-2:18.2.0-0.el9.x86_64 2024-02-16T19:17:15.224 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cephfs-2:18.2.0-0.el9.x86_64 2024-02-16T19:17:15.224 INFO:teuthology.orchestra.run.smithi117.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-02-16T19:17:15.224 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rados-2:18.2.0-0.el9.x86_64 2024-02-16T19:17:15.225 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rbd-2:18.2.0-0.el9.x86_64 2024-02-16T19:17:15.225 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rgw-2:18.2.0-0.el9.x86_64 2024-02-16T19:17:15.225 INFO:teuthology.orchestra.run.smithi117.stdout: re2-1:20211101-3.el9.x86_64 2024-02-16T19:17:15.225 INFO:teuthology.orchestra.run.smithi117.stdout: thrift-0.15.0-2.el9.x86_64 2024-02-16T19:17:15.225 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:17:15.225 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-02-16T19:17:15.235 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-02-16T19:17:15.236 INFO:teuthology.orchestra.run.smithi022.stdout:Nothing to do. 2024-02-16T19:17:15.236 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-02-16T19:17:15.320 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install libcephfs-devel 2024-02-16T19:17:15.594 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install ceph-test 2024-02-16T19:17:15.914 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:01:26 ago on Fri 16 Feb 2024 07:15:49 PM UTC. 2024-02-16T19:17:16.087 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-02-16T19:17:16.088 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-02-16T19:17:16.088 INFO:teuthology.orchestra.run.smithi022.stdout: Package Architecture Version Repository Size 2024-02-16T19:17:16.088 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-02-16T19:17:16.088 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2024-02-16T19:17:16.088 INFO:teuthology.orchestra.run.smithi022.stdout: libcephfs-devel x86_64 2:18.2.0-0.el9 ceph 30 k 2024-02-16T19:17:16.088 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:17:16.088 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2024-02-16T19:17:16.088 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-02-16T19:17:16.089 INFO:teuthology.orchestra.run.smithi022.stdout:Install 1 Package 2024-02-16T19:17:16.089 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:17:16.089 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 30 k 2024-02-16T19:17:16.089 INFO:teuthology.orchestra.run.smithi022.stdout:Installed size: 137 k 2024-02-16T19:17:16.089 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading Packages: 2024-02-16T19:17:16.154 INFO:teuthology.orchestra.run.smithi022.stdout:libcephfs-devel-18.2.0-0.el9.x86_64.rpm 468 kB/s | 30 kB 00:00 2024-02-16T19:17:16.155 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2024-02-16T19:17:16.155 INFO:teuthology.orchestra.run.smithi022.stdout:Total 459 kB/s | 30 kB 00:00 2024-02-16T19:17:16.155 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2024-02-16T19:17:16.159 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction check succeeded. 2024-02-16T19:17:16.159 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2024-02-16T19:17:16.183 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:00:26 ago on Fri 16 Feb 2024 07:16:50 PM UTC. 2024-02-16T19:17:16.187 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded. 2024-02-16T19:17:16.187 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2024-02-16T19:17:16.282 INFO:teuthology.orchestra.run.smithi022.stdout: Preparing : 1/1 2024-02-16T19:17:16.348 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-02-16T19:17:16.349 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-02-16T19:17:16.349 INFO:teuthology.orchestra.run.smithi117.stdout: Package Arch Version Repository Size 2024-02-16T19:17:16.349 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-02-16T19:17:16.349 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2024-02-16T19:17:16.349 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-test x86_64 2:18.2.0-0.el9 ceph 40 M 2024-02-16T19:17:16.349 INFO:teuthology.orchestra.run.smithi117.stdout:Installing dependencies: 2024-02-16T19:17:16.349 INFO:teuthology.orchestra.run.smithi117.stdout: jq x86_64 1.6-16.el9 CentOS-BaseOS 186 k 2024-02-16T19:17:16.349 INFO:teuthology.orchestra.run.smithi117.stdout: libcephsqlite x86_64 2:18.2.0-0.el9 ceph 161 k 2024-02-16T19:17:16.349 INFO:teuthology.orchestra.run.smithi117.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-BaseOS 217 k 2024-02-16T19:17:16.349 INFO:teuthology.orchestra.run.smithi117.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-02-16T19:17:16.349 INFO:teuthology.orchestra.run.smithi117.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-02-16T19:17:16.349 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:17:16.349 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-02-16T19:17:16.350 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-02-16T19:17:16.350 INFO:teuthology.orchestra.run.smithi117.stdout:Install 6 Packages 2024-02-16T19:17:16.350 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:17:16.350 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 40 M 2024-02-16T19:17:16.350 INFO:teuthology.orchestra.run.smithi117.stdout:Installed size: 167 M 2024-02-16T19:17:16.350 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2024-02-16T19:17:16.422 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libcephfs-devel-2:18.2.0-0.el9.x86_64 1/1 2024-02-16T19:17:16.479 INFO:teuthology.orchestra.run.smithi117.stdout:(1/6): libcephsqlite-18.2.0-0.el9.x86_64.rpm 1.2 MB/s | 161 kB 00:00 2024-02-16T19:17:16.816 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: libcephfs-devel-2:18.2.0-0.el9.x86_64 1/1 2024-02-16T19:17:16.946 INFO:teuthology.orchestra.run.smithi117.stdout:(2/6): ceph-test-18.2.0-0.el9.x86_64.rpm 67 MB/s | 40 MB 00:00 2024-02-16T19:17:16.971 INFO:teuthology.orchestra.run.smithi117.stdout:(3/6): xmlstarlet-1.6.1-20.el9.x86_64.rpm 128 kB/s | 63 kB 00:00 2024-02-16T19:17:16.996 INFO:teuthology.orchestra.run.smithi117.stdout:(4/6): socat-1.7.4.1-5.el9.x86_64.rpm 465 kB/s | 300 kB 00:00 2024-02-16T19:17:17.055 INFO:teuthology.orchestra.run.smithi117.stdout:(5/6): jq-1.6-16.el9.x86_64.rpm 1.7 MB/s | 186 kB 00:00 2024-02-16T19:17:17.082 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libcephfs-devel-2:18.2.0-0.el9.x86_64 1/1 2024-02-16T19:17:17.082 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:17:17.082 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2024-02-16T19:17:17.082 INFO:teuthology.orchestra.run.smithi022.stdout: libcephfs-devel-2:18.2.0-0.el9.x86_64 2024-02-16T19:17:17.082 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:17:17.082 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-02-16T19:17:17.096 INFO:teuthology.orchestra.run.smithi117.stdout:(6/6): oniguruma-6.9.6-1.el9.5.x86_64.rpm 1.7 MB/s | 217 kB 00:00 2024-02-16T19:17:17.099 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-02-16T19:17:17.100 INFO:teuthology.orchestra.run.smithi117.stdout:Total 54 MB/s | 40 MB 00:00 2024-02-16T19:17:17.100 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-02-16T19:17:17.120 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-02-16T19:17:17.120 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-02-16T19:17:17.286 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install librados2 2024-02-16T19:17:17.475 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-02-16T19:17:17.475 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-02-16T19:17:17.883 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:01:28 ago on Fri 16 Feb 2024 07:15:49 PM UTC. 2024-02-16T19:17:18.001 INFO:teuthology.orchestra.run.smithi022.stdout:Package librados2-2:18.2.0-0.el9.x86_64 is already installed. 2024-02-16T19:17:18.012 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-02-16T19:17:18.056 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-02-16T19:17:18.057 INFO:teuthology.orchestra.run.smithi022.stdout:Nothing to do. 2024-02-16T19:17:18.057 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-02-16T19:17:18.091 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 1/6 2024-02-16T19:17:18.129 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : jq-1.6-16.el9.x86_64 2/6 2024-02-16T19:17:18.145 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install librbd1 2024-02-16T19:17:18.174 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 3/6 2024-02-16T19:17:18.214 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 4/6 2024-02-16T19:17:18.242 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libcephsqlite-2:18.2.0-0.el9.x86_64 5/6 2024-02-16T19:17:18.730 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:01:29 ago on Fri 16 Feb 2024 07:15:49 PM UTC. 2024-02-16T19:17:18.847 INFO:teuthology.orchestra.run.smithi022.stdout:Package librbd1-2:18.2.0-0.el9.x86_64 is already installed. 2024-02-16T19:17:18.902 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-02-16T19:17:18.903 INFO:teuthology.orchestra.run.smithi022.stdout:Nothing to do. 2024-02-16T19:17:18.903 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-02-16T19:17:18.986 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install python3-rados 2024-02-16T19:17:19.579 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:01:30 ago on Fri 16 Feb 2024 07:15:49 PM UTC. 2024-02-16T19:17:19.699 INFO:teuthology.orchestra.run.smithi022.stdout:Package python3-rados-2:18.2.0-0.el9.x86_64 is already installed. 2024-02-16T19:17:19.755 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-02-16T19:17:19.756 INFO:teuthology.orchestra.run.smithi022.stdout:Nothing to do. 2024-02-16T19:17:19.756 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-02-16T19:17:19.847 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install python3-rgw 2024-02-16T19:17:20.433 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:01:31 ago on Fri 16 Feb 2024 07:15:49 PM UTC. 2024-02-16T19:17:20.554 INFO:teuthology.orchestra.run.smithi022.stdout:Package python3-rgw-2:18.2.0-0.el9.x86_64 is already installed. 2024-02-16T19:17:20.609 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-02-16T19:17:20.611 INFO:teuthology.orchestra.run.smithi022.stdout:Nothing to do. 2024-02-16T19:17:20.611 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-02-16T19:17:20.697 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install python3-cephfs 2024-02-16T19:17:21.285 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:01:32 ago on Fri 16 Feb 2024 07:15:49 PM UTC. 2024-02-16T19:17:21.403 INFO:teuthology.orchestra.run.smithi022.stdout:Package python3-cephfs-2:18.2.0-0.el9.x86_64 is already installed. 2024-02-16T19:17:21.461 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-02-16T19:17:21.462 INFO:teuthology.orchestra.run.smithi022.stdout:Nothing to do. 2024-02-16T19:17:21.463 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-02-16T19:17:21.556 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install python3-rbd 2024-02-16T19:17:21.952 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: libcephsqlite-2:18.2.0-0.el9.x86_64 5/6 2024-02-16T19:17:22.023 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-test-2:18.2.0-0.el9.x86_64 6/6 2024-02-16T19:17:22.138 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:01:33 ago on Fri 16 Feb 2024 07:15:49 PM UTC. 2024-02-16T19:17:22.257 INFO:teuthology.orchestra.run.smithi022.stdout:Package python3-rbd-2:18.2.0-0.el9.x86_64 is already installed. 2024-02-16T19:17:22.311 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-02-16T19:17:22.312 INFO:teuthology.orchestra.run.smithi022.stdout:Nothing to do. 2024-02-16T19:17:22.313 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-02-16T19:17:22.402 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install rbd-fuse 2024-02-16T19:17:22.990 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:01:33 ago on Fri 16 Feb 2024 07:15:49 PM UTC. 2024-02-16T19:17:23.165 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-02-16T19:17:23.166 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-02-16T19:17:23.166 INFO:teuthology.orchestra.run.smithi022.stdout: Package Architecture Version Repository Size 2024-02-16T19:17:23.166 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-02-16T19:17:23.166 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2024-02-16T19:17:23.166 INFO:teuthology.orchestra.run.smithi022.stdout: rbd-fuse x86_64 2:18.2.0-0.el9 ceph 86 k 2024-02-16T19:17:23.166 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:17:23.167 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2024-02-16T19:17:23.167 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-02-16T19:17:23.167 INFO:teuthology.orchestra.run.smithi022.stdout:Install 1 Package 2024-02-16T19:17:23.167 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:17:23.167 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 86 k 2024-02-16T19:17:23.167 INFO:teuthology.orchestra.run.smithi022.stdout:Installed size: 230 k 2024-02-16T19:17:23.167 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading Packages: 2024-02-16T19:17:23.232 INFO:teuthology.orchestra.run.smithi022.stdout:rbd-fuse-18.2.0-0.el9.x86_64.rpm 1.3 MB/s | 86 kB 00:00 2024-02-16T19:17:23.233 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2024-02-16T19:17:23.233 INFO:teuthology.orchestra.run.smithi022.stdout:Total 1.2 MB/s | 86 kB 00:00 2024-02-16T19:17:23.234 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2024-02-16T19:17:23.241 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction check succeeded. 2024-02-16T19:17:23.241 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2024-02-16T19:17:23.298 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded. 2024-02-16T19:17:23.298 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2024-02-16T19:17:23.438 INFO:teuthology.orchestra.run.smithi022.stdout: Preparing : 1/1 2024-02-16T19:17:23.550 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : rbd-fuse-2:18.2.0-0.el9.x86_64 1/1 2024-02-16T19:17:23.942 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: rbd-fuse-2:18.2.0-0.el9.x86_64 1/1 2024-02-16T19:17:24.090 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-test-2:18.2.0-0.el9.x86_64 6/6 2024-02-16T19:17:24.090 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-test-2:18.2.0-0.el9.x86_64 1/6 2024-02-16T19:17:24.090 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libcephsqlite-2:18.2.0-0.el9.x86_64 2/6 2024-02-16T19:17:24.090 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/6 2024-02-16T19:17:24.090 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/6 2024-02-16T19:17:24.090 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : jq-1.6-16.el9.x86_64 5/6 2024-02-16T19:17:24.182 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : rbd-fuse-2:18.2.0-0.el9.x86_64 1/1 2024-02-16T19:17:24.182 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:17:24.182 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2024-02-16T19:17:24.182 INFO:teuthology.orchestra.run.smithi022.stdout: rbd-fuse-2:18.2.0-0.el9.x86_64 2024-02-16T19:17:24.182 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:17:24.182 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-02-16T19:17:24.443 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install rbd-mirror 2024-02-16T19:17:24.519 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 6/6 2024-02-16T19:17:24.519 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:17:24.519 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2024-02-16T19:17:24.519 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-test-2:18.2.0-0.el9.x86_64 jq-1.6-16.el9.x86_64 2024-02-16T19:17:24.519 INFO:teuthology.orchestra.run.smithi117.stdout: libcephsqlite-2:18.2.0-0.el9.x86_64 oniguruma-6.9.6-1.el9.5.x86_64 2024-02-16T19:17:24.519 INFO:teuthology.orchestra.run.smithi117.stdout: socat-1.7.4.1-5.el9.x86_64 xmlstarlet-1.6.1-20.el9.x86_64 2024-02-16T19:17:24.519 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:17:24.519 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-02-16T19:17:24.808 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install ceph 2024-02-16T19:17:25.025 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:01:36 ago on Fri 16 Feb 2024 07:15:49 PM UTC. 2024-02-16T19:17:25.200 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-02-16T19:17:25.201 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-02-16T19:17:25.201 INFO:teuthology.orchestra.run.smithi022.stdout: Package Architecture Version Repository Size 2024-02-16T19:17:25.201 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-02-16T19:17:25.202 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2024-02-16T19:17:25.202 INFO:teuthology.orchestra.run.smithi022.stdout: rbd-mirror x86_64 2:18.2.0-0.el9 ceph 3.0 M 2024-02-16T19:17:25.202 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:17:25.202 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2024-02-16T19:17:25.202 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-02-16T19:17:25.202 INFO:teuthology.orchestra.run.smithi022.stdout:Install 1 Package 2024-02-16T19:17:25.202 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:17:25.202 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 3.0 M 2024-02-16T19:17:25.202 INFO:teuthology.orchestra.run.smithi022.stdout:Installed size: 12 M 2024-02-16T19:17:25.202 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading Packages: 2024-02-16T19:17:25.311 INFO:teuthology.orchestra.run.smithi022.stdout:rbd-mirror-18.2.0-0.el9.x86_64.rpm 27 MB/s | 3.0 MB 00:00 2024-02-16T19:17:25.312 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2024-02-16T19:17:25.312 INFO:teuthology.orchestra.run.smithi022.stdout:Total 27 MB/s | 3.0 MB 00:00 2024-02-16T19:17:25.312 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2024-02-16T19:17:25.322 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction check succeeded. 2024-02-16T19:17:25.322 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2024-02-16T19:17:25.388 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:00:35 ago on Fri 16 Feb 2024 07:16:50 PM UTC. 2024-02-16T19:17:25.396 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded. 2024-02-16T19:17:25.397 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2024-02-16T19:17:25.567 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-02-16T19:17:25.569 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-02-16T19:17:25.569 INFO:teuthology.orchestra.run.smithi117.stdout: Package Arch Version Repository Size 2024-02-16T19:17:25.569 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-02-16T19:17:25.569 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2024-02-16T19:17:25.569 INFO:teuthology.orchestra.run.smithi117.stdout: ceph x86_64 2:18.2.0-0.el9 ceph 6.4 k 2024-02-16T19:17:25.570 INFO:teuthology.orchestra.run.smithi117.stdout:Installing dependencies: 2024-02-16T19:17:25.570 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mds x86_64 2:18.2.0-0.el9 ceph 2.1 M 2024-02-16T19:17:25.570 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr x86_64 2:18.2.0-0.el9 ceph 1.4 M 2024-02-16T19:17:25.570 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-modules-core noarch 2:18.2.0-0.el9 ceph-noarch 240 k 2024-02-16T19:17:25.570 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mon x86_64 2:18.2.0-0.el9 ceph 4.4 M 2024-02-16T19:17:25.570 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-osd x86_64 2:18.2.0-0.el9 ceph 18 M 2024-02-16T19:17:25.570 INFO:teuthology.orchestra.run.smithi117.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-02-16T19:17:25.570 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-02-16T19:17:25.570 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-02-16T19:17:25.570 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-02-16T19:17:25.570 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-02-16T19:17:25.570 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-02-16T19:17:25.571 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-02-16T19:17:25.571 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-02-16T19:17:25.571 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-02-16T19:17:25.571 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-02-16T19:17:25.571 INFO:teuthology.orchestra.run.smithi117.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-02-16T19:17:25.571 INFO:teuthology.orchestra.run.smithi117.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-02-16T19:17:25.571 INFO:teuthology.orchestra.run.smithi117.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-02-16T19:17:25.571 INFO:teuthology.orchestra.run.smithi117.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-02-16T19:17:25.571 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-02-16T19:17:25.571 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-02-16T19:17:25.571 INFO:teuthology.orchestra.run.smithi117.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-02-16T19:17:25.571 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-02-16T19:17:25.571 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-02-16T19:17:25.571 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-02-16T19:17:25.571 INFO:teuthology.orchestra.run.smithi117.stdout: python3-requests noarch 2.25.1-8.el9 CentOS-BaseOS 113 k 2024-02-16T19:17:25.572 INFO:teuthology.orchestra.run.smithi117.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-02-16T19:17:25.572 INFO:teuthology.orchestra.run.smithi117.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-02-16T19:17:25.572 INFO:teuthology.orchestra.run.smithi117.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-02-16T19:17:25.572 INFO:teuthology.orchestra.run.smithi117.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-02-16T19:17:25.572 INFO:teuthology.orchestra.run.smithi117.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-02-16T19:17:25.572 INFO:teuthology.orchestra.run.smithi117.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-02-16T19:17:25.572 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:17:25.572 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-02-16T19:17:25.572 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-02-16T19:17:25.572 INFO:teuthology.orchestra.run.smithi117.stdout:Install 33 Packages 2024-02-16T19:17:25.572 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:17:25.573 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 30 M 2024-02-16T19:17:25.573 INFO:teuthology.orchestra.run.smithi117.stdout:Installed size: 109 M 2024-02-16T19:17:25.573 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2024-02-16T19:17:25.824 INFO:teuthology.orchestra.run.smithi022.stdout: Preparing : 1/1 2024-02-16T19:17:25.886 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : rbd-mirror-2:18.2.0-0.el9.x86_64 1/1 2024-02-16T19:17:25.911 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: rbd-mirror-2:18.2.0-0.el9.x86_64 1/1 2024-02-16T19:17:25.911 INFO:teuthology.orchestra.run.smithi022.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-16T19:17:25.911 INFO:teuthology.orchestra.run.smithi022.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-02-16T19:17:25.911 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-16T19:17:25.911 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-16T19:17:25.911 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:17:26.015 INFO:teuthology.orchestra.run.smithi117.stdout:(1/33): ceph-18.2.0-0.el9.x86_64.rpm 63 kB/s | 6.4 kB 00:00 2024-02-16T19:17:26.066 INFO:teuthology.orchestra.run.smithi117.stdout:(2/33): ceph-mgr-18.2.0-0.el9.x86_64.rpm 9.5 MB/s | 1.4 MB 00:00 2024-02-16T19:17:26.132 INFO:teuthology.orchestra.run.smithi117.stdout:(3/33): ceph-mds-18.2.0-0.el9.x86_64.rpm 9.6 MB/s | 2.1 MB 00:00 2024-02-16T19:17:26.158 INFO:teuthology.orchestra.run.smithi117.stdout:(4/33): ceph-mgr-modules-core-18.2.0-0.el9.noar 9.4 MB/s | 240 kB 00:00 2024-02-16T19:17:26.241 INFO:teuthology.orchestra.run.smithi117.stdout:(5/33): ceph-mon-18.2.0-0.el9.x86_64.rpm 20 MB/s | 4.4 MB 00:00 2024-02-16T19:17:26.492 INFO:teuthology.orchestra.run.smithi117.stdout:(6/33): ceph-osd-18.2.0-0.el9.x86_64.rpm 41 MB/s | 18 MB 00:00 2024-02-16T19:17:26.684 INFO:teuthology.orchestra.run.smithi117.stdout:(7/33): python3-markupsafe-1.1.1-12.el9.x86_64. 75 kB/s | 32 kB 00:00 2024-02-16T19:17:26.709 INFO:teuthology.orchestra.run.smithi117.stdout:(8/33): python3-mako-1.1.4-6.el9.noarch.rpm 262 kB/s | 144 kB 00:00 2024-02-16T19:17:26.734 INFO:teuthology.orchestra.run.smithi117.stdout:(9/33): python3-pytz-2021.1-5.el9.noarch.rpm 218 kB/s | 47 kB 00:00 2024-02-16T19:17:26.759 INFO:teuthology.orchestra.run.smithi117.stdout:(10/33): python3-toml-0.10.2-6.el9.noarch.rpm 511 kB/s | 39 kB 00:00 2024-02-16T19:17:26.843 INFO:teuthology.orchestra.run.smithi117.stdout:(11/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.8 MB/s | 241 kB 00:00 2024-02-16T19:17:26.943 INFO:teuthology.orchestra.run.smithi117.stdout:(12/33): python3-ply-3.11-14.el9.noarch.rpm 560 kB/s | 103 kB 00:00 2024-02-16T19:17:26.985 INFO:teuthology.orchestra.run.smithi117.stdout:(13/33): python3-cryptography-36.0.1-4.el9.x86_ 4.6 MB/s | 1.1 MB 00:00 2024-02-16T19:17:27.010 INFO:teuthology.orchestra.run.smithi117.stdout:(14/33): python3-pycparser-2.20-6.el9.noarch.rp 743 kB/s | 124 kB 00:00 2024-02-16T19:17:27.035 INFO:teuthology.orchestra.run.smithi117.stdout:(15/33): python3-requests-2.25.1-8.el9.noarch.r 1.2 MB/s | 113 kB 00:00 2024-02-16T19:17:27.065 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : rbd-mirror-2:18.2.0-0.el9.x86_64 1/1 2024-02-16T19:17:27.065 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:17:27.065 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2024-02-16T19:17:27.065 INFO:teuthology.orchestra.run.smithi022.stdout: rbd-mirror-2:18.2.0-0.el9.x86_64 2024-02-16T19:17:27.065 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:17:27.065 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-02-16T19:17:27.252 INFO:teuthology.orchestra.run.smithi117.stdout:(16/33): python3-urllib3-1.26.5-5.el9.noarch.rp 775 kB/s | 187 kB 00:00 2024-02-16T19:17:27.307 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install rbd-nbd 2024-02-16T19:17:27.519 INFO:teuthology.orchestra.run.smithi117.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 85 kB/s | 43 kB 00:00 2024-02-16T19:17:27.636 INFO:teuthology.orchestra.run.smithi117.stdout:(18/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 91 kB/s | 11 kB 00:00 2024-02-16T19:17:27.737 INFO:teuthology.orchestra.run.smithi117.stdout:(19/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 245 kB/s | 172 kB 00:00 2024-02-16T19:17:27.753 INFO:teuthology.orchestra.run.smithi117.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 151 kB/s | 18 kB 00:00 2024-02-16T19:17:27.820 INFO:teuthology.orchestra.run.smithi117.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 278 kB/s | 23 kB 00:00 2024-02-16T19:17:27.862 INFO:teuthology.orchestra.run.smithi117.stdout:(22/33): python3-jaraco-functools-3.5.0-2.el9.n 179 kB/s | 19 kB 00:00 2024-02-16T19:17:27.896 INFO:teuthology.orchestra.run.smithi117.stdout:(23/33): python3-jaraco-text-3.2.0-6.el9.noarch 262 kB/s | 20 kB 00:00 2024-02-16T19:17:27.896 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:01:38 ago on Fri 16 Feb 2024 07:15:49 PM UTC. 2024-02-16T19:17:27.921 INFO:teuthology.orchestra.run.smithi117.stdout:(24/33): python3-cherrypy-18.6.1-2.el9.noarch.r 537 kB/s | 358 kB 00:00 2024-02-16T19:17:28.022 INFO:teuthology.orchestra.run.smithi117.stdout:(25/33): python3-more-itertools-8.12.0-2.el9.no 629 kB/s | 79 kB 00:00 2024-02-16T19:17:28.047 INFO:teuthology.orchestra.run.smithi117.stdout:(26/33): python3-logutils-0.3.5-21.el9.noarch.r 251 kB/s | 46 kB 00:00 2024-02-16T19:17:28.070 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-02-16T19:17:28.070 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-02-16T19:17:28.070 INFO:teuthology.orchestra.run.smithi022.stdout: Package Architecture Version Repository Size 2024-02-16T19:17:28.070 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-02-16T19:17:28.071 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2024-02-16T19:17:28.071 INFO:teuthology.orchestra.run.smithi022.stdout: rbd-nbd x86_64 2:18.2.0-0.el9 ceph 169 k 2024-02-16T19:17:28.071 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:17:28.071 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2024-02-16T19:17:28.071 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-02-16T19:17:28.071 INFO:teuthology.orchestra.run.smithi022.stdout:Install 1 Package 2024-02-16T19:17:28.071 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:17:28.072 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 169 k 2024-02-16T19:17:28.072 INFO:teuthology.orchestra.run.smithi022.stdout:Installed size: 490 k 2024-02-16T19:17:28.072 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading Packages: 2024-02-16T19:17:28.072 INFO:teuthology.orchestra.run.smithi117.stdout:(27/33): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-02-16T19:17:28.122 INFO:teuthology.orchestra.run.smithi117.stdout:(28/33): python3-portend-3.1.0-2.el9.noarch.rpm 163 kB/s | 16 kB 00:00 2024-02-16T19:17:28.131 INFO:teuthology.orchestra.run.smithi022.stdout:rbd-nbd-18.2.0-0.el9.x86_64.rpm 2.8 MB/s | 169 kB 00:00 2024-02-16T19:17:28.132 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2024-02-16T19:17:28.132 INFO:teuthology.orchestra.run.smithi022.stdout:Total 2.7 MB/s | 169 kB 00:00 2024-02-16T19:17:28.132 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2024-02-16T19:17:28.140 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction check succeeded. 2024-02-16T19:17:28.140 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2024-02-16T19:17:28.197 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded. 2024-02-16T19:17:28.197 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2024-02-16T19:17:28.222 INFO:teuthology.orchestra.run.smithi117.stdout:(29/33): python3-tempora-5.0.0-2.el9.noarch.rpm 238 kB/s | 36 kB 00:00 2024-02-16T19:17:28.239 INFO:teuthology.orchestra.run.smithi117.stdout:(30/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 467 kB/s | 90 kB 00:00 2024-02-16T19:17:28.340 INFO:teuthology.orchestra.run.smithi117.stdout:(31/33): python3-zc-lockfile-2.0-10.el9.noarch. 200 kB/s | 20 kB 00:00 2024-02-16T19:17:28.354 INFO:teuthology.orchestra.run.smithi022.stdout: Preparing : 1/1 2024-02-16T19:17:28.365 INFO:teuthology.orchestra.run.smithi117.stdout:(32/33): python3-webob-1.8.7-6.el9.noarch.rpm 948 kB/s | 230 kB 00:00 2024-02-16T19:17:28.398 INFO:teuthology.orchestra.run.smithi117.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 2.4 MB/s | 427 kB 00:00 2024-02-16T19:17:28.405 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-02-16T19:17:28.406 INFO:teuthology.orchestra.run.smithi117.stdout:Total 10 MB/s | 30 MB 00:02 2024-02-16T19:17:28.406 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-02-16T19:17:28.445 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-02-16T19:17:28.445 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-02-16T19:17:28.451 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : rbd-nbd-2:18.2.0-0.el9.x86_64 1/1 2024-02-16T19:17:28.807 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-02-16T19:17:28.807 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-02-16T19:17:28.894 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: rbd-nbd-2:18.2.0-0.el9.x86_64 1/1 2024-02-16T19:17:29.184 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : rbd-nbd-2:18.2.0-0.el9.x86_64 1/1 2024-02-16T19:17:29.184 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:17:29.184 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2024-02-16T19:17:29.184 INFO:teuthology.orchestra.run.smithi022.stdout: rbd-nbd-2:18.2.0-0.el9.x86_64 2024-02-16T19:17:29.184 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:17:29.184 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-02-16T19:17:29.437 DEBUG:teuthology.parallel:result is None 2024-02-16T19:17:30.019 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-02-16T19:17:30.084 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-02-16T19:17:30.116 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-02-16T19:17:30.149 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-02-16T19:17:30.251 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-02-16T19:17:30.327 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-02-16T19:17:30.369 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-02-16T19:17:30.449 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-02-16T19:17:30.502 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 8/33 2024-02-16T19:17:30.541 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 9/33 2024-02-16T19:17:30.590 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-ply-3.11-14.el9.noarch 10/33 2024-02-16T19:17:30.656 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 11/33 2024-02-16T19:17:30.836 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 12/33 2024-02-16T19:17:30.886 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 13/33 2024-02-16T19:17:30.917 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 14/33 2024-02-16T19:17:30.952 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 15/33 2024-02-16T19:17:30.982 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 16/33 2024-02-16T19:17:31.058 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 17/33 2024-02-16T19:17:31.092 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-02-16T19:17:31.125 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 19/33 2024-02-16T19:17:31.166 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 20/33 2024-02-16T19:17:31.201 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/33 2024-02-16T19:17:31.236 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 22/33 2024-02-16T19:17:31.325 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 23/33 2024-02-16T19:17:31.366 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 24/33 2024-02-16T19:17:31.438 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/33 2024-02-16T19:17:31.574 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 26/33 2024-02-16T19:17:31.692 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-02-16T19:17:31.854 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-modules-core-2:18.2.0-0.el9.noarch 28/33 2024-02-16T19:17:31.888 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-2:18.2.0-0.el9.x86_64 29/33 2024-02-16T19:17:31.913 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-2:18.2.0-0.el9.x86_64 29/33 2024-02-16T19:17:31.913 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-16T19:17:31.913 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-02-16T19:17:31.913 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-16T19:17:31.913 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-16T19:17:31.913 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:17:33.448 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-osd-2:18.2.0-0.el9.x86_64 30/33 2024-02-16T19:17:33.477 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-osd-2:18.2.0-0.el9.x86_64 30/33 2024-02-16T19:17:33.477 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-16T19:17:33.477 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-02-16T19:17:33.477 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-16T19:17:33.477 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-16T19:17:33.477 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:17:33.932 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mon-2:18.2.0-0.el9.x86_64 31/33 2024-02-16T19:17:33.955 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mon-2:18.2.0-0.el9.x86_64 31/33 2024-02-16T19:17:33.956 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-16T19:17:33.956 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-02-16T19:17:33.956 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-16T19:17:33.956 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-16T19:17:33.956 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:17:34.182 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mds-2:18.2.0-0.el9.x86_64 32/33 2024-02-16T19:17:34.206 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mds-2:18.2.0-0.el9.x86_64 32/33 2024-02-16T19:17:34.206 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-16T19:17:34.206 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-02-16T19:17:34.206 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-16T19:17:34.206 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-16T19:17:34.206 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:17:34.272 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-2:18.2.0-0.el9.x86_64 33/33 2024-02-16T19:17:35.980 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-2:18.2.0-0.el9.x86_64 33/33 2024-02-16T19:17:35.980 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-2:18.2.0-0.el9.x86_64 1/33 2024-02-16T19:17:35.981 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mds-2:18.2.0-0.el9.x86_64 2/33 2024-02-16T19:17:35.981 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-2:18.2.0-0.el9.x86_64 3/33 2024-02-16T19:17:35.981 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mon-2:18.2.0-0.el9.x86_64 4/33 2024-02-16T19:17:35.981 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-osd-2:18.2.0-0.el9.x86_64 5/33 2024-02-16T19:17:35.981 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-modules-core-2:18.2.0-0.el9.noarch 6/33 2024-02-16T19:17:35.981 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 7/33 2024-02-16T19:17:35.981 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 8/33 2024-02-16T19:17:35.981 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 9/33 2024-02-16T19:17:35.981 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 10/33 2024-02-16T19:17:35.981 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 11/33 2024-02-16T19:17:35.981 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 12/33 2024-02-16T19:17:35.981 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-ply-3.11-14.el9.noarch 13/33 2024-02-16T19:17:35.981 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 14/33 2024-02-16T19:17:35.981 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 15/33 2024-02-16T19:17:35.981 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 16/33 2024-02-16T19:17:35.982 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-02-16T19:17:35.982 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-02-16T19:17:35.982 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-02-16T19:17:35.982 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-02-16T19:17:35.982 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-02-16T19:17:35.982 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-02-16T19:17:35.982 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-02-16T19:17:35.982 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-02-16T19:17:35.982 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-02-16T19:17:35.982 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-02-16T19:17:35.982 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-02-16T19:17:35.982 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-02-16T19:17:35.982 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-02-16T19:17:35.982 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-02-16T19:17:35.982 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-02-16T19:17:35.983 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-02-16T19:17:36.830 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-02-16T19:17:36.830 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:17:36.831 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2024-02-16T19:17:36.831 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-2:18.2.0-0.el9.x86_64 2024-02-16T19:17:36.831 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mds-2:18.2.0-0.el9.x86_64 2024-02-16T19:17:36.831 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-2:18.2.0-0.el9.x86_64 2024-02-16T19:17:36.831 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-modules-core-2:18.2.0-0.el9.noarch 2024-02-16T19:17:36.831 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mon-2:18.2.0-0.el9.x86_64 2024-02-16T19:17:36.831 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-osd-2:18.2.0-0.el9.x86_64 2024-02-16T19:17:36.831 INFO:teuthology.orchestra.run.smithi117.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-02-16T19:17:36.831 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-02-16T19:17:36.831 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-02-16T19:17:36.831 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-02-16T19:17:36.831 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-02-16T19:17:36.831 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-02-16T19:17:36.832 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-02-16T19:17:36.832 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-02-16T19:17:36.832 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-02-16T19:17:36.832 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-02-16T19:17:36.832 INFO:teuthology.orchestra.run.smithi117.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-02-16T19:17:36.832 INFO:teuthology.orchestra.run.smithi117.stdout: python3-mako-1.1.4-6.el9.noarch 2024-02-16T19:17:36.832 INFO:teuthology.orchestra.run.smithi117.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-02-16T19:17:36.832 INFO:teuthology.orchestra.run.smithi117.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-02-16T19:17:36.832 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-02-16T19:17:36.832 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ply-3.11-14.el9.noarch 2024-02-16T19:17:36.832 INFO:teuthology.orchestra.run.smithi117.stdout: python3-portend-3.1.0-2.el9.noarch 2024-02-16T19:17:36.832 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-02-16T19:17:36.832 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pycparser-2.20-6.el9.noarch 2024-02-16T19:17:36.833 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pytz-2021.1-5.el9.noarch 2024-02-16T19:17:36.833 INFO:teuthology.orchestra.run.smithi117.stdout: python3-requests-2.25.1-8.el9.noarch 2024-02-16T19:17:36.833 INFO:teuthology.orchestra.run.smithi117.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-02-16T19:17:36.833 INFO:teuthology.orchestra.run.smithi117.stdout: python3-toml-0.10.2-6.el9.noarch 2024-02-16T19:17:36.833 INFO:teuthology.orchestra.run.smithi117.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-02-16T19:17:36.833 INFO:teuthology.orchestra.run.smithi117.stdout: python3-webob-1.8.7-6.el9.noarch 2024-02-16T19:17:36.833 INFO:teuthology.orchestra.run.smithi117.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-02-16T19:17:36.833 INFO:teuthology.orchestra.run.smithi117.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-02-16T19:17:36.833 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:17:36.833 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-02-16T19:17:37.183 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install ceph-base 2024-02-16T19:17:37.758 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:00:47 ago on Fri 16 Feb 2024 07:16:50 PM UTC. 2024-02-16T19:17:37.871 INFO:teuthology.orchestra.run.smithi117.stdout:Package ceph-base-2:18.2.0-0.el9.x86_64 is already installed. 2024-02-16T19:17:37.923 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-02-16T19:17:37.923 INFO:teuthology.orchestra.run.smithi117.stdout:Nothing to do. 2024-02-16T19:17:37.924 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-02-16T19:17:37.985 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install cephadm 2024-02-16T19:17:38.564 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:00:48 ago on Fri 16 Feb 2024 07:16:50 PM UTC. 2024-02-16T19:17:38.727 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-02-16T19:17:38.728 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-02-16T19:17:38.728 INFO:teuthology.orchestra.run.smithi117.stdout: Package Architecture Version Repository Size 2024-02-16T19:17:38.728 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-02-16T19:17:38.728 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2024-02-16T19:17:38.728 INFO:teuthology.orchestra.run.smithi117.stdout: cephadm noarch 2:18.2.0-0.el9 ceph-noarch 209 k 2024-02-16T19:17:38.729 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:17:38.729 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-02-16T19:17:38.729 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-02-16T19:17:38.729 INFO:teuthology.orchestra.run.smithi117.stdout:Install 1 Package 2024-02-16T19:17:38.729 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:17:38.729 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 209 k 2024-02-16T19:17:38.729 INFO:teuthology.orchestra.run.smithi117.stdout:Installed size: 200 k 2024-02-16T19:17:38.729 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2024-02-16T19:17:38.816 INFO:teuthology.orchestra.run.smithi117.stdout:cephadm-18.2.0-0.el9.noarch.rpm 2.4 MB/s | 209 kB 00:00 2024-02-16T19:17:38.816 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-02-16T19:17:38.817 INFO:teuthology.orchestra.run.smithi117.stdout:Total 2.3 MB/s | 209 kB 00:00 2024-02-16T19:17:38.817 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-02-16T19:17:38.822 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-02-16T19:17:38.822 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-02-16T19:17:38.824 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-02-16T19:17:38.824 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-02-16T19:17:38.909 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-02-16T19:17:39.309 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: cephadm-2:18.2.0-0.el9.noarch 1/1 2024-02-16T19:17:39.412 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : cephadm-2:18.2.0-0.el9.noarch 1/1 2024-02-16T19:17:39.827 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: cephadm-2:18.2.0-0.el9.noarch 1/1 2024-02-16T19:17:40.154 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : cephadm-2:18.2.0-0.el9.noarch 1/1 2024-02-16T19:17:40.154 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:17:40.154 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2024-02-16T19:17:40.154 INFO:teuthology.orchestra.run.smithi117.stdout: cephadm-2:18.2.0-0.el9.noarch 2024-02-16T19:17:40.154 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:17:40.154 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-02-16T19:17:40.350 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install ceph-immutable-object-cache 2024-02-16T19:17:40.929 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:00:50 ago on Fri 16 Feb 2024 07:16:50 PM UTC. 2024-02-16T19:17:41.098 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-02-16T19:17:41.099 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-02-16T19:17:41.099 INFO:teuthology.orchestra.run.smithi117.stdout: Package Arch Version Repo Size 2024-02-16T19:17:41.099 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-02-16T19:17:41.099 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2024-02-16T19:17:41.099 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-immutable-object-cache x86_64 2:18.2.0-0.el9 ceph 142 k 2024-02-16T19:17:41.099 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:17:41.099 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-02-16T19:17:41.099 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-02-16T19:17:41.099 INFO:teuthology.orchestra.run.smithi117.stdout:Install 1 Package 2024-02-16T19:17:41.099 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:17:41.100 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 142 k 2024-02-16T19:17:41.100 INFO:teuthology.orchestra.run.smithi117.stdout:Installed size: 399 k 2024-02-16T19:17:41.100 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2024-02-16T19:17:41.173 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-immutable-object-cache-18.2.0-0.el9.x86_64 1.9 MB/s | 142 kB 00:00 2024-02-16T19:17:41.173 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-02-16T19:17:41.173 INFO:teuthology.orchestra.run.smithi117.stdout:Total 1.9 MB/s | 142 kB 00:00 2024-02-16T19:17:41.174 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-02-16T19:17:41.181 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-02-16T19:17:41.181 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-02-16T19:17:41.232 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-02-16T19:17:41.233 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-02-16T19:17:41.366 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-02-16T19:17:41.424 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-immutable-object-cache-2:18.2.0-0.el9.x86_64 1/1 2024-02-16T19:17:41.447 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.0-0.el9.x86_64 1/1 2024-02-16T19:17:41.447 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-16T19:17:41.447 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-02-16T19:17:41.447 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:17:42.403 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-immutable-object-cache-2:18.2.0-0.el9.x86_64 1/1 2024-02-16T19:17:42.404 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:17:42.404 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2024-02-16T19:17:42.404 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-immutable-object-cache-2:18.2.0-0.el9.x86_64 2024-02-16T19:17:42.404 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:17:42.404 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-02-16T19:17:42.666 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install ceph-mgr 2024-02-16T19:17:43.252 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:00:53 ago on Fri 16 Feb 2024 07:16:50 PM UTC. 2024-02-16T19:17:43.367 INFO:teuthology.orchestra.run.smithi117.stdout:Package ceph-mgr-2:18.2.0-0.el9.x86_64 is already installed. 2024-02-16T19:17:43.418 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-02-16T19:17:43.419 INFO:teuthology.orchestra.run.smithi117.stdout:Nothing to do. 2024-02-16T19:17:43.419 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-02-16T19:17:43.476 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install ceph-mgr-dashboard 2024-02-16T19:17:44.051 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:00:54 ago on Fri 16 Feb 2024 07:16:50 PM UTC. 2024-02-16T19:17:44.226 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-02-16T19:17:44.227 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-02-16T19:17:44.227 INFO:teuthology.orchestra.run.smithi117.stdout: Package Arch Version Repository Size 2024-02-16T19:17:44.227 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-02-16T19:17:44.227 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2024-02-16T19:17:44.227 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-dashboard noarch 2:18.2.0-0.el9 ceph-noarch 1.7 M 2024-02-16T19:17:44.227 INFO:teuthology.orchestra.run.smithi117.stdout:Installing dependencies: 2024-02-16T19:17:44.227 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-grafana-dashboards noarch 2:18.2.0-0.el9 ceph-noarch 23 k 2024-02-16T19:17:44.227 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-prometheus-alerts noarch 2:18.2.0-0.el9 ceph-noarch 15 k 2024-02-16T19:17:44.227 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-02-16T19:17:44.227 INFO:teuthology.orchestra.run.smithi117.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-02-16T19:17:44.227 INFO:teuthology.orchestra.run.smithi117.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-02-16T19:17:44.228 INFO:teuthology.orchestra.run.smithi117.stdout:Installing weak dependencies: 2024-02-16T19:17:44.228 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-02-16T19:17:44.228 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:17:44.228 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-02-16T19:17:44.228 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-02-16T19:17:44.228 INFO:teuthology.orchestra.run.smithi117.stdout:Install 7 Packages 2024-02-16T19:17:44.228 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:17:44.229 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 2.0 M 2024-02-16T19:17:44.229 INFO:teuthology.orchestra.run.smithi117.stdout:Installed size: 8.1 M 2024-02-16T19:17:44.229 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2024-02-16T19:17:44.376 INFO:teuthology.orchestra.run.smithi117.stdout:(1/7): ceph-prometheus-alerts-18.2.0-0.el9.noar 140 kB/s | 15 kB 00:00 2024-02-16T19:17:44.393 INFO:teuthology.orchestra.run.smithi117.stdout:(2/7): ceph-grafana-dashboards-18.2.0-0.el9.noa 191 kB/s | 23 kB 00:00 2024-02-16T19:17:44.443 INFO:teuthology.orchestra.run.smithi117.stdout:(3/7): ceph-mgr-dashboard-18.2.0-0.el9.noarch.r 9.9 MB/s | 1.7 MB 00:00 2024-02-16T19:17:44.652 INFO:teuthology.orchestra.run.smithi117.stdout:(4/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 33 kB/s | 9.0 kB 00:00 2024-02-16T19:17:44.719 INFO:teuthology.orchestra.run.smithi117.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 126 kB/s | 41 kB 00:00 2024-02-16T19:17:44.752 INFO:teuthology.orchestra.run.smithi117.stdout:(6/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 100 kB/s | 31 kB 00:00 2024-02-16T19:17:44.953 INFO:teuthology.orchestra.run.smithi117.stdout:(7/7): python3-routes-2.5.1-5.el9.noarch.rpm 628 kB/s | 188 kB 00:00 2024-02-16T19:17:44.956 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-02-16T19:17:44.957 INFO:teuthology.orchestra.run.smithi117.stdout:Total 2.7 MB/s | 2.0 MB 00:00 2024-02-16T19:17:44.957 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-02-16T19:17:44.964 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-02-16T19:17:44.964 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-02-16T19:17:45.033 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-02-16T19:17:45.033 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-02-16T19:17:45.328 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-02-16T19:17:45.396 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-02-16T19:17:45.429 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-02-16T19:17:45.475 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-02-16T19:17:45.508 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-02-16T19:17:45.537 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-prometheus-alerts-2:18.2.0-0.el9.noarch 5/7 2024-02-16T19:17:45.763 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-grafana-dashboards-2:18.2.0-0.el9.noarch 6/7 2024-02-16T19:17:45.798 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-dashboard-2:18.2.0-0.el9.noarch 7/7 2024-02-16T19:17:46.337 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.0-0.el9.noarch 7/7 2024-02-16T19:17:46.337 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-grafana-dashboards-2:18.2.0-0.el9.noarch 1/7 2024-02-16T19:17:46.337 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-dashboard-2:18.2.0-0.el9.noarch 2/7 2024-02-16T19:17:46.337 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-prometheus-alerts-2:18.2.0-0.el9.noarch 3/7 2024-02-16T19:17:46.337 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-02-16T19:17:46.337 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-02-16T19:17:46.337 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-02-16T19:17:46.644 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-02-16T19:17:46.644 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:17:46.644 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2024-02-16T19:17:46.644 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-grafana-dashboards-2:18.2.0-0.el9.noarch 2024-02-16T19:17:46.644 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-dashboard-2:18.2.0-0.el9.noarch 2024-02-16T19:17:46.644 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-prometheus-alerts-2:18.2.0-0.el9.noarch 2024-02-16T19:17:46.644 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-02-16T19:17:46.644 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-02-16T19:17:46.644 INFO:teuthology.orchestra.run.smithi117.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-02-16T19:17:46.644 INFO:teuthology.orchestra.run.smithi117.stdout: python3-routes-2.5.1-5.el9.noarch 2024-02-16T19:17:46.645 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:17:46.645 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-02-16T19:17:46.881 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install ceph-mgr-diskprediction-local 2024-02-16T19:17:47.454 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:00:57 ago on Fri 16 Feb 2024 07:16:50 PM UTC. 2024-02-16T19:17:47.619 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-02-16T19:17:47.620 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-02-16T19:17:47.620 INFO:teuthology.orchestra.run.smithi117.stdout: Package Arch Version Repository Size 2024-02-16T19:17:47.620 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-02-16T19:17:47.620 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2024-02-16T19:17:47.620 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.0-0.el9 ceph-noarch 7.4 M 2024-02-16T19:17:47.620 INFO:teuthology.orchestra.run.smithi117.stdout:Installing dependencies: 2024-02-16T19:17:47.620 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-02-16T19:17:47.620 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-02-16T19:17:47.621 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-02-16T19:17:47.621 INFO:teuthology.orchestra.run.smithi117.stdout: libgfortran x86_64 11.4.1-3.el9 CentOS-BaseOS 802 k 2024-02-16T19:17:47.621 INFO:teuthology.orchestra.run.smithi117.stdout: libquadmath x86_64 11.4.1-3.el9 CentOS-BaseOS 194 k 2024-02-16T19:17:47.621 INFO:teuthology.orchestra.run.smithi117.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-02-16T19:17:47.621 INFO:teuthology.orchestra.run.smithi117.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-02-16T19:17:47.621 INFO:teuthology.orchestra.run.smithi117.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-02-16T19:17:47.621 INFO:teuthology.orchestra.run.smithi117.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-02-16T19:17:47.621 INFO:teuthology.orchestra.run.smithi117.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-02-16T19:17:47.621 INFO:teuthology.orchestra.run.smithi117.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-02-16T19:17:47.621 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:17:47.621 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-02-16T19:17:47.621 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-02-16T19:17:47.621 INFO:teuthology.orchestra.run.smithi117.stdout:Install 12 Packages 2024-02-16T19:17:47.621 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:17:47.622 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 37 M 2024-02-16T19:17:47.622 INFO:teuthology.orchestra.run.smithi117.stdout:Installed size: 209 M 2024-02-16T19:17:47.622 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2024-02-16T19:17:47.859 INFO:teuthology.orchestra.run.smithi117.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 264 kB/s | 32 kB 00:00 2024-02-16T19:17:47.884 INFO:teuthology.orchestra.run.smithi117.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 689 kB/s | 17 kB 00:00 2024-02-16T19:17:47.985 INFO:teuthology.orchestra.run.smithi117.stdout:(3/12): ceph-mgr-diskprediction-local-18.2.0-0. 30 MB/s | 7.4 MB 00:00 2024-02-16T19:17:48.017 INFO:teuthology.orchestra.run.smithi117.stdout:(4/12): openblas-0.3.21-2.el9.x86_64.rpm 273 kB/s | 34 kB 00:00 2024-02-16T19:17:48.185 INFO:teuthology.orchestra.run.smithi117.stdout:(5/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 24 MB/s | 4.7 MB 00:00 2024-02-16T19:17:48.269 INFO:teuthology.orchestra.run.smithi117.stdout:(6/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 19 MB/s | 4.8 MB 00:00 2024-02-16T19:17:48.328 INFO:teuthology.orchestra.run.smithi117.stdout:(7/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 5.1 MB/s | 3.0 MB 00:00 2024-02-16T19:17:48.353 INFO:teuthology.orchestra.run.smithi117.stdout:(8/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 1.3 MB/s | 217 kB 00:00 2024-02-16T19:17:48.637 INFO:teuthology.orchestra.run.smithi117.stdout:(9/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 43 MB/s | 16 MB 00:00 2024-02-16T19:17:48.661 INFO:teuthology.orchestra.run.smithi117.stdout:(10/12): libquadmath-11.4.1-3.el9.x86_64.rpm 634 kB/s | 194 kB 00:00 2024-02-16T19:17:48.695 INFO:teuthology.orchestra.run.smithi117.stdout:(11/12): libgfortran-11.4.1-3.el9.x86_64.rpm 2.3 MB/s | 802 kB 00:00 2024-02-16T19:17:48.787 INFO:teuthology.orchestra.run.smithi117.stdout:(12/12): python3-devel-3.9.17-2.el9.x86_64.rpm 1.6 MB/s | 245 kB 00:00 2024-02-16T19:17:48.789 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-02-16T19:17:48.790 INFO:teuthology.orchestra.run.smithi117.stdout:Total 32 MB/s | 37 MB 00:01 2024-02-16T19:17:48.808 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-02-16T19:17:48.832 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-02-16T19:17:48.832 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-02-16T19:17:49.261 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-02-16T19:17:49.262 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-02-16T19:17:49.958 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-02-16T19:17:50.043 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 1/12 2024-02-16T19:17:50.074 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-02-16T19:17:50.173 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2024-02-16T19:17:50.228 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-02-16T19:17:50.406 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-02-16T19:17:50.435 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-02-16T19:17:50.511 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-02-16T19:17:51.187 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-02-16T19:17:51.298 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-02-16T19:17:52.531 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-02-16T19:17:53.570 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-02-16T19:17:53.601 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.0-0.el9.noarc 12/12 2024-02-16T19:17:56.107 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.0-0.el9.noarc 12/12 2024-02-16T19:17:56.108 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.0-0.el9.noarc 1/12 2024-02-16T19:17:56.108 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2024-02-16T19:17:56.108 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2024-02-16T19:17:56.108 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2024-02-16T19:17:56.108 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2024-02-16T19:17:56.108 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-02-16T19:17:56.108 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2024-02-16T19:17:56.108 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2024-02-16T19:17:56.108 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2024-02-16T19:17:56.108 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 10/12 2024-02-16T19:17:56.108 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 11/12 2024-02-16T19:17:56.554 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 12/12 2024-02-16T19:17:56.554 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:17:56.555 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2024-02-16T19:17:56.555 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-diskprediction-local-2:18.2.0-0.el9.noarch 2024-02-16T19:17:56.555 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-02-16T19:17:56.555 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-02-16T19:17:56.555 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-02-16T19:17:56.555 INFO:teuthology.orchestra.run.smithi117.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-02-16T19:17:56.555 INFO:teuthology.orchestra.run.smithi117.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-02-16T19:17:56.555 INFO:teuthology.orchestra.run.smithi117.stdout: openblas-0.3.21-2.el9.x86_64 2024-02-16T19:17:56.555 INFO:teuthology.orchestra.run.smithi117.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-02-16T19:17:56.555 INFO:teuthology.orchestra.run.smithi117.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-02-16T19:17:56.555 INFO:teuthology.orchestra.run.smithi117.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-02-16T19:17:56.555 INFO:teuthology.orchestra.run.smithi117.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-02-16T19:17:56.555 INFO:teuthology.orchestra.run.smithi117.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-02-16T19:17:56.555 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:17:56.555 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-02-16T19:17:56.921 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install ceph-mgr-rook 2024-02-16T19:17:57.501 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:01:07 ago on Fri 16 Feb 2024 07:16:50 PM UTC. 2024-02-16T19:17:57.666 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-02-16T19:17:57.667 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-02-16T19:17:57.667 INFO:teuthology.orchestra.run.smithi117.stdout: Package Arch Version Repository Size 2024-02-16T19:17:57.667 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-02-16T19:17:57.667 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2024-02-16T19:17:57.668 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-rook noarch 2:18.2.0-0.el9 ceph-noarch 47 k 2024-02-16T19:17:57.668 INFO:teuthology.orchestra.run.smithi117.stdout:Installing dependencies: 2024-02-16T19:17:57.668 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-02-16T19:17:57.668 INFO:teuthology.orchestra.run.smithi117.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-02-16T19:17:57.668 INFO:teuthology.orchestra.run.smithi117.stdout: python3-google-auth noarch 1:2.27.0-1.el9 epel 203 k 2024-02-16T19:17:57.668 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-02-16T19:17:57.668 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-02-16T19:17:57.668 INFO:teuthology.orchestra.run.smithi117.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-02-16T19:17:57.668 INFO:teuthology.orchestra.run.smithi117.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-02-16T19:17:57.668 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-02-16T19:17:57.668 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-02-16T19:17:57.668 INFO:teuthology.orchestra.run.smithi117.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-02-16T19:17:57.668 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-02-16T19:17:57.669 INFO:teuthology.orchestra.run.smithi117.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-02-16T19:17:57.669 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:17:57.669 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-02-16T19:17:57.669 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-02-16T19:17:57.669 INFO:teuthology.orchestra.run.smithi117.stdout:Install 13 Packages 2024-02-16T19:17:57.669 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:17:57.669 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 2.0 M 2024-02-16T19:17:57.670 INFO:teuthology.orchestra.run.smithi117.stdout:Installed size: 26 M 2024-02-16T19:17:57.670 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2024-02-16T19:17:57.817 INFO:teuthology.orchestra.run.smithi117.stdout:(1/13): ceph-mgr-rook-18.2.0-0.el9.noarch.rpm 474 kB/s | 47 kB 00:00 2024-02-16T19:17:57.842 INFO:teuthology.orchestra.run.smithi117.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 194 kB/s | 24 kB 00:00 2024-02-16T19:17:57.867 INFO:teuthology.orchestra.run.smithi117.stdout:(3/13): python3-jsonpointer-2.0-4.el9.noarch.rp 120 kB/s | 18 kB 00:00 2024-02-16T19:17:57.892 INFO:teuthology.orchestra.run.smithi117.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.3 MB/s | 174 kB 00:00 2024-02-16T19:17:57.918 INFO:teuthology.orchestra.run.smithi117.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2024-02-16T19:17:57.943 INFO:teuthology.orchestra.run.smithi117.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 2.7 MB/s | 211 kB 00:00 2024-02-16T19:17:57.968 INFO:teuthology.orchestra.run.smithi117.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 569 kB/s | 43 kB 00:00 2024-02-16T19:17:58.419 INFO:teuthology.orchestra.run.smithi117.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 32 kB/s | 15 kB 00:00 2024-02-16T19:17:58.477 INFO:teuthology.orchestra.run.smithi117.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 60 kB/s | 32 kB 00:00 2024-02-16T19:17:58.761 INFO:teuthology.orchestra.run.smithi117.stdout:(10/13): python3-google-auth-2.27.0-1.el9.noarc 256 kB/s | 203 kB 00:00 2024-02-16T19:17:59.003 INFO:teuthology.orchestra.run.smithi117.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 112 kB/s | 59 kB 00:00 2024-02-16T19:17:59.370 INFO:teuthology.orchestra.run.smithi117.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 147 kB/s | 90 kB 00:00 2024-02-16T19:17:59.719 INFO:teuthology.orchestra.run.smithi117.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 804 kB/s | 1.0 MB 00:01 2024-02-16T19:17:59.723 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-02-16T19:17:59.723 INFO:teuthology.orchestra.run.smithi117.stdout:Total 1.0 MB/s | 2.0 MB 00:02 2024-02-16T19:17:59.723 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-02-16T19:17:59.737 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-02-16T19:17:59.737 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-02-16T19:17:59.838 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-02-16T19:17:59.838 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-02-16T19:18:00.365 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-02-16T19:18:00.552 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-02-16T19:18:00.603 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-02-16T19:18:00.659 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-02-16T19:18:00.689 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-02-16T19:18:00.730 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-02-16T19:18:00.823 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-02-16T19:18:00.932 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-google-auth-1:2.27.0-1.el9.noarch 7/13 2024-02-16T19:18:00.980 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-02-16T19:18:01.705 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-02-16T19:18:01.753 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-02-16T19:18:01.787 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-02-16T19:18:01.832 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-02-16T19:18:01.857 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-rook-2:18.2.0-0.el9.noarch 13/13 2024-02-16T19:18:02.667 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.0-0.el9.noarch 13/13 2024-02-16T19:18:02.667 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-rook-2:18.2.0-0.el9.noarch 1/13 2024-02-16T19:18:02.668 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-02-16T19:18:02.668 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-02-16T19:18:02.668 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-02-16T19:18:02.668 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-02-16T19:18:02.668 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-02-16T19:18:02.668 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-02-16T19:18:02.668 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-02-16T19:18:02.668 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-02-16T19:18:02.668 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-google-auth-1:2.27.0-1.el9.noarch 10/13 2024-02-16T19:18:02.668 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-02-16T19:18:02.668 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-02-16T19:18:03.060 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-02-16T19:18:03.061 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:18:03.061 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2024-02-16T19:18:03.061 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-rook-2:18.2.0-0.el9.noarch 2024-02-16T19:18:03.061 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-02-16T19:18:03.061 INFO:teuthology.orchestra.run.smithi117.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-02-16T19:18:03.061 INFO:teuthology.orchestra.run.smithi117.stdout: python3-google-auth-1:2.27.0-1.el9.noarch 2024-02-16T19:18:03.061 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-02-16T19:18:03.061 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-02-16T19:18:03.061 INFO:teuthology.orchestra.run.smithi117.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-02-16T19:18:03.062 INFO:teuthology.orchestra.run.smithi117.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-02-16T19:18:03.062 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-02-16T19:18:03.062 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-02-16T19:18:03.062 INFO:teuthology.orchestra.run.smithi117.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-02-16T19:18:03.062 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rsa-4.9-2.el9.noarch 2024-02-16T19:18:03.062 INFO:teuthology.orchestra.run.smithi117.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-02-16T19:18:03.062 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:18:03.062 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-02-16T19:18:03.306 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install ceph-mgr-cephadm 2024-02-16T19:18:03.887 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:01:13 ago on Fri 16 Feb 2024 07:16:50 PM UTC. 2024-02-16T19:18:04.057 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-02-16T19:18:04.057 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-02-16T19:18:04.058 INFO:teuthology.orchestra.run.smithi117.stdout: Package Arch Version Repository Size 2024-02-16T19:18:04.058 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-02-16T19:18:04.058 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2024-02-16T19:18:04.058 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-cephadm noarch 2:18.2.0-0.el9 ceph-noarch 127 k 2024-02-16T19:18:04.058 INFO:teuthology.orchestra.run.smithi117.stdout:Installing dependencies: 2024-02-16T19:18:04.058 INFO:teuthology.orchestra.run.smithi117.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2024-02-16T19:18:04.059 INFO:teuthology.orchestra.run.smithi117.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-02-16T19:18:04.059 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jinja2 noarch 2.11.3-5.el9 CentOS-AppStream 227 k 2024-02-16T19:18:04.059 INFO:teuthology.orchestra.run.smithi117.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-02-16T19:18:04.059 INFO:teuthology.orchestra.run.smithi117.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-02-16T19:18:04.059 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:18:04.059 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-02-16T19:18:04.060 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-02-16T19:18:04.060 INFO:teuthology.orchestra.run.smithi117.stdout:Install 6 Packages 2024-02-16T19:18:04.060 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:18:04.061 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 6.8 M 2024-02-16T19:18:04.061 INFO:teuthology.orchestra.run.smithi117.stdout:Installed size: 33 M 2024-02-16T19:18:04.061 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2024-02-16T19:18:04.255 INFO:teuthology.orchestra.run.smithi117.stdout:(1/6): ceph-mgr-cephadm-18.2.0-0.el9.noarch.rpm 1.0 MB/s | 127 kB 00:00 2024-02-16T19:18:04.322 INFO:teuthology.orchestra.run.smithi117.stdout:(2/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 1.2 MB/s | 227 kB 00:00 2024-02-16T19:18:04.647 INFO:teuthology.orchestra.run.smithi117.stdout:(3/6): python3-babel-2.9.1-2.el9.noarch.rpm 11 MB/s | 5.8 MB 00:00 2024-02-16T19:18:04.805 INFO:teuthology.orchestra.run.smithi117.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 119 kB/s | 58 kB 00:00 2024-02-16T19:18:05.014 INFO:teuthology.orchestra.run.smithi117.stdout:(5/6): python3-asyncssh-2.13.2-3.el9.noarch.rpm 723 kB/s | 548 kB 00:00 2024-02-16T19:18:05.131 INFO:teuthology.orchestra.run.smithi117.stdout:(6/6): python3-typing-extensions-4.4.0-2.el9.no 105 kB/s | 51 kB 00:00 2024-02-16T19:18:05.135 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-02-16T19:18:05.135 INFO:teuthology.orchestra.run.smithi117.stdout:Total 6.3 MB/s | 6.8 MB 00:01 2024-02-16T19:18:05.136 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-02-16T19:18:05.145 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-02-16T19:18:05.145 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-02-16T19:18:05.235 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-02-16T19:18:05.235 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-02-16T19:18:05.531 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-02-16T19:18:05.696 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-02-16T19:18:05.741 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 2/6 2024-02-16T19:18:06.258 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-02-16T19:18:06.343 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-02-16T19:18:06.402 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-02-16T19:18:06.436 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-cephadm-2:18.2.0-0.el9.noarch 6/6 2024-02-16T19:18:07.228 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.0-0.el9.noarch 6/6 2024-02-16T19:18:07.228 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-cephadm-2:18.2.0-0.el9.noarch 1/6 2024-02-16T19:18:07.228 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-02-16T19:18:07.228 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-02-16T19:18:07.228 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 4/6 2024-02-16T19:18:07.228 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-02-16T19:18:07.550 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-02-16T19:18:07.550 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:18:07.550 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2024-02-16T19:18:07.550 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-cephadm-2:18.2.0-0.el9.noarch 2024-02-16T19:18:07.550 INFO:teuthology.orchestra.run.smithi117.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2024-02-16T19:18:07.550 INFO:teuthology.orchestra.run.smithi117.stdout: python3-babel-2.9.1-2.el9.noarch 2024-02-16T19:18:07.550 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-02-16T19:18:07.550 INFO:teuthology.orchestra.run.smithi117.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-02-16T19:18:07.551 INFO:teuthology.orchestra.run.smithi117.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-02-16T19:18:07.551 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:18:07.551 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-02-16T19:18:07.833 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install ceph-fuse 2024-02-16T19:18:08.414 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:01:18 ago on Fri 16 Feb 2024 07:16:50 PM UTC. 2024-02-16T19:18:08.580 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-02-16T19:18:08.580 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-02-16T19:18:08.580 INFO:teuthology.orchestra.run.smithi117.stdout: Package Architecture Version Repository Size 2024-02-16T19:18:08.581 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-02-16T19:18:08.581 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2024-02-16T19:18:08.581 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-fuse x86_64 2:18.2.0-0.el9 ceph 835 k 2024-02-16T19:18:08.581 INFO:teuthology.orchestra.run.smithi117.stdout:Installing dependencies: 2024-02-16T19:18:08.581 INFO:teuthology.orchestra.run.smithi117.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-02-16T19:18:08.581 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:18:08.581 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-02-16T19:18:08.581 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-02-16T19:18:08.581 INFO:teuthology.orchestra.run.smithi117.stdout:Install 2 Packages 2024-02-16T19:18:08.581 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:18:08.582 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 913 k 2024-02-16T19:18:08.582 INFO:teuthology.orchestra.run.smithi117.stdout:Installed size: 2.7 M 2024-02-16T19:18:08.582 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2024-02-16T19:18:08.682 INFO:teuthology.orchestra.run.smithi117.stdout:(1/2): ceph-fuse-18.2.0-0.el9.x86_64.rpm 8.1 MB/s | 835 kB 00:00 2024-02-16T19:18:08.799 INFO:teuthology.orchestra.run.smithi117.stdout:(2/2): fuse-2.9.9-15.el9.x86_64.rpm 362 kB/s | 79 kB 00:00 2024-02-16T19:18:08.801 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-02-16T19:18:08.801 INFO:teuthology.orchestra.run.smithi117.stdout:Total 4.1 MB/s | 913 kB 00:00 2024-02-16T19:18:08.801 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-02-16T19:18:08.821 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-02-16T19:18:08.821 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-02-16T19:18:08.902 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-02-16T19:18:08.902 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-02-16T19:18:09.088 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-02-16T19:18:09.213 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-02-16T19:18:09.283 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-fuse-2:18.2.0-0.el9.x86_64 2/2 2024-02-16T19:18:10.026 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-fuse-2:18.2.0-0.el9.x86_64 2/2 2024-02-16T19:18:10.026 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-fuse-2:18.2.0-0.el9.x86_64 1/2 2024-02-16T19:18:10.349 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-02-16T19:18:10.349 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:18:10.349 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2024-02-16T19:18:10.349 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-fuse-2:18.2.0-0.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-02-16T19:18:10.349 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:18:10.349 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-02-16T19:18:10.611 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install librados-devel 2024-02-16T19:18:11.205 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:01:21 ago on Fri 16 Feb 2024 07:16:50 PM UTC. 2024-02-16T19:18:11.374 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-02-16T19:18:11.374 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-02-16T19:18:11.374 INFO:teuthology.orchestra.run.smithi117.stdout: Package Architecture Version Repository Size 2024-02-16T19:18:11.375 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-02-16T19:18:11.375 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2024-02-16T19:18:11.375 INFO:teuthology.orchestra.run.smithi117.stdout: librados-devel x86_64 2:18.2.0-0.el9 ceph 126 k 2024-02-16T19:18:11.375 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:18:11.375 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-02-16T19:18:11.375 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-02-16T19:18:11.375 INFO:teuthology.orchestra.run.smithi117.stdout:Install 1 Package 2024-02-16T19:18:11.375 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:18:11.375 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 126 k 2024-02-16T19:18:11.375 INFO:teuthology.orchestra.run.smithi117.stdout:Installed size: 456 k 2024-02-16T19:18:11.375 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2024-02-16T19:18:11.445 INFO:teuthology.orchestra.run.smithi117.stdout:librados-devel-18.2.0-0.el9.x86_64.rpm 1.8 MB/s | 126 kB 00:00 2024-02-16T19:18:11.446 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-02-16T19:18:11.446 INFO:teuthology.orchestra.run.smithi117.stdout:Total 1.7 MB/s | 126 kB 00:00 2024-02-16T19:18:11.446 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-02-16T19:18:11.452 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-02-16T19:18:11.452 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-02-16T19:18:11.508 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-02-16T19:18:11.508 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-02-16T19:18:11.650 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-02-16T19:18:11.743 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : librados-devel-2:18.2.0-0.el9.x86_64 1/1 2024-02-16T19:18:12.198 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librados-devel-2:18.2.0-0.el9.x86_64 1/1 2024-02-16T19:18:12.472 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librados-devel-2:18.2.0-0.el9.x86_64 1/1 2024-02-16T19:18:12.472 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:18:12.472 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2024-02-16T19:18:12.472 INFO:teuthology.orchestra.run.smithi117.stdout: librados-devel-2:18.2.0-0.el9.x86_64 2024-02-16T19:18:12.472 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:18:12.472 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-02-16T19:18:12.755 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install libcephfs2 2024-02-16T19:18:13.352 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:01:23 ago on Fri 16 Feb 2024 07:16:50 PM UTC. 2024-02-16T19:18:13.478 INFO:teuthology.orchestra.run.smithi117.stdout:Package libcephfs2-2:18.2.0-0.el9.x86_64 is already installed. 2024-02-16T19:18:13.529 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-02-16T19:18:13.530 INFO:teuthology.orchestra.run.smithi117.stdout:Nothing to do. 2024-02-16T19:18:13.530 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-02-16T19:18:13.594 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install libcephfs-devel 2024-02-16T19:18:14.172 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:01:24 ago on Fri 16 Feb 2024 07:16:50 PM UTC. 2024-02-16T19:18:14.336 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-02-16T19:18:14.337 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-02-16T19:18:14.338 INFO:teuthology.orchestra.run.smithi117.stdout: Package Architecture Version Repository Size 2024-02-16T19:18:14.338 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-02-16T19:18:14.338 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2024-02-16T19:18:14.338 INFO:teuthology.orchestra.run.smithi117.stdout: libcephfs-devel x86_64 2:18.2.0-0.el9 ceph 30 k 2024-02-16T19:18:14.338 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:18:14.338 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-02-16T19:18:14.338 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-02-16T19:18:14.338 INFO:teuthology.orchestra.run.smithi117.stdout:Install 1 Package 2024-02-16T19:18:14.339 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:18:14.339 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 30 k 2024-02-16T19:18:14.339 INFO:teuthology.orchestra.run.smithi117.stdout:Installed size: 137 k 2024-02-16T19:18:14.339 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2024-02-16T19:18:14.409 INFO:teuthology.orchestra.run.smithi117.stdout:libcephfs-devel-18.2.0-0.el9.x86_64.rpm 430 kB/s | 30 kB 00:00 2024-02-16T19:18:14.410 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-02-16T19:18:14.410 INFO:teuthology.orchestra.run.smithi117.stdout:Total 423 kB/s | 30 kB 00:00 2024-02-16T19:18:14.410 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-02-16T19:18:14.413 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-02-16T19:18:14.413 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-02-16T19:18:14.447 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-02-16T19:18:14.447 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-02-16T19:18:14.532 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-02-16T19:18:14.639 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libcephfs-devel-2:18.2.0-0.el9.x86_64 1/1 2024-02-16T19:18:15.030 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: libcephfs-devel-2:18.2.0-0.el9.x86_64 1/1 2024-02-16T19:18:15.313 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libcephfs-devel-2:18.2.0-0.el9.x86_64 1/1 2024-02-16T19:18:15.313 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:18:15.314 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2024-02-16T19:18:15.314 INFO:teuthology.orchestra.run.smithi117.stdout: libcephfs-devel-2:18.2.0-0.el9.x86_64 2024-02-16T19:18:15.314 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:18:15.314 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-02-16T19:18:15.502 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install librados2 2024-02-16T19:18:16.075 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:01:26 ago on Fri 16 Feb 2024 07:16:50 PM UTC. 2024-02-16T19:18:16.188 INFO:teuthology.orchestra.run.smithi117.stdout:Package librados2-2:18.2.0-0.el9.x86_64 is already installed. 2024-02-16T19:18:16.240 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-02-16T19:18:16.241 INFO:teuthology.orchestra.run.smithi117.stdout:Nothing to do. 2024-02-16T19:18:16.241 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-02-16T19:18:16.306 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install librbd1 2024-02-16T19:18:16.879 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:01:26 ago on Fri 16 Feb 2024 07:16:50 PM UTC. 2024-02-16T19:18:16.992 INFO:teuthology.orchestra.run.smithi117.stdout:Package librbd1-2:18.2.0-0.el9.x86_64 is already installed. 2024-02-16T19:18:17.044 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-02-16T19:18:17.045 INFO:teuthology.orchestra.run.smithi117.stdout:Nothing to do. 2024-02-16T19:18:17.045 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-02-16T19:18:17.101 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install python3-rados 2024-02-16T19:18:17.671 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:01:27 ago on Fri 16 Feb 2024 07:16:50 PM UTC. 2024-02-16T19:18:17.785 INFO:teuthology.orchestra.run.smithi117.stdout:Package python3-rados-2:18.2.0-0.el9.x86_64 is already installed. 2024-02-16T19:18:17.837 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-02-16T19:18:17.838 INFO:teuthology.orchestra.run.smithi117.stdout:Nothing to do. 2024-02-16T19:18:17.838 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-02-16T19:18:17.905 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install python3-rgw 2024-02-16T19:18:18.494 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:01:28 ago on Fri 16 Feb 2024 07:16:50 PM UTC. 2024-02-16T19:18:18.620 INFO:teuthology.orchestra.run.smithi117.stdout:Package python3-rgw-2:18.2.0-0.el9.x86_64 is already installed. 2024-02-16T19:18:18.673 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-02-16T19:18:18.674 INFO:teuthology.orchestra.run.smithi117.stdout:Nothing to do. 2024-02-16T19:18:18.674 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-02-16T19:18:18.738 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install python3-cephfs 2024-02-16T19:18:19.315 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:01:29 ago on Fri 16 Feb 2024 07:16:50 PM UTC. 2024-02-16T19:18:19.431 INFO:teuthology.orchestra.run.smithi117.stdout:Package python3-cephfs-2:18.2.0-0.el9.x86_64 is already installed. 2024-02-16T19:18:19.485 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-02-16T19:18:19.486 INFO:teuthology.orchestra.run.smithi117.stdout:Nothing to do. 2024-02-16T19:18:19.486 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-02-16T19:18:19.554 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install python3-rbd 2024-02-16T19:18:20.128 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:01:30 ago on Fri 16 Feb 2024 07:16:50 PM UTC. 2024-02-16T19:18:20.241 INFO:teuthology.orchestra.run.smithi117.stdout:Package python3-rbd-2:18.2.0-0.el9.x86_64 is already installed. 2024-02-16T19:18:20.293 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-02-16T19:18:20.294 INFO:teuthology.orchestra.run.smithi117.stdout:Nothing to do. 2024-02-16T19:18:20.294 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-02-16T19:18:20.363 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install rbd-fuse 2024-02-16T19:18:20.944 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:01:30 ago on Fri 16 Feb 2024 07:16:50 PM UTC. 2024-02-16T19:18:21.108 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-02-16T19:18:21.109 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-02-16T19:18:21.109 INFO:teuthology.orchestra.run.smithi117.stdout: Package Architecture Version Repository Size 2024-02-16T19:18:21.109 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-02-16T19:18:21.110 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2024-02-16T19:18:21.110 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-fuse x86_64 2:18.2.0-0.el9 ceph 86 k 2024-02-16T19:18:21.110 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:18:21.110 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-02-16T19:18:21.110 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-02-16T19:18:21.110 INFO:teuthology.orchestra.run.smithi117.stdout:Install 1 Package 2024-02-16T19:18:21.110 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:18:21.110 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 86 k 2024-02-16T19:18:21.110 INFO:teuthology.orchestra.run.smithi117.stdout:Installed size: 230 k 2024-02-16T19:18:21.110 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2024-02-16T19:18:21.170 INFO:teuthology.orchestra.run.smithi117.stdout:rbd-fuse-18.2.0-0.el9.x86_64.rpm 1.4 MB/s | 86 kB 00:00 2024-02-16T19:18:21.171 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-02-16T19:18:21.171 INFO:teuthology.orchestra.run.smithi117.stdout:Total 1.4 MB/s | 86 kB 00:00 2024-02-16T19:18:21.171 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-02-16T19:18:21.178 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-02-16T19:18:21.178 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-02-16T19:18:21.232 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-02-16T19:18:21.232 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-02-16T19:18:21.366 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-02-16T19:18:21.476 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : rbd-fuse-2:18.2.0-0.el9.x86_64 1/1 2024-02-16T19:18:21.862 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: rbd-fuse-2:18.2.0-0.el9.x86_64 1/1 2024-02-16T19:18:22.147 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : rbd-fuse-2:18.2.0-0.el9.x86_64 1/1 2024-02-16T19:18:22.147 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:18:22.147 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2024-02-16T19:18:22.147 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-fuse-2:18.2.0-0.el9.x86_64 2024-02-16T19:18:22.147 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:18:22.147 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-02-16T19:18:22.363 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install rbd-mirror 2024-02-16T19:18:22.945 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:01:32 ago on Fri 16 Feb 2024 07:16:50 PM UTC. 2024-02-16T19:18:23.116 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-02-16T19:18:23.117 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-02-16T19:18:23.117 INFO:teuthology.orchestra.run.smithi117.stdout: Package Architecture Version Repository Size 2024-02-16T19:18:23.117 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-02-16T19:18:23.117 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2024-02-16T19:18:23.117 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-mirror x86_64 2:18.2.0-0.el9 ceph 3.0 M 2024-02-16T19:18:23.117 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:18:23.117 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-02-16T19:18:23.117 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-02-16T19:18:23.117 INFO:teuthology.orchestra.run.smithi117.stdout:Install 1 Package 2024-02-16T19:18:23.117 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:18:23.117 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 3.0 M 2024-02-16T19:18:23.117 INFO:teuthology.orchestra.run.smithi117.stdout:Installed size: 12 M 2024-02-16T19:18:23.118 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2024-02-16T19:18:23.215 INFO:teuthology.orchestra.run.smithi117.stdout:rbd-mirror-18.2.0-0.el9.x86_64.rpm 31 MB/s | 3.0 MB 00:00 2024-02-16T19:18:23.216 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-02-16T19:18:23.216 INFO:teuthology.orchestra.run.smithi117.stdout:Total 30 MB/s | 3.0 MB 00:00 2024-02-16T19:18:23.216 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-02-16T19:18:23.225 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-02-16T19:18:23.225 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-02-16T19:18:23.297 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-02-16T19:18:23.297 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-02-16T19:18:23.704 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-02-16T19:18:23.758 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : rbd-mirror-2:18.2.0-0.el9.x86_64 1/1 2024-02-16T19:18:23.782 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: rbd-mirror-2:18.2.0-0.el9.x86_64 1/1 2024-02-16T19:18:23.782 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-16T19:18:23.782 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-02-16T19:18:23.782 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-16T19:18:23.782 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-16T19:18:23.782 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:18:24.863 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : rbd-mirror-2:18.2.0-0.el9.x86_64 1/1 2024-02-16T19:18:24.863 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:18:24.863 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2024-02-16T19:18:24.863 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-mirror-2:18.2.0-0.el9.x86_64 2024-02-16T19:18:24.863 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:18:24.863 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-02-16T19:18:25.110 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install rbd-nbd 2024-02-16T19:18:25.695 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:01:35 ago on Fri 16 Feb 2024 07:16:50 PM UTC. 2024-02-16T19:18:25.860 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-02-16T19:18:25.861 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-02-16T19:18:25.861 INFO:teuthology.orchestra.run.smithi117.stdout: Package Architecture Version Repository Size 2024-02-16T19:18:25.861 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-02-16T19:18:25.861 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2024-02-16T19:18:25.861 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-nbd x86_64 2:18.2.0-0.el9 ceph 169 k 2024-02-16T19:18:25.861 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:18:25.861 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-02-16T19:18:25.861 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-02-16T19:18:25.861 INFO:teuthology.orchestra.run.smithi117.stdout:Install 1 Package 2024-02-16T19:18:25.861 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:18:25.862 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 169 k 2024-02-16T19:18:25.862 INFO:teuthology.orchestra.run.smithi117.stdout:Installed size: 490 k 2024-02-16T19:18:25.862 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2024-02-16T19:18:25.927 INFO:teuthology.orchestra.run.smithi117.stdout:rbd-nbd-18.2.0-0.el9.x86_64.rpm 2.6 MB/s | 169 kB 00:00 2024-02-16T19:18:25.927 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-02-16T19:18:25.928 INFO:teuthology.orchestra.run.smithi117.stdout:Total 2.5 MB/s | 169 kB 00:00 2024-02-16T19:18:25.928 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-02-16T19:18:25.935 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-02-16T19:18:25.935 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-02-16T19:18:25.990 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-02-16T19:18:25.990 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-02-16T19:18:26.146 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-02-16T19:18:26.249 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : rbd-nbd-2:18.2.0-0.el9.x86_64 1/1 2024-02-16T19:18:26.750 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: rbd-nbd-2:18.2.0-0.el9.x86_64 1/1 2024-02-16T19:18:27.077 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : rbd-nbd-2:18.2.0-0.el9.x86_64 1/1 2024-02-16T19:18:27.077 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:18:27.077 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2024-02-16T19:18:27.077 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-nbd-2:18.2.0-0.el9.x86_64 2024-02-16T19:18:27.077 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:18:27.077 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-02-16T19:18:27.280 DEBUG:teuthology.parallel:result is None 2024-02-16T19:18:27.281 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-02-16T19:18:27.281 INFO:teuthology.packaging:ref: None 2024-02-16T19:18:27.281 INFO:teuthology.packaging:tag: v18.2.0 2024-02-16T19:18:27.281 INFO:teuthology.packaging:branch: None 2024-02-16T19:18:27.281 INFO:teuthology.packaging:sha1: f78a58c0ffd401d1493058a1022c35f011d65275 2024-02-16T19:18:27.281 INFO:teuthology.packaging:Tag 'v18.2.0' not found in https://git.ceph.com/ceph-ci; will also look in https://git.ceph.com/ceph 2024-02-16T19:18:27.281 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5dd24139a1eada541a3bc16b6941c5dde975e26d 2024-02-16T19:18:27.509 DEBUG:teuthology.orchestra.run.smithi022:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-16T19:18:27.533 INFO:teuthology.orchestra.run.smithi022.stdout:18.2.0-0.el9 2024-02-16T19:18:27.533 INFO:teuthology.packaging:The installed version of ceph is 18.2.0-0.el9 2024-02-16T19:18:27.533 INFO:teuthology.task.install:The correct ceph version 18.2.0-0 is installed. 2024-02-16T19:18:27.535 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-02-16T19:18:27.535 INFO:teuthology.packaging:ref: None 2024-02-16T19:18:27.535 INFO:teuthology.packaging:tag: v18.2.0 2024-02-16T19:18:27.535 INFO:teuthology.packaging:branch: None 2024-02-16T19:18:27.536 INFO:teuthology.packaging:sha1: f78a58c0ffd401d1493058a1022c35f011d65275 2024-02-16T19:18:27.536 INFO:teuthology.packaging:Tag 'v18.2.0' not found in https://git.ceph.com/ceph-ci; will also look in https://git.ceph.com/ceph 2024-02-16T19:18:27.536 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5dd24139a1eada541a3bc16b6941c5dde975e26d 2024-02-16T19:18:27.679 DEBUG:teuthology.orchestra.run.smithi117:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-16T19:18:27.700 INFO:teuthology.orchestra.run.smithi117.stdout:18.2.0-0.el9 2024-02-16T19:18:27.701 INFO:teuthology.packaging:The installed version of ceph is 18.2.0-0.el9 2024-02-16T19:18:27.701 INFO:teuthology.task.install:The correct ceph version 18.2.0-0 is installed. 2024-02-16T19:18:27.702 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-02-16T19:18:27.702 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-02-16T19:18:27.703 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-16T19:18:27.735 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-02-16T19:18:27.736 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-16T19:18:27.768 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-02-16T19:18:27.768 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-02-16T19:18:27.768 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/usr/bin/daemon-helper 2024-02-16T19:18:27.805 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-16T19:18:27.875 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-02-16T19:18:27.875 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/usr/bin/daemon-helper 2024-02-16T19:18:27.920 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-16T19:18:27.987 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-02-16T19:18:27.988 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-02-16T19:18:27.988 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-16T19:18:28.017 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-16T19:18:28.085 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-02-16T19:18:28.085 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-16T19:18:28.111 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-16T19:18:28.178 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-02-16T19:18:28.178 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-02-16T19:18:28.178 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/usr/bin/stdin-killer 2024-02-16T19:18:28.205 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-16T19:18:28.272 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-02-16T19:18:28.272 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/usr/bin/stdin-killer 2024-02-16T19:18:28.299 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-16T19:18:28.365 INFO:teuthology.run_tasks:Running task print... 2024-02-16T19:18:28.374 INFO:teuthology.task.print:**** done install task... 2024-02-16T19:18:28.375 INFO:teuthology.run_tasks:Running task cephadm... 2024-02-16T19:18:28.465 INFO:tasks.cephadm:Config: {'compiled_cephadm_branch': 'reef', 'conf': {'osd': {'osd_class_default_list': '*', 'osd_class_load_list': '*', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'mon pg warn min per osd': 0}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}}, 'image': 'quay.io/ceph/ceph:v18.2.0', 'roleless': True, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'OSD_DOWN', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': 'f78a58c0ffd401d1493058a1022c35f011d65275'} 2024-02-16T19:18:28.465 INFO:tasks.cephadm:Cluster image is quay.io/ceph/ceph:v18.2.0 2024-02-16T19:18:28.466 INFO:tasks.cephadm:Cluster fsid is 2a1346e6-cd00-11ee-95ba-87774f69a715 2024-02-16T19:18:28.466 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-02-16T19:18:28.466 INFO:tasks.cephadm:No mon roles; fabricating mons 2024-02-16T19:18:28.466 INFO:tasks.cephadm:Monitor IPs: {'mon.smithi022': '172.21.15.22', 'mon.smithi117': '172.21.15.117'} 2024-02-16T19:18:28.466 INFO:tasks.cephadm:Normalizing hostnames... 2024-02-16T19:18:28.467 DEBUG:teuthology.orchestra.run.smithi022:> sudo hostname $(hostname -s) 2024-02-16T19:18:28.493 DEBUG:teuthology.orchestra.run.smithi117:> sudo hostname $(hostname -s) 2024-02-16T19:18:28.520 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra for reef 2024-02-16T19:18:28.520 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=f78a58c0ffd401d1493058a1022c35f011d65275 2024-02-16T19:18:28.660 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'f78a58c0ffd401d1493058a1022c35f011d65275', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/76790/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-1450-gf78a58c0', 'node_name': '172.21.2.1+braggi01', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-1450.gf78a58c0'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240215.160715/f78a58c0ffd401d1493058a1022c35f011d65275/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-02-15 19:25:41.455235', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-batrick-testing-20240215.160715', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-batrick-testing-20240215.160715/f78a58c0ffd401d1493058a1022c35f011d65275/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}, {'status': 'ready', 'sha1': 'f78a58c0ffd401d1493058a1022c35f011d65275', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/76792/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-1450-gf78a58c0', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-1450.gf78a58c0'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240215.160715/f78a58c0ffd401d1493058a1022c35f011d65275/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-02-15 19:17:06.436952', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-batrick-testing-20240215.160715', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-batrick-testing-20240215.160715/f78a58c0ffd401d1493058a1022c35f011d65275/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-02-16T19:18:28.997 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref reef, sha1 744c573dfc29e50959567861c524f9e6c038171f from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&ref=reef 2024-02-16T19:18:28.999 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/reef/744c573dfc29e50959567861c524f9e6c038171f/centos/9/x86_64/flavors/default/cephadm 2024-02-16T19:18:28.999 DEBUG:teuthology.orchestra.run.smithi022:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/reef/744c573dfc29e50959567861c524f9e6c038171f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-16T19:18:29.541 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-r--r--. 1 ubuntu ubuntu 214739 Feb 16 19:18 /home/ubuntu/cephtest/cephadm 2024-02-16T19:18:29.542 DEBUG:teuthology.orchestra.run.smithi117:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/reef/744c573dfc29e50959567861c524f9e6c038171f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-16T19:18:29.922 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r--. 1 ubuntu ubuntu 214739 Feb 16 19:18 /home/ubuntu/cephtest/cephadm 2024-02-16T19:18:29.922 DEBUG:teuthology.orchestra.run.smithi022:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-16T19:18:29.939 DEBUG:teuthology.orchestra.run.smithi117:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-16T19:18:29.969 INFO:tasks.cephadm:Pulling image quay.io/ceph/ceph:v18.2.0 on all hosts... 2024-02-16T19:18:29.969 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 pull 2024-02-16T19:18:29.981 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 pull 2024-02-16T19:18:30.717 INFO:teuthology.orchestra.run.smithi022.stderr:Pulling container image quay.io/ceph/ceph:v18.2.0... 2024-02-16T19:18:30.870 INFO:teuthology.orchestra.run.smithi117.stderr:Pulling container image quay.io/ceph/ceph:v18.2.0... 2024-02-16T19:18:48.630 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-02-16T19:18:48.631 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_version": "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)", 2024-02-16T19:18:48.631 INFO:teuthology.orchestra.run.smithi022.stdout: "image_id": "dc2bc1663786ad5862ef6efcff18243ab7dfeda0bcb911ddf757a0d20feca946", 2024-02-16T19:18:48.631 INFO:teuthology.orchestra.run.smithi022.stdout: "repo_digests": [ 2024-02-16T19:18:48.631 INFO:teuthology.orchestra.run.smithi022.stdout: "quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007", 2024-02-16T19:18:48.631 INFO:teuthology.orchestra.run.smithi022.stdout: "quay.io/ceph/ceph@sha256:d58cf65589d0abf9d5261cd46fb62be3bfb29098febc78c0fcdf116a15274d27" 2024-02-16T19:18:48.631 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-02-16T19:18:48.631 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-02-16T19:18:50.540 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-02-16T19:18:50.540 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_version": "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)", 2024-02-16T19:18:50.541 INFO:teuthology.orchestra.run.smithi117.stdout: "image_id": "dc2bc1663786ad5862ef6efcff18243ab7dfeda0bcb911ddf757a0d20feca946", 2024-02-16T19:18:50.541 INFO:teuthology.orchestra.run.smithi117.stdout: "repo_digests": [ 2024-02-16T19:18:50.541 INFO:teuthology.orchestra.run.smithi117.stdout: "quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007", 2024-02-16T19:18:50.541 INFO:teuthology.orchestra.run.smithi117.stdout: "quay.io/ceph/ceph@sha256:d58cf65589d0abf9d5261cd46fb62be3bfb29098febc78c0fcdf116a15274d27" 2024-02-16T19:18:50.541 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-02-16T19:18:50.541 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-02-16T19:18:50.559 DEBUG:teuthology.orchestra.run.smithi022:> sudo mkdir -p /etc/ceph 2024-02-16T19:18:50.594 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /etc/ceph 2024-02-16T19:18:50.623 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 777 /etc/ceph 2024-02-16T19:18:50.664 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 777 /etc/ceph 2024-02-16T19:18:50.690 INFO:tasks.cephadm:Writing seed config... 2024-02-16T19:18:50.691 INFO:tasks.cephadm: override: [osd] osd_class_default_list = * 2024-02-16T19:18:50.691 INFO:tasks.cephadm: override: [osd] osd_class_load_list = * 2024-02-16T19:18:50.691 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-02-16T19:18:50.691 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-02-16T19:18:50.692 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-02-16T19:18:50.692 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-02-16T19:18:50.692 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-02-16T19:18:50.692 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-02-16T19:18:50.692 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-02-16T19:18:50.692 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-02-16T19:18:50.692 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-02-16T19:18:50.692 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-02-16T19:18:50.692 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-02-16T19:18:50.692 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-02-16T19:18:50.692 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-02-16T19:18:50.692 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-02-16T19:18:50.692 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-02-16T19:18:50.692 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-02-16T19:18:50.693 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-02-16T19:18:50.693 INFO:tasks.cephadm: override: [client] debug client = 20 2024-02-16T19:18:50.693 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-02-16T19:18:50.693 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-02-16T19:18:50.693 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-02-16T19:18:50.693 INFO:tasks.cephadm: override: [global] mon pg warn min per osd = 0 2024-02-16T19:18:50.693 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-02-16T19:18:50.693 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-02-16T19:18:50.693 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-02-16T19:18:50.693 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-02-16T19:18:50.693 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-02-16T19:18:50.693 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-02-16T19:18:50.693 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-02-16T19:18:50.693 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-02-16T19:18:50.693 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-02-16T19:18:50.693 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-02-16T19:18:50.694 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-02-16T19:18:50.694 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-02-16T19:18:50.694 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-02-16T19:18:50.694 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-02-16T19:18:50.694 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-02-16T19:18:50.694 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-02-16T19:18:50.694 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-02-16T19:18:50.694 DEBUG:teuthology.orchestra.run.smithi022:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-02-16T19:18:50.720 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 2a1346e6-cd00-11ee-95ba-87774f69a715 mon pg warn min per osd = 0 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = true bdev debug aio = true osd sloppy crc = true osd_class_default_list = * osd_class_load_list = * bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-02-16T19:18:50.720 DEBUG:teuthology.orchestra.run.smithi022:mon.smithi022> sudo journalctl -f -n 0 -u ceph-2a1346e6-cd00-11ee-95ba-87774f69a715@mon.smithi022.service 2024-02-16T19:18:50.763 INFO:tasks.cephadm:Bootstrapping... 2024-02-16T19:18:50.763 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 -v bootstrap --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-ip 172.21.15.22 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-02-16T19:18:50.929 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2024-02-16T19:18:50.929 INFO:teuthology.orchestra.run.smithi022.stdout:cephadm ['--image', 'quay.io/ceph/ceph:v18.2.0', '-v', 'bootstrap', '--fsid', '2a1346e6-cd00-11ee-95ba-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-ip', '172.21.15.22', '--skip-admin-label'] 2024-02-16T19:18:50.953 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-16T19:18:50.954 INFO:teuthology.orchestra.run.smithi022.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-02-16T19:18:50.954 INFO:teuthology.orchestra.run.smithi022.stdout:Verifying podman|docker is present... 2024-02-16T19:18:50.976 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-16T19:18:50.976 INFO:teuthology.orchestra.run.smithi022.stdout:Verifying lvm2 is present... 2024-02-16T19:18:50.976 INFO:teuthology.orchestra.run.smithi022.stdout:Verifying time synchronization is in place... 2024-02-16T19:18:50.983 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-02-16T19:18:50.983 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-02-16T19:18:50.989 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-02-16T19:18:50.989 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout inactive 2024-02-16T19:18:50.995 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout enabled 2024-02-16T19:18:51.001 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout active 2024-02-16T19:18:51.001 INFO:teuthology.orchestra.run.smithi022.stdout:Unit chronyd.service is enabled and running 2024-02-16T19:18:51.001 INFO:teuthology.orchestra.run.smithi022.stdout:Repeating the final host check... 2024-02-16T19:18:51.024 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-16T19:18:51.024 INFO:teuthology.orchestra.run.smithi022.stdout:podman (/bin/podman) version 4.9.3 is present 2024-02-16T19:18:51.024 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl is present 2024-02-16T19:18:51.024 INFO:teuthology.orchestra.run.smithi022.stdout:lvcreate is present 2024-02-16T19:18:51.030 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-02-16T19:18:51.030 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-02-16T19:18:51.035 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-02-16T19:18:51.035 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout inactive 2024-02-16T19:18:51.041 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout enabled 2024-02-16T19:18:51.047 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout active 2024-02-16T19:18:51.047 INFO:teuthology.orchestra.run.smithi022.stdout:Unit chronyd.service is enabled and running 2024-02-16T19:18:51.048 INFO:teuthology.orchestra.run.smithi022.stdout:Host looks OK 2024-02-16T19:18:51.048 INFO:teuthology.orchestra.run.smithi022.stdout:Cluster fsid: 2a1346e6-cd00-11ee-95ba-87774f69a715 2024-02-16T19:18:51.048 INFO:teuthology.orchestra.run.smithi022.stdout:Acquiring lock 139863543826656 on /run/cephadm/2a1346e6-cd00-11ee-95ba-87774f69a715.lock 2024-02-16T19:18:51.048 INFO:teuthology.orchestra.run.smithi022.stdout:Lock 139863543826656 acquired on /run/cephadm/2a1346e6-cd00-11ee-95ba-87774f69a715.lock 2024-02-16T19:18:51.048 INFO:teuthology.orchestra.run.smithi022.stdout:Verifying IP 172.21.15.22 port 3300 ... 2024-02-16T19:18:51.049 INFO:teuthology.orchestra.run.smithi022.stdout:Verifying IP 172.21.15.22 port 6789 ... 2024-02-16T19:18:51.049 INFO:teuthology.orchestra.run.smithi022.stdout:Base mon IP(s) is [172.21.15.22:3300, 172.21.15.22:6789], mon addrv is [v2:172.21.15.22:3300,v1:172.21.15.22:6789] 2024-02-16T19:18:51.052 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.22 metric 100 2024-02-16T19:18:51.052 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.22 metric 100 2024-02-16T19:18:51.054 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-02-16T19:18:51.054 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-02-16T19:18:51.054 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-02-16T19:18:51.056 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-02-16T19:18:51.056 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-02-16T19:18:51.056 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-02-16T19:18:51.056 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-02-16T19:18:51.057 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe59:5de4/64 scope link noprefixroute 2024-02-16T19:18:51.057 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-02-16T19:18:51.057 INFO:teuthology.orchestra.run.smithi022.stdout:Mon IP `172.21.15.22` is in CIDR network `172.21.0.0/20` 2024-02-16T19:18:51.057 INFO:teuthology.orchestra.run.smithi022.stdout:Mon IP `172.21.15.22` is in CIDR network `172.21.0.0/20` 2024-02-16T19:18:51.057 INFO:teuthology.orchestra.run.smithi022.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-02-16T19:18:51.058 INFO:teuthology.orchestra.run.smithi022.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-02-16T19:18:51.058 INFO:teuthology.orchestra.run.smithi022.stdout:Pulling container image quay.io/ceph/ceph:v18.2.0... 2024-02-16T19:18:54.739 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/podman: stdout dc2bc1663786ad5862ef6efcff18243ab7dfeda0bcb911ddf757a0d20feca946 2024-02-16T19:18:54.739 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/podman: stderr Trying to pull quay.io/ceph/ceph:v18.2.0... 2024-02-16T19:18:54.739 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/podman: stderr Getting image source signatures 2024-02-16T19:18:54.739 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/podman: stderr Copying blob sha256:3bd20aeff60302f668275dc2005d10679ae56492967a3a5a54fd3dde85333aec 2024-02-16T19:18:54.739 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/podman: stderr Copying blob sha256:46af8f5390d4e94fc57efb422ccb97bb53dfe5b948546bfc191b46557eb2dbd9 2024-02-16T19:18:54.739 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/podman: stderr Copying config sha256:dc2bc1663786ad5862ef6efcff18243ab7dfeda0bcb911ddf757a0d20feca946 2024-02-16T19:18:54.739 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/podman: stderr Writing manifest to image destination 2024-02-16T19:19:03.544 INFO:teuthology.orchestra.run.smithi022.stdout:ceph: stdout ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable) 2024-02-16T19:19:03.544 INFO:teuthology.orchestra.run.smithi022.stdout:Ceph version: ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable) 2024-02-16T19:19:03.544 INFO:teuthology.orchestra.run.smithi022.stdout:Extracting ceph user uid/gid from container image... 2024-02-16T19:19:04.457 INFO:teuthology.orchestra.run.smithi022.stdout:stat: stdout 167 167 2024-02-16T19:19:04.457 INFO:teuthology.orchestra.run.smithi022.stdout:Creating initial keys... 2024-02-16T19:19:05.321 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph-authtool: stdout AQAotc9l2G8cMBAAdHEC/yDPXWaA/YoeJBJEgQ== 2024-02-16T19:19:06.252 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph-authtool: stdout AQAptc9ldcGBJRAA5wVc8ulrfCnFEEmXb6RASg== 2024-02-16T19:19:07.132 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph-authtool: stdout AQAqtc9lbgh6IhAA3XUM4imagOqDh54KGoMXeQ== 2024-02-16T19:19:07.133 INFO:teuthology.orchestra.run.smithi022.stdout:Creating initial monmap... 2024-02-16T19:19:07.989 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-16T19:19:07.989 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2024-02-16T19:19:07.989 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 2a1346e6-cd00-11ee-95ba-87774f69a715 2024-02-16T19:19:07.989 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-16T19:19:07.989 INFO:teuthology.orchestra.run.smithi022.stdout:monmaptool for smithi022 [v2:172.21.15.22:3300,v1:172.21.15.22:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-16T19:19:07.989 INFO:teuthology.orchestra.run.smithi022.stdout:setting min_mon_release = pacific 2024-02-16T19:19:07.989 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/monmaptool: set fsid to 2a1346e6-cd00-11ee-95ba-87774f69a715 2024-02-16T19:19:07.989 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-16T19:19:07.989 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:19:07.989 INFO:teuthology.orchestra.run.smithi022.stdout:Creating mon... 2024-02-16T19:19:10.003 INFO:teuthology.orchestra.run.smithi022.stdout:create mon.smithi022 on 2024-02-16T19:19:10.234 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-02-16T19:19:10.451 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-02-16T19:19:10.626 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-2a1346e6-cd00-11ee-95ba-87774f69a715.target → /etc/systemd/system/ceph-2a1346e6-cd00-11ee-95ba-87774f69a715.target. 2024-02-16T19:19:10.626 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-2a1346e6-cd00-11ee-95ba-87774f69a715.target → /etc/systemd/system/ceph-2a1346e6-cd00-11ee-95ba-87774f69a715.target. 2024-02-16T19:19:10.860 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-2a1346e6-cd00-11ee-95ba-87774f69a715@mon.smithi022 2024-02-16T19:19:10.860 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stderr Failed to reset failed state of unit ceph-2a1346e6-cd00-11ee-95ba-87774f69a715@mon.smithi022.service: Unit ceph-2a1346e6-cd00-11ee-95ba-87774f69a715@mon.smithi022.service not loaded. 2024-02-16T19:19:11.014 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-2a1346e6-cd00-11ee-95ba-87774f69a715.target.wants/ceph-2a1346e6-cd00-11ee-95ba-87774f69a715@mon.smithi022.service → /etc/systemd/system/ceph-2a1346e6-cd00-11ee-95ba-87774f69a715@.service. 2024-02-16T19:19:11.585 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-16T19:19:11.585 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout disabled 2024-02-16T19:19:11.591 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-16T19:19:11.591 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout inactive 2024-02-16T19:19:11.591 INFO:teuthology.orchestra.run.smithi022.stdout:firewalld.service is not enabled 2024-02-16T19:19:11.591 INFO:teuthology.orchestra.run.smithi022.stdout:Not possible to enable service . firewalld.service is not available 2024-02-16T19:19:11.591 INFO:teuthology.orchestra.run.smithi022.stdout:Waiting for mon to start... 2024-02-16T19:19:11.592 INFO:teuthology.orchestra.run.smithi022.stdout:Waiting for mon... 2024-02-16T19:19:11.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:11 smithi022 systemd[1]: Started Ceph mon.smithi022 for 2a1346e6-cd00-11ee-95ba-87774f69a715. 2024-02-16T19:19:12.654 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:12 smithi022 ceph-mon[28850]: mon.smithi022 is new leader, mons smithi022 in quorum (ranks 0) 2024-02-16T19:19:12.654 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:12 smithi022 ceph-mon[28850]: monmap e1: 1 mons at {smithi022=[v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0]} removed_ranks: {} 2024-02-16T19:19:12.654 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:12 smithi022 ceph-mon[28850]: fsmap 2024-02-16T19:19:12.654 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:12 smithi022 ceph-mon[28850]: osdmap e1: 0 total, 0 up, 0 in 2024-02-16T19:19:12.654 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:12 smithi022 ceph-mon[28850]: mgrmap e1: no daemons active 2024-02-16T19:19:12.654 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:12 smithi022 ceph-mon[28850]: from='client.? 172.21.15.22:0/4235149881' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-02-16T19:19:12.656 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout cluster: 2024-02-16T19:19:12.663 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout id: 2a1346e6-cd00-11ee-95ba-87774f69a715 2024-02-16T19:19:12.663 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-02-16T19:19:12.663 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-02-16T19:19:12.663 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout services: 2024-02-16T19:19:12.663 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum smithi022 (age 0.548782s) 2024-02-16T19:19:12.663 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-02-16T19:19:12.663 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-02-16T19:19:12.663 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-02-16T19:19:12.663 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout data: 2024-02-16T19:19:12.663 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-02-16T19:19:12.664 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-02-16T19:19:12.664 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-02-16T19:19:12.664 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout pgs: 2024-02-16T19:19:12.664 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-02-16T19:19:12.664 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:11.994+0000 7f031d3fd700 1 Processor -- start 2024-02-16T19:19:12.664 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:11.994+0000 7f031d3fd700 1 -- start start 2024-02-16T19:19:12.664 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:11.994+0000 7f031d3fd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0318107020 0x7f0318107430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:19:12.664 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:11.994+0000 7f031d3fd700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0318107970 con 0x7f0318107020 2024-02-16T19:19:12.664 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:11.994+0000 7f0316ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0318107020 0x7f0318107430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:19:12.664 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:11.994+0000 7f0316ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0318107020 0x7f0318107430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37972/0 (socket says 172.21.15.22:37972) 2024-02-16T19:19:12.664 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:11.994+0000 7f0316ffd700 1 -- 172.21.15.22:0/356238087 learned_addr learned my addr 172.21.15.22:0/356238087 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:19:12.664 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:11.995+0000 7f0316ffd700 1 -- 172.21.15.22:0/356238087 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0318107ab0 con 0x7f0318107020 2024-02-16T19:19:12.664 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:11.995+0000 7f0316ffd700 1 --2- 172.21.15.22:0/356238087 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0318107020 0x7f0318107430 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f03080098a0 tx=0x7f0308009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2858199a3e91e132 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:19:12.665 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:11.995+0000 7f0315ffb700 1 -- 172.21.15.22:0/356238087 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0308004180 con 0x7f0318107020 2024-02-16T19:19:12.665 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:11.995+0000 7f0315ffb700 1 -- 172.21.15.22:0/356238087 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03080042e0 con 0x7f0318107020 2024-02-16T19:19:12.665 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:11.995+0000 7f0315ffb700 1 -- 172.21.15.22:0/356238087 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f03080036a0 con 0x7f0318107020 2024-02-16T19:19:12.665 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:11.995+0000 7f031d3fd700 1 -- 172.21.15.22:0/356238087 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0318107020 msgr2=0x7f0318107430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:19:12.665 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:11.995+0000 7f031d3fd700 1 --2- 172.21.15.22:0/356238087 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0318107020 0x7f0318107430 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f03080098a0 tx=0x7f0308009bb0 comp rx=0 tx=0).stop 2024-02-16T19:19:12.665 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:11.995+0000 7f031d3fd700 1 -- 172.21.15.22:0/356238087 shutdown_connections 2024-02-16T19:19:12.665 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:11.995+0000 7f031d3fd700 1 --2- 172.21.15.22:0/356238087 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0318107020 0x7f0318107430 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:12.665 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:11.995+0000 7f031d3fd700 1 -- 172.21.15.22:0/356238087 >> 172.21.15.22:0/356238087 conn(0x7f03180ff670 msgr2=0x7f0318101ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:19:12.665 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:11.996+0000 7f031d3fd700 1 -- 172.21.15.22:0/356238087 shutdown_connections 2024-02-16T19:19:12.665 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:11.996+0000 7f031d3fd700 1 -- 172.21.15.22:0/356238087 wait complete. 2024-02-16T19:19:12.665 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:11.996+0000 7f031d3fd700 1 Processor -- start 2024-02-16T19:19:12.665 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:11.996+0000 7f031d3fd700 1 -- start start 2024-02-16T19:19:12.665 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:11.996+0000 7f031d3fd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0318107020 0x7f0318106720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:19:12.666 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:11.996+0000 7f031d3fd700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0318106c60 con 0x7f0318107020 2024-02-16T19:19:12.666 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:11.997+0000 7f0316ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0318107020 0x7f0318106720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:19:12.666 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:11.997+0000 7f0316ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0318107020 0x7f0318106720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37978/0 (socket says 172.21.15.22:37978) 2024-02-16T19:19:12.666 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:11.997+0000 7f0316ffd700 1 -- 172.21.15.22:0/4235149881 learned_addr learned my addr 172.21.15.22:0/4235149881 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:19:12.666 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:11.997+0000 7f0316ffd700 1 -- 172.21.15.22:0/4235149881 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0308009550 con 0x7f0318107020 2024-02-16T19:19:12.666 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:11.997+0000 7f0316ffd700 1 --2- 172.21.15.22:0/4235149881 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0318107020 0x7f0318106720 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f0318104b10 tx=0x7f0308004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:19:12.666 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:11.997+0000 7f02fffff700 1 -- 172.21.15.22:0/4235149881 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0308003ce0 con 0x7f0318107020 2024-02-16T19:19:12.666 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:11.997+0000 7f02fffff700 1 -- 172.21.15.22:0/4235149881 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0308020070 con 0x7f0318107020 2024-02-16T19:19:12.666 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:11.997+0000 7f031d3fd700 1 -- 172.21.15.22:0/4235149881 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0318104d70 con 0x7f0318107020 2024-02-16T19:19:12.666 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:11.997+0000 7f02fffff700 1 -- 172.21.15.22:0/4235149881 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0308011630 con 0x7f0318107020 2024-02-16T19:19:12.666 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:11.997+0000 7f031d3fd700 1 -- 172.21.15.22:0/4235149881 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0318105210 con 0x7f0318107020 2024-02-16T19:19:12.666 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:11.997+0000 7f02fffff700 1 -- 172.21.15.22:0/4235149881 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f0308022030 con 0x7f0318107020 2024-02-16T19:19:12.666 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:11.997+0000 7f02fffff700 1 -- 172.21.15.22:0/4235149881 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f0308025d00 con 0x7f0318107020 2024-02-16T19:19:12.666 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:11.998+0000 7f031d3fd700 1 -- 172.21.15.22:0/4235149881 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0318062380 con 0x7f0318107020 2024-02-16T19:19:12.667 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:12.000+0000 7f02fffff700 1 -- 172.21.15.22:0/4235149881 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f0308025480 con 0x7f0318107020 2024-02-16T19:19:12.667 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:12.075+0000 7f031d3fd700 1 -- 172.21.15.22:0/4235149881 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f0318192230 con 0x7f0318107020 2024-02-16T19:19:12.667 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:12.075+0000 7f02fffff700 1 -- 172.21.15.22:0/4235149881 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+325 (secure 0 0 0) 0x7f0308011c20 con 0x7f0318107020 2024-02-16T19:19:12.667 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:12.076+0000 7f031d3fd700 1 -- 172.21.15.22:0/4235149881 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0318107020 msgr2=0x7f0318106720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:19:12.667 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:12.076+0000 7f031d3fd700 1 --2- 172.21.15.22:0/4235149881 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0318107020 0x7f0318106720 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f0318104b10 tx=0x7f0308004750 comp rx=0 tx=0).stop 2024-02-16T19:19:12.667 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:12.076+0000 7f031d3fd700 1 -- 172.21.15.22:0/4235149881 shutdown_connections 2024-02-16T19:19:12.667 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:12.076+0000 7f031d3fd700 1 --2- 172.21.15.22:0/4235149881 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0318107020 0x7f0318106720 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:12.667 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:12.076+0000 7f031d3fd700 1 -- 172.21.15.22:0/4235149881 >> 172.21.15.22:0/4235149881 conn(0x7f03180ff670 msgr2=0x7f031818d580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:19:12.667 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:12.076+0000 7f031d3fd700 1 -- 172.21.15.22:0/4235149881 shutdown_connections 2024-02-16T19:19:12.667 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:12.076+0000 7f031d3fd700 1 -- 172.21.15.22:0/4235149881 wait complete. 2024-02-16T19:19:12.667 INFO:teuthology.orchestra.run.smithi022.stdout:mon is available 2024-02-16T19:19:12.667 INFO:teuthology.orchestra.run.smithi022.stdout:Assimilating anything we can from ceph.conf... 2024-02-16T19:19:13.785 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:13 smithi022 ceph-mon[28850]: from='client.? 172.21.15.22:0/4286665861' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-02-16T19:19:13.785 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:13 smithi022 ceph-mon[28850]: from='client.? 172.21.15.22:0/4286665861' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-02-16T19:19:13.794 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-02-16T19:19:13.794 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout [global] 2024-02-16T19:19:13.794 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout fsid = 2a1346e6-cd00-11ee-95ba-87774f69a715 2024-02-16T19:19:13.794 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.22:3300,v1:172.21.15.22:6789] 2024-02-16T19:19:13.794 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-02-16T19:19:13.794 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-02-16T19:19:13.794 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-02-16T19:19:13.794 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-02-16T19:19:13.794 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-02-16T19:19:13.794 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout [mgr] 2024-02-16T19:19:13.794 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-02-16T19:19:13.794 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-02-16T19:19:13.794 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout [osd] 2024-02-16T19:19:13.795 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-02-16T19:19:13.795 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-02-16T19:19:13.795 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-02-16T19:19:13.795 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-02-16T19:19:13.795 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-02-16T19:19:13.795 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:13.090+0000 7fa79aedd700 1 Processor -- start 2024-02-16T19:19:13.795 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:13.090+0000 7fa79aedd700 1 -- start start 2024-02-16T19:19:13.795 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:13.090+0000 7fa79aedd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa794106510 0x7fa7941088f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:19:13.795 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:13.090+0000 7fa79aedd700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa794079d10 con 0x7fa794106510 2024-02-16T19:19:13.795 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:13.090+0000 7fa798c79700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa794106510 0x7fa7941088f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:19:13.795 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:13.090+0000 7fa798c79700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa794106510 0x7fa7941088f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37986/0 (socket says 172.21.15.22:37986) 2024-02-16T19:19:13.795 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:13.090+0000 7fa798c79700 1 -- 172.21.15.22:0/440200667 learned_addr learned my addr 172.21.15.22:0/440200667 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:19:13.795 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:13.091+0000 7fa798c79700 1 -- 172.21.15.22:0/440200667 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa794079e50 con 0x7fa794106510 2024-02-16T19:19:13.795 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:13.091+0000 7fa798c79700 1 --2- 172.21.15.22:0/440200667 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa794106510 0x7fa7941088f0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fa7880098a0 tx=0x7fa788009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=216715c0e7ee70b4 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:19:13.796 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:13.091+0000 7fa7937fe700 1 -- 172.21.15.22:0/440200667 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa788004180 con 0x7fa794106510 2024-02-16T19:19:13.796 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:13.091+0000 7fa7937fe700 1 -- 172.21.15.22:0/440200667 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7880042e0 con 0x7fa794106510 2024-02-16T19:19:13.796 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:13.091+0000 7fa7937fe700 1 -- 172.21.15.22:0/440200667 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa7880036a0 con 0x7fa794106510 2024-02-16T19:19:13.796 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:13.091+0000 7fa79aedd700 1 -- 172.21.15.22:0/440200667 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa794106510 msgr2=0x7fa7941088f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:19:13.796 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:13.091+0000 7fa79aedd700 1 --2- 172.21.15.22:0/440200667 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa794106510 0x7fa7941088f0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fa7880098a0 tx=0x7fa788009bb0 comp rx=0 tx=0).stop 2024-02-16T19:19:13.796 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:13.091+0000 7fa79aedd700 1 -- 172.21.15.22:0/440200667 shutdown_connections 2024-02-16T19:19:13.796 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:13.091+0000 7fa79aedd700 1 --2- 172.21.15.22:0/440200667 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa794106510 0x7fa7941088f0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:13.796 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:13.091+0000 7fa79aedd700 1 -- 172.21.15.22:0/440200667 >> 172.21.15.22:0/440200667 conn(0x7fa7940ffed0 msgr2=0x7fa794102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:19:13.796 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:13.092+0000 7fa79aedd700 1 -- 172.21.15.22:0/440200667 shutdown_connections 2024-02-16T19:19:13.796 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:13.092+0000 7fa79aedd700 1 -- 172.21.15.22:0/440200667 wait complete. 2024-02-16T19:19:13.796 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:13.092+0000 7fa79aedd700 1 Processor -- start 2024-02-16T19:19:13.796 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:13.092+0000 7fa79aedd700 1 -- start start 2024-02-16T19:19:13.796 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:13.092+0000 7fa79aedd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa794106510 0x7fa794198e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:19:13.797 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:13.092+0000 7fa79aedd700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa794199380 con 0x7fa794106510 2024-02-16T19:19:13.797 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:13.092+0000 7fa798c79700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa794106510 0x7fa794198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:19:13.797 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:13.092+0000 7fa798c79700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa794106510 0x7fa794198e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38000/0 (socket says 172.21.15.22:38000) 2024-02-16T19:19:13.797 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:13.092+0000 7fa798c79700 1 -- 172.21.15.22:0/4286665861 learned_addr learned my addr 172.21.15.22:0/4286665861 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:19:13.797 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:13.093+0000 7fa798c79700 1 -- 172.21.15.22:0/4286665861 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa788009550 con 0x7fa794106510 2024-02-16T19:19:13.797 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:13.093+0000 7fa798c79700 1 --2- 172.21.15.22:0/4286665861 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa794106510 0x7fa794198e40 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fa788004000 tx=0x7fa788004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:19:13.797 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:13.093+0000 7fa791ffb700 1 -- 172.21.15.22:0/4286665861 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa788003ce0 con 0x7fa794106510 2024-02-16T19:19:13.797 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:13.093+0000 7fa791ffb700 1 -- 172.21.15.22:0/4286665861 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa788020070 con 0x7fa794106510 2024-02-16T19:19:13.797 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:13.093+0000 7fa79aedd700 1 -- 172.21.15.22:0/4286665861 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7941995e0 con 0x7fa794106510 2024-02-16T19:19:13.797 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:13.093+0000 7fa791ffb700 1 -- 172.21.15.22:0/4286665861 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa788011630 con 0x7fa794106510 2024-02-16T19:19:13.797 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:13.093+0000 7fa79aedd700 1 -- 172.21.15.22:0/4286665861 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa79419c3e0 con 0x7fa794106510 2024-02-16T19:19:13.797 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:13.093+0000 7fa791ffb700 1 -- 172.21.15.22:0/4286665861 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fa788022030 con 0x7fa794106510 2024-02-16T19:19:13.797 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:13.093+0000 7fa791ffb700 1 -- 172.21.15.22:0/4286665861 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fa788025cf0 con 0x7fa794106510 2024-02-16T19:19:13.798 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:13.094+0000 7fa79aedd700 1 -- 172.21.15.22:0/4286665861 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa794192c40 con 0x7fa794106510 2024-02-16T19:19:13.798 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:13.096+0000 7fa791ffb700 1 -- 172.21.15.22:0/4286665861 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7fa788025480 con 0x7fa794106510 2024-02-16T19:19:13.798 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:13.166+0000 7fa79aedd700 1 -- 172.21.15.22:0/4286665861 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fa79402cff0 con 0x7fa794106510 2024-02-16T19:19:13.798 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:13.170+0000 7fa791ffb700 1 -- 172.21.15.22:0/4286665861 <== mon.0 v2:172.21.15.22:3300/0 7 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa788011bd0 con 0x7fa794106510 2024-02-16T19:19:13.798 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:13.170+0000 7fa791ffb700 1 -- 172.21.15.22:0/4286665861 <== mon.0 v2:172.21.15.22:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7fa78801c500 con 0x7fa794106510 2024-02-16T19:19:13.798 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:13.171+0000 7fa79aedd700 1 -- 172.21.15.22:0/4286665861 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa794106510 msgr2=0x7fa794198e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:19:13.798 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:13.171+0000 7fa79aedd700 1 --2- 172.21.15.22:0/4286665861 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa794106510 0x7fa794198e40 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fa788004000 tx=0x7fa788004750 comp rx=0 tx=0).stop 2024-02-16T19:19:13.798 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:13.171+0000 7fa79aedd700 1 -- 172.21.15.22:0/4286665861 shutdown_connections 2024-02-16T19:19:13.798 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:13.171+0000 7fa79aedd700 1 --2- 172.21.15.22:0/4286665861 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa794106510 0x7fa794198e40 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:13.798 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:13.171+0000 7fa79aedd700 1 -- 172.21.15.22:0/4286665861 >> 172.21.15.22:0/4286665861 conn(0x7fa7940ffed0 msgr2=0x7fa794101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:19:13.798 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:13.171+0000 7fa79aedd700 1 -- 172.21.15.22:0/4286665861 shutdown_connections 2024-02-16T19:19:13.798 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:13.171+0000 7fa79aedd700 1 -- 172.21.15.22:0/4286665861 wait complete. 2024-02-16T19:19:13.798 INFO:teuthology.orchestra.run.smithi022.stdout:Generating new minimal ceph.conf... 2024-02-16T19:19:14.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:14 smithi022 ceph-mon[28850]: from='client.? 172.21.15.22:0/3662349623' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:19:15.165 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:14.514+0000 7f1559454700 1 Processor -- start 2024-02-16T19:19:15.165 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:14.514+0000 7f1559454700 1 -- start start 2024-02-16T19:19:15.165 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:14.514+0000 7f1559454700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f155407bdb0 0x7f155407a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:19:15.165 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:14.514+0000 7f1559454700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f155407a7f0 con 0x7f155407bdb0 2024-02-16T19:19:15.165 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:14.515+0000 7f1552ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f155407bdb0 0x7f155407a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:19:15.165 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:14.515+0000 7f1552ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f155407bdb0 0x7f155407a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43292/0 (socket says 172.21.15.22:43292) 2024-02-16T19:19:15.166 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:14.515+0000 7f1552ffd700 1 -- 172.21.15.22:0/2998606126 learned_addr learned my addr 172.21.15.22:0/2998606126 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:19:15.166 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:14.515+0000 7f1552ffd700 1 -- 172.21.15.22:0/2998606126 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f155407a930 con 0x7f155407bdb0 2024-02-16T19:19:15.166 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:14.515+0000 7f1552ffd700 1 --2- 172.21.15.22:0/2998606126 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f155407bdb0 0x7f155407a2b0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f154c009ed0 tx=0x7f154c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fb30bf248deedc4 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:19:15.166 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:14.515+0000 7f1551ffb700 1 -- 172.21.15.22:0/2998606126 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f154c00e070 con 0x7f155407bdb0 2024-02-16T19:19:15.166 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:14.515+0000 7f1551ffb700 1 -- 172.21.15.22:0/2998606126 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f154c003e90 con 0x7f155407bdb0 2024-02-16T19:19:15.166 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:14.515+0000 7f1551ffb700 1 -- 172.21.15.22:0/2998606126 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f154c004480 con 0x7f155407bdb0 2024-02-16T19:19:15.166 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:14.516+0000 7f1559454700 1 -- 172.21.15.22:0/2998606126 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f155407bdb0 msgr2=0x7f155407a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:19:15.166 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:14.516+0000 7f1559454700 1 --2- 172.21.15.22:0/2998606126 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f155407bdb0 0x7f155407a2b0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f154c009ed0 tx=0x7f154c00b330 comp rx=0 tx=0).stop 2024-02-16T19:19:15.166 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:14.516+0000 7f1559454700 1 -- 172.21.15.22:0/2998606126 shutdown_connections 2024-02-16T19:19:15.166 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:14.516+0000 7f1559454700 1 --2- 172.21.15.22:0/2998606126 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f155407bdb0 0x7f155407a2b0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:15.166 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:14.516+0000 7f1559454700 1 -- 172.21.15.22:0/2998606126 >> 172.21.15.22:0/2998606126 conn(0x7f1554101020 msgr2=0x7f1554103450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:19:15.166 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:14.516+0000 7f1559454700 1 -- 172.21.15.22:0/2998606126 shutdown_connections 2024-02-16T19:19:15.166 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:14.516+0000 7f1559454700 1 -- 172.21.15.22:0/2998606126 wait complete. 2024-02-16T19:19:15.167 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:14.517+0000 7f1559454700 1 Processor -- start 2024-02-16T19:19:15.167 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:14.517+0000 7f1559454700 1 -- start start 2024-02-16T19:19:15.167 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:14.517+0000 7f1559454700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f155407bdb0 0x7f155419b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:19:15.167 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:14.517+0000 7f1559454700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f155419bdc0 con 0x7f155407bdb0 2024-02-16T19:19:15.167 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:14.517+0000 7f1552ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f155407bdb0 0x7f155419b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:19:15.167 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:14.517+0000 7f1552ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f155407bdb0 0x7f155419b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43306/0 (socket says 172.21.15.22:43306) 2024-02-16T19:19:15.167 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:14.517+0000 7f1552ffd700 1 -- 172.21.15.22:0/3662349623 learned_addr learned my addr 172.21.15.22:0/3662349623 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:19:15.167 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:14.517+0000 7f1552ffd700 1 -- 172.21.15.22:0/3662349623 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f154c009b30 con 0x7f155407bdb0 2024-02-16T19:19:15.167 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:14.518+0000 7f1552ffd700 1 --2- 172.21.15.22:0/3662349623 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f155407bdb0 0x7f155419b880 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f154c00b7b0 tx=0x7f154c004a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:19:15.167 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:14.518+0000 7f154bfff700 1 -- 172.21.15.22:0/3662349623 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f154c00e040 con 0x7f155407bdb0 2024-02-16T19:19:15.167 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:14.518+0000 7f154bfff700 1 -- 172.21.15.22:0/3662349623 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f154c003cf0 con 0x7f155407bdb0 2024-02-16T19:19:15.167 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:14.518+0000 7f154bfff700 1 -- 172.21.15.22:0/3662349623 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f154c0127e0 con 0x7f155407bdb0 2024-02-16T19:19:15.167 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:14.518+0000 7f1559454700 1 -- 172.21.15.22:0/3662349623 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f155419bfc0 con 0x7f155407bdb0 2024-02-16T19:19:15.167 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:14.518+0000 7f1559454700 1 -- 172.21.15.22:0/3662349623 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f155419c460 con 0x7f155407bdb0 2024-02-16T19:19:15.167 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:14.518+0000 7f154bfff700 1 -- 172.21.15.22:0/3662349623 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f154c019040 con 0x7f155407bdb0 2024-02-16T19:19:15.167 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:14.518+0000 7f154bfff700 1 -- 172.21.15.22:0/3662349623 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f154c024d80 con 0x7f155407bdb0 2024-02-16T19:19:15.167 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:14.519+0000 7f1559454700 1 -- 172.21.15.22:0/3662349623 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1554062380 con 0x7f155407bdb0 2024-02-16T19:19:15.168 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:14.522+0000 7f154bfff700 1 -- 172.21.15.22:0/3662349623 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f154c01f020 con 0x7f155407bdb0 2024-02-16T19:19:15.168 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:14.593+0000 7f1559454700 1 -- 172.21.15.22:0/3662349623 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f155419ef10 con 0x7f155407bdb0 2024-02-16T19:19:15.168 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:14.593+0000 7f154bfff700 1 -- 172.21.15.22:0/3662349623 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f154c017070 con 0x7f155407bdb0 2024-02-16T19:19:15.168 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:14.594+0000 7f1559454700 1 -- 172.21.15.22:0/3662349623 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f155407bdb0 msgr2=0x7f155419b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:19:15.168 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:14.594+0000 7f1559454700 1 --2- 172.21.15.22:0/3662349623 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f155407bdb0 0x7f155419b880 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f154c00b7b0 tx=0x7f154c004a20 comp rx=0 tx=0).stop 2024-02-16T19:19:15.168 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:14.594+0000 7f1559454700 1 -- 172.21.15.22:0/3662349623 shutdown_connections 2024-02-16T19:19:15.168 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:14.594+0000 7f1559454700 1 --2- 172.21.15.22:0/3662349623 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f155407bdb0 0x7f155419b880 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:15.168 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:14.594+0000 7f1559454700 1 -- 172.21.15.22:0/3662349623 >> 172.21.15.22:0/3662349623 conn(0x7f1554101020 msgr2=0x7f1554101dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:19:15.168 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:14.594+0000 7f1559454700 1 -- 172.21.15.22:0/3662349623 shutdown_connections 2024-02-16T19:19:15.168 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:14.594+0000 7f1559454700 1 -- 172.21.15.22:0/3662349623 wait complete. 2024-02-16T19:19:15.168 INFO:teuthology.orchestra.run.smithi022.stdout:Restarting the monitor... 2024-02-16T19:19:15.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:15 smithi022 systemd[1]: Stopping Ceph mon.smithi022 for 2a1346e6-cd00-11ee-95ba-87774f69a715... 2024-02-16T19:19:15.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:15 smithi022 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-mon-smithi022[28846]: 2024-02-16T19:19:15.427+0000 7f903bd66700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi022 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-16T19:19:15.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:15 smithi022 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-mon-smithi022[28846]: 2024-02-16T19:19:15.427+0000 7f903bd66700 -1 mon.smithi022@0(leader) e1 *** Got Signal Terminated *** 2024-02-16T19:19:15.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:15 smithi022 podman[29093]: 2024-02-16 19:19:15.519439387 +0000 UTC m=+0.195894950 container died d474673341a776f8853c46b9621161df41bef3cda1ffc81513c367d94f496b7e (image=quay.io/ceph/ceph:v18.2.0, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-mon-smithi022, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.29.1, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=https://github.com/ceph/ceph-container.git, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.build-date=20231212, RELEASE=HEAD, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.vendor=CentOS) 2024-02-16T19:19:16.147 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:15 smithi022 podman[29093]: 2024-02-16 19:19:15.877434317 +0000 UTC m=+0.553889879 container cleanup d474673341a776f8853c46b9621161df41bef3cda1ffc81513c367d94f496b7e (image=quay.io/ceph/ceph:v18.2.0, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-mon-smithi022, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=-18.2.0, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.license=GPLv2, org.label-schema.build-date=20231212, GIT_REPO=https://github.com/ceph/ceph-container.git, RELEASE=HEAD, GIT_CLEAN=True, io.buildah.version=1.29.1, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-02-16T19:19:16.147 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:15 smithi022 bash[29093]: ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-mon-smithi022 2024-02-16T19:19:16.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 podman[29107]: 2024-02-16 19:19:16.147001998 +0000 UTC m=+0.628599725 container remove d474673341a776f8853c46b9621161df41bef3cda1ffc81513c367d94f496b7e (image=quay.io/ceph/ceph:v18.2.0, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-mon-smithi022, RELEASE=HEAD, GIT_CLEAN=True, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.build-date=20231212, org.label-schema.schema-version=1.0, io.buildah.version=1.29.1, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=https://github.com/ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS) 2024-02-16T19:19:16.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 systemd[1]: ceph-2a1346e6-cd00-11ee-95ba-87774f69a715@mon.smithi022.service: Deactivated successfully. 2024-02-16T19:19:16.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 systemd[1]: Stopped Ceph mon.smithi022 for 2a1346e6-cd00-11ee-95ba-87774f69a715. 2024-02-16T19:19:16.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 systemd[1]: Starting Ceph mon.smithi022 for 2a1346e6-cd00-11ee-95ba-87774f69a715... 2024-02-16T19:19:16.782 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 podman[29202]: 2024-02-16 19:19:16.422398368 +0000 UTC m=+0.017919592 image pull dc2bc1663786ad5862ef6efcff18243ab7dfeda0bcb911ddf757a0d20feca946 quay.io/ceph/ceph:v18.2.0 2024-02-16T19:19:16.782 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 podman[29202]: 2024-02-16 19:19:16.529333514 +0000 UTC m=+0.124854778 container create b4ffc3032e12371bfc1764880a8e6b8944c21341bd978ae792b2fbbd42152ee8 (image=quay.io/ceph/ceph:v18.2.0, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-mon-smithi022, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=-18.2.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=https://github.com/ceph/ceph-container.git, io.buildah.version=1.29.1, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.vendor=CentOS, org.label-schema.build-date=20231212, RELEASE=HEAD) 2024-02-16T19:19:16.782 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 podman[29202]: 2024-02-16 19:19:16.714753429 +0000 UTC m=+0.310274651 container init b4ffc3032e12371bfc1764880a8e6b8944c21341bd978ae792b2fbbd42152ee8 (image=quay.io/ceph/ceph:v18.2.0, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-mon-smithi022, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20231212, GIT_CLEAN=True, io.buildah.version=1.29.1, org.label-schema.license=GPLv2, GIT_REPO=https://github.com/ceph/ceph-container.git, RELEASE=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=-18.2.0, GIT_BRANCH=HEAD) 2024-02-16T19:19:16.782 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 podman[29202]: 2024-02-16 19:19:16.717617369 +0000 UTC m=+0.313138590 container start b4ffc3032e12371bfc1764880a8e6b8944c21341bd978ae792b2fbbd42152ee8 (image=quay.io/ceph/ceph:v18.2.0, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-mon-smithi022, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=HEAD, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=-18.2.0, io.buildah.version=1.29.1, org.label-schema.build-date=20231212, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-16T19:19:16.782 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: set uid:gid to 167:167 (ceph:ceph) 2024-02-16T19:19:16.782 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable), process ceph-mon, pid 2 2024-02-16T19:19:16.782 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: pidfile_write: ignore empty --pid-file 2024-02-16T19:19:16.783 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: load: jerasure load: lrc 2024-02-16T19:19:16.783 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: RocksDB version: 7.9.2 2024-02-16T19:19:16.783 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Git sha 0 2024-02-16T19:19:16.783 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Compile date 2023-08-03 19:21:13 2024-02-16T19:19:16.783 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: DB SUMMARY 2024-02-16T19:19:16.783 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: DB Session ID: 06IMZOHPWLS1EU33KG2E 2024-02-16T19:19:16.783 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: CURRENT file: CURRENT 2024-02-16T19:19:16.783 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: IDENTITY file: IDENTITY 2024-02-16T19:19:16.783 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-02-16T19:19:16.783 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi022/store.db dir, Total Num: 1, files: 000008.sst 2024-02-16T19:19:16.783 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi022/store.db: 000009.log size: 97396 ; 2024-02-16T19:19:16.783 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.error_if_exists: 0 2024-02-16T19:19:16.784 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.create_if_missing: 0 2024-02-16T19:19:16.784 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.paranoid_checks: 1 2024-02-16T19:19:16.784 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.flush_verify_memtable_count: 1 2024-02-16T19:19:16.784 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-16T19:19:16.784 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-02-16T19:19:16.784 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.env: 0x55f0d8211720 2024-02-16T19:19:16.784 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.fs: PosixFileSystem 2024-02-16T19:19:16.784 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.info_log: 0x55f0db65f2c0 2024-02-16T19:19:16.784 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.max_file_opening_threads: 16 2024-02-16T19:19:16.784 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.statistics: (nil) 2024-02-16T19:19:16.784 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.use_fsync: 0 2024-02-16T19:19:16.784 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.max_log_file_size: 0 2024-02-16T19:19:16.784 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-16T19:19:16.785 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-16T19:19:16.785 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.keep_log_file_num: 1000 2024-02-16T19:19:16.785 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.recycle_log_file_num: 0 2024-02-16T19:19:16.785 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.allow_fallocate: 1 2024-02-16T19:19:16.785 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.allow_mmap_reads: 0 2024-02-16T19:19:16.785 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.allow_mmap_writes: 0 2024-02-16T19:19:16.785 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.use_direct_reads: 0 2024-02-16T19:19:16.785 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-16T19:19:16.785 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.create_missing_column_families: 0 2024-02-16T19:19:16.785 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.db_log_dir: 2024-02-16T19:19:16.785 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.wal_dir: 2024-02-16T19:19:16.786 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-16T19:19:16.786 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-16T19:19:16.786 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-16T19:19:16.786 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-16T19:19:16.786 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-16T19:19:16.786 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-16T19:19:16.786 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.advise_random_on_open: 1 2024-02-16T19:19:16.786 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.db_write_buffer_size: 0 2024-02-16T19:19:16.786 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.write_buffer_manager: 0x55f0da8d03c0 2024-02-16T19:19:16.786 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-16T19:19:16.786 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-16T19:19:16.786 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-16T19:19:16.786 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.rate_limiter: (nil) 2024-02-16T19:19:16.787 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-16T19:19:16.787 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.wal_recovery_mode: 2 2024-02-16T19:19:16.787 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.enable_thread_tracking: 0 2024-02-16T19:19:16.787 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.enable_pipelined_write: 0 2024-02-16T19:19:16.788 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.unordered_write: 0 2024-02-16T19:19:16.788 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-16T19:19:16.788 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-16T19:19:16.788 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-16T19:19:16.788 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-16T19:19:16.789 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.row_cache: None 2024-02-16T19:19:16.789 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.wal_filter: None 2024-02-16T19:19:16.789 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-16T19:19:16.789 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.allow_ingest_behind: 0 2024-02-16T19:19:16.789 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.two_write_queues: 0 2024-02-16T19:19:16.789 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.manual_wal_flush: 0 2024-02-16T19:19:16.789 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.wal_compression: 0 2024-02-16T19:19:16.789 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.atomic_flush: 0 2024-02-16T19:19:16.789 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-16T19:19:16.789 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-16T19:19:16.789 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-16T19:19:16.789 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.log_readahead_size: 0 2024-02-16T19:19:16.789 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-16T19:19:16.789 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.best_efforts_recovery: 0 2024-02-16T19:19:16.790 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-16T19:19:16.790 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-16T19:19:16.790 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.allow_data_in_errors: 0 2024-02-16T19:19:16.790 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.db_host_id: __hostname__ 2024-02-16T19:19:16.790 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.enforce_single_del_contracts: true 2024-02-16T19:19:16.790 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.max_background_jobs: 2 2024-02-16T19:19:16.790 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.max_background_compactions: -1 2024-02-16T19:19:16.790 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.max_subcompactions: 1 2024-02-16T19:19:16.790 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-16T19:19:16.790 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-16T19:19:16.790 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-16T19:19:16.790 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.max_total_wal_size: 0 2024-02-16T19:19:16.790 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-16T19:19:16.790 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-16T19:19:16.791 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-16T19:19:16.791 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-16T19:19:16.791 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.max_open_files: -1 2024-02-16T19:19:16.791 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.bytes_per_sync: 0 2024-02-16T19:19:16.791 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-16T19:19:16.791 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-16T19:19:16.791 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.compaction_readahead_size: 0 2024-02-16T19:19:16.791 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.max_background_flushes: -1 2024-02-16T19:19:16.791 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Compression algorithms supported: 2024-02-16T19:19:16.791 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-16T19:19:16.791 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: kZSTD supported: 0 2024-02-16T19:19:16.791 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: kXpressCompression supported: 0 2024-02-16T19:19:16.792 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: kLZ4HCCompression supported: 1 2024-02-16T19:19:16.792 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: kZlibCompression supported: 1 2024-02-16T19:19:16.792 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: kSnappyCompression supported: 1 2024-02-16T19:19:16.792 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: kLZ4Compression supported: 1 2024-02-16T19:19:16.792 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: kBZip2Compression supported: 0 2024-02-16T19:19:16.792 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-16T19:19:16.792 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: DMutex implementation: pthread_mutex_t 2024-02-16T19:19:16.792 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi022/store.db/MANIFEST-000010 2024-02-16T19:19:16.792 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-02-16T19:19:16.792 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-16T19:19:16.792 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.merge_operator: 2024-02-16T19:19:16.792 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.compaction_filter: None 2024-02-16T19:19:16.792 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.compaction_filter_factory: None 2024-02-16T19:19:16.793 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.sst_partitioner_factory: None 2024-02-16T19:19:16.793 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-16T19:19:16.793 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-16T19:19:16.793 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55f0db65f3c0) 2024-02-16T19:19:16.793 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: cache_index_and_filter_blocks: 1 2024-02-16T19:19:16.793 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-16T19:19:16.793 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-16T19:19:16.793 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: pin_top_level_index_and_filter: 1 2024-02-16T19:19:16.793 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: index_type: 0 2024-02-16T19:19:16.793 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: data_block_index_type: 0 2024-02-16T19:19:16.793 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: index_shortening: 1 2024-02-16T19:19:16.793 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-16T19:19:16.793 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: checksum: 4 2024-02-16T19:19:16.793 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: no_block_cache: 0 2024-02-16T19:19:16.793 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: block_cache: 0x55f0da971350 2024-02-16T19:19:16.793 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: block_cache_name: BinnedLRUCache 2024-02-16T19:19:16.794 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: block_cache_options: 2024-02-16T19:19:16.794 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: capacity : 536870912 2024-02-16T19:19:16.794 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: num_shard_bits : 4 2024-02-16T19:19:16.794 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: strict_capacity_limit : 0 2024-02-16T19:19:16.794 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: high_pri_pool_ratio: 0.000 2024-02-16T19:19:16.794 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: block_cache_compressed: (nil) 2024-02-16T19:19:16.794 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: persistent_cache: (nil) 2024-02-16T19:19:16.794 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: block_size: 4096 2024-02-16T19:19:16.794 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: block_size_deviation: 10 2024-02-16T19:19:16.794 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: block_restart_interval: 16 2024-02-16T19:19:16.794 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: index_block_restart_interval: 1 2024-02-16T19:19:16.794 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: metadata_block_size: 4096 2024-02-16T19:19:16.794 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: partition_filters: 0 2024-02-16T19:19:16.794 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: use_delta_encoding: 1 2024-02-16T19:19:16.794 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: filter_policy: bloomfilter 2024-02-16T19:19:16.795 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: whole_key_filtering: 1 2024-02-16T19:19:16.795 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: verify_compression: 0 2024-02-16T19:19:16.795 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: read_amp_bytes_per_bit: 0 2024-02-16T19:19:16.795 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: format_version: 5 2024-02-16T19:19:16.795 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: enable_index_compression: 1 2024-02-16T19:19:16.795 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: block_align: 0 2024-02-16T19:19:16.795 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: max_auto_readahead_size: 262144 2024-02-16T19:19:16.795 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: prepopulate_block_cache: 0 2024-02-16T19:19:16.795 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: initial_auto_readahead_size: 8192 2024-02-16T19:19:16.795 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: num_file_reads_for_auto_readahead: 2 2024-02-16T19:19:16.795 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.write_buffer_size: 33554432 2024-02-16T19:19:16.795 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.max_write_buffer_number: 2 2024-02-16T19:19:16.795 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.compression: NoCompression 2024-02-16T19:19:16.795 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.bottommost_compression: Disabled 2024-02-16T19:19:16.795 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.prefix_extractor: nullptr 2024-02-16T19:19:16.796 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-16T19:19:16.796 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.num_levels: 7 2024-02-16T19:19:16.796 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-16T19:19:16.796 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-16T19:19:16.796 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-16T19:19:16.796 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-16T19:19:16.796 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-16T19:19:16.796 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-16T19:19:16.796 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-16T19:19:16.796 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-16T19:19:16.796 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-16T19:19:16.796 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-16T19:19:16.796 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-02-16T19:19:16.796 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-02-16T19:19:16.797 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-16T19:19:16.797 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.compression_opts.level: 32767 2024-02-16T19:19:16.797 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.compression_opts.strategy: 0 2024-02-16T19:19:16.797 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-16T19:19:16.797 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-16T19:19:16.797 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-02-16T19:19:16.797 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-16T19:19:16.797 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.compression_opts.enabled: false 2024-02-16T19:19:16.797 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-02-16T19:19:16.797 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-16T19:19:16.797 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-16T19:19:16.797 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-16T19:19:16.797 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.target_file_size_base: 67108864 2024-02-16T19:19:16.797 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-16T19:19:16.797 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-16T19:19:16.798 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-16T19:19:16.798 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-16T19:19:16.798 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-16T19:19:16.798 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-16T19:19:16.798 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-16T19:19:16.798 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-16T19:19:16.798 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-16T19:19:16.798 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-16T19:19:16.798 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-16T19:19:16.798 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-16T19:19:16.798 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-16T19:19:16.798 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-02-16T19:19:16.798 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.arena_block_size: 1048576 2024-02-16T19:19:16.798 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-16T19:19:16.798 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-16T19:19:16.799 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.disable_auto_compactions: 0 2024-02-16T19:19:16.799 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-16T19:19:16.799 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-16T19:19:16.799 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-16T19:19:16.799 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-16T19:19:16.799 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-16T19:19:16.799 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-16T19:19:16.799 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-16T19:19:16.799 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-16T19:19:16.799 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-16T19:19:16.799 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-16T19:19:16.799 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.table_properties_collectors: 2024-02-16T19:19:16.799 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.inplace_update_support: 0 2024-02-16T19:19:16.799 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-16T19:19:16.799 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-16T19:19:16.800 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-16T19:19:16.800 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-16T19:19:16.800 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.bloom_locality: 0 2024-02-16T19:19:16.800 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.max_successive_merges: 0 2024-02-16T19:19:16.800 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-16T19:19:16.800 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.paranoid_file_checks: 0 2024-02-16T19:19:16.800 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.force_consistency_checks: 1 2024-02-16T19:19:16.800 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.report_bg_io_stats: 0 2024-02-16T19:19:16.800 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.ttl: 2592000 2024-02-16T19:19:16.800 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-16T19:19:16.800 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-02-16T19:19:16.800 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-02-16T19:19:16.800 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.enable_blob_files: false 2024-02-16T19:19:16.800 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.min_blob_size: 0 2024-02-16T19:19:16.801 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.blob_file_size: 268435456 2024-02-16T19:19:16.801 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-16T19:19:16.801 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-16T19:19:16.801 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-16T19:19:16.801 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-02-16T19:19:16.801 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-02-16T19:19:16.801 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.blob_file_starting_level: 0 2024-02-16T19:19:16.801 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-02-16T19:19:16.801 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi022/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-02-16T19:19:16.801 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-02-16T19:19:16.802 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 47a8333a-8246-4f53-a0b9-cb19c23501f3 2024-02-16T19:19:16.802 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708111156745861, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-02-16T19:19:16.802 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-02-16T19:19:16.802 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708111156746825, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 92746, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 310, "table_properties": {"data_size": 90763, "index_size": 265, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13356, "raw_average_key_size": 49, "raw_value_size": 83478, "raw_average_value_size": 310, "num_data_blocks": 11, "num_entries": 269, "num_filter_entries": 269, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1708111156, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "47a8333a-8246-4f53-a0b9-cb19c23501f3", "db_session_id": "06IMZOHPWLS1EU33KG2E", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-02-16T19:19:16.802 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708111156746908, "job": 1, "event": "recovery_finished"} 2024-02-16T19:19:16.802 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-02-16T19:19:16.802 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi022/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-16T19:19:16.802 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55f0daa0e000 2024-02-16T19:19:16.802 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: DB pointer 0x55f0da9fa000 2024-02-16T19:19:16.802 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-02-16T19:19:16.802 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-02-16T19:19:16.802 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: ** DB Stats ** 2024-02-16T19:19:16.802 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-16T19:19:16.802 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-16T19:19:16.803 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-16T19:19:16.803 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-16T19:19:16.803 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-16T19:19:16.803 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-16T19:19:16.803 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-16T19:19:16.803 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: 2024-02-16T19:19:16.803 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: ** Compaction Stats [default] ** 2024-02-16T19:19:16.803 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-16T19:19:16.803 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-02-16T19:19:16.803 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: L0 2/0 92.45 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 140.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-16T19:19:16.803 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: Sum 2/0 92.45 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 140.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-16T19:19:16.803 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 140.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-16T19:19:16.803 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: 2024-02-16T19:19:16.803 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: ** Compaction Stats [default] ** 2024-02-16T19:19:16.803 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-16T19:19:16.804 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-16T19:19:16.804 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 140.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-16T19:19:16.804 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: 2024-02-16T19:19:16.804 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-02-16T19:19:16.804 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: 2024-02-16T19:19:16.804 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-16T19:19:16.804 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-02-16T19:19:16.804 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-16T19:19:16.804 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-16T19:19:16.804 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-16T19:19:16.804 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-16T19:19:16.804 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: Cumulative compaction: 0.00 GB write, 30.12 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-16T19:19:16.804 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: Interval compaction: 0.00 GB write, 30.12 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-16T19:19:16.805 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-16T19:19:16.805 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: Block cache BinnedLRUCache@0x55f0da971350#2 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-02-16T19:19:16.805 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-02-16T19:19:16.805 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: 2024-02-16T19:19:16.805 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-16T19:19:16.805 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: starting mon.smithi022 rank 0 at public addrs [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] at bind addrs [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi022 fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 2024-02-16T19:19:16.805 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: mon.smithi022@-1(???) e1 preinit fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 2024-02-16T19:19:16.805 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: mon.smithi022@-1(???).mds e1 new map 2024-02-16T19:19:16.805 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: mon.smithi022@-1(???).mds e1 print_map 2024-02-16T19:19:16.805 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: e1 2024-02-16T19:19:16.805 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T19:19:16.805 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:19:16.805 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: legacy client fscid: -1 2024-02-16T19:19:16.805 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: 2024-02-16T19:19:16.806 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: No filesystems configured 2024-02-16T19:19:16.806 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: mon.smithi022@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-16T19:19:16.806 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: mon.smithi022@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-16T19:19:16.806 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: mon.smithi022@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-16T19:19:16.806 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: mon.smithi022@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-16T19:19:16.806 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: mon.smithi022@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-02-16T19:19:16.806 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 bash[29202]: b4ffc3032e12371bfc1764880a8e6b8944c21341bd978ae792b2fbbd42152ee8 2024-02-16T19:19:16.806 INFO:teuthology.orchestra.run.smithi022.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-02-16T19:19:17.102 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 systemd[1]: Started Ceph mon.smithi022 for 2a1346e6-cd00-11ee-95ba-87774f69a715. 2024-02-16T19:19:17.102 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: mon.smithi022 is new leader, mons smithi022 in quorum (ranks 0) 2024-02-16T19:19:17.102 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: monmap e1: 1 mons at {smithi022=[v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0]} removed_ranks: {} 2024-02-16T19:19:17.102 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: fsmap 2024-02-16T19:19:17.102 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: osdmap e1: 0 total, 0 up, 0 in 2024-02-16T19:19:17.102 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:16 smithi022 ceph-mon[29221]: mgrmap e1: no daemons active 2024-02-16T19:19:17.862 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:17.205+0000 7f8992bab700 1 Processor -- start 2024-02-16T19:19:17.862 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:17.205+0000 7f8992bab700 1 -- start start 2024-02-16T19:19:17.862 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:17.205+0000 7f8992bab700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f898c1059d0 0x7f898c105de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:19:17.862 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:17.205+0000 7f8992bab700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f898c106320 con 0x7f898c1059d0 2024-02-16T19:19:17.863 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:17.205+0000 7f8990947700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f898c1059d0 0x7f898c105de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:19:17.863 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:17.205+0000 7f8990947700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f898c1059d0 0x7f898c105de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43316/0 (socket says 172.21.15.22:43316) 2024-02-16T19:19:17.863 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:17.205+0000 7f8990947700 1 -- 172.21.15.22:0/2685376507 learned_addr learned my addr 172.21.15.22:0/2685376507 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:19:17.863 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:17.206+0000 7f8990947700 1 -- 172.21.15.22:0/2685376507 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f898c106460 con 0x7f898c1059d0 2024-02-16T19:19:17.863 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:17.206+0000 7f8990947700 1 --2- 172.21.15.22:0/2685376507 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f898c1059d0 0x7f898c105de0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f8980009870 tx=0x7f8980009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=414eaf962499ccf2 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:19:17.863 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:17.206+0000 7f898b7fe700 1 -- 172.21.15.22:0/2685376507 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8980005c30 con 0x7f898c1059d0 2024-02-16T19:19:17.863 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:17.206+0000 7f898b7fe700 1 -- 172.21.15.22:0/2685376507 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8980005d90 con 0x7f898c1059d0 2024-02-16T19:19:17.863 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:17.206+0000 7f898b7fe700 1 -- 172.21.15.22:0/2685376507 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8980006030 con 0x7f898c1059d0 2024-02-16T19:19:17.863 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:17.206+0000 7f8992bab700 1 -- 172.21.15.22:0/2685376507 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f898c1059d0 msgr2=0x7f898c105de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:19:17.863 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:17.206+0000 7f8992bab700 1 --2- 172.21.15.22:0/2685376507 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f898c1059d0 0x7f898c105de0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f8980009870 tx=0x7f8980009b80 comp rx=0 tx=0).stop 2024-02-16T19:19:17.863 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:17.207+0000 7f8992bab700 1 -- 172.21.15.22:0/2685376507 shutdown_connections 2024-02-16T19:19:17.863 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:17.207+0000 7f8992bab700 1 --2- 172.21.15.22:0/2685376507 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f898c1059d0 0x7f898c105de0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:17.863 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:17.207+0000 7f8992bab700 1 -- 172.21.15.22:0/2685376507 >> 172.21.15.22:0/2685376507 conn(0x7f898c101020 msgr2=0x7f898c103450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:19:17.863 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:17.207+0000 7f8992bab700 1 -- 172.21.15.22:0/2685376507 shutdown_connections 2024-02-16T19:19:17.864 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:17.207+0000 7f8992bab700 1 -- 172.21.15.22:0/2685376507 wait complete. 2024-02-16T19:19:17.864 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:17.207+0000 7f8992bab700 1 Processor -- start 2024-02-16T19:19:17.864 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:17.207+0000 7f8992bab700 1 -- start start 2024-02-16T19:19:17.864 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:17.207+0000 7f8992bab700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f898c1059d0 0x7f898c07bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:19:17.864 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:17.207+0000 7f8992bab700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f898c07c160 con 0x7f898c1059d0 2024-02-16T19:19:17.864 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:17.207+0000 7f8990947700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f898c1059d0 0x7f898c07bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:19:17.864 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:17.207+0000 7f8990947700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f898c1059d0 0x7f898c07bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43318/0 (socket says 172.21.15.22:43318) 2024-02-16T19:19:17.864 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:17.207+0000 7f8990947700 1 -- 172.21.15.22:0/3692477948 learned_addr learned my addr 172.21.15.22:0/3692477948 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:19:17.864 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:17.208+0000 7f8990947700 1 -- 172.21.15.22:0/3692477948 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8980009500 con 0x7f898c1059d0 2024-02-16T19:19:17.864 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:17.208+0000 7f8990947700 1 --2- 172.21.15.22:0/3692477948 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f898c1059d0 0x7f898c07bc20 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f8980006160 tx=0x7f8980006190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:19:17.864 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:17.208+0000 7f8989ffb700 1 -- 172.21.15.22:0/3692477948 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8980004430 con 0x7f898c1059d0 2024-02-16T19:19:17.864 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:17.208+0000 7f8989ffb700 1 -- 172.21.15.22:0/3692477948 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8980006760 con 0x7f898c1059d0 2024-02-16T19:19:17.864 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:17.208+0000 7f8992bab700 1 -- 172.21.15.22:0/3692477948 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f898c07a270 con 0x7f898c1059d0 2024-02-16T19:19:17.864 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:17.208+0000 7f8989ffb700 1 -- 172.21.15.22:0/3692477948 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8980011d00 con 0x7f898c1059d0 2024-02-16T19:19:17.865 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:17.208+0000 7f8992bab700 1 -- 172.21.15.22:0/3692477948 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f898c07a710 con 0x7f898c1059d0 2024-02-16T19:19:17.865 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:17.208+0000 7f8989ffb700 1 -- 172.21.15.22:0/3692477948 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f8980018070 con 0x7f898c1059d0 2024-02-16T19:19:17.865 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:17.208+0000 7f8989ffb700 1 -- 172.21.15.22:0/3692477948 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f89800249b0 con 0x7f898c1059d0 2024-02-16T19:19:17.865 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:17.209+0000 7f8992bab700 1 -- 172.21.15.22:0/3692477948 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f898c062380 con 0x7f898c1059d0 2024-02-16T19:19:17.865 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:17.212+0000 7f8989ffb700 1 -- 172.21.15.22:0/3692477948 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f8980016070 con 0x7f898c1059d0 2024-02-16T19:19:17.865 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:17.282+0000 7f8992bab700 1 -- 172.21.15.22:0/3692477948 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f898c07aee0 con 0x7f898c1059d0 2024-02-16T19:19:17.865 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:17.284+0000 7f8989ffb700 1 -- 172.21.15.22:0/3692477948 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f89800068d0 con 0x7f898c1059d0 2024-02-16T19:19:17.865 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:17.285+0000 7f8992bab700 1 -- 172.21.15.22:0/3692477948 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f898c1059d0 msgr2=0x7f898c07bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:19:17.865 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:17.285+0000 7f8992bab700 1 --2- 172.21.15.22:0/3692477948 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f898c1059d0 0x7f898c07bc20 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f8980006160 tx=0x7f8980006190 comp rx=0 tx=0).stop 2024-02-16T19:19:17.865 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:17.285+0000 7f8992bab700 1 -- 172.21.15.22:0/3692477948 shutdown_connections 2024-02-16T19:19:17.865 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:17.285+0000 7f8992bab700 1 --2- 172.21.15.22:0/3692477948 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f898c1059d0 0x7f898c07bc20 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:17.865 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:17.285+0000 7f8992bab700 1 -- 172.21.15.22:0/3692477948 >> 172.21.15.22:0/3692477948 conn(0x7f898c101020 msgr2=0x7f898c101d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:19:17.865 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:17.286+0000 7f8992bab700 1 -- 172.21.15.22:0/3692477948 shutdown_connections 2024-02-16T19:19:17.865 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:17.286+0000 7f8992bab700 1 -- 172.21.15.22:0/3692477948 wait complete. 2024-02-16T19:19:17.866 INFO:teuthology.orchestra.run.smithi022.stdout:Wrote config to /etc/ceph/ceph.conf 2024-02-16T19:19:17.887 INFO:teuthology.orchestra.run.smithi022.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-02-16T19:19:17.888 INFO:teuthology.orchestra.run.smithi022.stdout:Creating mgr... 2024-02-16T19:19:17.888 INFO:teuthology.orchestra.run.smithi022.stdout:Verifying port 0.0.0.0:9283 ... 2024-02-16T19:19:17.888 INFO:teuthology.orchestra.run.smithi022.stdout:Verifying port 0.0.0.0:8765 ... 2024-02-16T19:19:17.888 INFO:teuthology.orchestra.run.smithi022.stdout:Verifying port 0.0.0.0:8443 ... 2024-02-16T19:19:18.120 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-2a1346e6-cd00-11ee-95ba-87774f69a715@mgr.smithi022.gmdcch 2024-02-16T19:19:18.120 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stderr Failed to reset failed state of unit ceph-2a1346e6-cd00-11ee-95ba-87774f69a715@mgr.smithi022.gmdcch.service: Unit ceph-2a1346e6-cd00-11ee-95ba-87774f69a715@mgr.smithi022.gmdcch.service not loaded. 2024-02-16T19:19:18.268 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-2a1346e6-cd00-11ee-95ba-87774f69a715.target.wants/ceph-2a1346e6-cd00-11ee-95ba-87774f69a715@mgr.smithi022.gmdcch.service → /etc/systemd/system/ceph-2a1346e6-cd00-11ee-95ba-87774f69a715@.service. 2024-02-16T19:19:18.600 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:18 smithi022 ceph-mon[29221]: from='client.? 172.21.15.22:0/3692477948' entity='client.admin' 2024-02-16T19:19:18.805 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-16T19:19:18.805 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout disabled 2024-02-16T19:19:18.812 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-16T19:19:18.812 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout inactive 2024-02-16T19:19:18.812 INFO:teuthology.orchestra.run.smithi022.stdout:firewalld.service is not enabled 2024-02-16T19:19:18.812 INFO:teuthology.orchestra.run.smithi022.stdout:Not possible to enable service . firewalld.service is not available 2024-02-16T19:19:18.820 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-16T19:19:18.820 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout disabled 2024-02-16T19:19:18.827 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-16T19:19:18.827 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout inactive 2024-02-16T19:19:18.827 INFO:teuthology.orchestra.run.smithi022.stdout:firewalld.service is not enabled 2024-02-16T19:19:18.827 INFO:teuthology.orchestra.run.smithi022.stdout:Not possible to open ports <[9283, 8765, 8443]>. firewalld.service is not available 2024-02-16T19:19:18.828 INFO:teuthology.orchestra.run.smithi022.stdout:Waiting for mgr to start... 2024-02-16T19:19:18.828 INFO:teuthology.orchestra.run.smithi022.stdout:Waiting for mgr... 2024-02-16T19:19:19.623 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:19 smithi022 ceph-mon[29221]: from='client.? 172.21.15.22:0/904449140' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-16T19:19:20.078 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-02-16T19:19:20.078 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout { 2024-02-16T19:19:20.079 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "fsid": "2a1346e6-cd00-11ee-95ba-87774f69a715", 2024-02-16T19:19:20.079 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "health": { 2024-02-16T19:19:20.079 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-16T19:19:20.079 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-16T19:19:20.079 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-16T19:19:20.079 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-02-16T19:19:20.079 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-16T19:19:20.079 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-16T19:19:20.079 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 0 2024-02-16T19:19:20.079 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2024-02-16T19:19:20.079 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-16T19:19:20.079 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "smithi022" 2024-02-16T19:19:20.079 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2024-02-16T19:19:20.079 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-02-16T19:19:20.079 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-16T19:19:20.079 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-16T19:19:20.079 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-02-16T19:19:20.080 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-16T19:19:20.080 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-02-16T19:19:20.080 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-16T19:19:20.080 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-16T19:19:20.080 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-16T19:19:20.080 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-16T19:19:20.080 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-16T19:19:20.080 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-16T19:19:20.080 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-16T19:19:20.080 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-16T19:19:20.080 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-02-16T19:19:20.080 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-16T19:19:20.080 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-16T19:19:20.080 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-16T19:19:20.081 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-16T19:19:20.081 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-16T19:19:20.081 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-16T19:19:20.081 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-16T19:19:20.081 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-16T19:19:20.081 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-16T19:19:20.081 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-02-16T19:19:20.081 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-16T19:19:20.081 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-16T19:19:20.081 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-16T19:19:20.081 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-16T19:19:20.081 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-02-16T19:19:20.081 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-16T19:19:20.081 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-16T19:19:20.081 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-16T19:19:20.082 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-16T19:19:20.082 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-16T19:19:20.082 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-16T19:19:20.082 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "restful" 2024-02-16T19:19:20.082 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2024-02-16T19:19:20.082 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-16T19:19:20.082 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-02-16T19:19:20.082 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-16T19:19:20.082 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-16T19:19:20.082 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "modified": "2024-02-16T19:19:11.528729+0000", 2024-02-16T19:19:20.082 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-16T19:19:20.082 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-02-16T19:19:20.082 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-16T19:19:20.082 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout } 2024-02-16T19:19:20.082 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:19.288+0000 7fb7ad7fe700 1 Processor -- start 2024-02-16T19:19:20.082 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:19.288+0000 7fb7ad7fe700 1 -- start start 2024-02-16T19:19:20.083 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:19.288+0000 7fb7ad7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb7a81064f0 0x7fb7a81088d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:19:20.083 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:19.288+0000 7fb7ad7fe700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a8074510 con 0x7fb7a81064f0 2024-02-16T19:19:20.083 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:19.289+0000 7fb7a6ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb7a81064f0 0x7fb7a81088d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:19:20.083 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:19.289+0000 7fb7a6ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb7a81064f0 0x7fb7a81088d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43352/0 (socket says 172.21.15.22:43352) 2024-02-16T19:19:20.083 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:19.289+0000 7fb7a6ffd700 1 -- 172.21.15.22:0/2608339791 learned_addr learned my addr 172.21.15.22:0/2608339791 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:19:20.083 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:19.289+0000 7fb7a6ffd700 1 -- 172.21.15.22:0/2608339791 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7a8074650 con 0x7fb7a81064f0 2024-02-16T19:19:20.083 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:19.289+0000 7fb7a6ffd700 1 --2- 172.21.15.22:0/2608339791 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb7a81064f0 0x7fb7a81088d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb7980098a0 tx=0x7fb798009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57daaf6fccfe6717 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:19:20.083 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:19.289+0000 7fb7a5ffb700 1 -- 172.21.15.22:0/2608339791 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb798004180 con 0x7fb7a81064f0 2024-02-16T19:19:20.083 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:19.289+0000 7fb7a5ffb700 1 -- 172.21.15.22:0/2608339791 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb7980042e0 con 0x7fb7a81064f0 2024-02-16T19:19:20.083 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:19.289+0000 7fb7a5ffb700 1 -- 172.21.15.22:0/2608339791 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb798011a50 con 0x7fb7a81064f0 2024-02-16T19:19:20.083 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:19.290+0000 7fb7ad7fe700 1 -- 172.21.15.22:0/2608339791 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb7a81064f0 msgr2=0x7fb7a81088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:19:20.083 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:19.290+0000 7fb7ad7fe700 1 --2- 172.21.15.22:0/2608339791 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb7a81064f0 0x7fb7a81088d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb7980098a0 tx=0x7fb798009bb0 comp rx=0 tx=0).stop 2024-02-16T19:19:20.083 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:19.290+0000 7fb7ad7fe700 1 -- 172.21.15.22:0/2608339791 shutdown_connections 2024-02-16T19:19:20.083 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:19.290+0000 7fb7ad7fe700 1 --2- 172.21.15.22:0/2608339791 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb7a81064f0 0x7fb7a81088d0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:20.083 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:19.290+0000 7fb7ad7fe700 1 -- 172.21.15.22:0/2608339791 >> 172.21.15.22:0/2608339791 conn(0x7fb7a80fff10 msgr2=0x7fb7a8102360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:19:20.083 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:19.290+0000 7fb7ad7fe700 1 -- 172.21.15.22:0/2608339791 shutdown_connections 2024-02-16T19:19:20.083 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:19.290+0000 7fb7ad7fe700 1 -- 172.21.15.22:0/2608339791 wait complete. 2024-02-16T19:19:20.084 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:19.291+0000 7fb7ad7fe700 1 Processor -- start 2024-02-16T19:19:20.084 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:19.291+0000 7fb7ad7fe700 1 -- start start 2024-02-16T19:19:20.084 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:19.291+0000 7fb7ad7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb7a81064f0 0x7fb7a8105e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:19:20.084 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:19.291+0000 7fb7ad7fe700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a8106390 con 0x7fb7a81064f0 2024-02-16T19:19:20.084 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:19.291+0000 7fb7a6ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb7a81064f0 0x7fb7a8105e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:19:20.084 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:19.291+0000 7fb7a6ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb7a81064f0 0x7fb7a8105e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43356/0 (socket says 172.21.15.22:43356) 2024-02-16T19:19:20.084 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:19.291+0000 7fb7a6ffd700 1 -- 172.21.15.22:0/904449140 learned_addr learned my addr 172.21.15.22:0/904449140 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:19:20.084 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:19.291+0000 7fb7a6ffd700 1 -- 172.21.15.22:0/904449140 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb798009550 con 0x7fb7a81064f0 2024-02-16T19:19:20.084 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:19.292+0000 7fb7a6ffd700 1 --2- 172.21.15.22:0/904449140 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb7a81064f0 0x7fb7a8105e50 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb798009520 tx=0x7fb798011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:19:20.084 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:19.292+0000 7fb79ffff700 1 -- 172.21.15.22:0/904449140 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb798003a80 con 0x7fb7a81064f0 2024-02-16T19:19:20.084 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:19.292+0000 7fb79ffff700 1 -- 172.21.15.22:0/904449140 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb798003be0 con 0x7fb7a81064f0 2024-02-16T19:19:20.084 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:19.292+0000 7fb79ffff700 1 -- 172.21.15.22:0/904449140 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb7980197b0 con 0x7fb7a81064f0 2024-02-16T19:19:20.084 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:19.292+0000 7fb7ad7fe700 1 -- 172.21.15.22:0/904449140 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7a81044a0 con 0x7fb7a81064f0 2024-02-16T19:19:20.085 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:19.292+0000 7fb7ad7fe700 1 -- 172.21.15.22:0/904449140 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7a8104940 con 0x7fb7a81064f0 2024-02-16T19:19:20.085 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:19.292+0000 7fb79ffff700 1 -- 172.21.15.22:0/904449140 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fb798019d20 con 0x7fb7a81064f0 2024-02-16T19:19:20.085 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:19.292+0000 7fb79ffff700 1 -- 172.21.15.22:0/904449140 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb7980247c0 con 0x7fb7a81064f0 2024-02-16T19:19:20.085 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:19.293+0000 7fb7ad7fe700 1 -- 172.21.15.22:0/904449140 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7a8109a60 con 0x7fb7a81064f0 2024-02-16T19:19:20.085 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:19.295+0000 7fb79ffff700 1 -- 172.21.15.22:0/904449140 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7fb79801ac30 con 0x7fb7a81064f0 2024-02-16T19:19:20.085 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:19.370+0000 7fb7ad7fe700 1 -- 172.21.15.22:0/904449140 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fb7a8104fa0 con 0x7fb7a81064f0 2024-02-16T19:19:20.085 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:19.371+0000 7fb79ffff700 1 -- 172.21.15.22:0/904449140 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7fb798024980 con 0x7fb7a81064f0 2024-02-16T19:19:20.085 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:19.372+0000 7fb7ad7fe700 1 -- 172.21.15.22:0/904449140 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb7a81064f0 msgr2=0x7fb7a8105e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:19:20.085 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:19.372+0000 7fb7ad7fe700 1 --2- 172.21.15.22:0/904449140 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb7a81064f0 0x7fb7a8105e50 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb798009520 tx=0x7fb798011ab0 comp rx=0 tx=0).stop 2024-02-16T19:19:20.085 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:19.372+0000 7fb7ad7fe700 1 -- 172.21.15.22:0/904449140 shutdown_connections 2024-02-16T19:19:20.085 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:19.372+0000 7fb7ad7fe700 1 --2- 172.21.15.22:0/904449140 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb7a81064f0 0x7fb7a8105e50 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:20.085 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:19.372+0000 7fb7ad7fe700 1 -- 172.21.15.22:0/904449140 >> 172.21.15.22:0/904449140 conn(0x7fb7a80fff10 msgr2=0x7fb7a8101600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:19:20.085 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:19.373+0000 7fb7ad7fe700 1 -- 172.21.15.22:0/904449140 shutdown_connections 2024-02-16T19:19:20.085 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:19.373+0000 7fb7ad7fe700 1 -- 172.21.15.22:0/904449140 wait complete. 2024-02-16T19:19:20.085 INFO:teuthology.orchestra.run.smithi022.stdout:mgr not available, waiting (1/15)... 2024-02-16T19:19:22.938 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:22 smithi022 ceph-mon[29221]: from='client.? 172.21.15.22:0/2551939038' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-16T19:19:23.127 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-02-16T19:19:23.127 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout { 2024-02-16T19:19:23.127 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "fsid": "2a1346e6-cd00-11ee-95ba-87774f69a715", 2024-02-16T19:19:23.127 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "health": { 2024-02-16T19:19:23.127 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-16T19:19:23.127 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-16T19:19:23.127 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-16T19:19:23.127 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-02-16T19:19:23.127 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-16T19:19:23.127 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-16T19:19:23.127 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 0 2024-02-16T19:19:23.128 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2024-02-16T19:19:23.128 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-16T19:19:23.128 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "smithi022" 2024-02-16T19:19:23.128 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2024-02-16T19:19:23.128 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-02-16T19:19:23.128 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-16T19:19:23.128 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-16T19:19:23.128 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-02-16T19:19:23.128 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-16T19:19:23.128 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-02-16T19:19:23.128 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-16T19:19:23.128 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-16T19:19:23.128 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-16T19:19:23.128 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-16T19:19:23.128 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-16T19:19:23.136 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-16T19:19:23.136 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-16T19:19:23.137 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-16T19:19:23.137 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-02-16T19:19:23.137 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-16T19:19:23.137 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-16T19:19:23.137 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-16T19:19:23.137 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-16T19:19:23.137 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-16T19:19:23.137 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-16T19:19:23.137 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-16T19:19:23.137 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-16T19:19:23.137 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-16T19:19:23.137 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-02-16T19:19:23.137 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-16T19:19:23.137 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-16T19:19:23.137 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-16T19:19:23.138 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-16T19:19:23.138 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-02-16T19:19:23.138 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-16T19:19:23.138 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-16T19:19:23.138 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-16T19:19:23.138 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-16T19:19:23.138 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-16T19:19:23.138 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-16T19:19:23.138 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "restful" 2024-02-16T19:19:23.138 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2024-02-16T19:19:23.138 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-16T19:19:23.138 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-02-16T19:19:23.138 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-16T19:19:23.138 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-16T19:19:23.138 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "modified": "2024-02-16T19:19:11.528729+0000", 2024-02-16T19:19:23.138 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-16T19:19:23.139 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-02-16T19:19:23.139 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-16T19:19:23.139 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout } 2024-02-16T19:19:23.139 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:22.511+0000 7f1959e7a700 1 Processor -- start 2024-02-16T19:19:23.139 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:22.511+0000 7f1959e7a700 1 -- start start 2024-02-16T19:19:23.139 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:22.511+0000 7f1959e7a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1954106520 0x7f1954108900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:19:23.139 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:22.511+0000 7f1959e7a700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1954079d10 con 0x7f1954106520 2024-02-16T19:19:23.139 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:22.511+0000 7f19537fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1954106520 0x7f1954108900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:19:23.139 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:22.511+0000 7f19537fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1954106520 0x7f1954108900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43362/0 (socket says 172.21.15.22:43362) 2024-02-16T19:19:23.139 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:22.511+0000 7f19537fe700 1 -- 172.21.15.22:0/1450590437 learned_addr learned my addr 172.21.15.22:0/1450590437 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:19:23.139 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:22.511+0000 7f19537fe700 1 -- 172.21.15.22:0/1450590437 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1954079e50 con 0x7f1954106520 2024-02-16T19:19:23.139 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:22.511+0000 7f19537fe700 1 --2- 172.21.15.22:0/1450590437 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1954106520 0x7f1954108900 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f19440098a0 tx=0x7f1944009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=595227bddc171924 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:19:23.139 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:22.511+0000 7f19527fc700 1 -- 172.21.15.22:0/1450590437 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1944004180 con 0x7f1954106520 2024-02-16T19:19:23.139 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:22.512+0000 7f19527fc700 1 -- 172.21.15.22:0/1450590437 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f19440042e0 con 0x7f1954106520 2024-02-16T19:19:23.139 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:22.512+0000 7f19527fc700 1 -- 172.21.15.22:0/1450590437 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1944011a50 con 0x7f1954106520 2024-02-16T19:19:23.140 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:22.512+0000 7f1959e7a700 1 -- 172.21.15.22:0/1450590437 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1954106520 msgr2=0x7f1954108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:19:23.140 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:22.512+0000 7f1959e7a700 1 --2- 172.21.15.22:0/1450590437 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1954106520 0x7f1954108900 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f19440098a0 tx=0x7f1944009bb0 comp rx=0 tx=0).stop 2024-02-16T19:19:23.140 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:22.512+0000 7f1959e7a700 1 -- 172.21.15.22:0/1450590437 shutdown_connections 2024-02-16T19:19:23.140 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:22.512+0000 7f1959e7a700 1 --2- 172.21.15.22:0/1450590437 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1954106520 0x7f1954108900 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:23.140 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:22.512+0000 7f1959e7a700 1 -- 172.21.15.22:0/1450590437 >> 172.21.15.22:0/1450590437 conn(0x7f19540fff00 msgr2=0x7f1954102350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:19:23.140 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:22.512+0000 7f1959e7a700 1 -- 172.21.15.22:0/1450590437 shutdown_connections 2024-02-16T19:19:23.140 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:22.512+0000 7f1959e7a700 1 -- 172.21.15.22:0/1450590437 wait complete. 2024-02-16T19:19:23.140 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:22.513+0000 7f1959e7a700 1 Processor -- start 2024-02-16T19:19:23.140 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:22.513+0000 7f1959e7a700 1 -- start start 2024-02-16T19:19:23.140 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:22.513+0000 7f1959e7a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1954106520 0x7f1954105e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:19:23.140 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:22.513+0000 7f1959e7a700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19541063c0 con 0x7f1954106520 2024-02-16T19:19:23.140 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:22.513+0000 7f19537fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1954106520 0x7f1954105e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:19:23.140 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:22.513+0000 7f19537fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1954106520 0x7f1954105e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43366/0 (socket says 172.21.15.22:43366) 2024-02-16T19:19:23.140 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:22.513+0000 7f19537fe700 1 -- 172.21.15.22:0/2551939038 learned_addr learned my addr 172.21.15.22:0/2551939038 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:19:23.140 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:22.513+0000 7f19537fe700 1 -- 172.21.15.22:0/2551939038 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1944009550 con 0x7f1954106520 2024-02-16T19:19:23.141 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:22.514+0000 7f19537fe700 1 --2- 172.21.15.22:0/2551939038 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1954106520 0x7f1954105e80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1944009520 tx=0x7f1944011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:19:23.141 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:22.514+0000 7f1950ff9700 1 -- 172.21.15.22:0/2551939038 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1944003a80 con 0x7f1954106520 2024-02-16T19:19:23.141 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:22.514+0000 7f1950ff9700 1 -- 172.21.15.22:0/2551939038 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1944003be0 con 0x7f1954106520 2024-02-16T19:19:23.141 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:22.514+0000 7f1959e7a700 1 -- 172.21.15.22:0/2551939038 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1954104530 con 0x7f1954106520 2024-02-16T19:19:23.141 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:22.514+0000 7f1950ff9700 1 -- 172.21.15.22:0/2551939038 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1944019750 con 0x7f1954106520 2024-02-16T19:19:23.141 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:22.514+0000 7f1959e7a700 1 -- 172.21.15.22:0/2551939038 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1954104a30 con 0x7f1954106520 2024-02-16T19:19:23.141 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:22.514+0000 7f1950ff9700 1 -- 172.21.15.22:0/2551939038 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f1944019d30 con 0x7f1954106520 2024-02-16T19:19:23.141 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:22.514+0000 7f1950ff9700 1 -- 172.21.15.22:0/2551939038 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f19440247c0 con 0x7f1954106520 2024-02-16T19:19:23.141 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:22.515+0000 7f1959e7a700 1 -- 172.21.15.22:0/2551939038 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19540623f0 con 0x7f1954106520 2024-02-16T19:19:23.141 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:22.517+0000 7f1950ff9700 1 -- 172.21.15.22:0/2551939038 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f19540623f0 con 0x7f1954106520 2024-02-16T19:19:23.141 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:22.592+0000 7f1959e7a700 1 -- 172.21.15.22:0/2551939038 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f1954104d30 con 0x7f1954106520 2024-02-16T19:19:23.141 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:22.593+0000 7f1950ff9700 1 -- 172.21.15.22:0/2551939038 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7f194401ac30 con 0x7f1954106520 2024-02-16T19:19:23.141 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:22.594+0000 7f1959e7a700 1 -- 172.21.15.22:0/2551939038 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1954106520 msgr2=0x7f1954105e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:19:23.141 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:22.594+0000 7f1959e7a700 1 --2- 172.21.15.22:0/2551939038 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1954106520 0x7f1954105e80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1944009520 tx=0x7f1944011ab0 comp rx=0 tx=0).stop 2024-02-16T19:19:23.142 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:22.594+0000 7f1959e7a700 1 -- 172.21.15.22:0/2551939038 shutdown_connections 2024-02-16T19:19:23.142 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:22.594+0000 7f1959e7a700 1 --2- 172.21.15.22:0/2551939038 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1954106520 0x7f1954105e80 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:23.142 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:22.594+0000 7f1959e7a700 1 -- 172.21.15.22:0/2551939038 >> 172.21.15.22:0/2551939038 conn(0x7f19540fff00 msgr2=0x7f19541015c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:19:23.142 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:22.594+0000 7f1959e7a700 1 -- 172.21.15.22:0/2551939038 shutdown_connections 2024-02-16T19:19:23.142 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:22.594+0000 7f1959e7a700 1 -- 172.21.15.22:0/2551939038 wait complete. 2024-02-16T19:19:23.142 INFO:teuthology.orchestra.run.smithi022.stdout:mgr not available, waiting (2/15)... 2024-02-16T19:19:25.927 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:25 smithi022 ceph-mon[29221]: Activating manager daemon smithi022.gmdcch 2024-02-16T19:19:25.927 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:25 smithi022 ceph-mon[29221]: mgrmap e2: smithi022.gmdcch(active, starting, since 0.00320896s) 2024-02-16T19:19:25.927 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:25 smithi022 ceph-mon[29221]: from='mgr.14100 172.21.15.22:0/3518558184' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-16T19:19:25.927 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:25 smithi022 ceph-mon[29221]: from='mgr.14100 172.21.15.22:0/3518558184' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-16T19:19:25.927 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:25 smithi022 ceph-mon[29221]: from='mgr.14100 172.21.15.22:0/3518558184' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-16T19:19:25.927 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:25 smithi022 ceph-mon[29221]: from='mgr.14100 172.21.15.22:0/3518558184' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mon metadata", "id": "smithi022"}]: dispatch 2024-02-16T19:19:25.928 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:25 smithi022 ceph-mon[29221]: from='mgr.14100 172.21.15.22:0/3518558184' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mgr metadata", "who": "smithi022.gmdcch", "id": "smithi022.gmdcch"}]: dispatch 2024-02-16T19:19:25.928 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:25 smithi022 ceph-mon[29221]: Manager daemon smithi022.gmdcch is now available 2024-02-16T19:19:25.928 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:25 smithi022 ceph-mon[29221]: from='mgr.14100 172.21.15.22:0/3518558184' entity='mgr.smithi022.gmdcch' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi022.gmdcch/mirror_snapshot_schedule"}]: dispatch 2024-02-16T19:19:25.928 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:25 smithi022 ceph-mon[29221]: from='mgr.14100 172.21.15.22:0/3518558184' entity='mgr.smithi022.gmdcch' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi022.gmdcch/trash_purge_schedule"}]: dispatch 2024-02-16T19:19:25.928 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:25 smithi022 ceph-mon[29221]: from='mgr.14100 172.21.15.22:0/3518558184' entity='mgr.smithi022.gmdcch' 2024-02-16T19:19:25.928 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:25 smithi022 ceph-mon[29221]: from='mgr.14100 172.21.15.22:0/3518558184' entity='mgr.smithi022.gmdcch' 2024-02-16T19:19:25.928 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:25 smithi022 ceph-mon[29221]: from='mgr.14100 172.21.15.22:0/3518558184' entity='mgr.smithi022.gmdcch' 2024-02-16T19:19:26.225 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-02-16T19:19:26.226 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout { 2024-02-16T19:19:26.226 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "fsid": "2a1346e6-cd00-11ee-95ba-87774f69a715", 2024-02-16T19:19:26.226 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "health": { 2024-02-16T19:19:26.226 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-16T19:19:26.226 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-16T19:19:26.226 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-16T19:19:26.226 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-02-16T19:19:26.226 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-16T19:19:26.226 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-16T19:19:26.226 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 0 2024-02-16T19:19:26.226 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2024-02-16T19:19:26.226 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-16T19:19:26.226 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "smithi022" 2024-02-16T19:19:26.226 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2024-02-16T19:19:26.227 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-02-16T19:19:26.227 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-16T19:19:26.227 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-16T19:19:26.227 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-02-16T19:19:26.227 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-16T19:19:26.227 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-02-16T19:19:26.227 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-16T19:19:26.227 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-16T19:19:26.227 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-16T19:19:26.227 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-16T19:19:26.227 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-16T19:19:26.235 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-16T19:19:26.235 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-16T19:19:26.236 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-16T19:19:26.236 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-02-16T19:19:26.236 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-16T19:19:26.236 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-16T19:19:26.236 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-16T19:19:26.236 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-16T19:19:26.236 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-16T19:19:26.236 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-16T19:19:26.236 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-16T19:19:26.236 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-16T19:19:26.236 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-16T19:19:26.236 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-02-16T19:19:26.236 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-16T19:19:26.237 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-16T19:19:26.237 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-16T19:19:26.237 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-16T19:19:26.237 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-02-16T19:19:26.237 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-16T19:19:26.237 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-16T19:19:26.237 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-16T19:19:26.237 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-16T19:19:26.237 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-16T19:19:26.237 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-16T19:19:26.237 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "restful" 2024-02-16T19:19:26.237 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2024-02-16T19:19:26.237 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-16T19:19:26.237 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-02-16T19:19:26.237 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-16T19:19:26.238 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-16T19:19:26.238 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "modified": "2024-02-16T19:19:11.528729+0000", 2024-02-16T19:19:26.238 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-16T19:19:26.238 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-02-16T19:19:26.238 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-16T19:19:26.238 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout } 2024-02-16T19:19:26.238 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:25.541+0000 7fc178b30700 1 Processor -- start 2024-02-16T19:19:26.238 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:25.541+0000 7fc178b30700 1 -- start start 2024-02-16T19:19:26.238 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:25.541+0000 7fc178b30700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc17407bdb0 0x7fc17407a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:19:26.238 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:25.541+0000 7fc178b30700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc17407a7f0 con 0x7fc17407bdb0 2024-02-16T19:19:26.238 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:25.542+0000 7fc17259c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc17407bdb0 0x7fc17407a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:19:26.238 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:25.542+0000 7fc17259c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc17407bdb0 0x7fc17407a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58652/0 (socket says 172.21.15.22:58652) 2024-02-16T19:19:26.238 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:25.542+0000 7fc17259c700 1 -- 172.21.15.22:0/1920171195 learned_addr learned my addr 172.21.15.22:0/1920171195 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:19:26.239 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:25.542+0000 7fc17259c700 1 -- 172.21.15.22:0/1920171195 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc17407a930 con 0x7fc17407bdb0 2024-02-16T19:19:26.239 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:25.542+0000 7fc17259c700 1 --2- 172.21.15.22:0/1920171195 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc17407bdb0 0x7fc17407a2b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc1640098a0 tx=0x7fc164009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d3ecc7a56cb93a0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:19:26.239 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:25.543+0000 7fc17159a700 1 -- 172.21.15.22:0/1920171195 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc164004180 con 0x7fc17407bdb0 2024-02-16T19:19:26.239 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:25.543+0000 7fc17159a700 1 -- 172.21.15.22:0/1920171195 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc1640042e0 con 0x7fc17407bdb0 2024-02-16T19:19:26.239 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:25.543+0000 7fc17159a700 1 -- 172.21.15.22:0/1920171195 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc164011a50 con 0x7fc17407bdb0 2024-02-16T19:19:26.239 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:25.544+0000 7fc178b30700 1 -- 172.21.15.22:0/1920171195 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc17407bdb0 msgr2=0x7fc17407a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:19:26.239 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:25.544+0000 7fc178b30700 1 --2- 172.21.15.22:0/1920171195 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc17407bdb0 0x7fc17407a2b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc1640098a0 tx=0x7fc164009bb0 comp rx=0 tx=0).stop 2024-02-16T19:19:26.239 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:25.544+0000 7fc178b30700 1 -- 172.21.15.22:0/1920171195 shutdown_connections 2024-02-16T19:19:26.239 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:25.544+0000 7fc178b30700 1 --2- 172.21.15.22:0/1920171195 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc17407bdb0 0x7fc17407a2b0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:26.239 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:25.544+0000 7fc178b30700 1 -- 172.21.15.22:0/1920171195 >> 172.21.15.22:0/1920171195 conn(0x7fc174100ff0 msgr2=0x7fc174103440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:19:26.239 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:25.544+0000 7fc178b30700 1 -- 172.21.15.22:0/1920171195 shutdown_connections 2024-02-16T19:19:26.239 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:25.544+0000 7fc178b30700 1 -- 172.21.15.22:0/1920171195 wait complete. 2024-02-16T19:19:26.239 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:25.545+0000 7fc178b30700 1 Processor -- start 2024-02-16T19:19:26.239 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:25.545+0000 7fc178b30700 1 -- start start 2024-02-16T19:19:26.240 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:25.545+0000 7fc178b30700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc17407bdb0 0x7fc17419b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:19:26.240 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:25.545+0000 7fc178b30700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc17419bf10 con 0x7fc17407bdb0 2024-02-16T19:19:26.240 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:25.545+0000 7fc17259c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc17407bdb0 0x7fc17419b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:19:26.240 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:25.545+0000 7fc17259c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc17407bdb0 0x7fc17419b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58668/0 (socket says 172.21.15.22:58668) 2024-02-16T19:19:26.240 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:25.545+0000 7fc17259c700 1 -- 172.21.15.22:0/2272325619 learned_addr learned my addr 172.21.15.22:0/2272325619 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:19:26.240 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:25.546+0000 7fc17259c700 1 -- 172.21.15.22:0/2272325619 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc164009550 con 0x7fc17407bdb0 2024-02-16T19:19:26.240 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:25.546+0000 7fc17259c700 1 --2- 172.21.15.22:0/2272325619 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc17407bdb0 0x7fc17419b9d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc164003de0 tx=0x7fc1640047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:19:26.240 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:25.546+0000 7fc1637fe700 1 -- 172.21.15.22:0/2272325619 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc1640038b0 con 0x7fc17407bdb0 2024-02-16T19:19:26.240 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:25.546+0000 7fc1637fe700 1 -- 172.21.15.22:0/2272325619 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc164003a10 con 0x7fc17407bdb0 2024-02-16T19:19:26.240 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:25.546+0000 7fc178b30700 1 -- 172.21.15.22:0/2272325619 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc17419c110 con 0x7fc17407bdb0 2024-02-16T19:19:26.240 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:25.546+0000 7fc1637fe700 1 -- 172.21.15.22:0/2272325619 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc164019600 con 0x7fc17407bdb0 2024-02-16T19:19:26.240 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:25.546+0000 7fc178b30700 1 -- 172.21.15.22:0/2272325619 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc17419c5b0 con 0x7fc17407bdb0 2024-02-16T19:19:26.240 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:25.547+0000 7fc1637fe700 1 -- 172.21.15.22:0/2272325619 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 2) v1 ==== 44840+0+0 (secure 0 0 0) 0x7fc164019800 con 0x7fc17407bdb0 2024-02-16T19:19:26.241 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:25.547+0000 7fc178b30700 1 -- 172.21.15.22:0/2272325619 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc17410bcb0 con 0x7fc17407bdb0 2024-02-16T19:19:26.241 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:25.548+0000 7fc1637fe700 1 -- 172.21.15.22:0/2272325619 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc16404baa0 con 0x7fc17407bdb0 2024-02-16T19:19:26.241 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:25.550+0000 7fc1637fe700 1 -- 172.21.15.22:0/2272325619 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7fc164019ab0 con 0x7fc17407bdb0 2024-02-16T19:19:26.241 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:25.630+0000 7fc178b30700 1 -- 172.21.15.22:0/2272325619 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fc17410bed0 con 0x7fc17407bdb0 2024-02-16T19:19:26.241 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:25.631+0000 7fc1637fe700 1 -- 172.21.15.22:0/2272325619 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7fc16401e120 con 0x7fc17407bdb0 2024-02-16T19:19:26.241 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:25.631+0000 7fc178b30700 1 -- 172.21.15.22:0/2272325619 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc17407bdb0 msgr2=0x7fc17419b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:19:26.241 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:25.632+0000 7fc178b30700 1 --2- 172.21.15.22:0/2272325619 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc17407bdb0 0x7fc17419b9d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc164003de0 tx=0x7fc1640047c0 comp rx=0 tx=0).stop 2024-02-16T19:19:26.241 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:25.632+0000 7fc178b30700 1 -- 172.21.15.22:0/2272325619 shutdown_connections 2024-02-16T19:19:26.241 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:25.632+0000 7fc178b30700 1 --2- 172.21.15.22:0/2272325619 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc17407bdb0 0x7fc17419b9d0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:26.241 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:25.632+0000 7fc178b30700 1 -- 172.21.15.22:0/2272325619 >> 172.21.15.22:0/2272325619 conn(0x7fc174100ff0 msgr2=0x7fc1741026e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:19:26.241 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:25.632+0000 7fc178b30700 1 -- 172.21.15.22:0/2272325619 shutdown_connections 2024-02-16T19:19:26.241 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:25.632+0000 7fc178b30700 1 -- 172.21.15.22:0/2272325619 wait complete. 2024-02-16T19:19:26.241 INFO:teuthology.orchestra.run.smithi022.stdout:mgr not available, waiting (3/15)... 2024-02-16T19:19:26.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:26 smithi022 ceph-mon[29221]: from='client.? 172.21.15.22:0/2272325619' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-16T19:19:26.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:26 smithi022 ceph-mon[29221]: mgrmap e3: smithi022.gmdcch(active, since 1.00754s) 2024-02-16T19:19:28.892 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:28 smithi022 ceph-mon[29221]: mgrmap e4: smithi022.gmdcch(active, since 2s) 2024-02-16T19:19:29.522 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-02-16T19:19:29.523 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout { 2024-02-16T19:19:29.523 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "fsid": "2a1346e6-cd00-11ee-95ba-87774f69a715", 2024-02-16T19:19:29.523 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "health": { 2024-02-16T19:19:29.523 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-16T19:19:29.523 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-16T19:19:29.523 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-16T19:19:29.523 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-02-16T19:19:29.523 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-16T19:19:29.523 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-16T19:19:29.523 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 0 2024-02-16T19:19:29.523 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2024-02-16T19:19:29.523 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-16T19:19:29.524 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "smithi022" 2024-02-16T19:19:29.524 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2024-02-16T19:19:29.524 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-02-16T19:19:29.524 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-16T19:19:29.524 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-16T19:19:29.524 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-02-16T19:19:29.524 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-16T19:19:29.524 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-02-16T19:19:29.524 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-16T19:19:29.524 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-16T19:19:29.524 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-16T19:19:29.524 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-16T19:19:29.524 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-16T19:19:29.533 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-16T19:19:29.533 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-16T19:19:29.533 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-16T19:19:29.533 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-02-16T19:19:29.533 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-16T19:19:29.533 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-16T19:19:29.533 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-16T19:19:29.534 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-16T19:19:29.534 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-16T19:19:29.534 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-16T19:19:29.534 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-16T19:19:29.534 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-16T19:19:29.534 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-16T19:19:29.534 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-02-16T19:19:29.534 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-16T19:19:29.534 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-16T19:19:29.534 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-16T19:19:29.534 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-16T19:19:29.534 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-02-16T19:19:29.534 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-16T19:19:29.534 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-16T19:19:29.534 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-16T19:19:29.534 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-16T19:19:29.535 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-16T19:19:29.535 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-16T19:19:29.535 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "restful" 2024-02-16T19:19:29.535 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2024-02-16T19:19:29.535 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-16T19:19:29.535 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-02-16T19:19:29.535 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-16T19:19:29.535 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-16T19:19:29.535 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "modified": "2024-02-16T19:19:11.528729+0000", 2024-02-16T19:19:29.535 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-16T19:19:29.535 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-02-16T19:19:29.535 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-16T19:19:29.535 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout } 2024-02-16T19:19:29.535 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:28.624+0000 7f797d159700 1 Processor -- start 2024-02-16T19:19:29.535 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:28.624+0000 7f797d159700 1 -- start start 2024-02-16T19:19:29.536 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:28.624+0000 7f797d159700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79781059d0 0x7f7978105de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:19:29.536 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:28.624+0000 7f797d159700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7978106320 con 0x7f79781059d0 2024-02-16T19:19:29.536 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:28.624+0000 7f7976d9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79781059d0 0x7f7978105de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:19:29.536 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:28.624+0000 7f7976d9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79781059d0 0x7f7978105de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58746/0 (socket says 172.21.15.22:58746) 2024-02-16T19:19:29.536 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:28.624+0000 7f7976d9d700 1 -- 172.21.15.22:0/1139008941 learned_addr learned my addr 172.21.15.22:0/1139008941 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:19:29.536 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:28.624+0000 7f7976d9d700 1 -- 172.21.15.22:0/1139008941 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7978106460 con 0x7f79781059d0 2024-02-16T19:19:29.536 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:28.625+0000 7f7976d9d700 1 --2- 172.21.15.22:0/1139008941 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79781059d0 0x7f7978105de0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7968009ed0 tx=0x7f796800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f116251e056c339 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:19:29.536 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:28.625+0000 7f7975d9b700 1 -- 172.21.15.22:0/1139008941 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f796800e070 con 0x7f79781059d0 2024-02-16T19:19:29.536 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:28.625+0000 7f7975d9b700 1 -- 172.21.15.22:0/1139008941 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7968003e90 con 0x7f79781059d0 2024-02-16T19:19:29.536 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:28.625+0000 7f7975d9b700 1 -- 172.21.15.22:0/1139008941 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7968004480 con 0x7f79781059d0 2024-02-16T19:19:29.536 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:28.625+0000 7f797d159700 1 -- 172.21.15.22:0/1139008941 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79781059d0 msgr2=0x7f7978105de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:19:29.536 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:28.625+0000 7f797d159700 1 --2- 172.21.15.22:0/1139008941 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79781059d0 0x7f7978105de0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7968009ed0 tx=0x7f796800b330 comp rx=0 tx=0).stop 2024-02-16T19:19:29.536 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:28.626+0000 7f797d159700 1 -- 172.21.15.22:0/1139008941 shutdown_connections 2024-02-16T19:19:29.537 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:28.626+0000 7f797d159700 1 --2- 172.21.15.22:0/1139008941 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79781059d0 0x7f7978105de0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:29.537 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:28.626+0000 7f797d159700 1 -- 172.21.15.22:0/1139008941 >> 172.21.15.22:0/1139008941 conn(0x7f7978101020 msgr2=0x7f7978103450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:19:29.537 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:28.626+0000 7f797d159700 1 -- 172.21.15.22:0/1139008941 shutdown_connections 2024-02-16T19:19:29.537 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:28.626+0000 7f797d159700 1 -- 172.21.15.22:0/1139008941 wait complete. 2024-02-16T19:19:29.537 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:28.627+0000 7f797d159700 1 Processor -- start 2024-02-16T19:19:29.537 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:28.627+0000 7f797d159700 1 -- start start 2024-02-16T19:19:29.537 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:28.627+0000 7f797d159700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79781059d0 0x7f7978199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:19:29.537 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:28.627+0000 7f797d159700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7978199ba0 con 0x7f79781059d0 2024-02-16T19:19:29.537 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:28.627+0000 7f7976d9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79781059d0 0x7f7978199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:19:29.537 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:28.627+0000 7f7976d9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79781059d0 0x7f7978199660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58760/0 (socket says 172.21.15.22:58760) 2024-02-16T19:19:29.537 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:28.627+0000 7f7976d9d700 1 -- 172.21.15.22:0/2248159169 learned_addr learned my addr 172.21.15.22:0/2248159169 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:19:29.537 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:28.628+0000 7f7976d9d700 1 -- 172.21.15.22:0/2248159169 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7968009b30 con 0x7f79781059d0 2024-02-16T19:19:29.537 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:28.628+0000 7f7976d9d700 1 --2- 172.21.15.22:0/2248159169 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79781059d0 0x7f7978199660 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f796800bd30 tx=0x7f7968004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:19:29.537 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:28.628+0000 7f796ffff700 1 -- 172.21.15.22:0/2248159169 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f796800e040 con 0x7f79781059d0 2024-02-16T19:19:29.537 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:28.628+0000 7f796ffff700 1 -- 172.21.15.22:0/2248159169 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f796801b8a0 con 0x7f79781059d0 2024-02-16T19:19:29.537 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:28.628+0000 7f797d159700 1 -- 172.21.15.22:0/2248159169 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7978199da0 con 0x7f79781059d0 2024-02-16T19:19:29.538 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:28.628+0000 7f796ffff700 1 -- 172.21.15.22:0/2248159169 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f79680128e0 con 0x7f79781059d0 2024-02-16T19:19:29.538 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:28.628+0000 7f797d159700 1 -- 172.21.15.22:0/2248159169 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f797819a240 con 0x7f79781059d0 2024-02-16T19:19:29.538 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:28.629+0000 7f796ffff700 1 -- 172.21.15.22:0/2248159169 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 4) v1 ==== 45073+0+0 (secure 0 0 0) 0x7f7968019040 con 0x7f79781059d0 2024-02-16T19:19:29.538 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:28.629+0000 7f797d159700 1 -- 172.21.15.22:0/2248159169 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7978062380 con 0x7f79781059d0 2024-02-16T19:19:29.538 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:28.629+0000 7f796ffff700 1 --2- 172.21.15.22:0/2248159169 >> [v2:172.21.15.22:6800/3109015836,v1:172.21.15.22:6801/3109015836] conn(0x7f7964038360 0x7f796403a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:19:29.538 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:28.629+0000 7f796ffff700 1 -- 172.21.15.22:0/2248159169 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f796804bd50 con 0x7f79781059d0 2024-02-16T19:19:29.538 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:28.629+0000 7f797659c700 1 --2- 172.21.15.22:0/2248159169 >> [v2:172.21.15.22:6800/3109015836,v1:172.21.15.22:6801/3109015836] conn(0x7f7964038360 0x7f796403a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:19:29.538 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:28.630+0000 7f797659c700 1 --2- 172.21.15.22:0/2248159169 >> [v2:172.21.15.22:6800/3109015836,v1:172.21.15.22:6801/3109015836] conn(0x7f7964038360 0x7f796403a810 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7960006fd0 tx=0x7f7960006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:19:29.538 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:28.635+0000 7f796ffff700 1 -- 172.21.15.22:0/2248159169 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f7968011df0 con 0x7f79781059d0 2024-02-16T19:19:29.538 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:28.918+0000 7f797d159700 1 -- 172.21.15.22:0/2248159169 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f7978107840 con 0x7f79781059d0 2024-02-16T19:19:29.538 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:28.919+0000 7f796ffff700 1 -- 172.21.15.22:0/2248159169 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7f796801f020 con 0x7f79781059d0 2024-02-16T19:19:29.538 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:28.921+0000 7f797d159700 1 -- 172.21.15.22:0/2248159169 >> [v2:172.21.15.22:6800/3109015836,v1:172.21.15.22:6801/3109015836] conn(0x7f7964038360 msgr2=0x7f796403a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:19:29.538 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:28.921+0000 7f797d159700 1 --2- 172.21.15.22:0/2248159169 >> [v2:172.21.15.22:6800/3109015836,v1:172.21.15.22:6801/3109015836] conn(0x7f7964038360 0x7f796403a810 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7960006fd0 tx=0x7f7960006e40 comp rx=0 tx=0).stop 2024-02-16T19:19:29.538 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:28.921+0000 7f797d159700 1 -- 172.21.15.22:0/2248159169 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79781059d0 msgr2=0x7f7978199660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:19:29.539 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:28.921+0000 7f797d159700 1 --2- 172.21.15.22:0/2248159169 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79781059d0 0x7f7978199660 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f796800bd30 tx=0x7f7968004570 comp rx=0 tx=0).stop 2024-02-16T19:19:29.539 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:28.921+0000 7f797d159700 1 -- 172.21.15.22:0/2248159169 shutdown_connections 2024-02-16T19:19:29.539 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:28.921+0000 7f797d159700 1 --2- 172.21.15.22:0/2248159169 >> [v2:172.21.15.22:6800/3109015836,v1:172.21.15.22:6801/3109015836] conn(0x7f7964038360 0x7f796403a810 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:29.539 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:28.921+0000 7f797d159700 1 --2- 172.21.15.22:0/2248159169 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79781059d0 0x7f7978199660 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:29.539 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:28.921+0000 7f797d159700 1 -- 172.21.15.22:0/2248159169 >> 172.21.15.22:0/2248159169 conn(0x7f7978101020 msgr2=0x7f7978101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:19:29.539 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:28.921+0000 7f797d159700 1 -- 172.21.15.22:0/2248159169 shutdown_connections 2024-02-16T19:19:29.539 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:28.921+0000 7f797d159700 1 -- 172.21.15.22:0/2248159169 wait complete. 2024-02-16T19:19:29.539 INFO:teuthology.orchestra.run.smithi022.stdout:mgr is available 2024-02-16T19:19:29.920 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:29 smithi022 ceph-mon[29221]: from='client.? 172.21.15.22:0/2248159169' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-16T19:19:30.708 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-02-16T19:19:30.708 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout [global] 2024-02-16T19:19:30.708 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout fsid = 2a1346e6-cd00-11ee-95ba-87774f69a715 2024-02-16T19:19:30.709 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-02-16T19:19:30.709 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-02-16T19:19:30.709 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-02-16T19:19:30.709 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-02-16T19:19:30.709 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-02-16T19:19:30.709 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout [mgr] 2024-02-16T19:19:30.709 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-02-16T19:19:30.709 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-02-16T19:19:30.709 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout [osd] 2024-02-16T19:19:30.709 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-02-16T19:19:30.709 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-02-16T19:19:30.709 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-02-16T19:19:30.709 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-02-16T19:19:30.710 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-02-16T19:19:30.710 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:29.944+0000 7f4131e8c700 1 Processor -- start 2024-02-16T19:19:30.710 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:29.944+0000 7f4131e8c700 1 -- start start 2024-02-16T19:19:30.710 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:29.944+0000 7f4131e8c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f412c07bdd0 0x7f412c07a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:19:30.710 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:29.944+0000 7f4131e8c700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f412c07a810 con 0x7f412c07bdd0 2024-02-16T19:19:30.710 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:29.944+0000 7f412b7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f412c07bdd0 0x7f412c07a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:19:30.710 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:29.945+0000 7f412b7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f412c07bdd0 0x7f412c07a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58764/0 (socket says 172.21.15.22:58764) 2024-02-16T19:19:30.710 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:29.945+0000 7f412b7fe700 1 -- 172.21.15.22:0/867767192 learned_addr learned my addr 172.21.15.22:0/867767192 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:19:30.710 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:29.945+0000 7f412b7fe700 1 -- 172.21.15.22:0/867767192 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f412c07a950 con 0x7f412c07bdd0 2024-02-16T19:19:30.710 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:29.945+0000 7f412b7fe700 1 --2- 172.21.15.22:0/867767192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f412c07bdd0 0x7f412c07a2d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f411c009870 tx=0x7f411c009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78e361ecccd859c1 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:19:30.710 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:29.945+0000 7f412a7fc700 1 -- 172.21.15.22:0/867767192 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f411c005c30 con 0x7f412c07bdd0 2024-02-16T19:19:30.710 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:29.945+0000 7f412a7fc700 1 -- 172.21.15.22:0/867767192 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f411c005d90 con 0x7f412c07bdd0 2024-02-16T19:19:30.710 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:29.945+0000 7f412a7fc700 1 -- 172.21.15.22:0/867767192 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f411c006030 con 0x7f412c07bdd0 2024-02-16T19:19:30.710 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:29.946+0000 7f4131e8c700 1 -- 172.21.15.22:0/867767192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f412c07bdd0 msgr2=0x7f412c07a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:19:30.711 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:29.946+0000 7f4131e8c700 1 --2- 172.21.15.22:0/867767192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f412c07bdd0 0x7f412c07a2d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f411c009870 tx=0x7f411c009b80 comp rx=0 tx=0).stop 2024-02-16T19:19:30.711 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:29.946+0000 7f4131e8c700 1 -- 172.21.15.22:0/867767192 shutdown_connections 2024-02-16T19:19:30.711 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:29.946+0000 7f4131e8c700 1 --2- 172.21.15.22:0/867767192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f412c07bdd0 0x7f412c07a2d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:30.711 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:29.946+0000 7f4131e8c700 1 -- 172.21.15.22:0/867767192 >> 172.21.15.22:0/867767192 conn(0x7f412c101020 msgr2=0x7f412c103450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:19:30.711 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:29.946+0000 7f4131e8c700 1 -- 172.21.15.22:0/867767192 shutdown_connections 2024-02-16T19:19:30.711 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:29.946+0000 7f4131e8c700 1 -- 172.21.15.22:0/867767192 wait complete. 2024-02-16T19:19:30.711 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:29.946+0000 7f4131e8c700 1 Processor -- start 2024-02-16T19:19:30.711 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:29.946+0000 7f4131e8c700 1 -- start start 2024-02-16T19:19:30.711 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:29.947+0000 7f4131e8c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f412c07bdd0 0x7f412c19b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:19:30.711 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:29.947+0000 7f4131e8c700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f412c19bdb0 con 0x7f412c07bdd0 2024-02-16T19:19:30.711 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:29.947+0000 7f412b7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f412c07bdd0 0x7f412c19b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:19:30.711 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:29.947+0000 7f412b7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f412c07bdd0 0x7f412c19b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58776/0 (socket says 172.21.15.22:58776) 2024-02-16T19:19:30.711 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:29.947+0000 7f412b7fe700 1 -- 172.21.15.22:0/2507847139 learned_addr learned my addr 172.21.15.22:0/2507847139 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:19:30.712 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:29.947+0000 7f412b7fe700 1 -- 172.21.15.22:0/2507847139 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f411c009500 con 0x7f412c07bdd0 2024-02-16T19:19:30.712 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:29.947+0000 7f412b7fe700 1 --2- 172.21.15.22:0/2507847139 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f412c07bdd0 0x7f412c19b870 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f411c006130 tx=0x7f411c006160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:19:30.712 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:29.947+0000 7f4128ff9700 1 -- 172.21.15.22:0/2507847139 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f411c0043b0 con 0x7f412c07bdd0 2024-02-16T19:19:30.712 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:29.947+0000 7f4128ff9700 1 -- 172.21.15.22:0/2507847139 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f411c004510 con 0x7f412c07bdd0 2024-02-16T19:19:30.712 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:29.947+0000 7f4131e8c700 1 -- 172.21.15.22:0/2507847139 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f412c19bfb0 con 0x7f412c07bdd0 2024-02-16T19:19:30.712 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:29.947+0000 7f4128ff9700 1 -- 172.21.15.22:0/2507847139 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f411c011c80 con 0x7f412c07bdd0 2024-02-16T19:19:30.712 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:29.947+0000 7f4131e8c700 1 -- 172.21.15.22:0/2507847139 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f412c19c450 con 0x7f412c07bdd0 2024-02-16T19:19:30.712 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:29.948+0000 7f4131e8c700 1 -- 172.21.15.22:0/2507847139 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f412c10bca0 con 0x7f412c07bdd0 2024-02-16T19:19:30.712 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:29.948+0000 7f4128ff9700 1 -- 172.21.15.22:0/2507847139 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 4) v1 ==== 45073+0+0 (secure 0 0 0) 0x7f411c018070 con 0x7f412c07bdd0 2024-02-16T19:19:30.712 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:29.949+0000 7f4128ff9700 1 --2- 172.21.15.22:0/2507847139 >> [v2:172.21.15.22:6800/3109015836,v1:172.21.15.22:6801/3109015836] conn(0x7f4114038360 0x7f411403a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:19:30.712 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:29.949+0000 7f4128ff9700 1 -- 172.21.15.22:0/2507847139 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f411c04b9b0 con 0x7f412c07bdd0 2024-02-16T19:19:30.712 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:29.949+0000 7f412affd700 1 --2- 172.21.15.22:0/2507847139 >> [v2:172.21.15.22:6800/3109015836,v1:172.21.15.22:6801/3109015836] conn(0x7f4114038360 0x7f411403a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:19:30.712 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:29.949+0000 7f412affd700 1 --2- 172.21.15.22:0/2507847139 >> [v2:172.21.15.22:6800/3109015836,v1:172.21.15.22:6801/3109015836] conn(0x7f4114038360 0x7f411403a810 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4120006fd0 tx=0x7f4120006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:19:30.713 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:29.953+0000 7f4128ff9700 1 -- 172.21.15.22:0/2507847139 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f411c018500 con 0x7f412c07bdd0 2024-02-16T19:19:30.713 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:30.153+0000 7f4131e8c700 1 -- 172.21.15.22:0/2507847139 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f412c10be90 con 0x7f412c07bdd0 2024-02-16T19:19:30.713 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:30.155+0000 7f4128ff9700 1 -- 172.21.15.22:0/2507847139 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f411c018320 con 0x7f412c07bdd0 2024-02-16T19:19:30.713 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:30.157+0000 7f4131e8c700 1 -- 172.21.15.22:0/2507847139 >> [v2:172.21.15.22:6800/3109015836,v1:172.21.15.22:6801/3109015836] conn(0x7f4114038360 msgr2=0x7f411403a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:19:30.713 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:30.157+0000 7f4131e8c700 1 --2- 172.21.15.22:0/2507847139 >> [v2:172.21.15.22:6800/3109015836,v1:172.21.15.22:6801/3109015836] conn(0x7f4114038360 0x7f411403a810 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4120006fd0 tx=0x7f4120006e40 comp rx=0 tx=0).stop 2024-02-16T19:19:30.713 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:30.157+0000 7f4131e8c700 1 -- 172.21.15.22:0/2507847139 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f412c07bdd0 msgr2=0x7f412c19b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:19:30.713 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:30.157+0000 7f4131e8c700 1 --2- 172.21.15.22:0/2507847139 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f412c07bdd0 0x7f412c19b870 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f411c006130 tx=0x7f411c006160 comp rx=0 tx=0).stop 2024-02-16T19:19:30.713 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:30.157+0000 7f4131e8c700 1 -- 172.21.15.22:0/2507847139 shutdown_connections 2024-02-16T19:19:30.713 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:30.157+0000 7f4131e8c700 1 --2- 172.21.15.22:0/2507847139 >> [v2:172.21.15.22:6800/3109015836,v1:172.21.15.22:6801/3109015836] conn(0x7f4114038360 0x7f411403a810 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:30.713 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:30.157+0000 7f4131e8c700 1 --2- 172.21.15.22:0/2507847139 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f412c07bdd0 0x7f412c19b870 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:30.713 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:30.157+0000 7f4131e8c700 1 -- 172.21.15.22:0/2507847139 >> 172.21.15.22:0/2507847139 conn(0x7f412c101020 msgr2=0x7f412c101dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:19:30.713 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:30.157+0000 7f4131e8c700 1 -- 172.21.15.22:0/2507847139 shutdown_connections 2024-02-16T19:19:30.713 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:30.157+0000 7f4131e8c700 1 -- 172.21.15.22:0/2507847139 wait complete. 2024-02-16T19:19:30.713 INFO:teuthology.orchestra.run.smithi022.stdout:Enabling cephadm module... 2024-02-16T19:19:31.043 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:30 smithi022 ceph-mon[29221]: from='client.? 172.21.15.22:0/2507847139' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-02-16T19:19:32.063 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:31 smithi022 ceph-mon[29221]: from='client.? 172.21.15.22:0/3761515316' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-02-16T19:19:32.355 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:31.149+0000 7effa1485700 1 Processor -- start 2024-02-16T19:19:32.355 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:31.149+0000 7effa1485700 1 -- start start 2024-02-16T19:19:32.355 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:31.149+0000 7effa1485700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff9c1064d0 0x7eff9c1088b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:19:32.355 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:31.149+0000 7effa1485700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff9c079d50 con 0x7eff9c1064d0 2024-02-16T19:19:32.355 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:31.150+0000 7eff9affd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff9c1064d0 0x7eff9c1088b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:19:32.355 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:31.150+0000 7eff9affd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff9c1064d0 0x7eff9c1088b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58780/0 (socket says 172.21.15.22:58780) 2024-02-16T19:19:32.355 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:31.150+0000 7eff9affd700 1 -- 172.21.15.22:0/1655897679 learned_addr learned my addr 172.21.15.22:0/1655897679 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:19:32.355 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:31.150+0000 7eff9affd700 1 -- 172.21.15.22:0/1655897679 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff9c079e90 con 0x7eff9c1064d0 2024-02-16T19:19:32.355 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:31.150+0000 7eff9affd700 1 --2- 172.21.15.22:0/1655897679 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff9c1064d0 0x7eff9c1088b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7eff94009e80 tx=0x7eff9400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f3ee88e5930d874 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:19:32.356 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:31.150+0000 7eff99ffb700 1 -- 172.21.15.22:0/1655897679 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7eff9400e070 con 0x7eff9c1064d0 2024-02-16T19:19:32.356 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:31.150+0000 7eff99ffb700 1 -- 172.21.15.22:0/1655897679 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7eff94003e90 con 0x7eff9c1064d0 2024-02-16T19:19:32.356 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:31.150+0000 7eff99ffb700 1 -- 172.21.15.22:0/1655897679 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7eff94004380 con 0x7eff9c1064d0 2024-02-16T19:19:32.356 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:31.151+0000 7effa1485700 1 -- 172.21.15.22:0/1655897679 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff9c1064d0 msgr2=0x7eff9c1088b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:19:32.356 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:31.151+0000 7effa1485700 1 --2- 172.21.15.22:0/1655897679 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff9c1064d0 0x7eff9c1088b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7eff94009e80 tx=0x7eff9400b330 comp rx=0 tx=0).stop 2024-02-16T19:19:32.356 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:31.151+0000 7effa1485700 1 -- 172.21.15.22:0/1655897679 shutdown_connections 2024-02-16T19:19:32.356 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:31.151+0000 7effa1485700 1 --2- 172.21.15.22:0/1655897679 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff9c1064d0 0x7eff9c1088b0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:32.356 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:31.151+0000 7effa1485700 1 -- 172.21.15.22:0/1655897679 >> 172.21.15.22:0/1655897679 conn(0x7eff9c0ffed0 msgr2=0x7eff9c102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:19:32.356 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:31.151+0000 7effa1485700 1 -- 172.21.15.22:0/1655897679 shutdown_connections 2024-02-16T19:19:32.356 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:31.151+0000 7effa1485700 1 -- 172.21.15.22:0/1655897679 wait complete. 2024-02-16T19:19:32.356 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:31.152+0000 7effa1485700 1 Processor -- start 2024-02-16T19:19:32.356 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:31.152+0000 7effa1485700 1 -- start start 2024-02-16T19:19:32.356 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:31.152+0000 7effa1485700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff9c1064d0 0x7eff9c105e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:19:32.357 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:31.152+0000 7effa1485700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff9c106370 con 0x7eff9c1064d0 2024-02-16T19:19:32.357 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:31.152+0000 7eff9affd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff9c1064d0 0x7eff9c105e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:19:32.357 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:31.152+0000 7eff9affd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff9c1064d0 0x7eff9c105e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58782/0 (socket says 172.21.15.22:58782) 2024-02-16T19:19:32.357 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:31.152+0000 7eff9affd700 1 -- 172.21.15.22:0/3761515316 learned_addr learned my addr 172.21.15.22:0/3761515316 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:19:32.357 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:31.152+0000 7eff9affd700 1 -- 172.21.15.22:0/3761515316 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff94009b30 con 0x7eff9c1064d0 2024-02-16T19:19:32.357 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:31.152+0000 7eff9affd700 1 --2- 172.21.15.22:0/3761515316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff9c1064d0 0x7eff9c105e30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7eff9400d010 tx=0x7eff94004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:19:32.357 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:31.153+0000 7eff93fff700 1 -- 172.21.15.22:0/3761515316 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7eff9400e040 con 0x7eff9c1064d0 2024-02-16T19:19:32.357 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:31.153+0000 7eff93fff700 1 -- 172.21.15.22:0/3761515316 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7eff94003c30 con 0x7eff9c1064d0 2024-02-16T19:19:32.357 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:31.153+0000 7effa1485700 1 -- 172.21.15.22:0/3761515316 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff9c104480 con 0x7eff9c1064d0 2024-02-16T19:19:32.357 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:31.153+0000 7eff93fff700 1 -- 172.21.15.22:0/3761515316 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7eff9401b790 con 0x7eff9c1064d0 2024-02-16T19:19:32.357 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:31.153+0000 7effa1485700 1 -- 172.21.15.22:0/3761515316 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff9c104980 con 0x7eff9c1064d0 2024-02-16T19:19:32.357 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:31.153+0000 7eff93fff700 1 -- 172.21.15.22:0/3761515316 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 4) v1 ==== 45073+0+0 (secure 0 0 0) 0x7eff94019040 con 0x7eff9c1064d0 2024-02-16T19:19:32.357 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:31.154+0000 7eff93fff700 1 --2- 172.21.15.22:0/3761515316 >> [v2:172.21.15.22:6800/3109015836,v1:172.21.15.22:6801/3109015836] conn(0x7eff880383b0 0x7eff8803a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:19:32.357 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:31.154+0000 7effa1485700 1 -- 172.21.15.22:0/3761515316 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff9c109a40 con 0x7eff9c1064d0 2024-02-16T19:19:32.358 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:31.154+0000 7eff93fff700 1 -- 172.21.15.22:0/3761515316 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7eff9404bd30 con 0x7eff9c1064d0 2024-02-16T19:19:32.358 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:31.154+0000 7eff9a7fc700 1 --2- 172.21.15.22:0/3761515316 >> [v2:172.21.15.22:6800/3109015836,v1:172.21.15.22:6801/3109015836] conn(0x7eff880383b0 0x7eff8803a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:19:32.358 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:31.154+0000 7eff9a7fc700 1 --2- 172.21.15.22:0/3761515316 >> [v2:172.21.15.22:6800/3109015836,v1:172.21.15.22:6801/3109015836] conn(0x7eff880383b0 0x7eff8803a860 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7eff84006fd0 tx=0x7eff84006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:19:32.358 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:31.159+0000 7eff93fff700 1 -- 172.21.15.22:0/3761515316 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7eff94011980 con 0x7eff9c1064d0 2024-02-16T19:19:32.358 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:31.423+0000 7effa1485700 1 -- 172.21.15.22:0/3761515316 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7eff9c109c60 con 0x7eff9c1064d0 2024-02-16T19:19:32.358 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:31.758+0000 7eff93fff700 1 -- 172.21.15.22:0/3761515316 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mgrmap(e 5) v1 ==== 45084+0+0 (secure 0 0 0) 0x7eff9401a390 con 0x7eff9c1064d0 2024-02-16T19:19:32.358 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:31.760+0000 7eff93fff700 1 -- 172.21.15.22:0/3761515316 <== mon.0 v2:172.21.15.22:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7eff9404b750 con 0x7eff9c1064d0 2024-02-16T19:19:32.358 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:31.763+0000 7effa1485700 1 -- 172.21.15.22:0/3761515316 >> [v2:172.21.15.22:6800/3109015836,v1:172.21.15.22:6801/3109015836] conn(0x7eff880383b0 msgr2=0x7eff8803a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:19:32.358 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:31.763+0000 7effa1485700 1 --2- 172.21.15.22:0/3761515316 >> [v2:172.21.15.22:6800/3109015836,v1:172.21.15.22:6801/3109015836] conn(0x7eff880383b0 0x7eff8803a860 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7eff84006fd0 tx=0x7eff84006e40 comp rx=0 tx=0).stop 2024-02-16T19:19:32.358 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:31.763+0000 7effa1485700 1 -- 172.21.15.22:0/3761515316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff9c1064d0 msgr2=0x7eff9c105e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:19:32.358 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:31.763+0000 7effa1485700 1 --2- 172.21.15.22:0/3761515316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff9c1064d0 0x7eff9c105e30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7eff9400d010 tx=0x7eff94004430 comp rx=0 tx=0).stop 2024-02-16T19:19:32.358 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:31.763+0000 7effa1485700 1 -- 172.21.15.22:0/3761515316 shutdown_connections 2024-02-16T19:19:32.358 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:31.763+0000 7effa1485700 1 --2- 172.21.15.22:0/3761515316 >> [v2:172.21.15.22:6800/3109015836,v1:172.21.15.22:6801/3109015836] conn(0x7eff880383b0 0x7eff8803a860 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:32.358 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:31.763+0000 7effa1485700 1 --2- 172.21.15.22:0/3761515316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff9c1064d0 0x7eff9c105e30 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:32.359 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:31.763+0000 7effa1485700 1 -- 172.21.15.22:0/3761515316 >> 172.21.15.22:0/3761515316 conn(0x7eff9c0ffed0 msgr2=0x7eff9c100d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:19:32.359 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:31.763+0000 7effa1485700 1 -- 172.21.15.22:0/3761515316 shutdown_connections 2024-02-16T19:19:32.359 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:31.763+0000 7effa1485700 1 -- 172.21.15.22:0/3761515316 wait complete. 2024-02-16T19:19:32.964 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:32 smithi022 ceph-mon[29221]: from='client.? 172.21.15.22:0/3761515316' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-02-16T19:19:32.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:32 smithi022 ceph-mon[29221]: mgrmap e5: smithi022.gmdcch(active, since 6s) 2024-02-16T19:19:33.662 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout { 2024-02-16T19:19:33.663 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-02-16T19:19:33.663 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-16T19:19:33.663 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "active_name": "smithi022.gmdcch", 2024-02-16T19:19:33.663 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-02-16T19:19:33.663 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout } 2024-02-16T19:19:33.663 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:32.832+0000 7f3e35941700 1 Processor -- start 2024-02-16T19:19:33.663 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:32.832+0000 7f3e35941700 1 -- start start 2024-02-16T19:19:33.663 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:32.832+0000 7f3e35941700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e301048e0 0x7f3e30104cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:19:33.663 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:32.832+0000 7f3e35941700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e30105230 con 0x7f3e301048e0 2024-02-16T19:19:33.663 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:32.832+0000 7f3e2effd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e301048e0 0x7f3e30104cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:19:33.663 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:32.832+0000 7f3e2effd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e301048e0 0x7f3e30104cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58812/0 (socket says 172.21.15.22:58812) 2024-02-16T19:19:33.663 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:32.832+0000 7f3e2effd700 1 -- 172.21.15.22:0/3222844070 learned_addr learned my addr 172.21.15.22:0/3222844070 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:19:33.663 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:32.833+0000 7f3e2effd700 1 -- 172.21.15.22:0/3222844070 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e30105370 con 0x7f3e301048e0 2024-02-16T19:19:33.664 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:32.833+0000 7f3e2effd700 1 --2- 172.21.15.22:0/3222844070 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e301048e0 0x7f3e30104cf0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3e240098a0 tx=0x7f3e24009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18edde70ed967ae3 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:19:33.664 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:32.833+0000 7f3e2dffb700 1 -- 172.21.15.22:0/3222844070 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3e24004180 con 0x7f3e301048e0 2024-02-16T19:19:33.664 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:32.833+0000 7f3e2dffb700 1 -- 172.21.15.22:0/3222844070 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3e240042e0 con 0x7f3e301048e0 2024-02-16T19:19:33.664 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:32.833+0000 7f3e2dffb700 1 -- 172.21.15.22:0/3222844070 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3e24011a50 con 0x7f3e301048e0 2024-02-16T19:19:33.664 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:32.834+0000 7f3e35941700 1 -- 172.21.15.22:0/3222844070 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e301048e0 msgr2=0x7f3e30104cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:19:33.664 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:32.834+0000 7f3e35941700 1 --2- 172.21.15.22:0/3222844070 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e301048e0 0x7f3e30104cf0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3e240098a0 tx=0x7f3e24009bb0 comp rx=0 tx=0).stop 2024-02-16T19:19:33.664 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:32.834+0000 7f3e35941700 1 -- 172.21.15.22:0/3222844070 shutdown_connections 2024-02-16T19:19:33.664 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:32.834+0000 7f3e35941700 1 --2- 172.21.15.22:0/3222844070 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e301048e0 0x7f3e30104cf0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:33.664 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:32.834+0000 7f3e35941700 1 -- 172.21.15.22:0/3222844070 >> 172.21.15.22:0/3222844070 conn(0x7f3e300fff10 msgr2=0x7f3e30102360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:19:33.664 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:32.834+0000 7f3e35941700 1 -- 172.21.15.22:0/3222844070 shutdown_connections 2024-02-16T19:19:33.664 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:32.834+0000 7f3e35941700 1 -- 172.21.15.22:0/3222844070 wait complete. 2024-02-16T19:19:33.664 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:32.834+0000 7f3e35941700 1 Processor -- start 2024-02-16T19:19:33.664 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:32.835+0000 7f3e35941700 1 -- start start 2024-02-16T19:19:33.665 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:32.835+0000 7f3e35941700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e301048e0 0x7f3e30199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:19:33.665 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:32.835+0000 7f3e35941700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e30199c80 con 0x7f3e301048e0 2024-02-16T19:19:33.665 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:32.835+0000 7f3e2effd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e301048e0 0x7f3e30199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:19:33.665 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:32.835+0000 7f3e2effd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e301048e0 0x7f3e30199740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58822/0 (socket says 172.21.15.22:58822) 2024-02-16T19:19:33.665 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:32.835+0000 7f3e2effd700 1 -- 172.21.15.22:0/4024987657 learned_addr learned my addr 172.21.15.22:0/4024987657 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:19:33.665 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:32.835+0000 7f3e2effd700 1 -- 172.21.15.22:0/4024987657 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e24009550 con 0x7f3e301048e0 2024-02-16T19:19:33.665 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:32.835+0000 7f3e2effd700 1 --2- 172.21.15.22:0/4024987657 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e301048e0 0x7f3e30199740 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3e24003f80 tx=0x7f3e24011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:19:33.665 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:32.836+0000 7f3e3493f700 1 -- 172.21.15.22:0/4024987657 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3e24003a80 con 0x7f3e301048e0 2024-02-16T19:19:33.665 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:32.836+0000 7f3e3493f700 1 -- 172.21.15.22:0/4024987657 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3e24003be0 con 0x7f3e301048e0 2024-02-16T19:19:33.665 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:32.836+0000 7f3e35941700 1 -- 172.21.15.22:0/4024987657 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e30199e80 con 0x7f3e301048e0 2024-02-16T19:19:33.665 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:32.836+0000 7f3e3493f700 1 -- 172.21.15.22:0/4024987657 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3e24019820 con 0x7f3e301048e0 2024-02-16T19:19:33.665 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:32.836+0000 7f3e35941700 1 -- 172.21.15.22:0/4024987657 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e3019a320 con 0x7f3e301048e0 2024-02-16T19:19:33.665 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:32.837+0000 7f3e3493f700 1 -- 172.21.15.22:0/4024987657 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f3e24019aa0 con 0x7f3e301048e0 2024-02-16T19:19:33.666 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:32.837+0000 7f3e35941700 1 -- 172.21.15.22:0/4024987657 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e30062380 con 0x7f3e301048e0 2024-02-16T19:19:33.666 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:32.837+0000 7f3e3493f700 1 --2- 172.21.15.22:0/4024987657 >> [v2:172.21.15.22:6800/3109015836,v1:172.21.15.22:6801/3109015836] conn(0x7f3e180383a0 0x7f3e1803a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:19:33.666 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:32.837+0000 7f3e3493f700 1 -- 172.21.15.22:0/4024987657 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f3e2404c7b0 con 0x7f3e301048e0 2024-02-16T19:19:33.666 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:32.837+0000 7f3e2e7fc700 1 -- 172.21.15.22:0/4024987657 >> [v2:172.21.15.22:6800/3109015836,v1:172.21.15.22:6801/3109015836] conn(0x7f3e180383a0 msgr2=0x7f3e1803a850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/3109015836 2024-02-16T19:19:33.666 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:32.837+0000 7f3e2e7fc700 1 --2- 172.21.15.22:0/4024987657 >> [v2:172.21.15.22:6800/3109015836,v1:172.21.15.22:6801/3109015836] conn(0x7f3e180383a0 0x7f3e1803a850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-16T19:19:33.666 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:32.842+0000 7f3e3493f700 1 -- 172.21.15.22:0/4024987657 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f3e2402b750 con 0x7f3e301048e0 2024-02-16T19:19:33.666 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:33.038+0000 7f3e2e7fc700 1 -- 172.21.15.22:0/4024987657 >> [v2:172.21.15.22:6800/3109015836,v1:172.21.15.22:6801/3109015836] conn(0x7f3e180383a0 msgr2=0x7f3e1803a850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/3109015836 2024-02-16T19:19:33.666 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:33.038+0000 7f3e2e7fc700 1 --2- 172.21.15.22:0/4024987657 >> [v2:172.21.15.22:6800/3109015836,v1:172.21.15.22:6801/3109015836] conn(0x7f3e180383a0 0x7f3e1803a850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-16T19:19:33.666 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:33.097+0000 7f3e35941700 1 -- 172.21.15.22:0/4024987657 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f3e3019d4b0 con 0x7f3e301048e0 2024-02-16T19:19:33.666 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:33.097+0000 7f3e3493f700 1 -- 172.21.15.22:0/4024987657 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+103 (secure 0 0 0) 0x7f3e24023390 con 0x7f3e301048e0 2024-02-16T19:19:33.666 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:33.099+0000 7f3e35941700 1 -- 172.21.15.22:0/4024987657 >> [v2:172.21.15.22:6800/3109015836,v1:172.21.15.22:6801/3109015836] conn(0x7f3e180383a0 msgr2=0x7f3e1803a850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-16T19:19:33.666 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:33.099+0000 7f3e35941700 1 --2- 172.21.15.22:0/4024987657 >> [v2:172.21.15.22:6800/3109015836,v1:172.21.15.22:6801/3109015836] conn(0x7f3e180383a0 0x7f3e1803a850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:33.666 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:33.099+0000 7f3e35941700 1 -- 172.21.15.22:0/4024987657 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e301048e0 msgr2=0x7f3e30199740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:19:33.666 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:33.099+0000 7f3e35941700 1 --2- 172.21.15.22:0/4024987657 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e301048e0 0x7f3e30199740 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3e24003f80 tx=0x7f3e24011ab0 comp rx=0 tx=0).stop 2024-02-16T19:19:33.667 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:33.099+0000 7f3e35941700 1 -- 172.21.15.22:0/4024987657 shutdown_connections 2024-02-16T19:19:33.667 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:33.099+0000 7f3e35941700 1 --2- 172.21.15.22:0/4024987657 >> [v2:172.21.15.22:6800/3109015836,v1:172.21.15.22:6801/3109015836] conn(0x7f3e180383a0 0x7f3e1803a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:33.667 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:33.099+0000 7f3e35941700 1 --2- 172.21.15.22:0/4024987657 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e301048e0 0x7f3e30199740 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:33.667 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:33.099+0000 7f3e35941700 1 -- 172.21.15.22:0/4024987657 >> 172.21.15.22:0/4024987657 conn(0x7f3e300fff10 msgr2=0x7f3e30074070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:19:33.667 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:33.100+0000 7f3e35941700 1 -- 172.21.15.22:0/4024987657 shutdown_connections 2024-02-16T19:19:33.667 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:33.100+0000 7f3e35941700 1 -- 172.21.15.22:0/4024987657 wait complete. 2024-02-16T19:19:33.667 INFO:teuthology.orchestra.run.smithi022.stdout:Waiting for the mgr to restart... 2024-02-16T19:19:33.667 INFO:teuthology.orchestra.run.smithi022.stdout:Waiting for mgr epoch 5... 2024-02-16T19:19:33.907 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:33 smithi022 ceph-mon[29221]: from='client.? 172.21.15.22:0/4024987657' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-16T19:19:38.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:38 smithi022 ceph-mon[29221]: Active manager daemon smithi022.gmdcch restarted 2024-02-16T19:19:38.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:38 smithi022 ceph-mon[29221]: Activating manager daemon smithi022.gmdcch 2024-02-16T19:19:38.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:38 smithi022 ceph-mon[29221]: osdmap e2: 0 total, 0 up, 0 in 2024-02-16T19:19:38.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:38 smithi022 ceph-mon[29221]: mgrmap e6: smithi022.gmdcch(active, starting, since 0.00347979s) 2024-02-16T19:19:38.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:38 smithi022 ceph-mon[29221]: from='mgr.14120 172.21.15.22:0/1425529966' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mon metadata", "id": "smithi022"}]: dispatch 2024-02-16T19:19:38.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:38 smithi022 ceph-mon[29221]: from='mgr.14120 172.21.15.22:0/1425529966' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mgr metadata", "who": "smithi022.gmdcch", "id": "smithi022.gmdcch"}]: dispatch 2024-02-16T19:19:38.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:38 smithi022 ceph-mon[29221]: from='mgr.14120 172.21.15.22:0/1425529966' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-16T19:19:38.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:38 smithi022 ceph-mon[29221]: from='mgr.14120 172.21.15.22:0/1425529966' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-16T19:19:38.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:38 smithi022 ceph-mon[29221]: from='mgr.14120 172.21.15.22:0/1425529966' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-16T19:19:38.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:38 smithi022 ceph-mon[29221]: Manager daemon smithi022.gmdcch is now available 2024-02-16T19:19:38.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:38 smithi022 ceph-mon[29221]: from='mgr.14120 172.21.15.22:0/1425529966' entity='mgr.smithi022.gmdcch' 2024-02-16T19:19:38.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:38 smithi022 ceph-mon[29221]: from='mgr.14120 172.21.15.22:0/1425529966' entity='mgr.smithi022.gmdcch' 2024-02-16T19:19:38.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:38 smithi022 ceph-mon[29221]: from='mgr.14120 172.21.15.22:0/1425529966' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:19:38.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:38 smithi022 ceph-mon[29221]: from='mgr.14120 172.21.15.22:0/1425529966' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:19:38.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:38 smithi022 ceph-mon[29221]: from='mgr.14120 172.21.15.22:0/1425529966' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:19:39.859 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:39 smithi022 ceph-mon[29221]: Found migration_current of "None". Setting to last migration. 2024-02-16T19:19:39.859 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:39 smithi022 ceph-mon[29221]: from='mgr.14120 172.21.15.22:0/1425529966' entity='mgr.smithi022.gmdcch' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi022.gmdcch/mirror_snapshot_schedule"}]: dispatch 2024-02-16T19:19:39.859 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:39 smithi022 ceph-mon[29221]: from='mgr.14120 172.21.15.22:0/1425529966' entity='mgr.smithi022.gmdcch' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi022.gmdcch/trash_purge_schedule"}]: dispatch 2024-02-16T19:19:39.859 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:39 smithi022 ceph-mon[29221]: mgrmap e7: smithi022.gmdcch(active, since 1.00543s) 2024-02-16T19:19:40.169 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout { 2024-02-16T19:19:40.170 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-02-16T19:19:40.170 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "initialized": true 2024-02-16T19:19:40.170 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout } 2024-02-16T19:19:40.170 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:34.106+0000 7fb71e91f700 1 Processor -- start 2024-02-16T19:19:40.170 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:34.106+0000 7fb71e91f700 1 -- start start 2024-02-16T19:19:40.170 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:34.106+0000 7fb71e91f700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb71807be10 0x7fb71807a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:19:40.170 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:34.106+0000 7fb71e91f700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb71807a850 con 0x7fb71807be10 2024-02-16T19:19:40.170 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:34.107+0000 7fb717fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb71807be10 0x7fb71807a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:19:40.170 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:34.107+0000 7fb717fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb71807be10 0x7fb71807a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58838/0 (socket says 172.21.15.22:58838) 2024-02-16T19:19:40.170 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:34.107+0000 7fb717fff700 1 -- 172.21.15.22:0/1457939585 learned_addr learned my addr 172.21.15.22:0/1457939585 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:19:40.170 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:34.107+0000 7fb717fff700 1 -- 172.21.15.22:0/1457939585 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb71807a990 con 0x7fb71807be10 2024-02-16T19:19:40.170 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:34.107+0000 7fb717fff700 1 --2- 172.21.15.22:0/1457939585 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb71807be10 0x7fb71807a310 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb7080098a0 tx=0x7fb708009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5447c98798e1f4b7 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:19:40.170 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:34.107+0000 7fb716ffd700 1 -- 172.21.15.22:0/1457939585 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb708004180 con 0x7fb71807be10 2024-02-16T19:19:40.171 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:34.107+0000 7fb716ffd700 1 -- 172.21.15.22:0/1457939585 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb7080042e0 con 0x7fb71807be10 2024-02-16T19:19:40.171 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:34.107+0000 7fb716ffd700 1 -- 172.21.15.22:0/1457939585 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb708011a50 con 0x7fb71807be10 2024-02-16T19:19:40.171 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:34.108+0000 7fb71e91f700 1 -- 172.21.15.22:0/1457939585 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb71807be10 msgr2=0x7fb71807a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:19:40.171 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:34.108+0000 7fb71e91f700 1 --2- 172.21.15.22:0/1457939585 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb71807be10 0x7fb71807a310 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb7080098a0 tx=0x7fb708009bb0 comp rx=0 tx=0).stop 2024-02-16T19:19:40.171 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:34.108+0000 7fb71e91f700 1 -- 172.21.15.22:0/1457939585 shutdown_connections 2024-02-16T19:19:40.171 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:34.108+0000 7fb71e91f700 1 --2- 172.21.15.22:0/1457939585 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb71807be10 0x7fb71807a310 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:40.171 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:34.108+0000 7fb71e91f700 1 -- 172.21.15.22:0/1457939585 >> 172.21.15.22:0/1457939585 conn(0x7fb718101020 msgr2=0x7fb718103430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:19:40.171 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:34.108+0000 7fb71e91f700 1 -- 172.21.15.22:0/1457939585 shutdown_connections 2024-02-16T19:19:40.171 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:34.108+0000 7fb71e91f700 1 -- 172.21.15.22:0/1457939585 wait complete. 2024-02-16T19:19:40.171 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:34.109+0000 7fb71e91f700 1 Processor -- start 2024-02-16T19:19:40.171 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:34.109+0000 7fb71e91f700 1 -- start start 2024-02-16T19:19:40.171 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:34.109+0000 7fb71e91f700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb71807be10 0x7fb71819b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:19:40.171 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:34.109+0000 7fb71e91f700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb71819be90 con 0x7fb71807be10 2024-02-16T19:19:40.172 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:34.109+0000 7fb717fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb71807be10 0x7fb71819b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:19:40.172 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:34.109+0000 7fb717fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb71807be10 0x7fb71819b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58852/0 (socket says 172.21.15.22:58852) 2024-02-16T19:19:40.172 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:34.109+0000 7fb717fff700 1 -- 172.21.15.22:0/2858700393 learned_addr learned my addr 172.21.15.22:0/2858700393 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:19:40.172 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:34.109+0000 7fb717fff700 1 -- 172.21.15.22:0/2858700393 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb708009550 con 0x7fb71807be10 2024-02-16T19:19:40.172 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:34.110+0000 7fb717fff700 1 --2- 172.21.15.22:0/2858700393 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb71807be10 0x7fb71819b950 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb708000c00 tx=0x7fb708011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:19:40.172 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:34.110+0000 7fb7157fa700 1 -- 172.21.15.22:0/2858700393 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb708003b90 con 0x7fb71807be10 2024-02-16T19:19:40.172 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:34.110+0000 7fb7157fa700 1 -- 172.21.15.22:0/2858700393 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb7080248b0 con 0x7fb71807be10 2024-02-16T19:19:40.172 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:34.110+0000 7fb71e91f700 1 -- 172.21.15.22:0/2858700393 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb71819c090 con 0x7fb71807be10 2024-02-16T19:19:40.172 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:34.111+0000 7fb7157fa700 1 -- 172.21.15.22:0/2858700393 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb70801a8e0 con 0x7fb71807be10 2024-02-16T19:19:40.172 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:34.111+0000 7fb71e91f700 1 -- 172.21.15.22:0/2858700393 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb71819c530 con 0x7fb71807be10 2024-02-16T19:19:40.172 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:34.111+0000 7fb7157fa700 1 -- 172.21.15.22:0/2858700393 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 45084+0+0 (secure 0 0 0) 0x7fb70801ab50 con 0x7fb71807be10 2024-02-16T19:19:40.172 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:34.111+0000 7fb7157fa700 1 --2- 172.21.15.22:0/2858700393 >> [v2:172.21.15.22:6800/3109015836,v1:172.21.15.22:6801/3109015836] conn(0x7fb704038440 0x7fb70403a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:19:40.172 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:34.111+0000 7fb7157fa700 1 -- 172.21.15.22:0/2858700393 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb708024440 con 0x7fb71807be10 2024-02-16T19:19:40.173 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:34.111+0000 7fb7177fe700 1 -- 172.21.15.22:0/2858700393 >> [v2:172.21.15.22:6800/3109015836,v1:172.21.15.22:6801/3109015836] conn(0x7fb704038440 msgr2=0x7fb70403a8f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/3109015836 2024-02-16T19:19:40.173 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:34.111+0000 7fb7177fe700 1 --2- 172.21.15.22:0/2858700393 >> [v2:172.21.15.22:6800/3109015836,v1:172.21.15.22:6801/3109015836] conn(0x7fb704038440 0x7fb70403a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-16T19:19:40.173 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:34.111+0000 7fb71e91f700 1 -- 172.21.15.22:0/2858700393 --> [v2:172.21.15.22:6800/3109015836,v1:172.21.15.22:6801/3109015836] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb71810bc70 con 0x7fb704038440 2024-02-16T19:19:40.173 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:34.312+0000 7fb7177fe700 1 -- 172.21.15.22:0/2858700393 >> [v2:172.21.15.22:6800/3109015836,v1:172.21.15.22:6801/3109015836] conn(0x7fb704038440 msgr2=0x7fb70403a8f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/3109015836 2024-02-16T19:19:40.173 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:34.312+0000 7fb7177fe700 1 --2- 172.21.15.22:0/2858700393 >> [v2:172.21.15.22:6800/3109015836,v1:172.21.15.22:6801/3109015836] conn(0x7fb704038440 0x7fb70403a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-16T19:19:40.173 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:34.713+0000 7fb7177fe700 1 -- 172.21.15.22:0/2858700393 >> [v2:172.21.15.22:6800/3109015836,v1:172.21.15.22:6801/3109015836] conn(0x7fb704038440 msgr2=0x7fb70403a8f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/3109015836 2024-02-16T19:19:40.173 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:34.713+0000 7fb7177fe700 1 --2- 172.21.15.22:0/2858700393 >> [v2:172.21.15.22:6800/3109015836,v1:172.21.15.22:6801/3109015836] conn(0x7fb704038440 0x7fb70403a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-02-16T19:19:40.173 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:35.514+0000 7fb7177fe700 1 -- 172.21.15.22:0/2858700393 >> [v2:172.21.15.22:6800/3109015836,v1:172.21.15.22:6801/3109015836] conn(0x7fb704038440 msgr2=0x7fb70403a8f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/3109015836 2024-02-16T19:19:40.173 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:35.514+0000 7fb7177fe700 1 --2- 172.21.15.22:0/2858700393 >> [v2:172.21.15.22:6800/3109015836,v1:172.21.15.22:6801/3109015836] conn(0x7fb704038440 0x7fb70403a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-02-16T19:19:40.173 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:37.115+0000 7fb7177fe700 1 -- 172.21.15.22:0/2858700393 >> [v2:172.21.15.22:6800/3109015836,v1:172.21.15.22:6801/3109015836] conn(0x7fb704038440 msgr2=0x7fb70403a8f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/3109015836 2024-02-16T19:19:40.173 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:37.115+0000 7fb7177fe700 1 --2- 172.21.15.22:0/2858700393 >> [v2:172.21.15.22:6800/3109015836,v1:172.21.15.22:6801/3109015836] conn(0x7fb704038440 0x7fb70403a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-02-16T19:19:40.173 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:38.545+0000 7fb7157fa700 1 -- 172.21.15.22:0/2858700393 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mgrmap(e 6) v1 ==== 44851+0+0 (secure 0 0 0) 0x7fb708024600 con 0x7fb71807be10 2024-02-16T19:19:40.174 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:38.545+0000 7fb7157fa700 1 -- 172.21.15.22:0/2858700393 >> [v2:172.21.15.22:6800/3109015836,v1:172.21.15.22:6801/3109015836] conn(0x7fb704038440 msgr2=0x7fb70403a8f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-16T19:19:40.174 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:38.545+0000 7fb7157fa700 1 --2- 172.21.15.22:0/2858700393 >> [v2:172.21.15.22:6800/3109015836,v1:172.21.15.22:6801/3109015836] conn(0x7fb704038440 0x7fb70403a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:40.174 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:39.546+0000 7fb7157fa700 1 -- 172.21.15.22:0/2858700393 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mgrmap(e 7) v1 ==== 44978+0+0 (secure 0 0 0) 0x7fb708040e10 con 0x7fb71807be10 2024-02-16T19:19:40.174 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:39.546+0000 7fb7157fa700 1 --2- 172.21.15.22:0/2858700393 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7fb70403bb60 0x7fb70403df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:19:40.174 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:39.547+0000 7fb7157fa700 1 -- 172.21.15.22:0/2858700393 --> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb71810bc70 con 0x7fb70403bb60 2024-02-16T19:19:40.174 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:39.547+0000 7fb7177fe700 1 --2- 172.21.15.22:0/2858700393 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7fb70403bb60 0x7fb70403df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:19:40.174 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:39.547+0000 7fb7177fe700 1 --2- 172.21.15.22:0/2858700393 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7fb70403bb60 0x7fb70403df40 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb700003a10 tx=0x7fb7000092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:19:40.174 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:39.548+0000 7fb7157fa700 1 -- 172.21.15.22:0/2858700393 <== mgr.14120 v2:172.21.15.22:6800/2755289428 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7fb71810bc70 con 0x7fb70403bb60 2024-02-16T19:19:40.174 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:39.554+0000 7fb71e91f700 1 -- 172.21.15.22:0/2858700393 --> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fb71819cc90 con 0x7fb70403bb60 2024-02-16T19:19:40.174 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:39.554+0000 7fb7157fa700 1 -- 172.21.15.22:0/2858700393 <== mgr.14120 v2:172.21.15.22:6800/2755289428 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fb71819cc90 con 0x7fb70403bb60 2024-02-16T19:19:40.174 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:39.555+0000 7fb71e91f700 1 -- 172.21.15.22:0/2858700393 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7fb70403bb60 msgr2=0x7fb70403df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:19:40.174 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:39.555+0000 7fb71e91f700 1 --2- 172.21.15.22:0/2858700393 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7fb70403bb60 0x7fb70403df40 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb700003a10 tx=0x7fb7000092b0 comp rx=0 tx=0).stop 2024-02-16T19:19:40.174 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:39.555+0000 7fb71e91f700 1 -- 172.21.15.22:0/2858700393 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb71807be10 msgr2=0x7fb71819b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:19:40.174 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:39.555+0000 7fb71e91f700 1 --2- 172.21.15.22:0/2858700393 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb71807be10 0x7fb71819b950 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb708000c00 tx=0x7fb708011ab0 comp rx=0 tx=0).stop 2024-02-16T19:19:40.175 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:39.555+0000 7fb71e91f700 1 -- 172.21.15.22:0/2858700393 shutdown_connections 2024-02-16T19:19:40.175 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:39.555+0000 7fb71e91f700 1 --2- 172.21.15.22:0/2858700393 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7fb70403bb60 0x7fb70403df40 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:40.175 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:39.555+0000 7fb71e91f700 1 --2- 172.21.15.22:0/2858700393 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb71807be10 0x7fb71819b950 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:40.175 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:39.555+0000 7fb71e91f700 1 --2- 172.21.15.22:0/2858700393 >> [v2:172.21.15.22:6800/3109015836,v1:172.21.15.22:6801/3109015836] conn(0x7fb704038440 0x7fb70403a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:40.175 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:39.555+0000 7fb71e91f700 1 -- 172.21.15.22:0/2858700393 >> 172.21.15.22:0/2858700393 conn(0x7fb718101020 msgr2=0x7fb718102690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:19:40.175 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:39.555+0000 7fb71e91f700 1 -- 172.21.15.22:0/2858700393 shutdown_connections 2024-02-16T19:19:40.175 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:39.555+0000 7fb71e91f700 1 -- 172.21.15.22:0/2858700393 wait complete. 2024-02-16T19:19:40.175 INFO:teuthology.orchestra.run.smithi022.stdout:mgr epoch 5 is available 2024-02-16T19:19:40.175 INFO:teuthology.orchestra.run.smithi022.stdout:Setting orchestrator backend to cephadm... 2024-02-16T19:19:40.793 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:40 smithi022 ceph-mon[29221]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-02-16T19:19:40.794 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:40 smithi022 ceph-mon[29221]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-02-16T19:19:40.794 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:40 smithi022 ceph-mon[29221]: from='mgr.14120 172.21.15.22:0/1425529966' entity='mgr.smithi022.gmdcch' 2024-02-16T19:19:40.794 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:40 smithi022 ceph-mon[29221]: from='mgr.14120 172.21.15.22:0/1425529966' entity='mgr.smithi022.gmdcch' 2024-02-16T19:19:41.581 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:40.571+0000 7f4028089700 1 Processor -- start 2024-02-16T19:19:41.581 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:40.571+0000 7f4028089700 1 -- start start 2024-02-16T19:19:41.581 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:40.571+0000 7f4028089700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4020107000 0x7f4020107410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:19:41.581 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:40.571+0000 7f4028089700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4020107950 con 0x7f4020107000 2024-02-16T19:19:41.581 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:40.571+0000 7f4025e25700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4020107000 0x7f4020107410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:19:41.581 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:40.571+0000 7f4025e25700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4020107000 0x7f4020107410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53552/0 (socket says 172.21.15.22:53552) 2024-02-16T19:19:41.581 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:40.571+0000 7f4025e25700 1 -- 172.21.15.22:0/4187617730 learned_addr learned my addr 172.21.15.22:0/4187617730 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:19:41.581 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:40.571+0000 7f4025e25700 1 -- 172.21.15.22:0/4187617730 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4020107a90 con 0x7f4020107000 2024-02-16T19:19:41.582 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:40.572+0000 7f4025e25700 1 --2- 172.21.15.22:0/4187617730 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4020107000 0x7f4020107410 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f401c0098a0 tx=0x7f401c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37d8a7960262eff1 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:19:41.582 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:40.572+0000 7f4024e23700 1 -- 172.21.15.22:0/4187617730 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f401c004180 con 0x7f4020107000 2024-02-16T19:19:41.582 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:40.572+0000 7f4024e23700 1 -- 172.21.15.22:0/4187617730 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f401c0042e0 con 0x7f4020107000 2024-02-16T19:19:41.582 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:40.572+0000 7f4024e23700 1 -- 172.21.15.22:0/4187617730 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f401c011a50 con 0x7f4020107000 2024-02-16T19:19:41.582 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:40.572+0000 7f4028089700 1 -- 172.21.15.22:0/4187617730 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4020107000 msgr2=0x7f4020107410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:19:41.582 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:40.572+0000 7f4028089700 1 --2- 172.21.15.22:0/4187617730 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4020107000 0x7f4020107410 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f401c0098a0 tx=0x7f401c009bb0 comp rx=0 tx=0).stop 2024-02-16T19:19:41.582 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:40.572+0000 7f4028089700 1 -- 172.21.15.22:0/4187617730 shutdown_connections 2024-02-16T19:19:41.582 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:40.572+0000 7f4028089700 1 --2- 172.21.15.22:0/4187617730 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4020107000 0x7f4020107410 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:41.582 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:40.572+0000 7f4028089700 1 -- 172.21.15.22:0/4187617730 >> 172.21.15.22:0/4187617730 conn(0x7f40200ff670 msgr2=0x7f4020101ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:19:41.582 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:40.572+0000 7f4028089700 1 -- 172.21.15.22:0/4187617730 shutdown_connections 2024-02-16T19:19:41.582 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:40.572+0000 7f4028089700 1 -- 172.21.15.22:0/4187617730 wait complete. 2024-02-16T19:19:41.582 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:40.573+0000 7f4028089700 1 Processor -- start 2024-02-16T19:19:41.582 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:40.573+0000 7f4028089700 1 -- start start 2024-02-16T19:19:41.582 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:40.573+0000 7f4028089700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4020107000 0x7f4020199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:19:41.582 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:40.573+0000 7f4028089700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4020199c80 con 0x7f4020107000 2024-02-16T19:19:41.583 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:40.573+0000 7f4025e25700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4020107000 0x7f4020199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:19:41.583 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:40.573+0000 7f4025e25700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4020107000 0x7f4020199740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53556/0 (socket says 172.21.15.22:53556) 2024-02-16T19:19:41.583 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:40.573+0000 7f4025e25700 1 -- 172.21.15.22:0/131943722 learned_addr learned my addr 172.21.15.22:0/131943722 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:19:41.583 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:40.573+0000 7f4025e25700 1 -- 172.21.15.22:0/131943722 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f401c009550 con 0x7f4020107000 2024-02-16T19:19:41.583 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:40.574+0000 7f4025e25700 1 --2- 172.21.15.22:0/131943722 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4020107000 0x7f4020199740 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f401c003de0 tx=0x7f401c011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:19:41.583 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:40.574+0000 7f4012ffd700 1 -- 172.21.15.22:0/131943722 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f401c003930 con 0x7f4020107000 2024-02-16T19:19:41.583 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:40.574+0000 7f4012ffd700 1 -- 172.21.15.22:0/131943722 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f401c003a90 con 0x7f4020107000 2024-02-16T19:19:41.583 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:40.574+0000 7f4028089700 1 -- 172.21.15.22:0/131943722 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4020199e80 con 0x7f4020107000 2024-02-16T19:19:41.583 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:40.574+0000 7f4012ffd700 1 -- 172.21.15.22:0/131943722 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f401c019730 con 0x7f4020107000 2024-02-16T19:19:41.583 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:40.574+0000 7f4028089700 1 -- 172.21.15.22:0/131943722 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f402019a320 con 0x7f4020107000 2024-02-16T19:19:41.583 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:40.575+0000 7f4012ffd700 1 -- 172.21.15.22:0/131943722 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 7) v1 ==== 44978+0+0 (secure 0 0 0) 0x7f401c019a30 con 0x7f4020107000 2024-02-16T19:19:41.583 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:40.575+0000 7f4012ffd700 1 --2- 172.21.15.22:0/131943722 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7f400c0382d0 0x7f400c03a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:19:41.583 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:40.575+0000 7f4028089700 1 -- 172.21.15.22:0/131943722 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4020104520 con 0x7f4020107000 2024-02-16T19:19:41.583 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:40.575+0000 7f4012ffd700 1 -- 172.21.15.22:0/131943722 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f401c04c2f0 con 0x7f4020107000 2024-02-16T19:19:41.584 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:40.575+0000 7f4025624700 1 --2- 172.21.15.22:0/131943722 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7f400c0382d0 0x7f400c03a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:19:41.584 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:40.575+0000 7f4025624700 1 --2- 172.21.15.22:0/131943722 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7f400c0382d0 0x7f400c03a780 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4014006fd0 tx=0x7f4014006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:19:41.584 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:40.580+0000 7f4012ffd700 1 -- 172.21.15.22:0/131943722 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f401c02c3a0 con 0x7f4020107000 2024-02-16T19:19:41.584 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:40.792+0000 7f4028089700 1 -- 172.21.15.22:0/131943722 --> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f4020101970 con 0x7f400c0382d0 2024-02-16T19:19:41.584 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:40.798+0000 7f4012ffd700 1 -- 172.21.15.22:0/131943722 <== mgr.14120 v2:172.21.15.22:6800/2755289428 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f4020101970 con 0x7f400c0382d0 2024-02-16T19:19:41.584 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:40.800+0000 7f4028089700 1 -- 172.21.15.22:0/131943722 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7f400c0382d0 msgr2=0x7f400c03a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:19:41.584 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:40.800+0000 7f4028089700 1 --2- 172.21.15.22:0/131943722 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7f400c0382d0 0x7f400c03a780 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4014006fd0 tx=0x7f4014006e40 comp rx=0 tx=0).stop 2024-02-16T19:19:41.584 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:40.800+0000 7f4028089700 1 -- 172.21.15.22:0/131943722 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4020107000 msgr2=0x7f4020199740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:19:41.584 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:40.800+0000 7f4028089700 1 --2- 172.21.15.22:0/131943722 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4020107000 0x7f4020199740 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f401c003de0 tx=0x7f401c011ab0 comp rx=0 tx=0).stop 2024-02-16T19:19:41.584 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:40.801+0000 7f4028089700 1 -- 172.21.15.22:0/131943722 shutdown_connections 2024-02-16T19:19:41.584 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:40.801+0000 7f4028089700 1 --2- 172.21.15.22:0/131943722 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7f400c0382d0 0x7f400c03a780 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:41.584 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:40.801+0000 7f4028089700 1 --2- 172.21.15.22:0/131943722 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4020107000 0x7f4020199740 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:41.584 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:40.801+0000 7f4028089700 1 -- 172.21.15.22:0/131943722 >> 172.21.15.22:0/131943722 conn(0x7f40200ff670 msgr2=0x7f4020100d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:19:41.584 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:40.801+0000 7f4028089700 1 -- 172.21.15.22:0/131943722 shutdown_connections 2024-02-16T19:19:41.585 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:40.801+0000 7f4028089700 1 -- 172.21.15.22:0/131943722 wait complete. 2024-02-16T19:19:41.885 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:41 smithi022 ceph-mon[29221]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:19:41.885 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:41 smithi022 ceph-mon[29221]: from='mgr.14120 172.21.15.22:0/1425529966' entity='mgr.smithi022.gmdcch' 2024-02-16T19:19:41.885 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:41 smithi022 ceph-mon[29221]: from='mgr.14120 172.21.15.22:0/1425529966' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:19:41.885 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:41 smithi022 ceph-mon[29221]: mgrmap e8: smithi022.gmdcch(active, since 2s) 2024-02-16T19:19:41.885 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:41 smithi022 ceph-mon[29221]: from='mgr.14120 172.21.15.22:0/1425529966' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:19:42.781 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout value unchanged 2024-02-16T19:19:42.781 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:41.983+0000 7f548cc46700 1 Processor -- start 2024-02-16T19:19:42.781 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:41.983+0000 7f548cc46700 1 -- start start 2024-02-16T19:19:42.781 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:41.984+0000 7f548cc46700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5488106fe0 0x7f54881073f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:19:42.781 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:41.984+0000 7f548cc46700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5488107930 con 0x7f5488106fe0 2024-02-16T19:19:42.781 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:41.984+0000 7f548659c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5488106fe0 0x7f54881073f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:19:42.782 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:41.984+0000 7f548659c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5488106fe0 0x7f54881073f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53568/0 (socket says 172.21.15.22:53568) 2024-02-16T19:19:42.782 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:41.984+0000 7f548659c700 1 -- 172.21.15.22:0/2902143997 learned_addr learned my addr 172.21.15.22:0/2902143997 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:19:42.782 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:41.984+0000 7f548659c700 1 -- 172.21.15.22:0/2902143997 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5488107a70 con 0x7f5488106fe0 2024-02-16T19:19:42.782 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:41.984+0000 7f548659c700 1 --2- 172.21.15.22:0/2902143997 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5488106fe0 0x7f54881073f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f54780098a0 tx=0x7f5478009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2526bc440f89e110 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:19:42.782 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:41.985+0000 7f548559a700 1 -- 172.21.15.22:0/2902143997 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5478004180 con 0x7f5488106fe0 2024-02-16T19:19:42.782 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:41.985+0000 7f548559a700 1 -- 172.21.15.22:0/2902143997 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f54780042e0 con 0x7f5488106fe0 2024-02-16T19:19:42.782 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:41.985+0000 7f548559a700 1 -- 172.21.15.22:0/2902143997 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5478011a50 con 0x7f5488106fe0 2024-02-16T19:19:42.782 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:41.985+0000 7f548cc46700 1 -- 172.21.15.22:0/2902143997 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5488106fe0 msgr2=0x7f54881073f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:19:42.782 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:41.985+0000 7f548cc46700 1 --2- 172.21.15.22:0/2902143997 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5488106fe0 0x7f54881073f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f54780098a0 tx=0x7f5478009bb0 comp rx=0 tx=0).stop 2024-02-16T19:19:42.782 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:41.985+0000 7f548cc46700 1 -- 172.21.15.22:0/2902143997 shutdown_connections 2024-02-16T19:19:42.782 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:41.985+0000 7f548cc46700 1 --2- 172.21.15.22:0/2902143997 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5488106fe0 0x7f54881073f0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:42.783 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:41.985+0000 7f548cc46700 1 -- 172.21.15.22:0/2902143997 >> 172.21.15.22:0/2902143997 conn(0x7f54880ff670 msgr2=0x7f5488101ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:19:42.783 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:41.985+0000 7f548cc46700 1 -- 172.21.15.22:0/2902143997 shutdown_connections 2024-02-16T19:19:42.783 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:41.985+0000 7f548cc46700 1 -- 172.21.15.22:0/2902143997 wait complete. 2024-02-16T19:19:42.783 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:41.986+0000 7f548cc46700 1 Processor -- start 2024-02-16T19:19:42.783 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:41.986+0000 7f548cc46700 1 -- start start 2024-02-16T19:19:42.783 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:41.986+0000 7f548cc46700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5488106fe0 0x7f54881066e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:19:42.783 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:41.986+0000 7f548cc46700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5488106c20 con 0x7f5488106fe0 2024-02-16T19:19:42.783 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:41.986+0000 7f548659c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5488106fe0 0x7f54881066e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:19:42.783 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:41.986+0000 7f548659c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5488106fe0 0x7f54881066e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53574/0 (socket says 172.21.15.22:53574) 2024-02-16T19:19:42.783 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:41.986+0000 7f548659c700 1 -- 172.21.15.22:0/2954797083 learned_addr learned my addr 172.21.15.22:0/2954797083 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:19:42.783 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:41.987+0000 7f548659c700 1 -- 172.21.15.22:0/2954797083 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5478009550 con 0x7f5488106fe0 2024-02-16T19:19:42.783 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:41.987+0000 7f548659c700 1 --2- 172.21.15.22:0/2954797083 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5488106fe0 0x7f54881066e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5478003f80 tx=0x7f5478011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:19:42.784 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:41.987+0000 7f54777fe700 1 -- 172.21.15.22:0/2954797083 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5478003a80 con 0x7f5488106fe0 2024-02-16T19:19:42.784 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:41.987+0000 7f548cc46700 1 -- 172.21.15.22:0/2954797083 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5488104d30 con 0x7f5488106fe0 2024-02-16T19:19:42.784 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:41.987+0000 7f54777fe700 1 -- 172.21.15.22:0/2954797083 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5478003be0 con 0x7f5488106fe0 2024-02-16T19:19:42.784 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:41.987+0000 7f54777fe700 1 -- 172.21.15.22:0/2954797083 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5478019820 con 0x7f5488106fe0 2024-02-16T19:19:42.784 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:41.987+0000 7f548cc46700 1 -- 172.21.15.22:0/2954797083 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54881051d0 con 0x7f5488106fe0 2024-02-16T19:19:42.784 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:41.988+0000 7f54777fe700 1 -- 172.21.15.22:0/2954797083 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f5478019aa0 con 0x7f5488106fe0 2024-02-16T19:19:42.784 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:41.988+0000 7f548cc46700 1 -- 172.21.15.22:0/2954797083 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54880623f0 con 0x7f5488106fe0 2024-02-16T19:19:42.784 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:41.988+0000 7f54777fe700 1 --2- 172.21.15.22:0/2954797083 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7f54700383a0 0x7f547003a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:19:42.784 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:41.988+0000 7f54777fe700 1 -- 172.21.15.22:0/2954797083 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f547804c5a0 con 0x7f5488106fe0 2024-02-16T19:19:42.784 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:41.988+0000 7f5485d9b700 1 --2- 172.21.15.22:0/2954797083 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7f54700383a0 0x7f547003a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:19:42.785 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:41.988+0000 7f5485d9b700 1 --2- 172.21.15.22:0/2954797083 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7f54700383a0 0x7f547003a850 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f547c006fd0 tx=0x7f547c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:19:42.785 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:41.993+0000 7f54777fe700 1 -- 172.21.15.22:0/2954797083 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f54880623f0 con 0x7f5488106fe0 2024-02-16T19:19:42.785 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:42.192+0000 7f548cc46700 1 -- 172.21.15.22:0/2954797083 --> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f5488062380 con 0x7f54700383a0 2024-02-16T19:19:42.785 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:42.193+0000 7f54777fe700 1 -- 172.21.15.22:0/2954797083 <== mgr.14120 v2:172.21.15.22:6800/2755289428 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f5488062380 con 0x7f54700383a0 2024-02-16T19:19:42.785 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:42.195+0000 7f548cc46700 1 -- 172.21.15.22:0/2954797083 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7f54700383a0 msgr2=0x7f547003a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:19:42.785 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:42.195+0000 7f548cc46700 1 --2- 172.21.15.22:0/2954797083 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7f54700383a0 0x7f547003a850 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f547c006fd0 tx=0x7f547c006e40 comp rx=0 tx=0).stop 2024-02-16T19:19:42.785 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:42.195+0000 7f548cc46700 1 -- 172.21.15.22:0/2954797083 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5488106fe0 msgr2=0x7f54881066e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:19:42.785 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:42.195+0000 7f548cc46700 1 --2- 172.21.15.22:0/2954797083 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5488106fe0 0x7f54881066e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5478003f80 tx=0x7f5478011ab0 comp rx=0 tx=0).stop 2024-02-16T19:19:42.785 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:42.195+0000 7f548cc46700 1 -- 172.21.15.22:0/2954797083 shutdown_connections 2024-02-16T19:19:42.785 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:42.195+0000 7f548cc46700 1 --2- 172.21.15.22:0/2954797083 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7f54700383a0 0x7f547003a850 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:42.785 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:42.195+0000 7f548cc46700 1 --2- 172.21.15.22:0/2954797083 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5488106fe0 0x7f54881066e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:42.785 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:42.195+0000 7f548cc46700 1 -- 172.21.15.22:0/2954797083 >> 172.21.15.22:0/2954797083 conn(0x7f54880ff670 msgr2=0x7f5488100d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:19:42.786 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:42.195+0000 7f548cc46700 1 -- 172.21.15.22:0/2954797083 shutdown_connections 2024-02-16T19:19:42.786 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:42.195+0000 7f548cc46700 1 -- 172.21.15.22:0/2954797083 wait complete. 2024-02-16T19:19:42.786 INFO:teuthology.orchestra.run.smithi022.stdout:Generating ssh key... 2024-02-16T19:19:43.099 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:42 smithi022 ceph-mon[29221]: [16/Feb/2024:19:19:41] ENGINE Bus STARTING 2024-02-16T19:19:43.099 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:42 smithi022 ceph-mon[29221]: [16/Feb/2024:19:19:41] ENGINE Serving on https://172.21.15.22:7150 2024-02-16T19:19:43.099 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:42 smithi022 ceph-mon[29221]: [16/Feb/2024:19:19:41] ENGINE Serving on http://172.21.15.22:8765 2024-02-16T19:19:43.099 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:42 smithi022 ceph-mon[29221]: [16/Feb/2024:19:19:41] ENGINE Bus STARTED 2024-02-16T19:19:44.094 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:43 smithi022 ceph-mon[29221]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:19:44.094 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:43 smithi022 ceph-mon[29221]: from='mgr.14120 172.21.15.22:0/1425529966' entity='mgr.smithi022.gmdcch' 2024-02-16T19:19:44.094 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:43 smithi022 ceph-mon[29221]: from='mgr.14120 172.21.15.22:0/1425529966' entity='mgr.smithi022.gmdcch' 2024-02-16T19:19:44.352 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:43.207+0000 7f3265a45700 1 Processor -- start 2024-02-16T19:19:44.352 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:43.207+0000 7f3265a45700 1 -- start start 2024-02-16T19:19:44.352 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:43.207+0000 7f3265a45700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32601059d0 0x7f3260105de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:19:44.352 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:43.207+0000 7f3265a45700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3260106320 con 0x7f32601059d0 2024-02-16T19:19:44.352 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:43.207+0000 7f325effd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32601059d0 0x7f3260105de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:19:44.353 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:43.207+0000 7f325effd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32601059d0 0x7f3260105de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53586/0 (socket says 172.21.15.22:53586) 2024-02-16T19:19:44.353 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:43.207+0000 7f325effd700 1 -- 172.21.15.22:0/2330795657 learned_addr learned my addr 172.21.15.22:0/2330795657 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:19:44.353 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:43.208+0000 7f325effd700 1 -- 172.21.15.22:0/2330795657 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3260106460 con 0x7f32601059d0 2024-02-16T19:19:44.353 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:43.208+0000 7f325effd700 1 --2- 172.21.15.22:0/2330795657 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32601059d0 0x7f3260105de0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3250009ed0 tx=0x7f325000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57883e80055ef162 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:19:44.353 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:43.208+0000 7f325dffb700 1 -- 172.21.15.22:0/2330795657 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f325000e070 con 0x7f32601059d0 2024-02-16T19:19:44.353 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:43.208+0000 7f325dffb700 1 -- 172.21.15.22:0/2330795657 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3250003e90 con 0x7f32601059d0 2024-02-16T19:19:44.353 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:43.208+0000 7f325dffb700 1 -- 172.21.15.22:0/2330795657 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3250004480 con 0x7f32601059d0 2024-02-16T19:19:44.353 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:43.209+0000 7f3265a45700 1 -- 172.21.15.22:0/2330795657 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32601059d0 msgr2=0x7f3260105de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:19:44.353 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:43.209+0000 7f3265a45700 1 --2- 172.21.15.22:0/2330795657 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32601059d0 0x7f3260105de0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3250009ed0 tx=0x7f325000b330 comp rx=0 tx=0).stop 2024-02-16T19:19:44.353 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:43.209+0000 7f3265a45700 1 -- 172.21.15.22:0/2330795657 shutdown_connections 2024-02-16T19:19:44.353 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:43.209+0000 7f3265a45700 1 --2- 172.21.15.22:0/2330795657 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32601059d0 0x7f3260105de0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:44.353 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:43.209+0000 7f3265a45700 1 -- 172.21.15.22:0/2330795657 >> 172.21.15.22:0/2330795657 conn(0x7f3260101020 msgr2=0x7f3260103450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:19:44.353 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:43.209+0000 7f3265a45700 1 -- 172.21.15.22:0/2330795657 shutdown_connections 2024-02-16T19:19:44.353 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:43.209+0000 7f3265a45700 1 -- 172.21.15.22:0/2330795657 wait complete. 2024-02-16T19:19:44.353 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:43.209+0000 7f3265a45700 1 Processor -- start 2024-02-16T19:19:44.354 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:43.210+0000 7f3265a45700 1 -- start start 2024-02-16T19:19:44.354 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:43.210+0000 7f3265a45700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32601059d0 0x7f326007bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:19:44.354 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:43.210+0000 7f3265a45700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f326007c160 con 0x7f32601059d0 2024-02-16T19:19:44.354 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:43.210+0000 7f325effd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32601059d0 0x7f326007bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:19:44.354 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:43.210+0000 7f325effd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32601059d0 0x7f326007bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53590/0 (socket says 172.21.15.22:53590) 2024-02-16T19:19:44.354 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:43.210+0000 7f325effd700 1 -- 172.21.15.22:0/3865029566 learned_addr learned my addr 172.21.15.22:0/3865029566 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:19:44.354 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:43.210+0000 7f325effd700 1 -- 172.21.15.22:0/3865029566 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3250009b30 con 0x7f32601059d0 2024-02-16T19:19:44.354 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:43.210+0000 7f325effd700 1 --2- 172.21.15.22:0/3865029566 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32601059d0 0x7f326007bc20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f325000bd30 tx=0x7f3250004550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:19:44.354 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:43.210+0000 7f3264a43700 1 -- 172.21.15.22:0/3865029566 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f325000e040 con 0x7f32601059d0 2024-02-16T19:19:44.354 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:43.210+0000 7f3264a43700 1 -- 172.21.15.22:0/3865029566 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f325001b8a0 con 0x7f32601059d0 2024-02-16T19:19:44.354 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:43.210+0000 7f3265a45700 1 -- 172.21.15.22:0/3865029566 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f326007a270 con 0x7f32601059d0 2024-02-16T19:19:44.354 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:43.211+0000 7f3264a43700 1 -- 172.21.15.22:0/3865029566 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3250012970 con 0x7f32601059d0 2024-02-16T19:19:44.354 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:43.211+0000 7f3265a45700 1 -- 172.21.15.22:0/3865029566 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f326007a710 con 0x7f32601059d0 2024-02-16T19:19:44.354 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:43.211+0000 7f3264a43700 1 -- 172.21.15.22:0/3865029566 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f3250019040 con 0x7f32601059d0 2024-02-16T19:19:44.355 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:43.212+0000 7f3265a45700 1 -- 172.21.15.22:0/3865029566 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3260062380 con 0x7f32601059d0 2024-02-16T19:19:44.355 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:43.212+0000 7f3264a43700 1 --2- 172.21.15.22:0/3865029566 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7f324c038440 0x7f324c03a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:19:44.355 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:43.212+0000 7f3264a43700 1 -- 172.21.15.22:0/3865029566 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f325004be20 con 0x7f32601059d0 2024-02-16T19:19:44.355 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:43.212+0000 7f325e7fc700 1 --2- 172.21.15.22:0/3865029566 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7f324c038440 0x7f324c03a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:19:44.355 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:43.212+0000 7f325e7fc700 1 --2- 172.21.15.22:0/3865029566 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7f324c038440 0x7f324c03a8f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3248006fd0 tx=0x7f3248006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:19:44.355 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:43.217+0000 7f3264a43700 1 -- 172.21.15.22:0/3865029566 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f325001ab40 con 0x7f32601059d0 2024-02-16T19:19:44.355 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:43.419+0000 7f3265a45700 1 -- 172.21.15.22:0/3865029566 --> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f3260102440 con 0x7f324c038440 2024-02-16T19:19:44.355 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:43.794+0000 7f3264a43700 1 -- 172.21.15.22:0/3865029566 <== mgr.14120 v2:172.21.15.22:6800/2755289428 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f3260102440 con 0x7f324c038440 2024-02-16T19:19:44.355 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:43.797+0000 7f3265a45700 1 -- 172.21.15.22:0/3865029566 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7f324c038440 msgr2=0x7f324c03a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:19:44.355 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:43.797+0000 7f3265a45700 1 --2- 172.21.15.22:0/3865029566 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7f324c038440 0x7f324c03a8f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3248006fd0 tx=0x7f3248006e40 comp rx=0 tx=0).stop 2024-02-16T19:19:44.355 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:43.797+0000 7f3265a45700 1 -- 172.21.15.22:0/3865029566 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32601059d0 msgr2=0x7f326007bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:19:44.355 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:43.797+0000 7f3265a45700 1 --2- 172.21.15.22:0/3865029566 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32601059d0 0x7f326007bc20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f325000bd30 tx=0x7f3250004550 comp rx=0 tx=0).stop 2024-02-16T19:19:44.355 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:43.797+0000 7f3265a45700 1 -- 172.21.15.22:0/3865029566 shutdown_connections 2024-02-16T19:19:44.355 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:43.797+0000 7f3265a45700 1 --2- 172.21.15.22:0/3865029566 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7f324c038440 0x7f324c03a8f0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:44.356 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:43.797+0000 7f3265a45700 1 --2- 172.21.15.22:0/3865029566 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32601059d0 0x7f326007bc20 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:44.356 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:43.797+0000 7f3265a45700 1 -- 172.21.15.22:0/3865029566 >> 172.21.15.22:0/3865029566 conn(0x7f3260101020 msgr2=0x7f3260101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:19:44.356 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:43.797+0000 7f3265a45700 1 -- 172.21.15.22:0/3865029566 shutdown_connections 2024-02-16T19:19:44.356 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:43.797+0000 7f3265a45700 1 -- 172.21.15.22:0/3865029566 wait complete. 2024-02-16T19:19:44.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:44 smithi022 ceph-mon[29221]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:19:44.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:44 smithi022 ceph-mon[29221]: Generating ssh key... 2024-02-16T19:19:45.550 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDvQbslCAhk7mMXzrGiXKtg1AoNmCl8HuGZevMJKL499hzGfD5iy3wyrLQg+D94evPSH9ToWEupNCcJsUFiEhE/A45TA2uGTD8mtqrEY0Abrm3Wt0x/nmLqq9GBjZ5CMf7LPlp8n8HdyqwTiDXZXD10QULQc27L3LqMdTSCSzlnEdXFjz7XHNytb5XJgi6WuYmTBnnGIWfM9ShTbPfOwmGbCAuDADspetUtlfhsaRe/hjAlPXAkKpkz5w46sMam51fK3fNi2XDGgNNoW8yJj/sYFLWPZxzd0EemrS5d7Q5qpgXlT6u/Nbfgn8ra7QyWT5TmOEN+vLWknQzRD7N+6Q64hRDFa2xhP0D4LXTN8ClNjOClGgfYBST73pKiWDVZvJxsFTXcdqRZKDHKk3E0aEenRtGvtmvQAluzjjFvDePlP7j/GBXj5XDOUH5xglXNIZFkaLuBJWpxXkMshW/Pkd4mUjyUmmhzjSC8bLBUAdvcNc81wvcviQAFLgc1nk387XU= ceph-2a1346e6-cd00-11ee-95ba-87774f69a715 2024-02-16T19:19:45.550 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:44.772+0000 7f0da73a0700 1 Processor -- start 2024-02-16T19:19:45.551 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:44.773+0000 7f0da73a0700 1 -- start start 2024-02-16T19:19:45.551 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:44.773+0000 7f0da73a0700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0da0105800 0x7f0da0105c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:19:45.551 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:44.773+0000 7f0da73a0700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0da0106150 con 0x7f0da0105800 2024-02-16T19:19:45.551 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:44.773+0000 7f0da639e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0da0105800 0x7f0da0105c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:19:45.551 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:44.773+0000 7f0da639e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0da0105800 0x7f0da0105c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59552/0 (socket says 172.21.15.22:59552) 2024-02-16T19:19:45.551 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:44.773+0000 7f0da639e700 1 -- 172.21.15.22:0/967547948 learned_addr learned my addr 172.21.15.22:0/967547948 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:19:45.551 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:44.773+0000 7f0da639e700 1 -- 172.21.15.22:0/967547948 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0da0106290 con 0x7f0da0105800 2024-02-16T19:19:45.551 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:44.774+0000 7f0da639e700 1 --2- 172.21.15.22:0/967547948 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0da0105800 0x7f0da0105c10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c0098a0 tx=0x7f0d9c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d64c80e7182127 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:19:45.551 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:44.774+0000 7f0da539c700 1 -- 172.21.15.22:0/967547948 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0d9c004180 con 0x7f0da0105800 2024-02-16T19:19:45.551 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:44.774+0000 7f0da539c700 1 -- 172.21.15.22:0/967547948 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0d9c0042e0 con 0x7f0da0105800 2024-02-16T19:19:45.551 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:44.774+0000 7f0da539c700 1 -- 172.21.15.22:0/967547948 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0d9c011a50 con 0x7f0da0105800 2024-02-16T19:19:45.551 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:44.774+0000 7f0da73a0700 1 -- 172.21.15.22:0/967547948 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0da0105800 msgr2=0x7f0da0105c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:19:45.552 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:44.774+0000 7f0da73a0700 1 --2- 172.21.15.22:0/967547948 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0da0105800 0x7f0da0105c10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c0098a0 tx=0x7f0d9c009bb0 comp rx=0 tx=0).stop 2024-02-16T19:19:45.552 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:44.774+0000 7f0da73a0700 1 -- 172.21.15.22:0/967547948 shutdown_connections 2024-02-16T19:19:45.552 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:44.774+0000 7f0da73a0700 1 --2- 172.21.15.22:0/967547948 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0da0105800 0x7f0da0105c10 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:45.552 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:44.774+0000 7f0da73a0700 1 -- 172.21.15.22:0/967547948 >> 172.21.15.22:0/967547948 conn(0x7f0da0100e50 msgr2=0x7f0da0103280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:19:45.552 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:44.774+0000 7f0da73a0700 1 -- 172.21.15.22:0/967547948 shutdown_connections 2024-02-16T19:19:45.552 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:44.774+0000 7f0da73a0700 1 -- 172.21.15.22:0/967547948 wait complete. 2024-02-16T19:19:45.552 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:44.775+0000 7f0da73a0700 1 Processor -- start 2024-02-16T19:19:45.552 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:44.775+0000 7f0da73a0700 1 -- start start 2024-02-16T19:19:45.552 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:44.775+0000 7f0da73a0700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0da0105800 0x7f0da01994f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:19:45.552 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:44.775+0000 7f0da73a0700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0da0199a30 con 0x7f0da0105800 2024-02-16T19:19:45.552 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:44.775+0000 7f0da639e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0da0105800 0x7f0da01994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:19:45.552 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:44.775+0000 7f0da639e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0da0105800 0x7f0da01994f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59560/0 (socket says 172.21.15.22:59560) 2024-02-16T19:19:45.552 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:44.775+0000 7f0da639e700 1 -- 172.21.15.22:0/3599235669 learned_addr learned my addr 172.21.15.22:0/3599235669 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:19:45.553 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:44.775+0000 7f0da639e700 1 -- 172.21.15.22:0/3599235669 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d9c009550 con 0x7f0da0105800 2024-02-16T19:19:45.553 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:44.776+0000 7f0da639e700 1 --2- 172.21.15.22:0/3599235669 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0da0105800 0x7f0da01994f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c0119e0 tx=0x7f0d9c011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:19:45.553 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:44.776+0000 7f0d937fe700 1 -- 172.21.15.22:0/3599235669 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0d9c003b90 con 0x7f0da0105800 2024-02-16T19:19:45.553 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:44.776+0000 7f0d937fe700 1 -- 172.21.15.22:0/3599235669 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0d9c01a8a0 con 0x7f0da0105800 2024-02-16T19:19:45.553 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:44.776+0000 7f0d937fe700 1 -- 172.21.15.22:0/3599235669 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0d9c019930 con 0x7f0da0105800 2024-02-16T19:19:45.553 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:44.776+0000 7f0da73a0700 1 -- 172.21.15.22:0/3599235669 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0da0199c30 con 0x7f0da0105800 2024-02-16T19:19:45.553 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:44.776+0000 7f0da73a0700 1 -- 172.21.15.22:0/3599235669 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0da019a0d0 con 0x7f0da0105800 2024-02-16T19:19:45.553 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:44.777+0000 7f0da73a0700 1 -- 172.21.15.22:0/3599235669 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0da0062380 con 0x7f0da0105800 2024-02-16T19:19:45.553 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:44.777+0000 7f0d937fe700 1 -- 172.21.15.22:0/3599235669 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f0d9c01aa10 con 0x7f0da0105800 2024-02-16T19:19:45.553 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:44.777+0000 7f0d937fe700 1 --2- 172.21.15.22:0/3599235669 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7f0d8c0383b0 0x7f0d8c03a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:19:45.553 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:44.777+0000 7f0d937fe700 1 -- 172.21.15.22:0/3599235669 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f0d9c04c6f0 con 0x7f0da0105800 2024-02-16T19:19:45.553 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:44.777+0000 7f0da5b9d700 1 --2- 172.21.15.22:0/3599235669 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7f0d8c0383b0 0x7f0d8c03a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:19:45.553 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:44.778+0000 7f0da5b9d700 1 --2- 172.21.15.22:0/3599235669 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7f0d8c0383b0 0x7f0d8c03a860 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0d94006fd0 tx=0x7f0d94006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:19:45.553 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:44.782+0000 7f0d937fe700 1 -- 172.21.15.22:0/3599235669 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f0d9c018530 con 0x7f0da0105800 2024-02-16T19:19:45.554 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:44.982+0000 7f0da73a0700 1 -- 172.21.15.22:0/3599235669 --> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f0da0102300 con 0x7f0d8c0383b0 2024-02-16T19:19:45.554 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:44.983+0000 7f0d937fe700 1 -- 172.21.15.22:0/3599235669 <== mgr.14120 v2:172.21.15.22:6800/2755289428 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f0da0102300 con 0x7f0d8c0383b0 2024-02-16T19:19:45.554 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:44.985+0000 7f0da73a0700 1 -- 172.21.15.22:0/3599235669 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7f0d8c0383b0 msgr2=0x7f0d8c03a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:19:45.554 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:44.985+0000 7f0da73a0700 1 --2- 172.21.15.22:0/3599235669 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7f0d8c0383b0 0x7f0d8c03a860 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0d94006fd0 tx=0x7f0d94006e40 comp rx=0 tx=0).stop 2024-02-16T19:19:45.554 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:44.985+0000 7f0da73a0700 1 -- 172.21.15.22:0/3599235669 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0da0105800 msgr2=0x7f0da01994f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:19:45.554 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:44.985+0000 7f0da73a0700 1 --2- 172.21.15.22:0/3599235669 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0da0105800 0x7f0da01994f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c0119e0 tx=0x7f0d9c011ac0 comp rx=0 tx=0).stop 2024-02-16T19:19:45.554 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:44.985+0000 7f0da73a0700 1 -- 172.21.15.22:0/3599235669 shutdown_connections 2024-02-16T19:19:45.554 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:44.985+0000 7f0da73a0700 1 --2- 172.21.15.22:0/3599235669 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7f0d8c0383b0 0x7f0d8c03a860 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:45.554 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:44.985+0000 7f0da73a0700 1 --2- 172.21.15.22:0/3599235669 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0da0105800 0x7f0da01994f0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:45.554 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:44.985+0000 7f0da73a0700 1 -- 172.21.15.22:0/3599235669 >> 172.21.15.22:0/3599235669 conn(0x7f0da0100e50 msgr2=0x7f0da0101bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:19:45.554 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:44.985+0000 7f0da73a0700 1 -- 172.21.15.22:0/3599235669 shutdown_connections 2024-02-16T19:19:45.554 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:44.985+0000 7f0da73a0700 1 -- 172.21.15.22:0/3599235669 wait complete. 2024-02-16T19:19:45.554 INFO:teuthology.orchestra.run.smithi022.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-02-16T19:19:45.554 INFO:teuthology.orchestra.run.smithi022.stdout:Adding key to root@localhost authorized_keys... 2024-02-16T19:19:45.554 INFO:teuthology.orchestra.run.smithi022.stdout:Adding host smithi022... 2024-02-16T19:19:46.161 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:45 smithi022 ceph-mon[29221]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:19:48.131 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:47 smithi022 ceph-mon[29221]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi022", "addr": "172.21.15.22", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:19:48.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:48 smithi022 ceph-mon[29221]: Deploying cephadm binary to smithi022 2024-02-16T19:19:48.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:48 smithi022 ceph-mon[29221]: from='mgr.14120 172.21.15.22:0/1425529966' entity='mgr.smithi022.gmdcch' 2024-02-16T19:19:48.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:48 smithi022 ceph-mon[29221]: from='mgr.14120 172.21.15.22:0/1425529966' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:19:49.268 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout Added host 'smithi022' with addr '172.21.15.22' 2024-02-16T19:19:49.268 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:45.948+0000 7fb883f43700 1 Processor -- start 2024-02-16T19:19:49.268 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:45.948+0000 7fb883f43700 1 -- start start 2024-02-16T19:19:49.268 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:45.949+0000 7fb883f43700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb87c1064d0 0x7fb87c1088b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:19:49.268 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:45.949+0000 7fb883f43700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb87c079d50 con 0x7fb87c1064d0 2024-02-16T19:19:49.268 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:45.949+0000 7fb881cdf700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb87c1064d0 0x7fb87c1088b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:19:49.268 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:45.949+0000 7fb881cdf700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb87c1064d0 0x7fb87c1088b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59572/0 (socket says 172.21.15.22:59572) 2024-02-16T19:19:49.268 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:45.949+0000 7fb881cdf700 1 -- 172.21.15.22:0/253144755 learned_addr learned my addr 172.21.15.22:0/253144755 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:19:49.268 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:45.949+0000 7fb881cdf700 1 -- 172.21.15.22:0/253144755 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb87c079e90 con 0x7fb87c1064d0 2024-02-16T19:19:49.268 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:45.949+0000 7fb881cdf700 1 --2- 172.21.15.22:0/253144755 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb87c1064d0 0x7fb87c1088b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb8780098a0 tx=0x7fb878009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8335aac063e5eda1 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:19:49.268 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:45.949+0000 7fb880cdd700 1 -- 172.21.15.22:0/253144755 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb878004180 con 0x7fb87c1064d0 2024-02-16T19:19:49.269 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:45.949+0000 7fb880cdd700 1 -- 172.21.15.22:0/253144755 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb8780042e0 con 0x7fb87c1064d0 2024-02-16T19:19:49.269 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:45.949+0000 7fb880cdd700 1 -- 172.21.15.22:0/253144755 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb878011a50 con 0x7fb87c1064d0 2024-02-16T19:19:49.269 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:45.950+0000 7fb883f43700 1 -- 172.21.15.22:0/253144755 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb87c1064d0 msgr2=0x7fb87c1088b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:19:49.269 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:45.950+0000 7fb883f43700 1 --2- 172.21.15.22:0/253144755 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb87c1064d0 0x7fb87c1088b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb8780098a0 tx=0x7fb878009bb0 comp rx=0 tx=0).stop 2024-02-16T19:19:49.269 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:45.950+0000 7fb883f43700 1 -- 172.21.15.22:0/253144755 shutdown_connections 2024-02-16T19:19:49.269 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:45.950+0000 7fb883f43700 1 --2- 172.21.15.22:0/253144755 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb87c1064d0 0x7fb87c1088b0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:49.269 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:45.950+0000 7fb883f43700 1 -- 172.21.15.22:0/253144755 >> 172.21.15.22:0/253144755 conn(0x7fb87c0ffed0 msgr2=0x7fb87c102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:19:49.269 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:45.950+0000 7fb883f43700 1 -- 172.21.15.22:0/253144755 shutdown_connections 2024-02-16T19:19:49.269 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:45.950+0000 7fb883f43700 1 -- 172.21.15.22:0/253144755 wait complete. 2024-02-16T19:19:49.269 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:45.951+0000 7fb883f43700 1 Processor -- start 2024-02-16T19:19:49.269 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:45.951+0000 7fb883f43700 1 -- start start 2024-02-16T19:19:49.269 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:45.951+0000 7fb883f43700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb87c1064d0 0x7fb87c105e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:19:49.269 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:45.951+0000 7fb883f43700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb87c106370 con 0x7fb87c1064d0 2024-02-16T19:19:49.269 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:45.951+0000 7fb881cdf700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb87c1064d0 0x7fb87c105e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:19:49.269 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:45.951+0000 7fb881cdf700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb87c1064d0 0x7fb87c105e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59574/0 (socket says 172.21.15.22:59574) 2024-02-16T19:19:49.270 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:45.951+0000 7fb881cdf700 1 -- 172.21.15.22:0/3500668534 learned_addr learned my addr 172.21.15.22:0/3500668534 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:19:49.270 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:45.951+0000 7fb881cdf700 1 -- 172.21.15.22:0/3500668534 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb878009550 con 0x7fb87c1064d0 2024-02-16T19:19:49.270 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:45.951+0000 7fb881cdf700 1 --2- 172.21.15.22:0/3500668534 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb87c1064d0 0x7fb87c105e30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb878003de0 tx=0x7fb878011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:19:49.270 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:45.951+0000 7fb872ffd700 1 -- 172.21.15.22:0/3500668534 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb878003930 con 0x7fb87c1064d0 2024-02-16T19:19:49.270 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:45.952+0000 7fb872ffd700 1 -- 172.21.15.22:0/3500668534 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb878003a90 con 0x7fb87c1064d0 2024-02-16T19:19:49.270 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:45.952+0000 7fb883f43700 1 -- 172.21.15.22:0/3500668534 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb87c104480 con 0x7fb87c1064d0 2024-02-16T19:19:49.270 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:45.952+0000 7fb872ffd700 1 -- 172.21.15.22:0/3500668534 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb8780196c0 con 0x7fb87c1064d0 2024-02-16T19:19:49.270 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:45.952+0000 7fb883f43700 1 -- 172.21.15.22:0/3500668534 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb87c104920 con 0x7fb87c1064d0 2024-02-16T19:19:49.270 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:45.952+0000 7fb872ffd700 1 -- 172.21.15.22:0/3500668534 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7fb87801abc0 con 0x7fb87c1064d0 2024-02-16T19:19:49.270 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:45.952+0000 7fb883f43700 1 -- 172.21.15.22:0/3500668534 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb87c109a40 con 0x7fb87c1064d0 2024-02-16T19:19:49.270 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:45.953+0000 7fb872ffd700 1 --2- 172.21.15.22:0/3500668534 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7fb8680383f0 0x7fb86803a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:19:49.270 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:45.953+0000 7fb872ffd700 1 -- 172.21.15.22:0/3500668534 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fb87804c420 con 0x7fb87c1064d0 2024-02-16T19:19:49.270 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:45.953+0000 7fb8814de700 1 --2- 172.21.15.22:0/3500668534 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7fb8680383f0 0x7fb86803a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:19:49.270 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:45.953+0000 7fb8814de700 1 --2- 172.21.15.22:0/3500668534 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7fb8680383f0 0x7fb86803a8a0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb86c006fd0 tx=0x7fb86c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:19:49.271 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:45.957+0000 7fb872ffd700 1 -- 172.21.15.22:0/3500668534 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fb878010770 con 0x7fb87c1064d0 2024-02-16T19:19:49.271 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:46.160+0000 7fb883f43700 1 -- 172.21.15.22:0/3500668534 --> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi022", "addr": "172.21.15.22", "target": ["mon-mgr", ""]}) v1 -- 0x7fb87c1021f0 con 0x7fb8680383f0 2024-02-16T19:19:49.271 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:48.706+0000 7fb872ffd700 1 -- 172.21.15.22:0/3500668534 <== mgr.14120 v2:172.21.15.22:6800/2755289428 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7fb87c1021f0 con 0x7fb8680383f0 2024-02-16T19:19:49.271 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:48.709+0000 7fb883f43700 1 -- 172.21.15.22:0/3500668534 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7fb8680383f0 msgr2=0x7fb86803a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:19:49.271 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:48.709+0000 7fb883f43700 1 --2- 172.21.15.22:0/3500668534 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7fb8680383f0 0x7fb86803a8a0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb86c006fd0 tx=0x7fb86c006e40 comp rx=0 tx=0).stop 2024-02-16T19:19:49.271 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:48.709+0000 7fb883f43700 1 -- 172.21.15.22:0/3500668534 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb87c1064d0 msgr2=0x7fb87c105e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:19:49.271 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:48.709+0000 7fb883f43700 1 --2- 172.21.15.22:0/3500668534 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb87c1064d0 0x7fb87c105e30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb878003de0 tx=0x7fb878011ab0 comp rx=0 tx=0).stop 2024-02-16T19:19:49.271 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:48.709+0000 7fb883f43700 1 -- 172.21.15.22:0/3500668534 shutdown_connections 2024-02-16T19:19:49.271 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:48.709+0000 7fb883f43700 1 --2- 172.21.15.22:0/3500668534 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7fb8680383f0 0x7fb86803a8a0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:49.271 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:48.709+0000 7fb883f43700 1 --2- 172.21.15.22:0/3500668534 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb87c1064d0 0x7fb87c105e30 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:49.271 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:48.709+0000 7fb883f43700 1 -- 172.21.15.22:0/3500668534 >> 172.21.15.22:0/3500668534 conn(0x7fb87c0ffed0 msgr2=0x7fb87c1015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:19:49.271 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:48.709+0000 7fb883f43700 1 -- 172.21.15.22:0/3500668534 shutdown_connections 2024-02-16T19:19:49.271 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:48.709+0000 7fb883f43700 1 -- 172.21.15.22:0/3500668534 wait complete. 2024-02-16T19:19:49.271 INFO:teuthology.orchestra.run.smithi022.stdout:Deploying mon service with default placement... 2024-02-16T19:19:49.948 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:49 smithi022 ceph-mon[29221]: Added host smithi022 2024-02-16T19:19:50.769 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-02-16T19:19:50.769 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:49.870+0000 7f613f333700 1 Processor -- start 2024-02-16T19:19:50.769 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:49.870+0000 7f613f333700 1 -- start start 2024-02-16T19:19:50.770 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:49.870+0000 7f613f333700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f613807bdd0 0x7f613807a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:19:50.770 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:49.870+0000 7f613f333700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f613807a810 con 0x7f613807bdd0 2024-02-16T19:19:50.770 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:49.870+0000 7f613d0cf700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f613807bdd0 0x7f613807a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:19:50.770 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:49.870+0000 7f613d0cf700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f613807bdd0 0x7f613807a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59590/0 (socket says 172.21.15.22:59590) 2024-02-16T19:19:50.770 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:49.870+0000 7f613d0cf700 1 -- 172.21.15.22:0/3069370123 learned_addr learned my addr 172.21.15.22:0/3069370123 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:19:50.770 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:49.871+0000 7f613d0cf700 1 -- 172.21.15.22:0/3069370123 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f613807a950 con 0x7f613807bdd0 2024-02-16T19:19:50.770 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:49.871+0000 7f613d0cf700 1 --2- 172.21.15.22:0/3069370123 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f613807bdd0 0x7f613807a2d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6134009870 tx=0x7f6134009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=752ce460d7d9677a server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:19:50.770 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:49.871+0000 7f612bfff700 1 -- 172.21.15.22:0/3069370123 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6134005c30 con 0x7f613807bdd0 2024-02-16T19:19:50.770 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:49.871+0000 7f612bfff700 1 -- 172.21.15.22:0/3069370123 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6134005d90 con 0x7f613807bdd0 2024-02-16T19:19:50.770 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:49.871+0000 7f612bfff700 1 -- 172.21.15.22:0/3069370123 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6134006030 con 0x7f613807bdd0 2024-02-16T19:19:50.770 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:49.871+0000 7f613f333700 1 -- 172.21.15.22:0/3069370123 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f613807bdd0 msgr2=0x7f613807a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:19:50.770 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:49.871+0000 7f613f333700 1 --2- 172.21.15.22:0/3069370123 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f613807bdd0 0x7f613807a2d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6134009870 tx=0x7f6134009b80 comp rx=0 tx=0).stop 2024-02-16T19:19:50.770 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:49.872+0000 7f613f333700 1 -- 172.21.15.22:0/3069370123 shutdown_connections 2024-02-16T19:19:50.770 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:49.872+0000 7f613f333700 1 --2- 172.21.15.22:0/3069370123 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f613807bdd0 0x7f613807a2d0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:50.771 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:49.872+0000 7f613f333700 1 -- 172.21.15.22:0/3069370123 >> 172.21.15.22:0/3069370123 conn(0x7f6138101020 msgr2=0x7f6138103450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:19:50.771 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:49.872+0000 7f613f333700 1 -- 172.21.15.22:0/3069370123 shutdown_connections 2024-02-16T19:19:50.771 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:49.872+0000 7f613f333700 1 -- 172.21.15.22:0/3069370123 wait complete. 2024-02-16T19:19:50.771 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:49.872+0000 7f613f333700 1 Processor -- start 2024-02-16T19:19:50.771 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:49.872+0000 7f613f333700 1 -- start start 2024-02-16T19:19:50.771 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:49.872+0000 7f613f333700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f613807bdd0 0x7f613819b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:19:50.771 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:49.872+0000 7f613f333700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f613819be00 con 0x7f613807bdd0 2024-02-16T19:19:50.771 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:49.872+0000 7f613d0cf700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f613807bdd0 0x7f613819b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:19:50.771 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:49.872+0000 7f613d0cf700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f613807bdd0 0x7f613819b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59604/0 (socket says 172.21.15.22:59604) 2024-02-16T19:19:50.771 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:49.872+0000 7f613d0cf700 1 -- 172.21.15.22:0/315141636 learned_addr learned my addr 172.21.15.22:0/315141636 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:19:50.771 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:49.873+0000 7f613d0cf700 1 -- 172.21.15.22:0/315141636 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6134009500 con 0x7f613807bdd0 2024-02-16T19:19:50.771 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:49.873+0000 7f613d0cf700 1 --2- 172.21.15.22:0/315141636 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f613807bdd0 0x7f613819b8c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f61340038f0 tx=0x7f61340061e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:19:50.771 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:49.873+0000 7f612a7fc700 1 -- 172.21.15.22:0/315141636 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6134004310 con 0x7f613807bdd0 2024-02-16T19:19:50.772 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:49.873+0000 7f612a7fc700 1 -- 172.21.15.22:0/315141636 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6134004470 con 0x7f613807bdd0 2024-02-16T19:19:50.772 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:49.873+0000 7f612a7fc700 1 -- 172.21.15.22:0/315141636 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6134011c90 con 0x7f613807bdd0 2024-02-16T19:19:50.772 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:49.873+0000 7f613f333700 1 -- 172.21.15.22:0/315141636 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f613819c000 con 0x7f613807bdd0 2024-02-16T19:19:50.772 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:49.873+0000 7f613f333700 1 -- 172.21.15.22:0/315141636 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f613819c4a0 con 0x7f613807bdd0 2024-02-16T19:19:50.772 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:49.874+0000 7f612a7fc700 1 -- 172.21.15.22:0/315141636 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f6134018070 con 0x7f613807bdd0 2024-02-16T19:19:50.772 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:49.874+0000 7f612a7fc700 1 --2- 172.21.15.22:0/315141636 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7f61240383b0 0x7f612403a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:19:50.772 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:49.874+0000 7f613f333700 1 -- 172.21.15.22:0/315141636 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f613810bca0 con 0x7f613807bdd0 2024-02-16T19:19:50.772 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:49.874+0000 7f613c8ce700 1 --2- 172.21.15.22:0/315141636 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7f61240383b0 0x7f612403a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:19:50.772 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:49.874+0000 7f612a7fc700 1 -- 172.21.15.22:0/315141636 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f613404b910 con 0x7f613807bdd0 2024-02-16T19:19:50.772 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:49.875+0000 7f613c8ce700 1 --2- 172.21.15.22:0/315141636 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7f61240383b0 0x7f612403a860 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f612c006fd0 tx=0x7f612c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:19:50.772 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:49.879+0000 7f612a7fc700 1 -- 172.21.15.22:0/315141636 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f61340186e0 con 0x7f613807bdd0 2024-02-16T19:19:50.772 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:50.101+0000 7f613f333700 1 -- 172.21.15.22:0/315141636 --> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}) v1 -- 0x7f613819f2e0 con 0x7f61240383b0 2024-02-16T19:19:50.772 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:50.106+0000 7f612a7fc700 1 -- 172.21.15.22:0/315141636 <== mgr.14120 v2:172.21.15.22:6800/2755289428 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f613819f2e0 con 0x7f61240383b0 2024-02-16T19:19:50.772 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:50.108+0000 7f613f333700 1 -- 172.21.15.22:0/315141636 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7f61240383b0 msgr2=0x7f612403a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:19:50.773 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:50.108+0000 7f613f333700 1 --2- 172.21.15.22:0/315141636 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7f61240383b0 0x7f612403a860 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f612c006fd0 tx=0x7f612c006e40 comp rx=0 tx=0).stop 2024-02-16T19:19:50.773 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:50.108+0000 7f613f333700 1 -- 172.21.15.22:0/315141636 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f613807bdd0 msgr2=0x7f613819b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:19:50.773 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:50.108+0000 7f613f333700 1 --2- 172.21.15.22:0/315141636 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f613807bdd0 0x7f613819b8c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f61340038f0 tx=0x7f61340061e0 comp rx=0 tx=0).stop 2024-02-16T19:19:50.773 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:50.108+0000 7f613f333700 1 -- 172.21.15.22:0/315141636 shutdown_connections 2024-02-16T19:19:50.773 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:50.108+0000 7f613f333700 1 --2- 172.21.15.22:0/315141636 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7f61240383b0 0x7f612403a860 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:50.773 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:50.108+0000 7f613f333700 1 --2- 172.21.15.22:0/315141636 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f613807bdd0 0x7f613819b8c0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:50.773 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:50.108+0000 7f613f333700 1 -- 172.21.15.22:0/315141636 >> 172.21.15.22:0/315141636 conn(0x7f6138101020 msgr2=0x7f6138101df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:19:50.773 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:50.108+0000 7f613f333700 1 -- 172.21.15.22:0/315141636 shutdown_connections 2024-02-16T19:19:50.773 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:50.108+0000 7f613f333700 1 -- 172.21.15.22:0/315141636 wait complete. 2024-02-16T19:19:50.773 INFO:teuthology.orchestra.run.smithi022.stdout:Deploying mgr service with default placement... 2024-02-16T19:19:51.452 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:51 smithi022 ceph-mon[29221]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:19:51.452 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:51 smithi022 ceph-mon[29221]: Saving service mon spec with placement count:5 2024-02-16T19:19:51.452 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:51 smithi022 ceph-mon[29221]: from='mgr.14120 172.21.15.22:0/1425529966' entity='mgr.smithi022.gmdcch' 2024-02-16T19:19:52.153 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-02-16T19:19:52.153 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:51.228+0000 7f8b3e4ca700 1 Processor -- start 2024-02-16T19:19:52.153 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:51.228+0000 7f8b3e4ca700 1 -- start start 2024-02-16T19:19:52.153 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:51.228+0000 7f8b3e4ca700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b381059c0 0x7f8b38105dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:19:52.153 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:51.228+0000 7f8b3e4ca700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b38106310 con 0x7f8b381059c0 2024-02-16T19:19:52.153 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:51.228+0000 7f8b37fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b381059c0 0x7f8b38105dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:19:52.153 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:51.228+0000 7f8b37fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b381059c0 0x7f8b38105dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59620/0 (socket says 172.21.15.22:59620) 2024-02-16T19:19:52.153 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:51.228+0000 7f8b37fff700 1 -- 172.21.15.22:0/999911279 learned_addr learned my addr 172.21.15.22:0/999911279 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:19:52.153 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:51.229+0000 7f8b37fff700 1 -- 172.21.15.22:0/999911279 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b38106450 con 0x7f8b381059c0 2024-02-16T19:19:52.153 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:51.229+0000 7f8b37fff700 1 --2- 172.21.15.22:0/999911279 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b381059c0 0x7f8b38105dd0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8b280098a0 tx=0x7f8b28009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5950647c74aa23b3 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:19:52.153 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:51.229+0000 7f8b36ffd700 1 -- 172.21.15.22:0/999911279 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8b28004180 con 0x7f8b381059c0 2024-02-16T19:19:52.153 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:51.229+0000 7f8b36ffd700 1 -- 172.21.15.22:0/999911279 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8b280042e0 con 0x7f8b381059c0 2024-02-16T19:19:52.154 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:51.229+0000 7f8b36ffd700 1 -- 172.21.15.22:0/999911279 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8b28011a50 con 0x7f8b381059c0 2024-02-16T19:19:52.154 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:51.229+0000 7f8b3e4ca700 1 -- 172.21.15.22:0/999911279 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b381059c0 msgr2=0x7f8b38105dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:19:52.154 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:51.229+0000 7f8b3e4ca700 1 --2- 172.21.15.22:0/999911279 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b381059c0 0x7f8b38105dd0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8b280098a0 tx=0x7f8b28009bb0 comp rx=0 tx=0).stop 2024-02-16T19:19:52.154 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:51.229+0000 7f8b3e4ca700 1 -- 172.21.15.22:0/999911279 shutdown_connections 2024-02-16T19:19:52.154 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:51.229+0000 7f8b3e4ca700 1 --2- 172.21.15.22:0/999911279 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b381059c0 0x7f8b38105dd0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:52.154 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:51.230+0000 7f8b3e4ca700 1 -- 172.21.15.22:0/999911279 >> 172.21.15.22:0/999911279 conn(0x7f8b38100ff0 msgr2=0x7f8b38103440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:19:52.154 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:51.230+0000 7f8b3e4ca700 1 -- 172.21.15.22:0/999911279 shutdown_connections 2024-02-16T19:19:52.154 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:51.230+0000 7f8b3e4ca700 1 -- 172.21.15.22:0/999911279 wait complete. 2024-02-16T19:19:52.154 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:51.230+0000 7f8b3e4ca700 1 Processor -- start 2024-02-16T19:19:52.154 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:51.230+0000 7f8b3e4ca700 1 -- start start 2024-02-16T19:19:52.154 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:51.230+0000 7f8b3e4ca700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b381059c0 0x7f8b3807bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:19:52.154 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:51.230+0000 7f8b3e4ca700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3807c160 con 0x7f8b381059c0 2024-02-16T19:19:52.154 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:51.231+0000 7f8b37fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b381059c0 0x7f8b3807bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:19:52.154 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:51.231+0000 7f8b37fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b381059c0 0x7f8b3807bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59634/0 (socket says 172.21.15.22:59634) 2024-02-16T19:19:52.155 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:51.231+0000 7f8b37fff700 1 -- 172.21.15.22:0/2637017820 learned_addr learned my addr 172.21.15.22:0/2637017820 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:19:52.155 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:51.231+0000 7f8b37fff700 1 -- 172.21.15.22:0/2637017820 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b28009550 con 0x7f8b381059c0 2024-02-16T19:19:52.155 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:51.231+0000 7f8b37fff700 1 --2- 172.21.15.22:0/2637017820 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b381059c0 0x7f8b3807bc20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8b28003f80 tx=0x7f8b28011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:19:52.155 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:51.231+0000 7f8b357fa700 1 -- 172.21.15.22:0/2637017820 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8b28003a80 con 0x7f8b381059c0 2024-02-16T19:19:52.155 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:51.231+0000 7f8b3e4ca700 1 -- 172.21.15.22:0/2637017820 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b3807a270 con 0x7f8b381059c0 2024-02-16T19:19:52.155 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:51.231+0000 7f8b357fa700 1 -- 172.21.15.22:0/2637017820 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8b28003be0 con 0x7f8b381059c0 2024-02-16T19:19:52.155 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:51.231+0000 7f8b357fa700 1 -- 172.21.15.22:0/2637017820 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8b28019820 con 0x7f8b381059c0 2024-02-16T19:19:52.155 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:51.231+0000 7f8b3e4ca700 1 -- 172.21.15.22:0/2637017820 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b3807a710 con 0x7f8b381059c0 2024-02-16T19:19:52.155 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:51.232+0000 7f8b357fa700 1 -- 172.21.15.22:0/2637017820 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f8b28019ac0 con 0x7f8b381059c0 2024-02-16T19:19:52.155 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:51.232+0000 7f8b357fa700 1 --2- 172.21.15.22:0/2637017820 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7f8b200383a0 0x7f8b2003a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:19:52.155 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:51.232+0000 7f8b357fa700 1 -- 172.21.15.22:0/2637017820 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f8b2804c5d0 con 0x7f8b381059c0 2024-02-16T19:19:52.155 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:51.232+0000 7f8b3e4ca700 1 -- 172.21.15.22:0/2637017820 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b38062380 con 0x7f8b381059c0 2024-02-16T19:19:52.155 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:51.232+0000 7f8b377fe700 1 --2- 172.21.15.22:0/2637017820 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7f8b200383a0 0x7f8b2003a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:19:52.155 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:51.233+0000 7f8b377fe700 1 --2- 172.21.15.22:0/2637017820 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7f8b200383a0 0x7f8b2003a850 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c006fd0 tx=0x7f8b2c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:19:52.155 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:51.237+0000 7f8b357fa700 1 -- 172.21.15.22:0/2637017820 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f8b2802c3a0 con 0x7f8b381059c0 2024-02-16T19:19:52.156 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:51.451+0000 7f8b3e4ca700 1 -- 172.21.15.22:0/2637017820 --> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7f8b38109e60 con 0x7f8b200383a0 2024-02-16T19:19:52.156 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:51.455+0000 7f8b357fa700 1 -- 172.21.15.22:0/2637017820 <== mgr.14120 v2:172.21.15.22:6800/2755289428 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f8b2c009e20 con 0x7f8b200383a0 2024-02-16T19:19:52.156 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:51.457+0000 7f8b3e4ca700 1 -- 172.21.15.22:0/2637017820 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7f8b200383a0 msgr2=0x7f8b2003a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:19:52.156 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:51.457+0000 7f8b3e4ca700 1 --2- 172.21.15.22:0/2637017820 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7f8b200383a0 0x7f8b2003a850 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c006fd0 tx=0x7f8b2c006e40 comp rx=0 tx=0).stop 2024-02-16T19:19:52.156 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:51.457+0000 7f8b3e4ca700 1 -- 172.21.15.22:0/2637017820 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b381059c0 msgr2=0x7f8b3807bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:19:52.156 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:51.457+0000 7f8b3e4ca700 1 --2- 172.21.15.22:0/2637017820 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b381059c0 0x7f8b3807bc20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8b28003f80 tx=0x7f8b28011ab0 comp rx=0 tx=0).stop 2024-02-16T19:19:52.156 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:51.457+0000 7f8b3e4ca700 1 -- 172.21.15.22:0/2637017820 shutdown_connections 2024-02-16T19:19:52.156 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:51.457+0000 7f8b3e4ca700 1 --2- 172.21.15.22:0/2637017820 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7f8b200383a0 0x7f8b2003a850 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:52.156 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:51.457+0000 7f8b3e4ca700 1 --2- 172.21.15.22:0/2637017820 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b381059c0 0x7f8b3807bc20 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:52.156 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:51.457+0000 7f8b3e4ca700 1 -- 172.21.15.22:0/2637017820 >> 172.21.15.22:0/2637017820 conn(0x7f8b38100ff0 msgr2=0x7f8b38102710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:19:52.156 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:51.457+0000 7f8b3e4ca700 1 -- 172.21.15.22:0/2637017820 shutdown_connections 2024-02-16T19:19:52.156 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:51.457+0000 7f8b3e4ca700 1 -- 172.21.15.22:0/2637017820 wait complete. 2024-02-16T19:19:52.156 INFO:teuthology.orchestra.run.smithi022.stdout:Deploying crash service with default placement... 2024-02-16T19:19:52.548 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:52 smithi022 ceph-mon[29221]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:19:52.548 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:52 smithi022 ceph-mon[29221]: Saving service mgr spec with placement count:2 2024-02-16T19:19:52.548 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:52 smithi022 ceph-mon[29221]: from='mgr.14120 172.21.15.22:0/1425529966' entity='mgr.smithi022.gmdcch' 2024-02-16T19:19:52.548 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:52 smithi022 ceph-mon[29221]: from='mgr.14120 172.21.15.22:0/1425529966' entity='mgr.smithi022.gmdcch' 2024-02-16T19:19:53.409 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout Scheduled crash update... 2024-02-16T19:19:53.409 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:52.566+0000 7fc4b2948700 1 Processor -- start 2024-02-16T19:19:53.409 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:52.566+0000 7fc4b2948700 1 -- start start 2024-02-16T19:19:53.409 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:52.566+0000 7fc4b2948700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4ac1059d0 0x7fc4ac105de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:19:53.409 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:52.566+0000 7fc4b2948700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4ac106320 con 0x7fc4ac1059d0 2024-02-16T19:19:53.409 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:52.566+0000 7fc4abfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4ac1059d0 0x7fc4ac105de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:19:53.409 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:52.566+0000 7fc4abfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4ac1059d0 0x7fc4ac105de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59640/0 (socket says 172.21.15.22:59640) 2024-02-16T19:19:53.409 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:52.566+0000 7fc4abfff700 1 -- 172.21.15.22:0/3286983317 learned_addr learned my addr 172.21.15.22:0/3286983317 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:19:53.409 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:52.566+0000 7fc4abfff700 1 -- 172.21.15.22:0/3286983317 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4ac106460 con 0x7fc4ac1059d0 2024-02-16T19:19:53.410 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:52.567+0000 7fc4abfff700 1 --2- 172.21.15.22:0/3286983317 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4ac1059d0 0x7fc4ac105de0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc49c009870 tx=0x7fc49c009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4e34a18f8a1f325 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:19:53.410 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:52.567+0000 7fc4aaffd700 1 -- 172.21.15.22:0/3286983317 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc49c005c30 con 0x7fc4ac1059d0 2024-02-16T19:19:53.410 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:52.567+0000 7fc4aaffd700 1 -- 172.21.15.22:0/3286983317 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc49c0044e0 con 0x7fc4ac1059d0 2024-02-16T19:19:53.410 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:52.567+0000 7fc4aaffd700 1 -- 172.21.15.22:0/3286983317 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc49c006030 con 0x7fc4ac1059d0 2024-02-16T19:19:53.410 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:52.567+0000 7fc4b2948700 1 -- 172.21.15.22:0/3286983317 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4ac1059d0 msgr2=0x7fc4ac105de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:19:53.410 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:52.567+0000 7fc4b2948700 1 --2- 172.21.15.22:0/3286983317 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4ac1059d0 0x7fc4ac105de0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc49c009870 tx=0x7fc49c009b80 comp rx=0 tx=0).stop 2024-02-16T19:19:53.410 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:52.567+0000 7fc4b2948700 1 -- 172.21.15.22:0/3286983317 shutdown_connections 2024-02-16T19:19:53.410 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:52.567+0000 7fc4b2948700 1 --2- 172.21.15.22:0/3286983317 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4ac1059d0 0x7fc4ac105de0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:53.410 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:52.567+0000 7fc4b2948700 1 -- 172.21.15.22:0/3286983317 >> 172.21.15.22:0/3286983317 conn(0x7fc4ac101020 msgr2=0x7fc4ac103450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:19:53.410 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:52.567+0000 7fc4b2948700 1 -- 172.21.15.22:0/3286983317 shutdown_connections 2024-02-16T19:19:53.410 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:52.567+0000 7fc4b2948700 1 -- 172.21.15.22:0/3286983317 wait complete. 2024-02-16T19:19:53.410 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:52.568+0000 7fc4b2948700 1 Processor -- start 2024-02-16T19:19:53.410 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:52.568+0000 7fc4b2948700 1 -- start start 2024-02-16T19:19:53.410 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:52.568+0000 7fc4b2948700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4ac1059d0 0x7fc4ac07bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:19:53.411 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:52.568+0000 7fc4b2948700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4ac07c160 con 0x7fc4ac1059d0 2024-02-16T19:19:53.411 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:52.568+0000 7fc4abfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4ac1059d0 0x7fc4ac07bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:19:53.411 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:52.568+0000 7fc4abfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4ac1059d0 0x7fc4ac07bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59642/0 (socket says 172.21.15.22:59642) 2024-02-16T19:19:53.411 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:52.568+0000 7fc4abfff700 1 -- 172.21.15.22:0/4251642888 learned_addr learned my addr 172.21.15.22:0/4251642888 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:19:53.411 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:52.569+0000 7fc4abfff700 1 -- 172.21.15.22:0/4251642888 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc49c009500 con 0x7fc4ac1059d0 2024-02-16T19:19:53.411 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:52.569+0000 7fc4abfff700 1 --2- 172.21.15.22:0/4251642888 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4ac1059d0 0x7fc4ac07bc20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc49c0046b0 tx=0x7fc49c0042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:19:53.411 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:52.569+0000 7fc4a97fa700 1 -- 172.21.15.22:0/4251642888 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc49c005e40 con 0x7fc4ac1059d0 2024-02-16T19:19:53.411 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:52.569+0000 7fc4b2948700 1 -- 172.21.15.22:0/4251642888 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4ac07a270 con 0x7fc4ac1059d0 2024-02-16T19:19:53.411 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:52.569+0000 7fc4a97fa700 1 -- 172.21.15.22:0/4251642888 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc49c006840 con 0x7fc4ac1059d0 2024-02-16T19:19:53.411 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:52.569+0000 7fc4a97fa700 1 -- 172.21.15.22:0/4251642888 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc49c011bc0 con 0x7fc4ac1059d0 2024-02-16T19:19:53.411 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:52.569+0000 7fc4b2948700 1 -- 172.21.15.22:0/4251642888 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4ac07a710 con 0x7fc4ac1059d0 2024-02-16T19:19:53.411 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:52.570+0000 7fc4a97fa700 1 -- 172.21.15.22:0/4251642888 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7fc49c018070 con 0x7fc4ac1059d0 2024-02-16T19:19:53.411 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:52.570+0000 7fc4b2948700 1 -- 172.21.15.22:0/4251642888 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4ac062380 con 0x7fc4ac1059d0 2024-02-16T19:19:53.411 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:52.570+0000 7fc4a97fa700 1 --2- 172.21.15.22:0/4251642888 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7fc4940383b0 0x7fc49403a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:19:53.412 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:52.570+0000 7fc4a97fa700 1 -- 172.21.15.22:0/4251642888 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fc49c04bef0 con 0x7fc4ac1059d0 2024-02-16T19:19:53.412 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:52.570+0000 7fc4ab7fe700 1 --2- 172.21.15.22:0/4251642888 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7fc4940383b0 0x7fc49403a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:19:53.412 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:52.571+0000 7fc4ab7fe700 1 --2- 172.21.15.22:0/4251642888 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7fc4940383b0 0x7fc49403a860 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0006fd0 tx=0x7fc4a0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:19:53.412 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:52.575+0000 7fc4a97fa700 1 -- 172.21.15.22:0/4251642888 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fc49c00bd70 con 0x7fc4ac1059d0 2024-02-16T19:19:53.412 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:52.793+0000 7fc4b2948700 1 -- 172.21.15.22:0/4251642888 --> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}) v1 -- 0x7fc4ac102480 con 0x7fc4940383b0 2024-02-16T19:19:53.412 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:52.797+0000 7fc4a97fa700 1 -- 172.21.15.22:0/4251642888 <== mgr.14120 v2:172.21.15.22:6800/2755289428 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7fc4ac102480 con 0x7fc4940383b0 2024-02-16T19:19:53.412 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:52.800+0000 7fc4b2948700 1 -- 172.21.15.22:0/4251642888 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7fc4940383b0 msgr2=0x7fc49403a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:19:53.412 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:52.800+0000 7fc4b2948700 1 --2- 172.21.15.22:0/4251642888 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7fc4940383b0 0x7fc49403a860 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0006fd0 tx=0x7fc4a0006e40 comp rx=0 tx=0).stop 2024-02-16T19:19:53.412 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:52.800+0000 7fc4b2948700 1 -- 172.21.15.22:0/4251642888 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4ac1059d0 msgr2=0x7fc4ac07bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:19:53.412 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:52.800+0000 7fc4b2948700 1 --2- 172.21.15.22:0/4251642888 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4ac1059d0 0x7fc4ac07bc20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc49c0046b0 tx=0x7fc49c0042e0 comp rx=0 tx=0).stop 2024-02-16T19:19:53.412 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:52.800+0000 7fc4b2948700 1 -- 172.21.15.22:0/4251642888 shutdown_connections 2024-02-16T19:19:53.412 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:52.800+0000 7fc4b2948700 1 --2- 172.21.15.22:0/4251642888 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7fc4940383b0 0x7fc49403a860 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:53.412 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:52.800+0000 7fc4b2948700 1 --2- 172.21.15.22:0/4251642888 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4ac1059d0 0x7fc4ac07bc20 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:53.413 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:52.800+0000 7fc4b2948700 1 -- 172.21.15.22:0/4251642888 >> 172.21.15.22:0/4251642888 conn(0x7fc4ac101020 msgr2=0x7fc4ac101d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:19:53.413 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:52.800+0000 7fc4b2948700 1 -- 172.21.15.22:0/4251642888 shutdown_connections 2024-02-16T19:19:53.413 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:52.800+0000 7fc4b2948700 1 -- 172.21.15.22:0/4251642888 wait complete. 2024-02-16T19:19:53.413 INFO:teuthology.orchestra.run.smithi022.stdout:Deploying ceph-exporter service with default placement... 2024-02-16T19:19:53.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:53 smithi022 ceph-mon[29221]: from='mgr.14120 172.21.15.22:0/1425529966' entity='mgr.smithi022.gmdcch' 2024-02-16T19:19:53.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:53 smithi022 ceph-mon[29221]: from='client.14146 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:19:53.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:53 smithi022 ceph-mon[29221]: Saving service crash spec with placement * 2024-02-16T19:19:53.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:53 smithi022 ceph-mon[29221]: from='mgr.14120 172.21.15.22:0/1425529966' entity='mgr.smithi022.gmdcch' 2024-02-16T19:19:55.186 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:55 smithi022 ceph-mon[29221]: from='client.14148 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "ceph-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:19:55.186 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:55 smithi022 ceph-mon[29221]: Saving service ceph-exporter spec with placement * 2024-02-16T19:19:55.186 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:55 smithi022 ceph-mon[29221]: from='mgr.14120 172.21.15.22:0/1425529966' entity='mgr.smithi022.gmdcch' 2024-02-16T19:19:55.186 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:55 smithi022 ceph-mon[29221]: from='mgr.14120 172.21.15.22:0/1425529966' entity='mgr.smithi022.gmdcch' 2024-02-16T19:19:55.193 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout Scheduled ceph-exporter update... 2024-02-16T19:19:55.193 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:53.851+0000 7f9c973a0700 1 Processor -- start 2024-02-16T19:19:55.193 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:53.851+0000 7f9c973a0700 1 -- start start 2024-02-16T19:19:55.193 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:53.852+0000 7f9c973a0700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c9007bdb0 0x7f9c9007a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:19:55.193 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:53.852+0000 7f9c973a0700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c9007a7f0 con 0x7f9c9007bdb0 2024-02-16T19:19:55.193 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:53.852+0000 7f9c9639e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c9007bdb0 0x7f9c9007a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:19:55.194 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:53.852+0000 7f9c9639e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c9007bdb0 0x7f9c9007a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59644/0 (socket says 172.21.15.22:59644) 2024-02-16T19:19:55.194 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:53.852+0000 7f9c9639e700 1 -- 172.21.15.22:0/2982959924 learned_addr learned my addr 172.21.15.22:0/2982959924 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:19:55.194 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:53.852+0000 7f9c9639e700 1 -- 172.21.15.22:0/2982959924 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c9007a930 con 0x7f9c9007bdb0 2024-02-16T19:19:55.194 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:53.852+0000 7f9c9639e700 1 --2- 172.21.15.22:0/2982959924 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c9007bdb0 0x7f9c9007a2b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9c800098a0 tx=0x7f9c80009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7abe88467ad13e07 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:19:55.194 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:53.852+0000 7f9c9539c700 1 -- 172.21.15.22:0/2982959924 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9c80004180 con 0x7f9c9007bdb0 2024-02-16T19:19:55.194 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:53.852+0000 7f9c9539c700 1 -- 172.21.15.22:0/2982959924 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9c800042e0 con 0x7f9c9007bdb0 2024-02-16T19:19:55.194 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:53.852+0000 7f9c9539c700 1 -- 172.21.15.22:0/2982959924 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9c80011af0 con 0x7f9c9007bdb0 2024-02-16T19:19:55.194 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:53.853+0000 7f9c973a0700 1 -- 172.21.15.22:0/2982959924 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c9007bdb0 msgr2=0x7f9c9007a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:19:55.194 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:53.853+0000 7f9c973a0700 1 --2- 172.21.15.22:0/2982959924 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c9007bdb0 0x7f9c9007a2b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9c800098a0 tx=0x7f9c80009bb0 comp rx=0 tx=0).stop 2024-02-16T19:19:55.194 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:53.853+0000 7f9c973a0700 1 -- 172.21.15.22:0/2982959924 shutdown_connections 2024-02-16T19:19:55.194 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:53.853+0000 7f9c973a0700 1 --2- 172.21.15.22:0/2982959924 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c9007bdb0 0x7f9c9007a2b0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:55.194 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:53.853+0000 7f9c973a0700 1 -- 172.21.15.22:0/2982959924 >> 172.21.15.22:0/2982959924 conn(0x7f9c90100e50 msgr2=0x7f9c90103280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:19:55.194 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:53.853+0000 7f9c973a0700 1 -- 172.21.15.22:0/2982959924 shutdown_connections 2024-02-16T19:19:55.194 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:53.853+0000 7f9c973a0700 1 -- 172.21.15.22:0/2982959924 wait complete. 2024-02-16T19:19:55.195 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:53.854+0000 7f9c973a0700 1 Processor -- start 2024-02-16T19:19:55.195 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:53.854+0000 7f9c973a0700 1 -- start start 2024-02-16T19:19:55.195 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:53.854+0000 7f9c973a0700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c9007bdb0 0x7f9c9019b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:19:55.195 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:53.854+0000 7f9c973a0700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c9019bcc0 con 0x7f9c9007bdb0 2024-02-16T19:19:55.195 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:53.854+0000 7f9c9639e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c9007bdb0 0x7f9c9019b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:19:55.195 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:53.854+0000 7f9c9639e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c9007bdb0 0x7f9c9019b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59646/0 (socket says 172.21.15.22:59646) 2024-02-16T19:19:55.195 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:53.854+0000 7f9c9639e700 1 -- 172.21.15.22:0/2056604684 learned_addr learned my addr 172.21.15.22:0/2056604684 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:19:55.195 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:53.854+0000 7f9c9639e700 1 -- 172.21.15.22:0/2056604684 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c80009550 con 0x7f9c9007bdb0 2024-02-16T19:19:55.195 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:53.854+0000 7f9c9639e700 1 --2- 172.21.15.22:0/2056604684 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c9007bdb0 0x7f9c9019b780 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9c80006ba0 tx=0x7f9c80011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:19:55.195 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:53.854+0000 7f9c877fe700 1 -- 172.21.15.22:0/2056604684 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9c80003b00 con 0x7f9c9007bdb0 2024-02-16T19:19:55.195 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:53.854+0000 7f9c877fe700 1 -- 172.21.15.22:0/2056604684 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9c8001a8e0 con 0x7f9c9007bdb0 2024-02-16T19:19:55.195 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:53.854+0000 7f9c973a0700 1 -- 172.21.15.22:0/2056604684 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c9019bec0 con 0x7f9c9007bdb0 2024-02-16T19:19:55.195 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:53.854+0000 7f9c877fe700 1 -- 172.21.15.22:0/2056604684 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9c80019990 con 0x7f9c9007bdb0 2024-02-16T19:19:55.195 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:53.854+0000 7f9c973a0700 1 -- 172.21.15.22:0/2056604684 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c9019c360 con 0x7f9c9007bdb0 2024-02-16T19:19:55.195 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:53.855+0000 7f9c877fe700 1 -- 172.21.15.22:0/2056604684 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f9c80019c00 con 0x7f9c9007bdb0 2024-02-16T19:19:55.195 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:53.855+0000 7f9c877fe700 1 --2- 172.21.15.22:0/2056604684 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7f9c7c0383f0 0x7f9c7c03a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:19:55.196 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:53.855+0000 7f9c973a0700 1 -- 172.21.15.22:0/2056604684 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c9010bb50 con 0x7f9c9007bdb0 2024-02-16T19:19:55.196 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:53.856+0000 7f9c877fe700 1 -- 172.21.15.22:0/2056604684 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f9c8004c660 con 0x7f9c9007bdb0 2024-02-16T19:19:55.196 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:53.856+0000 7f9c95b9d700 1 --2- 172.21.15.22:0/2056604684 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7f9c7c0383f0 0x7f9c7c03a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:19:55.196 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:53.856+0000 7f9c95b9d700 1 --2- 172.21.15.22:0/2056604684 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7f9c7c0383f0 0x7f9c7c03a8a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c006fd0 tx=0x7f9c8c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:19:55.196 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:53.860+0000 7f9c877fe700 1 -- 172.21.15.22:0/2056604684 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f9c8001aa50 con 0x7f9c9007bdb0 2024-02-16T19:19:55.196 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:54.095+0000 7f9c973a0700 1 -- 172.21.15.22:0/2056604684 --> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "ceph-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7f9c90102350 con 0x7f9c7c0383f0 2024-02-16T19:19:55.196 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:54.099+0000 7f9c877fe700 1 -- 172.21.15.22:0/2056604684 <== mgr.14120 v2:172.21.15.22:6800/2755289428 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7f9c90102350 con 0x7f9c7c0383f0 2024-02-16T19:19:55.196 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:54.102+0000 7f9c973a0700 1 -- 172.21.15.22:0/2056604684 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7f9c7c0383f0 msgr2=0x7f9c7c03a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:19:55.196 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:54.102+0000 7f9c973a0700 1 --2- 172.21.15.22:0/2056604684 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7f9c7c0383f0 0x7f9c7c03a8a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c006fd0 tx=0x7f9c8c006e40 comp rx=0 tx=0).stop 2024-02-16T19:19:55.196 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:54.102+0000 7f9c973a0700 1 -- 172.21.15.22:0/2056604684 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c9007bdb0 msgr2=0x7f9c9019b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:19:55.196 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:54.102+0000 7f9c973a0700 1 --2- 172.21.15.22:0/2056604684 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c9007bdb0 0x7f9c9019b780 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9c80006ba0 tx=0x7f9c80011b50 comp rx=0 tx=0).stop 2024-02-16T19:19:55.196 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:54.102+0000 7f9c973a0700 1 -- 172.21.15.22:0/2056604684 shutdown_connections 2024-02-16T19:19:55.196 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:54.102+0000 7f9c973a0700 1 --2- 172.21.15.22:0/2056604684 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7f9c7c0383f0 0x7f9c7c03a8a0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:55.196 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:54.102+0000 7f9c973a0700 1 --2- 172.21.15.22:0/2056604684 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c9007bdb0 0x7f9c9019b780 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:55.197 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:54.102+0000 7f9c973a0700 1 -- 172.21.15.22:0/2056604684 >> 172.21.15.22:0/2056604684 conn(0x7f9c90100e50 msgr2=0x7f9c90101c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:19:55.197 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:54.102+0000 7f9c973a0700 1 -- 172.21.15.22:0/2056604684 shutdown_connections 2024-02-16T19:19:55.197 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:54.102+0000 7f9c973a0700 1 -- 172.21.15.22:0/2056604684 wait complete. 2024-02-16T19:19:55.197 INFO:teuthology.orchestra.run.smithi022.stdout:Deploying prometheus service with default placement... 2024-02-16T19:19:56.483 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout Scheduled prometheus update... 2024-02-16T19:19:56.483 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:55.631+0000 7ff0852ad700 1 Processor -- start 2024-02-16T19:19:56.483 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:55.631+0000 7ff0852ad700 1 -- start start 2024-02-16T19:19:56.483 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:55.631+0000 7ff0852ad700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff080104040 0x7ff080104450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:19:56.483 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:55.631+0000 7ff0852ad700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff080104990 con 0x7ff080104040 2024-02-16T19:19:56.483 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:55.631+0000 7ff07effd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff080104040 0x7ff080104450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:19:56.483 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:55.631+0000 7ff07effd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff080104040 0x7ff080104450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49302/0 (socket says 172.21.15.22:49302) 2024-02-16T19:19:56.483 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:55.631+0000 7ff07effd700 1 -- 172.21.15.22:0/1128328412 learned_addr learned my addr 172.21.15.22:0/1128328412 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:19:56.483 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:55.632+0000 7ff07effd700 1 -- 172.21.15.22:0/1128328412 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff080104ad0 con 0x7ff080104040 2024-02-16T19:19:56.483 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:55.632+0000 7ff07effd700 1 --2- 172.21.15.22:0/1128328412 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff080104040 0x7ff080104450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff0700098a0 tx=0x7ff070009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb797281f6b700c8 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:19:56.483 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:55.632+0000 7ff07dffb700 1 -- 172.21.15.22:0/1128328412 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff070004180 con 0x7ff080104040 2024-02-16T19:19:56.483 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:55.632+0000 7ff07dffb700 1 -- 172.21.15.22:0/1128328412 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff0700042e0 con 0x7ff080104040 2024-02-16T19:19:56.484 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:55.632+0000 7ff07dffb700 1 -- 172.21.15.22:0/1128328412 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff070011af0 con 0x7ff080104040 2024-02-16T19:19:56.484 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:55.632+0000 7ff0852ad700 1 -- 172.21.15.22:0/1128328412 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff080104040 msgr2=0x7ff080104450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:19:56.484 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:55.632+0000 7ff0852ad700 1 --2- 172.21.15.22:0/1128328412 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff080104040 0x7ff080104450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff0700098a0 tx=0x7ff070009bb0 comp rx=0 tx=0).stop 2024-02-16T19:19:56.484 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:55.632+0000 7ff0852ad700 1 -- 172.21.15.22:0/1128328412 shutdown_connections 2024-02-16T19:19:56.484 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:55.632+0000 7ff0852ad700 1 --2- 172.21.15.22:0/1128328412 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff080104040 0x7ff080104450 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:56.484 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:55.632+0000 7ff0852ad700 1 -- 172.21.15.22:0/1128328412 >> 172.21.15.22:0/1128328412 conn(0x7ff0800ff670 msgr2=0x7ff080101ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:19:56.484 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:55.633+0000 7ff0852ad700 1 -- 172.21.15.22:0/1128328412 shutdown_connections 2024-02-16T19:19:56.484 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:55.633+0000 7ff0852ad700 1 -- 172.21.15.22:0/1128328412 wait complete. 2024-02-16T19:19:56.484 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:55.633+0000 7ff0852ad700 1 Processor -- start 2024-02-16T19:19:56.484 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:55.633+0000 7ff0852ad700 1 -- start start 2024-02-16T19:19:56.484 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:55.633+0000 7ff0852ad700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff080104040 0x7ff080198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:19:56.484 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:55.633+0000 7ff0852ad700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff080198b60 con 0x7ff080104040 2024-02-16T19:19:56.484 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:55.634+0000 7ff07effd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff080104040 0x7ff080198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:19:56.484 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:55.634+0000 7ff07effd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff080104040 0x7ff080198620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49308/0 (socket says 172.21.15.22:49308) 2024-02-16T19:19:56.485 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:55.634+0000 7ff07effd700 1 -- 172.21.15.22:0/3774069932 learned_addr learned my addr 172.21.15.22:0/3774069932 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:19:56.485 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:55.634+0000 7ff07effd700 1 -- 172.21.15.22:0/3774069932 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff070009550 con 0x7ff080104040 2024-02-16T19:19:56.485 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:55.634+0000 7ff07effd700 1 --2- 172.21.15.22:0/3774069932 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff080104040 0x7ff080198620 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff070011670 tx=0x7ff070011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:19:56.485 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:55.634+0000 7ff067fff700 1 -- 172.21.15.22:0/3774069932 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff070003a40 con 0x7ff080104040 2024-02-16T19:19:56.485 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:55.634+0000 7ff0852ad700 1 -- 172.21.15.22:0/3774069932 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff080198d60 con 0x7ff080104040 2024-02-16T19:19:56.485 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:55.634+0000 7ff067fff700 1 -- 172.21.15.22:0/3774069932 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff070003ba0 con 0x7ff080104040 2024-02-16T19:19:56.485 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:55.634+0000 7ff067fff700 1 -- 172.21.15.22:0/3774069932 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff070024790 con 0x7ff080104040 2024-02-16T19:19:56.485 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:55.634+0000 7ff0852ad700 1 -- 172.21.15.22:0/3774069932 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff080199200 con 0x7ff080104040 2024-02-16T19:19:56.485 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:55.635+0000 7ff067fff700 1 -- 172.21.15.22:0/3774069932 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7ff07002d3f0 con 0x7ff080104040 2024-02-16T19:19:56.485 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:55.635+0000 7ff0852ad700 1 -- 172.21.15.22:0/3774069932 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff080192580 con 0x7ff080104040 2024-02-16T19:19:56.485 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:55.635+0000 7ff067fff700 1 --2- 172.21.15.22:0/3774069932 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7ff0680383f0 0x7ff06803a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:19:56.485 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:55.635+0000 7ff067fff700 1 -- 172.21.15.22:0/3774069932 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7ff07004c570 con 0x7ff080104040 2024-02-16T19:19:56.485 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:55.635+0000 7ff07e7fc700 1 --2- 172.21.15.22:0/3774069932 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7ff0680383f0 0x7ff06803a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:19:56.485 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:55.636+0000 7ff07e7fc700 1 --2- 172.21.15.22:0/3774069932 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7ff0680383f0 0x7ff06803a8a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff074006fd0 tx=0x7ff074006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:19:56.486 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:55.640+0000 7ff067fff700 1 -- 172.21.15.22:0/3774069932 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7ff070010970 con 0x7ff080104040 2024-02-16T19:19:56.486 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:55.862+0000 7ff0852ad700 1 -- 172.21.15.22:0/3774069932 --> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}) v1 -- 0x7ff080061190 con 0x7ff0680383f0 2024-02-16T19:19:56.486 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:55.866+0000 7ff067fff700 1 -- 172.21.15.22:0/3774069932 <== mgr.14120 v2:172.21.15.22:6800/2755289428 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+31 (secure 0 0 0) 0x7ff080061190 con 0x7ff0680383f0 2024-02-16T19:19:56.486 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:55.868+0000 7ff0852ad700 1 -- 172.21.15.22:0/3774069932 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7ff0680383f0 msgr2=0x7ff06803a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:19:56.486 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:55.868+0000 7ff0852ad700 1 --2- 172.21.15.22:0/3774069932 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7ff0680383f0 0x7ff06803a8a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff074006fd0 tx=0x7ff074006e40 comp rx=0 tx=0).stop 2024-02-16T19:19:56.486 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:55.868+0000 7ff0852ad700 1 -- 172.21.15.22:0/3774069932 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff080104040 msgr2=0x7ff080198620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:19:56.486 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:55.868+0000 7ff0852ad700 1 --2- 172.21.15.22:0/3774069932 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff080104040 0x7ff080198620 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff070011670 tx=0x7ff070011b50 comp rx=0 tx=0).stop 2024-02-16T19:19:56.486 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:55.869+0000 7ff0852ad700 1 -- 172.21.15.22:0/3774069932 shutdown_connections 2024-02-16T19:19:56.486 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:55.869+0000 7ff0852ad700 1 --2- 172.21.15.22:0/3774069932 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7ff0680383f0 0x7ff06803a8a0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:56.486 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:55.869+0000 7ff0852ad700 1 --2- 172.21.15.22:0/3774069932 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff080104040 0x7ff080198620 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:56.486 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:55.869+0000 7ff0852ad700 1 -- 172.21.15.22:0/3774069932 >> 172.21.15.22:0/3774069932 conn(0x7ff0800ff670 msgr2=0x7ff08018f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:19:56.486 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:55.869+0000 7ff0852ad700 1 -- 172.21.15.22:0/3774069932 shutdown_connections 2024-02-16T19:19:56.486 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:55.869+0000 7ff0852ad700 1 -- 172.21.15.22:0/3774069932 wait complete. 2024-02-16T19:19:56.486 INFO:teuthology.orchestra.run.smithi022.stdout:Deploying grafana service with default placement... 2024-02-16T19:19:57.124 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:56 smithi022 ceph-mon[29221]: from='client.14150 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:19:57.124 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:56 smithi022 ceph-mon[29221]: Saving service prometheus spec with placement count:1 2024-02-16T19:19:57.124 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:56 smithi022 ceph-mon[29221]: from='mgr.14120 172.21.15.22:0/1425529966' entity='mgr.smithi022.gmdcch' 2024-02-16T19:19:57.125 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:56 smithi022 ceph-mon[29221]: from='mgr.14120 172.21.15.22:0/1425529966' entity='mgr.smithi022.gmdcch' 2024-02-16T19:19:58.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:58 smithi022 ceph-mon[29221]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:19:58.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:58 smithi022 ceph-mon[29221]: Saving service grafana spec with placement count:1 2024-02-16T19:19:58.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:58 smithi022 ceph-mon[29221]: from='mgr.14120 172.21.15.22:0/1425529966' entity='mgr.smithi022.gmdcch' 2024-02-16T19:19:58.520 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout Scheduled grafana update... 2024-02-16T19:19:58.520 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:56.977+0000 7f9f09f64700 1 Processor -- start 2024-02-16T19:19:58.520 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:56.977+0000 7f9f09f64700 1 -- start start 2024-02-16T19:19:58.520 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:56.977+0000 7f9f09f64700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f04104040 0x7f9f04104450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:19:58.520 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:56.977+0000 7f9f09f64700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f04104990 con 0x7f9f04104040 2024-02-16T19:19:58.521 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:56.978+0000 7f9f037fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f04104040 0x7f9f04104450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:19:58.521 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:56.978+0000 7f9f037fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f04104040 0x7f9f04104450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49318/0 (socket says 172.21.15.22:49318) 2024-02-16T19:19:58.521 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:56.978+0000 7f9f037fe700 1 -- 172.21.15.22:0/3721267700 learned_addr learned my addr 172.21.15.22:0/3721267700 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:19:58.521 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:56.978+0000 7f9f037fe700 1 -- 172.21.15.22:0/3721267700 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f04104ad0 con 0x7f9f04104040 2024-02-16T19:19:58.521 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:56.978+0000 7f9f037fe700 1 --2- 172.21.15.22:0/3721267700 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f04104040 0x7f9f04104450 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9eec0098a0 tx=0x7f9eec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f143f2b982dcdfbc server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:19:58.521 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:56.978+0000 7f9f027fc700 1 -- 172.21.15.22:0/3721267700 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9eec004180 con 0x7f9f04104040 2024-02-16T19:19:58.521 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:56.978+0000 7f9f027fc700 1 -- 172.21.15.22:0/3721267700 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9eec0042e0 con 0x7f9f04104040 2024-02-16T19:19:58.521 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:56.978+0000 7f9f027fc700 1 -- 172.21.15.22:0/3721267700 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9eec011af0 con 0x7f9f04104040 2024-02-16T19:19:58.521 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:56.979+0000 7f9f09f64700 1 -- 172.21.15.22:0/3721267700 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f04104040 msgr2=0x7f9f04104450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:19:58.521 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:56.979+0000 7f9f09f64700 1 --2- 172.21.15.22:0/3721267700 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f04104040 0x7f9f04104450 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9eec0098a0 tx=0x7f9eec009bb0 comp rx=0 tx=0).stop 2024-02-16T19:19:58.521 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:56.979+0000 7f9f09f64700 1 -- 172.21.15.22:0/3721267700 shutdown_connections 2024-02-16T19:19:58.521 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:56.979+0000 7f9f09f64700 1 --2- 172.21.15.22:0/3721267700 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f04104040 0x7f9f04104450 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:58.521 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:56.979+0000 7f9f09f64700 1 -- 172.21.15.22:0/3721267700 >> 172.21.15.22:0/3721267700 conn(0x7f9f040ff670 msgr2=0x7f9f04101ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:19:58.522 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:56.980+0000 7f9f09f64700 1 -- 172.21.15.22:0/3721267700 shutdown_connections 2024-02-16T19:19:58.522 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:56.980+0000 7f9f09f64700 1 -- 172.21.15.22:0/3721267700 wait complete. 2024-02-16T19:19:58.522 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:56.980+0000 7f9f09f64700 1 Processor -- start 2024-02-16T19:19:58.522 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:56.980+0000 7f9f09f64700 1 -- start start 2024-02-16T19:19:58.522 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:56.980+0000 7f9f09f64700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f04104040 0x7f9f04109070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:19:58.522 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:56.980+0000 7f9f09f64700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f0419c470 con 0x7f9f04104040 2024-02-16T19:19:58.522 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:56.981+0000 7f9f037fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f04104040 0x7f9f04109070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:19:58.522 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:56.981+0000 7f9f037fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f04104040 0x7f9f04109070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49324/0 (socket says 172.21.15.22:49324) 2024-02-16T19:19:58.522 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:56.981+0000 7f9f037fe700 1 -- 172.21.15.22:0/259441166 learned_addr learned my addr 172.21.15.22:0/259441166 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:19:58.522 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:56.981+0000 7f9f037fe700 1 -- 172.21.15.22:0/259441166 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eec009550 con 0x7f9f04104040 2024-02-16T19:19:58.522 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:56.981+0000 7f9f037fe700 1 --2- 172.21.15.22:0/259441166 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f04104040 0x7f9f04109070 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9eec011670 tx=0x7f9eec011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:19:58.522 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:56.981+0000 7f9f00ff9700 1 -- 172.21.15.22:0/259441166 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9eec003a40 con 0x7f9f04104040 2024-02-16T19:19:58.522 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:56.981+0000 7f9f00ff9700 1 -- 172.21.15.22:0/259441166 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9eec003ba0 con 0x7f9f04104040 2024-02-16T19:19:58.522 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:56.981+0000 7f9f00ff9700 1 -- 172.21.15.22:0/259441166 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9eec024790 con 0x7f9f04104040 2024-02-16T19:19:58.522 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:56.981+0000 7f9f09f64700 1 -- 172.21.15.22:0/259441166 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f0419c5b0 con 0x7f9f04104040 2024-02-16T19:19:58.522 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:56.981+0000 7f9f09f64700 1 -- 172.21.15.22:0/259441166 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f0419ca50 con 0x7f9f04104040 2024-02-16T19:19:58.523 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:56.982+0000 7f9f00ff9700 1 -- 172.21.15.22:0/259441166 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f9eec02d3f0 con 0x7f9f04104040 2024-02-16T19:19:58.523 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:56.982+0000 7f9f09f64700 1 -- 172.21.15.22:0/259441166 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f041056a0 con 0x7f9f04104040 2024-02-16T19:19:58.523 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:56.983+0000 7f9f00ff9700 1 --2- 172.21.15.22:0/259441166 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7f9ef0038440 0x7f9ef003a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:19:58.523 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:56.983+0000 7f9f00ff9700 1 -- 172.21.15.22:0/259441166 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f9eec04c450 con 0x7f9f04104040 2024-02-16T19:19:58.523 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:56.983+0000 7f9f02ffd700 1 --2- 172.21.15.22:0/259441166 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7f9ef0038440 0x7f9ef003a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:19:58.523 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:56.983+0000 7f9f02ffd700 1 --2- 172.21.15.22:0/259441166 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7f9ef0038440 0x7f9ef003a8f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9ef4006fd0 tx=0x7f9ef4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:19:58.523 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:56.988+0000 7f9f00ff9700 1 -- 172.21.15.22:0/259441166 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f9eec010bd0 con 0x7f9f04104040 2024-02-16T19:19:58.523 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:57.210+0000 7f9f09f64700 1 -- 172.21.15.22:0/259441166 --> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}) v1 -- 0x7f9f04100780 con 0x7f9ef0038440 2024-02-16T19:19:58.523 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:57.215+0000 7f9f00ff9700 1 -- 172.21.15.22:0/259441166 <== mgr.14120 v2:172.21.15.22:6800/2755289428 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f9f04100780 con 0x7f9ef0038440 2024-02-16T19:19:58.523 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:57.217+0000 7f9f09f64700 1 -- 172.21.15.22:0/259441166 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7f9ef0038440 msgr2=0x7f9ef003a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:19:58.523 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:57.217+0000 7f9f09f64700 1 --2- 172.21.15.22:0/259441166 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7f9ef0038440 0x7f9ef003a8f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9ef4006fd0 tx=0x7f9ef4006e40 comp rx=0 tx=0).stop 2024-02-16T19:19:58.523 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:57.217+0000 7f9f09f64700 1 -- 172.21.15.22:0/259441166 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f04104040 msgr2=0x7f9f04109070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:19:58.523 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:57.217+0000 7f9f09f64700 1 --2- 172.21.15.22:0/259441166 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f04104040 0x7f9f04109070 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9eec011670 tx=0x7f9eec011b50 comp rx=0 tx=0).stop 2024-02-16T19:19:58.523 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:57.217+0000 7f9f09f64700 1 -- 172.21.15.22:0/259441166 shutdown_connections 2024-02-16T19:19:58.524 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:57.217+0000 7f9f09f64700 1 --2- 172.21.15.22:0/259441166 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7f9ef0038440 0x7f9ef003a8f0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:58.524 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:57.217+0000 7f9f09f64700 1 --2- 172.21.15.22:0/259441166 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f04104040 0x7f9f04109070 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:58.524 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:57.217+0000 7f9f09f64700 1 -- 172.21.15.22:0/259441166 >> 172.21.15.22:0/259441166 conn(0x7f9f040ff670 msgr2=0x7f9f04100070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:19:58.524 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:57.217+0000 7f9f09f64700 1 -- 172.21.15.22:0/259441166 shutdown_connections 2024-02-16T19:19:58.524 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:57.217+0000 7f9f09f64700 1 -- 172.21.15.22:0/259441166 wait complete. 2024-02-16T19:19:58.524 INFO:teuthology.orchestra.run.smithi022.stdout:Deploying node-exporter service with default placement... 2024-02-16T19:19:59.810 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:59 smithi022 ceph-mon[29221]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:19:59.811 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:19:59 smithi022 ceph-mon[29221]: from='mgr.14120 172.21.15.22:0/1425529966' entity='mgr.smithi022.gmdcch' 2024-02-16T19:19:59.818 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout Scheduled node-exporter update... 2024-02-16T19:19:59.818 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:58.974+0000 7f2422b17700 1 Processor -- start 2024-02-16T19:19:59.818 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:58.974+0000 7f2422b17700 1 -- start start 2024-02-16T19:19:59.818 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:58.974+0000 7f2422b17700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f241c1059d0 0x7f241c105de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:19:59.818 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:58.974+0000 7f2422b17700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f241c106320 con 0x7f241c1059d0 2024-02-16T19:19:59.818 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:58.974+0000 7f24208b3700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f241c1059d0 0x7f241c105de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:19:59.818 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:58.974+0000 7f24208b3700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f241c1059d0 0x7f241c105de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49330/0 (socket says 172.21.15.22:49330) 2024-02-16T19:19:59.818 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:58.974+0000 7f24208b3700 1 -- 172.21.15.22:0/1343726613 learned_addr learned my addr 172.21.15.22:0/1343726613 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:19:59.819 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:58.974+0000 7f24208b3700 1 -- 172.21.15.22:0/1343726613 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f241c106460 con 0x7f241c1059d0 2024-02-16T19:19:59.819 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:58.975+0000 7f24208b3700 1 --2- 172.21.15.22:0/1343726613 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f241c1059d0 0x7f241c105de0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2410009870 tx=0x7f2410009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bbde50e511c5874 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:19:59.819 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:58.975+0000 7f241b7fe700 1 -- 172.21.15.22:0/1343726613 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2410005c30 con 0x7f241c1059d0 2024-02-16T19:19:59.819 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:58.975+0000 7f241b7fe700 1 -- 172.21.15.22:0/1343726613 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f24100044e0 con 0x7f241c1059d0 2024-02-16T19:19:59.819 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:58.975+0000 7f241b7fe700 1 -- 172.21.15.22:0/1343726613 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2410006030 con 0x7f241c1059d0 2024-02-16T19:19:59.819 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:58.975+0000 7f2422b17700 1 -- 172.21.15.22:0/1343726613 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f241c1059d0 msgr2=0x7f241c105de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:19:59.819 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:58.975+0000 7f2422b17700 1 --2- 172.21.15.22:0/1343726613 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f241c1059d0 0x7f241c105de0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2410009870 tx=0x7f2410009b80 comp rx=0 tx=0).stop 2024-02-16T19:19:59.819 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:58.975+0000 7f2422b17700 1 -- 172.21.15.22:0/1343726613 shutdown_connections 2024-02-16T19:19:59.819 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:58.975+0000 7f2422b17700 1 --2- 172.21.15.22:0/1343726613 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f241c1059d0 0x7f241c105de0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:59.819 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:58.975+0000 7f2422b17700 1 -- 172.21.15.22:0/1343726613 >> 172.21.15.22:0/1343726613 conn(0x7f241c101020 msgr2=0x7f241c103450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:19:59.819 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:58.975+0000 7f2422b17700 1 -- 172.21.15.22:0/1343726613 shutdown_connections 2024-02-16T19:19:59.819 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:58.975+0000 7f2422b17700 1 -- 172.21.15.22:0/1343726613 wait complete. 2024-02-16T19:19:59.819 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:58.976+0000 7f2422b17700 1 Processor -- start 2024-02-16T19:19:59.819 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:58.976+0000 7f2422b17700 1 -- start start 2024-02-16T19:19:59.820 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:58.976+0000 7f2422b17700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f241c1059d0 0x7f241c07bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:19:59.820 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:58.976+0000 7f2422b17700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f241c07c160 con 0x7f241c1059d0 2024-02-16T19:19:59.820 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:58.976+0000 7f24208b3700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f241c1059d0 0x7f241c07bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:19:59.820 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:58.976+0000 7f24208b3700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f241c1059d0 0x7f241c07bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49342/0 (socket says 172.21.15.22:49342) 2024-02-16T19:19:59.820 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:58.976+0000 7f24208b3700 1 -- 172.21.15.22:0/2198649373 learned_addr learned my addr 172.21.15.22:0/2198649373 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:19:59.820 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:58.977+0000 7f24208b3700 1 -- 172.21.15.22:0/2198649373 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2410009500 con 0x7f241c1059d0 2024-02-16T19:19:59.820 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:58.977+0000 7f24208b3700 1 --2- 172.21.15.22:0/2198649373 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f241c1059d0 0x7f241c07bc20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2410004840 tx=0x7f24100042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:19:59.820 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:58.977+0000 7f2419ffb700 1 -- 172.21.15.22:0/2198649373 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2410005e40 con 0x7f241c1059d0 2024-02-16T19:19:59.820 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:58.977+0000 7f2419ffb700 1 -- 172.21.15.22:0/2198649373 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2410005fa0 con 0x7f241c1059d0 2024-02-16T19:19:59.820 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:58.977+0000 7f2422b17700 1 -- 172.21.15.22:0/2198649373 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f241c07a270 con 0x7f241c1059d0 2024-02-16T19:19:59.820 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:58.977+0000 7f2419ffb700 1 -- 172.21.15.22:0/2198649373 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2410011c80 con 0x7f241c1059d0 2024-02-16T19:19:59.820 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:58.977+0000 7f2422b17700 1 -- 172.21.15.22:0/2198649373 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f241c07a710 con 0x7f241c1059d0 2024-02-16T19:19:59.820 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:58.978+0000 7f2419ffb700 1 -- 172.21.15.22:0/2198649373 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f2410018070 con 0x7f241c1059d0 2024-02-16T19:19:59.820 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:58.978+0000 7f2419ffb700 1 --2- 172.21.15.22:0/2198649373 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7f240c038400 0x7f240c03a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:19:59.821 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:58.978+0000 7f2422b17700 1 -- 172.21.15.22:0/2198649373 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f241c062380 con 0x7f241c1059d0 2024-02-16T19:19:59.821 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:58.978+0000 7f2419ffb700 1 -- 172.21.15.22:0/2198649373 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f241004ba70 con 0x7f241c1059d0 2024-02-16T19:19:59.821 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:58.978+0000 7f241bfff700 1 --2- 172.21.15.22:0/2198649373 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7f240c038400 0x7f240c03a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:19:59.821 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:58.979+0000 7f241bfff700 1 --2- 172.21.15.22:0/2198649373 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7f240c038400 0x7f240c03a8b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2408006fd0 tx=0x7f2408006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:19:59.821 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:58.983+0000 7f2419ffb700 1 -- 172.21.15.22:0/2198649373 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f241001f080 con 0x7f241c1059d0 2024-02-16T19:19:59.821 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:59.195+0000 7f2422b17700 1 -- 172.21.15.22:0/2198649373 --> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7f241c103300 con 0x7f240c038400 2024-02-16T19:19:59.821 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:59.199+0000 7f2419ffb700 1 -- 172.21.15.22:0/2198649373 <== mgr.14120 v2:172.21.15.22:6800/2755289428 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7f241c103300 con 0x7f240c038400 2024-02-16T19:19:59.821 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:59.201+0000 7f2422b17700 1 -- 172.21.15.22:0/2198649373 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7f240c038400 msgr2=0x7f240c03a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:19:59.821 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:59.201+0000 7f2422b17700 1 --2- 172.21.15.22:0/2198649373 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7f240c038400 0x7f240c03a8b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2408006fd0 tx=0x7f2408006e40 comp rx=0 tx=0).stop 2024-02-16T19:19:59.821 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:59.201+0000 7f2422b17700 1 -- 172.21.15.22:0/2198649373 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f241c1059d0 msgr2=0x7f241c07bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:19:59.821 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:59.201+0000 7f2422b17700 1 --2- 172.21.15.22:0/2198649373 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f241c1059d0 0x7f241c07bc20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2410004840 tx=0x7f24100042e0 comp rx=0 tx=0).stop 2024-02-16T19:19:59.821 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:59.202+0000 7f2422b17700 1 -- 172.21.15.22:0/2198649373 shutdown_connections 2024-02-16T19:19:59.821 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:59.202+0000 7f2422b17700 1 --2- 172.21.15.22:0/2198649373 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7f240c038400 0x7f240c03a8b0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:59.821 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:59.202+0000 7f2422b17700 1 --2- 172.21.15.22:0/2198649373 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f241c1059d0 0x7f241c07bc20 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:59.821 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:59.202+0000 7f2422b17700 1 -- 172.21.15.22:0/2198649373 >> 172.21.15.22:0/2198649373 conn(0x7f241c101020 msgr2=0x7f241c1026e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:19:59.822 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:59.202+0000 7f2422b17700 1 -- 172.21.15.22:0/2198649373 shutdown_connections 2024-02-16T19:19:59.822 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:19:59.202+0000 7f2422b17700 1 -- 172.21.15.22:0/2198649373 wait complete. 2024-02-16T19:19:59.822 INFO:teuthology.orchestra.run.smithi022.stdout:Deploying alertmanager service with default placement... 2024-02-16T19:20:00.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:00 smithi022 ceph-mon[29221]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:20:00.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:00 smithi022 ceph-mon[29221]: Saving service node-exporter spec with placement * 2024-02-16T19:20:00.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:00 smithi022 ceph-mon[29221]: from='mgr.14120 172.21.15.22:0/1425529966' entity='mgr.smithi022.gmdcch' 2024-02-16T19:20:01.070 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout Scheduled alertmanager update... 2024-02-16T19:20:01.075 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:00.256+0000 7f0e75ec6700 1 Processor -- start 2024-02-16T19:20:01.076 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:00.256+0000 7f0e75ec6700 1 -- start start 2024-02-16T19:20:01.076 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:00.256+0000 7f0e75ec6700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e70079cd0 0x7f0e7010a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:01.076 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:00.256+0000 7f0e75ec6700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e70106360 con 0x7f0e70079cd0 2024-02-16T19:20:01.076 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:00.256+0000 7f0e6f7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e70079cd0 0x7f0e7010a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:20:01.076 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:00.256+0000 7f0e6f7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e70079cd0 0x7f0e7010a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49358/0 (socket says 172.21.15.22:49358) 2024-02-16T19:20:01.076 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:00.256+0000 7f0e6f7fe700 1 -- 172.21.15.22:0/1732340575 learned_addr learned my addr 172.21.15.22:0/1732340575 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:20:01.076 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:00.257+0000 7f0e6f7fe700 1 -- 172.21.15.22:0/1732340575 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e7010aeb0 con 0x7f0e70079cd0 2024-02-16T19:20:01.076 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:00.257+0000 7f0e6f7fe700 1 --2- 172.21.15.22:0/1732340575 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e70079cd0 0x7f0e7010a970 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0e640098a0 tx=0x7f0e64009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aebb800874d2790 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:20:01.076 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:00.257+0000 7f0e6e7fc700 1 -- 172.21.15.22:0/1732340575 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0e64004180 con 0x7f0e70079cd0 2024-02-16T19:20:01.076 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:00.257+0000 7f0e6e7fc700 1 -- 172.21.15.22:0/1732340575 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0e640042e0 con 0x7f0e70079cd0 2024-02-16T19:20:01.076 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:00.257+0000 7f0e6e7fc700 1 -- 172.21.15.22:0/1732340575 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0e64011af0 con 0x7f0e70079cd0 2024-02-16T19:20:01.076 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:00.258+0000 7f0e75ec6700 1 -- 172.21.15.22:0/1732340575 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e70079cd0 msgr2=0x7f0e7010a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:01.077 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:00.258+0000 7f0e75ec6700 1 --2- 172.21.15.22:0/1732340575 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e70079cd0 0x7f0e7010a970 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0e640098a0 tx=0x7f0e64009bb0 comp rx=0 tx=0).stop 2024-02-16T19:20:01.077 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:00.258+0000 7f0e75ec6700 1 -- 172.21.15.22:0/1732340575 shutdown_connections 2024-02-16T19:20:01.077 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:00.258+0000 7f0e75ec6700 1 --2- 172.21.15.22:0/1732340575 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e70079cd0 0x7f0e7010a970 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:01.077 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:00.258+0000 7f0e75ec6700 1 -- 172.21.15.22:0/1732340575 >> 172.21.15.22:0/1732340575 conn(0x7f0e700fff00 msgr2=0x7f0e70102350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:20:01.077 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:00.258+0000 7f0e75ec6700 1 -- 172.21.15.22:0/1732340575 shutdown_connections 2024-02-16T19:20:01.077 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:00.258+0000 7f0e75ec6700 1 -- 172.21.15.22:0/1732340575 wait complete. 2024-02-16T19:20:01.077 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:00.258+0000 7f0e75ec6700 1 Processor -- start 2024-02-16T19:20:01.077 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:00.259+0000 7f0e75ec6700 1 -- start start 2024-02-16T19:20:01.077 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:00.259+0000 7f0e75ec6700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e7019b920 0x7f0e7019bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:01.077 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:00.259+0000 7f0e75ec6700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7019c270 con 0x7f0e7019b920 2024-02-16T19:20:01.077 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:00.259+0000 7f0e6f7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e7019b920 0x7f0e7019bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:20:01.077 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:00.259+0000 7f0e6f7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e7019b920 0x7f0e7019bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49368/0 (socket says 172.21.15.22:49368) 2024-02-16T19:20:01.077 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:00.259+0000 7f0e6f7fe700 1 -- 172.21.15.22:0/876710597 learned_addr learned my addr 172.21.15.22:0/876710597 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:20:01.078 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:00.259+0000 7f0e6f7fe700 1 -- 172.21.15.22:0/876710597 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e64009550 con 0x7f0e7019b920 2024-02-16T19:20:01.078 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:00.259+0000 7f0e6f7fe700 1 --2- 172.21.15.22:0/876710597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e7019b920 0x7f0e7019bd30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0e64011670 tx=0x7f0e64011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:20:01.078 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:00.259+0000 7f0e6cff9700 1 -- 172.21.15.22:0/876710597 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0e640039f0 con 0x7f0e7019b920 2024-02-16T19:20:01.078 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:00.259+0000 7f0e6cff9700 1 -- 172.21.15.22:0/876710597 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0e64003b50 con 0x7f0e7019b920 2024-02-16T19:20:01.078 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:00.259+0000 7f0e6cff9700 1 -- 172.21.15.22:0/876710597 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0e640247c0 con 0x7f0e7019b920 2024-02-16T19:20:01.078 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:00.259+0000 7f0e75ec6700 1 -- 172.21.15.22:0/876710597 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e7019c470 con 0x7f0e7019b920 2024-02-16T19:20:01.078 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:00.260+0000 7f0e75ec6700 1 -- 172.21.15.22:0/876710597 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e7019f0d0 con 0x7f0e7019b920 2024-02-16T19:20:01.078 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:00.260+0000 7f0e75ec6700 1 -- 172.21.15.22:0/876710597 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e70079cd0 con 0x7f0e7019b920 2024-02-16T19:20:01.078 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:00.260+0000 7f0e6cff9700 1 -- 172.21.15.22:0/876710597 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f0e6401ac00 con 0x7f0e7019b920 2024-02-16T19:20:01.078 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:00.261+0000 7f0e6cff9700 1 --2- 172.21.15.22:0/876710597 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7f0e58038440 0x7f0e5803a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:01.078 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:00.261+0000 7f0e6cff9700 1 -- 172.21.15.22:0/876710597 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f0e6404c3b0 con 0x7f0e7019b920 2024-02-16T19:20:01.078 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:00.261+0000 7f0e6effd700 1 --2- 172.21.15.22:0/876710597 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7f0e58038440 0x7f0e5803a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:20:01.078 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:00.261+0000 7f0e6effd700 1 --2- 172.21.15.22:0/876710597 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7f0e58038440 0x7f0e5803a8f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0e60006fd0 tx=0x7f0e60006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:20:01.078 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:00.265+0000 7f0e6cff9700 1 -- 172.21.15.22:0/876710597 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f0e6401a430 con 0x7f0e7019b920 2024-02-16T19:20:01.079 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:00.479+0000 7f0e75ec6700 1 -- 172.21.15.22:0/876710597 --> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}) v1 -- 0x7f0e701021e0 con 0x7f0e58038440 2024-02-16T19:20:01.079 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:00.483+0000 7f0e6cff9700 1 -- 172.21.15.22:0/876710597 <== mgr.14120 v2:172.21.15.22:6800/2755289428 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+33 (secure 0 0 0) 0x7f0e701021e0 con 0x7f0e58038440 2024-02-16T19:20:01.079 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:00.485+0000 7f0e75ec6700 1 -- 172.21.15.22:0/876710597 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7f0e58038440 msgr2=0x7f0e5803a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:01.079 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:00.485+0000 7f0e75ec6700 1 --2- 172.21.15.22:0/876710597 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7f0e58038440 0x7f0e5803a8f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0e60006fd0 tx=0x7f0e60006e40 comp rx=0 tx=0).stop 2024-02-16T19:20:01.079 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:00.485+0000 7f0e75ec6700 1 -- 172.21.15.22:0/876710597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e7019b920 msgr2=0x7f0e7019bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:01.079 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:00.485+0000 7f0e75ec6700 1 --2- 172.21.15.22:0/876710597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e7019b920 0x7f0e7019bd30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0e64011670 tx=0x7f0e64011b50 comp rx=0 tx=0).stop 2024-02-16T19:20:01.079 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:00.485+0000 7f0e75ec6700 1 -- 172.21.15.22:0/876710597 shutdown_connections 2024-02-16T19:20:01.079 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:00.485+0000 7f0e75ec6700 1 --2- 172.21.15.22:0/876710597 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7f0e58038440 0x7f0e5803a8f0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:01.079 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:00.485+0000 7f0e75ec6700 1 --2- 172.21.15.22:0/876710597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e7019b920 0x7f0e7019bd30 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:01.079 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:00.485+0000 7f0e75ec6700 1 -- 172.21.15.22:0/876710597 >> 172.21.15.22:0/876710597 conn(0x7f0e700fff00 msgr2=0x7f0e701015c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:20:01.079 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:00.485+0000 7f0e75ec6700 1 -- 172.21.15.22:0/876710597 shutdown_connections 2024-02-16T19:20:01.079 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:00.485+0000 7f0e75ec6700 1 -- 172.21.15.22:0/876710597 wait complete. 2024-02-16T19:20:01.705 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:01 smithi022 ceph-mon[29221]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:20:01.705 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:01 smithi022 ceph-mon[29221]: Saving service alertmanager spec with placement count:1 2024-02-16T19:20:01.705 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:01 smithi022 ceph-mon[29221]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:20:02.264 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:01.495+0000 7ff9c740c700 1 Processor -- start 2024-02-16T19:20:02.264 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:01.495+0000 7ff9c740c700 1 -- start start 2024-02-16T19:20:02.264 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:01.495+0000 7ff9c740c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9c0106520 0x7ff9c0108900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:02.264 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:01.495+0000 7ff9c740c700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c0079cf0 con 0x7ff9c0106520 2024-02-16T19:20:02.264 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:01.496+0000 7ff9c51a8700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9c0106520 0x7ff9c0108900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:20:02.264 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:01.496+0000 7ff9c51a8700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9c0106520 0x7ff9c0108900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49374/0 (socket says 172.21.15.22:49374) 2024-02-16T19:20:02.264 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:01.496+0000 7ff9c51a8700 1 -- 172.21.15.22:0/1982234907 learned_addr learned my addr 172.21.15.22:0/1982234907 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:20:02.264 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:01.496+0000 7ff9c51a8700 1 -- 172.21.15.22:0/1982234907 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9c0079e30 con 0x7ff9c0106520 2024-02-16T19:20:02.264 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:01.496+0000 7ff9c51a8700 1 --2- 172.21.15.22:0/1982234907 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9c0106520 0x7ff9c0108900 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8009e80 tx=0x7ff9b800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=600f9833bf66042a server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:20:02.265 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:01.496+0000 7ff9bffff700 1 -- 172.21.15.22:0/1982234907 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff9b800e070 con 0x7ff9c0106520 2024-02-16T19:20:02.265 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:01.496+0000 7ff9bffff700 1 -- 172.21.15.22:0/1982234907 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff9b800b8c0 con 0x7ff9c0106520 2024-02-16T19:20:02.265 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:01.496+0000 7ff9bffff700 1 -- 172.21.15.22:0/1982234907 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff9b80044d0 con 0x7ff9c0106520 2024-02-16T19:20:02.265 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:01.497+0000 7ff9c740c700 1 -- 172.21.15.22:0/1982234907 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9c0106520 msgr2=0x7ff9c0108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:02.265 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:01.497+0000 7ff9c740c700 1 --2- 172.21.15.22:0/1982234907 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9c0106520 0x7ff9c0108900 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8009e80 tx=0x7ff9b800b330 comp rx=0 tx=0).stop 2024-02-16T19:20:02.265 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:01.497+0000 7ff9c740c700 1 -- 172.21.15.22:0/1982234907 shutdown_connections 2024-02-16T19:20:02.265 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:01.497+0000 7ff9c740c700 1 --2- 172.21.15.22:0/1982234907 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9c0106520 0x7ff9c0108900 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:02.265 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:01.497+0000 7ff9c740c700 1 -- 172.21.15.22:0/1982234907 >> 172.21.15.22:0/1982234907 conn(0x7ff9c00ffee0 msgr2=0x7ff9c0102330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:20:02.265 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:01.497+0000 7ff9c740c700 1 -- 172.21.15.22:0/1982234907 shutdown_connections 2024-02-16T19:20:02.265 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:01.497+0000 7ff9c740c700 1 -- 172.21.15.22:0/1982234907 wait complete. 2024-02-16T19:20:02.265 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:01.498+0000 7ff9c740c700 1 Processor -- start 2024-02-16T19:20:02.265 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:01.498+0000 7ff9c740c700 1 -- start start 2024-02-16T19:20:02.265 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:01.498+0000 7ff9c740c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9c0106520 0x7ff9c0105e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:02.265 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:01.498+0000 7ff9c740c700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c01063c0 con 0x7ff9c0106520 2024-02-16T19:20:02.265 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:01.498+0000 7ff9c51a8700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9c0106520 0x7ff9c0105e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:20:02.265 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:01.498+0000 7ff9c51a8700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9c0106520 0x7ff9c0105e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49382/0 (socket says 172.21.15.22:49382) 2024-02-16T19:20:02.266 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:01.498+0000 7ff9c51a8700 1 -- 172.21.15.22:0/29900496 learned_addr learned my addr 172.21.15.22:0/29900496 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:20:02.266 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:01.498+0000 7ff9c51a8700 1 -- 172.21.15.22:0/29900496 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9b8009b30 con 0x7ff9c0106520 2024-02-16T19:20:02.266 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:01.498+0000 7ff9c51a8700 1 --2- 172.21.15.22:0/29900496 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9c0106520 0x7ff9c0105e80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8004940 tx=0x7ff9b8004a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:20:02.266 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:01.498+0000 7ff9be7fc700 1 -- 172.21.15.22:0/29900496 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff9b800e040 con 0x7ff9c0106520 2024-02-16T19:20:02.266 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:01.498+0000 7ff9be7fc700 1 -- 172.21.15.22:0/29900496 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff9b800bc40 con 0x7ff9c0106520 2024-02-16T19:20:02.266 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:01.498+0000 7ff9c740c700 1 -- 172.21.15.22:0/29900496 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9c01044d0 con 0x7ff9c0106520 2024-02-16T19:20:02.266 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:01.498+0000 7ff9be7fc700 1 -- 172.21.15.22:0/29900496 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff9b8012670 con 0x7ff9c0106520 2024-02-16T19:20:02.266 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:01.498+0000 7ff9c740c700 1 -- 172.21.15.22:0/29900496 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9c01049d0 con 0x7ff9c0106520 2024-02-16T19:20:02.266 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:01.499+0000 7ff9be7fc700 1 -- 172.21.15.22:0/29900496 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7ff9b8019040 con 0x7ff9c0106520 2024-02-16T19:20:02.266 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:01.499+0000 7ff9c740c700 1 -- 172.21.15.22:0/29900496 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9c0109aa0 con 0x7ff9c0106520 2024-02-16T19:20:02.266 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:01.499+0000 7ff9be7fc700 1 --2- 172.21.15.22:0/29900496 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7ff9a8038440 0x7ff9a803a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:02.266 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:01.499+0000 7ff9be7fc700 1 -- 172.21.15.22:0/29900496 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7ff9b804bba0 con 0x7ff9c0106520 2024-02-16T19:20:02.266 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:01.500+0000 7ff9c49a7700 1 --2- 172.21.15.22:0/29900496 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7ff9a8038440 0x7ff9a803a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:20:02.266 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:01.500+0000 7ff9c49a7700 1 --2- 172.21.15.22:0/29900496 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7ff9a8038440 0x7ff9a803a8f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4006fd0 tx=0x7ff9b4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:20:02.266 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:01.504+0000 7ff9be7fc700 1 -- 172.21.15.22:0/29900496 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7ff9b800bdb0 con 0x7ff9c0106520 2024-02-16T19:20:02.266 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:01.703+0000 7ff9c740c700 1 -- 172.21.15.22:0/29900496 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7ff9c0109cc0 con 0x7ff9c0106520 2024-02-16T19:20:02.267 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:01.707+0000 7ff9be7fc700 1 -- 172.21.15.22:0/29900496 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7ff9b801f020 con 0x7ff9c0106520 2024-02-16T19:20:02.267 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:01.709+0000 7ff9c740c700 1 -- 172.21.15.22:0/29900496 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7ff9a8038440 msgr2=0x7ff9a803a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:02.267 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:01.709+0000 7ff9c740c700 1 --2- 172.21.15.22:0/29900496 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7ff9a8038440 0x7ff9a803a8f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4006fd0 tx=0x7ff9b4006e40 comp rx=0 tx=0).stop 2024-02-16T19:20:02.267 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:01.709+0000 7ff9c740c700 1 -- 172.21.15.22:0/29900496 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9c0106520 msgr2=0x7ff9c0105e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:02.267 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:01.709+0000 7ff9c740c700 1 --2- 172.21.15.22:0/29900496 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9c0106520 0x7ff9c0105e80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8004940 tx=0x7ff9b8004a20 comp rx=0 tx=0).stop 2024-02-16T19:20:02.267 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:01.709+0000 7ff9c740c700 1 -- 172.21.15.22:0/29900496 shutdown_connections 2024-02-16T19:20:02.267 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:01.709+0000 7ff9c740c700 1 --2- 172.21.15.22:0/29900496 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7ff9a8038440 0x7ff9a803a8f0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:02.267 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:01.709+0000 7ff9c740c700 1 --2- 172.21.15.22:0/29900496 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9c0106520 0x7ff9c0105e80 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:02.267 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:01.709+0000 7ff9c740c700 1 -- 172.21.15.22:0/29900496 >> 172.21.15.22:0/29900496 conn(0x7ff9c00ffee0 msgr2=0x7ff9c01015c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:20:02.267 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:01.709+0000 7ff9c740c700 1 -- 172.21.15.22:0/29900496 shutdown_connections 2024-02-16T19:20:02.267 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:01.709+0000 7ff9c740c700 1 -- 172.21.15.22:0/29900496 wait complete. 2024-02-16T19:20:02.964 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:02 smithi022 ceph-mon[29221]: from='client.? 172.21.15.22:0/29900496' entity='client.admin' 2024-02-16T19:20:03.495 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:02.714+0000 7effd0709700 1 Processor -- start 2024-02-16T19:20:03.495 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:02.714+0000 7effd0709700 1 -- start start 2024-02-16T19:20:03.495 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:02.715+0000 7effd0709700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7effc81059d0 0x7effc8105de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:03.495 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:02.715+0000 7effd0709700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc8106320 con 0x7effc81059d0 2024-02-16T19:20:03.495 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:02.715+0000 7effce4a5700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7effc81059d0 0x7effc8105de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:20:03.496 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:02.715+0000 7effce4a5700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7effc81059d0 0x7effc8105de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49392/0 (socket says 172.21.15.22:49392) 2024-02-16T19:20:03.496 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:02.715+0000 7effce4a5700 1 -- 172.21.15.22:0/602109684 learned_addr learned my addr 172.21.15.22:0/602109684 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:20:03.496 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:02.715+0000 7effce4a5700 1 -- 172.21.15.22:0/602109684 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effc8106460 con 0x7effc81059d0 2024-02-16T19:20:03.496 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:02.715+0000 7effce4a5700 1 --2- 172.21.15.22:0/602109684 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7effc81059d0 0x7effc8105de0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7effc4009870 tx=0x7effc4009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60bf4e505c613e9b server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:20:03.496 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:02.716+0000 7effcd4a3700 1 -- 172.21.15.22:0/602109684 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7effc4005c30 con 0x7effc81059d0 2024-02-16T19:20:03.496 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:02.716+0000 7effcd4a3700 1 -- 172.21.15.22:0/602109684 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7effc40044e0 con 0x7effc81059d0 2024-02-16T19:20:03.496 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:02.716+0000 7effcd4a3700 1 -- 172.21.15.22:0/602109684 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7effc4006030 con 0x7effc81059d0 2024-02-16T19:20:03.496 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:02.716+0000 7effd0709700 1 -- 172.21.15.22:0/602109684 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7effc81059d0 msgr2=0x7effc8105de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:03.496 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:02.716+0000 7effd0709700 1 --2- 172.21.15.22:0/602109684 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7effc81059d0 0x7effc8105de0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7effc4009870 tx=0x7effc4009b80 comp rx=0 tx=0).stop 2024-02-16T19:20:03.496 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:02.716+0000 7effd0709700 1 -- 172.21.15.22:0/602109684 shutdown_connections 2024-02-16T19:20:03.496 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:02.716+0000 7effd0709700 1 --2- 172.21.15.22:0/602109684 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7effc81059d0 0x7effc8105de0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:03.496 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:02.716+0000 7effd0709700 1 -- 172.21.15.22:0/602109684 >> 172.21.15.22:0/602109684 conn(0x7effc8101020 msgr2=0x7effc8103450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:20:03.496 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:02.716+0000 7effd0709700 1 -- 172.21.15.22:0/602109684 shutdown_connections 2024-02-16T19:20:03.496 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:02.716+0000 7effd0709700 1 -- 172.21.15.22:0/602109684 wait complete. 2024-02-16T19:20:03.497 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:02.717+0000 7effd0709700 1 Processor -- start 2024-02-16T19:20:03.497 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:02.717+0000 7effd0709700 1 -- start start 2024-02-16T19:20:03.497 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:02.717+0000 7effd0709700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7effc81059d0 0x7effc8199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:03.497 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:02.717+0000 7effd0709700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc8199c60 con 0x7effc81059d0 2024-02-16T19:20:03.497 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:02.717+0000 7effce4a5700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7effc81059d0 0x7effc8199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:20:03.497 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:02.717+0000 7effce4a5700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7effc81059d0 0x7effc8199720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49400/0 (socket says 172.21.15.22:49400) 2024-02-16T19:20:03.497 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:02.717+0000 7effce4a5700 1 -- 172.21.15.22:0/2618891585 learned_addr learned my addr 172.21.15.22:0/2618891585 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:20:03.497 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:02.718+0000 7effce4a5700 1 -- 172.21.15.22:0/2618891585 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effc4009500 con 0x7effc81059d0 2024-02-16T19:20:03.497 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:02.718+0000 7effce4a5700 1 --2- 172.21.15.22:0/2618891585 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7effc81059d0 0x7effc8199720 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7effc4004840 tx=0x7effc40042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:20:03.497 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:02.718+0000 7effbb7fe700 1 -- 172.21.15.22:0/2618891585 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7effc4005e40 con 0x7effc81059d0 2024-02-16T19:20:03.497 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:02.718+0000 7effbb7fe700 1 -- 172.21.15.22:0/2618891585 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7effc4005fa0 con 0x7effc81059d0 2024-02-16T19:20:03.497 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:02.718+0000 7effd0709700 1 -- 172.21.15.22:0/2618891585 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effc8199e60 con 0x7effc81059d0 2024-02-16T19:20:03.497 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:02.718+0000 7effbb7fe700 1 -- 172.21.15.22:0/2618891585 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7effc4011c80 con 0x7effc81059d0 2024-02-16T19:20:03.498 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:02.718+0000 7effd0709700 1 -- 172.21.15.22:0/2618891585 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effc819a300 con 0x7effc81059d0 2024-02-16T19:20:03.498 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:02.719+0000 7effbb7fe700 1 -- 172.21.15.22:0/2618891585 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7effc4018070 con 0x7effc81059d0 2024-02-16T19:20:03.498 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:02.719+0000 7effbb7fe700 1 --2- 172.21.15.22:0/2618891585 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7effb4038400 0x7effb403a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:03.498 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:02.719+0000 7effbb7fe700 1 -- 172.21.15.22:0/2618891585 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7effc404bb40 con 0x7effc81059d0 2024-02-16T19:20:03.498 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:02.719+0000 7effd0709700 1 -- 172.21.15.22:0/2618891585 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effc8062380 con 0x7effc81059d0 2024-02-16T19:20:03.498 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:02.719+0000 7effcdca4700 1 --2- 172.21.15.22:0/2618891585 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7effb4038400 0x7effb403a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:20:03.498 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:02.720+0000 7effcdca4700 1 --2- 172.21.15.22:0/2618891585 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7effb4038400 0x7effb403a8b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7effbc006fd0 tx=0x7effbc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:20:03.498 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:02.724+0000 7effbb7fe700 1 -- 172.21.15.22:0/2618891585 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7effc401f020 con 0x7effc81059d0 2024-02-16T19:20:03.498 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:02.923+0000 7effd0709700 1 -- 172.21.15.22:0/2618891585 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7effc8107880 con 0x7effc81059d0 2024-02-16T19:20:03.498 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:02.926+0000 7effbb7fe700 1 -- 172.21.15.22:0/2618891585 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7effc40330c0 con 0x7effc81059d0 2024-02-16T19:20:03.498 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:02.928+0000 7effd0709700 1 -- 172.21.15.22:0/2618891585 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7effb4038400 msgr2=0x7effb403a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:03.498 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:02.928+0000 7effd0709700 1 --2- 172.21.15.22:0/2618891585 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7effb4038400 0x7effb403a8b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7effbc006fd0 tx=0x7effbc006e40 comp rx=0 tx=0).stop 2024-02-16T19:20:03.498 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:02.928+0000 7effd0709700 1 -- 172.21.15.22:0/2618891585 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7effc81059d0 msgr2=0x7effc8199720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:03.498 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:02.928+0000 7effd0709700 1 --2- 172.21.15.22:0/2618891585 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7effc81059d0 0x7effc8199720 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7effc4004840 tx=0x7effc40042e0 comp rx=0 tx=0).stop 2024-02-16T19:20:03.498 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:02.928+0000 7effd0709700 1 -- 172.21.15.22:0/2618891585 shutdown_connections 2024-02-16T19:20:03.499 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:02.928+0000 7effd0709700 1 --2- 172.21.15.22:0/2618891585 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7effb4038400 0x7effb403a8b0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:03.499 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:02.928+0000 7effd0709700 1 --2- 172.21.15.22:0/2618891585 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7effc81059d0 0x7effc8199720 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:03.499 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:02.928+0000 7effd0709700 1 -- 172.21.15.22:0/2618891585 >> 172.21.15.22:0/2618891585 conn(0x7effc8101020 msgr2=0x7effc8101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:20:03.499 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:02.928+0000 7effd0709700 1 -- 172.21.15.22:0/2618891585 shutdown_connections 2024-02-16T19:20:03.499 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:02.929+0000 7effd0709700 1 -- 172.21.15.22:0/2618891585 wait complete. 2024-02-16T19:20:03.499 INFO:teuthology.orchestra.run.smithi022.stdout:Enabling the dashboard module... 2024-02-16T19:20:04.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:03 smithi022 ceph-mon[29221]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:20:04.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:03 smithi022 ceph-mon[29221]: from='client.? 172.21.15.22:0/2618891585' entity='client.admin' 2024-02-16T19:20:05.085 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:04 smithi022 ceph-mon[29221]: from='client.? 172.21.15.22:0/1834881700' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-02-16T19:20:05.377 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:03.970+0000 7face08a4700 1 Processor -- start 2024-02-16T19:20:05.377 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:03.970+0000 7face08a4700 1 -- start start 2024-02-16T19:20:05.377 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:03.970+0000 7face08a4700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7facdc1064d0 0x7facdc1088b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:05.377 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:03.970+0000 7face08a4700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facdc079d50 con 0x7facdc1064d0 2024-02-16T19:20:05.377 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:03.970+0000 7facda59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7facdc1064d0 0x7facdc1088b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:20:05.377 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:03.970+0000 7facda59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7facdc1064d0 0x7facdc1088b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49412/0 (socket says 172.21.15.22:49412) 2024-02-16T19:20:05.377 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:03.970+0000 7facda59c700 1 -- 172.21.15.22:0/3403211721 learned_addr learned my addr 172.21.15.22:0/3403211721 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:20:05.377 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:03.970+0000 7facda59c700 1 -- 172.21.15.22:0/3403211721 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facdc079e90 con 0x7facdc1064d0 2024-02-16T19:20:05.377 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:03.971+0000 7facda59c700 1 --2- 172.21.15.22:0/3403211721 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7facdc1064d0 0x7facdc1088b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7faccc0098a0 tx=0x7faccc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97af71639fbf1865 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:20:05.378 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:03.971+0000 7facd959a700 1 -- 172.21.15.22:0/3403211721 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faccc004180 con 0x7facdc1064d0 2024-02-16T19:20:05.378 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:03.971+0000 7facd959a700 1 -- 172.21.15.22:0/3403211721 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7faccc0042e0 con 0x7facdc1064d0 2024-02-16T19:20:05.378 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:03.971+0000 7facd959a700 1 -- 172.21.15.22:0/3403211721 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faccc011af0 con 0x7facdc1064d0 2024-02-16T19:20:05.378 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:03.971+0000 7face08a4700 1 -- 172.21.15.22:0/3403211721 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7facdc1064d0 msgr2=0x7facdc1088b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:05.378 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:03.971+0000 7face08a4700 1 --2- 172.21.15.22:0/3403211721 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7facdc1064d0 0x7facdc1088b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7faccc0098a0 tx=0x7faccc009bb0 comp rx=0 tx=0).stop 2024-02-16T19:20:05.378 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:03.971+0000 7face08a4700 1 -- 172.21.15.22:0/3403211721 shutdown_connections 2024-02-16T19:20:05.378 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:03.971+0000 7face08a4700 1 --2- 172.21.15.22:0/3403211721 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7facdc1064d0 0x7facdc1088b0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:05.378 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:03.971+0000 7face08a4700 1 -- 172.21.15.22:0/3403211721 >> 172.21.15.22:0/3403211721 conn(0x7facdc0ffed0 msgr2=0x7facdc102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:20:05.378 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:03.971+0000 7face08a4700 1 -- 172.21.15.22:0/3403211721 shutdown_connections 2024-02-16T19:20:05.378 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:03.971+0000 7face08a4700 1 -- 172.21.15.22:0/3403211721 wait complete. 2024-02-16T19:20:05.378 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:03.972+0000 7face08a4700 1 Processor -- start 2024-02-16T19:20:05.378 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:03.972+0000 7face08a4700 1 -- start start 2024-02-16T19:20:05.378 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:03.972+0000 7face08a4700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7facdc1064d0 0x7facdc199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:05.378 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:03.972+0000 7face08a4700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facdc199ca0 con 0x7facdc1064d0 2024-02-16T19:20:05.379 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:03.972+0000 7facda59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7facdc1064d0 0x7facdc199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:20:05.379 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:03.972+0000 7facda59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7facdc1064d0 0x7facdc199760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49420/0 (socket says 172.21.15.22:49420) 2024-02-16T19:20:05.379 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:03.972+0000 7facda59c700 1 -- 172.21.15.22:0/1834881700 learned_addr learned my addr 172.21.15.22:0/1834881700 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:20:05.379 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:03.973+0000 7facda59c700 1 -- 172.21.15.22:0/1834881700 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faccc009550 con 0x7facdc1064d0 2024-02-16T19:20:05.379 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:03.973+0000 7facda59c700 1 --2- 172.21.15.22:0/1834881700 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7facdc1064d0 0x7facdc199760 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7faccc011670 tx=0x7faccc011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:20:05.379 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:03.973+0000 7faccb7fe700 1 -- 172.21.15.22:0/1834881700 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faccc0039f0 con 0x7facdc1064d0 2024-02-16T19:20:05.379 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:03.973+0000 7faccb7fe700 1 -- 172.21.15.22:0/1834881700 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7faccc003b50 con 0x7facdc1064d0 2024-02-16T19:20:05.379 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:03.973+0000 7face08a4700 1 -- 172.21.15.22:0/1834881700 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facdc199ea0 con 0x7facdc1064d0 2024-02-16T19:20:05.379 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:03.973+0000 7faccb7fe700 1 -- 172.21.15.22:0/1834881700 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faccc0247c0 con 0x7facdc1064d0 2024-02-16T19:20:05.379 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:03.973+0000 7face08a4700 1 -- 172.21.15.22:0/1834881700 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facdc19a340 con 0x7facdc1064d0 2024-02-16T19:20:05.379 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:03.974+0000 7faccb7fe700 1 -- 172.21.15.22:0/1834881700 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7faccc02d3f0 con 0x7facdc1064d0 2024-02-16T19:20:05.379 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:03.974+0000 7face08a4700 1 -- 172.21.15.22:0/1834881700 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facdc109ab0 con 0x7facdc1064d0 2024-02-16T19:20:05.379 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:03.974+0000 7faccb7fe700 1 --2- 172.21.15.22:0/1834881700 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7facc4038440 0x7facc403a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:05.379 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:03.974+0000 7faccb7fe700 1 -- 172.21.15.22:0/1834881700 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7faccc04c370 con 0x7facdc1064d0 2024-02-16T19:20:05.379 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:03.974+0000 7facd9d9b700 1 --2- 172.21.15.22:0/1834881700 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7facc4038440 0x7facc403a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:20:05.380 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:03.974+0000 7facd9d9b700 1 --2- 172.21.15.22:0/1834881700 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7facc4038440 0x7facc403a8f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7facd0006fd0 tx=0x7facd0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:20:05.380 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:03.979+0000 7faccb7fe700 1 -- 172.21.15.22:0/1834881700 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7faccc010970 con 0x7facdc1064d0 2024-02-16T19:20:05.380 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:04.235+0000 7face08a4700 1 -- 172.21.15.22:0/1834881700 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7facdc109cd0 con 0x7facdc1064d0 2024-02-16T19:20:05.380 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:04.763+0000 7faccb7fe700 1 -- 172.21.15.22:0/1834881700 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mgrmap(e 9) v1 ==== 45097+0+0 (secure 0 0 0) 0x7faccc01ac00 con 0x7facdc1064d0 2024-02-16T19:20:05.380 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:04.764+0000 7faccb7fe700 1 -- 172.21.15.22:0/1834881700 <== mon.0 v2:172.21.15.22:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7faccc016de0 con 0x7facdc1064d0 2024-02-16T19:20:05.380 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:04.767+0000 7face08a4700 1 -- 172.21.15.22:0/1834881700 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7facc4038440 msgr2=0x7facc403a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:05.380 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:04.767+0000 7face08a4700 1 --2- 172.21.15.22:0/1834881700 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7facc4038440 0x7facc403a8f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7facd0006fd0 tx=0x7facd0006e40 comp rx=0 tx=0).stop 2024-02-16T19:20:05.380 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:04.767+0000 7face08a4700 1 -- 172.21.15.22:0/1834881700 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7facdc1064d0 msgr2=0x7facdc199760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:05.380 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:04.767+0000 7face08a4700 1 --2- 172.21.15.22:0/1834881700 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7facdc1064d0 0x7facdc199760 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7faccc011670 tx=0x7faccc011b50 comp rx=0 tx=0).stop 2024-02-16T19:20:05.380 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:04.767+0000 7face08a4700 1 -- 172.21.15.22:0/1834881700 shutdown_connections 2024-02-16T19:20:05.380 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:04.767+0000 7face08a4700 1 --2- 172.21.15.22:0/1834881700 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7facc4038440 0x7facc403a8f0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:05.380 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:04.767+0000 7face08a4700 1 --2- 172.21.15.22:0/1834881700 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7facdc1064d0 0x7facdc199760 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:05.380 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:04.767+0000 7face08a4700 1 -- 172.21.15.22:0/1834881700 >> 172.21.15.22:0/1834881700 conn(0x7facdc0ffed0 msgr2=0x7facdc1015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:20:05.380 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:04.767+0000 7face08a4700 1 -- 172.21.15.22:0/1834881700 shutdown_connections 2024-02-16T19:20:05.380 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:04.767+0000 7face08a4700 1 -- 172.21.15.22:0/1834881700 wait complete. 2024-02-16T19:20:05.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:05 smithi022 ceph-mon[29221]: from='client.? 172.21.15.22:0/1834881700' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-02-16T19:20:05.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:05 smithi022 ceph-mon[29221]: mgrmap e9: smithi022.gmdcch(active, since 26s) 2024-02-16T19:20:06.726 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout { 2024-02-16T19:20:06.726 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-02-16T19:20:06.726 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-16T19:20:06.726 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "active_name": "smithi022.gmdcch", 2024-02-16T19:20:06.726 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-02-16T19:20:06.726 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout } 2024-02-16T19:20:06.727 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:05.830+0000 7f599552e700 1 Processor -- start 2024-02-16T19:20:06.727 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:05.830+0000 7f599552e700 1 -- start start 2024-02-16T19:20:06.727 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:05.830+0000 7f599552e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f59901048b0 0x7f5990104cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:06.727 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:05.830+0000 7f599552e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5990105200 con 0x7f59901048b0 2024-02-16T19:20:06.727 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:05.830+0000 7f598effd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f59901048b0 0x7f5990104cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:20:06.727 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:05.830+0000 7f598effd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f59901048b0 0x7f5990104cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42894/0 (socket says 172.21.15.22:42894) 2024-02-16T19:20:06.727 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:05.830+0000 7f598effd700 1 -- 172.21.15.22:0/1455319842 learned_addr learned my addr 172.21.15.22:0/1455319842 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:20:06.727 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:05.831+0000 7f598effd700 1 -- 172.21.15.22:0/1455319842 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5990105340 con 0x7f59901048b0 2024-02-16T19:20:06.727 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:05.831+0000 7f598effd700 1 --2- 172.21.15.22:0/1455319842 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f59901048b0 0x7f5990104cc0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f59840098a0 tx=0x7f5984009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be296266a553f706 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:20:06.727 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:05.831+0000 7f598dffb700 1 -- 172.21.15.22:0/1455319842 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5984004180 con 0x7f59901048b0 2024-02-16T19:20:06.727 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:05.831+0000 7f598dffb700 1 -- 172.21.15.22:0/1455319842 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f59840042e0 con 0x7f59901048b0 2024-02-16T19:20:06.727 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:05.831+0000 7f598dffb700 1 -- 172.21.15.22:0/1455319842 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5984011af0 con 0x7f59901048b0 2024-02-16T19:20:06.727 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:05.832+0000 7f599552e700 1 -- 172.21.15.22:0/1455319842 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f59901048b0 msgr2=0x7f5990104cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:06.727 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:05.832+0000 7f599552e700 1 --2- 172.21.15.22:0/1455319842 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f59901048b0 0x7f5990104cc0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f59840098a0 tx=0x7f5984009bb0 comp rx=0 tx=0).stop 2024-02-16T19:20:06.728 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:05.832+0000 7f599552e700 1 -- 172.21.15.22:0/1455319842 shutdown_connections 2024-02-16T19:20:06.728 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:05.832+0000 7f599552e700 1 --2- 172.21.15.22:0/1455319842 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f59901048b0 0x7f5990104cc0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:06.728 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:05.832+0000 7f599552e700 1 -- 172.21.15.22:0/1455319842 >> 172.21.15.22:0/1455319842 conn(0x7f59900ffee0 msgr2=0x7f5990102330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:20:06.728 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:05.832+0000 7f599552e700 1 -- 172.21.15.22:0/1455319842 shutdown_connections 2024-02-16T19:20:06.728 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:05.832+0000 7f599552e700 1 -- 172.21.15.22:0/1455319842 wait complete. 2024-02-16T19:20:06.728 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:05.832+0000 7f599552e700 1 Processor -- start 2024-02-16T19:20:06.728 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:05.832+0000 7f599552e700 1 -- start start 2024-02-16T19:20:06.728 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:05.833+0000 7f599552e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f59901048b0 0x7f5990199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:06.728 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:05.833+0000 7f599552e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5990199c50 con 0x7f59901048b0 2024-02-16T19:20:06.728 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:05.833+0000 7f598effd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f59901048b0 0x7f5990199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:20:06.728 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:05.833+0000 7f598effd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f59901048b0 0x7f5990199710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42896/0 (socket says 172.21.15.22:42896) 2024-02-16T19:20:06.728 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:05.833+0000 7f598effd700 1 -- 172.21.15.22:0/1668927691 learned_addr learned my addr 172.21.15.22:0/1668927691 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:20:06.728 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:05.833+0000 7f598effd700 1 -- 172.21.15.22:0/1668927691 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5984009550 con 0x7f59901048b0 2024-02-16T19:20:06.728 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:05.833+0000 7f598effd700 1 --2- 172.21.15.22:0/1668927691 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f59901048b0 0x7f5990199710 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5984011670 tx=0x7f5984011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:20:06.729 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:05.833+0000 7f5977fff700 1 -- 172.21.15.22:0/1668927691 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5984003a40 con 0x7f59901048b0 2024-02-16T19:20:06.729 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:05.833+0000 7f599552e700 1 -- 172.21.15.22:0/1668927691 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5990199e50 con 0x7f59901048b0 2024-02-16T19:20:06.729 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:05.833+0000 7f5977fff700 1 -- 172.21.15.22:0/1668927691 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5984003ba0 con 0x7f59901048b0 2024-02-16T19:20:06.729 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:05.833+0000 7f5977fff700 1 -- 172.21.15.22:0/1668927691 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5984024790 con 0x7f59901048b0 2024-02-16T19:20:06.729 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:05.833+0000 7f599552e700 1 -- 172.21.15.22:0/1668927691 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f599019a2f0 con 0x7f59901048b0 2024-02-16T19:20:06.729 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:05.834+0000 7f5977fff700 1 -- 172.21.15.22:0/1668927691 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 9) v1 ==== 45097+0+0 (secure 0 0 0) 0x7f598402d3f0 con 0x7f59901048b0 2024-02-16T19:20:06.729 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:05.834+0000 7f599552e700 1 -- 172.21.15.22:0/1668927691 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5990062380 con 0x7f59901048b0 2024-02-16T19:20:06.729 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:05.834+0000 7f5977fff700 1 --2- 172.21.15.22:0/1668927691 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7f597803c850 0x7f597803ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:06.729 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:05.835+0000 7f5977fff700 1 -- 172.21.15.22:0/1668927691 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f598404c5e0 con 0x7f59901048b0 2024-02-16T19:20:06.729 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:05.835+0000 7f598e7fc700 1 -- 172.21.15.22:0/1668927691 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7f597803c850 msgr2=0x7f597803ed00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/2755289428 2024-02-16T19:20:06.729 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:05.835+0000 7f598e7fc700 1 --2- 172.21.15.22:0/1668927691 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7f597803c850 0x7f597803ed00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-16T19:20:06.729 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:05.843+0000 7f5977fff700 1 -- 172.21.15.22:0/1668927691 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f59840109d0 con 0x7f59901048b0 2024-02-16T19:20:06.729 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:06.036+0000 7f598e7fc700 1 -- 172.21.15.22:0/1668927691 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7f597803c850 msgr2=0x7f597803ed00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/2755289428 2024-02-16T19:20:06.730 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:06.036+0000 7f598e7fc700 1 --2- 172.21.15.22:0/1668927691 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7f597803c850 0x7f597803ed00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-16T19:20:06.730 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:06.122+0000 7f599552e700 1 -- 172.21.15.22:0/1668927691 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f599018e6f0 con 0x7f59901048b0 2024-02-16T19:20:06.730 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:06.122+0000 7f5977fff700 1 -- 172.21.15.22:0/1668927691 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+103 (secure 0 0 0) 0x7f598404bf50 con 0x7f59901048b0 2024-02-16T19:20:06.730 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:06.124+0000 7f599552e700 1 -- 172.21.15.22:0/1668927691 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7f597803c850 msgr2=0x7f597803ed00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-16T19:20:06.730 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:06.124+0000 7f599552e700 1 --2- 172.21.15.22:0/1668927691 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7f597803c850 0x7f597803ed00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:06.730 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:06.124+0000 7f599552e700 1 -- 172.21.15.22:0/1668927691 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f59901048b0 msgr2=0x7f5990199710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:06.730 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:06.124+0000 7f599552e700 1 --2- 172.21.15.22:0/1668927691 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f59901048b0 0x7f5990199710 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5984011670 tx=0x7f5984011b50 comp rx=0 tx=0).stop 2024-02-16T19:20:06.730 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:06.124+0000 7f599552e700 1 -- 172.21.15.22:0/1668927691 shutdown_connections 2024-02-16T19:20:06.730 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:06.124+0000 7f599552e700 1 --2- 172.21.15.22:0/1668927691 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7f597803c850 0x7f597803ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:06.730 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:06.124+0000 7f599552e700 1 --2- 172.21.15.22:0/1668927691 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f59901048b0 0x7f5990199710 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:06.730 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:06.124+0000 7f599552e700 1 -- 172.21.15.22:0/1668927691 >> 172.21.15.22:0/1668927691 conn(0x7f59900ffee0 msgr2=0x7f5990079830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:20:06.730 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:06.125+0000 7f599552e700 1 -- 172.21.15.22:0/1668927691 shutdown_connections 2024-02-16T19:20:06.730 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:06.125+0000 7f599552e700 1 -- 172.21.15.22:0/1668927691 wait complete. 2024-02-16T19:20:06.730 INFO:teuthology.orchestra.run.smithi022.stdout:Waiting for the mgr to restart... 2024-02-16T19:20:06.731 INFO:teuthology.orchestra.run.smithi022.stdout:Waiting for mgr epoch 9... 2024-02-16T19:20:06.972 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:06 smithi022 ceph-mon[29221]: from='client.? 172.21.15.22:0/1668927691' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-16T19:20:11.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:11 smithi022 ceph-mon[29221]: Active manager daemon smithi022.gmdcch restarted 2024-02-16T19:20:11.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:11 smithi022 ceph-mon[29221]: Activating manager daemon smithi022.gmdcch 2024-02-16T19:20:11.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:11 smithi022 ceph-mon[29221]: osdmap e3: 0 total, 0 up, 0 in 2024-02-16T19:20:11.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:11 smithi022 ceph-mon[29221]: mgrmap e10: smithi022.gmdcch(active, starting, since 0.00323505s) 2024-02-16T19:20:11.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:11 smithi022 ceph-mon[29221]: from='mgr.14164 172.21.15.22:0/3311464841' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mon metadata", "id": "smithi022"}]: dispatch 2024-02-16T19:20:11.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:11 smithi022 ceph-mon[29221]: from='mgr.14164 172.21.15.22:0/3311464841' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mgr metadata", "who": "smithi022.gmdcch", "id": "smithi022.gmdcch"}]: dispatch 2024-02-16T19:20:11.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:11 smithi022 ceph-mon[29221]: from='mgr.14164 172.21.15.22:0/3311464841' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-16T19:20:11.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:11 smithi022 ceph-mon[29221]: from='mgr.14164 172.21.15.22:0/3311464841' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-16T19:20:11.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:11 smithi022 ceph-mon[29221]: from='mgr.14164 172.21.15.22:0/3311464841' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-16T19:20:11.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:11 smithi022 ceph-mon[29221]: Manager daemon smithi022.gmdcch is now available 2024-02-16T19:20:11.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:11 smithi022 ceph-mon[29221]: from='mgr.14164 172.21.15.22:0/3311464841' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:20:11.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:11 smithi022 ceph-mon[29221]: from='mgr.14164 172.21.15.22:0/3311464841' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:20:12.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:12 smithi022 ceph-mon[29221]: from='mgr.14164 172.21.15.22:0/3311464841' entity='mgr.smithi022.gmdcch' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi022.gmdcch/mirror_snapshot_schedule"}]: dispatch 2024-02-16T19:20:12.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:12 smithi022 ceph-mon[29221]: from='mgr.14164 172.21.15.22:0/3311464841' entity='mgr.smithi022.gmdcch' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi022.gmdcch/trash_purge_schedule"}]: dispatch 2024-02-16T19:20:12.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:12 smithi022 ceph-mon[29221]: mgrmap e11: smithi022.gmdcch(active, since 1.00583s) 2024-02-16T19:20:13.411 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout { 2024-02-16T19:20:13.411 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-02-16T19:20:13.411 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "initialized": true 2024-02-16T19:20:13.411 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout } 2024-02-16T19:20:13.411 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:07.204+0000 7fda62118700 1 Processor -- start 2024-02-16T19:20:13.412 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:07.204+0000 7fda62118700 1 -- start start 2024-02-16T19:20:13.412 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:07.205+0000 7fda62118700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda5c1048e0 0x7fda5c104cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:13.412 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:07.205+0000 7fda62118700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda5c105230 con 0x7fda5c1048e0 2024-02-16T19:20:13.412 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:07.205+0000 7fda5b7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda5c1048e0 0x7fda5c104cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:20:13.412 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:07.205+0000 7fda5b7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda5c1048e0 0x7fda5c104cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42910/0 (socket says 172.21.15.22:42910) 2024-02-16T19:20:13.412 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:07.205+0000 7fda5b7fe700 1 -- 172.21.15.22:0/842176830 learned_addr learned my addr 172.21.15.22:0/842176830 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:20:13.412 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:07.205+0000 7fda5b7fe700 1 -- 172.21.15.22:0/842176830 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda5c105370 con 0x7fda5c1048e0 2024-02-16T19:20:13.412 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:07.205+0000 7fda5b7fe700 1 --2- 172.21.15.22:0/842176830 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda5c1048e0 0x7fda5c104cf0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fda4c0098a0 tx=0x7fda4c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b194a8c5e758de94 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:20:13.412 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:07.205+0000 7fda5a7fc700 1 -- 172.21.15.22:0/842176830 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fda4c004180 con 0x7fda5c1048e0 2024-02-16T19:20:13.412 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:07.206+0000 7fda5a7fc700 1 -- 172.21.15.22:0/842176830 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fda4c0042e0 con 0x7fda5c1048e0 2024-02-16T19:20:13.412 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:07.206+0000 7fda5a7fc700 1 -- 172.21.15.22:0/842176830 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fda4c011af0 con 0x7fda5c1048e0 2024-02-16T19:20:13.412 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:07.206+0000 7fda62118700 1 -- 172.21.15.22:0/842176830 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda5c1048e0 msgr2=0x7fda5c104cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:13.412 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:07.206+0000 7fda62118700 1 --2- 172.21.15.22:0/842176830 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda5c1048e0 0x7fda5c104cf0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fda4c0098a0 tx=0x7fda4c009bb0 comp rx=0 tx=0).stop 2024-02-16T19:20:13.413 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:07.206+0000 7fda62118700 1 -- 172.21.15.22:0/842176830 shutdown_connections 2024-02-16T19:20:13.413 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:07.206+0000 7fda62118700 1 --2- 172.21.15.22:0/842176830 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda5c1048e0 0x7fda5c104cf0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:13.413 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:07.206+0000 7fda62118700 1 -- 172.21.15.22:0/842176830 >> 172.21.15.22:0/842176830 conn(0x7fda5c0fff10 msgr2=0x7fda5c102360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:20:13.413 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:07.206+0000 7fda62118700 1 -- 172.21.15.22:0/842176830 shutdown_connections 2024-02-16T19:20:13.413 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:07.207+0000 7fda62118700 1 -- 172.21.15.22:0/842176830 wait complete. 2024-02-16T19:20:13.413 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:07.207+0000 7fda62118700 1 Processor -- start 2024-02-16T19:20:13.413 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:07.207+0000 7fda62118700 1 -- start start 2024-02-16T19:20:13.413 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:07.207+0000 7fda62118700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda5c1048e0 0x7fda5c18fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:13.413 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:07.207+0000 7fda62118700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda5c190000 con 0x7fda5c1048e0 2024-02-16T19:20:13.413 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:07.208+0000 7fda5b7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda5c1048e0 0x7fda5c18fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:20:13.413 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:07.208+0000 7fda5b7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda5c1048e0 0x7fda5c18fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42924/0 (socket says 172.21.15.22:42924) 2024-02-16T19:20:13.413 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:07.208+0000 7fda5b7fe700 1 -- 172.21.15.22:0/904710180 learned_addr learned my addr 172.21.15.22:0/904710180 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:20:13.413 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:07.208+0000 7fda5b7fe700 1 -- 172.21.15.22:0/904710180 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda4c009550 con 0x7fda5c1048e0 2024-02-16T19:20:13.414 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:07.208+0000 7fda5b7fe700 1 --2- 172.21.15.22:0/904710180 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda5c1048e0 0x7fda5c18fac0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fda4c011670 tx=0x7fda4c011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:20:13.414 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:07.208+0000 7fda58ff9700 1 -- 172.21.15.22:0/904710180 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fda4c003a40 con 0x7fda5c1048e0 2024-02-16T19:20:13.414 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:07.208+0000 7fda62118700 1 -- 172.21.15.22:0/904710180 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda5c18e110 con 0x7fda5c1048e0 2024-02-16T19:20:13.414 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:07.208+0000 7fda62118700 1 -- 172.21.15.22:0/904710180 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda5c18e5b0 con 0x7fda5c1048e0 2024-02-16T19:20:13.414 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:07.208+0000 7fda58ff9700 1 -- 172.21.15.22:0/904710180 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fda4c003ba0 con 0x7fda5c1048e0 2024-02-16T19:20:13.414 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:07.208+0000 7fda58ff9700 1 -- 172.21.15.22:0/904710180 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fda4c024790 con 0x7fda5c1048e0 2024-02-16T19:20:13.414 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:07.209+0000 7fda58ff9700 1 -- 172.21.15.22:0/904710180 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 9) v1 ==== 45097+0+0 (secure 0 0 0) 0x7fda4c02d3f0 con 0x7fda5c1048e0 2024-02-16T19:20:13.414 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:07.209+0000 7fda58ff9700 1 --2- 172.21.15.22:0/904710180 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7fda44038490 0x7fda4403a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:13.414 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:07.209+0000 7fda58ff9700 1 -- 172.21.15.22:0/904710180 --> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fda4403b050 con 0x7fda44038490 2024-02-16T19:20:13.414 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:07.209+0000 7fda58ff9700 1 -- 172.21.15.22:0/904710180 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fda4c04c4b0 con 0x7fda5c1048e0 2024-02-16T19:20:13.414 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:07.209+0000 7fda5affd700 1 -- 172.21.15.22:0/904710180 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7fda44038490 msgr2=0x7fda4403a940 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/2755289428 2024-02-16T19:20:13.414 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:07.210+0000 7fda5affd700 1 --2- 172.21.15.22:0/904710180 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7fda44038490 0x7fda4403a940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-16T19:20:13.415 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:07.411+0000 7fda5affd700 1 -- 172.21.15.22:0/904710180 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7fda44038490 msgr2=0x7fda4403a940 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/2755289428 2024-02-16T19:20:13.415 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:07.411+0000 7fda5affd700 1 --2- 172.21.15.22:0/904710180 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7fda44038490 0x7fda4403a940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-16T19:20:13.415 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:07.812+0000 7fda5affd700 1 -- 172.21.15.22:0/904710180 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7fda44038490 msgr2=0x7fda4403a940 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/2755289428 2024-02-16T19:20:13.415 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:07.812+0000 7fda5affd700 1 --2- 172.21.15.22:0/904710180 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7fda44038490 0x7fda4403a940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-02-16T19:20:13.415 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:08.613+0000 7fda5affd700 1 -- 172.21.15.22:0/904710180 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7fda44038490 msgr2=0x7fda4403a940 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/2755289428 2024-02-16T19:20:13.415 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:08.613+0000 7fda5affd700 1 --2- 172.21.15.22:0/904710180 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7fda44038490 0x7fda4403a940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-02-16T19:20:13.415 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:10.214+0000 7fda5affd700 1 -- 172.21.15.22:0/904710180 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7fda44038490 msgr2=0x7fda4403a940 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/2755289428 2024-02-16T19:20:13.415 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:10.214+0000 7fda5affd700 1 --2- 172.21.15.22:0/904710180 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7fda44038490 0x7fda4403a940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-02-16T19:20:13.415 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:11.620+0000 7fda58ff9700 1 -- 172.21.15.22:0/904710180 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mgrmap(e 10) v1 ==== 44864+0+0 (secure 0 0 0) 0x7fda4c02d920 con 0x7fda5c1048e0 2024-02-16T19:20:13.415 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:11.620+0000 7fda58ff9700 1 -- 172.21.15.22:0/904710180 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7fda44038490 msgr2=0x7fda4403a940 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-16T19:20:13.415 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:11.620+0000 7fda58ff9700 1 --2- 172.21.15.22:0/904710180 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7fda44038490 0x7fda4403a940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:13.415 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:12.621+0000 7fda58ff9700 1 -- 172.21.15.22:0/904710180 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mgrmap(e 11) v1 ==== 44991+0+0 (secure 0 0 0) 0x7fda4c02dca0 con 0x7fda5c1048e0 2024-02-16T19:20:13.415 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:12.621+0000 7fda58ff9700 1 --2- 172.21.15.22:0/904710180 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7fda4403bef0 0x7fda4403e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:13.415 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:12.622+0000 7fda58ff9700 1 -- 172.21.15.22:0/904710180 --> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fda4403b050 con 0x7fda4403bef0 2024-02-16T19:20:13.415 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:12.622+0000 7fda5affd700 1 --2- 172.21.15.22:0/904710180 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7fda4403bef0 0x7fda4403e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:20:13.416 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:12.622+0000 7fda5affd700 1 --2- 172.21.15.22:0/904710180 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7fda4403bef0 0x7fda4403e2d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fda50003a10 tx=0x7fda50009270 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:20:13.416 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:12.623+0000 7fda58ff9700 1 -- 172.21.15.22:0/904710180 <== mgr.14164 v2:172.21.15.22:6800/2945983569 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7fda4403b050 con 0x7fda4403bef0 2024-02-16T19:20:13.416 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:12.630+0000 7fda62118700 1 -- 172.21.15.22:0/904710180 --> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fda48002630 con 0x7fda4403bef0 2024-02-16T19:20:13.416 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:12.630+0000 7fda58ff9700 1 -- 172.21.15.22:0/904710180 <== mgr.14164 v2:172.21.15.22:6800/2945983569 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7fda48002630 con 0x7fda4403bef0 2024-02-16T19:20:13.416 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:12.631+0000 7fda62118700 1 -- 172.21.15.22:0/904710180 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7fda4403bef0 msgr2=0x7fda4403e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:13.416 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:12.631+0000 7fda62118700 1 --2- 172.21.15.22:0/904710180 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7fda4403bef0 0x7fda4403e2d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fda50003a10 tx=0x7fda50009270 comp rx=0 tx=0).stop 2024-02-16T19:20:13.416 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:12.631+0000 7fda62118700 1 -- 172.21.15.22:0/904710180 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda5c1048e0 msgr2=0x7fda5c18fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:13.416 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:12.631+0000 7fda62118700 1 --2- 172.21.15.22:0/904710180 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda5c1048e0 0x7fda5c18fac0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fda4c011670 tx=0x7fda4c011b50 comp rx=0 tx=0).stop 2024-02-16T19:20:13.416 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:12.631+0000 7fda62118700 1 -- 172.21.15.22:0/904710180 shutdown_connections 2024-02-16T19:20:13.416 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:12.631+0000 7fda62118700 1 --2- 172.21.15.22:0/904710180 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7fda4403bef0 0x7fda4403e2d0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:13.416 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:12.631+0000 7fda62118700 1 --2- 172.21.15.22:0/904710180 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda5c1048e0 0x7fda5c18fac0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:13.416 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:12.631+0000 7fda62118700 1 --2- 172.21.15.22:0/904710180 >> [v2:172.21.15.22:6800/2755289428,v1:172.21.15.22:6801/2755289428] conn(0x7fda44038490 0x7fda4403a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:13.416 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:12.631+0000 7fda62118700 1 -- 172.21.15.22:0/904710180 >> 172.21.15.22:0/904710180 conn(0x7fda5c0fff10 msgr2=0x7fda5c1015e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:20:13.416 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:12.631+0000 7fda62118700 1 -- 172.21.15.22:0/904710180 shutdown_connections 2024-02-16T19:20:13.417 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:12.631+0000 7fda62118700 1 -- 172.21.15.22:0/904710180 wait complete. 2024-02-16T19:20:13.417 INFO:teuthology.orchestra.run.smithi022.stdout:mgr epoch 9 is available 2024-02-16T19:20:13.417 INFO:teuthology.orchestra.run.smithi022.stdout:Generating a dashboard self-signed certificate... 2024-02-16T19:20:13.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:13 smithi022 ceph-mon[29221]: [16/Feb/2024:19:20:11] ENGINE Bus STARTING 2024-02-16T19:20:13.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:13 smithi022 ceph-mon[29221]: [16/Feb/2024:19:20:12] ENGINE Serving on http://172.21.15.22:8765 2024-02-16T19:20:13.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:13 smithi022 ceph-mon[29221]: [16/Feb/2024:19:20:12] ENGINE Serving on https://172.21.15.22:7150 2024-02-16T19:20:13.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:13 smithi022 ceph-mon[29221]: [16/Feb/2024:19:20:12] ENGINE Bus STARTED 2024-02-16T19:20:13.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:13 smithi022 ceph-mon[29221]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-02-16T19:20:13.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:13 smithi022 ceph-mon[29221]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-02-16T19:20:13.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:13 smithi022 ceph-mon[29221]: from='mgr.14164 172.21.15.22:0/3311464841' entity='mgr.smithi022.gmdcch' 2024-02-16T19:20:14.757 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-02-16T19:20:14.758 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:13.848+0000 7f827d558700 1 Processor -- start 2024-02-16T19:20:14.758 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:13.848+0000 7f827d558700 1 -- start start 2024-02-16T19:20:14.758 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:13.848+0000 7f827d558700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82781059c0 0x7f8278105dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:14.758 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:13.848+0000 7f827d558700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8278106310 con 0x7f82781059c0 2024-02-16T19:20:14.758 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:13.849+0000 7f8276ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82781059c0 0x7f8278105dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:20:14.758 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:13.849+0000 7f8276ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82781059c0 0x7f8278105dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42998/0 (socket says 172.21.15.22:42998) 2024-02-16T19:20:14.758 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:13.849+0000 7f8276ffd700 1 -- 172.21.15.22:0/4022602107 learned_addr learned my addr 172.21.15.22:0/4022602107 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:20:14.758 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:13.849+0000 7f8276ffd700 1 -- 172.21.15.22:0/4022602107 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8278106450 con 0x7f82781059c0 2024-02-16T19:20:14.758 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:13.849+0000 7f8276ffd700 1 --2- 172.21.15.22:0/4022602107 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82781059c0 0x7f8278105dd0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f82680098a0 tx=0x7f8268009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=471f00276c0b9178 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:20:14.759 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:13.849+0000 7f8275ffb700 1 -- 172.21.15.22:0/4022602107 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8268004180 con 0x7f82781059c0 2024-02-16T19:20:14.759 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:13.849+0000 7f8275ffb700 1 -- 172.21.15.22:0/4022602107 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f82680042e0 con 0x7f82781059c0 2024-02-16T19:20:14.759 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:13.849+0000 7f8275ffb700 1 -- 172.21.15.22:0/4022602107 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8268011af0 con 0x7f82781059c0 2024-02-16T19:20:14.759 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:13.850+0000 7f827d558700 1 -- 172.21.15.22:0/4022602107 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82781059c0 msgr2=0x7f8278105dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:14.759 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:13.850+0000 7f827d558700 1 --2- 172.21.15.22:0/4022602107 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82781059c0 0x7f8278105dd0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f82680098a0 tx=0x7f8268009bb0 comp rx=0 tx=0).stop 2024-02-16T19:20:14.759 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:13.850+0000 7f827d558700 1 -- 172.21.15.22:0/4022602107 shutdown_connections 2024-02-16T19:20:14.759 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:13.850+0000 7f827d558700 1 --2- 172.21.15.22:0/4022602107 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82781059c0 0x7f8278105dd0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:14.759 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:13.850+0000 7f827d558700 1 -- 172.21.15.22:0/4022602107 >> 172.21.15.22:0/4022602107 conn(0x7f8278100ff0 msgr2=0x7f8278103440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:20:14.759 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:13.850+0000 7f827d558700 1 -- 172.21.15.22:0/4022602107 shutdown_connections 2024-02-16T19:20:14.759 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:13.850+0000 7f827d558700 1 -- 172.21.15.22:0/4022602107 wait complete. 2024-02-16T19:20:14.759 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:13.850+0000 7f827d558700 1 Processor -- start 2024-02-16T19:20:14.759 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:13.851+0000 7f827d558700 1 -- start start 2024-02-16T19:20:14.759 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:13.851+0000 7f827d558700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82781059c0 0x7f8278199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:14.759 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:13.851+0000 7f827d558700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8278199cb0 con 0x7f82781059c0 2024-02-16T19:20:14.760 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:13.851+0000 7f8276ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82781059c0 0x7f8278199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:20:14.760 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:13.851+0000 7f8276ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82781059c0 0x7f8278199770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43012/0 (socket says 172.21.15.22:43012) 2024-02-16T19:20:14.760 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:13.851+0000 7f8276ffd700 1 -- 172.21.15.22:0/2265360448 learned_addr learned my addr 172.21.15.22:0/2265360448 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:20:14.760 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:13.851+0000 7f8276ffd700 1 -- 172.21.15.22:0/2265360448 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8268009550 con 0x7f82781059c0 2024-02-16T19:20:14.760 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:13.851+0000 7f8276ffd700 1 --2- 172.21.15.22:0/2265360448 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82781059c0 0x7f8278199770 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8268011670 tx=0x7f8268011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:20:14.760 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:13.851+0000 7f825ffff700 1 -- 172.21.15.22:0/2265360448 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8268003a40 con 0x7f82781059c0 2024-02-16T19:20:14.760 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:13.852+0000 7f827d558700 1 -- 172.21.15.22:0/2265360448 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8278199eb0 con 0x7f82781059c0 2024-02-16T19:20:14.760 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:13.852+0000 7f825ffff700 1 -- 172.21.15.22:0/2265360448 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8268003ba0 con 0x7f82781059c0 2024-02-16T19:20:14.760 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:13.852+0000 7f825ffff700 1 -- 172.21.15.22:0/2265360448 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8268024820 con 0x7f82781059c0 2024-02-16T19:20:14.760 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:13.852+0000 7f827d558700 1 -- 172.21.15.22:0/2265360448 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f827819a350 con 0x7f82781059c0 2024-02-16T19:20:14.760 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:13.852+0000 7f825ffff700 1 -- 172.21.15.22:0/2265360448 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 11) v1 ==== 44991+0+0 (secure 0 0 0) 0x7f8268024ae0 con 0x7f82781059c0 2024-02-16T19:20:14.760 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:13.853+0000 7f827d558700 1 -- 172.21.15.22:0/2265360448 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8278062380 con 0x7f82781059c0 2024-02-16T19:20:14.760 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:13.853+0000 7f825ffff700 1 --2- 172.21.15.22:0/2265360448 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7f82600382e0 0x7f826003a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:14.761 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:13.853+0000 7f825ffff700 1 -- 172.21.15.22:0/2265360448 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f826804c290 con 0x7f82781059c0 2024-02-16T19:20:14.761 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:13.853+0000 7f82767fc700 1 --2- 172.21.15.22:0/2265360448 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7f82600382e0 0x7f826003a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:20:14.761 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:13.853+0000 7f82767fc700 1 --2- 172.21.15.22:0/2265360448 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7f82600382e0 0x7f826003a790 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f826c006fd0 tx=0x7f826c006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:20:14.761 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:13.858+0000 7f825ffff700 1 -- 172.21.15.22:0/2265360448 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f8268022310 con 0x7f82781059c0 2024-02-16T19:20:14.761 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:14.035+0000 7f825ffff700 1 -- 172.21.15.22:0/2265360448 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mgrmap(e 12) v1 ==== 45097+0+0 (secure 0 0 0) 0x7f826801ac70 con 0x7f82781059c0 2024-02-16T19:20:14.761 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:14.070+0000 7f827d558700 1 -- 172.21.15.22:0/2265360448 --> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f8278103330 con 0x7f82600382e0 2024-02-16T19:20:14.761 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:14.168+0000 7f825ffff700 1 -- 172.21.15.22:0/2265360448 <== mgr.14164 v2:172.21.15.22:6800/2945983569 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f8278103330 con 0x7f82600382e0 2024-02-16T19:20:14.761 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:14.171+0000 7f827d558700 1 -- 172.21.15.22:0/2265360448 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7f82600382e0 msgr2=0x7f826003a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:14.761 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:14.171+0000 7f827d558700 1 --2- 172.21.15.22:0/2265360448 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7f82600382e0 0x7f826003a790 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f826c006fd0 tx=0x7f826c006e40 comp rx=0 tx=0).stop 2024-02-16T19:20:14.761 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:14.171+0000 7f827d558700 1 -- 172.21.15.22:0/2265360448 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82781059c0 msgr2=0x7f8278199770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:14.761 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:14.171+0000 7f827d558700 1 --2- 172.21.15.22:0/2265360448 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82781059c0 0x7f8278199770 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8268011670 tx=0x7f8268011aa0 comp rx=0 tx=0).stop 2024-02-16T19:20:14.761 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:14.171+0000 7f827d558700 1 -- 172.21.15.22:0/2265360448 shutdown_connections 2024-02-16T19:20:14.761 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:14.171+0000 7f827d558700 1 --2- 172.21.15.22:0/2265360448 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7f82600382e0 0x7f826003a790 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:14.762 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:14.171+0000 7f827d558700 1 --2- 172.21.15.22:0/2265360448 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82781059c0 0x7f8278199770 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:14.762 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:14.171+0000 7f827d558700 1 -- 172.21.15.22:0/2265360448 >> 172.21.15.22:0/2265360448 conn(0x7f8278100ff0 msgr2=0x7f8278102710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:20:14.762 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:14.171+0000 7f827d558700 1 -- 172.21.15.22:0/2265360448 shutdown_connections 2024-02-16T19:20:14.762 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:14.171+0000 7f827d558700 1 -- 172.21.15.22:0/2265360448 wait complete. 2024-02-16T19:20:14.762 INFO:teuthology.orchestra.run.smithi022.stdout:Creating initial admin user... 2024-02-16T19:20:15.194 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:14 smithi022 ceph-mon[29221]: from='mgr.14164 172.21.15.22:0/3311464841' entity='mgr.smithi022.gmdcch' 2024-02-16T19:20:15.194 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:14 smithi022 ceph-mon[29221]: mgrmap e12: smithi022.gmdcch(active, since 2s) 2024-02-16T19:20:15.194 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:14 smithi022 ceph-mon[29221]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:20:15.195 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:14 smithi022 ceph-mon[29221]: from='mgr.14164 172.21.15.22:0/3311464841' entity='mgr.smithi022.gmdcch' 2024-02-16T19:20:15.195 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:14 smithi022 ceph-mon[29221]: from='mgr.14164 172.21.15.22:0/3311464841' entity='mgr.smithi022.gmdcch' 2024-02-16T19:20:16.347 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$AZQX.wLUJpVLwfCKNEAQZOCZ0mkUNiOW6ebZqbqGwWcYQMFnWcvUC", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1708111215, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-02-16T19:20:16.347 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:15.217+0000 7f5b087f8700 1 Processor -- start 2024-02-16T19:20:16.348 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:15.218+0000 7f5b087f8700 1 -- start start 2024-02-16T19:20:16.348 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:15.218+0000 7f5b087f8700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b0007bdd0 0x7f5b0007a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:16.348 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:15.218+0000 7f5b087f8700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0007a810 con 0x7f5b0007bdd0 2024-02-16T19:20:16.348 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:15.218+0000 7f5b06594700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b0007bdd0 0x7f5b0007a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:20:16.348 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:15.218+0000 7f5b06594700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b0007bdd0 0x7f5b0007a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59122/0 (socket says 172.21.15.22:59122) 2024-02-16T19:20:16.348 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:15.218+0000 7f5b06594700 1 -- 172.21.15.22:0/499874663 learned_addr learned my addr 172.21.15.22:0/499874663 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:20:16.348 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:15.218+0000 7f5b06594700 1 -- 172.21.15.22:0/499874663 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b0007a950 con 0x7f5b0007bdd0 2024-02-16T19:20:16.348 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:15.219+0000 7f5b06594700 1 --2- 172.21.15.22:0/499874663 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b0007bdd0 0x7f5b0007a2d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5afc0098a0 tx=0x7f5afc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bd5300b23383137 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:20:16.348 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:15.219+0000 7f5b05592700 1 -- 172.21.15.22:0/499874663 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5afc004180 con 0x7f5b0007bdd0 2024-02-16T19:20:16.348 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:15.219+0000 7f5b05592700 1 -- 172.21.15.22:0/499874663 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5afc0042e0 con 0x7f5b0007bdd0 2024-02-16T19:20:16.348 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:15.219+0000 7f5b05592700 1 -- 172.21.15.22:0/499874663 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5afc011af0 con 0x7f5b0007bdd0 2024-02-16T19:20:16.348 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:15.219+0000 7f5b087f8700 1 -- 172.21.15.22:0/499874663 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b0007bdd0 msgr2=0x7f5b0007a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:16.349 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:15.219+0000 7f5b087f8700 1 --2- 172.21.15.22:0/499874663 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b0007bdd0 0x7f5b0007a2d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5afc0098a0 tx=0x7f5afc009bb0 comp rx=0 tx=0).stop 2024-02-16T19:20:16.349 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:15.220+0000 7f5b087f8700 1 -- 172.21.15.22:0/499874663 shutdown_connections 2024-02-16T19:20:16.349 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:15.220+0000 7f5b087f8700 1 --2- 172.21.15.22:0/499874663 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b0007bdd0 0x7f5b0007a2d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:16.349 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:15.220+0000 7f5b087f8700 1 -- 172.21.15.22:0/499874663 >> 172.21.15.22:0/499874663 conn(0x7f5b00101020 msgr2=0x7f5b00103430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:20:16.349 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:15.220+0000 7f5b087f8700 1 -- 172.21.15.22:0/499874663 shutdown_connections 2024-02-16T19:20:16.349 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:15.220+0000 7f5b087f8700 1 -- 172.21.15.22:0/499874663 wait complete. 2024-02-16T19:20:16.349 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:15.220+0000 7f5b087f8700 1 Processor -- start 2024-02-16T19:20:16.349 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:15.220+0000 7f5b087f8700 1 -- start start 2024-02-16T19:20:16.349 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:15.220+0000 7f5b087f8700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b0007bdd0 0x7f5b0019b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:16.349 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:15.220+0000 7f5b087f8700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0019bf10 con 0x7f5b0007bdd0 2024-02-16T19:20:16.349 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:15.221+0000 7f5b06594700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b0007bdd0 0x7f5b0019b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:20:16.349 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:15.221+0000 7f5b06594700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b0007bdd0 0x7f5b0019b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59124/0 (socket says 172.21.15.22:59124) 2024-02-16T19:20:16.349 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:15.221+0000 7f5b06594700 1 -- 172.21.15.22:0/1277780466 learned_addr learned my addr 172.21.15.22:0/1277780466 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:20:16.350 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:15.221+0000 7f5b06594700 1 -- 172.21.15.22:0/1277780466 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5afc009550 con 0x7f5b0007bdd0 2024-02-16T19:20:16.350 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:15.221+0000 7f5b06594700 1 --2- 172.21.15.22:0/1277780466 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b0007bdd0 0x7f5b0019b9d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5afc006ba0 tx=0x7f5afc011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:20:16.350 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:15.221+0000 7f5af77fe700 1 -- 172.21.15.22:0/1277780466 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5afc003b00 con 0x7f5b0007bdd0 2024-02-16T19:20:16.350 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:15.221+0000 7f5af77fe700 1 -- 172.21.15.22:0/1277780466 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5afc003c60 con 0x7f5b0007bdd0 2024-02-16T19:20:16.350 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:15.221+0000 7f5b087f8700 1 -- 172.21.15.22:0/1277780466 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b0019c110 con 0x7f5b0007bdd0 2024-02-16T19:20:16.350 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:15.221+0000 7f5af77fe700 1 -- 172.21.15.22:0/1277780466 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5afc019820 con 0x7f5b0007bdd0 2024-02-16T19:20:16.350 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:15.221+0000 7f5b087f8700 1 -- 172.21.15.22:0/1277780466 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b0019c5b0 con 0x7f5b0007bdd0 2024-02-16T19:20:16.350 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:15.222+0000 7f5af77fe700 1 -- 172.21.15.22:0/1277780466 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 12) v1 ==== 45097+0+0 (secure 0 0 0) 0x7f5afc019a90 con 0x7f5b0007bdd0 2024-02-16T19:20:16.350 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:15.222+0000 7f5af77fe700 1 --2- 172.21.15.22:0/1277780466 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7f5aec038440 0x7f5aec03a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:16.350 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:15.223+0000 7f5af77fe700 1 -- 172.21.15.22:0/1277780466 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f5afc04c5b0 con 0x7f5b0007bdd0 2024-02-16T19:20:16.350 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:15.223+0000 7f5b087f8700 1 -- 172.21.15.22:0/1277780466 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b0010bce0 con 0x7f5b0007bdd0 2024-02-16T19:20:16.350 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:15.223+0000 7f5b05d93700 1 --2- 172.21.15.22:0/1277780466 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7f5aec038440 0x7f5aec03a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:20:16.351 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:15.223+0000 7f5b05d93700 1 --2- 172.21.15.22:0/1277780466 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7f5aec038440 0x7f5aec03a8f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5af0006fd0 tx=0x7f5af0006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:20:16.351 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:15.228+0000 7f5af77fe700 1 -- 172.21.15.22:0/1277780466 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f5afc02c3a0 con 0x7f5b0007bdd0 2024-02-16T19:20:16.351 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:15.444+0000 7f5b087f8700 1 -- 172.21.15.22:0/1277780466 --> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f5b000621a0 con 0x7f5aec038440 2024-02-16T19:20:16.351 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:15.691+0000 7f5af77fe700 1 -- 172.21.15.22:0/1277780466 <== mgr.14164 v2:172.21.15.22:6800/2945983569 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f5b000621a0 con 0x7f5aec038440 2024-02-16T19:20:16.351 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:15.693+0000 7f5b087f8700 1 -- 172.21.15.22:0/1277780466 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7f5aec038440 msgr2=0x7f5aec03a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:16.351 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:15.693+0000 7f5b087f8700 1 --2- 172.21.15.22:0/1277780466 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7f5aec038440 0x7f5aec03a8f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5af0006fd0 tx=0x7f5af0006e40 comp rx=0 tx=0).stop 2024-02-16T19:20:16.351 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:15.693+0000 7f5b087f8700 1 -- 172.21.15.22:0/1277780466 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b0007bdd0 msgr2=0x7f5b0019b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:16.351 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:15.693+0000 7f5b087f8700 1 --2- 172.21.15.22:0/1277780466 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b0007bdd0 0x7f5b0019b9d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5afc006ba0 tx=0x7f5afc011b50 comp rx=0 tx=0).stop 2024-02-16T19:20:16.351 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:15.694+0000 7f5b087f8700 1 -- 172.21.15.22:0/1277780466 shutdown_connections 2024-02-16T19:20:16.351 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:15.694+0000 7f5b087f8700 1 --2- 172.21.15.22:0/1277780466 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7f5aec038440 0x7f5aec03a8f0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:16.351 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:15.694+0000 7f5b087f8700 1 --2- 172.21.15.22:0/1277780466 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b0007bdd0 0x7f5b0019b9d0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:16.351 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:15.694+0000 7f5b087f8700 1 -- 172.21.15.22:0/1277780466 >> 172.21.15.22:0/1277780466 conn(0x7f5b00101020 msgr2=0x7f5b00102690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:20:16.351 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:15.694+0000 7f5b087f8700 1 -- 172.21.15.22:0/1277780466 shutdown_connections 2024-02-16T19:20:16.352 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:15.694+0000 7f5b087f8700 1 -- 172.21.15.22:0/1277780466 wait complete. 2024-02-16T19:20:16.352 INFO:teuthology.orchestra.run.smithi022.stdout:Fetching dashboard port number... 2024-02-16T19:20:16.815 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:16 smithi022 ceph-mon[29221]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:20:16.815 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:16 smithi022 ceph-mon[29221]: from='mgr.14164 172.21.15.22:0/3311464841' entity='mgr.smithi022.gmdcch' 2024-02-16T19:20:17.855 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:17 smithi022 ceph-mon[29221]: from='client.? 172.21.15.22:0/441901952' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-02-16T19:20:17.863 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 8443 2024-02-16T19:20:17.864 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:16.817+0000 7fc278c45700 1 Processor -- start 2024-02-16T19:20:17.864 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:16.817+0000 7fc278c45700 1 -- start start 2024-02-16T19:20:17.864 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:16.817+0000 7fc278c45700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc274106500 0x7fc2741088e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:17.864 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:16.818+0000 7fc278c45700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc274079d10 con 0x7fc274106500 2024-02-16T19:20:17.864 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:16.818+0000 7fc27259c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc274106500 0x7fc2741088e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:20:17.864 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:16.818+0000 7fc27259c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc274106500 0x7fc2741088e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59126/0 (socket says 172.21.15.22:59126) 2024-02-16T19:20:17.864 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:16.818+0000 7fc27259c700 1 -- 172.21.15.22:0/2355088705 learned_addr learned my addr 172.21.15.22:0/2355088705 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:20:17.864 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:16.818+0000 7fc27259c700 1 -- 172.21.15.22:0/2355088705 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc274079e50 con 0x7fc274106500 2024-02-16T19:20:17.864 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:16.818+0000 7fc27259c700 1 --2- 172.21.15.22:0/2355088705 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc274106500 0x7fc2741088e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc2640098a0 tx=0x7fc264009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2850b302f31902e server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:20:17.864 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:16.818+0000 7fc27159a700 1 -- 172.21.15.22:0/2355088705 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc264004180 con 0x7fc274106500 2024-02-16T19:20:17.864 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:16.818+0000 7fc27159a700 1 -- 172.21.15.22:0/2355088705 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc2640042e0 con 0x7fc274106500 2024-02-16T19:20:17.864 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:16.818+0000 7fc27159a700 1 -- 172.21.15.22:0/2355088705 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc264011af0 con 0x7fc274106500 2024-02-16T19:20:17.865 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:16.819+0000 7fc278c45700 1 -- 172.21.15.22:0/2355088705 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc274106500 msgr2=0x7fc2741088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:17.865 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:16.819+0000 7fc278c45700 1 --2- 172.21.15.22:0/2355088705 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc274106500 0x7fc2741088e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc2640098a0 tx=0x7fc264009bb0 comp rx=0 tx=0).stop 2024-02-16T19:20:17.865 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:16.819+0000 7fc278c45700 1 -- 172.21.15.22:0/2355088705 shutdown_connections 2024-02-16T19:20:17.865 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:16.819+0000 7fc278c45700 1 --2- 172.21.15.22:0/2355088705 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc274106500 0x7fc2741088e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:17.865 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:16.819+0000 7fc278c45700 1 -- 172.21.15.22:0/2355088705 >> 172.21.15.22:0/2355088705 conn(0x7fc2740ffee0 msgr2=0x7fc274102330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:20:17.865 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:16.819+0000 7fc278c45700 1 -- 172.21.15.22:0/2355088705 shutdown_connections 2024-02-16T19:20:17.865 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:16.819+0000 7fc278c45700 1 -- 172.21.15.22:0/2355088705 wait complete. 2024-02-16T19:20:17.865 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:16.820+0000 7fc278c45700 1 Processor -- start 2024-02-16T19:20:17.865 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:16.820+0000 7fc278c45700 1 -- start start 2024-02-16T19:20:17.865 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:16.820+0000 7fc278c45700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc274106500 0x7fc274199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:17.865 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:16.820+0000 7fc278c45700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc274199c50 con 0x7fc274106500 2024-02-16T19:20:17.865 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:16.820+0000 7fc27259c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc274106500 0x7fc274199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:20:17.865 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:16.821+0000 7fc27259c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc274106500 0x7fc274199710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59130/0 (socket says 172.21.15.22:59130) 2024-02-16T19:20:17.865 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:16.821+0000 7fc27259c700 1 -- 172.21.15.22:0/441901952 learned_addr learned my addr 172.21.15.22:0/441901952 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:20:17.865 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:16.821+0000 7fc27259c700 1 -- 172.21.15.22:0/441901952 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc264009550 con 0x7fc274106500 2024-02-16T19:20:17.866 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:16.821+0000 7fc27259c700 1 --2- 172.21.15.22:0/441901952 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc274106500 0x7fc274199710 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc264011670 tx=0x7fc264011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:20:17.866 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:16.821+0000 7fc2637fe700 1 -- 172.21.15.22:0/441901952 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc2640039c0 con 0x7fc274106500 2024-02-16T19:20:17.866 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:16.821+0000 7fc2637fe700 1 -- 172.21.15.22:0/441901952 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc264003b20 con 0x7fc274106500 2024-02-16T19:20:17.866 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:16.821+0000 7fc278c45700 1 -- 172.21.15.22:0/441901952 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc274199eb0 con 0x7fc274106500 2024-02-16T19:20:17.866 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:16.821+0000 7fc2637fe700 1 -- 172.21.15.22:0/441901952 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc2640247f0 con 0x7fc274106500 2024-02-16T19:20:17.866 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:16.821+0000 7fc278c45700 1 -- 172.21.15.22:0/441901952 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc27419ccb0 con 0x7fc274106500 2024-02-16T19:20:17.866 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:16.822+0000 7fc2637fe700 1 -- 172.21.15.22:0/441901952 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 12) v1 ==== 45097+0+0 (secure 0 0 0) 0x7fc26401ac70 con 0x7fc274106500 2024-02-16T19:20:17.866 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:16.822+0000 7fc278c45700 1 -- 172.21.15.22:0/441901952 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc274062380 con 0x7fc274106500 2024-02-16T19:20:17.866 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:16.822+0000 7fc2637fe700 1 --2- 172.21.15.22:0/441901952 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7fc25c038450 0x7fc25c03a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:17.866 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:16.822+0000 7fc2637fe700 1 -- 172.21.15.22:0/441901952 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc26404c3e0 con 0x7fc274106500 2024-02-16T19:20:17.866 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:16.823+0000 7fc271d9b700 1 --2- 172.21.15.22:0/441901952 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7fc25c038450 0x7fc25c03a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:20:17.866 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:16.823+0000 7fc271d9b700 1 --2- 172.21.15.22:0/441901952 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7fc25c038450 0x7fc25c03a900 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc268006fd0 tx=0x7fc268006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:20:17.866 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:16.831+0000 7fc2637fe700 1 -- 172.21.15.22:0/441901952 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fc264024a60 con 0x7fc274106500 2024-02-16T19:20:17.866 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:17.033+0000 7fc278c45700 1 -- 172.21.15.22:0/441901952 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7fc27419d540 con 0x7fc274106500 2024-02-16T19:20:17.866 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:17.033+0000 7fc2637fe700 1 -- 172.21.15.22:0/441901952 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7fc264018350 con 0x7fc274106500 2024-02-16T19:20:17.866 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:17.035+0000 7fc278c45700 1 -- 172.21.15.22:0/441901952 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7fc25c038450 msgr2=0x7fc25c03a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:17.867 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:17.035+0000 7fc278c45700 1 --2- 172.21.15.22:0/441901952 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7fc25c038450 0x7fc25c03a900 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc268006fd0 tx=0x7fc268006e40 comp rx=0 tx=0).stop 2024-02-16T19:20:17.867 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:17.035+0000 7fc278c45700 1 -- 172.21.15.22:0/441901952 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc274106500 msgr2=0x7fc274199710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:17.867 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:17.035+0000 7fc278c45700 1 --2- 172.21.15.22:0/441901952 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc274106500 0x7fc274199710 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc264011670 tx=0x7fc264011aa0 comp rx=0 tx=0).stop 2024-02-16T19:20:17.867 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:17.035+0000 7fc278c45700 1 -- 172.21.15.22:0/441901952 shutdown_connections 2024-02-16T19:20:17.867 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:17.035+0000 7fc278c45700 1 --2- 172.21.15.22:0/441901952 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7fc25c038450 0x7fc25c03a900 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:17.867 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:17.035+0000 7fc278c45700 1 --2- 172.21.15.22:0/441901952 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc274106500 0x7fc274199710 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:17.867 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:17.035+0000 7fc278c45700 1 -- 172.21.15.22:0/441901952 >> 172.21.15.22:0/441901952 conn(0x7fc2740ffee0 msgr2=0x7fc2741015e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:20:17.867 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:17.035+0000 7fc278c45700 1 -- 172.21.15.22:0/441901952 shutdown_connections 2024-02-16T19:20:17.867 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:17.035+0000 7fc278c45700 1 -- 172.21.15.22:0/441901952 wait complete. 2024-02-16T19:20:17.872 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-16T19:20:17.872 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout disabled 2024-02-16T19:20:17.880 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-16T19:20:17.880 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout inactive 2024-02-16T19:20:17.880 INFO:teuthology.orchestra.run.smithi022.stdout:firewalld.service is not enabled 2024-02-16T19:20:17.880 INFO:teuthology.orchestra.run.smithi022.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-02-16T19:20:17.882 INFO:teuthology.orchestra.run.smithi022.stdout:Ceph Dashboard is now available at: 2024-02-16T19:20:17.882 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:20:17.882 INFO:teuthology.orchestra.run.smithi022.stdout: URL: https://smithi022.front.sepia.ceph.com:8443/ 2024-02-16T19:20:17.882 INFO:teuthology.orchestra.run.smithi022.stdout: User: admin 2024-02-16T19:20:17.882 INFO:teuthology.orchestra.run.smithi022.stdout: Password: kbf6ehndh0 2024-02-16T19:20:17.882 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:20:17.883 INFO:teuthology.orchestra.run.smithi022.stdout:Saving cluster configuration to /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/config directory 2024-02-16T19:20:17.883 INFO:teuthology.orchestra.run.smithi022.stdout:Enabling autotune for osd_memory_target 2024-02-16T19:20:18.853 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:18 smithi022 ceph-mon[29221]: mgrmap e13: smithi022.gmdcch(active, since 6s) 2024-02-16T19:20:19.131 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:18.339+0000 7f30773ec700 1 Processor -- start 2024-02-16T19:20:19.131 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:18.339+0000 7f30773ec700 1 -- start start 2024-02-16T19:20:19.131 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:18.339+0000 7f30773ec700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3070105e70 0x7f3070106280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:19.131 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:18.339+0000 7f30773ec700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30701067c0 con 0x7f3070105e70 2024-02-16T19:20:19.131 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:18.339+0000 7f3075188700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3070105e70 0x7f3070106280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:20:19.131 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:18.339+0000 7f3075188700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3070105e70 0x7f3070106280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59146/0 (socket says 172.21.15.22:59146) 2024-02-16T19:20:19.131 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:18.339+0000 7f3075188700 1 -- 172.21.15.22:0/2480357829 learned_addr learned my addr 172.21.15.22:0/2480357829 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:20:19.131 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:18.340+0000 7f3075188700 1 -- 172.21.15.22:0/2480357829 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3070106900 con 0x7f3070105e70 2024-02-16T19:20:19.131 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:18.340+0000 7f3075188700 1 --2- 172.21.15.22:0/2480357829 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3070105e70 0x7f3070106280 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f306c0098a0 tx=0x7f306c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a78cb1dd6dc63a9 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:20:19.131 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:18.340+0000 7f3067fff700 1 -- 172.21.15.22:0/2480357829 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f306c004180 con 0x7f3070105e70 2024-02-16T19:20:19.131 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:18.340+0000 7f3067fff700 1 -- 172.21.15.22:0/2480357829 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f306c0042e0 con 0x7f3070105e70 2024-02-16T19:20:19.131 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:18.340+0000 7f3067fff700 1 -- 172.21.15.22:0/2480357829 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f306c011af0 con 0x7f3070105e70 2024-02-16T19:20:19.132 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:18.341+0000 7f30773ec700 1 -- 172.21.15.22:0/2480357829 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3070105e70 msgr2=0x7f3070106280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:19.132 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:18.341+0000 7f30773ec700 1 --2- 172.21.15.22:0/2480357829 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3070105e70 0x7f3070106280 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f306c0098a0 tx=0x7f306c009bb0 comp rx=0 tx=0).stop 2024-02-16T19:20:19.132 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:18.341+0000 7f30773ec700 1 -- 172.21.15.22:0/2480357829 shutdown_connections 2024-02-16T19:20:19.132 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:18.341+0000 7f30773ec700 1 --2- 172.21.15.22:0/2480357829 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3070105e70 0x7f3070106280 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:19.132 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:18.341+0000 7f30773ec700 1 -- 172.21.15.22:0/2480357829 >> 172.21.15.22:0/2480357829 conn(0x7f30700ff670 msgr2=0x7f3070101ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:20:19.132 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:18.341+0000 7f30773ec700 1 -- 172.21.15.22:0/2480357829 shutdown_connections 2024-02-16T19:20:19.132 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:18.341+0000 7f30773ec700 1 -- 172.21.15.22:0/2480357829 wait complete. 2024-02-16T19:20:19.132 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:18.341+0000 7f30773ec700 1 Processor -- start 2024-02-16T19:20:19.132 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:18.341+0000 7f30773ec700 1 -- start start 2024-02-16T19:20:19.132 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:18.342+0000 7f30773ec700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3070105e70 0x7f30701997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:19.132 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:18.342+0000 7f30773ec700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3070199ce0 con 0x7f3070105e70 2024-02-16T19:20:19.132 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:18.342+0000 7f3075188700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3070105e70 0x7f30701997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:20:19.132 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:18.342+0000 7f3075188700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3070105e70 0x7f30701997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59148/0 (socket says 172.21.15.22:59148) 2024-02-16T19:20:19.133 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:18.342+0000 7f3075188700 1 -- 172.21.15.22:0/2590806528 learned_addr learned my addr 172.21.15.22:0/2590806528 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:20:19.133 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:18.342+0000 7f3075188700 1 -- 172.21.15.22:0/2590806528 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f306c009550 con 0x7f3070105e70 2024-02-16T19:20:19.133 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:18.342+0000 7f3075188700 1 --2- 172.21.15.22:0/2590806528 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3070105e70 0x7f30701997a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f306c011670 tx=0x7f306c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:20:19.133 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:18.342+0000 7f30667fc700 1 -- 172.21.15.22:0/2590806528 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f306c0039c0 con 0x7f3070105e70 2024-02-16T19:20:19.133 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:18.342+0000 7f30667fc700 1 -- 172.21.15.22:0/2590806528 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f306c003b20 con 0x7f3070105e70 2024-02-16T19:20:19.133 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:18.342+0000 7f30773ec700 1 -- 172.21.15.22:0/2590806528 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3070199ee0 con 0x7f3070105e70 2024-02-16T19:20:19.133 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:18.342+0000 7f30667fc700 1 -- 172.21.15.22:0/2590806528 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f306c0247f0 con 0x7f3070105e70 2024-02-16T19:20:19.133 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:18.342+0000 7f30773ec700 1 -- 172.21.15.22:0/2590806528 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f307019a380 con 0x7f3070105e70 2024-02-16T19:20:19.133 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:18.343+0000 7f30667fc700 1 -- 172.21.15.22:0/2590806528 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7f306c01a430 con 0x7f3070105e70 2024-02-16T19:20:19.133 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:18.343+0000 7f30773ec700 1 -- 172.21.15.22:0/2590806528 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3070062380 con 0x7f3070105e70 2024-02-16T19:20:19.133 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:18.343+0000 7f30667fc700 1 --2- 172.21.15.22:0/2590806528 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7f305c0384f0 0x7f305c03a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:19.133 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:18.344+0000 7f30667fc700 1 -- 172.21.15.22:0/2590806528 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f306c04c4f0 con 0x7f3070105e70 2024-02-16T19:20:19.133 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:18.344+0000 7f3074987700 1 --2- 172.21.15.22:0/2590806528 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7f305c0384f0 0x7f305c03a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:20:19.133 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:18.344+0000 7f3074987700 1 --2- 172.21.15.22:0/2590806528 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7f305c0384f0 0x7f305c03a9a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3060006fd0 tx=0x7f3060006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:20:19.133 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:18.348+0000 7f30667fc700 1 -- 172.21.15.22:0/2590806528 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f306c018530 con 0x7f3070105e70 2024-02-16T19:20:19.134 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:18.547+0000 7f30773ec700 1 -- 172.21.15.22:0/2590806528 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f307019d280 con 0x7f3070105e70 2024-02-16T19:20:19.134 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:18.547+0000 7f30667fc700 1 -- 172.21.15.22:0/2590806528 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f306c018350 con 0x7f3070105e70 2024-02-16T19:20:19.134 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:18.549+0000 7f30773ec700 1 -- 172.21.15.22:0/2590806528 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7f305c0384f0 msgr2=0x7f305c03a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:19.134 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:18.549+0000 7f30773ec700 1 --2- 172.21.15.22:0/2590806528 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7f305c0384f0 0x7f305c03a9a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3060006fd0 tx=0x7f3060006e40 comp rx=0 tx=0).stop 2024-02-16T19:20:19.134 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:18.549+0000 7f30773ec700 1 -- 172.21.15.22:0/2590806528 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3070105e70 msgr2=0x7f30701997a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:19.134 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:18.549+0000 7f30773ec700 1 --2- 172.21.15.22:0/2590806528 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3070105e70 0x7f30701997a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f306c011670 tx=0x7f306c011aa0 comp rx=0 tx=0).stop 2024-02-16T19:20:19.134 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:18.549+0000 7f30773ec700 1 -- 172.21.15.22:0/2590806528 shutdown_connections 2024-02-16T19:20:19.134 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:18.549+0000 7f30773ec700 1 --2- 172.21.15.22:0/2590806528 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7f305c0384f0 0x7f305c03a9a0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:19.134 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:18.549+0000 7f30773ec700 1 --2- 172.21.15.22:0/2590806528 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3070105e70 0x7f30701997a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:19.134 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:18.549+0000 7f30773ec700 1 -- 172.21.15.22:0/2590806528 >> 172.21.15.22:0/2590806528 conn(0x7f30700ff670 msgr2=0x7f3070104330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:20:19.134 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:18.549+0000 7f30773ec700 1 -- 172.21.15.22:0/2590806528 shutdown_connections 2024-02-16T19:20:19.134 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:18.549+0000 7f30773ec700 1 -- 172.21.15.22:0/2590806528 wait complete. 2024-02-16T19:20:20.418 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:19.507+0000 7f662f0ce700 1 Processor -- start 2024-02-16T19:20:20.418 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:19.507+0000 7f662f0ce700 1 -- start start 2024-02-16T19:20:20.418 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:19.507+0000 7f662f0ce700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6628079cb0 0x7f6628108590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:20.418 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:19.507+0000 7f662f0ce700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6628108ad0 con 0x7f6628079cb0 2024-02-16T19:20:20.418 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:19.508+0000 7f662ce6a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6628079cb0 0x7f6628108590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:20:20.418 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:19.508+0000 7f662ce6a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6628079cb0 0x7f6628108590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59150/0 (socket says 172.21.15.22:59150) 2024-02-16T19:20:20.418 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:19.508+0000 7f662ce6a700 1 -- 172.21.15.22:0/2771438693 learned_addr learned my addr 172.21.15.22:0/2771438693 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:20:20.418 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:19.508+0000 7f662ce6a700 1 -- 172.21.15.22:0/2771438693 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6628108c10 con 0x7f6628079cb0 2024-02-16T19:20:20.418 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:19.508+0000 7f662ce6a700 1 --2- 172.21.15.22:0/2771438693 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6628079cb0 0x7f6628108590 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6620009e80 tx=0x7f662000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3dc251704a13983 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:20:20.418 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:19.508+0000 7f66277fe700 1 -- 172.21.15.22:0/2771438693 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f662000e070 con 0x7f6628079cb0 2024-02-16T19:20:20.418 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:19.508+0000 7f66277fe700 1 -- 172.21.15.22:0/2771438693 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f662000b8c0 con 0x7f6628079cb0 2024-02-16T19:20:20.419 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:19.508+0000 7f66277fe700 1 -- 172.21.15.22:0/2771438693 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f66200044d0 con 0x7f6628079cb0 2024-02-16T19:20:20.419 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:19.509+0000 7f662f0ce700 1 -- 172.21.15.22:0/2771438693 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6628079cb0 msgr2=0x7f6628108590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:20.419 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:19.509+0000 7f662f0ce700 1 --2- 172.21.15.22:0/2771438693 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6628079cb0 0x7f6628108590 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6620009e80 tx=0x7f662000b330 comp rx=0 tx=0).stop 2024-02-16T19:20:20.419 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:19.509+0000 7f662f0ce700 1 -- 172.21.15.22:0/2771438693 shutdown_connections 2024-02-16T19:20:20.419 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:19.509+0000 7f662f0ce700 1 --2- 172.21.15.22:0/2771438693 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6628079cb0 0x7f6628108590 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:20.419 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:19.509+0000 7f662f0ce700 1 -- 172.21.15.22:0/2771438693 >> 172.21.15.22:0/2771438693 conn(0x7f66280fff00 msgr2=0x7f6628102350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:20:20.419 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:19.509+0000 7f662f0ce700 1 -- 172.21.15.22:0/2771438693 shutdown_connections 2024-02-16T19:20:20.419 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:19.509+0000 7f662f0ce700 1 -- 172.21.15.22:0/2771438693 wait complete. 2024-02-16T19:20:20.419 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:19.510+0000 7f662f0ce700 1 Processor -- start 2024-02-16T19:20:20.419 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:19.510+0000 7f662f0ce700 1 -- start start 2024-02-16T19:20:20.419 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:19.510+0000 7f662f0ce700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6628079cb0 0x7f6628199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:20.419 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:19.510+0000 7f662f0ce700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6628199c90 con 0x7f6628079cb0 2024-02-16T19:20:20.419 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:19.510+0000 7f662ce6a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6628079cb0 0x7f6628199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:20:20.419 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:19.510+0000 7f662ce6a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6628079cb0 0x7f6628199750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59162/0 (socket says 172.21.15.22:59162) 2024-02-16T19:20:20.419 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:19.510+0000 7f662ce6a700 1 -- 172.21.15.22:0/3000963030 learned_addr learned my addr 172.21.15.22:0/3000963030 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:20:20.420 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:19.510+0000 7f662ce6a700 1 -- 172.21.15.22:0/3000963030 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6620009b30 con 0x7f6628079cb0 2024-02-16T19:20:20.420 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:19.510+0000 7f662ce6a700 1 --2- 172.21.15.22:0/3000963030 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6628079cb0 0x7f6628199750 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6620004070 tx=0x7f66200049f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:20:20.420 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:19.510+0000 7f6625ffb700 1 -- 172.21.15.22:0/3000963030 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f662000e040 con 0x7f6628079cb0 2024-02-16T19:20:20.420 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:19.510+0000 7f6625ffb700 1 -- 172.21.15.22:0/3000963030 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f662000bc40 con 0x7f6628079cb0 2024-02-16T19:20:20.420 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:19.510+0000 7f662f0ce700 1 -- 172.21.15.22:0/3000963030 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6628199e90 con 0x7f6628079cb0 2024-02-16T19:20:20.420 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:19.511+0000 7f6625ffb700 1 -- 172.21.15.22:0/3000963030 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f66200125e0 con 0x7f6628079cb0 2024-02-16T19:20:20.420 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:19.511+0000 7f662f0ce700 1 -- 172.21.15.22:0/3000963030 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f662819a390 con 0x7f6628079cb0 2024-02-16T19:20:20.420 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:19.511+0000 7f6625ffb700 1 -- 172.21.15.22:0/3000963030 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7f6620019040 con 0x7f6628079cb0 2024-02-16T19:20:20.420 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:19.512+0000 7f662f0ce700 1 -- 172.21.15.22:0/3000963030 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6628062380 con 0x7f6628079cb0 2024-02-16T19:20:20.420 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:19.512+0000 7f6625ffb700 1 --2- 172.21.15.22:0/3000963030 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7f6610038450 0x7f661003a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:20.420 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:19.512+0000 7f6625ffb700 1 -- 172.21.15.22:0/3000963030 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f662004be90 con 0x7f6628079cb0 2024-02-16T19:20:20.420 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:19.512+0000 7f6627fff700 1 --2- 172.21.15.22:0/3000963030 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7f6610038450 0x7f661003a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:20:20.420 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:19.512+0000 7f6627fff700 1 --2- 172.21.15.22:0/3000963030 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7f6610038450 0x7f661003a900 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f661c006fd0 tx=0x7f661c006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:20:20.421 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:19.517+0000 7f6625ffb700 1 -- 172.21.15.22:0/3000963030 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f662001a570 con 0x7f6628079cb0 2024-02-16T19:20:20.421 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:19.827+0000 7f662f0ce700 1 -- 172.21.15.22:0/3000963030 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f662819ce00 con 0x7f6628079cb0 2024-02-16T19:20:20.421 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:19.830+0000 7f6625ffb700 1 -- 172.21.15.22:0/3000963030 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v34)=0 set mgr/dashboard/cluster/status v34) v1 ==== 153+0+0 (secure 0 0 0) 0x7f662001f020 con 0x7f6628079cb0 2024-02-16T19:20:20.421 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:19.832+0000 7f662f0ce700 1 -- 172.21.15.22:0/3000963030 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7f6610038450 msgr2=0x7f661003a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:20.421 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:19.832+0000 7f662f0ce700 1 --2- 172.21.15.22:0/3000963030 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7f6610038450 0x7f661003a900 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f661c006fd0 tx=0x7f661c006e40 comp rx=0 tx=0).stop 2024-02-16T19:20:20.421 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:19.832+0000 7f662f0ce700 1 -- 172.21.15.22:0/3000963030 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6628079cb0 msgr2=0x7f6628199750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:20.421 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:19.832+0000 7f662f0ce700 1 --2- 172.21.15.22:0/3000963030 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6628079cb0 0x7f6628199750 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6620004070 tx=0x7f66200049f0 comp rx=0 tx=0).stop 2024-02-16T19:20:20.421 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:19.832+0000 7f662f0ce700 1 -- 172.21.15.22:0/3000963030 shutdown_connections 2024-02-16T19:20:20.421 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:19.832+0000 7f662f0ce700 1 --2- 172.21.15.22:0/3000963030 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7f6610038450 0x7f661003a900 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:20.421 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:19.832+0000 7f662f0ce700 1 --2- 172.21.15.22:0/3000963030 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6628079cb0 0x7f6628199750 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:20.421 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:19.832+0000 7f662f0ce700 1 -- 172.21.15.22:0/3000963030 >> 172.21.15.22:0/3000963030 conn(0x7f66280fff00 msgr2=0x7f6628100d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:20:20.421 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:19.832+0000 7f662f0ce700 1 -- 172.21.15.22:0/3000963030 shutdown_connections 2024-02-16T19:20:20.421 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-02-16T19:20:19.832+0000 7f662f0ce700 1 -- 172.21.15.22:0/3000963030 wait complete. 2024-02-16T19:20:20.421 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-02-16T19:20:20.422 INFO:teuthology.orchestra.run.smithi022.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-02-16T19:20:20.422 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:20:20.422 INFO:teuthology.orchestra.run.smithi022.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-02-16T19:20:20.422 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:20:20.422 INFO:teuthology.orchestra.run.smithi022.stdout:Or, if you are only running a single cluster on this host: 2024-02-16T19:20:20.422 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:20:20.422 INFO:teuthology.orchestra.run.smithi022.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-02-16T19:20:20.422 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:20:20.422 INFO:teuthology.orchestra.run.smithi022.stdout:Please consider enabling telemetry to help improve Ceph: 2024-02-16T19:20:20.422 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:20:20.422 INFO:teuthology.orchestra.run.smithi022.stdout: ceph telemetry on 2024-02-16T19:20:20.422 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:20:20.422 INFO:teuthology.orchestra.run.smithi022.stdout:For more information see: 2024-02-16T19:20:20.422 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:20:20.422 INFO:teuthology.orchestra.run.smithi022.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-02-16T19:20:20.423 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:20:20.423 INFO:teuthology.orchestra.run.smithi022.stdout:Bootstrap complete. 2024-02-16T19:20:20.453 INFO:tasks.cephadm:Fetching config... 2024-02-16T19:20:20.454 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-02-16T19:20:20.454 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-02-16T19:20:20.469 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-02-16T19:20:20.469 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-02-16T19:20:20.469 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-02-16T19:20:20.525 INFO:tasks.cephadm:Fetching mon keyring... 2024-02-16T19:20:20.525 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-02-16T19:20:20.525 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi022/keyring of=/dev/stdout 2024-02-16T19:20:20.591 INFO:tasks.cephadm:Fetching pub ssh key... 2024-02-16T19:20:20.592 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-02-16T19:20:20.592 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-02-16T19:20:20.646 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-02-16T19:20:20.647 DEBUG:teuthology.orchestra.run.smithi022:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDvQbslCAhk7mMXzrGiXKtg1AoNmCl8HuGZevMJKL499hzGfD5iy3wyrLQg+D94evPSH9ToWEupNCcJsUFiEhE/A45TA2uGTD8mtqrEY0Abrm3Wt0x/nmLqq9GBjZ5CMf7LPlp8n8HdyqwTiDXZXD10QULQc27L3LqMdTSCSzlnEdXFjz7XHNytb5XJgi6WuYmTBnnGIWfM9ShTbPfOwmGbCAuDADspetUtlfhsaRe/hjAlPXAkKpkz5w46sMam51fK3fNi2XDGgNNoW8yJj/sYFLWPZxzd0EemrS5d7Q5qpgXlT6u/Nbfgn8ra7QyWT5TmOEN+vLWknQzRD7N+6Q64hRDFa2xhP0D4LXTN8ClNjOClGgfYBST73pKiWDVZvJxsFTXcdqRZKDHKk3E0aEenRtGvtmvQAluzjjFvDePlP7j/GBXj5XDOUH5xglXNIZFkaLuBJWpxXkMshW/Pkd4mUjyUmmhzjSC8bLBUAdvcNc81wvcviQAFLgc1nk387XU= ceph-2a1346e6-cd00-11ee-95ba-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-16T19:20:20.729 INFO:teuthology.orchestra.run.smithi022.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDvQbslCAhk7mMXzrGiXKtg1AoNmCl8HuGZevMJKL499hzGfD5iy3wyrLQg+D94evPSH9ToWEupNCcJsUFiEhE/A45TA2uGTD8mtqrEY0Abrm3Wt0x/nmLqq9GBjZ5CMf7LPlp8n8HdyqwTiDXZXD10QULQc27L3LqMdTSCSzlnEdXFjz7XHNytb5XJgi6WuYmTBnnGIWfM9ShTbPfOwmGbCAuDADspetUtlfhsaRe/hjAlPXAkKpkz5w46sMam51fK3fNi2XDGgNNoW8yJj/sYFLWPZxzd0EemrS5d7Q5qpgXlT6u/Nbfgn8ra7QyWT5TmOEN+vLWknQzRD7N+6Q64hRDFa2xhP0D4LXTN8ClNjOClGgfYBST73pKiWDVZvJxsFTXcdqRZKDHKk3E0aEenRtGvtmvQAluzjjFvDePlP7j/GBXj5XDOUH5xglXNIZFkaLuBJWpxXkMshW/Pkd4mUjyUmmhzjSC8bLBUAdvcNc81wvcviQAFLgc1nk387XU= ceph-2a1346e6-cd00-11ee-95ba-87774f69a715 2024-02-16T19:20:20.744 DEBUG:teuthology.orchestra.run.smithi117:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDvQbslCAhk7mMXzrGiXKtg1AoNmCl8HuGZevMJKL499hzGfD5iy3wyrLQg+D94evPSH9ToWEupNCcJsUFiEhE/A45TA2uGTD8mtqrEY0Abrm3Wt0x/nmLqq9GBjZ5CMf7LPlp8n8HdyqwTiDXZXD10QULQc27L3LqMdTSCSzlnEdXFjz7XHNytb5XJgi6WuYmTBnnGIWfM9ShTbPfOwmGbCAuDADspetUtlfhsaRe/hjAlPXAkKpkz5w46sMam51fK3fNi2XDGgNNoW8yJj/sYFLWPZxzd0EemrS5d7Q5qpgXlT6u/Nbfgn8ra7QyWT5TmOEN+vLWknQzRD7N+6Q64hRDFa2xhP0D4LXTN8ClNjOClGgfYBST73pKiWDVZvJxsFTXcdqRZKDHKk3E0aEenRtGvtmvQAluzjjFvDePlP7j/GBXj5XDOUH5xglXNIZFkaLuBJWpxXkMshW/Pkd4mUjyUmmhzjSC8bLBUAdvcNc81wvcviQAFLgc1nk387XU= ceph-2a1346e6-cd00-11ee-95ba-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-16T19:20:20.796 INFO:teuthology.orchestra.run.smithi117.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDvQbslCAhk7mMXzrGiXKtg1AoNmCl8HuGZevMJKL499hzGfD5iy3wyrLQg+D94evPSH9ToWEupNCcJsUFiEhE/A45TA2uGTD8mtqrEY0Abrm3Wt0x/nmLqq9GBjZ5CMf7LPlp8n8HdyqwTiDXZXD10QULQc27L3LqMdTSCSzlnEdXFjz7XHNytb5XJgi6WuYmTBnnGIWfM9ShTbPfOwmGbCAuDADspetUtlfhsaRe/hjAlPXAkKpkz5w46sMam51fK3fNi2XDGgNNoW8yJj/sYFLWPZxzd0EemrS5d7Q5qpgXlT6u/Nbfgn8ra7QyWT5TmOEN+vLWknQzRD7N+6Q64hRDFa2xhP0D4LXTN8ClNjOClGgfYBST73pKiWDVZvJxsFTXcdqRZKDHKk3E0aEenRtGvtmvQAluzjjFvDePlP7j/GBXj5XDOUH5xglXNIZFkaLuBJWpxXkMshW/Pkd4mUjyUmmhzjSC8bLBUAdvcNc81wvcviQAFLgc1nk387XU= ceph-2a1346e6-cd00-11ee-95ba-87774f69a715 2024-02-16T19:20:20.811 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-02-16T19:20:20.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:20 smithi022 ceph-mon[29221]: from='client.? 172.21.15.22:0/3000963030' entity='client.admin' 2024-02-16T19:20:21.018 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi022/config 2024-02-16T19:20:23.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:23.563+0000 7fa368426700 1 -- 172.21.15.22:0/3602395354 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3600ffe20 msgr2=0x7fa360100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:23.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:23.563+0000 7fa368426700 1 --2- 172.21.15.22:0/3602395354 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3600ffe20 0x7fa360100230 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa35c009910 tx=0x7fa35c009c20 comp rx=0 tx=0).stop 2024-02-16T19:20:23.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:23.563+0000 7fa368426700 1 -- 172.21.15.22:0/3602395354 shutdown_connections 2024-02-16T19:20:23.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:23.563+0000 7fa368426700 1 --2- 172.21.15.22:0/3602395354 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3600ffe20 0x7fa360100230 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:23.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:23.563+0000 7fa368426700 1 -- 172.21.15.22:0/3602395354 >> 172.21.15.22:0/3602395354 conn(0x7fa3600fb3d0 msgr2=0x7fa3600fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:20:23.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:23.563+0000 7fa368426700 1 -- 172.21.15.22:0/3602395354 shutdown_connections 2024-02-16T19:20:23.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:23.563+0000 7fa368426700 1 -- 172.21.15.22:0/3602395354 wait complete. 2024-02-16T19:20:23.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:23.564+0000 7fa368426700 1 Processor -- start 2024-02-16T19:20:23.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:23.564+0000 7fa368426700 1 -- start start 2024-02-16T19:20:23.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:23.564+0000 7fa368426700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3600ffe20 0x7fa360194f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:23.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:23.564+0000 7fa368426700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3601954c0 con 0x7fa3600ffe20 2024-02-16T19:20:23.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:23.564+0000 7fa3661c2700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3600ffe20 0x7fa360194f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:20:23.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:23.564+0000 7fa3661c2700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3600ffe20 0x7fa360194f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59180/0 (socket says 172.21.15.22:59180) 2024-02-16T19:20:23.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:23.564+0000 7fa3661c2700 1 -- 172.21.15.22:0/237596502 learned_addr learned my addr 172.21.15.22:0/237596502 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:20:23.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:23.565+0000 7fa3661c2700 1 -- 172.21.15.22:0/237596502 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa35c0095f0 con 0x7fa3600ffe20 2024-02-16T19:20:23.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:23.565+0000 7fa3661c2700 1 --2- 172.21.15.22:0/237596502 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3600ffe20 0x7fa360194f80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa35c005b40 tx=0x7fa35c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:20:23.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:23.565+0000 7fa3577fe700 1 -- 172.21.15.22:0/237596502 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa35c01d070 con 0x7fa3600ffe20 2024-02-16T19:20:23.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:23.565+0000 7fa3577fe700 1 -- 172.21.15.22:0/237596502 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa35c00be10 con 0x7fa3600ffe20 2024-02-16T19:20:23.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:23.565+0000 7fa368426700 1 -- 172.21.15.22:0/237596502 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3601956c0 con 0x7fa3600ffe20 2024-02-16T19:20:23.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:23.565+0000 7fa368426700 1 -- 172.21.15.22:0/237596502 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa360195b60 con 0x7fa3600ffe20 2024-02-16T19:20:23.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:23.565+0000 7fa3577fe700 1 -- 172.21.15.22:0/237596502 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa35c00f460 con 0x7fa3600ffe20 2024-02-16T19:20:23.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:23.566+0000 7fa3577fe700 1 -- 172.21.15.22:0/237596502 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7fa35c00f5c0 con 0x7fa3600ffe20 2024-02-16T19:20:23.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:23.566+0000 7fa3577fe700 1 --2- 172.21.15.22:0/237596502 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7fa34c0384e0 0x7fa34c03a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:23.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:23.566+0000 7fa368426700 1 -- 172.21.15.22:0/237596502 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa36018ea80 con 0x7fa3600ffe20 2024-02-16T19:20:23.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:23.566+0000 7fa3577fe700 1 -- 172.21.15.22:0/237596502 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa35c04c2b0 con 0x7fa3600ffe20 2024-02-16T19:20:23.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:23.566+0000 7fa3659c1700 1 --2- 172.21.15.22:0/237596502 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7fa34c0384e0 0x7fa34c03a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:20:23.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:23.567+0000 7fa3659c1700 1 --2- 172.21.15.22:0/237596502 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7fa34c0384e0 0x7fa34c03a990 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa350006fd0 tx=0x7fa350006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:20:23.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:23.571+0000 7fa3577fe700 1 -- 172.21.15.22:0/237596502 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fa35c020960 con 0x7fa3600ffe20 2024-02-16T19:20:23.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:23.770+0000 7fa368426700 1 -- 172.21.15.22:0/237596502 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7fa36002cc30 con 0x7fa3600ffe20 2024-02-16T19:20:23.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:23.773+0000 7fa3577fe700 1 -- 172.21.15.22:0/237596502 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7fa35c027030 con 0x7fa3600ffe20 2024-02-16T19:20:23.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:23.776+0000 7fa368426700 1 -- 172.21.15.22:0/237596502 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7fa34c0384e0 msgr2=0x7fa34c03a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:23.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:23.776+0000 7fa368426700 1 --2- 172.21.15.22:0/237596502 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7fa34c0384e0 0x7fa34c03a990 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa350006fd0 tx=0x7fa350006e40 comp rx=0 tx=0).stop 2024-02-16T19:20:23.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:23.776+0000 7fa368426700 1 -- 172.21.15.22:0/237596502 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3600ffe20 msgr2=0x7fa360194f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:23.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:23.776+0000 7fa368426700 1 --2- 172.21.15.22:0/237596502 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3600ffe20 0x7fa360194f80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa35c005b40 tx=0x7fa35c004740 comp rx=0 tx=0).stop 2024-02-16T19:20:23.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:23.776+0000 7fa368426700 1 -- 172.21.15.22:0/237596502 shutdown_connections 2024-02-16T19:20:23.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:23.776+0000 7fa368426700 1 --2- 172.21.15.22:0/237596502 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7fa34c0384e0 0x7fa34c03a990 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:23.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:23.776+0000 7fa368426700 1 --2- 172.21.15.22:0/237596502 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3600ffe20 0x7fa360194f80 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:23.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:23.776+0000 7fa368426700 1 -- 172.21.15.22:0/237596502 >> 172.21.15.22:0/237596502 conn(0x7fa3600fb3d0 msgr2=0x7fa3600fc080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:20:23.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:23.776+0000 7fa368426700 1 -- 172.21.15.22:0/237596502 shutdown_connections 2024-02-16T19:20:23.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:23.776+0000 7fa368426700 1 -- 172.21.15.22:0/237596502 wait complete. 2024-02-16T19:20:24.711 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-02-16T19:20:24.712 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-02-16T19:20:24.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:24 smithi022 ceph-mon[29221]: from='client.? 172.21.15.22:0/237596502' entity='client.admin' 2024-02-16T19:20:24.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:24 smithi022 ceph-mon[29221]: from='mgr.14164 172.21.15.22:0/3311464841' entity='mgr.smithi022.gmdcch' 2024-02-16T19:20:24.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:24 smithi022 ceph-mon[29221]: from='mgr.14164 172.21.15.22:0/3311464841' entity='mgr.smithi022.gmdcch' 2024-02-16T19:20:24.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:24 smithi022 ceph-mon[29221]: from='mgr.14164 172.21.15.22:0/3311464841' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "who": "osd/host:smithi022", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:20:24.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:24 smithi022 ceph-mon[29221]: from='mgr.14164 172.21.15.22:0/3311464841' entity='mgr.smithi022.gmdcch' 2024-02-16T19:20:24.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:24 smithi022 ceph-mon[29221]: from='mgr.14164 172.21.15.22:0/3311464841' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi022", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-02-16T19:20:24.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:24 smithi022 ceph-mon[29221]: from='mgr.14164 172.21.15.22:0/3311464841' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi022", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-02-16T19:20:24.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:24 smithi022 ceph-mon[29221]: from='mgr.14164 172.21.15.22:0/3311464841' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:20:25.501 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi022/config 2024-02-16T19:20:25.887 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:25 smithi022 ceph-mon[29221]: Deploying daemon ceph-exporter.smithi022 on smithi022 2024-02-16T19:20:27.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:27.469+0000 7f008d898700 1 -- 172.21.15.22:0/3968633090 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0088074c60 msgr2=0x7f0088073160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:27.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:27.469+0000 7f008d898700 1 --2- 172.21.15.22:0/3968633090 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0088074c60 0x7f0088073160 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f008000b0a0 tx=0x7f008000b3b0 comp rx=0 tx=0).stop 2024-02-16T19:20:27.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:27.469+0000 7f008d898700 1 -- 172.21.15.22:0/3968633090 shutdown_connections 2024-02-16T19:20:27.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:27.469+0000 7f008d898700 1 --2- 172.21.15.22:0/3968633090 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0088074c60 0x7f0088073160 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:27.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:27.469+0000 7f008d898700 1 -- 172.21.15.22:0/3968633090 >> 172.21.15.22:0/3968633090 conn(0x7f00880fb470 msgr2=0x7f00880fd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:20:27.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:27.469+0000 7f008d898700 1 -- 172.21.15.22:0/3968633090 shutdown_connections 2024-02-16T19:20:27.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:27.469+0000 7f008d898700 1 -- 172.21.15.22:0/3968633090 wait complete. 2024-02-16T19:20:27.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:27.469+0000 7f008d898700 1 Processor -- start 2024-02-16T19:20:27.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:27.470+0000 7f008d898700 1 -- start start 2024-02-16T19:20:27.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:27.470+0000 7f008d898700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0088074c60 0x7f0088197180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:27.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:27.470+0000 7f008d898700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00881976c0 con 0x7f0088074c60 2024-02-16T19:20:27.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:27.470+0000 7f0086ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0088074c60 0x7f0088197180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:20:27.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:27.470+0000 7f0086ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0088074c60 0x7f0088197180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54062/0 (socket says 172.21.15.22:54062) 2024-02-16T19:20:27.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:27.470+0000 7f0086ffd700 1 -- 172.21.15.22:0/3586968090 learned_addr learned my addr 172.21.15.22:0/3586968090 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:20:27.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:27.470+0000 7f0086ffd700 1 -- 172.21.15.22:0/3586968090 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0080009d00 con 0x7f0088074c60 2024-02-16T19:20:27.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:27.470+0000 7f0086ffd700 1 --2- 172.21.15.22:0/3586968090 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0088074c60 0x7f0088197180 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f008000b830 tx=0x7f008000bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:20:27.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:27.471+0000 7f008c896700 1 -- 172.21.15.22:0/3586968090 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f008000e040 con 0x7f0088074c60 2024-02-16T19:20:27.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:27.471+0000 7f008c896700 1 -- 172.21.15.22:0/3586968090 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0080012af0 con 0x7f0088074c60 2024-02-16T19:20:27.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:27.471+0000 7f008d898700 1 -- 172.21.15.22:0/3586968090 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00881978c0 con 0x7f0088074c60 2024-02-16T19:20:27.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:27.471+0000 7f008c896700 1 -- 172.21.15.22:0/3586968090 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f008001d9f0 con 0x7f0088074c60 2024-02-16T19:20:27.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:27.471+0000 7f008d898700 1 -- 172.21.15.22:0/3586968090 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0088197ce0 con 0x7f0088074c60 2024-02-16T19:20:27.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:27.471+0000 7f008c896700 1 -- 172.21.15.22:0/3586968090 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7f008001dc60 con 0x7f0088074c60 2024-02-16T19:20:27.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:27.471+0000 7f008d898700 1 -- 172.21.15.22:0/3586968090 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00880623f0 con 0x7f0088074c60 2024-02-16T19:20:27.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:27.472+0000 7f008c896700 1 --2- 172.21.15.22:0/3586968090 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7f00740384e0 0x7f007403a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:27.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:27.472+0000 7f008c896700 1 -- 172.21.15.22:0/3586968090 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f008004c870 con 0x7f0088074c60 2024-02-16T19:20:27.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:27.472+0000 7f00867fc700 1 --2- 172.21.15.22:0/3586968090 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7f00740384e0 0x7f007403a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:20:27.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:27.472+0000 7f00867fc700 1 --2- 172.21.15.22:0/3586968090 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7f00740384e0 0x7f007403a990 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0070006fd0 tx=0x7f0070006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:20:27.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:27.477+0000 7f008c896700 1 -- 172.21.15.22:0/3586968090 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f00880623f0 con 0x7f0088074c60 2024-02-16T19:20:27.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:27.699+0000 7f008d898700 1 -- 172.21.15.22:0/3586968090 --> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f0088062410 con 0x7f00740384e0 2024-02-16T19:20:27.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:27.702+0000 7f008c896700 1 -- 172.21.15.22:0/3586968090 <== mgr.14164 v2:172.21.15.22:6800/2945983569 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f0070009e20 con 0x7f00740384e0 2024-02-16T19:20:27.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:27.704+0000 7f008d898700 1 -- 172.21.15.22:0/3586968090 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7f00740384e0 msgr2=0x7f007403a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:27.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:27.704+0000 7f008d898700 1 --2- 172.21.15.22:0/3586968090 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7f00740384e0 0x7f007403a990 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0070006fd0 tx=0x7f0070006e40 comp rx=0 tx=0).stop 2024-02-16T19:20:27.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:27.704+0000 7f008d898700 1 -- 172.21.15.22:0/3586968090 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0088074c60 msgr2=0x7f0088197180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:27.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:27.704+0000 7f008d898700 1 --2- 172.21.15.22:0/3586968090 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0088074c60 0x7f0088197180 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f008000b830 tx=0x7f008000bd40 comp rx=0 tx=0).stop 2024-02-16T19:20:27.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:27.704+0000 7f008d898700 1 -- 172.21.15.22:0/3586968090 shutdown_connections 2024-02-16T19:20:27.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:27.704+0000 7f008d898700 1 --2- 172.21.15.22:0/3586968090 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7f00740384e0 0x7f007403a990 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:27.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:27.704+0000 7f008d898700 1 --2- 172.21.15.22:0/3586968090 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0088074c60 0x7f0088197180 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:27.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:27.704+0000 7f008d898700 1 -- 172.21.15.22:0/3586968090 >> 172.21.15.22:0/3586968090 conn(0x7f00880fb470 msgr2=0x7f00880fc120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:20:27.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:27.705+0000 7f008d898700 1 -- 172.21.15.22:0/3586968090 shutdown_connections 2024-02-16T19:20:27.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:27.705+0000 7f008d898700 1 -- 172.21.15.22:0/3586968090 wait complete. 2024-02-16T19:20:28.641 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi117 2024-02-16T19:20:28.641 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-02-16T19:20:28.641 DEBUG:teuthology.orchestra.run.smithi117:> dd of=/etc/ceph/ceph.conf 2024-02-16T19:20:28.661 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-02-16T19:20:28.661 DEBUG:teuthology.orchestra.run.smithi117:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-16T19:20:28.718 INFO:tasks.cephadm:Adding host smithi117 to orchestrator... 2024-02-16T19:20:28.718 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -- ceph orch host add smithi117 2024-02-16T19:20:28.744 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:28 smithi022 ceph-mon[29221]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:20:28.744 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:28 smithi022 ceph-mon[29221]: from='mgr.14164 172.21.15.22:0/3311464841' entity='mgr.smithi022.gmdcch' 2024-02-16T19:20:28.744 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:28 smithi022 ceph-mon[29221]: from='mgr.14164 172.21.15.22:0/3311464841' entity='mgr.smithi022.gmdcch' 2024-02-16T19:20:28.744 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:28 smithi022 ceph-mon[29221]: from='mgr.14164 172.21.15.22:0/3311464841' entity='mgr.smithi022.gmdcch' 2024-02-16T19:20:28.744 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:28 smithi022 ceph-mon[29221]: from='mgr.14164 172.21.15.22:0/3311464841' entity='mgr.smithi022.gmdcch' 2024-02-16T19:20:28.744 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:28 smithi022 ceph-mon[29221]: from='mgr.14164 172.21.15.22:0/3311464841' entity='mgr.smithi022.gmdcch' 2024-02-16T19:20:28.745 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:28 smithi022 ceph-mon[29221]: from='mgr.14164 172.21.15.22:0/3311464841' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi022", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-16T19:20:28.745 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:28 smithi022 ceph-mon[29221]: from='mgr.14164 172.21.15.22:0/3311464841' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi022", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-02-16T19:20:28.745 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:28 smithi022 ceph-mon[29221]: from='mgr.14164 172.21.15.22:0/3311464841' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:20:29.814 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi022/config 2024-02-16T19:20:29.953 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:29 smithi022 ceph-mon[29221]: Deploying daemon crash.smithi022 on smithi022 2024-02-16T19:20:31.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:31.478+0000 7fc82ec23700 1 -- 172.21.15.22:0/2529723239 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc8280ffe30 msgr2=0x7fc828100240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:31.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:31.478+0000 7fc82ec23700 1 --2- 172.21.15.22:0/2529723239 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc8280ffe30 0x7fc828100240 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc81c009910 tx=0x7fc81c009c20 comp rx=0 tx=0).stop 2024-02-16T19:20:31.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:31.478+0000 7fc82ec23700 1 -- 172.21.15.22:0/2529723239 shutdown_connections 2024-02-16T19:20:31.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:31.478+0000 7fc82ec23700 1 --2- 172.21.15.22:0/2529723239 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc8280ffe30 0x7fc828100240 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:31.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:31.478+0000 7fc82ec23700 1 -- 172.21.15.22:0/2529723239 >> 172.21.15.22:0/2529723239 conn(0x7fc8280fb3c0 msgr2=0x7fc8280fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:20:31.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:31.478+0000 7fc82ec23700 1 -- 172.21.15.22:0/2529723239 shutdown_connections 2024-02-16T19:20:31.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:31.478+0000 7fc82ec23700 1 -- 172.21.15.22:0/2529723239 wait complete. 2024-02-16T19:20:31.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:31.479+0000 7fc82ec23700 1 Processor -- start 2024-02-16T19:20:31.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:31.479+0000 7fc82ec23700 1 -- start start 2024-02-16T19:20:31.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:31.479+0000 7fc82ec23700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc8280ffe30 0x7fc828074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:31.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:31.479+0000 7fc82ec23700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc828075030 con 0x7fc8280ffe30 2024-02-16T19:20:31.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:31.479+0000 7fc82c9bf700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc8280ffe30 0x7fc828074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:20:31.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:31.479+0000 7fc82c9bf700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc8280ffe30 0x7fc828074af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54090/0 (socket says 172.21.15.22:54090) 2024-02-16T19:20:31.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:31.479+0000 7fc82c9bf700 1 -- 172.21.15.22:0/3773557201 learned_addr learned my addr 172.21.15.22:0/3773557201 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:20:31.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:31.479+0000 7fc82c9bf700 1 -- 172.21.15.22:0/3773557201 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc81c0095f0 con 0x7fc8280ffe30 2024-02-16T19:20:31.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:31.479+0000 7fc82c9bf700 1 --2- 172.21.15.22:0/3773557201 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc8280ffe30 0x7fc828074af0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc81c005b40 tx=0x7fc81c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:20:31.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:31.480+0000 7fc825ffb700 1 -- 172.21.15.22:0/3773557201 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc81c01d070 con 0x7fc8280ffe30 2024-02-16T19:20:31.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:31.480+0000 7fc825ffb700 1 -- 172.21.15.22:0/3773557201 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc81c00be10 con 0x7fc8280ffe30 2024-02-16T19:20:31.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:31.480+0000 7fc82ec23700 1 -- 172.21.15.22:0/3773557201 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc828073140 con 0x7fc8280ffe30 2024-02-16T19:20:31.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:31.480+0000 7fc825ffb700 1 -- 172.21.15.22:0/3773557201 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc81c00f460 con 0x7fc8280ffe30 2024-02-16T19:20:31.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:31.480+0000 7fc82ec23700 1 -- 172.21.15.22:0/3773557201 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8280735e0 con 0x7fc8280ffe30 2024-02-16T19:20:31.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:31.481+0000 7fc825ffb700 1 -- 172.21.15.22:0/3773557201 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7fc81c00f5c0 con 0x7fc8280ffe30 2024-02-16T19:20:31.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:31.481+0000 7fc825ffb700 1 --2- 172.21.15.22:0/3773557201 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7fc8180384e0 0x7fc81803a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:31.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:31.481+0000 7fc825ffb700 1 -- 172.21.15.22:0/3773557201 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc81c0227e0 con 0x7fc8280ffe30 2024-02-16T19:20:31.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:31.481+0000 7fc827fff700 1 --2- 172.21.15.22:0/3773557201 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7fc8180384e0 0x7fc81803a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:20:31.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:31.481+0000 7fc82ec23700 1 -- 172.21.15.22:0/3773557201 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc828073ff0 con 0x7fc8280ffe30 2024-02-16T19:20:31.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:31.481+0000 7fc827fff700 1 --2- 172.21.15.22:0/3773557201 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7fc8180384e0 0x7fc81803a990 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc814006fd0 tx=0x7fc814006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:20:31.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:31.490+0000 7fc825ffb700 1 -- 172.21.15.22:0/3773557201 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fc81c027030 con 0x7fc8280ffe30 2024-02-16T19:20:31.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:31.710+0000 7fc82ec23700 1 -- 172.21.15.22:0/3773557201 --> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi117", "target": ["mon-mgr", ""]}) v1 -- 0x7fc82818ea20 con 0x7fc8180384e0 2024-02-16T19:20:32.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:32 smithi022 ceph-mon[29221]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:20:32.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:32 smithi022 ceph-mon[29221]: from='mgr.14164 172.21.15.22:0/3311464841' entity='mgr.smithi022.gmdcch' 2024-02-16T19:20:32.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:32 smithi022 ceph-mon[29221]: from='client.14191 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi117", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:20:32.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:32 smithi022 ceph-mon[29221]: from='mgr.14164 172.21.15.22:0/3311464841' entity='mgr.smithi022.gmdcch' 2024-02-16T19:20:32.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:32 smithi022 ceph-mon[29221]: from='mgr.14164 172.21.15.22:0/3311464841' entity='mgr.smithi022.gmdcch' 2024-02-16T19:20:32.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:32 smithi022 ceph-mon[29221]: from='mgr.14164 172.21.15.22:0/3311464841' entity='mgr.smithi022.gmdcch' 2024-02-16T19:20:32.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:32 smithi022 ceph-mon[29221]: from='mgr.14164 172.21.15.22:0/3311464841' entity='mgr.smithi022.gmdcch' 2024-02-16T19:20:33.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:33 smithi022 ceph-mon[29221]: Deploying daemon node-exporter.smithi022 on smithi022 2024-02-16T19:20:33.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:33 smithi022 ceph-mon[29221]: Deploying cephadm binary to smithi117 2024-02-16T19:20:34.372 INFO:teuthology.orchestra.run.smithi022.stdout:Added host 'smithi117' with addr '172.21.15.117' 2024-02-16T19:20:34.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:34.370+0000 7fc825ffb700 1 -- 172.21.15.22:0/3773557201 <== mgr.14164 v2:172.21.15.22:6800/2945983569 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fc82818ea20 con 0x7fc8180384e0 2024-02-16T19:20:34.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:34.372+0000 7fc82ec23700 1 -- 172.21.15.22:0/3773557201 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7fc8180384e0 msgr2=0x7fc81803a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:34.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:34.372+0000 7fc82ec23700 1 --2- 172.21.15.22:0/3773557201 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7fc8180384e0 0x7fc81803a990 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc814006fd0 tx=0x7fc814006e40 comp rx=0 tx=0).stop 2024-02-16T19:20:34.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:34.372+0000 7fc82ec23700 1 -- 172.21.15.22:0/3773557201 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc8280ffe30 msgr2=0x7fc828074af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:34.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:34.372+0000 7fc82ec23700 1 --2- 172.21.15.22:0/3773557201 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc8280ffe30 0x7fc828074af0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc81c005b40 tx=0x7fc81c004740 comp rx=0 tx=0).stop 2024-02-16T19:20:34.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:34.372+0000 7fc82ec23700 1 -- 172.21.15.22:0/3773557201 shutdown_connections 2024-02-16T19:20:34.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:34.372+0000 7fc82ec23700 1 --2- 172.21.15.22:0/3773557201 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7fc8180384e0 0x7fc81803a990 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:34.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:34.372+0000 7fc82ec23700 1 --2- 172.21.15.22:0/3773557201 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc8280ffe30 0x7fc828074af0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:34.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:34.372+0000 7fc82ec23700 1 -- 172.21.15.22:0/3773557201 >> 172.21.15.22:0/3773557201 conn(0x7fc8280fb3c0 msgr2=0x7fc8280fc090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:20:34.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:34.373+0000 7fc82ec23700 1 -- 172.21.15.22:0/3773557201 shutdown_connections 2024-02-16T19:20:34.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:34.373+0000 7fc82ec23700 1 -- 172.21.15.22:0/3773557201 wait complete. 2024-02-16T19:20:34.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:34 smithi022 ceph-mon[29221]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:20:34.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:34 smithi022 ceph-mon[29221]: from='mgr.14164 172.21.15.22:0/3311464841' entity='mgr.smithi022.gmdcch' 2024-02-16T19:20:35.409 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -- ceph orch host ls --format=json 2024-02-16T19:20:35.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:35 smithi022 ceph-mon[29221]: Added host smithi117 2024-02-16T19:20:36.100 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi022/config 2024-02-16T19:20:36.915 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:36 smithi022 ceph-mon[29221]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:20:36.916 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:36 smithi022 ceph-mon[29221]: from='mgr.14164 172.21.15.22:0/3311464841' entity='mgr.smithi022.gmdcch' 2024-02-16T19:20:36.916 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:36 smithi022 ceph-mon[29221]: from='mgr.14164 172.21.15.22:0/3311464841' entity='mgr.smithi022.gmdcch' 2024-02-16T19:20:36.916 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:36 smithi022 ceph-mon[29221]: from='mgr.14164 172.21.15.22:0/3311464841' entity='mgr.smithi022.gmdcch' 2024-02-16T19:20:36.916 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:36 smithi022 ceph-mon[29221]: from='mgr.14164 172.21.15.22:0/3311464841' entity='mgr.smithi022.gmdcch' 2024-02-16T19:20:36.916 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:36 smithi022 ceph-mon[29221]: from='mgr.14164 172.21.15.22:0/3311464841' entity='mgr.smithi022.gmdcch' 2024-02-16T19:20:37.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:37.696+0000 7f9abeb2e700 1 -- 172.21.15.22:0/2759120555 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ab80695b0 msgr2=0x7f9ab81029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:37.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:37.696+0000 7f9abeb2e700 1 --2- 172.21.15.22:0/2759120555 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ab80695b0 0x7f9ab81029b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9aac009910 tx=0x7f9aac009c20 comp rx=0 tx=0).stop 2024-02-16T19:20:37.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:37.696+0000 7f9abeb2e700 1 -- 172.21.15.22:0/2759120555 shutdown_connections 2024-02-16T19:20:37.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:37.696+0000 7f9abeb2e700 1 --2- 172.21.15.22:0/2759120555 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ab80695b0 0x7f9ab81029b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:37.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:37.696+0000 7f9abeb2e700 1 -- 172.21.15.22:0/2759120555 >> 172.21.15.22:0/2759120555 conn(0x7f9ab80fa380 msgr2=0x7f9ab80fc790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:20:37.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:37.697+0000 7f9abeb2e700 1 -- 172.21.15.22:0/2759120555 shutdown_connections 2024-02-16T19:20:37.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:37.697+0000 7f9abeb2e700 1 -- 172.21.15.22:0/2759120555 wait complete. 2024-02-16T19:20:37.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:37.697+0000 7f9abeb2e700 1 Processor -- start 2024-02-16T19:20:37.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:37.697+0000 7f9abeb2e700 1 -- start start 2024-02-16T19:20:37.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:37.697+0000 7f9abeb2e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ab81002c0 0x7f9ab80fe950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:37.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:37.697+0000 7f9abeb2e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab81006d0 con 0x7f9ab81002c0 2024-02-16T19:20:37.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:37.697+0000 7f9abc8ca700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ab81002c0 0x7f9ab80fe950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:20:37.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:37.697+0000 7f9abc8ca700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ab81002c0 0x7f9ab80fe950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45904/0 (socket says 172.21.15.22:45904) 2024-02-16T19:20:37.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:37.697+0000 7f9abc8ca700 1 -- 172.21.15.22:0/3487632326 learned_addr learned my addr 172.21.15.22:0/3487632326 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:20:37.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:37.698+0000 7f9abc8ca700 1 -- 172.21.15.22:0/3487632326 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aac0095f0 con 0x7f9ab81002c0 2024-02-16T19:20:37.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:37.698+0000 7f9abc8ca700 1 --2- 172.21.15.22:0/3487632326 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ab81002c0 0x7f9ab80fe950 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9aac005b40 tx=0x7f9aac004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:20:37.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:37.698+0000 7f9ab5ffb700 1 -- 172.21.15.22:0/3487632326 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9aac01d070 con 0x7f9ab81002c0 2024-02-16T19:20:37.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:37.698+0000 7f9abeb2e700 1 -- 172.21.15.22:0/3487632326 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ab80fee90 con 0x7f9ab81002c0 2024-02-16T19:20:37.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:37.698+0000 7f9ab5ffb700 1 -- 172.21.15.22:0/3487632326 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9aac00be10 con 0x7f9ab81002c0 2024-02-16T19:20:37.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:37.698+0000 7f9ab5ffb700 1 -- 172.21.15.22:0/3487632326 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9aac00f460 con 0x7f9ab81002c0 2024-02-16T19:20:37.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:37.698+0000 7f9abeb2e700 1 -- 172.21.15.22:0/3487632326 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ab80ff330 con 0x7f9ab81002c0 2024-02-16T19:20:37.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:37.699+0000 7f9ab5ffb700 1 -- 172.21.15.22:0/3487632326 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7f9aac00f6d0 con 0x7f9ab81002c0 2024-02-16T19:20:37.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:37.699+0000 7f9abeb2e700 1 -- 172.21.15.22:0/3487632326 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ab818ea70 con 0x7f9ab81002c0 2024-02-16T19:20:37.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:37.699+0000 7f9ab5ffb700 1 --2- 172.21.15.22:0/3487632326 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7f9aa8038490 0x7f9aa803a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:37.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:37.699+0000 7f9ab5ffb700 1 -- 172.21.15.22:0/3487632326 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f9aac04c3c0 con 0x7f9ab81002c0 2024-02-16T19:20:37.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:37.699+0000 7f9ab7fff700 1 --2- 172.21.15.22:0/3487632326 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7f9aa8038490 0x7f9aa803a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:20:37.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:37.700+0000 7f9ab7fff700 1 --2- 172.21.15.22:0/3487632326 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7f9aa8038490 0x7f9aa803a940 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4006fd0 tx=0x7f9aa4006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:20:37.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:37.704+0000 7f9ab5ffb700 1 -- 172.21.15.22:0/3487632326 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f9aac02a360 con 0x7f9ab81002c0 2024-02-16T19:20:37.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:37.916+0000 7f9abeb2e700 1 -- 172.21.15.22:0/3487632326 --> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9ab8105e10 con 0x7f9aa8038490 2024-02-16T19:20:37.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:37.917+0000 7f9ab5ffb700 1 -- 172.21.15.22:0/3487632326 <== mgr.14164 v2:172.21.15.22:6800/2945983569 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+160 (secure 0 0 0) 0x7f9ab8105e10 con 0x7f9aa8038490 2024-02-16T19:20:37.917 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:20:37.917 INFO:teuthology.orchestra.run.smithi022.stdout:[{"addr": "172.21.15.22", "hostname": "smithi022", "labels": [], "status": ""}, {"addr": "172.21.15.117", "hostname": "smithi117", "labels": [], "status": ""}] 2024-02-16T19:20:37.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:37.919+0000 7f9abeb2e700 1 -- 172.21.15.22:0/3487632326 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7f9aa8038490 msgr2=0x7f9aa803a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:37.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:37.919+0000 7f9abeb2e700 1 --2- 172.21.15.22:0/3487632326 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7f9aa8038490 0x7f9aa803a940 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4006fd0 tx=0x7f9aa4006e40 comp rx=0 tx=0).stop 2024-02-16T19:20:37.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:37.919+0000 7f9abeb2e700 1 -- 172.21.15.22:0/3487632326 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ab81002c0 msgr2=0x7f9ab80fe950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:37.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:37.919+0000 7f9abeb2e700 1 --2- 172.21.15.22:0/3487632326 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ab81002c0 0x7f9ab80fe950 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9aac005b40 tx=0x7f9aac004740 comp rx=0 tx=0).stop 2024-02-16T19:20:37.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:37.919+0000 7f9abeb2e700 1 -- 172.21.15.22:0/3487632326 shutdown_connections 2024-02-16T19:20:37.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:37.919+0000 7f9abeb2e700 1 --2- 172.21.15.22:0/3487632326 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7f9aa8038490 0x7f9aa803a940 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:37.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:37.919+0000 7f9abeb2e700 1 --2- 172.21.15.22:0/3487632326 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ab81002c0 0x7f9ab80fe950 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:37.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:37.919+0000 7f9abeb2e700 1 -- 172.21.15.22:0/3487632326 >> 172.21.15.22:0/3487632326 conn(0x7f9ab80fa380 msgr2=0x7f9ab80fafe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:20:37.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:37.919+0000 7f9abeb2e700 1 -- 172.21.15.22:0/3487632326 shutdown_connections 2024-02-16T19:20:37.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:37.919+0000 7f9abeb2e700 1 -- 172.21.15.22:0/3487632326 wait complete. 2024-02-16T19:20:38.141 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:37 smithi022 ceph-mon[29221]: Deploying daemon alertmanager.smithi022 on smithi022 2024-02-16T19:20:38.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:38 smithi022 ceph-mon[29221]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:20:39.035 INFO:tasks.cephadm:Setting crush tunables to default 2024-02-16T19:20:39.035 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -- ceph osd crush tunables default 2024-02-16T19:20:39.258 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi022/config 2024-02-16T19:20:40.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:39 smithi022 ceph-mon[29221]: from='client.14193 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-16T19:20:41.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:40 smithi022 ceph-mon[29221]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:20:42.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:42.271+0000 7f6ff4dee700 1 -- 172.21.15.22:0/3709723654 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ff00fed30 msgr2=0x7f6ff00ff140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:42.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:42.271+0000 7f6ff4dee700 1 --2- 172.21.15.22:0/3709723654 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ff00fed30 0x7f6ff00ff140 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6fe000b0a0 tx=0x7f6fe000b3b0 comp rx=0 tx=0).stop 2024-02-16T19:20:42.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:42.271+0000 7f6ff4dee700 1 -- 172.21.15.22:0/3709723654 shutdown_connections 2024-02-16T19:20:42.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:42.271+0000 7f6ff4dee700 1 --2- 172.21.15.22:0/3709723654 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ff00fed30 0x7f6ff00ff140 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:42.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:42.271+0000 7f6ff4dee700 1 -- 172.21.15.22:0/3709723654 >> 172.21.15.22:0/3709723654 conn(0x7f6ff00fa2c0 msgr2=0x7f6ff00fc710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:20:42.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:42.271+0000 7f6ff4dee700 1 -- 172.21.15.22:0/3709723654 shutdown_connections 2024-02-16T19:20:42.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:42.271+0000 7f6ff4dee700 1 -- 172.21.15.22:0/3709723654 wait complete. 2024-02-16T19:20:42.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:42.272+0000 7f6ff4dee700 1 Processor -- start 2024-02-16T19:20:42.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:42.272+0000 7f6ff4dee700 1 -- start start 2024-02-16T19:20:42.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:42.272+0000 7f6ff4dee700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ff00fed30 0x7f6ff0194fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:42.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:42.272+0000 7f6ff4dee700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff0195510 con 0x7f6ff00fed30 2024-02-16T19:20:42.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:42.272+0000 7f6fee59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ff00fed30 0x7f6ff0194fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:20:42.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:42.272+0000 7f6fee59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ff00fed30 0x7f6ff0194fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45916/0 (socket says 172.21.15.22:45916) 2024-02-16T19:20:42.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:42.272+0000 7f6fee59c700 1 -- 172.21.15.22:0/949230740 learned_addr learned my addr 172.21.15.22:0/949230740 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:20:42.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:42.272+0000 7f6fee59c700 1 -- 172.21.15.22:0/949230740 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fe0009d00 con 0x7f6ff00fed30 2024-02-16T19:20:42.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:42.272+0000 7f6fee59c700 1 --2- 172.21.15.22:0/949230740 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ff00fed30 0x7f6ff0194fd0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6fe0009fd0 tx=0x7f6fe000bb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:20:42.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:42.272+0000 7f6fe77fe700 1 -- 172.21.15.22:0/949230740 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6fe000e040 con 0x7f6ff00fed30 2024-02-16T19:20:42.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:42.272+0000 7f6fe77fe700 1 -- 172.21.15.22:0/949230740 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6fe000bdc0 con 0x7f6ff00fed30 2024-02-16T19:20:42.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:42.272+0000 7f6ff4dee700 1 -- 172.21.15.22:0/949230740 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ff0195710 con 0x7f6ff00fed30 2024-02-16T19:20:42.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:42.272+0000 7f6fe77fe700 1 -- 172.21.15.22:0/949230740 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6fe001da40 con 0x7f6ff00fed30 2024-02-16T19:20:42.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:42.273+0000 7f6ff4dee700 1 -- 172.21.15.22:0/949230740 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ff0195b30 con 0x7f6ff00fed30 2024-02-16T19:20:42.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:42.273+0000 7f6fe77fe700 1 -- 172.21.15.22:0/949230740 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7f6fe001dc80 con 0x7f6ff00fed30 2024-02-16T19:20:42.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:42.273+0000 7f6ff4dee700 1 -- 172.21.15.22:0/949230740 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ff018ea20 con 0x7f6ff00fed30 2024-02-16T19:20:42.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:42.274+0000 7f6fe77fe700 1 --2- 172.21.15.22:0/949230740 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7f6fdc0384e0 0x7f6fdc03a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:42.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:42.274+0000 7f6fe77fe700 1 -- 172.21.15.22:0/949230740 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f6fe004c810 con 0x7f6ff00fed30 2024-02-16T19:20:42.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:42.274+0000 7f6fedd9b700 1 --2- 172.21.15.22:0/949230740 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7f6fdc0384e0 0x7f6fdc03a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:20:42.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:42.274+0000 7f6fedd9b700 1 --2- 172.21.15.22:0/949230740 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7f6fdc0384e0 0x7f6fdc03a990 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6fd8006fd0 tx=0x7f6fd8006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:20:42.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:42.278+0000 7f6fe77fe700 1 -- 172.21.15.22:0/949230740 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f6fe0025770 con 0x7f6ff00fed30 2024-02-16T19:20:42.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:42.484+0000 7f6ff4dee700 1 -- 172.21.15.22:0/949230740 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f6ff0062380 con 0x7f6ff00fed30 2024-02-16T19:20:42.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:42.774+0000 7f6fe77fe700 1 -- 172.21.15.22:0/949230740 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f6fe001a3b0 con 0x7f6ff00fed30 2024-02-16T19:20:42.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:42.776+0000 7f6ff4dee700 1 -- 172.21.15.22:0/949230740 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7f6fdc0384e0 msgr2=0x7f6fdc03a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:42.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:42.776+0000 7f6ff4dee700 1 --2- 172.21.15.22:0/949230740 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7f6fdc0384e0 0x7f6fdc03a990 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6fd8006fd0 tx=0x7f6fd8006e40 comp rx=0 tx=0).stop 2024-02-16T19:20:42.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:42.776+0000 7f6ff4dee700 1 -- 172.21.15.22:0/949230740 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ff00fed30 msgr2=0x7f6ff0194fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:42.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:42.776+0000 7f6ff4dee700 1 --2- 172.21.15.22:0/949230740 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ff00fed30 0x7f6ff0194fd0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6fe0009fd0 tx=0x7f6fe000bb10 comp rx=0 tx=0).stop 2024-02-16T19:20:42.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:42.776+0000 7f6ff4dee700 1 -- 172.21.15.22:0/949230740 shutdown_connections 2024-02-16T19:20:42.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:42.777+0000 7f6ff4dee700 1 --2- 172.21.15.22:0/949230740 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7f6fdc0384e0 0x7f6fdc03a990 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:42.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:42.777+0000 7f6ff4dee700 1 --2- 172.21.15.22:0/949230740 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ff00fed30 0x7f6ff0194fd0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:42.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:42.777+0000 7f6ff4dee700 1 -- 172.21.15.22:0/949230740 >> 172.21.15.22:0/949230740 conn(0x7f6ff00fa2c0 msgr2=0x7f6ff0069070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:20:42.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:42.777+0000 7f6ff4dee700 1 -- 172.21.15.22:0/949230740 shutdown_connections 2024-02-16T19:20:42.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:20:42.777+0000 7f6ff4dee700 1 -- 172.21.15.22:0/949230740 wait complete. 2024-02-16T19:20:42.778 INFO:teuthology.orchestra.run.smithi022.stderr:adjusted tunables profile to default 2024-02-16T19:20:42.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:42 smithi022 ceph-mon[29221]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:20:42.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:42 smithi022 ceph-mon[29221]: from='client.? 172.21.15.22:0/949230740' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-02-16T19:20:43.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:43 smithi022 ceph-mon[29221]: from='client.? 172.21.15.22:0/949230740' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-02-16T19:20:43.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:43 smithi022 ceph-mon[29221]: osdmap e4: 0 total, 0 up, 0 in 2024-02-16T19:20:44.383 INFO:tasks.cephadm:Adding mon.smithi022 on smithi022 2024-02-16T19:20:44.383 INFO:tasks.cephadm:Adding mon.smithi117 on smithi117 2024-02-16T19:20:44.384 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -- ceph orch apply mon '2;smithi022:172.21.15.22=smithi022;smithi117:172.21.15.117=smithi117' 2024-02-16T19:20:44.583 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T19:20:44.628 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T19:20:45.069 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:44 smithi022 ceph-mon[29221]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:20:46.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:46.020+0000 7fc020a52700 1 -- 172.21.15.117:0/1395755196 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc01c0ffe20 msgr2=0x7fc01c100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:46.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:46.020+0000 7fc020a52700 1 --2- 172.21.15.117:0/1395755196 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc01c0ffe20 0x7fc01c100230 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc00c009910 tx=0x7fc00c009c20 comp rx=0 tx=0).stop 2024-02-16T19:20:46.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:46.020+0000 7fc020a52700 1 -- 172.21.15.117:0/1395755196 shutdown_connections 2024-02-16T19:20:46.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:46.020+0000 7fc020a52700 1 --2- 172.21.15.117:0/1395755196 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc01c0ffe20 0x7fc01c100230 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:46.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:46.020+0000 7fc020a52700 1 -- 172.21.15.117:0/1395755196 >> 172.21.15.117:0/1395755196 conn(0x7fc01c0fb3d0 msgr2=0x7fc01c0fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:20:46.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:46.020+0000 7fc020a52700 1 -- 172.21.15.117:0/1395755196 shutdown_connections 2024-02-16T19:20:46.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:46.020+0000 7fc020a52700 1 -- 172.21.15.117:0/1395755196 wait complete. 2024-02-16T19:20:46.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:46.021+0000 7fc020a52700 1 Processor -- start 2024-02-16T19:20:46.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:46.021+0000 7fc020a52700 1 -- start start 2024-02-16T19:20:46.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:46.021+0000 7fc020a52700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc01c0ffe20 0x7fc01c194f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:46.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:46.021+0000 7fc020a52700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc01c1954a0 con 0x7fc01c0ffe20 2024-02-16T19:20:46.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:46.021+0000 7fc01a59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc01c0ffe20 0x7fc01c194f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:20:46.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:46.021+0000 7fc01a59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc01c0ffe20 0x7fc01c194f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.117:41796/0 (socket says 172.21.15.117:41796) 2024-02-16T19:20:46.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:46.021+0000 7fc01a59c700 1 -- 172.21.15.117:0/2673385054 learned_addr learned my addr 172.21.15.117:0/2673385054 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-02-16T19:20:46.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:46.021+0000 7fc01a59c700 1 -- 172.21.15.117:0/2673385054 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc00c0095f0 con 0x7fc01c0ffe20 2024-02-16T19:20:46.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:46.021+0000 7fc01a59c700 1 --2- 172.21.15.117:0/2673385054 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc01c0ffe20 0x7fc01c194f60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc00c005b40 tx=0x7fc00c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:20:46.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:46.022+0000 7fc00b7fe700 1 -- 172.21.15.117:0/2673385054 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc00c01d070 con 0x7fc01c0ffe20 2024-02-16T19:20:46.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:46.022+0000 7fc020a52700 1 -- 172.21.15.117:0/2673385054 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc01c1956a0 con 0x7fc01c0ffe20 2024-02-16T19:20:46.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:46.022+0000 7fc00b7fe700 1 -- 172.21.15.117:0/2673385054 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc00c00be10 con 0x7fc01c0ffe20 2024-02-16T19:20:46.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:46.022+0000 7fc00b7fe700 1 -- 172.21.15.117:0/2673385054 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc00c00f460 con 0x7fc01c0ffe20 2024-02-16T19:20:46.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:46.022+0000 7fc020a52700 1 -- 172.21.15.117:0/2673385054 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc01c195b40 con 0x7fc01c0ffe20 2024-02-16T19:20:46.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:46.023+0000 7fc020a52700 1 -- 172.21.15.117:0/2673385054 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc01c18ea80 con 0x7fc01c0ffe20 2024-02-16T19:20:46.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:46.023+0000 7fc00b7fe700 1 -- 172.21.15.117:0/2673385054 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7fc00c0227e0 con 0x7fc01c0ffe20 2024-02-16T19:20:46.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:46.023+0000 7fc00b7fe700 1 --2- 172.21.15.117:0/2673385054 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7fc004038490 0x7fc00403a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:46.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:46.023+0000 7fc00b7fe700 1 -- 172.21.15.117:0/2673385054 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc00c04c2a0 con 0x7fc01c0ffe20 2024-02-16T19:20:46.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:46.023+0000 7fc019d9b700 1 --2- 172.21.15.117:0/2673385054 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7fc004038490 0x7fc00403a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:20:46.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:46.024+0000 7fc019d9b700 1 --2- 172.21.15.117:0/2673385054 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7fc004038490 0x7fc00403a940 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc010006fd0 tx=0x7fc010006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:20:46.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:46.028+0000 7fc00b7fe700 1 -- 172.21.15.117:0/2673385054 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fc00c02b590 con 0x7fc01c0ffe20 2024-02-16T19:20:46.224 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:46.223+0000 7fc020a52700 1 -- 172.21.15.117:0/2673385054 --> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi022:172.21.15.22=smithi022;smithi117:172.21.15.117=smithi117", "target": ["mon-mgr", ""]}) v1 -- 0x7fc01c061190 con 0x7fc004038490 2024-02-16T19:20:46.231 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:46.230+0000 7fc00b7fe700 1 -- 172.21.15.117:0/2673385054 <== mgr.14164 v2:172.21.15.22:6800/2945983569 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fc01c061190 con 0x7fc004038490 2024-02-16T19:20:46.231 INFO:teuthology.orchestra.run.smithi117.stdout:Scheduled mon update... 2024-02-16T19:20:46.233 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:46.232+0000 7fc020a52700 1 -- 172.21.15.117:0/2673385054 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7fc004038490 msgr2=0x7fc00403a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:46.234 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:46.232+0000 7fc020a52700 1 --2- 172.21.15.117:0/2673385054 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7fc004038490 0x7fc00403a940 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc010006fd0 tx=0x7fc010006e40 comp rx=0 tx=0).stop 2024-02-16T19:20:46.234 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:46.232+0000 7fc020a52700 1 -- 172.21.15.117:0/2673385054 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc01c0ffe20 msgr2=0x7fc01c194f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:46.234 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:46.232+0000 7fc020a52700 1 --2- 172.21.15.117:0/2673385054 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc01c0ffe20 0x7fc01c194f60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc00c005b40 tx=0x7fc00c004740 comp rx=0 tx=0).stop 2024-02-16T19:20:46.234 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:46.232+0000 7fc020a52700 1 -- 172.21.15.117:0/2673385054 shutdown_connections 2024-02-16T19:20:46.234 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:46.232+0000 7fc020a52700 1 --2- 172.21.15.117:0/2673385054 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7fc004038490 0x7fc00403a940 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:46.234 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:46.232+0000 7fc020a52700 1 --2- 172.21.15.117:0/2673385054 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc01c0ffe20 0x7fc01c194f60 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:46.234 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:46.232+0000 7fc020a52700 1 -- 172.21.15.117:0/2673385054 >> 172.21.15.117:0/2673385054 conn(0x7fc01c0fb3d0 msgr2=0x7fc01c0fc080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:20:46.234 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:46.232+0000 7fc020a52700 1 -- 172.21.15.117:0/2673385054 shutdown_connections 2024-02-16T19:20:46.234 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:46.232+0000 7fc020a52700 1 -- 172.21.15.117:0/2673385054 wait complete. 2024-02-16T19:20:46.769 DEBUG:teuthology.orchestra.run.smithi117:mon.smithi117> sudo journalctl -f -n 0 -u ceph-2a1346e6-cd00-11ee-95ba-87774f69a715@mon.smithi117.service 2024-02-16T19:20:46.771 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-16T19:20:46.771 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-16T19:20:46.999 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T19:20:47.032 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:46 smithi022 ceph-mon[29221]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:20:47.032 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:46 smithi022 ceph-mon[29221]: from='mgr.14164 172.21.15.22:0/3311464841' entity='mgr.smithi022.gmdcch' 2024-02-16T19:20:47.044 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T19:20:48.154 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:47 smithi022 ceph-mon[29221]: from='client.14197 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi022:172.21.15.22=smithi022;smithi117:172.21.15.117=smithi117", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:20:48.155 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:47 smithi022 ceph-mon[29221]: Saving service mon spec with placement smithi022:172.21.15.22=smithi022;smithi117:172.21.15.117=smithi117;count:2 2024-02-16T19:20:48.155 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:47 smithi022 ceph-mon[29221]: from='mgr.14164 172.21.15.22:0/3311464841' entity='mgr.smithi022.gmdcch' 2024-02-16T19:20:48.155 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:47 smithi022 ceph-mon[29221]: from='mgr.14164 172.21.15.22:0/3311464841' entity='mgr.smithi022.gmdcch' 2024-02-16T19:20:48.155 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:47 smithi022 ceph-mon[29221]: from='mgr.14164 172.21.15.22:0/3311464841' entity='mgr.smithi022.gmdcch' 2024-02-16T19:20:48.155 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:47 smithi022 ceph-mon[29221]: from='mgr.14164 172.21.15.22:0/3311464841' entity='mgr.smithi022.gmdcch' 2024-02-16T19:20:48.155 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:47 smithi022 ceph-mon[29221]: from='mgr.14164 172.21.15.22:0/3311464841' entity='mgr.smithi022.gmdcch' 2024-02-16T19:20:48.155 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:47 smithi022 ceph-mon[29221]: from='mgr.14164 172.21.15.22:0/3311464841' entity='mgr.smithi022.gmdcch' 2024-02-16T19:20:48.155 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:47 smithi022 ceph-mon[29221]: from='mgr.14164 172.21.15.22:0/3311464841' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-16T19:20:48.155 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:47 smithi022 ceph-mon[29221]: from='mgr.14164 172.21.15.22:0/3311464841' entity='mgr.smithi022.gmdcch' 2024-02-16T19:20:48.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:48.530+0000 7efd46da9700 1 -- 172.21.15.117:0/862168433 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efd400fed20 msgr2=0x7efd400ff130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:48.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:48.530+0000 7efd46da9700 1 --2- 172.21.15.117:0/862168433 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efd400fed20 0x7efd400ff130 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7efd34009910 tx=0x7efd34009c20 comp rx=0 tx=0).stop 2024-02-16T19:20:48.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:48.530+0000 7efd46da9700 1 -- 172.21.15.117:0/862168433 shutdown_connections 2024-02-16T19:20:48.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:48.530+0000 7efd46da9700 1 --2- 172.21.15.117:0/862168433 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efd400fed20 0x7efd400ff130 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:48.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:48.530+0000 7efd46da9700 1 -- 172.21.15.117:0/862168433 >> 172.21.15.117:0/862168433 conn(0x7efd400fa2b0 msgr2=0x7efd400fc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:20:48.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:48.530+0000 7efd46da9700 1 -- 172.21.15.117:0/862168433 shutdown_connections 2024-02-16T19:20:48.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:48.530+0000 7efd46da9700 1 -- 172.21.15.117:0/862168433 wait complete. 2024-02-16T19:20:48.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:48.530+0000 7efd46da9700 1 Processor -- start 2024-02-16T19:20:48.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:48.530+0000 7efd46da9700 1 -- start start 2024-02-16T19:20:48.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:48.530+0000 7efd46da9700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efd400fed20 0x7efd40069380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:48.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:48.530+0000 7efd46da9700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd40197cf0 con 0x7efd400fed20 2024-02-16T19:20:48.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:48.530+0000 7efd44b45700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efd400fed20 0x7efd40069380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:20:48.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:48.530+0000 7efd44b45700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efd400fed20 0x7efd40069380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.117:45204/0 (socket says 172.21.15.117:45204) 2024-02-16T19:20:48.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:48.530+0000 7efd44b45700 1 -- 172.21.15.117:0/3968005609 learned_addr learned my addr 172.21.15.117:0/3968005609 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-02-16T19:20:48.533 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:48.531+0000 7efd44b45700 1 -- 172.21.15.117:0/3968005609 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd340095f0 con 0x7efd400fed20 2024-02-16T19:20:48.533 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:48.531+0000 7efd44b45700 1 --2- 172.21.15.117:0/3968005609 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efd400fed20 0x7efd40069380 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7efd34005b40 tx=0x7efd34004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:20:48.533 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:48.531+0000 7efd3dffb700 1 -- 172.21.15.117:0/3968005609 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efd3401d070 con 0x7efd400fed20 2024-02-16T19:20:48.533 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:48.531+0000 7efd46da9700 1 -- 172.21.15.117:0/3968005609 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd40197e30 con 0x7efd400fed20 2024-02-16T19:20:48.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:48.531+0000 7efd3dffb700 1 -- 172.21.15.117:0/3968005609 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7efd3400be10 con 0x7efd400fed20 2024-02-16T19:20:48.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:48.531+0000 7efd3dffb700 1 -- 172.21.15.117:0/3968005609 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efd3400f460 con 0x7efd400fed20 2024-02-16T19:20:48.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:48.532+0000 7efd46da9700 1 -- 172.21.15.117:0/3968005609 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd40198270 con 0x7efd400fed20 2024-02-16T19:20:48.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:48.532+0000 7efd3dffb700 1 -- 172.21.15.117:0/3968005609 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7efd340227e0 con 0x7efd400fed20 2024-02-16T19:20:48.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:48.532+0000 7efd46da9700 1 -- 172.21.15.117:0/3968005609 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd4018da00 con 0x7efd400fed20 2024-02-16T19:20:48.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:48.533+0000 7efd3dffb700 1 --2- 172.21.15.117:0/3968005609 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7efd30038490 0x7efd3003a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:48.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:48.533+0000 7efd3dffb700 1 -- 172.21.15.117:0/3968005609 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7efd3404c3e0 con 0x7efd400fed20 2024-02-16T19:20:48.535 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:48.533+0000 7efd3ffff700 1 --2- 172.21.15.117:0/3968005609 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7efd30038490 0x7efd3003a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:20:48.535 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:48.533+0000 7efd3ffff700 1 --2- 172.21.15.117:0/3968005609 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7efd30038490 0x7efd3003a940 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7efd2c006fd0 tx=0x7efd2c006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:20:48.539 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:48.538+0000 7efd3dffb700 1 -- 172.21.15.117:0/3968005609 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7efd3402bba0 con 0x7efd400fed20 2024-02-16T19:20:48.804 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:48.803+0000 7efd46da9700 1 -- 172.21.15.117:0/3968005609 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7efd4002cc30 con 0x7efd400fed20 2024-02-16T19:20:48.804 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:48.804+0000 7efd3dffb700 1 -- 172.21.15.117:0/3968005609 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7efd34027070 con 0x7efd400fed20 2024-02-16T19:20:48.805 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:20:48.805 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":1,"fsid":"2a1346e6-cd00-11ee-95ba-87774f69a715","modified":"2024-02-16T19:19:07.447976Z","created":"2024-02-16T19:19:07.447976Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi022","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:3300","nonce":0},{"type":"v1","addr":"172.21.15.22:6789","nonce":0}]},"addr":"172.21.15.22:6789/0","public_addr":"172.21.15.22:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-16T19:20:48.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:48.806+0000 7efd46da9700 1 -- 172.21.15.117:0/3968005609 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7efd30038490 msgr2=0x7efd3003a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:48.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:48.806+0000 7efd46da9700 1 --2- 172.21.15.117:0/3968005609 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7efd30038490 0x7efd3003a940 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7efd2c006fd0 tx=0x7efd2c006e40 comp rx=0 tx=0).stop 2024-02-16T19:20:48.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:48.806+0000 7efd46da9700 1 -- 172.21.15.117:0/3968005609 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efd400fed20 msgr2=0x7efd40069380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:48.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:48.806+0000 7efd46da9700 1 --2- 172.21.15.117:0/3968005609 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efd400fed20 0x7efd40069380 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7efd34005b40 tx=0x7efd34004740 comp rx=0 tx=0).stop 2024-02-16T19:20:48.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:48.806+0000 7efd46da9700 1 -- 172.21.15.117:0/3968005609 shutdown_connections 2024-02-16T19:20:48.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:48.806+0000 7efd46da9700 1 --2- 172.21.15.117:0/3968005609 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7efd30038490 0x7efd3003a940 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:48.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:48.806+0000 7efd46da9700 1 --2- 172.21.15.117:0/3968005609 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efd400fed20 0x7efd40069380 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:48.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:48.806+0000 7efd46da9700 1 -- 172.21.15.117:0/3968005609 >> 172.21.15.117:0/3968005609 conn(0x7efd400fa2b0 msgr2=0x7efd400faf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:20:48.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:48.806+0000 7efd46da9700 1 -- 172.21.15.117:0/3968005609 shutdown_connections 2024-02-16T19:20:48.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:48.806+0000 7efd46da9700 1 -- 172.21.15.117:0/3968005609 wait complete. 2024-02-16T19:20:48.808 INFO:teuthology.orchestra.run.smithi117.stderr:dumped monmap epoch 1 2024-02-16T19:20:49.157 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:48 smithi022 ceph-mon[29221]: Regenerating cephadm self-signed grafana TLS certificates 2024-02-16T19:20:49.157 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:48 smithi022 ceph-mon[29221]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-16T19:20:49.157 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:48 smithi022 ceph-mon[29221]: Deploying daemon grafana.smithi022 on smithi022 2024-02-16T19:20:49.157 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:48 smithi022 ceph-mon[29221]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:20:50.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:49 smithi022 ceph-mon[29221]: from='client.? 172.21.15.117:0/3968005609' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-16T19:20:50.398 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-16T19:20:50.398 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-16T19:20:50.608 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T19:20:50.653 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T19:20:51.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:50 smithi022 ceph-mon[29221]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:20:52.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:52.026+0000 7f57bb975700 1 -- 172.21.15.117:0/3701567135 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57b4100910 msgr2=0x7f57b4102cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:52.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:52.026+0000 7f57bb975700 1 --2- 172.21.15.117:0/3701567135 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57b4100910 0x7f57b4102cf0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f57b0009910 tx=0x7f57b0009c20 comp rx=0 tx=0).stop 2024-02-16T19:20:52.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:52.026+0000 7f57bb975700 1 -- 172.21.15.117:0/3701567135 shutdown_connections 2024-02-16T19:20:52.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:52.026+0000 7f57bb975700 1 --2- 172.21.15.117:0/3701567135 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57b4100910 0x7f57b4102cf0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:52.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:52.026+0000 7f57bb975700 1 -- 172.21.15.117:0/3701567135 >> 172.21.15.117:0/3701567135 conn(0x7f57b40fa2b0 msgr2=0x7f57b40fc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:20:52.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:52.026+0000 7f57bb975700 1 -- 172.21.15.117:0/3701567135 shutdown_connections 2024-02-16T19:20:52.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:52.026+0000 7f57bb975700 1 -- 172.21.15.117:0/3701567135 wait complete. 2024-02-16T19:20:52.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:52.026+0000 7f57bb975700 1 Processor -- start 2024-02-16T19:20:52.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:52.026+0000 7f57bb975700 1 -- start start 2024-02-16T19:20:52.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:52.026+0000 7f57bb975700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57b4100910 0x7f57b4100270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:52.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:52.026+0000 7f57bb975700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b41007b0 con 0x7f57b4100910 2024-02-16T19:20:52.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:52.027+0000 7f57b9711700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57b4100910 0x7f57b4100270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:20:52.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:52.027+0000 7f57b9711700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57b4100910 0x7f57b4100270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.117:45224/0 (socket says 172.21.15.117:45224) 2024-02-16T19:20:52.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:52.027+0000 7f57b9711700 1 -- 172.21.15.117:0/2865046461 learned_addr learned my addr 172.21.15.117:0/2865046461 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-02-16T19:20:52.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:52.027+0000 7f57b9711700 1 -- 172.21.15.117:0/2865046461 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57b00095f0 con 0x7f57b4100910 2024-02-16T19:20:52.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:52.027+0000 7f57b9711700 1 --2- 172.21.15.117:0/2865046461 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57b4100910 0x7f57b4100270 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f57b0005b40 tx=0x7f57b0004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:20:52.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:52.027+0000 7f57aa7fc700 1 -- 172.21.15.117:0/2865046461 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f57b001d070 con 0x7f57b4100910 2024-02-16T19:20:52.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:52.027+0000 7f57aa7fc700 1 -- 172.21.15.117:0/2865046461 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f57b000be10 con 0x7f57b4100910 2024-02-16T19:20:52.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:52.027+0000 7f57bb975700 1 -- 172.21.15.117:0/2865046461 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57b40fe8c0 con 0x7f57b4100910 2024-02-16T19:20:52.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:52.027+0000 7f57aa7fc700 1 -- 172.21.15.117:0/2865046461 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f57b000f460 con 0x7f57b4100910 2024-02-16T19:20:52.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:52.027+0000 7f57bb975700 1 -- 172.21.15.117:0/2865046461 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57b40fed60 con 0x7f57b4100910 2024-02-16T19:20:52.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:52.028+0000 7f57aa7fc700 1 -- 172.21.15.117:0/2865046461 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7f57b000b960 con 0x7f57b4100910 2024-02-16T19:20:52.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:52.028+0000 7f57bb975700 1 -- 172.21.15.117:0/2865046461 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57b4062380 con 0x7f57b4100910 2024-02-16T19:20:52.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:52.028+0000 7f57aa7fc700 1 --2- 172.21.15.117:0/2865046461 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7f57a00384e0 0x7f57a003a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:52.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:52.028+0000 7f57aa7fc700 1 -- 172.21.15.117:0/2865046461 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f57b004c470 con 0x7f57b4100910 2024-02-16T19:20:52.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:52.028+0000 7f57b8f10700 1 --2- 172.21.15.117:0/2865046461 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7f57a00384e0 0x7f57a003a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:20:52.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:52.029+0000 7f57b8f10700 1 --2- 172.21.15.117:0/2865046461 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7f57a00384e0 0x7f57a003a990 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f57a4006fd0 tx=0x7f57a4006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:20:52.036 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:52.036+0000 7f57aa7fc700 1 -- 172.21.15.117:0/2865046461 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f57b002bb60 con 0x7f57b4100910 2024-02-16T19:20:52.303 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:52.302+0000 7f57bb975700 1 -- 172.21.15.117:0/2865046461 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f57b418e4e0 con 0x7f57b4100910 2024-02-16T19:20:52.414 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:52.303+0000 7f57aa7fc700 1 -- 172.21.15.117:0/2865046461 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f57b0027070 con 0x7f57b4100910 2024-02-16T19:20:52.415 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:20:52.415 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":1,"fsid":"2a1346e6-cd00-11ee-95ba-87774f69a715","modified":"2024-02-16T19:19:07.447976Z","created":"2024-02-16T19:19:07.447976Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi022","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:3300","nonce":0},{"type":"v1","addr":"172.21.15.22:6789","nonce":0}]},"addr":"172.21.15.22:6789/0","public_addr":"172.21.15.22:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-16T19:20:52.415 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:52.305+0000 7f57bb975700 1 -- 172.21.15.117:0/2865046461 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7f57a00384e0 msgr2=0x7f57a003a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:52.415 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:52.305+0000 7f57bb975700 1 --2- 172.21.15.117:0/2865046461 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7f57a00384e0 0x7f57a003a990 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f57a4006fd0 tx=0x7f57a4006e40 comp rx=0 tx=0).stop 2024-02-16T19:20:52.415 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:52.305+0000 7f57bb975700 1 -- 172.21.15.117:0/2865046461 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57b4100910 msgr2=0x7f57b4100270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:52.416 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:52.305+0000 7f57bb975700 1 --2- 172.21.15.117:0/2865046461 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57b4100910 0x7f57b4100270 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f57b0005b40 tx=0x7f57b0004740 comp rx=0 tx=0).stop 2024-02-16T19:20:52.416 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:52.305+0000 7f57bb975700 1 -- 172.21.15.117:0/2865046461 shutdown_connections 2024-02-16T19:20:52.416 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:52.305+0000 7f57bb975700 1 --2- 172.21.15.117:0/2865046461 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7f57a00384e0 0x7f57a003a990 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:52.416 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:52.305+0000 7f57bb975700 1 --2- 172.21.15.117:0/2865046461 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57b4100910 0x7f57b4100270 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:52.416 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:52.305+0000 7f57bb975700 1 -- 172.21.15.117:0/2865046461 >> 172.21.15.117:0/2865046461 conn(0x7f57b40fa2b0 msgr2=0x7f57b40faf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:20:52.416 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:52.305+0000 7f57bb975700 1 -- 172.21.15.117:0/2865046461 shutdown_connections 2024-02-16T19:20:52.416 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:52.305+0000 7f57bb975700 1 -- 172.21.15.117:0/2865046461 wait complete. 2024-02-16T19:20:52.416 INFO:teuthology.orchestra.run.smithi117.stderr:dumped monmap epoch 1 2024-02-16T19:20:52.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:52 smithi022 ceph-mon[29221]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:20:52.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:52 smithi022 ceph-mon[29221]: from='mgr.14164 172.21.15.22:0/3311464841' entity='mgr.smithi022.gmdcch' 2024-02-16T19:20:52.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:52 smithi022 ceph-mon[29221]: from='client.? 172.21.15.117:0/2865046461' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-16T19:20:53.844 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-16T19:20:53.845 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-16T19:20:54.047 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T19:20:54.093 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T19:20:54.964 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:54 smithi022 ceph-mon[29221]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:20:55.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:55.507+0000 7fba024a9700 1 -- 172.21.15.117:0/3577042951 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb9fc0ffe20 msgr2=0x7fb9fc100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:55.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:55.507+0000 7fba024a9700 1 --2- 172.21.15.117:0/3577042951 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb9fc0ffe20 0x7fb9fc100230 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec009910 tx=0x7fb9ec009c20 comp rx=0 tx=0).stop 2024-02-16T19:20:55.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:55.507+0000 7fba024a9700 1 -- 172.21.15.117:0/3577042951 shutdown_connections 2024-02-16T19:20:55.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:55.507+0000 7fba024a9700 1 --2- 172.21.15.117:0/3577042951 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb9fc0ffe20 0x7fb9fc100230 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:55.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:55.507+0000 7fba024a9700 1 -- 172.21.15.117:0/3577042951 >> 172.21.15.117:0/3577042951 conn(0x7fb9fc0fb3d0 msgr2=0x7fb9fc0fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:20:55.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:55.507+0000 7fba024a9700 1 -- 172.21.15.117:0/3577042951 shutdown_connections 2024-02-16T19:20:55.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:55.507+0000 7fba024a9700 1 -- 172.21.15.117:0/3577042951 wait complete. 2024-02-16T19:20:55.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:55.508+0000 7fba024a9700 1 Processor -- start 2024-02-16T19:20:55.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:55.508+0000 7fba024a9700 1 -- start start 2024-02-16T19:20:55.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:55.508+0000 7fba024a9700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb9fc0ffe20 0x7fb9fc074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:55.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:55.508+0000 7fba024a9700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9fc075030 con 0x7fb9fc0ffe20 2024-02-16T19:20:55.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:55.508+0000 7fb9fbfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb9fc0ffe20 0x7fb9fc074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:20:55.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:55.508+0000 7fb9fbfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb9fc0ffe20 0x7fb9fc074af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.117:45242/0 (socket says 172.21.15.117:45242) 2024-02-16T19:20:55.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:55.508+0000 7fb9fbfff700 1 -- 172.21.15.117:0/3067816950 learned_addr learned my addr 172.21.15.117:0/3067816950 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-02-16T19:20:55.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:55.509+0000 7fb9fbfff700 1 -- 172.21.15.117:0/3067816950 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9ec0095f0 con 0x7fb9fc0ffe20 2024-02-16T19:20:55.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:55.509+0000 7fb9fbfff700 1 --2- 172.21.15.117:0/3067816950 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb9fc0ffe20 0x7fb9fc074af0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec0092e0 tx=0x7fb9ec009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:20:55.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:55.509+0000 7fb9f97fa700 1 -- 172.21.15.117:0/3067816950 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb9ec01d070 con 0x7fb9fc0ffe20 2024-02-16T19:20:55.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:55.509+0000 7fba024a9700 1 -- 172.21.15.117:0/3067816950 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9fc073140 con 0x7fb9fc0ffe20 2024-02-16T19:20:55.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:55.509+0000 7fb9f97fa700 1 -- 172.21.15.117:0/3067816950 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb9ec00be10 con 0x7fb9fc0ffe20 2024-02-16T19:20:55.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:55.509+0000 7fb9f97fa700 1 -- 172.21.15.117:0/3067816950 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb9ec00f460 con 0x7fb9fc0ffe20 2024-02-16T19:20:55.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:55.509+0000 7fba024a9700 1 -- 172.21.15.117:0/3067816950 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9fc0735e0 con 0x7fb9fc0ffe20 2024-02-16T19:20:55.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:55.510+0000 7fb9f97fa700 1 -- 172.21.15.117:0/3067816950 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7fb9ec00b960 con 0x7fb9fc0ffe20 2024-02-16T19:20:55.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:55.510+0000 7fba024a9700 1 -- 172.21.15.117:0/3067816950 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9fc062380 con 0x7fb9fc0ffe20 2024-02-16T19:20:55.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:55.510+0000 7fb9f97fa700 1 --2- 172.21.15.117:0/3067816950 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7fb9e4038490 0x7fb9e403a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:55.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:55.510+0000 7fb9f97fa700 1 -- 172.21.15.117:0/3067816950 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb9ec04c3b0 con 0x7fb9fc0ffe20 2024-02-16T19:20:55.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:55.510+0000 7fb9fb7fe700 1 --2- 172.21.15.117:0/3067816950 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7fb9e4038490 0x7fb9e403a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:20:55.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:55.511+0000 7fb9fb7fe700 1 --2- 172.21.15.117:0/3067816950 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7fb9e4038490 0x7fb9e403a940 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb9f0006fd0 tx=0x7fb9f0006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:20:55.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:55.516+0000 7fb9f97fa700 1 -- 172.21.15.117:0/3067816950 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fb9ec022790 con 0x7fb9fc0ffe20 2024-02-16T19:20:55.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:55.779+0000 7fba024a9700 1 -- 172.21.15.117:0/3067816950 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fb9fc18e460 con 0x7fb9fc0ffe20 2024-02-16T19:20:55.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:55.780+0000 7fb9f97fa700 1 -- 172.21.15.117:0/3067816950 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7fb9ec027070 con 0x7fb9fc0ffe20 2024-02-16T19:20:55.781 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:20:55.781 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":1,"fsid":"2a1346e6-cd00-11ee-95ba-87774f69a715","modified":"2024-02-16T19:19:07.447976Z","created":"2024-02-16T19:19:07.447976Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi022","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:3300","nonce":0},{"type":"v1","addr":"172.21.15.22:6789","nonce":0}]},"addr":"172.21.15.22:6789/0","public_addr":"172.21.15.22:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-16T19:20:55.782 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:55.782+0000 7fba024a9700 1 -- 172.21.15.117:0/3067816950 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7fb9e4038490 msgr2=0x7fb9e403a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:55.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:55.782+0000 7fba024a9700 1 --2- 172.21.15.117:0/3067816950 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7fb9e4038490 0x7fb9e403a940 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb9f0006fd0 tx=0x7fb9f0006e40 comp rx=0 tx=0).stop 2024-02-16T19:20:55.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:55.782+0000 7fba024a9700 1 -- 172.21.15.117:0/3067816950 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb9fc0ffe20 msgr2=0x7fb9fc074af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:55.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:55.782+0000 7fba024a9700 1 --2- 172.21.15.117:0/3067816950 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb9fc0ffe20 0x7fb9fc074af0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec0092e0 tx=0x7fb9ec009e60 comp rx=0 tx=0).stop 2024-02-16T19:20:55.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:55.782+0000 7fba024a9700 1 -- 172.21.15.117:0/3067816950 shutdown_connections 2024-02-16T19:20:55.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:55.782+0000 7fba024a9700 1 --2- 172.21.15.117:0/3067816950 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7fb9e4038490 0x7fb9e403a940 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:55.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:55.782+0000 7fba024a9700 1 --2- 172.21.15.117:0/3067816950 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb9fc0ffe20 0x7fb9fc074af0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:55.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:55.782+0000 7fba024a9700 1 -- 172.21.15.117:0/3067816950 >> 172.21.15.117:0/3067816950 conn(0x7fb9fc0fb3d0 msgr2=0x7fb9fc0fc080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:20:55.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:55.782+0000 7fba024a9700 1 -- 172.21.15.117:0/3067816950 shutdown_connections 2024-02-16T19:20:55.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:55.782+0000 7fba024a9700 1 -- 172.21.15.117:0/3067816950 wait complete. 2024-02-16T19:20:55.783 INFO:teuthology.orchestra.run.smithi117.stderr:dumped monmap epoch 1 2024-02-16T19:20:56.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:56 smithi022 ceph-mon[29221]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:20:56.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:56 smithi022 ceph-mon[29221]: from='client.? 172.21.15.117:0/3067816950' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-16T19:20:57.367 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-16T19:20:57.368 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-16T19:20:57.572 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T19:20:57.618 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T19:20:59.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:59.083+0000 7fdc65919700 1 -- 172.21.15.117:0/131217358 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc600fed30 msgr2=0x7fdc600ff140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:59.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:59.083+0000 7fdc65919700 1 --2- 172.21.15.117:0/131217358 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc600fed30 0x7fdc600ff140 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fdc50009910 tx=0x7fdc50009c20 comp rx=0 tx=0).stop 2024-02-16T19:20:59.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:59.083+0000 7fdc65919700 1 -- 172.21.15.117:0/131217358 shutdown_connections 2024-02-16T19:20:59.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:59.083+0000 7fdc65919700 1 --2- 172.21.15.117:0/131217358 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc600fed30 0x7fdc600ff140 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:59.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:59.083+0000 7fdc65919700 1 -- 172.21.15.117:0/131217358 >> 172.21.15.117:0/131217358 conn(0x7fdc600fa380 msgr2=0x7fdc600fc7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:20:59.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:59.083+0000 7fdc65919700 1 -- 172.21.15.117:0/131217358 shutdown_connections 2024-02-16T19:20:59.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:59.083+0000 7fdc65919700 1 -- 172.21.15.117:0/131217358 wait complete. 2024-02-16T19:20:59.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:59.084+0000 7fdc65919700 1 Processor -- start 2024-02-16T19:20:59.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:59.084+0000 7fdc65919700 1 -- start start 2024-02-16T19:20:59.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:59.084+0000 7fdc65919700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc600fed30 0x7fdc600693c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:59.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:59.084+0000 7fdc65919700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc60197cf0 con 0x7fdc600fed30 2024-02-16T19:20:59.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:59.084+0000 7fdc5effd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc600fed30 0x7fdc600693c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:20:59.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:59.084+0000 7fdc5effd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc600fed30 0x7fdc600693c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.117:55162/0 (socket says 172.21.15.117:55162) 2024-02-16T19:20:59.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:59.084+0000 7fdc5effd700 1 -- 172.21.15.117:0/1150061435 learned_addr learned my addr 172.21.15.117:0/1150061435 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-02-16T19:20:59.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:59.084+0000 7fdc5effd700 1 -- 172.21.15.117:0/1150061435 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc500095f0 con 0x7fdc600fed30 2024-02-16T19:20:59.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:59.085+0000 7fdc5effd700 1 --2- 172.21.15.117:0/1150061435 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc600fed30 0x7fdc600693c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdc50005b40 tx=0x7fdc50004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:20:59.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:59.085+0000 7fdc64917700 1 -- 172.21.15.117:0/1150061435 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdc5001d070 con 0x7fdc600fed30 2024-02-16T19:20:59.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:59.085+0000 7fdc64917700 1 -- 172.21.15.117:0/1150061435 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fdc5000be10 con 0x7fdc600fed30 2024-02-16T19:20:59.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:59.085+0000 7fdc65919700 1 -- 172.21.15.117:0/1150061435 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc60197e30 con 0x7fdc600fed30 2024-02-16T19:20:59.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:59.085+0000 7fdc64917700 1 -- 172.21.15.117:0/1150061435 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdc5000f460 con 0x7fdc600fed30 2024-02-16T19:20:59.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:59.085+0000 7fdc65919700 1 -- 172.21.15.117:0/1150061435 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc60198270 con 0x7fdc600fed30 2024-02-16T19:20:59.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:59.086+0000 7fdc64917700 1 -- 172.21.15.117:0/1150061435 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7fdc500227e0 con 0x7fdc600fed30 2024-02-16T19:20:59.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:59.086+0000 7fdc65919700 1 -- 172.21.15.117:0/1150061435 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc6018d990 con 0x7fdc600fed30 2024-02-16T19:20:59.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:59.086+0000 7fdc64917700 1 --2- 172.21.15.117:0/1150061435 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7fdc48038490 0x7fdc4803a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:59.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:59.086+0000 7fdc64917700 1 -- 172.21.15.117:0/1150061435 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fdc5004c430 con 0x7fdc600fed30 2024-02-16T19:20:59.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:59.086+0000 7fdc5e7fc700 1 --2- 172.21.15.117:0/1150061435 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7fdc48038490 0x7fdc4803a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:20:59.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:59.087+0000 7fdc5e7fc700 1 --2- 172.21.15.117:0/1150061435 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7fdc48038490 0x7fdc4803a940 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fdc54006fd0 tx=0x7fdc54006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:20:59.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:59.091+0000 7fdc64917700 1 -- 172.21.15.117:0/1150061435 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fdc5002bba0 con 0x7fdc600fed30 2024-02-16T19:20:59.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:58 smithi022 ceph-mon[29221]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:20:59.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:59.355+0000 7fdc65919700 1 -- 172.21.15.117:0/1150061435 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fdc6002cc30 con 0x7fdc600fed30 2024-02-16T19:20:59.357 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:59.356+0000 7fdc64917700 1 -- 172.21.15.117:0/1150061435 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7fdc50027070 con 0x7fdc600fed30 2024-02-16T19:20:59.357 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:20:59.357 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":1,"fsid":"2a1346e6-cd00-11ee-95ba-87774f69a715","modified":"2024-02-16T19:19:07.447976Z","created":"2024-02-16T19:19:07.447976Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi022","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:3300","nonce":0},{"type":"v1","addr":"172.21.15.22:6789","nonce":0}]},"addr":"172.21.15.22:6789/0","public_addr":"172.21.15.22:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-16T19:20:59.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:59.358+0000 7fdc65919700 1 -- 172.21.15.117:0/1150061435 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7fdc48038490 msgr2=0x7fdc4803a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:59.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:59.358+0000 7fdc65919700 1 --2- 172.21.15.117:0/1150061435 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7fdc48038490 0x7fdc4803a940 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fdc54006fd0 tx=0x7fdc54006e40 comp rx=0 tx=0).stop 2024-02-16T19:20:59.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:59.358+0000 7fdc65919700 1 -- 172.21.15.117:0/1150061435 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc600fed30 msgr2=0x7fdc600693c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:59.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:59.358+0000 7fdc65919700 1 --2- 172.21.15.117:0/1150061435 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc600fed30 0x7fdc600693c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdc50005b40 tx=0x7fdc50004740 comp rx=0 tx=0).stop 2024-02-16T19:20:59.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:59.358+0000 7fdc65919700 1 -- 172.21.15.117:0/1150061435 shutdown_connections 2024-02-16T19:20:59.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:59.358+0000 7fdc65919700 1 --2- 172.21.15.117:0/1150061435 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7fdc48038490 0x7fdc4803a940 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:59.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:59.358+0000 7fdc65919700 1 --2- 172.21.15.117:0/1150061435 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc600fed30 0x7fdc600693c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:59.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:59.358+0000 7fdc65919700 1 -- 172.21.15.117:0/1150061435 >> 172.21.15.117:0/1150061435 conn(0x7fdc600fa380 msgr2=0x7fdc600fb030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:20:59.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:59.358+0000 7fdc65919700 1 -- 172.21.15.117:0/1150061435 shutdown_connections 2024-02-16T19:20:59.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:20:59.359+0000 7fdc65919700 1 -- 172.21.15.117:0/1150061435 wait complete. 2024-02-16T19:20:59.361 INFO:teuthology.orchestra.run.smithi117.stderr:dumped monmap epoch 1 2024-02-16T19:20:59.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:20:59 smithi022 ceph-mon[29221]: from='client.? 172.21.15.117:0/1150061435' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-16T19:21:00.970 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-16T19:21:00.970 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-16T19:21:01.022 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:00 smithi022 ceph-mon[29221]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:21:01.171 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T19:21:01.218 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T19:21:02.075 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:01 smithi022 ceph-mon[29221]: from='mgr.14164 172.21.15.22:0/3311464841' entity='mgr.smithi022.gmdcch' 2024-02-16T19:21:02.076 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:01 smithi022 ceph-mon[29221]: from='mgr.14164 172.21.15.22:0/3311464841' entity='mgr.smithi022.gmdcch' 2024-02-16T19:21:02.076 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:01 smithi022 ceph-mon[29221]: from='mgr.14164 172.21.15.22:0/3311464841' entity='mgr.smithi022.gmdcch' 2024-02-16T19:21:02.076 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:01 smithi022 ceph-mon[29221]: from='mgr.14164 172.21.15.22:0/3311464841' entity='mgr.smithi022.gmdcch' 2024-02-16T19:21:02.076 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:01 smithi022 ceph-mon[29221]: from='mgr.14164 172.21.15.22:0/3311464841' entity='mgr.smithi022.gmdcch' 2024-02-16T19:21:02.076 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:01 smithi022 ceph-mon[29221]: from='mgr.14164 172.21.15.22:0/3311464841' entity='mgr.smithi022.gmdcch' 2024-02-16T19:21:02.076 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:01 smithi022 ceph-mon[29221]: from='mgr.14164 172.21.15.22:0/3311464841' entity='mgr.smithi022.gmdcch' 2024-02-16T19:21:02.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:02.587+0000 7fcfac968700 1 -- 172.21.15.117:0/3437327706 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcfa80ffe20 msgr2=0x7fcfa8100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:02.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:02.587+0000 7fcfac968700 1 --2- 172.21.15.117:0/3437327706 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcfa80ffe20 0x7fcfa8100230 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fcfa000b0a0 tx=0x7fcfa000b3b0 comp rx=0 tx=0).stop 2024-02-16T19:21:02.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:02.587+0000 7fcfac968700 1 -- 172.21.15.117:0/3437327706 shutdown_connections 2024-02-16T19:21:02.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:02.587+0000 7fcfac968700 1 --2- 172.21.15.117:0/3437327706 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcfa80ffe20 0x7fcfa8100230 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:02.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:02.587+0000 7fcfac968700 1 -- 172.21.15.117:0/3437327706 >> 172.21.15.117:0/3437327706 conn(0x7fcfa80fb3d0 msgr2=0x7fcfa80fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:21:02.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:02.587+0000 7fcfac968700 1 -- 172.21.15.117:0/3437327706 shutdown_connections 2024-02-16T19:21:02.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:02.587+0000 7fcfac968700 1 -- 172.21.15.117:0/3437327706 wait complete. 2024-02-16T19:21:02.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:02.588+0000 7fcfac968700 1 Processor -- start 2024-02-16T19:21:02.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:02.588+0000 7fcfac968700 1 -- start start 2024-02-16T19:21:02.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:02.588+0000 7fcfac968700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcfa80ffe20 0x7fcfa8074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:02.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:02.588+0000 7fcfac968700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa8075030 con 0x7fcfa80ffe20 2024-02-16T19:21:02.591 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:02.589+0000 7fcfa659c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcfa80ffe20 0x7fcfa8074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:02.591 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:02.589+0000 7fcfa659c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcfa80ffe20 0x7fcfa8074af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.117:55174/0 (socket says 172.21.15.117:55174) 2024-02-16T19:21:02.591 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:02.589+0000 7fcfa659c700 1 -- 172.21.15.117:0/875372957 learned_addr learned my addr 172.21.15.117:0/875372957 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-02-16T19:21:02.591 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:02.589+0000 7fcfa659c700 1 -- 172.21.15.117:0/875372957 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfa0009d00 con 0x7fcfa80ffe20 2024-02-16T19:21:02.591 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:02.589+0000 7fcfa659c700 1 --2- 172.21.15.117:0/875372957 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcfa80ffe20 0x7fcfa8074af0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fcfa0015040 tx=0x7fcfa000bb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:21:02.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:02.589+0000 7fcf9f7fe700 1 -- 172.21.15.117:0/875372957 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcfa000e040 con 0x7fcfa80ffe20 2024-02-16T19:21:02.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:02.589+0000 7fcfac968700 1 -- 172.21.15.117:0/875372957 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfa8073140 con 0x7fcfa80ffe20 2024-02-16T19:21:02.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:02.589+0000 7fcf9f7fe700 1 -- 172.21.15.117:0/875372957 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fcfa001d930 con 0x7fcfa80ffe20 2024-02-16T19:21:02.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:02.589+0000 7fcf9f7fe700 1 -- 172.21.15.117:0/875372957 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcfa00129d0 con 0x7fcfa80ffe20 2024-02-16T19:21:02.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:02.589+0000 7fcfac968700 1 -- 172.21.15.117:0/875372957 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfa8073560 con 0x7fcfa80ffe20 2024-02-16T19:21:02.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:02.590+0000 7fcfac968700 1 -- 172.21.15.117:0/875372957 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfa8073f70 con 0x7fcfa80ffe20 2024-02-16T19:21:02.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:02.590+0000 7fcf9f7fe700 1 -- 172.21.15.117:0/875372957 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7fcfa0012c90 con 0x7fcfa80ffe20 2024-02-16T19:21:02.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:02.590+0000 7fcf9f7fe700 1 --2- 172.21.15.117:0/875372957 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7fcf94038490 0x7fcf9403a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:02.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:02.590+0000 7fcf9f7fe700 1 -- 172.21.15.117:0/875372957 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fcfa004c780 con 0x7fcfa80ffe20 2024-02-16T19:21:02.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:02.591+0000 7fcfa5d9b700 1 --2- 172.21.15.117:0/875372957 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7fcf94038490 0x7fcf9403a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:02.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:02.591+0000 7fcfa5d9b700 1 --2- 172.21.15.117:0/875372957 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7fcf94038490 0x7fcf9403a940 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fcf90006fd0 tx=0x7fcf90006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:21:02.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:02.595+0000 7fcf9f7fe700 1 -- 172.21.15.117:0/875372957 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fcfa001c3e0 con 0x7fcfa80ffe20 2024-02-16T19:21:02.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:02.860+0000 7fcfac968700 1 -- 172.21.15.117:0/875372957 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fcfa81008b0 con 0x7fcfa80ffe20 2024-02-16T19:21:02.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:02.861+0000 7fcf9f7fe700 1 -- 172.21.15.117:0/875372957 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7fcfa81008b0 con 0x7fcfa80ffe20 2024-02-16T19:21:02.862 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:21:02.862 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":1,"fsid":"2a1346e6-cd00-11ee-95ba-87774f69a715","modified":"2024-02-16T19:19:07.447976Z","created":"2024-02-16T19:19:07.447976Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi022","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:3300","nonce":0},{"type":"v1","addr":"172.21.15.22:6789","nonce":0}]},"addr":"172.21.15.22:6789/0","public_addr":"172.21.15.22:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-16T19:21:02.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:02.863+0000 7fcfac968700 1 -- 172.21.15.117:0/875372957 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7fcf94038490 msgr2=0x7fcf9403a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:02.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:02.863+0000 7fcfac968700 1 --2- 172.21.15.117:0/875372957 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7fcf94038490 0x7fcf9403a940 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fcf90006fd0 tx=0x7fcf90006e40 comp rx=0 tx=0).stop 2024-02-16T19:21:02.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:02.863+0000 7fcfac968700 1 -- 172.21.15.117:0/875372957 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcfa80ffe20 msgr2=0x7fcfa8074af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:02.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:02.863+0000 7fcfac968700 1 --2- 172.21.15.117:0/875372957 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcfa80ffe20 0x7fcfa8074af0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fcfa0015040 tx=0x7fcfa000bb10 comp rx=0 tx=0).stop 2024-02-16T19:21:02.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:02.863+0000 7fcfac968700 1 -- 172.21.15.117:0/875372957 shutdown_connections 2024-02-16T19:21:02.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:02.863+0000 7fcfac968700 1 --2- 172.21.15.117:0/875372957 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7fcf94038490 0x7fcf9403a940 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:02.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:02.863+0000 7fcfac968700 1 --2- 172.21.15.117:0/875372957 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcfa80ffe20 0x7fcfa8074af0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:02.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:02.863+0000 7fcfac968700 1 -- 172.21.15.117:0/875372957 >> 172.21.15.117:0/875372957 conn(0x7fcfa80fb3d0 msgr2=0x7fcfa80fc080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:21:02.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:02.863+0000 7fcfac968700 1 -- 172.21.15.117:0/875372957 shutdown_connections 2024-02-16T19:21:02.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:02.863+0000 7fcfac968700 1 -- 172.21.15.117:0/875372957 wait complete. 2024-02-16T19:21:02.865 INFO:teuthology.orchestra.run.smithi117.stderr:dumped monmap epoch 1 2024-02-16T19:21:03.173 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:02 smithi022 ceph-mon[29221]: Deploying daemon prometheus.smithi022 on smithi022 2024-02-16T19:21:03.173 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:02 smithi022 ceph-mon[29221]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:21:04.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:03 smithi022 ceph-mon[29221]: from='client.? 172.21.15.117:0/875372957' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-16T19:21:04.407 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-16T19:21:04.407 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-16T19:21:04.606 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T19:21:04.653 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T19:21:05.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:04 smithi022 ceph-mon[29221]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:21:06.132 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:06.130+0000 7fc7b964e700 1 -- 172.21.15.117:0/4228616036 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc7b4101420 msgr2=0x7fc7b4101830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:06.132 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:06.130+0000 7fc7b964e700 1 --2- 172.21.15.117:0/4228616036 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc7b4101420 0x7fc7b4101830 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc7a400b0a0 tx=0x7fc7a400b3b0 comp rx=0 tx=0).stop 2024-02-16T19:21:06.133 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:06.130+0000 7fc7b964e700 1 -- 172.21.15.117:0/4228616036 shutdown_connections 2024-02-16T19:21:06.133 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:06.130+0000 7fc7b964e700 1 --2- 172.21.15.117:0/4228616036 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc7b4101420 0x7fc7b4101830 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:06.133 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:06.130+0000 7fc7b964e700 1 -- 172.21.15.117:0/4228616036 >> 172.21.15.117:0/4228616036 conn(0x7fc7b40f9a50 msgr2=0x7fc7b40fbe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:21:06.133 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:06.130+0000 7fc7b964e700 1 -- 172.21.15.117:0/4228616036 shutdown_connections 2024-02-16T19:21:06.133 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:06.130+0000 7fc7b964e700 1 -- 172.21.15.117:0/4228616036 wait complete. 2024-02-16T19:21:06.133 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:06.131+0000 7fc7b964e700 1 Processor -- start 2024-02-16T19:21:06.133 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:06.131+0000 7fc7b964e700 1 -- start start 2024-02-16T19:21:06.133 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:06.131+0000 7fc7b964e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc7b4101420 0x7fc7b4194fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:06.133 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:06.131+0000 7fc7b964e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b4195500 con 0x7fc7b4101420 2024-02-16T19:21:06.133 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:06.131+0000 7fc7b2ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc7b4101420 0x7fc7b4194fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:06.133 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:06.132+0000 7fc7b2ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc7b4101420 0x7fc7b4194fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.117:55188/0 (socket says 172.21.15.117:55188) 2024-02-16T19:21:06.133 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:06.132+0000 7fc7b2ffd700 1 -- 172.21.15.117:0/2788384986 learned_addr learned my addr 172.21.15.117:0/2788384986 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-02-16T19:21:06.133 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:06.132+0000 7fc7b2ffd700 1 -- 172.21.15.117:0/2788384986 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7a4009d00 con 0x7fc7b4101420 2024-02-16T19:21:06.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:06.132+0000 7fc7b2ffd700 1 --2- 172.21.15.117:0/2788384986 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc7b4101420 0x7fc7b4194fc0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc7a400b830 tx=0x7fc7a400bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:21:06.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:06.132+0000 7fc7abfff700 1 -- 172.21.15.117:0/2788384986 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc7a400e040 con 0x7fc7b4101420 2024-02-16T19:21:06.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:06.132+0000 7fc7abfff700 1 -- 172.21.15.117:0/2788384986 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc7a4012af0 con 0x7fc7b4101420 2024-02-16T19:21:06.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:06.132+0000 7fc7b964e700 1 -- 172.21.15.117:0/2788384986 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7b4195700 con 0x7fc7b4101420 2024-02-16T19:21:06.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:06.132+0000 7fc7abfff700 1 -- 172.21.15.117:0/2788384986 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc7a401d9f0 con 0x7fc7b4101420 2024-02-16T19:21:06.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:06.132+0000 7fc7b964e700 1 -- 172.21.15.117:0/2788384986 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7b4195b20 con 0x7fc7b4101420 2024-02-16T19:21:06.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:06.133+0000 7fc7abfff700 1 -- 172.21.15.117:0/2788384986 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7fc7a401dcb0 con 0x7fc7b4101420 2024-02-16T19:21:06.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:06.133+0000 7fc7b964e700 1 -- 172.21.15.117:0/2788384986 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7b418eb00 con 0x7fc7b4101420 2024-02-16T19:21:06.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:06.133+0000 7fc7abfff700 1 --2- 172.21.15.117:0/2788384986 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7fc7a0038490 0x7fc7a003a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:06.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:06.133+0000 7fc7abfff700 1 -- 172.21.15.117:0/2788384986 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc7a404caf0 con 0x7fc7b4101420 2024-02-16T19:21:06.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:06.133+0000 7fc7b27fc700 1 --2- 172.21.15.117:0/2788384986 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7fc7a0038490 0x7fc7a003a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:06.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:06.134+0000 7fc7b27fc700 1 --2- 172.21.15.117:0/2788384986 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7fc7a0038490 0x7fc7a003a940 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc79c006fd0 tx=0x7fc79c006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:21:06.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:06.138+0000 7fc7abfff700 1 -- 172.21.15.117:0/2788384986 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fc7a401a820 con 0x7fc7b4101420 2024-02-16T19:21:06.406 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:06.405+0000 7fc7b964e700 1 -- 172.21.15.117:0/2788384986 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fc7b4062380 con 0x7fc7b4101420 2024-02-16T19:21:06.406 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:06.405+0000 7fc7abfff700 1 -- 172.21.15.117:0/2788384986 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7fc7a401a820 con 0x7fc7b4101420 2024-02-16T19:21:06.406 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:21:06.407 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":1,"fsid":"2a1346e6-cd00-11ee-95ba-87774f69a715","modified":"2024-02-16T19:19:07.447976Z","created":"2024-02-16T19:19:07.447976Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi022","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:3300","nonce":0},{"type":"v1","addr":"172.21.15.22:6789","nonce":0}]},"addr":"172.21.15.22:6789/0","public_addr":"172.21.15.22:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-16T19:21:06.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:06.407+0000 7fc7b964e700 1 -- 172.21.15.117:0/2788384986 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7fc7a0038490 msgr2=0x7fc7a003a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:06.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:06.407+0000 7fc7b964e700 1 --2- 172.21.15.117:0/2788384986 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7fc7a0038490 0x7fc7a003a940 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc79c006fd0 tx=0x7fc79c006e40 comp rx=0 tx=0).stop 2024-02-16T19:21:06.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:06.407+0000 7fc7b964e700 1 -- 172.21.15.117:0/2788384986 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc7b4101420 msgr2=0x7fc7b4194fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:06.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:06.407+0000 7fc7b964e700 1 --2- 172.21.15.117:0/2788384986 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc7b4101420 0x7fc7b4194fc0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc7a400b830 tx=0x7fc7a400bd40 comp rx=0 tx=0).stop 2024-02-16T19:21:06.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:06.407+0000 7fc7b964e700 1 -- 172.21.15.117:0/2788384986 shutdown_connections 2024-02-16T19:21:06.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:06.407+0000 7fc7b964e700 1 --2- 172.21.15.117:0/2788384986 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7fc7a0038490 0x7fc7a003a940 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:06.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:06.407+0000 7fc7b964e700 1 --2- 172.21.15.117:0/2788384986 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc7b4101420 0x7fc7b4194fc0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:06.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:06.407+0000 7fc7b964e700 1 -- 172.21.15.117:0/2788384986 >> 172.21.15.117:0/2788384986 conn(0x7fc7b40f9a50 msgr2=0x7fc7b40fa700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:21:06.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:06.408+0000 7fc7b964e700 1 -- 172.21.15.117:0/2788384986 shutdown_connections 2024-02-16T19:21:06.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:06.408+0000 7fc7b964e700 1 -- 172.21.15.117:0/2788384986 wait complete. 2024-02-16T19:21:06.410 INFO:teuthology.orchestra.run.smithi117.stderr:dumped monmap epoch 1 2024-02-16T19:21:07.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:06 smithi022 ceph-mon[29221]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:21:07.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:06 smithi022 ceph-mon[29221]: from='client.? 172.21.15.117:0/2788384986' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-16T19:21:07.994 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-16T19:21:07.995 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-16T19:21:08.195 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T19:21:08.241 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T19:21:09.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:08 smithi022 ceph-mon[29221]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:21:09.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:09.584+0000 7f51f3601700 1 -- 172.21.15.117:0/1565795259 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51ec0fed20 msgr2=0x7f51ec0ff130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:09.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:09.584+0000 7f51f3601700 1 --2- 172.21.15.117:0/1565795259 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51ec0fed20 0x7f51ec0ff130 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f51dc009910 tx=0x7f51dc009c20 comp rx=0 tx=0).stop 2024-02-16T19:21:09.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:09.584+0000 7f51f3601700 1 -- 172.21.15.117:0/1565795259 shutdown_connections 2024-02-16T19:21:09.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:09.584+0000 7f51f3601700 1 --2- 172.21.15.117:0/1565795259 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51ec0fed20 0x7f51ec0ff130 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:09.587 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:09.584+0000 7f51f3601700 1 -- 172.21.15.117:0/1565795259 >> 172.21.15.117:0/1565795259 conn(0x7f51ec0fa2b0 msgr2=0x7f51ec0fc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:21:09.587 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:09.584+0000 7f51f3601700 1 -- 172.21.15.117:0/1565795259 shutdown_connections 2024-02-16T19:21:09.587 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:09.585+0000 7f51f3601700 1 -- 172.21.15.117:0/1565795259 wait complete. 2024-02-16T19:21:09.587 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:09.585+0000 7f51f3601700 1 Processor -- start 2024-02-16T19:21:09.587 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:09.585+0000 7f51f3601700 1 -- start start 2024-02-16T19:21:09.587 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:09.585+0000 7f51f3601700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51ec0fed20 0x7f51ec192d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:09.587 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:09.585+0000 7f51f3601700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51ec1932d0 con 0x7f51ec0fed20 2024-02-16T19:21:09.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:09.586+0000 7f51f139d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51ec0fed20 0x7f51ec192d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:09.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:09.586+0000 7f51f139d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51ec0fed20 0x7f51ec192d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.117:58426/0 (socket says 172.21.15.117:58426) 2024-02-16T19:21:09.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:09.586+0000 7f51f139d700 1 -- 172.21.15.117:0/1775251306 learned_addr learned my addr 172.21.15.117:0/1775251306 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-02-16T19:21:09.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:09.586+0000 7f51f139d700 1 -- 172.21.15.117:0/1775251306 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51dc0095f0 con 0x7f51ec0fed20 2024-02-16T19:21:09.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:09.586+0000 7f51f139d700 1 --2- 172.21.15.117:0/1775251306 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51ec0fed20 0x7f51ec192d90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f51dc005b40 tx=0x7f51dc004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:21:09.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:09.586+0000 7f51e27fc700 1 -- 172.21.15.117:0/1775251306 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f51dc01d070 con 0x7f51ec0fed20 2024-02-16T19:21:09.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:09.586+0000 7f51e27fc700 1 -- 172.21.15.117:0/1775251306 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f51dc00be10 con 0x7f51ec0fed20 2024-02-16T19:21:09.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:09.587+0000 7f51f3601700 1 -- 172.21.15.117:0/1775251306 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51ec1934d0 con 0x7f51ec0fed20 2024-02-16T19:21:09.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:09.587+0000 7f51e27fc700 1 -- 172.21.15.117:0/1775251306 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f51dc00f460 con 0x7f51ec0fed20 2024-02-16T19:21:09.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:09.587+0000 7f51f3601700 1 -- 172.21.15.117:0/1775251306 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51ec193970 con 0x7f51ec0fed20 2024-02-16T19:21:09.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:09.587+0000 7f51e27fc700 1 -- 172.21.15.117:0/1775251306 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7f51dc0227e0 con 0x7f51ec0fed20 2024-02-16T19:21:09.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:09.587+0000 7f51f3601700 1 -- 172.21.15.117:0/1775251306 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51ec18c900 con 0x7f51ec0fed20 2024-02-16T19:21:09.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:09.588+0000 7f51e27fc700 1 --2- 172.21.15.117:0/1775251306 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7f51d803a6f0 0x7f51d803cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:09.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:09.588+0000 7f51e27fc700 1 -- 172.21.15.117:0/1775251306 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f51dc04c480 con 0x7f51ec0fed20 2024-02-16T19:21:09.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:09.588+0000 7f51f0b9c700 1 --2- 172.21.15.117:0/1775251306 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7f51d803a6f0 0x7f51d803cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:09.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:09.588+0000 7f51f0b9c700 1 --2- 172.21.15.117:0/1775251306 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7f51d803a6f0 0x7f51d803cba0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f51e8006fd0 tx=0x7f51e8006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:21:09.594 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:09.593+0000 7f51e27fc700 1 -- 172.21.15.117:0/1775251306 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f51dc02bba0 con 0x7f51ec0fed20 2024-02-16T19:21:09.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:09.857+0000 7f51f3601700 1 -- 172.21.15.117:0/1775251306 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f51ec02cc30 con 0x7f51ec0fed20 2024-02-16T19:21:09.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:09.857+0000 7f51e27fc700 1 -- 172.21.15.117:0/1775251306 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f51dc027030 con 0x7f51ec0fed20 2024-02-16T19:21:09.858 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:21:09.859 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":1,"fsid":"2a1346e6-cd00-11ee-95ba-87774f69a715","modified":"2024-02-16T19:19:07.447976Z","created":"2024-02-16T19:19:07.447976Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi022","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:3300","nonce":0},{"type":"v1","addr":"172.21.15.22:6789","nonce":0}]},"addr":"172.21.15.22:6789/0","public_addr":"172.21.15.22:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-16T19:21:09.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:09.860+0000 7f51f3601700 1 -- 172.21.15.117:0/1775251306 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7f51d803a6f0 msgr2=0x7f51d803cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:09.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:09.860+0000 7f51f3601700 1 --2- 172.21.15.117:0/1775251306 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7f51d803a6f0 0x7f51d803cba0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f51e8006fd0 tx=0x7f51e8006e40 comp rx=0 tx=0).stop 2024-02-16T19:21:09.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:09.860+0000 7f51f3601700 1 -- 172.21.15.117:0/1775251306 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51ec0fed20 msgr2=0x7f51ec192d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:09.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:09.860+0000 7f51f3601700 1 --2- 172.21.15.117:0/1775251306 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51ec0fed20 0x7f51ec192d90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f51dc005b40 tx=0x7f51dc004740 comp rx=0 tx=0).stop 2024-02-16T19:21:09.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:09.860+0000 7f51f3601700 1 -- 172.21.15.117:0/1775251306 shutdown_connections 2024-02-16T19:21:09.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:09.860+0000 7f51f3601700 1 --2- 172.21.15.117:0/1775251306 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7f51d803a6f0 0x7f51d803cba0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:09.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:09.860+0000 7f51f3601700 1 --2- 172.21.15.117:0/1775251306 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51ec0fed20 0x7f51ec192d90 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:09.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:09.860+0000 7f51f3601700 1 -- 172.21.15.117:0/1775251306 >> 172.21.15.117:0/1775251306 conn(0x7f51ec0fa2b0 msgr2=0x7f51ec0faf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:21:09.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:09.860+0000 7f51f3601700 1 -- 172.21.15.117:0/1775251306 shutdown_connections 2024-02-16T19:21:09.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:09.860+0000 7f51f3601700 1 -- 172.21.15.117:0/1775251306 wait complete. 2024-02-16T19:21:09.862 INFO:teuthology.orchestra.run.smithi117.stderr:dumped monmap epoch 1 2024-02-16T19:21:11.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:10 smithi022 ceph-mon[29221]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:21:11.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:10 smithi022 ceph-mon[29221]: from='client.? 172.21.15.117:0/1775251306' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-16T19:21:11.396 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-16T19:21:11.397 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-16T19:21:11.596 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T19:21:11.642 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T19:21:13.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:13.093+0000 7ff6a2504700 1 -- 172.21.15.117:0/2638886151 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff69c0ffe30 msgr2=0x7ff69c100240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:13.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:13.093+0000 7ff6a2504700 1 --2- 172.21.15.117:0/2638886151 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff69c0ffe30 0x7ff69c100240 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff690009910 tx=0x7ff690009c20 comp rx=0 tx=0).stop 2024-02-16T19:21:13.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:13.093+0000 7ff6a2504700 1 -- 172.21.15.117:0/2638886151 shutdown_connections 2024-02-16T19:21:13.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:13.093+0000 7ff6a2504700 1 --2- 172.21.15.117:0/2638886151 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff69c0ffe30 0x7ff69c100240 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:13.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:13.093+0000 7ff6a2504700 1 -- 172.21.15.117:0/2638886151 >> 172.21.15.117:0/2638886151 conn(0x7ff69c0fb3c0 msgr2=0x7ff69c0fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:21:13.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:13.093+0000 7ff6a2504700 1 -- 172.21.15.117:0/2638886151 shutdown_connections 2024-02-16T19:21:13.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:13.093+0000 7ff6a2504700 1 -- 172.21.15.117:0/2638886151 wait complete. 2024-02-16T19:21:13.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:13.094+0000 7ff6a2504700 1 Processor -- start 2024-02-16T19:21:13.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:13.094+0000 7ff6a2504700 1 -- start start 2024-02-16T19:21:13.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:13.094+0000 7ff6a2504700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff69c0ffe30 0x7ff69c194f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:13.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:13.094+0000 7ff6a2504700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff69c1954c0 con 0x7ff69c0ffe30 2024-02-16T19:21:13.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:13.095+0000 7ff69bfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff69c0ffe30 0x7ff69c194f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:13.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:13.095+0000 7ff69bfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff69c0ffe30 0x7ff69c194f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.117:58436/0 (socket says 172.21.15.117:58436) 2024-02-16T19:21:13.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:13.095+0000 7ff69bfff700 1 -- 172.21.15.117:0/1731543739 learned_addr learned my addr 172.21.15.117:0/1731543739 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-02-16T19:21:13.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:13.095+0000 7ff69bfff700 1 -- 172.21.15.117:0/1731543739 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6900095f0 con 0x7ff69c0ffe30 2024-02-16T19:21:13.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:13.095+0000 7ff69bfff700 1 --2- 172.21.15.117:0/1731543739 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff69c0ffe30 0x7ff69c194f80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff690005b40 tx=0x7ff690004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:21:13.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:13.095+0000 7ff6997fa700 1 -- 172.21.15.117:0/1731543739 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff69001d070 con 0x7ff69c0ffe30 2024-02-16T19:21:13.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:13.095+0000 7ff6a2504700 1 -- 172.21.15.117:0/1731543739 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff69c1956c0 con 0x7ff69c0ffe30 2024-02-16T19:21:13.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:13.095+0000 7ff6997fa700 1 -- 172.21.15.117:0/1731543739 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff69000be10 con 0x7ff69c0ffe30 2024-02-16T19:21:13.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:13.095+0000 7ff6997fa700 1 -- 172.21.15.117:0/1731543739 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff69000f460 con 0x7ff69c0ffe30 2024-02-16T19:21:13.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:13.095+0000 7ff6a2504700 1 -- 172.21.15.117:0/1731543739 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff69c195b60 con 0x7ff69c0ffe30 2024-02-16T19:21:13.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:13.096+0000 7ff6997fa700 1 -- 172.21.15.117:0/1731543739 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7ff69000b960 con 0x7ff69c0ffe30 2024-02-16T19:21:13.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:13.096+0000 7ff6a2504700 1 -- 172.21.15.117:0/1731543739 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff69c18ea90 con 0x7ff69c0ffe30 2024-02-16T19:21:13.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:13.096+0000 7ff6997fa700 1 --2- 172.21.15.117:0/1731543739 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7ff684038490 0x7ff68403a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:13.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:13.096+0000 7ff6997fa700 1 -- 172.21.15.117:0/1731543739 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff69004c380 con 0x7ff69c0ffe30 2024-02-16T19:21:13.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:13.096+0000 7ff69b7fe700 1 --2- 172.21.15.117:0/1731543739 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7ff684038490 0x7ff68403a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:13.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:13.097+0000 7ff69b7fe700 1 --2- 172.21.15.117:0/1731543739 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7ff684038490 0x7ff68403a940 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff68c006fd0 tx=0x7ff68c006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:21:13.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:13.102+0000 7ff6997fa700 1 -- 172.21.15.117:0/1731543739 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7ff69002bba0 con 0x7ff69c0ffe30 2024-02-16T19:21:13.187 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:12 smithi022 ceph-mon[29221]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:21:13.365 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:13.363+0000 7ff6a2504700 1 -- 172.21.15.117:0/1731543739 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ff69c02cc30 con 0x7ff69c0ffe30 2024-02-16T19:21:13.366 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:13.364+0000 7ff6997fa700 1 -- 172.21.15.117:0/1731543739 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7ff690027070 con 0x7ff69c0ffe30 2024-02-16T19:21:13.366 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:21:13.366 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":1,"fsid":"2a1346e6-cd00-11ee-95ba-87774f69a715","modified":"2024-02-16T19:19:07.447976Z","created":"2024-02-16T19:19:07.447976Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi022","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:3300","nonce":0},{"type":"v1","addr":"172.21.15.22:6789","nonce":0}]},"addr":"172.21.15.22:6789/0","public_addr":"172.21.15.22:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-16T19:21:13.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:13.366+0000 7ff6a2504700 1 -- 172.21.15.117:0/1731543739 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7ff684038490 msgr2=0x7ff68403a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:13.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:13.366+0000 7ff6a2504700 1 --2- 172.21.15.117:0/1731543739 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7ff684038490 0x7ff68403a940 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff68c006fd0 tx=0x7ff68c006e40 comp rx=0 tx=0).stop 2024-02-16T19:21:13.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:13.366+0000 7ff6a2504700 1 -- 172.21.15.117:0/1731543739 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff69c0ffe30 msgr2=0x7ff69c194f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:13.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:13.366+0000 7ff6a2504700 1 --2- 172.21.15.117:0/1731543739 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff69c0ffe30 0x7ff69c194f80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff690005b40 tx=0x7ff690004740 comp rx=0 tx=0).stop 2024-02-16T19:21:13.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:13.366+0000 7ff6a2504700 1 -- 172.21.15.117:0/1731543739 shutdown_connections 2024-02-16T19:21:13.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:13.366+0000 7ff6a2504700 1 --2- 172.21.15.117:0/1731543739 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7ff684038490 0x7ff68403a940 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:13.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:13.366+0000 7ff6a2504700 1 --2- 172.21.15.117:0/1731543739 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff69c0ffe30 0x7ff69c194f80 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:13.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:13.366+0000 7ff6a2504700 1 -- 172.21.15.117:0/1731543739 >> 172.21.15.117:0/1731543739 conn(0x7ff69c0fb3c0 msgr2=0x7ff69c0fc090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:21:13.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:13.366+0000 7ff6a2504700 1 -- 172.21.15.117:0/1731543739 shutdown_connections 2024-02-16T19:21:13.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:13.366+0000 7ff6a2504700 1 -- 172.21.15.117:0/1731543739 wait complete. 2024-02-16T19:21:13.369 INFO:teuthology.orchestra.run.smithi117.stderr:dumped monmap epoch 1 2024-02-16T19:21:14.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:13 smithi022 ceph-mon[29221]: from='client.? 172.21.15.117:0/1731543739' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-16T19:21:14.931 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-16T19:21:14.932 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-16T19:21:15.131 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T19:21:15.176 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T19:21:15.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:14 smithi022 ceph-mon[29221]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:21:16.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:16.506+0000 7efc9e6e6700 1 -- 172.21.15.117:0/2800493998 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc980ffe20 msgr2=0x7efc98100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:16.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:16.506+0000 7efc9e6e6700 1 --2- 172.21.15.117:0/2800493998 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc980ffe20 0x7efc98100230 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7efc8c009910 tx=0x7efc8c009c20 comp rx=0 tx=0).stop 2024-02-16T19:21:16.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:16.506+0000 7efc9e6e6700 1 -- 172.21.15.117:0/2800493998 shutdown_connections 2024-02-16T19:21:16.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:16.506+0000 7efc9e6e6700 1 --2- 172.21.15.117:0/2800493998 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc980ffe20 0x7efc98100230 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:16.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:16.506+0000 7efc9e6e6700 1 -- 172.21.15.117:0/2800493998 >> 172.21.15.117:0/2800493998 conn(0x7efc980fb3d0 msgr2=0x7efc980fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:21:16.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:16.506+0000 7efc9e6e6700 1 -- 172.21.15.117:0/2800493998 shutdown_connections 2024-02-16T19:21:16.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:16.506+0000 7efc9e6e6700 1 -- 172.21.15.117:0/2800493998 wait complete. 2024-02-16T19:21:16.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:16.506+0000 7efc9e6e6700 1 Processor -- start 2024-02-16T19:21:16.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:16.506+0000 7efc9e6e6700 1 -- start start 2024-02-16T19:21:16.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:16.506+0000 7efc9e6e6700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc980ffe20 0x7efc98194f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:16.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:16.506+0000 7efc9e6e6700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc981954d0 con 0x7efc980ffe20 2024-02-16T19:21:16.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:16.507+0000 7efc97fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc980ffe20 0x7efc98194f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:16.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:16.507+0000 7efc97fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc980ffe20 0x7efc98194f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.117:57344/0 (socket says 172.21.15.117:57344) 2024-02-16T19:21:16.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:16.507+0000 7efc97fff700 1 -- 172.21.15.117:0/2952380897 learned_addr learned my addr 172.21.15.117:0/2952380897 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-02-16T19:21:16.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:16.507+0000 7efc97fff700 1 -- 172.21.15.117:0/2952380897 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc8c0095f0 con 0x7efc980ffe20 2024-02-16T19:21:16.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:16.507+0000 7efc97fff700 1 --2- 172.21.15.117:0/2952380897 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc980ffe20 0x7efc98194f90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7efc8c005b40 tx=0x7efc8c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:21:16.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:16.507+0000 7efc957fa700 1 -- 172.21.15.117:0/2952380897 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efc8c01d070 con 0x7efc980ffe20 2024-02-16T19:21:16.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:16.507+0000 7efc957fa700 1 -- 172.21.15.117:0/2952380897 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7efc8c00be10 con 0x7efc980ffe20 2024-02-16T19:21:16.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:16.507+0000 7efc9e6e6700 1 -- 172.21.15.117:0/2952380897 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc981956d0 con 0x7efc980ffe20 2024-02-16T19:21:16.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:16.507+0000 7efc957fa700 1 -- 172.21.15.117:0/2952380897 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efc8c00f460 con 0x7efc980ffe20 2024-02-16T19:21:16.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:16.507+0000 7efc9e6e6700 1 -- 172.21.15.117:0/2952380897 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc98195b70 con 0x7efc980ffe20 2024-02-16T19:21:16.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:16.508+0000 7efc9e6e6700 1 -- 172.21.15.117:0/2952380897 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc9818ea80 con 0x7efc980ffe20 2024-02-16T19:21:16.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:16.508+0000 7efc957fa700 1 -- 172.21.15.117:0/2952380897 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7efc8c0227e0 con 0x7efc980ffe20 2024-02-16T19:21:16.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:16.508+0000 7efc957fa700 1 --2- 172.21.15.117:0/2952380897 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7efc80038490 0x7efc8003a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:16.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:16.508+0000 7efc957fa700 1 -- 172.21.15.117:0/2952380897 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7efc8c04c2a0 con 0x7efc980ffe20 2024-02-16T19:21:16.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:16.509+0000 7efc977fe700 1 --2- 172.21.15.117:0/2952380897 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7efc80038490 0x7efc8003a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:16.528 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:16.509+0000 7efc977fe700 1 --2- 172.21.15.117:0/2952380897 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7efc80038490 0x7efc8003a940 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7efc88006fd0 tx=0x7efc88006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:21:16.528 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:16.514+0000 7efc957fa700 1 -- 172.21.15.117:0/2952380897 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7efc8c02b590 con 0x7efc980ffe20 2024-02-16T19:21:16.782 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:16.780+0000 7efc9e6e6700 1 -- 172.21.15.117:0/2952380897 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7efc9802cc30 con 0x7efc980ffe20 2024-02-16T19:21:16.782 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:16.780+0000 7efc957fa700 1 -- 172.21.15.117:0/2952380897 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7efc8c027070 con 0x7efc980ffe20 2024-02-16T19:21:16.782 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:21:16.782 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":1,"fsid":"2a1346e6-cd00-11ee-95ba-87774f69a715","modified":"2024-02-16T19:19:07.447976Z","created":"2024-02-16T19:19:07.447976Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi022","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:3300","nonce":0},{"type":"v1","addr":"172.21.15.22:6789","nonce":0}]},"addr":"172.21.15.22:6789/0","public_addr":"172.21.15.22:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-16T19:21:16.784 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:16.782+0000 7efc9e6e6700 1 -- 172.21.15.117:0/2952380897 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7efc80038490 msgr2=0x7efc8003a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:16.785 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:16.782+0000 7efc9e6e6700 1 --2- 172.21.15.117:0/2952380897 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7efc80038490 0x7efc8003a940 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7efc88006fd0 tx=0x7efc88006e40 comp rx=0 tx=0).stop 2024-02-16T19:21:16.785 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:16.782+0000 7efc9e6e6700 1 -- 172.21.15.117:0/2952380897 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc980ffe20 msgr2=0x7efc98194f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:16.785 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:16.782+0000 7efc9e6e6700 1 --2- 172.21.15.117:0/2952380897 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc980ffe20 0x7efc98194f90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7efc8c005b40 tx=0x7efc8c004740 comp rx=0 tx=0).stop 2024-02-16T19:21:16.785 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:16.783+0000 7efc9e6e6700 1 -- 172.21.15.117:0/2952380897 shutdown_connections 2024-02-16T19:21:16.785 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:16.783+0000 7efc9e6e6700 1 --2- 172.21.15.117:0/2952380897 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7efc80038490 0x7efc8003a940 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:16.785 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:16.783+0000 7efc9e6e6700 1 --2- 172.21.15.117:0/2952380897 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc980ffe20 0x7efc98194f90 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:16.785 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:16.783+0000 7efc9e6e6700 1 -- 172.21.15.117:0/2952380897 >> 172.21.15.117:0/2952380897 conn(0x7efc980fb3d0 msgr2=0x7efc980fc080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:21:16.785 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:16.783+0000 7efc9e6e6700 1 -- 172.21.15.117:0/2952380897 shutdown_connections 2024-02-16T19:21:16.785 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:16.783+0000 7efc9e6e6700 1 -- 172.21.15.117:0/2952380897 wait complete. 2024-02-16T19:21:16.785 INFO:teuthology.orchestra.run.smithi117.stderr:dumped monmap epoch 1 2024-02-16T19:21:17.143 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:16 smithi022 ceph-mon[29221]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:21:17.144 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:16 smithi022 ceph-mon[29221]: from='client.? 172.21.15.117:0/2952380897' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-16T19:21:18.316 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-16T19:21:18.317 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-16T19:21:18.516 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T19:21:18.561 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T19:21:19.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:18 smithi022 ceph-mon[29221]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:21:20.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:20.020+0000 7f1312dd2700 1 -- 172.21.15.117:0/3853889041 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f130c100910 msgr2=0x7f130c102cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:20.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:20.020+0000 7f1312dd2700 1 --2- 172.21.15.117:0/3853889041 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f130c100910 0x7f130c102cf0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1300009910 tx=0x7f1300009c20 comp rx=0 tx=0).stop 2024-02-16T19:21:20.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:20.021+0000 7f1312dd2700 1 -- 172.21.15.117:0/3853889041 shutdown_connections 2024-02-16T19:21:20.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:20.021+0000 7f1312dd2700 1 --2- 172.21.15.117:0/3853889041 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f130c100910 0x7f130c102cf0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:20.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:20.021+0000 7f1312dd2700 1 -- 172.21.15.117:0/3853889041 >> 172.21.15.117:0/3853889041 conn(0x7f130c0fa2b0 msgr2=0x7f130c0fc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:21:20.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:20.021+0000 7f1312dd2700 1 -- 172.21.15.117:0/3853889041 shutdown_connections 2024-02-16T19:21:20.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:20.021+0000 7f1312dd2700 1 -- 172.21.15.117:0/3853889041 wait complete. 2024-02-16T19:21:20.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:20.021+0000 7f1312dd2700 1 Processor -- start 2024-02-16T19:21:20.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:20.021+0000 7f1312dd2700 1 -- start start 2024-02-16T19:21:20.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:20.021+0000 7f1312dd2700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f130c100910 0x7f130c194f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:20.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:20.021+0000 7f1312dd2700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f130c195470 con 0x7f130c100910 2024-02-16T19:21:20.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:20.022+0000 7f1310b6e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f130c100910 0x7f130c194f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:20.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:20.022+0000 7f1310b6e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f130c100910 0x7f130c194f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.117:57364/0 (socket says 172.21.15.117:57364) 2024-02-16T19:21:20.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:20.022+0000 7f1310b6e700 1 -- 172.21.15.117:0/2136919515 learned_addr learned my addr 172.21.15.117:0/2136919515 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-02-16T19:21:20.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:20.022+0000 7f1310b6e700 1 -- 172.21.15.117:0/2136919515 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13000095f0 con 0x7f130c100910 2024-02-16T19:21:20.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:20.022+0000 7f1310b6e700 1 --2- 172.21.15.117:0/2136919515 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f130c100910 0x7f130c194f30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1300005b40 tx=0x7f1300004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:21:20.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:20.022+0000 7f1309ffb700 1 -- 172.21.15.117:0/2136919515 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f130001d070 con 0x7f130c100910 2024-02-16T19:21:20.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:20.022+0000 7f1312dd2700 1 -- 172.21.15.117:0/2136919515 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f130c195670 con 0x7f130c100910 2024-02-16T19:21:20.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:20.022+0000 7f1309ffb700 1 -- 172.21.15.117:0/2136919515 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f130000be10 con 0x7f130c100910 2024-02-16T19:21:20.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:20.022+0000 7f1309ffb700 1 -- 172.21.15.117:0/2136919515 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f130000f460 con 0x7f130c100910 2024-02-16T19:21:20.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:20.022+0000 7f1312dd2700 1 -- 172.21.15.117:0/2136919515 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f130c195b10 con 0x7f130c100910 2024-02-16T19:21:20.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:20.023+0000 7f1309ffb700 1 -- 172.21.15.117:0/2136919515 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 14) v1 ==== 45154+0+0 (secure 0 0 0) 0x7f130000b960 con 0x7f130c100910 2024-02-16T19:21:20.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:20.023+0000 7f1312dd2700 1 -- 172.21.15.117:0/2136919515 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f130c18eb00 con 0x7f130c100910 2024-02-16T19:21:20.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:20.023+0000 7f1309ffb700 1 --2- 172.21.15.117:0/2136919515 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7f12fc0384e0 0x7f12fc03a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:20.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:20.023+0000 7f1309ffb700 1 -- 172.21.15.117:0/2136919515 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f130004c540 con 0x7f130c100910 2024-02-16T19:21:20.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:20.023+0000 7f130bfff700 1 -- 172.21.15.117:0/2136919515 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7f12fc0384e0 msgr2=0x7f12fc03a990 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/2945983569 2024-02-16T19:21:20.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:20.023+0000 7f130bfff700 1 --2- 172.21.15.117:0/2136919515 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7f12fc0384e0 0x7f12fc03a990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-16T19:21:20.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:20.029+0000 7f1309ffb700 1 -- 172.21.15.117:0/2136919515 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f130002a360 con 0x7f130c100910 2024-02-16T19:21:20.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:19 smithi022 ceph-mon[29221]: from='mgr.14164 172.21.15.22:0/3311464841' entity='mgr.smithi022.gmdcch' 2024-02-16T19:21:20.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:19 smithi022 ceph-mon[29221]: from='mgr.14164 172.21.15.22:0/3311464841' entity='mgr.smithi022.gmdcch' 2024-02-16T19:21:20.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:19 smithi022 ceph-mon[29221]: from='mgr.14164 172.21.15.22:0/3311464841' entity='mgr.smithi022.gmdcch' 2024-02-16T19:21:20.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:19 smithi022 ceph-mon[29221]: from='mgr.14164 172.21.15.22:0/3311464841' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-02-16T19:21:20.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:19 smithi022 ceph-mon[29221]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:21:20.225 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:20.225+0000 7f130bfff700 1 -- 172.21.15.117:0/2136919515 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7f12fc0384e0 msgr2=0x7f12fc03a990 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/2945983569 2024-02-16T19:21:20.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:20.225+0000 7f130bfff700 1 --2- 172.21.15.117:0/2136919515 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7f12fc0384e0 0x7f12fc03a990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-16T19:21:20.296 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:20.296+0000 7f1312dd2700 1 -- 172.21.15.117:0/2136919515 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f130c02cc30 con 0x7f130c100910 2024-02-16T19:21:20.297 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:20.296+0000 7f1309ffb700 1 -- 172.21.15.117:0/2136919515 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f1300027030 con 0x7f130c100910 2024-02-16T19:21:20.297 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:21:20.297 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":1,"fsid":"2a1346e6-cd00-11ee-95ba-87774f69a715","modified":"2024-02-16T19:19:07.447976Z","created":"2024-02-16T19:19:07.447976Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi022","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:3300","nonce":0},{"type":"v1","addr":"172.21.15.22:6789","nonce":0}]},"addr":"172.21.15.22:6789/0","public_addr":"172.21.15.22:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-16T19:21:20.299 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:20.298+0000 7f1312dd2700 1 -- 172.21.15.117:0/2136919515 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7f12fc0384e0 msgr2=0x7f12fc03a990 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-16T19:21:20.299 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:20.298+0000 7f1312dd2700 1 --2- 172.21.15.117:0/2136919515 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7f12fc0384e0 0x7f12fc03a990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:20.299 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:20.298+0000 7f1312dd2700 1 -- 172.21.15.117:0/2136919515 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f130c100910 msgr2=0x7f130c194f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:20.299 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:20.298+0000 7f1312dd2700 1 --2- 172.21.15.117:0/2136919515 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f130c100910 0x7f130c194f30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1300005b40 tx=0x7f1300004740 comp rx=0 tx=0).stop 2024-02-16T19:21:20.299 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:20.298+0000 7f1312dd2700 1 -- 172.21.15.117:0/2136919515 shutdown_connections 2024-02-16T19:21:20.299 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:20.298+0000 7f1312dd2700 1 --2- 172.21.15.117:0/2136919515 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7f12fc0384e0 0x7f12fc03a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:20.300 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:20.298+0000 7f1312dd2700 1 --2- 172.21.15.117:0/2136919515 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f130c100910 0x7f130c194f30 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:20.300 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:20.298+0000 7f1312dd2700 1 -- 172.21.15.117:0/2136919515 >> 172.21.15.117:0/2136919515 conn(0x7f130c0fa2b0 msgr2=0x7f130c0faf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:21:20.300 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:20.298+0000 7f1312dd2700 1 -- 172.21.15.117:0/2136919515 shutdown_connections 2024-02-16T19:21:20.300 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:20.298+0000 7f1312dd2700 1 -- 172.21.15.117:0/2136919515 wait complete. 2024-02-16T19:21:20.300 INFO:teuthology.orchestra.run.smithi117.stderr:dumped monmap epoch 1 2024-02-16T19:21:21.200 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:20 smithi022 ceph-mon[29221]: from='mgr.14164 172.21.15.22:0/3311464841' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-02-16T19:21:21.200 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:20 smithi022 ceph-mon[29221]: mgrmap e14: smithi022.gmdcch(active, since 68s) 2024-02-16T19:21:21.200 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:20 smithi022 ceph-mon[29221]: from='client.? 172.21.15.117:0/2136919515' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-16T19:21:21.880 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-16T19:21:21.880 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-16T19:21:22.074 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T19:21:22.121 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T19:21:23.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:23.613+0000 7f353b514700 1 -- 172.21.15.117:0/1715007494 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35340fed10 msgr2=0x7f35340ff120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:23.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:23.613+0000 7f353b514700 1 --2- 172.21.15.117:0/1715007494 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35340fed10 0x7f35340ff120 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3530009910 tx=0x7f3530009c20 comp rx=0 tx=0).stop 2024-02-16T19:21:23.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:23.614+0000 7f353b514700 1 -- 172.21.15.117:0/1715007494 shutdown_connections 2024-02-16T19:21:23.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:23.614+0000 7f353b514700 1 --2- 172.21.15.117:0/1715007494 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35340fed10 0x7f35340ff120 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:23.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:23.614+0000 7f353b514700 1 -- 172.21.15.117:0/1715007494 >> 172.21.15.117:0/1715007494 conn(0x7f35340fa380 msgr2=0x7f35340fc790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:21:23.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:23.614+0000 7f353b514700 1 -- 172.21.15.117:0/1715007494 shutdown_connections 2024-02-16T19:21:23.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:23.614+0000 7f353b514700 1 -- 172.21.15.117:0/1715007494 wait complete. 2024-02-16T19:21:23.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:23.614+0000 7f353b514700 1 Processor -- start 2024-02-16T19:21:23.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:23.614+0000 7f353b514700 1 -- start start 2024-02-16T19:21:23.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:23.614+0000 7f353b514700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35340fed10 0x7f3534194f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:23.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:23.614+0000 7f353b514700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3534195490 con 0x7f35340fed10 2024-02-16T19:21:23.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:23.615+0000 7f35392b0700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35340fed10 0x7f3534194f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:23.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:23.615+0000 7f35392b0700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35340fed10 0x7f3534194f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.117:57374/0 (socket says 172.21.15.117:57374) 2024-02-16T19:21:23.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:23.615+0000 7f35392b0700 1 -- 172.21.15.117:0/2267122537 learned_addr learned my addr 172.21.15.117:0/2267122537 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-02-16T19:21:23.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:23.615+0000 7f35392b0700 1 -- 172.21.15.117:0/2267122537 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35300095f0 con 0x7f35340fed10 2024-02-16T19:21:23.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:23.615+0000 7f35392b0700 1 --2- 172.21.15.117:0/2267122537 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35340fed10 0x7f3534194f50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3530005b40 tx=0x7f3530004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:21:23.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:23.615+0000 7f352a7fc700 1 -- 172.21.15.117:0/2267122537 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f353001d070 con 0x7f35340fed10 2024-02-16T19:21:23.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:23.615+0000 7f353b514700 1 -- 172.21.15.117:0/2267122537 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3534195690 con 0x7f35340fed10 2024-02-16T19:21:23.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:23.615+0000 7f352a7fc700 1 -- 172.21.15.117:0/2267122537 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f353000be10 con 0x7f35340fed10 2024-02-16T19:21:23.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:23.615+0000 7f352a7fc700 1 -- 172.21.15.117:0/2267122537 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f353000f460 con 0x7f35340fed10 2024-02-16T19:21:23.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:23.615+0000 7f353b514700 1 -- 172.21.15.117:0/2267122537 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3534195b30 con 0x7f35340fed10 2024-02-16T19:21:23.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:23.616+0000 7f353b514700 1 -- 172.21.15.117:0/2267122537 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f353418e980 con 0x7f35340fed10 2024-02-16T19:21:23.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:23.616+0000 7f352a7fc700 1 -- 172.21.15.117:0/2267122537 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 14) v1 ==== 45154+0+0 (secure 0 0 0) 0x7f35300227e0 con 0x7f35340fed10 2024-02-16T19:21:23.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:23.616+0000 7f352a7fc700 1 --2- 172.21.15.117:0/2267122537 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7f35200384e0 0x7f352003a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:23.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:23.616+0000 7f352a7fc700 1 -- 172.21.15.117:0/2267122537 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f353004c530 con 0x7f35340fed10 2024-02-16T19:21:23.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:23.616+0000 7f3538aaf700 1 -- 172.21.15.117:0/2267122537 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7f35200384e0 msgr2=0x7f352003a990 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/2945983569 2024-02-16T19:21:23.620 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:23.616+0000 7f3538aaf700 1 --2- 172.21.15.117:0/2267122537 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7f35200384e0 0x7f352003a990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-16T19:21:23.622 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:23.621+0000 7f352a7fc700 1 -- 172.21.15.117:0/2267122537 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f353002bba0 con 0x7f35340fed10 2024-02-16T19:21:23.818 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:23.817+0000 7f3538aaf700 1 -- 172.21.15.117:0/2267122537 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7f35200384e0 msgr2=0x7f352003a990 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/2945983569 2024-02-16T19:21:23.818 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:23.817+0000 7f3538aaf700 1 --2- 172.21.15.117:0/2267122537 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7f35200384e0 0x7f352003a990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-16T19:21:23.888 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:23.886+0000 7f353b514700 1 -- 172.21.15.117:0/2267122537 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f353402cc30 con 0x7f35340fed10 2024-02-16T19:21:23.888 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:23.887+0000 7f352a7fc700 1 -- 172.21.15.117:0/2267122537 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f3530027070 con 0x7f35340fed10 2024-02-16T19:21:23.888 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:21:23.888 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":1,"fsid":"2a1346e6-cd00-11ee-95ba-87774f69a715","modified":"2024-02-16T19:19:07.447976Z","created":"2024-02-16T19:19:07.447976Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi022","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:3300","nonce":0},{"type":"v1","addr":"172.21.15.22:6789","nonce":0}]},"addr":"172.21.15.22:6789/0","public_addr":"172.21.15.22:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-16T19:21:23.890 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:23.889+0000 7f353b514700 1 -- 172.21.15.117:0/2267122537 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7f35200384e0 msgr2=0x7f352003a990 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-16T19:21:23.890 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:23.889+0000 7f353b514700 1 --2- 172.21.15.117:0/2267122537 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7f35200384e0 0x7f352003a990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:23.890 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:23.889+0000 7f353b514700 1 -- 172.21.15.117:0/2267122537 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35340fed10 msgr2=0x7f3534194f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:23.890 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:23.889+0000 7f353b514700 1 --2- 172.21.15.117:0/2267122537 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35340fed10 0x7f3534194f50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3530005b40 tx=0x7f3530004740 comp rx=0 tx=0).stop 2024-02-16T19:21:23.890 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:23.889+0000 7f353b514700 1 -- 172.21.15.117:0/2267122537 shutdown_connections 2024-02-16T19:21:23.890 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:23.889+0000 7f353b514700 1 --2- 172.21.15.117:0/2267122537 >> [v2:172.21.15.22:6800/2945983569,v1:172.21.15.22:6801/2945983569] conn(0x7f35200384e0 0x7f352003a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:23.890 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:23.889+0000 7f353b514700 1 --2- 172.21.15.117:0/2267122537 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35340fed10 0x7f3534194f50 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:23.890 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:23.889+0000 7f353b514700 1 -- 172.21.15.117:0/2267122537 >> 172.21.15.117:0/2267122537 conn(0x7f35340fa380 msgr2=0x7f35340690d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:21:23.890 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:23.889+0000 7f353b514700 1 -- 172.21.15.117:0/2267122537 shutdown_connections 2024-02-16T19:21:23.890 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:23.889+0000 7f353b514700 1 -- 172.21.15.117:0/2267122537 wait complete. 2024-02-16T19:21:23.891 INFO:teuthology.orchestra.run.smithi117.stderr:dumped monmap epoch 1 2024-02-16T19:21:24.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:23 smithi022 ceph-mon[29221]: from='client.? 172.21.15.117:0/2267122537' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-16T19:21:25.447 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-16T19:21:25.447 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-16T19:21:25.650 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T19:21:25.696 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T19:21:27.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:27.179+0000 7fcd9d164700 1 -- 172.21.15.117:0/3175783079 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcd980fed30 msgr2=0x7fcd980ff140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:27.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:27.179+0000 7fcd9d164700 1 --2- 172.21.15.117:0/3175783079 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcd980fed30 0x7fcd980ff140 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fcd88009910 tx=0x7fcd88009c20 comp rx=0 tx=0).stop 2024-02-16T19:21:27.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:27.179+0000 7fcd9d164700 1 -- 172.21.15.117:0/3175783079 shutdown_connections 2024-02-16T19:21:27.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:27.179+0000 7fcd9d164700 1 --2- 172.21.15.117:0/3175783079 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcd980fed30 0x7fcd980ff140 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:27.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:27.179+0000 7fcd9d164700 1 -- 172.21.15.117:0/3175783079 >> 172.21.15.117:0/3175783079 conn(0x7fcd980fa2c0 msgr2=0x7fcd980fc710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:21:27.182 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:27.179+0000 7fcd9d164700 1 -- 172.21.15.117:0/3175783079 shutdown_connections 2024-02-16T19:21:27.182 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:27.179+0000 7fcd9d164700 1 -- 172.21.15.117:0/3175783079 wait complete. 2024-02-16T19:21:27.182 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:27.180+0000 7fcd9d164700 1 Processor -- start 2024-02-16T19:21:27.183 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:27.180+0000 7fcd9d164700 1 -- start start 2024-02-16T19:21:27.183 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:27.180+0000 7fcd9d164700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcd980fed30 0x7fcd98069360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:27.183 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:27.180+0000 7fcd9d164700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd98197d00 con 0x7fcd980fed30 2024-02-16T19:21:27.183 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:27.181+0000 7fcd96d9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcd980fed30 0x7fcd98069360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:27.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:27.181+0000 7fcd96d9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcd980fed30 0x7fcd98069360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.117:36776/0 (socket says 172.21.15.117:36776) 2024-02-16T19:21:27.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:27.181+0000 7fcd96d9d700 1 -- 172.21.15.117:0/778492240 learned_addr learned my addr 172.21.15.117:0/778492240 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-02-16T19:21:27.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:27.181+0000 7fcd96d9d700 1 -- 172.21.15.117:0/778492240 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd880095f0 con 0x7fcd980fed30 2024-02-16T19:21:27.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:27.181+0000 7fcd96d9d700 1 --2- 172.21.15.117:0/778492240 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcd980fed30 0x7fcd98069360 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fcd88005b40 tx=0x7fcd88004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:21:27.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:27.181+0000 7fcd7ffff700 1 -- 172.21.15.117:0/778492240 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcd8801d070 con 0x7fcd980fed30 2024-02-16T19:21:27.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:27.181+0000 7fcd7ffff700 1 -- 172.21.15.117:0/778492240 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fcd8800be10 con 0x7fcd980fed30 2024-02-16T19:21:27.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:27.181+0000 7fcd9d164700 1 -- 172.21.15.117:0/778492240 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd98197e40 con 0x7fcd980fed30 2024-02-16T19:21:27.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:27.181+0000 7fcd7ffff700 1 -- 172.21.15.117:0/778492240 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcd8800f460 con 0x7fcd980fed30 2024-02-16T19:21:27.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:27.181+0000 7fcd9d164700 1 -- 172.21.15.117:0/778492240 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd981982e0 con 0x7fcd980fed30 2024-02-16T19:21:27.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:27.182+0000 7fcd7ffff700 1 -- 172.21.15.117:0/778492240 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 15) v1 ==== 44878+0+0 (secure 0 0 0) 0x7fcd8800f5c0 con 0x7fcd980fed30 2024-02-16T19:21:27.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:27.182+0000 7fcd9d164700 1 -- 172.21.15.117:0/778492240 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd9818da10 con 0x7fcd980fed30 2024-02-16T19:21:27.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:27.182+0000 7fcd7ffff700 1 -- 172.21.15.117:0/778492240 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fcd8804c070 con 0x7fcd980fed30 2024-02-16T19:21:27.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:27.188+0000 7fcd7ffff700 1 -- 172.21.15.117:0/778492240 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fcd88027030 con 0x7fcd980fed30 2024-02-16T19:21:27.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:26 smithi022 ceph-mon[29221]: Active manager daemon smithi022.gmdcch restarted 2024-02-16T19:21:27.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:26 smithi022 ceph-mon[29221]: Activating manager daemon smithi022.gmdcch 2024-02-16T19:21:27.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:26 smithi022 ceph-mon[29221]: osdmap e5: 0 total, 0 up, 0 in 2024-02-16T19:21:27.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:26 smithi022 ceph-mon[29221]: mgrmap e15: smithi022.gmdcch(active, starting, since 0.00340187s) 2024-02-16T19:21:27.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:26 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mon metadata", "id": "smithi022"}]: dispatch 2024-02-16T19:21:27.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:26 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mgr metadata", "who": "smithi022.gmdcch", "id": "smithi022.gmdcch"}]: dispatch 2024-02-16T19:21:27.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:26 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-16T19:21:27.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:26 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-16T19:21:27.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:26 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-16T19:21:27.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:26 smithi022 ceph-mon[29221]: Manager daemon smithi022.gmdcch is now available 2024-02-16T19:21:27.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:26 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:21:27.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:26 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:21:27.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:27.465+0000 7fcd9d164700 1 -- 172.21.15.117:0/778492240 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fcd9802cc30 con 0x7fcd980fed30 2024-02-16T19:21:27.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:27.466+0000 7fcd7ffff700 1 -- 172.21.15.117:0/778492240 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7fcd8802c440 con 0x7fcd980fed30 2024-02-16T19:21:27.467 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:21:27.467 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":1,"fsid":"2a1346e6-cd00-11ee-95ba-87774f69a715","modified":"2024-02-16T19:19:07.447976Z","created":"2024-02-16T19:19:07.447976Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi022","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:3300","nonce":0},{"type":"v1","addr":"172.21.15.22:6789","nonce":0}]},"addr":"172.21.15.22:6789/0","public_addr":"172.21.15.22:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-16T19:21:27.469 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:27.468+0000 7fcd9d164700 1 -- 172.21.15.117:0/778492240 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcd980fed30 msgr2=0x7fcd98069360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:27.469 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:27.468+0000 7fcd9d164700 1 --2- 172.21.15.117:0/778492240 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcd980fed30 0x7fcd98069360 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fcd88005b40 tx=0x7fcd88004740 comp rx=0 tx=0).stop 2024-02-16T19:21:27.469 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:27.468+0000 7fcd9d164700 1 -- 172.21.15.117:0/778492240 shutdown_connections 2024-02-16T19:21:27.469 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:27.468+0000 7fcd9d164700 1 --2- 172.21.15.117:0/778492240 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcd980fed30 0x7fcd98069360 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:27.469 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:27.468+0000 7fcd9d164700 1 -- 172.21.15.117:0/778492240 >> 172.21.15.117:0/778492240 conn(0x7fcd980fa2c0 msgr2=0x7fcd980faf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:21:27.469 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:27.468+0000 7fcd9d164700 1 -- 172.21.15.117:0/778492240 shutdown_connections 2024-02-16T19:21:27.469 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:27.468+0000 7fcd9d164700 1 -- 172.21.15.117:0/778492240 wait complete. 2024-02-16T19:21:27.470 INFO:teuthology.orchestra.run.smithi117.stderr:dumped monmap epoch 1 2024-02-16T19:21:27.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:27 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:21:27.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:27 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:21:27.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:27 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi022.gmdcch/mirror_snapshot_schedule"}]: dispatch 2024-02-16T19:21:27.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:27 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi022.gmdcch/trash_purge_schedule"}]: dispatch 2024-02-16T19:21:27.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:27 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:21:27.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:27 smithi022 ceph-mon[29221]: from='client.? 172.21.15.117:0/778492240' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-16T19:21:27.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:27 smithi022 ceph-mon[29221]: mgrmap e16: smithi022.gmdcch(active, since 1.01596s) 2024-02-16T19:21:29.044 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-16T19:21:29.044 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-16T19:21:29.240 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T19:21:29.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:29 smithi022 ceph-mon[29221]: [16/Feb/2024:19:21:28] ENGINE Bus STARTING 2024-02-16T19:21:29.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:29 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:21:29.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:29 smithi022 ceph-mon[29221]: [16/Feb/2024:19:21:28] ENGINE Serving on http://172.21.15.22:8765 2024-02-16T19:21:29.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:29 smithi022 ceph-mon[29221]: [16/Feb/2024:19:21:28] ENGINE Serving on https://172.21.15.22:7150 2024-02-16T19:21:29.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:29 smithi022 ceph-mon[29221]: [16/Feb/2024:19:21:28] ENGINE Bus STARTED 2024-02-16T19:21:29.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:29 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:21:30.249 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T19:21:30.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:30 smithi022 ceph-mon[29221]: mgrmap e17: smithi022.gmdcch(active, since 2s) 2024-02-16T19:21:32.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:32 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:21:32.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:32 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:21:32.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:32 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:21:32.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:32 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:21:32.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:32 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "who": "osd/host:smithi022", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:21:32.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:32 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:21:32.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:32 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:21:32.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:32 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:21:32.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:32 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:21:32.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:32 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "who": "osd/host:smithi117", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:21:32.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:32 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:21:32.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:32 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:21:32.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:32 smithi022 ceph-mon[29221]: Updating smithi022:/etc/ceph/ceph.conf 2024-02-16T19:21:32.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:32 smithi022 ceph-mon[29221]: Updating smithi117:/etc/ceph/ceph.conf 2024-02-16T19:21:32.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:32.896+0000 7fa7b453f700 1 -- 172.21.15.117:0/1498245673 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa7ac100f30 msgr2=0x7fa7ac101300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:32.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:32.896+0000 7fa7b453f700 1 --2- 172.21.15.117:0/1498245673 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa7ac100f30 0x7fa7ac101300 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa79c009910 tx=0x7fa79c009c20 comp rx=0 tx=0).stop 2024-02-16T19:21:32.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:32.896+0000 7fa7b453f700 1 -- 172.21.15.117:0/1498245673 shutdown_connections 2024-02-16T19:21:32.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:32.896+0000 7fa7b453f700 1 --2- 172.21.15.117:0/1498245673 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa7ac100f30 0x7fa7ac101300 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:32.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:32.896+0000 7fa7b453f700 1 -- 172.21.15.117:0/1498245673 >> 172.21.15.117:0/1498245673 conn(0x7fa7ac074af0 msgr2=0x7fa7ac074ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:21:32.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:32.896+0000 7fa7b453f700 1 -- 172.21.15.117:0/1498245673 shutdown_connections 2024-02-16T19:21:32.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:32.896+0000 7fa7b453f700 1 -- 172.21.15.117:0/1498245673 wait complete. 2024-02-16T19:21:32.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:32.896+0000 7fa7b453f700 1 Processor -- start 2024-02-16T19:21:32.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:32.896+0000 7fa7b453f700 1 -- start start 2024-02-16T19:21:32.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:32.897+0000 7fa7b453f700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa7ac100f30 0x7fa7ac195390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:32.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:32.897+0000 7fa7b453f700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7ac1958d0 con 0x7fa7ac100f30 2024-02-16T19:21:32.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:32.897+0000 7fa7b22db700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa7ac100f30 0x7fa7ac195390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:32.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:32.897+0000 7fa7b22db700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa7ac100f30 0x7fa7ac195390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.117:36788/0 (socket says 172.21.15.117:36788) 2024-02-16T19:21:32.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:32.897+0000 7fa7b22db700 1 -- 172.21.15.117:0/3643104627 learned_addr learned my addr 172.21.15.117:0/3643104627 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-02-16T19:21:32.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:32.897+0000 7fa7b22db700 1 -- 172.21.15.117:0/3643104627 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa79c0095f0 con 0x7fa7ac100f30 2024-02-16T19:21:32.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:32.897+0000 7fa7b22db700 1 --2- 172.21.15.117:0/3643104627 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa7ac100f30 0x7fa7ac195390 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa79c005b40 tx=0x7fa79c004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:21:32.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:32.897+0000 7fa7a37fe700 1 -- 172.21.15.117:0/3643104627 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa79c01d070 con 0x7fa7ac100f30 2024-02-16T19:21:32.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:32.897+0000 7fa7b453f700 1 -- 172.21.15.117:0/3643104627 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7ac195ad0 con 0x7fa7ac100f30 2024-02-16T19:21:32.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:32.897+0000 7fa7a37fe700 1 -- 172.21.15.117:0/3643104627 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa79c0044d0 con 0x7fa7ac100f30 2024-02-16T19:21:32.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:32.897+0000 7fa7a37fe700 1 -- 172.21.15.117:0/3643104627 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa79c003930 con 0x7fa7ac100f30 2024-02-16T19:21:32.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:32.898+0000 7fa7b453f700 1 -- 172.21.15.117:0/3643104627 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7ac195e90 con 0x7fa7ac100f30 2024-02-16T19:21:32.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:32.898+0000 7fa7a37fe700 1 -- 172.21.15.117:0/3643104627 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) v1 ==== 45197+0+0 (secure 0 0 0) 0x7fa79c003bf0 con 0x7fa7ac100f30 2024-02-16T19:21:32.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:32.898+0000 7fa7b453f700 1 -- 172.21.15.117:0/3643104627 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7ac062380 con 0x7fa7ac100f30 2024-02-16T19:21:32.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:32.899+0000 7fa7a37fe700 1 --2- 172.21.15.117:0/3643104627 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fa798038580 0x7fa79803aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:32.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:32.899+0000 7fa7a37fe700 1 -- 172.21.15.117:0/3643104627 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fa79c04c5f0 con 0x7fa7ac100f30 2024-02-16T19:21:32.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:32.899+0000 7fa7b1ada700 1 --2- 172.21.15.117:0/3643104627 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fa798038580 0x7fa79803aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:32.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:32.899+0000 7fa7b1ada700 1 --2- 172.21.15.117:0/3643104627 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fa798038580 0x7fa79803aa30 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8006fd0 tx=0x7fa7a8006e40 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:21:32.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:32.904+0000 7fa7a37fe700 1 -- 172.21.15.117:0/3643104627 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fa79c02a950 con 0x7fa7ac100f30 2024-02-16T19:21:33.180 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:33.179+0000 7fa7b453f700 1 -- 172.21.15.117:0/3643104627 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fa7ac107130 con 0x7fa7ac100f30 2024-02-16T19:21:33.183 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:33.180+0000 7fa7a37fe700 1 -- 172.21.15.117:0/3643104627 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7fa79c02a360 con 0x7fa7ac100f30 2024-02-16T19:21:33.184 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:21:33.184 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":1,"fsid":"2a1346e6-cd00-11ee-95ba-87774f69a715","modified":"2024-02-16T19:19:07.447976Z","created":"2024-02-16T19:19:07.447976Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi022","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:3300","nonce":0},{"type":"v1","addr":"172.21.15.22:6789","nonce":0}]},"addr":"172.21.15.22:6789/0","public_addr":"172.21.15.22:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-16T19:21:33.185 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:33.182+0000 7fa7b453f700 1 -- 172.21.15.117:0/3643104627 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fa798038580 msgr2=0x7fa79803aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:33.185 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:33.182+0000 7fa7b453f700 1 --2- 172.21.15.117:0/3643104627 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fa798038580 0x7fa79803aa30 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8006fd0 tx=0x7fa7a8006e40 comp rx=0 tx=0).stop 2024-02-16T19:21:33.185 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:33.182+0000 7fa7b453f700 1 -- 172.21.15.117:0/3643104627 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa7ac100f30 msgr2=0x7fa7ac195390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:33.185 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:33.182+0000 7fa7b453f700 1 --2- 172.21.15.117:0/3643104627 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa7ac100f30 0x7fa7ac195390 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa79c005b40 tx=0x7fa79c004c80 comp rx=0 tx=0).stop 2024-02-16T19:21:33.185 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:33.182+0000 7fa7b453f700 1 -- 172.21.15.117:0/3643104627 shutdown_connections 2024-02-16T19:21:33.185 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:33.182+0000 7fa7b453f700 1 --2- 172.21.15.117:0/3643104627 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fa798038580 0x7fa79803aa30 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:33.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:33.182+0000 7fa7b453f700 1 --2- 172.21.15.117:0/3643104627 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa7ac100f30 0x7fa7ac195390 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:33.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:33.182+0000 7fa7b453f700 1 -- 172.21.15.117:0/3643104627 >> 172.21.15.117:0/3643104627 conn(0x7fa7ac074af0 msgr2=0x7fa7ac104750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:21:33.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:33.182+0000 7fa7b453f700 1 -- 172.21.15.117:0/3643104627 shutdown_connections 2024-02-16T19:21:33.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:33.182+0000 7fa7b453f700 1 -- 172.21.15.117:0/3643104627 wait complete. 2024-02-16T19:21:33.186 INFO:teuthology.orchestra.run.smithi117.stderr:dumped monmap epoch 1 2024-02-16T19:21:33.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:33 smithi022 ceph-mon[29221]: Updating smithi022:/var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T19:21:33.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:33 smithi022 ceph-mon[29221]: Updating smithi117:/var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T19:21:33.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:33 smithi022 ceph-mon[29221]: Updating smithi022:/etc/ceph/ceph.client.admin.keyring 2024-02-16T19:21:33.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:33 smithi022 ceph-mon[29221]: Updating smithi117:/etc/ceph/ceph.client.admin.keyring 2024-02-16T19:21:33.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:33 smithi022 ceph-mon[29221]: Updating smithi022:/var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-16T19:21:33.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:33 smithi022 ceph-mon[29221]: Updating smithi117:/var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-16T19:21:33.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:33 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:21:33.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:33 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:21:33.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:33 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:21:33.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:33 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:21:33.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:33 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:21:33.467 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:33 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi117", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-02-16T19:21:33.467 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:33 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi117", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-02-16T19:21:33.467 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:33 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:21:34.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:34 smithi022 ceph-mon[29221]: Deploying daemon ceph-exporter.smithi117 on smithi117 2024-02-16T19:21:34.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:34 smithi022 ceph-mon[29221]: from='client.? 172.21.15.117:0/3643104627' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-16T19:21:34.788 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-16T19:21:34.789 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-16T19:21:34.992 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T19:21:36.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:36.593+0000 7fbb4192e700 1 -- 172.21.15.117:0/2115085728 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb3c0fff90 msgr2=0x7fbb3c100360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:36.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:36.593+0000 7fbb4192e700 1 --2- 172.21.15.117:0/2115085728 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb3c0fff90 0x7fbb3c100360 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c009910 tx=0x7fbb2c009c20 comp rx=0 tx=0).stop 2024-02-16T19:21:36.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:36.593+0000 7fbb4192e700 1 -- 172.21.15.117:0/2115085728 shutdown_connections 2024-02-16T19:21:36.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:36.593+0000 7fbb4192e700 1 --2- 172.21.15.117:0/2115085728 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb3c0fff90 0x7fbb3c100360 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:36.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:36.593+0000 7fbb4192e700 1 -- 172.21.15.117:0/2115085728 >> 172.21.15.117:0/2115085728 conn(0x7fbb3c075550 msgr2=0x7fbb3c075950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:21:36.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:36.593+0000 7fbb4192e700 1 -- 172.21.15.117:0/2115085728 shutdown_connections 2024-02-16T19:21:36.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:36.593+0000 7fbb4192e700 1 -- 172.21.15.117:0/2115085728 wait complete. 2024-02-16T19:21:36.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:36.593+0000 7fbb4192e700 1 Processor -- start 2024-02-16T19:21:36.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:36.593+0000 7fbb4192e700 1 -- start start 2024-02-16T19:21:36.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:36.594+0000 7fbb4192e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb3c0fff90 0x7fbb3c1953a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:36.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:36.594+0000 7fbb4192e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3c1958e0 con 0x7fbb3c0fff90 2024-02-16T19:21:36.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:36.594+0000 7fbb3affd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb3c0fff90 0x7fbb3c1953a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:36.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:36.594+0000 7fbb3affd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb3c0fff90 0x7fbb3c1953a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.117:52610/0 (socket says 172.21.15.117:52610) 2024-02-16T19:21:36.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:36.594+0000 7fbb3affd700 1 -- 172.21.15.117:0/3048359889 learned_addr learned my addr 172.21.15.117:0/3048359889 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-02-16T19:21:36.597 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:36.594+0000 7fbb3affd700 1 -- 172.21.15.117:0/3048359889 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb2c0095f0 con 0x7fbb3c0fff90 2024-02-16T19:21:36.597 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:36.594+0000 7fbb3affd700 1 --2- 172.21.15.117:0/3048359889 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb3c0fff90 0x7fbb3c1953a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c005b40 tx=0x7fbb2c004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:21:36.597 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:36.595+0000 7fbb4092c700 1 -- 172.21.15.117:0/3048359889 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbb2c01d070 con 0x7fbb3c0fff90 2024-02-16T19:21:36.597 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:36.595+0000 7fbb4192e700 1 -- 172.21.15.117:0/3048359889 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb3c195ae0 con 0x7fbb3c0fff90 2024-02-16T19:21:36.597 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:36.595+0000 7fbb4092c700 1 -- 172.21.15.117:0/3048359889 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fbb2c0044d0 con 0x7fbb3c0fff90 2024-02-16T19:21:36.597 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:36.595+0000 7fbb4092c700 1 -- 172.21.15.117:0/3048359889 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbb2c003930 con 0x7fbb3c0fff90 2024-02-16T19:21:36.597 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:36.595+0000 7fbb4192e700 1 -- 172.21.15.117:0/3048359889 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb3c195ea0 con 0x7fbb3c0fff90 2024-02-16T19:21:36.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:36.596+0000 7fbb4192e700 1 -- 172.21.15.117:0/3048359889 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb3c062380 con 0x7fbb3c0fff90 2024-02-16T19:21:36.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:36.596+0000 7fbb4092c700 1 -- 172.21.15.117:0/3048359889 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) v1 ==== 45197+0+0 (secure 0 0 0) 0x7fbb2c00b960 con 0x7fbb3c0fff90 2024-02-16T19:21:36.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:36.596+0000 7fbb4092c700 1 --2- 172.21.15.117:0/3048359889 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fbb28038580 0x7fbb2803aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:36.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:36.596+0000 7fbb4092c700 1 -- 172.21.15.117:0/3048359889 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fbb2c04c3f0 con 0x7fbb3c0fff90 2024-02-16T19:21:36.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:36.596+0000 7fbb327fc700 1 --2- 172.21.15.117:0/3048359889 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fbb28038580 0x7fbb2803aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:36.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:36.596+0000 7fbb327fc700 1 --2- 172.21.15.117:0/3048359889 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fbb28038580 0x7fbb2803aa30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbb24006fd0 tx=0x7fbb24006e40 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:21:36.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:36.601+0000 7fbb4092c700 1 -- 172.21.15.117:0/3048359889 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fbb2c02a540 con 0x7fbb3c0fff90 2024-02-16T19:21:36.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:36.883+0000 7fbb4192e700 1 -- 172.21.15.117:0/3048359889 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fbb3c199df0 con 0x7fbb3c0fff90 2024-02-16T19:21:36.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:36.883+0000 7fbb4092c700 1 -- 172.21.15.117:0/3048359889 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7fbb2c02a360 con 0x7fbb3c0fff90 2024-02-16T19:21:36.885 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:21:36.885 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":1,"fsid":"2a1346e6-cd00-11ee-95ba-87774f69a715","modified":"2024-02-16T19:19:07.447976Z","created":"2024-02-16T19:19:07.447976Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi022","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:3300","nonce":0},{"type":"v1","addr":"172.21.15.22:6789","nonce":0}]},"addr":"172.21.15.22:6789/0","public_addr":"172.21.15.22:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-16T19:21:36.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:36.885+0000 7fbb4192e700 1 -- 172.21.15.117:0/3048359889 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fbb28038580 msgr2=0x7fbb2803aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:36.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:36.885+0000 7fbb4192e700 1 --2- 172.21.15.117:0/3048359889 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fbb28038580 0x7fbb2803aa30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbb24006fd0 tx=0x7fbb24006e40 comp rx=0 tx=0).stop 2024-02-16T19:21:36.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:36.885+0000 7fbb4192e700 1 -- 172.21.15.117:0/3048359889 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb3c0fff90 msgr2=0x7fbb3c1953a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:36.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:36.886+0000 7fbb4192e700 1 --2- 172.21.15.117:0/3048359889 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb3c0fff90 0x7fbb3c1953a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c005b40 tx=0x7fbb2c004c80 comp rx=0 tx=0).stop 2024-02-16T19:21:36.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:36.886+0000 7fbb4192e700 1 -- 172.21.15.117:0/3048359889 shutdown_connections 2024-02-16T19:21:36.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:36.886+0000 7fbb4192e700 1 --2- 172.21.15.117:0/3048359889 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fbb28038580 0x7fbb2803aa30 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:36.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:36.886+0000 7fbb4192e700 1 --2- 172.21.15.117:0/3048359889 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb3c0fff90 0x7fbb3c1953a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:36.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:36.886+0000 7fbb4192e700 1 -- 172.21.15.117:0/3048359889 >> 172.21.15.117:0/3048359889 conn(0x7fbb3c075550 msgr2=0x7fbb3c0fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:21:36.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:36.886+0000 7fbb4192e700 1 -- 172.21.15.117:0/3048359889 shutdown_connections 2024-02-16T19:21:36.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:36.886+0000 7fbb4192e700 1 -- 172.21.15.117:0/3048359889 wait complete. 2024-02-16T19:21:36.887 INFO:teuthology.orchestra.run.smithi117.stderr:dumped monmap epoch 1 2024-02-16T19:21:38.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:37 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:21:38.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:37 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:21:38.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:37 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:21:38.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:37 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:21:38.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:37 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi117", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-16T19:21:38.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:37 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi117", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-02-16T19:21:38.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:37 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:21:38.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:37 smithi022 ceph-mon[29221]: Deploying daemon crash.smithi117 on smithi117 2024-02-16T19:21:38.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:37 smithi022 ceph-mon[29221]: from='client.? 172.21.15.117:0/3048359889' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-16T19:21:38.461 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-16T19:21:38.461 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-16T19:21:38.817 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T19:21:40.431 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:40.429+0000 7fc494d47700 1 -- 172.21.15.117:0/856676066 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc490100090 msgr2=0x7fc490100460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:40.432 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:40.429+0000 7fc494d47700 1 --2- 172.21.15.117:0/856676066 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc490100090 0x7fc490100460 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc48800b0a0 tx=0x7fc48800b3b0 comp rx=0 tx=0).stop 2024-02-16T19:21:40.432 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:40.429+0000 7fc494d47700 1 -- 172.21.15.117:0/856676066 shutdown_connections 2024-02-16T19:21:40.432 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:40.429+0000 7fc494d47700 1 --2- 172.21.15.117:0/856676066 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc490100090 0x7fc490100460 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:40.432 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:40.429+0000 7fc494d47700 1 -- 172.21.15.117:0/856676066 >> 172.21.15.117:0/856676066 conn(0x7fc4900fb970 msgr2=0x7fc4900fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:21:40.432 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:40.429+0000 7fc494d47700 1 -- 172.21.15.117:0/856676066 shutdown_connections 2024-02-16T19:21:40.432 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:40.429+0000 7fc494d47700 1 -- 172.21.15.117:0/856676066 wait complete. 2024-02-16T19:21:40.432 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:40.430+0000 7fc494d47700 1 Processor -- start 2024-02-16T19:21:40.432 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:40.430+0000 7fc494d47700 1 -- start start 2024-02-16T19:21:40.432 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:40.430+0000 7fc494d47700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc490100090 0x7fc4901953a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:40.432 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:40.430+0000 7fc494d47700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4901958e0 con 0x7fc490100090 2024-02-16T19:21:40.432 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:40.430+0000 7fc48e59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc490100090 0x7fc4901953a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:40.432 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:40.430+0000 7fc48e59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc490100090 0x7fc4901953a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.117:52642/0 (socket says 172.21.15.117:52642) 2024-02-16T19:21:40.432 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:40.430+0000 7fc48e59c700 1 -- 172.21.15.117:0/1089156272 learned_addr learned my addr 172.21.15.117:0/1089156272 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-02-16T19:21:40.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:40.431+0000 7fc48e59c700 1 -- 172.21.15.117:0/1089156272 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc488009d00 con 0x7fc490100090 2024-02-16T19:21:40.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:40.431+0000 7fc48e59c700 1 --2- 172.21.15.117:0/1089156272 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc490100090 0x7fc4901953a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc4880062a0 tx=0x7fc488003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:21:40.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:40.431+0000 7fc47f7fe700 1 -- 172.21.15.117:0/1089156272 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc48800e040 con 0x7fc490100090 2024-02-16T19:21:40.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:40.431+0000 7fc494d47700 1 -- 172.21.15.117:0/1089156272 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc490195ae0 con 0x7fc490100090 2024-02-16T19:21:40.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:40.431+0000 7fc47f7fe700 1 -- 172.21.15.117:0/1089156272 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc48800bdd0 con 0x7fc490100090 2024-02-16T19:21:40.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:40.431+0000 7fc47f7fe700 1 -- 172.21.15.117:0/1089156272 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc48801d940 con 0x7fc490100090 2024-02-16T19:21:40.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:40.431+0000 7fc494d47700 1 -- 172.21.15.117:0/1089156272 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc490195f60 con 0x7fc490100090 2024-02-16T19:21:40.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:40.432+0000 7fc494d47700 1 -- 172.21.15.117:0/1089156272 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc490062380 con 0x7fc490100090 2024-02-16T19:21:40.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:40.432+0000 7fc47f7fe700 1 -- 172.21.15.117:0/1089156272 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) v1 ==== 45197+0+0 (secure 0 0 0) 0x7fc488004250 con 0x7fc490100090 2024-02-16T19:21:40.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:40.432+0000 7fc47f7fe700 1 --2- 172.21.15.117:0/1089156272 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fc478038580 0x7fc47803aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:40.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:40.432+0000 7fc47f7fe700 1 -- 172.21.15.117:0/1089156272 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fc48804c4b0 con 0x7fc490100090 2024-02-16T19:21:40.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:40.432+0000 7fc48dd9b700 1 --2- 172.21.15.117:0/1089156272 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fc478038580 0x7fc47803aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:40.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:40.433+0000 7fc48dd9b700 1 --2- 172.21.15.117:0/1089156272 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fc478038580 0x7fc47803aa30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc484006fd0 tx=0x7fc484006e40 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:21:40.438 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:40.438+0000 7fc47f7fe700 1 -- 172.21.15.117:0/1089156272 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fc488011570 con 0x7fc490100090 2024-02-16T19:21:40.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:40.713+0000 7fc494d47700 1 -- 172.21.15.117:0/1089156272 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fc490107e00 con 0x7fc490100090 2024-02-16T19:21:40.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:40.714+0000 7fc47f7fe700 1 -- 172.21.15.117:0/1089156272 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7fc488011390 con 0x7fc490100090 2024-02-16T19:21:40.715 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:21:40.715 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":1,"fsid":"2a1346e6-cd00-11ee-95ba-87774f69a715","modified":"2024-02-16T19:19:07.447976Z","created":"2024-02-16T19:19:07.447976Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi022","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:3300","nonce":0},{"type":"v1","addr":"172.21.15.22:6789","nonce":0}]},"addr":"172.21.15.22:6789/0","public_addr":"172.21.15.22:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-16T19:21:40.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:40.716+0000 7fc494d47700 1 -- 172.21.15.117:0/1089156272 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fc478038580 msgr2=0x7fc47803aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:40.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:40.716+0000 7fc494d47700 1 --2- 172.21.15.117:0/1089156272 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fc478038580 0x7fc47803aa30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc484006fd0 tx=0x7fc484006e40 comp rx=0 tx=0).stop 2024-02-16T19:21:40.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:40.716+0000 7fc494d47700 1 -- 172.21.15.117:0/1089156272 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc490100090 msgr2=0x7fc4901953a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:40.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:40.716+0000 7fc494d47700 1 --2- 172.21.15.117:0/1089156272 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc490100090 0x7fc4901953a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc4880062a0 tx=0x7fc488003ce0 comp rx=0 tx=0).stop 2024-02-16T19:21:40.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:40.716+0000 7fc494d47700 1 -- 172.21.15.117:0/1089156272 shutdown_connections 2024-02-16T19:21:40.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:40.716+0000 7fc494d47700 1 --2- 172.21.15.117:0/1089156272 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fc478038580 0x7fc47803aa30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:40.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:40.716+0000 7fc494d47700 1 --2- 172.21.15.117:0/1089156272 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc490100090 0x7fc4901953a0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:40.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:40.716+0000 7fc494d47700 1 -- 172.21.15.117:0/1089156272 >> 172.21.15.117:0/1089156272 conn(0x7fc4900fb970 msgr2=0x7fc490104270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:21:40.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:40.716+0000 7fc494d47700 1 -- 172.21.15.117:0/1089156272 shutdown_connections 2024-02-16T19:21:40.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:40.716+0000 7fc494d47700 1 -- 172.21.15.117:0/1089156272 wait complete. 2024-02-16T19:21:40.718 INFO:teuthology.orchestra.run.smithi117.stderr:dumped monmap epoch 1 2024-02-16T19:21:41.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:41 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:21:41.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:41 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:21:41.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:41 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:21:41.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:41 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:21:41.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:41 smithi022 ceph-mon[29221]: Deploying daemon node-exporter.smithi117 on smithi117 2024-02-16T19:21:41.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:41 smithi022 ceph-mon[29221]: from='client.? 172.21.15.117:0/1089156272' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-16T19:21:42.306 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-16T19:21:42.306 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-16T19:21:42.508 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T19:21:43.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:42 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:21:43.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:42 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:21:45.361 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:45.359+0000 7f9def936700 1 -- 172.21.15.117:0/175702096 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9de80fff90 msgr2=0x7f9de8100360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:45.362 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:45.359+0000 7f9def936700 1 --2- 172.21.15.117:0/175702096 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9de80fff90 0x7f9de8100360 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9de4009910 tx=0x7f9de4009c20 comp rx=0 tx=0).stop 2024-02-16T19:21:45.362 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:45.359+0000 7f9def936700 1 -- 172.21.15.117:0/175702096 shutdown_connections 2024-02-16T19:21:45.362 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:45.359+0000 7f9def936700 1 --2- 172.21.15.117:0/175702096 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9de80fff90 0x7f9de8100360 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:45.362 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:45.359+0000 7f9def936700 1 -- 172.21.15.117:0/175702096 >> 172.21.15.117:0/175702096 conn(0x7f9de8075550 msgr2=0x7f9de8075950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:21:45.362 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:45.359+0000 7f9def936700 1 -- 172.21.15.117:0/175702096 shutdown_connections 2024-02-16T19:21:45.362 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:45.359+0000 7f9def936700 1 -- 172.21.15.117:0/175702096 wait complete. 2024-02-16T19:21:45.362 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:45.359+0000 7f9def936700 1 Processor -- start 2024-02-16T19:21:45.362 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:45.360+0000 7f9def936700 1 -- start start 2024-02-16T19:21:45.362 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:45.360+0000 7f9def936700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9de80fff90 0x7f9de81961c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:45.362 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:45.360+0000 7f9def936700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de8196700 con 0x7f9de80fff90 2024-02-16T19:21:45.362 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:45.360+0000 7f9ded6d2700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9de80fff90 0x7f9de81961c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:45.362 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:45.360+0000 7f9ded6d2700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9de80fff90 0x7f9de81961c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.117:52676/0 (socket says 172.21.15.117:52676) 2024-02-16T19:21:45.362 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:45.360+0000 7f9ded6d2700 1 -- 172.21.15.117:0/2223834154 learned_addr learned my addr 172.21.15.117:0/2223834154 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-02-16T19:21:45.362 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:45.360+0000 7f9ded6d2700 1 -- 172.21.15.117:0/2223834154 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9de40095f0 con 0x7f9de80fff90 2024-02-16T19:21:45.362 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:45.360+0000 7f9ded6d2700 1 --2- 172.21.15.117:0/2223834154 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9de80fff90 0x7f9de81961c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9de4005b40 tx=0x7f9de4004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:21:45.363 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:45.361+0000 7f9dde7fc700 1 -- 172.21.15.117:0/2223834154 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9de401d070 con 0x7f9de80fff90 2024-02-16T19:21:45.363 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:45.361+0000 7f9def936700 1 -- 172.21.15.117:0/2223834154 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9de8196900 con 0x7f9de80fff90 2024-02-16T19:21:45.363 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:45.361+0000 7f9dde7fc700 1 -- 172.21.15.117:0/2223834154 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9de40044d0 con 0x7f9de80fff90 2024-02-16T19:21:45.363 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:45.361+0000 7f9dde7fc700 1 -- 172.21.15.117:0/2223834154 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9de4003930 con 0x7f9de80fff90 2024-02-16T19:21:45.363 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:45.361+0000 7f9def936700 1 -- 172.21.15.117:0/2223834154 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9de8196cc0 con 0x7f9de80fff90 2024-02-16T19:21:45.364 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:45.362+0000 7f9def936700 1 -- 172.21.15.117:0/2223834154 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9de8106700 con 0x7f9de80fff90 2024-02-16T19:21:45.364 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:45.362+0000 7f9dde7fc700 1 -- 172.21.15.117:0/2223834154 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) v1 ==== 45197+0+0 (secure 0 0 0) 0x7f9de400b960 con 0x7f9de80fff90 2024-02-16T19:21:45.364 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:45.362+0000 7f9dde7fc700 1 --2- 172.21.15.117:0/2223834154 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f9dd4038580 0x7f9dd403aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:45.364 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:45.362+0000 7f9dde7fc700 1 -- 172.21.15.117:0/2223834154 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f9de404c3f0 con 0x7f9de80fff90 2024-02-16T19:21:45.364 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:45.362+0000 7f9deced1700 1 --2- 172.21.15.117:0/2223834154 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f9dd4038580 0x7f9dd403aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:45.364 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:45.363+0000 7f9deced1700 1 --2- 172.21.15.117:0/2223834154 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f9dd4038580 0x7f9dd403aa30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9dd8006fd0 tx=0x7f9dd8006e40 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:21:45.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:45.367+0000 7f9dde7fc700 1 -- 172.21.15.117:0/2223834154 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f9de402a540 con 0x7f9de80fff90 2024-02-16T19:21:45.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:45.651+0000 7f9def936700 1 -- 172.21.15.117:0/2223834154 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f9de80ff470 con 0x7f9de80fff90 2024-02-16T19:21:45.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:45.652+0000 7f9dde7fc700 1 -- 172.21.15.117:0/2223834154 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f9de402a360 con 0x7f9de80fff90 2024-02-16T19:21:45.653 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:21:45.653 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":1,"fsid":"2a1346e6-cd00-11ee-95ba-87774f69a715","modified":"2024-02-16T19:19:07.447976Z","created":"2024-02-16T19:19:07.447976Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi022","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:3300","nonce":0},{"type":"v1","addr":"172.21.15.22:6789","nonce":0}]},"addr":"172.21.15.22:6789/0","public_addr":"172.21.15.22:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-16T19:21:45.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:45.654+0000 7f9def936700 1 -- 172.21.15.117:0/2223834154 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f9dd4038580 msgr2=0x7f9dd403aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:45.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:45.654+0000 7f9def936700 1 --2- 172.21.15.117:0/2223834154 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f9dd4038580 0x7f9dd403aa30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9dd8006fd0 tx=0x7f9dd8006e40 comp rx=0 tx=0).stop 2024-02-16T19:21:45.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:45.654+0000 7f9def936700 1 -- 172.21.15.117:0/2223834154 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9de80fff90 msgr2=0x7f9de81961c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:45.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:45.654+0000 7f9def936700 1 --2- 172.21.15.117:0/2223834154 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9de80fff90 0x7f9de81961c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9de4005b40 tx=0x7f9de4004c80 comp rx=0 tx=0).stop 2024-02-16T19:21:45.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:45.654+0000 7f9def936700 1 -- 172.21.15.117:0/2223834154 shutdown_connections 2024-02-16T19:21:45.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:45.654+0000 7f9def936700 1 --2- 172.21.15.117:0/2223834154 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f9dd4038580 0x7f9dd403aa30 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:45.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:45.654+0000 7f9def936700 1 --2- 172.21.15.117:0/2223834154 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9de80fff90 0x7f9de81961c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:45.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:45.654+0000 7f9def936700 1 -- 172.21.15.117:0/2223834154 >> 172.21.15.117:0/2223834154 conn(0x7f9de8075550 msgr2=0x7f9de8103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:21:45.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:45.654+0000 7f9def936700 1 -- 172.21.15.117:0/2223834154 shutdown_connections 2024-02-16T19:21:45.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:45.654+0000 7f9def936700 1 -- 172.21.15.117:0/2223834154 wait complete. 2024-02-16T19:21:45.656 INFO:teuthology.orchestra.run.smithi117.stderr:dumped monmap epoch 1 2024-02-16T19:21:46.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:46 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:21:46.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:46 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:21:46.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:46 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:21:46.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:46 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:21:46.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:46 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi117.cohbtv", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-16T19:21:46.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:46 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi117.cohbtv", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-16T19:21:46.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:46 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-16T19:21:46.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:46 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:21:46.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:46 smithi022 ceph-mon[29221]: Deploying daemon mgr.smithi117.cohbtv on smithi117 2024-02-16T19:21:46.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:46 smithi022 ceph-mon[29221]: from='client.? 172.21.15.117:0/2223834154' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-16T19:21:47.793 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-16T19:21:47.794 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-16T19:21:48.185 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T19:21:48.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:47 smithi022 ceph-mon[29221]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:21:48.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:47 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:21:49.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:49 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:21:49.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:49 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:21:49.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:49 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:21:49.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:49 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:21:49.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:49 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-16T19:21:49.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:49 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:21:49.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:49 smithi022 ceph-mon[29221]: Deploying daemon mon.smithi117 on smithi117 2024-02-16T19:21:49.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:49 smithi022 ceph-mon[29221]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:21:49.845 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:49.844+0000 7fbfa6096700 1 -- 172.21.15.117:0/2251499057 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbfa00fef90 msgr2=0x7fbfa00ff360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:49.845 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:49.844+0000 7fbfa6096700 1 --2- 172.21.15.117:0/2251499057 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbfa00fef90 0x7fbfa00ff360 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbf90009910 tx=0x7fbf90009c20 comp rx=0 tx=0).stop 2024-02-16T19:21:49.846 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:49.844+0000 7fbfa6096700 1 -- 172.21.15.117:0/2251499057 shutdown_connections 2024-02-16T19:21:49.846 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:49.844+0000 7fbfa6096700 1 --2- 172.21.15.117:0/2251499057 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbfa00fef90 0x7fbfa00ff360 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:49.846 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:49.844+0000 7fbfa6096700 1 -- 172.21.15.117:0/2251499057 >> 172.21.15.117:0/2251499057 conn(0x7fbfa00fa850 msgr2=0x7fbfa00fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:21:49.846 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:49.844+0000 7fbfa6096700 1 -- 172.21.15.117:0/2251499057 shutdown_connections 2024-02-16T19:21:49.846 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:49.844+0000 7fbfa6096700 1 -- 172.21.15.117:0/2251499057 wait complete. 2024-02-16T19:21:49.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:49.845+0000 7fbfa6096700 1 Processor -- start 2024-02-16T19:21:49.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:49.845+0000 7fbfa6096700 1 -- start start 2024-02-16T19:21:49.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:49.846+0000 7fbfa6096700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbfa00fef90 0x7fbfa0194210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:49.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:49.846+0000 7fbfa6096700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa0194750 con 0x7fbfa00fef90 2024-02-16T19:21:49.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:49.846+0000 7fbf9f7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbfa00fef90 0x7fbfa0194210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:49.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:49.846+0000 7fbf9f7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbfa00fef90 0x7fbfa0194210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.117:54634/0 (socket says 172.21.15.117:54634) 2024-02-16T19:21:49.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:49.846+0000 7fbf9f7fe700 1 -- 172.21.15.117:0/3527210886 learned_addr learned my addr 172.21.15.117:0/3527210886 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-02-16T19:21:49.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:49.846+0000 7fbf9f7fe700 1 -- 172.21.15.117:0/3527210886 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf900095f0 con 0x7fbfa00fef90 2024-02-16T19:21:49.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:49.847+0000 7fbf9f7fe700 1 --2- 172.21.15.117:0/3527210886 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbfa00fef90 0x7fbfa0194210 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fbf900092e0 tx=0x7fbf90004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:21:49.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:49.847+0000 7fbf9cff9700 1 -- 172.21.15.117:0/3527210886 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbf9001d070 con 0x7fbfa00fef90 2024-02-16T19:21:49.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:49.847+0000 7fbfa6096700 1 -- 172.21.15.117:0/3527210886 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfa0194950 con 0x7fbfa00fef90 2024-02-16T19:21:49.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:49.847+0000 7fbf9cff9700 1 -- 172.21.15.117:0/3527210886 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fbf9000be10 con 0x7fbfa00fef90 2024-02-16T19:21:49.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:49.847+0000 7fbf9cff9700 1 -- 172.21.15.117:0/3527210886 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbf90022e10 con 0x7fbfa00fef90 2024-02-16T19:21:49.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:49.847+0000 7fbfa6096700 1 -- 172.21.15.117:0/3527210886 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfa0194d70 con 0x7fbfa00fef90 2024-02-16T19:21:49.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:49.848+0000 7fbfa6096700 1 -- 172.21.15.117:0/3527210886 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfa0062380 con 0x7fbfa00fef90 2024-02-16T19:21:49.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:49.848+0000 7fbf9cff9700 1 -- 172.21.15.117:0/3527210886 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) v1 ==== 45197+0+0 (secure 0 0 0) 0x7fbf9000f460 con 0x7fbfa00fef90 2024-02-16T19:21:49.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:49.848+0000 7fbf9cff9700 1 --2- 172.21.15.117:0/3527210886 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fbf88038530 0x7fbf8803a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:49.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:49.848+0000 7fbf9cff9700 1 -- 172.21.15.117:0/3527210886 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fbf9004c650 con 0x7fbfa00fef90 2024-02-16T19:21:49.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:49.848+0000 7fbf9effd700 1 --2- 172.21.15.117:0/3527210886 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fbf88038530 0x7fbf8803a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:49.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:49.849+0000 7fbf9effd700 1 --2- 172.21.15.117:0/3527210886 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fbf88038530 0x7fbf8803a9e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbf94006fd0 tx=0x7fbf94006e40 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:21:49.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:49.853+0000 7fbf9cff9700 1 -- 172.21.15.117:0/3527210886 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fbf9000fe60 con 0x7fbfa00fef90 2024-02-16T19:21:50.144 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:50.142+0000 7fbfa6096700 1 -- 172.21.15.117:0/3527210886 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fbfa0106d30 con 0x7fbfa00fef90 2024-02-16T19:21:50.144 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:50.143+0000 7fbf9cff9700 1 -- 172.21.15.117:0/3527210886 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7fbf90027030 con 0x7fbfa00fef90 2024-02-16T19:21:50.144 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:21:50.144 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":1,"fsid":"2a1346e6-cd00-11ee-95ba-87774f69a715","modified":"2024-02-16T19:19:07.447976Z","created":"2024-02-16T19:19:07.447976Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi022","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:3300","nonce":0},{"type":"v1","addr":"172.21.15.22:6789","nonce":0}]},"addr":"172.21.15.22:6789/0","public_addr":"172.21.15.22:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-16T19:21:50.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:50.145+0000 7fbfa6096700 1 -- 172.21.15.117:0/3527210886 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fbf88038530 msgr2=0x7fbf8803a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:50.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:50.145+0000 7fbfa6096700 1 --2- 172.21.15.117:0/3527210886 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fbf88038530 0x7fbf8803a9e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbf94006fd0 tx=0x7fbf94006e40 comp rx=0 tx=0).stop 2024-02-16T19:21:50.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:50.145+0000 7fbfa6096700 1 -- 172.21.15.117:0/3527210886 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbfa00fef90 msgr2=0x7fbfa0194210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:50.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:50.145+0000 7fbfa6096700 1 --2- 172.21.15.117:0/3527210886 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbfa00fef90 0x7fbfa0194210 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fbf900092e0 tx=0x7fbf90004c80 comp rx=0 tx=0).stop 2024-02-16T19:21:50.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:50.145+0000 7fbfa6096700 1 -- 172.21.15.117:0/3527210886 shutdown_connections 2024-02-16T19:21:50.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:50.145+0000 7fbfa6096700 1 --2- 172.21.15.117:0/3527210886 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fbf88038530 0x7fbf8803a9e0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:50.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:50.145+0000 7fbfa6096700 1 --2- 172.21.15.117:0/3527210886 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbfa00fef90 0x7fbfa0194210 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:50.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:50.145+0000 7fbfa6096700 1 -- 172.21.15.117:0/3527210886 >> 172.21.15.117:0/3527210886 conn(0x7fbfa00fa850 msgr2=0x7fbfa0103170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:21:50.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:50.145+0000 7fbfa6096700 1 -- 172.21.15.117:0/3527210886 shutdown_connections 2024-02-16T19:21:50.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:21:50.145+0000 7fbfa6096700 1 -- 172.21.15.117:0/3527210886 wait complete. 2024-02-16T19:21:50.147 INFO:teuthology.orchestra.run.smithi117.stderr:dumped monmap epoch 1 2024-02-16T19:21:50.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:50 smithi022 ceph-mon[29221]: from='client.? 172.21.15.117:0/3527210886' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-16T19:21:51.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:51 smithi022 ceph-mon[29221]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:21:52.344 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-16T19:21:52.345 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-16T19:21:52.592 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi117/config 2024-02-16T19:21:53.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:52 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:21:53.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:53 smithi117 podman[35276]: 2024-02-16 19:21:53.820788607 +0000 UTC m=+0.145948356 container create aff3e21a73a066313217f8602d22198de2ba1116dd786752ae3316405b220af0 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-mon-smithi117, ceph=True, CEPH_POINT_RELEASE=-18.2.0, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.29.1, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20231212) 2024-02-16T19:21:54.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:53 smithi022 ceph-mon[29221]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:21:54.249 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:53 smithi117 podman[35276]: 2024-02-16 19:21:53.989782644 +0000 UTC m=+0.314942396 container init aff3e21a73a066313217f8602d22198de2ba1116dd786752ae3316405b220af0 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-mon-smithi117, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.build-date=20231212, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.29.1, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-02-16T19:21:54.249 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:53 smithi117 podman[35276]: 2024-02-16 19:21:53.992775828 +0000 UTC m=+0.317935577 container start aff3e21a73a066313217f8602d22198de2ba1116dd786752ae3316405b220af0 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-mon-smithi117, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=HEAD, ceph=True, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, io.buildah.version=1.29.1, GIT_REPO=https://github.com/ceph/ceph-container.git, CEPH_POINT_RELEASE=-18.2.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20231212) 2024-02-16T19:21:54.249 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: set uid:gid to 167:167 (ceph:ceph) 2024-02-16T19:21:54.249 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable), process ceph-mon, pid 2 2024-02-16T19:21:54.250 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: pidfile_write: ignore empty --pid-file 2024-02-16T19:21:54.250 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: load: jerasure load: lrc 2024-02-16T19:21:54.250 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: RocksDB version: 7.9.2 2024-02-16T19:21:54.250 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Git sha 0 2024-02-16T19:21:54.250 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Compile date 2023-08-03 19:21:13 2024-02-16T19:21:54.250 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: DB SUMMARY 2024-02-16T19:21:54.250 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: DB Session ID: PW3S6WUKD5A4NNZ6MI0N 2024-02-16T19:21:54.250 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: CURRENT file: CURRENT 2024-02-16T19:21:54.250 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: IDENTITY file: IDENTITY 2024-02-16T19:21:54.250 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-02-16T19:21:54.250 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi117/store.db dir, Total Num: 0, files: 2024-02-16T19:21:54.250 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi117/store.db: 000004.log size: 511 ; 2024-02-16T19:21:54.250 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.error_if_exists: 0 2024-02-16T19:21:54.250 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.create_if_missing: 0 2024-02-16T19:21:54.251 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.paranoid_checks: 1 2024-02-16T19:21:54.251 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.flush_verify_memtable_count: 1 2024-02-16T19:21:54.251 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-16T19:21:54.251 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-02-16T19:21:54.251 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.env: 0x55ab028bc720 2024-02-16T19:21:54.251 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.fs: PosixFileSystem 2024-02-16T19:21:54.251 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.info_log: 0x55ab041332c0 2024-02-16T19:21:54.251 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.max_file_opening_threads: 16 2024-02-16T19:21:54.251 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.statistics: (nil) 2024-02-16T19:21:54.253 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.use_fsync: 0 2024-02-16T19:21:54.253 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.max_log_file_size: 0 2024-02-16T19:21:54.253 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-16T19:21:54.253 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-16T19:21:54.253 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.keep_log_file_num: 1000 2024-02-16T19:21:54.253 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.recycle_log_file_num: 0 2024-02-16T19:21:54.254 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.allow_fallocate: 1 2024-02-16T19:21:54.254 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.allow_mmap_reads: 0 2024-02-16T19:21:54.254 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.allow_mmap_writes: 0 2024-02-16T19:21:54.254 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.use_direct_reads: 0 2024-02-16T19:21:54.254 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-16T19:21:54.254 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.create_missing_column_families: 0 2024-02-16T19:21:54.254 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.db_log_dir: 2024-02-16T19:21:54.254 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.wal_dir: 2024-02-16T19:21:54.254 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-16T19:21:54.254 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-16T19:21:54.254 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-16T19:21:54.254 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-16T19:21:54.254 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-16T19:21:54.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-16T19:21:54.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.advise_random_on_open: 1 2024-02-16T19:21:54.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.db_write_buffer_size: 0 2024-02-16T19:21:54.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.write_buffer_manager: 0x55ab033a43c0 2024-02-16T19:21:54.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-16T19:21:54.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-16T19:21:54.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-16T19:21:54.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.rate_limiter: (nil) 2024-02-16T19:21:54.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-16T19:21:54.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.wal_recovery_mode: 2 2024-02-16T19:21:54.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.enable_thread_tracking: 0 2024-02-16T19:21:54.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.enable_pipelined_write: 0 2024-02-16T19:21:54.255 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.unordered_write: 0 2024-02-16T19:21:54.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-16T19:21:54.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-16T19:21:54.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-16T19:21:54.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-16T19:21:54.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.row_cache: None 2024-02-16T19:21:54.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.wal_filter: None 2024-02-16T19:21:54.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-16T19:21:54.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.allow_ingest_behind: 0 2024-02-16T19:21:54.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.two_write_queues: 0 2024-02-16T19:21:54.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.manual_wal_flush: 0 2024-02-16T19:21:54.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.wal_compression: 0 2024-02-16T19:21:54.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.atomic_flush: 0 2024-02-16T19:21:54.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-16T19:21:54.256 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-16T19:21:54.257 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-16T19:21:54.257 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.log_readahead_size: 0 2024-02-16T19:21:54.257 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-16T19:21:54.257 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.best_efforts_recovery: 0 2024-02-16T19:21:54.257 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-16T19:21:54.257 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-16T19:21:54.257 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.allow_data_in_errors: 0 2024-02-16T19:21:54.257 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.db_host_id: __hostname__ 2024-02-16T19:21:54.257 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.enforce_single_del_contracts: true 2024-02-16T19:21:54.257 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.max_background_jobs: 2 2024-02-16T19:21:54.257 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.max_background_compactions: -1 2024-02-16T19:21:54.257 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.max_subcompactions: 1 2024-02-16T19:21:54.257 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-16T19:21:54.257 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-16T19:21:54.258 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-16T19:21:54.258 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.max_total_wal_size: 0 2024-02-16T19:21:54.258 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-16T19:21:54.258 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-16T19:21:54.258 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-16T19:21:54.258 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-16T19:21:54.258 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.max_open_files: -1 2024-02-16T19:21:54.258 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.bytes_per_sync: 0 2024-02-16T19:21:54.258 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-16T19:21:54.258 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-16T19:21:54.258 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.compaction_readahead_size: 0 2024-02-16T19:21:54.258 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.max_background_flushes: -1 2024-02-16T19:21:54.258 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Compression algorithms supported: 2024-02-16T19:21:54.258 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-16T19:21:54.259 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: kZSTD supported: 0 2024-02-16T19:21:54.259 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: kXpressCompression supported: 0 2024-02-16T19:21:54.259 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: kLZ4HCCompression supported: 1 2024-02-16T19:21:54.259 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: kZlibCompression supported: 1 2024-02-16T19:21:54.259 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: kSnappyCompression supported: 1 2024-02-16T19:21:54.259 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: kLZ4Compression supported: 1 2024-02-16T19:21:54.259 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: kBZip2Compression supported: 0 2024-02-16T19:21:54.259 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-16T19:21:54.259 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: DMutex implementation: pthread_mutex_t 2024-02-16T19:21:54.259 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi117/store.db/MANIFEST-000005 2024-02-16T19:21:54.259 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-02-16T19:21:54.259 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-16T19:21:54.259 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.merge_operator: 2024-02-16T19:21:54.259 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.compaction_filter: None 2024-02-16T19:21:54.260 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.compaction_filter_factory: None 2024-02-16T19:21:54.260 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.sst_partitioner_factory: None 2024-02-16T19:21:54.260 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-16T19:21:54.260 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-16T19:21:54.260 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55ab041333c0) 2024-02-16T19:21:54.260 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: cache_index_and_filter_blocks: 1 2024-02-16T19:21:54.260 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-16T19:21:54.260 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-16T19:21:54.260 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: pin_top_level_index_and_filter: 1 2024-02-16T19:21:54.260 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: index_type: 0 2024-02-16T19:21:54.260 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: data_block_index_type: 0 2024-02-16T19:21:54.260 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: index_shortening: 1 2024-02-16T19:21:54.260 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-16T19:21:54.260 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: checksum: 4 2024-02-16T19:21:54.261 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: no_block_cache: 0 2024-02-16T19:21:54.261 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: block_cache: 0x55ab034451f0 2024-02-16T19:21:54.261 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: block_cache_name: BinnedLRUCache 2024-02-16T19:21:54.261 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: block_cache_options: 2024-02-16T19:21:54.261 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: capacity : 536870912 2024-02-16T19:21:54.261 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: num_shard_bits : 4 2024-02-16T19:21:54.261 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: strict_capacity_limit : 0 2024-02-16T19:21:54.261 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: high_pri_pool_ratio: 0.000 2024-02-16T19:21:54.261 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: block_cache_compressed: (nil) 2024-02-16T19:21:54.261 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: persistent_cache: (nil) 2024-02-16T19:21:54.261 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: block_size: 4096 2024-02-16T19:21:54.261 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: block_size_deviation: 10 2024-02-16T19:21:54.261 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: block_restart_interval: 16 2024-02-16T19:21:54.262 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: index_block_restart_interval: 1 2024-02-16T19:21:54.262 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: metadata_block_size: 4096 2024-02-16T19:21:54.262 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: partition_filters: 0 2024-02-16T19:21:54.262 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: use_delta_encoding: 1 2024-02-16T19:21:54.262 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: filter_policy: bloomfilter 2024-02-16T19:21:54.262 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: whole_key_filtering: 1 2024-02-16T19:21:54.262 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: verify_compression: 0 2024-02-16T19:21:54.262 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: read_amp_bytes_per_bit: 0 2024-02-16T19:21:54.262 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: format_version: 5 2024-02-16T19:21:54.262 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: enable_index_compression: 1 2024-02-16T19:21:54.262 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: block_align: 0 2024-02-16T19:21:54.262 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: max_auto_readahead_size: 262144 2024-02-16T19:21:54.262 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: prepopulate_block_cache: 0 2024-02-16T19:21:54.262 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: initial_auto_readahead_size: 8192 2024-02-16T19:21:54.262 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: num_file_reads_for_auto_readahead: 2 2024-02-16T19:21:54.262 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.write_buffer_size: 33554432 2024-02-16T19:21:54.263 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.max_write_buffer_number: 2 2024-02-16T19:21:54.263 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.compression: NoCompression 2024-02-16T19:21:54.263 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.bottommost_compression: Disabled 2024-02-16T19:21:54.263 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.prefix_extractor: nullptr 2024-02-16T19:21:54.263 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-16T19:21:54.263 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.num_levels: 7 2024-02-16T19:21:54.263 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-16T19:21:54.263 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-16T19:21:54.263 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-16T19:21:54.263 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-16T19:21:54.263 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-16T19:21:54.263 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-16T19:21:54.263 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-16T19:21:54.263 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-16T19:21:54.263 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-16T19:21:54.263 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-16T19:21:54.264 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-02-16T19:21:54.264 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-02-16T19:21:54.264 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-16T19:21:54.264 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.compression_opts.level: 32767 2024-02-16T19:21:54.264 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.compression_opts.strategy: 0 2024-02-16T19:21:54.264 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-16T19:21:54.264 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-16T19:21:54.264 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-02-16T19:21:54.264 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-16T19:21:54.264 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.compression_opts.enabled: false 2024-02-16T19:21:54.264 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-02-16T19:21:54.264 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-16T19:21:54.264 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-16T19:21:54.264 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-16T19:21:54.264 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.target_file_size_base: 67108864 2024-02-16T19:21:54.264 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-16T19:21:54.265 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-16T19:21:54.265 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-16T19:21:54.265 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-16T19:21:54.265 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-16T19:21:54.265 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-16T19:21:54.265 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-16T19:21:54.265 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-16T19:21:54.265 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-16T19:21:54.265 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-16T19:21:54.265 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-16T19:21:54.265 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-16T19:21:54.265 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-16T19:21:54.265 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-02-16T19:21:54.265 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.arena_block_size: 1048576 2024-02-16T19:21:54.265 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-16T19:21:54.266 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-16T19:21:54.266 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.disable_auto_compactions: 0 2024-02-16T19:21:54.266 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-16T19:21:54.266 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-16T19:21:54.266 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-16T19:21:54.266 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-16T19:21:54.266 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-16T19:21:54.266 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-16T19:21:54.266 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-16T19:21:54.266 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-16T19:21:54.266 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-16T19:21:54.266 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-16T19:21:54.266 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.table_properties_collectors: 2024-02-16T19:21:54.266 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.inplace_update_support: 0 2024-02-16T19:21:54.266 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-16T19:21:54.267 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-16T19:21:54.267 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-16T19:21:54.267 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-16T19:21:54.267 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.bloom_locality: 0 2024-02-16T19:21:54.267 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.max_successive_merges: 0 2024-02-16T19:21:54.267 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-16T19:21:54.267 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.paranoid_file_checks: 0 2024-02-16T19:21:54.267 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.force_consistency_checks: 1 2024-02-16T19:21:54.267 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.report_bg_io_stats: 0 2024-02-16T19:21:54.267 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.ttl: 2592000 2024-02-16T19:21:54.267 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-16T19:21:54.267 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-02-16T19:21:54.267 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-02-16T19:21:54.267 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.enable_blob_files: false 2024-02-16T19:21:54.267 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.min_blob_size: 0 2024-02-16T19:21:54.267 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.blob_file_size: 268435456 2024-02-16T19:21:54.268 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-16T19:21:54.268 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-16T19:21:54.268 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-16T19:21:54.268 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-02-16T19:21:54.268 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-02-16T19:21:54.268 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.blob_file_starting_level: 0 2024-02-16T19:21:54.268 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-02-16T19:21:54.268 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi117/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-02-16T19:21:54.268 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-02-16T19:21:54.268 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 056bb860-300c-4be7-a776-5fb6a0ad3677 2024-02-16T19:21:54.268 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708111314021125, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-02-16T19:21:54.268 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-02-16T19:21:54.268 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708111314021548, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1708111314, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "056bb860-300c-4be7-a776-5fb6a0ad3677", "db_session_id": "PW3S6WUKD5A4NNZ6MI0N", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-02-16T19:21:54.268 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708111314021628, "job": 1, "event": "recovery_finished"} 2024-02-16T19:21:54.269 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-02-16T19:21:54.269 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi117/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-16T19:21:54.269 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55ab034e2000 2024-02-16T19:21:54.269 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: DB pointer 0x55ab034ce000 2024-02-16T19:21:54.269 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-02-16T19:21:54.269 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-02-16T19:21:54.269 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: ** DB Stats ** 2024-02-16T19:21:54.269 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-16T19:21:54.269 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-16T19:21:54.269 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-16T19:21:54.269 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-16T19:21:54.269 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-16T19:21:54.269 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-16T19:21:54.269 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-16T19:21:54.269 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: 2024-02-16T19:21:54.270 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: ** Compaction Stats [default] ** 2024-02-16T19:21:54.270 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-16T19:21:54.270 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-02-16T19:21:54.270 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-16T19:21:54.270 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-16T19:21:54.270 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-16T19:21:54.270 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: 2024-02-16T19:21:54.270 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: ** Compaction Stats [default] ** 2024-02-16T19:21:54.270 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-16T19:21:54.270 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-16T19:21:54.270 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-16T19:21:54.270 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: 2024-02-16T19:21:54.270 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-02-16T19:21:54.270 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: 2024-02-16T19:21:54.270 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-16T19:21:54.271 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-02-16T19:21:54.271 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-16T19:21:54.271 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-16T19:21:54.271 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-16T19:21:54.271 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-16T19:21:54.271 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Cumulative compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-16T19:21:54.271 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Interval compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-16T19:21:54.271 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-16T19:21:54.271 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Block cache BinnedLRUCache@0x55ab034451f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-02-16T19:21:54.271 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-02-16T19:21:54.271 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: 2024-02-16T19:21:54.271 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-16T19:21:54.271 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: mon.smithi117 does not exist in monmap, will attempt to join an existing cluster 2024-02-16T19:21:54.271 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: using public_addr v2:172.21.15.117:0/0 -> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] 2024-02-16T19:21:54.271 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: starting mon.smithi117 rank -1 at public addrs [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] at bind addrs [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi117 fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 2024-02-16T19:21:54.272 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: mon.smithi117@-1(???) e0 preinit fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 2024-02-16T19:21:54.272 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: mon.smithi117@-1(synchronizing).mds e1 new map 2024-02-16T19:21:54.272 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: mon.smithi117@-1(synchronizing).mds e1 print_map 2024-02-16T19:21:54.272 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: e1 2024-02-16T19:21:54.272 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T19:21:54.272 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:21:54.272 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: legacy client fscid: -1 2024-02-16T19:21:54.272 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: 2024-02-16T19:21:54.272 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: No filesystems configured 2024-02-16T19:21:54.272 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: mon.smithi117@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-02-16T19:21:54.272 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: mon.smithi117@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-02-16T19:21:54.272 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: mon.smithi117@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-02-16T19:21:54.272 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: mon.smithi117@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-02-16T19:21:54.272 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: mon.smithi117@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-02-16T19:21:54.273 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: mon.smithi117@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-02-16T19:21:54.273 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: mon.smithi117@-1(synchronizing).osd e5 e5: 0 total, 0 up, 0 in 2024-02-16T19:21:54.273 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: mon.smithi117@-1(synchronizing).osd e5 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-16T19:21:54.273 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: mon.smithi117@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-02-16T19:21:54.273 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: mon.smithi117@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-02-16T19:21:54.273 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: mon.smithi117@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-02-16T19:21:54.273 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:21:54.273 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:21:54.273 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:21:54.273 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:21:54.273 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi117.cohbtv", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-16T19:21:54.273 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi117.cohbtv", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-16T19:21:54.273 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-16T19:21:54.273 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:21:54.273 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: Deploying daemon mgr.smithi117.cohbtv on smithi117 2024-02-16T19:21:54.274 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: from='client.? 172.21.15.117:0/2223834154' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-16T19:21:54.274 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:21:54.274 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:21:54.274 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:21:54.274 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:21:54.274 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:21:54.274 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:21:54.274 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-16T19:21:54.274 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:21:54.274 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: Deploying daemon mon.smithi117 on smithi117 2024-02-16T19:21:54.274 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:21:54.274 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: from='client.? 172.21.15.117:0/3527210886' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-16T19:21:54.274 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:21:54.274 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:21:54.275 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:21:54.275 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: mon.smithi117@-1(synchronizing).paxosservice(auth 1..8) refresh upgraded, format 0 -> 3 2024-02-16T19:21:54.275 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: mon.smithi117@-1(synchronizing) e1 handle_conf_change mon_allow_pool_delete,mon_cluster_log_to_file 2024-02-16T19:21:54.275 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: expand_channel_meta expand map: {default=false} 2024-02-16T19:21:54.275 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: expand_channel_meta from 'false' to 'false' 2024-02-16T19:21:54.275 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: expand_channel_meta expanded map: {default=false} 2024-02-16T19:21:54.275 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: expand_channel_meta expand map: {default=info} 2024-02-16T19:21:54.275 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: expand_channel_meta from 'info' to 'info' 2024-02-16T19:21:54.275 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: expand_channel_meta expanded map: {default=info} 2024-02-16T19:21:54.275 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: expand_channel_meta expand map: {default=daemon} 2024-02-16T19:21:54.275 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: expand_channel_meta from 'daemon' to 'daemon' 2024-02-16T19:21:54.275 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: expand_channel_meta expanded map: {default=daemon} 2024-02-16T19:21:54.275 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: expand_channel_meta expand map: {default=debug} 2024-02-16T19:21:54.275 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: expand_channel_meta from 'debug' to 'debug' 2024-02-16T19:21:54.275 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 ceph-mon[35309]: expand_channel_meta expanded map: {default=debug} 2024-02-16T19:21:54.276 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 bash[35276]: aff3e21a73a066313217f8602d22198de2ba1116dd786752ae3316405b220af0 2024-02-16T19:21:54.276 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:54 smithi117 systemd[1]: Started Ceph mon.smithi117 for 2a1346e6-cd00-11ee-95ba-87774f69a715. 2024-02-16T19:21:59.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:59 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mon metadata", "id": "smithi022"}]: dispatch 2024-02-16T19:21:59.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:59 smithi117 ceph-mon[35309]: mon.smithi022 calling monitor election 2024-02-16T19:21:59.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:59 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mon metadata", "id": "smithi117"}]: dispatch 2024-02-16T19:21:59.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:59 smithi117 ceph-mon[35309]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:21:59.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:59 smithi117 ceph-mon[35309]: from='mgr.? 172.21.15.117:0/3502066616' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi117.cohbtv/crt"}]: dispatch 2024-02-16T19:21:59.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:59 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mon metadata", "id": "smithi117"}]: dispatch 2024-02-16T19:21:59.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:59 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mon metadata", "id": "smithi117"}]: dispatch 2024-02-16T19:21:59.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:59 smithi117 ceph-mon[35309]: mon.smithi117 calling monitor election 2024-02-16T19:21:59.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:59 smithi117 ceph-mon[35309]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:21:59.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:59 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:21:59.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:59 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mon metadata", "id": "smithi117"}]: dispatch 2024-02-16T19:21:59.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:59 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mon metadata", "id": "smithi117"}]: dispatch 2024-02-16T19:21:59.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:59 smithi117 ceph-mon[35309]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:21:59.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:59 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mon metadata", "id": "smithi117"}]: dispatch 2024-02-16T19:21:59.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:59 smithi117 ceph-mon[35309]: mon.smithi022 is new leader, mons smithi022,smithi117 in quorum (ranks 0,1) 2024-02-16T19:21:59.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:59 smithi117 ceph-mon[35309]: monmap e2: 2 mons at {smithi022=[v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0],smithi117=[v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0]} removed_ranks: {} 2024-02-16T19:21:59.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:59 smithi117 ceph-mon[35309]: fsmap 2024-02-16T19:21:59.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:59 smithi117 ceph-mon[35309]: osdmap e5: 0 total, 0 up, 0 in 2024-02-16T19:21:59.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:59 smithi117 ceph-mon[35309]: mgrmap e17: smithi022.gmdcch(active, since 32s) 2024-02-16T19:21:59.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:59 smithi117 ceph-mon[35309]: Standby manager daemon smithi117.cohbtv started 2024-02-16T19:21:59.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:59 smithi117 ceph-mon[35309]: overall HEALTH_OK 2024-02-16T19:21:59.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:59 smithi117 ceph-mon[35309]: from='mgr.? 172.21.15.117:0/3502066616' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-16T19:21:59.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:59 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:21:59.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:59 smithi117 ceph-mon[35309]: from='mgr.? 172.21.15.117:0/3502066616' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi117.cohbtv/key"}]: dispatch 2024-02-16T19:21:59.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:59 smithi117 ceph-mon[35309]: from='mgr.? 172.21.15.117:0/3502066616' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-16T19:21:59.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:59 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:21:59.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:59 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:21:59.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:59 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:21:59.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:21:59 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:21:59.406 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:59 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mon metadata", "id": "smithi022"}]: dispatch 2024-02-16T19:21:59.407 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:59 smithi022 ceph-mon[29221]: mon.smithi022 calling monitor election 2024-02-16T19:21:59.407 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:59 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mon metadata", "id": "smithi117"}]: dispatch 2024-02-16T19:21:59.407 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:59 smithi022 ceph-mon[29221]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:21:59.407 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:59 smithi022 ceph-mon[29221]: from='mgr.? 172.21.15.117:0/3502066616' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi117.cohbtv/crt"}]: dispatch 2024-02-16T19:21:59.407 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:59 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mon metadata", "id": "smithi117"}]: dispatch 2024-02-16T19:21:59.407 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:59 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mon metadata", "id": "smithi117"}]: dispatch 2024-02-16T19:21:59.407 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:59 smithi022 ceph-mon[29221]: mon.smithi117 calling monitor election 2024-02-16T19:21:59.407 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:59 smithi022 ceph-mon[29221]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:21:59.407 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:59 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:21:59.407 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:59 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mon metadata", "id": "smithi117"}]: dispatch 2024-02-16T19:21:59.407 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:59 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mon metadata", "id": "smithi117"}]: dispatch 2024-02-16T19:21:59.407 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:59 smithi022 ceph-mon[29221]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:21:59.407 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:59 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mon metadata", "id": "smithi117"}]: dispatch 2024-02-16T19:21:59.408 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:59 smithi022 ceph-mon[29221]: mon.smithi022 is new leader, mons smithi022,smithi117 in quorum (ranks 0,1) 2024-02-16T19:21:59.408 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:59 smithi022 ceph-mon[29221]: monmap e2: 2 mons at {smithi022=[v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0],smithi117=[v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0]} removed_ranks: {} 2024-02-16T19:21:59.408 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:59 smithi022 ceph-mon[29221]: fsmap 2024-02-16T19:21:59.408 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:59 smithi022 ceph-mon[29221]: osdmap e5: 0 total, 0 up, 0 in 2024-02-16T19:21:59.408 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:59 smithi022 ceph-mon[29221]: mgrmap e17: smithi022.gmdcch(active, since 32s) 2024-02-16T19:21:59.408 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:59 smithi022 ceph-mon[29221]: Standby manager daemon smithi117.cohbtv started 2024-02-16T19:21:59.408 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:59 smithi022 ceph-mon[29221]: overall HEALTH_OK 2024-02-16T19:21:59.408 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:59 smithi022 ceph-mon[29221]: from='mgr.? 172.21.15.117:0/3502066616' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-16T19:21:59.408 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:59 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:21:59.408 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:59 smithi022 ceph-mon[29221]: from='mgr.? 172.21.15.117:0/3502066616' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi117.cohbtv/key"}]: dispatch 2024-02-16T19:21:59.408 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:59 smithi022 ceph-mon[29221]: from='mgr.? 172.21.15.117:0/3502066616' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-16T19:21:59.408 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:59 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:21:59.408 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:59 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:21:59.409 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:59 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:21:59.409 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:21:59 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:00.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:00 smithi022 ceph-mon[29221]: mgrmap e18: smithi022.gmdcch(active, since 32s), standbys: smithi117.cohbtv 2024-02-16T19:22:00.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:00 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mgr metadata", "who": "smithi117.cohbtv", "id": "smithi117.cohbtv"}]: dispatch 2024-02-16T19:22:00.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:00 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:22:00.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:00 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mon metadata", "id": "smithi117"}]: dispatch 2024-02-16T19:22:00.513 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:00 smithi117 ceph-mon[35309]: mgrmap e18: smithi022.gmdcch(active, since 32s), standbys: smithi117.cohbtv 2024-02-16T19:22:00.513 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:00 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mgr metadata", "who": "smithi117.cohbtv", "id": "smithi117.cohbtv"}]: dispatch 2024-02-16T19:22:00.513 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:00 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:22:00.513 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:00 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mon metadata", "id": "smithi117"}]: dispatch 2024-02-16T19:22:00.640 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:22:00.638+0000 7f943cd62700 1 -- 172.21.15.117:0/1699784917 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9420006230 msgr2=0x7f9420008690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:00.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:22:00.638+0000 7f943cd62700 1 --2- 172.21.15.117:0/1699784917 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9420006230 0x7f9420008690 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9438068f20 tx=0x7f942800fe90 comp rx=0 tx=0).stop 2024-02-16T19:22:00.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:22:00.638+0000 7f943cd62700 1 -- 172.21.15.117:0/1699784917 shutdown_connections 2024-02-16T19:22:00.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:22:00.638+0000 7f943cd62700 1 --2- 172.21.15.117:0/1699784917 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9420006230 0x7f9420008690 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:00.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:22:00.638+0000 7f943cd62700 1 -- 172.21.15.117:0/1699784917 >> 172.21.15.117:0/1699784917 conn(0x7f94380fa9a0 msgr2=0x7f94380fcdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:22:00.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:22:00.638+0000 7f943cd62700 1 -- 172.21.15.117:0/1699784917 shutdown_connections 2024-02-16T19:22:00.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:22:00.638+0000 7f943cd62700 1 -- 172.21.15.117:0/1699784917 wait complete. 2024-02-16T19:22:00.642 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:22:00.639+0000 7f943cd62700 1 Processor -- start 2024-02-16T19:22:00.643 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:22:00.639+0000 7f943cd62700 1 -- start start 2024-02-16T19:22:00.643 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:22:00.639+0000 7f943cd62700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9420006230 0x7f943819d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:00.643 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:22:00.639+0000 7f943cd62700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f943819dbf0 0x7f9438197730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:00.643 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:22:00.639+0000 7f943cd62700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f943819e200 con 0x7f9420006230 2024-02-16T19:22:00.643 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:22:00.639+0000 7f943cd62700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9438197c70 con 0x7f943819dbf0 2024-02-16T19:22:00.643 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:22:00.639+0000 7f9435d9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f943819dbf0 0x7f9438197730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:00.643 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:22:00.640+0000 7f943659c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9420006230 0x7f943819d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:00.643 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:22:00.640+0000 7f9435d9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f943819dbf0 0x7f9438197730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:48486/0 (socket says 172.21.15.117:48486) 2024-02-16T19:22:00.643 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:22:00.640+0000 7f943659c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9420006230 0x7f943819d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.117:53878/0 (socket says 172.21.15.117:53878) 2024-02-16T19:22:00.643 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:22:00.640+0000 7f9435d9b700 1 -- 172.21.15.117:0/3302750362 learned_addr learned my addr 172.21.15.117:0/3302750362 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-02-16T19:22:00.643 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:22:00.640+0000 7f9435d9b700 1 -- 172.21.15.117:0/3302750362 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f943819dbf0 msgr2=0x7f9438197730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-02-16T19:22:00.643 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:22:00.640+0000 7f9435d9b700 1 -- 172.21.15.117:0/3302750362 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f943819dbf0 msgr2=0x7f9438197730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-02-16T19:22:00.643 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:22:00.640+0000 7f9435d9b700 1 --2- 172.21.15.117:0/3302750362 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f943819dbf0 0x7f9438197730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-02-16T19:22:00.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:22:00.640+0000 7f9435d9b700 1 --2- 172.21.15.117:0/3302750362 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f943819dbf0 0x7f9438197730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-16T19:22:00.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:22:00.640+0000 7f943659c700 1 -- 172.21.15.117:0/3302750362 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f943819dbf0 msgr2=0x7f9438197730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-16T19:22:00.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:22:00.640+0000 7f943659c700 1 --2- 172.21.15.117:0/3302750362 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f943819dbf0 0x7f9438197730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:00.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:22:00.640+0000 7f943659c700 1 -- 172.21.15.117:0/3302750362 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f942800f8e0 con 0x7f9420006230 2024-02-16T19:22:00.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:22:00.640+0000 7f943659c700 1 --2- 172.21.15.117:0/3302750362 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9420006230 0x7f943819d6b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9438068f20 tx=0x7f942801d9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:22:00.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:22:00.640+0000 7f94277fe700 1 -- 172.21.15.117:0/3302750362 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9428005da0 con 0x7f9420006230 2024-02-16T19:22:00.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:22:00.640+0000 7f94277fe700 1 -- 172.21.15.117:0/3302750362 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9428005430 con 0x7f9420006230 2024-02-16T19:22:00.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:22:00.640+0000 7f94277fe700 1 -- 172.21.15.117:0/3302750362 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9428011850 con 0x7f9420006230 2024-02-16T19:22:00.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:22:00.640+0000 7f943cd62700 1 -- 172.21.15.117:0/3302750362 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9438197e90 con 0x7f9420006230 2024-02-16T19:22:00.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:22:00.640+0000 7f943cd62700 1 -- 172.21.15.117:0/3302750362 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94381983e0 con 0x7f9420006230 2024-02-16T19:22:00.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:22:00.641+0000 7f943cd62700 1 -- 172.21.15.117:0/3302750362 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9438062380 con 0x7f9420006230 2024-02-16T19:22:00.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:22:00.642+0000 7f94277fe700 1 -- 172.21.15.117:0/3302750362 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 89858+0+0 (secure 0 0 0) 0x7f9428003680 con 0x7f9420006230 2024-02-16T19:22:00.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:22:00.642+0000 7f94277fe700 1 --2- 172.21.15.117:0/3302750362 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f941806c750 0x7f941806ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:00.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:22:00.642+0000 7f94277fe700 1 -- 172.21.15.117:0/3302750362 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f942808d6d0 con 0x7f9420006230 2024-02-16T19:22:00.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:22:00.642+0000 7f9435d9b700 1 --2- 172.21.15.117:0/3302750362 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f941806c750 0x7f941806ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:00.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:22:00.643+0000 7f9435d9b700 1 --2- 172.21.15.117:0/3302750362 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f941806c750 0x7f941806ec00 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f942c009180 tx=0x7f942c008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:22:00.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:22:00.647+0000 7f94277fe700 1 -- 172.21.15.117:0/3302750362 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f942805ca50 con 0x7f9420006230 2024-02-16T19:22:00.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:22:00.916+0000 7f943cd62700 1 -- 172.21.15.117:0/3302750362 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f9438198c40 con 0x7f9420006230 2024-02-16T19:22:00.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:22:00.917+0000 7f94277fe700 1 -- 172.21.15.117:0/3302750362 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1022 (secure 0 0 0) 0x7f9428035090 con 0x7f9420006230 2024-02-16T19:22:00.919 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-02-16T19:22:00.919 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":2,"fsid":"2a1346e6-cd00-11ee-95ba-87774f69a715","modified":"2024-02-16T19:21:54.047971Z","created":"2024-02-16T19:19:07.447976Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi022","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:3300","nonce":0},{"type":"v1","addr":"172.21.15.22:6789","nonce":0}]},"addr":"172.21.15.22:6789/0","public_addr":"172.21.15.22:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"smithi117","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:3300","nonce":0},{"type":"v1","addr":"172.21.15.117:6789","nonce":0}]},"addr":"172.21.15.117:6789/0","public_addr":"172.21.15.117:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-02-16T19:22:00.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:22:00.919+0000 7f943cd62700 1 -- 172.21.15.117:0/3302750362 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f941806c750 msgr2=0x7f941806ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:00.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:22:00.919+0000 7f943cd62700 1 --2- 172.21.15.117:0/3302750362 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f941806c750 0x7f941806ec00 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f942c009180 tx=0x7f942c008040 comp rx=0 tx=0).stop 2024-02-16T19:22:00.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:22:00.919+0000 7f943cd62700 1 -- 172.21.15.117:0/3302750362 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9420006230 msgr2=0x7f943819d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:00.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:22:00.919+0000 7f943cd62700 1 --2- 172.21.15.117:0/3302750362 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9420006230 0x7f943819d6b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9438068f20 tx=0x7f942801d9b0 comp rx=0 tx=0).stop 2024-02-16T19:22:00.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:22:00.919+0000 7f943cd62700 1 -- 172.21.15.117:0/3302750362 shutdown_connections 2024-02-16T19:22:00.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:22:00.919+0000 7f943cd62700 1 --2- 172.21.15.117:0/3302750362 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9420006230 0x7f943819d6b0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:00.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:22:00.919+0000 7f943cd62700 1 --2- 172.21.15.117:0/3302750362 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f941806c750 0x7f941806ec00 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:00.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:22:00.919+0000 7f943cd62700 1 --2- 172.21.15.117:0/3302750362 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f943819dbf0 0x7f9438197730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:00.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:22:00.919+0000 7f943cd62700 1 -- 172.21.15.117:0/3302750362 >> 172.21.15.117:0/3302750362 conn(0x7f94380fa9a0 msgr2=0x7f94380fbf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:22:00.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:22:00.919+0000 7f943cd62700 1 -- 172.21.15.117:0/3302750362 shutdown_connections 2024-02-16T19:22:00.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:22:00.919+0000 7f943cd62700 1 -- 172.21.15.117:0/3302750362 wait complete. 2024-02-16T19:22:00.922 INFO:teuthology.orchestra.run.smithi117.stderr:dumped monmap epoch 2 2024-02-16T19:22:01.236 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:01 smithi117 ceph-mon[35309]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:22:01.236 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:01 smithi117 ceph-mon[35309]: from='client.? 172.21.15.117:0/3302750362' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-16T19:22:01.464 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:01 smithi022 ceph-mon[29221]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:22:01.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:01 smithi022 ceph-mon[29221]: from='client.? 172.21.15.117:0/3302750362' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-16T19:22:01.507 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-02-16T19:22:01.508 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -- ceph config generate-minimal-conf 2024-02-16T19:22:01.746 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi022/config 2024-02-16T19:22:02.885 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:02 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:02.885 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:02 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:02.885 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:02 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:22:02.885 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:02 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:22:02.885 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:02 smithi022 ceph-mon[29221]: Updating smithi022:/etc/ceph/ceph.conf 2024-02-16T19:22:02.885 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:02 smithi022 ceph-mon[29221]: Updating smithi117:/etc/ceph/ceph.conf 2024-02-16T19:22:02.885 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:02 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:02.885 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:02 smithi022 ceph-mon[29221]: Updating smithi022:/var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T19:22:02.885 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:02 smithi022 ceph-mon[29221]: Updating smithi117:/var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T19:22:02.885 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:02 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:02.886 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:02 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:02.886 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:02 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:02.886 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:02 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:02.886 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:02 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:02.886 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:02 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-16T19:22:02.886 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:02 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-16T19:22:02.886 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:02 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:22:02.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:02 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:02.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:02 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:02.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:02 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:22:02.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:02 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:22:02.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:02 smithi117 ceph-mon[35309]: Updating smithi022:/etc/ceph/ceph.conf 2024-02-16T19:22:02.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:02 smithi117 ceph-mon[35309]: Updating smithi117:/etc/ceph/ceph.conf 2024-02-16T19:22:02.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:02 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:02.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:02 smithi117 ceph-mon[35309]: Updating smithi022:/var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T19:22:02.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:02 smithi117 ceph-mon[35309]: Updating smithi117:/var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T19:22:02.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:02 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:02.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:02 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:02.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:02 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:02.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:02 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:02.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:02 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:02.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:02 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-16T19:22:02.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:02 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-16T19:22:02.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:02 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:22:03.874 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:03 smithi022 ceph-mon[29221]: Reconfiguring mon.smithi022 (unknown last config time)... 2024-02-16T19:22:03.874 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:03 smithi022 ceph-mon[29221]: Reconfiguring daemon mon.smithi022 on smithi022 2024-02-16T19:22:03.874 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:03 smithi022 ceph-mon[29221]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:22:03.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:03 smithi117 ceph-mon[35309]: Reconfiguring mon.smithi022 (unknown last config time)... 2024-02-16T19:22:03.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:03 smithi117 ceph-mon[35309]: Reconfiguring daemon mon.smithi022 on smithi022 2024-02-16T19:22:03.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:03 smithi117 ceph-mon[35309]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:22:04.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:04.431+0000 7f50288c8700 1 -- 172.21.15.22:0/105384675 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f50240687a0 msgr2=0x7f5024068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:04.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:04.431+0000 7f50288c8700 1 --2- 172.21.15.22:0/105384675 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f50240687a0 0x7f5024068b70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5014009910 tx=0x7f5014009c20 comp rx=0 tx=0).stop 2024-02-16T19:22:04.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:04.431+0000 7f50288c8700 1 -- 172.21.15.22:0/105384675 shutdown_connections 2024-02-16T19:22:04.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:04.431+0000 7f50288c8700 1 --2- 172.21.15.22:0/105384675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5024069140 0x7f5024104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:04.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:04.431+0000 7f50288c8700 1 --2- 172.21.15.22:0/105384675 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f50240687a0 0x7f5024068b70 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:04.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:04.431+0000 7f50288c8700 1 -- 172.21.15.22:0/105384675 >> 172.21.15.22:0/105384675 conn(0x7f5024075130 msgr2=0x7f5024075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:22:04.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:04.432+0000 7f50288c8700 1 -- 172.21.15.22:0/105384675 shutdown_connections 2024-02-16T19:22:04.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:04.432+0000 7f50288c8700 1 -- 172.21.15.22:0/105384675 wait complete. 2024-02-16T19:22:04.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:04.432+0000 7f50288c8700 1 Processor -- start 2024-02-16T19:22:04.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:04.433+0000 7f50288c8700 1 -- start start 2024-02-16T19:22:04.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:04.433+0000 7f50288c8700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f50240687a0 0x7f5024195f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:04.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:04.433+0000 7f50288c8700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5024069140 0x7f5024196480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:04.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:04.433+0000 7f50288c8700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5024196b60 con 0x7f50240687a0 2024-02-16T19:22:04.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:04.433+0000 7f50288c8700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f502419a8f0 con 0x7f5024069140 2024-02-16T19:22:04.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:04.433+0000 7f502259c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f50240687a0 0x7f5024195f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:04.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:04.433+0000 7f502259c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f50240687a0 0x7f5024195f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:56204/0 (socket says 172.21.15.22:56204) 2024-02-16T19:22:04.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:04.433+0000 7f502259c700 1 -- 172.21.15.22:0/3713685038 learned_addr learned my addr 172.21.15.22:0/3713685038 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:22:04.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:04.433+0000 7f5021d9b700 1 --2- 172.21.15.22:0/3713685038 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5024069140 0x7f5024196480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:04.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:04.433+0000 7f502259c700 1 -- 172.21.15.22:0/3713685038 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5024069140 msgr2=0x7f5024196480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:04.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:04.433+0000 7f502259c700 1 --2- 172.21.15.22:0/3713685038 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5024069140 0x7f5024196480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:04.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:04.433+0000 7f502259c700 1 -- 172.21.15.22:0/3713685038 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50140095f0 con 0x7f50240687a0 2024-02-16T19:22:04.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:04.433+0000 7f5021d9b700 1 --2- 172.21.15.22:0/3713685038 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5024069140 0x7f5024196480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-16T19:22:04.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:04.433+0000 7f502259c700 1 --2- 172.21.15.22:0/3713685038 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f50240687a0 0x7f5024195f40 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5014006010 tx=0x7f501400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:22:04.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:04.434+0000 7f50137fe700 1 -- 172.21.15.22:0/3713685038 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f501401d070 con 0x7f50240687a0 2024-02-16T19:22:04.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:04.434+0000 7f50137fe700 1 -- 172.21.15.22:0/3713685038 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f501400f460 con 0x7f50240687a0 2024-02-16T19:22:04.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:04.434+0000 7f50288c8700 1 -- 172.21.15.22:0/3713685038 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f502419ab70 con 0x7f50240687a0 2024-02-16T19:22:04.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:04.434+0000 7f50137fe700 1 -- 172.21.15.22:0/3713685038 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5014017630 con 0x7f50240687a0 2024-02-16T19:22:04.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:04.434+0000 7f50288c8700 1 -- 172.21.15.22:0/3713685038 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f502419b060 con 0x7f50240687a0 2024-02-16T19:22:04.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:04.435+0000 7f50137fe700 1 -- 172.21.15.22:0/3713685038 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 89858+0+0 (secure 0 0 0) 0x7f5014004b70 con 0x7f50240687a0 2024-02-16T19:22:04.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:04.435+0000 7f50288c8700 1 -- 172.21.15.22:0/3713685038 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5024108790 con 0x7f50240687a0 2024-02-16T19:22:04.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:04.435+0000 7f50137fe700 1 --2- 172.21.15.22:0/3713685038 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f500c06c610 0x7f500c06eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:04.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:04.436+0000 7f50137fe700 1 -- 172.21.15.22:0/3713685038 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f501408be00 con 0x7f50240687a0 2024-02-16T19:22:04.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:04.436+0000 7f5021d9b700 1 --2- 172.21.15.22:0/3713685038 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f500c06c610 0x7f500c06eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:04.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:04.436+0000 7f5021d9b700 1 --2- 172.21.15.22:0/3713685038 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f500c06c610 0x7f500c06eac0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f5024197560 tx=0x7f5018008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:22:04.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:04.440+0000 7f50137fe700 1 -- 172.21.15.22:0/3713685038 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f50140575c0 con 0x7f50240687a0 2024-02-16T19:22:04.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:04.642+0000 7f50288c8700 1 -- 172.21.15.22:0/3713685038 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f502404f2a0 con 0x7f50240687a0 2024-02-16T19:22:04.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:04.642+0000 7f50137fe700 1 -- 172.21.15.22:0/3713685038 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v10) v1 ==== 76+0+225 (secure 0 0 0) 0x7f501408b5c0 con 0x7f50240687a0 2024-02-16T19:22:04.644 INFO:teuthology.orchestra.run.smithi022.stdout:# minimal ceph.conf for 2a1346e6-cd00-11ee-95ba-87774f69a715 2024-02-16T19:22:04.644 INFO:teuthology.orchestra.run.smithi022.stdout:[global] 2024-02-16T19:22:04.644 INFO:teuthology.orchestra.run.smithi022.stdout: fsid = 2a1346e6-cd00-11ee-95ba-87774f69a715 2024-02-16T19:22:04.644 INFO:teuthology.orchestra.run.smithi022.stdout: mon_host = [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] 2024-02-16T19:22:04.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:04.644+0000 7f50288c8700 1 -- 172.21.15.22:0/3713685038 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f500c06c610 msgr2=0x7f500c06eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:04.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:04.644+0000 7f50288c8700 1 --2- 172.21.15.22:0/3713685038 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f500c06c610 0x7f500c06eac0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f5024197560 tx=0x7f5018008040 comp rx=0 tx=0).stop 2024-02-16T19:22:04.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:04.644+0000 7f50288c8700 1 -- 172.21.15.22:0/3713685038 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f50240687a0 msgr2=0x7f5024195f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:04.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:04.644+0000 7f50288c8700 1 --2- 172.21.15.22:0/3713685038 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f50240687a0 0x7f5024195f40 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5014006010 tx=0x7f501400be20 comp rx=0 tx=0).stop 2024-02-16T19:22:04.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:04.645+0000 7f50288c8700 1 -- 172.21.15.22:0/3713685038 shutdown_connections 2024-02-16T19:22:04.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:04.645+0000 7f50288c8700 1 --2- 172.21.15.22:0/3713685038 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f50240687a0 0x7f5024195f40 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:04.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:04.645+0000 7f50288c8700 1 --2- 172.21.15.22:0/3713685038 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f500c06c610 0x7f500c06eac0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:04.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:04.645+0000 7f50288c8700 1 --2- 172.21.15.22:0/3713685038 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5024069140 0x7f5024196480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:04.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:04.645+0000 7f50288c8700 1 -- 172.21.15.22:0/3713685038 >> 172.21.15.22:0/3713685038 conn(0x7f5024075130 msgr2=0x7f50240fe7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:22:04.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:04.645+0000 7f50288c8700 1 -- 172.21.15.22:0/3713685038 shutdown_connections 2024-02-16T19:22:04.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:04.645+0000 7f50288c8700 1 -- 172.21.15.22:0/3713685038 wait complete. 2024-02-16T19:22:05.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:04 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:05.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:04 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:05.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:04 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi022.gmdcch", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-16T19:22:05.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:04 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-16T19:22:05.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:04 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:22:05.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:04 smithi022 ceph-mon[29221]: from='client.? 172.21.15.22:0/3713685038' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:22:05.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:04 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:05.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:04 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:05.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:04 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi022.gmdcch", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-16T19:22:05.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:04 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-16T19:22:05.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:04 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:22:05.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:04 smithi117 ceph-mon[35309]: from='client.? 172.21.15.22:0/3713685038' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:22:05.545 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-02-16T19:22:05.545 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-02-16T19:22:05.545 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/etc/ceph/ceph.conf 2024-02-16T19:22:05.577 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-02-16T19:22:05.577 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-16T19:22:05.646 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-02-16T19:22:05.646 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/ceph/ceph.conf 2024-02-16T19:22:05.680 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-02-16T19:22:05.680 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-16T19:22:05.751 INFO:tasks.cephadm:Deploying OSDs... 2024-02-16T19:22:05.751 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-02-16T19:22:05.751 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/scratch_devs of=/dev/stdout 2024-02-16T19:22:05.770 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-16T19:22:05.770 DEBUG:teuthology.orchestra.run.smithi022:> stat /dev/vg_nvme/lv_1 2024-02-16T19:22:05.825 INFO:teuthology.orchestra.run.smithi022.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-16T19:22:05.825 INFO:teuthology.orchestra.run.smithi022.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-16T19:22:05.825 INFO:teuthology.orchestra.run.smithi022.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-02-16T19:22:05.825 INFO:teuthology.orchestra.run.smithi022.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-16T19:22:05.825 INFO:teuthology.orchestra.run.smithi022.stdout:Context: system_u:object_r:device_t:s0 2024-02-16T19:22:05.825 INFO:teuthology.orchestra.run.smithi022.stdout:Access: 2024-02-16 19:21:18.183593817 +0000 2024-02-16T19:22:05.825 INFO:teuthology.orchestra.run.smithi022.stdout:Modify: 2024-02-16 19:14:49.658838272 +0000 2024-02-16T19:22:05.826 INFO:teuthology.orchestra.run.smithi022.stdout:Change: 2024-02-16 19:14:49.658838272 +0000 2024-02-16T19:22:05.826 INFO:teuthology.orchestra.run.smithi022.stdout: Birth: 2024-02-16 19:14:49.658838272 +0000 2024-02-16T19:22:05.826 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-16T19:22:05.891 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records in 2024-02-16T19:22:05.891 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records out 2024-02-16T19:22:05.891 INFO:teuthology.orchestra.run.smithi022.stderr:512 bytes copied, 8.838e-05 s, 5.8 MB/s 2024-02-16T19:22:05.892 DEBUG:teuthology.orchestra.run.smithi022:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-16T19:22:05.925 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:05 smithi117 ceph-mon[35309]: Reconfiguring mgr.smithi022.gmdcch (unknown last config time)... 2024-02-16T19:22:05.925 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:05 smithi117 ceph-mon[35309]: Reconfiguring daemon mgr.smithi022.gmdcch on smithi022 2024-02-16T19:22:05.925 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:05 smithi117 ceph-mon[35309]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:22:05.950 DEBUG:teuthology.orchestra.run.smithi022:> stat /dev/vg_nvme/lv_2 2024-02-16T19:22:06.005 INFO:teuthology.orchestra.run.smithi022.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-16T19:22:06.005 INFO:teuthology.orchestra.run.smithi022.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-16T19:22:06.005 INFO:teuthology.orchestra.run.smithi022.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-02-16T19:22:06.005 INFO:teuthology.orchestra.run.smithi022.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-16T19:22:06.005 INFO:teuthology.orchestra.run.smithi022.stdout:Context: system_u:object_r:device_t:s0 2024-02-16T19:22:06.005 INFO:teuthology.orchestra.run.smithi022.stdout:Access: 2024-02-16 19:21:18.184593804 +0000 2024-02-16T19:22:06.005 INFO:teuthology.orchestra.run.smithi022.stdout:Modify: 2024-02-16 19:14:49.989833923 +0000 2024-02-16T19:22:06.006 INFO:teuthology.orchestra.run.smithi022.stdout:Change: 2024-02-16 19:14:49.989833923 +0000 2024-02-16T19:22:06.006 INFO:teuthology.orchestra.run.smithi022.stdout: Birth: 2024-02-16 19:14:49.989833923 +0000 2024-02-16T19:22:06.006 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-16T19:22:06.070 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records in 2024-02-16T19:22:06.070 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records out 2024-02-16T19:22:06.070 INFO:teuthology.orchestra.run.smithi022.stderr:512 bytes copied, 9.2546e-05 s, 5.5 MB/s 2024-02-16T19:22:06.071 DEBUG:teuthology.orchestra.run.smithi022:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-16T19:22:06.127 DEBUG:teuthology.orchestra.run.smithi022:> stat /dev/vg_nvme/lv_3 2024-02-16T19:22:06.183 INFO:teuthology.orchestra.run.smithi022.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-16T19:22:06.183 INFO:teuthology.orchestra.run.smithi022.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-16T19:22:06.183 INFO:teuthology.orchestra.run.smithi022.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-02-16T19:22:06.183 INFO:teuthology.orchestra.run.smithi022.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-16T19:22:06.184 INFO:teuthology.orchestra.run.smithi022.stdout:Context: system_u:object_r:device_t:s0 2024-02-16T19:22:06.184 INFO:teuthology.orchestra.run.smithi022.stdout:Access: 2024-02-16 19:21:18.184593804 +0000 2024-02-16T19:22:06.184 INFO:teuthology.orchestra.run.smithi022.stdout:Modify: 2024-02-16 19:14:50.330829442 +0000 2024-02-16T19:22:06.184 INFO:teuthology.orchestra.run.smithi022.stdout:Change: 2024-02-16 19:14:50.330829442 +0000 2024-02-16T19:22:06.184 INFO:teuthology.orchestra.run.smithi022.stdout: Birth: 2024-02-16 19:14:50.330829442 +0000 2024-02-16T19:22:06.185 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-16T19:22:06.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:05 smithi022 ceph-mon[29221]: Reconfiguring mgr.smithi022.gmdcch (unknown last config time)... 2024-02-16T19:22:06.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:05 smithi022 ceph-mon[29221]: Reconfiguring daemon mgr.smithi022.gmdcch on smithi022 2024-02-16T19:22:06.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:05 smithi022 ceph-mon[29221]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:22:06.240 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records in 2024-02-16T19:22:06.240 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records out 2024-02-16T19:22:06.240 INFO:teuthology.orchestra.run.smithi022.stderr:512 bytes copied, 9.2259e-05 s, 5.5 MB/s 2024-02-16T19:22:06.241 DEBUG:teuthology.orchestra.run.smithi022:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-16T19:22:06.296 DEBUG:teuthology.orchestra.run.smithi022:> stat /dev/vg_nvme/lv_4 2024-02-16T19:22:06.352 INFO:teuthology.orchestra.run.smithi022.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-16T19:22:06.352 INFO:teuthology.orchestra.run.smithi022.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-16T19:22:06.352 INFO:teuthology.orchestra.run.smithi022.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-02-16T19:22:06.353 INFO:teuthology.orchestra.run.smithi022.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-16T19:22:06.353 INFO:teuthology.orchestra.run.smithi022.stdout:Context: system_u:object_r:device_t:s0 2024-02-16T19:22:06.353 INFO:teuthology.orchestra.run.smithi022.stdout:Access: 2024-02-16 19:21:18.184593804 +0000 2024-02-16T19:22:06.353 INFO:teuthology.orchestra.run.smithi022.stdout:Modify: 2024-02-16 19:14:50.670824975 +0000 2024-02-16T19:22:06.353 INFO:teuthology.orchestra.run.smithi022.stdout:Change: 2024-02-16 19:14:50.670824975 +0000 2024-02-16T19:22:06.353 INFO:teuthology.orchestra.run.smithi022.stdout: Birth: 2024-02-16 19:14:50.670824975 +0000 2024-02-16T19:22:06.353 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-16T19:22:06.418 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records in 2024-02-16T19:22:06.419 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records out 2024-02-16T19:22:06.419 INFO:teuthology.orchestra.run.smithi022.stderr:512 bytes copied, 0.000101389 s, 5.0 MB/s 2024-02-16T19:22:06.420 DEBUG:teuthology.orchestra.run.smithi022:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-16T19:22:06.475 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-02-16T19:22:06.476 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/scratch_devs of=/dev/stdout 2024-02-16T19:22:06.494 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-16T19:22:06.494 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_1 2024-02-16T19:22:06.552 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-16T19:22:06.552 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-16T19:22:06.552 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-02-16T19:22:06.552 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-16T19:22:06.552 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:device_t:s0 2024-02-16T19:22:06.552 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-02-16 19:21:52.589160612 +0000 2024-02-16T19:22:06.552 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-02-16 19:14:49.314975571 +0000 2024-02-16T19:22:06.553 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-02-16 19:14:49.314975571 +0000 2024-02-16T19:22:06.553 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: 2024-02-16 19:14:49.314975571 +0000 2024-02-16T19:22:06.553 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-16T19:22:06.622 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-02-16T19:22:06.622 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-02-16T19:22:06.622 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000234058 s, 2.2 MB/s 2024-02-16T19:22:06.623 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-16T19:22:06.681 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_2 2024-02-16T19:22:06.738 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-16T19:22:06.738 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-16T19:22:06.738 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-02-16T19:22:06.738 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-16T19:22:06.738 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:device_t:s0 2024-02-16T19:22:06.738 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-02-16 19:21:52.589160612 +0000 2024-02-16T19:22:06.738 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-02-16 19:14:49.659968541 +0000 2024-02-16T19:22:06.739 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-02-16 19:14:49.659968541 +0000 2024-02-16T19:22:06.739 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: 2024-02-16 19:14:49.659968541 +0000 2024-02-16T19:22:06.739 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-16T19:22:06.806 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-02-16T19:22:06.806 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-02-16T19:22:06.806 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000226786 s, 2.3 MB/s 2024-02-16T19:22:06.807 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-16T19:22:06.865 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_3 2024-02-16T19:22:06.920 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-16T19:22:06.920 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-16T19:22:06.921 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-02-16T19:22:06.921 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-16T19:22:06.921 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:device_t:s0 2024-02-16T19:22:06.921 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-02-16 19:21:52.590160591 +0000 2024-02-16T19:22:06.921 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-02-16 19:14:49.992961754 +0000 2024-02-16T19:22:06.921 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-02-16 19:14:49.992961754 +0000 2024-02-16T19:22:06.921 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: 2024-02-16 19:14:49.992961754 +0000 2024-02-16T19:22:06.921 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-16T19:22:06.987 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-02-16T19:22:06.987 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-02-16T19:22:06.987 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000226026 s, 2.3 MB/s 2024-02-16T19:22:06.989 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-16T19:22:07.044 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_4 2024-02-16T19:22:07.100 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-16T19:22:07.100 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-16T19:22:07.100 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-02-16T19:22:07.100 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-16T19:22:07.100 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:device_t:s0 2024-02-16T19:22:07.100 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-02-16 19:21:52.590160591 +0000 2024-02-16T19:22:07.100 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-02-16 19:14:50.323955019 +0000 2024-02-16T19:22:07.100 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-02-16 19:14:50.323955019 +0000 2024-02-16T19:22:07.100 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: 2024-02-16 19:14:50.323955019 +0000 2024-02-16T19:22:07.100 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-16T19:22:07.165 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-02-16T19:22:07.165 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-02-16T19:22:07.165 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.00021079 s, 2.4 MB/s 2024-02-16T19:22:07.167 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-16T19:22:07.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:07 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:07.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:07 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:07.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:07 smithi022 ceph-mon[29221]: Reconfiguring ceph-exporter.smithi022 (monmap changed)... 2024-02-16T19:22:07.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:07 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi022", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-02-16T19:22:07.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:07 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:22:07.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:07 smithi022 ceph-mon[29221]: Reconfiguring daemon ceph-exporter.smithi022 on smithi022 2024-02-16T19:22:07.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:07 smithi022 ceph-mon[29221]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:22:07.222 INFO:tasks.cephadm:Deploying osd.0 on smithi022 with /dev/vg_nvme/lv_4... 2024-02-16T19:22:07.222 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-02-16T19:22:07.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:07 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:07.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:07 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:07.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:07 smithi117 ceph-mon[35309]: Reconfiguring ceph-exporter.smithi022 (monmap changed)... 2024-02-16T19:22:07.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:07 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi022", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-02-16T19:22:07.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:07 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:22:07.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:07 smithi117 ceph-mon[35309]: Reconfiguring daemon ceph-exporter.smithi022 on smithi022 2024-02-16T19:22:07.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:07 smithi117 ceph-mon[35309]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:22:07.471 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi022/config 2024-02-16T19:22:08.619 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:08 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:08.619 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:08 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:08.619 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:08 smithi022 ceph-mon[29221]: Reconfiguring crash.smithi022 (monmap changed)... 2024-02-16T19:22:08.619 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:08 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi022", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-16T19:22:08.619 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:08 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:22:08.619 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:08 smithi022 ceph-mon[29221]: Reconfiguring daemon crash.smithi022 on smithi022 2024-02-16T19:22:08.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:08 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:08.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:08 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:08.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:08 smithi117 ceph-mon[35309]: Reconfiguring crash.smithi022 (monmap changed)... 2024-02-16T19:22:08.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:08 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi022", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-16T19:22:08.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:08 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:22:08.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:08 smithi117 ceph-mon[35309]: Reconfiguring daemon crash.smithi022 on smithi022 2024-02-16T19:22:09.678 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:09 smithi022 ceph-mon[29221]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:22:09.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:09 smithi117 ceph-mon[35309]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:22:10.569 INFO:teuthology.orchestra.run.smithi022.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-02-16T19:22:10.569 INFO:teuthology.orchestra.run.smithi022.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-02-16T19:22:10.570 INFO:teuthology.orchestra.run.smithi022.stderr: stderr: 10+0 records in 2024-02-16T19:22:10.570 INFO:teuthology.orchestra.run.smithi022.stderr:10+0 records out 2024-02-16T19:22:10.570 INFO:teuthology.orchestra.run.smithi022.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00834269 s, 1.3 GB/s 2024-02-16T19:22:10.570 INFO:teuthology.orchestra.run.smithi022.stderr:--> Zapping successful for: 2024-02-16T19:22:11.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:11 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:11.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:11 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:11.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:11 smithi117 ceph-mon[35309]: Reconfiguring alertmanager.smithi022 (dependencies changed)... 2024-02-16T19:22:11.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:11 smithi117 ceph-mon[35309]: Reconfiguring daemon alertmanager.smithi022 on smithi022 2024-02-16T19:22:11.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:11 smithi117 ceph-mon[35309]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:22:11.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:11 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:11.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:11 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:11.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:11 smithi022 ceph-mon[29221]: Reconfiguring alertmanager.smithi022 (dependencies changed)... 2024-02-16T19:22:11.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:11 smithi022 ceph-mon[29221]: Reconfiguring daemon alertmanager.smithi022 on smithi022 2024-02-16T19:22:11.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:11 smithi022 ceph-mon[29221]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:22:11.832 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi022:vg_nvme/lv_4 2024-02-16T19:22:13.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:12 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:22:13.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:12 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:22:13.720 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi022/config 2024-02-16T19:22:14.093 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:13 smithi022 ceph-mon[29221]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:22:14.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:13 smithi117 ceph-mon[35309]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:22:16.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:16.036+0000 7fbaa6bc3700 1 -- 172.21.15.22:0/2290454665 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbaa0100240 msgr2=0x7fbaa0100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:16.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:16.036+0000 7fbaa6bc3700 1 --2- 172.21.15.22:0/2290454665 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbaa0100240 0x7fbaa0100650 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fba94009910 tx=0x7fba94009c20 comp rx=0 tx=0).stop 2024-02-16T19:22:16.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:16.036+0000 7fbaa6bc3700 1 -- 172.21.15.22:0/2290454665 shutdown_connections 2024-02-16T19:22:16.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:16.036+0000 7fbaa6bc3700 1 --2- 172.21.15.22:0/2290454665 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbaa0101440 0x7fbaa0101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:16.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:16.036+0000 7fbaa6bc3700 1 --2- 172.21.15.22:0/2290454665 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbaa0100240 0x7fbaa0100650 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:16.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:16.036+0000 7fbaa6bc3700 1 -- 172.21.15.22:0/2290454665 >> 172.21.15.22:0/2290454665 conn(0x7fbaa00fb7f0 msgr2=0x7fbaa00fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:22:16.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:16.036+0000 7fbaa6bc3700 1 -- 172.21.15.22:0/2290454665 shutdown_connections 2024-02-16T19:22:16.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:16.037+0000 7fbaa6bc3700 1 -- 172.21.15.22:0/2290454665 wait complete. 2024-02-16T19:22:16.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:16.037+0000 7fbaa6bc3700 1 Processor -- start 2024-02-16T19:22:16.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:16.037+0000 7fbaa6bc3700 1 -- start start 2024-02-16T19:22:16.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:16.037+0000 7fbaa6bc3700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbaa0100240 0x7fbaa0195ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:16.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:16.037+0000 7fbaa6bc3700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbaa0101440 0x7fbaa0196010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:16.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:16.037+0000 7fbaa6bc3700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa0196630 con 0x7fbaa0101440 2024-02-16T19:22:16.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:16.037+0000 7fbaa6bc3700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa0196770 con 0x7fbaa0100240 2024-02-16T19:22:16.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:16.037+0000 7fba9ffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbaa0101440 0x7fbaa0196010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:16.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:16.037+0000 7fba9ffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbaa0101440 0x7fbaa0196010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45200/0 (socket says 172.21.15.22:45200) 2024-02-16T19:22:16.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:16.037+0000 7fba9ffff700 1 -- 172.21.15.22:0/2213533246 learned_addr learned my addr 172.21.15.22:0/2213533246 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:22:16.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:16.037+0000 7fbaa495f700 1 --2- 172.21.15.22:0/2213533246 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbaa0100240 0x7fbaa0195ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:16.041 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:15 smithi022 ceph-mon[29221]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:22:16.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:16.038+0000 7fba9ffff700 1 -- 172.21.15.22:0/2213533246 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbaa0100240 msgr2=0x7fbaa0195ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:16.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:16.038+0000 7fba9ffff700 1 --2- 172.21.15.22:0/2213533246 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbaa0100240 0x7fbaa0195ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:16.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:16.038+0000 7fba9ffff700 1 -- 172.21.15.22:0/2213533246 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba940095f0 con 0x7fbaa0101440 2024-02-16T19:22:16.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:16.038+0000 7fba9ffff700 1 --2- 172.21.15.22:0/2213533246 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbaa0101440 0x7fbaa0196010 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fba8c00d8d0 tx=0x7fba8c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:22:16.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:16.038+0000 7fba9dffb700 1 -- 172.21.15.22:0/2213533246 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fba8c0098a0 con 0x7fbaa0101440 2024-02-16T19:22:16.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:16.038+0000 7fba9dffb700 1 -- 172.21.15.22:0/2213533246 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fba8c010460 con 0x7fbaa0101440 2024-02-16T19:22:16.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:16.038+0000 7fbaa6bc3700 1 -- 172.21.15.22:0/2213533246 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbaa019b220 con 0x7fbaa0101440 2024-02-16T19:22:16.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:16.038+0000 7fba9dffb700 1 -- 172.21.15.22:0/2213533246 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fba8c00f5f0 con 0x7fbaa0101440 2024-02-16T19:22:16.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:16.039+0000 7fbaa6bc3700 1 -- 172.21.15.22:0/2213533246 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbaa019b770 con 0x7fbaa0101440 2024-02-16T19:22:16.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:16.040+0000 7fbaa6bc3700 1 -- 172.21.15.22:0/2213533246 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbaa0108d90 con 0x7fbaa0101440 2024-02-16T19:22:16.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:16.040+0000 7fba9dffb700 1 -- 172.21.15.22:0/2213533246 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 89858+0+0 (secure 0 0 0) 0x7fba8c009a00 con 0x7fbaa0101440 2024-02-16T19:22:16.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:16.041+0000 7fba9dffb700 1 --2- 172.21.15.22:0/2213533246 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fba9006c600 0x7fba9006eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:16.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:16.041+0000 7fba9dffb700 1 -- 172.21.15.22:0/2213533246 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fba8c08a600 con 0x7fbaa0101440 2024-02-16T19:22:16.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:16.041+0000 7fbaa495f700 1 --2- 172.21.15.22:0/2213533246 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fba9006c600 0x7fba9006eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:16.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:16.041+0000 7fbaa495f700 1 --2- 172.21.15.22:0/2213533246 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fba9006c600 0x7fba9006eab0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fba94006010 tx=0x7fba9400b540 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:22:16.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:16.046+0000 7fba9dffb700 1 -- 172.21.15.22:0/2213533246 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fba8c059870 con 0x7fbaa0101440 2024-02-16T19:22:16.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:15 smithi117 ceph-mon[35309]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:22:16.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:16.264+0000 7fbaa6bc3700 1 -- 172.21.15.22:0/2213533246 --> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fbaa0105d90 con 0x7fba9006c600 2024-02-16T19:22:17.119 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:16 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-16T19:22:17.120 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:16 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-16T19:22:17.120 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:16 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:22:17.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:16 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-16T19:22:17.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:16 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-16T19:22:17.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:16 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:22:18.118 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:17 smithi022 ceph-mon[29221]: from='client.14254 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:22:18.118 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:17 smithi022 ceph-mon[29221]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:22:18.118 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:17 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:18.118 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:17 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:18.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:17 smithi117 ceph-mon[35309]: from='client.14254 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:22:18.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:17 smithi117 ceph-mon[35309]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:22:18.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:17 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:18.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:17 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:19.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:18 smithi117 ceph-mon[35309]: Reconfiguring grafana.smithi022 (dependencies changed)... 2024-02-16T19:22:19.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:18 smithi117 ceph-mon[35309]: Reconfiguring daemon grafana.smithi022 on smithi022 2024-02-16T19:22:19.140 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:18 smithi022 ceph-mon[29221]: Reconfiguring grafana.smithi022 (dependencies changed)... 2024-02-16T19:22:19.141 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:18 smithi022 ceph-mon[29221]: Reconfiguring daemon grafana.smithi022 on smithi022 2024-02-16T19:22:20.052 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:19 smithi117 ceph-mon[35309]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:22:20.054 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:19 smithi022 ceph-mon[29221]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:22:21.888 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:21 smithi022 ceph-mon[29221]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:22:21.888 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:21 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:21.888 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:21 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:22.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:21 smithi117 ceph-mon[35309]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:22:22.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:21 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:22.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:21 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:23.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:22 smithi117 ceph-mon[35309]: Reconfiguring prometheus.smithi022 (dependencies changed)... 2024-02-16T19:22:23.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:22 smithi022 ceph-mon[29221]: Reconfiguring prometheus.smithi022 (dependencies changed)... 2024-02-16T19:22:23.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:23 smithi022 ceph-mon[29221]: Reconfiguring daemon prometheus.smithi022 on smithi022 2024-02-16T19:22:23.968 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:23 smithi022 ceph-mon[29221]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:22:23.968 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:23 smithi022 ceph-mon[29221]: from='client.? 172.21.15.22:0/1856738144' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d16e47f2-7209-45f4-ad50-1e0f6789ffad"}]: dispatch 2024-02-16T19:22:23.968 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:23 smithi022 ceph-mon[29221]: from='client.? 172.21.15.22:0/1856738144' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d16e47f2-7209-45f4-ad50-1e0f6789ffad"}]': finished 2024-02-16T19:22:23.968 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:23 smithi022 ceph-mon[29221]: osdmap e6: 1 total, 0 up, 1 in 2024-02-16T19:22:23.968 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:23 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-16T19:22:23.968 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:23 smithi022 ceph-mon[29221]: from='client.? 172.21.15.22:0/518727221' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-16T19:22:24.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:23 smithi117 ceph-mon[35309]: Reconfiguring daemon prometheus.smithi022 on smithi022 2024-02-16T19:22:24.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:23 smithi117 ceph-mon[35309]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:22:24.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:23 smithi117 ceph-mon[35309]: from='client.? 172.21.15.22:0/1856738144' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d16e47f2-7209-45f4-ad50-1e0f6789ffad"}]: dispatch 2024-02-16T19:22:24.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:23 smithi117 ceph-mon[35309]: from='client.? 172.21.15.22:0/1856738144' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d16e47f2-7209-45f4-ad50-1e0f6789ffad"}]': finished 2024-02-16T19:22:24.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:23 smithi117 ceph-mon[35309]: osdmap e6: 1 total, 0 up, 1 in 2024-02-16T19:22:24.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:23 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-16T19:22:24.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:23 smithi117 ceph-mon[35309]: from='client.? 172.21.15.22:0/518727221' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-16T19:22:26.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:25 smithi117 ceph-mon[35309]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:22:26.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:25 smithi022 ceph-mon[29221]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:22:27.899 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:27 smithi022 ceph-mon[29221]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:22:27.900 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:27 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:22:28.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:27 smithi117 ceph-mon[35309]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:22:28.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:27 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:22:30.016 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:29 smithi022 ceph-mon[29221]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:22:30.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:29 smithi117 ceph-mon[35309]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:22:31.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:31 smithi022 ceph-mon[29221]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:22:32.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:31 smithi117 ceph-mon[35309]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:22:33.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:33 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:33.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:33 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:33.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:33 smithi117 ceph-mon[35309]: Reconfiguring ceph-exporter.smithi117 (monmap changed)... 2024-02-16T19:22:33.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:33 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi117", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-02-16T19:22:33.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:33 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:22:33.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:33 smithi117 ceph-mon[35309]: Reconfiguring daemon ceph-exporter.smithi117 on smithi117 2024-02-16T19:22:33.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:33 smithi117 ceph-mon[35309]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:22:33.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:33 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:33.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:33 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:33.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:33 smithi022 ceph-mon[29221]: Reconfiguring ceph-exporter.smithi117 (monmap changed)... 2024-02-16T19:22:33.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:33 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi117", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-02-16T19:22:33.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:33 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:22:33.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:33 smithi022 ceph-mon[29221]: Reconfiguring daemon ceph-exporter.smithi117 on smithi117 2024-02-16T19:22:33.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:33 smithi022 ceph-mon[29221]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:22:35.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:34 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:35.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:34 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:35.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:34 smithi022 ceph-mon[29221]: Reconfiguring crash.smithi117 (monmap changed)... 2024-02-16T19:22:35.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:34 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi117", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-16T19:22:35.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:34 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:22:35.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:34 smithi022 ceph-mon[29221]: Reconfiguring daemon crash.smithi117 on smithi117 2024-02-16T19:22:35.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:34 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-16T19:22:35.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:34 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:22:35.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:34 smithi022 ceph-mon[29221]: Deploying daemon osd.0 on smithi022 2024-02-16T19:22:35.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:34 smithi022 ceph-mon[29221]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:22:35.361 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:34 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:35.361 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:34 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:35.361 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:34 smithi117 ceph-mon[35309]: Reconfiguring crash.smithi117 (monmap changed)... 2024-02-16T19:22:35.361 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:34 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi117", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-16T19:22:35.361 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:34 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:22:35.361 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:34 smithi117 ceph-mon[35309]: Reconfiguring daemon crash.smithi117 on smithi117 2024-02-16T19:22:35.361 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:34 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-16T19:22:35.362 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:34 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:22:35.362 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:34 smithi117 ceph-mon[35309]: Deploying daemon osd.0 on smithi022 2024-02-16T19:22:35.362 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:34 smithi117 ceph-mon[35309]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:22:36.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:36 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:36.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:36 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:36.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:36 smithi022 ceph-mon[29221]: Reconfiguring mgr.smithi117.cohbtv (monmap changed)... 2024-02-16T19:22:36.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:36 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi117.cohbtv", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-16T19:22:36.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:36 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-16T19:22:36.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:36 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:22:36.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:36 smithi022 ceph-mon[29221]: Reconfiguring daemon mgr.smithi117.cohbtv on smithi117 2024-02-16T19:22:36.757 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:36 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:36.758 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:36 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:36.758 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:36 smithi117 ceph-mon[35309]: Reconfiguring mgr.smithi117.cohbtv (monmap changed)... 2024-02-16T19:22:36.758 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:36 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi117.cohbtv", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-16T19:22:36.758 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:36 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-16T19:22:36.758 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:36 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:22:36.758 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:36 smithi117 ceph-mon[35309]: Reconfiguring daemon mgr.smithi117.cohbtv on smithi117 2024-02-16T19:22:37.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:37 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:37.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:37 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:37.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:37 smithi022 ceph-mon[29221]: Reconfiguring mon.smithi117 (monmap changed)... 2024-02-16T19:22:37.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:37 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-16T19:22:37.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:37 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-16T19:22:37.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:37 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:22:37.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:37 smithi022 ceph-mon[29221]: Reconfiguring daemon mon.smithi117 on smithi117 2024-02-16T19:22:37.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:37 smithi022 ceph-mon[29221]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:22:38.112 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:37 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:38.112 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:37 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:38.112 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:37 smithi117 ceph-mon[35309]: Reconfiguring mon.smithi117 (monmap changed)... 2024-02-16T19:22:38.112 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:37 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-16T19:22:38.113 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:37 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-16T19:22:38.113 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:37 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:22:38.113 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:37 smithi117 ceph-mon[35309]: Reconfiguring daemon mon.smithi117 on smithi117 2024-02-16T19:22:38.113 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:37 smithi117 ceph-mon[35309]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:22:38.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:38 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:38.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:38 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:38.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:38 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-16T19:22:38.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:38 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi022.front.sepia.ceph.com:9093"}]: dispatch 2024-02-16T19:22:38.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:38 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:38.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:38 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-16T19:22:38.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:38 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi022.front.sepia.ceph.com:3000"}]: dispatch 2024-02-16T19:22:38.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:38 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:38.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:38 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-16T19:22:38.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:38 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi022.front.sepia.ceph.com:9095"}]: dispatch 2024-02-16T19:22:38.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:38 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:38.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:38 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:22:39.311 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:38 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:39.311 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:38 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:39.311 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:38 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-16T19:22:39.311 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:38 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi022.front.sepia.ceph.com:9093"}]: dispatch 2024-02-16T19:22:39.311 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:38 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:39.311 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:38 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-16T19:22:39.311 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:38 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi022.front.sepia.ceph.com:3000"}]: dispatch 2024-02-16T19:22:39.311 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:38 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:39.312 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:38 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-16T19:22:39.312 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:38 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi022.front.sepia.ceph.com:9095"}]: dispatch 2024-02-16T19:22:39.312 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:38 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:39.312 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:38 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:22:40.186 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:39 smithi022 ceph-mon[29221]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-16T19:22:40.187 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:39 smithi022 ceph-mon[29221]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi022.front.sepia.ceph.com:9093"}]: dispatch 2024-02-16T19:22:40.187 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:39 smithi022 ceph-mon[29221]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-16T19:22:40.187 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:39 smithi022 ceph-mon[29221]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi022.front.sepia.ceph.com:3000"}]: dispatch 2024-02-16T19:22:40.187 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:39 smithi022 ceph-mon[29221]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-16T19:22:40.187 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:39 smithi022 ceph-mon[29221]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi022.front.sepia.ceph.com:9095"}]: dispatch 2024-02-16T19:22:40.187 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:39 smithi022 ceph-mon[29221]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:22:40.187 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:39 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:40.187 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:39 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:40.187 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:39 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:40.187 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:39 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:40.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:39 smithi117 ceph-mon[35309]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-16T19:22:40.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:39 smithi117 ceph-mon[35309]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi022.front.sepia.ceph.com:9093"}]: dispatch 2024-02-16T19:22:40.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:39 smithi117 ceph-mon[35309]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-16T19:22:40.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:39 smithi117 ceph-mon[35309]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi022.front.sepia.ceph.com:3000"}]: dispatch 2024-02-16T19:22:40.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:39 smithi117 ceph-mon[35309]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-16T19:22:40.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:39 smithi117 ceph-mon[35309]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi022.front.sepia.ceph.com:9095"}]: dispatch 2024-02-16T19:22:40.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:39 smithi117 ceph-mon[35309]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:22:40.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:39 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:40.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:39 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:40.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:39 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:40.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:39 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:42.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:41 smithi117 ceph-mon[35309]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:22:42.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:41 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:22:42.204 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:41 smithi022 ceph-mon[29221]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:22:42.204 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:41 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:22:44.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:43 smithi022 ceph-mon[29221]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:22:44.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:43 smithi022 ceph-mon[29221]: from='osd.0 [v2:172.21.15.22:6802/2126256017,v1:172.21.15.22:6803/2126256017]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-16T19:22:44.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:43 smithi117 ceph-mon[35309]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:22:44.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:43 smithi117 ceph-mon[35309]: from='osd.0 [v2:172.21.15.22:6802/2126256017,v1:172.21.15.22:6803/2126256017]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-16T19:22:44.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:44.949+0000 7fba9dffb700 1 -- 172.21.15.22:0/2213533246 <== mgr.14217 v2:172.21.15.22:6800/2253867279 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fbaa0105d90 con 0x7fba9006c600 2024-02-16T19:22:44.955 INFO:teuthology.orchestra.run.smithi022.stdout:Created osd(s) 0 on host 'smithi022' 2024-02-16T19:22:44.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:44.951+0000 7fbaa6bc3700 1 -- 172.21.15.22:0/2213533246 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fba9006c600 msgr2=0x7fba9006eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:44.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:44.951+0000 7fbaa6bc3700 1 --2- 172.21.15.22:0/2213533246 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fba9006c600 0x7fba9006eab0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fba94006010 tx=0x7fba9400b540 comp rx=0 tx=0).stop 2024-02-16T19:22:44.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:44.951+0000 7fbaa6bc3700 1 -- 172.21.15.22:0/2213533246 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbaa0101440 msgr2=0x7fbaa0196010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:44.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:44.951+0000 7fbaa6bc3700 1 --2- 172.21.15.22:0/2213533246 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbaa0101440 0x7fbaa0196010 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fba8c00d8d0 tx=0x7fba8c00dbe0 comp rx=0 tx=0).stop 2024-02-16T19:22:44.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:44.951+0000 7fbaa6bc3700 1 -- 172.21.15.22:0/2213533246 shutdown_connections 2024-02-16T19:22:44.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:44.951+0000 7fbaa6bc3700 1 --2- 172.21.15.22:0/2213533246 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fba9006c600 0x7fba9006eab0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:44.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:44.951+0000 7fbaa6bc3700 1 --2- 172.21.15.22:0/2213533246 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbaa0100240 0x7fbaa0195ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:44.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:44.951+0000 7fbaa6bc3700 1 --2- 172.21.15.22:0/2213533246 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbaa0101440 0x7fbaa0196010 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:44.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:44.951+0000 7fbaa6bc3700 1 -- 172.21.15.22:0/2213533246 >> 172.21.15.22:0/2213533246 conn(0x7fbaa00fb7f0 msgr2=0x7fbaa0104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:22:44.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:44.951+0000 7fbaa6bc3700 1 -- 172.21.15.22:0/2213533246 shutdown_connections 2024-02-16T19:22:44.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:44.951+0000 7fbaa6bc3700 1 -- 172.21.15.22:0/2213533246 wait complete. 2024-02-16T19:22:45.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:44 smithi022 ceph-mon[29221]: from='osd.0 [v2:172.21.15.22:6802/2126256017,v1:172.21.15.22:6803/2126256017]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-16T19:22:45.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:44 smithi022 ceph-mon[29221]: osdmap e7: 1 total, 0 up, 1 in 2024-02-16T19:22:45.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:44 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-16T19:22:45.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:44 smithi022 ceph-mon[29221]: from='osd.0 [v2:172.21.15.22:6802/2126256017,v1:172.21.15.22:6803/2126256017]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-02-16T19:22:45.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:44 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:45.216 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:44 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:45.216 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:44 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:45.216 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:44 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:45.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:44 smithi117 ceph-mon[35309]: from='osd.0 [v2:172.21.15.22:6802/2126256017,v1:172.21.15.22:6803/2126256017]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-16T19:22:45.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:44 smithi117 ceph-mon[35309]: osdmap e7: 1 total, 0 up, 1 in 2024-02-16T19:22:45.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:44 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-16T19:22:45.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:44 smithi117 ceph-mon[35309]: from='osd.0 [v2:172.21.15.22:6802/2126256017,v1:172.21.15.22:6803/2126256017]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-02-16T19:22:45.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:44 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:45.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:44 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:45.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:44 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:45.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:44 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:45.634 DEBUG:teuthology.orchestra.run.smithi022:osd.0> sudo journalctl -f -n 0 -u ceph-2a1346e6-cd00-11ee-95ba-87774f69a715@osd.0.service 2024-02-16T19:22:45.637 INFO:tasks.cephadm:Deploying osd.1 on smithi022 with /dev/vg_nvme/lv_3... 2024-02-16T19:22:45.637 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-02-16T19:22:46.201 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:45 smithi022 ceph-mon[29221]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:22:46.202 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:45 smithi022 ceph-mon[29221]: from='osd.0 [v2:172.21.15.22:6802/2126256017,v1:172.21.15.22:6803/2126256017]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-02-16T19:22:46.202 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:45 smithi022 ceph-mon[29221]: osdmap e8: 1 total, 0 up, 1 in 2024-02-16T19:22:46.202 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:45 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-16T19:22:46.202 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:45 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-16T19:22:46.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:45 smithi117 ceph-mon[35309]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:22:46.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:45 smithi117 ceph-mon[35309]: from='osd.0 [v2:172.21.15.22:6802/2126256017,v1:172.21.15.22:6803/2126256017]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-02-16T19:22:46.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:45 smithi117 ceph-mon[35309]: osdmap e8: 1 total, 0 up, 1 in 2024-02-16T19:22:46.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:45 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-16T19:22:46.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:45 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-16T19:22:47.105 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:46 smithi022 ceph-mon[29221]: purged_snaps scrub starts 2024-02-16T19:22:47.105 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:46 smithi022 ceph-mon[29221]: purged_snaps scrub ok 2024-02-16T19:22:47.105 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:46 smithi022 ceph-mon[29221]: osd.0 [v2:172.21.15.22:6802/2126256017,v1:172.21.15.22:6803/2126256017] boot 2024-02-16T19:22:47.105 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:46 smithi022 ceph-mon[29221]: osdmap e9: 1 total, 1 up, 1 in 2024-02-16T19:22:47.105 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:46 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-16T19:22:47.105 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:46 smithi022 ceph-mon[29221]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:22:47.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:46 smithi117 ceph-mon[35309]: purged_snaps scrub starts 2024-02-16T19:22:47.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:46 smithi117 ceph-mon[35309]: purged_snaps scrub ok 2024-02-16T19:22:47.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:46 smithi117 ceph-mon[35309]: osd.0 [v2:172.21.15.22:6802/2126256017,v1:172.21.15.22:6803/2126256017] boot 2024-02-16T19:22:47.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:46 smithi117 ceph-mon[35309]: osdmap e9: 1 total, 1 up, 1 in 2024-02-16T19:22:47.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:46 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-16T19:22:47.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:46 smithi117 ceph-mon[35309]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:22:47.340 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi022/config 2024-02-16T19:22:48.318 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:47 smithi022 ceph-mon[29221]: osdmap e10: 1 total, 1 up, 1 in 2024-02-16T19:22:48.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:47 smithi117 ceph-mon[35309]: osdmap e10: 1 total, 1 up, 1 in 2024-02-16T19:22:49.856 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:49 smithi022 ceph-mon[29221]: Detected new or changed devices on smithi022 2024-02-16T19:22:49.856 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:49 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:49.856 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:49 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:49.856 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:49 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:22:49.856 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:49 smithi022 ceph-mon[29221]: Adjusting osd_memory_target on smithi022 to 11845M 2024-02-16T19:22:49.856 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:49 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:49.856 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:49 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:22:49.856 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:49 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:22:49.856 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:49 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:49.856 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:49 smithi022 ceph-mon[29221]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-16T19:22:49.857 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:49 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:22:49.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:49 smithi117 ceph-mon[35309]: Detected new or changed devices on smithi022 2024-02-16T19:22:49.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:49 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:49.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:49 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:49.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:49 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:22:49.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:49 smithi117 ceph-mon[35309]: Adjusting osd_memory_target on smithi022 to 11845M 2024-02-16T19:22:49.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:49 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:49.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:49 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:22:49.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:49 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:22:49.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:49 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:49.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:49 smithi117 ceph-mon[35309]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-16T19:22:49.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:49 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:22:49.888 INFO:teuthology.orchestra.run.smithi022.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-02-16T19:22:49.888 INFO:teuthology.orchestra.run.smithi022.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-02-16T19:22:49.888 INFO:teuthology.orchestra.run.smithi022.stderr: stderr: 10+0 records in 2024-02-16T19:22:49.888 INFO:teuthology.orchestra.run.smithi022.stderr:10+0 records out 2024-02-16T19:22:49.888 INFO:teuthology.orchestra.run.smithi022.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00852243 s, 1.2 GB/s 2024-02-16T19:22:49.888 INFO:teuthology.orchestra.run.smithi022.stderr:--> Zapping successful for: 2024-02-16T19:22:50.596 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi022:vg_nvme/lv_3 2024-02-16T19:22:51.174 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi022/config 2024-02-16T19:22:51.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:51 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:51.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:51 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:51.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:51 smithi117 ceph-mon[35309]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-16T19:22:51.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:51 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:51.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:51 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:51.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:51 smithi022 ceph-mon[29221]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-16T19:22:53.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:53.291+0000 7f5287a5b700 1 -- 172.21.15.22:0/573331270 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5280100240 msgr2=0x7f5280100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:53.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:53.291+0000 7f5287a5b700 1 --2- 172.21.15.22:0/573331270 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5280100240 0x7f5280100650 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f527c009910 tx=0x7f527c009c20 comp rx=0 tx=0).stop 2024-02-16T19:22:53.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:53.291+0000 7f5287a5b700 1 -- 172.21.15.22:0/573331270 shutdown_connections 2024-02-16T19:22:53.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:53.291+0000 7f5287a5b700 1 --2- 172.21.15.22:0/573331270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5280101440 0x7f5280101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:53.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:53.291+0000 7f5287a5b700 1 --2- 172.21.15.22:0/573331270 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5280100240 0x7f5280100650 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:53.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:53.291+0000 7f5287a5b700 1 -- 172.21.15.22:0/573331270 >> 172.21.15.22:0/573331270 conn(0x7f52800fb7f0 msgr2=0x7f52800fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:22:53.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:53.291+0000 7f5287a5b700 1 -- 172.21.15.22:0/573331270 shutdown_connections 2024-02-16T19:22:53.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:53.291+0000 7f5287a5b700 1 -- 172.21.15.22:0/573331270 wait complete. 2024-02-16T19:22:53.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:53.291+0000 7f5287a5b700 1 Processor -- start 2024-02-16T19:22:53.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:53.291+0000 7f5287a5b700 1 -- start start 2024-02-16T19:22:53.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:53.292+0000 7f5287a5b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5280100240 0x7f5280195b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:53.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:53.292+0000 7f5287a5b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5280101440 0x7f52801960a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:53.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:53.292+0000 7f5287a5b700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52801966c0 con 0x7f5280101440 2024-02-16T19:22:53.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:53.292+0000 7f5287a5b700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5280196800 con 0x7f5280100240 2024-02-16T19:22:53.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:53.292+0000 7f5284ff6700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5280101440 0x7f52801960a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:53.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:53.292+0000 7f5284ff6700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5280101440 0x7f52801960a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42276/0 (socket says 172.21.15.22:42276) 2024-02-16T19:22:53.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:53.292+0000 7f5284ff6700 1 -- 172.21.15.22:0/2807068776 learned_addr learned my addr 172.21.15.22:0/2807068776 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:22:53.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:53.292+0000 7f52857f7700 1 --2- 172.21.15.22:0/2807068776 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5280100240 0x7f5280195b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:53.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:53.292+0000 7f5284ff6700 1 -- 172.21.15.22:0/2807068776 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5280100240 msgr2=0x7f5280195b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:53.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:53.292+0000 7f5284ff6700 1 --2- 172.21.15.22:0/2807068776 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5280100240 0x7f5280195b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:53.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:53.292+0000 7f5284ff6700 1 -- 172.21.15.22:0/2807068776 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f527c0095f0 con 0x7f5280101440 2024-02-16T19:22:53.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:53.292+0000 7f5284ff6700 1 --2- 172.21.15.22:0/2807068776 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5280101440 0x7f52801960a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f527000b4b0 tx=0x7f527000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:22:53.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:53.292+0000 7f52767fc700 1 -- 172.21.15.22:0/2807068776 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5270011800 con 0x7f5280101440 2024-02-16T19:22:53.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:53.292+0000 7f52767fc700 1 -- 172.21.15.22:0/2807068776 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5270011e10 con 0x7f5280101440 2024-02-16T19:22:53.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:53.292+0000 7f52767fc700 1 -- 172.21.15.22:0/2807068776 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f527000f2e0 con 0x7f5280101440 2024-02-16T19:22:53.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:53.292+0000 7f5287a5b700 1 -- 172.21.15.22:0/2807068776 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f528019b2b0 con 0x7f5280101440 2024-02-16T19:22:53.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:53.293+0000 7f5287a5b700 1 -- 172.21.15.22:0/2807068776 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f528019b800 con 0x7f5280101440 2024-02-16T19:22:53.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:53.294+0000 7f5287a5b700 1 -- 172.21.15.22:0/2807068776 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5280108e10 con 0x7f5280101440 2024-02-16T19:22:53.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:53.294+0000 7f52767fc700 1 -- 172.21.15.22:0/2807068776 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 89858+0+0 (secure 0 0 0) 0x7f5270010380 con 0x7f5280101440 2024-02-16T19:22:53.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:53.294+0000 7f52767fc700 1 --2- 172.21.15.22:0/2807068776 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f526c06c600 0x7f526c06eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:53.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:53.294+0000 7f52857f7700 1 --2- 172.21.15.22:0/2807068776 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f526c06c600 0x7f526c06eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:53.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:53.294+0000 7f52767fc700 1 -- 172.21.15.22:0/2807068776 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1982+0+0 (secure 0 0 0) 0x7f527008a520 con 0x7f5280101440 2024-02-16T19:22:53.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:53.295+0000 7f52857f7700 1 --2- 172.21.15.22:0/2807068776 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f526c06c600 0x7f526c06eab0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f527c006010 tx=0x7f527c005920 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:22:53.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:53.299+0000 7f52767fc700 1 -- 172.21.15.22:0/2807068776 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f5270059640 con 0x7f5280101440 2024-02-16T19:22:53.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:22:53.514+0000 7f5287a5b700 1 -- 172.21.15.22:0/2807068776 --> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f528019bf50 con 0x7f526c06c600 2024-02-16T19:22:54.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:53 smithi117 ceph-mon[35309]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-16T19:22:54.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:53 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-16T19:22:54.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:53 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-16T19:22:54.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:53 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:22:54.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:53 smithi022 ceph-mon[29221]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-16T19:22:54.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:53 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-16T19:22:54.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:53 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-16T19:22:54.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:53 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:22:54.859 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:54 smithi022 ceph-mon[29221]: from='client.14272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:22:55.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:54 smithi117 ceph-mon[35309]: from='client.14272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:22:55.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:55 smithi022 ceph-mon[29221]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-16T19:22:56.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:55 smithi117 ceph-mon[35309]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-16T19:22:57.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:57 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:57.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:57 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:57.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:57 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:22:57.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:57 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:22:57.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:57 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:57.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:57 smithi117 ceph-mon[35309]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-16T19:22:57.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:57 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:22:57.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:57 smithi117 ceph-mon[35309]: from='client.? 172.21.15.22:0/3484017390' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "59c3d085-3422-4e92-9048-46d7a39e31a3"}]: dispatch 2024-02-16T19:22:57.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:57 smithi117 ceph-mon[35309]: from='client.? 172.21.15.22:0/3484017390' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "59c3d085-3422-4e92-9048-46d7a39e31a3"}]': finished 2024-02-16T19:22:57.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:57 smithi117 ceph-mon[35309]: osdmap e11: 2 total, 1 up, 2 in 2024-02-16T19:22:57.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:57 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-16T19:22:57.894 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:57 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:57.894 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:57 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:57.894 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:57 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:22:57.895 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:57 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:22:57.895 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:57 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:22:57.895 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:57 smithi022 ceph-mon[29221]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-16T19:22:57.895 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:57 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:22:57.895 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:57 smithi022 ceph-mon[29221]: from='client.? 172.21.15.22:0/3484017390' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "59c3d085-3422-4e92-9048-46d7a39e31a3"}]: dispatch 2024-02-16T19:22:57.895 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:57 smithi022 ceph-mon[29221]: from='client.? 172.21.15.22:0/3484017390' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "59c3d085-3422-4e92-9048-46d7a39e31a3"}]': finished 2024-02-16T19:22:57.895 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:57 smithi022 ceph-mon[29221]: osdmap e11: 2 total, 1 up, 2 in 2024-02-16T19:22:57.895 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:57 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-16T19:22:58.906 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:58 smithi022 ceph-mon[29221]: from='client.? 172.21.15.22:0/2987872136' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-16T19:22:59.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:58 smithi117 ceph-mon[35309]: from='client.? 172.21.15.22:0/2987872136' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-16T19:22:59.964 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:22:59 smithi022 ceph-mon[29221]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-16T19:23:00.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:22:59 smithi117 ceph-mon[35309]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-16T19:23:01.959 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:01 smithi022 ceph-mon[29221]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-16T19:23:02.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:01 smithi117 ceph-mon[35309]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-16T19:23:03.964 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:03 smithi022 ceph-mon[29221]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-16T19:23:04.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:03 smithi117 ceph-mon[35309]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-16T19:23:04.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:04 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-16T19:23:04.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:04 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:23:05.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:04 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-16T19:23:05.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:04 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:23:06.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:05 smithi117 ceph-mon[35309]: Deploying daemon osd.1 on smithi022 2024-02-16T19:23:06.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:05 smithi117 ceph-mon[35309]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-16T19:23:06.153 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:05 smithi022 ceph-mon[29221]: Deploying daemon osd.1 on smithi022 2024-02-16T19:23:06.154 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:05 smithi022 ceph-mon[29221]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-16T19:23:07.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:07 smithi022 ceph-mon[29221]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-16T19:23:08.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:07 smithi117 ceph-mon[35309]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-16T19:23:10.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:09 smithi117 ceph-mon[35309]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-16T19:23:10.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:09 smithi022 ceph-mon[29221]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-16T19:23:11.316 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:11 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:23:11.316 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:11 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:23:11.316 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:11 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:23:11.316 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:11 smithi022 ceph-mon[29221]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-16T19:23:11.385 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:11 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:23:11.385 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:11 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:23:11.385 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:11 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:23:11.385 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:11 smithi117 ceph-mon[35309]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-16T19:23:13.062 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:12 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:23:13.062 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:12 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:23:13.062 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:12 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:23:13.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:12 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:23:13.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:12 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:23:13.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:12 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:23:13.964 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:13 smithi022 ceph-mon[29221]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-16T19:23:13.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:23:13.971+0000 7f52767fc700 1 -- 172.21.15.22:0/2807068776 <== mgr.14217 v2:172.21.15.22:6800/2253867279 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f528019bf50 con 0x7f526c06c600 2024-02-16T19:23:13.972 INFO:teuthology.orchestra.run.smithi022.stdout:Created osd(s) 1 on host 'smithi022' 2024-02-16T19:23:13.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:23:13.974+0000 7f5287a5b700 1 -- 172.21.15.22:0/2807068776 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f526c06c600 msgr2=0x7f526c06eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:23:13.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:23:13.974+0000 7f5287a5b700 1 --2- 172.21.15.22:0/2807068776 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f526c06c600 0x7f526c06eab0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f527c006010 tx=0x7f527c005920 comp rx=0 tx=0).stop 2024-02-16T19:23:13.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:23:13.974+0000 7f5287a5b700 1 -- 172.21.15.22:0/2807068776 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5280101440 msgr2=0x7f52801960a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:23:13.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:23:13.974+0000 7f5287a5b700 1 --2- 172.21.15.22:0/2807068776 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5280101440 0x7f52801960a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f527000b4b0 tx=0x7f527000b870 comp rx=0 tx=0).stop 2024-02-16T19:23:13.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:23:13.974+0000 7f5287a5b700 1 -- 172.21.15.22:0/2807068776 shutdown_connections 2024-02-16T19:23:13.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:23:13.974+0000 7f5287a5b700 1 --2- 172.21.15.22:0/2807068776 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f526c06c600 0x7f526c06eab0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:23:13.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:23:13.974+0000 7f5287a5b700 1 --2- 172.21.15.22:0/2807068776 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5280100240 0x7f5280195b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:23:13.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:23:13.974+0000 7f5287a5b700 1 --2- 172.21.15.22:0/2807068776 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5280101440 0x7f52801960a0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:23:13.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:23:13.974+0000 7f5287a5b700 1 -- 172.21.15.22:0/2807068776 >> 172.21.15.22:0/2807068776 conn(0x7f52800fb7f0 msgr2=0x7f5280104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:23:13.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:23:13.974+0000 7f5287a5b700 1 -- 172.21.15.22:0/2807068776 shutdown_connections 2024-02-16T19:23:13.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:23:13.974+0000 7f5287a5b700 1 -- 172.21.15.22:0/2807068776 wait complete. 2024-02-16T19:23:14.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:13 smithi117 ceph-mon[35309]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-16T19:23:15.083 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:14 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:23:15.083 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:14 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:23:15.083 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:14 smithi022 ceph-mon[29221]: from='osd.1 [v2:172.21.15.22:6810/3355370212,v1:172.21.15.22:6811/3355370212]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-16T19:23:15.111 DEBUG:teuthology.orchestra.run.smithi022:osd.1> sudo journalctl -f -n 0 -u ceph-2a1346e6-cd00-11ee-95ba-87774f69a715@osd.1.service 2024-02-16T19:23:15.114 INFO:tasks.cephadm:Deploying osd.2 on smithi022 with /dev/vg_nvme/lv_2... 2024-02-16T19:23:15.114 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-02-16T19:23:15.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:14 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:23:15.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:14 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:23:15.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:14 smithi117 ceph-mon[35309]: from='osd.1 [v2:172.21.15.22:6810/3355370212,v1:172.21.15.22:6811/3355370212]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-16T19:23:16.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:15 smithi022 ceph-mon[29221]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-16T19:23:16.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:15 smithi022 ceph-mon[29221]: from='osd.1 [v2:172.21.15.22:6810/3355370212,v1:172.21.15.22:6811/3355370212]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-16T19:23:16.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:15 smithi022 ceph-mon[29221]: osdmap e12: 2 total, 1 up, 2 in 2024-02-16T19:23:16.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:15 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-16T19:23:16.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:15 smithi022 ceph-mon[29221]: from='osd.1 [v2:172.21.15.22:6810/3355370212,v1:172.21.15.22:6811/3355370212]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-02-16T19:23:16.216 INFO:journalctl@ceph.osd.1.smithi022.stdout:Feb 16 19:23:15 smithi022 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-1[49665]: 2024-02-16T19:23:15.977+0000 7f309c576700 -1 osd.1 0 waiting for initial osdmap 2024-02-16T19:23:16.216 INFO:journalctl@ceph.osd.1.smithi022.stdout:Feb 16 19:23:15 smithi022 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-1[49665]: 2024-02-16T19:23:15.981+0000 7f3096b68700 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-16T19:23:16.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:15 smithi117 ceph-mon[35309]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-16T19:23:16.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:15 smithi117 ceph-mon[35309]: from='osd.1 [v2:172.21.15.22:6810/3355370212,v1:172.21.15.22:6811/3355370212]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-16T19:23:16.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:15 smithi117 ceph-mon[35309]: osdmap e12: 2 total, 1 up, 2 in 2024-02-16T19:23:16.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:15 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-16T19:23:16.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:15 smithi117 ceph-mon[35309]: from='osd.1 [v2:172.21.15.22:6810/3355370212,v1:172.21.15.22:6811/3355370212]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-02-16T19:23:16.561 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi022/config 2024-02-16T19:23:17.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:16 smithi117 ceph-mon[35309]: purged_snaps scrub starts 2024-02-16T19:23:17.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:16 smithi117 ceph-mon[35309]: purged_snaps scrub ok 2024-02-16T19:23:17.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:16 smithi117 ceph-mon[35309]: from='osd.1 [v2:172.21.15.22:6810/3355370212,v1:172.21.15.22:6811/3355370212]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-02-16T19:23:17.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:16 smithi117 ceph-mon[35309]: osdmap e13: 2 total, 1 up, 2 in 2024-02-16T19:23:17.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:16 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-16T19:23:17.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:16 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-16T19:23:17.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:16 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:23:17.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:16 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:23:17.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:16 smithi117 ceph-mon[35309]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-16T19:23:17.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:16 smithi022 ceph-mon[29221]: purged_snaps scrub starts 2024-02-16T19:23:17.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:16 smithi022 ceph-mon[29221]: purged_snaps scrub ok 2024-02-16T19:23:17.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:16 smithi022 ceph-mon[29221]: from='osd.1 [v2:172.21.15.22:6810/3355370212,v1:172.21.15.22:6811/3355370212]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-02-16T19:23:17.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:16 smithi022 ceph-mon[29221]: osdmap e13: 2 total, 1 up, 2 in 2024-02-16T19:23:17.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:16 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-16T19:23:17.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:16 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-16T19:23:17.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:16 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:23:17.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:16 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:23:17.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:16 smithi022 ceph-mon[29221]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-16T19:23:18.200 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:17 smithi022 ceph-mon[29221]: osd.1 [v2:172.21.15.22:6810/3355370212,v1:172.21.15.22:6811/3355370212] boot 2024-02-16T19:23:18.200 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:17 smithi022 ceph-mon[29221]: osdmap e14: 2 total, 2 up, 2 in 2024-02-16T19:23:18.200 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:17 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-16T19:23:18.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:17 smithi117 ceph-mon[35309]: osd.1 [v2:172.21.15.22:6810/3355370212,v1:172.21.15.22:6811/3355370212] boot 2024-02-16T19:23:18.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:17 smithi117 ceph-mon[35309]: osdmap e14: 2 total, 2 up, 2 in 2024-02-16T19:23:18.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:17 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-16T19:23:19.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:18 smithi022 ceph-mon[29221]: osdmap e15: 2 total, 2 up, 2 in 2024-02-16T19:23:19.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:18 smithi022 ceph-mon[29221]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-16T19:23:19.220 INFO:teuthology.orchestra.run.smithi022.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-02-16T19:23:19.220 INFO:teuthology.orchestra.run.smithi022.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-02-16T19:23:19.220 INFO:teuthology.orchestra.run.smithi022.stderr: stderr: 10+0 records in 2024-02-16T19:23:19.220 INFO:teuthology.orchestra.run.smithi022.stderr:10+0 records out 2024-02-16T19:23:19.220 INFO:teuthology.orchestra.run.smithi022.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00862837 s, 1.2 GB/s 2024-02-16T19:23:19.220 INFO:teuthology.orchestra.run.smithi022.stderr:--> Zapping successful for: 2024-02-16T19:23:19.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:18 smithi117 ceph-mon[35309]: osdmap e15: 2 total, 2 up, 2 in 2024-02-16T19:23:19.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:18 smithi117 ceph-mon[35309]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-16T19:23:19.915 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi022:vg_nvme/lv_2 2024-02-16T19:23:20.438 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi022/config 2024-02-16T19:23:22.090 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:21 smithi117 ceph-mon[35309]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-16T19:23:22.125 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:21 smithi022 ceph-mon[29221]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-16T19:23:22.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:23:22.610+0000 7f2f02758700 1 -- 172.21.15.22:0/2605371586 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2efc0ff150 msgr2=0x7f2efc0ff560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:23:22.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:23:22.610+0000 7f2f02758700 1 --2- 172.21.15.22:0/2605371586 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2efc0ff150 0x7f2efc0ff560 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2eec009910 tx=0x7f2eec009c20 comp rx=0 tx=0).stop 2024-02-16T19:23:22.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:23:22.610+0000 7f2f02758700 1 -- 172.21.15.22:0/2605371586 shutdown_connections 2024-02-16T19:23:22.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:23:22.610+0000 7f2f02758700 1 --2- 172.21.15.22:0/2605371586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2efc100350 0x7f2efc1007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:23:22.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:23:22.610+0000 7f2f02758700 1 --2- 172.21.15.22:0/2605371586 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2efc0ff150 0x7f2efc0ff560 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:23:22.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:23:22.610+0000 7f2f02758700 1 -- 172.21.15.22:0/2605371586 >> 172.21.15.22:0/2605371586 conn(0x7f2efc0fa700 msgr2=0x7f2efc0fcb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:23:22.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:23:22.610+0000 7f2f02758700 1 -- 172.21.15.22:0/2605371586 shutdown_connections 2024-02-16T19:23:22.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:23:22.610+0000 7f2f02758700 1 -- 172.21.15.22:0/2605371586 wait complete. 2024-02-16T19:23:22.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:23:22.611+0000 7f2f02758700 1 Processor -- start 2024-02-16T19:23:22.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:23:22.611+0000 7f2f02758700 1 -- start start 2024-02-16T19:23:22.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:23:22.611+0000 7f2f02758700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2efc0ff150 0x7f2efc195ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:23:22.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:23:22.611+0000 7f2f02758700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2efc100350 0x7f2efc196000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:23:22.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:23:22.611+0000 7f2f02758700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2efc196620 con 0x7f2efc0ff150 2024-02-16T19:23:22.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:23:22.611+0000 7f2f02758700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2efc196760 con 0x7f2efc100350 2024-02-16T19:23:22.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:23:22.611+0000 7f2efbfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2efc0ff150 0x7f2efc195ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:23:22.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:23:22.611+0000 7f2efbfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2efc0ff150 0x7f2efc195ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41952/0 (socket says 172.21.15.22:41952) 2024-02-16T19:23:22.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:23:22.611+0000 7f2efbfff700 1 -- 172.21.15.22:0/881539366 learned_addr learned my addr 172.21.15.22:0/881539366 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:23:22.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:23:22.611+0000 7f2efb7fe700 1 --2- 172.21.15.22:0/881539366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2efc100350 0x7f2efc196000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:23:22.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:23:22.612+0000 7f2efbfff700 1 -- 172.21.15.22:0/881539366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2efc100350 msgr2=0x7f2efc196000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:23:22.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:23:22.612+0000 7f2efbfff700 1 --2- 172.21.15.22:0/881539366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2efc100350 0x7f2efc196000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:23:22.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:23:22.612+0000 7f2efbfff700 1 -- 172.21.15.22:0/881539366 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eec0095f0 con 0x7f2efc0ff150 2024-02-16T19:23:22.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:23:22.612+0000 7f2efbfff700 1 --2- 172.21.15.22:0/881539366 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2efc0ff150 0x7f2efc195ac0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2eec00bda0 tx=0x7f2eec00bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:23:22.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:23:22.612+0000 7f2ef97fa700 1 -- 172.21.15.22:0/881539366 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2eec01d070 con 0x7f2efc0ff150 2024-02-16T19:23:22.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:23:22.612+0000 7f2ef97fa700 1 -- 172.21.15.22:0/881539366 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2eec00f460 con 0x7f2efc0ff150 2024-02-16T19:23:22.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:23:22.612+0000 7f2f02758700 1 -- 172.21.15.22:0/881539366 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2efc19b1b0 con 0x7f2efc0ff150 2024-02-16T19:23:22.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:23:22.612+0000 7f2ef97fa700 1 -- 172.21.15.22:0/881539366 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2eec021640 con 0x7f2efc0ff150 2024-02-16T19:23:22.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:23:22.612+0000 7f2f02758700 1 -- 172.21.15.22:0/881539366 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2efc19b6a0 con 0x7f2efc0ff150 2024-02-16T19:23:22.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:23:22.613+0000 7f2f02758700 1 -- 172.21.15.22:0/881539366 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2efc107ca0 con 0x7f2efc0ff150 2024-02-16T19:23:22.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:23:22.614+0000 7f2ef97fa700 1 -- 172.21.15.22:0/881539366 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 89858+0+0 (secure 0 0 0) 0x7f2eec00f5d0 con 0x7f2efc0ff150 2024-02-16T19:23:22.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:23:22.614+0000 7f2ef97fa700 1 --2- 172.21.15.22:0/881539366 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f2ee806c600 0x7f2ee806eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:23:22.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:23:22.615+0000 7f2ef97fa700 1 -- 172.21.15.22:0/881539366 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2414+0+0 (secure 0 0 0) 0x7f2eec08be40 con 0x7f2efc0ff150 2024-02-16T19:23:22.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:23:22.615+0000 7f2efb7fe700 1 --2- 172.21.15.22:0/881539366 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f2ee806c600 0x7f2ee806eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:23:22.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:23:22.615+0000 7f2efb7fe700 1 --2- 172.21.15.22:0/881539366 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f2ee806c600 0x7f2ee806eab0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2ee4005950 tx=0x7f2ee400b410 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:23:22.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:23:22.623+0000 7f2ef97fa700 1 -- 172.21.15.22:0/881539366 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f2eec05ac80 con 0x7f2efc0ff150 2024-02-16T19:23:22.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:23:22.849+0000 7f2f02758700 1 -- 172.21.15.22:0/881539366 --> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f2efc0694c0 con 0x7f2ee806c600 2024-02-16T19:23:23.331 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:23 smithi117 ceph-mon[35309]: Detected new or changed devices on smithi022 2024-02-16T19:23:23.331 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:23 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:23:23.331 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:23 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:23:23.331 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:23 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:23:23.331 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:23 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:23:23.331 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:23 smithi117 ceph-mon[35309]: Adjusting osd_memory_target on smithi022 to 5922M 2024-02-16T19:23:23.331 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:23 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:23:23.331 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:23 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:23:23.331 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:23 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:23:23.332 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:23 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:23:23.332 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:23 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:23:23.332 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:23 smithi117 ceph-mon[35309]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-16T19:23:23.332 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:23 smithi117 ceph-mon[35309]: from='client.14290 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:23:23.332 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:23 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-16T19:23:23.332 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:23 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-16T19:23:23.332 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:23 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:23:23.462 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:23 smithi022 ceph-mon[29221]: Detected new or changed devices on smithi022 2024-02-16T19:23:23.462 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:23 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:23:23.463 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:23 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:23:23.463 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:23 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:23:23.463 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:23 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:23:23.463 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:23 smithi022 ceph-mon[29221]: Adjusting osd_memory_target on smithi022 to 5922M 2024-02-16T19:23:23.463 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:23 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:23:23.463 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:23 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:23:23.463 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:23 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:23:23.463 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:23 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:23:23.463 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:23 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:23:23.463 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:23 smithi022 ceph-mon[29221]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-16T19:23:23.463 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:23 smithi022 ceph-mon[29221]: from='client.14290 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:23:23.463 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:23 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-16T19:23:23.463 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:23 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-16T19:23:23.464 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:23 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:23:25.432 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:25 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:23:25.432 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:25 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:23:25.433 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:25 smithi022 ceph-mon[29221]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-16T19:23:25.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:25 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:23:25.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:25 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:23:25.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:25 smithi117 ceph-mon[35309]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-16T19:23:27.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:27 smithi022 ceph-mon[29221]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-16T19:23:27.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:27 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:23:27.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:27 smithi022 ceph-mon[29221]: from='client.? 172.21.15.22:0/3713386484' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5952e710-f0b6-406c-855a-00f1cb8059c6"}]: dispatch 2024-02-16T19:23:27.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:27 smithi022 ceph-mon[29221]: from='client.? 172.21.15.22:0/3713386484' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5952e710-f0b6-406c-855a-00f1cb8059c6"}]': finished 2024-02-16T19:23:27.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:27 smithi022 ceph-mon[29221]: osdmap e16: 3 total, 2 up, 3 in 2024-02-16T19:23:27.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:27 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-16T19:23:27.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:27 smithi022 ceph-mon[29221]: from='client.? 172.21.15.22:0/512771349' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-16T19:23:28.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:27 smithi117 ceph-mon[35309]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-16T19:23:28.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:27 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:23:28.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:27 smithi117 ceph-mon[35309]: from='client.? 172.21.15.22:0/3713386484' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5952e710-f0b6-406c-855a-00f1cb8059c6"}]: dispatch 2024-02-16T19:23:28.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:27 smithi117 ceph-mon[35309]: from='client.? 172.21.15.22:0/3713386484' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5952e710-f0b6-406c-855a-00f1cb8059c6"}]': finished 2024-02-16T19:23:28.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:27 smithi117 ceph-mon[35309]: osdmap e16: 3 total, 2 up, 3 in 2024-02-16T19:23:28.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:27 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-16T19:23:28.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:27 smithi117 ceph-mon[35309]: from='client.? 172.21.15.22:0/512771349' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-16T19:23:29.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:28 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:23:29.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:28 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:23:29.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:28 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:23:29.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:28 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:23:29.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:28 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:23:29.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:28 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:23:29.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:28 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:23:29.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:28 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:23:29.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:28 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:23:29.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:28 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:23:30.131 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:29 smithi022 ceph-mon[29221]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-16T19:23:30.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:29 smithi117 ceph-mon[35309]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-16T19:23:32.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:31 smithi117 ceph-mon[35309]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-16T19:23:32.204 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:31 smithi022 ceph-mon[29221]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-16T19:23:34.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:33 smithi117 ceph-mon[35309]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-16T19:23:34.198 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:33 smithi022 ceph-mon[29221]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-16T19:23:34.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:34 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-16T19:23:34.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:34 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:23:35.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:34 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-16T19:23:35.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:34 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:23:35.916 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:35 smithi022 ceph-mon[29221]: Deploying daemon osd.2 on smithi022 2024-02-16T19:23:35.916 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:35 smithi022 ceph-mon[29221]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-16T19:23:36.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:35 smithi117 ceph-mon[35309]: Deploying daemon osd.2 on smithi022 2024-02-16T19:23:36.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:35 smithi117 ceph-mon[35309]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-16T19:23:38.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:37 smithi022 ceph-mon[29221]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-16T19:23:38.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:37 smithi117 ceph-mon[35309]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-16T19:23:40.111 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:39 smithi022 ceph-mon[29221]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-16T19:23:40.111 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:39 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:23:40.111 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:39 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:23:40.111 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:39 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:23:40.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:39 smithi117 ceph-mon[35309]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-16T19:23:40.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:39 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:23:40.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:39 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:23:40.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:39 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:23:42.091 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:41 smithi022 ceph-mon[29221]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-16T19:23:42.091 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:41 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:23:42.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:41 smithi117 ceph-mon[35309]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-16T19:23:42.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:41 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:23:42.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:23:42.805+0000 7f2ef97fa700 1 -- 172.21.15.22:0/881539366 <== mgr.14217 v2:172.21.15.22:6800/2253867279 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f2efc0694c0 con 0x7f2ee806c600 2024-02-16T19:23:42.807 INFO:teuthology.orchestra.run.smithi022.stdout:Created osd(s) 2 on host 'smithi022' 2024-02-16T19:23:42.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:23:42.808+0000 7f2f02758700 1 -- 172.21.15.22:0/881539366 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f2ee806c600 msgr2=0x7f2ee806eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:23:42.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:23:42.808+0000 7f2f02758700 1 --2- 172.21.15.22:0/881539366 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f2ee806c600 0x7f2ee806eab0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2ee4005950 tx=0x7f2ee400b410 comp rx=0 tx=0).stop 2024-02-16T19:23:42.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:23:42.808+0000 7f2f02758700 1 -- 172.21.15.22:0/881539366 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2efc0ff150 msgr2=0x7f2efc195ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:23:42.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:23:42.808+0000 7f2f02758700 1 --2- 172.21.15.22:0/881539366 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2efc0ff150 0x7f2efc195ac0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2eec00bda0 tx=0x7f2eec00bdd0 comp rx=0 tx=0).stop 2024-02-16T19:23:42.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:23:42.808+0000 7f2f02758700 1 -- 172.21.15.22:0/881539366 shutdown_connections 2024-02-16T19:23:42.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:23:42.808+0000 7f2f02758700 1 --2- 172.21.15.22:0/881539366 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2efc0ff150 0x7f2efc195ac0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:23:42.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:23:42.808+0000 7f2f02758700 1 --2- 172.21.15.22:0/881539366 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f2ee806c600 0x7f2ee806eab0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:23:42.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:23:42.808+0000 7f2f02758700 1 --2- 172.21.15.22:0/881539366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2efc100350 0x7f2efc196000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:23:42.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:23:42.808+0000 7f2f02758700 1 -- 172.21.15.22:0/881539366 >> 172.21.15.22:0/881539366 conn(0x7f2efc0fa700 msgr2=0x7f2efc068db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:23:42.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:23:42.808+0000 7f2f02758700 1 -- 172.21.15.22:0/881539366 shutdown_connections 2024-02-16T19:23:42.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:23:42.808+0000 7f2f02758700 1 -- 172.21.15.22:0/881539366 wait complete. 2024-02-16T19:23:43.486 DEBUG:teuthology.orchestra.run.smithi022:osd.2> sudo journalctl -f -n 0 -u ceph-2a1346e6-cd00-11ee-95ba-87774f69a715@osd.2.service 2024-02-16T19:23:43.488 INFO:tasks.cephadm:Deploying osd.3 on smithi117 with /dev/vg_nvme/lv_4... 2024-02-16T19:23:43.488 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-02-16T19:23:43.701 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi117/config 2024-02-16T19:23:43.715 INFO:journalctl@ceph.osd.2.smithi022.stdout:Feb 16 19:23:43 smithi022 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-2[55140]: 2024-02-16T19:23:43.520+0000 7ff8f71ae640 -1 osd.2 0 log_to_monitors true 2024-02-16T19:23:44.045 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:43 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:23:44.045 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:43 smithi022 ceph-mon[29221]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-16T19:23:44.046 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:43 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:23:44.046 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:43 smithi022 ceph-mon[29221]: from='osd.2 [v2:172.21.15.22:6818/3627861698,v1:172.21.15.22:6819/3627861698]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-16T19:23:44.099 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:43 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:23:44.100 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:43 smithi117 ceph-mon[35309]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-16T19:23:44.100 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:43 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:23:44.100 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:43 smithi117 ceph-mon[35309]: from='osd.2 [v2:172.21.15.22:6818/3627861698,v1:172.21.15.22:6819/3627861698]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-16T19:23:44.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:44 smithi022 ceph-mon[29221]: from='osd.2 [v2:172.21.15.22:6818/3627861698,v1:172.21.15.22:6819/3627861698]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-16T19:23:44.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:44 smithi022 ceph-mon[29221]: osdmap e17: 3 total, 2 up, 3 in 2024-02-16T19:23:44.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:44 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-16T19:23:44.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:44 smithi022 ceph-mon[29221]: from='osd.2 [v2:172.21.15.22:6818/3627861698,v1:172.21.15.22:6819/3627861698]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-02-16T19:23:44.965 INFO:journalctl@ceph.osd.2.smithi022.stdout:Feb 16 19:23:44 smithi022 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-2[55140]: 2024-02-16T19:23:44.793+0000 7ff8ec024700 -1 osd.2 0 waiting for initial osdmap 2024-02-16T19:23:44.965 INFO:journalctl@ceph.osd.2.smithi022.stdout:Feb 16 19:23:44 smithi022 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-2[55140]: 2024-02-16T19:23:44.799+0000 7ff8e5614700 -1 osd.2 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-16T19:23:45.002 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:44 smithi117 ceph-mon[35309]: from='osd.2 [v2:172.21.15.22:6818/3627861698,v1:172.21.15.22:6819/3627861698]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-16T19:23:45.002 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:44 smithi117 ceph-mon[35309]: osdmap e17: 3 total, 2 up, 3 in 2024-02-16T19:23:45.003 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:44 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-16T19:23:45.003 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:44 smithi117 ceph-mon[35309]: from='osd.2 [v2:172.21.15.22:6818/3627861698,v1:172.21.15.22:6819/3627861698]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-02-16T19:23:45.485 INFO:teuthology.orchestra.run.smithi117.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-02-16T19:23:45.485 INFO:teuthology.orchestra.run.smithi117.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-02-16T19:23:45.485 INFO:teuthology.orchestra.run.smithi117.stderr: stderr: 10+0 records in 2024-02-16T19:23:45.485 INFO:teuthology.orchestra.run.smithi117.stderr:10+0 records out 2024-02-16T19:23:45.486 INFO:teuthology.orchestra.run.smithi117.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0122913 s, 853 MB/s 2024-02-16T19:23:45.486 INFO:teuthology.orchestra.run.smithi117.stderr:--> Zapping successful for: 2024-02-16T19:23:46.014 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi117:vg_nvme/lv_4 2024-02-16T19:23:46.133 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:45 smithi022 ceph-mon[29221]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-16T19:23:46.134 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:45 smithi022 ceph-mon[29221]: from='osd.2 [v2:172.21.15.22:6818/3627861698,v1:172.21.15.22:6819/3627861698]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-02-16T19:23:46.134 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:45 smithi022 ceph-mon[29221]: osdmap e18: 3 total, 2 up, 3 in 2024-02-16T19:23:46.134 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:45 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-16T19:23:46.134 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:45 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-16T19:23:46.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:45 smithi117 ceph-mon[35309]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-16T19:23:46.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:45 smithi117 ceph-mon[35309]: from='osd.2 [v2:172.21.15.22:6818/3627861698,v1:172.21.15.22:6819/3627861698]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-02-16T19:23:46.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:45 smithi117 ceph-mon[35309]: osdmap e18: 3 total, 2 up, 3 in 2024-02-16T19:23:46.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:45 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-16T19:23:46.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:45 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-16T19:23:46.232 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi117/config 2024-02-16T19:23:46.847 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:46 smithi117 ceph-mon[35309]: purged_snaps scrub starts 2024-02-16T19:23:46.848 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:46 smithi117 ceph-mon[35309]: purged_snaps scrub ok 2024-02-16T19:23:46.848 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:46 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-16T19:23:46.848 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:46 smithi117 ceph-mon[35309]: osd.2 [v2:172.21.15.22:6818/3627861698,v1:172.21.15.22:6819/3627861698] boot 2024-02-16T19:23:46.848 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:46 smithi117 ceph-mon[35309]: osdmap e19: 3 total, 3 up, 3 in 2024-02-16T19:23:46.848 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:46 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-16T19:23:46.848 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:46 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:23:46.848 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:46 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:23:47.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:46 smithi022 ceph-mon[29221]: purged_snaps scrub starts 2024-02-16T19:23:47.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:46 smithi022 ceph-mon[29221]: purged_snaps scrub ok 2024-02-16T19:23:47.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:46 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-16T19:23:47.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:46 smithi022 ceph-mon[29221]: osd.2 [v2:172.21.15.22:6818/3627861698,v1:172.21.15.22:6819/3627861698] boot 2024-02-16T19:23:47.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:46 smithi022 ceph-mon[29221]: osdmap e19: 3 total, 3 up, 3 in 2024-02-16T19:23:47.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:46 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-16T19:23:47.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:46 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:23:47.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:46 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:23:47.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:23:47.736+0000 7f8adca3e700 1 -- 172.21.15.117:0/2274922576 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ad80ff150 msgr2=0x7f8ad80ff560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:23:47.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:23:47.736+0000 7f8adca3e700 1 --2- 172.21.15.117:0/2274922576 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ad80ff150 0x7f8ad80ff560 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f8ac8009910 tx=0x7f8ac8009c20 comp rx=0 tx=0).stop 2024-02-16T19:23:47.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:23:47.737+0000 7f8adca3e700 1 -- 172.21.15.117:0/2274922576 shutdown_connections 2024-02-16T19:23:47.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:23:47.737+0000 7f8adca3e700 1 --2- 172.21.15.117:0/2274922576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8ad81003f0 0x7f8ad8100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:23:47.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:23:47.737+0000 7f8adca3e700 1 --2- 172.21.15.117:0/2274922576 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ad80ff150 0x7f8ad80ff560 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:23:47.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:23:47.737+0000 7f8adca3e700 1 -- 172.21.15.117:0/2274922576 >> 172.21.15.117:0/2274922576 conn(0x7f8ad80fa7a0 msgr2=0x7f8ad80fcbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:23:47.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:23:47.737+0000 7f8adca3e700 1 -- 172.21.15.117:0/2274922576 shutdown_connections 2024-02-16T19:23:47.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:23:47.737+0000 7f8adca3e700 1 -- 172.21.15.117:0/2274922576 wait complete. 2024-02-16T19:23:47.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:23:47.737+0000 7f8adca3e700 1 Processor -- start 2024-02-16T19:23:47.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:23:47.737+0000 7f8adca3e700 1 -- start start 2024-02-16T19:23:47.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:23:47.737+0000 7f8adca3e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ad80ff150 0x7f8ad8069930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:23:47.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:23:47.737+0000 7f8adca3e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8ad81003f0 0x7f8ad819cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:23:47.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:23:47.737+0000 7f8adca3e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad8196910 con 0x7f8ad81003f0 2024-02-16T19:23:47.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:23:47.737+0000 7f8adca3e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad819d3e0 con 0x7f8ad80ff150 2024-02-16T19:23:47.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:23:47.738+0000 7f8ad659c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ad80ff150 0x7f8ad8069930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:23:47.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:23:47.738+0000 7f8ad659c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ad80ff150 0x7f8ad8069930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:35192/0 (socket says 172.21.15.117:35192) 2024-02-16T19:23:47.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:23:47.738+0000 7f8ad659c700 1 -- 172.21.15.117:0/164413716 learned_addr learned my addr 172.21.15.117:0/164413716 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-02-16T19:23:47.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:23:47.738+0000 7f8ad5d9b700 1 --2- 172.21.15.117:0/164413716 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8ad81003f0 0x7f8ad819cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:23:47.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:23:47.738+0000 7f8ad659c700 1 -- 172.21.15.117:0/164413716 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8ad81003f0 msgr2=0x7f8ad819cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:23:47.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:23:47.738+0000 7f8ad659c700 1 --2- 172.21.15.117:0/164413716 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8ad81003f0 0x7f8ad819cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:23:47.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:23:47.738+0000 7f8ad659c700 1 -- 172.21.15.117:0/164413716 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ac80095f0 con 0x7f8ad80ff150 2024-02-16T19:23:47.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:23:47.738+0000 7f8ad659c700 1 --2- 172.21.15.117:0/164413716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ad80ff150 0x7f8ad8069930 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f8ac8000c00 tx=0x7f8ac800f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:23:47.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:23:47.738+0000 7f8ac77fe700 1 -- 172.21.15.117:0/164413716 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ac801d070 con 0x7f8ad80ff150 2024-02-16T19:23:47.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:23:47.738+0000 7f8ac77fe700 1 -- 172.21.15.117:0/164413716 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8ac8005dc0 con 0x7f8ad80ff150 2024-02-16T19:23:47.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:23:47.738+0000 7f8ac77fe700 1 -- 172.21.15.117:0/164413716 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ac80226a0 con 0x7f8ad80ff150 2024-02-16T19:23:47.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:23:47.738+0000 7f8adca3e700 1 -- 172.21.15.117:0/164413716 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ad819d630 con 0x7f8ad80ff150 2024-02-16T19:23:47.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:23:47.738+0000 7f8adca3e700 1 -- 172.21.15.117:0/164413716 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ad819dad0 con 0x7f8ad80ff150 2024-02-16T19:23:47.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:23:47.739+0000 7f8adca3e700 1 -- 172.21.15.117:0/164413716 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ad8068ab0 con 0x7f8ad80ff150 2024-02-16T19:23:47.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:23:47.740+0000 7f8ac77fe700 1 -- 172.21.15.117:0/164413716 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) v1 ==== 89858+0+0 (secure 0 0 0) 0x7f8ac800faf0 con 0x7f8ad80ff150 2024-02-16T19:23:47.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:23:47.740+0000 7f8ac77fe700 1 --2- 172.21.15.117:0/164413716 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f8ac006c610 0x7f8ac006eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:23:47.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:23:47.740+0000 7f8ac77fe700 1 -- 172.21.15.117:0/164413716 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 2846+0+0 (secure 0 0 0) 0x7f8ac808c210 con 0x7f8ad80ff150 2024-02-16T19:23:47.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:23:47.740+0000 7f8ad5d9b700 1 --2- 172.21.15.117:0/164413716 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f8ac006c610 0x7f8ac006eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:23:47.743 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:23:47.740+0000 7f8ad5d9b700 1 --2- 172.21.15.117:0/164413716 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f8ac006c610 0x7f8ac006eac0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8069eb0 tx=0x7f8acc009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:23:47.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:23:47.745+0000 7f8ac77fe700 1 -- 172.21.15.117:0/164413716 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f8ac805aea0 con 0x7f8ad80ff150 2024-02-16T19:23:47.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:23:47.943+0000 7f8adca3e700 1 -- 172.21.15.117:0/164413716 --> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f8ad8104e40 con 0x7f8ac006c610 2024-02-16T19:23:48.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:47 smithi117 ceph-mon[35309]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-16T19:23:48.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:47 smithi117 ceph-mon[35309]: osdmap e20: 3 total, 3 up, 3 in 2024-02-16T19:23:48.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:47 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-16T19:23:48.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:47 smithi022 ceph-mon[29221]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-16T19:23:48.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:47 smithi022 ceph-mon[29221]: osdmap e20: 3 total, 3 up, 3 in 2024-02-16T19:23:48.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:47 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-16T19:23:49.033 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:48 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-16T19:23:49.033 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:48 smithi117 ceph-mon[35309]: osdmap e21: 3 total, 3 up, 3 in 2024-02-16T19:23:49.033 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:48 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-16T19:23:49.033 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:48 smithi117 ceph-mon[35309]: from='client.24137 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:23:49.033 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:48 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-16T19:23:49.033 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:48 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-16T19:23:49.033 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:48 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:23:49.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:48 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-16T19:23:49.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:48 smithi022 ceph-mon[29221]: osdmap e21: 3 total, 3 up, 3 in 2024-02-16T19:23:49.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:48 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-16T19:23:49.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:48 smithi022 ceph-mon[29221]: from='client.24137 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:23:49.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:48 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-16T19:23:49.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:48 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-16T19:23:49.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:48 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:23:49.816 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:49 smithi117 ceph-mon[35309]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2024-02-16T19:23:49.816 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:49 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-16T19:23:49.816 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:49 smithi117 ceph-mon[35309]: osdmap e22: 3 total, 3 up, 3 in 2024-02-16T19:23:49.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:23:49.814+0000 7f8ac77fe700 1 -- 172.21.15.117:0/164413716 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f8ac80577e0 con 0x7f8ad80ff150 2024-02-16T19:23:49.945 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:49 smithi022 ceph-mon[29221]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2024-02-16T19:23:49.945 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:49 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-16T19:23:49.945 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:49 smithi022 ceph-mon[29221]: osdmap e22: 3 total, 3 up, 3 in 2024-02-16T19:23:49.945 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 16 19:23:49 smithi022 sudo[57481]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-16T19:23:49.946 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 16 19:23:49 smithi022 sudo[57481]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-16T19:23:49.946 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 16 19:23:49 smithi022 sudo[57481]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-16T19:23:49.946 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 16 19:23:49 smithi022 sudo[57481]: pam_unix(sudo:session): session closed for user root 2024-02-16T19:23:50.214 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 16 19:23:49 smithi022 sudo[57484]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-16T19:23:50.215 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 16 19:23:49 smithi022 sudo[57484]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-16T19:23:50.215 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 16 19:23:49 smithi022 sudo[57484]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-16T19:23:50.215 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 16 19:23:49 smithi022 sudo[57484]: pam_unix(sudo:session): session closed for user root 2024-02-16T19:23:50.499 INFO:journalctl@ceph.osd.1.smithi022.stdout:Feb 16 19:23:50 smithi022 sudo[57585]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-16T19:23:50.499 INFO:journalctl@ceph.osd.1.smithi022.stdout:Feb 16 19:23:50 smithi022 sudo[57585]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-16T19:23:50.499 INFO:journalctl@ceph.osd.1.smithi022.stdout:Feb 16 19:23:50 smithi022 sudo[57585]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-16T19:23:50.499 INFO:journalctl@ceph.osd.1.smithi022.stdout:Feb 16 19:23:50 smithi022 sudo[57585]: pam_unix(sudo:session): session closed for user root 2024-02-16T19:23:50.753 INFO:journalctl@ceph.osd.1.smithi022.stdout:Feb 16 19:23:50 smithi022 sudo[57638]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-16T19:23:50.753 INFO:journalctl@ceph.osd.1.smithi022.stdout:Feb 16 19:23:50 smithi022 sudo[57638]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-16T19:23:50.753 INFO:journalctl@ceph.osd.1.smithi022.stdout:Feb 16 19:23:50 smithi022 sudo[57638]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-16T19:23:50.753 INFO:journalctl@ceph.osd.1.smithi022.stdout:Feb 16 19:23:50 smithi022 sudo[57638]: pam_unix(sudo:session): session closed for user root 2024-02-16T19:23:51.030 INFO:journalctl@ceph.osd.2.smithi022.stdout:Feb 16 19:23:50 smithi022 sudo[57683]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-16T19:23:51.030 INFO:journalctl@ceph.osd.2.smithi022.stdout:Feb 16 19:23:50 smithi022 sudo[57683]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-16T19:23:51.030 INFO:journalctl@ceph.osd.2.smithi022.stdout:Feb 16 19:23:50 smithi022 sudo[57683]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-16T19:23:51.030 INFO:journalctl@ceph.osd.2.smithi022.stdout:Feb 16 19:23:50 smithi022 sudo[57683]: pam_unix(sudo:session): session closed for user root 2024-02-16T19:23:51.030 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:50 smithi022 ceph-mon[29221]: mgrmap e19: smithi022.gmdcch(active, since 2m), standbys: smithi117.cohbtv 2024-02-16T19:23:51.030 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:50 smithi022 ceph-mon[29221]: osdmap e23: 3 total, 3 up, 3 in 2024-02-16T19:23:51.030 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:50 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:23:51.031 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:50 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:23:51.031 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:50 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:23:51.031 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:50 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:23:51.031 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:50 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:23:51.031 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:50 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:23:51.031 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:50 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:23:51.031 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:50 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:23:51.031 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:50 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:23:51.031 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:50 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:23:51.031 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:50 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:23:51.031 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:50 smithi022 ceph-mon[29221]: from='client.? 172.21.15.117:0/717284274' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5dc87494-e400-4111-91a7-306d9905febb"}]: dispatch 2024-02-16T19:23:51.031 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:50 smithi022 ceph-mon[29221]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5dc87494-e400-4111-91a7-306d9905febb"}]: dispatch 2024-02-16T19:23:51.031 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:50 smithi022 ceph-mon[29221]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5dc87494-e400-4111-91a7-306d9905febb"}]': finished 2024-02-16T19:23:51.032 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:50 smithi022 ceph-mon[29221]: osdmap e24: 4 total, 3 up, 4 in 2024-02-16T19:23:51.032 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:50 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-16T19:23:51.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:50 smithi117 ceph-mon[35309]: mgrmap e19: smithi022.gmdcch(active, since 2m), standbys: smithi117.cohbtv 2024-02-16T19:23:51.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:50 smithi117 ceph-mon[35309]: osdmap e23: 3 total, 3 up, 3 in 2024-02-16T19:23:51.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:50 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:23:51.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:50 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:23:51.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:50 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:23:51.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:50 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:23:51.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:50 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:23:51.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:50 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:23:51.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:50 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:23:51.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:50 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:23:51.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:50 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:23:51.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:50 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:23:51.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:50 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:23:51.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:50 smithi117 ceph-mon[35309]: from='client.? 172.21.15.117:0/717284274' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5dc87494-e400-4111-91a7-306d9905febb"}]: dispatch 2024-02-16T19:23:51.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:50 smithi117 ceph-mon[35309]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5dc87494-e400-4111-91a7-306d9905febb"}]: dispatch 2024-02-16T19:23:51.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:50 smithi117 ceph-mon[35309]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5dc87494-e400-4111-91a7-306d9905febb"}]': finished 2024-02-16T19:23:51.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:50 smithi117 ceph-mon[35309]: osdmap e24: 4 total, 3 up, 4 in 2024-02-16T19:23:51.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:50 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-16T19:23:51.292 INFO:journalctl@ceph.osd.2.smithi022.stdout:Feb 16 19:23:51 smithi022 sudo[57700]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-16T19:23:51.292 INFO:journalctl@ceph.osd.2.smithi022.stdout:Feb 16 19:23:51 smithi022 sudo[57700]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-16T19:23:51.293 INFO:journalctl@ceph.osd.2.smithi022.stdout:Feb 16 19:23:51 smithi022 sudo[57700]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-16T19:23:51.293 INFO:journalctl@ceph.osd.2.smithi022.stdout:Feb 16 19:23:51 smithi022 sudo[57700]: pam_unix(sudo:session): session closed for user root 2024-02-16T19:23:51.556 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:51 smithi022 sudo[57757]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-16T19:23:51.557 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:51 smithi022 sudo[57757]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-16T19:23:51.557 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:51 smithi022 sudo[57757]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-16T19:23:51.557 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:51 smithi022 sudo[57757]: pam_unix(sudo:session): session closed for user root 2024-02-16T19:23:51.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:51 smithi022 sudo[57816]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-16T19:23:51.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:51 smithi022 sudo[57816]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-16T19:23:51.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:51 smithi022 sudo[57816]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-16T19:23:51.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:51 smithi022 sudo[57816]: pam_unix(sudo:session): session closed for user root 2024-02-16T19:23:52.072 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:51 smithi117 sudo[39233]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-16T19:23:52.072 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:51 smithi117 sudo[39233]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-16T19:23:52.072 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:51 smithi117 sudo[39233]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-16T19:23:52.072 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:51 smithi117 sudo[39233]: pam_unix(sudo:session): session closed for user root 2024-02-16T19:23:52.384 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:52 smithi022 ceph-mon[29221]: Detected new or changed devices on smithi022 2024-02-16T19:23:52.384 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:52 smithi022 ceph-mon[29221]: Adjusting osd_memory_target on smithi022 to 3948M 2024-02-16T19:23:52.384 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:52 smithi022 ceph-mon[29221]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2024-02-16T19:23:52.384 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:52 smithi022 ceph-mon[29221]: from='client.? 172.21.15.117:0/3554691755' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-16T19:23:52.384 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:52 smithi022 ceph-mon[29221]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-16T19:23:52.384 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:52 smithi022 ceph-mon[29221]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-16T19:23:52.384 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:52 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mon metadata", "id": "smithi022"}]: dispatch 2024-02-16T19:23:52.384 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:52 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mon metadata", "id": "smithi117"}]: dispatch 2024-02-16T19:23:52.385 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:52 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mon metadata", "id": "smithi022"}]: dispatch 2024-02-16T19:23:52.385 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:52 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mon metadata", "id": "smithi117"}]: dispatch 2024-02-16T19:23:52.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:52 smithi117 sudo[39421]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-16T19:23:52.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:52 smithi117 sudo[39421]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-16T19:23:52.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:52 smithi117 sudo[39421]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-16T19:23:52.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:52 smithi117 sudo[39421]: pam_unix(sudo:session): session closed for user root 2024-02-16T19:23:52.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:52 smithi117 ceph-mon[35309]: Detected new or changed devices on smithi022 2024-02-16T19:23:52.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:52 smithi117 ceph-mon[35309]: Adjusting osd_memory_target on smithi022 to 3948M 2024-02-16T19:23:52.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:52 smithi117 ceph-mon[35309]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2024-02-16T19:23:52.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:52 smithi117 ceph-mon[35309]: from='client.? 172.21.15.117:0/3554691755' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-16T19:23:52.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:52 smithi117 ceph-mon[35309]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-16T19:23:52.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:52 smithi117 ceph-mon[35309]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-16T19:23:52.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:52 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mon metadata", "id": "smithi022"}]: dispatch 2024-02-16T19:23:52.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:52 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mon metadata", "id": "smithi117"}]: dispatch 2024-02-16T19:23:52.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:52 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mon metadata", "id": "smithi022"}]: dispatch 2024-02-16T19:23:52.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:52 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mon metadata", "id": "smithi117"}]: dispatch 2024-02-16T19:23:53.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:53 smithi117 ceph-mon[35309]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-16T19:23:53.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:53 smithi117 ceph-mon[35309]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-16T19:23:53.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:53 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:23:53.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:53 smithi117 ceph-mon[35309]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-16T19:23:53.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:53 smithi022 ceph-mon[29221]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-16T19:23:53.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:53 smithi022 ceph-mon[29221]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-16T19:23:53.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:53 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:23:53.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:53 smithi022 ceph-mon[29221]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-16T19:23:55.712 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:55 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:23:55.712 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:55 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:23:55.712 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:55 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:23:55.712 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:55 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:23:55.712 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:55 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:23:55.712 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:55 smithi117 ceph-mon[35309]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-16T19:23:55.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:55 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:23:55.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:55 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:23:55.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:55 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:23:55.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:55 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:23:55.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:55 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:23:55.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:55 smithi022 ceph-mon[29221]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-16T19:23:58.110 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:57 smithi117 ceph-mon[35309]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-16T19:23:58.110 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:57 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:23:58.110 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:57 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-16T19:23:58.110 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:57 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:23:58.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:57 smithi022 ceph-mon[29221]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-16T19:23:58.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:57 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:23:58.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:57 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-16T19:23:58.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:57 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:23:58.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:58 smithi022 ceph-mon[29221]: Deploying daemon osd.3 on smithi117 2024-02-16T19:23:59.125 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:58 smithi117 ceph-mon[35309]: Deploying daemon osd.3 on smithi117 2024-02-16T19:24:00.117 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:23:59 smithi117 ceph-mon[35309]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-02-16T19:24:00.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:23:59 smithi022 ceph-mon[29221]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-02-16T19:24:02.082 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:01 smithi117 ceph-mon[35309]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-02-16T19:24:02.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:01 smithi022 ceph-mon[29221]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-02-16T19:24:03.775 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:03 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:24:03.776 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:03 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:24:03.776 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:03 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:24:03.776 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:03 smithi117 ceph-mon[35309]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-02-16T19:24:03.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:03 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:24:03.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:03 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:24:03.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:03 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:24:03.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:03 smithi022 ceph-mon[29221]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-02-16T19:24:05.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:05.203+0000 7f8ac77fe700 1 -- 172.21.15.117:0/164413716 <== mgr.14217 v2:172.21.15.22:6800/2253867279 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f8ad8104e40 con 0x7f8ac006c610 2024-02-16T19:24:05.207 INFO:teuthology.orchestra.run.smithi117.stdout:Created osd(s) 3 on host 'smithi117' 2024-02-16T19:24:05.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:05.205+0000 7f8adca3e700 1 -- 172.21.15.117:0/164413716 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f8ac006c610 msgr2=0x7f8ac006eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:24:05.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:05.205+0000 7f8adca3e700 1 --2- 172.21.15.117:0/164413716 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f8ac006c610 0x7f8ac006eac0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8069eb0 tx=0x7f8acc009260 comp rx=0 tx=0).stop 2024-02-16T19:24:05.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:05.205+0000 7f8adca3e700 1 -- 172.21.15.117:0/164413716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ad80ff150 msgr2=0x7f8ad8069930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:24:05.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:05.205+0000 7f8adca3e700 1 --2- 172.21.15.117:0/164413716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ad80ff150 0x7f8ad8069930 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f8ac8000c00 tx=0x7f8ac800f690 comp rx=0 tx=0).stop 2024-02-16T19:24:05.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:05.205+0000 7f8adca3e700 1 -- 172.21.15.117:0/164413716 shutdown_connections 2024-02-16T19:24:05.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:05.205+0000 7f8adca3e700 1 --2- 172.21.15.117:0/164413716 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f8ac006c610 0x7f8ac006eac0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:05.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:05.205+0000 7f8adca3e700 1 --2- 172.21.15.117:0/164413716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ad80ff150 0x7f8ad8069930 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:05.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:05.205+0000 7f8adca3e700 1 --2- 172.21.15.117:0/164413716 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8ad81003f0 0x7f8ad819cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:05.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:05.205+0000 7f8adca3e700 1 -- 172.21.15.117:0/164413716 >> 172.21.15.117:0/164413716 conn(0x7f8ad80fa7a0 msgr2=0x7f8ad81036c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:24:05.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:05.205+0000 7f8adca3e700 1 -- 172.21.15.117:0/164413716 shutdown_connections 2024-02-16T19:24:05.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:05.205+0000 7f8adca3e700 1 -- 172.21.15.117:0/164413716 wait complete. 2024-02-16T19:24:06.060 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:05 smithi117 ceph-mon[35309]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-02-16T19:24:06.060 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:05 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:24:06.060 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:05 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:24:06.081 DEBUG:teuthology.orchestra.run.smithi117:osd.3> sudo journalctl -f -n 0 -u ceph-2a1346e6-cd00-11ee-95ba-87774f69a715@osd.3.service 2024-02-16T19:24:06.083 INFO:tasks.cephadm:Deploying osd.4 on smithi117 with /dev/vg_nvme/lv_3... 2024-02-16T19:24:06.083 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-02-16T19:24:06.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:05 smithi022 ceph-mon[29221]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-02-16T19:24:06.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:05 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:24:06.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:05 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:24:06.637 INFO:journalctl@ceph.osd.3.smithi117.stdout:Feb 16 19:24:06 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-3[41233]: 2024-02-16T19:24:06.414+0000 7fc7547a9640 -1 osd.3 0 log_to_monitors true 2024-02-16T19:24:06.866 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi117/config 2024-02-16T19:24:07.102 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:06 smithi117 ceph-mon[35309]: from='osd.3 [v2:172.21.15.117:6800/2012503812,v1:172.21.15.117:6801/2012503812]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-16T19:24:07.102 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:06 smithi117 ceph-mon[35309]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-16T19:24:07.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:06 smithi022 ceph-mon[29221]: from='osd.3 [v2:172.21.15.117:6800/2012503812,v1:172.21.15.117:6801/2012503812]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-16T19:24:07.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:06 smithi022 ceph-mon[29221]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-16T19:24:07.940 INFO:journalctl@ceph.osd.3.smithi117.stdout:Feb 16 19:24:07 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-3[41233]: 2024-02-16T19:24:07.856+0000 7fc74961f700 -1 osd.3 0 waiting for initial osdmap 2024-02-16T19:24:07.940 INFO:journalctl@ceph.osd.3.smithi117.stdout:Feb 16 19:24:07 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-3[41233]: 2024-02-16T19:24:07.862+0000 7fc743410700 -1 osd.3 26 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-16T19:24:07.940 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:07 smithi117 ceph-mon[35309]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-02-16T19:24:07.941 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:07 smithi117 ceph-mon[35309]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-16T19:24:07.941 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:07 smithi117 ceph-mon[35309]: osdmap e25: 4 total, 3 up, 4 in 2024-02-16T19:24:07.941 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:07 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-16T19:24:07.941 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:07 smithi117 ceph-mon[35309]: from='osd.3 [v2:172.21.15.117:6800/2012503812,v1:172.21.15.117:6801/2012503812]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-02-16T19:24:07.941 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:07 smithi117 ceph-mon[35309]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-02-16T19:24:07.941 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:07 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:24:07.941 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:07 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:24:08.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:07 smithi022 ceph-mon[29221]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-02-16T19:24:08.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:07 smithi022 ceph-mon[29221]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-16T19:24:08.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:07 smithi022 ceph-mon[29221]: osdmap e25: 4 total, 3 up, 4 in 2024-02-16T19:24:08.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:07 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-16T19:24:08.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:07 smithi022 ceph-mon[29221]: from='osd.3 [v2:172.21.15.117:6800/2012503812,v1:172.21.15.117:6801/2012503812]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-02-16T19:24:08.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:07 smithi022 ceph-mon[29221]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-02-16T19:24:08.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:07 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:24:08.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:07 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:24:08.937 INFO:teuthology.orchestra.run.smithi117.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-02-16T19:24:08.937 INFO:teuthology.orchestra.run.smithi117.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-02-16T19:24:08.937 INFO:teuthology.orchestra.run.smithi117.stderr: stderr: 10+0 records in 2024-02-16T19:24:08.937 INFO:teuthology.orchestra.run.smithi117.stderr:10+0 records out 2024-02-16T19:24:08.937 INFO:teuthology.orchestra.run.smithi117.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0143151 s, 732 MB/s 2024-02-16T19:24:08.937 INFO:teuthology.orchestra.run.smithi117.stderr:--> Zapping successful for: 2024-02-16T19:24:09.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:08 smithi117 ceph-mon[35309]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-02-16T19:24:09.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:08 smithi117 ceph-mon[35309]: osdmap e26: 4 total, 3 up, 4 in 2024-02-16T19:24:09.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:08 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-16T19:24:09.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:08 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-16T19:24:09.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:08 smithi022 ceph-mon[29221]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-02-16T19:24:09.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:08 smithi022 ceph-mon[29221]: osdmap e26: 4 total, 3 up, 4 in 2024-02-16T19:24:09.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:08 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-16T19:24:09.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:08 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-16T19:24:09.883 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:09 smithi117 ceph-mon[35309]: purged_snaps scrub starts 2024-02-16T19:24:09.883 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:09 smithi117 ceph-mon[35309]: purged_snaps scrub ok 2024-02-16T19:24:09.883 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:09 smithi117 ceph-mon[35309]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-16T19:24:09.883 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:09 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-16T19:24:09.883 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:09 smithi117 ceph-mon[35309]: osd.3 [v2:172.21.15.117:6800/2012503812,v1:172.21.15.117:6801/2012503812] boot 2024-02-16T19:24:09.884 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:09 smithi117 ceph-mon[35309]: osdmap e27: 4 total, 4 up, 4 in 2024-02-16T19:24:09.884 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:09 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-16T19:24:09.885 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi117:vg_nvme/lv_3 2024-02-16T19:24:10.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:09 smithi022 ceph-mon[29221]: purged_snaps scrub starts 2024-02-16T19:24:10.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:09 smithi022 ceph-mon[29221]: purged_snaps scrub ok 2024-02-16T19:24:10.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:09 smithi022 ceph-mon[29221]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-16T19:24:10.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:09 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-16T19:24:10.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:09 smithi022 ceph-mon[29221]: osd.3 [v2:172.21.15.117:6800/2012503812,v1:172.21.15.117:6801/2012503812] boot 2024-02-16T19:24:10.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:09 smithi022 ceph-mon[29221]: osdmap e27: 4 total, 4 up, 4 in 2024-02-16T19:24:10.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:09 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-16T19:24:10.358 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi117/config 2024-02-16T19:24:11.130 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:10 smithi117 ceph-mon[35309]: osdmap e28: 4 total, 4 up, 4 in 2024-02-16T19:24:11.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:10 smithi022 ceph-mon[29221]: osdmap e28: 4 total, 4 up, 4 in 2024-02-16T19:24:12.120 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:11 smithi117 ceph-mon[35309]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-16T19:24:12.120 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:11 smithi117 ceph-mon[35309]: osdmap e29: 4 total, 4 up, 4 in 2024-02-16T19:24:12.120 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:11 smithi117 ceph-mon[35309]: osdmap e30: 4 total, 4 up, 4 in 2024-02-16T19:24:12.120 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:11 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:24:12.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:11 smithi022 ceph-mon[29221]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-16T19:24:12.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:11 smithi022 ceph-mon[29221]: osdmap e29: 4 total, 4 up, 4 in 2024-02-16T19:24:12.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:11 smithi022 ceph-mon[29221]: osdmap e30: 4 total, 4 up, 4 in 2024-02-16T19:24:12.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:11 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:24:12.430 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:12.429+0000 7fa6c11a6700 1 -- 172.21.15.117:0/1353752942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6bc102250 msgr2=0x7fa6bc0fe380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:24:12.430 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:12.429+0000 7fa6c11a6700 1 --2- 172.21.15.117:0/1353752942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6bc102250 0x7fa6bc0fe380 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa6b0009910 tx=0x7fa6b0009c20 comp rx=0 tx=0).stop 2024-02-16T19:24:12.430 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:12.429+0000 7fa6c11a6700 1 -- 172.21.15.117:0/1353752942 shutdown_connections 2024-02-16T19:24:12.431 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:12.429+0000 7fa6c11a6700 1 --2- 172.21.15.117:0/1353752942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6bc102250 0x7fa6bc0fe380 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:12.431 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:12.429+0000 7fa6c11a6700 1 --2- 172.21.15.117:0/1353752942 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6bc101900 0x7fa6bc101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:12.431 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:12.429+0000 7fa6c11a6700 1 -- 172.21.15.117:0/1353752942 >> 172.21.15.117:0/1353752942 conn(0x7fa6bc0f9f10 msgr2=0x7fa6bc0fc340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:24:12.431 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:12.429+0000 7fa6c11a6700 1 -- 172.21.15.117:0/1353752942 shutdown_connections 2024-02-16T19:24:12.431 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:12.429+0000 7fa6c11a6700 1 -- 172.21.15.117:0/1353752942 wait complete. 2024-02-16T19:24:12.432 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:12.430+0000 7fa6c11a6700 1 Processor -- start 2024-02-16T19:24:12.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:12.430+0000 7fa6c11a6700 1 -- start start 2024-02-16T19:24:12.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:12.430+0000 7fa6c11a6700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6bc101900 0x7fa6bc195b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:24:12.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:12.430+0000 7fa6c11a6700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6bc102250 0x7fa6bc1960c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:24:12.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:12.430+0000 7fa6c11a6700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6bc1966e0 con 0x7fa6bc101900 2024-02-16T19:24:12.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:12.430+0000 7fa6c11a6700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6bc196820 con 0x7fa6bc102250 2024-02-16T19:24:12.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:12.431+0000 7fa6ba59c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6bc102250 0x7fa6bc1960c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:24:12.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:12.431+0000 7fa6ba59c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6bc102250 0x7fa6bc1960c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:33420/0 (socket says 172.21.15.117:33420) 2024-02-16T19:24:12.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:12.431+0000 7fa6ba59c700 1 -- 172.21.15.117:0/3485681380 learned_addr learned my addr 172.21.15.117:0/3485681380 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-02-16T19:24:12.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:12.431+0000 7fa6bad9d700 1 --2- 172.21.15.117:0/3485681380 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6bc101900 0x7fa6bc195b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:24:12.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:12.431+0000 7fa6ba59c700 1 -- 172.21.15.117:0/3485681380 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6bc101900 msgr2=0x7fa6bc195b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:24:12.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:12.431+0000 7fa6ba59c700 1 --2- 172.21.15.117:0/3485681380 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6bc101900 0x7fa6bc195b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:12.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:12.431+0000 7fa6ba59c700 1 -- 172.21.15.117:0/3485681380 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6b00095f0 con 0x7fa6bc102250 2024-02-16T19:24:12.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:12.431+0000 7fa6ba59c700 1 --2- 172.21.15.117:0/3485681380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6bc102250 0x7fa6bc1960c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa6b000bd70 tx=0x7fa6b000be50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:24:12.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:12.431+0000 7fa6a3fff700 1 -- 172.21.15.117:0/3485681380 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa6b001d070 con 0x7fa6bc102250 2024-02-16T19:24:12.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:12.431+0000 7fa6a3fff700 1 -- 172.21.15.117:0/3485681380 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa6b0022470 con 0x7fa6bc102250 2024-02-16T19:24:12.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:12.431+0000 7fa6c11a6700 1 -- 172.21.15.117:0/3485681380 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6bc19b270 con 0x7fa6bc102250 2024-02-16T19:24:12.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:12.431+0000 7fa6a3fff700 1 -- 172.21.15.117:0/3485681380 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa6b000f670 con 0x7fa6bc102250 2024-02-16T19:24:12.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:12.431+0000 7fa6c11a6700 1 -- 172.21.15.117:0/3485681380 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6bc19b760 con 0x7fa6bc102250 2024-02-16T19:24:12.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:12.432+0000 7fa6c11a6700 1 -- 172.21.15.117:0/3485681380 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6bc108db0 con 0x7fa6bc102250 2024-02-16T19:24:12.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:12.433+0000 7fa6a3fff700 1 -- 172.21.15.117:0/3485681380 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fa6b0022a80 con 0x7fa6bc102250 2024-02-16T19:24:12.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:12.433+0000 7fa6a3fff700 1 --2- 172.21.15.117:0/3485681380 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fa6a406c680 0x7fa6a406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:24:12.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:12.433+0000 7fa6a3fff700 1 -- 172.21.15.117:0/3485681380 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 3774+0+0 (secure 0 0 0) 0x7fa6b008c810 con 0x7fa6bc102250 2024-02-16T19:24:12.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:12.433+0000 7fa6bad9d700 1 --2- 172.21.15.117:0/3485681380 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fa6a406c680 0x7fa6a406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:24:12.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:12.434+0000 7fa6bad9d700 1 --2- 172.21.15.117:0/3485681380 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fa6a406c680 0x7fa6a406eb30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac006fd0 tx=0x7fa6ac008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:24:12.438 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:12.438+0000 7fa6a3fff700 1 -- 172.21.15.117:0/3485681380 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fa6b005b0d0 con 0x7fa6bc102250 2024-02-16T19:24:12.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:12.632+0000 7fa6c11a6700 1 -- 172.21.15.117:0/3485681380 --> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fa6bc19bda0 con 0x7fa6a406c680 2024-02-16T19:24:13.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:12 smithi022 ceph-mon[29221]: Detected new or changed devices on smithi117 2024-02-16T19:24:13.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:12 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:24:13.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:12 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:24:13.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:12 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:24:13.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:12 smithi022 ceph-mon[29221]: Adjusting osd_memory_target on smithi117 to 14912M 2024-02-16T19:24:13.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:12 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:24:13.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:12 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:24:13.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:12 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:24:13.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:12 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:24:13.216 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:12 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:24:13.216 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:12 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:24:13.216 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:12 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:24:13.216 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:12 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:24:13.216 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:12 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-16T19:24:13.216 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:12 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-16T19:24:13.216 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:12 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:24:13.216 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:12 smithi022 ceph-mon[29221]: osdmap e31: 4 total, 4 up, 4 in 2024-02-16T19:24:13.273 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:12 smithi117 ceph-mon[35309]: Detected new or changed devices on smithi117 2024-02-16T19:24:13.273 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:12 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:24:13.273 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:12 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:24:13.273 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:12 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:24:13.273 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:12 smithi117 ceph-mon[35309]: Adjusting osd_memory_target on smithi117 to 14912M 2024-02-16T19:24:13.273 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:12 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:24:13.273 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:12 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:24:13.273 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:12 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:24:13.273 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:12 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:24:13.273 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:12 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:24:13.273 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:12 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:24:13.273 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:12 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:24:13.274 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:12 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:24:13.274 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:12 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-16T19:24:13.274 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:12 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-16T19:24:13.274 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:12 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:24:13.274 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:12 smithi117 ceph-mon[35309]: osdmap e31: 4 total, 4 up, 4 in 2024-02-16T19:24:14.084 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:13 smithi117 ceph-mon[35309]: from='client.24159 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:24:14.084 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:13 smithi117 ceph-mon[35309]: pgmap v101: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-02-16T19:24:14.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:13 smithi022 ceph-mon[29221]: from='client.24159 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:24:14.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:13 smithi022 ceph-mon[29221]: pgmap v101: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-02-16T19:24:16.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:16 smithi117 ceph-mon[35309]: pgmap v103: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-02-16T19:24:16.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:16 smithi117 ceph-mon[35309]: from='client.? 172.21.15.117:0/970237571' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d27bde8d-1a77-4f95-acb6-7d05becd9480"}]: dispatch 2024-02-16T19:24:16.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:16 smithi117 ceph-mon[35309]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d27bde8d-1a77-4f95-acb6-7d05becd9480"}]: dispatch 2024-02-16T19:24:16.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:16 smithi117 ceph-mon[35309]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d27bde8d-1a77-4f95-acb6-7d05becd9480"}]': finished 2024-02-16T19:24:16.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:16 smithi117 ceph-mon[35309]: osdmap e32: 5 total, 4 up, 5 in 2024-02-16T19:24:16.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:16 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-16T19:24:16.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:16 smithi117 ceph-mon[35309]: from='client.? 172.21.15.117:0/113820153' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-16T19:24:16.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:15 smithi022 ceph-mon[29221]: pgmap v103: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-02-16T19:24:16.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:15 smithi022 ceph-mon[29221]: from='client.? 172.21.15.117:0/970237571' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d27bde8d-1a77-4f95-acb6-7d05becd9480"}]: dispatch 2024-02-16T19:24:16.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:15 smithi022 ceph-mon[29221]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d27bde8d-1a77-4f95-acb6-7d05becd9480"}]: dispatch 2024-02-16T19:24:16.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:15 smithi022 ceph-mon[29221]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d27bde8d-1a77-4f95-acb6-7d05becd9480"}]': finished 2024-02-16T19:24:16.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:15 smithi022 ceph-mon[29221]: osdmap e32: 5 total, 4 up, 5 in 2024-02-16T19:24:16.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:15 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-16T19:24:16.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:15 smithi022 ceph-mon[29221]: from='client.? 172.21.15.117:0/113820153' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-16T19:24:18.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:17 smithi117 ceph-mon[35309]: pgmap v105: 1 pgs: 1 peering; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-02-16T19:24:18.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:17 smithi022 ceph-mon[29221]: pgmap v105: 1 pgs: 1 peering; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-02-16T19:24:19.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:19 smithi117 ceph-mon[35309]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-02-16T19:24:20.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:19 smithi022 ceph-mon[29221]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-02-16T19:24:21.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:21 smithi117 ceph-mon[35309]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-16T19:24:22.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:21 smithi022 ceph-mon[29221]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-16T19:24:23.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:22 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-16T19:24:23.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:22 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:24:23.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:22 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-16T19:24:23.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:22 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:24:23.966 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:23 smithi117 ceph-mon[35309]: Deploying daemon osd.4 on smithi117 2024-02-16T19:24:23.966 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:23 smithi117 ceph-mon[35309]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-16T19:24:24.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:23 smithi022 ceph-mon[29221]: Deploying daemon osd.4 on smithi117 2024-02-16T19:24:24.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:23 smithi022 ceph-mon[29221]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-16T19:24:26.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:25 smithi117 ceph-mon[35309]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-16T19:24:26.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:25 smithi022 ceph-mon[29221]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-16T19:24:28.042 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:27 smithi117 ceph-mon[35309]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-16T19:24:28.042 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:27 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:24:28.042 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:27 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:24:28.042 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:27 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:24:28.042 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:27 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:24:28.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:27 smithi022 ceph-mon[29221]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-16T19:24:28.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:27 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:24:28.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:27 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:24:28.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:27 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:24:28.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:27 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:24:30.093 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:29 smithi117 ceph-mon[35309]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-16T19:24:30.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:29 smithi022 ceph-mon[29221]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-16T19:24:31.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:31.046+0000 7fa6a3fff700 1 -- 172.21.15.117:0/3485681380 <== mgr.14217 v2:172.21.15.22:6800/2253867279 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fa6bc19bda0 con 0x7fa6a406c680 2024-02-16T19:24:31.048 INFO:teuthology.orchestra.run.smithi117.stdout:Created osd(s) 4 on host 'smithi117' 2024-02-16T19:24:31.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:31.048+0000 7fa6c11a6700 1 -- 172.21.15.117:0/3485681380 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fa6a406c680 msgr2=0x7fa6a406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:24:31.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:31.048+0000 7fa6c11a6700 1 --2- 172.21.15.117:0/3485681380 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fa6a406c680 0x7fa6a406eb30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac006fd0 tx=0x7fa6ac008040 comp rx=0 tx=0).stop 2024-02-16T19:24:31.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:31.048+0000 7fa6c11a6700 1 -- 172.21.15.117:0/3485681380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6bc102250 msgr2=0x7fa6bc1960c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:24:31.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:31.048+0000 7fa6c11a6700 1 --2- 172.21.15.117:0/3485681380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6bc102250 0x7fa6bc1960c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa6b000bd70 tx=0x7fa6b000be50 comp rx=0 tx=0).stop 2024-02-16T19:24:31.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:31.048+0000 7fa6c11a6700 1 -- 172.21.15.117:0/3485681380 shutdown_connections 2024-02-16T19:24:31.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:31.048+0000 7fa6c11a6700 1 --2- 172.21.15.117:0/3485681380 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6bc101900 0x7fa6bc195b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:31.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:31.048+0000 7fa6c11a6700 1 --2- 172.21.15.117:0/3485681380 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fa6a406c680 0x7fa6a406eb30 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:31.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:31.048+0000 7fa6c11a6700 1 --2- 172.21.15.117:0/3485681380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6bc102250 0x7fa6bc1960c0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:31.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:31.048+0000 7fa6c11a6700 1 -- 172.21.15.117:0/3485681380 >> 172.21.15.117:0/3485681380 conn(0x7fa6bc0f9f10 msgr2=0x7fa6bc1048d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:24:31.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:31.048+0000 7fa6c11a6700 1 -- 172.21.15.117:0/3485681380 shutdown_connections 2024-02-16T19:24:31.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:31.048+0000 7fa6c11a6700 1 -- 172.21.15.117:0/3485681380 wait complete. 2024-02-16T19:24:31.618 DEBUG:teuthology.orchestra.run.smithi117:osd.4> sudo journalctl -f -n 0 -u ceph-2a1346e6-cd00-11ee-95ba-87774f69a715@osd.4.service 2024-02-16T19:24:31.620 INFO:tasks.cephadm:Deploying osd.5 on smithi117 with /dev/vg_nvme/lv_2... 2024-02-16T19:24:31.621 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-02-16T19:24:31.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:31 smithi117 ceph-mon[35309]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-16T19:24:31.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:31 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:24:31.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:31 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:24:31.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:31 smithi117 ceph-mon[35309]: from='osd.4 [v2:172.21.15.117:6808/1856566932,v1:172.21.15.117:6809/1856566932]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-16T19:24:31.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:31 smithi117 ceph-mon[35309]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-16T19:24:31.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:31 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:24:31.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:31 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:24:32.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:31 smithi022 ceph-mon[29221]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-16T19:24:32.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:31 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:24:32.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:31 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:24:32.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:31 smithi022 ceph-mon[29221]: from='osd.4 [v2:172.21.15.117:6808/1856566932,v1:172.21.15.117:6809/1856566932]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-16T19:24:32.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:31 smithi022 ceph-mon[29221]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-16T19:24:32.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:31 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:24:32.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:31 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:24:32.423 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi117/config 2024-02-16T19:24:33.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:33 smithi117 ceph-mon[35309]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-16T19:24:33.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:33 smithi117 ceph-mon[35309]: osdmap e33: 5 total, 4 up, 5 in 2024-02-16T19:24:33.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:33 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-16T19:24:33.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:33 smithi117 ceph-mon[35309]: from='osd.4 [v2:172.21.15.117:6808/1856566932,v1:172.21.15.117:6809/1856566932]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-02-16T19:24:33.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:33 smithi117 ceph-mon[35309]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-02-16T19:24:33.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:33 smithi117 ceph-mon[35309]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-16T19:24:33.138 INFO:journalctl@ceph.osd.4.smithi117.stdout:Feb 16 19:24:33 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-4[45912]: 2024-02-16T19:24:33.004+0000 7f57ed310700 -1 osd.4 0 waiting for initial osdmap 2024-02-16T19:24:33.138 INFO:journalctl@ceph.osd.4.smithi117.stdout:Feb 16 19:24:33 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-4[45912]: 2024-02-16T19:24:33.011+0000 7f57e7902700 -1 osd.4 34 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-16T19:24:33.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:33 smithi022 ceph-mon[29221]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-16T19:24:33.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:33 smithi022 ceph-mon[29221]: osdmap e33: 5 total, 4 up, 5 in 2024-02-16T19:24:33.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:33 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-16T19:24:33.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:33 smithi022 ceph-mon[29221]: from='osd.4 [v2:172.21.15.117:6808/1856566932,v1:172.21.15.117:6809/1856566932]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-02-16T19:24:33.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:33 smithi022 ceph-mon[29221]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-02-16T19:24:33.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:33 smithi022 ceph-mon[29221]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-16T19:24:34.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:34 smithi117 ceph-mon[35309]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-02-16T19:24:34.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:34 smithi117 ceph-mon[35309]: osdmap e34: 5 total, 4 up, 5 in 2024-02-16T19:24:34.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:34 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-16T19:24:34.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:34 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-16T19:24:34.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:34 smithi022 ceph-mon[29221]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-02-16T19:24:34.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:34 smithi022 ceph-mon[29221]: osdmap e34: 5 total, 4 up, 5 in 2024-02-16T19:24:34.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:34 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-16T19:24:34.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:34 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-16T19:24:35.135 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:35 smithi117 ceph-mon[35309]: purged_snaps scrub starts 2024-02-16T19:24:35.136 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:35 smithi117 ceph-mon[35309]: purged_snaps scrub ok 2024-02-16T19:24:35.136 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:35 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-16T19:24:35.136 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:35 smithi117 ceph-mon[35309]: osd.4 [v2:172.21.15.117:6808/1856566932,v1:172.21.15.117:6809/1856566932] boot 2024-02-16T19:24:35.136 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:35 smithi117 ceph-mon[35309]: osdmap e35: 5 total, 5 up, 5 in 2024-02-16T19:24:35.136 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:35 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-16T19:24:35.136 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:35 smithi117 ceph-mon[35309]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-16T19:24:35.136 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:35 smithi117 ceph-mon[35309]: Detected new or changed devices on smithi117 2024-02-16T19:24:35.136 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:35 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:24:35.136 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:35 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:24:35.136 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:35 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:24:35.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:35 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:24:35.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:35 smithi117 ceph-mon[35309]: Adjusting osd_memory_target on smithi117 to 7456M 2024-02-16T19:24:35.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:35 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:24:35.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:35 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:24:35.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:35 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:24:35.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:35 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:24:35.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:35 smithi022 ceph-mon[29221]: purged_snaps scrub starts 2024-02-16T19:24:35.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:35 smithi022 ceph-mon[29221]: purged_snaps scrub ok 2024-02-16T19:24:35.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:35 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-16T19:24:35.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:35 smithi022 ceph-mon[29221]: osd.4 [v2:172.21.15.117:6808/1856566932,v1:172.21.15.117:6809/1856566932] boot 2024-02-16T19:24:35.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:35 smithi022 ceph-mon[29221]: osdmap e35: 5 total, 5 up, 5 in 2024-02-16T19:24:35.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:35 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-16T19:24:35.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:35 smithi022 ceph-mon[29221]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-16T19:24:35.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:35 smithi022 ceph-mon[29221]: Detected new or changed devices on smithi117 2024-02-16T19:24:35.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:35 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:24:35.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:35 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:24:35.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:35 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:24:35.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:35 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:24:35.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:35 smithi022 ceph-mon[29221]: Adjusting osd_memory_target on smithi117 to 7456M 2024-02-16T19:24:35.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:35 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:24:35.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:35 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:24:35.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:35 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:24:35.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:35 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:24:35.525 INFO:teuthology.orchestra.run.smithi117.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-02-16T19:24:35.525 INFO:teuthology.orchestra.run.smithi117.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-02-16T19:24:35.525 INFO:teuthology.orchestra.run.smithi117.stderr: stderr: 10+0 records in 2024-02-16T19:24:35.525 INFO:teuthology.orchestra.run.smithi117.stderr:10+0 records out 2024-02-16T19:24:35.525 INFO:teuthology.orchestra.run.smithi117.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0151002 s, 694 MB/s 2024-02-16T19:24:35.525 INFO:teuthology.orchestra.run.smithi117.stderr:--> Zapping successful for: 2024-02-16T19:24:36.114 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:36 smithi117 ceph-mon[35309]: osdmap e36: 5 total, 5 up, 5 in 2024-02-16T19:24:36.114 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:36 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:24:36.114 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:36 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:24:36.114 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:36 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:24:36.115 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:36 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:24:36.138 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi117:vg_nvme/lv_2 2024-02-16T19:24:36.347 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi117/config 2024-02-16T19:24:36.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:36 smithi022 ceph-mon[29221]: osdmap e36: 5 total, 5 up, 5 in 2024-02-16T19:24:36.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:36 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:24:36.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:36 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:24:36.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:36 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:24:36.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:36 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:24:37.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:37.866+0000 7f106389a700 1 -- 172.21.15.117:0/2408234401 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f105c0fe500 msgr2=0x7f105c0fe910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:24:37.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:37.866+0000 7f106389a700 1 --2- 172.21.15.117:0/2408234401 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f105c0fe500 0x7f105c0fe910 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1058009910 tx=0x7f1058009c20 comp rx=0 tx=0).stop 2024-02-16T19:24:37.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:37.866+0000 7f106389a700 1 -- 172.21.15.117:0/2408234401 shutdown_connections 2024-02-16T19:24:37.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:37.866+0000 7f106389a700 1 --2- 172.21.15.117:0/2408234401 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f105c0fee50 0x7f105c101580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:37.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:37.866+0000 7f106389a700 1 --2- 172.21.15.117:0/2408234401 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f105c0fe500 0x7f105c0fe910 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:37.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:37.866+0000 7f106389a700 1 -- 172.21.15.117:0/2408234401 >> 172.21.15.117:0/2408234401 conn(0x7f105c0f9f10 msgr2=0x7f105c0fc340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:24:37.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:37.866+0000 7f106389a700 1 -- 172.21.15.117:0/2408234401 shutdown_connections 2024-02-16T19:24:37.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:37.866+0000 7f106389a700 1 -- 172.21.15.117:0/2408234401 wait complete. 2024-02-16T19:24:37.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:37.867+0000 7f106389a700 1 Processor -- start 2024-02-16T19:24:37.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:37.867+0000 7f106389a700 1 -- start start 2024-02-16T19:24:37.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:37.867+0000 7f106389a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f105c0fe500 0x7f105c195b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:24:37.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:37.867+0000 7f106389a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f105c0fee50 0x7f105c1960a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:24:37.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:37.867+0000 7f106389a700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f105c1966c0 con 0x7f105c0fee50 2024-02-16T19:24:37.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:37.867+0000 7f106389a700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f105c196800 con 0x7f105c0fe500 2024-02-16T19:24:37.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:37.867+0000 7f1061636700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f105c0fe500 0x7f105c195b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:24:37.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:37.867+0000 7f1061636700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f105c0fe500 0x7f105c195b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:49312/0 (socket says 172.21.15.117:49312) 2024-02-16T19:24:37.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:37.867+0000 7f1061636700 1 -- 172.21.15.117:0/1357375261 learned_addr learned my addr 172.21.15.117:0/1357375261 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-02-16T19:24:37.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:37.867+0000 7f1060e35700 1 --2- 172.21.15.117:0/1357375261 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f105c0fee50 0x7f105c1960a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:24:37.871 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:37.867+0000 7f1061636700 1 -- 172.21.15.117:0/1357375261 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f105c0fee50 msgr2=0x7f105c1960a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:24:37.871 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:37.867+0000 7f1061636700 1 --2- 172.21.15.117:0/1357375261 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f105c0fee50 0x7f105c1960a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:37.871 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:37.867+0000 7f1061636700 1 -- 172.21.15.117:0/1357375261 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10580095f0 con 0x7f105c0fe500 2024-02-16T19:24:37.871 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:37.868+0000 7f1061636700 1 --2- 172.21.15.117:0/1357375261 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f105c0fe500 0x7f105c195b60 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f105800bd70 tx=0x7f105800be50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:24:37.871 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:37.868+0000 7f10527fc700 1 -- 172.21.15.117:0/1357375261 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f105801d070 con 0x7f105c0fe500 2024-02-16T19:24:37.871 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:37.868+0000 7f10527fc700 1 -- 172.21.15.117:0/1357375261 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f1058022470 con 0x7f105c0fe500 2024-02-16T19:24:37.871 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:37.868+0000 7f10527fc700 1 -- 172.21.15.117:0/1357375261 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f105800f650 con 0x7f105c0fe500 2024-02-16T19:24:37.871 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:37.868+0000 7f106389a700 1 -- 172.21.15.117:0/1357375261 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f105c19b250 con 0x7f105c0fe500 2024-02-16T19:24:37.871 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:37.868+0000 7f106389a700 1 -- 172.21.15.117:0/1357375261 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f105c19b740 con 0x7f105c0fe500 2024-02-16T19:24:37.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:37.869+0000 7f106389a700 1 -- 172.21.15.117:0/1357375261 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f105c108db0 con 0x7f105c0fe500 2024-02-16T19:24:37.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:37.869+0000 7f10527fc700 1 -- 172.21.15.117:0/1357375261 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f1058004b10 con 0x7f105c0fe500 2024-02-16T19:24:37.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:37.870+0000 7f10527fc700 1 --2- 172.21.15.117:0/1357375261 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f104806c680 0x7f104806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:24:37.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:37.870+0000 7f10527fc700 1 -- 172.21.15.117:0/1357375261 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4206+0+0 (secure 0 0 0) 0x7f105808cac0 con 0x7f105c0fe500 2024-02-16T19:24:37.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:37.870+0000 7f1060e35700 1 --2- 172.21.15.117:0/1357375261 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f104806c680 0x7f104806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:24:37.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:37.870+0000 7f1060e35700 1 --2- 172.21.15.117:0/1357375261 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f104806c680 0x7f104806eb30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f104c006fd0 tx=0x7f104c008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:24:37.875 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:37.874+0000 7f10527fc700 1 -- 172.21.15.117:0/1357375261 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f105805b1d0 con 0x7f105c0fe500 2024-02-16T19:24:38.072 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:38.071+0000 7f106389a700 1 -- 172.21.15.117:0/1357375261 --> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f105c19bd80 con 0x7f104806c680 2024-02-16T19:24:38.072 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:37 smithi117 ceph-mon[35309]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-16T19:24:38.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:37 smithi022 ceph-mon[29221]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-16T19:24:39.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:38 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-16T19:24:39.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:38 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-16T19:24:39.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:38 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:24:39.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:38 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-16T19:24:39.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:38 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-16T19:24:39.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:38 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:24:40.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:39 smithi117 ceph-mon[35309]: from='client.24177 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:24:40.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:39 smithi117 ceph-mon[35309]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-16T19:24:40.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:39 smithi022 ceph-mon[29221]: from='client.24177 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:24:40.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:39 smithi022 ceph-mon[29221]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-16T19:24:41.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:40 smithi117 ceph-mon[35309]: from='client.? 172.21.15.117:0/3844011923' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5873798f-3a2c-4a29-becd-6280b8866b6b"}]: dispatch 2024-02-16T19:24:41.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:40 smithi117 ceph-mon[35309]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5873798f-3a2c-4a29-becd-6280b8866b6b"}]: dispatch 2024-02-16T19:24:41.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:40 smithi117 ceph-mon[35309]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5873798f-3a2c-4a29-becd-6280b8866b6b"}]': finished 2024-02-16T19:24:41.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:40 smithi117 ceph-mon[35309]: osdmap e37: 6 total, 5 up, 6 in 2024-02-16T19:24:41.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:40 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-16T19:24:41.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:40 smithi022 ceph-mon[29221]: from='client.? 172.21.15.117:0/3844011923' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5873798f-3a2c-4a29-becd-6280b8866b6b"}]: dispatch 2024-02-16T19:24:41.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:40 smithi022 ceph-mon[29221]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5873798f-3a2c-4a29-becd-6280b8866b6b"}]: dispatch 2024-02-16T19:24:41.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:40 smithi022 ceph-mon[29221]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5873798f-3a2c-4a29-becd-6280b8866b6b"}]': finished 2024-02-16T19:24:41.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:40 smithi022 ceph-mon[29221]: osdmap e37: 6 total, 5 up, 6 in 2024-02-16T19:24:41.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:40 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-16T19:24:42.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:41 smithi117 ceph-mon[35309]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-16T19:24:42.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:41 smithi117 ceph-mon[35309]: from='client.? 172.21.15.117:0/3716289612' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-16T19:24:42.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:41 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:24:42.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:41 smithi022 ceph-mon[29221]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-16T19:24:42.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:41 smithi022 ceph-mon[29221]: from='client.? 172.21.15.117:0/3716289612' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-16T19:24:42.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:41 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:24:44.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:43 smithi117 ceph-mon[35309]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-16T19:24:44.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:43 smithi022 ceph-mon[29221]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-16T19:24:46.016 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:45 smithi117 ceph-mon[35309]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-16T19:24:46.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:45 smithi022 ceph-mon[29221]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-16T19:24:48.136 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:47 smithi117 ceph-mon[35309]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-16T19:24:48.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:47 smithi022 ceph-mon[29221]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-16T19:24:49.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:48 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-16T19:24:49.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:48 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:24:49.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:48 smithi117 ceph-mon[35309]: Deploying daemon osd.5 on smithi117 2024-02-16T19:24:49.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:48 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-16T19:24:49.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:48 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:24:49.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:48 smithi022 ceph-mon[29221]: Deploying daemon osd.5 on smithi117 2024-02-16T19:24:50.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:49 smithi117 ceph-mon[35309]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-16T19:24:50.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:49 smithi022 ceph-mon[29221]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-16T19:24:52.121 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:51 smithi117 ceph-mon[35309]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-16T19:24:52.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:51 smithi022 ceph-mon[29221]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-16T19:24:54.003 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:53 smithi117 ceph-mon[35309]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-16T19:24:54.003 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:53 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:24:54.004 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:53 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:24:54.004 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:53 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:24:54.111 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:53 smithi022 ceph-mon[29221]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-16T19:24:54.111 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:53 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:24:54.111 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:53 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:24:54.111 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:53 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:24:56.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:55 smithi117 ceph-mon[35309]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-16T19:24:56.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:55 smithi022 ceph-mon[29221]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-16T19:24:56.236 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:56.235+0000 7f10527fc700 1 -- 172.21.15.117:0/1357375261 <== mgr.14217 v2:172.21.15.22:6800/2253867279 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f105c19bd80 con 0x7f104806c680 2024-02-16T19:24:56.236 INFO:teuthology.orchestra.run.smithi117.stdout:Created osd(s) 5 on host 'smithi117' 2024-02-16T19:24:56.238 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:56.238+0000 7f106389a700 1 -- 172.21.15.117:0/1357375261 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f104806c680 msgr2=0x7f104806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:24:56.239 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:56.238+0000 7f106389a700 1 --2- 172.21.15.117:0/1357375261 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f104806c680 0x7f104806eb30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f104c006fd0 tx=0x7f104c008040 comp rx=0 tx=0).stop 2024-02-16T19:24:56.239 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:56.238+0000 7f106389a700 1 -- 172.21.15.117:0/1357375261 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f105c0fe500 msgr2=0x7f105c195b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:24:56.239 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:56.238+0000 7f106389a700 1 --2- 172.21.15.117:0/1357375261 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f105c0fe500 0x7f105c195b60 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f105800bd70 tx=0x7f105800be50 comp rx=0 tx=0).stop 2024-02-16T19:24:56.239 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:56.238+0000 7f106389a700 1 -- 172.21.15.117:0/1357375261 shutdown_connections 2024-02-16T19:24:56.239 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:56.238+0000 7f106389a700 1 --2- 172.21.15.117:0/1357375261 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f104806c680 0x7f104806eb30 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:56.239 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:56.238+0000 7f106389a700 1 --2- 172.21.15.117:0/1357375261 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f105c0fe500 0x7f105c195b60 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:56.239 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:56.238+0000 7f106389a700 1 --2- 172.21.15.117:0/1357375261 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f105c0fee50 0x7f105c1960a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:56.239 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:56.238+0000 7f106389a700 1 -- 172.21.15.117:0/1357375261 >> 172.21.15.117:0/1357375261 conn(0x7f105c0f9f10 msgr2=0x7f105c1047b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:24:56.239 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:56.238+0000 7f106389a700 1 -- 172.21.15.117:0/1357375261 shutdown_connections 2024-02-16T19:24:56.239 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:24:56.238+0000 7f106389a700 1 -- 172.21.15.117:0/1357375261 wait complete. 2024-02-16T19:24:56.881 DEBUG:teuthology.orchestra.run.smithi117:osd.5> sudo journalctl -f -n 0 -u ceph-2a1346e6-cd00-11ee-95ba-87774f69a715@osd.5.service 2024-02-16T19:24:56.884 INFO:tasks.cephadm:Waiting for 6 OSDs to come up... 2024-02-16T19:24:56.884 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -- ceph osd stat -f json 2024-02-16T19:24:57.105 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi022/config 2024-02-16T19:24:57.137 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:24:56 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-5[50556]: 2024-02-16T19:24:56.931+0000 7f173f986640 -1 osd.5 0 log_to_monitors true 2024-02-16T19:24:57.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:57 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:24:57.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:57 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:24:57.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:57 smithi022 ceph-mon[29221]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-16T19:24:57.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:57 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:24:57.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:57 smithi022 ceph-mon[29221]: from='osd.5 [v2:172.21.15.117:6816/38288182,v1:172.21.15.117:6817/38288182]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-16T19:24:57.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:57 smithi022 ceph-mon[29221]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-16T19:24:57.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:57 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:24:57.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:57 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:24:57.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:57 smithi117 ceph-mon[35309]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-16T19:24:57.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:57 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:24:57.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:57 smithi117 ceph-mon[35309]: from='osd.5 [v2:172.21.15.117:6816/38288182,v1:172.21.15.117:6817/38288182]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-16T19:24:57.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:57 smithi117 ceph-mon[35309]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-16T19:24:58.809 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:58 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:24:58.809 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:58 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:24:58.809 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:58 smithi022 ceph-mon[29221]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-16T19:24:58.809 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:58 smithi022 ceph-mon[29221]: osdmap e38: 6 total, 5 up, 6 in 2024-02-16T19:24:58.809 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:58 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-16T19:24:58.809 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:58 smithi022 ceph-mon[29221]: from='osd.5 [v2:172.21.15.117:6816/38288182,v1:172.21.15.117:6817/38288182]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-02-16T19:24:58.809 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:58 smithi022 ceph-mon[29221]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-02-16T19:24:58.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:24:58.818+0000 7f15dd868700 1 -- 172.21.15.22:0/579807425 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15d8101440 msgr2=0x7f15d8101890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:24:58.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:24:58.818+0000 7f15dd868700 1 --2- 172.21.15.22:0/579807425 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15d8101440 0x7f15d8101890 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f15c8009910 tx=0x7f15c8009c20 comp rx=0 tx=0).stop 2024-02-16T19:24:58.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:24:58.818+0000 7f15dd868700 1 -- 172.21.15.22:0/579807425 shutdown_connections 2024-02-16T19:24:58.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:24:58.818+0000 7f15dd868700 1 --2- 172.21.15.22:0/579807425 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15d8101440 0x7f15d8101890 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:58.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:24:58.818+0000 7f15dd868700 1 --2- 172.21.15.22:0/579807425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15d8100240 0x7f15d8100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:58.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:24:58.818+0000 7f15dd868700 1 -- 172.21.15.22:0/579807425 >> 172.21.15.22:0/579807425 conn(0x7f15d80fb7f0 msgr2=0x7f15d80fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:24:58.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:24:58.818+0000 7f15dd868700 1 -- 172.21.15.22:0/579807425 shutdown_connections 2024-02-16T19:24:58.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:24:58.819+0000 7f15dd868700 1 -- 172.21.15.22:0/579807425 wait complete. 2024-02-16T19:24:58.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:24:58.819+0000 7f15dd868700 1 Processor -- start 2024-02-16T19:24:58.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:24:58.819+0000 7f15dd868700 1 -- start start 2024-02-16T19:24:58.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:24:58.820+0000 7f15dd868700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15d8100240 0x7f15d8195b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:24:58.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:24:58.820+0000 7f15dd868700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15d8101440 0x7f15d8196070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:24:58.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:24:58.820+0000 7f15dd868700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d8196690 con 0x7f15d8100240 2024-02-16T19:24:58.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:24:58.820+0000 7f15dd868700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d81967d0 con 0x7f15d8101440 2024-02-16T19:24:58.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:24:58.820+0000 7f15d6ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15d8100240 0x7f15d8195b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:24:58.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:24:58.820+0000 7f15d6ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15d8100240 0x7f15d8195b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58250/0 (socket says 172.21.15.22:58250) 2024-02-16T19:24:58.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:24:58.820+0000 7f15d67fc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15d8101440 0x7f15d8196070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:24:58.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:24:58.820+0000 7f15d6ffd700 1 -- 172.21.15.22:0/1432343512 learned_addr learned my addr 172.21.15.22:0/1432343512 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:24:58.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:24:58.820+0000 7f15d6ffd700 1 -- 172.21.15.22:0/1432343512 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15d8101440 msgr2=0x7f15d8196070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:24:58.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:24:58.820+0000 7f15d6ffd700 1 --2- 172.21.15.22:0/1432343512 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15d8101440 0x7f15d8196070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:58.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:24:58.820+0000 7f15d6ffd700 1 -- 172.21.15.22:0/1432343512 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15c80095f0 con 0x7f15d8100240 2024-02-16T19:24:58.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:24:58.820+0000 7f15d6ffd700 1 --2- 172.21.15.22:0/1432343512 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15d8100240 0x7f15d8195b30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f15cc00b480 tx=0x7f15cc00b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:24:58.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:24:58.821+0000 7f15dc866700 1 -- 172.21.15.22:0/1432343512 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15cc011800 con 0x7f15d8100240 2024-02-16T19:24:58.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:24:58.821+0000 7f15dd868700 1 -- 172.21.15.22:0/1432343512 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15d819b280 con 0x7f15d8100240 2024-02-16T19:24:58.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:24:58.821+0000 7f15dc866700 1 -- 172.21.15.22:0/1432343512 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f15cc011e10 con 0x7f15d8100240 2024-02-16T19:24:58.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:24:58.821+0000 7f15dc866700 1 -- 172.21.15.22:0/1432343512 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15cc010ea0 con 0x7f15d8100240 2024-02-16T19:24:58.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:24:58.821+0000 7f15dd868700 1 -- 172.21.15.22:0/1432343512 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15d819b7d0 con 0x7f15d8100240 2024-02-16T19:24:58.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:24:58.822+0000 7f15dd868700 1 -- 172.21.15.22:0/1432343512 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15d8108e30 con 0x7f15d8100240 2024-02-16T19:24:58.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:24:58.823+0000 7f15dc866700 1 -- 172.21.15.22:0/1432343512 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f15cc010380 con 0x7f15d8100240 2024-02-16T19:24:58.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:24:58.823+0000 7f15dc866700 1 --2- 172.21.15.22:0/1432343512 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f15c006c680 0x7f15c006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:24:58.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:24:58.823+0000 7f15dc866700 1 -- 172.21.15.22:0/1432343512 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4358+0+0 (secure 0 0 0) 0x7f15cc08af90 con 0x7f15d8100240 2024-02-16T19:24:58.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:24:58.823+0000 7f15d67fc700 1 --2- 172.21.15.22:0/1432343512 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f15c006c680 0x7f15c006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:24:58.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:24:58.824+0000 7f15d67fc700 1 --2- 172.21.15.22:0/1432343512 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f15c006c680 0x7f15c006eb30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f15c8006010 tx=0x7f15c800b540 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:24:58.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:24:58.828+0000 7f15dc866700 1 -- 172.21.15.22:0/1432343512 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f15cc059600 con 0x7f15d8100240 2024-02-16T19:24:58.851 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:24:58 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-5[50556]: 2024-02-16T19:24:58.825+0000 7f17347fc700 -1 osd.5 0 waiting for initial osdmap 2024-02-16T19:24:58.851 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:24:58 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-5[50556]: 2024-02-16T19:24:58.835+0000 7f172e5ed700 -1 osd.5 39 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-16T19:24:58.852 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:58 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:24:58.852 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:58 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:24:58.852 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:58 smithi117 ceph-mon[35309]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-16T19:24:58.852 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:58 smithi117 ceph-mon[35309]: osdmap e38: 6 total, 5 up, 6 in 2024-02-16T19:24:58.852 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:58 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-16T19:24:58.852 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:58 smithi117 ceph-mon[35309]: from='osd.5 [v2:172.21.15.117:6816/38288182,v1:172.21.15.117:6817/38288182]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-02-16T19:24:58.852 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:58 smithi117 ceph-mon[35309]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-02-16T19:24:59.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:24:59.034+0000 7f15dd868700 1 -- 172.21.15.22:0/1432343512 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f15d8108ff0 con 0x7f15d8100240 2024-02-16T19:24:59.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:24:59.035+0000 7f15dc866700 1 -- 172.21.15.22:0/1432343512 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v39) v1 ==== 74+0+130 (secure 0 0 0) 0x7f15cc059190 con 0x7f15d8100240 2024-02-16T19:24:59.036 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:24:59.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:24:59.037+0000 7f15dd868700 1 -- 172.21.15.22:0/1432343512 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f15c006c680 msgr2=0x7f15c006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:24:59.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:24:59.037+0000 7f15dd868700 1 --2- 172.21.15.22:0/1432343512 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f15c006c680 0x7f15c006eb30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f15c8006010 tx=0x7f15c800b540 comp rx=0 tx=0).stop 2024-02-16T19:24:59.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:24:59.037+0000 7f15dd868700 1 -- 172.21.15.22:0/1432343512 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15d8100240 msgr2=0x7f15d8195b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:24:59.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:24:59.037+0000 7f15dd868700 1 --2- 172.21.15.22:0/1432343512 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15d8100240 0x7f15d8195b30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f15cc00b480 tx=0x7f15cc00b790 comp rx=0 tx=0).stop 2024-02-16T19:24:59.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:24:59.037+0000 7f15dd868700 1 -- 172.21.15.22:0/1432343512 shutdown_connections 2024-02-16T19:24:59.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:24:59.037+0000 7f15dd868700 1 --2- 172.21.15.22:0/1432343512 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15d8100240 0x7f15d8195b30 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:59.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:24:59.037+0000 7f15dd868700 1 --2- 172.21.15.22:0/1432343512 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f15c006c680 0x7f15c006eb30 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:59.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:24:59.037+0000 7f15dd868700 1 --2- 172.21.15.22:0/1432343512 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15d8101440 0x7f15d8196070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:59.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:24:59.037+0000 7f15dd868700 1 -- 172.21.15.22:0/1432343512 >> 172.21.15.22:0/1432343512 conn(0x7f15d80fb7f0 msgr2=0x7f15d8104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:24:59.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:24:59.037+0000 7f15dd868700 1 -- 172.21.15.22:0/1432343512 shutdown_connections 2024-02-16T19:24:59.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:24:59.037+0000 7f15dd868700 1 -- 172.21.15.22:0/1432343512 wait complete. 2024-02-16T19:24:59.846 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":39,"num_osds":6,"num_up_osds":5,"osd_up_since":1708111474,"num_in_osds":6,"osd_in_since":1708111480,"num_remapped_pgs":0} 2024-02-16T19:25:00.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:59 smithi117 ceph-mon[35309]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-16T19:25:00.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:59 smithi117 ceph-mon[35309]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-02-16T19:25:00.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:59 smithi117 ceph-mon[35309]: osdmap e39: 6 total, 5 up, 6 in 2024-02-16T19:25:00.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:59 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-16T19:25:00.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:59 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-16T19:25:00.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:24:59 smithi117 ceph-mon[35309]: from='client.? 172.21.15.22:0/1432343512' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-16T19:25:00.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:59 smithi022 ceph-mon[29221]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-16T19:25:00.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:59 smithi022 ceph-mon[29221]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-02-16T19:25:00.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:59 smithi022 ceph-mon[29221]: osdmap e39: 6 total, 5 up, 6 in 2024-02-16T19:25:00.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:59 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-16T19:25:00.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:59 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-16T19:25:00.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:24:59 smithi022 ceph-mon[29221]: from='client.? 172.21.15.22:0/1432343512' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-16T19:25:00.847 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -- ceph osd stat -f json 2024-02-16T19:25:00.988 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:25:00 smithi117 ceph-mon[35309]: purged_snaps scrub starts 2024-02-16T19:25:00.988 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:25:00 smithi117 ceph-mon[35309]: purged_snaps scrub ok 2024-02-16T19:25:00.988 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:25:00 smithi117 ceph-mon[35309]: osd.5 [v2:172.21.15.117:6816/38288182,v1:172.21.15.117:6817/38288182] boot 2024-02-16T19:25:00.988 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:25:00 smithi117 ceph-mon[35309]: osdmap e40: 6 total, 6 up, 6 in 2024-02-16T19:25:00.988 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:25:00 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-16T19:25:01.068 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi022/config 2024-02-16T19:25:01.190 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:25:00 smithi022 ceph-mon[29221]: purged_snaps scrub starts 2024-02-16T19:25:01.191 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:25:00 smithi022 ceph-mon[29221]: purged_snaps scrub ok 2024-02-16T19:25:01.191 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:25:00 smithi022 ceph-mon[29221]: osd.5 [v2:172.21.15.117:6816/38288182,v1:172.21.15.117:6817/38288182] boot 2024-02-16T19:25:01.191 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:25:00 smithi022 ceph-mon[29221]: osdmap e40: 6 total, 6 up, 6 in 2024-02-16T19:25:01.191 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:25:00 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-16T19:25:02.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:25:01 smithi117 ceph-mon[35309]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:25:02.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:25:01 smithi117 ceph-mon[35309]: osdmap e41: 6 total, 6 up, 6 in 2024-02-16T19:25:02.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:25:01 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:25:02.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:25:01 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:25:02.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:25:01 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:25:02.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:25:01 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:25:02.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:25:01 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:25:02.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:25:01 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:25:02.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:25:01 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:25:02.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:25:01 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:25:02.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:25:01 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:25:02.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:25:01 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:25:02.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:25:01 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:25:02.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:25:01 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:25:02.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:25:01 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:25:02.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:25:01 smithi022 ceph-mon[29221]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:25:02.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:25:01 smithi022 ceph-mon[29221]: osdmap e41: 6 total, 6 up, 6 in 2024-02-16T19:25:02.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:25:01 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:25:02.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:25:01 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:25:02.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:25:01 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:25:02.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:25:01 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:25:02.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:25:01 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:25:02.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:25:01 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:25:02.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:25:01 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:25:02.216 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:25:01 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:25:02.216 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:25:01 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:25:02.216 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:25:01 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:25:02.216 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:25:01 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:25:02.216 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:25:01 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:25:02.216 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:25:01 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:25:02.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:02.822+0000 7f2f291c2700 1 -- 172.21.15.22:0/143387212 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f240fffb0 msgr2=0x7f2f24100440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:02.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:02.822+0000 7f2f291c2700 1 --2- 172.21.15.22:0/143387212 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f240fffb0 0x7f2f24100440 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c009910 tx=0x7f2f0c009c20 comp rx=0 tx=0).stop 2024-02-16T19:25:02.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:02.822+0000 7f2f291c2700 1 -- 172.21.15.22:0/143387212 shutdown_connections 2024-02-16T19:25:02.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:02.822+0000 7f2f291c2700 1 --2- 172.21.15.22:0/143387212 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f240fffb0 0x7f2f24100440 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:02.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:02.822+0000 7f2f291c2700 1 --2- 172.21.15.22:0/143387212 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f240ff660 0x7f2f240ffa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:02.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:02.822+0000 7f2f291c2700 1 -- 172.21.15.22:0/143387212 >> 172.21.15.22:0/143387212 conn(0x7f2f240f9f10 msgr2=0x7f2f240fc340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:25:02.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:02.822+0000 7f2f291c2700 1 -- 172.21.15.22:0/143387212 shutdown_connections 2024-02-16T19:25:02.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:02.822+0000 7f2f291c2700 1 -- 172.21.15.22:0/143387212 wait complete. 2024-02-16T19:25:02.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:02.822+0000 7f2f291c2700 1 Processor -- start 2024-02-16T19:25:02.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:02.823+0000 7f2f291c2700 1 -- start start 2024-02-16T19:25:02.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:02.823+0000 7f2f291c2700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f240ff660 0x7f2f24195af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:02.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:02.823+0000 7f2f291c2700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f240fffb0 0x7f2f24196030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:02.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:02.823+0000 7f2f291c2700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f241965c0 con 0x7f2f240fffb0 2024-02-16T19:25:02.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:02.823+0000 7f2f291c2700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f24196700 con 0x7f2f240ff660 2024-02-16T19:25:02.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:02.823+0000 7f2f2259c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f240fffb0 0x7f2f24196030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:02.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:02.823+0000 7f2f2259c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f240fffb0 0x7f2f24196030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58266/0 (socket says 172.21.15.22:58266) 2024-02-16T19:25:02.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:02.823+0000 7f2f2259c700 1 -- 172.21.15.22:0/1597472130 learned_addr learned my addr 172.21.15.22:0/1597472130 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:25:02.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:02.823+0000 7f2f22d9d700 1 --2- 172.21.15.22:0/1597472130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f240ff660 0x7f2f24195af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:02.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:02.823+0000 7f2f2259c700 1 -- 172.21.15.22:0/1597472130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f240ff660 msgr2=0x7f2f24195af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:02.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:02.823+0000 7f2f2259c700 1 --2- 172.21.15.22:0/1597472130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f240ff660 0x7f2f24195af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:02.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:02.823+0000 7f2f2259c700 1 -- 172.21.15.22:0/1597472130 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f0c0095f0 con 0x7f2f240fffb0 2024-02-16T19:25:02.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:02.823+0000 7f2f2259c700 1 --2- 172.21.15.22:0/1597472130 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f240fffb0 0x7f2f24196030 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c00bda0 tx=0x7f2f0c00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:02.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:02.824+0000 7f2f1bfff700 1 -- 172.21.15.22:0/1597472130 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f0c01d070 con 0x7f2f240fffb0 2024-02-16T19:25:02.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:02.824+0000 7f2f1bfff700 1 -- 172.21.15.22:0/1597472130 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2f0c004c50 con 0x7f2f240fffb0 2024-02-16T19:25:02.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:02.824+0000 7f2f291c2700 1 -- 172.21.15.22:0/1597472130 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f2419b160 con 0x7f2f240fffb0 2024-02-16T19:25:02.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:02.824+0000 7f2f1bfff700 1 -- 172.21.15.22:0/1597472130 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f0c00f690 con 0x7f2f240fffb0 2024-02-16T19:25:02.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:02.824+0000 7f2f291c2700 1 -- 172.21.15.22:0/1597472130 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f2419b620 con 0x7f2f240fffb0 2024-02-16T19:25:02.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:02.825+0000 7f2f291c2700 1 -- 172.21.15.22:0/1597472130 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f24107c00 con 0x7f2f240fffb0 2024-02-16T19:25:02.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:02.825+0000 7f2f1bfff700 1 -- 172.21.15.22:0/1597472130 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f2f0c022a20 con 0x7f2f240fffb0 2024-02-16T19:25:02.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:02.826+0000 7f2f1bfff700 1 --2- 172.21.15.22:0/1597472130 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f2f1006c680 0x7f2f1006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:02.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:02.826+0000 7f2f1bfff700 1 -- 172.21.15.22:0/1597472130 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f2f0c08cac0 con 0x7f2f240fffb0 2024-02-16T19:25:02.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:02.826+0000 7f2f22d9d700 1 --2- 172.21.15.22:0/1597472130 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f2f1006c680 0x7f2f1006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:02.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:02.826+0000 7f2f22d9d700 1 --2- 172.21.15.22:0/1597472130 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f2f1006c680 0x7f2f1006eb30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c005fd0 tx=0x7f2f1c005e50 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:02.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:02.830+0000 7f2f1bfff700 1 -- 172.21.15.22:0/1597472130 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f2f0c05b0a0 con 0x7f2f240fffb0 2024-02-16T19:25:03.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:03.030+0000 7f2f291c2700 1 -- 172.21.15.22:0/1597472130 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f2f24107df0 con 0x7f2f240fffb0 2024-02-16T19:25:03.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:03.030+0000 7f2f1bfff700 1 -- 172.21.15.22:0/1597472130 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v41) v1 ==== 74+0+130 (secure 0 0 0) 0x7f2f0c0270a0 con 0x7f2f240fffb0 2024-02-16T19:25:03.031 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:25:03.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:03.032+0000 7f2f291c2700 1 -- 172.21.15.22:0/1597472130 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f2f1006c680 msgr2=0x7f2f1006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:03.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:03.032+0000 7f2f291c2700 1 --2- 172.21.15.22:0/1597472130 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f2f1006c680 0x7f2f1006eb30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c005fd0 tx=0x7f2f1c005e50 comp rx=0 tx=0).stop 2024-02-16T19:25:03.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:03.032+0000 7f2f291c2700 1 -- 172.21.15.22:0/1597472130 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f240fffb0 msgr2=0x7f2f24196030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:03.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:03.032+0000 7f2f291c2700 1 --2- 172.21.15.22:0/1597472130 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f240fffb0 0x7f2f24196030 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c00bda0 tx=0x7f2f0c00be80 comp rx=0 tx=0).stop 2024-02-16T19:25:03.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:03.032+0000 7f2f291c2700 1 -- 172.21.15.22:0/1597472130 shutdown_connections 2024-02-16T19:25:03.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:03.032+0000 7f2f291c2700 1 --2- 172.21.15.22:0/1597472130 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f2f1006c680 0x7f2f1006eb30 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:03.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:03.032+0000 7f2f291c2700 1 --2- 172.21.15.22:0/1597472130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f240ff660 0x7f2f24195af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:03.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:03.032+0000 7f2f291c2700 1 --2- 172.21.15.22:0/1597472130 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f240fffb0 0x7f2f24196030 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:03.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:03.032+0000 7f2f291c2700 1 -- 172.21.15.22:0/1597472130 >> 172.21.15.22:0/1597472130 conn(0x7f2f240f9f10 msgr2=0x7f2f240fe260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:25:03.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:03.032+0000 7f2f291c2700 1 -- 172.21.15.22:0/1597472130 shutdown_connections 2024-02-16T19:25:03.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:03.032+0000 7f2f291c2700 1 -- 172.21.15.22:0/1597472130 wait complete. 2024-02-16T19:25:03.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:25:02 smithi117 ceph-mon[35309]: Detected new or changed devices on smithi117 2024-02-16T19:25:03.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:25:02 smithi117 ceph-mon[35309]: Adjusting osd_memory_target on smithi117 to 4970M 2024-02-16T19:25:03.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:25:02 smithi022 ceph-mon[29221]: Detected new or changed devices on smithi117 2024-02-16T19:25:03.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:25:02 smithi022 ceph-mon[29221]: Adjusting osd_memory_target on smithi117 to 4970M 2024-02-16T19:25:03.700 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":41,"num_osds":6,"num_up_osds":6,"osd_up_since":1708111499,"num_in_osds":6,"osd_in_since":1708111480,"num_remapped_pgs":0} 2024-02-16T19:25:03.701 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -- ceph osd dump --format=json 2024-02-16T19:25:03.911 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi022/config 2024-02-16T19:25:03.964 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:25:03 smithi022 ceph-mon[29221]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:25:03.964 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:25:03 smithi022 ceph-mon[29221]: from='client.? 172.21.15.22:0/1597472130' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-16T19:25:04.112 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:25:03 smithi117 ceph-mon[35309]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:25:04.112 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:25:03 smithi117 ceph-mon[35309]: from='client.? 172.21.15.22:0/1597472130' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-16T19:25:05.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:05.640+0000 7f1984522700 1 -- 172.21.15.22:0/17552910 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f197c100e80 msgr2=0x7f197c104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:05.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:05.640+0000 7f1984522700 1 --2- 172.21.15.22:0/17552910 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f197c100e80 0x7f197c104e50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1978009910 tx=0x7f1978009c20 comp rx=0 tx=0).stop 2024-02-16T19:25:05.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:05.640+0000 7f1984522700 1 -- 172.21.15.22:0/17552910 shutdown_connections 2024-02-16T19:25:05.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:05.640+0000 7f1984522700 1 --2- 172.21.15.22:0/17552910 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f197c100e80 0x7f197c104e50 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:05.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:05.640+0000 7f1984522700 1 --2- 172.21.15.22:0/17552910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f197c1004e0 0x7f197c1008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:05.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:05.640+0000 7f1984522700 1 -- 172.21.15.22:0/17552910 >> 172.21.15.22:0/17552910 conn(0x7f197c0fbda0 msgr2=0x7f197c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:25:05.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:05.640+0000 7f1984522700 1 -- 172.21.15.22:0/17552910 shutdown_connections 2024-02-16T19:25:05.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:05.640+0000 7f1984522700 1 -- 172.21.15.22:0/17552910 wait complete. 2024-02-16T19:25:05.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:05.641+0000 7f1984522700 1 Processor -- start 2024-02-16T19:25:05.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:05.641+0000 7f1984522700 1 -- start start 2024-02-16T19:25:05.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:05.641+0000 7f1984522700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f197c1004e0 0x7f197c195f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:05.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:05.641+0000 7f1984522700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f197c100e80 0x7f197c1964d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:05.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:05.641+0000 7f1984522700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f197c196bb0 con 0x7f197c100e80 2024-02-16T19:25:05.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:05.641+0000 7f1984522700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f197c19a940 con 0x7f197c1004e0 2024-02-16T19:25:05.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:05.641+0000 7f1981abd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f197c100e80 0x7f197c1964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:05.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:05.641+0000 7f1981abd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f197c100e80 0x7f197c1964d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51810/0 (socket says 172.21.15.22:51810) 2024-02-16T19:25:05.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:05.641+0000 7f1981abd700 1 -- 172.21.15.22:0/1369097948 learned_addr learned my addr 172.21.15.22:0/1369097948 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:25:05.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:05.642+0000 7f19822be700 1 --2- 172.21.15.22:0/1369097948 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f197c1004e0 0x7f197c195f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:05.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:05.642+0000 7f1981abd700 1 -- 172.21.15.22:0/1369097948 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f197c1004e0 msgr2=0x7f197c195f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:05.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:05.642+0000 7f1981abd700 1 --2- 172.21.15.22:0/1369097948 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f197c1004e0 0x7f197c195f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:05.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:05.642+0000 7f1981abd700 1 -- 172.21.15.22:0/1369097948 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19780095f0 con 0x7f197c100e80 2024-02-16T19:25:05.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:05.642+0000 7f1981abd700 1 --2- 172.21.15.22:0/1369097948 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f197c100e80 0x7f197c1964d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f19780092e0 tx=0x7f197800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:05.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:05.642+0000 7f19737fe700 1 -- 172.21.15.22:0/1369097948 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f197801d070 con 0x7f197c100e80 2024-02-16T19:25:05.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:05.642+0000 7f1984522700 1 -- 172.21.15.22:0/1369097948 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f197c19abc0 con 0x7f197c100e80 2024-02-16T19:25:05.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:05.642+0000 7f19737fe700 1 -- 172.21.15.22:0/1369097948 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f1978022470 con 0x7f197c100e80 2024-02-16T19:25:05.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:05.642+0000 7f19737fe700 1 -- 172.21.15.22:0/1369097948 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f197800f690 con 0x7f197c100e80 2024-02-16T19:25:05.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:05.642+0000 7f1984522700 1 -- 172.21.15.22:0/1369097948 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f197c19b0b0 con 0x7f197c100e80 2024-02-16T19:25:05.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:05.643+0000 7f1984522700 1 -- 172.21.15.22:0/1369097948 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f197c1087a0 con 0x7f197c100e80 2024-02-16T19:25:05.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:05.644+0000 7f19737fe700 1 -- 172.21.15.22:0/1369097948 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f19780225e0 con 0x7f197c100e80 2024-02-16T19:25:05.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:05.644+0000 7f19737fe700 1 --2- 172.21.15.22:0/1369097948 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f196806c680 0x7f196806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:05.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:05.644+0000 7f19737fe700 1 -- 172.21.15.22:0/1369097948 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f197808cb60 con 0x7f197c100e80 2024-02-16T19:25:05.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:05.644+0000 7f19822be700 1 --2- 172.21.15.22:0/1369097948 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f196806c680 0x7f196806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:05.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:05.644+0000 7f19822be700 1 --2- 172.21.15.22:0/1369097948 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f196806c680 0x7f196806eb30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f196c006fd0 tx=0x7f196c008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:05.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:05.649+0000 7f19737fe700 1 -- 172.21.15.22:0/1369097948 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f1978057630 con 0x7f197c100e80 2024-02-16T19:25:05.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:05.847+0000 7f1984522700 1 -- 172.21.15.22:0/1369097948 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f197c197390 con 0x7f197c100e80 2024-02-16T19:25:05.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:05.847+0000 7f19737fe700 1 -- 172.21.15.22:0/1369097948 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+11323 (secure 0 0 0) 0x7f197805ac50 con 0x7f197c100e80 2024-02-16T19:25:05.849 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:25:05.849 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":41,"fsid":"2a1346e6-cd00-11ee-95ba-87774f69a715","created":"2024-02-16T19:19:11.528280+0000","modified":"2024-02-16T19:25:00.824993+0000","last_up_change":"2024-02-16T19:24:59.819598+0000","last_in_change":"2024-02-16T19:24:40.575373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-16T19:23:47.101426+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d16e47f2-7209-45f4-ad50-1e0f6789ffad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2126256017},{"type":"v1","addr":"172.21.15.22:6803","nonce":2126256017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2126256017},{"type":"v1","addr":"172.21.15.22:6805","nonce":2126256017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2126256017},{"type":"v1","addr":"172.21.15.22:6809","nonce":2126256017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2126256017},{"type":"v1","addr":"172.21.15.22:6807","nonce":2126256017}]},"public_addr":"172.21.15.22:6803/2126256017","cluster_addr":"172.21.15.22:6805/2126256017","heartbeat_back_addr":"172.21.15.22:6809/2126256017","heartbeat_front_addr":"172.21.15.22:6807/2126256017","state":["exists","up"]},{"osd":1,"uuid":"59c3d085-3422-4e92-9048-46d7a39e31a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":3355370212},{"type":"v1","addr":"172.21.15.22:6811","nonce":3355370212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":3355370212},{"type":"v1","addr":"172.21.15.22:6813","nonce":3355370212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":3355370212},{"type":"v1","addr":"172.21.15.22:6817","nonce":3355370212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":3355370212},{"type":"v1","addr":"172.21.15.22:6815","nonce":3355370212}]},"public_addr":"172.21.15.22:6811/3355370212","cluster_addr":"172.21.15.22:6813/3355370212","heartbeat_back_addr":"172.21.15.22:6817/3355370212","heartbeat_front_addr":"172.21.15.22:6815/3355370212","state":["exists","up"]},{"osd":2,"uuid":"5952e710-f0b6-406c-855a-00f1cb8059c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":3627861698},{"type":"v1","addr":"172.21.15.22:6819","nonce":3627861698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":3627861698},{"type":"v1","addr":"172.21.15.22:6821","nonce":3627861698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":3627861698},{"type":"v1","addr":"172.21.15.22:6825","nonce":3627861698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":3627861698},{"type":"v1","addr":"172.21.15.22:6823","nonce":3627861698}]},"public_addr":"172.21.15.22:6819/3627861698","cluster_addr":"172.21.15.22:6821/3627861698","heartbeat_back_addr":"172.21.15.22:6825/3627861698","heartbeat_front_addr":"172.21.15.22:6823/3627861698","state":["exists","up"]},{"osd":3,"uuid":"5dc87494-e400-4111-91a7-306d9905febb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":2012503812},{"type":"v1","addr":"172.21.15.117:6801","nonce":2012503812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2012503812},{"type":"v1","addr":"172.21.15.117:6803","nonce":2012503812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2012503812},{"type":"v1","addr":"172.21.15.117:6807","nonce":2012503812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":2012503812},{"type":"v1","addr":"172.21.15.117:6805","nonce":2012503812}]},"public_addr":"172.21.15.117:6801/2012503812","cluster_addr":"172.21.15.117:6803/2012503812","heartbeat_back_addr":"172.21.15.117:6807/2012503812","heartbeat_front_addr":"172.21.15.117:6805/2012503812","state":["exists","up"]},{"osd":4,"uuid":"d27bde8d-1a77-4f95-acb6-7d05becd9480","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1856566932},{"type":"v1","addr":"172.21.15.117:6809","nonce":1856566932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":1856566932},{"type":"v1","addr":"172.21.15.117:6811","nonce":1856566932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":1856566932},{"type":"v1","addr":"172.21.15.117:6815","nonce":1856566932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1856566932},{"type":"v1","addr":"172.21.15.117:6813","nonce":1856566932}]},"public_addr":"172.21.15.117:6809/1856566932","cluster_addr":"172.21.15.117:6811/1856566932","heartbeat_back_addr":"172.21.15.117:6815/1856566932","heartbeat_front_addr":"172.21.15.117:6813/1856566932","state":["exists","up"]},{"osd":5,"uuid":"5873798f-3a2c-4a29-becd-6280b8866b6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":38288182},{"type":"v1","addr":"172.21.15.117:6817","nonce":38288182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":38288182},{"type":"v1","addr":"172.21.15.117:6819","nonce":38288182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":38288182},{"type":"v1","addr":"172.21.15.117:6823","nonce":38288182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":38288182},{"type":"v1","addr":"172.21.15.117:6821","nonce":38288182}]},"public_addr":"172.21.15.117:6817/38288182","cluster_addr":"172.21.15.117:6819/38288182","heartbeat_back_addr":"172.21.15.117:6823/38288182","heartbeat_front_addr":"172.21.15.117:6821/38288182","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:22:44.777427+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:23:14.935167+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:23:44.496184+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:24:07.447481+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:24:32.205064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:24:57.982560+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:6800/2945983569":"2024-02-17T19:21:26.760165+0000","172.21.15.22:0/668279955":"2024-02-17T19:21:26.760165+0000","172.21.15.22:0/2017591724":"2024-02-17T19:20:11.619146+0000","172.21.15.22:0/2384083375":"2024-02-17T19:21:26.760165+0000","172.21.15.22:6801/2755289428":"2024-02-17T19:20:11.619146+0000","172.21.15.22:6800/2755289428":"2024-02-17T19:20:11.619146+0000","172.21.15.22:6801/2945983569":"2024-02-17T19:21:26.760165+0000","172.21.15.22:6801/3109015836":"2024-02-17T19:19:38.544291+0000","172.21.15.22:6800/3109015836":"2024-02-17T19:19:38.544291+0000","172.21.15.22:0/1603216752":"2024-02-17T19:19:38.544291+0000","172.21.15.22:0/2099601555":"2024-02-17T19:20:11.619146+0000","172.21.15.22:0/1229330195":"2024-02-17T19:20:11.619146+0000","172.21.15.22:0/2371245756":"2024-02-17T19:19:38.544291+0000","172.21.15.22:0/3200629226":"2024-02-17T19:21:26.760165+0000","172.21.15.22:0/8466130":"2024-02-17T19:19:38.544291+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-16T19:25:05.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:05.849+0000 7f1984522700 1 -- 172.21.15.22:0/1369097948 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f196806c680 msgr2=0x7f196806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:05.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:05.849+0000 7f1984522700 1 --2- 172.21.15.22:0/1369097948 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f196806c680 0x7f196806eb30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f196c006fd0 tx=0x7f196c008040 comp rx=0 tx=0).stop 2024-02-16T19:25:05.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:05.849+0000 7f1984522700 1 -- 172.21.15.22:0/1369097948 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f197c100e80 msgr2=0x7f197c1964d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:05.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:05.849+0000 7f1984522700 1 --2- 172.21.15.22:0/1369097948 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f197c100e80 0x7f197c1964d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f19780092e0 tx=0x7f197800be20 comp rx=0 tx=0).stop 2024-02-16T19:25:05.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:05.850+0000 7f1984522700 1 -- 172.21.15.22:0/1369097948 shutdown_connections 2024-02-16T19:25:05.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:05.850+0000 7f1984522700 1 --2- 172.21.15.22:0/1369097948 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f196806c680 0x7f196806eb30 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:05.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:05.850+0000 7f1984522700 1 --2- 172.21.15.22:0/1369097948 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f197c1004e0 0x7f197c195f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:05.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:05.850+0000 7f1984522700 1 --2- 172.21.15.22:0/1369097948 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f197c100e80 0x7f197c1964d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:05.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:05.850+0000 7f1984522700 1 -- 172.21.15.22:0/1369097948 >> 172.21.15.22:0/1369097948 conn(0x7f197c0fbda0 msgr2=0x7f197c0fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:25:05.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:05.850+0000 7f1984522700 1 -- 172.21.15.22:0/1369097948 shutdown_connections 2024-02-16T19:25:05.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:05.850+0000 7f1984522700 1 -- 172.21.15.22:0/1369097948 wait complete. 2024-02-16T19:25:06.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:25:05 smithi117 ceph-mon[35309]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:25:06.158 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:25:05 smithi022 ceph-mon[29221]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:25:06.567 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-16T19:23:47.101426+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-16T19:25:06.567 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-02-16T19:25:06.781 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi022/config 2024-02-16T19:25:07.129 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:25:06 smithi022 ceph-mon[29221]: from='client.? 172.21.15.22:0/1369097948' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-16T19:25:07.136 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:25:06 smithi117 ceph-mon[35309]: from='client.? 172.21.15.22:0/1369097948' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-16T19:25:08.136 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:25:07 smithi117 ceph-mon[35309]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:25:08.194 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:25:07 smithi022 ceph-mon[29221]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:25:08.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:08.406+0000 7fabf6958700 1 -- 172.21.15.22:0/49603049 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fabf00ff3c0 msgr2=0x7fabf00ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:08.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:08.406+0000 7fabf6958700 1 --2- 172.21.15.22:0/49603049 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fabf00ff3c0 0x7fabf00ff790 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fabe0009910 tx=0x7fabe0009c20 comp rx=0 tx=0).stop 2024-02-16T19:25:08.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:08.407+0000 7fabf6958700 1 -- 172.21.15.22:0/49603049 shutdown_connections 2024-02-16T19:25:08.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:08.407+0000 7fabf6958700 1 --2- 172.21.15.22:0/49603049 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fabf00ffd60 0x7fabf0103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:08.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:08.407+0000 7fabf6958700 1 --2- 172.21.15.22:0/49603049 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fabf00ff3c0 0x7fabf00ff790 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:08.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:08.407+0000 7fabf6958700 1 -- 172.21.15.22:0/49603049 >> 172.21.15.22:0/49603049 conn(0x7fabf00fac80 msgr2=0x7fabf00fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:25:08.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:08.407+0000 7fabf6958700 1 -- 172.21.15.22:0/49603049 shutdown_connections 2024-02-16T19:25:08.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:08.407+0000 7fabf6958700 1 -- 172.21.15.22:0/49603049 wait complete. 2024-02-16T19:25:08.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:08.407+0000 7fabf6958700 1 Processor -- start 2024-02-16T19:25:08.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:08.407+0000 7fabf6958700 1 -- start start 2024-02-16T19:25:08.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:08.408+0000 7fabf6958700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fabf00ff3c0 0x7fabf0196de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:08.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:08.408+0000 7fabf6958700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fabf00ffd60 0x7fabf0197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:08.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:08.408+0000 7fabf6958700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf0197940 con 0x7fabf00ff3c0 2024-02-16T19:25:08.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:08.408+0000 7fabf6958700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf0197ab0 con 0x7fabf00ffd60 2024-02-16T19:25:08.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:08.408+0000 7fabeffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fabf00ff3c0 0x7fabf0196de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:08.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:08.408+0000 7fabeffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fabf00ff3c0 0x7fabf0196de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51830/0 (socket says 172.21.15.22:51830) 2024-02-16T19:25:08.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:08.408+0000 7fabeffff700 1 -- 172.21.15.22:0/521895471 learned_addr learned my addr 172.21.15.22:0/521895471 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:25:08.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:08.408+0000 7fabef7fe700 1 --2- 172.21.15.22:0/521895471 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fabf00ffd60 0x7fabf0197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:08.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:08.408+0000 7fabeffff700 1 -- 172.21.15.22:0/521895471 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fabf00ffd60 msgr2=0x7fabf0197320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:08.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:08.408+0000 7fabeffff700 1 --2- 172.21.15.22:0/521895471 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fabf00ffd60 0x7fabf0197320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:08.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:08.408+0000 7fabeffff700 1 -- 172.21.15.22:0/521895471 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabe00095f0 con 0x7fabf00ff3c0 2024-02-16T19:25:08.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:08.408+0000 7fabeffff700 1 --2- 172.21.15.22:0/521895471 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fabf00ff3c0 0x7fabf0196de0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fabe0006010 tx=0x7fabe000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:08.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:08.408+0000 7fabed7fa700 1 -- 172.21.15.22:0/521895471 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fabe001d070 con 0x7fabf00ff3c0 2024-02-16T19:25:08.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:08.408+0000 7fabed7fa700 1 -- 172.21.15.22:0/521895471 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fabe000f460 con 0x7fabf00ff3c0 2024-02-16T19:25:08.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:08.408+0000 7fabf6958700 1 -- 172.21.15.22:0/521895471 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabf019fe50 con 0x7fabf00ff3c0 2024-02-16T19:25:08.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:08.408+0000 7fabed7fa700 1 -- 172.21.15.22:0/521895471 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fabe0021640 con 0x7fabf00ff3c0 2024-02-16T19:25:08.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:08.408+0000 7fabf6958700 1 -- 172.21.15.22:0/521895471 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabf01a02c0 con 0x7fabf00ff3c0 2024-02-16T19:25:08.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:08.409+0000 7fabf6958700 1 -- 172.21.15.22:0/521895471 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabf01076d0 con 0x7fabf00ff3c0 2024-02-16T19:25:08.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:08.410+0000 7fabed7fa700 1 -- 172.21.15.22:0/521895471 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fabe000faf0 con 0x7fabf00ff3c0 2024-02-16T19:25:08.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:08.410+0000 7fabed7fa700 1 --2- 172.21.15.22:0/521895471 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fabd806c680 0x7fabd806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:08.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:08.410+0000 7fabed7fa700 1 -- 172.21.15.22:0/521895471 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fabe008c8f0 con 0x7fabf00ff3c0 2024-02-16T19:25:08.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:08.410+0000 7fabef7fe700 1 --2- 172.21.15.22:0/521895471 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fabd806c680 0x7fabd806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:08.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:08.411+0000 7fabef7fe700 1 --2- 172.21.15.22:0/521895471 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fabd806c680 0x7fabd806eb30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fabe4006fd0 tx=0x7fabe4008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:08.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:08.415+0000 7fabed7fa700 1 -- 172.21.15.22:0/521895471 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fabe0057310 con 0x7fabf00ff3c0 2024-02-16T19:25:08.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:08.614+0000 7fabf6958700 1 -- 172.21.15.22:0/521895471 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fabf01070b0 con 0x7fabf00ff3c0 2024-02-16T19:25:08.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:08.614+0000 7fabed7fa700 1 -- 172.21.15.22:0/521895471 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) v1 ==== 93+0+10 (secure 0 0 0) 0x7fabe00260a0 con 0x7fabf00ff3c0 2024-02-16T19:25:08.682 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-02-16T19:25:08.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:08.617+0000 7fabf6958700 1 -- 172.21.15.22:0/521895471 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fabd806c680 msgr2=0x7fabd806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:08.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:08.617+0000 7fabf6958700 1 --2- 172.21.15.22:0/521895471 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fabd806c680 0x7fabd806eb30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fabe4006fd0 tx=0x7fabe4008040 comp rx=0 tx=0).stop 2024-02-16T19:25:08.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:08.617+0000 7fabf6958700 1 -- 172.21.15.22:0/521895471 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fabf00ff3c0 msgr2=0x7fabf0196de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:08.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:08.617+0000 7fabf6958700 1 --2- 172.21.15.22:0/521895471 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fabf00ff3c0 0x7fabf0196de0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fabe0006010 tx=0x7fabe000be20 comp rx=0 tx=0).stop 2024-02-16T19:25:08.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:08.617+0000 7fabf6958700 1 -- 172.21.15.22:0/521895471 shutdown_connections 2024-02-16T19:25:08.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:08.617+0000 7fabf6958700 1 --2- 172.21.15.22:0/521895471 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fabf00ff3c0 0x7fabf0196de0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:08.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:08.617+0000 7fabf6958700 1 --2- 172.21.15.22:0/521895471 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fabd806c680 0x7fabd806eb30 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:08.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:08.617+0000 7fabf6958700 1 --2- 172.21.15.22:0/521895471 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fabf00ffd60 0x7fabf0197320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:08.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:08.617+0000 7fabf6958700 1 -- 172.21.15.22:0/521895471 >> 172.21.15.22:0/521895471 conn(0x7fabf00fac80 msgr2=0x7fabf00fc2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:25:08.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:08.617+0000 7fabf6958700 1 -- 172.21.15.22:0/521895471 shutdown_connections 2024-02-16T19:25:08.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:08.617+0000 7fabf6958700 1 -- 172.21.15.22:0/521895471 wait complete. 2024-02-16T19:25:08.947 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:25:08 smithi022 ceph-mon[29221]: from='client.? 172.21.15.22:0/521895471' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-16T19:25:09.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:25:08 smithi117 ceph-mon[35309]: from='client.? 172.21.15.22:0/521895471' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-16T19:25:09.229 INFO:tasks.cephadm:Setting up client nodes... 2024-02-16T19:25:09.229 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-02-16T19:25:09.436 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi022/config 2024-02-16T19:25:10.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:25:09 smithi117 ceph-mon[35309]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:25:10.188 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:25:09 smithi022 ceph-mon[29221]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:25:11.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:11.035+0000 7f0e65f57700 1 -- 172.21.15.22:0/1800290300 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e601003d0 msgr2=0x7f0e60100840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:11.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:11.035+0000 7f0e65f57700 1 --2- 172.21.15.22:0/1800290300 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e601003d0 0x7f0e60100840 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0e54009910 tx=0x7f0e54009c20 comp rx=0 tx=0).stop 2024-02-16T19:25:11.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:11.035+0000 7f0e65f57700 1 -- 172.21.15.22:0/1800290300 shutdown_connections 2024-02-16T19:25:11.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:11.035+0000 7f0e65f57700 1 --2- 172.21.15.22:0/1800290300 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e601003d0 0x7f0e60100840 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:11.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:11.035+0000 7f0e65f57700 1 --2- 172.21.15.22:0/1800290300 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e600ff130 0x7f0e600ff540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:11.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:11.035+0000 7f0e65f57700 1 -- 172.21.15.22:0/1800290300 >> 172.21.15.22:0/1800290300 conn(0x7f0e600fa780 msgr2=0x7f0e600fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:25:11.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:11.035+0000 7f0e65f57700 1 -- 172.21.15.22:0/1800290300 shutdown_connections 2024-02-16T19:25:11.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:11.035+0000 7f0e65f57700 1 -- 172.21.15.22:0/1800290300 wait complete. 2024-02-16T19:25:11.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:11.036+0000 7f0e65f57700 1 Processor -- start 2024-02-16T19:25:11.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:11.036+0000 7f0e65f57700 1 -- start start 2024-02-16T19:25:11.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:11.036+0000 7f0e65f57700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e600ff130 0x7f0e600690a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:11.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:11.036+0000 7f0e65f57700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e601003d0 0x7f0e6019a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:11.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:11.036+0000 7f0e65f57700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6019ad90 con 0x7f0e601003d0 2024-02-16T19:25:11.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:11.036+0000 7f0e65f57700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6019aed0 con 0x7f0e600ff130 2024-02-16T19:25:11.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:11.036+0000 7f0e5effd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e601003d0 0x7f0e6019a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:11.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:11.036+0000 7f0e5effd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e601003d0 0x7f0e6019a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51860/0 (socket says 172.21.15.22:51860) 2024-02-16T19:25:11.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:11.036+0000 7f0e5effd700 1 -- 172.21.15.22:0/2884613806 learned_addr learned my addr 172.21.15.22:0/2884613806 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:25:11.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:11.036+0000 7f0e5f7fe700 1 --2- 172.21.15.22:0/2884613806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e600ff130 0x7f0e600690a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:11.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:11.036+0000 7f0e5effd700 1 -- 172.21.15.22:0/2884613806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e600ff130 msgr2=0x7f0e600690a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:11.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:11.036+0000 7f0e5effd700 1 --2- 172.21.15.22:0/2884613806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e600ff130 0x7f0e600690a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:11.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:11.036+0000 7f0e5effd700 1 -- 172.21.15.22:0/2884613806 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e540095f0 con 0x7f0e601003d0 2024-02-16T19:25:11.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:11.037+0000 7f0e5effd700 1 --2- 172.21.15.22:0/2884613806 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e601003d0 0x7f0e6019a7c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0e54006010 tx=0x7f0e5400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:11.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:11.037+0000 7f0e5cff9700 1 -- 172.21.15.22:0/2884613806 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e5401d070 con 0x7f0e601003d0 2024-02-16T19:25:11.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:11.037+0000 7f0e5cff9700 1 -- 172.21.15.22:0/2884613806 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0e5400f460 con 0x7f0e601003d0 2024-02-16T19:25:11.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:11.037+0000 7f0e65f57700 1 -- 172.21.15.22:0/2884613806 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e6019b150 con 0x7f0e601003d0 2024-02-16T19:25:11.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:11.037+0000 7f0e5cff9700 1 -- 172.21.15.22:0/2884613806 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e54021640 con 0x7f0e601003d0 2024-02-16T19:25:11.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:11.037+0000 7f0e65f57700 1 -- 172.21.15.22:0/2884613806 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e6019b640 con 0x7f0e601003d0 2024-02-16T19:25:11.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:11.038+0000 7f0e5cff9700 1 -- 172.21.15.22:0/2884613806 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f0e5400f5d0 con 0x7f0e601003d0 2024-02-16T19:25:11.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:11.038+0000 7f0e65f57700 1 -- 172.21.15.22:0/2884613806 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e60107c10 con 0x7f0e601003d0 2024-02-16T19:25:11.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:11.039+0000 7f0e5cff9700 1 --2- 172.21.15.22:0/2884613806 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f0e4806c680 0x7f0e4806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:11.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:11.039+0000 7f0e5cff9700 1 -- 172.21.15.22:0/2884613806 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f0e5408c790 con 0x7f0e601003d0 2024-02-16T19:25:11.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:11.039+0000 7f0e5f7fe700 1 --2- 172.21.15.22:0/2884613806 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f0e4806c680 0x7f0e4806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:11.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:11.039+0000 7f0e5f7fe700 1 --2- 172.21.15.22:0/2884613806 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f0e4806c680 0x7f0e4806eb30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0e50005950 tx=0x7f0e5000b410 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:11.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:11.044+0000 7f0e5cff9700 1 -- 172.21.15.22:0/2884613806 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f0e5405ac40 con 0x7f0e601003d0 2024-02-16T19:25:11.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:11.325+0000 7f0e65f57700 1 -- 172.21.15.22:0/2884613806 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f0e6019bd70 con 0x7f0e601003d0 2024-02-16T19:25:11.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:11.328+0000 7f0e5cff9700 1 -- 172.21.15.22:0/2884613806 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v16) v1 ==== 170+0+59 (secure 0 0 0) 0x7f0e5405a7d0 con 0x7f0e601003d0 2024-02-16T19:25:11.329 INFO:teuthology.orchestra.run.smithi022.stdout:[client.0] 2024-02-16T19:25:11.330 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQCXts9ldUB4ExAACjNAbeVwKmgmOD6TdBfPHw== 2024-02-16T19:25:11.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:11.330+0000 7f0e65f57700 1 -- 172.21.15.22:0/2884613806 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f0e4806c680 msgr2=0x7f0e4806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:11.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:11.330+0000 7f0e65f57700 1 --2- 172.21.15.22:0/2884613806 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f0e4806c680 0x7f0e4806eb30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0e50005950 tx=0x7f0e5000b410 comp rx=0 tx=0).stop 2024-02-16T19:25:11.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:11.330+0000 7f0e65f57700 1 -- 172.21.15.22:0/2884613806 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e601003d0 msgr2=0x7f0e6019a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:11.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:11.330+0000 7f0e65f57700 1 --2- 172.21.15.22:0/2884613806 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e601003d0 0x7f0e6019a7c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0e54006010 tx=0x7f0e5400bd70 comp rx=0 tx=0).stop 2024-02-16T19:25:11.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:11.330+0000 7f0e65f57700 1 -- 172.21.15.22:0/2884613806 shutdown_connections 2024-02-16T19:25:11.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:11.330+0000 7f0e65f57700 1 --2- 172.21.15.22:0/2884613806 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f0e4806c680 0x7f0e4806eb30 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:11.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:11.330+0000 7f0e65f57700 1 --2- 172.21.15.22:0/2884613806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e600ff130 0x7f0e600690a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:11.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:11.330+0000 7f0e65f57700 1 --2- 172.21.15.22:0/2884613806 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e601003d0 0x7f0e6019a7c0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:11.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:11.330+0000 7f0e65f57700 1 -- 172.21.15.22:0/2884613806 >> 172.21.15.22:0/2884613806 conn(0x7f0e600fa780 msgr2=0x7f0e60103600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:25:11.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:11.331+0000 7f0e65f57700 1 -- 172.21.15.22:0/2884613806 shutdown_connections 2024-02-16T19:25:11.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:11.331+0000 7f0e65f57700 1 -- 172.21.15.22:0/2884613806 wait complete. 2024-02-16T19:25:11.936 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:25:11 smithi022 ceph-mon[29221]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:25:11.937 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:25:11 smithi022 ceph-mon[29221]: from='client.? 172.21.15.22:0/2884613806' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-16T19:25:11.937 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:25:11 smithi022 ceph-mon[29221]: from='client.? 172.21.15.22:0/2884613806' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-16T19:25:11.937 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:25:11 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:25:11.958 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-02-16T19:25:11.959 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-02-16T19:25:11.959 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-02-16T19:25:12.002 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-02-16T19:25:12.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:25:11 smithi117 ceph-mon[35309]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:25:12.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:25:11 smithi117 ceph-mon[35309]: from='client.? 172.21.15.22:0/2884613806' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-16T19:25:12.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:25:11 smithi117 ceph-mon[35309]: from='client.? 172.21.15.22:0/2884613806' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-16T19:25:12.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:25:11 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:25:12.221 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi117/config 2024-02-16T19:25:13.604 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:25:13.602+0000 7fd4efe50700 1 -- 172.21.15.117:0/671642521 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4e81003f0 msgr2=0x7fd4e8100860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:13.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:25:13.602+0000 7fd4efe50700 1 --2- 172.21.15.117:0/671642521 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4e81003f0 0x7fd4e8100860 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd4d8009910 tx=0x7fd4d8009c20 comp rx=0 tx=0).stop 2024-02-16T19:25:13.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:25:13.602+0000 7fd4efe50700 1 -- 172.21.15.117:0/671642521 shutdown_connections 2024-02-16T19:25:13.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:25:13.602+0000 7fd4efe50700 1 --2- 172.21.15.117:0/671642521 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4e81003f0 0x7fd4e8100860 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:13.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:25:13.602+0000 7fd4efe50700 1 --2- 172.21.15.117:0/671642521 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd4e80ff150 0x7fd4e80ff560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:13.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:25:13.602+0000 7fd4efe50700 1 -- 172.21.15.117:0/671642521 >> 172.21.15.117:0/671642521 conn(0x7fd4e80fa7a0 msgr2=0x7fd4e80fcbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:25:13.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:25:13.602+0000 7fd4efe50700 1 -- 172.21.15.117:0/671642521 shutdown_connections 2024-02-16T19:25:13.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:25:13.602+0000 7fd4efe50700 1 -- 172.21.15.117:0/671642521 wait complete. 2024-02-16T19:25:13.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:25:13.602+0000 7fd4efe50700 1 Processor -- start 2024-02-16T19:25:13.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:25:13.602+0000 7fd4efe50700 1 -- start start 2024-02-16T19:25:13.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:25:13.603+0000 7fd4efe50700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4e80ff150 0x7fd4e81071a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:13.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:25:13.603+0000 7fd4efe50700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd4e81003f0 0x7fd4e81057f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:13.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:25:13.603+0000 7fd4efe50700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e8105d30 con 0x7fd4e81003f0 2024-02-16T19:25:13.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:25:13.603+0000 7fd4efe50700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e8105e70 con 0x7fd4e80ff150 2024-02-16T19:25:13.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:25:13.603+0000 7fd4edbec700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4e80ff150 0x7fd4e81071a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:13.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:25:13.603+0000 7fd4edbec700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4e80ff150 0x7fd4e81071a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:54402/0 (socket says 172.21.15.117:54402) 2024-02-16T19:25:13.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:25:13.603+0000 7fd4edbec700 1 -- 172.21.15.117:0/4083686817 learned_addr learned my addr 172.21.15.117:0/4083686817 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-02-16T19:25:13.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:25:13.603+0000 7fd4ed3eb700 1 --2- 172.21.15.117:0/4083686817 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd4e81003f0 0x7fd4e81057f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:13.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:25:13.603+0000 7fd4edbec700 1 -- 172.21.15.117:0/4083686817 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd4e81003f0 msgr2=0x7fd4e81057f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:13.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:25:13.603+0000 7fd4edbec700 1 --2- 172.21.15.117:0/4083686817 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd4e81003f0 0x7fd4e81057f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:13.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:25:13.603+0000 7fd4edbec700 1 -- 172.21.15.117:0/4083686817 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4d80095f0 con 0x7fd4e80ff150 2024-02-16T19:25:13.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:25:13.603+0000 7fd4ed3eb700 1 --2- 172.21.15.117:0/4083686817 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd4e81003f0 0x7fd4e81057f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T19:25:13.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:25:13.603+0000 7fd4edbec700 1 --2- 172.21.15.117:0/4083686817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4e80ff150 0x7fd4e81071a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd4e400b4b0 tx=0x7fd4e400b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:13.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:25:13.603+0000 7fd4deffd700 1 -- 172.21.15.117:0/4083686817 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd4e4011800 con 0x7fd4e80ff150 2024-02-16T19:25:13.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:25:13.603+0000 7fd4efe50700 1 -- 172.21.15.117:0/4083686817 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4e8106150 con 0x7fd4e80ff150 2024-02-16T19:25:13.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:25:13.603+0000 7fd4deffd700 1 -- 172.21.15.117:0/4083686817 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd4e4011e10 con 0x7fd4e80ff150 2024-02-16T19:25:13.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:25:13.603+0000 7fd4deffd700 1 -- 172.21.15.117:0/4083686817 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd4e400f2e0 con 0x7fd4e80ff150 2024-02-16T19:25:13.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:25:13.603+0000 7fd4efe50700 1 -- 172.21.15.117:0/4083686817 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4e819deb0 con 0x7fd4e80ff150 2024-02-16T19:25:13.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:25:13.605+0000 7fd4efe50700 1 -- 172.21.15.117:0/4083686817 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4e8068ab0 con 0x7fd4e80ff150 2024-02-16T19:25:13.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:25:13.605+0000 7fd4deffd700 1 -- 172.21.15.117:0/4083686817 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fd4e4010380 con 0x7fd4e80ff150 2024-02-16T19:25:13.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:25:13.605+0000 7fd4deffd700 1 --2- 172.21.15.117:0/4083686817 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fd4d406c680 0x7fd4d406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:13.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:25:13.605+0000 7fd4deffd700 1 -- 172.21.15.117:0/4083686817 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fd4e408b180 con 0x7fd4e80ff150 2024-02-16T19:25:13.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:25:13.606+0000 7fd4ed3eb700 1 --2- 172.21.15.117:0/4083686817 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fd4d406c680 0x7fd4d406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:13.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:25:13.606+0000 7fd4ed3eb700 1 --2- 172.21.15.117:0/4083686817 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fd4d406c680 0x7fd4d406eb30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd4d80059d0 tx=0x7fd4d8005920 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:13.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:25:13.610+0000 7fd4deffd700 1 -- 172.21.15.117:0/4083686817 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fd4e4059630 con 0x7fd4e80ff150 2024-02-16T19:25:13.874 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:25:13.873+0000 7fd4efe50700 1 -- 172.21.15.117:0/4083686817 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7fd4e819e040 con 0x7fd4e80ff150 2024-02-16T19:25:13.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:25:13.878+0000 7fd4deffd700 1 -- 172.21.15.117:0/4083686817 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v17) v1 ==== 170+0+59 (secure 0 0 0) 0x7fd4e40591c0 con 0x7fd4e80ff150 2024-02-16T19:25:13.878 INFO:teuthology.orchestra.run.smithi117.stdout:[client.1] 2024-02-16T19:25:13.879 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQCZts9ltyEwNBAAhBn5LDgQeXRuIQ1/IzNfvw== 2024-02-16T19:25:13.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:25:13.880+0000 7fd4efe50700 1 -- 172.21.15.117:0/4083686817 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fd4d406c680 msgr2=0x7fd4d406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:13.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:25:13.880+0000 7fd4efe50700 1 --2- 172.21.15.117:0/4083686817 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fd4d406c680 0x7fd4d406eb30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd4d80059d0 tx=0x7fd4d8005920 comp rx=0 tx=0).stop 2024-02-16T19:25:13.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:25:13.880+0000 7fd4efe50700 1 -- 172.21.15.117:0/4083686817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4e80ff150 msgr2=0x7fd4e81071a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:13.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:25:13.880+0000 7fd4efe50700 1 --2- 172.21.15.117:0/4083686817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4e80ff150 0x7fd4e81071a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd4e400b4b0 tx=0x7fd4e400b870 comp rx=0 tx=0).stop 2024-02-16T19:25:13.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:25:13.880+0000 7fd4efe50700 1 -- 172.21.15.117:0/4083686817 shutdown_connections 2024-02-16T19:25:13.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:25:13.880+0000 7fd4efe50700 1 --2- 172.21.15.117:0/4083686817 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fd4d406c680 0x7fd4d406eb30 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:13.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:25:13.880+0000 7fd4efe50700 1 --2- 172.21.15.117:0/4083686817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4e80ff150 0x7fd4e81071a0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:13.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:25:13.880+0000 7fd4efe50700 1 --2- 172.21.15.117:0/4083686817 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd4e81003f0 0x7fd4e81057f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:13.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:25:13.880+0000 7fd4efe50700 1 -- 172.21.15.117:0/4083686817 >> 172.21.15.117:0/4083686817 conn(0x7fd4e80fa7a0 msgr2=0x7fd4e81036c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:25:13.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:25:13.880+0000 7fd4efe50700 1 -- 172.21.15.117:0/4083686817 shutdown_connections 2024-02-16T19:25:13.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:25:13.880+0000 7fd4efe50700 1 -- 172.21.15.117:0/4083686817 wait complete. 2024-02-16T19:25:14.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:25:13 smithi117 ceph-mon[35309]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:25:14.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:25:13 smithi022 ceph-mon[29221]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:25:14.468 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-02-16T19:25:14.468 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-02-16T19:25:14.469 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-02-16T19:25:14.510 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-02-16T19:25:14.510 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-02-16T19:25:14.510 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -- ceph mgr dump --format=json 2024-02-16T19:25:14.732 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi022/config 2024-02-16T19:25:15.104 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:25:14 smithi022 ceph-mon[29221]: from='client.? 172.21.15.117:0/4083686817' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-16T19:25:15.104 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:25:14 smithi022 ceph-mon[29221]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-16T19:25:15.104 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:25:14 smithi022 ceph-mon[29221]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-16T19:25:15.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:25:14 smithi117 ceph-mon[35309]: from='client.? 172.21.15.117:0/4083686817' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-16T19:25:15.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:25:14 smithi117 ceph-mon[35309]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-16T19:25:15.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:25:14 smithi117 ceph-mon[35309]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-16T19:25:16.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:25:15 smithi117 ceph-mon[35309]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:25:16.157 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:25:15 smithi022 ceph-mon[29221]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:25:16.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:16.517+0000 7fa378295700 1 -- 172.21.15.22:0/1787802707 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3700ffd60 msgr2=0x7fa370103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:16.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:16.517+0000 7fa378295700 1 --2- 172.21.15.22:0/1787802707 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3700ffd60 0x7fa370103d30 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa364009910 tx=0x7fa364009c20 comp rx=0 tx=0).stop 2024-02-16T19:25:16.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:16.517+0000 7fa378295700 1 -- 172.21.15.22:0/1787802707 shutdown_connections 2024-02-16T19:25:16.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:16.517+0000 7fa378295700 1 --2- 172.21.15.22:0/1787802707 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3700ffd60 0x7fa370103d30 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:16.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:16.517+0000 7fa378295700 1 --2- 172.21.15.22:0/1787802707 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3700ff3c0 0x7fa3700ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:16.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:16.517+0000 7fa378295700 1 -- 172.21.15.22:0/1787802707 >> 172.21.15.22:0/1787802707 conn(0x7fa3700fac80 msgr2=0x7fa3700fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:25:16.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:16.517+0000 7fa378295700 1 -- 172.21.15.22:0/1787802707 shutdown_connections 2024-02-16T19:25:16.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:16.517+0000 7fa378295700 1 -- 172.21.15.22:0/1787802707 wait complete. 2024-02-16T19:25:16.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:16.518+0000 7fa378295700 1 Processor -- start 2024-02-16T19:25:16.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:16.518+0000 7fa378295700 1 -- start start 2024-02-16T19:25:16.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:16.518+0000 7fa378295700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3700ff3c0 0x7fa370100dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:16.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:16.518+0000 7fa378295700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3700ffd60 0x7fa370101310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:16.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:16.518+0000 7fa378295700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa370101850 con 0x7fa3700ff3c0 2024-02-16T19:25:16.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:16.518+0000 7fa378295700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa370101990 con 0x7fa3700ffd60 2024-02-16T19:25:16.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:16.518+0000 7fa376031700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3700ff3c0 0x7fa370100dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:16.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:16.518+0000 7fa376031700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3700ff3c0 0x7fa370100dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42344/0 (socket says 172.21.15.22:42344) 2024-02-16T19:25:16.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:16.518+0000 7fa376031700 1 -- 172.21.15.22:0/4102242790 learned_addr learned my addr 172.21.15.22:0/4102242790 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:25:16.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:16.518+0000 7fa375830700 1 --2- 172.21.15.22:0/4102242790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3700ffd60 0x7fa370101310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:16.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:16.518+0000 7fa376031700 1 -- 172.21.15.22:0/4102242790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3700ffd60 msgr2=0x7fa370101310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:16.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:16.518+0000 7fa376031700 1 --2- 172.21.15.22:0/4102242790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3700ffd60 0x7fa370101310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:16.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:16.518+0000 7fa376031700 1 -- 172.21.15.22:0/4102242790 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3640095f0 con 0x7fa3700ff3c0 2024-02-16T19:25:16.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:16.518+0000 7fa376031700 1 --2- 172.21.15.22:0/4102242790 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3700ff3c0 0x7fa370100dd0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa36800ada0 tx=0x7fa36800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:16.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:16.519+0000 7fa36f7fe700 1 -- 172.21.15.22:0/4102242790 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa36800f070 con 0x7fa3700ff3c0 2024-02-16T19:25:16.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:16.519+0000 7fa378295700 1 -- 172.21.15.22:0/4102242790 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3701a2070 con 0x7fa3700ff3c0 2024-02-16T19:25:16.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:16.519+0000 7fa36f7fe700 1 -- 172.21.15.22:0/4102242790 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa368007bf0 con 0x7fa3700ff3c0 2024-02-16T19:25:16.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:16.519+0000 7fa36f7fe700 1 -- 172.21.15.22:0/4102242790 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa368005670 con 0x7fa3700ff3c0 2024-02-16T19:25:16.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:16.519+0000 7fa378295700 1 -- 172.21.15.22:0/4102242790 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3701a2480 con 0x7fa3700ff3c0 2024-02-16T19:25:16.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:16.520+0000 7fa378295700 1 -- 172.21.15.22:0/4102242790 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa37004f2a0 con 0x7fa3700ff3c0 2024-02-16T19:25:16.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:16.520+0000 7fa36f7fe700 1 -- 172.21.15.22:0/4102242790 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fa368005940 con 0x7fa3700ff3c0 2024-02-16T19:25:16.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:16.520+0000 7fa36f7fe700 1 --2- 172.21.15.22:0/4102242790 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fa35806c680 0x7fa35806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:16.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:16.521+0000 7fa36f7fe700 1 -- 172.21.15.22:0/4102242790 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fa36808af80 con 0x7fa3700ff3c0 2024-02-16T19:25:16.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:16.521+0000 7fa375830700 1 --2- 172.21.15.22:0/4102242790 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fa35806c680 0x7fa35806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:16.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:16.521+0000 7fa375830700 1 --2- 172.21.15.22:0/4102242790 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fa35806c680 0x7fa35806eb30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa364006010 tx=0x7fa36400b540 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:16.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:16.525+0000 7fa36f7fe700 1 -- 172.21.15.22:0/4102242790 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fa3680559a0 con 0x7fa3700ff3c0 2024-02-16T19:25:16.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:16.781+0000 7fa378295700 1 -- 172.21.15.22:0/4102242790 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fa3701a26c0 con 0x7fa3700ff3c0 2024-02-16T19:25:16.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:16.784+0000 7fa36f7fe700 1 -- 172.21.15.22:0/4102242790 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v19) v1 ==== 74+0+172853 (secure 0 0 0) 0x7fa368058fc0 con 0x7fa3700ff3c0 2024-02-16T19:25:16.788 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:25:16.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:16.788+0000 7fa378295700 1 -- 172.21.15.22:0/4102242790 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fa35806c680 msgr2=0x7fa35806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:16.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:16.788+0000 7fa378295700 1 --2- 172.21.15.22:0/4102242790 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fa35806c680 0x7fa35806eb30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa364006010 tx=0x7fa36400b540 comp rx=0 tx=0).stop 2024-02-16T19:25:16.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:16.788+0000 7fa378295700 1 -- 172.21.15.22:0/4102242790 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3700ff3c0 msgr2=0x7fa370100dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:16.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:16.788+0000 7fa378295700 1 --2- 172.21.15.22:0/4102242790 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3700ff3c0 0x7fa370100dd0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa36800ada0 tx=0x7fa36800c330 comp rx=0 tx=0).stop 2024-02-16T19:25:16.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:16.788+0000 7fa378295700 1 -- 172.21.15.22:0/4102242790 shutdown_connections 2024-02-16T19:25:16.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:16.788+0000 7fa378295700 1 --2- 172.21.15.22:0/4102242790 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3700ff3c0 0x7fa370100dd0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:16.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:16.788+0000 7fa378295700 1 --2- 172.21.15.22:0/4102242790 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fa35806c680 0x7fa35806eb30 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:16.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:16.788+0000 7fa378295700 1 --2- 172.21.15.22:0/4102242790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3700ffd60 0x7fa370101310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:16.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:16.788+0000 7fa378295700 1 -- 172.21.15.22:0/4102242790 >> 172.21.15.22:0/4102242790 conn(0x7fa3700fac80 msgr2=0x7fa370068e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:25:16.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:16.788+0000 7fa378295700 1 -- 172.21.15.22:0/4102242790 shutdown_connections 2024-02-16T19:25:16.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:16.788+0000 7fa378295700 1 -- 172.21.15.22:0/4102242790 wait complete. 2024-02-16T19:25:17.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:25:16 smithi117 ceph-mon[35309]: from='client.? 172.21.15.22:0/4102242790' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-16T19:25:17.143 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:25:16 smithi022 ceph-mon[29221]: from='client.? 172.21.15.22:0/4102242790' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-16T19:25:17.568 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":19,"active_gid":14217,"active_name":"smithi022.gmdcch","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":2253867279},{"type":"v1","addr":"172.21.15.22:6801","nonce":2253867279}]},"active_addr":"172.21.15.22:6801/2253867279","active_change":"2024-02-16T19:21:26.760333+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":14240,"name":"smithi117.cohbtv","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph:v18","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph:v18","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.22:8443/","prometheus":"http://172.21.15.22:9283/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.22:0","nonce":206933569}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.22:0","nonce":402850490}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.22:0","nonce":2705605267}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.22:0","nonce":2301835367}]}]} 2024-02-16T19:25:17.571 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-02-16T19:25:17.572 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-02-16T19:25:17.572 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -- ceph osd dump --format=json 2024-02-16T19:25:17.785 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi022/config 2024-02-16T19:25:18.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:25:17 smithi117 ceph-mon[35309]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:25:18.140 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:25:17 smithi022 ceph-mon[29221]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:25:19.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:19.571+0000 7feadb84f700 1 -- 172.21.15.22:0/2286475656 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fead41004e0 msgr2=0x7fead41008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:19.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:19.571+0000 7feadb84f700 1 --2- 172.21.15.22:0/2286475656 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fead41004e0 0x7fead41008b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7feacc009910 tx=0x7feacc009c20 comp rx=0 tx=0).stop 2024-02-16T19:25:19.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:19.571+0000 7feadb84f700 1 -- 172.21.15.22:0/2286475656 shutdown_connections 2024-02-16T19:25:19.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:19.571+0000 7feadb84f700 1 --2- 172.21.15.22:0/2286475656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fead4100e80 0x7fead4104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:19.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:19.571+0000 7feadb84f700 1 --2- 172.21.15.22:0/2286475656 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fead41004e0 0x7fead41008b0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:19.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:19.571+0000 7feadb84f700 1 -- 172.21.15.22:0/2286475656 >> 172.21.15.22:0/2286475656 conn(0x7fead40fbda0 msgr2=0x7fead40fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:25:19.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:19.571+0000 7feadb84f700 1 -- 172.21.15.22:0/2286475656 shutdown_connections 2024-02-16T19:25:19.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:19.571+0000 7feadb84f700 1 -- 172.21.15.22:0/2286475656 wait complete. 2024-02-16T19:25:19.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:19.572+0000 7feadb84f700 1 Processor -- start 2024-02-16T19:25:19.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:19.572+0000 7feadb84f700 1 -- start start 2024-02-16T19:25:19.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:19.572+0000 7feadb84f700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fead41004e0 0x7fead4073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:19.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:19.572+0000 7feadb84f700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fead4100e80 0x7fead4073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:19.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:19.572+0000 7feadb84f700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fead4073ad0 con 0x7fead4100e80 2024-02-16T19:25:19.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:19.572+0000 7feadb84f700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fead4073c10 con 0x7fead41004e0 2024-02-16T19:25:19.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:19.572+0000 7fead8dea700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fead4100e80 0x7fead4073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:19.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:19.572+0000 7fead8dea700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fead4100e80 0x7fead4073590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42366/0 (socket says 172.21.15.22:42366) 2024-02-16T19:25:19.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:19.572+0000 7fead8dea700 1 -- 172.21.15.22:0/497720813 learned_addr learned my addr 172.21.15.22:0/497720813 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:25:19.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:19.572+0000 7fead95eb700 1 --2- 172.21.15.22:0/497720813 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fead41004e0 0x7fead4073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:19.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:19.572+0000 7fead8dea700 1 -- 172.21.15.22:0/497720813 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fead41004e0 msgr2=0x7fead4073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:19.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:19.572+0000 7fead8dea700 1 --2- 172.21.15.22:0/497720813 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fead41004e0 0x7fead4073050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:19.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:19.572+0000 7fead8dea700 1 -- 172.21.15.22:0/497720813 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feacc0095f0 con 0x7fead4100e80 2024-02-16T19:25:19.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:19.573+0000 7fead8dea700 1 --2- 172.21.15.22:0/497720813 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fead4100e80 0x7fead4073590 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7feac800b480 tx=0x7feac800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:19.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:19.573+0000 7fead27fc700 1 -- 172.21.15.22:0/497720813 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feac80118c0 con 0x7fead4100e80 2024-02-16T19:25:19.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:19.573+0000 7fead27fc700 1 -- 172.21.15.22:0/497720813 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7feac8010380 con 0x7fead4100e80 2024-02-16T19:25:19.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:19.573+0000 7feadb84f700 1 -- 172.21.15.22:0/497720813 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fead41a2140 con 0x7fead4100e80 2024-02-16T19:25:19.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:19.573+0000 7fead27fc700 1 -- 172.21.15.22:0/497720813 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feac800f4f0 con 0x7fead4100e80 2024-02-16T19:25:19.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:19.573+0000 7feadb84f700 1 -- 172.21.15.22:0/497720813 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fead41a2550 con 0x7fead4100e80 2024-02-16T19:25:19.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:19.574+0000 7feadb84f700 1 -- 172.21.15.22:0/497720813 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fead41087f0 con 0x7fead4100e80 2024-02-16T19:25:19.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:19.575+0000 7fead27fc700 1 -- 172.21.15.22:0/497720813 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7feac80104f0 con 0x7fead4100e80 2024-02-16T19:25:19.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:19.575+0000 7fead27fc700 1 --2- 172.21.15.22:0/497720813 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7feabc06c680 0x7feabc06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:19.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:19.575+0000 7fead27fc700 1 -- 172.21.15.22:0/497720813 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7feac808b180 con 0x7fead4100e80 2024-02-16T19:25:19.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:19.575+0000 7fead95eb700 1 --2- 172.21.15.22:0/497720813 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7feabc06c680 0x7feabc06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:19.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:19.576+0000 7fead95eb700 1 --2- 172.21.15.22:0/497720813 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7feabc06c680 0x7feabc06eb30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7feacc00b5c0 tx=0x7feacc01a040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:19.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:19.580+0000 7fead27fc700 1 -- 172.21.15.22:0/497720813 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7feac8055ba0 con 0x7fead4100e80 2024-02-16T19:25:19.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:19.781+0000 7feadb84f700 1 -- 172.21.15.22:0/497720813 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fead404f2a0 con 0x7fead4100e80 2024-02-16T19:25:19.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:19.782+0000 7fead27fc700 1 -- 172.21.15.22:0/497720813 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+11323 (secure 0 0 0) 0x7feac80591c0 con 0x7fead4100e80 2024-02-16T19:25:19.784 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:25:19.784 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":41,"fsid":"2a1346e6-cd00-11ee-95ba-87774f69a715","created":"2024-02-16T19:19:11.528280+0000","modified":"2024-02-16T19:25:00.824993+0000","last_up_change":"2024-02-16T19:24:59.819598+0000","last_in_change":"2024-02-16T19:24:40.575373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-16T19:23:47.101426+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d16e47f2-7209-45f4-ad50-1e0f6789ffad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2126256017},{"type":"v1","addr":"172.21.15.22:6803","nonce":2126256017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2126256017},{"type":"v1","addr":"172.21.15.22:6805","nonce":2126256017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2126256017},{"type":"v1","addr":"172.21.15.22:6809","nonce":2126256017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2126256017},{"type":"v1","addr":"172.21.15.22:6807","nonce":2126256017}]},"public_addr":"172.21.15.22:6803/2126256017","cluster_addr":"172.21.15.22:6805/2126256017","heartbeat_back_addr":"172.21.15.22:6809/2126256017","heartbeat_front_addr":"172.21.15.22:6807/2126256017","state":["exists","up"]},{"osd":1,"uuid":"59c3d085-3422-4e92-9048-46d7a39e31a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":3355370212},{"type":"v1","addr":"172.21.15.22:6811","nonce":3355370212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":3355370212},{"type":"v1","addr":"172.21.15.22:6813","nonce":3355370212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":3355370212},{"type":"v1","addr":"172.21.15.22:6817","nonce":3355370212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":3355370212},{"type":"v1","addr":"172.21.15.22:6815","nonce":3355370212}]},"public_addr":"172.21.15.22:6811/3355370212","cluster_addr":"172.21.15.22:6813/3355370212","heartbeat_back_addr":"172.21.15.22:6817/3355370212","heartbeat_front_addr":"172.21.15.22:6815/3355370212","state":["exists","up"]},{"osd":2,"uuid":"5952e710-f0b6-406c-855a-00f1cb8059c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":3627861698},{"type":"v1","addr":"172.21.15.22:6819","nonce":3627861698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":3627861698},{"type":"v1","addr":"172.21.15.22:6821","nonce":3627861698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":3627861698},{"type":"v1","addr":"172.21.15.22:6825","nonce":3627861698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":3627861698},{"type":"v1","addr":"172.21.15.22:6823","nonce":3627861698}]},"public_addr":"172.21.15.22:6819/3627861698","cluster_addr":"172.21.15.22:6821/3627861698","heartbeat_back_addr":"172.21.15.22:6825/3627861698","heartbeat_front_addr":"172.21.15.22:6823/3627861698","state":["exists","up"]},{"osd":3,"uuid":"5dc87494-e400-4111-91a7-306d9905febb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":2012503812},{"type":"v1","addr":"172.21.15.117:6801","nonce":2012503812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2012503812},{"type":"v1","addr":"172.21.15.117:6803","nonce":2012503812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2012503812},{"type":"v1","addr":"172.21.15.117:6807","nonce":2012503812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":2012503812},{"type":"v1","addr":"172.21.15.117:6805","nonce":2012503812}]},"public_addr":"172.21.15.117:6801/2012503812","cluster_addr":"172.21.15.117:6803/2012503812","heartbeat_back_addr":"172.21.15.117:6807/2012503812","heartbeat_front_addr":"172.21.15.117:6805/2012503812","state":["exists","up"]},{"osd":4,"uuid":"d27bde8d-1a77-4f95-acb6-7d05becd9480","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1856566932},{"type":"v1","addr":"172.21.15.117:6809","nonce":1856566932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":1856566932},{"type":"v1","addr":"172.21.15.117:6811","nonce":1856566932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":1856566932},{"type":"v1","addr":"172.21.15.117:6815","nonce":1856566932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1856566932},{"type":"v1","addr":"172.21.15.117:6813","nonce":1856566932}]},"public_addr":"172.21.15.117:6809/1856566932","cluster_addr":"172.21.15.117:6811/1856566932","heartbeat_back_addr":"172.21.15.117:6815/1856566932","heartbeat_front_addr":"172.21.15.117:6813/1856566932","state":["exists","up"]},{"osd":5,"uuid":"5873798f-3a2c-4a29-becd-6280b8866b6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":38288182},{"type":"v1","addr":"172.21.15.117:6817","nonce":38288182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":38288182},{"type":"v1","addr":"172.21.15.117:6819","nonce":38288182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":38288182},{"type":"v1","addr":"172.21.15.117:6823","nonce":38288182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":38288182},{"type":"v1","addr":"172.21.15.117:6821","nonce":38288182}]},"public_addr":"172.21.15.117:6817/38288182","cluster_addr":"172.21.15.117:6819/38288182","heartbeat_back_addr":"172.21.15.117:6823/38288182","heartbeat_front_addr":"172.21.15.117:6821/38288182","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:22:44.777427+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:23:14.935167+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:23:44.496184+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:24:07.447481+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:24:32.205064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:24:57.982560+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:6800/2945983569":"2024-02-17T19:21:26.760165+0000","172.21.15.22:0/668279955":"2024-02-17T19:21:26.760165+0000","172.21.15.22:0/2017591724":"2024-02-17T19:20:11.619146+0000","172.21.15.22:0/2384083375":"2024-02-17T19:21:26.760165+0000","172.21.15.22:6801/2755289428":"2024-02-17T19:20:11.619146+0000","172.21.15.22:6800/2755289428":"2024-02-17T19:20:11.619146+0000","172.21.15.22:6801/2945983569":"2024-02-17T19:21:26.760165+0000","172.21.15.22:6801/3109015836":"2024-02-17T19:19:38.544291+0000","172.21.15.22:6800/3109015836":"2024-02-17T19:19:38.544291+0000","172.21.15.22:0/1603216752":"2024-02-17T19:19:38.544291+0000","172.21.15.22:0/2099601555":"2024-02-17T19:20:11.619146+0000","172.21.15.22:0/1229330195":"2024-02-17T19:20:11.619146+0000","172.21.15.22:0/2371245756":"2024-02-17T19:19:38.544291+0000","172.21.15.22:0/3200629226":"2024-02-17T19:21:26.760165+0000","172.21.15.22:0/8466130":"2024-02-17T19:19:38.544291+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-16T19:25:19.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:19.785+0000 7feadb84f700 1 -- 172.21.15.22:0/497720813 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7feabc06c680 msgr2=0x7feabc06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:19.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:19.785+0000 7feadb84f700 1 --2- 172.21.15.22:0/497720813 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7feabc06c680 0x7feabc06eb30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7feacc00b5c0 tx=0x7feacc01a040 comp rx=0 tx=0).stop 2024-02-16T19:25:19.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:19.785+0000 7feadb84f700 1 -- 172.21.15.22:0/497720813 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fead4100e80 msgr2=0x7fead4073590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:19.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:19.785+0000 7feadb84f700 1 --2- 172.21.15.22:0/497720813 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fead4100e80 0x7fead4073590 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7feac800b480 tx=0x7feac800b840 comp rx=0 tx=0).stop 2024-02-16T19:25:19.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:19.785+0000 7feadb84f700 1 -- 172.21.15.22:0/497720813 shutdown_connections 2024-02-16T19:25:19.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:19.785+0000 7feadb84f700 1 --2- 172.21.15.22:0/497720813 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7feabc06c680 0x7feabc06eb30 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:19.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:19.785+0000 7feadb84f700 1 --2- 172.21.15.22:0/497720813 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fead41004e0 0x7fead4073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:19.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:19.785+0000 7feadb84f700 1 --2- 172.21.15.22:0/497720813 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fead4100e80 0x7fead4073590 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:19.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:19.785+0000 7feadb84f700 1 -- 172.21.15.22:0/497720813 >> 172.21.15.22:0/497720813 conn(0x7fead40fbda0 msgr2=0x7fead40fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:25:19.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:19.785+0000 7feadb84f700 1 -- 172.21.15.22:0/497720813 shutdown_connections 2024-02-16T19:25:19.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:19.785+0000 7feadb84f700 1 -- 172.21.15.22:0/497720813 wait complete. 2024-02-16T19:25:20.197 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:25:19 smithi022 ceph-mon[29221]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:25:20.198 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:25:19 smithi022 ceph-mon[29221]: from='client.? 172.21.15.22:0/497720813' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-16T19:25:20.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:25:19 smithi117 ceph-mon[35309]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:25:20.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:25:19 smithi117 ceph-mon[35309]: from='client.? 172.21.15.22:0/497720813' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-16T19:25:20.581 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-02-16T19:25:20.581 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -- ceph osd dump --format=json 2024-02-16T19:25:20.793 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi022/config 2024-02-16T19:25:22.160 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:25:21 smithi022 ceph-mon[29221]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:25:22.188 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:25:21 smithi117 ceph-mon[35309]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:25:22.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:22.489+0000 7f921705d700 1 -- 172.21.15.22:0/1529286702 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f92101004e0 msgr2=0x7f92101008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:22.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:22.489+0000 7f921705d700 1 --2- 172.21.15.22:0/1529286702 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f92101004e0 0x7f92101008b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f920c009910 tx=0x7f920c009c20 comp rx=0 tx=0).stop 2024-02-16T19:25:22.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:22.490+0000 7f921705d700 1 -- 172.21.15.22:0/1529286702 shutdown_connections 2024-02-16T19:25:22.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:22.490+0000 7f921705d700 1 --2- 172.21.15.22:0/1529286702 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9210100e80 0x7f9210104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:22.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:22.490+0000 7f921705d700 1 --2- 172.21.15.22:0/1529286702 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f92101004e0 0x7f92101008b0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:22.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:22.490+0000 7f921705d700 1 -- 172.21.15.22:0/1529286702 >> 172.21.15.22:0/1529286702 conn(0x7f92100fbda0 msgr2=0x7f92100fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:25:22.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:22.490+0000 7f921705d700 1 -- 172.21.15.22:0/1529286702 shutdown_connections 2024-02-16T19:25:22.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:22.490+0000 7f921705d700 1 -- 172.21.15.22:0/1529286702 wait complete. 2024-02-16T19:25:22.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:22.490+0000 7f921705d700 1 Processor -- start 2024-02-16T19:25:22.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:22.490+0000 7f921705d700 1 -- start start 2024-02-16T19:25:22.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:22.490+0000 7f921705d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92101004e0 0x7f9210195ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:22.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:22.490+0000 7f921705d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9210100e80 0x7f9210196430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:22.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:22.490+0000 7f921705d700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9210196b10 con 0x7f9210100e80 2024-02-16T19:25:22.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:22.490+0000 7f921705d700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f921019a8a0 con 0x7f92101004e0 2024-02-16T19:25:22.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:22.491+0000 7f9207fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9210100e80 0x7f9210196430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:22.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:22.491+0000 7f9207fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9210100e80 0x7f9210196430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42386/0 (socket says 172.21.15.22:42386) 2024-02-16T19:25:22.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:22.491+0000 7f9207fff700 1 -- 172.21.15.22:0/4088921635 learned_addr learned my addr 172.21.15.22:0/4088921635 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:25:22.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:22.491+0000 7f9214df9700 1 --2- 172.21.15.22:0/4088921635 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92101004e0 0x7f9210195ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:22.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:22.491+0000 7f9207fff700 1 -- 172.21.15.22:0/4088921635 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92101004e0 msgr2=0x7f9210195ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:22.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:22.491+0000 7f9207fff700 1 --2- 172.21.15.22:0/4088921635 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92101004e0 0x7f9210195ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:22.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:22.491+0000 7f9207fff700 1 -- 172.21.15.22:0/4088921635 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f920c0095f0 con 0x7f9210100e80 2024-02-16T19:25:22.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:22.491+0000 7f9207fff700 1 --2- 172.21.15.22:0/4088921635 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9210100e80 0x7f9210196430 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f920000d900 tx=0x7f920000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:22.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:22.491+0000 7f9205ffb700 1 -- 172.21.15.22:0/4088921635 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9200009900 con 0x7f9210100e80 2024-02-16T19:25:22.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:22.491+0000 7f921705d700 1 -- 172.21.15.22:0/4088921635 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f921019ab80 con 0x7f9210100e80 2024-02-16T19:25:22.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:22.491+0000 7f9205ffb700 1 -- 172.21.15.22:0/4088921635 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9200010460 con 0x7f9210100e80 2024-02-16T19:25:22.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:22.491+0000 7f9205ffb700 1 -- 172.21.15.22:0/4088921635 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f920000f5f0 con 0x7f9210100e80 2024-02-16T19:25:22.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:22.491+0000 7f921705d700 1 -- 172.21.15.22:0/4088921635 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f921019b0d0 con 0x7f9210100e80 2024-02-16T19:25:22.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:22.492+0000 7f921705d700 1 -- 172.21.15.22:0/4088921635 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92101087f0 con 0x7f9210100e80 2024-02-16T19:25:22.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:22.493+0000 7f9205ffb700 1 -- 172.21.15.22:0/4088921635 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f92000105d0 con 0x7f9210100e80 2024-02-16T19:25:22.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:22.493+0000 7f9205ffb700 1 --2- 172.21.15.22:0/4088921635 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f91f806c680 0x7f91f806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:22.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:22.493+0000 7f9205ffb700 1 -- 172.21.15.22:0/4088921635 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f920008b0d0 con 0x7f9210100e80 2024-02-16T19:25:22.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:22.493+0000 7f9214df9700 1 --2- 172.21.15.22:0/4088921635 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f91f806c680 0x7f91f806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:22.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:22.494+0000 7f9214df9700 1 --2- 172.21.15.22:0/4088921635 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f91f806c680 0x7f91f806eb30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f920c006010 tx=0x7f920c01a040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:22.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:22.498+0000 7f9205ffb700 1 -- 172.21.15.22:0/4088921635 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f9200055af0 con 0x7f9210100e80 2024-02-16T19:25:22.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:22.699+0000 7f921705d700 1 -- 172.21.15.22:0/4088921635 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f921019b2f0 con 0x7f9210100e80 2024-02-16T19:25:22.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:22.700+0000 7f9205ffb700 1 -- 172.21.15.22:0/4088921635 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+11323 (secure 0 0 0) 0x7f9200059110 con 0x7f9210100e80 2024-02-16T19:25:22.701 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:25:22.701 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":41,"fsid":"2a1346e6-cd00-11ee-95ba-87774f69a715","created":"2024-02-16T19:19:11.528280+0000","modified":"2024-02-16T19:25:00.824993+0000","last_up_change":"2024-02-16T19:24:59.819598+0000","last_in_change":"2024-02-16T19:24:40.575373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-16T19:23:47.101426+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d16e47f2-7209-45f4-ad50-1e0f6789ffad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2126256017},{"type":"v1","addr":"172.21.15.22:6803","nonce":2126256017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2126256017},{"type":"v1","addr":"172.21.15.22:6805","nonce":2126256017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2126256017},{"type":"v1","addr":"172.21.15.22:6809","nonce":2126256017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2126256017},{"type":"v1","addr":"172.21.15.22:6807","nonce":2126256017}]},"public_addr":"172.21.15.22:6803/2126256017","cluster_addr":"172.21.15.22:6805/2126256017","heartbeat_back_addr":"172.21.15.22:6809/2126256017","heartbeat_front_addr":"172.21.15.22:6807/2126256017","state":["exists","up"]},{"osd":1,"uuid":"59c3d085-3422-4e92-9048-46d7a39e31a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":3355370212},{"type":"v1","addr":"172.21.15.22:6811","nonce":3355370212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":3355370212},{"type":"v1","addr":"172.21.15.22:6813","nonce":3355370212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":3355370212},{"type":"v1","addr":"172.21.15.22:6817","nonce":3355370212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":3355370212},{"type":"v1","addr":"172.21.15.22:6815","nonce":3355370212}]},"public_addr":"172.21.15.22:6811/3355370212","cluster_addr":"172.21.15.22:6813/3355370212","heartbeat_back_addr":"172.21.15.22:6817/3355370212","heartbeat_front_addr":"172.21.15.22:6815/3355370212","state":["exists","up"]},{"osd":2,"uuid":"5952e710-f0b6-406c-855a-00f1cb8059c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":3627861698},{"type":"v1","addr":"172.21.15.22:6819","nonce":3627861698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":3627861698},{"type":"v1","addr":"172.21.15.22:6821","nonce":3627861698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":3627861698},{"type":"v1","addr":"172.21.15.22:6825","nonce":3627861698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":3627861698},{"type":"v1","addr":"172.21.15.22:6823","nonce":3627861698}]},"public_addr":"172.21.15.22:6819/3627861698","cluster_addr":"172.21.15.22:6821/3627861698","heartbeat_back_addr":"172.21.15.22:6825/3627861698","heartbeat_front_addr":"172.21.15.22:6823/3627861698","state":["exists","up"]},{"osd":3,"uuid":"5dc87494-e400-4111-91a7-306d9905febb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":2012503812},{"type":"v1","addr":"172.21.15.117:6801","nonce":2012503812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2012503812},{"type":"v1","addr":"172.21.15.117:6803","nonce":2012503812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2012503812},{"type":"v1","addr":"172.21.15.117:6807","nonce":2012503812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":2012503812},{"type":"v1","addr":"172.21.15.117:6805","nonce":2012503812}]},"public_addr":"172.21.15.117:6801/2012503812","cluster_addr":"172.21.15.117:6803/2012503812","heartbeat_back_addr":"172.21.15.117:6807/2012503812","heartbeat_front_addr":"172.21.15.117:6805/2012503812","state":["exists","up"]},{"osd":4,"uuid":"d27bde8d-1a77-4f95-acb6-7d05becd9480","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1856566932},{"type":"v1","addr":"172.21.15.117:6809","nonce":1856566932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":1856566932},{"type":"v1","addr":"172.21.15.117:6811","nonce":1856566932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":1856566932},{"type":"v1","addr":"172.21.15.117:6815","nonce":1856566932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1856566932},{"type":"v1","addr":"172.21.15.117:6813","nonce":1856566932}]},"public_addr":"172.21.15.117:6809/1856566932","cluster_addr":"172.21.15.117:6811/1856566932","heartbeat_back_addr":"172.21.15.117:6815/1856566932","heartbeat_front_addr":"172.21.15.117:6813/1856566932","state":["exists","up"]},{"osd":5,"uuid":"5873798f-3a2c-4a29-becd-6280b8866b6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":38288182},{"type":"v1","addr":"172.21.15.117:6817","nonce":38288182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":38288182},{"type":"v1","addr":"172.21.15.117:6819","nonce":38288182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":38288182},{"type":"v1","addr":"172.21.15.117:6823","nonce":38288182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":38288182},{"type":"v1","addr":"172.21.15.117:6821","nonce":38288182}]},"public_addr":"172.21.15.117:6817/38288182","cluster_addr":"172.21.15.117:6819/38288182","heartbeat_back_addr":"172.21.15.117:6823/38288182","heartbeat_front_addr":"172.21.15.117:6821/38288182","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:22:44.777427+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:23:14.935167+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:23:44.496184+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:24:07.447481+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:24:32.205064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:24:57.982560+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:6800/2945983569":"2024-02-17T19:21:26.760165+0000","172.21.15.22:0/668279955":"2024-02-17T19:21:26.760165+0000","172.21.15.22:0/2017591724":"2024-02-17T19:20:11.619146+0000","172.21.15.22:0/2384083375":"2024-02-17T19:21:26.760165+0000","172.21.15.22:6801/2755289428":"2024-02-17T19:20:11.619146+0000","172.21.15.22:6800/2755289428":"2024-02-17T19:20:11.619146+0000","172.21.15.22:6801/2945983569":"2024-02-17T19:21:26.760165+0000","172.21.15.22:6801/3109015836":"2024-02-17T19:19:38.544291+0000","172.21.15.22:6800/3109015836":"2024-02-17T19:19:38.544291+0000","172.21.15.22:0/1603216752":"2024-02-17T19:19:38.544291+0000","172.21.15.22:0/2099601555":"2024-02-17T19:20:11.619146+0000","172.21.15.22:0/1229330195":"2024-02-17T19:20:11.619146+0000","172.21.15.22:0/2371245756":"2024-02-17T19:19:38.544291+0000","172.21.15.22:0/3200629226":"2024-02-17T19:21:26.760165+0000","172.21.15.22:0/8466130":"2024-02-17T19:19:38.544291+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-16T19:25:22.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:22.702+0000 7f921705d700 1 -- 172.21.15.22:0/4088921635 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f91f806c680 msgr2=0x7f91f806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:22.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:22.702+0000 7f921705d700 1 --2- 172.21.15.22:0/4088921635 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f91f806c680 0x7f91f806eb30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f920c006010 tx=0x7f920c01a040 comp rx=0 tx=0).stop 2024-02-16T19:25:22.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:22.702+0000 7f921705d700 1 -- 172.21.15.22:0/4088921635 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9210100e80 msgr2=0x7f9210196430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:22.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:22.702+0000 7f921705d700 1 --2- 172.21.15.22:0/4088921635 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9210100e80 0x7f9210196430 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f920000d900 tx=0x7f920000dc10 comp rx=0 tx=0).stop 2024-02-16T19:25:22.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:22.702+0000 7f921705d700 1 -- 172.21.15.22:0/4088921635 shutdown_connections 2024-02-16T19:25:22.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:22.702+0000 7f921705d700 1 --2- 172.21.15.22:0/4088921635 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f91f806c680 0x7f91f806eb30 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:22.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:22.702+0000 7f921705d700 1 --2- 172.21.15.22:0/4088921635 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92101004e0 0x7f9210195ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:22.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:22.702+0000 7f921705d700 1 --2- 172.21.15.22:0/4088921635 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9210100e80 0x7f9210196430 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:22.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:22.702+0000 7f921705d700 1 -- 172.21.15.22:0/4088921635 >> 172.21.15.22:0/4088921635 conn(0x7f92100fbda0 msgr2=0x7f92100fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:25:22.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:22.702+0000 7f921705d700 1 -- 172.21.15.22:0/4088921635 shutdown_connections 2024-02-16T19:25:22.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:22.702+0000 7f921705d700 1 -- 172.21.15.22:0/4088921635 wait complete. 2024-02-16T19:25:23.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:25:22 smithi022 ceph-mon[29221]: from='client.? 172.21.15.22:0/4088921635' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-16T19:25:23.370 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-02-16T19:25:23.370 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-02-16T19:25:23.370 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-02-16T19:25:23.371 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-02-16T19:25:23.371 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-02-16T19:25:23.372 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-02-16T19:25:23.382 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:25:22 smithi117 ceph-mon[35309]: from='client.? 172.21.15.22:0/4088921635' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-16T19:25:23.687 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi022/config 2024-02-16T19:25:23.687 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi022/config 2024-02-16T19:25:23.749 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi022/config 2024-02-16T19:25:23.752 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi022/config 2024-02-16T19:25:23.752 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi022/config 2024-02-16T19:25:23.756 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi022/config 2024-02-16T19:25:24.083 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:25:23 smithi022 ceph-mon[29221]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:25:24.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:25:23 smithi117 ceph-mon[35309]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:25:25.964 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:25:25 smithi022 ceph-mon[29221]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:25:26.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:25:25 smithi117 ceph-mon[35309]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:25:27.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:25:26 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:25:27.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:25:26 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:25:28.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:25:27 smithi022 ceph-mon[29221]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:25:28.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:25:27 smithi117 ceph-mon[35309]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:25:29.964 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:25:29 smithi022 ceph-mon[29221]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:25:30.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:25:29 smithi117 ceph-mon[35309]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:25:30.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:30.838+0000 7ffaf9c16700 1 -- 172.21.15.22:0/1339715385 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffaf4068540 msgr2=0x7ffaf40689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:30.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:30.838+0000 7ffaf9c16700 1 --2- 172.21.15.22:0/1339715385 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffaf4068540 0x7ffaf40689b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ffae8009910 tx=0x7ffae8009c20 comp rx=0 tx=0).stop 2024-02-16T19:25:30.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:30.838+0000 7ffaf9c16700 1 -- 172.21.15.22:0/1339715385 shutdown_connections 2024-02-16T19:25:30.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:30.838+0000 7ffaf9c16700 1 --2- 172.21.15.22:0/1339715385 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffaf4068540 0x7ffaf40689b0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:30.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:30.838+0000 7ffaf9c16700 1 --2- 172.21.15.22:0/1339715385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffaf4106320 0x7ffaf41066f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:30.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:30.838+0000 7ffaf9c16700 1 -- 172.21.15.22:0/1339715385 >> 172.21.15.22:0/1339715385 conn(0x7ffaf4075130 msgr2=0x7ffaf4075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:25:30.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:30.838+0000 7ffaf9c16700 1 -- 172.21.15.22:0/1339715385 shutdown_connections 2024-02-16T19:25:30.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:30.838+0000 7ffaf9c16700 1 -- 172.21.15.22:0/1339715385 wait complete. 2024-02-16T19:25:30.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:30.839+0000 7ffaf9c16700 1 Processor -- start 2024-02-16T19:25:30.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:30.839+0000 7ffaf9c16700 1 -- start start 2024-02-16T19:25:30.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:30.839+0000 7ffaf9c16700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffaf4068540 0x7ffaf4195ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:30.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:30.839+0000 7ffaf9c16700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffaf4106320 0x7ffaf4196400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:30.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:30.839+0000 7ffaf9c16700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf4196ae0 con 0x7ffaf4106320 2024-02-16T19:25:30.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:30.839+0000 7ffaf9c16700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf419a870 con 0x7ffaf4068540 2024-02-16T19:25:30.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:30.839+0000 7ffaf2ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffaf4106320 0x7ffaf4196400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:30.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:30.839+0000 7ffaf2ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffaf4106320 0x7ffaf4196400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33724/0 (socket says 172.21.15.22:33724) 2024-02-16T19:25:30.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:30.839+0000 7ffaf2ffd700 1 -- 172.21.15.22:0/1811038954 learned_addr learned my addr 172.21.15.22:0/1811038954 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:25:30.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:30.839+0000 7ffaf37fe700 1 --2- 172.21.15.22:0/1811038954 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffaf4068540 0x7ffaf4195ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:30.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:30.839+0000 7ffaf2ffd700 1 -- 172.21.15.22:0/1811038954 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffaf4068540 msgr2=0x7ffaf4195ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:30.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:30.839+0000 7ffaf2ffd700 1 --2- 172.21.15.22:0/1811038954 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffaf4068540 0x7ffaf4195ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:30.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:30.839+0000 7ffaf2ffd700 1 -- 172.21.15.22:0/1811038954 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffae80095f0 con 0x7ffaf4106320 2024-02-16T19:25:30.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:30.840+0000 7ffaf2ffd700 1 --2- 172.21.15.22:0/1811038954 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffaf4106320 0x7ffaf4196400 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ffae8006010 tx=0x7ffae800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:30.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:30.840+0000 7ffaf0ff9700 1 -- 172.21.15.22:0/1811038954 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffae801d070 con 0x7ffaf4106320 2024-02-16T19:25:30.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:30.840+0000 7ffaf0ff9700 1 -- 172.21.15.22:0/1811038954 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ffae800f460 con 0x7ffaf4106320 2024-02-16T19:25:30.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:30.840+0000 7ffaf9c16700 1 -- 172.21.15.22:0/1811038954 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaf419aaf0 con 0x7ffaf4106320 2024-02-16T19:25:30.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:30.840+0000 7ffaf0ff9700 1 -- 172.21.15.22:0/1811038954 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffae8021640 con 0x7ffaf4106320 2024-02-16T19:25:30.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:30.840+0000 7ffaf9c16700 1 -- 172.21.15.22:0/1811038954 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaf419afe0 con 0x7ffaf4106320 2024-02-16T19:25:30.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:30.841+0000 7ffaf9c16700 1 -- 172.21.15.22:0/1811038954 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ffaf410d690 con 0x7ffaf4106320 2024-02-16T19:25:30.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:30.841+0000 7ffaf0ff9700 1 -- 172.21.15.22:0/1811038954 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7ffae800faf0 con 0x7ffaf4106320 2024-02-16T19:25:30.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:30.842+0000 7ffaf0ff9700 1 --2- 172.21.15.22:0/1811038954 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7ffadc06c680 0x7ffadc06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:30.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:30.842+0000 7ffaf0ff9700 1 -- 172.21.15.22:0/1811038954 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7ffae808cf80 con 0x7ffaf4106320 2024-02-16T19:25:30.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:30.842+0000 7ffaf37fe700 1 --2- 172.21.15.22:0/1811038954 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7ffadc06c680 0x7ffadc06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:30.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:30.842+0000 7ffaf0ff9700 1 --2- 172.21.15.22:0/1811038954 >> [v2:172.21.15.117:6800/2012503812,v1:172.21.15.117:6801/2012503812] conn(0x7ffadc072350 0x7ffadc074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:30.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:30.842+0000 7ffaf0ff9700 1 -- 172.21.15.22:0/1811038954 --> [v2:172.21.15.117:6800/2012503812,v1:172.21.15.117:6801/2012503812] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ffadc074e10 con 0x7ffadc072350 2024-02-16T19:25:30.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:30.842+0000 7ffaf0ff9700 1 -- 172.21.15.22:0/1811038954 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffae808d3e0 con 0x7ffaf4106320 2024-02-16T19:25:30.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:30.842+0000 7ffaf3fff700 1 --2- 172.21.15.22:0/1811038954 >> [v2:172.21.15.117:6800/2012503812,v1:172.21.15.117:6801/2012503812] conn(0x7ffadc072350 0x7ffadc074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:30.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:30.842+0000 7ffaf37fe700 1 --2- 172.21.15.22:0/1811038954 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7ffadc06c680 0x7ffadc06eb30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ffae4006fd0 tx=0x7ffae4008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:30.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:30.843+0000 7ffaf3fff700 1 --2- 172.21.15.22:0/1811038954 >> [v2:172.21.15.117:6800/2012503812,v1:172.21.15.117:6801/2012503812] conn(0x7ffadc072350 0x7ffadc074760 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:30.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:30.843+0000 7ffaf0ff9700 1 -- 172.21.15.22:0/1811038954 <== osd.3 v2:172.21.15.117:6800/2012503812 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7ffadc074e10 con 0x7ffadc072350 2024-02-16T19:25:30.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:30.860+0000 7ffaf9c16700 1 -- 172.21.15.22:0/1811038954 --> [v2:172.21.15.117:6800/2012503812,v1:172.21.15.117:6801/2012503812] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ffaf404f2a0 con 0x7ffadc072350 2024-02-16T19:25:30.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:30.860+0000 7ffaf0ff9700 1 -- 172.21.15.22:0/1811038954 <== osd.3 v2:172.21.15.117:6800/2012503812 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7ffaf404f2a0 con 0x7ffadc072350 2024-02-16T19:25:30.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:30.860+0000 7ffaf9c16700 1 -- 172.21.15.22:0/1811038954 >> [v2:172.21.15.117:6800/2012503812,v1:172.21.15.117:6801/2012503812] conn(0x7ffadc072350 msgr2=0x7ffadc074760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:30.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:30.860+0000 7ffaf9c16700 1 --2- 172.21.15.22:0/1811038954 >> [v2:172.21.15.117:6800/2012503812,v1:172.21.15.117:6801/2012503812] conn(0x7ffadc072350 0x7ffadc074760 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:30.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:30.860+0000 7ffaf9c16700 1 -- 172.21.15.22:0/1811038954 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7ffadc06c680 msgr2=0x7ffadc06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:30.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:30.860+0000 7ffaf9c16700 1 --2- 172.21.15.22:0/1811038954 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7ffadc06c680 0x7ffadc06eb30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ffae4006fd0 tx=0x7ffae4008040 comp rx=0 tx=0).stop 2024-02-16T19:25:30.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:30.860+0000 7ffaf9c16700 1 -- 172.21.15.22:0/1811038954 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffaf4106320 msgr2=0x7ffaf4196400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:30.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:30.860+0000 7ffaf9c16700 1 --2- 172.21.15.22:0/1811038954 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffaf4106320 0x7ffaf4196400 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ffae8006010 tx=0x7ffae800be20 comp rx=0 tx=0).stop 2024-02-16T19:25:30.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:30.860+0000 7ffaf9c16700 1 -- 172.21.15.22:0/1811038954 shutdown_connections 2024-02-16T19:25:30.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:30.860+0000 7ffaf9c16700 1 --2- 172.21.15.22:0/1811038954 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7ffadc06c680 0x7ffadc06eb30 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:30.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:30.860+0000 7ffaf9c16700 1 --2- 172.21.15.22:0/1811038954 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffaf4068540 0x7ffaf4195ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:30.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:30.860+0000 7ffaf9c16700 1 --2- 172.21.15.22:0/1811038954 >> [v2:172.21.15.117:6800/2012503812,v1:172.21.15.117:6801/2012503812] conn(0x7ffadc072350 0x7ffadc074760 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:30.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:30.860+0000 7ffaf9c16700 1 --2- 172.21.15.22:0/1811038954 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffaf4106320 0x7ffaf4196400 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:30.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:30.860+0000 7ffaf9c16700 1 -- 172.21.15.22:0/1811038954 >> 172.21.15.22:0/1811038954 conn(0x7ffaf4075130 msgr2=0x7ffaf40feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:25:30.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:30.861+0000 7ffaf9c16700 1 -- 172.21.15.22:0/1811038954 shutdown_connections 2024-02-16T19:25:30.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:30.861+0000 7ffaf9c16700 1 -- 172.21.15.22:0/1811038954 wait complete. 2024-02-16T19:25:31.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.022+0000 7f031bddf700 1 -- 172.21.15.22:0/2090578986 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03140ffbd0 msgr2=0x7f0314100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:31.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.022+0000 7f031bddf700 1 --2- 172.21.15.22:0/2090578986 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03140ffbd0 0x7f0314100040 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0304009910 tx=0x7f0304009c20 comp rx=0 tx=0).stop 2024-02-16T19:25:31.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.022+0000 7f031bddf700 1 -- 172.21.15.22:0/2090578986 shutdown_connections 2024-02-16T19:25:31.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.022+0000 7f031bddf700 1 --2- 172.21.15.22:0/2090578986 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03140ffbd0 0x7f0314100040 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:31.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.022+0000 7f031bddf700 1 --2- 172.21.15.22:0/2090578986 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0314104130 0x7f0314104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:31.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.022+0000 7f031bddf700 1 -- 172.21.15.22:0/2090578986 >> 172.21.15.22:0/2090578986 conn(0x7f0314075130 msgr2=0x7f0314075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:25:31.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.022+0000 7f031bddf700 1 -- 172.21.15.22:0/2090578986 shutdown_connections 2024-02-16T19:25:31.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.022+0000 7f031bddf700 1 -- 172.21.15.22:0/2090578986 wait complete. 2024-02-16T19:25:31.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.023+0000 7f031bddf700 1 Processor -- start 2024-02-16T19:25:31.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.023+0000 7f031bddf700 1 -- start start 2024-02-16T19:25:31.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.023+0000 7f031bddf700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03140ffbd0 0x7f0314196d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:31.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.023+0000 7f031bddf700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0314104130 0x7f0314197260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:31.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.023+0000 7f031bddf700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0314197880 con 0x7f03140ffbd0 2024-02-16T19:25:31.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.023+0000 7f031bddf700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03141979c0 con 0x7f0314104130 2024-02-16T19:25:31.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.023+0000 7f0319b7b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03140ffbd0 0x7f0314196d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:31.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.023+0000 7f0319b7b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03140ffbd0 0x7f0314196d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33750/0 (socket says 172.21.15.22:33750) 2024-02-16T19:25:31.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.023+0000 7f0319b7b700 1 -- 172.21.15.22:0/1625782406 learned_addr learned my addr 172.21.15.22:0/1625782406 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:25:31.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.023+0000 7f031937a700 1 --2- 172.21.15.22:0/1625782406 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0314104130 0x7f0314197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:31.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.024+0000 7f0319b7b700 1 -- 172.21.15.22:0/1625782406 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0314104130 msgr2=0x7f0314197260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:31.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.024+0000 7f0319b7b700 1 --2- 172.21.15.22:0/1625782406 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0314104130 0x7f0314197260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:31.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.024+0000 7f0319b7b700 1 -- 172.21.15.22:0/1625782406 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03040095f0 con 0x7f03140ffbd0 2024-02-16T19:25:31.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.024+0000 7f0319b7b700 1 --2- 172.21.15.22:0/1625782406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03140ffbd0 0x7f0314196d20 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f031000b4b0 tx=0x7f031000b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:31.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.024+0000 7f030affd700 1 -- 172.21.15.22:0/1625782406 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0310011800 con 0x7f03140ffbd0 2024-02-16T19:25:31.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.024+0000 7f031bddf700 1 -- 172.21.15.22:0/1625782406 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f031419fd90 con 0x7f03140ffbd0 2024-02-16T19:25:31.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.024+0000 7f030affd700 1 -- 172.21.15.22:0/1625782406 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0310011e10 con 0x7f03140ffbd0 2024-02-16T19:25:31.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.024+0000 7f031bddf700 1 -- 172.21.15.22:0/1625782406 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03141a0250 con 0x7f03140ffbd0 2024-02-16T19:25:31.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.024+0000 7f030affd700 1 -- 172.21.15.22:0/1625782406 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f031000f2e0 con 0x7f03140ffbd0 2024-02-16T19:25:31.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.025+0000 7f031bddf700 1 -- 172.21.15.22:0/1625782406 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f031410c600 con 0x7f03140ffbd0 2024-02-16T19:25:31.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.026+0000 7f030affd700 1 -- 172.21.15.22:0/1625782406 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f0310010380 con 0x7f03140ffbd0 2024-02-16T19:25:31.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.027+0000 7f030affd700 1 --2- 172.21.15.22:0/1625782406 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f030006c680 0x7f030006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:31.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.027+0000 7f030affd700 1 -- 172.21.15.22:0/1625782406 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f031008b670 con 0x7f03140ffbd0 2024-02-16T19:25:31.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.027+0000 7f031937a700 1 --2- 172.21.15.22:0/1625782406 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f030006c680 0x7f030006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:31.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.027+0000 7f030affd700 1 --2- 172.21.15.22:0/1625782406 >> [v2:172.21.15.22:6802/2126256017,v1:172.21.15.22:6803/2126256017] conn(0x7f0300072350 0x7f0300074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:31.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.027+0000 7f030affd700 1 -- 172.21.15.22:0/1625782406 --> [v2:172.21.15.22:6802/2126256017,v1:172.21.15.22:6803/2126256017] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0300074e10 con 0x7f0300072350 2024-02-16T19:25:31.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.027+0000 7f030affd700 1 -- 172.21.15.22:0/1625782406 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f031008bad0 con 0x7f03140ffbd0 2024-02-16T19:25:31.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.027+0000 7f031a37c700 1 --2- 172.21.15.22:0/1625782406 >> [v2:172.21.15.22:6802/2126256017,v1:172.21.15.22:6803/2126256017] conn(0x7f0300072350 0x7f0300074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:31.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.027+0000 7f031937a700 1 --2- 172.21.15.22:0/1625782406 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f030006c680 0x7f030006eb30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0314068df0 tx=0x7f030401a040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:31.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.027+0000 7f031a37c700 1 --2- 172.21.15.22:0/1625782406 >> [v2:172.21.15.22:6802/2126256017,v1:172.21.15.22:6803/2126256017] conn(0x7f0300072350 0x7f0300074760 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:31.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.028+0000 7f030affd700 1 -- 172.21.15.22:0/1625782406 <== osd.0 v2:172.21.15.22:6802/2126256017 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f0300074e10 con 0x7f0300072350 2024-02-16T19:25:31.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.045+0000 7f031bddf700 1 -- 172.21.15.22:0/1625782406 --> [v2:172.21.15.22:6802/2126256017,v1:172.21.15.22:6803/2126256017] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f031404f2a0 con 0x7f0300072350 2024-02-16T19:25:31.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.045+0000 7f030affd700 1 -- 172.21.15.22:0/1625782406 <== osd.0 v2:172.21.15.22:6802/2126256017 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f031404f2a0 con 0x7f0300072350 2024-02-16T19:25:31.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.046+0000 7f031bddf700 1 -- 172.21.15.22:0/1625782406 >> [v2:172.21.15.22:6802/2126256017,v1:172.21.15.22:6803/2126256017] conn(0x7f0300072350 msgr2=0x7f0300074760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:31.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.046+0000 7f031bddf700 1 --2- 172.21.15.22:0/1625782406 >> [v2:172.21.15.22:6802/2126256017,v1:172.21.15.22:6803/2126256017] conn(0x7f0300072350 0x7f0300074760 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:31.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.046+0000 7f031bddf700 1 -- 172.21.15.22:0/1625782406 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f030006c680 msgr2=0x7f030006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:31.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.046+0000 7f031bddf700 1 --2- 172.21.15.22:0/1625782406 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f030006c680 0x7f030006eb30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0314068df0 tx=0x7f030401a040 comp rx=0 tx=0).stop 2024-02-16T19:25:31.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.046+0000 7f031bddf700 1 -- 172.21.15.22:0/1625782406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03140ffbd0 msgr2=0x7f0314196d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:31.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.046+0000 7f031bddf700 1 --2- 172.21.15.22:0/1625782406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03140ffbd0 0x7f0314196d20 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f031000b4b0 tx=0x7f031000b7c0 comp rx=0 tx=0).stop 2024-02-16T19:25:31.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.046+0000 7f031bddf700 1 -- 172.21.15.22:0/1625782406 shutdown_connections 2024-02-16T19:25:31.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.046+0000 7f031bddf700 1 --2- 172.21.15.22:0/1625782406 >> [v2:172.21.15.22:6802/2126256017,v1:172.21.15.22:6803/2126256017] conn(0x7f0300072350 0x7f0300074760 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:31.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.046+0000 7f031bddf700 1 --2- 172.21.15.22:0/1625782406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03140ffbd0 0x7f0314196d20 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:31.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.046+0000 7f031bddf700 1 --2- 172.21.15.22:0/1625782406 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f030006c680 0x7f030006eb30 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:31.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.046+0000 7f031bddf700 1 --2- 172.21.15.22:0/1625782406 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0314104130 0x7f0314197260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:31.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.046+0000 7f031bddf700 1 -- 172.21.15.22:0/1625782406 >> 172.21.15.22:0/1625782406 conn(0x7f0314075130 msgr2=0x7f03140fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:25:31.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.046+0000 7f031bddf700 1 -- 172.21.15.22:0/1625782406 shutdown_connections 2024-02-16T19:25:31.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.046+0000 7f031bddf700 1 -- 172.21.15.22:0/1625782406 wait complete. 2024-02-16T19:25:31.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.240+0000 7f995721e700 1 -- 172.21.15.22:0/3152354289 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99500ff3c0 msgr2=0x7f99500ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:31.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.240+0000 7f995721e700 1 --2- 172.21.15.22:0/3152354289 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99500ff3c0 0x7f99500ff790 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f994800b0a0 tx=0x7f994800b3b0 comp rx=0 tx=0).stop 2024-02-16T19:25:31.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.240+0000 7f995721e700 1 -- 172.21.15.22:0/3152354289 shutdown_connections 2024-02-16T19:25:31.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.240+0000 7f995721e700 1 --2- 172.21.15.22:0/3152354289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99500ffd60 0x7f9950103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:31.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.240+0000 7f995721e700 1 --2- 172.21.15.22:0/3152354289 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99500ff3c0 0x7f99500ff790 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:31.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.240+0000 7f995721e700 1 -- 172.21.15.22:0/3152354289 >> 172.21.15.22:0/3152354289 conn(0x7f99500fac80 msgr2=0x7f99500fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:25:31.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.240+0000 7f995721e700 1 -- 172.21.15.22:0/3152354289 shutdown_connections 2024-02-16T19:25:31.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.240+0000 7f995721e700 1 -- 172.21.15.22:0/3152354289 wait complete. 2024-02-16T19:25:31.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.240+0000 7f995721e700 1 Processor -- start 2024-02-16T19:25:31.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.240+0000 7f995721e700 1 -- start start 2024-02-16T19:25:31.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.241+0000 7f995721e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99500ff3c0 0x7f9950195f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:31.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.241+0000 7f995721e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99500ffd60 0x7f9950196460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:31.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.241+0000 7f995721e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9950196b40 con 0x7f99500ff3c0 2024-02-16T19:25:31.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.241+0000 7f995721e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f995019a8d0 con 0x7f99500ffd60 2024-02-16T19:25:31.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.241+0000 7f9954fba700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99500ff3c0 0x7f9950195f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:31.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.241+0000 7f9954fba700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99500ff3c0 0x7f9950195f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33768/0 (socket says 172.21.15.22:33768) 2024-02-16T19:25:31.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.241+0000 7f9954fba700 1 -- 172.21.15.22:0/3557707535 learned_addr learned my addr 172.21.15.22:0/3557707535 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:25:31.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.241+0000 7f9954fba700 1 -- 172.21.15.22:0/3557707535 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99500ffd60 msgr2=0x7f9950196460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:31.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.241+0000 7f9954fba700 1 --2- 172.21.15.22:0/3557707535 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99500ffd60 0x7f9950196460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:31.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.241+0000 7f9954fba700 1 -- 172.21.15.22:0/3557707535 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9948009d00 con 0x7f99500ff3c0 2024-02-16T19:25:31.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.241+0000 7f9954fba700 1 --2- 172.21.15.22:0/3557707535 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99500ff3c0 0x7f9950195f20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9948000f80 tx=0x7f994800cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:31.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.241+0000 7f994dffb700 1 -- 172.21.15.22:0/3557707535 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f994800e040 con 0x7f99500ff3c0 2024-02-16T19:25:31.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.241+0000 7f994dffb700 1 -- 172.21.15.22:0/3557707535 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9948003de0 con 0x7f99500ff3c0 2024-02-16T19:25:31.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.241+0000 7f995721e700 1 -- 172.21.15.22:0/3557707535 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f995019ab50 con 0x7f99500ff3c0 2024-02-16T19:25:31.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.241+0000 7f994dffb700 1 -- 172.21.15.22:0/3557707535 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f994801d9a0 con 0x7f99500ff3c0 2024-02-16T19:25:31.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.242+0000 7f995721e700 1 -- 172.21.15.22:0/3557707535 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f995019b040 con 0x7f99500ff3c0 2024-02-16T19:25:31.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.242+0000 7f995721e700 1 -- 172.21.15.22:0/3557707535 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f995010c520 con 0x7f99500ff3c0 2024-02-16T19:25:31.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.243+0000 7f994dffb700 1 -- 172.21.15.22:0/3557707535 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f994801db00 con 0x7f99500ff3c0 2024-02-16T19:25:31.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.243+0000 7f994dffb700 1 --2- 172.21.15.22:0/3557707535 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f993806c680 0x7f993806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:31.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.243+0000 7f994dffb700 1 -- 172.21.15.22:0/3557707535 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f994808d9b0 con 0x7f99500ff3c0 2024-02-16T19:25:31.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.243+0000 7f994ffff700 1 --2- 172.21.15.22:0/3557707535 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f993806c680 0x7f993806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:31.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.244+0000 7f994dffb700 1 --2- 172.21.15.22:0/3557707535 >> [v2:172.21.15.117:6808/1856566932,v1:172.21.15.117:6809/1856566932] conn(0x7f9938072350 0x7f9938074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:31.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.244+0000 7f994dffb700 1 -- 172.21.15.22:0/3557707535 --> [v2:172.21.15.117:6808/1856566932,v1:172.21.15.117:6809/1856566932] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9938074e10 con 0x7f9938072350 2024-02-16T19:25:31.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.244+0000 7f994dffb700 1 -- 172.21.15.22:0/3557707535 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f994808dd70 con 0x7f99500ff3c0 2024-02-16T19:25:31.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.244+0000 7f994ffff700 1 --2- 172.21.15.22:0/3557707535 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f993806c680 0x7f993806eb30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9944005950 tx=0x7f99440058e0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:31.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.244+0000 7f99557bb700 1 --2- 172.21.15.22:0/3557707535 >> [v2:172.21.15.117:6808/1856566932,v1:172.21.15.117:6809/1856566932] conn(0x7f9938072350 0x7f9938074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:31.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.244+0000 7f99557bb700 1 --2- 172.21.15.22:0/3557707535 >> [v2:172.21.15.117:6808/1856566932,v1:172.21.15.117:6809/1856566932] conn(0x7f9938072350 0x7f9938074760 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:31.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.245+0000 7f994dffb700 1 -- 172.21.15.22:0/3557707535 <== osd.4 v2:172.21.15.117:6808/1856566932 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f9938074e10 con 0x7f9938072350 2024-02-16T19:25:31.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.261+0000 7f995721e700 1 -- 172.21.15.22:0/3557707535 --> [v2:172.21.15.117:6808/1856566932,v1:172.21.15.117:6809/1856566932] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f995004f2a0 con 0x7f9938072350 2024-02-16T19:25:31.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.262+0000 7f994dffb700 1 -- 172.21.15.22:0/3557707535 <== osd.4 v2:172.21.15.117:6808/1856566932 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f995004f2a0 con 0x7f9938072350 2024-02-16T19:25:31.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.262+0000 7f995721e700 1 -- 172.21.15.22:0/3557707535 >> [v2:172.21.15.117:6808/1856566932,v1:172.21.15.117:6809/1856566932] conn(0x7f9938072350 msgr2=0x7f9938074760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:31.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.262+0000 7f995721e700 1 --2- 172.21.15.22:0/3557707535 >> [v2:172.21.15.117:6808/1856566932,v1:172.21.15.117:6809/1856566932] conn(0x7f9938072350 0x7f9938074760 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:31.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.262+0000 7f995721e700 1 -- 172.21.15.22:0/3557707535 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f993806c680 msgr2=0x7f993806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:31.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.262+0000 7f995721e700 1 --2- 172.21.15.22:0/3557707535 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f993806c680 0x7f993806eb30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9944005950 tx=0x7f99440058e0 comp rx=0 tx=0).stop 2024-02-16T19:25:31.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.262+0000 7f995721e700 1 -- 172.21.15.22:0/3557707535 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99500ff3c0 msgr2=0x7f9950195f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:31.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.262+0000 7f995721e700 1 --2- 172.21.15.22:0/3557707535 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99500ff3c0 0x7f9950195f20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9948000f80 tx=0x7f994800cdd0 comp rx=0 tx=0).stop 2024-02-16T19:25:31.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.262+0000 7f995721e700 1 -- 172.21.15.22:0/3557707535 shutdown_connections 2024-02-16T19:25:31.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.262+0000 7f995721e700 1 --2- 172.21.15.22:0/3557707535 >> [v2:172.21.15.117:6808/1856566932,v1:172.21.15.117:6809/1856566932] conn(0x7f9938072350 0x7f9938074760 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:31.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.262+0000 7f995721e700 1 --2- 172.21.15.22:0/3557707535 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99500ff3c0 0x7f9950195f20 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:31.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.262+0000 7f995721e700 1 --2- 172.21.15.22:0/3557707535 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f993806c680 0x7f993806eb30 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:31.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.262+0000 7f995721e700 1 --2- 172.21.15.22:0/3557707535 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99500ffd60 0x7f9950196460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:31.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.262+0000 7f995721e700 1 -- 172.21.15.22:0/3557707535 >> 172.21.15.22:0/3557707535 conn(0x7f99500fac80 msgr2=0x7f9950069050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:25:31.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.262+0000 7f995721e700 1 -- 172.21.15.22:0/3557707535 shutdown_connections 2024-02-16T19:25:31.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:31.262+0000 7f995721e700 1 -- 172.21.15.22:0/3557707535 wait complete. 2024-02-16T19:25:32.043 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:25:31 smithi022 ceph-mon[29221]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:25:32.386 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:25:31 smithi117 ceph-mon[35309]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:25:32.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.441+0000 7f842955b700 1 -- 172.21.15.22:0/1340720435 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84241004e0 msgr2=0x7f84241008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:32.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.441+0000 7f842955b700 1 --2- 172.21.15.22:0/1340720435 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84241004e0 0x7f84241008b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8414009910 tx=0x7f8414009c20 comp rx=0 tx=0).stop 2024-02-16T19:25:32.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.441+0000 7f842955b700 1 -- 172.21.15.22:0/1340720435 shutdown_connections 2024-02-16T19:25:32.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.441+0000 7f842955b700 1 --2- 172.21.15.22:0/1340720435 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8424100e80 0x7f8424104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:32.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.441+0000 7f842955b700 1 --2- 172.21.15.22:0/1340720435 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84241004e0 0x7f84241008b0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:32.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.441+0000 7f842955b700 1 -- 172.21.15.22:0/1340720435 >> 172.21.15.22:0/1340720435 conn(0x7f84240fbda0 msgr2=0x7f84240fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:25:32.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.441+0000 7f842955b700 1 -- 172.21.15.22:0/1340720435 shutdown_connections 2024-02-16T19:25:32.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.441+0000 7f842955b700 1 -- 172.21.15.22:0/1340720435 wait complete. 2024-02-16T19:25:32.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.442+0000 7f842955b700 1 Processor -- start 2024-02-16T19:25:32.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.442+0000 7f842955b700 1 -- start start 2024-02-16T19:25:32.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.442+0000 7f842955b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84241004e0 0x7f8424073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:32.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.442+0000 7f842955b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8424100e80 0x7f8424073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:32.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.442+0000 7f842955b700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8424073ad0 con 0x7f84241004e0 2024-02-16T19:25:32.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.442+0000 7f842955b700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8424073c10 con 0x7f8424100e80 2024-02-16T19:25:32.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.442+0000 7f8422ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84241004e0 0x7f8424073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:32.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.442+0000 7f8422ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84241004e0 0x7f8424073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33784/0 (socket says 172.21.15.22:33784) 2024-02-16T19:25:32.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.442+0000 7f8422ffd700 1 -- 172.21.15.22:0/3947612827 learned_addr learned my addr 172.21.15.22:0/3947612827 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:25:32.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.442+0000 7f84227fc700 1 --2- 172.21.15.22:0/3947612827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8424100e80 0x7f8424073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:32.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.442+0000 7f8422ffd700 1 -- 172.21.15.22:0/3947612827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8424100e80 msgr2=0x7f8424073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:32.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.442+0000 7f8422ffd700 1 --2- 172.21.15.22:0/3947612827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8424100e80 0x7f8424073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:32.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.442+0000 7f8422ffd700 1 -- 172.21.15.22:0/3947612827 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84140095f0 con 0x7f84241004e0 2024-02-16T19:25:32.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.443+0000 7f84227fc700 1 --2- 172.21.15.22:0/3947612827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8424100e80 0x7f8424073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-16T19:25:32.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.443+0000 7f8422ffd700 1 --2- 172.21.15.22:0/3947612827 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84241004e0 0x7f8424073050 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8414006010 tx=0x7f841400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:32.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.443+0000 7f840bfff700 1 -- 172.21.15.22:0/3947612827 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f841401d070 con 0x7f84241004e0 2024-02-16T19:25:32.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.443+0000 7f842955b700 1 -- 172.21.15.22:0/3947612827 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8424073e90 con 0x7f84241004e0 2024-02-16T19:25:32.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.443+0000 7f840bfff700 1 -- 172.21.15.22:0/3947612827 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f841400f460 con 0x7f84241004e0 2024-02-16T19:25:32.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.443+0000 7f840bfff700 1 -- 172.21.15.22:0/3947612827 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8414021640 con 0x7f84241004e0 2024-02-16T19:25:32.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.443+0000 7f842955b700 1 -- 172.21.15.22:0/3947612827 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84241a2490 con 0x7f84241004e0 2024-02-16T19:25:32.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.444+0000 7f842955b700 1 -- 172.21.15.22:0/3947612827 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f842410d6f0 con 0x7f84241004e0 2024-02-16T19:25:32.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.444+0000 7f840bfff700 1 -- 172.21.15.22:0/3947612827 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f841400faf0 con 0x7f84241004e0 2024-02-16T19:25:32.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.445+0000 7f840bfff700 1 --2- 172.21.15.22:0/3947612827 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f840c06c680 0x7f840c06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:32.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.445+0000 7f840bfff700 1 -- 172.21.15.22:0/3947612827 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f841408c8f0 con 0x7f84241004e0 2024-02-16T19:25:32.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.445+0000 7f84227fc700 1 --2- 172.21.15.22:0/3947612827 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f840c06c680 0x7f840c06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:32.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.445+0000 7f840bfff700 1 --2- 172.21.15.22:0/3947612827 >> [v2:172.21.15.22:6818/3627861698,v1:172.21.15.22:6819/3627861698] conn(0x7f840c072350 0x7f840c074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:32.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.445+0000 7f840bfff700 1 -- 172.21.15.22:0/3947612827 --> [v2:172.21.15.22:6818/3627861698,v1:172.21.15.22:6819/3627861698] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f840c074e10 con 0x7f840c072350 2024-02-16T19:25:32.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.445+0000 7f840bfff700 1 -- 172.21.15.22:0/3947612827 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84140912a0 con 0x7f84241004e0 2024-02-16T19:25:32.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.445+0000 7f84237fe700 1 --2- 172.21.15.22:0/3947612827 >> [v2:172.21.15.22:6818/3627861698,v1:172.21.15.22:6819/3627861698] conn(0x7f840c072350 0x7f840c074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:32.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.445+0000 7f84227fc700 1 --2- 172.21.15.22:0/3947612827 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f840c06c680 0x7f840c06eb30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8424074840 tx=0x7f8418008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:32.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.445+0000 7f84237fe700 1 --2- 172.21.15.22:0/3947612827 >> [v2:172.21.15.22:6818/3627861698,v1:172.21.15.22:6819/3627861698] conn(0x7f840c072350 0x7f840c074760 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:32.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.446+0000 7f840bfff700 1 -- 172.21.15.22:0/3947612827 <== osd.2 v2:172.21.15.22:6818/3627861698 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f840c074e10 con 0x7f840c072350 2024-02-16T19:25:32.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.471+0000 7f842955b700 1 -- 172.21.15.22:0/3947612827 --> [v2:172.21.15.22:6818/3627861698,v1:172.21.15.22:6819/3627861698] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f842404f2a0 con 0x7f840c072350 2024-02-16T19:25:32.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.472+0000 7f840bfff700 1 -- 172.21.15.22:0/3947612827 <== osd.2 v2:172.21.15.22:6818/3627861698 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f842404f2a0 con 0x7f840c072350 2024-02-16T19:25:32.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.472+0000 7f842955b700 1 -- 172.21.15.22:0/3947612827 >> [v2:172.21.15.22:6818/3627861698,v1:172.21.15.22:6819/3627861698] conn(0x7f840c072350 msgr2=0x7f840c074760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:32.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.472+0000 7f842955b700 1 --2- 172.21.15.22:0/3947612827 >> [v2:172.21.15.22:6818/3627861698,v1:172.21.15.22:6819/3627861698] conn(0x7f840c072350 0x7f840c074760 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:32.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.472+0000 7f842955b700 1 -- 172.21.15.22:0/3947612827 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f840c06c680 msgr2=0x7f840c06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:32.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.472+0000 7f842955b700 1 --2- 172.21.15.22:0/3947612827 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f840c06c680 0x7f840c06eb30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8424074840 tx=0x7f8418008040 comp rx=0 tx=0).stop 2024-02-16T19:25:32.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.472+0000 7f842955b700 1 -- 172.21.15.22:0/3947612827 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84241004e0 msgr2=0x7f8424073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:32.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.472+0000 7f842955b700 1 --2- 172.21.15.22:0/3947612827 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84241004e0 0x7f8424073050 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8414006010 tx=0x7f841400be20 comp rx=0 tx=0).stop 2024-02-16T19:25:32.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.472+0000 7f842955b700 1 -- 172.21.15.22:0/3947612827 shutdown_connections 2024-02-16T19:25:32.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.472+0000 7f842955b700 1 --2- 172.21.15.22:0/3947612827 >> [v2:172.21.15.22:6818/3627861698,v1:172.21.15.22:6819/3627861698] conn(0x7f840c072350 0x7f840c074760 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:32.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.472+0000 7f842955b700 1 --2- 172.21.15.22:0/3947612827 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84241004e0 0x7f8424073050 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:32.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.472+0000 7f842955b700 1 --2- 172.21.15.22:0/3947612827 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f840c06c680 0x7f840c06eb30 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:32.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.472+0000 7f842955b700 1 --2- 172.21.15.22:0/3947612827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8424100e80 0x7f8424073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:32.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.472+0000 7f842955b700 1 -- 172.21.15.22:0/3947612827 >> 172.21.15.22:0/3947612827 conn(0x7f84240fbda0 msgr2=0x7f84240fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:25:32.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.472+0000 7f842955b700 1 -- 172.21.15.22:0/3947612827 shutdown_connections 2024-02-16T19:25:32.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.473+0000 7f842955b700 1 -- 172.21.15.22:0/3947612827 wait complete. 2024-02-16T19:25:32.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.684+0000 7f7be693c700 1 -- 172.21.15.22:0/2063559573 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7be0104130 msgr2=0x7f7be0104500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:32.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.684+0000 7f7be693c700 1 --2- 172.21.15.22:0/2063559573 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7be0104130 0x7f7be0104500 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7bc8009910 tx=0x7f7bc8009c20 comp rx=0 tx=0).stop 2024-02-16T19:25:32.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.684+0000 7f7be693c700 1 -- 172.21.15.22:0/2063559573 shutdown_connections 2024-02-16T19:25:32.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.684+0000 7f7be693c700 1 --2- 172.21.15.22:0/2063559573 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7be00ffbd0 0x7f7be0100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:32.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.684+0000 7f7be693c700 1 --2- 172.21.15.22:0/2063559573 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7be0104130 0x7f7be0104500 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:32.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.684+0000 7f7be693c700 1 -- 172.21.15.22:0/2063559573 >> 172.21.15.22:0/2063559573 conn(0x7f7be0075130 msgr2=0x7f7be0075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:25:32.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.684+0000 7f7be693c700 1 -- 172.21.15.22:0/2063559573 shutdown_connections 2024-02-16T19:25:32.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.684+0000 7f7be693c700 1 -- 172.21.15.22:0/2063559573 wait complete. 2024-02-16T19:25:32.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.685+0000 7f7be693c700 1 Processor -- start 2024-02-16T19:25:32.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.685+0000 7f7be693c700 1 -- start start 2024-02-16T19:25:32.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.685+0000 7f7be693c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7be00ffbd0 0x7f7be01980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:32.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.685+0000 7f7be693c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7be0104130 0x7f7be0198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:32.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.685+0000 7f7be693c700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7be0198ce0 con 0x7f7be00ffbd0 2024-02-16T19:25:32.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.685+0000 7f7be693c700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7be019ca70 con 0x7f7be0104130 2024-02-16T19:25:32.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.685+0000 7f7bdffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7be00ffbd0 0x7f7be01980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:32.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.685+0000 7f7bdffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7be00ffbd0 0x7f7be01980c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33800/0 (socket says 172.21.15.22:33800) 2024-02-16T19:25:32.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.685+0000 7f7bdffff700 1 -- 172.21.15.22:0/1757074465 learned_addr learned my addr 172.21.15.22:0/1757074465 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:25:32.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.685+0000 7f7bdffff700 1 -- 172.21.15.22:0/1757074465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7be0104130 msgr2=0x7f7be0198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:32.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.685+0000 7f7bdf7fe700 1 --2- 172.21.15.22:0/1757074465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7be0104130 0x7f7be0198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:32.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.685+0000 7f7bdffff700 1 --2- 172.21.15.22:0/1757074465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7be0104130 0x7f7be0198600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:32.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.685+0000 7f7bdffff700 1 -- 172.21.15.22:0/1757074465 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bc80095f0 con 0x7f7be00ffbd0 2024-02-16T19:25:32.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.685+0000 7f7bdffff700 1 --2- 172.21.15.22:0/1757074465 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7be00ffbd0 0x7f7be01980c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7bc8006010 tx=0x7f7bc800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:32.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.686+0000 7f7bdd7fa700 1 -- 172.21.15.22:0/1757074465 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7bc801d070 con 0x7f7be00ffbd0 2024-02-16T19:25:32.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.686+0000 7f7bdd7fa700 1 -- 172.21.15.22:0/1757074465 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7bc800f460 con 0x7f7be00ffbd0 2024-02-16T19:25:32.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.686+0000 7f7be693c700 1 -- 172.21.15.22:0/1757074465 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7be019ccf0 con 0x7f7be00ffbd0 2024-02-16T19:25:32.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.686+0000 7f7bdd7fa700 1 -- 172.21.15.22:0/1757074465 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7bc8021620 con 0x7f7be00ffbd0 2024-02-16T19:25:32.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.686+0000 7f7be693c700 1 -- 172.21.15.22:0/1757074465 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7be019d1e0 con 0x7f7be00ffbd0 2024-02-16T19:25:32.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.687+0000 7f7be693c700 1 -- 172.21.15.22:0/1757074465 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7be010c640 con 0x7f7be00ffbd0 2024-02-16T19:25:32.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.687+0000 7f7bdd7fa700 1 -- 172.21.15.22:0/1757074465 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f7bc800faf0 con 0x7f7be00ffbd0 2024-02-16T19:25:32.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.688+0000 7f7bdd7fa700 1 --2- 172.21.15.22:0/1757074465 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f7bcc06c680 0x7f7bcc06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:32.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.688+0000 7f7bdd7fa700 1 -- 172.21.15.22:0/1757074465 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f7bc808c7c0 con 0x7f7be00ffbd0 2024-02-16T19:25:32.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.688+0000 7f7bdf7fe700 1 --2- 172.21.15.22:0/1757074465 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f7bcc06c680 0x7f7bcc06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:32.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.688+0000 7f7bdd7fa700 1 --2- 172.21.15.22:0/1757074465 >> [v2:172.21.15.117:6816/38288182,v1:172.21.15.117:6817/38288182] conn(0x7f7bcc072350 0x7f7bcc074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:32.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.688+0000 7f7bdd7fa700 1 -- 172.21.15.22:0/1757074465 --> [v2:172.21.15.117:6816/38288182,v1:172.21.15.117:6817/38288182] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7bcc074e10 con 0x7f7bcc072350 2024-02-16T19:25:32.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.688+0000 7f7bdd7fa700 1 -- 172.21.15.22:0/1757074465 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7bc80912a0 con 0x7f7be00ffbd0 2024-02-16T19:25:32.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.688+0000 7f7be4ed9700 1 --2- 172.21.15.22:0/1757074465 >> [v2:172.21.15.117:6816/38288182,v1:172.21.15.117:6817/38288182] conn(0x7f7bcc072350 0x7f7bcc074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:32.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.688+0000 7f7bdf7fe700 1 --2- 172.21.15.22:0/1757074465 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f7bcc06c680 0x7f7bcc06eb30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7bd0006fd0 tx=0x7f7bd0008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:32.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.689+0000 7f7be4ed9700 1 --2- 172.21.15.22:0/1757074465 >> [v2:172.21.15.117:6816/38288182,v1:172.21.15.117:6817/38288182] conn(0x7f7bcc072350 0x7f7bcc074760 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:32.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.689+0000 7f7bdd7fa700 1 -- 172.21.15.22:0/1757074465 <== osd.5 v2:172.21.15.117:6816/38288182 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f7bcc074e10 con 0x7f7bcc072350 2024-02-16T19:25:32.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.705+0000 7f7be693c700 1 -- 172.21.15.22:0/1757074465 --> [v2:172.21.15.117:6816/38288182,v1:172.21.15.117:6817/38288182] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7be004f2a0 con 0x7f7bcc072350 2024-02-16T19:25:32.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.705+0000 7f7bdd7fa700 1 -- 172.21.15.22:0/1757074465 <== osd.5 v2:172.21.15.117:6816/38288182 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f7be004f2a0 con 0x7f7bcc072350 2024-02-16T19:25:32.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.706+0000 7f7be693c700 1 -- 172.21.15.22:0/1757074465 >> [v2:172.21.15.117:6816/38288182,v1:172.21.15.117:6817/38288182] conn(0x7f7bcc072350 msgr2=0x7f7bcc074760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:32.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.706+0000 7f7be693c700 1 --2- 172.21.15.22:0/1757074465 >> [v2:172.21.15.117:6816/38288182,v1:172.21.15.117:6817/38288182] conn(0x7f7bcc072350 0x7f7bcc074760 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:32.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.706+0000 7f7be693c700 1 -- 172.21.15.22:0/1757074465 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f7bcc06c680 msgr2=0x7f7bcc06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:32.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.706+0000 7f7be693c700 1 --2- 172.21.15.22:0/1757074465 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f7bcc06c680 0x7f7bcc06eb30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7bd0006fd0 tx=0x7f7bd0008040 comp rx=0 tx=0).stop 2024-02-16T19:25:32.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.706+0000 7f7be693c700 1 -- 172.21.15.22:0/1757074465 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7be00ffbd0 msgr2=0x7f7be01980c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:32.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.706+0000 7f7be693c700 1 --2- 172.21.15.22:0/1757074465 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7be00ffbd0 0x7f7be01980c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7bc8006010 tx=0x7f7bc800be20 comp rx=0 tx=0).stop 2024-02-16T19:25:32.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.706+0000 7f7be693c700 1 -- 172.21.15.22:0/1757074465 shutdown_connections 2024-02-16T19:25:32.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.706+0000 7f7be693c700 1 --2- 172.21.15.22:0/1757074465 >> [v2:172.21.15.117:6816/38288182,v1:172.21.15.117:6817/38288182] conn(0x7f7bcc072350 0x7f7bcc074760 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:32.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.706+0000 7f7be693c700 1 --2- 172.21.15.22:0/1757074465 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7be00ffbd0 0x7f7be01980c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:32.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.706+0000 7f7be693c700 1 --2- 172.21.15.22:0/1757074465 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f7bcc06c680 0x7f7bcc06eb30 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:32.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.706+0000 7f7be693c700 1 --2- 172.21.15.22:0/1757074465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7be0104130 0x7f7be0198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:32.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.706+0000 7f7be693c700 1 -- 172.21.15.22:0/1757074465 >> 172.21.15.22:0/1757074465 conn(0x7f7be0075130 msgr2=0x7f7be0068dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:25:32.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.706+0000 7f7be693c700 1 -- 172.21.15.22:0/1757074465 shutdown_connections 2024-02-16T19:25:32.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:32.706+0000 7f7be693c700 1 -- 172.21.15.22:0/1757074465 wait complete. 2024-02-16T19:25:33.250 INFO:teuthology.orchestra.run.smithi022.stdout:38654705699 2024-02-16T19:25:33.251 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-02-16T19:25:33.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:33.684+0000 7f07fa07e700 1 -- 172.21.15.22:0/2042432324 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f07f4100e60 msgr2=0x7f07f4104e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:33.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:33.684+0000 7f07fa07e700 1 --2- 172.21.15.22:0/2042432324 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f07f4100e60 0x7f07f4104e30 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f07e4009910 tx=0x7f07e4009c20 comp rx=0 tx=0).stop 2024-02-16T19:25:33.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:33.684+0000 7f07fa07e700 1 -- 172.21.15.22:0/2042432324 shutdown_connections 2024-02-16T19:25:33.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:33.684+0000 7f07fa07e700 1 --2- 172.21.15.22:0/2042432324 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f07f4100e60 0x7f07f4104e30 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:33.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:33.684+0000 7f07fa07e700 1 --2- 172.21.15.22:0/2042432324 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07f41004c0 0x7f07f4100890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:33.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:33.684+0000 7f07fa07e700 1 -- 172.21.15.22:0/2042432324 >> 172.21.15.22:0/2042432324 conn(0x7f07f40fbd80 msgr2=0x7f07f40fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:25:33.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:33.684+0000 7f07fa07e700 1 -- 172.21.15.22:0/2042432324 shutdown_connections 2024-02-16T19:25:33.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:33.684+0000 7f07fa07e700 1 -- 172.21.15.22:0/2042432324 wait complete. 2024-02-16T19:25:33.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:33.685+0000 7f07fa07e700 1 Processor -- start 2024-02-16T19:25:33.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:33.685+0000 7f07fa07e700 1 -- start start 2024-02-16T19:25:33.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:33.685+0000 7f07fa07e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f07f41004c0 0x7f07f4195fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:33.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:33.685+0000 7f07fa07e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07f4100e60 0x7f07f4196510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:33.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:33.685+0000 7f07fa07e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f4196bf0 con 0x7f07f41004c0 2024-02-16T19:25:33.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:33.685+0000 7f07fa07e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f419a980 con 0x7f07f4100e60 2024-02-16T19:25:33.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:33.685+0000 7f07f37fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f07f41004c0 0x7f07f4195fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:33.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:33.685+0000 7f07f37fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f07f41004c0 0x7f07f4195fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33828/0 (socket says 172.21.15.22:33828) 2024-02-16T19:25:33.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:33.685+0000 7f07f37fe700 1 -- 172.21.15.22:0/1463547207 learned_addr learned my addr 172.21.15.22:0/1463547207 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:25:33.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:33.685+0000 7f07f2ffd700 1 --2- 172.21.15.22:0/1463547207 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07f4100e60 0x7f07f4196510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:33.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:33.685+0000 7f07f37fe700 1 -- 172.21.15.22:0/1463547207 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07f4100e60 msgr2=0x7f07f4196510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:33.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:33.685+0000 7f07f37fe700 1 --2- 172.21.15.22:0/1463547207 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07f4100e60 0x7f07f4196510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:33.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:33.686+0000 7f07f37fe700 1 -- 172.21.15.22:0/1463547207 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07e40095f0 con 0x7f07f41004c0 2024-02-16T19:25:33.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:33.686+0000 7f07f2ffd700 1 --2- 172.21.15.22:0/1463547207 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07f4100e60 0x7f07f4196510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T19:25:33.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:33.686+0000 7f07f37fe700 1 --2- 172.21.15.22:0/1463547207 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f07f41004c0 0x7f07f4195fd0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f07dc00b4b0 tx=0x7f07dc00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:33.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:33.686+0000 7f07f0ff9700 1 -- 172.21.15.22:0/1463547207 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f07dc011800 con 0x7f07f41004c0 2024-02-16T19:25:33.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:33.686+0000 7f07f0ff9700 1 -- 172.21.15.22:0/1463547207 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f07dc011e10 con 0x7f07f41004c0 2024-02-16T19:25:33.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:33.686+0000 7f07fa07e700 1 -- 172.21.15.22:0/1463547207 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07f419ac60 con 0x7f07f41004c0 2024-02-16T19:25:33.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:33.686+0000 7f07f0ff9700 1 -- 172.21.15.22:0/1463547207 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f07dc00f2e0 con 0x7f07f41004c0 2024-02-16T19:25:33.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:33.686+0000 7f07fa07e700 1 -- 172.21.15.22:0/1463547207 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07f419b1b0 con 0x7f07f41004c0 2024-02-16T19:25:33.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:33.687+0000 7f07fa07e700 1 -- 172.21.15.22:0/1463547207 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f07f410d6b0 con 0x7f07f41004c0 2024-02-16T19:25:33.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:33.687+0000 7f07f0ff9700 1 -- 172.21.15.22:0/1463547207 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f07dc010380 con 0x7f07f41004c0 2024-02-16T19:25:33.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:33.688+0000 7f07f0ff9700 1 --2- 172.21.15.22:0/1463547207 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f07e006c680 0x7f07e006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:33.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:33.688+0000 7f07f0ff9700 1 -- 172.21.15.22:0/1463547207 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f07dc08b1a0 con 0x7f07f41004c0 2024-02-16T19:25:33.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:33.688+0000 7f07f2ffd700 1 --2- 172.21.15.22:0/1463547207 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f07e006c680 0x7f07e006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:33.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:33.688+0000 7f07f0ff9700 1 --2- 172.21.15.22:0/1463547207 >> [v2:172.21.15.22:6810/3355370212,v1:172.21.15.22:6811/3355370212] conn(0x7f07e0072350 0x7f07e0074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:33.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:33.688+0000 7f07f0ff9700 1 -- 172.21.15.22:0/1463547207 --> [v2:172.21.15.22:6810/3355370212,v1:172.21.15.22:6811/3355370212] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f07e0074e10 con 0x7f07e0072350 2024-02-16T19:25:33.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:33.688+0000 7f07f0ff9700 1 -- 172.21.15.22:0/1463547207 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f07dc08d870 con 0x7f07f41004c0 2024-02-16T19:25:33.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:33.688+0000 7f07f3fff700 1 --2- 172.21.15.22:0/1463547207 >> [v2:172.21.15.22:6810/3355370212,v1:172.21.15.22:6811/3355370212] conn(0x7f07e0072350 0x7f07e0074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:33.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:33.688+0000 7f07f2ffd700 1 --2- 172.21.15.22:0/1463547207 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f07e006c680 0x7f07e006eb30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f07f41975f0 tx=0x7f07e401a040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:33.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:33.688+0000 7f07f3fff700 1 --2- 172.21.15.22:0/1463547207 >> [v2:172.21.15.22:6810/3355370212,v1:172.21.15.22:6811/3355370212] conn(0x7f07e0072350 0x7f07e0074760 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:33.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:33.689+0000 7f07f0ff9700 1 -- 172.21.15.22:0/1463547207 <== osd.1 v2:172.21.15.22:6810/3355370212 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f07e0074e10 con 0x7f07e0072350 2024-02-16T19:25:33.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:33.705+0000 7f07fa07e700 1 -- 172.21.15.22:0/1463547207 --> [v2:172.21.15.22:6810/3355370212,v1:172.21.15.22:6811/3355370212] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f07f404f2a0 con 0x7f07e0072350 2024-02-16T19:25:33.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:33.705+0000 7f07f0ff9700 1 -- 172.21.15.22:0/1463547207 <== osd.1 v2:172.21.15.22:6810/3355370212 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f07f404f2a0 con 0x7f07e0072350 2024-02-16T19:25:33.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:33.705+0000 7f07fa07e700 1 -- 172.21.15.22:0/1463547207 >> [v2:172.21.15.22:6810/3355370212,v1:172.21.15.22:6811/3355370212] conn(0x7f07e0072350 msgr2=0x7f07e0074760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:33.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:33.705+0000 7f07fa07e700 1 --2- 172.21.15.22:0/1463547207 >> [v2:172.21.15.22:6810/3355370212,v1:172.21.15.22:6811/3355370212] conn(0x7f07e0072350 0x7f07e0074760 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:33.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:33.705+0000 7f07fa07e700 1 -- 172.21.15.22:0/1463547207 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f07e006c680 msgr2=0x7f07e006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:33.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:33.705+0000 7f07fa07e700 1 --2- 172.21.15.22:0/1463547207 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f07e006c680 0x7f07e006eb30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f07f41975f0 tx=0x7f07e401a040 comp rx=0 tx=0).stop 2024-02-16T19:25:33.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:33.705+0000 7f07fa07e700 1 -- 172.21.15.22:0/1463547207 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f07f41004c0 msgr2=0x7f07f4195fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:33.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:33.705+0000 7f07fa07e700 1 --2- 172.21.15.22:0/1463547207 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f07f41004c0 0x7f07f4195fd0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f07dc00b4b0 tx=0x7f07dc00b870 comp rx=0 tx=0).stop 2024-02-16T19:25:33.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:33.705+0000 7f07fa07e700 1 -- 172.21.15.22:0/1463547207 shutdown_connections 2024-02-16T19:25:33.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:33.705+0000 7f07fa07e700 1 --2- 172.21.15.22:0/1463547207 >> [v2:172.21.15.22:6810/3355370212,v1:172.21.15.22:6811/3355370212] conn(0x7f07e0072350 0x7f07e0074760 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:33.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:33.705+0000 7f07fa07e700 1 --2- 172.21.15.22:0/1463547207 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f07f41004c0 0x7f07f4195fd0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:33.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:33.705+0000 7f07fa07e700 1 --2- 172.21.15.22:0/1463547207 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f07e006c680 0x7f07e006eb30 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:33.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:33.706+0000 7f07fa07e700 1 --2- 172.21.15.22:0/1463547207 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07f4100e60 0x7f07f4196510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:33.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:33.706+0000 7f07fa07e700 1 -- 172.21.15.22:0/1463547207 >> 172.21.15.22:0/1463547207 conn(0x7f07f40fbd80 msgr2=0x7f07f40fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:25:33.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:33.706+0000 7f07fa07e700 1 -- 172.21.15.22:0/1463547207 shutdown_connections 2024-02-16T19:25:33.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:33.706+0000 7f07fa07e700 1 -- 172.21.15.22:0/1463547207 wait complete. 2024-02-16T19:25:34.079 INFO:teuthology.orchestra.run.smithi022.stdout:171798691849 2024-02-16T19:25:34.080 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-02-16T19:25:34.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:25:33 smithi022 ceph-mon[29221]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:25:34.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:25:33 smithi117 ceph-mon[35309]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:25:35.055 INFO:teuthology.orchestra.run.smithi022.stdout:115964117012 2024-02-16T19:25:35.056 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-02-16T19:25:35.654 INFO:teuthology.orchestra.run.smithi022.stdout:60129542174 2024-02-16T19:25:35.655 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-02-16T19:25:36.056 INFO:teuthology.orchestra.run.smithi022.stdout:81604378649 2024-02-16T19:25:36.056 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-02-16T19:25:36.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:25:35 smithi022 ceph-mon[29221]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:25:36.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:25:35 smithi117 ceph-mon[35309]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:25:36.860 INFO:teuthology.orchestra.run.smithi022.stdout:150323855375 2024-02-16T19:25:36.860 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-02-16T19:25:36.902 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi022/config 2024-02-16T19:25:36.959 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi022/config 2024-02-16T19:25:36.960 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi022/config 2024-02-16T19:25:36.961 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi022/config 2024-02-16T19:25:36.962 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi022/config 2024-02-16T19:25:38.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:25:37 smithi022 ceph-mon[29221]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:25:38.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:25:37 smithi117 ceph-mon[35309]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:25:40.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:25:39 smithi022 ceph-mon[29221]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:25:40.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:25:39 smithi117 ceph-mon[35309]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:25:41.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:41.523+0000 7f6b02c71700 1 -- 172.21.15.22:0/2581083807 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6afc100e80 msgr2=0x7f6afc104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:41.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:41.523+0000 7f6b02c71700 1 --2- 172.21.15.22:0/2581083807 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6afc100e80 0x7f6afc104e50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6af0009910 tx=0x7f6af0009c20 comp rx=0 tx=0).stop 2024-02-16T19:25:41.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:41.523+0000 7f6b02c71700 1 -- 172.21.15.22:0/2581083807 shutdown_connections 2024-02-16T19:25:41.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:41.523+0000 7f6b02c71700 1 --2- 172.21.15.22:0/2581083807 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6afc100e80 0x7f6afc104e50 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:41.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:41.523+0000 7f6b02c71700 1 --2- 172.21.15.22:0/2581083807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6afc1004e0 0x7f6afc1008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:41.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:41.523+0000 7f6b02c71700 1 -- 172.21.15.22:0/2581083807 >> 172.21.15.22:0/2581083807 conn(0x7f6afc0fbda0 msgr2=0x7f6afc0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:25:41.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:41.523+0000 7f6b02c71700 1 -- 172.21.15.22:0/2581083807 shutdown_connections 2024-02-16T19:25:41.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:41.523+0000 7f6b02c71700 1 -- 172.21.15.22:0/2581083807 wait complete. 2024-02-16T19:25:41.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:41.524+0000 7f6b02c71700 1 Processor -- start 2024-02-16T19:25:41.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:41.524+0000 7f6b02c71700 1 -- start start 2024-02-16T19:25:41.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:41.524+0000 7f6b02c71700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6afc1004e0 0x7f6afc195f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:41.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:41.524+0000 7f6b02c71700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6afc100e80 0x7f6afc196450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:41.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:41.524+0000 7f6b02c71700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6afc196b30 con 0x7f6afc1004e0 2024-02-16T19:25:41.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:41.524+0000 7f6b02c71700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6afc19a8c0 con 0x7f6afc100e80 2024-02-16T19:25:41.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:41.524+0000 7f6b00a0d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6afc1004e0 0x7f6afc195f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:41.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:41.524+0000 7f6b00a0d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6afc1004e0 0x7f6afc195f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37990/0 (socket says 172.21.15.22:37990) 2024-02-16T19:25:41.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:41.524+0000 7f6b00a0d700 1 -- 172.21.15.22:0/620535630 learned_addr learned my addr 172.21.15.22:0/620535630 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:25:41.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:41.524+0000 7f6afbfff700 1 --2- 172.21.15.22:0/620535630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6afc100e80 0x7f6afc196450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:41.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:41.524+0000 7f6b00a0d700 1 -- 172.21.15.22:0/620535630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6afc100e80 msgr2=0x7f6afc196450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:41.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:41.524+0000 7f6b00a0d700 1 --2- 172.21.15.22:0/620535630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6afc100e80 0x7f6afc196450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:41.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:41.524+0000 7f6b00a0d700 1 -- 172.21.15.22:0/620535630 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6af00095f0 con 0x7f6afc1004e0 2024-02-16T19:25:41.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:41.525+0000 7f6b00a0d700 1 --2- 172.21.15.22:0/620535630 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6afc1004e0 0x7f6afc195f10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6af40060b0 tx=0x7f6af400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:41.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:41.525+0000 7f6af9ffb700 1 -- 172.21.15.22:0/620535630 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6af400dda0 con 0x7f6afc1004e0 2024-02-16T19:25:41.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:41.525+0000 7f6af9ffb700 1 -- 172.21.15.22:0/620535630 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6af400f040 con 0x7f6afc1004e0 2024-02-16T19:25:41.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:41.525+0000 7f6af9ffb700 1 -- 172.21.15.22:0/620535630 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6af4014790 con 0x7f6afc1004e0 2024-02-16T19:25:41.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:41.525+0000 7f6b02c71700 1 -- 172.21.15.22:0/620535630 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6afc19aba0 con 0x7f6afc1004e0 2024-02-16T19:25:41.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:41.525+0000 7f6b02c71700 1 -- 172.21.15.22:0/620535630 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6afc19b0f0 con 0x7f6afc1004e0 2024-02-16T19:25:41.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:41.526+0000 7f6b02c71700 1 -- 172.21.15.22:0/620535630 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6afc108850 con 0x7f6afc1004e0 2024-02-16T19:25:41.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:41.527+0000 7f6af9ffb700 1 -- 172.21.15.22:0/620535630 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f6af40148f0 con 0x7f6afc1004e0 2024-02-16T19:25:41.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:41.527+0000 7f6af9ffb700 1 --2- 172.21.15.22:0/620535630 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f6ae406c680 0x7f6ae406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:41.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:41.527+0000 7f6af9ffb700 1 -- 172.21.15.22:0/620535630 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f6af408af50 con 0x7f6afc1004e0 2024-02-16T19:25:41.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:41.527+0000 7f6afbfff700 1 --2- 172.21.15.22:0/620535630 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f6ae406c680 0x7f6ae406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:41.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:41.527+0000 7f6afbfff700 1 --2- 172.21.15.22:0/620535630 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f6ae406c680 0x7f6ae406eb30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6af00059d0 tx=0x7f6af0005960 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:41.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:41.532+0000 7f6af9ffb700 1 -- 172.21.15.22:0/620535630 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f6af404f720 con 0x7f6afc1004e0 2024-02-16T19:25:41.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:41.732+0000 7f6b02c71700 1 -- 172.21.15.22:0/620535630 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f6afc04f2a0 con 0x7f6afc1004e0 2024-02-16T19:25:41.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:41.733+0000 7f6af9ffb700 1 -- 172.21.15.22:0/620535630 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6af4055aa0 con 0x7f6afc1004e0 2024-02-16T19:25:41.734 INFO:teuthology.orchestra.run.smithi022.stdout:38654705701 2024-02-16T19:25:41.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:41.735+0000 7f6b02c71700 1 -- 172.21.15.22:0/620535630 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f6ae406c680 msgr2=0x7f6ae406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:41.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:41.735+0000 7f6b02c71700 1 --2- 172.21.15.22:0/620535630 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f6ae406c680 0x7f6ae406eb30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6af00059d0 tx=0x7f6af0005960 comp rx=0 tx=0).stop 2024-02-16T19:25:41.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:41.735+0000 7f6b02c71700 1 -- 172.21.15.22:0/620535630 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6afc1004e0 msgr2=0x7f6afc195f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:41.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:41.735+0000 7f6b02c71700 1 --2- 172.21.15.22:0/620535630 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6afc1004e0 0x7f6afc195f10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6af40060b0 tx=0x7f6af400d3b0 comp rx=0 tx=0).stop 2024-02-16T19:25:41.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:41.735+0000 7f6b02c71700 1 -- 172.21.15.22:0/620535630 shutdown_connections 2024-02-16T19:25:41.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:41.735+0000 7f6b02c71700 1 --2- 172.21.15.22:0/620535630 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6afc1004e0 0x7f6afc195f10 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:41.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:41.735+0000 7f6b02c71700 1 --2- 172.21.15.22:0/620535630 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f6ae406c680 0x7f6ae406eb30 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:41.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:41.735+0000 7f6b02c71700 1 --2- 172.21.15.22:0/620535630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6afc100e80 0x7f6afc196450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:41.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:41.735+0000 7f6b02c71700 1 -- 172.21.15.22:0/620535630 >> 172.21.15.22:0/620535630 conn(0x7f6afc0fbda0 msgr2=0x7f6afc0fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:25:41.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:41.735+0000 7f6b02c71700 1 -- 172.21.15.22:0/620535630 shutdown_connections 2024-02-16T19:25:41.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:41.735+0000 7f6b02c71700 1 -- 172.21.15.22:0/620535630 wait complete. 2024-02-16T19:25:41.964 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:25:41 smithi022 ceph-mon[29221]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:25:41.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:25:41 smithi022 ceph-mon[29221]: from='client.? 172.21.15.22:0/620535630' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-16T19:25:41.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:25:41 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:25:42.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:42.204+0000 7fab4b0a6700 1 -- 172.21.15.22:0/21338553 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab44101d30 msgr2=0x7fab4410a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:42.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:42.204+0000 7fab4b0a6700 1 --2- 172.21.15.22:0/21338553 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab44101d30 0x7fab4410a230 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fab30009910 tx=0x7fab30009c20 comp rx=0 tx=0).stop 2024-02-16T19:25:42.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:42.204+0000 7fab4b0a6700 1 -- 172.21.15.22:0/21338553 shutdown_connections 2024-02-16T19:25:42.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:42.204+0000 7fab4b0a6700 1 --2- 172.21.15.22:0/21338553 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab44101d30 0x7fab4410a230 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:42.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:42.204+0000 7fab4b0a6700 1 --2- 172.21.15.22:0/21338553 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab44101420 0x7fab441017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:42.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:42.204+0000 7fab4b0a6700 1 -- 172.21.15.22:0/21338553 >> 172.21.15.22:0/21338553 conn(0x7fab440fac60 msgr2=0x7fab440fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:25:42.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:42.204+0000 7fab4b0a6700 1 -- 172.21.15.22:0/21338553 shutdown_connections 2024-02-16T19:25:42.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:42.204+0000 7fab4b0a6700 1 -- 172.21.15.22:0/21338553 wait complete. 2024-02-16T19:25:42.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:42.205+0000 7fab4b0a6700 1 Processor -- start 2024-02-16T19:25:42.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:42.205+0000 7fab4b0a6700 1 -- start start 2024-02-16T19:25:42.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:42.205+0000 7fab4b0a6700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab44101420 0x7fab44195ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:42.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:42.205+0000 7fab4b0a6700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab44101d30 0x7fab44196430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:42.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:42.205+0000 7fab4b0a6700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab44196b10 con 0x7fab44101420 2024-02-16T19:25:42.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:42.205+0000 7fab4b0a6700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab4419a8a0 con 0x7fab44101d30 2024-02-16T19:25:42.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:42.206+0000 7fab48e42700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab44101420 0x7fab44195ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:42.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:42.206+0000 7fab48e42700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab44101420 0x7fab44195ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38000/0 (socket says 172.21.15.22:38000) 2024-02-16T19:25:42.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:42.206+0000 7fab48e42700 1 -- 172.21.15.22:0/3658303150 learned_addr learned my addr 172.21.15.22:0/3658303150 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:25:42.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:42.206+0000 7fab43fff700 1 --2- 172.21.15.22:0/3658303150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab44101d30 0x7fab44196430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:42.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:42.206+0000 7fab48e42700 1 -- 172.21.15.22:0/3658303150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab44101d30 msgr2=0x7fab44196430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:42.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:42.206+0000 7fab48e42700 1 --2- 172.21.15.22:0/3658303150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab44101d30 0x7fab44196430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:42.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:42.206+0000 7fab48e42700 1 -- 172.21.15.22:0/3658303150 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab300095f0 con 0x7fab44101420 2024-02-16T19:25:42.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:42.206+0000 7fab43fff700 1 --2- 172.21.15.22:0/3658303150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab44101d30 0x7fab44196430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T19:25:42.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:42.206+0000 7fab48e42700 1 --2- 172.21.15.22:0/3658303150 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab44101420 0x7fab44195ef0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fab3800b4b0 tx=0x7fab3800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:42.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:42.207+0000 7fab41ffb700 1 -- 172.21.15.22:0/3658303150 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab38011800 con 0x7fab44101420 2024-02-16T19:25:42.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:42.207+0000 7fab41ffb700 1 -- 172.21.15.22:0/3658303150 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fab38011e10 con 0x7fab44101420 2024-02-16T19:25:42.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:42.207+0000 7fab4b0a6700 1 -- 172.21.15.22:0/3658303150 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab4419ab80 con 0x7fab44101420 2024-02-16T19:25:42.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:42.207+0000 7fab41ffb700 1 -- 172.21.15.22:0/3658303150 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab3800f2e0 con 0x7fab44101420 2024-02-16T19:25:42.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:42.207+0000 7fab4b0a6700 1 -- 172.21.15.22:0/3658303150 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab4419b0d0 con 0x7fab44101420 2024-02-16T19:25:42.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:42.208+0000 7fab41ffb700 1 -- 172.21.15.22:0/3658303150 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fab38010380 con 0x7fab44101420 2024-02-16T19:25:42.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:42.208+0000 7fab4b0a6700 1 -- 172.21.15.22:0/3658303150 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab441078a0 con 0x7fab44101420 2024-02-16T19:25:42.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:42.209+0000 7fab41ffb700 1 --2- 172.21.15.22:0/3658303150 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fab3406c6d0 0x7fab3406eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:42.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:42.209+0000 7fab41ffb700 1 -- 172.21.15.22:0/3658303150 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fab3808b120 con 0x7fab44101420 2024-02-16T19:25:42.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:42.209+0000 7fab43fff700 1 --2- 172.21.15.22:0/3658303150 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fab3406c6d0 0x7fab3406eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:42.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:42.209+0000 7fab43fff700 1 --2- 172.21.15.22:0/3658303150 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fab3406c6d0 0x7fab3406eb80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fab3000b5c0 tx=0x7fab3001a040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:42.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:42.213+0000 7fab41ffb700 1 -- 172.21.15.22:0/3658303150 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fab38055b20 con 0x7fab44101420 2024-02-16T19:25:42.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:25:41 smithi117 ceph-mon[35309]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:25:42.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:25:41 smithi117 ceph-mon[35309]: from='client.? 172.21.15.22:0/620535630' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-16T19:25:42.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:25:41 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:25:42.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:42.413+0000 7fab4b0a6700 1 -- 172.21.15.22:0/3658303150 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fab4404f2a0 con 0x7fab44101420 2024-02-16T19:25:42.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:42.414+0000 7fab41ffb700 1 -- 172.21.15.22:0/3658303150 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fab38059140 con 0x7fab44101420 2024-02-16T19:25:42.415 INFO:teuthology.orchestra.run.smithi022.stdout:115964117014 2024-02-16T19:25:42.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:42.417+0000 7fab4b0a6700 1 -- 172.21.15.22:0/3658303150 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fab3406c6d0 msgr2=0x7fab3406eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:42.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:42.417+0000 7fab4b0a6700 1 --2- 172.21.15.22:0/3658303150 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fab3406c6d0 0x7fab3406eb80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fab3000b5c0 tx=0x7fab3001a040 comp rx=0 tx=0).stop 2024-02-16T19:25:42.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:42.417+0000 7fab4b0a6700 1 -- 172.21.15.22:0/3658303150 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab44101420 msgr2=0x7fab44195ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:42.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:42.417+0000 7fab4b0a6700 1 --2- 172.21.15.22:0/3658303150 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab44101420 0x7fab44195ef0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fab3800b4b0 tx=0x7fab3800b870 comp rx=0 tx=0).stop 2024-02-16T19:25:42.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:42.417+0000 7fab4b0a6700 1 -- 172.21.15.22:0/3658303150 shutdown_connections 2024-02-16T19:25:42.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:42.417+0000 7fab4b0a6700 1 --2- 172.21.15.22:0/3658303150 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab44101420 0x7fab44195ef0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:42.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:42.417+0000 7fab4b0a6700 1 --2- 172.21.15.22:0/3658303150 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fab3406c6d0 0x7fab3406eb80 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:42.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:42.417+0000 7fab4b0a6700 1 --2- 172.21.15.22:0/3658303150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab44101d30 0x7fab44196430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:42.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:42.417+0000 7fab4b0a6700 1 -- 172.21.15.22:0/3658303150 >> 172.21.15.22:0/3658303150 conn(0x7fab440fac60 msgr2=0x7fab441049e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:25:42.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:42.417+0000 7fab4b0a6700 1 -- 172.21.15.22:0/3658303150 shutdown_connections 2024-02-16T19:25:42.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:42.417+0000 7fab4b0a6700 1 -- 172.21.15.22:0/3658303150 wait complete. 2024-02-16T19:25:42.652 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705699 got 38654705701 for osd.0 2024-02-16T19:25:42.653 DEBUG:teuthology.parallel:result is None 2024-02-16T19:25:42.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:25:42 smithi022 ceph-mon[29221]: from='client.? 172.21.15.22:0/3658303150' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-16T19:25:43.239 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi022/config 2024-02-16T19:25:43.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:25:42 smithi117 ceph-mon[35309]: from='client.? 172.21.15.22:0/3658303150' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-16T19:25:43.964 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:25:43 smithi022 ceph-mon[29221]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:25:44.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.369+0000 7f662bf2f700 1 -- 172.21.15.22:0/628360403 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66240ff3c0 msgr2=0x7f66240ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:44.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.369+0000 7f662bf2f700 1 --2- 172.21.15.22:0/628360403 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66240ff3c0 0x7f66240ff790 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f661400b0a0 tx=0x7f661400b3b0 comp rx=0 tx=0).stop 2024-02-16T19:25:44.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.369+0000 7f662bf2f700 1 -- 172.21.15.22:0/628360403 shutdown_connections 2024-02-16T19:25:44.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.369+0000 7f662bf2f700 1 --2- 172.21.15.22:0/628360403 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66240ffd60 0x7f6624103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:44.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.369+0000 7f662bf2f700 1 --2- 172.21.15.22:0/628360403 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66240ff3c0 0x7f66240ff790 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:44.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.369+0000 7f662bf2f700 1 -- 172.21.15.22:0/628360403 >> 172.21.15.22:0/628360403 conn(0x7f66240fac80 msgr2=0x7f66240fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:25:44.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.369+0000 7f662bf2f700 1 -- 172.21.15.22:0/628360403 shutdown_connections 2024-02-16T19:25:44.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.369+0000 7f662bf2f700 1 -- 172.21.15.22:0/628360403 wait complete. 2024-02-16T19:25:44.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.370+0000 7f662bf2f700 1 Processor -- start 2024-02-16T19:25:44.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.370+0000 7f662bf2f700 1 -- start start 2024-02-16T19:25:44.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.370+0000 7f662bf2f700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66240ff3c0 0x7f6624196000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:44.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.370+0000 7f662bf2f700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66240ffd60 0x7f6624196540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:44.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.370+0000 7f662bf2f700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6624196c20 con 0x7f66240ff3c0 2024-02-16T19:25:44.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.370+0000 7f662bf2f700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f662419a9b0 con 0x7f66240ffd60 2024-02-16T19:25:44.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.370+0000 7f6629ccb700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66240ff3c0 0x7f6624196000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:44.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.370+0000 7f6629ccb700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66240ff3c0 0x7f6624196000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35754/0 (socket says 172.21.15.22:35754) 2024-02-16T19:25:44.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.370+0000 7f6629ccb700 1 -- 172.21.15.22:0/445308017 learned_addr learned my addr 172.21.15.22:0/445308017 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:25:44.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.370+0000 7f66294ca700 1 --2- 172.21.15.22:0/445308017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66240ffd60 0x7f6624196540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:44.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.371+0000 7f6629ccb700 1 -- 172.21.15.22:0/445308017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66240ffd60 msgr2=0x7f6624196540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:44.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.371+0000 7f6629ccb700 1 --2- 172.21.15.22:0/445308017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66240ffd60 0x7f6624196540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:44.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.371+0000 7f6629ccb700 1 -- 172.21.15.22:0/445308017 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6614009d00 con 0x7f66240ff3c0 2024-02-16T19:25:44.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.371+0000 7f6629ccb700 1 --2- 172.21.15.22:0/445308017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66240ff3c0 0x7f6624196000 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6614000f80 tx=0x7f661400cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:44.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.371+0000 7f6622ffd700 1 -- 172.21.15.22:0/445308017 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f661400e040 con 0x7f66240ff3c0 2024-02-16T19:25:44.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.371+0000 7f662bf2f700 1 -- 172.21.15.22:0/445308017 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f662419ac30 con 0x7f66240ff3c0 2024-02-16T19:25:44.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.371+0000 7f6622ffd700 1 -- 172.21.15.22:0/445308017 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f66140044f0 con 0x7f66240ff3c0 2024-02-16T19:25:44.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.371+0000 7f6622ffd700 1 -- 172.21.15.22:0/445308017 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f661401d9f0 con 0x7f66240ff3c0 2024-02-16T19:25:44.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.371+0000 7f662bf2f700 1 -- 172.21.15.22:0/445308017 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f662419b120 con 0x7f66240ff3c0 2024-02-16T19:25:44.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.372+0000 7f662bf2f700 1 -- 172.21.15.22:0/445308017 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f662404f2a0 con 0x7f66240ff3c0 2024-02-16T19:25:44.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.372+0000 7f6622ffd700 1 -- 172.21.15.22:0/445308017 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f6614004040 con 0x7f66240ff3c0 2024-02-16T19:25:44.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.373+0000 7f6622ffd700 1 --2- 172.21.15.22:0/445308017 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f660c06c470 0x7f660c06e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:44.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.373+0000 7f6622ffd700 1 -- 172.21.15.22:0/445308017 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f661408ca50 con 0x7f66240ff3c0 2024-02-16T19:25:44.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.373+0000 7f66294ca700 1 --2- 172.21.15.22:0/445308017 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f660c06c470 0x7f660c06e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:44.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.373+0000 7f66294ca700 1 --2- 172.21.15.22:0/445308017 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f660c06c470 0x7f660c06e920 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6618006fd0 tx=0x7f6618008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:44.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.381+0000 7f6622ffd700 1 -- 172.21.15.22:0/445308017 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f6614057470 con 0x7f66240ff3c0 2024-02-16T19:25:44.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:25:43 smithi117 ceph-mon[35309]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:25:44.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.599+0000 7f662bf2f700 1 -- 172.21.15.22:0/445308017 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f662419b360 con 0x7f66240ff3c0 2024-02-16T19:25:44.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.600+0000 7f6622ffd700 1 -- 172.21.15.22:0/445308017 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f661405aa90 con 0x7f66240ff3c0 2024-02-16T19:25:44.600 INFO:teuthology.orchestra.run.smithi022.stdout:81604378651 2024-02-16T19:25:44.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.602+0000 7f662bf2f700 1 -- 172.21.15.22:0/445308017 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f660c06c470 msgr2=0x7f660c06e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:44.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.602+0000 7f662bf2f700 1 --2- 172.21.15.22:0/445308017 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f660c06c470 0x7f660c06e920 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6618006fd0 tx=0x7f6618008040 comp rx=0 tx=0).stop 2024-02-16T19:25:44.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.602+0000 7f662bf2f700 1 -- 172.21.15.22:0/445308017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66240ff3c0 msgr2=0x7f6624196000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:44.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.602+0000 7f662bf2f700 1 --2- 172.21.15.22:0/445308017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66240ff3c0 0x7f6624196000 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6614000f80 tx=0x7f661400cdb0 comp rx=0 tx=0).stop 2024-02-16T19:25:44.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.602+0000 7f662bf2f700 1 -- 172.21.15.22:0/445308017 shutdown_connections 2024-02-16T19:25:44.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.602+0000 7f662bf2f700 1 --2- 172.21.15.22:0/445308017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66240ff3c0 0x7f6624196000 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:44.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.603+0000 7f662bf2f700 1 --2- 172.21.15.22:0/445308017 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f660c06c470 0x7f660c06e920 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:44.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.603+0000 7f662bf2f700 1 --2- 172.21.15.22:0/445308017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66240ffd60 0x7f6624196540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:44.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.603+0000 7f662bf2f700 1 -- 172.21.15.22:0/445308017 >> 172.21.15.22:0/445308017 conn(0x7f66240fac80 msgr2=0x7f66241034e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:25:44.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.604+0000 7f662bf2f700 1 -- 172.21.15.22:0/445308017 shutdown_connections 2024-02-16T19:25:44.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.604+0000 7f662bf2f700 1 -- 172.21.15.22:0/445308017 wait complete. 2024-02-16T19:25:44.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.873+0000 7f4bd8cb7700 1 -- 172.21.15.22:0/2252061249 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4bd4101d30 msgr2=0x7f4bd410a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:44.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.873+0000 7f4bd8cb7700 1 --2- 172.21.15.22:0/2252061249 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4bd4101d30 0x7f4bd410a230 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc009910 tx=0x7f4bbc009c20 comp rx=0 tx=0).stop 2024-02-16T19:25:44.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.873+0000 7f4bd8cb7700 1 -- 172.21.15.22:0/2252061249 shutdown_connections 2024-02-16T19:25:44.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.873+0000 7f4bd8cb7700 1 --2- 172.21.15.22:0/2252061249 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4bd4101d30 0x7f4bd410a230 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:44.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.873+0000 7f4bd8cb7700 1 --2- 172.21.15.22:0/2252061249 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4bd4101420 0x7f4bd41017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:44.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.873+0000 7f4bd8cb7700 1 -- 172.21.15.22:0/2252061249 >> 172.21.15.22:0/2252061249 conn(0x7f4bd40fac60 msgr2=0x7f4bd40fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:25:44.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.873+0000 7f4bd8cb7700 1 -- 172.21.15.22:0/2252061249 shutdown_connections 2024-02-16T19:25:44.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.873+0000 7f4bd8cb7700 1 -- 172.21.15.22:0/2252061249 wait complete. 2024-02-16T19:25:44.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.874+0000 7f4bd8cb7700 1 Processor -- start 2024-02-16T19:25:44.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.874+0000 7f4bd8cb7700 1 -- start start 2024-02-16T19:25:44.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.874+0000 7f4bd8cb7700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4bd4101420 0x7f4bd4195e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:44.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.874+0000 7f4bd8cb7700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4bd4101d30 0x7f4bd4196390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:44.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.874+0000 7f4bd8cb7700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd41969e0 con 0x7f4bd4101d30 2024-02-16T19:25:44.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.874+0000 7f4bd8cb7700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd4196b20 con 0x7f4bd4101420 2024-02-16T19:25:44.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.874+0000 7f4bd1d9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4bd4101d30 0x7f4bd4196390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:44.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.874+0000 7f4bd1d9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4bd4101d30 0x7f4bd4196390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35770/0 (socket says 172.21.15.22:35770) 2024-02-16T19:25:44.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.874+0000 7f4bd1d9b700 1 -- 172.21.15.22:0/782450048 learned_addr learned my addr 172.21.15.22:0/782450048 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:25:44.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.874+0000 7f4bd1d9b700 1 -- 172.21.15.22:0/782450048 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4bd4101420 msgr2=0x7f4bd4195e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:44.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.874+0000 7f4bd1d9b700 1 --2- 172.21.15.22:0/782450048 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4bd4101420 0x7f4bd4195e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:44.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.874+0000 7f4bd1d9b700 1 -- 172.21.15.22:0/782450048 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bbc0095f0 con 0x7f4bd4101d30 2024-02-16T19:25:44.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.874+0000 7f4bd1d9b700 1 --2- 172.21.15.22:0/782450048 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4bd4101d30 0x7f4bd4196390 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc006010 tx=0x7f4bbc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:44.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.874+0000 7f4bcb7fe700 1 -- 172.21.15.22:0/782450048 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4bbc01d070 con 0x7f4bd4101d30 2024-02-16T19:25:44.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.875+0000 7f4bd8cb7700 1 -- 172.21.15.22:0/782450048 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bd419a910 con 0x7f4bd4101d30 2024-02-16T19:25:44.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.875+0000 7f4bcb7fe700 1 -- 172.21.15.22:0/782450048 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4bbc00f460 con 0x7f4bd4101d30 2024-02-16T19:25:44.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.875+0000 7f4bcb7fe700 1 -- 172.21.15.22:0/782450048 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4bbc021640 con 0x7f4bd4101d30 2024-02-16T19:25:44.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.875+0000 7f4bd8cb7700 1 -- 172.21.15.22:0/782450048 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bd419ae00 con 0x7f4bd4101d30 2024-02-16T19:25:44.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.875+0000 7f4bd8cb7700 1 -- 172.21.15.22:0/782450048 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bd41078a0 con 0x7f4bd4101d30 2024-02-16T19:25:44.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.876+0000 7f4bcb7fe700 1 -- 172.21.15.22:0/782450048 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f4bbc00fab0 con 0x7f4bd4101d30 2024-02-16T19:25:44.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.877+0000 7f4bcb7fe700 1 --2- 172.21.15.22:0/782450048 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f4bc006c680 0x7f4bc006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:44.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.877+0000 7f4bcb7fe700 1 -- 172.21.15.22:0/782450048 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f4bbc08c860 con 0x7f4bd4101d30 2024-02-16T19:25:44.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.877+0000 7f4bd259c700 1 --2- 172.21.15.22:0/782450048 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f4bc006c680 0x7f4bc006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:44.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.877+0000 7f4bd259c700 1 --2- 172.21.15.22:0/782450048 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f4bc006c680 0x7f4bc006eb30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc005fd0 tx=0x7f4bcc005d00 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:44.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:44.882+0000 7f4bcb7fe700 1 -- 172.21.15.22:0/782450048 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f4bbc057280 con 0x7f4bd4101d30 2024-02-16T19:25:45.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:45.085+0000 7f4bd8cb7700 1 -- 172.21.15.22:0/782450048 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f4bd404f2a0 con 0x7f4bd4101d30 2024-02-16T19:25:45.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:45.085+0000 7f4bcb7fe700 1 -- 172.21.15.22:0/782450048 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4bbc026090 con 0x7f4bd4101d30 2024-02-16T19:25:45.085 INFO:teuthology.orchestra.run.smithi022.stdout:60129542176 2024-02-16T19:25:45.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:45.088+0000 7f4bd8cb7700 1 -- 172.21.15.22:0/782450048 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f4bc006c680 msgr2=0x7f4bc006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:45.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:45.088+0000 7f4bd8cb7700 1 --2- 172.21.15.22:0/782450048 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f4bc006c680 0x7f4bc006eb30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc005fd0 tx=0x7f4bcc005d00 comp rx=0 tx=0).stop 2024-02-16T19:25:45.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:45.088+0000 7f4bd8cb7700 1 -- 172.21.15.22:0/782450048 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4bd4101d30 msgr2=0x7f4bd4196390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:45.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:45.088+0000 7f4bd8cb7700 1 --2- 172.21.15.22:0/782450048 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4bd4101d30 0x7f4bd4196390 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc006010 tx=0x7f4bbc00be20 comp rx=0 tx=0).stop 2024-02-16T19:25:45.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:45.088+0000 7f4bd8cb7700 1 -- 172.21.15.22:0/782450048 shutdown_connections 2024-02-16T19:25:45.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:45.088+0000 7f4bd8cb7700 1 --2- 172.21.15.22:0/782450048 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f4bc006c680 0x7f4bc006eb30 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:45.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:45.088+0000 7f4bd8cb7700 1 --2- 172.21.15.22:0/782450048 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4bd4101420 0x7f4bd4195e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:45.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:45.088+0000 7f4bd8cb7700 1 --2- 172.21.15.22:0/782450048 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4bd4101d30 0x7f4bd4196390 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:45.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:45.088+0000 7f4bd8cb7700 1 -- 172.21.15.22:0/782450048 >> 172.21.15.22:0/782450048 conn(0x7f4bd40fac60 msgr2=0x7f4bd41049e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:25:45.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:45.088+0000 7f4bd8cb7700 1 -- 172.21.15.22:0/782450048 shutdown_connections 2024-02-16T19:25:45.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:45.088+0000 7f4bd8cb7700 1 -- 172.21.15.22:0/782450048 wait complete. 2024-02-16T19:25:45.129 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:25:44 smithi022 ceph-mon[29221]: from='client.? 172.21.15.22:0/445308017' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-16T19:25:45.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:25:44 smithi117 ceph-mon[35309]: from='client.? 172.21.15.22:0/445308017' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-16T19:25:46.045 INFO:tasks.cephadm.ceph_manager.ceph:need seq 115964117012 got 115964117014 for osd.3 2024-02-16T19:25:46.045 DEBUG:teuthology.parallel:result is None 2024-02-16T19:25:46.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:25:45 smithi022 ceph-mon[29221]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:25:46.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:25:45 smithi022 ceph-mon[29221]: from='client.? 172.21.15.22:0/782450048' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-16T19:25:46.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:46.266+0000 7ff7b3389700 1 -- 172.21.15.22:0/965292356 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff7ac100e80 msgr2=0x7ff7ac104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:46.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:46.266+0000 7ff7b3389700 1 --2- 172.21.15.22:0/965292356 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff7ac100e80 0x7ff7ac104e50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff7a8009910 tx=0x7ff7a8009c20 comp rx=0 tx=0).stop 2024-02-16T19:25:46.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:46.266+0000 7ff7b3389700 1 -- 172.21.15.22:0/965292356 shutdown_connections 2024-02-16T19:25:46.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:46.266+0000 7ff7b3389700 1 --2- 172.21.15.22:0/965292356 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff7ac100e80 0x7ff7ac104e50 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:46.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:46.266+0000 7ff7b3389700 1 --2- 172.21.15.22:0/965292356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7ac1004e0 0x7ff7ac1008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:46.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:46.266+0000 7ff7b3389700 1 -- 172.21.15.22:0/965292356 >> 172.21.15.22:0/965292356 conn(0x7ff7ac0fbda0 msgr2=0x7ff7ac0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:25:46.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:46.266+0000 7ff7b3389700 1 -- 172.21.15.22:0/965292356 shutdown_connections 2024-02-16T19:25:46.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:46.266+0000 7ff7b3389700 1 -- 172.21.15.22:0/965292356 wait complete. 2024-02-16T19:25:46.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:46.267+0000 7ff7b3389700 1 Processor -- start 2024-02-16T19:25:46.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:46.267+0000 7ff7b3389700 1 -- start start 2024-02-16T19:25:46.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:46.267+0000 7ff7b3389700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7ac1004e0 0x7ff7ac073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:46.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:46.267+0000 7ff7b3389700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff7ac100e80 0x7ff7ac073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:46.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:46.267+0000 7ff7b3389700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7ac073ad0 con 0x7ff7ac100e80 2024-02-16T19:25:46.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:46.267+0000 7ff7b3389700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7ac073c10 con 0x7ff7ac1004e0 2024-02-16T19:25:46.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:46.267+0000 7ff7b0924700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff7ac100e80 0x7ff7ac073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:46.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:46.267+0000 7ff7b0924700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff7ac100e80 0x7ff7ac073590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35784/0 (socket says 172.21.15.22:35784) 2024-02-16T19:25:46.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:46.267+0000 7ff7b0924700 1 -- 172.21.15.22:0/239937266 learned_addr learned my addr 172.21.15.22:0/239937266 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:25:46.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:46.267+0000 7ff7b1125700 1 --2- 172.21.15.22:0/239937266 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7ac1004e0 0x7ff7ac073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:46.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:46.268+0000 7ff7b0924700 1 -- 172.21.15.22:0/239937266 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7ac1004e0 msgr2=0x7ff7ac073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:46.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:46.268+0000 7ff7b0924700 1 --2- 172.21.15.22:0/239937266 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7ac1004e0 0x7ff7ac073050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:46.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:46.268+0000 7ff7b0924700 1 -- 172.21.15.22:0/239937266 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7a80095f0 con 0x7ff7ac100e80 2024-02-16T19:25:46.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:46.268+0000 7ff7b1125700 1 --2- 172.21.15.22:0/239937266 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7ac1004e0 0x7ff7ac073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-16T19:25:46.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:46.268+0000 7ff7b0924700 1 --2- 172.21.15.22:0/239937266 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff7ac100e80 0x7ff7ac073590 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff7a80092e0 tx=0x7ff7a800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:46.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:46.268+0000 7ff7a27fc700 1 -- 172.21.15.22:0/239937266 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff7a801d070 con 0x7ff7ac100e80 2024-02-16T19:25:46.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:46.268+0000 7ff7a27fc700 1 -- 172.21.15.22:0/239937266 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff7a800f460 con 0x7ff7ac100e80 2024-02-16T19:25:46.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:46.268+0000 7ff7b3389700 1 -- 172.21.15.22:0/239937266 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7ac073e90 con 0x7ff7ac100e80 2024-02-16T19:25:46.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:46.268+0000 7ff7a27fc700 1 -- 172.21.15.22:0/239937266 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff7a8017630 con 0x7ff7ac100e80 2024-02-16T19:25:46.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:46.268+0000 7ff7b3389700 1 -- 172.21.15.22:0/239937266 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7ac1a2410 con 0x7ff7ac100e80 2024-02-16T19:25:46.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:46.269+0000 7ff7b3389700 1 -- 172.21.15.22:0/239937266 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7ac108850 con 0x7ff7ac100e80 2024-02-16T19:25:46.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:46.270+0000 7ff7a27fc700 1 -- 172.21.15.22:0/239937266 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7ff7a800f5d0 con 0x7ff7ac100e80 2024-02-16T19:25:46.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:46.270+0000 7ff7a27fc700 1 --2- 172.21.15.22:0/239937266 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7ff79806c680 0x7ff79806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:46.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:46.270+0000 7ff7a27fc700 1 -- 172.21.15.22:0/239937266 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7ff7a808ca20 con 0x7ff7ac100e80 2024-02-16T19:25:46.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:46.270+0000 7ff7b1125700 1 --2- 172.21.15.22:0/239937266 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7ff79806c680 0x7ff79806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:46.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:46.271+0000 7ff7b1125700 1 --2- 172.21.15.22:0/239937266 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7ff79806c680 0x7ff79806eb30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff79c005fd0 tx=0x7ff79c005ee0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:46.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:46.275+0000 7ff7a27fc700 1 -- 172.21.15.22:0/239937266 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7ff7a80573e0 con 0x7ff7ac100e80 2024-02-16T19:25:46.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:25:45 smithi117 ceph-mon[35309]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:25:46.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:25:45 smithi117 ceph-mon[35309]: from='client.? 172.21.15.22:0/782450048' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-16T19:25:46.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:46.477+0000 7ff7b3389700 1 -- 172.21.15.22:0/239937266 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7ff7ac074580 con 0x7ff7ac100e80 2024-02-16T19:25:46.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:46.477+0000 7ff7a27fc700 1 -- 172.21.15.22:0/239937266 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff7a805aa00 con 0x7ff7ac100e80 2024-02-16T19:25:46.478 INFO:teuthology.orchestra.run.smithi022.stdout:171798691851 2024-02-16T19:25:46.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:46.479+0000 7ff7b3389700 1 -- 172.21.15.22:0/239937266 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7ff79806c680 msgr2=0x7ff79806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:46.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:46.479+0000 7ff7b3389700 1 --2- 172.21.15.22:0/239937266 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7ff79806c680 0x7ff79806eb30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff79c005fd0 tx=0x7ff79c005ee0 comp rx=0 tx=0).stop 2024-02-16T19:25:46.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:46.479+0000 7ff7b3389700 1 -- 172.21.15.22:0/239937266 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff7ac100e80 msgr2=0x7ff7ac073590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:46.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:46.479+0000 7ff7b3389700 1 --2- 172.21.15.22:0/239937266 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff7ac100e80 0x7ff7ac073590 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff7a80092e0 tx=0x7ff7a800be20 comp rx=0 tx=0).stop 2024-02-16T19:25:46.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:46.479+0000 7ff7b3389700 1 -- 172.21.15.22:0/239937266 shutdown_connections 2024-02-16T19:25:46.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:46.479+0000 7ff7b3389700 1 --2- 172.21.15.22:0/239937266 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7ff79806c680 0x7ff79806eb30 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:46.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:46.479+0000 7ff7b3389700 1 --2- 172.21.15.22:0/239937266 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7ac1004e0 0x7ff7ac073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:46.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:46.479+0000 7ff7b3389700 1 --2- 172.21.15.22:0/239937266 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff7ac100e80 0x7ff7ac073590 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:46.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:46.480+0000 7ff7b3389700 1 -- 172.21.15.22:0/239937266 >> 172.21.15.22:0/239937266 conn(0x7ff7ac0fbda0 msgr2=0x7ff7ac0fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:25:46.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:46.480+0000 7ff7b3389700 1 -- 172.21.15.22:0/239937266 shutdown_connections 2024-02-16T19:25:46.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:46.480+0000 7ff7b3389700 1 -- 172.21.15.22:0/239937266 wait complete. 2024-02-16T19:25:47.051 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:25:46 smithi022 ceph-mon[29221]: from='client.? 172.21.15.22:0/239937266' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-16T19:25:47.073 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542174 got 60129542176 for osd.1 2024-02-16T19:25:47.073 DEBUG:teuthology.parallel:result is None 2024-02-16T19:25:47.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:25:46 smithi117 ceph-mon[35309]: from='client.? 172.21.15.22:0/239937266' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-16T19:25:47.679 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378649 got 81604378651 for osd.2 2024-02-16T19:25:47.679 DEBUG:teuthology.parallel:result is None 2024-02-16T19:25:48.136 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:25:47 smithi117 ceph-mon[35309]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:25:48.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:25:47 smithi022 ceph-mon[29221]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:25:48.250 INFO:tasks.cephadm.ceph_manager.ceph:need seq 171798691849 got 171798691851 for osd.5 2024-02-16T19:25:48.250 DEBUG:teuthology.parallel:result is None 2024-02-16T19:25:49.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:49.249+0000 7ff7f514b700 1 -- 172.21.15.22:0/2169376780 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff7f00ffd60 msgr2=0x7ff7f0103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:49.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:49.249+0000 7ff7f514b700 1 --2- 172.21.15.22:0/2169376780 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff7f00ffd60 0x7ff7f0103d30 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff7e0009910 tx=0x7ff7e0009c20 comp rx=0 tx=0).stop 2024-02-16T19:25:49.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:49.249+0000 7ff7f514b700 1 -- 172.21.15.22:0/2169376780 shutdown_connections 2024-02-16T19:25:49.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:49.249+0000 7ff7f514b700 1 --2- 172.21.15.22:0/2169376780 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff7f00ffd60 0x7ff7f0103d30 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:49.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:49.249+0000 7ff7f514b700 1 --2- 172.21.15.22:0/2169376780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7f00ff3c0 0x7ff7f00ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:49.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:49.249+0000 7ff7f514b700 1 -- 172.21.15.22:0/2169376780 >> 172.21.15.22:0/2169376780 conn(0x7ff7f00fac80 msgr2=0x7ff7f00fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:25:49.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:49.249+0000 7ff7f514b700 1 -- 172.21.15.22:0/2169376780 shutdown_connections 2024-02-16T19:25:49.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:49.249+0000 7ff7f514b700 1 -- 172.21.15.22:0/2169376780 wait complete. 2024-02-16T19:25:49.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:49.250+0000 7ff7f514b700 1 Processor -- start 2024-02-16T19:25:49.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:49.250+0000 7ff7f514b700 1 -- start start 2024-02-16T19:25:49.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:49.250+0000 7ff7f514b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7f00ff3c0 0x7ff7f0195eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:49.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:49.250+0000 7ff7f514b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff7f00ffd60 0x7ff7f01963f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:49.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:49.250+0000 7ff7f514b700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f0196ad0 con 0x7ff7f00ffd60 2024-02-16T19:25:49.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:49.250+0000 7ff7f514b700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f019a860 con 0x7ff7f00ff3c0 2024-02-16T19:25:49.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:49.250+0000 7ff7ee59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff7f00ffd60 0x7ff7f01963f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:49.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:49.250+0000 7ff7ee59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff7f00ffd60 0x7ff7f01963f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35806/0 (socket says 172.21.15.22:35806) 2024-02-16T19:25:49.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:49.250+0000 7ff7ee59c700 1 -- 172.21.15.22:0/1175763941 learned_addr learned my addr 172.21.15.22:0/1175763941 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:25:49.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:49.251+0000 7ff7eed9d700 1 --2- 172.21.15.22:0/1175763941 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7f00ff3c0 0x7ff7f0195eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:49.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:49.251+0000 7ff7ee59c700 1 -- 172.21.15.22:0/1175763941 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7f00ff3c0 msgr2=0x7ff7f0195eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:49.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:49.251+0000 7ff7ee59c700 1 --2- 172.21.15.22:0/1175763941 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7f00ff3c0 0x7ff7f0195eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:49.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:49.251+0000 7ff7ee59c700 1 -- 172.21.15.22:0/1175763941 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7e00095f0 con 0x7ff7f00ffd60 2024-02-16T19:25:49.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:49.251+0000 7ff7ee59c700 1 --2- 172.21.15.22:0/1175763941 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff7f00ffd60 0x7ff7f01963f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff7e0005f50 tx=0x7ff7e000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:49.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:49.251+0000 7ff7d7fff700 1 -- 172.21.15.22:0/1175763941 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff7e001d070 con 0x7ff7f00ffd60 2024-02-16T19:25:49.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:49.251+0000 7ff7d7fff700 1 -- 172.21.15.22:0/1175763941 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff7e000f460 con 0x7ff7f00ffd60 2024-02-16T19:25:49.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:49.251+0000 7ff7f514b700 1 -- 172.21.15.22:0/1175763941 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7f019aae0 con 0x7ff7f00ffd60 2024-02-16T19:25:49.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:49.251+0000 7ff7d7fff700 1 -- 172.21.15.22:0/1175763941 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff7e0017630 con 0x7ff7f00ffd60 2024-02-16T19:25:49.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:49.251+0000 7ff7f514b700 1 -- 172.21.15.22:0/1175763941 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7f019afd0 con 0x7ff7f00ffd60 2024-02-16T19:25:49.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:49.252+0000 7ff7f514b700 1 -- 172.21.15.22:0/1175763941 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7f004f2a0 con 0x7ff7f00ffd60 2024-02-16T19:25:49.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:49.253+0000 7ff7d7fff700 1 -- 172.21.15.22:0/1175763941 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7ff7e0003a60 con 0x7ff7f00ffd60 2024-02-16T19:25:49.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:49.253+0000 7ff7d7fff700 1 --2- 172.21.15.22:0/1175763941 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7ff7d806c680 0x7ff7d806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:49.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:49.253+0000 7ff7eed9d700 1 --2- 172.21.15.22:0/1175763941 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7ff7d806c680 0x7ff7d806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:49.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:49.253+0000 7ff7d7fff700 1 -- 172.21.15.22:0/1175763941 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7ff7e008ca20 con 0x7ff7f00ffd60 2024-02-16T19:25:49.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:49.254+0000 7ff7eed9d700 1 --2- 172.21.15.22:0/1175763941 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7ff7d806c680 0x7ff7d806eb30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4005950 tx=0x7ff7e400a300 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:49.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:49.258+0000 7ff7d7fff700 1 -- 172.21.15.22:0/1175763941 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7ff7e0057440 con 0x7ff7f00ffd60 2024-02-16T19:25:49.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:49.458+0000 7ff7f514b700 1 -- 172.21.15.22:0/1175763941 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7ff7f019b2b0 con 0x7ff7f00ffd60 2024-02-16T19:25:49.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:49.459+0000 7ff7d7fff700 1 -- 172.21.15.22:0/1175763941 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff7e005aa60 con 0x7ff7f00ffd60 2024-02-16T19:25:49.459 INFO:teuthology.orchestra.run.smithi022.stdout:150323855378 2024-02-16T19:25:49.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:49.461+0000 7ff7f514b700 1 -- 172.21.15.22:0/1175763941 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7ff7d806c680 msgr2=0x7ff7d806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:49.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:49.461+0000 7ff7f514b700 1 --2- 172.21.15.22:0/1175763941 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7ff7d806c680 0x7ff7d806eb30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4005950 tx=0x7ff7e400a300 comp rx=0 tx=0).stop 2024-02-16T19:25:49.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:49.461+0000 7ff7f514b700 1 -- 172.21.15.22:0/1175763941 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff7f00ffd60 msgr2=0x7ff7f01963f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:49.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:49.461+0000 7ff7f514b700 1 --2- 172.21.15.22:0/1175763941 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff7f00ffd60 0x7ff7f01963f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff7e0005f50 tx=0x7ff7e000bd70 comp rx=0 tx=0).stop 2024-02-16T19:25:49.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:49.461+0000 7ff7f514b700 1 -- 172.21.15.22:0/1175763941 shutdown_connections 2024-02-16T19:25:49.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:49.461+0000 7ff7f514b700 1 --2- 172.21.15.22:0/1175763941 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7ff7d806c680 0x7ff7d806eb30 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:49.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:49.461+0000 7ff7f514b700 1 --2- 172.21.15.22:0/1175763941 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7f00ff3c0 0x7ff7f0195eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:49.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:49.461+0000 7ff7f514b700 1 --2- 172.21.15.22:0/1175763941 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff7f00ffd60 0x7ff7f01963f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:49.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:49.461+0000 7ff7f514b700 1 -- 172.21.15.22:0/1175763941 >> 172.21.15.22:0/1175763941 conn(0x7ff7f00fac80 msgr2=0x7ff7f0068ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:25:49.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:49.461+0000 7ff7f514b700 1 -- 172.21.15.22:0/1175763941 shutdown_connections 2024-02-16T19:25:49.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:49.461+0000 7ff7f514b700 1 -- 172.21.15.22:0/1175763941 wait complete. 2024-02-16T19:25:50.061 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:25:49 smithi022 ceph-mon[29221]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:25:50.061 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:25:49 smithi022 ceph-mon[29221]: from='client.? 172.21.15.22:0/1175763941' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-16T19:25:50.080 INFO:tasks.cephadm.ceph_manager.ceph:need seq 150323855375 got 150323855378 for osd.4 2024-02-16T19:25:50.080 DEBUG:teuthology.parallel:result is None 2024-02-16T19:25:50.080 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-02-16T19:25:50.080 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -- ceph pg dump --format=json 2024-02-16T19:25:50.291 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi022/config 2024-02-16T19:25:50.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:25:49 smithi117 ceph-mon[35309]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:25:50.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:25:49 smithi117 ceph-mon[35309]: from='client.? 172.21.15.22:0/1175763941' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-16T19:25:51.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:51.914+0000 7ff204be0700 1 -- 172.21.15.22:0/2406454690 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2001004e0 msgr2=0x7ff2001008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:51.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:51.914+0000 7ff204be0700 1 --2- 172.21.15.22:0/2406454690 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2001004e0 0x7ff2001008b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff1f8009910 tx=0x7ff1f8009c20 comp rx=0 tx=0).stop 2024-02-16T19:25:51.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:51.914+0000 7ff204be0700 1 -- 172.21.15.22:0/2406454690 shutdown_connections 2024-02-16T19:25:51.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:51.914+0000 7ff204be0700 1 --2- 172.21.15.22:0/2406454690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff200100e80 0x7ff200104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:51.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:51.914+0000 7ff204be0700 1 --2- 172.21.15.22:0/2406454690 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2001004e0 0x7ff2001008b0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:51.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:51.914+0000 7ff204be0700 1 -- 172.21.15.22:0/2406454690 >> 172.21.15.22:0/2406454690 conn(0x7ff2000fbda0 msgr2=0x7ff2000fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:25:51.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:51.914+0000 7ff204be0700 1 -- 172.21.15.22:0/2406454690 shutdown_connections 2024-02-16T19:25:51.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:51.914+0000 7ff204be0700 1 -- 172.21.15.22:0/2406454690 wait complete. 2024-02-16T19:25:51.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:51.915+0000 7ff204be0700 1 Processor -- start 2024-02-16T19:25:51.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:51.915+0000 7ff204be0700 1 -- start start 2024-02-16T19:25:51.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:51.915+0000 7ff204be0700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2001004e0 0x7ff200073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:51.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:51.915+0000 7ff204be0700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff200100e80 0x7ff200073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:51.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:51.915+0000 7ff204be0700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff200073ad0 con 0x7ff200100e80 2024-02-16T19:25:51.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:51.915+0000 7ff204be0700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff200073c10 con 0x7ff2001004e0 2024-02-16T19:25:51.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:51.915+0000 7ff1fdd9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff200100e80 0x7ff200073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:51.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:51.915+0000 7ff1fdd9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff200100e80 0x7ff200073590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35834/0 (socket says 172.21.15.22:35834) 2024-02-16T19:25:51.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:51.915+0000 7ff1fdd9b700 1 -- 172.21.15.22:0/206075871 learned_addr learned my addr 172.21.15.22:0/206075871 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:25:51.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:51.915+0000 7ff1fe59c700 1 --2- 172.21.15.22:0/206075871 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2001004e0 0x7ff200073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:51.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:51.915+0000 7ff1fdd9b700 1 -- 172.21.15.22:0/206075871 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2001004e0 msgr2=0x7ff200073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:51.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:51.915+0000 7ff1fdd9b700 1 --2- 172.21.15.22:0/206075871 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2001004e0 0x7ff200073050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:51.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:51.915+0000 7ff1fdd9b700 1 -- 172.21.15.22:0/206075871 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1f80095f0 con 0x7ff200100e80 2024-02-16T19:25:51.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:51.916+0000 7ff1fdd9b700 1 --2- 172.21.15.22:0/206075871 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff200100e80 0x7ff200073590 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff1e800b480 tx=0x7ff1e800b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:51.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:51.916+0000 7ff1f77fe700 1 -- 172.21.15.22:0/206075871 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff1e8011800 con 0x7ff200100e80 2024-02-16T19:25:51.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:51.916+0000 7ff1f77fe700 1 -- 172.21.15.22:0/206075871 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff1e8011e10 con 0x7ff200100e80 2024-02-16T19:25:51.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:51.916+0000 7ff204be0700 1 -- 172.21.15.22:0/206075871 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2001a2140 con 0x7ff200100e80 2024-02-16T19:25:51.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:51.916+0000 7ff1f77fe700 1 -- 172.21.15.22:0/206075871 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff1e8010ea0 con 0x7ff200100e80 2024-02-16T19:25:51.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:51.916+0000 7ff204be0700 1 -- 172.21.15.22:0/206075871 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2001a2550 con 0x7ff200100e80 2024-02-16T19:25:51.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:51.917+0000 7ff204be0700 1 -- 172.21.15.22:0/206075871 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2001087f0 con 0x7ff200100e80 2024-02-16T19:25:51.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:51.917+0000 7ff1f77fe700 1 -- 172.21.15.22:0/206075871 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7ff1e8010380 con 0x7ff200100e80 2024-02-16T19:25:51.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:51.918+0000 7ff1f77fe700 1 --2- 172.21.15.22:0/206075871 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7ff1ec06c680 0x7ff1ec06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:51.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:51.918+0000 7ff1f77fe700 1 -- 172.21.15.22:0/206075871 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7ff1e808b0b0 con 0x7ff200100e80 2024-02-16T19:25:51.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:51.918+0000 7ff1fe59c700 1 --2- 172.21.15.22:0/206075871 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7ff1ec06c680 0x7ff1ec06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:51.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:51.918+0000 7ff1fe59c700 1 --2- 172.21.15.22:0/206075871 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7ff1ec06c680 0x7ff1ec06eb30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff1f8009fd0 tx=0x7ff1f801a040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:51.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:51.922+0000 7ff1f77fe700 1 -- 172.21.15.22:0/206075871 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7ff1e8055ad0 con 0x7ff200100e80 2024-02-16T19:25:52.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:52.118+0000 7ff204be0700 1 -- 172.21.15.22:0/206075871 --> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff200061190 con 0x7ff1ec06c680 2024-02-16T19:25:52.119 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:25:51 smithi022 ceph-mon[29221]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:25:52.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:52.121+0000 7ff1f77fe700 1 -- 172.21.15.22:0/206075871 <== mgr.14217 v2:172.21.15.22:6800/2253867279 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+27806 (secure 0 0 0) 0x7ff200061190 con 0x7ff1ec06c680 2024-02-16T19:25:52.121 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:25:52.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:52.123+0000 7ff204be0700 1 -- 172.21.15.22:0/206075871 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7ff1ec06c680 msgr2=0x7ff1ec06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:52.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:52.123+0000 7ff204be0700 1 --2- 172.21.15.22:0/206075871 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7ff1ec06c680 0x7ff1ec06eb30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff1f8009fd0 tx=0x7ff1f801a040 comp rx=0 tx=0).stop 2024-02-16T19:25:52.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:52.123+0000 7ff204be0700 1 -- 172.21.15.22:0/206075871 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff200100e80 msgr2=0x7ff200073590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:52.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:52.123+0000 7ff204be0700 1 --2- 172.21.15.22:0/206075871 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff200100e80 0x7ff200073590 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff1e800b480 tx=0x7ff1e800b790 comp rx=0 tx=0).stop 2024-02-16T19:25:52.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:52.123+0000 7ff204be0700 1 -- 172.21.15.22:0/206075871 shutdown_connections 2024-02-16T19:25:52.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:52.123+0000 7ff204be0700 1 --2- 172.21.15.22:0/206075871 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7ff1ec06c680 0x7ff1ec06eb30 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:52.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:52.123+0000 7ff204be0700 1 --2- 172.21.15.22:0/206075871 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2001004e0 0x7ff200073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:52.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:52.123+0000 7ff204be0700 1 --2- 172.21.15.22:0/206075871 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff200100e80 0x7ff200073590 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:52.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:52.123+0000 7ff204be0700 1 -- 172.21.15.22:0/206075871 >> 172.21.15.22:0/206075871 conn(0x7ff2000fbda0 msgr2=0x7ff2000fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:25:52.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:52.123+0000 7ff204be0700 1 -- 172.21.15.22:0/206075871 shutdown_connections 2024-02-16T19:25:52.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:52.123+0000 7ff204be0700 1 -- 172.21.15.22:0/206075871 wait complete. 2024-02-16T19:25:52.124 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-02-16T19:25:52.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:25:51 smithi117 ceph-mon[35309]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:25:52.720 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":161,"stamp":"2024-02-16T19:25:50.809923+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":165636,"kb_used_data":3924,"kb_used_omap":0,"kb_used_meta":161664,"kb_avail":562305276,"statfs":{"total":575970213888,"available":575800602624,"internally_reserved":0,"allocated":4018176,"data_stored":2627442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":165543936},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002206"},"pg_stats":[{"pgid":"1.0","version":"24'117","reported_seq":187,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-02-16T19:24:59.822602+0000","last_change":"2024-02-16T19:24:34.012127+0000","last_active":"2024-02-16T19:24:59.822602+0000","last_peered":"2024-02-16T19:24:59.822602+0000","last_clean":"2024-02-16T19:24:59.822602+0000","last_became_active":"2024-02-16T19:24:34.011862+0000","last_became_peered":"2024-02-16T19:24:34.011862+0000","last_unstale":"2024-02-16T19:24:59.822602+0000","last_undegraded":"2024-02-16T19:24:59.822602+0000","last_fullsized":"2024-02-16T19:24:59.822602+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-16T19:23:47.802583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-16T19:23:47.802583+0000","last_clean_scrub_stamp":"2024-02-16T19:23:47.802583+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T03:33:21.979600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":40,"seq":171798691853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27316,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717836,"statfs":{"total":95995035648,"available":95967064064,"internally_reserved":0,"allocated":372736,"data_stored":142723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41599999999999998}]}]},{"osd":4,"up_from":35,"seq":150323855379,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27316,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717836,"statfs":{"total":95995035648,"available":95967064064,"internally_reserved":0,"allocated":372736,"data_stored":142723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 16 19:25:37 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.36799999999999999}]},{"osd":1,"last update":"Fri Feb 16 19:25:37 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.45200000000000001}]},{"osd":2,"last update":"Fri Feb 16 19:25:37 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.436},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.51100000000000001}]},{"osd":3,"last update":"Fri Feb 16 19:25:37 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.495}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]}]},{"osd":3,"up_from":27,"seq":115964117016,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":733091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 16 19:25:11 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.56299999999999994}]},{"osd":1,"last update":"Fri Feb 16 19:25:11 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.57399999999999995}]},{"osd":2,"last update":"Fri Feb 16 19:25:11 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.60299999999999998}]},{"osd":4,"last update":"Fri Feb 16 19:25:36 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.48899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]}]},{"osd":2,"up_from":19,"seq":81604378652,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27316,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717836,"statfs":{"total":95995035648,"available":95967064064,"internally_reserved":0,"allocated":372736,"data_stored":142723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 16 19:24:51 2024","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.495,"5min":0.495,"15min":0.495},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.505,"5min":0.505,"15min":0.505},"last":0.32400000000000001}]},{"osd":1,"last update":"Fri Feb 16 19:24:51 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.376,"5min":0.376,"15min":0.376},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"last":0.309}]},{"osd":3,"last update":"Fri Feb 16 19:25:14 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.69299999999999995}]},{"osd":4,"last update":"Fri Feb 16 19:25:37 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.628}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]}]},{"osd":0,"up_from":9,"seq":38654705703,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":733091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Feb 16 19:25:23 2024","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.32700000000000001,"15min":0.32500000000000001},"min":{"1min":0.22700000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.438,"5min":0.48899999999999999,"15min":0.48899999999999999},"last":0.23000000000000001},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.34000000000000002,"15min":0.33700000000000002},"min":{"1min":0.245,"5min":0.224,"15min":0.224},"max":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"last":0.34200000000000003}]},{"osd":2,"last update":"Fri Feb 16 19:24:48 2024","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"last":0.29799999999999999}]},{"osd":3,"last update":"Fri Feb 16 19:25:12 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.66500000000000004}]},{"osd":4,"last update":"Fri Feb 16 19:25:35 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.53000000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]}]},{"osd":1,"up_from":14,"seq":60129542177,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":733091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 16 19:25:22 2024","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.32400000000000001,"15min":0.32000000000000001},"min":{"1min":0.255,"5min":0.192,"15min":0.192},"max":{"1min":0.47199999999999998,"5min":0.55600000000000005,"15min":0.55600000000000005},"last":0.26700000000000002},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.32300000000000001,"15min":0.315},"min":{"1min":0.23799999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.30299999999999999}]},{"osd":2,"last update":"Fri Feb 16 19:24:50 2024","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.372,"15min":0.372},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.497,"5min":0.497,"15min":0.497},"last":0.249},{"interface":"front","average":{"1min":0.38,"5min":0.38,"15min":0.38},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.29399999999999998}]},{"osd":3,"last update":"Fri Feb 16 19:25:14 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.54200000000000004}]},{"osd":4,"last update":"Fri Feb 16 19:25:36 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.53100000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-16T19:25:52.721 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -- ceph pg dump --format=json 2024-02-16T19:25:52.931 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi022/config 2024-02-16T19:25:53.997 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:25:53 smithi022 ceph-mon[29221]: from='client.14432 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-16T19:25:53.997 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:25:53 smithi022 ceph-mon[29221]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:25:54.366 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:25:53 smithi117 ceph-mon[35309]: from='client.14432 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-16T19:25:54.366 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:25:53 smithi117 ceph-mon[35309]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:25:54.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:54.552+0000 7f1c1a55d700 1 -- 172.21.15.22:0/1369863131 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1c140ff3c0 msgr2=0x7f1c140ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:54.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:54.552+0000 7f1c1a55d700 1 --2- 172.21.15.22:0/1369863131 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1c140ff3c0 0x7f1c140ff790 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc009910 tx=0x7f1bfc009c20 comp rx=0 tx=0).stop 2024-02-16T19:25:54.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:54.553+0000 7f1c1a55d700 1 -- 172.21.15.22:0/1369863131 shutdown_connections 2024-02-16T19:25:54.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:54.553+0000 7f1c1a55d700 1 --2- 172.21.15.22:0/1369863131 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c140ffd60 0x7f1c14103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:54.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:54.553+0000 7f1c1a55d700 1 --2- 172.21.15.22:0/1369863131 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1c140ff3c0 0x7f1c140ff790 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:54.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:54.553+0000 7f1c1a55d700 1 -- 172.21.15.22:0/1369863131 >> 172.21.15.22:0/1369863131 conn(0x7f1c140fac80 msgr2=0x7f1c140fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:25:54.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:54.553+0000 7f1c1a55d700 1 -- 172.21.15.22:0/1369863131 shutdown_connections 2024-02-16T19:25:54.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:54.553+0000 7f1c1a55d700 1 -- 172.21.15.22:0/1369863131 wait complete. 2024-02-16T19:25:54.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:54.553+0000 7f1c1a55d700 1 Processor -- start 2024-02-16T19:25:54.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:54.553+0000 7f1c1a55d700 1 -- start start 2024-02-16T19:25:54.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:54.554+0000 7f1c1a55d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c140ff3c0 0x7f1c14194d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:54.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:54.554+0000 7f1c1a55d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1c140ffd60 0x7f1c141952c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:54.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:54.554+0000 7f1c1a55d700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c141959a0 con 0x7f1c140ffd60 2024-02-16T19:25:54.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:54.554+0000 7f1c1a55d700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c14199730 con 0x7f1c140ff3c0 2024-02-16T19:25:54.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:54.554+0000 7f1c137fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1c140ffd60 0x7f1c141952c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:54.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:54.554+0000 7f1c137fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1c140ffd60 0x7f1c141952c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:56082/0 (socket says 172.21.15.22:56082) 2024-02-16T19:25:54.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:54.554+0000 7f1c137fe700 1 -- 172.21.15.22:0/4016052103 learned_addr learned my addr 172.21.15.22:0/4016052103 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:25:54.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:54.554+0000 7f1c13fff700 1 --2- 172.21.15.22:0/4016052103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c140ff3c0 0x7f1c14194d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:54.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:54.554+0000 7f1c137fe700 1 -- 172.21.15.22:0/4016052103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c140ff3c0 msgr2=0x7f1c14194d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:54.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:54.554+0000 7f1c137fe700 1 --2- 172.21.15.22:0/4016052103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c140ff3c0 0x7f1c14194d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:54.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:54.554+0000 7f1c137fe700 1 -- 172.21.15.22:0/4016052103 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bfc0095f0 con 0x7f1c140ffd60 2024-02-16T19:25:54.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:54.555+0000 7f1c137fe700 1 --2- 172.21.15.22:0/4016052103 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1c140ffd60 0x7f1c141952c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1c040098f0 tx=0x7f1c04009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:54.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:54.555+0000 7f1c117fa700 1 -- 172.21.15.22:0/4016052103 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c04012070 con 0x7f1c140ffd60 2024-02-16T19:25:54.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:54.555+0000 7f1c117fa700 1 -- 172.21.15.22:0/4016052103 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f1c0400d1f0 con 0x7f1c140ffd60 2024-02-16T19:25:54.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:54.555+0000 7f1c117fa700 1 -- 172.21.15.22:0/4016052103 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c0400c8f0 con 0x7f1c140ffd60 2024-02-16T19:25:54.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:54.555+0000 7f1c1a55d700 1 -- 172.21.15.22:0/4016052103 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c141999b0 con 0x7f1c140ffd60 2024-02-16T19:25:54.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:54.555+0000 7f1c1a55d700 1 -- 172.21.15.22:0/4016052103 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c14199f00 con 0x7f1c140ffd60 2024-02-16T19:25:54.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:54.556+0000 7f1c1a55d700 1 -- 172.21.15.22:0/4016052103 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c1404f2a0 con 0x7f1c140ffd60 2024-02-16T19:25:54.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:54.556+0000 7f1c117fa700 1 -- 172.21.15.22:0/4016052103 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f1c04016070 con 0x7f1c140ffd60 2024-02-16T19:25:54.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:54.557+0000 7f1c117fa700 1 --2- 172.21.15.22:0/4016052103 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f1c0006c680 0x7f1c0006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:54.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:54.557+0000 7f1c117fa700 1 -- 172.21.15.22:0/4016052103 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f1c0408c660 con 0x7f1c140ffd60 2024-02-16T19:25:54.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:54.557+0000 7f1c13fff700 1 --2- 172.21.15.22:0/4016052103 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f1c0006c680 0x7f1c0006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:54.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:54.557+0000 7f1c13fff700 1 --2- 172.21.15.22:0/4016052103 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f1c0006c680 0x7f1c0006eb30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc00b5c0 tx=0x7f1bfc01a040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:54.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:54.561+0000 7f1c117fa700 1 -- 172.21.15.22:0/4016052103 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f1c04051720 con 0x7f1c140ffd60 2024-02-16T19:25:54.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:54.769+0000 7f1c1a55d700 1 -- 172.21.15.22:0/4016052103 --> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1c1419a2a0 con 0x7f1c0006c680 2024-02-16T19:25:54.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:54.770+0000 7f1c117fa700 1 -- 172.21.15.22:0/4016052103 <== mgr.14217 v2:172.21.15.22:6800/2253867279 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+27778 (secure 0 0 0) 0x7f1c1419a2a0 con 0x7f1c0006c680 2024-02-16T19:25:54.771 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:25:54.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:54.772+0000 7f1c1a55d700 1 -- 172.21.15.22:0/4016052103 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f1c0006c680 msgr2=0x7f1c0006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:54.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:54.772+0000 7f1c1a55d700 1 --2- 172.21.15.22:0/4016052103 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f1c0006c680 0x7f1c0006eb30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc00b5c0 tx=0x7f1bfc01a040 comp rx=0 tx=0).stop 2024-02-16T19:25:54.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:54.772+0000 7f1c1a55d700 1 -- 172.21.15.22:0/4016052103 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1c140ffd60 msgr2=0x7f1c141952c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:54.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:54.772+0000 7f1c1a55d700 1 --2- 172.21.15.22:0/4016052103 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1c140ffd60 0x7f1c141952c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1c040098f0 tx=0x7f1c04009c00 comp rx=0 tx=0).stop 2024-02-16T19:25:54.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:54.772+0000 7f1c1a55d700 1 -- 172.21.15.22:0/4016052103 shutdown_connections 2024-02-16T19:25:54.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:54.772+0000 7f1c1a55d700 1 --2- 172.21.15.22:0/4016052103 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f1c0006c680 0x7f1c0006eb30 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:54.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:54.772+0000 7f1c1a55d700 1 --2- 172.21.15.22:0/4016052103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c140ff3c0 0x7f1c14194d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:54.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:54.772+0000 7f1c1a55d700 1 --2- 172.21.15.22:0/4016052103 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1c140ffd60 0x7f1c141952c0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:54.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:54.772+0000 7f1c1a55d700 1 -- 172.21.15.22:0/4016052103 >> 172.21.15.22:0/4016052103 conn(0x7f1c140fac80 msgr2=0x7f1c140fc300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:25:54.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:54.772+0000 7f1c1a55d700 1 -- 172.21.15.22:0/4016052103 shutdown_connections 2024-02-16T19:25:54.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:54.772+0000 7f1c1a55d700 1 -- 172.21.15.22:0/4016052103 wait complete. 2024-02-16T19:25:54.773 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-02-16T19:25:55.340 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":162,"stamp":"2024-02-16T19:25:52.810243+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":165636,"kb_used_data":3924,"kb_used_omap":0,"kb_used_meta":161664,"kb_avail":562305276,"statfs":{"total":575970213888,"available":575800602624,"internally_reserved":0,"allocated":4018176,"data_stored":2627442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":165543936},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002101"},"pg_stats":[{"pgid":"1.0","version":"24'117","reported_seq":187,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-02-16T19:24:59.822602+0000","last_change":"2024-02-16T19:24:34.012127+0000","last_active":"2024-02-16T19:24:59.822602+0000","last_peered":"2024-02-16T19:24:59.822602+0000","last_clean":"2024-02-16T19:24:59.822602+0000","last_became_active":"2024-02-16T19:24:34.011862+0000","last_became_peered":"2024-02-16T19:24:34.011862+0000","last_unstale":"2024-02-16T19:24:59.822602+0000","last_undegraded":"2024-02-16T19:24:59.822602+0000","last_fullsized":"2024-02-16T19:24:59.822602+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-16T19:23:47.802583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-16T19:23:47.802583+0000","last_clean_scrub_stamp":"2024-02-16T19:23:47.802583+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T03:33:21.979600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":40,"seq":171798691853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27316,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717836,"statfs":{"total":95995035648,"available":95967064064,"internally_reserved":0,"allocated":372736,"data_stored":142723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41599999999999998}]}]},{"osd":4,"up_from":35,"seq":150323855379,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27316,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717836,"statfs":{"total":95995035648,"available":95967064064,"internally_reserved":0,"allocated":372736,"data_stored":142723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 16 19:25:37 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.36799999999999999}]},{"osd":1,"last update":"Fri Feb 16 19:25:37 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.45200000000000001}]},{"osd":2,"last update":"Fri Feb 16 19:25:37 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.436},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.51100000000000001}]},{"osd":3,"last update":"Fri Feb 16 19:25:37 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.495}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]}]},{"osd":3,"up_from":27,"seq":115964117016,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":733091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 16 19:25:11 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.56299999999999994}]},{"osd":1,"last update":"Fri Feb 16 19:25:11 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.57399999999999995}]},{"osd":2,"last update":"Fri Feb 16 19:25:11 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.60299999999999998}]},{"osd":4,"last update":"Fri Feb 16 19:25:36 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.48899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]}]},{"osd":2,"up_from":19,"seq":81604378653,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27316,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717836,"statfs":{"total":95995035648,"available":95967064064,"internally_reserved":0,"allocated":372736,"data_stored":142723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 16 19:24:51 2024","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.495,"5min":0.495,"15min":0.495},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.505,"5min":0.505,"15min":0.505},"last":0.48299999999999998}]},{"osd":1,"last update":"Fri Feb 16 19:24:51 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.376,"5min":0.376,"15min":0.376},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"last":0.437}]},{"osd":3,"last update":"Fri Feb 16 19:25:14 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.82099999999999995}]},{"osd":4,"last update":"Fri Feb 16 19:25:37 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.78000000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71099999999999997}]}]},{"osd":0,"up_from":9,"seq":38654705704,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":733091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Feb 16 19:25:23 2024","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.32700000000000001,"15min":0.32500000000000001},"min":{"1min":0.22700000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.438,"5min":0.48899999999999999,"15min":0.48899999999999999},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.34000000000000002,"15min":0.33700000000000002},"min":{"1min":0.245,"5min":0.224,"15min":0.224},"max":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"last":0.42799999999999999}]},{"osd":2,"last update":"Fri Feb 16 19:25:48 2024","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.48399999999999999,"5min":0.50900000000000001,"15min":0.50900000000000001},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.39100000000000001,"15min":0.39600000000000002},"min":{"1min":0.26700000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.504,"5min":0.504,"15min":0.504},"last":0.46700000000000003}]},{"osd":3,"last update":"Fri Feb 16 19:25:12 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.59299999999999997}]},{"osd":4,"last update":"Fri Feb 16 19:25:35 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.60199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]}]},{"osd":1,"up_from":14,"seq":60129542178,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":733091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 16 19:25:22 2024","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.32400000000000001,"15min":0.32000000000000001},"min":{"1min":0.255,"5min":0.192,"15min":0.192},"max":{"1min":0.47199999999999998,"5min":0.55600000000000005,"15min":0.55600000000000005},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.32300000000000001,"15min":0.315},"min":{"1min":0.23799999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.245}]},{"osd":2,"last update":"Fri Feb 16 19:24:50 2024","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.372,"15min":0.372},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.497,"5min":0.497,"15min":0.497},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.38,"5min":0.38,"15min":0.38},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.307}]},{"osd":3,"last update":"Fri Feb 16 19:25:14 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.56799999999999995}]},{"osd":4,"last update":"Fri Feb 16 19:25:36 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.49299999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-16T19:25:55.341 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-02-16T19:25:55.341 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-02-16T19:25:55.341 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-02-16T19:25:55.342 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -- ceph health --format=json 2024-02-16T19:25:55.551 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi022/config 2024-02-16T19:25:56.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:25:55 smithi022 ceph-mon[29221]: from='client.14436 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-16T19:25:56.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:25:55 smithi022 ceph-mon[29221]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:25:56.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:25:55 smithi117 ceph-mon[35309]: from='client.14436 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-16T19:25:56.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:25:55 smithi117 ceph-mon[35309]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:25:57.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:57.190+0000 7f30d75af700 1 -- 172.21.15.22:0/1531929232 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30d0104130 msgr2=0x7f30d0104500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:57.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:57.190+0000 7f30d75af700 1 --2- 172.21.15.22:0/1531929232 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30d0104130 0x7f30d0104500 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f30c8009910 tx=0x7f30c8009c20 comp rx=0 tx=0).stop 2024-02-16T19:25:57.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:57.190+0000 7f30d75af700 1 -- 172.21.15.22:0/1531929232 shutdown_connections 2024-02-16T19:25:57.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:57.190+0000 7f30d75af700 1 --2- 172.21.15.22:0/1531929232 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30d00ffbd0 0x7f30d0100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:57.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:57.190+0000 7f30d75af700 1 --2- 172.21.15.22:0/1531929232 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30d0104130 0x7f30d0104500 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:57.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:57.190+0000 7f30d75af700 1 -- 172.21.15.22:0/1531929232 >> 172.21.15.22:0/1531929232 conn(0x7f30d0075130 msgr2=0x7f30d0075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:25:57.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:57.190+0000 7f30d75af700 1 -- 172.21.15.22:0/1531929232 shutdown_connections 2024-02-16T19:25:57.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:57.190+0000 7f30d75af700 1 -- 172.21.15.22:0/1531929232 wait complete. 2024-02-16T19:25:57.192 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:25:56 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:25:57.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:57.191+0000 7f30d75af700 1 Processor -- start 2024-02-16T19:25:57.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:57.191+0000 7f30d75af700 1 -- start start 2024-02-16T19:25:57.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:57.191+0000 7f30d75af700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30d00ffbd0 0x7f30d0100e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:57.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:57.191+0000 7f30d75af700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30d0104130 0x7f30d0101370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:57.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:57.191+0000 7f30d75af700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d01018b0 con 0x7f30d0104130 2024-02-16T19:25:57.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:57.191+0000 7f30d75af700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d01019f0 con 0x7f30d00ffbd0 2024-02-16T19:25:57.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:57.191+0000 7f30d4b4a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30d0104130 0x7f30d0101370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:57.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:57.191+0000 7f30d4b4a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30d0104130 0x7f30d0101370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:56104/0 (socket says 172.21.15.22:56104) 2024-02-16T19:25:57.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:57.191+0000 7f30d534b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30d00ffbd0 0x7f30d0100e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:57.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:57.191+0000 7f30d4b4a700 1 -- 172.21.15.22:0/1366041314 learned_addr learned my addr 172.21.15.22:0/1366041314 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:25:57.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:57.191+0000 7f30d4b4a700 1 -- 172.21.15.22:0/1366041314 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30d00ffbd0 msgr2=0x7f30d0100e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:57.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:57.191+0000 7f30d4b4a700 1 --2- 172.21.15.22:0/1366041314 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30d00ffbd0 0x7f30d0100e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:57.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:57.191+0000 7f30d4b4a700 1 -- 172.21.15.22:0/1366041314 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30c80095f0 con 0x7f30d0104130 2024-02-16T19:25:57.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:57.192+0000 7f30d4b4a700 1 --2- 172.21.15.22:0/1366041314 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30d0104130 0x7f30d0101370 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f30c400b4b0 tx=0x7f30c400b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:57.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:57.192+0000 7f30ce7fc700 1 -- 172.21.15.22:0/1366041314 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f30c4011800 con 0x7f30d0104130 2024-02-16T19:25:57.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:57.192+0000 7f30ce7fc700 1 -- 172.21.15.22:0/1366041314 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f30c4011e10 con 0x7f30d0104130 2024-02-16T19:25:57.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:57.192+0000 7f30ce7fc700 1 -- 172.21.15.22:0/1366041314 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f30c400f2e0 con 0x7f30d0104130 2024-02-16T19:25:57.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:57.192+0000 7f30d75af700 1 -- 172.21.15.22:0/1366041314 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30d01a20a0 con 0x7f30d0104130 2024-02-16T19:25:57.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:57.192+0000 7f30d75af700 1 -- 172.21.15.22:0/1366041314 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30d01a24b0 con 0x7f30d0104130 2024-02-16T19:25:57.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:57.193+0000 7f30d75af700 1 -- 172.21.15.22:0/1366041314 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30d01086a0 con 0x7f30d0104130 2024-02-16T19:25:57.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:57.193+0000 7f30ce7fc700 1 -- 172.21.15.22:0/1366041314 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f30c4010380 con 0x7f30d0104130 2024-02-16T19:25:57.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:57.194+0000 7f30ce7fc700 1 --2- 172.21.15.22:0/1366041314 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f30b806c680 0x7f30b806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:57.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:57.194+0000 7f30ce7fc700 1 -- 172.21.15.22:0/1366041314 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f30c408aff0 con 0x7f30d0104130 2024-02-16T19:25:57.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:57.194+0000 7f30d534b700 1 --2- 172.21.15.22:0/1366041314 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f30b806c680 0x7f30b806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:57.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:57.194+0000 7f30d534b700 1 --2- 172.21.15.22:0/1366041314 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f30b806c680 0x7f30b806eb30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f30c8000c00 tx=0x7f30c801a040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:57.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:57.198+0000 7f30ce7fc700 1 -- 172.21.15.22:0/1366041314 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f30c4055a90 con 0x7f30d0104130 2024-02-16T19:25:57.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:25:56 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:25:57.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:57.445+0000 7f30d75af700 1 -- 172.21.15.22:0/1366041314 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f30d004f2a0 con 0x7f30d0104130 2024-02-16T19:25:57.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:57.446+0000 7f30ce7fc700 1 -- 172.21.15.22:0/1366041314 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f30c40590b0 con 0x7f30d0104130 2024-02-16T19:25:57.447 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:25:57.447 INFO:teuthology.orchestra.run.smithi022.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-02-16T19:25:57.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:57.448+0000 7f30d75af700 1 -- 172.21.15.22:0/1366041314 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f30b806c680 msgr2=0x7f30b806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:57.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:57.448+0000 7f30d75af700 1 --2- 172.21.15.22:0/1366041314 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f30b806c680 0x7f30b806eb30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f30c8000c00 tx=0x7f30c801a040 comp rx=0 tx=0).stop 2024-02-16T19:25:57.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:57.448+0000 7f30d75af700 1 -- 172.21.15.22:0/1366041314 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30d0104130 msgr2=0x7f30d0101370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:57.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:57.448+0000 7f30d75af700 1 --2- 172.21.15.22:0/1366041314 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30d0104130 0x7f30d0101370 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f30c400b4b0 tx=0x7f30c400b7c0 comp rx=0 tx=0).stop 2024-02-16T19:25:57.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:57.448+0000 7f30d75af700 1 -- 172.21.15.22:0/1366041314 shutdown_connections 2024-02-16T19:25:57.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:57.448+0000 7f30d75af700 1 --2- 172.21.15.22:0/1366041314 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f30b806c680 0x7f30b806eb30 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:57.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:57.448+0000 7f30d75af700 1 --2- 172.21.15.22:0/1366041314 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30d00ffbd0 0x7f30d0100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:57.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:57.448+0000 7f30d75af700 1 --2- 172.21.15.22:0/1366041314 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30d0104130 0x7f30d0101370 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:57.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:57.448+0000 7f30d75af700 1 -- 172.21.15.22:0/1366041314 >> 172.21.15.22:0/1366041314 conn(0x7f30d0075130 msgr2=0x7f30d00fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:25:57.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:57.448+0000 7f30d75af700 1 -- 172.21.15.22:0/1366041314 shutdown_connections 2024-02-16T19:25:57.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:25:57.448+0000 7f30d75af700 1 -- 172.21.15.22:0/1366041314 wait complete. 2024-02-16T19:25:58.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:25:57 smithi117 ceph-mon[35309]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:25:58.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:25:57 smithi117 ceph-mon[35309]: from='client.? 172.21.15.22:0/1366041314' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-16T19:25:58.181 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:25:57 smithi022 ceph-mon[29221]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:25:58.181 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:25:57 smithi022 ceph-mon[29221]: from='client.? 172.21.15.22:0/1366041314' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-16T19:25:58.206 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-02-16T19:25:58.206 INFO:tasks.cephadm:Setup complete, yielding 2024-02-16T19:25:58.206 INFO:teuthology.run_tasks:Running task print... 2024-02-16T19:25:58.215 INFO:teuthology.task.print:**** done end installing v18.2.0 cephadm ... 2024-02-16T19:25:58.215 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-16T19:25:58.236 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi022.front.sepia.ceph.com 2024-02-16T19:25:58.236 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -- bash -c 'ceph config set mgr mgr/cephadm/use_repo_digest true --force' 2024-02-16T19:25:58.453 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi022/config 2024-02-16T19:26:00.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:00.099+0000 7fedd7980700 1 -- 172.21.15.22:0/3138843515 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fedd0101440 msgr2=0x7fedd0101890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:00.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:00.099+0000 7fedd7980700 1 --2- 172.21.15.22:0/3138843515 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fedd0101440 0x7fedd0101890 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fedc0009910 tx=0x7fedc0009c20 comp rx=0 tx=0).stop 2024-02-16T19:26:00.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:00.099+0000 7fedd7980700 1 -- 172.21.15.22:0/3138843515 shutdown_connections 2024-02-16T19:26:00.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:00.099+0000 7fedd7980700 1 --2- 172.21.15.22:0/3138843515 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fedd0101440 0x7fedd0101890 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:00.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:00.099+0000 7fedd7980700 1 --2- 172.21.15.22:0/3138843515 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fedd0100240 0x7fedd0100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:00.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:00.099+0000 7fedd7980700 1 -- 172.21.15.22:0/3138843515 >> 172.21.15.22:0/3138843515 conn(0x7fedd00fb7f0 msgr2=0x7fedd00fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:26:00.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:00.099+0000 7fedd7980700 1 -- 172.21.15.22:0/3138843515 shutdown_connections 2024-02-16T19:26:00.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:00.099+0000 7fedd7980700 1 -- 172.21.15.22:0/3138843515 wait complete. 2024-02-16T19:26:00.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:00.099+0000 7fedd7980700 1 Processor -- start 2024-02-16T19:26:00.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:00.099+0000 7fedd7980700 1 -- start start 2024-02-16T19:26:00.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:00.100+0000 7fedd7980700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fedd0100240 0x7fedd0195b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:00.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:00.100+0000 7fedd7980700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fedd0101440 0x7fedd01960c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:00.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:00.100+0000 7fedd7980700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedd01966e0 con 0x7fedd0100240 2024-02-16T19:26:00.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:00.100+0000 7fedd7980700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedd0196820 con 0x7fedd0101440 2024-02-16T19:26:00.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:00.100+0000 7fedd571c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fedd0100240 0x7fedd0195b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:00.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:00.100+0000 7fedd571c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fedd0100240 0x7fedd0195b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:56114/0 (socket says 172.21.15.22:56114) 2024-02-16T19:26:00.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:00.100+0000 7fedd571c700 1 -- 172.21.15.22:0/2442302616 learned_addr learned my addr 172.21.15.22:0/2442302616 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:26:00.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:00.100+0000 7fedd4f1b700 1 --2- 172.21.15.22:0/2442302616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fedd0101440 0x7fedd01960c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:00.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:00.100+0000 7fedd571c700 1 -- 172.21.15.22:0/2442302616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fedd0101440 msgr2=0x7fedd01960c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:00.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:00.100+0000 7fedd571c700 1 --2- 172.21.15.22:0/2442302616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fedd0101440 0x7fedd01960c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:00.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:00.100+0000 7fedd571c700 1 -- 172.21.15.22:0/2442302616 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedc00095f0 con 0x7fedd0100240 2024-02-16T19:26:00.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:00.100+0000 7fedd4f1b700 1 --2- 172.21.15.22:0/2442302616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fedd0101440 0x7fedd01960c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T19:26:00.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:00.100+0000 7fedd571c700 1 --2- 172.21.15.22:0/2442302616 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fedd0100240 0x7fedd0195b80 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fedcc00b480 tx=0x7fedcc00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:26:00.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:00.100+0000 7fedc67fc700 1 -- 172.21.15.22:0/2442302616 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fedcc0118c0 con 0x7fedd0100240 2024-02-16T19:26:00.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:00.100+0000 7fedd7980700 1 -- 172.21.15.22:0/2442302616 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedd019b2d0 con 0x7fedd0100240 2024-02-16T19:26:00.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:00.100+0000 7fedc67fc700 1 -- 172.21.15.22:0/2442302616 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fedcc010380 con 0x7fedd0100240 2024-02-16T19:26:00.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:00.100+0000 7fedc67fc700 1 -- 172.21.15.22:0/2442302616 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fedcc00f4f0 con 0x7fedd0100240 2024-02-16T19:26:00.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:00.100+0000 7fedd7980700 1 -- 172.21.15.22:0/2442302616 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedd019b820 con 0x7fedd0100240 2024-02-16T19:26:00.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:00.101+0000 7fedd7980700 1 -- 172.21.15.22:0/2442302616 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedd0108e30 con 0x7fedd0100240 2024-02-16T19:26:00.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:00.102+0000 7fedc67fc700 1 -- 172.21.15.22:0/2442302616 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fedcc0104f0 con 0x7fedd0100240 2024-02-16T19:26:00.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:00.102+0000 7fedc67fc700 1 --2- 172.21.15.22:0/2442302616 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fedbc06c680 0x7fedbc06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:00.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:00.102+0000 7fedc67fc700 1 -- 172.21.15.22:0/2442302616 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fedcc08b190 con 0x7fedd0100240 2024-02-16T19:26:00.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:00.102+0000 7fedd4f1b700 1 --2- 172.21.15.22:0/2442302616 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fedbc06c680 0x7fedbc06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:00.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:00.103+0000 7fedd4f1b700 1 --2- 172.21.15.22:0/2442302616 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fedbc06c680 0x7fedbc06eb30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fedc0005b40 tx=0x7fedc0005ab0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:26:00.105 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:25:59 smithi022 ceph-mon[29221]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:26:00.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:00.111+0000 7fedc67fc700 1 -- 172.21.15.22:0/2442302616 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fedcc059640 con 0x7fedd0100240 2024-02-16T19:26:00.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:00.314+0000 7fedd7980700 1 -- 172.21.15.22:0/2442302616 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/use_repo_digest}] v 0) v1 -- 0x7fedd019bec0 con 0x7fedd0100240 2024-02-16T19:26:00.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:00.318+0000 7fedc67fc700 1 -- 172.21.15.22:0/2442302616 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/use_repo_digest}]=0 v20)=0 v20) v1 ==== 143+0+0 (secure 0 0 0) 0x7fedcc0591d0 con 0x7fedd0100240 2024-02-16T19:26:00.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:00.321+0000 7fedd7980700 1 -- 172.21.15.22:0/2442302616 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fedbc06c680 msgr2=0x7fedbc06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:00.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:00.321+0000 7fedd7980700 1 --2- 172.21.15.22:0/2442302616 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fedbc06c680 0x7fedbc06eb30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fedc0005b40 tx=0x7fedc0005ab0 comp rx=0 tx=0).stop 2024-02-16T19:26:00.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:00.321+0000 7fedd7980700 1 -- 172.21.15.22:0/2442302616 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fedd0100240 msgr2=0x7fedd0195b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:00.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:00.321+0000 7fedd7980700 1 --2- 172.21.15.22:0/2442302616 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fedd0100240 0x7fedd0195b80 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fedcc00b480 tx=0x7fedcc00b840 comp rx=0 tx=0).stop 2024-02-16T19:26:00.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:00.321+0000 7fedd7980700 1 -- 172.21.15.22:0/2442302616 shutdown_connections 2024-02-16T19:26:00.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:00.321+0000 7fedd7980700 1 --2- 172.21.15.22:0/2442302616 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fedd0100240 0x7fedd0195b80 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:00.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:00.321+0000 7fedd7980700 1 --2- 172.21.15.22:0/2442302616 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fedbc06c680 0x7fedbc06eb30 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:00.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:00.321+0000 7fedd7980700 1 --2- 172.21.15.22:0/2442302616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fedd0101440 0x7fedd01960c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:00.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:00.321+0000 7fedd7980700 1 -- 172.21.15.22:0/2442302616 >> 172.21.15.22:0/2442302616 conn(0x7fedd00fb7f0 msgr2=0x7fedd0104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:26:00.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:00.321+0000 7fedd7980700 1 -- 172.21.15.22:0/2442302616 shutdown_connections 2024-02-16T19:26:00.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:00.321+0000 7fedd7980700 1 -- 172.21.15.22:0/2442302616 wait complete. 2024-02-16T19:26:00.386 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:25:59 smithi117 ceph-mon[35309]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:26:00.915 INFO:teuthology.run_tasks:Running task print... 2024-02-16T19:26:00.927 INFO:teuthology.task.print:**** done cephadm.shell ceph config set mgr... 2024-02-16T19:26:00.927 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-16T19:26:00.936 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi022.front.sepia.ceph.com 2024-02-16T19:26:00.936 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -- bash -c 'ceph orch status' 2024-02-16T19:26:01.161 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi022/config 2024-02-16T19:26:01.381 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:01 smithi022 ceph-mon[29221]: from='client.? 172.21.15.22:0/2442302616' entity='client.admin' 2024-02-16T19:26:01.381 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:01 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:26:01.382 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:01 smithi022 ceph-mon[29221]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:26:01.382 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:01 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:26:01.382 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:01 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:26:01.382 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:01 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:26:01.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:01 smithi117 ceph-mon[35309]: from='client.? 172.21.15.22:0/2442302616' entity='client.admin' 2024-02-16T19:26:01.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:01 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:26:01.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:01 smithi117 ceph-mon[35309]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:26:01.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:01 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:26:01.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:01 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:26:01.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:01 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:26:02.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:02.731+0000 7f4cb294d700 1 -- 172.21.15.22:0/3749462258 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4cac1036a0 msgr2=0x7f4cac105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:02.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:02.731+0000 7f4cb294d700 1 --2- 172.21.15.22:0/3749462258 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4cac1036a0 0x7f4cac105a80 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c009910 tx=0x7f4c9c009c20 comp rx=0 tx=0).stop 2024-02-16T19:26:02.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:02.731+0000 7f4cb294d700 1 -- 172.21.15.22:0/3749462258 shutdown_connections 2024-02-16T19:26:02.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:02.731+0000 7f4cb294d700 1 --2- 172.21.15.22:0/3749462258 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4cac1036a0 0x7f4cac105a80 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:02.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:02.731+0000 7f4cb294d700 1 --2- 172.21.15.22:0/3749462258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cac100d80 0x7f4cac103160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:02.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:02.731+0000 7f4cb294d700 1 -- 172.21.15.22:0/3749462258 >> 172.21.15.22:0/3749462258 conn(0x7f4cac0fa7a0 msgr2=0x7f4cac0fcbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:26:02.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:02.731+0000 7f4cb294d700 1 -- 172.21.15.22:0/3749462258 shutdown_connections 2024-02-16T19:26:02.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:02.731+0000 7f4cb294d700 1 -- 172.21.15.22:0/3749462258 wait complete. 2024-02-16T19:26:02.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:02.732+0000 7f4cb294d700 1 Processor -- start 2024-02-16T19:26:02.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:02.732+0000 7f4cb294d700 1 -- start start 2024-02-16T19:26:02.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:02.732+0000 7f4cb294d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cac100d80 0x7f4cac195dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:02.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:02.732+0000 7f4cb294d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4cac1036a0 0x7f4cac196300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:02.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:02.732+0000 7f4cb294d700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cac196920 con 0x7f4cac1036a0 2024-02-16T19:26:02.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:02.732+0000 7f4cb294d700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cac196a60 con 0x7f4cac100d80 2024-02-16T19:26:02.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:02.732+0000 7f4cab7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4cac1036a0 0x7f4cac196300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:02.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:02.732+0000 7f4cab7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4cac1036a0 0x7f4cac196300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:56136/0 (socket says 172.21.15.22:56136) 2024-02-16T19:26:02.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:02.732+0000 7f4cab7fe700 1 -- 172.21.15.22:0/3107286303 learned_addr learned my addr 172.21.15.22:0/3107286303 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:26:02.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:02.732+0000 7f4cabfff700 1 --2- 172.21.15.22:0/3107286303 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cac100d80 0x7f4cac195dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:02.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:02.732+0000 7f4cab7fe700 1 -- 172.21.15.22:0/3107286303 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cac100d80 msgr2=0x7f4cac195dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:02.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:02.732+0000 7f4cab7fe700 1 --2- 172.21.15.22:0/3107286303 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cac100d80 0x7f4cac195dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:02.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:02.732+0000 7f4cab7fe700 1 -- 172.21.15.22:0/3107286303 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c9c0095f0 con 0x7f4cac1036a0 2024-02-16T19:26:02.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:02.733+0000 7f4cab7fe700 1 --2- 172.21.15.22:0/3107286303 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4cac1036a0 0x7f4cac196300 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c00bd70 tx=0x7f4c9c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:26:02.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:02.733+0000 7f4ca97fa700 1 -- 172.21.15.22:0/3107286303 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c9c01d070 con 0x7f4cac1036a0 2024-02-16T19:26:02.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:02.733+0000 7f4cb294d700 1 -- 172.21.15.22:0/3107286303 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cac19b4b0 con 0x7f4cac1036a0 2024-02-16T19:26:02.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:02.733+0000 7f4ca97fa700 1 -- 172.21.15.22:0/3107286303 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4c9c022470 con 0x7f4cac1036a0 2024-02-16T19:26:02.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:02.733+0000 7f4ca97fa700 1 -- 172.21.15.22:0/3107286303 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c9c00f670 con 0x7f4cac1036a0 2024-02-16T19:26:02.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:02.733+0000 7f4cb294d700 1 -- 172.21.15.22:0/3107286303 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cac19b9a0 con 0x7f4cac1036a0 2024-02-16T19:26:02.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:02.734+0000 7f4cb294d700 1 -- 172.21.15.22:0/3107286303 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cac18fc00 con 0x7f4cac1036a0 2024-02-16T19:26:02.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:02.734+0000 7f4ca97fa700 1 -- 172.21.15.22:0/3107286303 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f4c9c022a50 con 0x7f4cac1036a0 2024-02-16T19:26:02.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:02.734+0000 7f4ca97fa700 1 --2- 172.21.15.22:0/3107286303 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f4c9806c680 0x7f4c9806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:02.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:02.735+0000 7f4ca97fa700 1 -- 172.21.15.22:0/3107286303 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f4c9c08cb90 con 0x7f4cac1036a0 2024-02-16T19:26:02.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:02.735+0000 7f4cabfff700 1 --2- 172.21.15.22:0/3107286303 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f4c9806c680 0x7f4c9806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:02.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:02.735+0000 7f4cabfff700 1 --2- 172.21.15.22:0/3107286303 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f4c9806c680 0x7f4c9806eb30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4c94005fd0 tx=0x7f4c9400a560 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:26:02.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:02.739+0000 7f4ca97fa700 1 -- 172.21.15.22:0/3107286303 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f4c9c05b0f0 con 0x7f4cac1036a0 2024-02-16T19:26:02.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:02.953+0000 7f4cb294d700 1 -- 172.21.15.22:0/3107286303 --> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4cac0008d0 con 0x7f4c9806c680 2024-02-16T19:26:02.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:02.954+0000 7f4ca97fa700 1 -- 172.21.15.22:0/3107286303 <== mgr.14217 v2:172.21.15.22:6800/2253867279 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f4cac0008d0 con 0x7f4c9806c680 2024-02-16T19:26:02.955 INFO:teuthology.orchestra.run.smithi022.stdout:Backend: cephadm 2024-02-16T19:26:02.956 INFO:teuthology.orchestra.run.smithi022.stdout:Available: Yes 2024-02-16T19:26:02.956 INFO:teuthology.orchestra.run.smithi022.stdout:Paused: No 2024-02-16T19:26:02.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:02.956+0000 7f4cb294d700 1 -- 172.21.15.22:0/3107286303 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f4c9806c680 msgr2=0x7f4c9806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:02.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:02.956+0000 7f4cb294d700 1 --2- 172.21.15.22:0/3107286303 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f4c9806c680 0x7f4c9806eb30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4c94005fd0 tx=0x7f4c9400a560 comp rx=0 tx=0).stop 2024-02-16T19:26:02.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:02.956+0000 7f4cb294d700 1 -- 172.21.15.22:0/3107286303 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4cac1036a0 msgr2=0x7f4cac196300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:02.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:02.956+0000 7f4cb294d700 1 --2- 172.21.15.22:0/3107286303 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4cac1036a0 0x7f4cac196300 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c00bd70 tx=0x7f4c9c00be50 comp rx=0 tx=0).stop 2024-02-16T19:26:02.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:02.957+0000 7f4cb294d700 1 -- 172.21.15.22:0/3107286303 shutdown_connections 2024-02-16T19:26:02.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:02.957+0000 7f4cb294d700 1 --2- 172.21.15.22:0/3107286303 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f4c9806c680 0x7f4c9806eb30 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:02.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:02.957+0000 7f4cb294d700 1 --2- 172.21.15.22:0/3107286303 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cac100d80 0x7f4cac195dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:02.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:02.957+0000 7f4cb294d700 1 --2- 172.21.15.22:0/3107286303 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4cac1036a0 0x7f4cac196300 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:02.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:02.957+0000 7f4cb294d700 1 -- 172.21.15.22:0/3107286303 >> 172.21.15.22:0/3107286303 conn(0x7f4cac0fa7a0 msgr2=0x7f4cac0fcbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:26:02.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:02.957+0000 7f4cb294d700 1 -- 172.21.15.22:0/3107286303 shutdown_connections 2024-02-16T19:26:02.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:02.957+0000 7f4cb294d700 1 -- 172.21.15.22:0/3107286303 wait complete. 2024-02-16T19:26:03.606 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -- bash -c 'ceph orch ps' 2024-02-16T19:26:03.818 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi022/config 2024-02-16T19:26:04.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:03 smithi117 ceph-mon[35309]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:26:04.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:03 smithi117 ceph-mon[35309]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:26:04.143 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:03 smithi022 ceph-mon[29221]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:26:04.143 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:03 smithi022 ceph-mon[29221]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:26:05.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:05.387+0000 7fa1365e8700 1 -- 172.21.15.22:0/2265967386 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa130100240 msgr2=0x7fa130100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:05.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:05.387+0000 7fa1365e8700 1 --2- 172.21.15.22:0/2265967386 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa130100240 0x7fa130100650 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa120009910 tx=0x7fa120009c20 comp rx=0 tx=0).stop 2024-02-16T19:26:05.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:05.387+0000 7fa1365e8700 1 -- 172.21.15.22:0/2265967386 shutdown_connections 2024-02-16T19:26:05.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:05.387+0000 7fa1365e8700 1 --2- 172.21.15.22:0/2265967386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa130101440 0x7fa130101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:05.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:05.387+0000 7fa1365e8700 1 --2- 172.21.15.22:0/2265967386 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa130100240 0x7fa130100650 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:05.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:05.387+0000 7fa1365e8700 1 -- 172.21.15.22:0/2265967386 >> 172.21.15.22:0/2265967386 conn(0x7fa1300fb7f0 msgr2=0x7fa1300fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:26:05.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:05.387+0000 7fa1365e8700 1 -- 172.21.15.22:0/2265967386 shutdown_connections 2024-02-16T19:26:05.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:05.387+0000 7fa1365e8700 1 -- 172.21.15.22:0/2265967386 wait complete. 2024-02-16T19:26:05.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:05.387+0000 7fa1365e8700 1 Processor -- start 2024-02-16T19:26:05.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:05.387+0000 7fa1365e8700 1 -- start start 2024-02-16T19:26:05.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:05.388+0000 7fa1365e8700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa130100240 0x7fa130195b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:05.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:05.388+0000 7fa1365e8700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa130101440 0x7fa1301960a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:05.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:05.388+0000 7fa1365e8700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1301966c0 con 0x7fa130100240 2024-02-16T19:26:05.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:05.388+0000 7fa1365e8700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa130196800 con 0x7fa130101440 2024-02-16T19:26:05.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:05.388+0000 7fa12ffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa130100240 0x7fa130195b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:05.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:05.388+0000 7fa12ffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa130100240 0x7fa130195b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35530/0 (socket says 172.21.15.22:35530) 2024-02-16T19:26:05.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:05.388+0000 7fa12ffff700 1 -- 172.21.15.22:0/955318597 learned_addr learned my addr 172.21.15.22:0/955318597 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:26:05.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:05.388+0000 7fa12ffff700 1 -- 172.21.15.22:0/955318597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa130101440 msgr2=0x7fa1301960a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:05.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:05.388+0000 7fa12f7fe700 1 --2- 172.21.15.22:0/955318597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa130101440 0x7fa1301960a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:05.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:05.388+0000 7fa12ffff700 1 --2- 172.21.15.22:0/955318597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa130101440 0x7fa1301960a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:05.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:05.388+0000 7fa12ffff700 1 -- 172.21.15.22:0/955318597 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1200095f0 con 0x7fa130100240 2024-02-16T19:26:05.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:05.388+0000 7fa12f7fe700 1 --2- 172.21.15.22:0/955318597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa130101440 0x7fa1301960a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-16T19:26:05.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:05.388+0000 7fa12ffff700 1 --2- 172.21.15.22:0/955318597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa130100240 0x7fa130195b60 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa12000bda0 tx=0x7fa12000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:26:05.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:05.388+0000 7fa12d7fa700 1 -- 172.21.15.22:0/955318597 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa12001d070 con 0x7fa130100240 2024-02-16T19:26:05.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:05.388+0000 7fa1365e8700 1 -- 172.21.15.22:0/955318597 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa13019b250 con 0x7fa130100240 2024-02-16T19:26:05.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:05.388+0000 7fa12d7fa700 1 -- 172.21.15.22:0/955318597 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa120004c50 con 0x7fa130100240 2024-02-16T19:26:05.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:05.388+0000 7fa12d7fa700 1 -- 172.21.15.22:0/955318597 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa12000f690 con 0x7fa130100240 2024-02-16T19:26:05.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:05.388+0000 7fa1365e8700 1 -- 172.21.15.22:0/955318597 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa13019b740 con 0x7fa130100240 2024-02-16T19:26:05.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:05.389+0000 7fa1365e8700 1 -- 172.21.15.22:0/955318597 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa130108e10 con 0x7fa130100240 2024-02-16T19:26:05.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:05.390+0000 7fa12d7fa700 1 -- 172.21.15.22:0/955318597 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fa12000f900 con 0x7fa130100240 2024-02-16T19:26:05.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:05.390+0000 7fa12d7fa700 1 --2- 172.21.15.22:0/955318597 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fa11806c680 0x7fa11806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:05.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:05.391+0000 7fa12d7fa700 1 -- 172.21.15.22:0/955318597 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fa12008cd50 con 0x7fa130100240 2024-02-16T19:26:05.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:05.391+0000 7fa12f7fe700 1 --2- 172.21.15.22:0/955318597 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fa11806c680 0x7fa11806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:05.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:05.391+0000 7fa12f7fe700 1 --2- 172.21.15.22:0/955318597 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fa11806c680 0x7fa11806eb30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa124007900 tx=0x7fa124008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:26:05.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:05.395+0000 7fa12d7fa700 1 -- 172.21.15.22:0/955318597 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fa12005b200 con 0x7fa130100240 2024-02-16T19:26:05.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:05.603+0000 7fa1365e8700 1 -- 172.21.15.22:0/955318597 --> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa130105d90 con 0x7fa11806c680 2024-02-16T19:26:05.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:05.612+0000 7fa12d7fa700 1 -- 172.21.15.22:0/955318597 <== mgr.14217 v2:172.21.15.22:6800/2253867279 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+2820 (secure 0 0 0) 0x7fa130105d90 con 0x7fa11806c680 2024-02-16T19:26:05.613 INFO:teuthology.orchestra.run.smithi022.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T19:26:05.613 INFO:teuthology.orchestra.run.smithi022.stdout:alertmanager.smithi022 smithi022 *:9093,9094 running (3m) 2m ago 5m 26.4M - 0.25.0 c8568f914cd2 4dac585d8372 2024-02-16T19:26:05.613 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-exporter.smithi022 smithi022 running (5m) 2m ago 5m 7042k - 18.2.0 dc2bc1663786 3f8bef4349ba 2024-02-16T19:26:05.613 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-exporter.smithi117 smithi117 running (4m) 67s ago 4m 6941k - 18.2.0 dc2bc1663786 59b65a7e309f 2024-02-16T19:26:05.613 INFO:teuthology.orchestra.run.smithi022.stdout:crash.smithi022 smithi022 running (5m) 2m ago 5m 6669k - 18.2.0 dc2bc1663786 efef476aa486 2024-02-16T19:26:05.613 INFO:teuthology.orchestra.run.smithi022.stdout:crash.smithi117 smithi117 running (4m) 67s ago 4m 6660k - 18.2.0 dc2bc1663786 5466453eb149 2024-02-16T19:26:05.613 INFO:teuthology.orchestra.run.smithi022.stdout:grafana.smithi022 smithi022 *:3000 running (3m) 2m ago 5m 78.2M - 9.4.7 954c08fa6188 a48872ecbbf2 2024-02-16T19:26:05.613 INFO:teuthology.orchestra.run.smithi022.stdout:mgr.smithi022.gmdcch smithi022 *:9283,8765,8443 running (6m) 2m ago 6m 479M - 18.2.0 dc2bc1663786 e8435b394d04 2024-02-16T19:26:05.613 INFO:teuthology.orchestra.run.smithi022.stdout:mgr.smithi117.cohbtv smithi117 *:8443,9283,8765 running (4m) 67s ago 4m 437M - 18.2.0 dc2bc1663786 ddc46b07de2f 2024-02-16T19:26:05.614 INFO:teuthology.orchestra.run.smithi022.stdout:mon.smithi022 smithi022 running (6m) 2m ago 6m 45.1M 2048M 18.2.0 dc2bc1663786 b4ffc3032e12 2024-02-16T19:26:05.614 INFO:teuthology.orchestra.run.smithi022.stdout:mon.smithi117 smithi117 running (4m) 67s ago 4m 40.7M 2048M 18.2.0 dc2bc1663786 aff3e21a73a0 2024-02-16T19:26:05.614 INFO:teuthology.orchestra.run.smithi022.stdout:node-exporter.smithi022 smithi022 *:9100 running (5m) 2m ago 5m 15.7M - 1.5.0 0da6a335fe13 d727f51df007 2024-02-16T19:26:05.614 INFO:teuthology.orchestra.run.smithi022.stdout:node-exporter.smithi117 smithi117 *:9100 running (4m) 67s ago 4m 17.3M - 1.5.0 0da6a335fe13 fc8b2df5a29d 2024-02-16T19:26:05.614 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0 smithi022 running (3m) 2m ago 3m 40.2M 3948M 18.2.0 dc2bc1663786 0027f86f16ce 2024-02-16T19:26:05.614 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1 smithi022 running (2m) 2m ago 2m 43.7M 3948M 18.2.0 dc2bc1663786 87300382b8a7 2024-02-16T19:26:05.614 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2 smithi022 running (2m) 2m ago 2m 40.5M 3948M 18.2.0 dc2bc1663786 be8a631b4f93 2024-02-16T19:26:05.614 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3 smithi117 running (2m) 67s ago 2m 44.2M 4970M 18.2.0 dc2bc1663786 9c0ce9a981cf 2024-02-16T19:26:05.614 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4 smithi117 running (98s) 67s ago 98s 39.2M 4970M 18.2.0 dc2bc1663786 1779c3496f4d 2024-02-16T19:26:05.614 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5 smithi117 running (73s) 67s ago 72s 10.8M 4970M 18.2.0 dc2bc1663786 fefaabca0638 2024-02-16T19:26:05.614 INFO:teuthology.orchestra.run.smithi022.stdout:prometheus.smithi022 smithi022 *:9095 running (3m) 2m ago 4m 42.5M - 2.43.0 a07b618ecd1d a1a5b92b5ef7 2024-02-16T19:26:05.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:05.614+0000 7fa1365e8700 1 -- 172.21.15.22:0/955318597 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fa11806c680 msgr2=0x7fa11806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:05.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:05.614+0000 7fa1365e8700 1 --2- 172.21.15.22:0/955318597 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fa11806c680 0x7fa11806eb30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa124007900 tx=0x7fa124008040 comp rx=0 tx=0).stop 2024-02-16T19:26:05.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:05.614+0000 7fa1365e8700 1 -- 172.21.15.22:0/955318597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa130100240 msgr2=0x7fa130195b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:05.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:05.614+0000 7fa1365e8700 1 --2- 172.21.15.22:0/955318597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa130100240 0x7fa130195b60 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa12000bda0 tx=0x7fa12000be80 comp rx=0 tx=0).stop 2024-02-16T19:26:05.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:05.614+0000 7fa1365e8700 1 -- 172.21.15.22:0/955318597 shutdown_connections 2024-02-16T19:26:05.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:05.614+0000 7fa1365e8700 1 --2- 172.21.15.22:0/955318597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa130100240 0x7fa130195b60 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:05.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:05.614+0000 7fa1365e8700 1 --2- 172.21.15.22:0/955318597 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fa11806c680 0x7fa11806eb30 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:05.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:05.614+0000 7fa1365e8700 1 --2- 172.21.15.22:0/955318597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa130101440 0x7fa1301960a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:05.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:05.614+0000 7fa1365e8700 1 -- 172.21.15.22:0/955318597 >> 172.21.15.22:0/955318597 conn(0x7fa1300fb7f0 msgr2=0x7fa130104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:26:05.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:05.614+0000 7fa1365e8700 1 -- 172.21.15.22:0/955318597 shutdown_connections 2024-02-16T19:26:05.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:05.614+0000 7fa1365e8700 1 -- 172.21.15.22:0/955318597 wait complete. 2024-02-16T19:26:05.964 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:05 smithi022 ceph-mon[29221]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:26:06.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:05 smithi117 ceph-mon[35309]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:26:06.192 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -- bash -c 'ceph orch ls' 2024-02-16T19:26:06.399 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi022/config 2024-02-16T19:26:07.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:06 smithi117 ceph-mon[35309]: from='client.14452 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:26:07.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:06 smithi022 ceph-mon[29221]: from='client.14452 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:26:08.013 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:07 smithi022 ceph-mon[29221]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:26:08.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:08.035+0000 7f4b88910700 1 -- 172.21.15.22:0/3512444175 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4b84100240 msgr2=0x7f4b84100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:08.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:08.035+0000 7f4b88910700 1 --2- 172.21.15.22:0/3512444175 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4b84100240 0x7f4b84100650 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c009910 tx=0x7f4b7c009c20 comp rx=0 tx=0).stop 2024-02-16T19:26:08.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:08.035+0000 7f4b88910700 1 -- 172.21.15.22:0/3512444175 shutdown_connections 2024-02-16T19:26:08.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:08.035+0000 7f4b88910700 1 --2- 172.21.15.22:0/3512444175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b84101440 0x7f4b84101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:08.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:08.035+0000 7f4b88910700 1 --2- 172.21.15.22:0/3512444175 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4b84100240 0x7f4b84100650 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:08.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:08.035+0000 7f4b88910700 1 -- 172.21.15.22:0/3512444175 >> 172.21.15.22:0/3512444175 conn(0x7f4b840fb7f0 msgr2=0x7f4b840fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:26:08.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:08.036+0000 7f4b88910700 1 -- 172.21.15.22:0/3512444175 shutdown_connections 2024-02-16T19:26:08.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:08.036+0000 7f4b88910700 1 -- 172.21.15.22:0/3512444175 wait complete. 2024-02-16T19:26:08.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:08.036+0000 7f4b88910700 1 Processor -- start 2024-02-16T19:26:08.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:08.036+0000 7f4b88910700 1 -- start start 2024-02-16T19:26:08.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:08.036+0000 7f4b88910700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b84100240 0x7f4b84074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:08.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:08.036+0000 7f4b88910700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4b84101440 0x7f4b84073140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:08.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:08.036+0000 7f4b88910700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b84073680 con 0x7f4b84101440 2024-02-16T19:26:08.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:08.036+0000 7f4b88910700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b840737c0 con 0x7f4b84100240 2024-02-16T19:26:08.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:08.037+0000 7f4b81d9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4b84101440 0x7f4b84073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:08.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:08.037+0000 7f4b81d9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4b84101440 0x7f4b84073140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35556/0 (socket says 172.21.15.22:35556) 2024-02-16T19:26:08.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:08.037+0000 7f4b81d9b700 1 -- 172.21.15.22:0/448529539 learned_addr learned my addr 172.21.15.22:0/448529539 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:26:08.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:08.037+0000 7f4b8259c700 1 --2- 172.21.15.22:0/448529539 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b84100240 0x7f4b84074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:08.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:08.037+0000 7f4b81d9b700 1 -- 172.21.15.22:0/448529539 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b84100240 msgr2=0x7f4b84074af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:08.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:08.037+0000 7f4b81d9b700 1 --2- 172.21.15.22:0/448529539 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b84100240 0x7f4b84074af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:08.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:08.037+0000 7f4b81d9b700 1 -- 172.21.15.22:0/448529539 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b7c0095f0 con 0x7f4b84101440 2024-02-16T19:26:08.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:08.037+0000 7f4b81d9b700 1 --2- 172.21.15.22:0/448529539 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4b84101440 0x7f4b84073140 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c00b4b0 tx=0x7f4b6c00b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:26:08.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:08.037+0000 7f4b7b7fe700 1 -- 172.21.15.22:0/448529539 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b6c011800 con 0x7f4b84101440 2024-02-16T19:26:08.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:08.037+0000 7f4b7b7fe700 1 -- 172.21.15.22:0/448529539 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4b6c011e10 con 0x7f4b84101440 2024-02-16T19:26:08.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:08.037+0000 7f4b88910700 1 -- 172.21.15.22:0/448529539 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b84073aa0 con 0x7f4b84101440 2024-02-16T19:26:08.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:08.037+0000 7f4b7b7fe700 1 -- 172.21.15.22:0/448529539 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b6c00f2e0 con 0x7f4b84101440 2024-02-16T19:26:08.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:08.037+0000 7f4b88910700 1 -- 172.21.15.22:0/448529539 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b8419dc50 con 0x7f4b84101440 2024-02-16T19:26:08.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:08.038+0000 7f4b88910700 1 -- 172.21.15.22:0/448529539 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b84108e10 con 0x7f4b84101440 2024-02-16T19:26:08.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:08.039+0000 7f4b7b7fe700 1 -- 172.21.15.22:0/448529539 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f4b6c010380 con 0x7f4b84101440 2024-02-16T19:26:08.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:08.039+0000 7f4b7b7fe700 1 --2- 172.21.15.22:0/448529539 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f4b7006c680 0x7f4b7006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:08.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:08.040+0000 7f4b7b7fe700 1 -- 172.21.15.22:0/448529539 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f4b6c08b110 con 0x7f4b84101440 2024-02-16T19:26:08.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:08.040+0000 7f4b8259c700 1 --2- 172.21.15.22:0/448529539 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f4b7006c680 0x7f4b7006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:08.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:08.040+0000 7f4b8259c700 1 --2- 172.21.15.22:0/448529539 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f4b7006c680 0x7f4b7006eb30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c005090 tx=0x7f4b7c0058e0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:26:08.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:08.044+0000 7f4b7b7fe700 1 -- 172.21.15.22:0/448529539 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f4b6c0595c0 con 0x7f4b84101440 2024-02-16T19:26:08.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:07 smithi117 ceph-mon[35309]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:26:08.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:08.250+0000 7f4b88910700 1 -- 172.21.15.22:0/448529539 --> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f4b84105d90 con 0x7f4b7006c680 2024-02-16T19:26:08.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:08.254+0000 7f4b7b7fe700 1 -- 172.21.15.22:0/448529539 <== mgr.14217 v2:172.21.15.22:6800/2253867279 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+1300 (secure 0 0 0) 0x7f4b84105d90 con 0x7f4b7006c680 2024-02-16T19:26:08.256 INFO:teuthology.orchestra.run.smithi022.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-02-16T19:26:08.256 INFO:teuthology.orchestra.run.smithi022.stdout:alertmanager ?:9093,9094 1/1 2m ago 6m count:1 2024-02-16T19:26:08.256 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-exporter 2/2 2m ago 6m * 2024-02-16T19:26:08.256 INFO:teuthology.orchestra.run.smithi022.stdout:crash 2/2 2m ago 6m * 2024-02-16T19:26:08.256 INFO:teuthology.orchestra.run.smithi022.stdout:grafana ?:3000 1/1 2m ago 6m count:1 2024-02-16T19:26:08.256 INFO:teuthology.orchestra.run.smithi022.stdout:mgr 2/2 2m ago 6m count:2 2024-02-16T19:26:08.256 INFO:teuthology.orchestra.run.smithi022.stdout:mon 2/2 2m ago 5m smithi022:172.21.15.22=smithi022;smithi117:172.21.15.117=smithi117;count:2 2024-02-16T19:26:08.256 INFO:teuthology.orchestra.run.smithi022.stdout:node-exporter ?:9100 2/2 2m ago 6m * 2024-02-16T19:26:08.256 INFO:teuthology.orchestra.run.smithi022.stdout:osd 6 2m ago - 2024-02-16T19:26:08.256 INFO:teuthology.orchestra.run.smithi022.stdout:prometheus ?:9095 1/1 2m ago 6m count:1 2024-02-16T19:26:08.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:08.256+0000 7f4b88910700 1 -- 172.21.15.22:0/448529539 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f4b7006c680 msgr2=0x7f4b7006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:08.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:08.256+0000 7f4b88910700 1 --2- 172.21.15.22:0/448529539 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f4b7006c680 0x7f4b7006eb30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c005090 tx=0x7f4b7c0058e0 comp rx=0 tx=0).stop 2024-02-16T19:26:08.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:08.256+0000 7f4b88910700 1 -- 172.21.15.22:0/448529539 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4b84101440 msgr2=0x7f4b84073140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:08.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:08.256+0000 7f4b88910700 1 --2- 172.21.15.22:0/448529539 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4b84101440 0x7f4b84073140 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c00b4b0 tx=0x7f4b6c00b7c0 comp rx=0 tx=0).stop 2024-02-16T19:26:08.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:08.256+0000 7f4b88910700 1 -- 172.21.15.22:0/448529539 shutdown_connections 2024-02-16T19:26:08.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:08.256+0000 7f4b88910700 1 --2- 172.21.15.22:0/448529539 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f4b7006c680 0x7f4b7006eb30 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:08.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:08.256+0000 7f4b88910700 1 --2- 172.21.15.22:0/448529539 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b84100240 0x7f4b84074af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:08.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:08.256+0000 7f4b88910700 1 --2- 172.21.15.22:0/448529539 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4b84101440 0x7f4b84073140 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:08.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:08.256+0000 7f4b88910700 1 -- 172.21.15.22:0/448529539 >> 172.21.15.22:0/448529539 conn(0x7f4b840fb7f0 msgr2=0x7f4b84104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:26:08.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:08.257+0000 7f4b88910700 1 -- 172.21.15.22:0/448529539 shutdown_connections 2024-02-16T19:26:08.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:08.257+0000 7f4b88910700 1 -- 172.21.15.22:0/448529539 wait complete. 2024-02-16T19:26:08.890 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -- bash -c 'ceph orch host ls' 2024-02-16T19:26:09.098 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi022/config 2024-02-16T19:26:10.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:09 smithi117 ceph-mon[35309]: from='client.14456 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:26:10.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:09 smithi117 ceph-mon[35309]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:26:10.199 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:09 smithi022 ceph-mon[29221]: from='client.14456 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:26:10.199 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:09 smithi022 ceph-mon[29221]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:26:10.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:10.750+0000 7fc45d4da700 1 -- 172.21.15.22:0/4166444089 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc458100240 msgr2=0x7fc458100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:10.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:10.750+0000 7fc45d4da700 1 --2- 172.21.15.22:0/4166444089 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc458100240 0x7fc458100650 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc448009910 tx=0x7fc448009c20 comp rx=0 tx=0).stop 2024-02-16T19:26:10.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:10.750+0000 7fc45d4da700 1 -- 172.21.15.22:0/4166444089 shutdown_connections 2024-02-16T19:26:10.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:10.750+0000 7fc45d4da700 1 --2- 172.21.15.22:0/4166444089 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc458101440 0x7fc458101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:10.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:10.750+0000 7fc45d4da700 1 --2- 172.21.15.22:0/4166444089 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc458100240 0x7fc458100650 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:10.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:10.750+0000 7fc45d4da700 1 -- 172.21.15.22:0/4166444089 >> 172.21.15.22:0/4166444089 conn(0x7fc4580fb7f0 msgr2=0x7fc4580fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:26:10.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:10.750+0000 7fc45d4da700 1 -- 172.21.15.22:0/4166444089 shutdown_connections 2024-02-16T19:26:10.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:10.750+0000 7fc45d4da700 1 -- 172.21.15.22:0/4166444089 wait complete. 2024-02-16T19:26:10.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:10.750+0000 7fc45d4da700 1 Processor -- start 2024-02-16T19:26:10.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:10.751+0000 7fc45d4da700 1 -- start start 2024-02-16T19:26:10.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:10.751+0000 7fc45d4da700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc458100240 0x7fc458074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:10.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:10.751+0000 7fc45d4da700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc458101440 0x7fc458073140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:10.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:10.751+0000 7fc45d4da700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc458073680 con 0x7fc458101440 2024-02-16T19:26:10.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:10.751+0000 7fc45d4da700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4580737c0 con 0x7fc458100240 2024-02-16T19:26:10.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:10.751+0000 7fc4567fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc458101440 0x7fc458073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:10.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:10.751+0000 7fc4567fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc458101440 0x7fc458073140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35566/0 (socket says 172.21.15.22:35566) 2024-02-16T19:26:10.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:10.751+0000 7fc4567fc700 1 -- 172.21.15.22:0/736235988 learned_addr learned my addr 172.21.15.22:0/736235988 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:26:10.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:10.751+0000 7fc456ffd700 1 --2- 172.21.15.22:0/736235988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc458100240 0x7fc458074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:10.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:10.751+0000 7fc4567fc700 1 -- 172.21.15.22:0/736235988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc458100240 msgr2=0x7fc458074af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:10.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:10.751+0000 7fc4567fc700 1 --2- 172.21.15.22:0/736235988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc458100240 0x7fc458074af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:10.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:10.751+0000 7fc4567fc700 1 -- 172.21.15.22:0/736235988 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4480095f0 con 0x7fc458101440 2024-02-16T19:26:10.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:10.751+0000 7fc456ffd700 1 --2- 172.21.15.22:0/736235988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc458100240 0x7fc458074af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-16T19:26:10.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:10.751+0000 7fc4567fc700 1 --2- 172.21.15.22:0/736235988 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc458101440 0x7fc458073140 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc44c00d8d0 tx=0x7fc44c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:26:10.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:10.752+0000 7fc43ffff700 1 -- 172.21.15.22:0/736235988 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc44c009940 con 0x7fc458101440 2024-02-16T19:26:10.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:10.752+0000 7fc43ffff700 1 -- 172.21.15.22:0/736235988 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc44c010460 con 0x7fc458101440 2024-02-16T19:26:10.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:10.752+0000 7fc45d4da700 1 -- 172.21.15.22:0/736235988 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc458073aa0 con 0x7fc458101440 2024-02-16T19:26:10.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:10.752+0000 7fc43ffff700 1 -- 172.21.15.22:0/736235988 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc44c00f5f0 con 0x7fc458101440 2024-02-16T19:26:10.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:10.752+0000 7fc45d4da700 1 -- 172.21.15.22:0/736235988 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc45819db30 con 0x7fc458101440 2024-02-16T19:26:10.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:10.753+0000 7fc45d4da700 1 -- 172.21.15.22:0/736235988 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc458108d90 con 0x7fc458101440 2024-02-16T19:26:10.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:10.753+0000 7fc43ffff700 1 -- 172.21.15.22:0/736235988 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fc44c009aa0 con 0x7fc458101440 2024-02-16T19:26:10.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:10.753+0000 7fc43ffff700 1 --2- 172.21.15.22:0/736235988 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fc44006c680 0x7fc44006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:10.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:10.754+0000 7fc43ffff700 1 -- 172.21.15.22:0/736235988 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fc44c08b560 con 0x7fc458101440 2024-02-16T19:26:10.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:10.754+0000 7fc456ffd700 1 --2- 172.21.15.22:0/736235988 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fc44006c680 0x7fc44006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:10.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:10.754+0000 7fc456ffd700 1 --2- 172.21.15.22:0/736235988 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fc44006c680 0x7fc44006eb30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc44800b5c0 tx=0x7fc448005e20 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:26:10.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:10.758+0000 7fc43ffff700 1 -- 172.21.15.22:0/736235988 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fc44c059a10 con 0x7fc458101440 2024-02-16T19:26:10.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:10.964+0000 7fc45d4da700 1 -- 172.21.15.22:0/736235988 --> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fc458105d90 con 0x7fc44006c680 2024-02-16T19:26:10.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:10.966+0000 7fc43ffff700 1 -- 172.21.15.22:0/736235988 <== mgr.14217 v2:172.21.15.22:6800/2253867279 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+148 (secure 0 0 0) 0x7fc458105d90 con 0x7fc44006c680 2024-02-16T19:26:10.967 INFO:teuthology.orchestra.run.smithi022.stdout:HOST ADDR LABELS STATUS 2024-02-16T19:26:10.967 INFO:teuthology.orchestra.run.smithi022.stdout:smithi022 172.21.15.22 2024-02-16T19:26:10.967 INFO:teuthology.orchestra.run.smithi022.stdout:smithi117 172.21.15.117 2024-02-16T19:26:10.967 INFO:teuthology.orchestra.run.smithi022.stdout:2 hosts in cluster 2024-02-16T19:26:10.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:10.968+0000 7fc45d4da700 1 -- 172.21.15.22:0/736235988 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fc44006c680 msgr2=0x7fc44006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:10.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:10.968+0000 7fc45d4da700 1 --2- 172.21.15.22:0/736235988 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fc44006c680 0x7fc44006eb30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc44800b5c0 tx=0x7fc448005e20 comp rx=0 tx=0).stop 2024-02-16T19:26:10.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:10.968+0000 7fc45d4da700 1 -- 172.21.15.22:0/736235988 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc458101440 msgr2=0x7fc458073140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:10.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:10.968+0000 7fc45d4da700 1 --2- 172.21.15.22:0/736235988 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc458101440 0x7fc458073140 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc44c00d8d0 tx=0x7fc44c00dc90 comp rx=0 tx=0).stop 2024-02-16T19:26:10.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:10.968+0000 7fc45d4da700 1 -- 172.21.15.22:0/736235988 shutdown_connections 2024-02-16T19:26:10.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:10.968+0000 7fc45d4da700 1 --2- 172.21.15.22:0/736235988 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fc44006c680 0x7fc44006eb30 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:10.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:10.968+0000 7fc45d4da700 1 --2- 172.21.15.22:0/736235988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc458100240 0x7fc458074af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:10.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:10.968+0000 7fc45d4da700 1 --2- 172.21.15.22:0/736235988 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc458101440 0x7fc458073140 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:10.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:10.968+0000 7fc45d4da700 1 -- 172.21.15.22:0/736235988 >> 172.21.15.22:0/736235988 conn(0x7fc4580fb7f0 msgr2=0x7fc458104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:26:10.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:10.968+0000 7fc45d4da700 1 -- 172.21.15.22:0/736235988 shutdown_connections 2024-02-16T19:26:10.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:10.968+0000 7fc45d4da700 1 -- 172.21.15.22:0/736235988 wait complete. 2024-02-16T19:26:11.576 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -- bash -c 'ceph orch device ls' 2024-02-16T19:26:11.785 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi022/config 2024-02-16T19:26:12.133 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:11 smithi022 ceph-mon[29221]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:26:12.133 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:11 smithi022 ceph-mon[29221]: from='client.14460 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:26:12.133 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:11 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:26:12.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:11 smithi117 ceph-mon[35309]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:26:12.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:11 smithi117 ceph-mon[35309]: from='client.14460 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:26:12.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:11 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:26:13.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:13.396+0000 7fe8a6ba9700 1 -- 172.21.15.22:0/1747940360 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe8a0101440 msgr2=0x7fe8a0101890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:13.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:13.396+0000 7fe8a6ba9700 1 --2- 172.21.15.22:0/1747940360 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe8a0101440 0x7fe8a0101890 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe88c009910 tx=0x7fe88c009c20 comp rx=0 tx=0).stop 2024-02-16T19:26:13.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:13.396+0000 7fe8a6ba9700 1 -- 172.21.15.22:0/1747940360 shutdown_connections 2024-02-16T19:26:13.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:13.396+0000 7fe8a6ba9700 1 --2- 172.21.15.22:0/1747940360 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe8a0101440 0x7fe8a0101890 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:13.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:13.396+0000 7fe8a6ba9700 1 --2- 172.21.15.22:0/1747940360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8a0100240 0x7fe8a0100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:13.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:13.396+0000 7fe8a6ba9700 1 -- 172.21.15.22:0/1747940360 >> 172.21.15.22:0/1747940360 conn(0x7fe8a00fb7f0 msgr2=0x7fe8a00fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:26:13.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:13.396+0000 7fe8a6ba9700 1 -- 172.21.15.22:0/1747940360 shutdown_connections 2024-02-16T19:26:13.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:13.396+0000 7fe8a6ba9700 1 -- 172.21.15.22:0/1747940360 wait complete. 2024-02-16T19:26:13.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:13.396+0000 7fe8a6ba9700 1 Processor -- start 2024-02-16T19:26:13.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:13.396+0000 7fe8a6ba9700 1 -- start start 2024-02-16T19:26:13.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:13.397+0000 7fe8a6ba9700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe8a0100240 0x7fe8a0074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:13.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:13.397+0000 7fe8a6ba9700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8a0101440 0x7fe8a0073140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:13.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:13.397+0000 7fe8a6ba9700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a0073680 con 0x7fe8a0100240 2024-02-16T19:26:13.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:13.397+0000 7fe8a6ba9700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a00737c0 con 0x7fe8a0101440 2024-02-16T19:26:13.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:13.397+0000 7fe8a4945700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe8a0100240 0x7fe8a0074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:13.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:13.397+0000 7fe8a4945700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe8a0100240 0x7fe8a0074af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35574/0 (socket says 172.21.15.22:35574) 2024-02-16T19:26:13.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:13.397+0000 7fe8a4945700 1 -- 172.21.15.22:0/1205645719 learned_addr learned my addr 172.21.15.22:0/1205645719 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:26:13.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:13.397+0000 7fe89ffff700 1 --2- 172.21.15.22:0/1205645719 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8a0101440 0x7fe8a0073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:13.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:13.397+0000 7fe8a4945700 1 -- 172.21.15.22:0/1205645719 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8a0101440 msgr2=0x7fe8a0073140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:13.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:13.397+0000 7fe8a4945700 1 --2- 172.21.15.22:0/1205645719 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8a0101440 0x7fe8a0073140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:13.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:13.397+0000 7fe8a4945700 1 -- 172.21.15.22:0/1205645719 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe88c0095f0 con 0x7fe8a0100240 2024-02-16T19:26:13.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:13.397+0000 7fe8a4945700 1 --2- 172.21.15.22:0/1205645719 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe8a0100240 0x7fe8a0074af0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe89400c8f0 tx=0x7fe89400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:26:13.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:13.398+0000 7fe89dffb700 1 -- 172.21.15.22:0/1205645719 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe894007a70 con 0x7fe8a0100240 2024-02-16T19:26:13.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:13.398+0000 7fe8a6ba9700 1 -- 172.21.15.22:0/1205645719 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8a0073aa0 con 0x7fe8a0100240 2024-02-16T19:26:13.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:13.398+0000 7fe89dffb700 1 -- 172.21.15.22:0/1205645719 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe894007bd0 con 0x7fe8a0100240 2024-02-16T19:26:13.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:13.398+0000 7fe89dffb700 1 -- 172.21.15.22:0/1205645719 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe894018630 con 0x7fe8a0100240 2024-02-16T19:26:13.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:13.398+0000 7fe8a6ba9700 1 -- 172.21.15.22:0/1205645719 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8a019daf0 con 0x7fe8a0100240 2024-02-16T19:26:13.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:13.399+0000 7fe8a6ba9700 1 -- 172.21.15.22:0/1205645719 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8a0108d90 con 0x7fe8a0100240 2024-02-16T19:26:13.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:13.399+0000 7fe89dffb700 1 -- 172.21.15.22:0/1205645719 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fe89400f450 con 0x7fe8a0100240 2024-02-16T19:26:13.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:13.399+0000 7fe89dffb700 1 --2- 172.21.15.22:0/1205645719 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fe89006c680 0x7fe89006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:13.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:13.400+0000 7fe89dffb700 1 -- 172.21.15.22:0/1205645719 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fe89408b550 con 0x7fe8a0100240 2024-02-16T19:26:13.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:13.400+0000 7fe89ffff700 1 --2- 172.21.15.22:0/1205645719 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fe89006c680 0x7fe89006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:13.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:13.400+0000 7fe89ffff700 1 --2- 172.21.15.22:0/1205645719 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fe89006c680 0x7fe89006eb30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe88c00b5c0 tx=0x7fe88c005960 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:26:13.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:13.404+0000 7fe89dffb700 1 -- 172.21.15.22:0/1205645719 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fe894059ab0 con 0x7fe8a0100240 2024-02-16T19:26:13.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:13.608+0000 7fe8a6ba9700 1 -- 172.21.15.22:0/1205645719 --> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fe8a0105d90 con 0x7fe89006c680 2024-02-16T19:26:13.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:13.610+0000 7fe89dffb700 1 -- 172.21.15.22:0/1205645719 <== mgr.14217 v2:172.21.15.22:6800/2253867279 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+369 (secure 0 0 0) 0x7fe8a0105d90 con 0x7fe89006c680 2024-02-16T19:26:13.611 INFO:teuthology.orchestra.run.smithi022.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-02-16T19:26:13.611 INFO:teuthology.orchestra.run.smithi022.stdout:smithi022 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB747402EC480DGN 447G No 2m ago LVM detected, locked 2024-02-16T19:26:13.611 INFO:teuthology.orchestra.run.smithi022.stdout:smithi117 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400V6400BGN 372G No 72s ago LVM detected, locked 2024-02-16T19:26:13.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:13.612+0000 7fe8a6ba9700 1 -- 172.21.15.22:0/1205645719 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fe89006c680 msgr2=0x7fe89006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:13.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:13.612+0000 7fe8a6ba9700 1 --2- 172.21.15.22:0/1205645719 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fe89006c680 0x7fe89006eb30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe88c00b5c0 tx=0x7fe88c005960 comp rx=0 tx=0).stop 2024-02-16T19:26:13.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:13.612+0000 7fe8a6ba9700 1 -- 172.21.15.22:0/1205645719 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe8a0100240 msgr2=0x7fe8a0074af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:13.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:13.612+0000 7fe8a6ba9700 1 --2- 172.21.15.22:0/1205645719 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe8a0100240 0x7fe8a0074af0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe89400c8f0 tx=0x7fe89400ccb0 comp rx=0 tx=0).stop 2024-02-16T19:26:13.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:13.612+0000 7fe8a6ba9700 1 -- 172.21.15.22:0/1205645719 shutdown_connections 2024-02-16T19:26:13.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:13.612+0000 7fe8a6ba9700 1 --2- 172.21.15.22:0/1205645719 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe8a0100240 0x7fe8a0074af0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:13.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:13.612+0000 7fe8a6ba9700 1 --2- 172.21.15.22:0/1205645719 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fe89006c680 0x7fe89006eb30 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:13.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:13.612+0000 7fe8a6ba9700 1 --2- 172.21.15.22:0/1205645719 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8a0101440 0x7fe8a0073140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:13.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:13.612+0000 7fe8a6ba9700 1 -- 172.21.15.22:0/1205645719 >> 172.21.15.22:0/1205645719 conn(0x7fe8a00fb7f0 msgr2=0x7fe8a0104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:26:13.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:13.612+0000 7fe8a6ba9700 1 -- 172.21.15.22:0/1205645719 shutdown_connections 2024-02-16T19:26:13.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:13.612+0000 7fe8a6ba9700 1 -- 172.21.15.22:0/1205645719 wait complete. 2024-02-16T19:26:14.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:13 smithi117 ceph-mon[35309]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:26:14.209 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:13 smithi022 ceph-mon[29221]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:26:14.232 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-02-16T19:26:14.243 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-16T19:26:14.263 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi022.front.sepia.ceph.com 2024-02-16T19:26:14.263 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -- bash -c 'ceph fs volume create cephfs --placement=4' 2024-02-16T19:26:14.475 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi022/config 2024-02-16T19:26:15.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:14 smithi117 ceph-mon[35309]: from='client.14464 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:26:15.166 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:14 smithi022 ceph-mon[29221]: from='client.14464 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:26:16.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:16.049+0000 7f5e77892700 1 -- 172.21.15.22:0/3261235811 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5e70074d00 msgr2=0x7f5e70073160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:16.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:16.049+0000 7f5e77892700 1 --2- 172.21.15.22:0/3261235811 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5e70074d00 0x7f5e70073160 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5e68009910 tx=0x7f5e68009c20 comp rx=0 tx=0).stop 2024-02-16T19:26:16.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:16.049+0000 7f5e77892700 1 -- 172.21.15.22:0/3261235811 shutdown_connections 2024-02-16T19:26:16.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:16.049+0000 7f5e77892700 1 --2- 172.21.15.22:0/3261235811 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e70073730 0x7f5e70073ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:16.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:16.049+0000 7f5e77892700 1 --2- 172.21.15.22:0/3261235811 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5e70074d00 0x7f5e70073160 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:16.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:16.049+0000 7f5e77892700 1 -- 172.21.15.22:0/3261235811 >> 172.21.15.22:0/3261235811 conn(0x7f5e700fb7f0 msgr2=0x7f5e700fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:26:16.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:16.049+0000 7f5e77892700 1 -- 172.21.15.22:0/3261235811 shutdown_connections 2024-02-16T19:26:16.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:16.049+0000 7f5e77892700 1 -- 172.21.15.22:0/3261235811 wait complete. 2024-02-16T19:26:16.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:16.049+0000 7f5e77892700 1 Processor -- start 2024-02-16T19:26:16.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:16.049+0000 7f5e77892700 1 -- start start 2024-02-16T19:26:16.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:16.050+0000 7f5e77892700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e70073730 0x7f5e70197d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:16.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:16.050+0000 7f5e77892700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5e70074d00 0x7f5e70198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:16.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:16.050+0000 7f5e77892700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e70198890 con 0x7f5e70074d00 2024-02-16T19:26:16.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:16.050+0000 7f5e77892700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e701989d0 con 0x7f5e70073730 2024-02-16T19:26:16.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:16.050+0000 7f5e74e2d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5e70074d00 0x7f5e70198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:16.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:16.050+0000 7f5e74e2d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5e70074d00 0x7f5e70198270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50354/0 (socket says 172.21.15.22:50354) 2024-02-16T19:26:16.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:16.050+0000 7f5e74e2d700 1 -- 172.21.15.22:0/3559178456 learned_addr learned my addr 172.21.15.22:0/3559178456 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:26:16.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:16.050+0000 7f5e7562e700 1 --2- 172.21.15.22:0/3559178456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e70073730 0x7f5e70197d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:16.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:16.050+0000 7f5e74e2d700 1 -- 172.21.15.22:0/3559178456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e70073730 msgr2=0x7f5e70197d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:16.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:16.050+0000 7f5e74e2d700 1 --2- 172.21.15.22:0/3559178456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e70073730 0x7f5e70197d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:16.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:16.050+0000 7f5e74e2d700 1 -- 172.21.15.22:0/3559178456 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e680095f0 con 0x7f5e70074d00 2024-02-16T19:26:16.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:16.050+0000 7f5e74e2d700 1 --2- 172.21.15.22:0/3559178456 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5e70074d00 0x7f5e70198270 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5e6400b4b0 tx=0x7f5e6400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:26:16.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:16.050+0000 7f5e6e7fc700 1 -- 172.21.15.22:0/3559178456 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e64011800 con 0x7f5e70074d00 2024-02-16T19:26:16.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:16.050+0000 7f5e77892700 1 -- 172.21.15.22:0/3559178456 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e7019d480 con 0x7f5e70074d00 2024-02-16T19:26:16.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:16.050+0000 7f5e6e7fc700 1 -- 172.21.15.22:0/3559178456 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5e64011e10 con 0x7f5e70074d00 2024-02-16T19:26:16.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:16.050+0000 7f5e6e7fc700 1 -- 172.21.15.22:0/3559178456 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e6400f2e0 con 0x7f5e70074d00 2024-02-16T19:26:16.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:16.050+0000 7f5e77892700 1 -- 172.21.15.22:0/3559178456 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e7019d9d0 con 0x7f5e70074d00 2024-02-16T19:26:16.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:16.051+0000 7f5e77892700 1 -- 172.21.15.22:0/3559178456 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e7010aff0 con 0x7f5e70074d00 2024-02-16T19:26:16.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:16.052+0000 7f5e6e7fc700 1 -- 172.21.15.22:0/3559178456 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f5e64010380 con 0x7f5e70074d00 2024-02-16T19:26:16.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:16.052+0000 7f5e6e7fc700 1 --2- 172.21.15.22:0/3559178456 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f5e5806c680 0x7f5e5806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:16.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:16.052+0000 7f5e6e7fc700 1 -- 172.21.15.22:0/3559178456 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f5e6408b1d0 con 0x7f5e70074d00 2024-02-16T19:26:16.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:16.052+0000 7f5e7562e700 1 --2- 172.21.15.22:0/3559178456 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f5e5806c680 0x7f5e5806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:16.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:16.053+0000 7f5e7562e700 1 --2- 172.21.15.22:0/3559178456 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f5e5806c680 0x7f5e5806eb30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5e680059d0 tx=0x7f5e68005920 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:26:16.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:16.057+0000 7f5e6e7fc700 1 -- 172.21.15.22:0/3559178456 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f5e64059680 con 0x7f5e70074d00 2024-02-16T19:26:16.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:15 smithi117 ceph-mon[35309]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:26:16.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:15 smithi022 ceph-mon[29221]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:26:16.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:16.285+0000 7f5e77892700 1 -- 172.21.15.22:0/3559178456 --> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f5e701077c0 con 0x7f5e5806c680 2024-02-16T19:26:17.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:16 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-02-16T19:26:17.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:16 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-02-16T19:26:17.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:17.898+0000 7f5e6e7fc700 1 -- 172.21.15.22:0/3559178456 <== mgr.14217 v2:172.21.15.22:6800/2253867279 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f5e701077c0 con 0x7f5e5806c680 2024-02-16T19:26:17.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:17.900+0000 7f5e77892700 1 -- 172.21.15.22:0/3559178456 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f5e5806c680 msgr2=0x7f5e5806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:17.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:17.900+0000 7f5e77892700 1 --2- 172.21.15.22:0/3559178456 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f5e5806c680 0x7f5e5806eb30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5e680059d0 tx=0x7f5e68005920 comp rx=0 tx=0).stop 2024-02-16T19:26:17.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:17.900+0000 7f5e77892700 1 -- 172.21.15.22:0/3559178456 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5e70074d00 msgr2=0x7f5e70198270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:17.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:17.900+0000 7f5e77892700 1 --2- 172.21.15.22:0/3559178456 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5e70074d00 0x7f5e70198270 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5e6400b4b0 tx=0x7f5e6400b870 comp rx=0 tx=0).stop 2024-02-16T19:26:17.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:17.901+0000 7f5e77892700 1 -- 172.21.15.22:0/3559178456 shutdown_connections 2024-02-16T19:26:17.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:17.901+0000 7f5e77892700 1 --2- 172.21.15.22:0/3559178456 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f5e5806c680 0x7f5e5806eb30 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:17.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:17.901+0000 7f5e77892700 1 --2- 172.21.15.22:0/3559178456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e70073730 0x7f5e70197d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:17.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:17.901+0000 7f5e77892700 1 --2- 172.21.15.22:0/3559178456 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5e70074d00 0x7f5e70198270 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:17.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:17.901+0000 7f5e77892700 1 -- 172.21.15.22:0/3559178456 >> 172.21.15.22:0/3559178456 conn(0x7f5e700fb7f0 msgr2=0x7f5e701060a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:26:17.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:17.901+0000 7f5e77892700 1 -- 172.21.15.22:0/3559178456 shutdown_connections 2024-02-16T19:26:17.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:17.901+0000 7f5e77892700 1 -- 172.21.15.22:0/3559178456 wait complete. 2024-02-16T19:26:18.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:17 smithi117 ceph-mon[35309]: from='client.14468 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:26:18.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:17 smithi117 ceph-mon[35309]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:26:18.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:17 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-02-16T19:26:18.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:17 smithi117 ceph-mon[35309]: osdmap e42: 6 total, 6 up, 6 in 2024-02-16T19:26:18.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:17 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-02-16T19:26:18.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:17 smithi022 ceph-mon[29221]: from='client.14468 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:26:18.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:17 smithi022 ceph-mon[29221]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:26:18.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:17 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-02-16T19:26:18.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:17 smithi022 ceph-mon[29221]: osdmap e42: 6 total, 6 up, 6 in 2024-02-16T19:26:18.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:17 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-02-16T19:26:18.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:17 smithi022 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-mon-smithi022[29217]: 2024-02-16T19:26:17.872+0000 7f440052c700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-16T19:26:18.595 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -- bash -c 'ceph fs dump' 2024-02-16T19:26:19.126 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:18 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-02-16T19:26:19.127 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:18 smithi022 ceph-mon[29221]: osdmap e43: 6 total, 6 up, 6 in 2024-02-16T19:26:19.127 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:18 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-02-16T19:26:19.127 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:18 smithi022 ceph-mon[29221]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-16T19:26:19.127 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:18 smithi022 ceph-mon[29221]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-02-16T19:26:19.127 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:18 smithi022 ceph-mon[29221]: osdmap e44: 6 total, 6 up, 6 in 2024-02-16T19:26:19.127 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:18 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-02-16T19:26:19.127 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:18 smithi022 ceph-mon[29221]: fsmap cephfs:0 2024-02-16T19:26:19.127 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:18 smithi022 ceph-mon[29221]: Saving service mds.cephfs spec with placement count:4 2024-02-16T19:26:19.127 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:18 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:26:19.127 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:18 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:26:19.127 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:18 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:26:19.127 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:18 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:26:19.127 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:18 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:26:19.127 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:18 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi022.ocukvs", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T19:26:19.127 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:18 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi022.ocukvs", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-16T19:26:19.127 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:18 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:26:19.128 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:18 smithi022 ceph-mon[29221]: Deploying daemon mds.cephfs.smithi022.ocukvs on smithi022 2024-02-16T19:26:19.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:18 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-02-16T19:26:19.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:18 smithi117 ceph-mon[35309]: osdmap e43: 6 total, 6 up, 6 in 2024-02-16T19:26:19.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:18 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-02-16T19:26:19.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:18 smithi117 ceph-mon[35309]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-16T19:26:19.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:18 smithi117 ceph-mon[35309]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-02-16T19:26:19.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:18 smithi117 ceph-mon[35309]: osdmap e44: 6 total, 6 up, 6 in 2024-02-16T19:26:19.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:18 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-02-16T19:26:19.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:18 smithi117 ceph-mon[35309]: fsmap cephfs:0 2024-02-16T19:26:19.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:18 smithi117 ceph-mon[35309]: Saving service mds.cephfs spec with placement count:4 2024-02-16T19:26:19.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:18 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:26:19.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:18 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:26:19.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:18 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:26:19.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:18 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:26:19.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:18 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:26:19.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:18 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi022.ocukvs", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T19:26:19.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:18 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi022.ocukvs", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-16T19:26:19.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:18 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:26:19.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:18 smithi117 ceph-mon[35309]: Deploying daemon mds.cephfs.smithi022.ocukvs on smithi022 2024-02-16T19:26:19.432 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi022/config 2024-02-16T19:26:20.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:19 smithi117 ceph-mon[35309]: pgmap v178: 65 pgs: 10 creating+peering, 1 active+clean, 54 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:26:20.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:19 smithi117 ceph-mon[35309]: osdmap e45: 6 total, 6 up, 6 in 2024-02-16T19:26:20.141 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:19 smithi022 ceph-mon[29221]: pgmap v178: 65 pgs: 10 creating+peering, 1 active+clean, 54 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:26:20.141 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:19 smithi022 ceph-mon[29221]: osdmap e45: 6 total, 6 up, 6 in 2024-02-16T19:26:20.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:20.994+0000 7f38864eb700 1 -- 172.21.15.22:0/3739062866 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3880100240 msgr2=0x7f3880100690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:20.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:20.994+0000 7f38864eb700 1 --2- 172.21.15.22:0/3739062866 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3880100240 0x7f3880100690 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3874009910 tx=0x7f3874009c20 comp rx=0 tx=0).stop 2024-02-16T19:26:20.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:20.994+0000 7f38864eb700 1 -- 172.21.15.22:0/3739062866 shutdown_connections 2024-02-16T19:26:20.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:20.994+0000 7f38864eb700 1 --2- 172.21.15.22:0/3739062866 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3880100240 0x7f3880100690 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:20.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:20.994+0000 7f38864eb700 1 --2- 172.21.15.22:0/3739062866 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38800ff040 0x7f38800ff450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:20.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:20.994+0000 7f38864eb700 1 -- 172.21.15.22:0/3739062866 >> 172.21.15.22:0/3739062866 conn(0x7f38800fa610 msgr2=0x7f38800fca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:26:20.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:20.994+0000 7f38864eb700 1 -- 172.21.15.22:0/3739062866 shutdown_connections 2024-02-16T19:26:20.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:20.994+0000 7f38864eb700 1 -- 172.21.15.22:0/3739062866 wait complete. 2024-02-16T19:26:20.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:20.994+0000 7f38864eb700 1 Processor -- start 2024-02-16T19:26:20.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:20.994+0000 7f38864eb700 1 -- start start 2024-02-16T19:26:20.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:20.995+0000 7f38864eb700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f38800ff040 0x7f3880197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:20.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:20.995+0000 7f38864eb700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3880100240 0x7f3880198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:20.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:20.995+0000 7f38864eb700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3880198680 con 0x7f38800ff040 2024-02-16T19:26:20.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:20.995+0000 7f38864eb700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38801987c0 con 0x7f3880100240 2024-02-16T19:26:20.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:20.995+0000 7f387ffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f38800ff040 0x7f3880197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:20.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:20.995+0000 7f387ffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f38800ff040 0x7f3880197b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50368/0 (socket says 172.21.15.22:50368) 2024-02-16T19:26:20.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:20.995+0000 7f387ffff700 1 -- 172.21.15.22:0/2753454352 learned_addr learned my addr 172.21.15.22:0/2753454352 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:26:20.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:20.995+0000 7f387f7fe700 1 --2- 172.21.15.22:0/2753454352 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3880100240 0x7f3880198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:20.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:20.995+0000 7f387ffff700 1 -- 172.21.15.22:0/2753454352 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3880100240 msgr2=0x7f3880198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:20.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:20.995+0000 7f387ffff700 1 --2- 172.21.15.22:0/2753454352 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3880100240 0x7f3880198060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:20.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:20.995+0000 7f387ffff700 1 -- 172.21.15.22:0/2753454352 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38740095f0 con 0x7f38800ff040 2024-02-16T19:26:20.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:20.995+0000 7f387ffff700 1 --2- 172.21.15.22:0/2753454352 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f38800ff040 0x7f3880197b20 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f387000d900 tx=0x7f387000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:26:20.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:20.995+0000 7f387d7fa700 1 -- 172.21.15.22:0/2753454352 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f38700041f0 con 0x7f38800ff040 2024-02-16T19:26:20.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:20.995+0000 7f387d7fa700 1 -- 172.21.15.22:0/2753454352 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3870004350 con 0x7f38800ff040 2024-02-16T19:26:20.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:20.995+0000 7f387d7fa700 1 -- 172.21.15.22:0/2753454352 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3870003da0 con 0x7f38800ff040 2024-02-16T19:26:20.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:20.995+0000 7f38864eb700 1 -- 172.21.15.22:0/2753454352 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f388019d270 con 0x7f38800ff040 2024-02-16T19:26:20.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:20.996+0000 7f38864eb700 1 -- 172.21.15.22:0/2753454352 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f388019d7c0 con 0x7f38800ff040 2024-02-16T19:26:20.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:20.997+0000 7f38864eb700 1 -- 172.21.15.22:0/2753454352 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3880107b90 con 0x7f38800ff040 2024-02-16T19:26:20.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:20.997+0000 7f387d7fa700 1 -- 172.21.15.22:0/2753454352 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f3870009730 con 0x7f38800ff040 2024-02-16T19:26:20.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:20.997+0000 7f387d7fa700 1 --2- 172.21.15.22:0/2753454352 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f386806c680 0x7f386806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:20.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:20.997+0000 7f387d7fa700 1 -- 172.21.15.22:0/2753454352 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f387008ba50 con 0x7f38800ff040 2024-02-16T19:26:20.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:20.997+0000 7f387f7fe700 1 --2- 172.21.15.22:0/2753454352 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f386806c680 0x7f386806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:20.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:20.998+0000 7f387f7fe700 1 --2- 172.21.15.22:0/2753454352 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f386806c680 0x7f386806eb30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f387400b5c0 tx=0x7f3874005960 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:26:21.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:21.002+0000 7f387d7fa700 1 -- 172.21.15.22:0/2753454352 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f3870059bc0 con 0x7f38800ff040 2024-02-16T19:26:21.123 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:20 smithi022 ceph-mon[29221]: osdmap e46: 6 total, 6 up, 6 in 2024-02-16T19:26:21.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:21.245+0000 7f38864eb700 1 -- 172.21.15.22:0/2753454352 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f388019e300 con 0x7f38800ff040 2024-02-16T19:26:21.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:21.246+0000 7f387d7fa700 1 -- 172.21.15.22:0/2753454352 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 2 v2) v1 ==== 75+0+1093 (secure 0 0 0) 0x7f3870059750 con 0x7f38800ff040 2024-02-16T19:26:21.247 INFO:teuthology.orchestra.run.smithi022.stdout:e2 2024-02-16T19:26:21.247 INFO:teuthology.orchestra.run.smithi022.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T19:26:21.247 INFO:teuthology.orchestra.run.smithi022.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:26:21.247 INFO:teuthology.orchestra.run.smithi022.stdout:legacy client fscid: 1 2024-02-16T19:26:21.247 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:26:21.247 INFO:teuthology.orchestra.run.smithi022.stdout:Filesystem 'cephfs' (1) 2024-02-16T19:26:21.247 INFO:teuthology.orchestra.run.smithi022.stdout:fs_name cephfs 2024-02-16T19:26:21.247 INFO:teuthology.orchestra.run.smithi022.stdout:epoch 2 2024-02-16T19:26:21.248 INFO:teuthology.orchestra.run.smithi022.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T19:26:21.248 INFO:teuthology.orchestra.run.smithi022.stdout:created 2024-02-16T19:26:17.873644+0000 2024-02-16T19:26:21.248 INFO:teuthology.orchestra.run.smithi022.stdout:modified 2024-02-16T19:26:17.873844+0000 2024-02-16T19:26:21.248 INFO:teuthology.orchestra.run.smithi022.stdout:tableserver 0 2024-02-16T19:26:21.248 INFO:teuthology.orchestra.run.smithi022.stdout:root 0 2024-02-16T19:26:21.248 INFO:teuthology.orchestra.run.smithi022.stdout:session_timeout 60 2024-02-16T19:26:21.248 INFO:teuthology.orchestra.run.smithi022.stdout:session_autoclose 300 2024-02-16T19:26:21.248 INFO:teuthology.orchestra.run.smithi022.stdout:max_file_size 1099511627776 2024-02-16T19:26:21.248 INFO:teuthology.orchestra.run.smithi022.stdout:required_client_features {} 2024-02-16T19:26:21.248 INFO:teuthology.orchestra.run.smithi022.stdout:last_failure 0 2024-02-16T19:26:21.248 INFO:teuthology.orchestra.run.smithi022.stdout:last_failure_osd_epoch 0 2024-02-16T19:26:21.248 INFO:teuthology.orchestra.run.smithi022.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:26:21.248 INFO:teuthology.orchestra.run.smithi022.stdout:max_mds 1 2024-02-16T19:26:21.248 INFO:teuthology.orchestra.run.smithi022.stdout:in 2024-02-16T19:26:21.248 INFO:teuthology.orchestra.run.smithi022.stdout:up {} 2024-02-16T19:26:21.249 INFO:teuthology.orchestra.run.smithi022.stdout:failed 2024-02-16T19:26:21.249 INFO:teuthology.orchestra.run.smithi022.stdout:damaged 2024-02-16T19:26:21.249 INFO:teuthology.orchestra.run.smithi022.stdout:stopped 2024-02-16T19:26:21.249 INFO:teuthology.orchestra.run.smithi022.stdout:data_pools [3] 2024-02-16T19:26:21.249 INFO:teuthology.orchestra.run.smithi022.stdout:metadata_pool 2 2024-02-16T19:26:21.249 INFO:teuthology.orchestra.run.smithi022.stdout:inline_data disabled 2024-02-16T19:26:21.249 INFO:teuthology.orchestra.run.smithi022.stdout:balancer 2024-02-16T19:26:21.249 INFO:teuthology.orchestra.run.smithi022.stdout:bal_rank_mask -1 2024-02-16T19:26:21.249 INFO:teuthology.orchestra.run.smithi022.stdout:standby_count_wanted 0 2024-02-16T19:26:21.249 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:26:21.249 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:26:21.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:21.248+0000 7f38864eb700 1 -- 172.21.15.22:0/2753454352 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f386806c680 msgr2=0x7f386806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:21.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:21.248+0000 7f38864eb700 1 --2- 172.21.15.22:0/2753454352 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f386806c680 0x7f386806eb30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f387400b5c0 tx=0x7f3874005960 comp rx=0 tx=0).stop 2024-02-16T19:26:21.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:21.248+0000 7f38864eb700 1 -- 172.21.15.22:0/2753454352 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f38800ff040 msgr2=0x7f3880197b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:21.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:21.248+0000 7f38864eb700 1 --2- 172.21.15.22:0/2753454352 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f38800ff040 0x7f3880197b20 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f387000d900 tx=0x7f387000dcc0 comp rx=0 tx=0).stop 2024-02-16T19:26:21.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:21.248+0000 7f38864eb700 1 -- 172.21.15.22:0/2753454352 shutdown_connections 2024-02-16T19:26:21.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:21.248+0000 7f38864eb700 1 --2- 172.21.15.22:0/2753454352 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f38800ff040 0x7f3880197b20 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:21.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:21.248+0000 7f38864eb700 1 --2- 172.21.15.22:0/2753454352 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f386806c680 0x7f386806eb30 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:21.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:21.248+0000 7f38864eb700 1 --2- 172.21.15.22:0/2753454352 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3880100240 0x7f3880198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:21.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:21.248+0000 7f38864eb700 1 -- 172.21.15.22:0/2753454352 >> 172.21.15.22:0/2753454352 conn(0x7f38800fa610 msgr2=0x7f3880068dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:26:21.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:21.248+0000 7f38864eb700 1 -- 172.21.15.22:0/2753454352 shutdown_connections 2024-02-16T19:26:21.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:21.248+0000 7f38864eb700 1 -- 172.21.15.22:0/2753454352 wait complete. 2024-02-16T19:26:21.251 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 2 2024-02-16T19:26:21.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:20 smithi117 ceph-mon[35309]: osdmap e46: 6 total, 6 up, 6 in 2024-02-16T19:26:22.023 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-16T19:26:22.033 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi022.front.sepia.ceph.com 2024-02-16T19:26:22.033 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -- bash -c 'ceph fs set cephfs max_mds 1' 2024-02-16T19:26:22.059 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:21 smithi022 ceph-mon[29221]: pgmap v181: 65 pgs: 35 active+clean, 10 creating+peering, 20 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:26:22.059 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:21 smithi022 ceph-mon[29221]: from='client.? 172.21.15.22:0/2753454352' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:26:22.288 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:21 smithi117 ceph-mon[35309]: pgmap v181: 65 pgs: 35 active+clean, 10 creating+peering, 20 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:26:22.288 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:21 smithi117 ceph-mon[35309]: from='client.? 172.21.15.22:0/2753454352' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:26:22.684 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi022/config 2024-02-16T19:26:23.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:23 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:26:23.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:23 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:26:23.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:23 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:26:23.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:23 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi117.ibmppn", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T19:26:23.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:23 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi117.ibmppn", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-16T19:26:23.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:23 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:26:23.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:23 smithi022 ceph-mon[29221]: Deploying daemon mds.cephfs.smithi117.ibmppn on smithi117 2024-02-16T19:26:23.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:23 smithi022 ceph-mon[29221]: pgmap v182: 65 pgs: 65 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:26:23.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:23 smithi022 ceph-mon[29221]: mds.? [v2:172.21.15.22:6826/149959963,v1:172.21.15.22:6827/149959963] up:boot 2024-02-16T19:26:23.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:23 smithi022 ceph-mon[29221]: daemon mds.cephfs.smithi022.ocukvs assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-02-16T19:26:23.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:23 smithi022 ceph-mon[29221]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-16T19:26:23.716 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:23 smithi022 ceph-mon[29221]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-02-16T19:26:23.716 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:23 smithi022 ceph-mon[29221]: Cluster is now healthy 2024-02-16T19:26:23.716 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:23 smithi022 ceph-mon[29221]: fsmap cephfs:0 1 up:standby 2024-02-16T19:26:23.716 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:23 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi022.ocukvs"}]: dispatch 2024-02-16T19:26:23.716 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:23 smithi022 ceph-mon[29221]: fsmap cephfs:1 {0=cephfs.smithi022.ocukvs=up:creating} 2024-02-16T19:26:23.716 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:23 smithi022 ceph-mon[29221]: daemon mds.cephfs.smithi022.ocukvs is now active in filesystem cephfs as rank 0 2024-02-16T19:26:23.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:23 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:26:23.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:23 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:26:23.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:23 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:26:23.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:23 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi117.ibmppn", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T19:26:23.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:23 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi117.ibmppn", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-16T19:26:23.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:23 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:26:23.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:23 smithi117 ceph-mon[35309]: Deploying daemon mds.cephfs.smithi117.ibmppn on smithi117 2024-02-16T19:26:23.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:23 smithi117 ceph-mon[35309]: pgmap v182: 65 pgs: 65 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:26:23.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:23 smithi117 ceph-mon[35309]: mds.? [v2:172.21.15.22:6826/149959963,v1:172.21.15.22:6827/149959963] up:boot 2024-02-16T19:26:23.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:23 smithi117 ceph-mon[35309]: daemon mds.cephfs.smithi022.ocukvs assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-02-16T19:26:23.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:23 smithi117 ceph-mon[35309]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-16T19:26:23.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:23 smithi117 ceph-mon[35309]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-02-16T19:26:23.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:23 smithi117 ceph-mon[35309]: Cluster is now healthy 2024-02-16T19:26:23.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:23 smithi117 ceph-mon[35309]: fsmap cephfs:0 1 up:standby 2024-02-16T19:26:23.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:23 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi022.ocukvs"}]: dispatch 2024-02-16T19:26:23.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:23 smithi117 ceph-mon[35309]: fsmap cephfs:1 {0=cephfs.smithi022.ocukvs=up:creating} 2024-02-16T19:26:23.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:23 smithi117 ceph-mon[35309]: daemon mds.cephfs.smithi022.ocukvs is now active in filesystem cephfs as rank 0 2024-02-16T19:26:24.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:24.147+0000 7ff5b8fa0700 1 -- 172.21.15.22:0/3083413416 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5b4069b00 msgr2=0x7ff5b4102df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:24.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:24.147+0000 7ff5b8fa0700 1 --2- 172.21.15.22:0/3083413416 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5b4069b00 0x7ff5b4102df0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac00b0a0 tx=0x7ff5ac00b3b0 comp rx=0 tx=0).stop 2024-02-16T19:26:24.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:24.147+0000 7ff5b8fa0700 1 -- 172.21.15.22:0/3083413416 shutdown_connections 2024-02-16T19:26:24.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:24.147+0000 7ff5b8fa0700 1 --2- 172.21.15.22:0/3083413416 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff5b4103330 0x7ff5b41057b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:24.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:24.147+0000 7ff5b8fa0700 1 --2- 172.21.15.22:0/3083413416 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5b4069b00 0x7ff5b4102df0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:24.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:24.147+0000 7ff5b8fa0700 1 -- 172.21.15.22:0/3083413416 >> 172.21.15.22:0/3083413416 conn(0x7ff5b40fa7a0 msgr2=0x7ff5b40fcbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:26:24.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:24.147+0000 7ff5b8fa0700 1 -- 172.21.15.22:0/3083413416 shutdown_connections 2024-02-16T19:26:24.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:24.147+0000 7ff5b8fa0700 1 -- 172.21.15.22:0/3083413416 wait complete. 2024-02-16T19:26:24.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:24.148+0000 7ff5b8fa0700 1 Processor -- start 2024-02-16T19:26:24.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:24.148+0000 7ff5b8fa0700 1 -- start start 2024-02-16T19:26:24.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:24.148+0000 7ff5b8fa0700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5b4103330 0x7ff5b4100740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:24.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:24.148+0000 7ff5b8fa0700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff5b40fed90 0x7ff5b40ff200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:24.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:24.148+0000 7ff5b8fa0700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b40ff740 con 0x7ff5b4103330 2024-02-16T19:26:24.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:24.148+0000 7ff5b8fa0700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b40ff8b0 con 0x7ff5b40fed90 2024-02-16T19:26:24.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:24.148+0000 7ff5b259c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5b4103330 0x7ff5b4100740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:24.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:24.148+0000 7ff5b259c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5b4103330 0x7ff5b4100740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50406/0 (socket says 172.21.15.22:50406) 2024-02-16T19:26:24.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:24.148+0000 7ff5b259c700 1 -- 172.21.15.22:0/741975961 learned_addr learned my addr 172.21.15.22:0/741975961 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:26:24.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:24.148+0000 7ff5b1d9b700 1 --2- 172.21.15.22:0/741975961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff5b40fed90 0x7ff5b40ff200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:24.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:24.148+0000 7ff5b259c700 1 -- 172.21.15.22:0/741975961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff5b40fed90 msgr2=0x7ff5b40ff200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:24.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:24.148+0000 7ff5b259c700 1 --2- 172.21.15.22:0/741975961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff5b40fed90 0x7ff5b40ff200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:24.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:24.148+0000 7ff5b259c700 1 -- 172.21.15.22:0/741975961 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5ac009d00 con 0x7ff5b4103330 2024-02-16T19:26:24.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:24.148+0000 7ff5b259c700 1 --2- 172.21.15.22:0/741975961 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5b4103330 0x7ff5b4100740 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac0077b0 tx=0x7ff5ac003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:26:24.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:24.148+0000 7ff5ab7fe700 1 -- 172.21.15.22:0/741975961 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff5ac00e040 con 0x7ff5b4103330 2024-02-16T19:26:24.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:24.148+0000 7ff5ab7fe700 1 -- 172.21.15.22:0/741975961 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff5ac00ce60 con 0x7ff5b4103330 2024-02-16T19:26:24.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:24.148+0000 7ff5ab7fe700 1 -- 172.21.15.22:0/741975961 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff5ac01db10 con 0x7ff5b4103330 2024-02-16T19:26:24.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:24.148+0000 7ff5b8fa0700 1 -- 172.21.15.22:0/741975961 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5b40ffb30 con 0x7ff5b4103330 2024-02-16T19:26:24.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:24.149+0000 7ff5b8fa0700 1 -- 172.21.15.22:0/741975961 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5b419e150 con 0x7ff5b4103330 2024-02-16T19:26:24.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:24.149+0000 7ff5b8fa0700 1 -- 172.21.15.22:0/741975961 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5b418fc70 con 0x7ff5b4103330 2024-02-16T19:26:24.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:24.150+0000 7ff5ab7fe700 1 -- 172.21.15.22:0/741975961 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7ff5ac01dc90 con 0x7ff5b4103330 2024-02-16T19:26:24.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:24.150+0000 7ff5ab7fe700 1 --2- 172.21.15.22:0/741975961 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7ff5a006c680 0x7ff5a006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:24.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:24.151+0000 7ff5b1d9b700 1 --2- 172.21.15.22:0/741975961 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7ff5a006c680 0x7ff5a006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:24.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:24.151+0000 7ff5ab7fe700 1 -- 172.21.15.22:0/741975961 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7ff5ac08d330 con 0x7ff5b4103330 2024-02-16T19:26:24.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:24.151+0000 7ff5b1d9b700 1 --2- 172.21.15.22:0/741975961 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7ff5a006c680 0x7ff5a006eb30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff59c006fd0 tx=0x7ff59c008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:26:24.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:24.155+0000 7ff5ab7fe700 1 -- 172.21.15.22:0/741975961 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7ff5ac05b4a0 con 0x7ff5b4103330 2024-02-16T19:26:24.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:24.401+0000 7ff5b8fa0700 1 -- 172.21.15.22:0/741975961 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"} v 0) v1 -- 0x7ff5b4069b00 con 0x7ff5b4103330 2024-02-16T19:26:24.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:24.901+0000 7ff5ab7fe700 1 -- 172.21.15.22:0/741975961 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]=0 v6) v1 ==== 105+0+0 (secure 0 0 0) 0x7ff5ac05b030 con 0x7ff5b4103330 2024-02-16T19:26:24.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:24.903+0000 7ff5b8fa0700 1 -- 172.21.15.22:0/741975961 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7ff5a006c680 msgr2=0x7ff5a006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:24.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:24.903+0000 7ff5b8fa0700 1 --2- 172.21.15.22:0/741975961 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7ff5a006c680 0x7ff5a006eb30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff59c006fd0 tx=0x7ff59c008040 comp rx=0 tx=0).stop 2024-02-16T19:26:24.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:24.903+0000 7ff5b8fa0700 1 -- 172.21.15.22:0/741975961 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5b4103330 msgr2=0x7ff5b4100740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:24.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:24.903+0000 7ff5b8fa0700 1 --2- 172.21.15.22:0/741975961 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5b4103330 0x7ff5b4100740 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac0077b0 tx=0x7ff5ac003ce0 comp rx=0 tx=0).stop 2024-02-16T19:26:24.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:24.903+0000 7ff5b8fa0700 1 -- 172.21.15.22:0/741975961 shutdown_connections 2024-02-16T19:26:24.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:24.903+0000 7ff5b8fa0700 1 --2- 172.21.15.22:0/741975961 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5b4103330 0x7ff5b4100740 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:24.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:24.903+0000 7ff5b8fa0700 1 --2- 172.21.15.22:0/741975961 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7ff5a006c680 0x7ff5a006eb30 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:24.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:24.903+0000 7ff5b8fa0700 1 --2- 172.21.15.22:0/741975961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff5b40fed90 0x7ff5b40ff200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:24.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:24.903+0000 7ff5b8fa0700 1 -- 172.21.15.22:0/741975961 >> 172.21.15.22:0/741975961 conn(0x7ff5b40fa7a0 msgr2=0x7ff5b40fcbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:26:24.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:24.903+0000 7ff5b8fa0700 1 -- 172.21.15.22:0/741975961 shutdown_connections 2024-02-16T19:26:24.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:24.903+0000 7ff5b8fa0700 1 -- 172.21.15.22:0/741975961 wait complete. 2024-02-16T19:26:25.183 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:24 smithi117 ceph-mon[35309]: mds.? [v2:172.21.15.22:6826/149959963,v1:172.21.15.22:6827/149959963] up:active 2024-02-16T19:26:25.184 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:24 smithi117 ceph-mon[35309]: fsmap cephfs:1 {0=cephfs.smithi022.ocukvs=up:active} 2024-02-16T19:26:25.184 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:24 smithi117 ceph-mon[35309]: from='client.? 172.21.15.22:0/741975961' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]: dispatch 2024-02-16T19:26:25.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:24 smithi022 ceph-mon[29221]: mds.? [v2:172.21.15.22:6826/149959963,v1:172.21.15.22:6827/149959963] up:active 2024-02-16T19:26:25.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:24 smithi022 ceph-mon[29221]: fsmap cephfs:1 {0=cephfs.smithi022.ocukvs=up:active} 2024-02-16T19:26:25.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:24 smithi022 ceph-mon[29221]: from='client.? 172.21.15.22:0/741975961' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]: dispatch 2024-02-16T19:26:25.525 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-16T19:26:25.535 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi022.front.sepia.ceph.com 2024-02-16T19:26:25.536 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -- bash -c 'ceph fs set cephfs allow_standby_replay false' 2024-02-16T19:26:26.072 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:25 smithi022 ceph-mon[29221]: pgmap v183: 65 pgs: 65 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:26:26.072 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:25 smithi022 ceph-mon[29221]: from='client.? 172.21.15.22:0/741975961' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]': finished 2024-02-16T19:26:26.072 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:25 smithi022 ceph-mon[29221]: fsmap cephfs:1 {0=cephfs.smithi022.ocukvs=up:active} 2024-02-16T19:26:26.072 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:25 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:26:26.072 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:25 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:26:26.072 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:25 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:26:26.072 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:25 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi022.btppwm", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T19:26:26.072 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:25 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi022.btppwm", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-16T19:26:26.072 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:25 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:26:26.241 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:25 smithi117 ceph-mon[35309]: pgmap v183: 65 pgs: 65 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:26:26.241 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:25 smithi117 ceph-mon[35309]: from='client.? 172.21.15.22:0/741975961' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]': finished 2024-02-16T19:26:26.241 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:25 smithi117 ceph-mon[35309]: fsmap cephfs:1 {0=cephfs.smithi022.ocukvs=up:active} 2024-02-16T19:26:26.241 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:25 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:26:26.242 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:25 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:26:26.242 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:25 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:26:26.242 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:25 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi022.btppwm", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T19:26:26.242 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:25 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi022.btppwm", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-16T19:26:26.242 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:25 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:26:26.827 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi022/config 2024-02-16T19:26:26.953 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:26 smithi022 ceph-mon[29221]: Deploying daemon mds.cephfs.smithi022.btppwm on smithi022 2024-02-16T19:26:26.953 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:26 smithi022 ceph-mon[29221]: mds.? [v2:172.21.15.117:6824/565408102,v1:172.21.15.117:6825/565408102] up:boot 2024-02-16T19:26:26.953 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:26 smithi022 ceph-mon[29221]: fsmap cephfs:1 {0=cephfs.smithi022.ocukvs=up:active} 1 up:standby 2024-02-16T19:26:26.953 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:26 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi117.ibmppn"}]: dispatch 2024-02-16T19:26:26.953 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:26 smithi022 ceph-mon[29221]: fsmap cephfs:1 {0=cephfs.smithi022.ocukvs=up:active} 1 up:standby 2024-02-16T19:26:26.953 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:26 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:26:27.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:26 smithi117 ceph-mon[35309]: Deploying daemon mds.cephfs.smithi022.btppwm on smithi022 2024-02-16T19:26:27.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:26 smithi117 ceph-mon[35309]: mds.? [v2:172.21.15.117:6824/565408102,v1:172.21.15.117:6825/565408102] up:boot 2024-02-16T19:26:27.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:26 smithi117 ceph-mon[35309]: fsmap cephfs:1 {0=cephfs.smithi022.ocukvs=up:active} 1 up:standby 2024-02-16T19:26:27.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:26 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi117.ibmppn"}]: dispatch 2024-02-16T19:26:27.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:26 smithi117 ceph-mon[35309]: fsmap cephfs:1 {0=cephfs.smithi022.ocukvs=up:active} 1 up:standby 2024-02-16T19:26:27.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:26 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:26:28.175 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:27 smithi022 ceph-mon[29221]: pgmap v184: 65 pgs: 65 active+clean; 578 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 511 B/s wr, 3 op/s 2024-02-16T19:26:28.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:27 smithi117 ceph-mon[35309]: pgmap v184: 65 pgs: 65 active+clean; 578 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 511 B/s wr, 3 op/s 2024-02-16T19:26:29.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:28.601+0000 7f8c27675700 1 -- 172.21.15.22:0/3180196628 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8c20069210 msgr2=0x7f8c20102e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:29.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:28.601+0000 7f8c27675700 1 --2- 172.21.15.22:0/3180196628 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8c20069210 0x7f8c20102e30 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c009910 tx=0x7f8c1c009c20 comp rx=0 tx=0).stop 2024-02-16T19:26:29.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:28.601+0000 7f8c27675700 1 -- 172.21.15.22:0/3180196628 shutdown_connections 2024-02-16T19:26:29.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:28.601+0000 7f8c27675700 1 --2- 172.21.15.22:0/3180196628 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c20103370 0x7f8c20105790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:29.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:28.601+0000 7f8c27675700 1 --2- 172.21.15.22:0/3180196628 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8c20069210 0x7f8c20102e30 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:29.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:28.601+0000 7f8c27675700 1 -- 172.21.15.22:0/3180196628 >> 172.21.15.22:0/3180196628 conn(0x7f8c200fa780 msgr2=0x7f8c200fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:26:29.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:28.601+0000 7f8c27675700 1 -- 172.21.15.22:0/3180196628 shutdown_connections 2024-02-16T19:26:29.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:28.601+0000 7f8c27675700 1 -- 172.21.15.22:0/3180196628 wait complete. 2024-02-16T19:26:29.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:28.601+0000 7f8c27675700 1 Processor -- start 2024-02-16T19:26:29.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:28.602+0000 7f8c27675700 1 -- start start 2024-02-16T19:26:29.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:28.602+0000 7f8c27675700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8c20069210 0x7f8c20100780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:29.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:28.602+0000 7f8c27675700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c20103370 0x7f8c200fedd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:29.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:28.602+0000 7f8c27675700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c200ff310 con 0x7f8c20069210 2024-02-16T19:26:29.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:28.602+0000 7f8c27675700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c200ff450 con 0x7f8c20103370 2024-02-16T19:26:29.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:28.602+0000 7f8c25411700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8c20069210 0x7f8c20100780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:29.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:28.602+0000 7f8c25411700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8c20069210 0x7f8c20100780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49100/0 (socket says 172.21.15.22:49100) 2024-02-16T19:26:29.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:28.602+0000 7f8c25411700 1 -- 172.21.15.22:0/927368668 learned_addr learned my addr 172.21.15.22:0/927368668 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:26:29.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:28.602+0000 7f8c25411700 1 -- 172.21.15.22:0/927368668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c20103370 msgr2=0x7f8c200fedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:29.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:28.602+0000 7f8c25411700 1 --2- 172.21.15.22:0/927368668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c20103370 0x7f8c200fedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:29.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:28.602+0000 7f8c25411700 1 -- 172.21.15.22:0/927368668 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c1c0095f0 con 0x7f8c20069210 2024-02-16T19:26:29.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:28.602+0000 7f8c25411700 1 --2- 172.21.15.22:0/927368668 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8c20069210 0x7f8c20100780 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c00bda0 tx=0x7f8c1c00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:26:29.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:28.602+0000 7f8c167fc700 1 -- 172.21.15.22:0/927368668 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c1c01d070 con 0x7f8c20069210 2024-02-16T19:26:29.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:28.602+0000 7f8c167fc700 1 -- 172.21.15.22:0/927368668 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8c1c004c50 con 0x7f8c20069210 2024-02-16T19:26:29.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:28.602+0000 7f8c27675700 1 -- 172.21.15.22:0/927368668 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c200ff6d0 con 0x7f8c20069210 2024-02-16T19:26:29.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:28.602+0000 7f8c167fc700 1 -- 172.21.15.22:0/927368668 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c1c00f690 con 0x7f8c20069210 2024-02-16T19:26:29.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:28.602+0000 7f8c27675700 1 -- 172.21.15.22:0/927368668 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c200ffbc0 con 0x7f8c20069210 2024-02-16T19:26:29.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:28.603+0000 7f8c27675700 1 -- 172.21.15.22:0/927368668 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c2018fc50 con 0x7f8c20069210 2024-02-16T19:26:29.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:28.604+0000 7f8c167fc700 1 -- 172.21.15.22:0/927368668 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f8c1c0229b0 con 0x7f8c20069210 2024-02-16T19:26:29.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:28.604+0000 7f8c167fc700 1 --2- 172.21.15.22:0/927368668 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f8c0c06c680 0x7f8c0c06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:29.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:28.604+0000 7f8c167fc700 1 -- 172.21.15.22:0/927368668 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f8c1c08cf70 con 0x7f8c20069210 2024-02-16T19:26:29.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:28.604+0000 7f8c24c10700 1 --2- 172.21.15.22:0/927368668 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f8c0c06c680 0x7f8c0c06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:29.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:28.605+0000 7f8c24c10700 1 --2- 172.21.15.22:0/927368668 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f8c0c06c680 0x7f8c0c06eb30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8c100097b0 tx=0x7f8c10006d20 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:26:29.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:28.609+0000 7f8c167fc700 1 -- 172.21.15.22:0/927368668 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f8c1c0bf720 con 0x7f8c20069210 2024-02-16T19:26:29.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:28.861+0000 7f8c27675700 1 -- 172.21.15.22:0/927368668 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f8c2002cc30 con 0x7f8c20069210 2024-02-16T19:26:29.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:28.916+0000 7f8c167fc700 1 -- 172.21.15.22:0/927368668 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) v1 ==== 122+0+0 (secure 0 0 0) 0x7f8c1c05b190 con 0x7f8c20069210 2024-02-16T19:26:29.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:28.918+0000 7f8c27675700 1 -- 172.21.15.22:0/927368668 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f8c0c06c680 msgr2=0x7f8c0c06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:29.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:28.918+0000 7f8c27675700 1 --2- 172.21.15.22:0/927368668 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f8c0c06c680 0x7f8c0c06eb30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8c100097b0 tx=0x7f8c10006d20 comp rx=0 tx=0).stop 2024-02-16T19:26:29.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:28.918+0000 7f8c27675700 1 -- 172.21.15.22:0/927368668 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8c20069210 msgr2=0x7f8c20100780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:29.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:28.918+0000 7f8c27675700 1 --2- 172.21.15.22:0/927368668 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8c20069210 0x7f8c20100780 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c00bda0 tx=0x7f8c1c00be80 comp rx=0 tx=0).stop 2024-02-16T19:26:29.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:28.918+0000 7f8c27675700 1 -- 172.21.15.22:0/927368668 shutdown_connections 2024-02-16T19:26:29.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:28.918+0000 7f8c27675700 1 --2- 172.21.15.22:0/927368668 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8c20069210 0x7f8c20100780 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:29.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:28.918+0000 7f8c27675700 1 --2- 172.21.15.22:0/927368668 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f8c0c06c680 0x7f8c0c06eb30 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:29.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:28.918+0000 7f8c27675700 1 --2- 172.21.15.22:0/927368668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c20103370 0x7f8c200fedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:29.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:28.918+0000 7f8c27675700 1 -- 172.21.15.22:0/927368668 >> 172.21.15.22:0/927368668 conn(0x7f8c200fa780 msgr2=0x7f8c200fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:26:29.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:28.918+0000 7f8c27675700 1 -- 172.21.15.22:0/927368668 shutdown_connections 2024-02-16T19:26:29.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:28.918+0000 7f8c27675700 1 -- 172.21.15.22:0/927368668 wait complete. 2024-02-16T19:26:29.101 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:28 smithi022 ceph-mon[29221]: from='client.? 172.21.15.22:0/927368668' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]: dispatch 2024-02-16T19:26:29.386 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:28 smithi117 ceph-mon[35309]: from='client.? 172.21.15.22:0/927368668' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]: dispatch 2024-02-16T19:26:29.658 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-16T19:26:29.668 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi022.front.sepia.ceph.com 2024-02-16T19:26:29.668 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -- bash -c 'ceph fs set cephfs inline_data true --yes-i-really-really-mean-it' 2024-02-16T19:26:29.893 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi022/config 2024-02-16T19:26:30.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:29 smithi117 ceph-mon[35309]: pgmap v185: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.4 KiB/s wr, 4 op/s 2024-02-16T19:26:30.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:29 smithi117 ceph-mon[35309]: from='client.? 172.21.15.22:0/927368668' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]': finished 2024-02-16T19:26:30.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:29 smithi117 ceph-mon[35309]: mds.? [v2:172.21.15.22:6828/3597685751,v1:172.21.15.22:6829/3597685751] up:boot 2024-02-16T19:26:30.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:29 smithi117 ceph-mon[35309]: fsmap cephfs:1 {0=cephfs.smithi022.ocukvs=up:active} 2 up:standby 2024-02-16T19:26:30.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:29 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi022.btppwm"}]: dispatch 2024-02-16T19:26:30.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:29 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:26:30.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:29 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:26:30.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:29 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:26:30.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:29 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi117.orzors", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T19:26:30.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:29 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi117.orzors", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-16T19:26:30.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:29 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:26:30.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:29 smithi022 ceph-mon[29221]: pgmap v185: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.4 KiB/s wr, 4 op/s 2024-02-16T19:26:30.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:29 smithi022 ceph-mon[29221]: from='client.? 172.21.15.22:0/927368668' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]': finished 2024-02-16T19:26:30.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:29 smithi022 ceph-mon[29221]: mds.? [v2:172.21.15.22:6828/3597685751,v1:172.21.15.22:6829/3597685751] up:boot 2024-02-16T19:26:30.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:29 smithi022 ceph-mon[29221]: fsmap cephfs:1 {0=cephfs.smithi022.ocukvs=up:active} 2 up:standby 2024-02-16T19:26:30.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:29 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi022.btppwm"}]: dispatch 2024-02-16T19:26:30.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:29 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:26:30.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:29 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:26:30.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:29 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:26:30.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:29 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi117.orzors", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T19:26:30.216 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:29 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi117.orzors", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-16T19:26:30.216 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:29 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:26:30.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:30 smithi022 ceph-mon[29221]: Deploying daemon mds.cephfs.smithi117.orzors on smithi117 2024-02-16T19:26:31.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:30 smithi117 ceph-mon[35309]: Deploying daemon mds.cephfs.smithi117.orzors on smithi117 2024-02-16T19:26:31.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:31.537+0000 7ff56a669700 1 -- 172.21.15.22:0/105480299 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5641018e0 msgr2=0x7ff564101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:31.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:31.537+0000 7ff56a669700 1 --2- 172.21.15.22:0/105480299 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5641018e0 0x7ff564101cf0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff554009910 tx=0x7ff554009c20 comp rx=0 tx=0).stop 2024-02-16T19:26:31.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:31.537+0000 7ff56a669700 1 -- 172.21.15.22:0/105480299 shutdown_connections 2024-02-16T19:26:31.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:31.537+0000 7ff56a669700 1 --2- 172.21.15.22:0/105480299 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff564102230 0x7ff5640fe360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:31.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:31.537+0000 7ff56a669700 1 --2- 172.21.15.22:0/105480299 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5641018e0 0x7ff564101cf0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:31.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:31.537+0000 7ff56a669700 1 -- 172.21.15.22:0/105480299 >> 172.21.15.22:0/105480299 conn(0x7ff5640f9f10 msgr2=0x7ff5640fc340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:26:31.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:31.537+0000 7ff56a669700 1 -- 172.21.15.22:0/105480299 shutdown_connections 2024-02-16T19:26:31.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:31.537+0000 7ff56a669700 1 -- 172.21.15.22:0/105480299 wait complete. 2024-02-16T19:26:31.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:31.538+0000 7ff56a669700 1 Processor -- start 2024-02-16T19:26:31.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:31.538+0000 7ff56a669700 1 -- start start 2024-02-16T19:26:31.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:31.538+0000 7ff56a669700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5641018e0 0x7ff564195b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:31.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:31.538+0000 7ff56a669700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff564102230 0x7ff564196060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:31.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:31.538+0000 7ff56a669700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff564196680 con 0x7ff5641018e0 2024-02-16T19:26:31.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:31.538+0000 7ff56a669700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5641967c0 con 0x7ff564102230 2024-02-16T19:26:31.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:31.539+0000 7ff563fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5641018e0 0x7ff564195b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:31.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:31.539+0000 7ff563fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5641018e0 0x7ff564195b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49152/0 (socket says 172.21.15.22:49152) 2024-02-16T19:26:31.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:31.539+0000 7ff563fff700 1 -- 172.21.15.22:0/3244151036 learned_addr learned my addr 172.21.15.22:0/3244151036 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:26:31.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:31.539+0000 7ff5637fe700 1 --2- 172.21.15.22:0/3244151036 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff564102230 0x7ff564196060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:31.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:31.539+0000 7ff563fff700 1 -- 172.21.15.22:0/3244151036 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff564102230 msgr2=0x7ff564196060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:31.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:31.539+0000 7ff563fff700 1 --2- 172.21.15.22:0/3244151036 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff564102230 0x7ff564196060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:31.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:31.539+0000 7ff563fff700 1 -- 172.21.15.22:0/3244151036 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5540095f0 con 0x7ff5641018e0 2024-02-16T19:26:31.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:31.539+0000 7ff563fff700 1 --2- 172.21.15.22:0/3244151036 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5641018e0 0x7ff564195b20 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff554006010 tx=0x7ff55400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:26:31.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:31.539+0000 7ff5617fa700 1 -- 172.21.15.22:0/3244151036 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff55401d070 con 0x7ff5641018e0 2024-02-16T19:26:31.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:31.539+0000 7ff56a669700 1 -- 172.21.15.22:0/3244151036 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff56419b210 con 0x7ff5641018e0 2024-02-16T19:26:31.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:31.539+0000 7ff5617fa700 1 -- 172.21.15.22:0/3244151036 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff55400f460 con 0x7ff5641018e0 2024-02-16T19:26:31.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:31.539+0000 7ff5617fa700 1 -- 172.21.15.22:0/3244151036 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff554021640 con 0x7ff5641018e0 2024-02-16T19:26:31.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:31.539+0000 7ff56a669700 1 -- 172.21.15.22:0/3244151036 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff56419b700 con 0x7ff5641018e0 2024-02-16T19:26:31.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:31.540+0000 7ff56a669700 1 -- 172.21.15.22:0/3244151036 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff564108d60 con 0x7ff5641018e0 2024-02-16T19:26:31.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:31.541+0000 7ff5617fa700 1 -- 172.21.15.22:0/3244151036 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7ff55400fab0 con 0x7ff5641018e0 2024-02-16T19:26:31.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:31.541+0000 7ff5617fa700 1 --2- 172.21.15.22:0/3244151036 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7ff54c06c6d0 0x7ff54c06eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:31.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:31.541+0000 7ff5637fe700 1 --2- 172.21.15.22:0/3244151036 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7ff54c06c6d0 0x7ff54c06eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:31.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:31.541+0000 7ff5617fa700 1 -- 172.21.15.22:0/3244151036 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7ff55408cba0 con 0x7ff5641018e0 2024-02-16T19:26:31.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:31.542+0000 7ff5637fe700 1 --2- 172.21.15.22:0/3244151036 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7ff54c06c6d0 0x7ff54c06eb80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff558005950 tx=0x7ff55800b410 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:26:31.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:31.546+0000 7ff5617fa700 1 -- 172.21.15.22:0/3244151036 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7ff55405ac70 con 0x7ff5641018e0 2024-02-16T19:26:31.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:31.795+0000 7ff56a669700 1 -- 172.21.15.22:0/3244151036 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff564108f50 con 0x7ff5641018e0 2024-02-16T19:26:31.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:31.923+0000 7ff5617fa700 1 -- 172.21.15.22:0/3244151036 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]=0 inline data enabled v10) v1 ==== 168+0+0 (secure 0 0 0) 0x7ff55405a800 con 0x7ff5641018e0 2024-02-16T19:26:31.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:31.926+0000 7ff56a669700 1 -- 172.21.15.22:0/3244151036 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7ff54c06c6d0 msgr2=0x7ff54c06eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:31.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:31.926+0000 7ff56a669700 1 --2- 172.21.15.22:0/3244151036 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7ff54c06c6d0 0x7ff54c06eb80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff558005950 tx=0x7ff55800b410 comp rx=0 tx=0).stop 2024-02-16T19:26:31.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:31.926+0000 7ff56a669700 1 -- 172.21.15.22:0/3244151036 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5641018e0 msgr2=0x7ff564195b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:31.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:31.926+0000 7ff56a669700 1 --2- 172.21.15.22:0/3244151036 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5641018e0 0x7ff564195b20 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff554006010 tx=0x7ff55400bd70 comp rx=0 tx=0).stop 2024-02-16T19:26:31.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:31.926+0000 7ff56a669700 1 -- 172.21.15.22:0/3244151036 shutdown_connections 2024-02-16T19:26:31.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:31.926+0000 7ff56a669700 1 --2- 172.21.15.22:0/3244151036 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5641018e0 0x7ff564195b20 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:31.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:31.926+0000 7ff56a669700 1 --2- 172.21.15.22:0/3244151036 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7ff54c06c6d0 0x7ff54c06eb80 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:31.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:31.926+0000 7ff56a669700 1 --2- 172.21.15.22:0/3244151036 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff564102230 0x7ff564196060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:31.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:31.926+0000 7ff56a669700 1 -- 172.21.15.22:0/3244151036 >> 172.21.15.22:0/3244151036 conn(0x7ff5640f9f10 msgr2=0x7ff5641048b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:26:31.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:31.927+0000 7ff56a669700 1 -- 172.21.15.22:0/3244151036 shutdown_connections 2024-02-16T19:26:31.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:31.927+0000 7ff56a669700 1 -- 172.21.15.22:0/3244151036 wait complete. 2024-02-16T19:26:31.929 INFO:teuthology.orchestra.run.smithi022.stderr:inline data enabled 2024-02-16T19:26:32.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:31 smithi022 ceph-mon[29221]: pgmap v186: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.3 KiB/s wr, 4 op/s 2024-02-16T19:26:32.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:31 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:26:32.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:31 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:26:32.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:31 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:26:32.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:31 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:26:32.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:31 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:26:32.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:31 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:26:32.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:31 smithi022 ceph-mon[29221]: from='client.? 172.21.15.22:0/3244151036' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-16T19:26:32.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:31 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:26:32.294 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:31 smithi117 ceph-mon[35309]: pgmap v186: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.3 KiB/s wr, 4 op/s 2024-02-16T19:26:32.294 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:31 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:26:32.294 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:31 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:26:32.294 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:31 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:26:32.294 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:31 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:26:32.294 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:31 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:26:32.294 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:31 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:26:32.294 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:31 smithi117 ceph-mon[35309]: from='client.? 172.21.15.22:0/3244151036' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-16T19:26:32.294 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:31 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:26:32.578 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-16T19:26:32.589 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi022.front.sepia.ceph.com 2024-02-16T19:26:32.589 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -- bash -c 'ceph fs dump' 2024-02-16T19:26:33.135 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:32 smithi117 ceph-mon[35309]: Health check failed: 1 filesystem with deprecated feature inline_data (FS_INLINE_DATA_DEPRECATED) 2024-02-16T19:26:33.135 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:32 smithi117 ceph-mon[35309]: mds.? [v2:172.21.15.117:6826/3122342028,v1:172.21.15.117:6827/3122342028] up:boot 2024-02-16T19:26:33.135 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:32 smithi117 ceph-mon[35309]: from='client.? 172.21.15.22:0/3244151036' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]': finished 2024-02-16T19:26:33.135 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:32 smithi117 ceph-mon[35309]: fsmap cephfs:1 {0=cephfs.smithi022.ocukvs=up:active} 3 up:standby 2024-02-16T19:26:33.135 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:32 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi117.orzors"}]: dispatch 2024-02-16T19:26:33.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:32 smithi022 ceph-mon[29221]: Health check failed: 1 filesystem with deprecated feature inline_data (FS_INLINE_DATA_DEPRECATED) 2024-02-16T19:26:33.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:32 smithi022 ceph-mon[29221]: mds.? [v2:172.21.15.117:6826/3122342028,v1:172.21.15.117:6827/3122342028] up:boot 2024-02-16T19:26:33.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:32 smithi022 ceph-mon[29221]: from='client.? 172.21.15.22:0/3244151036' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]': finished 2024-02-16T19:26:33.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:32 smithi022 ceph-mon[29221]: fsmap cephfs:1 {0=cephfs.smithi022.ocukvs=up:active} 3 up:standby 2024-02-16T19:26:33.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:32 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi117.orzors"}]: dispatch 2024-02-16T19:26:33.348 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi022/config 2024-02-16T19:26:34.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:33 smithi022 ceph-mon[29221]: pgmap v187: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s wr, 3 op/s 2024-02-16T19:26:34.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:33 smithi022 ceph-mon[29221]: mds.? [v2:172.21.15.22:6828/3597685751,v1:172.21.15.22:6829/3597685751] up:standby 2024-02-16T19:26:34.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:33 smithi022 ceph-mon[29221]: Dropping low affinity active daemon mds.cephfs.smithi022.ocukvs in favor of higher affinity standby. 2024-02-16T19:26:34.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:33 smithi022 ceph-mon[29221]: Replacing daemon mds.cephfs.smithi022.ocukvs as rank 0 with standby daemon mds.cephfs.smithi022.btppwm 2024-02-16T19:26:34.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:33 smithi022 ceph-mon[29221]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-02-16T19:26:34.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:33 smithi022 ceph-mon[29221]: fsmap cephfs:1 {0=cephfs.smithi022.ocukvs=up:active} 3 up:standby 2024-02-16T19:26:34.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:33 smithi022 ceph-mon[29221]: osdmap e47: 6 total, 6 up, 6 in 2024-02-16T19:26:34.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:33 smithi022 ceph-mon[29221]: fsmap cephfs:1/1 {0=cephfs.smithi022.btppwm=up:replay} 2 up:standby 2024-02-16T19:26:34.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:33 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:26:34.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:33 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:26:34.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:33 smithi117 ceph-mon[35309]: pgmap v187: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s wr, 3 op/s 2024-02-16T19:26:34.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:33 smithi117 ceph-mon[35309]: mds.? [v2:172.21.15.22:6828/3597685751,v1:172.21.15.22:6829/3597685751] up:standby 2024-02-16T19:26:34.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:33 smithi117 ceph-mon[35309]: Dropping low affinity active daemon mds.cephfs.smithi022.ocukvs in favor of higher affinity standby. 2024-02-16T19:26:34.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:33 smithi117 ceph-mon[35309]: Replacing daemon mds.cephfs.smithi022.ocukvs as rank 0 with standby daemon mds.cephfs.smithi022.btppwm 2024-02-16T19:26:34.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:33 smithi117 ceph-mon[35309]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-02-16T19:26:34.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:33 smithi117 ceph-mon[35309]: fsmap cephfs:1 {0=cephfs.smithi022.ocukvs=up:active} 3 up:standby 2024-02-16T19:26:34.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:33 smithi117 ceph-mon[35309]: osdmap e47: 6 total, 6 up, 6 in 2024-02-16T19:26:34.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:33 smithi117 ceph-mon[35309]: fsmap cephfs:1/1 {0=cephfs.smithi022.btppwm=up:replay} 2 up:standby 2024-02-16T19:26:34.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:33 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:26:34.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:33 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:26:34.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:34 smithi022 ceph-mon[29221]: mds.? [v2:172.21.15.22:6828/3597685751,v1:172.21.15.22:6829/3597685751] up:reconnect 2024-02-16T19:26:34.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:34 smithi022 ceph-mon[29221]: mds.? [v2:172.21.15.22:6826/242865843,v1:172.21.15.22:6827/242865843] up:boot 2024-02-16T19:26:34.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:34 smithi022 ceph-mon[29221]: mds.? [v2:172.21.15.117:6824/565408102,v1:172.21.15.117:6825/565408102] up:standby 2024-02-16T19:26:34.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:34 smithi022 ceph-mon[29221]: fsmap cephfs:1/1 {0=cephfs.smithi022.btppwm=up:reconnect} 3 up:standby 2024-02-16T19:26:34.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:34 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi022.ocukvs"}]: dispatch 2024-02-16T19:26:35.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:34 smithi117 ceph-mon[35309]: mds.? [v2:172.21.15.22:6828/3597685751,v1:172.21.15.22:6829/3597685751] up:reconnect 2024-02-16T19:26:35.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:34 smithi117 ceph-mon[35309]: mds.? [v2:172.21.15.22:6826/242865843,v1:172.21.15.22:6827/242865843] up:boot 2024-02-16T19:26:35.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:34 smithi117 ceph-mon[35309]: mds.? [v2:172.21.15.117:6824/565408102,v1:172.21.15.117:6825/565408102] up:standby 2024-02-16T19:26:35.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:34 smithi117 ceph-mon[35309]: fsmap cephfs:1/1 {0=cephfs.smithi022.btppwm=up:reconnect} 3 up:standby 2024-02-16T19:26:35.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:34 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi022.ocukvs"}]: dispatch 2024-02-16T19:26:35.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:35.268+0000 7f4772c10700 1 -- 172.21.15.22:0/2958038849 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f476c101440 msgr2=0x7f476c101890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:35.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:35.268+0000 7f4772c10700 1 --2- 172.21.15.22:0/2958038849 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f476c101440 0x7f476c101890 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4758009910 tx=0x7f4758009c20 comp rx=0 tx=0).stop 2024-02-16T19:26:35.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:35.268+0000 7f4772c10700 1 -- 172.21.15.22:0/2958038849 shutdown_connections 2024-02-16T19:26:35.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:35.268+0000 7f4772c10700 1 --2- 172.21.15.22:0/2958038849 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f476c101440 0x7f476c101890 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:35.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:35.268+0000 7f4772c10700 1 --2- 172.21.15.22:0/2958038849 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f476c100240 0x7f476c100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:35.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:35.268+0000 7f4772c10700 1 -- 172.21.15.22:0/2958038849 >> 172.21.15.22:0/2958038849 conn(0x7f476c0fb7f0 msgr2=0x7f476c0fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:26:35.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:35.268+0000 7f4772c10700 1 -- 172.21.15.22:0/2958038849 shutdown_connections 2024-02-16T19:26:35.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:35.268+0000 7f4772c10700 1 -- 172.21.15.22:0/2958038849 wait complete. 2024-02-16T19:26:35.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:35.269+0000 7f4772c10700 1 Processor -- start 2024-02-16T19:26:35.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:35.269+0000 7f4772c10700 1 -- start start 2024-02-16T19:26:35.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:35.269+0000 7f4772c10700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f476c100240 0x7f476c195ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:35.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:35.269+0000 7f4772c10700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f476c101440 0x7f476c195ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:35.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:35.269+0000 7f4772c10700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f476c196580 con 0x7f476c101440 2024-02-16T19:26:35.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:35.269+0000 7f4772c10700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f476c1966c0 con 0x7f476c100240 2024-02-16T19:26:35.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:35.269+0000 7f476bfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f476c101440 0x7f476c195ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:35.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:35.269+0000 7f476bfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f476c101440 0x7f476c195ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53120/0 (socket says 172.21.15.22:53120) 2024-02-16T19:26:35.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:35.269+0000 7f476bfff700 1 -- 172.21.15.22:0/65365778 learned_addr learned my addr 172.21.15.22:0/65365778 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:26:35.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:35.269+0000 7f47709ac700 1 --2- 172.21.15.22:0/65365778 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f476c100240 0x7f476c195ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:35.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:35.269+0000 7f476bfff700 1 -- 172.21.15.22:0/65365778 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f476c100240 msgr2=0x7f476c195ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:35.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:35.269+0000 7f476bfff700 1 --2- 172.21.15.22:0/65365778 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f476c100240 0x7f476c195ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:35.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:35.269+0000 7f476bfff700 1 -- 172.21.15.22:0/65365778 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47580095f0 con 0x7f476c101440 2024-02-16T19:26:35.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:35.269+0000 7f476bfff700 1 --2- 172.21.15.22:0/65365778 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f476c101440 0x7f476c195ff0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4758006010 tx=0x7f475800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:26:35.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:35.269+0000 7f4769ffb700 1 -- 172.21.15.22:0/65365778 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f475801d070 con 0x7f476c101440 2024-02-16T19:26:35.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:35.269+0000 7f4769ffb700 1 -- 172.21.15.22:0/65365778 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f475800f460 con 0x7f476c101440 2024-02-16T19:26:35.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:35.270+0000 7f4772c10700 1 -- 172.21.15.22:0/65365778 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f476c19b120 con 0x7f476c101440 2024-02-16T19:26:35.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:35.270+0000 7f4769ffb700 1 -- 172.21.15.22:0/65365778 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4758021640 con 0x7f476c101440 2024-02-16T19:26:35.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:35.270+0000 7f4772c10700 1 -- 172.21.15.22:0/65365778 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f476c19b5e0 con 0x7f476c101440 2024-02-16T19:26:35.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:35.271+0000 7f4772c10700 1 -- 172.21.15.22:0/65365778 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f476c108d90 con 0x7f476c101440 2024-02-16T19:26:35.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:35.271+0000 7f4769ffb700 1 -- 172.21.15.22:0/65365778 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f475800fab0 con 0x7f476c101440 2024-02-16T19:26:35.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:35.271+0000 7f4769ffb700 1 --2- 172.21.15.22:0/65365778 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f475c06c680 0x7f475c06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:35.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:35.271+0000 7f4769ffb700 1 -- 172.21.15.22:0/65365778 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f475808cca0 con 0x7f476c101440 2024-02-16T19:26:35.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:35.272+0000 7f47709ac700 1 --2- 172.21.15.22:0/65365778 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f475c06c680 0x7f475c06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:35.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:35.272+0000 7f47709ac700 1 --2- 172.21.15.22:0/65365778 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f475c06c680 0x7f475c06eb30 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4760005fd0 tx=0x7f4760005e50 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:26:35.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:35.276+0000 7f4769ffb700 1 -- 172.21.15.22:0/65365778 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f4758091070 con 0x7f476c101440 2024-02-16T19:26:35.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:35.523+0000 7f4772c10700 1 -- 172.21.15.22:0/65365778 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f476c108f50 con 0x7f476c101440 2024-02-16T19:26:35.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:35.524+0000 7f4769ffb700 1 -- 172.21.15.22:0/65365778 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1811 (secure 0 0 0) 0x7f4758026070 con 0x7f476c101440 2024-02-16T19:26:35.604 INFO:teuthology.orchestra.run.smithi022.stdout:e14 2024-02-16T19:26:35.604 INFO:teuthology.orchestra.run.smithi022.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T19:26:35.604 INFO:teuthology.orchestra.run.smithi022.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:26:35.605 INFO:teuthology.orchestra.run.smithi022.stdout:legacy client fscid: 1 2024-02-16T19:26:35.605 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:26:35.605 INFO:teuthology.orchestra.run.smithi022.stdout:Filesystem 'cephfs' (1) 2024-02-16T19:26:35.605 INFO:teuthology.orchestra.run.smithi022.stdout:fs_name cephfs 2024-02-16T19:26:35.605 INFO:teuthology.orchestra.run.smithi022.stdout:epoch 14 2024-02-16T19:26:35.605 INFO:teuthology.orchestra.run.smithi022.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T19:26:35.605 INFO:teuthology.orchestra.run.smithi022.stdout:created 2024-02-16T19:26:17.873644+0000 2024-02-16T19:26:35.605 INFO:teuthology.orchestra.run.smithi022.stdout:modified 2024-02-16T19:26:34.938118+0000 2024-02-16T19:26:35.605 INFO:teuthology.orchestra.run.smithi022.stdout:tableserver 0 2024-02-16T19:26:35.605 INFO:teuthology.orchestra.run.smithi022.stdout:root 0 2024-02-16T19:26:35.605 INFO:teuthology.orchestra.run.smithi022.stdout:session_timeout 60 2024-02-16T19:26:35.605 INFO:teuthology.orchestra.run.smithi022.stdout:session_autoclose 300 2024-02-16T19:26:35.605 INFO:teuthology.orchestra.run.smithi022.stdout:max_file_size 1099511627776 2024-02-16T19:26:35.605 INFO:teuthology.orchestra.run.smithi022.stdout:required_client_features {} 2024-02-16T19:26:35.605 INFO:teuthology.orchestra.run.smithi022.stdout:last_failure 0 2024-02-16T19:26:35.605 INFO:teuthology.orchestra.run.smithi022.stdout:last_failure_osd_epoch 47 2024-02-16T19:26:35.606 INFO:teuthology.orchestra.run.smithi022.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:26:35.606 INFO:teuthology.orchestra.run.smithi022.stdout:max_mds 1 2024-02-16T19:26:35.606 INFO:teuthology.orchestra.run.smithi022.stdout:in 0 2024-02-16T19:26:35.606 INFO:teuthology.orchestra.run.smithi022.stdout:up {0=14490} 2024-02-16T19:26:35.606 INFO:teuthology.orchestra.run.smithi022.stdout:failed 2024-02-16T19:26:35.606 INFO:teuthology.orchestra.run.smithi022.stdout:damaged 2024-02-16T19:26:35.606 INFO:teuthology.orchestra.run.smithi022.stdout:stopped 2024-02-16T19:26:35.606 INFO:teuthology.orchestra.run.smithi022.stdout:data_pools [3] 2024-02-16T19:26:35.606 INFO:teuthology.orchestra.run.smithi022.stdout:metadata_pool 2 2024-02-16T19:26:35.606 INFO:teuthology.orchestra.run.smithi022.stdout:inline_data enabled 2024-02-16T19:26:35.606 INFO:teuthology.orchestra.run.smithi022.stdout:balancer 2024-02-16T19:26:35.606 INFO:teuthology.orchestra.run.smithi022.stdout:bal_rank_mask -1 2024-02-16T19:26:35.606 INFO:teuthology.orchestra.run.smithi022.stdout:standby_count_wanted 1 2024-02-16T19:26:35.606 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.cephfs.smithi022.btppwm{0:14490} state up:rejoin seq 4 join_fscid=1 addr [v2:172.21.15.22:6828/3597685751,v1:172.21.15.22:6829/3597685751] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:26:35.606 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:26:35.607 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:26:35.607 INFO:teuthology.orchestra.run.smithi022.stdout:Standby daemons: 2024-02-16T19:26:35.607 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:26:35.607 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.cephfs.smithi022.ocukvs{-1:14500} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.22:6826/242865843,v1:172.21.15.22:6827/242865843] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:26:35.607 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.cephfs.smithi117.ibmppn{-1:24259} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.117:6824/565408102,v1:172.21.15.117:6825/565408102] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:26:35.607 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.cephfs.smithi117.orzors{-1:24265} state up:standby seq 1 addr [v2:172.21.15.117:6826/3122342028,v1:172.21.15.117:6827/3122342028] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:26:35.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:35.526+0000 7f4772c10700 1 -- 172.21.15.22:0/65365778 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f475c06c680 msgr2=0x7f475c06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:35.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:35.526+0000 7f4772c10700 1 --2- 172.21.15.22:0/65365778 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f475c06c680 0x7f475c06eb30 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4760005fd0 tx=0x7f4760005e50 comp rx=0 tx=0).stop 2024-02-16T19:26:35.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:35.526+0000 7f4772c10700 1 -- 172.21.15.22:0/65365778 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f476c101440 msgr2=0x7f476c195ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:35.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:35.526+0000 7f4772c10700 1 --2- 172.21.15.22:0/65365778 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f476c101440 0x7f476c195ff0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4758006010 tx=0x7f475800be20 comp rx=0 tx=0).stop 2024-02-16T19:26:35.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:35.526+0000 7f4772c10700 1 -- 172.21.15.22:0/65365778 shutdown_connections 2024-02-16T19:26:35.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:35.526+0000 7f4772c10700 1 --2- 172.21.15.22:0/65365778 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f475c06c680 0x7f475c06eb30 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:35.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:35.526+0000 7f4772c10700 1 --2- 172.21.15.22:0/65365778 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f476c100240 0x7f476c195ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:35.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:35.526+0000 7f4772c10700 1 --2- 172.21.15.22:0/65365778 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f476c101440 0x7f476c195ff0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:35.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:35.526+0000 7f4772c10700 1 -- 172.21.15.22:0/65365778 >> 172.21.15.22:0/65365778 conn(0x7f476c0fb7f0 msgr2=0x7f476c104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:26:35.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:35.526+0000 7f4772c10700 1 -- 172.21.15.22:0/65365778 shutdown_connections 2024-02-16T19:26:35.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:35.526+0000 7f4772c10700 1 -- 172.21.15.22:0/65365778 wait complete. 2024-02-16T19:26:35.608 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 14 2024-02-16T19:26:36.155 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:35 smithi022 ceph-mon[29221]: pgmap v189: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.4 KiB/s wr, 4 op/s 2024-02-16T19:26:36.155 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:35 smithi022 ceph-mon[29221]: mds.? [v2:172.21.15.22:6828/3597685751,v1:172.21.15.22:6829/3597685751] up:rejoin 2024-02-16T19:26:36.155 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:35 smithi022 ceph-mon[29221]: fsmap cephfs:1/1 {0=cephfs.smithi022.btppwm=up:rejoin} 3 up:standby 2024-02-16T19:26:36.155 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:35 smithi022 ceph-mon[29221]: daemon mds.cephfs.smithi022.btppwm is now active in filesystem cephfs as rank 0 2024-02-16T19:26:36.155 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:35 smithi022 ceph-mon[29221]: from='client.? 172.21.15.22:0/65365778' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:26:36.177 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -- bash -c 'ceph --format=json fs dump | jq -e ".filesystems | length == 1"' 2024-02-16T19:26:36.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:35 smithi117 ceph-mon[35309]: pgmap v189: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.4 KiB/s wr, 4 op/s 2024-02-16T19:26:36.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:35 smithi117 ceph-mon[35309]: mds.? [v2:172.21.15.22:6828/3597685751,v1:172.21.15.22:6829/3597685751] up:rejoin 2024-02-16T19:26:36.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:35 smithi117 ceph-mon[35309]: fsmap cephfs:1/1 {0=cephfs.smithi022.btppwm=up:rejoin} 3 up:standby 2024-02-16T19:26:36.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:35 smithi117 ceph-mon[35309]: daemon mds.cephfs.smithi022.btppwm is now active in filesystem cephfs as rank 0 2024-02-16T19:26:36.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:35 smithi117 ceph-mon[35309]: from='client.? 172.21.15.22:0/65365778' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:26:37.211 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:36 smithi022 ceph-mon[29221]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-02-16T19:26:37.211 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:36 smithi022 ceph-mon[29221]: mds.? [v2:172.21.15.22:6828/3597685751,v1:172.21.15.22:6829/3597685751] up:active 2024-02-16T19:26:37.211 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:36 smithi022 ceph-mon[29221]: mds.? [v2:172.21.15.117:6826/3122342028,v1:172.21.15.117:6827/3122342028] up:standby 2024-02-16T19:26:37.211 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:36 smithi022 ceph-mon[29221]: fsmap cephfs:1 {0=cephfs.smithi022.btppwm=up:active} 3 up:standby 2024-02-16T19:26:37.357 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi022/config 2024-02-16T19:26:37.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:36 smithi117 ceph-mon[35309]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-02-16T19:26:37.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:36 smithi117 ceph-mon[35309]: mds.? [v2:172.21.15.22:6828/3597685751,v1:172.21.15.22:6829/3597685751] up:active 2024-02-16T19:26:37.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:36 smithi117 ceph-mon[35309]: mds.? [v2:172.21.15.117:6826/3122342028,v1:172.21.15.117:6827/3122342028] up:standby 2024-02-16T19:26:37.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:36 smithi117 ceph-mon[35309]: fsmap cephfs:1 {0=cephfs.smithi022.btppwm=up:active} 3 up:standby 2024-02-16T19:26:38.091 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:37 smithi022 ceph-mon[29221]: pgmap v190: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 3.6 KiB/s rd, 1023 B/s wr, 4 op/s 2024-02-16T19:26:38.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:37 smithi117 ceph-mon[35309]: pgmap v190: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 3.6 KiB/s rd, 1023 B/s wr, 4 op/s 2024-02-16T19:26:39.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:39.834+0000 7f85027f0700 1 -- 172.21.15.22:0/1439744493 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84fc100d60 msgr2=0x7f84fc103140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:39.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:39.834+0000 7f85027f0700 1 --2- 172.21.15.22:0/1439744493 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84fc100d60 0x7f84fc103140 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f84ec009910 tx=0x7f84ec009c20 comp rx=0 tx=0).stop 2024-02-16T19:26:39.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:39.834+0000 7f85027f0700 1 -- 172.21.15.22:0/1439744493 shutdown_connections 2024-02-16T19:26:39.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:39.834+0000 7f85027f0700 1 --2- 172.21.15.22:0/1439744493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f84fc103680 0x7f84fc105a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:39.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:39.834+0000 7f85027f0700 1 --2- 172.21.15.22:0/1439744493 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84fc100d60 0x7f84fc103140 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:39.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:39.834+0000 7f85027f0700 1 -- 172.21.15.22:0/1439744493 >> 172.21.15.22:0/1439744493 conn(0x7f84fc0fa780 msgr2=0x7f84fc0fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:26:39.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:39.834+0000 7f85027f0700 1 -- 172.21.15.22:0/1439744493 shutdown_connections 2024-02-16T19:26:39.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:39.834+0000 7f85027f0700 1 -- 172.21.15.22:0/1439744493 wait complete. 2024-02-16T19:26:39.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:39.834+0000 7f85027f0700 1 Processor -- start 2024-02-16T19:26:39.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:39.834+0000 7f85027f0700 1 -- start start 2024-02-16T19:26:39.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:39.835+0000 7f85027f0700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f84fc100d60 0x7f84fc195ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:39.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:39.835+0000 7f85027f0700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84fc103680 0x7f84fc195ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:39.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:39.835+0000 7f85027f0700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84fc196610 con 0x7f84fc103680 2024-02-16T19:26:39.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:39.835+0000 7f85027f0700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84fc196750 con 0x7f84fc100d60 2024-02-16T19:26:39.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:39.835+0000 7f84fb7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84fc103680 0x7f84fc195ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:39.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:39.835+0000 7f84fb7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84fc103680 0x7f84fc195ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53128/0 (socket says 172.21.15.22:53128) 2024-02-16T19:26:39.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:39.835+0000 7f84fb7fe700 1 -- 172.21.15.22:0/3441587641 learned_addr learned my addr 172.21.15.22:0/3441587641 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:26:39.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:39.835+0000 7f84fbfff700 1 --2- 172.21.15.22:0/3441587641 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f84fc100d60 0x7f84fc195ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:39.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:39.835+0000 7f84fb7fe700 1 -- 172.21.15.22:0/3441587641 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f84fc100d60 msgr2=0x7f84fc195ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:39.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:39.835+0000 7f84fb7fe700 1 --2- 172.21.15.22:0/3441587641 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f84fc100d60 0x7f84fc195ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:39.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:39.835+0000 7f84fb7fe700 1 -- 172.21.15.22:0/3441587641 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84ec0095f0 con 0x7f84fc103680 2024-02-16T19:26:39.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:39.835+0000 7f84fb7fe700 1 --2- 172.21.15.22:0/3441587641 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84fc103680 0x7f84fc195ff0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f84f000e920 tx=0x7f84f000ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:26:39.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:39.836+0000 7f84f97fa700 1 -- 172.21.15.22:0/3441587641 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f84f0009790 con 0x7f84fc103680 2024-02-16T19:26:39.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:39.836+0000 7f84f97fa700 1 -- 172.21.15.22:0/3441587641 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f84f000cba0 con 0x7f84fc103680 2024-02-16T19:26:39.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:39.836+0000 7f84f97fa700 1 -- 172.21.15.22:0/3441587641 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f84f0018630 con 0x7f84fc103680 2024-02-16T19:26:39.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:39.836+0000 7f85027f0700 1 -- 172.21.15.22:0/3441587641 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84fc19b200 con 0x7f84fc103680 2024-02-16T19:26:39.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:39.836+0000 7f85027f0700 1 -- 172.21.15.22:0/3441587641 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84fc19b750 con 0x7f84fc103680 2024-02-16T19:26:39.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:39.837+0000 7f85027f0700 1 -- 172.21.15.22:0/3441587641 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84fc18fbc0 con 0x7f84fc103680 2024-02-16T19:26:39.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:39.837+0000 7f84f97fa700 1 -- 172.21.15.22:0/3441587641 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f84f000cd10 con 0x7f84fc103680 2024-02-16T19:26:39.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:39.838+0000 7f84f97fa700 1 --2- 172.21.15.22:0/3441587641 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f84e406c680 0x7f84e406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:39.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:39.838+0000 7f84f97fa700 1 -- 172.21.15.22:0/3441587641 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f84f0014070 con 0x7f84fc103680 2024-02-16T19:26:39.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:39.838+0000 7f84fbfff700 1 --2- 172.21.15.22:0/3441587641 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f84e406c680 0x7f84e406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:39.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:39.838+0000 7f84fbfff700 1 --2- 172.21.15.22:0/3441587641 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f84e406c680 0x7f84e406eb30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f84ec00b6f0 tx=0x7f84ec009f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:26:39.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:39.844+0000 7f84f97fa700 1 -- 172.21.15.22:0/3441587641 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f84f0051720 con 0x7f84fc103680 2024-02-16T19:26:40.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:40.119+0000 7f85027f0700 1 -- 172.21.15.22:0/3441587641 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f84fc02d0a0 con 0x7f84fc103680 2024-02-16T19:26:40.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:40.119+0000 7f84f97fa700 1 -- 172.21.15.22:0/3441587641 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4739 (secure 0 0 0) 0x7f84f0059bd0 con 0x7f84fc103680 2024-02-16T19:26:40.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:40.122+0000 7f85027f0700 1 -- 172.21.15.22:0/3441587641 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f84e406c680 msgr2=0x7f84e406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:40.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:40.122+0000 7f85027f0700 1 --2- 172.21.15.22:0/3441587641 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f84e406c680 0x7f84e406eb30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f84ec00b6f0 tx=0x7f84ec009f90 comp rx=0 tx=0).stop 2024-02-16T19:26:40.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:40.122+0000 7f85027f0700 1 -- 172.21.15.22:0/3441587641 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84fc103680 msgr2=0x7f84fc195ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:40.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:40.122+0000 7f85027f0700 1 --2- 172.21.15.22:0/3441587641 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84fc103680 0x7f84fc195ff0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f84f000e920 tx=0x7f84f000ece0 comp rx=0 tx=0).stop 2024-02-16T19:26:40.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:40.122+0000 7f85027f0700 1 -- 172.21.15.22:0/3441587641 shutdown_connections 2024-02-16T19:26:40.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:40.122+0000 7f85027f0700 1 --2- 172.21.15.22:0/3441587641 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f84e406c680 0x7f84e406eb30 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:40.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:40.122+0000 7f85027f0700 1 --2- 172.21.15.22:0/3441587641 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f84fc100d60 0x7f84fc195ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:40.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:40.122+0000 7f85027f0700 1 --2- 172.21.15.22:0/3441587641 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84fc103680 0x7f84fc195ff0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:40.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:40.122+0000 7f85027f0700 1 -- 172.21.15.22:0/3441587641 >> 172.21.15.22:0/3441587641 conn(0x7f84fc0fa780 msgr2=0x7f84fc0fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:26:40.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:40.122+0000 7f85027f0700 1 -- 172.21.15.22:0/3441587641 shutdown_connections 2024-02-16T19:26:40.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:40.122+0000 7f85027f0700 1 -- 172.21.15.22:0/3441587641 wait complete. 2024-02-16T19:26:40.124 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 15 2024-02-16T19:26:40.133 INFO:teuthology.orchestra.run.smithi022.stdout:true 2024-02-16T19:26:40.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:39 smithi022 ceph-mon[29221]: pgmap v191: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 5.8 KiB/s rd, 0 B/s wr, 5 op/s 2024-02-16T19:26:40.386 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:39 smithi117 ceph-mon[35309]: pgmap v191: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 5.8 KiB/s rd, 0 B/s wr, 5 op/s 2024-02-16T19:26:40.758 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -- bash -c 'while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done' 2024-02-16T19:26:41.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:41 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:26:41.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:41 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:26:41.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:41 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:26:41.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:41 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:26:41.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:41 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:26:41.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:41 smithi117 ceph-mon[35309]: from='client.? 172.21.15.22:0/3441587641' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-16T19:26:41.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:41 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:26:41.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:41 smithi117 ceph-mon[35309]: pgmap v192: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 8.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-02-16T19:26:41.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:41 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:26:41.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:41 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:26:41.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:41 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:26:41.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:41 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:26:41.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:41 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:26:41.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:41 smithi022 ceph-mon[29221]: from='client.? 172.21.15.22:0/3441587641' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-16T19:26:41.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:41 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:26:41.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:41 smithi022 ceph-mon[29221]: pgmap v192: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 8.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-02-16T19:26:41.514 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi022/config 2024-02-16T19:26:43.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:43.046+0000 7f88c7767700 1 -- 172.21.15.22:0/2951406374 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f88c00fe8c0 msgr2=0x7f88c00fecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:43.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:43.046+0000 7f88c7767700 1 --2- 172.21.15.22:0/2951406374 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f88c00fe8c0 0x7f88c00fecd0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f88bc009910 tx=0x7f88bc009c20 comp rx=0 tx=0).stop 2024-02-16T19:26:43.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:43.046+0000 7f88c7767700 1 -- 172.21.15.22:0/2951406374 shutdown_connections 2024-02-16T19:26:43.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:43.046+0000 7f88c7767700 1 --2- 172.21.15.22:0/2951406374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88c00ffac0 0x7f88c00fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:43.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:43.046+0000 7f88c7767700 1 --2- 172.21.15.22:0/2951406374 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f88c00fe8c0 0x7f88c00fecd0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:43.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:43.046+0000 7f88c7767700 1 -- 172.21.15.22:0/2951406374 >> 172.21.15.22:0/2951406374 conn(0x7f88c00f9e50 msgr2=0x7f88c00fc2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:26:43.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:43.046+0000 7f88c7767700 1 -- 172.21.15.22:0/2951406374 shutdown_connections 2024-02-16T19:26:43.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:43.046+0000 7f88c7767700 1 -- 172.21.15.22:0/2951406374 wait complete. 2024-02-16T19:26:43.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:43.046+0000 7f88c7767700 1 Processor -- start 2024-02-16T19:26:43.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:43.046+0000 7f88c7767700 1 -- start start 2024-02-16T19:26:43.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:43.046+0000 7f88c7767700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f88c00fe8c0 0x7f88c01938c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:43.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:43.046+0000 7f88c7767700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88c00ffac0 0x7f88c0193e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:43.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:43.046+0000 7f88c7767700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c0194420 con 0x7f88c00fe8c0 2024-02-16T19:26:43.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:43.046+0000 7f88c7767700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c0194560 con 0x7f88c00ffac0 2024-02-16T19:26:43.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:43.047+0000 7f88c5503700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f88c00fe8c0 0x7f88c01938c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:43.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:43.047+0000 7f88c5503700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f88c00fe8c0 0x7f88c01938c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53146/0 (socket says 172.21.15.22:53146) 2024-02-16T19:26:43.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:43.047+0000 7f88c5503700 1 -- 172.21.15.22:0/1333829604 learned_addr learned my addr 172.21.15.22:0/1333829604 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:26:43.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:43.047+0000 7f88c4d02700 1 --2- 172.21.15.22:0/1333829604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88c00ffac0 0x7f88c0193e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:43.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:43.047+0000 7f88c5503700 1 -- 172.21.15.22:0/1333829604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88c00ffac0 msgr2=0x7f88c0193e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:43.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:43.047+0000 7f88c5503700 1 --2- 172.21.15.22:0/1333829604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88c00ffac0 0x7f88c0193e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:43.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:43.047+0000 7f88c5503700 1 -- 172.21.15.22:0/1333829604 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88bc0095f0 con 0x7f88c00fe8c0 2024-02-16T19:26:43.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:43.047+0000 7f88c4d02700 1 --2- 172.21.15.22:0/1333829604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88c00ffac0 0x7f88c0193e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T19:26:43.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:43.047+0000 7f88c5503700 1 --2- 172.21.15.22:0/1333829604 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f88c00fe8c0 0x7f88c01938c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f88bc00bda0 tx=0x7f88bc00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:26:43.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:43.047+0000 7f88b67fc700 1 -- 172.21.15.22:0/1333829604 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f88bc01d070 con 0x7f88c00fe8c0 2024-02-16T19:26:43.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:43.047+0000 7f88c7767700 1 -- 172.21.15.22:0/1333829604 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88c0198fb0 con 0x7f88c00fe8c0 2024-02-16T19:26:43.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:43.047+0000 7f88b67fc700 1 -- 172.21.15.22:0/1333829604 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f88bc00f460 con 0x7f88c00fe8c0 2024-02-16T19:26:43.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:43.047+0000 7f88b67fc700 1 -- 172.21.15.22:0/1333829604 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f88bc021640 con 0x7f88c00fe8c0 2024-02-16T19:26:43.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:43.047+0000 7f88c7767700 1 -- 172.21.15.22:0/1333829604 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88c0199420 con 0x7f88c00fe8c0 2024-02-16T19:26:43.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:43.048+0000 7f88c7767700 1 -- 172.21.15.22:0/1333829604 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88c0106b80 con 0x7f88c00fe8c0 2024-02-16T19:26:43.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:43.049+0000 7f88b67fc700 1 -- 172.21.15.22:0/1333829604 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f88bc00fa00 con 0x7f88c00fe8c0 2024-02-16T19:26:43.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:43.049+0000 7f88b67fc700 1 --2- 172.21.15.22:0/1333829604 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f88ac06c680 0x7f88ac06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:43.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:43.049+0000 7f88c4d02700 1 --2- 172.21.15.22:0/1333829604 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f88ac06c680 0x7f88ac06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:43.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:43.049+0000 7f88b67fc700 1 -- 172.21.15.22:0/1333829604 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f88bc08cbd0 con 0x7f88c00fe8c0 2024-02-16T19:26:43.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:43.050+0000 7f88c4d02700 1 --2- 172.21.15.22:0/1333829604 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f88ac06c680 0x7f88ac06eb30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f88b0005950 tx=0x7f88b00058e0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:26:43.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:43.054+0000 7f88b67fc700 1 -- 172.21.15.22:0/1333829604 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f88bc05ace0 con 0x7f88c00fe8c0 2024-02-16T19:26:43.134 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:42 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:26:43.134 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:42 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:26:43.134 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:42 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:26:43.134 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:42 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:26:43.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:42 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:26:43.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:42 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:26:43.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:42 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:26:43.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:42 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:26:43.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:43.355+0000 7f88c7767700 1 -- 172.21.15.22:0/1333829604 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mds versions", "format": "json"} v 0) v1 -- 0x7f88c0199b20 con 0x7f88c00fe8c0 2024-02-16T19:26:43.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:43.356+0000 7f88b67fc700 1 -- 172.21.15.22:0/1333829604 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "mds versions", "format": "json"}]=0 v15) v1 ==== 78+0+83 (secure 0 0 0) 0x7f88bc05a870 con 0x7f88c00fe8c0 2024-02-16T19:26:43.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:43.358+0000 7f88c7767700 1 -- 172.21.15.22:0/1333829604 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f88ac06c680 msgr2=0x7f88ac06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:43.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:43.358+0000 7f88c7767700 1 --2- 172.21.15.22:0/1333829604 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f88ac06c680 0x7f88ac06eb30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f88b0005950 tx=0x7f88b00058e0 comp rx=0 tx=0).stop 2024-02-16T19:26:43.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:43.358+0000 7f88c7767700 1 -- 172.21.15.22:0/1333829604 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f88c00fe8c0 msgr2=0x7f88c01938c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:43.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:43.358+0000 7f88c7767700 1 --2- 172.21.15.22:0/1333829604 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f88c00fe8c0 0x7f88c01938c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f88bc00bda0 tx=0x7f88bc00be80 comp rx=0 tx=0).stop 2024-02-16T19:26:43.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:43.358+0000 7f88c7767700 1 -- 172.21.15.22:0/1333829604 shutdown_connections 2024-02-16T19:26:43.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:43.358+0000 7f88c7767700 1 --2- 172.21.15.22:0/1333829604 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f88c00fe8c0 0x7f88c01938c0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:43.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:43.358+0000 7f88c7767700 1 --2- 172.21.15.22:0/1333829604 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f88ac06c680 0x7f88ac06eb30 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:43.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:43.358+0000 7f88c7767700 1 --2- 172.21.15.22:0/1333829604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88c00ffac0 0x7f88c0193e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:43.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:43.358+0000 7f88c7767700 1 -- 172.21.15.22:0/1333829604 >> 172.21.15.22:0/1333829604 conn(0x7f88c00f9e50 msgr2=0x7f88c0102cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:26:43.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:43.358+0000 7f88c7767700 1 -- 172.21.15.22:0/1333829604 shutdown_connections 2024-02-16T19:26:43.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:43.358+0000 7f88c7767700 1 -- 172.21.15.22:0/1333829604 wait complete. 2024-02-16T19:26:43.369 INFO:teuthology.orchestra.run.smithi022.stdout:true 2024-02-16T19:26:44.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:43 smithi117 ceph-mon[35309]: pgmap v193: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 8.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-02-16T19:26:44.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:43 smithi117 ceph-mon[35309]: from='client.? 172.21.15.22:0/1333829604' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-02-16T19:26:44.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:43 smithi022 ceph-mon[29221]: pgmap v193: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 8.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-02-16T19:26:44.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:43 smithi022 ceph-mon[29221]: from='client.? 172.21.15.22:0/1333829604' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-02-16T19:26:44.497 INFO:teuthology.run_tasks:Running task fs.pre_upgrade_save... 2024-02-16T19:26:44.510 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-16T19:26:44.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.653+0000 7fa1007a4640 1 -- 172.21.15.22:0/4108708596 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0f80fd950 msgr2=0x7fa0f80fdd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:44.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.653+0000 7fa1007a4640 1 --2- 172.21.15.22:0/4108708596 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0f80fd950 0x7fa0f80fdd60 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa0f4009880 tx=0x7fa0f402bce0 comp rx=0 tx=0).stop 2024-02-16T19:26:44.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.653+0000 7fa1007a4640 1 -- 172.21.15.22:0/4108708596 shutdown_connections 2024-02-16T19:26:44.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.653+0000 7fa1007a4640 1 --2- 172.21.15.22:0/4108708596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0f80fe2a0 0x7fa0f81009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:44.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.653+0000 7fa1007a4640 1 --2- 172.21.15.22:0/4108708596 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0f80fd950 0x7fa0f80fdd60 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:44.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.653+0000 7fa1007a4640 1 -- 172.21.15.22:0/4108708596 >> 172.21.15.22:0/4108708596 conn(0x7fa0f80f9350 msgr2=0x7fa0f80fb760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:26:44.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.653+0000 7fa1007a4640 1 -- 172.21.15.22:0/4108708596 shutdown_connections 2024-02-16T19:26:44.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.653+0000 7fa1007a4640 1 -- 172.21.15.22:0/4108708596 wait complete. 2024-02-16T19:26:44.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.654+0000 7fa1007a4640 1 Processor -- start 2024-02-16T19:26:44.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.654+0000 7fa1007a4640 1 -- start start 2024-02-16T19:26:44.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.654+0000 7fa1007a4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0f80fd950 0x7fa0f8194f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:44.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.654+0000 7fa1007a4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0f80fe2a0 0x7fa0f8195450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:44.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.654+0000 7fa1007a4640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f8195a20 con 0x7fa0f80fd950 2024-02-16T19:26:44.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.654+0000 7fa1007a4640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f8195b90 con 0x7fa0f80fe2a0 2024-02-16T19:26:44.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.654+0000 7fa0fe540640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0f80fd950 0x7fa0f8194f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:44.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.654+0000 7fa0fe540640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0f80fd950 0x7fa0f8194f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42880/0 (socket says 172.21.15.22:42880) 2024-02-16T19:26:44.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.654+0000 7fa0fe540640 1 -- 172.21.15.22:0/851257052 learned_addr learned my addr 172.21.15.22:0/851257052 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:26:44.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.654+0000 7fa0fdd3f640 1 --2- 172.21.15.22:0/851257052 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0f80fe2a0 0x7fa0f8195450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:44.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.654+0000 7fa0fe540640 1 -- 172.21.15.22:0/851257052 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0f80fe2a0 msgr2=0x7fa0f8195450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:44.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.654+0000 7fa0fe540640 1 --2- 172.21.15.22:0/851257052 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0f80fe2a0 0x7fa0f8195450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:44.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.654+0000 7fa0fe540640 1 -- 172.21.15.22:0/851257052 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0f4009560 con 0x7fa0f80fd950 2024-02-16T19:26:44.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.655+0000 7fa0fe540640 1 --2- 172.21.15.22:0/851257052 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0f80fd950 0x7fa0f8194f10 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa0f4009230 tx=0x7fa0f4004f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:26:44.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.655+0000 7fa0ef7fe640 1 -- 172.21.15.22:0/851257052 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa0f4004520 con 0x7fa0f80fd950 2024-02-16T19:26:44.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.655+0000 7fa1007a4640 1 -- 172.21.15.22:0/851257052 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0f819a5c0 con 0x7fa0f80fd950 2024-02-16T19:26:44.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.655+0000 7fa0ef7fe640 1 -- 172.21.15.22:0/851257052 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa0f40054b0 con 0x7fa0f80fd950 2024-02-16T19:26:44.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.655+0000 7fa0ef7fe640 1 -- 172.21.15.22:0/851257052 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa0f4034440 con 0x7fa0f80fd950 2024-02-16T19:26:44.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.655+0000 7fa1007a4640 1 -- 172.21.15.22:0/851257052 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0f819aac0 con 0x7fa0f80fd950 2024-02-16T19:26:44.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.656+0000 7fa1007a4640 1 -- 172.21.15.22:0/851257052 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0f8108380 con 0x7fa0f80fd950 2024-02-16T19:26:44.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.656+0000 7fa0ef7fe640 1 -- 172.21.15.22:0/851257052 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fa0f40037f0 con 0x7fa0f80fd950 2024-02-16T19:26:44.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.657+0000 7fa0ef7fe640 1 --2- 172.21.15.22:0/851257052 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fa0e406c6b0 0x7fa0e406eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:44.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.657+0000 7fa0ef7fe640 1 -- 172.21.15.22:0/851257052 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fa0f40ade60 con 0x7fa0f80fd950 2024-02-16T19:26:44.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.657+0000 7fa0fdd3f640 1 --2- 172.21.15.22:0/851257052 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fa0e406c6b0 0x7fa0e406eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:44.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.657+0000 7fa0fdd3f640 1 --2- 172.21.15.22:0/851257052 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fa0e406c6b0 0x7fa0e406eb60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa0e8005fd0 tx=0x7fa0e8005850 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:26:44.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.661+0000 7fa0ef7fe640 1 -- 172.21.15.22:0/851257052 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fa0f40784e0 con 0x7fa0f80fd950 2024-02-16T19:26:44.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.842+0000 7fa1007a4640 1 -- 172.21.15.22:0/851257052 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa0f819add0 con 0x7fa0f80fd950 2024-02-16T19:26:44.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.843+0000 7fa0ef7fe640 1 -- 172.21.15.22:0/851257052 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+15114 (secure 0 0 0) 0x7fa0f407bb00 con 0x7fa0f80fd950 2024-02-16T19:26:44.845 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:26:44.845 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":47,"fsid":"2a1346e6-cd00-11ee-95ba-87774f69a715","created":"2024-02-16T19:19:11.528280+0000","modified":"2024-02-16T19:26:32.932599+0000","last_up_change":"2024-02-16T19:24:59.819598+0000","last_in_change":"2024-02-16T19:24:40.575373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-16T19:23:47.101426+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-02-16T19:26:16.533404+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-02-16T19:26:17.113076+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d16e47f2-7209-45f4-ad50-1e0f6789ffad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2126256017},{"type":"v1","addr":"172.21.15.22:6803","nonce":2126256017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2126256017},{"type":"v1","addr":"172.21.15.22:6805","nonce":2126256017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2126256017},{"type":"v1","addr":"172.21.15.22:6809","nonce":2126256017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2126256017},{"type":"v1","addr":"172.21.15.22:6807","nonce":2126256017}]},"public_addr":"172.21.15.22:6803/2126256017","cluster_addr":"172.21.15.22:6805/2126256017","heartbeat_back_addr":"172.21.15.22:6809/2126256017","heartbeat_front_addr":"172.21.15.22:6807/2126256017","state":["exists","up"]},{"osd":1,"uuid":"59c3d085-3422-4e92-9048-46d7a39e31a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":3355370212},{"type":"v1","addr":"172.21.15.22:6811","nonce":3355370212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":3355370212},{"type":"v1","addr":"172.21.15.22:6813","nonce":3355370212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":3355370212},{"type":"v1","addr":"172.21.15.22:6817","nonce":3355370212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":3355370212},{"type":"v1","addr":"172.21.15.22:6815","nonce":3355370212}]},"public_addr":"172.21.15.22:6811/3355370212","cluster_addr":"172.21.15.22:6813/3355370212","heartbeat_back_addr":"172.21.15.22:6817/3355370212","heartbeat_front_addr":"172.21.15.22:6815/3355370212","state":["exists","up"]},{"osd":2,"uuid":"5952e710-f0b6-406c-855a-00f1cb8059c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":3627861698},{"type":"v1","addr":"172.21.15.22:6819","nonce":3627861698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":3627861698},{"type":"v1","addr":"172.21.15.22:6821","nonce":3627861698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":3627861698},{"type":"v1","addr":"172.21.15.22:6825","nonce":3627861698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":3627861698},{"type":"v1","addr":"172.21.15.22:6823","nonce":3627861698}]},"public_addr":"172.21.15.22:6819/3627861698","cluster_addr":"172.21.15.22:6821/3627861698","heartbeat_back_addr":"172.21.15.22:6825/3627861698","heartbeat_front_addr":"172.21.15.22:6823/3627861698","state":["exists","up"]},{"osd":3,"uuid":"5dc87494-e400-4111-91a7-306d9905febb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":2012503812},{"type":"v1","addr":"172.21.15.117:6801","nonce":2012503812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2012503812},{"type":"v1","addr":"172.21.15.117:6803","nonce":2012503812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2012503812},{"type":"v1","addr":"172.21.15.117:6807","nonce":2012503812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":2012503812},{"type":"v1","addr":"172.21.15.117:6805","nonce":2012503812}]},"public_addr":"172.21.15.117:6801/2012503812","cluster_addr":"172.21.15.117:6803/2012503812","heartbeat_back_addr":"172.21.15.117:6807/2012503812","heartbeat_front_addr":"172.21.15.117:6805/2012503812","state":["exists","up"]},{"osd":4,"uuid":"d27bde8d-1a77-4f95-acb6-7d05becd9480","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1856566932},{"type":"v1","addr":"172.21.15.117:6809","nonce":1856566932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":1856566932},{"type":"v1","addr":"172.21.15.117:6811","nonce":1856566932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":1856566932},{"type":"v1","addr":"172.21.15.117:6815","nonce":1856566932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1856566932},{"type":"v1","addr":"172.21.15.117:6813","nonce":1856566932}]},"public_addr":"172.21.15.117:6809/1856566932","cluster_addr":"172.21.15.117:6811/1856566932","heartbeat_back_addr":"172.21.15.117:6815/1856566932","heartbeat_front_addr":"172.21.15.117:6813/1856566932","state":["exists","up"]},{"osd":5,"uuid":"5873798f-3a2c-4a29-becd-6280b8866b6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":38288182},{"type":"v1","addr":"172.21.15.117:6817","nonce":38288182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":38288182},{"type":"v1","addr":"172.21.15.117:6819","nonce":38288182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":38288182},{"type":"v1","addr":"172.21.15.117:6823","nonce":38288182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":38288182},{"type":"v1","addr":"172.21.15.117:6821","nonce":38288182}]},"public_addr":"172.21.15.117:6817/38288182","cluster_addr":"172.21.15.117:6819/38288182","heartbeat_back_addr":"172.21.15.117:6823/38288182","heartbeat_front_addr":"172.21.15.117:6821/38288182","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:22:44.777427+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:23:14.935167+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:23:44.496184+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:24:07.447481+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:24:32.205064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:24:57.982560+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/8466130":"2024-02-17T19:19:38.544291+0000","172.21.15.22:0/3200629226":"2024-02-17T19:21:26.760165+0000","172.21.15.22:6827/149959963":"2024-02-17T19:26:32.932573+0000","172.21.15.22:0/1229330195":"2024-02-17T19:20:11.619146+0000","172.21.15.22:0/2099601555":"2024-02-17T19:20:11.619146+0000","172.21.15.22:0/1603216752":"2024-02-17T19:19:38.544291+0000","172.21.15.22:6800/3109015836":"2024-02-17T19:19:38.544291+0000","172.21.15.22:6801/3109015836":"2024-02-17T19:19:38.544291+0000","172.21.15.22:6801/2945983569":"2024-02-17T19:21:26.760165+0000","172.21.15.22:6800/2755289428":"2024-02-17T19:20:11.619146+0000","172.21.15.22:0/2384083375":"2024-02-17T19:21:26.760165+0000","172.21.15.22:0/2371245756":"2024-02-17T19:19:38.544291+0000","172.21.15.22:0/2017591724":"2024-02-17T19:20:11.619146+0000","172.21.15.22:0/668279955":"2024-02-17T19:21:26.760165+0000","172.21.15.22:6800/2945983569":"2024-02-17T19:21:26.760165+0000","172.21.15.22:6801/2755289428":"2024-02-17T19:20:11.619146+0000","172.21.15.22:6826/149959963":"2024-02-17T19:26:32.932573+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-16T19:26:44.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.845+0000 7fa1007a4640 1 -- 172.21.15.22:0/851257052 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fa0e406c6b0 msgr2=0x7fa0e406eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:44.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.845+0000 7fa1007a4640 1 --2- 172.21.15.22:0/851257052 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fa0e406c6b0 0x7fa0e406eb60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa0e8005fd0 tx=0x7fa0e8005850 comp rx=0 tx=0).stop 2024-02-16T19:26:44.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.845+0000 7fa1007a4640 1 -- 172.21.15.22:0/851257052 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0f80fd950 msgr2=0x7fa0f8194f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:44.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.845+0000 7fa1007a4640 1 --2- 172.21.15.22:0/851257052 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0f80fd950 0x7fa0f8194f10 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa0f4009230 tx=0x7fa0f4004f00 comp rx=0 tx=0).stop 2024-02-16T19:26:44.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.845+0000 7fa1007a4640 1 -- 172.21.15.22:0/851257052 shutdown_connections 2024-02-16T19:26:44.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.845+0000 7fa1007a4640 1 --2- 172.21.15.22:0/851257052 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fa0e406c6b0 0x7fa0e406eb60 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:44.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.845+0000 7fa1007a4640 1 --2- 172.21.15.22:0/851257052 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0f80fe2a0 0x7fa0f8195450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:44.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.845+0000 7fa1007a4640 1 --2- 172.21.15.22:0/851257052 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0f80fd950 0x7fa0f8194f10 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:44.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.845+0000 7fa1007a4640 1 -- 172.21.15.22:0/851257052 >> 172.21.15.22:0/851257052 conn(0x7fa0f80f9350 msgr2=0x7fa0f8103bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:26:44.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.845+0000 7fa1007a4640 1 -- 172.21.15.22:0/851257052 shutdown_connections 2024-02-16T19:26:44.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.845+0000 7fa1007a4640 1 -- 172.21.15.22:0/851257052 wait complete. 2024-02-16T19:26:44.858 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-16T19:23:47.101426+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-02-16T19:26:16.533404+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-02-16T19:26:17.113076+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-16T19:26:44.858 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-16T19:26:44.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.964+0000 7f6cb368b640 1 -- 172.21.15.22:0/747620626 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6cac069080 msgr2=0x7f6cac104610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:44.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.964+0000 7f6cb368b640 1 --2- 172.21.15.22:0/747620626 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6cac069080 0x7f6cac104610 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8009880 tx=0x7f6ca802bce0 comp rx=0 tx=0).stop 2024-02-16T19:26:44.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.964+0000 7f6cb368b640 1 -- 172.21.15.22:0/747620626 shutdown_connections 2024-02-16T19:26:44.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.964+0000 7f6cb368b640 1 --2- 172.21.15.22:0/747620626 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cac104b50 0x7f6cac106f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:44.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.964+0000 7f6cb368b640 1 --2- 172.21.15.22:0/747620626 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6cac069080 0x7f6cac104610 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:44.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.964+0000 7f6cb368b640 1 -- 172.21.15.22:0/747620626 >> 172.21.15.22:0/747620626 conn(0x7f6cac0f9ba0 msgr2=0x7f6cac0fbfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:26:44.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.964+0000 7f6cb368b640 1 -- 172.21.15.22:0/747620626 shutdown_connections 2024-02-16T19:26:44.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.964+0000 7f6cb368b640 1 -- 172.21.15.22:0/747620626 wait complete. 2024-02-16T19:26:44.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.965+0000 7f6cb368b640 1 Processor -- start 2024-02-16T19:26:44.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.965+0000 7f6cb368b640 1 -- start start 2024-02-16T19:26:44.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.965+0000 7f6cb368b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cac069080 0x7f6cac1971a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:44.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.965+0000 7f6cb368b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6cac104b50 0x7f6cac1976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:44.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.965+0000 7f6cb368b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cac197cb0 con 0x7f6cac104b50 2024-02-16T19:26:44.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.965+0000 7f6cb368b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cac197e20 con 0x7f6cac069080 2024-02-16T19:26:44.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.965+0000 7f6cb0c26640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6cac104b50 0x7f6cac1976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:44.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.965+0000 7f6cb0c26640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6cac104b50 0x7f6cac1976e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42896/0 (socket says 172.21.15.22:42896) 2024-02-16T19:26:44.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.965+0000 7f6cb0c26640 1 -- 172.21.15.22:0/976569272 learned_addr learned my addr 172.21.15.22:0/976569272 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:26:44.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.965+0000 7f6cb1427640 1 --2- 172.21.15.22:0/976569272 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cac069080 0x7f6cac1971a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:44.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.965+0000 7f6cb0c26640 1 -- 172.21.15.22:0/976569272 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cac069080 msgr2=0x7f6cac1971a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:44.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.965+0000 7f6cb0c26640 1 --2- 172.21.15.22:0/976569272 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cac069080 0x7f6cac1971a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:44.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.965+0000 7f6cb0c26640 1 -- 172.21.15.22:0/976569272 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ca8009560 con 0x7f6cac104b50 2024-02-16T19:26:44.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.965+0000 7f6cb1427640 1 --2- 172.21.15.22:0/976569272 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cac069080 0x7f6cac1971a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-16T19:26:44.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.965+0000 7f6cb0c26640 1 --2- 172.21.15.22:0/976569272 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6cac104b50 0x7f6cac1976e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c00d8d0 tx=0x7f6c9c00dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:26:44.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.966+0000 7f6ca27fc640 1 -- 172.21.15.22:0/976569272 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c9c00bea0 con 0x7f6cac104b50 2024-02-16T19:26:44.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.966+0000 7f6cb368b640 1 -- 172.21.15.22:0/976569272 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cac19c8b0 con 0x7f6cac104b50 2024-02-16T19:26:44.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.966+0000 7f6ca27fc640 1 -- 172.21.15.22:0/976569272 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6c9c009c00 con 0x7f6cac104b50 2024-02-16T19:26:44.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.966+0000 7f6ca27fc640 1 -- 172.21.15.22:0/976569272 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c9c00f710 con 0x7f6cac104b50 2024-02-16T19:26:44.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.966+0000 7f6cb368b640 1 -- 172.21.15.22:0/976569272 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cac19ce10 con 0x7f6cac104b50 2024-02-16T19:26:44.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.966+0000 7f6cb368b640 1 -- 172.21.15.22:0/976569272 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cac1911e0 con 0x7f6cac104b50 2024-02-16T19:26:44.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.967+0000 7f6ca27fc640 1 -- 172.21.15.22:0/976569272 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f6c9c00f990 con 0x7f6cac104b50 2024-02-16T19:26:44.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.967+0000 7f6ca27fc640 1 --2- 172.21.15.22:0/976569272 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f6c9806c5f0 0x7f6c9806eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:44.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.967+0000 7f6ca27fc640 1 -- 172.21.15.22:0/976569272 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f6c9c08b6f0 con 0x7f6cac104b50 2024-02-16T19:26:44.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.968+0000 7f6cb1427640 1 --2- 172.21.15.22:0/976569272 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f6c9806c5f0 0x7f6c9806eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:44.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.968+0000 7f6cb1427640 1 --2- 172.21.15.22:0/976569272 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f6c9806c5f0 0x7f6c9806eaa0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8036040 tx=0x7f6ca8005fd0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:26:44.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:44.972+0000 7f6ca27fc640 1 -- 172.21.15.22:0/976569272 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f6c9c059800 con 0x7f6cac104b50 2024-02-16T19:26:45.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.148+0000 7f6cb368b640 1 -- 172.21.15.22:0/976569272 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6cac02c4b0 con 0x7f6cac104b50 2024-02-16T19:26:45.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.148+0000 7f6ca27fc640 1 -- 172.21.15.22:0/976569272 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v47) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6c9c0200a0 con 0x7f6cac104b50 2024-02-16T19:26:45.149 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-02-16T19:26:45.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.150+0000 7f6cb368b640 1 -- 172.21.15.22:0/976569272 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f6c9806c5f0 msgr2=0x7f6c9806eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:45.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.150+0000 7f6cb368b640 1 --2- 172.21.15.22:0/976569272 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f6c9806c5f0 0x7f6c9806eaa0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8036040 tx=0x7f6ca8005fd0 comp rx=0 tx=0).stop 2024-02-16T19:26:45.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.150+0000 7f6cb368b640 1 -- 172.21.15.22:0/976569272 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6cac104b50 msgr2=0x7f6cac1976e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:45.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.150+0000 7f6cb368b640 1 --2- 172.21.15.22:0/976569272 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6cac104b50 0x7f6cac1976e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c00d8d0 tx=0x7f6c9c00dd90 comp rx=0 tx=0).stop 2024-02-16T19:26:45.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.150+0000 7f6cb368b640 1 -- 172.21.15.22:0/976569272 shutdown_connections 2024-02-16T19:26:45.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.150+0000 7f6cb368b640 1 --2- 172.21.15.22:0/976569272 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6cac104b50 0x7f6cac1976e0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:45.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.150+0000 7f6cb368b640 1 --2- 172.21.15.22:0/976569272 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f6c9806c5f0 0x7f6c9806eaa0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:45.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.150+0000 7f6cb368b640 1 --2- 172.21.15.22:0/976569272 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cac069080 0x7f6cac1971a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:45.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.150+0000 7f6cb368b640 1 -- 172.21.15.22:0/976569272 >> 172.21.15.22:0/976569272 conn(0x7f6cac0f9ba0 msgr2=0x7f6cac0fbf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:26:45.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.150+0000 7f6cb368b640 1 -- 172.21.15.22:0/976569272 shutdown_connections 2024-02-16T19:26:45.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.150+0000 7f6cb368b640 1 -- 172.21.15.22:0/976569272 wait complete. 2024-02-16T19:26:45.162 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-02-16T19:26:45.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.272+0000 7f96f205f640 1 -- 172.21.15.22:0/3735945219 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96ec0ff670 msgr2=0x7f96ec0ffa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:45.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.272+0000 7f96f205f640 1 --2- 172.21.15.22:0/3735945219 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96ec0ff670 0x7f96ec0ffa80 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f96dc009880 tx=0x7f96dc02bce0 comp rx=0 tx=0).stop 2024-02-16T19:26:45.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.272+0000 7f96f205f640 1 -- 172.21.15.22:0/3735945219 shutdown_connections 2024-02-16T19:26:45.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.272+0000 7f96f205f640 1 --2- 172.21.15.22:0/3735945219 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96ec100860 0x7f96ec100cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:45.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.272+0000 7f96f205f640 1 --2- 172.21.15.22:0/3735945219 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96ec0ff670 0x7f96ec0ffa80 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:45.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.272+0000 7f96f205f640 1 -- 172.21.15.22:0/3735945219 >> 172.21.15.22:0/3735945219 conn(0x7f96ec0fabf0 msgr2=0x7f96ec0fd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:26:45.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.272+0000 7f96f205f640 1 -- 172.21.15.22:0/3735945219 shutdown_connections 2024-02-16T19:26:45.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.272+0000 7f96f205f640 1 -- 172.21.15.22:0/3735945219 wait complete. 2024-02-16T19:26:45.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.272+0000 7f96f205f640 1 Processor -- start 2024-02-16T19:26:45.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.273+0000 7f96f205f640 1 -- start start 2024-02-16T19:26:45.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.273+0000 7f96f205f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96ec0ff670 0x7f96ec194fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:45.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.273+0000 7f96f205f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96ec100860 0x7f96ec195510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:45.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.273+0000 7f96f205f640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96ec195ae0 con 0x7f96ec100860 2024-02-16T19:26:45.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.273+0000 7f96f205f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96ec195c50 con 0x7f96ec0ff670 2024-02-16T19:26:45.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.273+0000 7f96eaffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96ec100860 0x7f96ec195510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:45.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.273+0000 7f96eaffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96ec100860 0x7f96ec195510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42924/0 (socket says 172.21.15.22:42924) 2024-02-16T19:26:45.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.273+0000 7f96eaffd640 1 -- 172.21.15.22:0/2269084230 learned_addr learned my addr 172.21.15.22:0/2269084230 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:26:45.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.273+0000 7f96eb7fe640 1 --2- 172.21.15.22:0/2269084230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96ec0ff670 0x7f96ec194fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:45.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.273+0000 7f96eaffd640 1 -- 172.21.15.22:0/2269084230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96ec0ff670 msgr2=0x7f96ec194fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:45.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.273+0000 7f96eaffd640 1 --2- 172.21.15.22:0/2269084230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96ec0ff670 0x7f96ec194fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:45.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.273+0000 7f96eaffd640 1 -- 172.21.15.22:0/2269084230 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96dc009560 con 0x7f96ec100860 2024-02-16T19:26:45.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.273+0000 7f96eb7fe640 1 --2- 172.21.15.22:0/2269084230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96ec0ff670 0x7f96ec194fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T19:26:45.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.273+0000 7f96eaffd640 1 --2- 172.21.15.22:0/2269084230 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96ec100860 0x7f96ec195510 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f96e000c8f0 tx=0x7f96e000cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:26:45.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.274+0000 7f96e8ff9640 1 -- 172.21.15.22:0/2269084230 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f96e0007b80 con 0x7f96ec100860 2024-02-16T19:26:45.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.274+0000 7f96f205f640 1 -- 172.21.15.22:0/2269084230 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96ec19a6e0 con 0x7f96ec100860 2024-02-16T19:26:45.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.274+0000 7f96e8ff9640 1 -- 172.21.15.22:0/2269084230 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f96e0007ce0 con 0x7f96ec100860 2024-02-16T19:26:45.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.274+0000 7f96e8ff9640 1 -- 172.21.15.22:0/2269084230 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f96e000f450 con 0x7f96ec100860 2024-02-16T19:26:45.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.274+0000 7f96f205f640 1 -- 172.21.15.22:0/2269084230 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96ec19ac40 con 0x7f96ec100860 2024-02-16T19:26:45.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.275+0000 7f96f205f640 1 -- 172.21.15.22:0/2269084230 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96ec1083a0 con 0x7f96ec100860 2024-02-16T19:26:45.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.275+0000 7f96e8ff9640 1 -- 172.21.15.22:0/2269084230 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f96e0004f60 con 0x7f96ec100860 2024-02-16T19:26:45.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.275+0000 7f96e8ff9640 1 --2- 172.21.15.22:0/2269084230 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f96d406c5f0 0x7f96d406eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:45.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.276+0000 7f96e8ff9640 1 -- 172.21.15.22:0/2269084230 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f96e008b700 con 0x7f96ec100860 2024-02-16T19:26:45.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.276+0000 7f96eb7fe640 1 --2- 172.21.15.22:0/2269084230 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f96d406c5f0 0x7f96d406eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:45.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.276+0000 7f96eb7fe640 1 --2- 172.21.15.22:0/2269084230 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f96d406c5f0 0x7f96d406eaa0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f96dc006010 tx=0x7f96dc02cf90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:26:45.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.280+0000 7f96e8ff9640 1 -- 172.21.15.22:0/2269084230 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f96e0055d80 con 0x7f96ec100860 2024-02-16T19:26:45.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.458+0000 7f96f205f640 1 -- 172.21.15.22:0/2269084230 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f96ec19aec0 con 0x7f96ec100860 2024-02-16T19:26:45.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.459+0000 7f96e8ff9640 1 -- 172.21.15.22:0/2269084230 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v47) v1 ==== 107+0+11 (secure 0 0 0) 0x7f96e00593a0 con 0x7f96ec100860 2024-02-16T19:26:45.460 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-02-16T19:26:45.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.461+0000 7f96f205f640 1 -- 172.21.15.22:0/2269084230 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f96d406c5f0 msgr2=0x7f96d406eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:45.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.461+0000 7f96f205f640 1 --2- 172.21.15.22:0/2269084230 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f96d406c5f0 0x7f96d406eaa0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f96dc006010 tx=0x7f96dc02cf90 comp rx=0 tx=0).stop 2024-02-16T19:26:45.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.461+0000 7f96f205f640 1 -- 172.21.15.22:0/2269084230 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96ec100860 msgr2=0x7f96ec195510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:45.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.461+0000 7f96f205f640 1 --2- 172.21.15.22:0/2269084230 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96ec100860 0x7f96ec195510 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f96e000c8f0 tx=0x7f96e000cdb0 comp rx=0 tx=0).stop 2024-02-16T19:26:45.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.461+0000 7f96f205f640 1 -- 172.21.15.22:0/2269084230 shutdown_connections 2024-02-16T19:26:45.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.461+0000 7f96f205f640 1 --2- 172.21.15.22:0/2269084230 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96ec100860 0x7f96ec195510 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:45.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.461+0000 7f96f205f640 1 --2- 172.21.15.22:0/2269084230 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f96d406c5f0 0x7f96d406eaa0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:45.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.461+0000 7f96f205f640 1 --2- 172.21.15.22:0/2269084230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96ec0ff670 0x7f96ec194fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:45.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.461+0000 7f96f205f640 1 -- 172.21.15.22:0/2269084230 >> 172.21.15.22:0/2269084230 conn(0x7f96ec0fabf0 msgr2=0x7f96ec103a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:26:45.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.461+0000 7f96f205f640 1 -- 172.21.15.22:0/2269084230 shutdown_connections 2024-02-16T19:26:45.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.461+0000 7f96f205f640 1 -- 172.21.15.22:0/2269084230 wait complete. 2024-02-16T19:26:45.473 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-02-16T19:26:45.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.583+0000 7f87e51b1640 1 -- 172.21.15.22:0/411614771 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87e01000c0 msgr2=0x7f87e01024a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:45.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.583+0000 7f87e51b1640 1 --2- 172.21.15.22:0/411614771 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87e01000c0 0x7f87e01024a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f87d00098b0 tx=0x7f87d002bcd0 comp rx=0 tx=0).stop 2024-02-16T19:26:45.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.583+0000 7f87e51b1640 1 -- 172.21.15.22:0/411614771 shutdown_connections 2024-02-16T19:26:45.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.583+0000 7f87e51b1640 1 --2- 172.21.15.22:0/411614771 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87e01029e0 0x7f87e0104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:45.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.583+0000 7f87e51b1640 1 --2- 172.21.15.22:0/411614771 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87e01000c0 0x7f87e01024a0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:45.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.583+0000 7f87e51b1640 1 -- 172.21.15.22:0/411614771 >> 172.21.15.22:0/411614771 conn(0x7f87e00f9a80 msgr2=0x7f87e00fbef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:26:45.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.583+0000 7f87e51b1640 1 -- 172.21.15.22:0/411614771 shutdown_connections 2024-02-16T19:26:45.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.583+0000 7f87e51b1640 1 -- 172.21.15.22:0/411614771 wait complete. 2024-02-16T19:26:45.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.583+0000 7f87e51b1640 1 Processor -- start 2024-02-16T19:26:45.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.584+0000 7f87e51b1640 1 -- start start 2024-02-16T19:26:45.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.584+0000 7f87e51b1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87e01000c0 0x7f87e0194ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:45.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.584+0000 7f87e51b1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87e01029e0 0x7f87e0195530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:45.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.584+0000 7f87e51b1640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e0195a70 con 0x7f87e01000c0 2024-02-16T19:26:45.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.584+0000 7f87e51b1640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e0195be0 con 0x7f87e01029e0 2024-02-16T19:26:45.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.584+0000 7f87ded9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87e01000c0 0x7f87e0194ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:45.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.584+0000 7f87ded9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87e01000c0 0x7f87e0194ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42938/0 (socket says 172.21.15.22:42938) 2024-02-16T19:26:45.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.584+0000 7f87ded9d640 1 -- 172.21.15.22:0/275002218 learned_addr learned my addr 172.21.15.22:0/275002218 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:26:45.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.584+0000 7f87de59c640 1 --2- 172.21.15.22:0/275002218 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87e01029e0 0x7f87e0195530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:45.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.584+0000 7f87ded9d640 1 -- 172.21.15.22:0/275002218 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87e01029e0 msgr2=0x7f87e0195530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:45.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.584+0000 7f87ded9d640 1 --2- 172.21.15.22:0/275002218 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87e01029e0 0x7f87e0195530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:45.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.584+0000 7f87ded9d640 1 -- 172.21.15.22:0/275002218 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87d0009560 con 0x7f87e01000c0 2024-02-16T19:26:45.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.584+0000 7f87ded9d640 1 --2- 172.21.15.22:0/275002218 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87e01000c0 0x7f87e0194ff0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f87d002c210 tx=0x7f87d0004f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:26:45.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.585+0000 7f87c7fff640 1 -- 172.21.15.22:0/275002218 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87d0005000 con 0x7f87e01000c0 2024-02-16T19:26:45.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.585+0000 7f87e51b1640 1 -- 172.21.15.22:0/275002218 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87e019a650 con 0x7f87e01000c0 2024-02-16T19:26:45.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.585+0000 7f87c7fff640 1 -- 172.21.15.22:0/275002218 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f87d0005610 con 0x7f87e01000c0 2024-02-16T19:26:45.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.585+0000 7f87e51b1640 1 -- 172.21.15.22:0/275002218 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87e019ab80 con 0x7f87e01000c0 2024-02-16T19:26:45.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.585+0000 7f87c7fff640 1 -- 172.21.15.22:0/275002218 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87d003e680 con 0x7f87e01000c0 2024-02-16T19:26:45.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.586+0000 7f87e51b1640 1 -- 172.21.15.22:0/275002218 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87e0108370 con 0x7f87e01000c0 2024-02-16T19:26:45.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.587+0000 7f87c7fff640 1 -- 172.21.15.22:0/275002218 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f87d0003a10 con 0x7f87e01000c0 2024-02-16T19:26:45.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.588+0000 7f87c7fff640 1 --2- 172.21.15.22:0/275002218 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f87c806c5f0 0x7f87c806eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:45.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.588+0000 7f87c7fff640 1 -- 172.21.15.22:0/275002218 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f87d00aea50 con 0x7f87e01000c0 2024-02-16T19:26:45.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.588+0000 7f87de59c640 1 --2- 172.21.15.22:0/275002218 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f87c806c5f0 0x7f87c806eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:45.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.588+0000 7f87de59c640 1 --2- 172.21.15.22:0/275002218 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f87c806c5f0 0x7f87c806eaa0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f87e0196500 tx=0x7f87d4005cc0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:26:45.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.591+0000 7f87c7fff640 1 -- 172.21.15.22:0/275002218 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f87d0079150 con 0x7f87e01000c0 2024-02-16T19:26:45.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.770+0000 7f87e51b1640 1 -- 172.21.15.22:0/275002218 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f87e019b000 con 0x7f87e01000c0 2024-02-16T19:26:45.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.770+0000 7f87c7fff640 1 -- 172.21.15.22:0/275002218 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v47) v1 ==== 107+0+11 (secure 0 0 0) 0x7f87d007c770 con 0x7f87e01000c0 2024-02-16T19:26:45.771 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-02-16T19:26:45.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.772+0000 7f87e51b1640 1 -- 172.21.15.22:0/275002218 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f87c806c5f0 msgr2=0x7f87c806eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:45.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.772+0000 7f87e51b1640 1 --2- 172.21.15.22:0/275002218 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f87c806c5f0 0x7f87c806eaa0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f87e0196500 tx=0x7f87d4005cc0 comp rx=0 tx=0).stop 2024-02-16T19:26:45.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.772+0000 7f87e51b1640 1 -- 172.21.15.22:0/275002218 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87e01000c0 msgr2=0x7f87e0194ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:45.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.772+0000 7f87e51b1640 1 --2- 172.21.15.22:0/275002218 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87e01000c0 0x7f87e0194ff0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f87d002c210 tx=0x7f87d0004f00 comp rx=0 tx=0).stop 2024-02-16T19:26:45.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.772+0000 7f87e51b1640 1 -- 172.21.15.22:0/275002218 shutdown_connections 2024-02-16T19:26:45.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.772+0000 7f87e51b1640 1 --2- 172.21.15.22:0/275002218 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f87c806c5f0 0x7f87c806eaa0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:45.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.772+0000 7f87e51b1640 1 --2- 172.21.15.22:0/275002218 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87e01029e0 0x7f87e0195530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:45.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.772+0000 7f87e51b1640 1 --2- 172.21.15.22:0/275002218 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87e01000c0 0x7f87e0194ff0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:45.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.772+0000 7f87e51b1640 1 -- 172.21.15.22:0/275002218 >> 172.21.15.22:0/275002218 conn(0x7f87e00f9a80 msgr2=0x7f87e00fbef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:26:45.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.772+0000 7f87e51b1640 1 -- 172.21.15.22:0/275002218 shutdown_connections 2024-02-16T19:26:45.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.772+0000 7f87e51b1640 1 -- 172.21.15.22:0/275002218 wait complete. 2024-02-16T19:26:45.784 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-16T19:26:45.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.885+0000 7f86d3d77640 1 -- 172.21.15.22:0/3433210723 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86cc0fe530 msgr2=0x7f86cc0fe940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:45.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.885+0000 7f86d3d77640 1 --2- 172.21.15.22:0/3433210723 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86cc0fe530 0x7f86cc0fe940 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f86c8009880 tx=0x7f86c802bce0 comp rx=0 tx=0).stop 2024-02-16T19:26:45.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.885+0000 7f86d3d77640 1 -- 172.21.15.22:0/3433210723 shutdown_connections 2024-02-16T19:26:45.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.885+0000 7f86d3d77640 1 --2- 172.21.15.22:0/3433210723 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86cc0ff720 0x7f86cc0ffb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:45.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.885+0000 7f86d3d77640 1 --2- 172.21.15.22:0/3433210723 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86cc0fe530 0x7f86cc0fe940 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:45.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.885+0000 7f86d3d77640 1 -- 172.21.15.22:0/3433210723 >> 172.21.15.22:0/3433210723 conn(0x7f86cc0f9a90 msgr2=0x7f86cc0fbee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:26:45.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.885+0000 7f86d3d77640 1 -- 172.21.15.22:0/3433210723 shutdown_connections 2024-02-16T19:26:45.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.885+0000 7f86d3d77640 1 -- 172.21.15.22:0/3433210723 wait complete. 2024-02-16T19:26:45.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.886+0000 7f86d3d77640 1 Processor -- start 2024-02-16T19:26:45.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.886+0000 7f86d3d77640 1 -- start start 2024-02-16T19:26:45.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.886+0000 7f86d3d77640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86cc0fe530 0x7f86cc197110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:45.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.886+0000 7f86d3d77640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86cc0ff720 0x7f86cc197650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:45.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.886+0000 7f86d3d77640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86cc197c20 con 0x7f86cc0ff720 2024-02-16T19:26:45.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.886+0000 7f86d3d77640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86cc197d90 con 0x7f86cc0fe530 2024-02-16T19:26:45.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.886+0000 7f86d1312640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86cc0ff720 0x7f86cc197650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:45.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.886+0000 7f86d1312640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86cc0ff720 0x7f86cc197650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42952/0 (socket says 172.21.15.22:42952) 2024-02-16T19:26:45.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.886+0000 7f86d1312640 1 -- 172.21.15.22:0/3062682816 learned_addr learned my addr 172.21.15.22:0/3062682816 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:26:45.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.886+0000 7f86d1b13640 1 --2- 172.21.15.22:0/3062682816 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86cc0fe530 0x7f86cc197110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:45.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.886+0000 7f86d1312640 1 -- 172.21.15.22:0/3062682816 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86cc0fe530 msgr2=0x7f86cc197110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:45.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.886+0000 7f86d1312640 1 --2- 172.21.15.22:0/3062682816 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86cc0fe530 0x7f86cc197110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:45.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.886+0000 7f86d1312640 1 -- 172.21.15.22:0/3062682816 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86c8009560 con 0x7f86cc0ff720 2024-02-16T19:26:45.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.886+0000 7f86d1312640 1 --2- 172.21.15.22:0/3062682816 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86cc0ff720 0x7f86cc197650 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f86bc00d6f0 tx=0x7f86bc00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:26:45.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.886+0000 7f86c2ffd640 1 -- 172.21.15.22:0/3062682816 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f86bc009a30 con 0x7f86cc0ff720 2024-02-16T19:26:45.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.886+0000 7f86d3d77640 1 -- 172.21.15.22:0/3062682816 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86cc19c820 con 0x7f86cc0ff720 2024-02-16T19:26:45.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.886+0000 7f86c2ffd640 1 -- 172.21.15.22:0/3062682816 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f86bc0054d0 con 0x7f86cc0ff720 2024-02-16T19:26:45.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.887+0000 7f86c2ffd640 1 -- 172.21.15.22:0/3062682816 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f86bc010460 con 0x7f86cc0ff720 2024-02-16T19:26:45.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.887+0000 7f86d3d77640 1 -- 172.21.15.22:0/3062682816 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86cc19cd80 con 0x7f86cc0ff720 2024-02-16T19:26:45.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.888+0000 7f86d3d77640 1 -- 172.21.15.22:0/3062682816 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86cc107260 con 0x7f86cc0ff720 2024-02-16T19:26:45.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.888+0000 7f86c2ffd640 1 -- 172.21.15.22:0/3062682816 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f86bc009b90 con 0x7f86cc0ff720 2024-02-16T19:26:45.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.888+0000 7f86c2ffd640 1 --2- 172.21.15.22:0/3062682816 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f86b806c6b0 0x7f86b806eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:45.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.888+0000 7f86c2ffd640 1 -- 172.21.15.22:0/3062682816 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f86bc08b3d0 con 0x7f86cc0ff720 2024-02-16T19:26:45.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.888+0000 7f86d1b13640 1 --2- 172.21.15.22:0/3062682816 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f86b806c6b0 0x7f86b806eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:45.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.889+0000 7f86d1b13640 1 --2- 172.21.15.22:0/3062682816 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f86b806c6b0 0x7f86b806eb60 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f86c8000c00 tx=0x7f86c8005fd0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:26:45.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:45.893+0000 7f86c2ffd640 1 -- 172.21.15.22:0/3062682816 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f86bc055b00 con 0x7f86cc0ff720 2024-02-16T19:26:46.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:46.114+0000 7f86d3d77640 1 -- 172.21.15.22:0/3062682816 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f86cc19cfa0 con 0x7f86cc0ff720 2024-02-16T19:26:46.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:46.114+0000 7f86c2ffd640 1 -- 172.21.15.22:0/3062682816 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4739 (secure 0 0 0) 0x7f86bc059120 con 0x7f86cc0ff720 2024-02-16T19:26:46.115 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:26:46.115 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14500,"name":"cephfs.smithi022.ocukvs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6827/242865843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":242865843},{"type":"v1","addr":"172.21.15.22:6827","nonce":242865843}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":24259,"name":"cephfs.smithi117.ibmppn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.117:6825/565408102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":565408102},{"type":"v1","addr":"172.21.15.117:6825","nonce":565408102}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":24265,"name":"cephfs.smithi117.orzors","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.117:6827/3122342028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":3122342028},{"type":"v1","addr":"172.21.15.117:6827","nonce":3122342028}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T19:26:17.873644+0000","modified":"2024-02-16T19:26:35.943353+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14490":{"gid":14490,"name":"cephfs.smithi022.btppwm","rank":0,"incarnation":12,"state":"up:active","state_seq":5,"addr":"172.21.15.22:6829/3597685751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3597685751},{"type":"v1","addr":"172.21.15.22:6829","nonce":3597685751}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T19:26:46.116 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 15 2024-02-16T19:26:46.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:46.116+0000 7f86d3d77640 1 -- 172.21.15.22:0/3062682816 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f86b806c6b0 msgr2=0x7f86b806eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:46.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:46.116+0000 7f86d3d77640 1 --2- 172.21.15.22:0/3062682816 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f86b806c6b0 0x7f86b806eb60 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f86c8000c00 tx=0x7f86c8005fd0 comp rx=0 tx=0).stop 2024-02-16T19:26:46.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:46.116+0000 7f86d3d77640 1 -- 172.21.15.22:0/3062682816 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86cc0ff720 msgr2=0x7f86cc197650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:46.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:46.116+0000 7f86d3d77640 1 --2- 172.21.15.22:0/3062682816 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86cc0ff720 0x7f86cc197650 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f86bc00d6f0 tx=0x7f86bc00dbb0 comp rx=0 tx=0).stop 2024-02-16T19:26:46.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:46.116+0000 7f86d3d77640 1 -- 172.21.15.22:0/3062682816 shutdown_connections 2024-02-16T19:26:46.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:46.116+0000 7f86d3d77640 1 --2- 172.21.15.22:0/3062682816 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86cc0ff720 0x7f86cc197650 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:46.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:46.116+0000 7f86d3d77640 1 --2- 172.21.15.22:0/3062682816 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f86b806c6b0 0x7f86b806eb60 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:46.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:46.116+0000 7f86d3d77640 1 --2- 172.21.15.22:0/3062682816 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86cc0fe530 0x7f86cc197110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:46.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:46.116+0000 7f86d3d77640 1 -- 172.21.15.22:0/3062682816 >> 172.21.15.22:0/3062682816 conn(0x7f86cc0f9a90 msgr2=0x7f86cc068410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:26:46.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:46.116+0000 7f86d3d77640 1 -- 172.21.15.22:0/3062682816 shutdown_connections 2024-02-16T19:26:46.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:46.116+0000 7f86d3d77640 1 -- 172.21.15.22:0/3062682816 wait complete. 2024-02-16T19:26:46.127 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:45 smithi022 ceph-mon[29221]: pgmap v194: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 6.9 KiB/s rd, 0 B/s wr, 6 op/s 2024-02-16T19:26:46.127 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:45 smithi022 ceph-mon[29221]: from='client.? 172.21.15.22:0/851257052' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-16T19:26:46.127 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:45 smithi022 ceph-mon[29221]: from='client.? 172.21.15.22:0/976569272' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-16T19:26:46.127 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:45 smithi022 ceph-mon[29221]: from='client.? 172.21.15.22:0/2269084230' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-02-16T19:26:46.127 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:45 smithi022 ceph-mon[29221]: from='client.? 172.21.15.22:0/275002218' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-02-16T19:26:46.128 DEBUG:tasks.fs:fs fscid=1,name=cephfs state = {'epoch': 15, 'max_mds': 1, 'flags': 18} 2024-02-16T19:26:46.128 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-02-16T19:26:46.155 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-02-16T19:26:46.155 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.1': {}} 2024-02-16T19:26:46.156 INFO:tasks.ceph_fuse:client.0 config is {} 2024-02-16T19:26:46.156 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-02-16T19:26:46.156 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-02-16T19:26:46.156 INFO:tasks.ceph_fuse:client.1 config is {} 2024-02-16T19:26:46.156 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-02-16T19:26:46.156 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-02-16T19:26:46.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:26:46.156 DEBUG:teuthology.orchestra.run.smithi117:> ip netns list 2024-02-16T19:26:46.157 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:45 smithi117 ceph-mon[35309]: pgmap v194: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 6.9 KiB/s rd, 0 B/s wr, 6 op/s 2024-02-16T19:26:46.157 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:45 smithi117 ceph-mon[35309]: from='client.? 172.21.15.22:0/851257052' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-16T19:26:46.157 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:45 smithi117 ceph-mon[35309]: from='client.? 172.21.15.22:0/976569272' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-16T19:26:46.157 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:45 smithi117 ceph-mon[35309]: from='client.? 172.21.15.22:0/2269084230' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-02-16T19:26:46.157 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:45 smithi117 ceph-mon[35309]: from='client.? 172.21.15.22:0/275002218' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-02-16T19:26:46.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:26:46.176 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link delete ceph-brx 2024-02-16T19:26:46.252 INFO:teuthology.orchestra.run.smithi117.stderr:Cannot find device "ceph-brx" 2024-02-16T19:26:46.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-16T19:26:46.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:26:46.253 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list 2024-02-16T19:26:46.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:26:46.268 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link delete ceph-brx 2024-02-16T19:26:46.337 INFO:teuthology.orchestra.run.smithi022.stderr:Cannot find device "ceph-brx" 2024-02-16T19:26:46.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-16T19:26:46.339 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-02-16T19:26:46.339 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -- ceph fs ls 2024-02-16T19:26:46.724 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi022/config 2024-02-16T19:26:47.091 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:46 smithi022 ceph-mon[29221]: from='client.? 172.21.15.22:0/3062682816' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-16T19:26:47.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:46 smithi117 ceph-mon[35309]: from='client.? 172.21.15.22:0/3062682816' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-16T19:26:48.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:47 smithi117 ceph-mon[35309]: pgmap v195: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 6.8 KiB/s rd, 0 B/s wr, 6 op/s 2024-02-16T19:26:48.144 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:47 smithi022 ceph-mon[29221]: pgmap v195: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 6.8 KiB/s rd, 0 B/s wr, 6 op/s 2024-02-16T19:26:48.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:48.354+0000 7fef5e1f8700 1 -- 172.21.15.22:0/2087688911 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fef580ff940 msgr2=0x7fef5810c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:48.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:48.354+0000 7fef5e1f8700 1 --2- 172.21.15.22:0/2087688911 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fef580ff940 0x7fef5810c6a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fef40009910 tx=0x7fef40009c20 comp rx=0 tx=0).stop 2024-02-16T19:26:48.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:48.354+0000 7fef5e1f8700 1 -- 172.21.15.22:0/2087688911 shutdown_connections 2024-02-16T19:26:48.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:48.354+0000 7fef5e1f8700 1 --2- 172.21.15.22:0/2087688911 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fef580ff940 0x7fef5810c6a0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:48.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:48.354+0000 7fef5e1f8700 1 --2- 172.21.15.22:0/2087688911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef580ff030 0x7fef580ff400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:48.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:48.354+0000 7fef5e1f8700 1 -- 172.21.15.22:0/2087688911 >> 172.21.15.22:0/2087688911 conn(0x7fef580fac80 msgr2=0x7fef580fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:26:48.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:48.354+0000 7fef5e1f8700 1 -- 172.21.15.22:0/2087688911 shutdown_connections 2024-02-16T19:26:48.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:48.354+0000 7fef5e1f8700 1 -- 172.21.15.22:0/2087688911 wait complete. 2024-02-16T19:26:48.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:48.354+0000 7fef5e1f8700 1 Processor -- start 2024-02-16T19:26:48.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:48.355+0000 7fef5e1f8700 1 -- start start 2024-02-16T19:26:48.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:48.355+0000 7fef5e1f8700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fef580ff030 0x7fef58198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:48.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:48.355+0000 7fef5e1f8700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef580ff940 0x7fef581986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:48.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:48.355+0000 7fef5e1f8700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef58198d90 con 0x7fef580ff030 2024-02-16T19:26:48.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:48.355+0000 7fef5e1f8700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef5819cb20 con 0x7fef580ff940 2024-02-16T19:26:48.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:48.355+0000 7fef577fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fef580ff030 0x7fef58198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:48.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:48.355+0000 7fef577fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fef580ff030 0x7fef58198170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42966/0 (socket says 172.21.15.22:42966) 2024-02-16T19:26:48.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:48.355+0000 7fef577fe700 1 -- 172.21.15.22:0/4139262937 learned_addr learned my addr 172.21.15.22:0/4139262937 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:26:48.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:48.355+0000 7fef4effd700 1 --2- 172.21.15.22:0/4139262937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef580ff940 0x7fef581986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:48.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:48.355+0000 7fef577fe700 1 -- 172.21.15.22:0/4139262937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef580ff940 msgr2=0x7fef581986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:48.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:48.355+0000 7fef577fe700 1 --2- 172.21.15.22:0/4139262937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef580ff940 0x7fef581986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:48.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:48.355+0000 7fef577fe700 1 -- 172.21.15.22:0/4139262937 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef400095f0 con 0x7fef580ff030 2024-02-16T19:26:48.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:48.356+0000 7fef577fe700 1 --2- 172.21.15.22:0/4139262937 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fef580ff030 0x7fef58198170 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fef4800e920 tx=0x7fef4800ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:26:48.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:48.356+0000 7fef557fa700 1 -- 172.21.15.22:0/4139262937 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef48009790 con 0x7fef580ff030 2024-02-16T19:26:48.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:48.356+0000 7fef5e1f8700 1 -- 172.21.15.22:0/4139262937 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef5819ce00 con 0x7fef580ff030 2024-02-16T19:26:48.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:48.356+0000 7fef557fa700 1 -- 172.21.15.22:0/4139262937 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fef4800cba0 con 0x7fef580ff030 2024-02-16T19:26:48.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:48.356+0000 7fef557fa700 1 -- 172.21.15.22:0/4139262937 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef48018630 con 0x7fef580ff030 2024-02-16T19:26:48.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:48.356+0000 7fef5e1f8700 1 -- 172.21.15.22:0/4139262937 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef5819d350 con 0x7fef580ff030 2024-02-16T19:26:48.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:48.357+0000 7fef5e1f8700 1 -- 172.21.15.22:0/4139262937 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef58109e10 con 0x7fef580ff030 2024-02-16T19:26:48.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:48.357+0000 7fef557fa700 1 -- 172.21.15.22:0/4139262937 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fef480188f0 con 0x7fef580ff030 2024-02-16T19:26:48.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:48.358+0000 7fef557fa700 1 --2- 172.21.15.22:0/4139262937 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fef4406c680 0x7fef4406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:48.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:48.358+0000 7fef557fa700 1 -- 172.21.15.22:0/4139262937 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fef48014070 con 0x7fef580ff030 2024-02-16T19:26:48.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:48.358+0000 7fef4effd700 1 --2- 172.21.15.22:0/4139262937 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fef4406c680 0x7fef4406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:48.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:48.358+0000 7fef4effd700 1 --2- 172.21.15.22:0/4139262937 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fef4406c680 0x7fef4406eb30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fef400059d0 tx=0x7fef40005940 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:26:48.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:48.363+0000 7fef557fa700 1 -- 172.21.15.22:0/4139262937 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fef48051720 con 0x7fef580ff030 2024-02-16T19:26:48.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:48.613+0000 7fef5e1f8700 1 -- 172.21.15.22:0/4139262937 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fef5804f2a0 con 0x7fef580ff030 2024-02-16T19:26:48.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:48.614+0000 7fef557fa700 1 -- 172.21.15.22:0/4139262937 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v15) v1 ==== 53+0+83 (secure 0 0 0) 0x7fef48091020 con 0x7fef580ff030 2024-02-16T19:26:48.616 INFO:teuthology.orchestra.run.smithi022.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-02-16T19:26:48.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:48.616+0000 7fef5e1f8700 1 -- 172.21.15.22:0/4139262937 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fef4406c680 msgr2=0x7fef4406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:48.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:48.616+0000 7fef5e1f8700 1 --2- 172.21.15.22:0/4139262937 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fef4406c680 0x7fef4406eb30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fef400059d0 tx=0x7fef40005940 comp rx=0 tx=0).stop 2024-02-16T19:26:48.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:48.616+0000 7fef5e1f8700 1 -- 172.21.15.22:0/4139262937 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fef580ff030 msgr2=0x7fef58198170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:48.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:48.616+0000 7fef5e1f8700 1 --2- 172.21.15.22:0/4139262937 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fef580ff030 0x7fef58198170 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fef4800e920 tx=0x7fef4800ece0 comp rx=0 tx=0).stop 2024-02-16T19:26:48.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:48.616+0000 7fef5e1f8700 1 -- 172.21.15.22:0/4139262937 shutdown_connections 2024-02-16T19:26:48.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:48.616+0000 7fef5e1f8700 1 --2- 172.21.15.22:0/4139262937 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fef580ff030 0x7fef58198170 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:48.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:48.616+0000 7fef5e1f8700 1 --2- 172.21.15.22:0/4139262937 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fef4406c680 0x7fef4406eb30 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:48.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:48.616+0000 7fef5e1f8700 1 --2- 172.21.15.22:0/4139262937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef580ff940 0x7fef581986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:48.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:48.616+0000 7fef5e1f8700 1 -- 172.21.15.22:0/4139262937 >> 172.21.15.22:0/4139262937 conn(0x7fef580fac80 msgr2=0x7fef580fc3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:26:48.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:48.616+0000 7fef5e1f8700 1 -- 172.21.15.22:0/4139262937 shutdown_connections 2024-02-16T19:26:48.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:48.616+0000 7fef5e1f8700 1 -- 172.21.15.22:0/4139262937 wait complete. 2024-02-16T19:26:49.215 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-02-16T19:26:49.215 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-16T19:26:49.215 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi022.front.sepia.ceph.com 2024-02-16T19:26:49.215 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-02-16T19:26:49.215 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-02-16T19:26:49.215 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-02-16T19:26:49.215 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-02-16T19:26:49.216 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-02-16T19:26:49.216 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-02-16T19:26:49.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:26:49.216 DEBUG:teuthology.orchestra.run.smithi022:> ip addr 2024-02-16T19:26:49.232 INFO:teuthology.orchestra.run.smithi022.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-16T19:26:49.232 INFO:teuthology.orchestra.run.smithi022.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-16T19:26:49.232 INFO:teuthology.orchestra.run.smithi022.stdout: inet 127.0.0.1/8 scope host lo 2024-02-16T19:26:49.232 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-02-16T19:26:49.232 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 ::1/128 scope host 2024-02-16T19:26:49.232 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-02-16T19:26:49.232 INFO:teuthology.orchestra.run.smithi022.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-16T19:26:49.232 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9a brd ff:ff:ff:ff:ff:ff 2024-02-16T19:26:49.232 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp7s0f0 2024-02-16T19:26:49.233 INFO:teuthology.orchestra.run.smithi022.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-16T19:26:49.233 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9b brd ff:ff:ff:ff:ff:ff 2024-02-16T19:26:49.233 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp7s0f1 2024-02-16T19:26:49.233 INFO:teuthology.orchestra.run.smithi022.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-16T19:26:49.233 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e4 brd ff:ff:ff:ff:ff:ff 2024-02-16T19:26:49.233 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp1s0f0 2024-02-16T19:26:49.233 INFO:teuthology.orchestra.run.smithi022.stdout: inet 172.21.15.22/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-02-16T19:26:49.233 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft 42212sec preferred_lft 42212sec 2024-02-16T19:26:49.233 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::ec4:7aff:fe59:5de4/64 scope link noprefixroute 2024-02-16T19:26:49.233 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-02-16T19:26:49.233 INFO:teuthology.orchestra.run.smithi022.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-16T19:26:49.233 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e5 brd ff:ff:ff:ff:ff:ff 2024-02-16T19:26:49.233 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp1s0f1 2024-02-16T19:26:49.234 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-02-16T19:26:49.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:26:49.234 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec bash -c ' 2024-02-16T19:26:49.234 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2024-02-16T19:26:49.234 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link add name ceph-brx type bridge 2024-02-16T19:26:49.234 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip addr flush dev ceph-brx 2024-02-16T19:26:49.234 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link set ceph-brx up 2024-02-16T19:26:49.234 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-02-16T19:26:49.234 DEBUG:teuthology.orchestra.run.smithi022:> ') 2024-02-16T19:26:49.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:26:49.400 DEBUG:teuthology.orchestra.run.smithi022:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-16T19:26:49.427 INFO:teuthology.orchestra.run.smithi022.stdout:1 2024-02-16T19:26:49.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:26:49.429 DEBUG:teuthology.orchestra.run.smithi022:> route 2024-02-16T19:26:49.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:49 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:26:49.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:49 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:26:49.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:49 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:26:49.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:49 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:26:49.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:49 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:26:49.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:49 smithi022 ceph-mon[29221]: from='client.? 172.21.15.22:0/4139262937' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-16T19:26:49.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:49 smithi022 ceph-mon[29221]: pgmap v196: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 3.8 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-16T19:26:49.509 INFO:teuthology.orchestra.run.smithi022.stdout:Kernel IP routing table 2024-02-16T19:26:49.509 INFO:teuthology.orchestra.run.smithi022.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-16T19:26:49.509 INFO:teuthology.orchestra.run.smithi022.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-02-16T19:26:49.509 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-02-16T19:26:49.509 INFO:teuthology.orchestra.run.smithi022.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-16T19:26:49.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:26:49.510 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec bash -c ' 2024-02-16T19:26:49.510 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2024-02-16T19:26:49.510 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-02-16T19:26:49.510 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-02-16T19:26:49.510 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-02-16T19:26:49.510 DEBUG:teuthology.orchestra.run.smithi022:> ') 2024-02-16T19:26:49.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:26:49.772 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list 2024-02-16T19:26:49.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:26:49.787 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list-id 2024-02-16T19:26:49.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:26:49.841 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec bash -c ' 2024-02-16T19:26:49.841 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2024-02-16T19:26:49.841 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-16T19:26:49.841 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-02-16T19:26:49.841 DEBUG:teuthology.orchestra.run.smithi022:> ') 2024-02-16T19:26:49.886 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:49 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:26:49.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:49 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:26:49.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:49 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:26:49.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:49 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:26:49.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:49 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:26:49.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:49 smithi117 ceph-mon[35309]: from='client.? 172.21.15.22:0/4139262937' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-16T19:26:49.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:49 smithi117 ceph-mon[35309]: pgmap v196: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 3.8 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-16T19:26:49.924 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-02-16T19:26:49.924 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:26:49.924 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec bash -c ' 2024-02-16T19:26:49.925 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2024-02-16T19:26:49.925 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-02-16T19:26:49.925 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-02-16T19:26:49.925 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-02-16T19:26:49.925 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-02-16T19:26:49.925 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-02-16T19:26:49.925 DEBUG:teuthology.orchestra.run.smithi022:> ') 2024-02-16T19:26:50.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:26:50.071 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec bash -c ' 2024-02-16T19:26:50.071 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2024-02-16T19:26:50.071 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link set brx.0 up 2024-02-16T19:26:50.071 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link set dev brx.0 master ceph-brx 2024-02-16T19:26:50.071 DEBUG:teuthology.orchestra.run.smithi022:> ') 2024-02-16T19:26:50.114 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-02-16T19:26:50.114 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-16T19:26:50.114 DEBUG:teuthology.orchestra.run.smithi022:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-02-16T19:26:50.168 INFO:teuthology.orchestra.run.smithi022.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-02-16T19:26:50.168 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-16T19:26:50.169 DEBUG:teuthology.orchestra.run.smithi022:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-02-16T19:26:50.223 DEBUG:teuthology.orchestra.run.smithi022:> sudo modprobe fuse 2024-02-16T19:26:50.290 DEBUG:teuthology.orchestra.run.smithi022:> cat /proc/self/mounts | awk '{print $2}' 2024-02-16T19:26:50.346 INFO:teuthology.orchestra.run.smithi022.stdout:/proc 2024-02-16T19:26:50.346 INFO:teuthology.orchestra.run.smithi022.stdout:/sys 2024-02-16T19:26:50.346 INFO:teuthology.orchestra.run.smithi022.stdout:/dev 2024-02-16T19:26:50.346 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/security 2024-02-16T19:26:50.346 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/shm 2024-02-16T19:26:50.346 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/pts 2024-02-16T19:26:50.346 INFO:teuthology.orchestra.run.smithi022.stdout:/run 2024-02-16T19:26:50.347 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/cgroup 2024-02-16T19:26:50.347 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/pstore 2024-02-16T19:26:50.347 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/bpf 2024-02-16T19:26:50.347 INFO:teuthology.orchestra.run.smithi022.stdout:/ 2024-02-16T19:26:50.347 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/selinux 2024-02-16T19:26:50.347 INFO:teuthology.orchestra.run.smithi022.stdout:/proc/sys/fs/binfmt_misc 2024-02-16T19:26:50.347 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/tracing 2024-02-16T19:26:50.347 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/hugepages 2024-02-16T19:26:50.347 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/debug 2024-02-16T19:26:50.347 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/mqueue 2024-02-16T19:26:50.347 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/fuse/connections 2024-02-16T19:26:50.347 INFO:teuthology.orchestra.run.smithi022.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-02-16T19:26:50.347 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/config 2024-02-16T19:26:50.347 INFO:teuthology.orchestra.run.smithi022.stdout:/run/credentials/systemd-sysctl.service 2024-02-16T19:26:50.348 INFO:teuthology.orchestra.run.smithi022.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-02-16T19:26:50.348 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/nfs/rpc_pipefs 2024-02-16T19:26:50.348 INFO:teuthology.orchestra.run.smithi022.stdout:/run/user/1000 2024-02-16T19:26:50.348 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph 2024-02-16T19:26:50.348 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay 2024-02-16T19:26:50.348 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay/779e5ad49c23f8737e2e1cb1a3afdddd67954a1d9add5670267fe35b7aa5ee88/merged 2024-02-16T19:26:50.348 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay/5bd75dab3529fda51ff4f07670e96d6ccb7554fd8901bcdf1df1f87c5e66a98e/merged 2024-02-16T19:26:50.348 INFO:teuthology.orchestra.run.smithi022.stdout:/run/user/0 2024-02-16T19:26:50.348 INFO:teuthology.orchestra.run.smithi022.stdout:/proc/sys/fs/binfmt_misc 2024-02-16T19:26:50.348 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay/0732eb03dd1af7c2fffb5008ba4292e4b64b5d4c77ef1e8c46510d79a7953a0a/merged 2024-02-16T19:26:50.348 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay/782929ed89ecb4ab332a75d2ddaa1b9ba1f9b638c22487385abfcbc6ad2f3d3f/merged 2024-02-16T19:26:50.348 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay/d7a9327fbcfb0837cf086427871ac7f67cb2dd3e1fc798e3de9e902640d8d01a/merged 2024-02-16T19:26:50.348 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay/838b0e0b475f1da25e3b8f7ec63b83aa51fd9afed5ec6c69fd3e55aa0a0e4e0f/merged 2024-02-16T19:26:50.348 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay/e5513881a353bcb1c566b8e9c3807b3baa26ab93dee8e060ac7262111eff78e9/merged 2024-02-16T19:26:50.349 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay/50ed921ce470159be421bd32781570fe45ec4bfab23b6a9e7c5aeccee2b1d2cb/merged 2024-02-16T19:26:50.349 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay/aac65383523c04e641427d9d37d68b1d1ef5e8f8cba0bcc064c798e604b397e2/merged 2024-02-16T19:26:50.349 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay/b89becca6c82ef5a084c850d2bb9a88abe2c5af7f2f19a6c1b328a8b76f1e6b4/merged 2024-02-16T19:26:50.349 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay/1a7068743fbe5e41786c05a76cf924af46161994f396971c131dda7db1521eb8/merged 2024-02-16T19:26:50.349 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay/6e77c4ef5fde3cebf48b5dc1ff62f86f802c57857b2564401f0fa3e297200b82/merged 2024-02-16T19:26:50.349 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay/7dba9aa470789eb6cb79c7cd4172cdcd307339b0a471d24e39c798f6b738d2d9/merged 2024-02-16T19:26:50.349 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns 2024-02-16T19:26:50.349 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-16T19:26:50.349 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-16T19:26:50.349 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:26:50.349 DEBUG:teuthology.orchestra.run.smithi022:> ls /sys/fs/fuse/connections 2024-02-16T19:26:50.401 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-02-16T19:26:50.401 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-02-16T19:26:50.443 DEBUG:teuthology.orchestra.run.smithi022:> sudo modprobe fuse 2024-02-16T19:26:50.469 DEBUG:teuthology.orchestra.run.smithi022:> cat /proc/self/mounts | awk '{print $2}' 2024-02-16T19:26:50.525 INFO:teuthology.orchestra.run.smithi022.stdout:/proc 2024-02-16T19:26:50.526 INFO:teuthology.orchestra.run.smithi022.stdout:/sys 2024-02-16T19:26:50.526 INFO:teuthology.orchestra.run.smithi022.stdout:/dev 2024-02-16T19:26:50.526 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/security 2024-02-16T19:26:50.526 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/shm 2024-02-16T19:26:50.526 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/pts 2024-02-16T19:26:50.526 INFO:teuthology.orchestra.run.smithi022.stdout:/run 2024-02-16T19:26:50.526 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/cgroup 2024-02-16T19:26:50.526 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/pstore 2024-02-16T19:26:50.526 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/bpf 2024-02-16T19:26:50.526 INFO:teuthology.orchestra.run.smithi022.stdout:/ 2024-02-16T19:26:50.526 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/selinux 2024-02-16T19:26:50.526 INFO:teuthology.orchestra.run.smithi022.stdout:/proc/sys/fs/binfmt_misc 2024-02-16T19:26:50.526 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/tracing 2024-02-16T19:26:50.526 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/hugepages 2024-02-16T19:26:50.527 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/debug 2024-02-16T19:26:50.527 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/mqueue 2024-02-16T19:26:50.527 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/fuse/connections 2024-02-16T19:26:50.527 INFO:teuthology.orchestra.run.smithi022.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-02-16T19:26:50.527 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/config 2024-02-16T19:26:50.527 INFO:teuthology.orchestra.run.smithi022.stdout:/run/credentials/systemd-sysctl.service 2024-02-16T19:26:50.527 INFO:teuthology.orchestra.run.smithi022.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-02-16T19:26:50.527 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/nfs/rpc_pipefs 2024-02-16T19:26:50.527 INFO:teuthology.orchestra.run.smithi022.stdout:/run/user/1000 2024-02-16T19:26:50.527 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph 2024-02-16T19:26:50.527 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay 2024-02-16T19:26:50.527 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay/779e5ad49c23f8737e2e1cb1a3afdddd67954a1d9add5670267fe35b7aa5ee88/merged 2024-02-16T19:26:50.527 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay/5bd75dab3529fda51ff4f07670e96d6ccb7554fd8901bcdf1df1f87c5e66a98e/merged 2024-02-16T19:26:50.527 INFO:teuthology.orchestra.run.smithi022.stdout:/run/user/0 2024-02-16T19:26:50.528 INFO:teuthology.orchestra.run.smithi022.stdout:/proc/sys/fs/binfmt_misc 2024-02-16T19:26:50.528 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay/0732eb03dd1af7c2fffb5008ba4292e4b64b5d4c77ef1e8c46510d79a7953a0a/merged 2024-02-16T19:26:50.528 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay/782929ed89ecb4ab332a75d2ddaa1b9ba1f9b638c22487385abfcbc6ad2f3d3f/merged 2024-02-16T19:26:50.528 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay/d7a9327fbcfb0837cf086427871ac7f67cb2dd3e1fc798e3de9e902640d8d01a/merged 2024-02-16T19:26:50.528 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay/838b0e0b475f1da25e3b8f7ec63b83aa51fd9afed5ec6c69fd3e55aa0a0e4e0f/merged 2024-02-16T19:26:50.528 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay/e5513881a353bcb1c566b8e9c3807b3baa26ab93dee8e060ac7262111eff78e9/merged 2024-02-16T19:26:50.528 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay/50ed921ce470159be421bd32781570fe45ec4bfab23b6a9e7c5aeccee2b1d2cb/merged 2024-02-16T19:26:50.528 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay/aac65383523c04e641427d9d37d68b1d1ef5e8f8cba0bcc064c798e604b397e2/merged 2024-02-16T19:26:50.528 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay/b89becca6c82ef5a084c850d2bb9a88abe2c5af7f2f19a6c1b328a8b76f1e6b4/merged 2024-02-16T19:26:50.528 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay/1a7068743fbe5e41786c05a76cf924af46161994f396971c131dda7db1521eb8/merged 2024-02-16T19:26:50.528 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay/6e77c4ef5fde3cebf48b5dc1ff62f86f802c57857b2564401f0fa3e297200b82/merged 2024-02-16T19:26:50.528 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay/7dba9aa470789eb6cb79c7cd4172cdcd307339b0a471d24e39c798f6b738d2d9/merged 2024-02-16T19:26:50.528 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns 2024-02-16T19:26:50.528 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-16T19:26:50.528 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-16T19:26:50.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:26:50.529 DEBUG:teuthology.orchestra.run.smithi022:> ls /sys/fs/fuse/connections 2024-02-16T19:26:50.568 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi022.stderr:2024-02-16T19:26:50.567+0000 7f3f6e380480 -1 init, newargv = 0x55f343dc1380 newargc=15 2024-02-16T19:26:50.568 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi022.stderr:ceph-fuse[67263]: starting ceph client 2024-02-16T19:26:50.573 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi022.stderr:ceph-fuse[67263]: starting fuse 2024-02-16T19:26:50.581 INFO:teuthology.orchestra.run.smithi022.stdout:148 2024-02-16T19:26:50.581 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [148] 2024-02-16T19:26:50.581 DEBUG:teuthology.orchestra.run.smithi022:> sudo stdin-killer -- python3 -c ' 2024-02-16T19:26:50.581 DEBUG:teuthology.orchestra.run.smithi022:> import glob 2024-02-16T19:26:50.581 DEBUG:teuthology.orchestra.run.smithi022:> import re 2024-02-16T19:26:50.581 DEBUG:teuthology.orchestra.run.smithi022:> import os 2024-02-16T19:26:50.581 DEBUG:teuthology.orchestra.run.smithi022:> import subprocess 2024-02-16T19:26:50.582 DEBUG:teuthology.orchestra.run.smithi022:> 2024-02-16T19:26:50.582 DEBUG:teuthology.orchestra.run.smithi022:> def _find_admin_socket(client_name): 2024-02-16T19:26:50.582 DEBUG:teuthology.orchestra.run.smithi022:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-02-16T19:26:50.582 DEBUG:teuthology.orchestra.run.smithi022:> files = glob.glob(asok_path) 2024-02-16T19:26:50.582 DEBUG:teuthology.orchestra.run.smithi022:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-02-16T19:26:50.582 DEBUG:teuthology.orchestra.run.smithi022:> 2024-02-16T19:26:50.582 DEBUG:teuthology.orchestra.run.smithi022:> # Given a non-glob path, it better be there 2024-02-16T19:26:50.582 DEBUG:teuthology.orchestra.run.smithi022:> if "*" not in asok_path: 2024-02-16T19:26:50.582 DEBUG:teuthology.orchestra.run.smithi022:> assert(len(files) == 1) 2024-02-16T19:26:50.582 DEBUG:teuthology.orchestra.run.smithi022:> return files[0] 2024-02-16T19:26:50.582 DEBUG:teuthology.orchestra.run.smithi022:> 2024-02-16T19:26:50.582 DEBUG:teuthology.orchestra.run.smithi022:> for f in files: 2024-02-16T19:26:50.582 DEBUG:teuthology.orchestra.run.smithi022:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-02-16T19:26:50.582 DEBUG:teuthology.orchestra.run.smithi022:> if os.path.exists("/proc/{0}".format(pid)): 2024-02-16T19:26:50.582 DEBUG:teuthology.orchestra.run.smithi022:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-02-16T19:26:50.582 DEBUG:teuthology.orchestra.run.smithi022:> contents = proc_f.read() 2024-02-16T19:26:50.583 DEBUG:teuthology.orchestra.run.smithi022:> if mountpoint in contents: 2024-02-16T19:26:50.583 DEBUG:teuthology.orchestra.run.smithi022:> return f 2024-02-16T19:26:50.583 DEBUG:teuthology.orchestra.run.smithi022:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-02-16T19:26:50.583 DEBUG:teuthology.orchestra.run.smithi022:> 2024-02-16T19:26:50.583 DEBUG:teuthology.orchestra.run.smithi022:> print(_find_admin_socket("client.0")) 2024-02-16T19:26:50.583 DEBUG:teuthology.orchestra.run.smithi022:> ' 2024-02-16T19:26:50.701 INFO:teuthology.orchestra.run.smithi022.stdout:/var/run/ceph/ceph-client.0.67263.asok 2024-02-16T19:26:50.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-02-16T19:26:50.711 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.67263.asok 2024-02-16T19:26:50.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:26:50.712 DEBUG:teuthology.orchestra.run.smithi022:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.67263.asok status 2024-02-16T19:26:50.848 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-02-16T19:26:50.848 INFO:teuthology.orchestra.run.smithi022.stdout: "metadata": { 2024-02-16T19:26:50.848 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_sha1": "5dd24139a1eada541a3bc16b6941c5dde975e26d", 2024-02-16T19:26:50.848 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_version": "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)", 2024-02-16T19:26:50.848 INFO:teuthology.orchestra.run.smithi022.stdout: "entity_id": "0", 2024-02-16T19:26:50.848 INFO:teuthology.orchestra.run.smithi022.stdout: "hostname": "smithi022", 2024-02-16T19:26:50.848 INFO:teuthology.orchestra.run.smithi022.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-02-16T19:26:50.848 INFO:teuthology.orchestra.run.smithi022.stdout: "pid": "67263", 2024-02-16T19:26:50.848 INFO:teuthology.orchestra.run.smithi022.stdout: "root": "/" 2024-02-16T19:26:50.848 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:26:50.848 INFO:teuthology.orchestra.run.smithi022.stdout: "dentry_count": 0, 2024-02-16T19:26:50.849 INFO:teuthology.orchestra.run.smithi022.stdout: "dentry_pinned_count": 0, 2024-02-16T19:26:50.849 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 14540, 2024-02-16T19:26:50.849 INFO:teuthology.orchestra.run.smithi022.stdout: "inst": { 2024-02-16T19:26:50.849 INFO:teuthology.orchestra.run.smithi022.stdout: "name": { 2024-02-16T19:26:50.849 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "client", 2024-02-16T19:26:50.849 INFO:teuthology.orchestra.run.smithi022.stdout: "num": 14540 2024-02-16T19:26:50.849 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:26:50.849 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": { 2024-02-16T19:26:50.849 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-02-16T19:26:50.849 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "192.168.0.1:0", 2024-02-16T19:26:50.849 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1593109629 2024-02-16T19:26:50.849 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-02-16T19:26:50.849 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:26:50.849 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": { 2024-02-16T19:26:50.849 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-02-16T19:26:50.850 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "192.168.0.1:0", 2024-02-16T19:26:50.850 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1593109629 2024-02-16T19:26:50.850 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:26:50.850 INFO:teuthology.orchestra.run.smithi022.stdout: "inst_str": "client.14540 192.168.0.1:0/1593109629", 2024-02-16T19:26:50.850 INFO:teuthology.orchestra.run.smithi022.stdout: "addr_str": "192.168.0.1:0/1593109629", 2024-02-16T19:26:50.850 INFO:teuthology.orchestra.run.smithi022.stdout: "inode_count": 1, 2024-02-16T19:26:50.850 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_epoch": 15, 2024-02-16T19:26:50.850 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_epoch": 47, 2024-02-16T19:26:50.850 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_epoch_barrier": 0, 2024-02-16T19:26:50.850 INFO:teuthology.orchestra.run.smithi022.stdout: "blocklisted": false, 2024-02-16T19:26:50.850 INFO:teuthology.orchestra.run.smithi022.stdout: "fs_name": "cephfs" 2024-02-16T19:26:50.850 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-02-16T19:26:50.859 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -- ceph fs ls 2024-02-16T19:26:51.070 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi022/config 2024-02-16T19:26:52.110 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:51 smithi022 ceph-mon[29221]: pgmap v197: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-16T19:26:52.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:51 smithi117 ceph-mon[35309]: pgmap v197: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-16T19:26:52.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:52.650+0000 7f5ca2508700 1 -- 172.21.15.22:0/413667474 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5c9c100e80 msgr2=0x7f5c9c104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:52.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:52.650+0000 7f5ca2508700 1 --2- 172.21.15.22:0/413667474 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5c9c100e80 0x7f5c9c104e50 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5c90009910 tx=0x7f5c90009c20 comp rx=0 tx=0).stop 2024-02-16T19:26:52.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:52.650+0000 7f5ca2508700 1 -- 172.21.15.22:0/413667474 shutdown_connections 2024-02-16T19:26:52.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:52.650+0000 7f5ca2508700 1 --2- 172.21.15.22:0/413667474 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5c9c100e80 0x7f5c9c104e50 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:52.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:52.650+0000 7f5ca2508700 1 --2- 172.21.15.22:0/413667474 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c9c1004e0 0x7f5c9c1008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:52.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:52.650+0000 7f5ca2508700 1 -- 172.21.15.22:0/413667474 >> 172.21.15.22:0/413667474 conn(0x7f5c9c0fbda0 msgr2=0x7f5c9c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:26:52.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:52.651+0000 7f5ca2508700 1 -- 172.21.15.22:0/413667474 shutdown_connections 2024-02-16T19:26:52.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:52.651+0000 7f5ca2508700 1 -- 172.21.15.22:0/413667474 wait complete. 2024-02-16T19:26:52.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:52.651+0000 7f5ca2508700 1 Processor -- start 2024-02-16T19:26:52.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:52.651+0000 7f5ca2508700 1 -- start start 2024-02-16T19:26:52.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:52.652+0000 7f5ca2508700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c9c1004e0 0x7f5c9c195f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:52.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:52.652+0000 7f5ca2508700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5c9c100e80 0x7f5c9c196450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:52.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:52.652+0000 7f5ca2508700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9c196b30 con 0x7f5c9c100e80 2024-02-16T19:26:52.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:52.652+0000 7f5ca2508700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9c19a8c0 con 0x7f5c9c1004e0 2024-02-16T19:26:52.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:52.652+0000 7f5c9b7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5c9c100e80 0x7f5c9c196450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:52.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:52.652+0000 7f5c9b7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5c9c100e80 0x7f5c9c196450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42992/0 (socket says 172.21.15.22:42992) 2024-02-16T19:26:52.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:52.652+0000 7f5c9b7fe700 1 -- 172.21.15.22:0/2518842914 learned_addr learned my addr 172.21.15.22:0/2518842914 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:26:52.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:52.652+0000 7f5c9bfff700 1 --2- 172.21.15.22:0/2518842914 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c9c1004e0 0x7f5c9c195f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:52.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:52.652+0000 7f5c9b7fe700 1 -- 172.21.15.22:0/2518842914 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c9c1004e0 msgr2=0x7f5c9c195f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:52.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:52.652+0000 7f5c9b7fe700 1 --2- 172.21.15.22:0/2518842914 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c9c1004e0 0x7f5c9c195f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:52.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:52.652+0000 7f5c9b7fe700 1 -- 172.21.15.22:0/2518842914 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c900095f0 con 0x7f5c9c100e80 2024-02-16T19:26:52.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:52.652+0000 7f5c9b7fe700 1 --2- 172.21.15.22:0/2518842914 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5c9c100e80 0x7f5c9c196450 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5c90009fd0 tx=0x7f5c9000fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:26:52.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:52.652+0000 7f5c997fa700 1 -- 172.21.15.22:0/2518842914 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c9001d070 con 0x7f5c9c100e80 2024-02-16T19:26:52.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:52.652+0000 7f5c997fa700 1 -- 172.21.15.22:0/2518842914 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5c90005640 con 0x7f5c9c100e80 2024-02-16T19:26:52.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:52.652+0000 7f5ca2508700 1 -- 172.21.15.22:0/2518842914 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c9c19ab40 con 0x7f5c9c100e80 2024-02-16T19:26:52.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:52.652+0000 7f5c997fa700 1 -- 172.21.15.22:0/2518842914 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c90017630 con 0x7f5c9c100e80 2024-02-16T19:26:52.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:52.652+0000 7f5ca2508700 1 -- 172.21.15.22:0/2518842914 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c9c19afd0 con 0x7f5c9c100e80 2024-02-16T19:26:52.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:52.653+0000 7f5ca2508700 1 -- 172.21.15.22:0/2518842914 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c9c108850 con 0x7f5c9c100e80 2024-02-16T19:26:52.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:52.654+0000 7f5c997fa700 1 -- 172.21.15.22:0/2518842914 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f5c90005080 con 0x7f5c9c100e80 2024-02-16T19:26:52.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:52.654+0000 7f5c997fa700 1 --2- 172.21.15.22:0/2518842914 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f5c8406c680 0x7f5c8406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:52.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:52.654+0000 7f5c997fa700 1 -- 172.21.15.22:0/2518842914 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f5c9008cf60 con 0x7f5c9c100e80 2024-02-16T19:26:52.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:52.654+0000 7f5c9bfff700 1 --2- 172.21.15.22:0/2518842914 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f5c8406c680 0x7f5c8406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:52.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:52.655+0000 7f5c9bfff700 1 --2- 172.21.15.22:0/2518842914 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f5c8406c680 0x7f5c8406eb30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c009c60 tx=0x7f5c8c009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:26:52.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:52.659+0000 7f5c997fa700 1 -- 172.21.15.22:0/2518842914 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f5c90057540 con 0x7f5c9c100e80 2024-02-16T19:26:52.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:52.907+0000 7f5ca2508700 1 -- 172.21.15.22:0/2518842914 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f5c9c04f2a0 con 0x7f5c9c100e80 2024-02-16T19:26:52.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:52.908+0000 7f5c997fa700 1 -- 172.21.15.22:0/2518842914 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v15) v1 ==== 53+0+83 (secure 0 0 0) 0x7f5c90005330 con 0x7f5c9c100e80 2024-02-16T19:26:52.909 INFO:teuthology.orchestra.run.smithi022.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-02-16T19:26:52.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:52.910+0000 7f5ca2508700 1 -- 172.21.15.22:0/2518842914 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f5c8406c680 msgr2=0x7f5c8406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:52.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:52.910+0000 7f5ca2508700 1 --2- 172.21.15.22:0/2518842914 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f5c8406c680 0x7f5c8406eb30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c009c60 tx=0x7f5c8c009260 comp rx=0 tx=0).stop 2024-02-16T19:26:52.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:52.910+0000 7f5ca2508700 1 -- 172.21.15.22:0/2518842914 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5c9c100e80 msgr2=0x7f5c9c196450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:52.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:52.910+0000 7f5ca2508700 1 --2- 172.21.15.22:0/2518842914 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5c9c100e80 0x7f5c9c196450 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5c90009fd0 tx=0x7f5c9000fa00 comp rx=0 tx=0).stop 2024-02-16T19:26:52.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:52.910+0000 7f5ca2508700 1 -- 172.21.15.22:0/2518842914 shutdown_connections 2024-02-16T19:26:52.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:52.910+0000 7f5ca2508700 1 --2- 172.21.15.22:0/2518842914 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f5c8406c680 0x7f5c8406eb30 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:52.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:52.910+0000 7f5ca2508700 1 --2- 172.21.15.22:0/2518842914 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c9c1004e0 0x7f5c9c195f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:52.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:52.910+0000 7f5ca2508700 1 --2- 172.21.15.22:0/2518842914 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5c9c100e80 0x7f5c9c196450 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:52.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:52.910+0000 7f5ca2508700 1 -- 172.21.15.22:0/2518842914 >> 172.21.15.22:0/2518842914 conn(0x7f5c9c0fbda0 msgr2=0x7f5c9c0fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:26:52.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:52.910+0000 7f5ca2508700 1 -- 172.21.15.22:0/2518842914 shutdown_connections 2024-02-16T19:26:52.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:52.910+0000 7f5ca2508700 1 -- 172.21.15.22:0/2518842914 wait complete. 2024-02-16T19:26:53.569 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-02-16T19:26:53.569 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-16T19:26:53.569 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi117.front.sepia.ceph.com 2024-02-16T19:26:53.569 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-02-16T19:26:53.569 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-02-16T19:26:53.569 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-02-16T19:26:53.569 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-02-16T19:26:53.570 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-02-16T19:26:53.570 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-02-16T19:26:53.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:26:53.570 DEBUG:teuthology.orchestra.run.smithi117:> ip addr 2024-02-16T19:26:53.587 INFO:teuthology.orchestra.run.smithi117.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-16T19:26:53.587 INFO:teuthology.orchestra.run.smithi117.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-16T19:26:53.587 INFO:teuthology.orchestra.run.smithi117.stdout: inet 127.0.0.1/8 scope host lo 2024-02-16T19:26:53.587 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-02-16T19:26:53.587 INFO:teuthology.orchestra.run.smithi117.stdout: inet6 ::1/128 scope host 2024-02-16T19:26:53.587 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-02-16T19:26:53.587 INFO:teuthology.orchestra.run.smithi117.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-16T19:26:53.587 INFO:teuthology.orchestra.run.smithi117.stdout: link/ether 0c:c4:7a:d8:a8:8a brd ff:ff:ff:ff:ff:ff 2024-02-16T19:26:53.587 INFO:teuthology.orchestra.run.smithi117.stdout: altname enp7s0f0 2024-02-16T19:26:53.588 INFO:teuthology.orchestra.run.smithi117.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-16T19:26:53.588 INFO:teuthology.orchestra.run.smithi117.stdout: link/ether 0c:c4:7a:d8:a8:8b brd ff:ff:ff:ff:ff:ff 2024-02-16T19:26:53.588 INFO:teuthology.orchestra.run.smithi117.stdout: altname enp7s0f1 2024-02-16T19:26:53.588 INFO:teuthology.orchestra.run.smithi117.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-16T19:26:53.588 INFO:teuthology.orchestra.run.smithi117.stdout: link/ether 0c:c4:7a:8f:44:62 brd ff:ff:ff:ff:ff:ff 2024-02-16T19:26:53.588 INFO:teuthology.orchestra.run.smithi117.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-16T19:26:53.588 INFO:teuthology.orchestra.run.smithi117.stdout: link/ether 0c:c4:7a:8f:44:63 brd ff:ff:ff:ff:ff:ff 2024-02-16T19:26:53.588 INFO:teuthology.orchestra.run.smithi117.stdout: inet 172.21.15.117/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-02-16T19:26:53.588 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft 42205sec preferred_lft 42205sec 2024-02-16T19:26:53.588 INFO:teuthology.orchestra.run.smithi117.stdout: inet6 fe80::ec4:7aff:fe8f:4463/64 scope link 2024-02-16T19:26:53.588 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-02-16T19:26:53.588 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-02-16T19:26:53.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:26:53.589 DEBUG:teuthology.orchestra.run.smithi117:> (cd / && exec bash -c ' 2024-02-16T19:26:53.589 DEBUG:teuthology.orchestra.run.smithi117:> set -e 2024-02-16T19:26:53.589 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link add name ceph-brx type bridge 2024-02-16T19:26:53.589 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip addr flush dev ceph-brx 2024-02-16T19:26:53.589 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link set ceph-brx up 2024-02-16T19:26:53.589 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-02-16T19:26:53.589 DEBUG:teuthology.orchestra.run.smithi117:> ') 2024-02-16T19:26:53.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:26:53.764 DEBUG:teuthology.orchestra.run.smithi117:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-16T19:26:53.790 INFO:teuthology.orchestra.run.smithi117.stdout:1 2024-02-16T19:26:53.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:26:53.792 DEBUG:teuthology.orchestra.run.smithi117:> route 2024-02-16T19:26:53.861 INFO:teuthology.orchestra.run.smithi117.stdout:Kernel IP routing table 2024-02-16T19:26:53.861 INFO:teuthology.orchestra.run.smithi117.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-16T19:26:53.861 INFO:teuthology.orchestra.run.smithi117.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-02-16T19:26:53.861 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-02-16T19:26:53.861 INFO:teuthology.orchestra.run.smithi117.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-16T19:26:53.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:26:53.861 DEBUG:teuthology.orchestra.run.smithi117:> (cd / && exec bash -c ' 2024-02-16T19:26:53.861 DEBUG:teuthology.orchestra.run.smithi117:> set -e 2024-02-16T19:26:53.861 DEBUG:teuthology.orchestra.run.smithi117:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-02-16T19:26:53.861 DEBUG:teuthology.orchestra.run.smithi117:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-02-16T19:26:53.862 DEBUG:teuthology.orchestra.run.smithi117:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-02-16T19:26:53.862 DEBUG:teuthology.orchestra.run.smithi117:> ') 2024-02-16T19:26:53.925 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:53 smithi117 ceph-mon[35309]: pgmap v198: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 0 B/s wr, 0 op/s 2024-02-16T19:26:53.926 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:53 smithi117 ceph-mon[35309]: from='client.? 172.21.15.22:0/2518842914' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-16T19:26:54.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:26:54.106 DEBUG:teuthology.orchestra.run.smithi117:> ip netns list 2024-02-16T19:26:54.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:26:54.162 DEBUG:teuthology.orchestra.run.smithi117:> ip netns list-id 2024-02-16T19:26:54.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:53 smithi022 ceph-mon[29221]: pgmap v198: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 0 B/s wr, 0 op/s 2024-02-16T19:26:54.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:53 smithi022 ceph-mon[29221]: from='client.? 172.21.15.22:0/2518842914' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-16T19:26:54.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:26:54.216 DEBUG:teuthology.orchestra.run.smithi117:> (cd / && exec bash -c ' 2024-02-16T19:26:54.216 DEBUG:teuthology.orchestra.run.smithi117:> set -e 2024-02-16T19:26:54.216 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-02-16T19:26:54.216 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-02-16T19:26:54.216 DEBUG:teuthology.orchestra.run.smithi117:> ') 2024-02-16T19:26:54.299 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-02-16T19:26:54.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:26:54.299 DEBUG:teuthology.orchestra.run.smithi117:> (cd / && exec bash -c ' 2024-02-16T19:26:54.299 DEBUG:teuthology.orchestra.run.smithi117:> set -e 2024-02-16T19:26:54.299 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-02-16T19:26:54.299 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-02-16T19:26:54.299 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-02-16T19:26:54.299 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-02-16T19:26:54.299 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-02-16T19:26:54.300 DEBUG:teuthology.orchestra.run.smithi117:> ') 2024-02-16T19:26:54.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:26:54.431 DEBUG:teuthology.orchestra.run.smithi117:> (cd / && exec bash -c ' 2024-02-16T19:26:54.431 DEBUG:teuthology.orchestra.run.smithi117:> set -e 2024-02-16T19:26:54.431 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link set brx.0 up 2024-02-16T19:26:54.431 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link set dev brx.0 master ceph-brx 2024-02-16T19:26:54.431 DEBUG:teuthology.orchestra.run.smithi117:> ') 2024-02-16T19:26:54.472 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-02-16T19:26:54.472 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-16T19:26:54.472 DEBUG:teuthology.orchestra.run.smithi117:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-02-16T19:26:54.531 INFO:teuthology.orchestra.run.smithi117.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-02-16T19:26:54.531 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-16T19:26:54.531 DEBUG:teuthology.orchestra.run.smithi117:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-02-16T19:26:54.585 DEBUG:teuthology.orchestra.run.smithi117:> sudo modprobe fuse 2024-02-16T19:26:54.653 DEBUG:teuthology.orchestra.run.smithi117:> cat /proc/self/mounts | awk '{print $2}' 2024-02-16T19:26:54.709 INFO:teuthology.orchestra.run.smithi117.stdout:/proc 2024-02-16T19:26:54.709 INFO:teuthology.orchestra.run.smithi117.stdout:/sys 2024-02-16T19:26:54.709 INFO:teuthology.orchestra.run.smithi117.stdout:/dev 2024-02-16T19:26:54.709 INFO:teuthology.orchestra.run.smithi117.stdout:/sys/kernel/security 2024-02-16T19:26:54.709 INFO:teuthology.orchestra.run.smithi117.stdout:/dev/shm 2024-02-16T19:26:54.709 INFO:teuthology.orchestra.run.smithi117.stdout:/dev/pts 2024-02-16T19:26:54.709 INFO:teuthology.orchestra.run.smithi117.stdout:/run 2024-02-16T19:26:54.709 INFO:teuthology.orchestra.run.smithi117.stdout:/sys/fs/cgroup 2024-02-16T19:26:54.709 INFO:teuthology.orchestra.run.smithi117.stdout:/sys/fs/pstore 2024-02-16T19:26:54.709 INFO:teuthology.orchestra.run.smithi117.stdout:/sys/fs/bpf 2024-02-16T19:26:54.709 INFO:teuthology.orchestra.run.smithi117.stdout:/ 2024-02-16T19:26:54.709 INFO:teuthology.orchestra.run.smithi117.stdout:/sys/fs/selinux 2024-02-16T19:26:54.710 INFO:teuthology.orchestra.run.smithi117.stdout:/proc/sys/fs/binfmt_misc 2024-02-16T19:26:54.710 INFO:teuthology.orchestra.run.smithi117.stdout:/dev/hugepages 2024-02-16T19:26:54.710 INFO:teuthology.orchestra.run.smithi117.stdout:/dev/mqueue 2024-02-16T19:26:54.710 INFO:teuthology.orchestra.run.smithi117.stdout:/sys/kernel/debug 2024-02-16T19:26:54.710 INFO:teuthology.orchestra.run.smithi117.stdout:/sys/kernel/tracing 2024-02-16T19:26:54.710 INFO:teuthology.orchestra.run.smithi117.stdout:/sys/fs/fuse/connections 2024-02-16T19:26:54.710 INFO:teuthology.orchestra.run.smithi117.stdout:/sys/kernel/config 2024-02-16T19:26:54.710 INFO:teuthology.orchestra.run.smithi117.stdout:/run/credentials/systemd-sysctl.service 2024-02-16T19:26:54.710 INFO:teuthology.orchestra.run.smithi117.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-02-16T19:26:54.710 INFO:teuthology.orchestra.run.smithi117.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-02-16T19:26:54.710 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/nfs/rpc_pipefs 2024-02-16T19:26:54.710 INFO:teuthology.orchestra.run.smithi117.stdout:/run/user/1000 2024-02-16T19:26:54.710 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/ceph 2024-02-16T19:26:54.710 INFO:teuthology.orchestra.run.smithi117.stdout:/run/user/0 2024-02-16T19:26:54.710 INFO:teuthology.orchestra.run.smithi117.stdout:/proc/sys/fs/binfmt_misc 2024-02-16T19:26:54.711 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay 2024-02-16T19:26:54.711 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/7b17aad3b30a116aedb1bb0b55e28b0436849e4cc7d8263fe202c99794786b4a/merged 2024-02-16T19:26:54.711 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/2885557ff311542a9d533c95b758d606d0da52daa171ab3e3b1c8f7b97ee1af7/merged 2024-02-16T19:26:54.711 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/c40bc59ec87651de9d35a11666d74d57c3a903c38243a6c2f47b1066bf5362f7/merged 2024-02-16T19:26:54.711 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/9b9b9c48af8ae24b6fcd04d84dc061eaa7fb803dbe65719790a421a35c73fe9b/merged 2024-02-16T19:26:54.711 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/3262e6eaada32dd1159384022cfd9f38a9e2ea43f49bc3d3b284f780d048075d/merged 2024-02-16T19:26:54.711 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/07143fa755c910691a4058a44397d241553c6ec7aa7b29e82b14e8a55c264341/merged 2024-02-16T19:26:54.711 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/42c9757745c01c39ad17c602ea796e2f0a4dc40b1d5192e05cc6a7961eb7b7c6/merged 2024-02-16T19:26:54.711 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/4b585d897a03162ccbca68fa1c3e4955ab83500a2e2833cc280e8d4efcb00d2a/merged 2024-02-16T19:26:54.711 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/5d05969183086e4f83fe52ff1b6b853713e9a544d0df7238d723d3f63ab02454/merged 2024-02-16T19:26:54.711 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/f421afdac9bee96a19dca264f099deffaf753cb36e9ded2fa9cad1c32407a1d6/merged 2024-02-16T19:26:54.711 INFO:teuthology.orchestra.run.smithi117.stdout:/run/netns 2024-02-16T19:26:54.711 INFO:teuthology.orchestra.run.smithi117.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-02-16T19:26:54.711 INFO:teuthology.orchestra.run.smithi117.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-02-16T19:26:54.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:26:54.712 DEBUG:teuthology.orchestra.run.smithi117:> ls /sys/fs/fuse/connections 2024-02-16T19:26:54.763 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-02-16T19:26:54.764 DEBUG:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-02-16T19:26:54.805 DEBUG:teuthology.orchestra.run.smithi117:> sudo modprobe fuse 2024-02-16T19:26:54.831 DEBUG:teuthology.orchestra.run.smithi117:> cat /proc/self/mounts | awk '{print $2}' 2024-02-16T19:26:54.886 INFO:teuthology.orchestra.run.smithi117.stdout:/proc 2024-02-16T19:26:54.886 INFO:teuthology.orchestra.run.smithi117.stdout:/sys 2024-02-16T19:26:54.886 INFO:teuthology.orchestra.run.smithi117.stdout:/dev 2024-02-16T19:26:54.886 INFO:teuthology.orchestra.run.smithi117.stdout:/sys/kernel/security 2024-02-16T19:26:54.887 INFO:teuthology.orchestra.run.smithi117.stdout:/dev/shm 2024-02-16T19:26:54.887 INFO:teuthology.orchestra.run.smithi117.stdout:/dev/pts 2024-02-16T19:26:54.887 INFO:teuthology.orchestra.run.smithi117.stdout:/run 2024-02-16T19:26:54.887 INFO:teuthology.orchestra.run.smithi117.stdout:/sys/fs/cgroup 2024-02-16T19:26:54.887 INFO:teuthology.orchestra.run.smithi117.stdout:/sys/fs/pstore 2024-02-16T19:26:54.887 INFO:teuthology.orchestra.run.smithi117.stdout:/sys/fs/bpf 2024-02-16T19:26:54.887 INFO:teuthology.orchestra.run.smithi117.stdout:/ 2024-02-16T19:26:54.887 INFO:teuthology.orchestra.run.smithi117.stdout:/sys/fs/selinux 2024-02-16T19:26:54.887 INFO:teuthology.orchestra.run.smithi117.stdout:/proc/sys/fs/binfmt_misc 2024-02-16T19:26:54.887 INFO:teuthology.orchestra.run.smithi117.stdout:/dev/hugepages 2024-02-16T19:26:54.887 INFO:teuthology.orchestra.run.smithi117.stdout:/dev/mqueue 2024-02-16T19:26:54.887 INFO:teuthology.orchestra.run.smithi117.stdout:/sys/kernel/debug 2024-02-16T19:26:54.887 INFO:teuthology.orchestra.run.smithi117.stdout:/sys/kernel/tracing 2024-02-16T19:26:54.887 INFO:teuthology.orchestra.run.smithi117.stdout:/sys/fs/fuse/connections 2024-02-16T19:26:54.887 INFO:teuthology.orchestra.run.smithi117.stdout:/sys/kernel/config 2024-02-16T19:26:54.888 INFO:teuthology.orchestra.run.smithi117.stdout:/run/credentials/systemd-sysctl.service 2024-02-16T19:26:54.888 INFO:teuthology.orchestra.run.smithi117.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-02-16T19:26:54.888 INFO:teuthology.orchestra.run.smithi117.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-02-16T19:26:54.888 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/nfs/rpc_pipefs 2024-02-16T19:26:54.888 INFO:teuthology.orchestra.run.smithi117.stdout:/run/user/1000 2024-02-16T19:26:54.888 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/ceph 2024-02-16T19:26:54.888 INFO:teuthology.orchestra.run.smithi117.stdout:/run/user/0 2024-02-16T19:26:54.888 INFO:teuthology.orchestra.run.smithi117.stdout:/proc/sys/fs/binfmt_misc 2024-02-16T19:26:54.888 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay 2024-02-16T19:26:54.888 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/7b17aad3b30a116aedb1bb0b55e28b0436849e4cc7d8263fe202c99794786b4a/merged 2024-02-16T19:26:54.888 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/2885557ff311542a9d533c95b758d606d0da52daa171ab3e3b1c8f7b97ee1af7/merged 2024-02-16T19:26:54.888 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/c40bc59ec87651de9d35a11666d74d57c3a903c38243a6c2f47b1066bf5362f7/merged 2024-02-16T19:26:54.888 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/9b9b9c48af8ae24b6fcd04d84dc061eaa7fb803dbe65719790a421a35c73fe9b/merged 2024-02-16T19:26:54.888 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/3262e6eaada32dd1159384022cfd9f38a9e2ea43f49bc3d3b284f780d048075d/merged 2024-02-16T19:26:54.888 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/07143fa755c910691a4058a44397d241553c6ec7aa7b29e82b14e8a55c264341/merged 2024-02-16T19:26:54.889 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/42c9757745c01c39ad17c602ea796e2f0a4dc40b1d5192e05cc6a7961eb7b7c6/merged 2024-02-16T19:26:54.889 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/4b585d897a03162ccbca68fa1c3e4955ab83500a2e2833cc280e8d4efcb00d2a/merged 2024-02-16T19:26:54.889 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/5d05969183086e4f83fe52ff1b6b853713e9a544d0df7238d723d3f63ab02454/merged 2024-02-16T19:26:54.889 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/f421afdac9bee96a19dca264f099deffaf753cb36e9ded2fa9cad1c32407a1d6/merged 2024-02-16T19:26:54.889 INFO:teuthology.orchestra.run.smithi117.stdout:/run/netns 2024-02-16T19:26:54.889 INFO:teuthology.orchestra.run.smithi117.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-02-16T19:26:54.889 INFO:teuthology.orchestra.run.smithi117.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-02-16T19:26:54.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:26:54.889 DEBUG:teuthology.orchestra.run.smithi117:> ls /sys/fs/fuse/connections 2024-02-16T19:26:54.931 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi117.stderr:2024-02-16T19:26:54.929+0000 7f6b66268480 -1 init, newargv = 0x559fc3351360 newargc=15 2024-02-16T19:26:54.931 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi117.stderr:ceph-fuse[55016]: starting ceph client 2024-02-16T19:26:54.936 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi117.stderr:ceph-fuse[55016]: starting fuse 2024-02-16T19:26:54.942 INFO:teuthology.orchestra.run.smithi117.stdout:126 2024-02-16T19:26:54.942 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [126] 2024-02-16T19:26:54.942 DEBUG:teuthology.orchestra.run.smithi117:> sudo stdin-killer -- python3 -c ' 2024-02-16T19:26:54.942 DEBUG:teuthology.orchestra.run.smithi117:> import glob 2024-02-16T19:26:54.942 DEBUG:teuthology.orchestra.run.smithi117:> import re 2024-02-16T19:26:54.942 DEBUG:teuthology.orchestra.run.smithi117:> import os 2024-02-16T19:26:54.942 DEBUG:teuthology.orchestra.run.smithi117:> import subprocess 2024-02-16T19:26:54.943 DEBUG:teuthology.orchestra.run.smithi117:> 2024-02-16T19:26:54.943 DEBUG:teuthology.orchestra.run.smithi117:> def _find_admin_socket(client_name): 2024-02-16T19:26:54.943 DEBUG:teuthology.orchestra.run.smithi117:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-02-16T19:26:54.943 DEBUG:teuthology.orchestra.run.smithi117:> files = glob.glob(asok_path) 2024-02-16T19:26:54.943 DEBUG:teuthology.orchestra.run.smithi117:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-02-16T19:26:54.943 DEBUG:teuthology.orchestra.run.smithi117:> 2024-02-16T19:26:54.943 DEBUG:teuthology.orchestra.run.smithi117:> # Given a non-glob path, it better be there 2024-02-16T19:26:54.943 DEBUG:teuthology.orchestra.run.smithi117:> if "*" not in asok_path: 2024-02-16T19:26:54.943 DEBUG:teuthology.orchestra.run.smithi117:> assert(len(files) == 1) 2024-02-16T19:26:54.943 DEBUG:teuthology.orchestra.run.smithi117:> return files[0] 2024-02-16T19:26:54.943 DEBUG:teuthology.orchestra.run.smithi117:> 2024-02-16T19:26:54.943 DEBUG:teuthology.orchestra.run.smithi117:> for f in files: 2024-02-16T19:26:54.943 DEBUG:teuthology.orchestra.run.smithi117:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-02-16T19:26:54.943 DEBUG:teuthology.orchestra.run.smithi117:> if os.path.exists("/proc/{0}".format(pid)): 2024-02-16T19:26:54.943 DEBUG:teuthology.orchestra.run.smithi117:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-02-16T19:26:54.943 DEBUG:teuthology.orchestra.run.smithi117:> contents = proc_f.read() 2024-02-16T19:26:54.943 DEBUG:teuthology.orchestra.run.smithi117:> if mountpoint in contents: 2024-02-16T19:26:54.944 DEBUG:teuthology.orchestra.run.smithi117:> return f 2024-02-16T19:26:54.944 DEBUG:teuthology.orchestra.run.smithi117:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-02-16T19:26:54.944 DEBUG:teuthology.orchestra.run.smithi117:> 2024-02-16T19:26:54.944 DEBUG:teuthology.orchestra.run.smithi117:> print(_find_admin_socket("client.1")) 2024-02-16T19:26:54.944 DEBUG:teuthology.orchestra.run.smithi117:> ' 2024-02-16T19:26:55.062 INFO:teuthology.orchestra.run.smithi117.stdout:/var/run/ceph/ceph-client.1.55016.asok 2024-02-16T19:26:55.065 INFO:teuthology.orchestra.run.smithi117.stderr:2024-02-16T19:26:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-02-16T19:26:55.073 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.55016.asok 2024-02-16T19:26:55.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:26:55.073 DEBUG:teuthology.orchestra.run.smithi117:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.55016.asok status 2024-02-16T19:26:55.208 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-02-16T19:26:55.208 INFO:teuthology.orchestra.run.smithi117.stdout: "metadata": { 2024-02-16T19:26:55.208 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_sha1": "5dd24139a1eada541a3bc16b6941c5dde975e26d", 2024-02-16T19:26:55.208 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_version": "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)", 2024-02-16T19:26:55.208 INFO:teuthology.orchestra.run.smithi117.stdout: "entity_id": "1", 2024-02-16T19:26:55.208 INFO:teuthology.orchestra.run.smithi117.stdout: "hostname": "smithi117", 2024-02-16T19:26:55.209 INFO:teuthology.orchestra.run.smithi117.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-02-16T19:26:55.209 INFO:teuthology.orchestra.run.smithi117.stdout: "pid": "55016", 2024-02-16T19:26:55.209 INFO:teuthology.orchestra.run.smithi117.stdout: "root": "/" 2024-02-16T19:26:55.209 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-02-16T19:26:55.209 INFO:teuthology.orchestra.run.smithi117.stdout: "dentry_count": 0, 2024-02-16T19:26:55.209 INFO:teuthology.orchestra.run.smithi117.stdout: "dentry_pinned_count": 0, 2024-02-16T19:26:55.209 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 24307, 2024-02-16T19:26:55.209 INFO:teuthology.orchestra.run.smithi117.stdout: "inst": { 2024-02-16T19:26:55.209 INFO:teuthology.orchestra.run.smithi117.stdout: "name": { 2024-02-16T19:26:55.209 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "client", 2024-02-16T19:26:55.209 INFO:teuthology.orchestra.run.smithi117.stdout: "num": 24307 2024-02-16T19:26:55.209 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-02-16T19:26:55.209 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": { 2024-02-16T19:26:55.209 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v1", 2024-02-16T19:26:55.209 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "192.168.0.1:0", 2024-02-16T19:26:55.210 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 2099531446 2024-02-16T19:26:55.210 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-02-16T19:26:55.210 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-02-16T19:26:55.210 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": { 2024-02-16T19:26:55.210 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v1", 2024-02-16T19:26:55.210 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "192.168.0.1:0", 2024-02-16T19:26:55.210 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 2099531446 2024-02-16T19:26:55.210 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-02-16T19:26:55.210 INFO:teuthology.orchestra.run.smithi117.stdout: "inst_str": "client.24307 192.168.0.1:0/2099531446", 2024-02-16T19:26:55.210 INFO:teuthology.orchestra.run.smithi117.stdout: "addr_str": "192.168.0.1:0/2099531446", 2024-02-16T19:26:55.210 INFO:teuthology.orchestra.run.smithi117.stdout: "inode_count": 1, 2024-02-16T19:26:55.210 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_epoch": 15, 2024-02-16T19:26:55.210 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_epoch": 47, 2024-02-16T19:26:55.210 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_epoch_barrier": 0, 2024-02-16T19:26:55.210 INFO:teuthology.orchestra.run.smithi117.stdout: "blocklisted": false, 2024-02-16T19:26:55.210 INFO:teuthology.orchestra.run.smithi117.stdout: "fs_name": "cephfs" 2024-02-16T19:26:55.211 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-02-16T19:26:55.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:26:55.217 DEBUG:teuthology.orchestra.run.smithi022:> stat --file-system '--printf=%T 2024-02-16T19:26:55.217 DEBUG:teuthology.orchestra.run.smithi022:> ' -- /home/ubuntu/cephtest/mnt.0 2024-02-16T19:26:55.235 INFO:teuthology.orchestra.run.smithi022.stdout:fuseblk 2024-02-16T19:26:55.235 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-02-16T19:26:55.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:26:55.235 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-02-16T19:26:55.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:26:55.310 DEBUG:teuthology.orchestra.run.smithi117:> stat --file-system '--printf=%T 2024-02-16T19:26:55.310 DEBUG:teuthology.orchestra.run.smithi117:> ' -- /home/ubuntu/cephtest/mnt.1 2024-02-16T19:26:55.325 INFO:teuthology.orchestra.run.smithi117.stdout:fuseblk 2024-02-16T19:26:55.325 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-02-16T19:26:55.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:26:55.326 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-02-16T19:26:55.395 INFO:teuthology.run_tasks:Running task print... 2024-02-16T19:26:55.406 INFO:teuthology.task.print:**** done client 2024-02-16T19:26:55.406 INFO:teuthology.run_tasks:Running task parallel... 2024-02-16T19:26:55.420 INFO:teuthology.task.parallel:starting parallel... 2024-02-16T19:26:55.421 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-02-16T19:26:55.421 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-02-16T19:26:55.422 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi022.front.sepia.ceph.com 2024-02-16T19:26:55.422 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-02-16T19:26:55.422 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-02-16T19:26:55.422 INFO:teuthology.task.sequential:In sequential, running task workunit... 2024-02-16T19:26:55.426 INFO:tasks.workunit:Pulling workunits from ref ed7ab725e1a43615009dfb74a42bf33d80081bc8 2024-02-16T19:26:55.426 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-02-16T19:26:55.426 INFO:tasks.workunit:timeout=3h 2024-02-16T19:26:55.426 INFO:tasks.workunit:cleanup=True 2024-02-16T19:26:55.426 DEBUG:teuthology.orchestra.run.smithi022:> stat -- /home/ubuntu/cephtest/mnt.0 2024-02-16T19:26:55.444 INFO:teuthology.orchestra.run.smithi022.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-02-16T19:26:55.444 INFO:teuthology.orchestra.run.smithi022.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-02-16T19:26:55.444 INFO:teuthology.orchestra.run.smithi022.stdout:Device: 94h/148d Inode: 1 Links: 2 2024-02-16T19:26:55.444 INFO:teuthology.orchestra.run.smithi022.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-16T19:26:55.444 INFO:teuthology.orchestra.run.smithi022.stdout:Context: system_u:object_r:fusefs_t:s0 2024-02-16T19:26:55.445 INFO:teuthology.orchestra.run.smithi022.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-02-16T19:26:55.445 INFO:teuthology.orchestra.run.smithi022.stdout:Modify: 2024-02-16 19:26:22.898565314 +0000 2024-02-16T19:26:55.445 INFO:teuthology.orchestra.run.smithi022.stdout:Change: 2024-02-16 19:26:55.307294282 +0000 2024-02-16T19:26:55.445 INFO:teuthology.orchestra.run.smithi022.stdout: Birth: - 2024-02-16T19:26:55.445 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-02-16T19:26:55.445 DEBUG:teuthology.orchestra.run.smithi022:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-02-16T19:26:55.518 DEBUG:teuthology.orchestra.run.smithi117:> stat -- /home/ubuntu/cephtest/mnt.1 2024-02-16T19:26:55.534 INFO:teuthology.orchestra.run.smithi117.stdout: File: /home/ubuntu/cephtest/mnt.1 2024-02-16T19:26:55.534 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-02-16T19:26:55.535 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 7eh/126d Inode: 1 Links: 3 2024-02-16T19:26:55.535 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-16T19:26:55.535 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:fusefs_t:s0 2024-02-16T19:26:55.535 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-02-16T19:26:55.535 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-02-16 19:26:55.513466097 +0000 2024-02-16T19:26:55.535 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-02-16 19:26:55.513466097 +0000 2024-02-16T19:26:55.535 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: - 2024-02-16T19:26:55.535 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.1 2024-02-16T19:26:55.535 DEBUG:teuthology.orchestra.run.smithi117:> cd -- /home/ubuntu/cephtest/mnt.1 && sudo install -d -m 0755 --owner=ubuntu -- client.1 2024-02-16T19:26:55.606 DEBUG:teuthology.orchestra.run.smithi022:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/batrick/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout ed7ab725e1a43615009dfb74a42bf33d80081bc8 2024-02-16T19:26:55.606 DEBUG:teuthology.orchestra.run.smithi117:> rm -rf /home/ubuntu/cephtest/clone.client.1 && git clone https://github.com/batrick/ceph.git /home/ubuntu/cephtest/clone.client.1 && cd /home/ubuntu/cephtest/clone.client.1 && git checkout ed7ab725e1a43615009dfb74a42bf33d80081bc8 2024-02-16T19:26:55.621 INFO:tasks.workunit.client.0.smithi022.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-02-16T19:26:55.648 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi022/config 2024-02-16T19:26:55.662 INFO:tasks.workunit.client.1.smithi117.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.1'... 2024-02-16T19:26:56.021 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:55 smithi022 ceph-mon[29221]: pgmap v199: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:26:56.136 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:55 smithi117 ceph-mon[35309]: pgmap v199: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:26:57.086 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:56 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:26:57.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:56 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:26:57.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:57.323+0000 7f4246b1b700 1 -- 172.21.15.22:0/526851978 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f42401007f0 msgr2=0x7f4240100c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:57.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:57.323+0000 7f4246b1b700 1 --2- 172.21.15.22:0/526851978 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f42401007f0 0x7f4240100c00 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4234009910 tx=0x7f4234009c20 comp rx=0 tx=0).stop 2024-02-16T19:26:57.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:57.323+0000 7f4246b1b700 1 -- 172.21.15.22:0/526851978 shutdown_connections 2024-02-16T19:26:57.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:57.323+0000 7f4246b1b700 1 --2- 172.21.15.22:0/526851978 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4240101140 0x7f4240107a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:57.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:57.323+0000 7f4246b1b700 1 --2- 172.21.15.22:0/526851978 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f42401007f0 0x7f4240100c00 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:57.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:57.323+0000 7f4246b1b700 1 -- 172.21.15.22:0/526851978 >> 172.21.15.22:0/526851978 conn(0x7f42400f9f10 msgr2=0x7f42400fc340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:26:57.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:57.324+0000 7f4246b1b700 1 -- 172.21.15.22:0/526851978 shutdown_connections 2024-02-16T19:26:57.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:57.324+0000 7f4246b1b700 1 -- 172.21.15.22:0/526851978 wait complete. 2024-02-16T19:26:57.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:57.324+0000 7f4246b1b700 1 Processor -- start 2024-02-16T19:26:57.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:57.324+0000 7f4246b1b700 1 -- start start 2024-02-16T19:26:57.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:57.324+0000 7f4246b1b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f42401007f0 0x7f4240197d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:57.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:57.324+0000 7f4246b1b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4240101140 0x7f4240198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:57.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:57.325+0000 7f4246b1b700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4240198870 con 0x7f42401007f0 2024-02-16T19:26:57.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:57.325+0000 7f4246b1b700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f424019d280 con 0x7f4240101140 2024-02-16T19:26:57.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:57.325+0000 7f42448b7700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f42401007f0 0x7f4240197d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:57.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:57.325+0000 7f42448b7700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f42401007f0 0x7f4240197d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46948/0 (socket says 172.21.15.22:46948) 2024-02-16T19:26:57.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:57.325+0000 7f42448b7700 1 -- 172.21.15.22:0/3785823339 learned_addr learned my addr 172.21.15.22:0/3785823339 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:26:57.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:57.325+0000 7f423ffff700 1 --2- 172.21.15.22:0/3785823339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4240101140 0x7f4240198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:57.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:57.325+0000 7f42448b7700 1 -- 172.21.15.22:0/3785823339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4240101140 msgr2=0x7f4240198250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:57.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:57.325+0000 7f42448b7700 1 --2- 172.21.15.22:0/3785823339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4240101140 0x7f4240198250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:57.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:57.325+0000 7f42448b7700 1 -- 172.21.15.22:0/3785823339 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42340095f0 con 0x7f42401007f0 2024-02-16T19:26:57.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:57.325+0000 7f423ffff700 1 --2- 172.21.15.22:0/3785823339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4240101140 0x7f4240198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T19:26:57.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:57.325+0000 7f42448b7700 1 --2- 172.21.15.22:0/3785823339 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f42401007f0 0x7f4240197d10 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f423400b5c0 tx=0x7f423400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:26:57.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:57.325+0000 7f423dffb700 1 -- 172.21.15.22:0/3785823339 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f423401d070 con 0x7f42401007f0 2024-02-16T19:26:57.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:57.325+0000 7f423dffb700 1 -- 172.21.15.22:0/3785823339 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4234022470 con 0x7f42401007f0 2024-02-16T19:26:57.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:57.325+0000 7f423dffb700 1 -- 172.21.15.22:0/3785823339 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f423400f690 con 0x7f42401007f0 2024-02-16T19:26:57.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:57.325+0000 7f4246b1b700 1 -- 172.21.15.22:0/3785823339 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f424019d420 con 0x7f42401007f0 2024-02-16T19:26:57.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:57.326+0000 7f4246b1b700 1 -- 172.21.15.22:0/3785823339 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f424019d890 con 0x7f42401007f0 2024-02-16T19:26:57.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:57.326+0000 7f4246b1b700 1 -- 172.21.15.22:0/3785823339 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4240191e10 con 0x7f42401007f0 2024-02-16T19:26:57.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:57.327+0000 7f423dffb700 1 -- 172.21.15.22:0/3785823339 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f4234004b70 con 0x7f42401007f0 2024-02-16T19:26:57.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:57.327+0000 7f423dffb700 1 --2- 172.21.15.22:0/3785823339 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f423006c680 0x7f423006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:57.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:57.327+0000 7f423dffb700 1 -- 172.21.15.22:0/3785823339 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f423408cfb0 con 0x7f42401007f0 2024-02-16T19:26:57.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:57.327+0000 7f423ffff700 1 --2- 172.21.15.22:0/3785823339 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f423006c680 0x7f423006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:57.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:57.328+0000 7f423ffff700 1 --2- 172.21.15.22:0/3785823339 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f423006c680 0x7f423006eb30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f422c005950 tx=0x7f422c005f50 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:26:57.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:57.332+0000 7f423dffb700 1 -- 172.21.15.22:0/3785823339 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f42340576e0 con 0x7f42401007f0 2024-02-16T19:26:57.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:57.530+0000 7f4246b1b700 1 -- 172.21.15.22:0/3785823339 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7f424002cc30 con 0x7f42401007f0 2024-02-16T19:26:57.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:57.530+0000 7f423dffb700 1 -- 172.21.15.22:0/3785823339 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v21)=0 v21) v1 ==== 155+0+0 (secure 0 0 0) 0x7f423405ad00 con 0x7f42401007f0 2024-02-16T19:26:57.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:57.533+0000 7f4246b1b700 1 -- 172.21.15.22:0/3785823339 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f423006c680 msgr2=0x7f423006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:57.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:57.533+0000 7f4246b1b700 1 --2- 172.21.15.22:0/3785823339 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f423006c680 0x7f423006eb30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f422c005950 tx=0x7f422c005f50 comp rx=0 tx=0).stop 2024-02-16T19:26:57.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:57.533+0000 7f4246b1b700 1 -- 172.21.15.22:0/3785823339 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f42401007f0 msgr2=0x7f4240197d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:57.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:57.533+0000 7f4246b1b700 1 --2- 172.21.15.22:0/3785823339 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f42401007f0 0x7f4240197d10 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f423400b5c0 tx=0x7f423400be20 comp rx=0 tx=0).stop 2024-02-16T19:26:57.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:57.533+0000 7f4246b1b700 1 -- 172.21.15.22:0/3785823339 shutdown_connections 2024-02-16T19:26:57.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:57.533+0000 7f4246b1b700 1 --2- 172.21.15.22:0/3785823339 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f42401007f0 0x7f4240197d10 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:57.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:57.533+0000 7f4246b1b700 1 --2- 172.21.15.22:0/3785823339 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f423006c680 0x7f423006eb30 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:57.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:57.533+0000 7f4246b1b700 1 --2- 172.21.15.22:0/3785823339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4240101140 0x7f4240198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:57.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:57.533+0000 7f4246b1b700 1 -- 172.21.15.22:0/3785823339 >> 172.21.15.22:0/3785823339 conn(0x7f42400f9f10 msgr2=0x7f42400fc340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:26:57.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:57.533+0000 7f4246b1b700 1 -- 172.21.15.22:0/3785823339 shutdown_connections 2024-02-16T19:26:57.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:26:57.533+0000 7f4246b1b700 1 -- 172.21.15.22:0/3785823339 wait complete. 2024-02-16T19:26:58.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:57 smithi117 ceph-mon[35309]: pgmap v200: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-02-16T19:26:58.182 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:57 smithi022 ceph-mon[29221]: pgmap v200: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-02-16T19:26:58.208 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-02-16T19:26:58.462 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi022/config 2024-02-16T19:26:59.903 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:26:59 smithi022 ceph-mon[29221]: pgmap v201: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-02-16T19:27:00.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:00.241+0000 7f80173b3700 1 -- 172.21.15.22:0/860022306 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8010103680 msgr2=0x7f8010105a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:27:00.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:00.241+0000 7f80173b3700 1 --2- 172.21.15.22:0/860022306 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8010103680 0x7f8010105a60 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8000009910 tx=0x7f8000009c20 comp rx=0 tx=0).stop 2024-02-16T19:27:00.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:00.241+0000 7f80173b3700 1 -- 172.21.15.22:0/860022306 shutdown_connections 2024-02-16T19:27:00.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:00.241+0000 7f80173b3700 1 --2- 172.21.15.22:0/860022306 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8010103680 0x7f8010105a60 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:00.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:00.241+0000 7f80173b3700 1 --2- 172.21.15.22:0/860022306 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8010100d60 0x7f8010103140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:00.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:00.241+0000 7f80173b3700 1 -- 172.21.15.22:0/860022306 >> 172.21.15.22:0/860022306 conn(0x7f80100fa780 msgr2=0x7f80100fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:27:00.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:00.241+0000 7f80173b3700 1 -- 172.21.15.22:0/860022306 shutdown_connections 2024-02-16T19:27:00.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:00.242+0000 7f80173b3700 1 -- 172.21.15.22:0/860022306 wait complete. 2024-02-16T19:27:00.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:00.242+0000 7f80173b3700 1 Processor -- start 2024-02-16T19:27:00.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:00.242+0000 7f80173b3700 1 -- start start 2024-02-16T19:27:00.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:00.242+0000 7f80173b3700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8010100d60 0x7f8010195ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:27:00.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:00.242+0000 7f80173b3700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8010103680 0x7f8010196020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:27:00.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:00.242+0000 7f80173b3700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8010196640 con 0x7f8010100d60 2024-02-16T19:27:00.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:00.242+0000 7f80173b3700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8010196780 con 0x7f8010103680 2024-02-16T19:27:00.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:00.243+0000 7f801494e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8010103680 0x7f8010196020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:27:00.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:00.243+0000 7f801514f700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8010100d60 0x7f8010195ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:27:00.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:00.243+0000 7f801514f700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8010100d60 0x7f8010195ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46964/0 (socket says 172.21.15.22:46964) 2024-02-16T19:27:00.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:00.243+0000 7f801514f700 1 -- 172.21.15.22:0/710405211 learned_addr learned my addr 172.21.15.22:0/710405211 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:27:00.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:00.243+0000 7f801494e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8010103680 0x7f8010196020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.22:36448/0 (socket says 172.21.15.22:36448) 2024-02-16T19:27:00.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:00.243+0000 7f801514f700 1 -- 172.21.15.22:0/710405211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8010103680 msgr2=0x7f8010196020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:27:00.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:00.243+0000 7f801514f700 1 --2- 172.21.15.22:0/710405211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8010103680 0x7f8010196020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:00.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:00.243+0000 7f801514f700 1 -- 172.21.15.22:0/710405211 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80000095f0 con 0x7f8010100d60 2024-02-16T19:27:00.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:00.243+0000 7f801514f700 1 --2- 172.21.15.22:0/710405211 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8010100d60 0x7f8010195ae0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f800c00e8f0 tx=0x7f800c00ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:27:00.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:00.243+0000 7f80067fc700 1 -- 172.21.15.22:0/710405211 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f800c009790 con 0x7f8010100d60 2024-02-16T19:27:00.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:00.243+0000 7f80067fc700 1 -- 172.21.15.22:0/710405211 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f800c00c810 con 0x7f8010100d60 2024-02-16T19:27:00.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:00.243+0000 7f80067fc700 1 -- 172.21.15.22:0/710405211 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f800c0055b0 con 0x7f8010100d60 2024-02-16T19:27:00.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:00.243+0000 7f80173b3700 1 -- 172.21.15.22:0/710405211 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f801019b230 con 0x7f8010100d60 2024-02-16T19:27:00.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:00.244+0000 7f80173b3700 1 -- 172.21.15.22:0/710405211 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f801019b730 con 0x7f8010100d60 2024-02-16T19:27:00.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:00.245+0000 7f80173b3700 1 -- 172.21.15.22:0/710405211 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f801018fbe0 con 0x7f8010100d60 2024-02-16T19:27:00.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:00.246+0000 7f80067fc700 1 -- 172.21.15.22:0/710405211 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f800c00c980 con 0x7f8010100d60 2024-02-16T19:27:00.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:00.246+0000 7f80067fc700 1 --2- 172.21.15.22:0/710405211 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f7ffc06c680 0x7f7ffc06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:27:00.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:00.247+0000 7f80067fc700 1 -- 172.21.15.22:0/710405211 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f800c014070 con 0x7f8010100d60 2024-02-16T19:27:00.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:00.247+0000 7f801494e700 1 --2- 172.21.15.22:0/710405211 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f7ffc06c680 0x7f7ffc06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:27:00.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:00.247+0000 7f801494e700 1 --2- 172.21.15.22:0/710405211 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f7ffc06c680 0x7f7ffc06eb30 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f800000b6f0 tx=0x7f8000005fb0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:27:00.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:00.251+0000 7f80067fc700 1 -- 172.21.15.22:0/710405211 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f800c05a010 con 0x7f8010100d60 2024-02-16T19:27:00.386 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:26:59 smithi117 ceph-mon[35309]: pgmap v201: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-02-16T19:27:00.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:00.450+0000 7f80173b3700 1 -- 172.21.15.22:0/710405211 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7f801002cd90 con 0x7f8010100d60 2024-02-16T19:27:00.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:00.450+0000 7f80067fc700 1 -- 172.21.15.22:0/710405211 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v21)=0 v21) v1 ==== 163+0+0 (secure 0 0 0) 0x7f800c059ba0 con 0x7f8010100d60 2024-02-16T19:27:00.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:00.453+0000 7f80173b3700 1 -- 172.21.15.22:0/710405211 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f7ffc06c680 msgr2=0x7f7ffc06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:27:00.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:00.453+0000 7f80173b3700 1 --2- 172.21.15.22:0/710405211 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f7ffc06c680 0x7f7ffc06eb30 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f800000b6f0 tx=0x7f8000005fb0 comp rx=0 tx=0).stop 2024-02-16T19:27:00.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:00.453+0000 7f80173b3700 1 -- 172.21.15.22:0/710405211 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8010100d60 msgr2=0x7f8010195ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:27:00.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:00.453+0000 7f80173b3700 1 --2- 172.21.15.22:0/710405211 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8010100d60 0x7f8010195ae0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f800c00e8f0 tx=0x7f800c00ec00 comp rx=0 tx=0).stop 2024-02-16T19:27:00.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:00.453+0000 7f80173b3700 1 -- 172.21.15.22:0/710405211 shutdown_connections 2024-02-16T19:27:00.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:00.453+0000 7f80173b3700 1 --2- 172.21.15.22:0/710405211 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8010100d60 0x7f8010195ae0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:00.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:00.453+0000 7f80173b3700 1 --2- 172.21.15.22:0/710405211 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f7ffc06c680 0x7f7ffc06eb30 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:00.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:00.453+0000 7f80173b3700 1 --2- 172.21.15.22:0/710405211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8010103680 0x7f8010196020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:00.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:00.453+0000 7f80173b3700 1 -- 172.21.15.22:0/710405211 >> 172.21.15.22:0/710405211 conn(0x7f80100fa780 msgr2=0x7f80100fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:27:00.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:00.453+0000 7f80173b3700 1 -- 172.21.15.22:0/710405211 shutdown_connections 2024-02-16T19:27:00.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:00.453+0000 7f80173b3700 1 -- 172.21.15.22:0/710405211 wait complete. 2024-02-16T19:27:01.113 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph config set global log_to_journald false --force' 2024-02-16T19:27:01.338 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi022/config 2024-02-16T19:27:02.182 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:27:01 smithi022 ceph-mon[29221]: pgmap v202: 65 pgs: 65 active+clean; 585 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 682 B/s wr, 0 op/s 2024-02-16T19:27:02.386 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:27:01 smithi117 ceph-mon[35309]: pgmap v202: 65 pgs: 65 active+clean; 585 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 682 B/s wr, 0 op/s 2024-02-16T19:27:03.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:03.147+0000 7f65b09a7700 1 -- 172.21.15.22:0/3125600276 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f65ac100240 msgr2=0x7f65ac100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:27:03.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:03.147+0000 7f65b09a7700 1 --2- 172.21.15.22:0/3125600276 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f65ac100240 0x7f65ac100650 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f659c009910 tx=0x7f659c009c20 comp rx=0 tx=0).stop 2024-02-16T19:27:03.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:03.147+0000 7f65b09a7700 1 -- 172.21.15.22:0/3125600276 shutdown_connections 2024-02-16T19:27:03.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:03.147+0000 7f65b09a7700 1 --2- 172.21.15.22:0/3125600276 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65ac101440 0x7f65ac101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:03.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:03.147+0000 7f65b09a7700 1 --2- 172.21.15.22:0/3125600276 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f65ac100240 0x7f65ac100650 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:03.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:03.147+0000 7f65b09a7700 1 -- 172.21.15.22:0/3125600276 >> 172.21.15.22:0/3125600276 conn(0x7f65ac0fb7f0 msgr2=0x7f65ac0fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:27:03.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:03.147+0000 7f65b09a7700 1 -- 172.21.15.22:0/3125600276 shutdown_connections 2024-02-16T19:27:03.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:03.147+0000 7f65b09a7700 1 -- 172.21.15.22:0/3125600276 wait complete. 2024-02-16T19:27:03.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:03.148+0000 7f65b09a7700 1 Processor -- start 2024-02-16T19:27:03.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:03.148+0000 7f65b09a7700 1 -- start start 2024-02-16T19:27:03.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:03.148+0000 7f65b09a7700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f65ac100240 0x7f65ac195ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:27:03.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:03.148+0000 7f65b09a7700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65ac101440 0x7f65ac195ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:27:03.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:03.148+0000 7f65b09a7700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65ac196610 con 0x7f65ac100240 2024-02-16T19:27:03.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:03.149+0000 7f65b09a7700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65ac196750 con 0x7f65ac101440 2024-02-16T19:27:03.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:03.149+0000 7f65aa59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f65ac100240 0x7f65ac195ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:27:03.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:03.149+0000 7f65aa59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f65ac100240 0x7f65ac195ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46978/0 (socket says 172.21.15.22:46978) 2024-02-16T19:27:03.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:03.149+0000 7f65aa59c700 1 -- 172.21.15.22:0/142233966 learned_addr learned my addr 172.21.15.22:0/142233966 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:27:03.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:03.149+0000 7f65a9d9b700 1 --2- 172.21.15.22:0/142233966 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65ac101440 0x7f65ac195ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:27:03.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:03.149+0000 7f65aa59c700 1 -- 172.21.15.22:0/142233966 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65ac101440 msgr2=0x7f65ac195ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:27:03.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:03.149+0000 7f65aa59c700 1 --2- 172.21.15.22:0/142233966 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65ac101440 0x7f65ac195ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:03.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:03.149+0000 7f65aa59c700 1 -- 172.21.15.22:0/142233966 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f659c0095f0 con 0x7f65ac100240 2024-02-16T19:27:03.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:03.149+0000 7f65aa59c700 1 --2- 172.21.15.22:0/142233966 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f65ac100240 0x7f65ac195ab0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f659c000c00 tx=0x7f659c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:27:03.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:03.149+0000 7f659b7fe700 1 -- 172.21.15.22:0/142233966 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f659c01d070 con 0x7f65ac100240 2024-02-16T19:27:03.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:03.149+0000 7f659b7fe700 1 -- 172.21.15.22:0/142233966 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f659c005dc0 con 0x7f65ac100240 2024-02-16T19:27:03.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:03.149+0000 7f65b09a7700 1 -- 172.21.15.22:0/142233966 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65ac19b1a0 con 0x7f65ac100240 2024-02-16T19:27:03.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:03.149+0000 7f659b7fe700 1 -- 172.21.15.22:0/142233966 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f659c022470 con 0x7f65ac100240 2024-02-16T19:27:03.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:03.149+0000 7f65b09a7700 1 -- 172.21.15.22:0/142233966 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65ac19b690 con 0x7f65ac100240 2024-02-16T19:27:03.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:03.150+0000 7f65b09a7700 1 -- 172.21.15.22:0/142233966 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65ac108d90 con 0x7f65ac100240 2024-02-16T19:27:03.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:03.151+0000 7f659b7fe700 1 -- 172.21.15.22:0/142233966 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f659c003810 con 0x7f65ac100240 2024-02-16T19:27:03.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:03.151+0000 7f659b7fe700 1 --2- 172.21.15.22:0/142233966 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f659406c680 0x7f659406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:27:03.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:03.151+0000 7f659b7fe700 1 -- 172.21.15.22:0/142233966 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f659c08cdc0 con 0x7f65ac100240 2024-02-16T19:27:03.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:03.151+0000 7f65a9d9b700 1 --2- 172.21.15.22:0/142233966 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f659406c680 0x7f659406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:27:03.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:03.152+0000 7f65a9d9b700 1 --2- 172.21.15.22:0/142233966 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f659406c680 0x7f659406eb30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f65a0009c60 tx=0x7f65a0009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:27:03.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:03.156+0000 7f659b7fe700 1 -- 172.21.15.22:0/142233966 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f659c05aed0 con 0x7f65ac100240 2024-02-16T19:27:03.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:03.359+0000 7f65b09a7700 1 -- 172.21.15.22:0/142233966 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7f65ac108f50 con 0x7f65ac100240 2024-02-16T19:27:03.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:03.360+0000 7f659b7fe700 1 -- 172.21.15.22:0/142233966 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v21)=0 v21) v1 ==== 135+0+0 (secure 0 0 0) 0x7f659c05aa60 con 0x7f65ac100240 2024-02-16T19:27:03.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:03.362+0000 7f65b09a7700 1 -- 172.21.15.22:0/142233966 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f659406c680 msgr2=0x7f659406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:27:03.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:03.362+0000 7f65b09a7700 1 --2- 172.21.15.22:0/142233966 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f659406c680 0x7f659406eb30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f65a0009c60 tx=0x7f65a0009260 comp rx=0 tx=0).stop 2024-02-16T19:27:03.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:03.362+0000 7f65b09a7700 1 -- 172.21.15.22:0/142233966 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f65ac100240 msgr2=0x7f65ac195ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:27:03.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:03.362+0000 7f65b09a7700 1 --2- 172.21.15.22:0/142233966 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f65ac100240 0x7f65ac195ab0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f659c000c00 tx=0x7f659c00f690 comp rx=0 tx=0).stop 2024-02-16T19:27:03.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:03.362+0000 7f65b09a7700 1 -- 172.21.15.22:0/142233966 shutdown_connections 2024-02-16T19:27:03.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:03.362+0000 7f65b09a7700 1 --2- 172.21.15.22:0/142233966 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f65ac100240 0x7f65ac195ab0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:03.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:03.362+0000 7f65b09a7700 1 --2- 172.21.15.22:0/142233966 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f659406c680 0x7f659406eb30 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:03.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:03.362+0000 7f65b09a7700 1 --2- 172.21.15.22:0/142233966 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65ac101440 0x7f65ac195ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:03.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:03.362+0000 7f65b09a7700 1 -- 172.21.15.22:0/142233966 >> 172.21.15.22:0/142233966 conn(0x7f65ac0fb7f0 msgr2=0x7f65ac104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:27:03.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:03.362+0000 7f65b09a7700 1 -- 172.21.15.22:0/142233966 shutdown_connections 2024-02-16T19:27:03.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:03.362+0000 7f65b09a7700 1 -- 172.21.15.22:0/142233966 wait complete. 2024-02-16T19:27:04.041 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:27:03 smithi022 ceph-mon[29221]: pgmap v203: 65 pgs: 65 active+clean; 585 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-02-16T19:27:04.065 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mgr' 2024-02-16T19:27:04.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:27:03 smithi117 ceph-mon[35309]: pgmap v203: 65 pgs: 65 active+clean; 585 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-02-16T19:27:04.286 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi022/config 2024-02-16T19:27:06.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:06.099+0000 7fec3fd45700 1 -- 172.21.15.22:0/718444333 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec380fe500 msgr2=0x7fec380fe910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:27:06.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:06.099+0000 7fec3fd45700 1 --2- 172.21.15.22:0/718444333 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec380fe500 0x7fec380fe910 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fec34009910 tx=0x7fec34009c20 comp rx=0 tx=0).stop 2024-02-16T19:27:06.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:06.099+0000 7fec3fd45700 1 -- 172.21.15.22:0/718444333 shutdown_connections 2024-02-16T19:27:06.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:06.099+0000 7fec3fd45700 1 --2- 172.21.15.22:0/718444333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec380fee50 0x7fec38101580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:06.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:06.099+0000 7fec3fd45700 1 --2- 172.21.15.22:0/718444333 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec380fe500 0x7fec380fe910 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:06.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:06.099+0000 7fec3fd45700 1 -- 172.21.15.22:0/718444333 >> 172.21.15.22:0/718444333 conn(0x7fec380f9f10 msgr2=0x7fec380fc340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:27:06.101 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:27:05 smithi022 ceph-mon[29221]: pgmap v204: 65 pgs: 65 active+clean; 585 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-02-16T19:27:06.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:06.099+0000 7fec3fd45700 1 -- 172.21.15.22:0/718444333 shutdown_connections 2024-02-16T19:27:06.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:06.100+0000 7fec3fd45700 1 -- 172.21.15.22:0/718444333 wait complete. 2024-02-16T19:27:06.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:06.101+0000 7fec3fd45700 1 Processor -- start 2024-02-16T19:27:06.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:06.101+0000 7fec3fd45700 1 -- start start 2024-02-16T19:27:06.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:06.101+0000 7fec3fd45700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec380fe500 0x7fec38100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:27:06.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:06.101+0000 7fec3fd45700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec380fee50 0x7fec380ff530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:27:06.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:06.101+0000 7fec3fd45700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec380ffa70 con 0x7fec380fee50 2024-02-16T19:27:06.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:06.101+0000 7fec3fd45700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec380ffbb0 con 0x7fec380fe500 2024-02-16T19:27:06.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:06.101+0000 7fec3d2e0700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec380fee50 0x7fec380ff530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:27:06.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:06.101+0000 7fec3d2e0700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec380fee50 0x7fec380ff530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42464/0 (socket says 172.21.15.22:42464) 2024-02-16T19:27:06.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:06.101+0000 7fec3d2e0700 1 -- 172.21.15.22:0/3511543299 learned_addr learned my addr 172.21.15.22:0/3511543299 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:27:06.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:06.101+0000 7fec3dae1700 1 --2- 172.21.15.22:0/3511543299 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec380fe500 0x7fec38100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:27:06.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:06.101+0000 7fec3d2e0700 1 -- 172.21.15.22:0/3511543299 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec380fe500 msgr2=0x7fec38100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:27:06.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:06.101+0000 7fec3d2e0700 1 --2- 172.21.15.22:0/3511543299 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec380fe500 0x7fec38100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:06.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:06.101+0000 7fec3d2e0700 1 -- 172.21.15.22:0/3511543299 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec340095f0 con 0x7fec380fee50 2024-02-16T19:27:06.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:06.101+0000 7fec3dae1700 1 --2- 172.21.15.22:0/3511543299 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec380fe500 0x7fec38100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T19:27:06.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:06.102+0000 7fec3d2e0700 1 --2- 172.21.15.22:0/3511543299 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec380fee50 0x7fec380ff530 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fec2800e8c0 tx=0x7fec2800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:27:06.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:06.102+0000 7fec2effd700 1 -- 172.21.15.22:0/3511543299 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec2800c9b0 con 0x7fec380fee50 2024-02-16T19:27:06.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:06.102+0000 7fec3fd45700 1 -- 172.21.15.22:0/3511543299 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec380ffe90 con 0x7fec380fee50 2024-02-16T19:27:06.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:06.102+0000 7fec2effd700 1 -- 172.21.15.22:0/3511543299 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fec28010430 con 0x7fec380fee50 2024-02-16T19:27:06.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:06.102+0000 7fec3fd45700 1 -- 172.21.15.22:0/3511543299 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec3819db90 con 0x7fec380fee50 2024-02-16T19:27:06.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:06.102+0000 7fec2effd700 1 -- 172.21.15.22:0/3511543299 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec28018650 con 0x7fec380fee50 2024-02-16T19:27:06.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:06.103+0000 7fec3fd45700 1 -- 172.21.15.22:0/3511543299 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec38108d50 con 0x7fec380fee50 2024-02-16T19:27:06.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:06.103+0000 7fec2effd700 1 -- 172.21.15.22:0/3511543299 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fec2800cb10 con 0x7fec380fee50 2024-02-16T19:27:06.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:06.104+0000 7fec2effd700 1 --2- 172.21.15.22:0/3511543299 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fec2406c680 0x7fec2406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:27:06.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:06.104+0000 7fec2effd700 1 -- 172.21.15.22:0/3511543299 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fec28014070 con 0x7fec380fee50 2024-02-16T19:27:06.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:06.104+0000 7fec3dae1700 1 --2- 172.21.15.22:0/3511543299 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fec2406c680 0x7fec2406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:27:06.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:06.104+0000 7fec3dae1700 1 --2- 172.21.15.22:0/3511543299 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fec2406c680 0x7fec2406eb30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fec34009fd0 tx=0x7fec34005940 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:27:06.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:06.108+0000 7fec2effd700 1 -- 172.21.15.22:0/3511543299 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fec28051720 con 0x7fec380fee50 2024-02-16T19:27:06.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:06.358+0000 7fec3fd45700 1 -- 172.21.15.22:0/3511543299 --> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:f78a58c0ffd401d1493058a1022c35f011d65275", "daemon_types": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7fec38105f80 con 0x7fec2406c680 2024-02-16T19:27:06.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:27:05 smithi117 ceph-mon[35309]: pgmap v204: 65 pgs: 65 active+clean; 585 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-02-16T19:27:08.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:27:07 smithi022 ceph-mon[29221]: from='client.14562 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:f78a58c0ffd401d1493058a1022c35f011d65275", "daemon_types": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:27:08.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:27:07 smithi022 ceph-mon[29221]: pgmap v205: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.1 KiB/s wr, 0 op/s 2024-02-16T19:27:08.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:27:07 smithi117 ceph-mon[35309]: from='client.14562 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:f78a58c0ffd401d1493058a1022c35f011d65275", "daemon_types": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:27:08.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:27:07 smithi117 ceph-mon[35309]: pgmap v205: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.1 KiB/s wr, 0 op/s 2024-02-16T19:27:10.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:27:09 smithi022 ceph-mon[29221]: pgmap v206: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s 2024-02-16T19:27:10.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:27:09 smithi117 ceph-mon[35309]: pgmap v206: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s 2024-02-16T19:27:12.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:27:11 smithi022 ceph-mon[29221]: pgmap v207: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s 2024-02-16T19:27:12.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:27:11 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:27:12.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:27:11 smithi117 ceph-mon[35309]: pgmap v207: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s 2024-02-16T19:27:12.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:27:11 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:27:14.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:27:13 smithi022 ceph-mon[29221]: pgmap v208: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-02-16T19:27:14.386 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:27:13 smithi117 ceph-mon[35309]: pgmap v208: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-02-16T19:27:16.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:27:15 smithi022 ceph-mon[29221]: pgmap v209: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-02-16T19:27:16.386 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:27:15 smithi117 ceph-mon[35309]: pgmap v209: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-02-16T19:27:18.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:27:17 smithi022 ceph-mon[29221]: pgmap v210: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-02-16T19:27:18.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:27:17 smithi117 ceph-mon[35309]: pgmap v210: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-02-16T19:27:20.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:27:19 smithi022 ceph-mon[29221]: pgmap v211: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-02-16T19:27:20.386 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:27:19 smithi117 ceph-mon[35309]: pgmap v211: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-02-16T19:27:22.189 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:27:21 smithi117 ceph-mon[35309]: pgmap v212: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:27:22.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:27:21 smithi022 ceph-mon[29221]: pgmap v212: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:27:24.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:27:23 smithi022 ceph-mon[29221]: pgmap v213: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:27:24.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:27:23 smithi117 ceph-mon[35309]: pgmap v213: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:27:26.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:27:25 smithi022 ceph-mon[29221]: pgmap v214: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:27:26.386 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:27:25 smithi117 ceph-mon[35309]: pgmap v214: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:27:27.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:27:26 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:27:27.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:27:26 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:27:28.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:27:27 smithi022 ceph-mon[29221]: pgmap v215: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:27:28.386 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:27:27 smithi117 ceph-mon[35309]: pgmap v215: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:27:30.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:27:29 smithi022 ceph-mon[29221]: pgmap v216: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:27:30.386 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:27:29 smithi117 ceph-mon[35309]: pgmap v216: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:27:32.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:27:31 smithi022 ceph-mon[29221]: pgmap v217: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:27:32.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:27:31 smithi117 ceph-mon[35309]: pgmap v217: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:27:34.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:27:33 smithi022 ceph-mon[29221]: pgmap v218: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:27:34.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:27:33 smithi117 ceph-mon[35309]: pgmap v218: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:27:35.925 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:27:35 smithi022 ceph-mon[29221]: pgmap v219: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:27:36.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:27:35 smithi117 ceph-mon[35309]: pgmap v219: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:27:38.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:27:37 smithi022 ceph-mon[29221]: pgmap v220: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:27:38.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:27:37 smithi117 ceph-mon[35309]: pgmap v220: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:27:40.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:27:39 smithi022 ceph-mon[29221]: pgmap v221: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:27:40.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:27:39 smithi117 ceph-mon[35309]: pgmap v221: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:27:42.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:27:41 smithi022 ceph-mon[29221]: pgmap v222: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:27:42.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:27:41 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:27:42.386 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:27:41 smithi117 ceph-mon[35309]: pgmap v222: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:27:42.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:27:41 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:27:44.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:27:43 smithi117 ceph-mon[35309]: pgmap v223: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:27:44.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:27:43 smithi022 ceph-mon[29221]: pgmap v223: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:27:46.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:27:45 smithi022 ceph-mon[29221]: pgmap v224: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:27:46.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:27:45 smithi117 ceph-mon[35309]: pgmap v224: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:27:48.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:27:47 smithi022 ceph-mon[29221]: pgmap v225: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:27:48.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:27:47 smithi117 ceph-mon[35309]: pgmap v225: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:27:49.136 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:27:48 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:27:49.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:27:48 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:27:49.639 INFO:tasks.workunit.client.1.smithi117.stderr:Note: switching to 'ed7ab725e1a43615009dfb74a42bf33d80081bc8'. 2024-02-16T19:27:49.640 INFO:tasks.workunit.client.1.smithi117.stderr: 2024-02-16T19:27:49.640 INFO:tasks.workunit.client.1.smithi117.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-02-16T19:27:49.640 INFO:tasks.workunit.client.1.smithi117.stderr:changes and commit them, and you can discard any commits you make in this 2024-02-16T19:27:49.640 INFO:tasks.workunit.client.1.smithi117.stderr:state without impacting any branches by switching back to a branch. 2024-02-16T19:27:49.640 INFO:tasks.workunit.client.1.smithi117.stderr: 2024-02-16T19:27:49.640 INFO:tasks.workunit.client.1.smithi117.stderr:If you want to create a new branch to retain commits you create, you may 2024-02-16T19:27:49.640 INFO:tasks.workunit.client.1.smithi117.stderr:do so (now or later) by using -c with the switch command. Example: 2024-02-16T19:27:49.640 INFO:tasks.workunit.client.1.smithi117.stderr: 2024-02-16T19:27:49.640 INFO:tasks.workunit.client.1.smithi117.stderr: git switch -c 2024-02-16T19:27:49.640 INFO:tasks.workunit.client.1.smithi117.stderr: 2024-02-16T19:27:49.640 INFO:tasks.workunit.client.1.smithi117.stderr:Or undo this operation with: 2024-02-16T19:27:49.640 INFO:tasks.workunit.client.1.smithi117.stderr: 2024-02-16T19:27:49.640 INFO:tasks.workunit.client.1.smithi117.stderr: git switch - 2024-02-16T19:27:49.640 INFO:tasks.workunit.client.1.smithi117.stderr: 2024-02-16T19:27:49.641 INFO:tasks.workunit.client.1.smithi117.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-02-16T19:27:49.641 INFO:tasks.workunit.client.1.smithi117.stderr: 2024-02-16T19:27:49.641 INFO:tasks.workunit.client.1.smithi117.stderr:HEAD is now at ed7ab725e1a qa: test fuse/kclient for mds upgrade seq 2024-02-16T19:27:49.644 DEBUG:teuthology.orchestra.run.smithi117:> cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.1 2024-02-16T19:27:49.708 INFO:tasks.workunit.client.1.smithi117.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-02-16T19:27:49.709 INFO:tasks.workunit.client.1.smithi117.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-02-16T19:27:49.710 INFO:tasks.workunit.client.1.smithi117.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-02-16T19:27:50.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:27:49 smithi022 ceph-mon[29221]: pgmap v226: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:27:50.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:27:49 smithi117 ceph-mon[35309]: pgmap v226: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:27:50.476 INFO:tasks.workunit.client.1.smithi117.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-02-16T19:27:50.554 INFO:tasks.workunit.client.1.smithi117.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-02-16T19:27:50.595 INFO:tasks.workunit.client.1.smithi117.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-02-16T19:27:50.597 INFO:tasks.workunit.client.1.smithi117.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-02-16T19:27:50.597 INFO:tasks.workunit.client.1.smithi117.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-02-16T19:27:50.635 INFO:tasks.workunit.client.1.smithi117.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-02-16T19:27:50.638 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-02-16T19:27:50.639 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/home/ubuntu/cephtest/workunits.list.client.1 of=/dev/stdout 2024-02-16T19:27:50.694 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.1... 2024-02-16T19:27:50.695 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-02-16T19:27:50.695 DEBUG:teuthology.orchestra.run.smithi117:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ed7ab725e1a43615009dfb74a42bf33d80081bc8 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh 2024-02-16T19:27:50.758 INFO:tasks.workunit.client.1.smithi117.stderr:+ mkdir -p fsstress 2024-02-16T19:27:50.761 INFO:tasks.workunit.client.1.smithi117.stderr:+ pushd fsstress 2024-02-16T19:27:50.762 INFO:tasks.workunit.client.1.smithi117.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-02-16T19:27:50.762 INFO:tasks.workunit.client.1.smithi117.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-02-16T19:27:52.203 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:27:51 smithi117 ceph-mon[35309]: pgmap v227: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:27:52.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:27:51 smithi022 ceph-mon[29221]: pgmap v227: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:27:52.863 INFO:tasks.workunit.client.1.smithi117.stderr:+ tar xzf ltp-full.tgz 2024-02-16T19:27:54.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:27:53 smithi022 ceph-mon[29221]: pgmap v228: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:27:54.386 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:27:53 smithi117 ceph-mon[35309]: pgmap v228: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:27:55.188 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:27:54 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:27:55.189 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:27:54 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:27:55.189 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:27:54 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:27:55.386 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:27:54 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:27:55.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:27:54 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:27:55.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:27:54 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:27:56.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:27:55 smithi022 ceph-mon[29221]: pgmap v229: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:27:56.386 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:27:55 smithi117 ceph-mon[35309]: pgmap v229: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:27:56.465 INFO:teuthology.orchestra.run.smithi022.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:f78a58c0ffd401d1493058a1022c35f011d65275 2024-02-16T19:27:56.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:56.463+0000 7fec2effd700 1 -- 172.21.15.22:0/3511543299 <== mgr.14217 v2:172.21.15.22:6800/2253867279 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7fec38105f80 con 0x7fec2406c680 2024-02-16T19:27:56.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:56.466+0000 7fec3fd45700 1 -- 172.21.15.22:0/3511543299 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fec2406c680 msgr2=0x7fec2406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:27:56.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:56.466+0000 7fec3fd45700 1 --2- 172.21.15.22:0/3511543299 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fec2406c680 0x7fec2406eb30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fec34009fd0 tx=0x7fec34005940 comp rx=0 tx=0).stop 2024-02-16T19:27:56.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:56.466+0000 7fec3fd45700 1 -- 172.21.15.22:0/3511543299 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec380fee50 msgr2=0x7fec380ff530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:27:56.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:56.466+0000 7fec3fd45700 1 --2- 172.21.15.22:0/3511543299 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec380fee50 0x7fec380ff530 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fec2800e8c0 tx=0x7fec2800ebd0 comp rx=0 tx=0).stop 2024-02-16T19:27:56.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:56.466+0000 7fec3fd45700 1 -- 172.21.15.22:0/3511543299 shutdown_connections 2024-02-16T19:27:56.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:56.466+0000 7fec3fd45700 1 --2- 172.21.15.22:0/3511543299 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fec2406c680 0x7fec2406eb30 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:56.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:56.466+0000 7fec3fd45700 1 --2- 172.21.15.22:0/3511543299 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec380fe500 0x7fec38100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:56.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:56.466+0000 7fec3fd45700 1 --2- 172.21.15.22:0/3511543299 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec380fee50 0x7fec380ff530 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:56.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:56.466+0000 7fec3fd45700 1 -- 172.21.15.22:0/3511543299 >> 172.21.15.22:0/3511543299 conn(0x7fec380f9f10 msgr2=0x7fec381047b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:27:56.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:56.466+0000 7fec3fd45700 1 -- 172.21.15.22:0/3511543299 shutdown_connections 2024-02-16T19:27:56.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:27:56.466+0000 7fec3fd45700 1 -- 172.21.15.22:0/3511543299 wait complete. 2024-02-16T19:27:57.279 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done' 2024-02-16T19:27:57.460 INFO:tasks.workunit.client.0.smithi022.stderr:Note: switching to 'ed7ab725e1a43615009dfb74a42bf33d80081bc8'. 2024-02-16T19:27:57.460 INFO:tasks.workunit.client.0.smithi022.stderr: 2024-02-16T19:27:57.461 INFO:tasks.workunit.client.0.smithi022.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-02-16T19:27:57.461 INFO:tasks.workunit.client.0.smithi022.stderr:changes and commit them, and you can discard any commits you make in this 2024-02-16T19:27:57.461 INFO:tasks.workunit.client.0.smithi022.stderr:state without impacting any branches by switching back to a branch. 2024-02-16T19:27:57.461 INFO:tasks.workunit.client.0.smithi022.stderr: 2024-02-16T19:27:57.461 INFO:tasks.workunit.client.0.smithi022.stderr:If you want to create a new branch to retain commits you create, you may 2024-02-16T19:27:57.461 INFO:tasks.workunit.client.0.smithi022.stderr:do so (now or later) by using -c with the switch command. Example: 2024-02-16T19:27:57.461 INFO:tasks.workunit.client.0.smithi022.stderr: 2024-02-16T19:27:57.461 INFO:tasks.workunit.client.0.smithi022.stderr: git switch -c 2024-02-16T19:27:57.461 INFO:tasks.workunit.client.0.smithi022.stderr: 2024-02-16T19:27:57.461 INFO:tasks.workunit.client.0.smithi022.stderr:Or undo this operation with: 2024-02-16T19:27:57.461 INFO:tasks.workunit.client.0.smithi022.stderr: 2024-02-16T19:27:57.461 INFO:tasks.workunit.client.0.smithi022.stderr: git switch - 2024-02-16T19:27:57.461 INFO:tasks.workunit.client.0.smithi022.stderr: 2024-02-16T19:27:57.462 INFO:tasks.workunit.client.0.smithi022.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-02-16T19:27:57.462 INFO:tasks.workunit.client.0.smithi022.stderr: 2024-02-16T19:27:57.462 INFO:tasks.workunit.client.0.smithi022.stderr:HEAD is now at ed7ab725e1a qa: test fuse/kclient for mds upgrade seq 2024-02-16T19:27:57.466 DEBUG:teuthology.orchestra.run.smithi022:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-02-16T19:27:57.714 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:27:57 smithi022 ceph-mon[29221]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:f78a58c0ffd401d1493058a1022c35f011d65275 2024-02-16T19:27:57.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:27:57 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:27:57.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:27:57 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:27:57.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:27:57 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:27:57.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:27:57 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:27:57.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:27:57 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:27:57.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:27:57 smithi022 ceph-mon[29221]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:f78a58c0ffd401d1493058a1022c35f011d65275 2024-02-16T19:27:57.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:27:57 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:27:57.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:27:57 smithi022 ceph-mon[29221]: pgmap v230: 65 pgs: 65 active+clean; 6.6 MiB data, 185 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 517 KiB/s wr, 17 op/s 2024-02-16T19:27:57.762 INFO:tasks.workunit.client.0.smithi022.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-02-16T19:27:57.765 INFO:tasks.workunit.client.0.smithi022.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-02-16T19:27:57.765 INFO:tasks.workunit.client.0.smithi022.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-02-16T19:27:57.886 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:27:57 smithi117 ceph-mon[35309]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:f78a58c0ffd401d1493058a1022c35f011d65275 2024-02-16T19:27:57.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:27:57 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:27:57.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:27:57 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:27:57.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:27:57 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:27:57.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:27:57 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:27:57.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:27:57 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:27:57.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:27:57 smithi117 ceph-mon[35309]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:f78a58c0ffd401d1493058a1022c35f011d65275 2024-02-16T19:27:57.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:27:57 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:27:57.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:27:57 smithi117 ceph-mon[35309]: pgmap v230: 65 pgs: 65 active+clean; 6.6 MiB data, 185 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 517 KiB/s wr, 17 op/s 2024-02-16T19:27:58.597 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi022/config 2024-02-16T19:27:59.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:27:59 smithi022 ceph-mon[29221]: pgmap v231: 65 pgs: 65 active+clean; 14 MiB data, 203 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.1 MiB/s wr, 32 op/s 2024-02-16T19:27:59.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:27:59 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:27:59.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:27:59 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:27:59.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:27:59 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:28:00.198 INFO:tasks.workunit.client.0.smithi022.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-02-16T19:28:00.327 INFO:tasks.workunit.client.0.smithi022.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-02-16T19:28:00.367 INFO:tasks.workunit.client.0.smithi022.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-02-16T19:28:00.369 INFO:tasks.workunit.client.0.smithi022.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-02-16T19:28:00.369 INFO:tasks.workunit.client.0.smithi022.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-02-16T19:28:00.386 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:27:59 smithi117 ceph-mon[35309]: pgmap v231: 65 pgs: 65 active+clean; 14 MiB data, 203 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.1 MiB/s wr, 32 op/s 2024-02-16T19:28:00.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:27:59 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:28:00.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:27:59 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:28:00.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:27:59 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:28:00.415 INFO:tasks.workunit.client.0.smithi022.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-02-16T19:28:00.418 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-02-16T19:28:00.418 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-02-16T19:28:00.433 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.0... 2024-02-16T19:28:00.434 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-02-16T19:28:00.434 DEBUG:teuthology.orchestra.run.smithi022:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ed7ab725e1a43615009dfb74a42bf33d80081bc8 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh 2024-02-16T19:28:00.551 INFO:tasks.workunit.client.0.smithi022.stderr:+ mkdir -p fsstress 2024-02-16T19:28:00.554 INFO:tasks.workunit.client.0.smithi022.stderr:+ pushd fsstress 2024-02-16T19:28:00.556 INFO:tasks.workunit.client.0.smithi022.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-02-16T19:28:00.556 INFO:tasks.workunit.client.0.smithi022.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-02-16T19:28:01.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:00 smithi022 ceph-mon[29221]: Upgrade: Target is version 19.0.0-1450-gf78a58c0 (unknown) 2024-02-16T19:28:01.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:00 smithi022 ceph-mon[29221]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, digests ['quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526'] 2024-02-16T19:28:01.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:00 smithi022 ceph-mon[29221]: Upgrade: Need to upgrade myself (mgr.smithi022.gmdcch) 2024-02-16T19:28:01.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:00 smithi022 ceph-mon[29221]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 on smithi117 2024-02-16T19:28:01.386 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:00 smithi117 ceph-mon[35309]: Upgrade: Target is version 19.0.0-1450-gf78a58c0 (unknown) 2024-02-16T19:28:01.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:00 smithi117 ceph-mon[35309]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, digests ['quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526'] 2024-02-16T19:28:01.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:00 smithi117 ceph-mon[35309]: Upgrade: Need to upgrade myself (mgr.smithi022.gmdcch) 2024-02-16T19:28:01.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:00 smithi117 ceph-mon[35309]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 on smithi117 2024-02-16T19:28:01.559 INFO:tasks.workunit.client.0.smithi022.stderr:+ tar xzf ltp-full.tgz 2024-02-16T19:28:02.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:01 smithi022 ceph-mon[29221]: pgmap v232: 65 pgs: 65 active+clean; 19 MiB data, 219 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.6 MiB/s wr, 58 op/s 2024-02-16T19:28:02.386 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:01 smithi117 ceph-mon[35309]: pgmap v232: 65 pgs: 65 active+clean; 19 MiB data, 219 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.6 MiB/s wr, 58 op/s 2024-02-16T19:28:03.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:03.825+0000 7f5b393fa700 1 -- 172.21.15.22:0/2596818820 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b34100350 msgr2=0x7f5b341007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:03.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:03.825+0000 7f5b393fa700 1 --2- 172.21.15.22:0/2596818820 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b34100350 0x7f5b341007a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f5b28009910 tx=0x7f5b28009c20 comp rx=0 tx=0).stop 2024-02-16T19:28:03.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:03.825+0000 7f5b393fa700 1 -- 172.21.15.22:0/2596818820 shutdown_connections 2024-02-16T19:28:03.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:03.825+0000 7f5b393fa700 1 --2- 172.21.15.22:0/2596818820 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b34100350 0x7f5b341007a0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:03.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:03.825+0000 7f5b393fa700 1 --2- 172.21.15.22:0/2596818820 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b340ff150 0x7f5b340ff560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:03.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:03.825+0000 7f5b393fa700 1 -- 172.21.15.22:0/2596818820 >> 172.21.15.22:0/2596818820 conn(0x7f5b340fa700 msgr2=0x7f5b340fcb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:28:03.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:03.825+0000 7f5b393fa700 1 -- 172.21.15.22:0/2596818820 shutdown_connections 2024-02-16T19:28:03.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:03.825+0000 7f5b393fa700 1 -- 172.21.15.22:0/2596818820 wait complete. 2024-02-16T19:28:03.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:03.826+0000 7f5b393fa700 1 Processor -- start 2024-02-16T19:28:03.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:03.826+0000 7f5b393fa700 1 -- start start 2024-02-16T19:28:03.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:03.827+0000 7f5b393fa700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b340ff150 0x7f5b341030e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:28:03.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:03.827+0000 7f5b32ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b340ff150 0x7f5b341030e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:28:03.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:03.827+0000 7f5b393fa700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b34100350 0x7f5b34101730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:28:03.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:03.827+0000 7f5b32ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b340ff150 0x7f5b341030e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58784/0 (socket says 172.21.15.22:58784) 2024-02-16T19:28:03.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:03.827+0000 7f5b393fa700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b34101c70 con 0x7f5b340ff150 2024-02-16T19:28:03.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:03.827+0000 7f5b393fa700 1 -- 172.21.15.22:0/1887613946 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b34101db0 con 0x7f5b34100350 2024-02-16T19:28:03.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:03.827+0000 7f5b32ffd700 1 -- 172.21.15.22:0/1887613946 learned_addr learned my addr 172.21.15.22:0/1887613946 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:28:03.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:03.827+0000 7f5b327fc700 1 --2- 172.21.15.22:0/1887613946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b34100350 0x7f5b34101730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:28:03.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:03.827+0000 7f5b32ffd700 1 -- 172.21.15.22:0/1887613946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b34100350 msgr2=0x7f5b34101730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:03.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:03.827+0000 7f5b32ffd700 1 --2- 172.21.15.22:0/1887613946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b34100350 0x7f5b34101730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:03.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:03.827+0000 7f5b32ffd700 1 -- 172.21.15.22:0/1887613946 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b280095f0 con 0x7f5b340ff150 2024-02-16T19:28:03.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:03.828+0000 7f5b32ffd700 1 --2- 172.21.15.22:0/1887613946 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b340ff150 0x7f5b341030e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f5b2400ecb0 tx=0x7f5b24009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:28:03.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:03.828+0000 7f5b1bfff700 1 -- 172.21.15.22:0/1887613946 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b24009790 con 0x7f5b340ff150 2024-02-16T19:28:03.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:03.828+0000 7f5b393fa700 1 -- 172.21.15.22:0/1887613946 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b34102090 con 0x7f5b340ff150 2024-02-16T19:28:03.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:03.828+0000 7f5b393fa700 1 -- 172.21.15.22:0/1887613946 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b3419db00 con 0x7f5b340ff150 2024-02-16T19:28:03.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:03.828+0000 7f5b1bfff700 1 -- 172.21.15.22:0/1887613946 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5b2400c8e0 con 0x7f5b340ff150 2024-02-16T19:28:03.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:03.828+0000 7f5b1bfff700 1 -- 172.21.15.22:0/1887613946 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b24003dc0 con 0x7f5b340ff150 2024-02-16T19:28:03.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:03.829+0000 7f5b393fa700 1 -- 172.21.15.22:0/1887613946 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b34107ca0 con 0x7f5b340ff150 2024-02-16T19:28:03.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:03.830+0000 7f5b1bfff700 1 -- 172.21.15.22:0/1887613946 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f5b24018430 con 0x7f5b340ff150 2024-02-16T19:28:03.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:03.830+0000 7f5b1bfff700 1 --2- 172.21.15.22:0/1887613946 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f5b1c06c680 0x7f5b1c06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:28:03.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:03.830+0000 7f5b1bfff700 1 -- 172.21.15.22:0/1887613946 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f5b24014070 con 0x7f5b340ff150 2024-02-16T19:28:03.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:03.830+0000 7f5b327fc700 1 --2- 172.21.15.22:0/1887613946 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f5b1c06c680 0x7f5b1c06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:28:03.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:03.831+0000 7f5b327fc700 1 --2- 172.21.15.22:0/1887613946 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f5b1c06c680 0x7f5b1c06eb30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5b28009fd0 tx=0x7f5b28005fd0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:28:03.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:03.835+0000 7f5b1bfff700 1 -- 172.21.15.22:0/1887613946 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f5b2405aec0 con 0x7f5b340ff150 2024-02-16T19:28:04.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.175+0000 7f5b393fa700 1 -- 172.21.15.22:0/1887613946 --> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5b340694e0 con 0x7f5b1c06c680 2024-02-16T19:28:04.176 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:03 smithi022 ceph-mon[29221]: pgmap v233: 65 pgs: 65 active+clean; 29 MiB data, 245 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 2.4 MiB/s wr, 83 op/s 2024-02-16T19:28:04.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.176+0000 7f5b1bfff700 1 -- 172.21.15.22:0/1887613946 <== mgr.14217 v2:172.21.15.22:6800/2253867279 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f5b340694e0 con 0x7f5b1c06c680 2024-02-16T19:28:04.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.179+0000 7f5b393fa700 1 -- 172.21.15.22:0/1887613946 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f5b1c06c680 msgr2=0x7f5b1c06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:04.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.179+0000 7f5b393fa700 1 --2- 172.21.15.22:0/1887613946 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f5b1c06c680 0x7f5b1c06eb30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5b28009fd0 tx=0x7f5b28005fd0 comp rx=0 tx=0).stop 2024-02-16T19:28:04.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.179+0000 7f5b393fa700 1 -- 172.21.15.22:0/1887613946 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b340ff150 msgr2=0x7f5b341030e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:04.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.179+0000 7f5b393fa700 1 --2- 172.21.15.22:0/1887613946 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b340ff150 0x7f5b341030e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f5b2400ecb0 tx=0x7f5b24009ea0 comp rx=0 tx=0).stop 2024-02-16T19:28:04.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.179+0000 7f5b393fa700 1 -- 172.21.15.22:0/1887613946 shutdown_connections 2024-02-16T19:28:04.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.179+0000 7f5b393fa700 1 --2- 172.21.15.22:0/1887613946 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b340ff150 0x7f5b341030e0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:04.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.179+0000 7f5b393fa700 1 --2- 172.21.15.22:0/1887613946 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f5b1c06c680 0x7f5b1c06eb30 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:04.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.179+0000 7f5b393fa700 1 --2- 172.21.15.22:0/1887613946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b34100350 0x7f5b34101730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:04.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.179+0000 7f5b393fa700 1 -- 172.21.15.22:0/1887613946 >> 172.21.15.22:0/1887613946 conn(0x7f5b340fa700 msgr2=0x7f5b34068dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:28:04.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.179+0000 7f5b393fa700 1 -- 172.21.15.22:0/1887613946 shutdown_connections 2024-02-16T19:28:04.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.179+0000 7f5b393fa700 1 -- 172.21.15.22:0/1887613946 wait complete. 2024-02-16T19:28:04.194 INFO:teuthology.orchestra.run.smithi022.stdout:true 2024-02-16T19:28:04.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.324+0000 7fb1a78c5700 1 -- 172.21.15.22:0/3183235340 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1a01003f0 msgr2=0x7fb1a0100860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:04.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.324+0000 7fb1a78c5700 1 --2- 172.21.15.22:0/3183235340 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1a01003f0 0x7fb1a0100860 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb190009910 tx=0x7fb190009c20 comp rx=0 tx=0).stop 2024-02-16T19:28:04.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.324+0000 7fb1a78c5700 1 -- 172.21.15.22:0/3183235340 shutdown_connections 2024-02-16T19:28:04.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.324+0000 7fb1a78c5700 1 --2- 172.21.15.22:0/3183235340 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1a01003f0 0x7fb1a0100860 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:04.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.324+0000 7fb1a78c5700 1 --2- 172.21.15.22:0/3183235340 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1a00ff150 0x7fb1a00ff560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:04.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.324+0000 7fb1a78c5700 1 -- 172.21.15.22:0/3183235340 >> 172.21.15.22:0/3183235340 conn(0x7fb1a00fa7a0 msgr2=0x7fb1a00fcbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:28:04.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.324+0000 7fb1a78c5700 1 -- 172.21.15.22:0/3183235340 shutdown_connections 2024-02-16T19:28:04.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.324+0000 7fb1a78c5700 1 -- 172.21.15.22:0/3183235340 wait complete. 2024-02-16T19:28:04.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.325+0000 7fb1a78c5700 1 Processor -- start 2024-02-16T19:28:04.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.325+0000 7fb1a78c5700 1 -- start start 2024-02-16T19:28:04.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.325+0000 7fb1a78c5700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1a00ff150 0x7fb1a0197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:28:04.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.325+0000 7fb1a78c5700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1a01003f0 0x7fb1a01984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:28:04.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.325+0000 7fb1a78c5700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a0198b10 con 0x7fb1a01003f0 2024-02-16T19:28:04.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.325+0000 7fb1a78c5700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a0198c50 con 0x7fb1a00ff150 2024-02-16T19:28:04.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.325+0000 7fb1a5661700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1a00ff150 0x7fb1a0197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:28:04.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.325+0000 7fb1a5661700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1a00ff150 0x7fb1a0197fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.22:52390/0 (socket says 172.21.15.22:52390) 2024-02-16T19:28:04.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.325+0000 7fb1a5661700 1 -- 172.21.15.22:0/2988561938 learned_addr learned my addr 172.21.15.22:0/2988561938 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:28:04.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.325+0000 7fb1a4e60700 1 --2- 172.21.15.22:0/2988561938 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1a01003f0 0x7fb1a01984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:28:04.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.326+0000 7fb1a5661700 1 -- 172.21.15.22:0/2988561938 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1a01003f0 msgr2=0x7fb1a01984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:04.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.326+0000 7fb1a5661700 1 --2- 172.21.15.22:0/2988561938 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1a01003f0 0x7fb1a01984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:04.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.326+0000 7fb1a5661700 1 -- 172.21.15.22:0/2988561938 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1900095f0 con 0x7fb1a00ff150 2024-02-16T19:28:04.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.326+0000 7fb1a5661700 1 --2- 172.21.15.22:0/2988561938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1a00ff150 0x7fb1a0197fb0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb19c00e920 tx=0x7fb19c00ec30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:28:04.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.326+0000 7fb1967fc700 1 -- 172.21.15.22:0/2988561938 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb19c00c9d0 con 0x7fb1a00ff150 2024-02-16T19:28:04.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.326+0000 7fb1967fc700 1 -- 172.21.15.22:0/2988561938 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb19c010430 con 0x7fb1a00ff150 2024-02-16T19:28:04.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.326+0000 7fb1967fc700 1 -- 172.21.15.22:0/2988561938 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb19c018650 con 0x7fb1a00ff150 2024-02-16T19:28:04.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.326+0000 7fb1a78c5700 1 -- 172.21.15.22:0/2988561938 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1a019d700 con 0x7fb1a00ff150 2024-02-16T19:28:04.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.326+0000 7fb1a78c5700 1 -- 172.21.15.22:0/2988561938 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1a019dc00 con 0x7fb1a00ff150 2024-02-16T19:28:04.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.327+0000 7fb1a78c5700 1 -- 172.21.15.22:0/2988561938 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1a0068ab0 con 0x7fb1a00ff150 2024-02-16T19:28:04.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.328+0000 7fb1967fc700 1 -- 172.21.15.22:0/2988561938 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fb19c0105a0 con 0x7fb1a00ff150 2024-02-16T19:28:04.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.329+0000 7fb1967fc700 1 --2- 172.21.15.22:0/2988561938 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fb18c06c6d0 0x7fb18c06eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:28:04.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.329+0000 7fb1967fc700 1 -- 172.21.15.22:0/2988561938 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fb19c014070 con 0x7fb1a00ff150 2024-02-16T19:28:04.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.329+0000 7fb1a4e60700 1 --2- 172.21.15.22:0/2988561938 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fb18c06c6d0 0x7fb18c06eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:28:04.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.330+0000 7fb1a4e60700 1 --2- 172.21.15.22:0/2988561938 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fb18c06c6d0 0x7fb18c06eb80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb190009fd0 tx=0x7fb190005940 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:28:04.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.333+0000 7fb1967fc700 1 -- 172.21.15.22:0/2988561938 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fb19c05a1c0 con 0x7fb1a00ff150 2024-02-16T19:28:04.386 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:03 smithi117 ceph-mon[35309]: pgmap v233: 65 pgs: 65 active+clean; 29 MiB data, 245 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 2.4 MiB/s wr, 83 op/s 2024-02-16T19:28:04.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.638+0000 7fb1a78c5700 1 -- 172.21.15.22:0/2988561938 --> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb1a00694e0 con 0x7fb18c06c6d0 2024-02-16T19:28:04.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.640+0000 7fb1967fc700 1 -- 172.21.15.22:0/2988561938 <== mgr.14217 v2:172.21.15.22:6800/2253867279 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7fb1a00694e0 con 0x7fb18c06c6d0 2024-02-16T19:28:04.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.643+0000 7fb1a78c5700 1 -- 172.21.15.22:0/2988561938 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fb18c06c6d0 msgr2=0x7fb18c06eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:04.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.643+0000 7fb1a78c5700 1 --2- 172.21.15.22:0/2988561938 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fb18c06c6d0 0x7fb18c06eb80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb190009fd0 tx=0x7fb190005940 comp rx=0 tx=0).stop 2024-02-16T19:28:04.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.643+0000 7fb1a78c5700 1 -- 172.21.15.22:0/2988561938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1a00ff150 msgr2=0x7fb1a0197fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:04.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.643+0000 7fb1a78c5700 1 --2- 172.21.15.22:0/2988561938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1a00ff150 0x7fb1a0197fb0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb19c00e920 tx=0x7fb19c00ec30 comp rx=0 tx=0).stop 2024-02-16T19:28:04.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.643+0000 7fb1a78c5700 1 -- 172.21.15.22:0/2988561938 shutdown_connections 2024-02-16T19:28:04.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.643+0000 7fb1a78c5700 1 --2- 172.21.15.22:0/2988561938 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fb18c06c6d0 0x7fb18c06eb80 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:04.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.643+0000 7fb1a78c5700 1 --2- 172.21.15.22:0/2988561938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1a00ff150 0x7fb1a0197fb0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:04.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.643+0000 7fb1a78c5700 1 --2- 172.21.15.22:0/2988561938 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1a01003f0 0x7fb1a01984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:04.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.643+0000 7fb1a78c5700 1 -- 172.21.15.22:0/2988561938 >> 172.21.15.22:0/2988561938 conn(0x7fb1a00fa7a0 msgr2=0x7fb1a0068dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:28:04.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.643+0000 7fb1a78c5700 1 -- 172.21.15.22:0/2988561938 shutdown_connections 2024-02-16T19:28:04.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.644+0000 7fb1a78c5700 1 -- 172.21.15.22:0/2988561938 wait complete. 2024-02-16T19:28:04.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.794+0000 7fc360eff700 1 -- 172.21.15.22:0/1323552687 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc35c0ffb60 msgr2=0x7fc35c0fffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:04.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.794+0000 7fc360eff700 1 --2- 172.21.15.22:0/1323552687 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc35c0ffb60 0x7fc35c0fffd0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc350009910 tx=0x7fc350009c20 comp rx=0 tx=0).stop 2024-02-16T19:28:04.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.794+0000 7fc360eff700 1 -- 172.21.15.22:0/1323552687 shutdown_connections 2024-02-16T19:28:04.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.794+0000 7fc360eff700 1 --2- 172.21.15.22:0/1323552687 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc35c0ffb60 0x7fc35c0fffd0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:04.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.794+0000 7fc360eff700 1 --2- 172.21.15.22:0/1323552687 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc35c0fe8c0 0x7fc35c0fecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:04.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.794+0000 7fc360eff700 1 -- 172.21.15.22:0/1323552687 >> 172.21.15.22:0/1323552687 conn(0x7fc35c0f9f10 msgr2=0x7fc35c0fc340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:28:04.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.794+0000 7fc360eff700 1 -- 172.21.15.22:0/1323552687 shutdown_connections 2024-02-16T19:28:04.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.794+0000 7fc360eff700 1 -- 172.21.15.22:0/1323552687 wait complete. 2024-02-16T19:28:04.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.795+0000 7fc360eff700 1 Processor -- start 2024-02-16T19:28:04.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.795+0000 7fc360eff700 1 -- start start 2024-02-16T19:28:04.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.795+0000 7fc360eff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc35c0fe8c0 0x7fc35c1028f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:28:04.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.795+0000 7fc360eff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc35c0ffb60 0x7fc35c100f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:28:04.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.795+0000 7fc360eff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc35c101480 con 0x7fc35c0ffb60 2024-02-16T19:28:04.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.795+0000 7fc360eff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc35c1015c0 con 0x7fc35c0fe8c0 2024-02-16T19:28:04.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.795+0000 7fc359d9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc35c0ffb60 0x7fc35c100f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:28:04.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.795+0000 7fc359d9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc35c0ffb60 0x7fc35c100f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36420/0 (socket says 172.21.15.22:36420) 2024-02-16T19:28:04.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.795+0000 7fc35a59c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc35c0fe8c0 0x7fc35c1028f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:28:04.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.795+0000 7fc359d9b700 1 -- 172.21.15.22:0/1636390899 learned_addr learned my addr 172.21.15.22:0/1636390899 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:28:04.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.796+0000 7fc35a59c700 1 -- 172.21.15.22:0/1636390899 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc35c0ffb60 msgr2=0x7fc35c100f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:04.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.796+0000 7fc35a59c700 1 --2- 172.21.15.22:0/1636390899 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc35c0ffb60 0x7fc35c100f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:04.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.796+0000 7fc35a59c700 1 -- 172.21.15.22:0/1636390899 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3500095f0 con 0x7fc35c0fe8c0 2024-02-16T19:28:04.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.796+0000 7fc359d9b700 1 --2- 172.21.15.22:0/1636390899 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc35c0ffb60 0x7fc35c100f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T19:28:04.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.796+0000 7fc35a59c700 1 --2- 172.21.15.22:0/1636390899 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc35c0fe8c0 0x7fc35c1028f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc34c00ea10 tx=0x7fc34c00ed20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:28:04.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.796+0000 7fc34b7fe700 1 -- 172.21.15.22:0/1636390899 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc34c00cca0 con 0x7fc35c0fe8c0 2024-02-16T19:28:04.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.796+0000 7fc360eff700 1 -- 172.21.15.22:0/1636390899 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc35c1018a0 con 0x7fc35c0fe8c0 2024-02-16T19:28:04.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.796+0000 7fc34b7fe700 1 -- 172.21.15.22:0/1636390899 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc34c00ce00 con 0x7fc35c0fe8c0 2024-02-16T19:28:04.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.796+0000 7fc34b7fe700 1 -- 172.21.15.22:0/1636390899 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc34c0157f0 con 0x7fc35c0fe8c0 2024-02-16T19:28:04.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.796+0000 7fc360eff700 1 -- 172.21.15.22:0/1636390899 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc35c19d920 con 0x7fc35c0fe8c0 2024-02-16T19:28:04.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.797+0000 7fc360eff700 1 -- 172.21.15.22:0/1636390899 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc35c18f9f0 con 0x7fc35c0fe8c0 2024-02-16T19:28:04.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.798+0000 7fc34b7fe700 1 -- 172.21.15.22:0/1636390899 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fc34c0163e0 con 0x7fc35c0fe8c0 2024-02-16T19:28:04.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.799+0000 7fc34b7fe700 1 --2- 172.21.15.22:0/1636390899 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fc34406c680 0x7fc34406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:28:04.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.799+0000 7fc34b7fe700 1 -- 172.21.15.22:0/1636390899 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fc34c08bcb0 con 0x7fc35c0fe8c0 2024-02-16T19:28:04.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.799+0000 7fc359d9b700 1 --2- 172.21.15.22:0/1636390899 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fc34406c680 0x7fc34406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:28:04.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.799+0000 7fc359d9b700 1 --2- 172.21.15.22:0/1636390899 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fc34406c680 0x7fc34406eb30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc350009fd0 tx=0x7fc350000bc0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:28:04.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:04.804+0000 7fc34b7fe700 1 -- 172.21.15.22:0/1636390899 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fc34c059d20 con 0x7fc35c0fe8c0 2024-02-16T19:28:05.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.115+0000 7fc360eff700 1 -- 172.21.15.22:0/1636390899 --> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc35c061190 con 0x7fc34406c680 2024-02-16T19:28:05.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.127+0000 7fc34b7fe700 1 -- 172.21.15.22:0/1636390899 <== mgr.14217 v2:172.21.15.22:6800/2253867279 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3528 (secure 0 0 0) 0x7fc35c061190 con 0x7fc34406c680 2024-02-16T19:28:05.128 INFO:teuthology.orchestra.run.smithi022.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T19:28:05.128 INFO:teuthology.orchestra.run.smithi022.stdout:alertmanager.smithi022 smithi022 *:9093,9094 running (5m) 76s ago 7m 27.3M - 0.25.0 c8568f914cd2 4dac585d8372 2024-02-16T19:28:05.128 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-exporter.smithi022 smithi022 running (7m) 76s ago 7m 7428k - 18.2.0 dc2bc1663786 3f8bef4349ba 2024-02-16T19:28:05.128 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-exporter.smithi117 smithi117 running (6m) 82s ago 6m 7159k - 18.2.0 dc2bc1663786 59b65a7e309f 2024-02-16T19:28:05.128 INFO:teuthology.orchestra.run.smithi022.stdout:crash.smithi022 smithi022 running (7m) 76s ago 7m 6669k - 18.2.0 dc2bc1663786 efef476aa486 2024-02-16T19:28:05.128 INFO:teuthology.orchestra.run.smithi022.stdout:crash.smithi117 smithi117 running (6m) 82s ago 6m 6660k - 18.2.0 dc2bc1663786 5466453eb149 2024-02-16T19:28:05.129 INFO:teuthology.orchestra.run.smithi022.stdout:grafana.smithi022 smithi022 *:3000 running (5m) 76s ago 7m 84.6M - 9.4.7 954c08fa6188 a48872ecbbf2 2024-02-16T19:28:05.129 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi022.btppwm smithi022 running (96s) 76s ago 96s 14.9M - 18.2.0 dc2bc1663786 22971fb3a907 2024-02-16T19:28:05.129 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi022.ocukvs smithi022 running (102s) 76s ago 102s 11.5M - 18.2.0 dc2bc1663786 81ee7bd5fd39 2024-02-16T19:28:05.129 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi117.ibmppn smithi117 running (100s) 82s ago 99s 13.3M - 18.2.0 dc2bc1663786 8c606cbeeb86 2024-02-16T19:28:05.129 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi117.orzors smithi117 running (94s) 82s ago 93s 12.2M - 18.2.0 dc2bc1663786 7d2bc4df3feb 2024-02-16T19:28:05.129 INFO:teuthology.orchestra.run.smithi022.stdout:mgr.smithi022.gmdcch smithi022 *:9283,8765,8443 running (8m) 76s ago 8m 487M - 18.2.0 dc2bc1663786 e8435b394d04 2024-02-16T19:28:05.129 INFO:teuthology.orchestra.run.smithi022.stdout:mgr.smithi117.cohbtv smithi117 *:8443,9283,8765 running (6m) 82s ago 6m 438M - 18.2.0 dc2bc1663786 ddc46b07de2f 2024-02-16T19:28:05.129 INFO:teuthology.orchestra.run.smithi022.stdout:mon.smithi022 smithi022 running (8m) 76s ago 8m 45.2M 2048M 18.2.0 dc2bc1663786 b4ffc3032e12 2024-02-16T19:28:05.129 INFO:teuthology.orchestra.run.smithi022.stdout:mon.smithi117 smithi117 running (6m) 82s ago 6m 35.5M 2048M 18.2.0 dc2bc1663786 aff3e21a73a0 2024-02-16T19:28:05.129 INFO:teuthology.orchestra.run.smithi022.stdout:node-exporter.smithi022 smithi022 *:9100 running (7m) 76s ago 7m 16.0M - 1.5.0 0da6a335fe13 d727f51df007 2024-02-16T19:28:05.129 INFO:teuthology.orchestra.run.smithi022.stdout:node-exporter.smithi117 smithi117 *:9100 running (6m) 82s ago 6m 17.0M - 1.5.0 0da6a335fe13 fc8b2df5a29d 2024-02-16T19:28:05.129 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0 smithi022 running (5m) 76s ago 5m 47.4M 3948M 18.2.0 dc2bc1663786 0027f86f16ce 2024-02-16T19:28:05.129 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1 smithi022 running (4m) 76s ago 4m 48.5M 3948M 18.2.0 dc2bc1663786 87300382b8a7 2024-02-16T19:28:05.129 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2 smithi022 running (4m) 76s ago 4m 46.7M 3948M 18.2.0 dc2bc1663786 be8a631b4f93 2024-02-16T19:28:05.129 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3 smithi117 running (4m) 82s ago 4m 48.9M 4970M 18.2.0 dc2bc1663786 9c0ce9a981cf 2024-02-16T19:28:05.129 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4 smithi117 running (3m) 82s ago 3m 44.8M 4970M 18.2.0 dc2bc1663786 1779c3496f4d 2024-02-16T19:28:05.130 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5 smithi117 running (3m) 82s ago 3m 43.5M 4970M 18.2.0 dc2bc1663786 fefaabca0638 2024-02-16T19:28:05.130 INFO:teuthology.orchestra.run.smithi022.stdout:prometheus.smithi022 smithi022 *:9095 running (5m) 76s ago 6m 60.2M - 2.43.0 a07b618ecd1d a1a5b92b5ef7 2024-02-16T19:28:05.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.130+0000 7fc360eff700 1 -- 172.21.15.22:0/1636390899 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fc34406c680 msgr2=0x7fc34406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:05.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.130+0000 7fc360eff700 1 --2- 172.21.15.22:0/1636390899 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fc34406c680 0x7fc34406eb30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc350009fd0 tx=0x7fc350000bc0 comp rx=0 tx=0).stop 2024-02-16T19:28:05.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.130+0000 7fc360eff700 1 -- 172.21.15.22:0/1636390899 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc35c0fe8c0 msgr2=0x7fc35c1028f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:05.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.130+0000 7fc360eff700 1 --2- 172.21.15.22:0/1636390899 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc35c0fe8c0 0x7fc35c1028f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc34c00ea10 tx=0x7fc34c00ed20 comp rx=0 tx=0).stop 2024-02-16T19:28:05.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.130+0000 7fc360eff700 1 -- 172.21.15.22:0/1636390899 shutdown_connections 2024-02-16T19:28:05.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.130+0000 7fc360eff700 1 --2- 172.21.15.22:0/1636390899 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fc34406c680 0x7fc34406eb30 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:05.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.130+0000 7fc360eff700 1 --2- 172.21.15.22:0/1636390899 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc35c0fe8c0 0x7fc35c1028f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:05.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.130+0000 7fc360eff700 1 --2- 172.21.15.22:0/1636390899 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc35c0ffb60 0x7fc35c100f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:05.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.130+0000 7fc360eff700 1 -- 172.21.15.22:0/1636390899 >> 172.21.15.22:0/1636390899 conn(0x7fc35c0f9f10 msgr2=0x7fc35c0fabc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:28:05.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.130+0000 7fc360eff700 1 -- 172.21.15.22:0/1636390899 shutdown_connections 2024-02-16T19:28:05.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.130+0000 7fc360eff700 1 -- 172.21.15.22:0/1636390899 wait complete. 2024-02-16T19:28:05.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.262+0000 7efd84b04700 1 -- 172.21.15.22:0/3785165818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd80069ae0 msgr2=0x7efd80102e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:05.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.262+0000 7efd84b04700 1 --2- 172.21.15.22:0/3785165818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd80069ae0 0x7efd80102e10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7efd78009910 tx=0x7efd78009c20 comp rx=0 tx=0).stop 2024-02-16T19:28:05.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.262+0000 7efd84b04700 1 -- 172.21.15.22:0/3785165818 shutdown_connections 2024-02-16T19:28:05.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.262+0000 7efd84b04700 1 --2- 172.21.15.22:0/3785165818 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efd80103350 0x7efd801057b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:05.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.262+0000 7efd84b04700 1 --2- 172.21.15.22:0/3785165818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd80069ae0 0x7efd80102e10 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:05.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.262+0000 7efd84b04700 1 -- 172.21.15.22:0/3785165818 >> 172.21.15.22:0/3785165818 conn(0x7efd800fa7a0 msgr2=0x7efd800fcbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:28:05.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.262+0000 7efd84b04700 1 -- 172.21.15.22:0/3785165818 shutdown_connections 2024-02-16T19:28:05.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.262+0000 7efd84b04700 1 -- 172.21.15.22:0/3785165818 wait complete. 2024-02-16T19:28:05.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.263+0000 7efd84b04700 1 Processor -- start 2024-02-16T19:28:05.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.263+0000 7efd84b04700 1 -- start start 2024-02-16T19:28:05.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.263+0000 7efd84b04700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efd80069ae0 0x7efd80100760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:28:05.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.263+0000 7efd84b04700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd80103350 0x7efd800fedb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:28:05.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.263+0000 7efd84b04700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd800ff2f0 con 0x7efd80069ae0 2024-02-16T19:28:05.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.263+0000 7efd84b04700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd800ff430 con 0x7efd80103350 2024-02-16T19:28:05.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.264+0000 7efd7e59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efd80069ae0 0x7efd80100760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:28:05.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.264+0000 7efd7dd9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd80103350 0x7efd800fedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:28:05.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.264+0000 7efd7e59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efd80069ae0 0x7efd80100760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36426/0 (socket says 172.21.15.22:36426) 2024-02-16T19:28:05.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.264+0000 7efd7e59c700 1 -- 172.21.15.22:0/2490776009 learned_addr learned my addr 172.21.15.22:0/2490776009 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:28:05.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.264+0000 7efd7e59c700 1 -- 172.21.15.22:0/2490776009 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd80103350 msgr2=0x7efd800fedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:05.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.264+0000 7efd7e59c700 1 --2- 172.21.15.22:0/2490776009 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd80103350 0x7efd800fedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:05.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.264+0000 7efd7e59c700 1 -- 172.21.15.22:0/2490776009 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd70009c40 con 0x7efd80069ae0 2024-02-16T19:28:05.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.264+0000 7efd7dd9b700 1 --2- 172.21.15.22:0/2490776009 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd80103350 0x7efd800fedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T19:28:05.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.264+0000 7efd7e59c700 1 --2- 172.21.15.22:0/2490776009 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efd80069ae0 0x7efd80100760 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7efd78000c00 tx=0x7efd7800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:28:05.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.264+0000 7efd777fe700 1 -- 172.21.15.22:0/2490776009 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd7801d070 con 0x7efd80069ae0 2024-02-16T19:28:05.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.264+0000 7efd84b04700 1 -- 172.21.15.22:0/2490776009 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd780095f0 con 0x7efd80069ae0 2024-02-16T19:28:05.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.264+0000 7efd777fe700 1 -- 172.21.15.22:0/2490776009 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7efd78005dc0 con 0x7efd80069ae0 2024-02-16T19:28:05.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.265+0000 7efd777fe700 1 -- 172.21.15.22:0/2490776009 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd78017400 con 0x7efd80069ae0 2024-02-16T19:28:05.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.265+0000 7efd84b04700 1 -- 172.21.15.22:0/2490776009 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd800ffa10 con 0x7efd80069ae0 2024-02-16T19:28:05.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.266+0000 7efd84b04700 1 -- 172.21.15.22:0/2490776009 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd8018fbf0 con 0x7efd80069ae0 2024-02-16T19:28:05.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.267+0000 7efd777fe700 1 -- 172.21.15.22:0/2490776009 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7efd78003810 con 0x7efd80069ae0 2024-02-16T19:28:05.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.267+0000 7efd777fe700 1 --2- 172.21.15.22:0/2490776009 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7efd6c06c680 0x7efd6c06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:28:05.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.267+0000 7efd777fe700 1 -- 172.21.15.22:0/2490776009 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7efd7808cd60 con 0x7efd80069ae0 2024-02-16T19:28:05.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.267+0000 7efd7dd9b700 1 --2- 172.21.15.22:0/2490776009 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7efd6c06c680 0x7efd6c06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:28:05.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.268+0000 7efd7dd9b700 1 --2- 172.21.15.22:0/2490776009 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7efd6c06c680 0x7efd6c06eb30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7efd70007610 tx=0x7efd7000b040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:28:05.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.273+0000 7efd777fe700 1 -- 172.21.15.22:0/2490776009 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7efd7805ae70 con 0x7efd80069ae0 2024-02-16T19:28:05.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.722+0000 7efd84b04700 1 -- 172.21.15.22:0/2490776009 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7efd8004f2a0 con 0x7efd80069ae0 2024-02-16T19:28:05.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.723+0000 7efd777fe700 1 -- 172.21.15.22:0/2490776009 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+558 (secure 0 0 0) 0x7efd7805aa00 con 0x7efd80069ae0 2024-02-16T19:28:05.725 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-02-16T19:28:05.725 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": { 2024-02-16T19:28:05.725 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-02-16T19:28:05.725 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:28:05.725 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": { 2024-02-16T19:28:05.725 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-02-16T19:28:05.725 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:28:05.725 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": { 2024-02-16T19:28:05.725 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-02-16T19:28:05.725 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:28:05.725 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": { 2024-02-16T19:28:05.725 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-02-16T19:28:05.725 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:28:05.725 INFO:teuthology.orchestra.run.smithi022.stdout: "overall": { 2024-02-16T19:28:05.725 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 14 2024-02-16T19:28:05.725 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-02-16T19:28:05.726 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-02-16T19:28:05.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.726+0000 7efd84b04700 1 -- 172.21.15.22:0/2490776009 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7efd6c06c680 msgr2=0x7efd6c06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:05.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.726+0000 7efd84b04700 1 --2- 172.21.15.22:0/2490776009 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7efd6c06c680 0x7efd6c06eb30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7efd70007610 tx=0x7efd7000b040 comp rx=0 tx=0).stop 2024-02-16T19:28:05.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.726+0000 7efd84b04700 1 -- 172.21.15.22:0/2490776009 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efd80069ae0 msgr2=0x7efd80100760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:05.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.726+0000 7efd84b04700 1 --2- 172.21.15.22:0/2490776009 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efd80069ae0 0x7efd80100760 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7efd78000c00 tx=0x7efd7800f690 comp rx=0 tx=0).stop 2024-02-16T19:28:05.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.726+0000 7efd84b04700 1 -- 172.21.15.22:0/2490776009 shutdown_connections 2024-02-16T19:28:05.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.726+0000 7efd84b04700 1 --2- 172.21.15.22:0/2490776009 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efd80069ae0 0x7efd80100760 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:05.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.726+0000 7efd84b04700 1 --2- 172.21.15.22:0/2490776009 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7efd6c06c680 0x7efd6c06eb30 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:05.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.726+0000 7efd84b04700 1 --2- 172.21.15.22:0/2490776009 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd80103350 0x7efd800fedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:05.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.726+0000 7efd84b04700 1 -- 172.21.15.22:0/2490776009 >> 172.21.15.22:0/2490776009 conn(0x7efd800fa7a0 msgr2=0x7efd800fcbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:28:05.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.726+0000 7efd84b04700 1 -- 172.21.15.22:0/2490776009 shutdown_connections 2024-02-16T19:28:05.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.726+0000 7efd84b04700 1 -- 172.21.15.22:0/2490776009 wait complete. 2024-02-16T19:28:05.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.870+0000 7f6c6ef59700 1 -- 172.21.15.22:0/1408994737 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c68103600 msgr2=0x7f6c681059e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:05.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.870+0000 7f6c6ef59700 1 --2- 172.21.15.22:0/1408994737 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c68103600 0x7f6c681059e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6c54009910 tx=0x7f6c54009c20 comp rx=0 tx=0).stop 2024-02-16T19:28:05.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.870+0000 7f6c6ef59700 1 -- 172.21.15.22:0/1408994737 shutdown_connections 2024-02-16T19:28:05.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.870+0000 7f6c6ef59700 1 --2- 172.21.15.22:0/1408994737 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c68103600 0x7f6c681059e0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:05.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.870+0000 7f6c6ef59700 1 --2- 172.21.15.22:0/1408994737 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c68100ce0 0x7f6c681030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:05.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.870+0000 7f6c6ef59700 1 -- 172.21.15.22:0/1408994737 >> 172.21.15.22:0/1408994737 conn(0x7f6c680fa700 msgr2=0x7f6c680fcb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:28:05.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.870+0000 7f6c6ef59700 1 -- 172.21.15.22:0/1408994737 shutdown_connections 2024-02-16T19:28:05.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.870+0000 7f6c6ef59700 1 -- 172.21.15.22:0/1408994737 wait complete. 2024-02-16T19:28:05.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.871+0000 7f6c6ef59700 1 Processor -- start 2024-02-16T19:28:05.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.871+0000 7f6c6ef59700 1 -- start start 2024-02-16T19:28:05.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.871+0000 7f6c6ef59700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c68100ce0 0x7f6c681005f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:28:05.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.871+0000 7f6c6ef59700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c68103600 0x7f6c680fec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:28:05.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.871+0000 7f6c6ef59700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c680ff1d0 con 0x7f6c68100ce0 2024-02-16T19:28:05.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.871+0000 7f6c6ef59700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c680ff310 con 0x7f6c68103600 2024-02-16T19:28:05.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.871+0000 7f6c6ccf5700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c68100ce0 0x7f6c681005f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:28:05.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.871+0000 7f6c67fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c68103600 0x7f6c680fec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:28:05.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.871+0000 7f6c6ccf5700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c68100ce0 0x7f6c681005f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36442/0 (socket says 172.21.15.22:36442) 2024-02-16T19:28:05.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.871+0000 7f6c6ccf5700 1 -- 172.21.15.22:0/1501813325 learned_addr learned my addr 172.21.15.22:0/1501813325 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:28:05.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.872+0000 7f6c67fff700 1 -- 172.21.15.22:0/1501813325 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c68100ce0 msgr2=0x7f6c681005f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:05.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.872+0000 7f6c67fff700 1 --2- 172.21.15.22:0/1501813325 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c68100ce0 0x7f6c681005f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:05.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.872+0000 7f6c67fff700 1 -- 172.21.15.22:0/1501813325 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c540095f0 con 0x7f6c68103600 2024-02-16T19:28:05.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.872+0000 7f6c6ccf5700 1 --2- 172.21.15.22:0/1501813325 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c68100ce0 0x7f6c681005f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T19:28:05.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.872+0000 7f6c67fff700 1 --2- 172.21.15.22:0/1501813325 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c68103600 0x7f6c680fec90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6c54000c00 tx=0x7f6c5400f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:28:05.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.872+0000 7f6c65ffb700 1 -- 172.21.15.22:0/1501813325 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c5401d070 con 0x7f6c68103600 2024-02-16T19:28:05.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.872+0000 7f6c65ffb700 1 -- 172.21.15.22:0/1501813325 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6c54005dc0 con 0x7f6c68103600 2024-02-16T19:28:05.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.873+0000 7f6c65ffb700 1 -- 172.21.15.22:0/1501813325 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c54022470 con 0x7f6c68103600 2024-02-16T19:28:05.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.873+0000 7f6c6ef59700 1 -- 172.21.15.22:0/1501813325 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c680ff590 con 0x7f6c68103600 2024-02-16T19:28:05.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.873+0000 7f6c6ef59700 1 -- 172.21.15.22:0/1501813325 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c680ffa80 con 0x7f6c68103600 2024-02-16T19:28:05.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.874+0000 7f6c6ef59700 1 -- 172.21.15.22:0/1501813325 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c6818fc70 con 0x7f6c68103600 2024-02-16T19:28:05.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.875+0000 7f6c65ffb700 1 -- 172.21.15.22:0/1501813325 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f6c54003810 con 0x7f6c68103600 2024-02-16T19:28:05.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.875+0000 7f6c65ffb700 1 --2- 172.21.15.22:0/1501813325 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f6c5806c680 0x7f6c5806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:28:05.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.875+0000 7f6c65ffb700 1 -- 172.21.15.22:0/1501813325 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f6c5408c960 con 0x7f6c68103600 2024-02-16T19:28:05.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.875+0000 7f6c6ccf5700 1 --2- 172.21.15.22:0/1501813325 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f6c5806c680 0x7f6c5806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:28:05.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.876+0000 7f6c6ccf5700 1 --2- 172.21.15.22:0/1501813325 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f6c5806c680 0x7f6c5806eb30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c011550 tx=0x7f6c5c009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:28:05.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:05.881+0000 7f6c65ffb700 1 -- 172.21.15.22:0/1501813325 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f6c5405aa50 con 0x7f6c68103600 2024-02-16T19:28:06.033 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:05 smithi022 ceph-mon[29221]: from='client.14566 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:28:06.033 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:05 smithi022 ceph-mon[29221]: from='client.24329 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:28:06.033 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:05 smithi022 ceph-mon[29221]: pgmap v234: 65 pgs: 65 active+clean; 29 MiB data, 245 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 2.4 MiB/s wr, 83 op/s 2024-02-16T19:28:06.033 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:05 smithi022 ceph-mon[29221]: from='client.? 172.21.15.22:0/2490776009' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:28:06.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:06.170+0000 7f6c6ef59700 1 -- 172.21.15.22:0/1501813325 --> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6c68061190 con 0x7f6c5806c680 2024-02-16T19:28:06.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:06.172+0000 7f6c65ffb700 1 -- 172.21.15.22:0/1501813325 <== mgr.14217 v2:172.21.15.22:6800/2253867279 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f6c68061190 con 0x7f6c5806c680 2024-02-16T19:28:06.173 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-02-16T19:28:06.173 INFO:teuthology.orchestra.run.smithi022.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T19:28:06.173 INFO:teuthology.orchestra.run.smithi022.stdout: "in_progress": true, 2024-02-16T19:28:06.173 INFO:teuthology.orchestra.run.smithi022.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-02-16T19:28:06.173 INFO:teuthology.orchestra.run.smithi022.stdout: "services_complete": [], 2024-02-16T19:28:06.173 INFO:teuthology.orchestra.run.smithi022.stdout: "progress": "0/2 daemons upgraded", 2024-02-16T19:28:06.173 INFO:teuthology.orchestra.run.smithi022.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 image on host smithi117", 2024-02-16T19:28:06.173 INFO:teuthology.orchestra.run.smithi022.stdout: "is_paused": false 2024-02-16T19:28:06.173 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-02-16T19:28:06.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:06.174+0000 7f6c6ef59700 1 -- 172.21.15.22:0/1501813325 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f6c5806c680 msgr2=0x7f6c5806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:06.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:06.174+0000 7f6c6ef59700 1 --2- 172.21.15.22:0/1501813325 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f6c5806c680 0x7f6c5806eb30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c011550 tx=0x7f6c5c009260 comp rx=0 tx=0).stop 2024-02-16T19:28:06.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:06.174+0000 7f6c6ef59700 1 -- 172.21.15.22:0/1501813325 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c68103600 msgr2=0x7f6c680fec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:06.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:06.174+0000 7f6c6ef59700 1 --2- 172.21.15.22:0/1501813325 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c68103600 0x7f6c680fec90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6c54000c00 tx=0x7f6c5400f690 comp rx=0 tx=0).stop 2024-02-16T19:28:06.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:06.174+0000 7f6c6ef59700 1 -- 172.21.15.22:0/1501813325 shutdown_connections 2024-02-16T19:28:06.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:06.174+0000 7f6c6ef59700 1 --2- 172.21.15.22:0/1501813325 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c68100ce0 0x7f6c681005f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:06.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:06.174+0000 7f6c6ef59700 1 --2- 172.21.15.22:0/1501813325 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f6c5806c680 0x7f6c5806eb30 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:06.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:06.174+0000 7f6c6ef59700 1 --2- 172.21.15.22:0/1501813325 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c68103600 0x7f6c680fec90 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:06.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:06.175+0000 7f6c6ef59700 1 -- 172.21.15.22:0/1501813325 >> 172.21.15.22:0/1501813325 conn(0x7f6c680fa700 msgr2=0x7f6c680fcb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:28:06.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:06.175+0000 7f6c6ef59700 1 -- 172.21.15.22:0/1501813325 shutdown_connections 2024-02-16T19:28:06.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:06.175+0000 7f6c6ef59700 1 -- 172.21.15.22:0/1501813325 wait complete. 2024-02-16T19:28:06.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:05 smithi117 ceph-mon[35309]: from='client.14566 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:28:06.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:05 smithi117 ceph-mon[35309]: from='client.24329 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:28:06.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:05 smithi117 ceph-mon[35309]: pgmap v234: 65 pgs: 65 active+clean; 29 MiB data, 245 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 2.4 MiB/s wr, 83 op/s 2024-02-16T19:28:06.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:05 smithi117 ceph-mon[35309]: from='client.? 172.21.15.22:0/2490776009' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:28:07.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:06 smithi022 ceph-mon[29221]: from='client.24333 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:28:07.386 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:06 smithi117 ceph-mon[35309]: from='client.24333 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:28:08.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:07 smithi022 ceph-mon[29221]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:28:08.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:07 smithi022 ceph-mon[29221]: pgmap v235: 65 pgs: 65 active+clean; 46 MiB data, 316 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 3.8 MiB/s wr, 131 op/s 2024-02-16T19:28:08.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:07 smithi117 ceph-mon[35309]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:28:08.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:07 smithi117 ceph-mon[35309]: pgmap v235: 65 pgs: 65 active+clean; 46 MiB data, 316 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 3.8 MiB/s wr, 131 op/s 2024-02-16T19:28:10.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:09 smithi022 ceph-mon[29221]: pgmap v236: 65 pgs: 65 active+clean; 54 MiB data, 346 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 4.1 MiB/s wr, 147 op/s 2024-02-16T19:28:10.386 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:09 smithi117 ceph-mon[35309]: pgmap v236: 65 pgs: 65 active+clean; 54 MiB data, 346 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 4.1 MiB/s wr, 147 op/s 2024-02-16T19:28:12.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:11 smithi022 ceph-mon[29221]: pgmap v237: 65 pgs: 65 active+clean; 66 MiB data, 384 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 4.4 MiB/s wr, 178 op/s 2024-02-16T19:28:12.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:11 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:28:12.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:11 smithi117 ceph-mon[35309]: pgmap v237: 65 pgs: 65 active+clean; 66 MiB data, 384 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 4.4 MiB/s wr, 178 op/s 2024-02-16T19:28:12.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:11 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:28:14.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:13 smithi022 ceph-mon[29221]: pgmap v238: 65 pgs: 65 active+clean; 71 MiB data, 406 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 4.4 MiB/s wr, 182 op/s 2024-02-16T19:28:14.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:13 smithi117 ceph-mon[35309]: pgmap v238: 65 pgs: 65 active+clean; 71 MiB data, 406 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 4.4 MiB/s wr, 182 op/s 2024-02-16T19:28:16.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:15 smithi022 ceph-mon[29221]: pgmap v239: 65 pgs: 65 active+clean; 71 MiB data, 406 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 3.6 MiB/s wr, 157 op/s 2024-02-16T19:28:16.386 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:15 smithi117 ceph-mon[35309]: pgmap v239: 65 pgs: 65 active+clean; 71 MiB data, 406 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 3.6 MiB/s wr, 157 op/s 2024-02-16T19:28:18.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:17 smithi022 ceph-mon[29221]: pgmap v240: 65 pgs: 65 active+clean; 84 MiB data, 441 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 4.7 MiB/s wr, 197 op/s 2024-02-16T19:28:18.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:17 smithi117 ceph-mon[35309]: pgmap v240: 65 pgs: 65 active+clean; 84 MiB data, 441 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 4.7 MiB/s wr, 197 op/s 2024-02-16T19:28:20.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:19 smithi022 ceph-mon[29221]: pgmap v241: 65 pgs: 65 active+clean; 92 MiB data, 476 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 3.9 MiB/s wr, 171 op/s 2024-02-16T19:28:20.386 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:19 smithi117 ceph-mon[35309]: pgmap v241: 65 pgs: 65 active+clean; 92 MiB data, 476 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 3.9 MiB/s wr, 171 op/s 2024-02-16T19:28:22.188 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:21 smithi117 ceph-mon[35309]: pgmap v242: 65 pgs: 65 active+clean; 106 MiB data, 522 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 4.4 MiB/s wr, 190 op/s 2024-02-16T19:28:22.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:21 smithi022 ceph-mon[29221]: pgmap v242: 65 pgs: 65 active+clean; 106 MiB data, 522 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 4.4 MiB/s wr, 190 op/s 2024-02-16T19:28:24.206 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:23 smithi117 ceph-mon[35309]: pgmap v243: 65 pgs: 65 active+clean; 113 MiB data, 557 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 4.0 MiB/s wr, 176 op/s 2024-02-16T19:28:24.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:23 smithi022 ceph-mon[29221]: pgmap v243: 65 pgs: 65 active+clean; 113 MiB data, 557 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 4.0 MiB/s wr, 176 op/s 2024-02-16T19:28:26.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:25 smithi022 ceph-mon[29221]: pgmap v244: 65 pgs: 65 active+clean; 113 MiB data, 557 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 3.6 MiB/s wr, 146 op/s 2024-02-16T19:28:26.386 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:25 smithi117 ceph-mon[35309]: pgmap v244: 65 pgs: 65 active+clean; 113 MiB data, 557 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 3.6 MiB/s wr, 146 op/s 2024-02-16T19:28:27.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:26 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:28:27.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:26 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:28:28.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:27 smithi022 ceph-mon[29221]: pgmap v245: 65 pgs: 65 active+clean; 125 MiB data, 604 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 4.6 MiB/s wr, 220 op/s 2024-02-16T19:28:28.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:27 smithi117 ceph-mon[35309]: pgmap v245: 65 pgs: 65 active+clean; 125 MiB data, 604 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 4.6 MiB/s wr, 220 op/s 2024-02-16T19:28:30.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:29 smithi022 ceph-mon[29221]: pgmap v246: 65 pgs: 65 active+clean; 132 MiB data, 628 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 4.1 MiB/s wr, 212 op/s 2024-02-16T19:28:30.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:29 smithi117 ceph-mon[35309]: pgmap v246: 65 pgs: 65 active+clean; 132 MiB data, 628 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 4.1 MiB/s wr, 212 op/s 2024-02-16T19:28:32.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:31 smithi022 ceph-mon[29221]: pgmap v247: 65 pgs: 65 active+clean; 144 MiB data, 672 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 4.4 MiB/s wr, 255 op/s 2024-02-16T19:28:32.386 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:31 smithi117 ceph-mon[35309]: pgmap v247: 65 pgs: 65 active+clean; 144 MiB data, 672 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 4.4 MiB/s wr, 255 op/s 2024-02-16T19:28:33.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:33 smithi117 ceph-mon[35309]: Upgrade: Updating mgr.smithi117.cohbtv 2024-02-16T19:28:33.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:33 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:28:33.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:33 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi117.cohbtv", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-16T19:28:33.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:33 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-16T19:28:33.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:33 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:28:33.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:33 smithi117 ceph-mon[35309]: Deploying daemon mgr.smithi117.cohbtv on smithi117 2024-02-16T19:28:33.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:33 smithi117 ceph-mon[35309]: pgmap v248: 65 pgs: 65 active+clean; 151 MiB data, 711 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 3.8 MiB/s wr, 273 op/s 2024-02-16T19:28:33.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:33 smithi022 ceph-mon[29221]: Upgrade: Updating mgr.smithi117.cohbtv 2024-02-16T19:28:33.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:33 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:28:33.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:33 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi117.cohbtv", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-16T19:28:33.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:33 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-16T19:28:33.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:33 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:28:33.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:33 smithi022 ceph-mon[29221]: Deploying daemon mgr.smithi117.cohbtv on smithi117 2024-02-16T19:28:33.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:33 smithi022 ceph-mon[29221]: pgmap v248: 65 pgs: 65 active+clean; 151 MiB data, 711 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 3.8 MiB/s wr, 273 op/s 2024-02-16T19:28:36.096 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:35 smithi117 ceph-mon[35309]: pgmap v249: 65 pgs: 65 active+clean; 151 MiB data, 711 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 3.2 MiB/s wr, 241 op/s 2024-02-16T19:28:36.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:35 smithi022 ceph-mon[29221]: pgmap v249: 65 pgs: 65 active+clean; 151 MiB data, 711 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 3.2 MiB/s wr, 241 op/s 2024-02-16T19:28:36.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.309+0000 7fe848831700 1 -- 172.21.15.22:0/1271933318 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe844100d60 msgr2=0x7fe844103140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:36.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.309+0000 7fe848831700 1 --2- 172.21.15.22:0/1271933318 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe844100d60 0x7fe844103140 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe834009910 tx=0x7fe834009c20 comp rx=0 tx=0).stop 2024-02-16T19:28:36.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.309+0000 7fe848831700 1 -- 172.21.15.22:0/1271933318 shutdown_connections 2024-02-16T19:28:36.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.309+0000 7fe848831700 1 --2- 172.21.15.22:0/1271933318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe844103680 0x7fe844105a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:36.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.309+0000 7fe848831700 1 --2- 172.21.15.22:0/1271933318 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe844100d60 0x7fe844103140 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:36.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.309+0000 7fe848831700 1 -- 172.21.15.22:0/1271933318 >> 172.21.15.22:0/1271933318 conn(0x7fe8440fa780 msgr2=0x7fe8440fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:28:36.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.309+0000 7fe848831700 1 -- 172.21.15.22:0/1271933318 shutdown_connections 2024-02-16T19:28:36.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.309+0000 7fe848831700 1 -- 172.21.15.22:0/1271933318 wait complete. 2024-02-16T19:28:36.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.310+0000 7fe848831700 1 Processor -- start 2024-02-16T19:28:36.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.310+0000 7fe848831700 1 -- start start 2024-02-16T19:28:36.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.310+0000 7fe848831700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe844100d60 0x7fe844195ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:28:36.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.310+0000 7fe848831700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe844103680 0x7fe844195ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:28:36.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.310+0000 7fe848831700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe844196610 con 0x7fe844100d60 2024-02-16T19:28:36.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.310+0000 7fe848831700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe844196750 con 0x7fe844103680 2024-02-16T19:28:36.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.310+0000 7fe84259c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe844100d60 0x7fe844195ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:28:36.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.310+0000 7fe84259c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe844100d60 0x7fe844195ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37216/0 (socket says 172.21.15.22:37216) 2024-02-16T19:28:36.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.310+0000 7fe84259c700 1 -- 172.21.15.22:0/4250972120 learned_addr learned my addr 172.21.15.22:0/4250972120 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:28:36.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.310+0000 7fe841d9b700 1 --2- 172.21.15.22:0/4250972120 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe844103680 0x7fe844195ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:28:36.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.311+0000 7fe84259c700 1 -- 172.21.15.22:0/4250972120 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe844103680 msgr2=0x7fe844195ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:36.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.311+0000 7fe84259c700 1 --2- 172.21.15.22:0/4250972120 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe844103680 0x7fe844195ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:36.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.311+0000 7fe84259c700 1 -- 172.21.15.22:0/4250972120 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8340095f0 con 0x7fe844100d60 2024-02-16T19:28:36.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.311+0000 7fe84259c700 1 --2- 172.21.15.22:0/4250972120 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe844100d60 0x7fe844195ab0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe834000c00 tx=0x7fe83400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:28:36.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.311+0000 7fe8337fe700 1 -- 172.21.15.22:0/4250972120 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe83401d070 con 0x7fe844100d60 2024-02-16T19:28:36.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.311+0000 7fe848831700 1 -- 172.21.15.22:0/4250972120 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe84419b1a0 con 0x7fe844100d60 2024-02-16T19:28:36.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.311+0000 7fe8337fe700 1 -- 172.21.15.22:0/4250972120 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe834005dc0 con 0x7fe844100d60 2024-02-16T19:28:36.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.311+0000 7fe8337fe700 1 -- 172.21.15.22:0/4250972120 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe834022470 con 0x7fe844100d60 2024-02-16T19:28:36.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.311+0000 7fe848831700 1 -- 172.21.15.22:0/4250972120 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe84419b690 con 0x7fe844100d60 2024-02-16T19:28:36.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.312+0000 7fe848831700 1 -- 172.21.15.22:0/4250972120 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe84418fbc0 con 0x7fe844100d60 2024-02-16T19:28:36.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.313+0000 7fe8337fe700 1 -- 172.21.15.22:0/4250972120 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fe834003810 con 0x7fe844100d60 2024-02-16T19:28:36.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.313+0000 7fe8337fe700 1 --2- 172.21.15.22:0/4250972120 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fe82c06c680 0x7fe82c06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:28:36.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.314+0000 7fe8337fe700 1 -- 172.21.15.22:0/4250972120 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fe83408cdc0 con 0x7fe844100d60 2024-02-16T19:28:36.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.314+0000 7fe841d9b700 1 --2- 172.21.15.22:0/4250972120 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fe82c06c680 0x7fe82c06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:28:36.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.314+0000 7fe841d9b700 1 --2- 172.21.15.22:0/4250972120 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fe82c06c680 0x7fe82c06eb30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe838009c60 tx=0x7fe838009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:28:36.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.321+0000 7fe8337fe700 1 -- 172.21.15.22:0/4250972120 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fe83405aed0 con 0x7fe844100d60 2024-02-16T19:28:36.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.619+0000 7fe848831700 1 -- 172.21.15.22:0/4250972120 --> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe844061190 con 0x7fe82c06c680 2024-02-16T19:28:36.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.620+0000 7fe8337fe700 1 -- 172.21.15.22:0/4250972120 <== mgr.14217 v2:172.21.15.22:6800/2253867279 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7fe844061190 con 0x7fe82c06c680 2024-02-16T19:28:36.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.624+0000 7fe848831700 1 -- 172.21.15.22:0/4250972120 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fe82c06c680 msgr2=0x7fe82c06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:36.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.624+0000 7fe848831700 1 --2- 172.21.15.22:0/4250972120 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fe82c06c680 0x7fe82c06eb30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe838009c60 tx=0x7fe838009260 comp rx=0 tx=0).stop 2024-02-16T19:28:36.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.624+0000 7fe848831700 1 -- 172.21.15.22:0/4250972120 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe844100d60 msgr2=0x7fe844195ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:36.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.624+0000 7fe848831700 1 --2- 172.21.15.22:0/4250972120 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe844100d60 0x7fe844195ab0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe834000c00 tx=0x7fe83400f690 comp rx=0 tx=0).stop 2024-02-16T19:28:36.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.624+0000 7fe848831700 1 -- 172.21.15.22:0/4250972120 shutdown_connections 2024-02-16T19:28:36.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.624+0000 7fe848831700 1 --2- 172.21.15.22:0/4250972120 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe844100d60 0x7fe844195ab0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:36.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.624+0000 7fe848831700 1 --2- 172.21.15.22:0/4250972120 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fe82c06c680 0x7fe82c06eb30 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:36.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.624+0000 7fe848831700 1 --2- 172.21.15.22:0/4250972120 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe844103680 0x7fe844195ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:36.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.624+0000 7fe848831700 1 -- 172.21.15.22:0/4250972120 >> 172.21.15.22:0/4250972120 conn(0x7fe8440fa780 msgr2=0x7fe8440fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:28:36.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.624+0000 7fe848831700 1 -- 172.21.15.22:0/4250972120 shutdown_connections 2024-02-16T19:28:36.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.624+0000 7fe848831700 1 -- 172.21.15.22:0/4250972120 wait complete. 2024-02-16T19:28:36.637 INFO:teuthology.orchestra.run.smithi022.stdout:true 2024-02-16T19:28:36.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.778+0000 7fc4e511c700 1 -- 172.21.15.22:0/2813468028 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4e0100350 msgr2=0x7fc4e01007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:36.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.778+0000 7fc4e511c700 1 --2- 172.21.15.22:0/2813468028 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4e0100350 0x7fc4e01007a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc4d4009910 tx=0x7fc4d4009c20 comp rx=0 tx=0).stop 2024-02-16T19:28:36.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.778+0000 7fc4e511c700 1 -- 172.21.15.22:0/2813468028 shutdown_connections 2024-02-16T19:28:36.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.778+0000 7fc4e511c700 1 --2- 172.21.15.22:0/2813468028 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4e0100350 0x7fc4e01007a0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:36.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.778+0000 7fc4e511c700 1 --2- 172.21.15.22:0/2813468028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4e00ff150 0x7fc4e00ff560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:36.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.778+0000 7fc4e511c700 1 -- 172.21.15.22:0/2813468028 >> 172.21.15.22:0/2813468028 conn(0x7fc4e00fa700 msgr2=0x7fc4e00fcb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:28:36.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.778+0000 7fc4e511c700 1 -- 172.21.15.22:0/2813468028 shutdown_connections 2024-02-16T19:28:36.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.778+0000 7fc4e511c700 1 -- 172.21.15.22:0/2813468028 wait complete. 2024-02-16T19:28:36.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.779+0000 7fc4e511c700 1 Processor -- start 2024-02-16T19:28:36.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.779+0000 7fc4e511c700 1 -- start start 2024-02-16T19:28:36.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.779+0000 7fc4e511c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4e00ff150 0x7fc4e0069930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:28:36.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.779+0000 7fc4e511c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4e0100350 0x7fc4e019a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:28:36.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.779+0000 7fc4e511c700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e019add0 con 0x7fc4e00ff150 2024-02-16T19:28:36.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.779+0000 7fc4e511c700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e019af10 con 0x7fc4e0100350 2024-02-16T19:28:36.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.780+0000 7fc4ded9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4e00ff150 0x7fc4e0069930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:28:36.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.780+0000 7fc4de59c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4e0100350 0x7fc4e019a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:28:36.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.780+0000 7fc4de59c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4e0100350 0x7fc4e019a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.22:33702/0 (socket says 172.21.15.22:33702) 2024-02-16T19:28:36.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.780+0000 7fc4de59c700 1 -- 172.21.15.22:0/1058983266 learned_addr learned my addr 172.21.15.22:0/1058983266 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:28:36.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.780+0000 7fc4ded9d700 1 --2- 172.21.15.22:0/1058983266 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4e00ff150 0x7fc4e0069930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37242/0 (socket says 172.21.15.22:37242) 2024-02-16T19:28:36.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.780+0000 7fc4ded9d700 1 -- 172.21.15.22:0/1058983266 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4e0100350 msgr2=0x7fc4e019a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:36.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.780+0000 7fc4ded9d700 1 --2- 172.21.15.22:0/1058983266 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4e0100350 0x7fc4e019a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:36.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.780+0000 7fc4ded9d700 1 -- 172.21.15.22:0/1058983266 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4d40095f0 con 0x7fc4e00ff150 2024-02-16T19:28:36.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.780+0000 7fc4de59c700 1 --2- 172.21.15.22:0/1058983266 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4e0100350 0x7fc4e019a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T19:28:36.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.780+0000 7fc4ded9d700 1 --2- 172.21.15.22:0/1058983266 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4e00ff150 0x7fc4e0069930 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc4d000e940 tx=0x7fc4d000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:28:36.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.781+0000 7fc4c7fff700 1 -- 172.21.15.22:0/1058983266 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc4d00041f0 con 0x7fc4e00ff150 2024-02-16T19:28:36.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.781+0000 7fc4c7fff700 1 -- 172.21.15.22:0/1058983266 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc4d0004350 con 0x7fc4e00ff150 2024-02-16T19:28:36.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.781+0000 7fc4c7fff700 1 -- 172.21.15.22:0/1058983266 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc4d0003cb0 con 0x7fc4e00ff150 2024-02-16T19:28:36.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.781+0000 7fc4e511c700 1 -- 172.21.15.22:0/1058983266 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4e019b1f0 con 0x7fc4e00ff150 2024-02-16T19:28:36.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.781+0000 7fc4e511c700 1 -- 172.21.15.22:0/1058983266 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4e019b6f0 con 0x7fc4e00ff150 2024-02-16T19:28:36.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.782+0000 7fc4e511c700 1 -- 172.21.15.22:0/1058983266 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4e0107ca0 con 0x7fc4e00ff150 2024-02-16T19:28:36.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.783+0000 7fc4c7fff700 1 -- 172.21.15.22:0/1058983266 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fc4d000c7c0 con 0x7fc4e00ff150 2024-02-16T19:28:36.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.783+0000 7fc4c7fff700 1 --2- 172.21.15.22:0/1058983266 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fc4c806c680 0x7fc4c806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:28:36.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.784+0000 7fc4c7fff700 1 -- 172.21.15.22:0/1058983266 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fc4d0014070 con 0x7fc4e00ff150 2024-02-16T19:28:36.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.784+0000 7fc4de59c700 1 --2- 172.21.15.22:0/1058983266 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fc4c806c680 0x7fc4c806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:28:36.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.784+0000 7fc4de59c700 1 --2- 172.21.15.22:0/1058983266 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fc4c806c680 0x7fc4c806eb30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc4e0069eb0 tx=0x7fc4d4005940 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:28:36.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:36.789+0000 7fc4c7fff700 1 -- 172.21.15.22:0/1058983266 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fc4d005a030 con 0x7fc4e00ff150 2024-02-16T19:28:37.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.085+0000 7fc4e511c700 1 -- 172.21.15.22:0/1058983266 --> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc4e0104ca0 con 0x7fc4c806c680 2024-02-16T19:28:37.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.086+0000 7fc4c7fff700 1 -- 172.21.15.22:0/1058983266 <== mgr.14217 v2:172.21.15.22:6800/2253867279 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+321 (secure 0 0 0) 0x7fc4e0104ca0 con 0x7fc4c806c680 2024-02-16T19:28:37.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.089+0000 7fc4e511c700 1 -- 172.21.15.22:0/1058983266 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fc4c806c680 msgr2=0x7fc4c806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:37.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.089+0000 7fc4e511c700 1 --2- 172.21.15.22:0/1058983266 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fc4c806c680 0x7fc4c806eb30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc4e0069eb0 tx=0x7fc4d4005940 comp rx=0 tx=0).stop 2024-02-16T19:28:37.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.089+0000 7fc4e511c700 1 -- 172.21.15.22:0/1058983266 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4e00ff150 msgr2=0x7fc4e0069930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:37.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.089+0000 7fc4e511c700 1 --2- 172.21.15.22:0/1058983266 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4e00ff150 0x7fc4e0069930 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc4d000e940 tx=0x7fc4d000ec50 comp rx=0 tx=0).stop 2024-02-16T19:28:37.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.089+0000 7fc4e511c700 1 -- 172.21.15.22:0/1058983266 shutdown_connections 2024-02-16T19:28:37.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.090+0000 7fc4e511c700 1 --2- 172.21.15.22:0/1058983266 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4e00ff150 0x7fc4e0069930 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:37.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.090+0000 7fc4e511c700 1 --2- 172.21.15.22:0/1058983266 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7fc4c806c680 0x7fc4c806eb30 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:37.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.090+0000 7fc4e511c700 1 --2- 172.21.15.22:0/1058983266 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4e0100350 0x7fc4e019a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:37.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.090+0000 7fc4e511c700 1 -- 172.21.15.22:0/1058983266 >> 172.21.15.22:0/1058983266 conn(0x7fc4e00fa700 msgr2=0x7fc4e0103580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:28:37.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.090+0000 7fc4e511c700 1 -- 172.21.15.22:0/1058983266 shutdown_connections 2024-02-16T19:28:37.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.090+0000 7fc4e511c700 1 -- 172.21.15.22:0/1058983266 wait complete. 2024-02-16T19:28:37.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.232+0000 7f427be07700 1 -- 172.21.15.22:0/1494364829 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4274100680 msgr2=0x7f4274100a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:37.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.232+0000 7f427be07700 1 --2- 172.21.15.22:0/1494364829 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4274100680 0x7f4274100a90 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4270009910 tx=0x7f4270009c20 comp rx=0 tx=0).stop 2024-02-16T19:28:37.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.232+0000 7f427be07700 1 -- 172.21.15.22:0/1494364829 shutdown_connections 2024-02-16T19:28:37.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.232+0000 7f427be07700 1 --2- 172.21.15.22:0/1494364829 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42740ff250 0x7f42740ff6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:37.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.232+0000 7f427be07700 1 --2- 172.21.15.22:0/1494364829 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4274100680 0x7f4274100a90 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:37.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.232+0000 7f427be07700 1 -- 172.21.15.22:0/1494364829 >> 172.21.15.22:0/1494364829 conn(0x7f42740f9d30 msgr2=0x7f42740fc160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:28:37.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.232+0000 7f427be07700 1 -- 172.21.15.22:0/1494364829 shutdown_connections 2024-02-16T19:28:37.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.232+0000 7f427be07700 1 -- 172.21.15.22:0/1494364829 wait complete. 2024-02-16T19:28:37.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.233+0000 7f427be07700 1 Processor -- start 2024-02-16T19:28:37.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.233+0000 7f427be07700 1 -- start start 2024-02-16T19:28:37.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.233+0000 7f427be07700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42740ff250 0x7f42740fed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:28:37.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.233+0000 7f427be07700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4274100680 0x7f427419a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:28:37.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.233+0000 7f427be07700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f427419ac00 con 0x7f4274100680 2024-02-16T19:28:37.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.233+0000 7f427be07700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f427419ad40 con 0x7f42740ff250 2024-02-16T19:28:37.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.233+0000 7f4279ba3700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42740ff250 0x7f42740fed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:28:37.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.233+0000 7f42793a2700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4274100680 0x7f427419a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:28:37.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.233+0000 7f4279ba3700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42740ff250 0x7f42740fed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.22:33722/0 (socket says 172.21.15.22:33722) 2024-02-16T19:28:37.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.233+0000 7f4279ba3700 1 -- 172.21.15.22:0/2776745397 learned_addr learned my addr 172.21.15.22:0/2776745397 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:28:37.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.234+0000 7f4279ba3700 1 -- 172.21.15.22:0/2776745397 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4274100680 msgr2=0x7f427419a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:37.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.234+0000 7f4279ba3700 1 --2- 172.21.15.22:0/2776745397 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4274100680 0x7f427419a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:37.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.234+0000 7f4279ba3700 1 -- 172.21.15.22:0/2776745397 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42700095f0 con 0x7f42740ff250 2024-02-16T19:28:37.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.234+0000 7f42793a2700 1 --2- 172.21.15.22:0/2776745397 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4274100680 0x7f427419a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T19:28:37.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.234+0000 7f4279ba3700 1 --2- 172.21.15.22:0/2776745397 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42740ff250 0x7f42740fed00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f42700092e0 tx=0x7f427000f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:28:37.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.234+0000 7f426affd700 1 -- 172.21.15.22:0/2776745397 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f427001d070 con 0x7f42740ff250 2024-02-16T19:28:37.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.234+0000 7f427be07700 1 -- 172.21.15.22:0/2776745397 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f427419afc0 con 0x7f42740ff250 2024-02-16T19:28:37.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.234+0000 7f426affd700 1 -- 172.21.15.22:0/2776745397 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4270005dc0 con 0x7f42740ff250 2024-02-16T19:28:37.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.234+0000 7f427be07700 1 -- 172.21.15.22:0/2776745397 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f427419b430 con 0x7f42740ff250 2024-02-16T19:28:37.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.234+0000 7f426affd700 1 -- 172.21.15.22:0/2776745397 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4270017630 con 0x7f42740ff250 2024-02-16T19:28:37.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.235+0000 7f427be07700 1 -- 172.21.15.22:0/2776745397 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4274107bf0 con 0x7f42740ff250 2024-02-16T19:28:37.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.236+0000 7f426affd700 1 -- 172.21.15.22:0/2776745397 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f427000faf0 con 0x7f42740ff250 2024-02-16T19:28:37.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.236+0000 7f426affd700 1 --2- 172.21.15.22:0/2776745397 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f426006c680 0x7f426006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:28:37.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.236+0000 7f426affd700 1 -- 172.21.15.22:0/2776745397 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f427008cdf0 con 0x7f42740ff250 2024-02-16T19:28:37.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.236+0000 7f42793a2700 1 --2- 172.21.15.22:0/2776745397 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f426006c680 0x7f426006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:28:37.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.237+0000 7f42793a2700 1 --2- 172.21.15.22:0/2776745397 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f426006c680 0x7f426006eb30 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4264011490 tx=0x7f4264009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:28:37.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.242+0000 7f426affd700 1 -- 172.21.15.22:0/2776745397 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f427005ae60 con 0x7f42740ff250 2024-02-16T19:28:37.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.555+0000 7f427be07700 1 -- 172.21.15.22:0/2776745397 --> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f4274104bb0 con 0x7f426006c680 2024-02-16T19:28:37.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.569+0000 7f426affd700 1 -- 172.21.15.22:0/2776745397 <== mgr.14217 v2:172.21.15.22:6800/2253867279 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3504 (secure 0 0 0) 0x7f4274104bb0 con 0x7f426006c680 2024-02-16T19:28:37.570 INFO:teuthology.orchestra.run.smithi022.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T19:28:37.570 INFO:teuthology.orchestra.run.smithi022.stdout:alertmanager.smithi022 smithi022 *:9093,9094 running (6m) 109s ago 7m 27.3M - 0.25.0 c8568f914cd2 4dac585d8372 2024-02-16T19:28:37.571 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-exporter.smithi022 smithi022 running (8m) 109s ago 8m 7428k - 18.2.0 dc2bc1663786 3f8bef4349ba 2024-02-16T19:28:37.571 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-exporter.smithi117 smithi117 running (7m) 115s ago 7m 7159k - 18.2.0 dc2bc1663786 59b65a7e309f 2024-02-16T19:28:37.571 INFO:teuthology.orchestra.run.smithi022.stdout:crash.smithi022 smithi022 running (8m) 109s ago 8m 6669k - 18.2.0 dc2bc1663786 efef476aa486 2024-02-16T19:28:37.571 INFO:teuthology.orchestra.run.smithi022.stdout:crash.smithi117 smithi117 running (6m) 115s ago 6m 6660k - 18.2.0 dc2bc1663786 5466453eb149 2024-02-16T19:28:37.571 INFO:teuthology.orchestra.run.smithi022.stdout:grafana.smithi022 smithi022 *:3000 running (6m) 109s ago 7m 84.6M - 9.4.7 954c08fa6188 a48872ecbbf2 2024-02-16T19:28:37.571 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi022.btppwm smithi022 running (2m) 109s ago 2m 14.9M - 18.2.0 dc2bc1663786 22971fb3a907 2024-02-16T19:28:37.571 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi022.ocukvs smithi022 running (2m) 109s ago 2m 11.5M - 18.2.0 dc2bc1663786 81ee7bd5fd39 2024-02-16T19:28:37.571 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi117.ibmppn smithi117 running (2m) 115s ago 2m 13.3M - 18.2.0 dc2bc1663786 8c606cbeeb86 2024-02-16T19:28:37.572 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi117.orzors smithi117 running (2m) 115s ago 2m 12.2M - 18.2.0 dc2bc1663786 7d2bc4df3feb 2024-02-16T19:28:37.572 INFO:teuthology.orchestra.run.smithi022.stdout:mgr.smithi022.gmdcch smithi022 *:9283,8765,8443 running (9m) 109s ago 9m 487M - 18.2.0 dc2bc1663786 e8435b394d04 2024-02-16T19:28:37.572 INFO:teuthology.orchestra.run.smithi022.stdout:mgr.smithi117.cohbtv smithi117 *:8443,9283,8765 starting - - - - 2024-02-16T19:28:37.572 INFO:teuthology.orchestra.run.smithi022.stdout:mon.smithi022 smithi022 running (9m) 109s ago 9m 45.2M 2048M 18.2.0 dc2bc1663786 b4ffc3032e12 2024-02-16T19:28:37.572 INFO:teuthology.orchestra.run.smithi022.stdout:mon.smithi117 smithi117 running (6m) 115s ago 6m 35.5M 2048M 18.2.0 dc2bc1663786 aff3e21a73a0 2024-02-16T19:28:37.572 INFO:teuthology.orchestra.run.smithi022.stdout:node-exporter.smithi022 smithi022 *:9100 running (8m) 109s ago 8m 16.0M - 1.5.0 0da6a335fe13 d727f51df007 2024-02-16T19:28:37.572 INFO:teuthology.orchestra.run.smithi022.stdout:node-exporter.smithi117 smithi117 *:9100 running (6m) 115s ago 6m 17.0M - 1.5.0 0da6a335fe13 fc8b2df5a29d 2024-02-16T19:28:37.572 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0 smithi022 running (5m) 109s ago 5m 47.4M 3948M 18.2.0 dc2bc1663786 0027f86f16ce 2024-02-16T19:28:37.572 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1 smithi022 running (5m) 109s ago 5m 48.5M 3948M 18.2.0 dc2bc1663786 87300382b8a7 2024-02-16T19:28:37.572 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2 smithi022 running (4m) 109s ago 4m 46.7M 3948M 18.2.0 dc2bc1663786 be8a631b4f93 2024-02-16T19:28:37.572 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3 smithi117 running (4m) 115s ago 4m 48.9M 4970M 18.2.0 dc2bc1663786 9c0ce9a981cf 2024-02-16T19:28:37.572 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4 smithi117 running (4m) 115s ago 4m 44.8M 4970M 18.2.0 dc2bc1663786 1779c3496f4d 2024-02-16T19:28:37.573 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5 smithi117 running (3m) 115s ago 3m 43.5M 4970M 18.2.0 dc2bc1663786 fefaabca0638 2024-02-16T19:28:37.573 INFO:teuthology.orchestra.run.smithi022.stdout:prometheus.smithi022 smithi022 *:9095 running (6m) 109s ago 7m 60.2M - 2.43.0 a07b618ecd1d a1a5b92b5ef7 2024-02-16T19:28:37.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.571+0000 7f427be07700 1 -- 172.21.15.22:0/2776745397 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f426006c680 msgr2=0x7f426006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:37.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.571+0000 7f427be07700 1 --2- 172.21.15.22:0/2776745397 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f426006c680 0x7f426006eb30 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4264011490 tx=0x7f4264009260 comp rx=0 tx=0).stop 2024-02-16T19:28:37.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.571+0000 7f427be07700 1 -- 172.21.15.22:0/2776745397 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42740ff250 msgr2=0x7f42740fed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:37.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.571+0000 7f427be07700 1 --2- 172.21.15.22:0/2776745397 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42740ff250 0x7f42740fed00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f42700092e0 tx=0x7f427000f690 comp rx=0 tx=0).stop 2024-02-16T19:28:37.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.571+0000 7f427be07700 1 -- 172.21.15.22:0/2776745397 shutdown_connections 2024-02-16T19:28:37.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.571+0000 7f427be07700 1 --2- 172.21.15.22:0/2776745397 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f426006c680 0x7f426006eb30 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:37.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.571+0000 7f427be07700 1 --2- 172.21.15.22:0/2776745397 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42740ff250 0x7f42740fed00 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:37.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.571+0000 7f427be07700 1 --2- 172.21.15.22:0/2776745397 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4274100680 0x7f427419a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:37.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.571+0000 7f427be07700 1 -- 172.21.15.22:0/2776745397 >> 172.21.15.22:0/2776745397 conn(0x7f42740f9d30 msgr2=0x7f4274103490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:28:37.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.571+0000 7f427be07700 1 -- 172.21.15.22:0/2776745397 shutdown_connections 2024-02-16T19:28:37.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.571+0000 7f427be07700 1 -- 172.21.15.22:0/2776745397 wait complete. 2024-02-16T19:28:37.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.714+0000 7f843b678700 1 -- 172.21.15.22:0/2643376999 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84340ff130 msgr2=0x7f84340ff540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:37.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.714+0000 7f843b678700 1 --2- 172.21.15.22:0/2643376999 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84340ff130 0x7f84340ff540 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8430009910 tx=0x7f8430009c20 comp rx=0 tx=0).stop 2024-02-16T19:28:37.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.714+0000 7f843b678700 1 -- 172.21.15.22:0/2643376999 shutdown_connections 2024-02-16T19:28:37.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.714+0000 7f843b678700 1 --2- 172.21.15.22:0/2643376999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f84341003d0 0x7f8434100840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:37.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.714+0000 7f843b678700 1 --2- 172.21.15.22:0/2643376999 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84340ff130 0x7f84340ff540 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:37.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.714+0000 7f843b678700 1 -- 172.21.15.22:0/2643376999 >> 172.21.15.22:0/2643376999 conn(0x7f84340fa780 msgr2=0x7f84340fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:28:37.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.714+0000 7f843b678700 1 -- 172.21.15.22:0/2643376999 shutdown_connections 2024-02-16T19:28:37.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.714+0000 7f843b678700 1 -- 172.21.15.22:0/2643376999 wait complete. 2024-02-16T19:28:37.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.714+0000 7f843b678700 1 Processor -- start 2024-02-16T19:28:37.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.715+0000 7f843b678700 1 -- start start 2024-02-16T19:28:37.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.715+0000 7f843b678700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84340ff130 0x7f84341949b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:28:37.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.715+0000 7f843b678700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f84341003d0 0x7f8434194ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:28:37.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.715+0000 7f843b678700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8434195510 con 0x7f84340ff130 2024-02-16T19:28:37.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.715+0000 7f843b678700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8434195650 con 0x7f84341003d0 2024-02-16T19:28:37.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.715+0000 7f8439414700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84340ff130 0x7f84341949b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:28:37.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.715+0000 7f8439414700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84340ff130 0x7f84341949b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37282/0 (socket says 172.21.15.22:37282) 2024-02-16T19:28:37.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.715+0000 7f8439414700 1 -- 172.21.15.22:0/1323046511 learned_addr learned my addr 172.21.15.22:0/1323046511 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:28:37.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.715+0000 7f8438c13700 1 --2- 172.21.15.22:0/1323046511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f84341003d0 0x7f8434194ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:28:37.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.715+0000 7f8439414700 1 -- 172.21.15.22:0/1323046511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f84341003d0 msgr2=0x7f8434194ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:37.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.715+0000 7f8439414700 1 --2- 172.21.15.22:0/1323046511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f84341003d0 0x7f8434194ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:37.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.715+0000 7f8439414700 1 -- 172.21.15.22:0/1323046511 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84300095f0 con 0x7f84340ff130 2024-02-16T19:28:37.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.716+0000 7f8439414700 1 --2- 172.21.15.22:0/1323046511 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84340ff130 0x7f84341949b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f8430000c00 tx=0x7f843000f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:28:37.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.716+0000 7f842a7fc700 1 -- 172.21.15.22:0/1323046511 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f843001d070 con 0x7f84340ff130 2024-02-16T19:28:37.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.716+0000 7f842a7fc700 1 -- 172.21.15.22:0/1323046511 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8430005dc0 con 0x7f84340ff130 2024-02-16T19:28:37.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.716+0000 7f843b678700 1 -- 172.21.15.22:0/1323046511 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8434068680 con 0x7f84340ff130 2024-02-16T19:28:37.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.716+0000 7f842a7fc700 1 -- 172.21.15.22:0/1323046511 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f84300226a0 con 0x7f84340ff130 2024-02-16T19:28:37.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.716+0000 7f843b678700 1 -- 172.21.15.22:0/1323046511 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8434068b70 con 0x7f84340ff130 2024-02-16T19:28:37.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.717+0000 7f843b678700 1 -- 172.21.15.22:0/1323046511 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8418005320 con 0x7f84340ff130 2024-02-16T19:28:37.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.718+0000 7f842a7fc700 1 -- 172.21.15.22:0/1323046511 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f843000fba0 con 0x7f84340ff130 2024-02-16T19:28:37.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.719+0000 7f842a7fc700 1 --2- 172.21.15.22:0/1323046511 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f842006c680 0x7f842006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:28:37.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.719+0000 7f842a7fc700 1 -- 172.21.15.22:0/1323046511 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f843008cd20 con 0x7f84340ff130 2024-02-16T19:28:37.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.719+0000 7f8438c13700 1 --2- 172.21.15.22:0/1323046511 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f842006c680 0x7f842006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:28:37.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.720+0000 7f8438c13700 1 --2- 172.21.15.22:0/1323046511 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f842006c680 0x7f842006eb30 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8424009bf0 tx=0x7f8424009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:28:37.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:37.725+0000 7f842a7fc700 1 -- 172.21.15.22:0/1323046511 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f843005aeb0 con 0x7f84340ff130 2024-02-16T19:28:37.877 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:37 smithi117 ceph-mon[35309]: from='client.14586 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:28:37.878 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:37 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:28:37.878 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:37 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:28:37.878 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:37 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:28:37.878 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:37 smithi117 ceph-mon[35309]: pgmap v250: 65 pgs: 65 active+clean; 167 MiB data, 735 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 4.6 MiB/s wr, 339 op/s 2024-02-16T19:28:37.964 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:37 smithi022 ceph-mon[29221]: from='client.14586 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:28:37.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:37 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:28:37.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:37 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:28:37.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:37 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:28:37.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:37 smithi022 ceph-mon[29221]: pgmap v250: 65 pgs: 65 active+clean; 167 MiB data, 735 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 4.6 MiB/s wr, 339 op/s 2024-02-16T19:28:38.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:38.188+0000 7f843b678700 1 -- 172.21.15.22:0/1323046511 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f8418006200 con 0x7f84340ff130 2024-02-16T19:28:38.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:38.189+0000 7f842a7fc700 1 -- 172.21.15.22:0/1323046511 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+558 (secure 0 0 0) 0x7f843005aa40 con 0x7f84340ff130 2024-02-16T19:28:38.190 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-02-16T19:28:38.190 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": { 2024-02-16T19:28:38.190 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-02-16T19:28:38.190 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:28:38.190 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": { 2024-02-16T19:28:38.190 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-02-16T19:28:38.190 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:28:38.190 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": { 2024-02-16T19:28:38.190 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-02-16T19:28:38.190 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:28:38.191 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": { 2024-02-16T19:28:38.191 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-02-16T19:28:38.191 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:28:38.191 INFO:teuthology.orchestra.run.smithi022.stdout: "overall": { 2024-02-16T19:28:38.191 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 14 2024-02-16T19:28:38.191 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-02-16T19:28:38.191 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-02-16T19:28:38.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:38.192+0000 7f843b678700 1 -- 172.21.15.22:0/1323046511 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f842006c680 msgr2=0x7f842006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:38.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:38.192+0000 7f843b678700 1 --2- 172.21.15.22:0/1323046511 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f842006c680 0x7f842006eb30 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8424009bf0 tx=0x7f8424009260 comp rx=0 tx=0).stop 2024-02-16T19:28:38.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:38.192+0000 7f843b678700 1 -- 172.21.15.22:0/1323046511 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84340ff130 msgr2=0x7f84341949b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:38.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:38.192+0000 7f843b678700 1 --2- 172.21.15.22:0/1323046511 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84340ff130 0x7f84341949b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f8430000c00 tx=0x7f843000f740 comp rx=0 tx=0).stop 2024-02-16T19:28:38.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:38.192+0000 7f843b678700 1 -- 172.21.15.22:0/1323046511 shutdown_connections 2024-02-16T19:28:38.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:38.192+0000 7f843b678700 1 --2- 172.21.15.22:0/1323046511 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84340ff130 0x7f84341949b0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:38.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:38.192+0000 7f843b678700 1 --2- 172.21.15.22:0/1323046511 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f842006c680 0x7f842006eb30 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:38.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:38.192+0000 7f843b678700 1 --2- 172.21.15.22:0/1323046511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f84341003d0 0x7f8434194ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:38.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:38.192+0000 7f843b678700 1 -- 172.21.15.22:0/1323046511 >> 172.21.15.22:0/1323046511 conn(0x7f84340fa780 msgr2=0x7f8434103600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:28:38.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:38.192+0000 7f843b678700 1 -- 172.21.15.22:0/1323046511 shutdown_connections 2024-02-16T19:28:38.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:38.192+0000 7f843b678700 1 -- 172.21.15.22:0/1323046511 wait complete. 2024-02-16T19:28:38.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:38.319+0000 7f8361d28700 1 -- 172.21.15.22:0/768556796 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f835c100340 msgr2=0x7f835c100790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:38.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:38.319+0000 7f8361d28700 1 --2- 172.21.15.22:0/768556796 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f835c100340 0x7f835c100790 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f834c009910 tx=0x7f834c009c20 comp rx=0 tx=0).stop 2024-02-16T19:28:38.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:38.319+0000 7f8361d28700 1 -- 172.21.15.22:0/768556796 shutdown_connections 2024-02-16T19:28:38.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:38.319+0000 7f8361d28700 1 --2- 172.21.15.22:0/768556796 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f835c100340 0x7f835c100790 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:38.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:38.319+0000 7f8361d28700 1 --2- 172.21.15.22:0/768556796 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f835c0ff140 0x7f835c0ff550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:38.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:38.319+0000 7f8361d28700 1 -- 172.21.15.22:0/768556796 >> 172.21.15.22:0/768556796 conn(0x7f835c0fa6d0 msgr2=0x7f835c0fcb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:28:38.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:38.319+0000 7f8361d28700 1 -- 172.21.15.22:0/768556796 shutdown_connections 2024-02-16T19:28:38.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:38.319+0000 7f8361d28700 1 -- 172.21.15.22:0/768556796 wait complete. 2024-02-16T19:28:38.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:38.320+0000 7f8361d28700 1 Processor -- start 2024-02-16T19:28:38.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:38.320+0000 7f8361d28700 1 -- start start 2024-02-16T19:28:38.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:38.320+0000 7f8361d28700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f835c0ff140 0x7f835c1030d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:28:38.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:38.320+0000 7f8361d28700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f835c100340 0x7f835c101720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:28:38.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:38.320+0000 7f8361d28700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f835c101c60 con 0x7f835c100340 2024-02-16T19:28:38.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:38.320+0000 7f8361d28700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f835c101da0 con 0x7f835c0ff140 2024-02-16T19:28:38.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:38.320+0000 7f835affd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f835c100340 0x7f835c101720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:28:38.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:38.320+0000 7f835affd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f835c100340 0x7f835c101720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37296/0 (socket says 172.21.15.22:37296) 2024-02-16T19:28:38.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:38.320+0000 7f835affd700 1 -- 172.21.15.22:0/2052031836 learned_addr learned my addr 172.21.15.22:0/2052031836 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:28:38.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:38.320+0000 7f835b7fe700 1 --2- 172.21.15.22:0/2052031836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f835c0ff140 0x7f835c1030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:28:38.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:38.321+0000 7f835affd700 1 -- 172.21.15.22:0/2052031836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f835c0ff140 msgr2=0x7f835c1030d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:38.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:38.321+0000 7f835affd700 1 --2- 172.21.15.22:0/2052031836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f835c0ff140 0x7f835c1030d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:38.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:38.321+0000 7f835affd700 1 -- 172.21.15.22:0/2052031836 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f834c0095f0 con 0x7f835c100340 2024-02-16T19:28:38.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:38.321+0000 7f835affd700 1 --2- 172.21.15.22:0/2052031836 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f835c100340 0x7f835c101720 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f834c00bd70 tx=0x7f834c00bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:28:38.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:38.321+0000 7f8358ff9700 1 -- 172.21.15.22:0/2052031836 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f834c01d070 con 0x7f835c100340 2024-02-16T19:28:38.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:38.321+0000 7f8361d28700 1 -- 172.21.15.22:0/2052031836 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f835c102020 con 0x7f835c100340 2024-02-16T19:28:38.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:38.321+0000 7f8358ff9700 1 -- 172.21.15.22:0/2052031836 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f834c005dc0 con 0x7f835c100340 2024-02-16T19:28:38.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:38.321+0000 7f8358ff9700 1 -- 172.21.15.22:0/2052031836 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f834c021410 con 0x7f835c100340 2024-02-16T19:28:38.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:38.321+0000 7f8361d28700 1 -- 172.21.15.22:0/2052031836 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f835c102510 con 0x7f835c100340 2024-02-16T19:28:38.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:38.322+0000 7f8361d28700 1 -- 172.21.15.22:0/2052031836 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f835c107c90 con 0x7f835c100340 2024-02-16T19:28:38.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:38.324+0000 7f8358ff9700 1 -- 172.21.15.22:0/2052031836 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f834c00f7d0 con 0x7f835c100340 2024-02-16T19:28:38.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:38.324+0000 7f8358ff9700 1 --2- 172.21.15.22:0/2052031836 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f834406c6d0 0x7f834406eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:28:38.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:38.324+0000 7f8358ff9700 1 -- 172.21.15.22:0/2052031836 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f834c08d010 con 0x7f835c100340 2024-02-16T19:28:38.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:38.325+0000 7f835b7fe700 1 --2- 172.21.15.22:0/2052031836 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f834406c6d0 0x7f834406eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:28:38.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:38.326+0000 7f835b7fe700 1 --2- 172.21.15.22:0/2052031836 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f834406c6d0 0x7f834406eb80 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f835c1001a0 tx=0x7f8350007400 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:28:38.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:38.330+0000 7f8358ff9700 1 -- 172.21.15.22:0/2052031836 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f834c05b100 con 0x7f835c100340 2024-02-16T19:28:38.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:38.656+0000 7f8361d28700 1 -- 172.21.15.22:0/2052031836 --> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f835c19df70 con 0x7f834406c6d0 2024-02-16T19:28:38.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:38.658+0000 7f8358ff9700 1 -- 172.21.15.22:0/2052031836 <== mgr.14217 v2:172.21.15.22:6800/2253867279 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+321 (secure 0 0 0) 0x7f835c19df70 con 0x7f834406c6d0 2024-02-16T19:28:38.659 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-02-16T19:28:38.659 INFO:teuthology.orchestra.run.smithi022.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T19:28:38.659 INFO:teuthology.orchestra.run.smithi022.stdout: "in_progress": true, 2024-02-16T19:28:38.659 INFO:teuthology.orchestra.run.smithi022.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-02-16T19:28:38.659 INFO:teuthology.orchestra.run.smithi022.stdout: "services_complete": [], 2024-02-16T19:28:38.659 INFO:teuthology.orchestra.run.smithi022.stdout: "progress": "", 2024-02-16T19:28:38.659 INFO:teuthology.orchestra.run.smithi022.stdout: "message": "Currently upgrading mgr daemons", 2024-02-16T19:28:38.659 INFO:teuthology.orchestra.run.smithi022.stdout: "is_paused": false 2024-02-16T19:28:38.659 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-02-16T19:28:38.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:38.660+0000 7f8361d28700 1 -- 172.21.15.22:0/2052031836 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f834406c6d0 msgr2=0x7f834406eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:38.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:38.660+0000 7f8361d28700 1 --2- 172.21.15.22:0/2052031836 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f834406c6d0 0x7f834406eb80 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f835c1001a0 tx=0x7f8350007400 comp rx=0 tx=0).stop 2024-02-16T19:28:38.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:38.660+0000 7f8361d28700 1 -- 172.21.15.22:0/2052031836 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f835c100340 msgr2=0x7f835c101720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:38.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:38.660+0000 7f8361d28700 1 --2- 172.21.15.22:0/2052031836 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f835c100340 0x7f835c101720 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f834c00bd70 tx=0x7f834c00bda0 comp rx=0 tx=0).stop 2024-02-16T19:28:38.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:38.660+0000 7f8361d28700 1 -- 172.21.15.22:0/2052031836 shutdown_connections 2024-02-16T19:28:38.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:38.660+0000 7f8361d28700 1 --2- 172.21.15.22:0/2052031836 >> [v2:172.21.15.22:6800/2253867279,v1:172.21.15.22:6801/2253867279] conn(0x7f834406c6d0 0x7f834406eb80 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:38.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:38.660+0000 7f8361d28700 1 --2- 172.21.15.22:0/2052031836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f835c0ff140 0x7f835c1030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:38.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:38.660+0000 7f8361d28700 1 --2- 172.21.15.22:0/2052031836 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f835c100340 0x7f835c101720 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:38.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:38.660+0000 7f8361d28700 1 -- 172.21.15.22:0/2052031836 >> 172.21.15.22:0/2052031836 conn(0x7f835c0fa6d0 msgr2=0x7f835c068dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:28:38.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:38.660+0000 7f8361d28700 1 -- 172.21.15.22:0/2052031836 shutdown_connections 2024-02-16T19:28:38.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:28:38.660+0000 7f8361d28700 1 -- 172.21.15.22:0/2052031836 wait complete. 2024-02-16T19:28:38.725 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:38 smithi117 ceph-mon[35309]: from='client.14594 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:28:38.725 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:38 smithi117 ceph-mon[35309]: from='client.24357 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:28:38.725 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:38 smithi117 ceph-mon[35309]: from='client.? 172.21.15.22:0/1323046511' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:28:38.964 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:38 smithi022 ceph-mon[29221]: from='client.14594 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:28:38.964 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:38 smithi022 ceph-mon[29221]: from='client.24357 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:28:38.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:38 smithi022 ceph-mon[29221]: from='client.? 172.21.15.22:0/1323046511' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:28:39.964 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:39 smithi022 ceph-mon[29221]: from='client.14606 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:28:39.964 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:39 smithi022 ceph-mon[29221]: pgmap v251: 65 pgs: 65 active+clean; 175 MiB data, 742 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 4.2 MiB/s wr, 311 op/s 2024-02-16T19:28:40.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:39 smithi117 ceph-mon[35309]: from='client.14606 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:28:40.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:39 smithi117 ceph-mon[35309]: pgmap v251: 65 pgs: 65 active+clean; 175 MiB data, 742 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 4.2 MiB/s wr, 311 op/s 2024-02-16T19:28:40.490 INFO:tasks.workunit.client.1.smithi117.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-02-16T19:28:40.495 INFO:tasks.workunit.client.1.smithi117.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-02-16T19:28:40.495 INFO:tasks.workunit.client.1.smithi117.stderr:+ make 2024-02-16T19:28:40.843 INFO:tasks.workunit.client.1.smithi117.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-02-16T19:28:42.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:41 smithi117 ceph-mon[35309]: pgmap v252: 65 pgs: 65 active+clean; 191 MiB data, 790 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 5.0 MiB/s wr, 357 op/s 2024-02-16T19:28:42.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:41 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:28:42.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:41 smithi022 ceph-mon[29221]: pgmap v252: 65 pgs: 65 active+clean; 191 MiB data, 790 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 5.0 MiB/s wr, 357 op/s 2024-02-16T19:28:42.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:41 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:28:43.453 INFO:tasks.workunit.client.1.smithi117.stderr:++ readlink -f fsstress 2024-02-16T19:28:43.455 INFO:tasks.workunit.client.1.smithi117.stderr:+ BIN=/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-02-16T19:28:43.455 INFO:tasks.workunit.client.1.smithi117.stderr:+ popd 2024-02-16T19:28:43.456 INFO:tasks.workunit.client.1.smithi117.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-02-16T19:28:43.456 INFO:tasks.workunit.client.1.smithi117.stderr:+ popd 2024-02-16T19:28:43.456 INFO:tasks.workunit.client.1.smithi117.stdout:~/cephtest/mnt.1/client.1/tmp 2024-02-16T19:28:43.457 INFO:tasks.workunit.client.1.smithi117.stderr:++ mktemp -d -p . 2024-02-16T19:28:43.459 INFO:tasks.workunit.client.1.smithi117.stderr:+ T=./tmp.hytcJGc2wO 2024-02-16T19:28:43.459 INFO:tasks.workunit.client.1.smithi117.stderr:+ /home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.hytcJGc2wO -l 1 -n 1000 -p 10 -v 2024-02-16T19:28:43.463 INFO:tasks.workunit.client.1.smithi117.stdout:seed = 1707647676 2024-02-16T19:28:43.468 INFO:tasks.workunit.client.1.smithi117.stdout:0/0: write - no filename 2024-02-16T19:28:43.470 INFO:tasks.workunit.client.1.smithi117.stdout:1/0: dread - no filename 2024-02-16T19:28:43.470 INFO:tasks.workunit.client.1.smithi117.stdout:1/1: dread - no filename 2024-02-16T19:28:43.473 INFO:tasks.workunit.client.1.smithi117.stdout:2/0: write - no filename 2024-02-16T19:28:43.474 INFO:tasks.workunit.client.1.smithi117.stdout:0/1: mkdir d0 0 2024-02-16T19:28:43.474 INFO:tasks.workunit.client.1.smithi117.stdout:0/2: read - no filename 2024-02-16T19:28:43.476 INFO:tasks.workunit.client.1.smithi117.stdout:1/2: symlink l0 0 2024-02-16T19:28:43.477 INFO:tasks.workunit.client.1.smithi117.stdout:3/0: readlink - no filename 2024-02-16T19:28:43.477 INFO:tasks.workunit.client.1.smithi117.stdout:3/1: dwrite - no filename 2024-02-16T19:28:43.477 INFO:tasks.workunit.client.1.smithi117.stdout:3/2: chown . 334030585 1 2024-02-16T19:28:43.477 INFO:tasks.workunit.client.1.smithi117.stdout:3/3: fsync - no filename 2024-02-16T19:28:43.480 INFO:tasks.workunit.client.1.smithi117.stdout:1/3: rename l0 to l1 0 2024-02-16T19:28:43.480 INFO:tasks.workunit.client.1.smithi117.stdout:2/1: mkdir d0 0 2024-02-16T19:28:43.482 INFO:tasks.workunit.client.1.smithi117.stdout:4/0: stat - no entries 2024-02-16T19:28:43.483 INFO:tasks.workunit.client.1.smithi117.stdout:4/1: rmdir - no directory 2024-02-16T19:28:43.483 INFO:tasks.workunit.client.1.smithi117.stdout:4/2: chown . 41286 1 2024-02-16T19:28:43.483 INFO:tasks.workunit.client.1.smithi117.stdout:4/3: dread - no filename 2024-02-16T19:28:43.483 INFO:tasks.workunit.client.1.smithi117.stdout:4/4: dread - no filename 2024-02-16T19:28:43.483 INFO:tasks.workunit.client.1.smithi117.stdout:4/5: write - no filename 2024-02-16T19:28:43.484 INFO:tasks.workunit.client.1.smithi117.stdout:0/3: creat d0/f1 x:0 0 0 2024-02-16T19:28:43.485 INFO:tasks.workunit.client.1.smithi117.stdout:0/4: write d0/f1 [161996,37378] 0 2024-02-16T19:28:43.485 INFO:tasks.workunit.client.1.smithi117.stdout:1/4: mkdir d2 0 2024-02-16T19:28:43.489 INFO:tasks.workunit.client.1.smithi117.stdout:5/0: write - no filename 2024-02-16T19:28:43.489 INFO:tasks.workunit.client.1.smithi117.stdout:3/4: getdents . 0 2024-02-16T19:28:43.489 INFO:tasks.workunit.client.1.smithi117.stdout:3/5: truncate - no filename 2024-02-16T19:28:43.490 INFO:tasks.workunit.client.1.smithi117.stdout:3/6: dread - no filename 2024-02-16T19:28:43.493 INFO:tasks.workunit.client.1.smithi117.stdout:2/2: creat d0/f1 x:0 0 0 2024-02-16T19:28:43.493 INFO:tasks.workunit.client.1.smithi117.stdout:2/3: chown d0/f1 2271 1 2024-02-16T19:28:43.493 INFO:tasks.workunit.client.1.smithi117.stdout:2/4: read - d0/f1 zero size 2024-02-16T19:28:43.494 INFO:tasks.workunit.client.1.smithi117.stdout:4/6: mkdir d0 0 2024-02-16T19:28:43.494 INFO:tasks.workunit.client.1.smithi117.stdout:4/7: write - no filename 2024-02-16T19:28:43.494 INFO:tasks.workunit.client.1.smithi117.stdout:4/8: write - no filename 2024-02-16T19:28:43.494 INFO:tasks.workunit.client.1.smithi117.stdout:2/5: write d0/f1 [218448,86301] 0 2024-02-16T19:28:43.495 INFO:tasks.workunit.client.1.smithi117.stdout:2/6: stat d0/f1 0 2024-02-16T19:28:43.497 INFO:tasks.workunit.client.1.smithi117.stdout:2/7: write d0/f1 [151654,75089] 0 2024-02-16T19:28:43.497 INFO:tasks.workunit.client.1.smithi117.stdout:6/0: rename - no filename 2024-02-16T19:28:43.502 INFO:tasks.workunit.client.1.smithi117.stdout:5/1: symlink l0 0 2024-02-16T19:28:43.502 INFO:tasks.workunit.client.1.smithi117.stdout:5/2: fdatasync - no filename 2024-02-16T19:28:43.505 INFO:tasks.workunit.client.1.smithi117.stdout:3/7: creat f0 x:0 0 0 2024-02-16T19:28:43.508 INFO:tasks.workunit.client.1.smithi117.stdout:2/8: creat d0/f2 x:0 0 0 2024-02-16T19:28:43.508 INFO:tasks.workunit.client.1.smithi117.stdout:4/9: mkdir d0/d1 0 2024-02-16T19:28:43.508 INFO:tasks.workunit.client.1.smithi117.stdout:4/10: dwrite - no filename 2024-02-16T19:28:43.509 INFO:tasks.workunit.client.1.smithi117.stdout:5/3: mknod c1 0 2024-02-16T19:28:43.511 INFO:tasks.workunit.client.1.smithi117.stdout:8/0: truncate - no filename 2024-02-16T19:28:43.512 INFO:tasks.workunit.client.1.smithi117.stdout:8/1: write - no filename 2024-02-16T19:28:43.512 INFO:tasks.workunit.client.1.smithi117.stdout:8/2: dwrite - no filename 2024-02-16T19:28:43.514 INFO:tasks.workunit.client.1.smithi117.stdout:3/8: mkdir d1 0 2024-02-16T19:28:43.515 INFO:tasks.workunit.client.1.smithi117.stdout:6/1: creat f0 x:0 0 0 2024-02-16T19:28:43.516 INFO:tasks.workunit.client.1.smithi117.stdout:4/11: symlink d0/l2 0 2024-02-16T19:28:43.516 INFO:tasks.workunit.client.1.smithi117.stdout:4/12: read - no filename 2024-02-16T19:28:43.516 INFO:tasks.workunit.client.1.smithi117.stdout:4/13: fdatasync - no filename 2024-02-16T19:28:43.516 INFO:tasks.workunit.client.1.smithi117.stdout:2/9: dwrite d0/f2 [0,4194304] 0 2024-02-16T19:28:43.517 INFO:tasks.workunit.client.1.smithi117.stdout:5/4: mkdir d2 0 2024-02-16T19:28:43.517 INFO:tasks.workunit.client.1.smithi117.stdout:5/5: read - no filename 2024-02-16T19:28:43.517 INFO:tasks.workunit.client.1.smithi117.stdout:5/6: dwrite - no filename 2024-02-16T19:28:43.517 INFO:tasks.workunit.client.1.smithi117.stdout:4/14: rename d0/d1 to d0/d1/d3 22 2024-02-16T19:28:43.517 INFO:tasks.workunit.client.1.smithi117.stdout:4/15: dwrite - no filename 2024-02-16T19:28:43.517 INFO:tasks.workunit.client.1.smithi117.stdout:4/16: dread - no filename 2024-02-16T19:28:43.517 INFO:tasks.workunit.client.1.smithi117.stdout:4/17: dread - no filename 2024-02-16T19:28:43.517 INFO:tasks.workunit.client.1.smithi117.stdout:4/18: write - no filename 2024-02-16T19:28:43.518 INFO:tasks.workunit.client.1.smithi117.stdout:5/7: chown l0 6019943 1 2024-02-16T19:28:43.518 INFO:tasks.workunit.client.1.smithi117.stdout:2/10: write d0/f2 [2583562,94026] 0 2024-02-16T19:28:43.518 INFO:tasks.workunit.client.1.smithi117.stdout:7/0: creat f0 x:0 0 0 2024-02-16T19:28:43.518 INFO:tasks.workunit.client.1.smithi117.stdout:4/19: stat d0/d1 0 2024-02-16T19:28:43.518 INFO:tasks.workunit.client.1.smithi117.stdout:4/20: read - no filename 2024-02-16T19:28:43.518 INFO:tasks.workunit.client.1.smithi117.stdout:4/21: dread - no filename 2024-02-16T19:28:43.519 INFO:tasks.workunit.client.1.smithi117.stdout:4/22: dread - no filename 2024-02-16T19:28:43.519 INFO:tasks.workunit.client.1.smithi117.stdout:7/1: stat f0 0 2024-02-16T19:28:43.519 INFO:tasks.workunit.client.1.smithi117.stdout:9/0: unlink - no file 2024-02-16T19:28:43.524 INFO:tasks.workunit.client.1.smithi117.stdout:6/2: rename f0 to f1 0 2024-02-16T19:28:43.528 INFO:tasks.workunit.client.1.smithi117.stdout:2/11: mkdir d0/d3 0 2024-02-16T19:28:43.528 INFO:tasks.workunit.client.1.smithi117.stdout:2/12: chown d0/f1 27063 1 2024-02-16T19:28:43.529 INFO:tasks.workunit.client.1.smithi117.stdout:5/8: unlink c1 0 2024-02-16T19:28:43.529 INFO:tasks.workunit.client.1.smithi117.stdout:5/9: dwrite - no filename 2024-02-16T19:28:43.533 INFO:tasks.workunit.client.1.smithi117.stdout:7/2: mknod c1 0 2024-02-16T19:28:43.533 INFO:tasks.workunit.client.1.smithi117.stdout:8/3: mknod c0 0 2024-02-16T19:28:43.533 INFO:tasks.workunit.client.1.smithi117.stdout:8/4: dwrite - no filename 2024-02-16T19:28:43.536 INFO:tasks.workunit.client.1.smithi117.stdout:6/3: creat f2 x:0 0 0 2024-02-16T19:28:43.541 INFO:tasks.workunit.client.1.smithi117.stdout:7/3: dwrite f0 [0,4194304] 0 2024-02-16T19:28:43.541 INFO:tasks.workunit.client.1.smithi117.stdout:8/5: rename c0 to c1 0 2024-02-16T19:28:43.542 INFO:tasks.workunit.client.1.smithi117.stdout:2/13: mknod d0/d3/c4 0 2024-02-16T19:28:43.542 INFO:tasks.workunit.client.1.smithi117.stdout:2/14: chown d0 16157 1 2024-02-16T19:28:43.543 INFO:tasks.workunit.client.1.smithi117.stdout:6/4: write f1 [631656,34678] 0 2024-02-16T19:28:43.545 INFO:tasks.workunit.client.1.smithi117.stdout:5/10: mkdir d2/d3 0 2024-02-16T19:28:43.548 INFO:tasks.workunit.client.1.smithi117.stdout:4/23: rmdir d0/d1 0 2024-02-16T19:28:43.548 INFO:tasks.workunit.client.1.smithi117.stdout:4/24: write - no filename 2024-02-16T19:28:43.548 INFO:tasks.workunit.client.1.smithi117.stdout:4/25: chown d0 106495681 1 2024-02-16T19:28:43.548 INFO:tasks.workunit.client.1.smithi117.stdout:4/26: dwrite - no filename 2024-02-16T19:28:43.550 INFO:tasks.workunit.client.1.smithi117.stdout:7/4: creat f2 x:0 0 0 2024-02-16T19:28:43.553 INFO:tasks.workunit.client.1.smithi117.stdout:6/5: mknod c3 0 2024-02-16T19:28:43.554 INFO:tasks.workunit.client.1.smithi117.stdout:6/6: truncate f1 743538 0 2024-02-16T19:28:43.554 INFO:tasks.workunit.client.1.smithi117.stdout:5/11: mkdir d2/d4 0 2024-02-16T19:28:43.554 INFO:tasks.workunit.client.1.smithi117.stdout:5/12: write - no filename 2024-02-16T19:28:43.554 INFO:tasks.workunit.client.1.smithi117.stdout:5/13: dread - no filename 2024-02-16T19:28:43.555 INFO:tasks.workunit.client.1.smithi117.stdout:5/14: chown l0 2270 1 2024-02-16T19:28:43.556 INFO:tasks.workunit.client.1.smithi117.stdout:4/27: mknod d0/c4 0 2024-02-16T19:28:43.557 INFO:tasks.workunit.client.1.smithi117.stdout:7/5: creat f3 x:0 0 0 2024-02-16T19:28:43.560 INFO:tasks.workunit.client.1.smithi117.stdout:4/28: rename d0/l2 to d0/l5 0 2024-02-16T19:28:43.561 INFO:tasks.workunit.client.1.smithi117.stdout:7/6: mkdir d4 0 2024-02-16T19:28:43.562 INFO:tasks.workunit.client.1.smithi117.stdout:7/7: write f0 [303319,107843] 0 2024-02-16T19:28:43.562 INFO:tasks.workunit.client.1.smithi117.stdout:6/7: link f2 f4 0 2024-02-16T19:28:43.563 INFO:tasks.workunit.client.1.smithi117.stdout:5/15: creat d2/d3/f5 x:0 0 0 2024-02-16T19:28:43.564 INFO:tasks.workunit.client.1.smithi117.stdout:6/8: read f1 [326791,16686] 0 2024-02-16T19:28:43.564 INFO:tasks.workunit.client.1.smithi117.stdout:4/29: rmdir d0 39 2024-02-16T19:28:43.564 INFO:tasks.workunit.client.1.smithi117.stdout:4/30: write - no filename 2024-02-16T19:28:43.569 INFO:tasks.workunit.client.1.smithi117.stdout:6/9: fdatasync f2 0 2024-02-16T19:28:43.572 INFO:tasks.workunit.client.1.smithi117.stdout:7/8: creat d4/f5 x:0 0 0 2024-02-16T19:28:43.576 INFO:tasks.workunit.client.1.smithi117.stdout:6/10: dwrite f4 [0,4194304] 0 2024-02-16T19:28:44.177 INFO:tasks.workunit.client.1.smithi117.stdout:1/5: stat l1 0 2024-02-16T19:28:44.177 INFO:tasks.workunit.client.1.smithi117.stdout:1/6: readlink l1 0 2024-02-16T19:28:44.177 INFO:tasks.workunit.client.1.smithi117.stdout:1/7: write - no filename 2024-02-16T19:28:44.183 INFO:tasks.workunit.client.1.smithi117.stdout:1/8: getdents d2 0 2024-02-16T19:28:44.184 INFO:tasks.workunit.client.1.smithi117.stdout:2/15: fsync d0/f1 0 2024-02-16T19:28:44.186 INFO:tasks.workunit.client.1.smithi117.stdout:4/31: stat d0 0 2024-02-16T19:28:44.189 INFO:tasks.workunit.client.1.smithi117.stdout:2/16: creat d0/f5 x:0 0 0 2024-02-16T19:28:44.190 INFO:tasks.workunit.client.1.smithi117.stdout:0/5: truncate d0/f1 69263 0 2024-02-16T19:28:44.190 INFO:tasks.workunit.client.1.smithi117.stdout:1/9: mknod d2/c3 0 2024-02-16T19:28:44.191 INFO:tasks.workunit.client.1.smithi117.stdout:1/10: chown d2 29727895 1 2024-02-16T19:28:44.191 INFO:tasks.workunit.client.1.smithi117.stdout:1/11: write - no filename 2024-02-16T19:28:44.192 INFO:tasks.workunit.client.1.smithi117.stdout:4/32: mkdir d0/d6 0 2024-02-16T19:28:44.192 INFO:tasks.workunit.client.1.smithi117.stdout:4/33: truncate - no filename 2024-02-16T19:28:44.194 INFO:tasks.workunit.client.1.smithi117.stdout:2/17: rename d0/f2 to d0/d3/f6 0 2024-02-16T19:28:44.194 INFO:tasks.workunit.client.1.smithi117.stdout:3/9: fsync f0 0 2024-02-16T19:28:44.196 INFO:tasks.workunit.client.1.smithi117.stdout:0/6: mknod d0/c2 0 2024-02-16T19:28:44.197 INFO:tasks.workunit.client.1.smithi117.stdout:1/12: mkdir d2/d4 0 2024-02-16T19:28:44.197 INFO:tasks.workunit.client.1.smithi117.stdout:1/13: truncate - no filename 2024-02-16T19:28:44.206 INFO:tasks.workunit.client.1.smithi117.stdout:4/34: mkdir d0/d6/d7 0 2024-02-16T19:28:44.209 INFO:tasks.workunit.client.1.smithi117.stdout:2/18: link d0/f5 d0/d3/f7 0 2024-02-16T19:28:44.209 INFO:tasks.workunit.client.1.smithi117.stdout:2/19: stat d0/d3/f7 0 2024-02-16T19:28:44.211 INFO:tasks.workunit.client.1.smithi117.stdout:3/10: creat d1/f2 x:0 0 0 2024-02-16T19:28:44.211 INFO:tasks.workunit.client.1.smithi117.stdout:3/11: read - f0 zero size 2024-02-16T19:28:44.214 INFO:tasks.workunit.client.1.smithi117.stdout:2/20: mknod d0/d3/c8 0 2024-02-16T19:28:44.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:43 smithi022 ceph-mon[29221]: pgmap v253: 65 pgs: 65 active+clean; 196 MiB data, 802 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 4.5 MiB/s wr, 353 op/s 2024-02-16T19:28:44.215 INFO:tasks.workunit.client.1.smithi117.stdout:3/12: mknod d1/c3 0 2024-02-16T19:28:44.216 INFO:tasks.workunit.client.1.smithi117.stdout:3/13: write f0 [14460,50487] 0 2024-02-16T19:28:44.220 INFO:tasks.workunit.client.1.smithi117.stdout:3/14: symlink d1/l4 0 2024-02-16T19:28:44.221 INFO:tasks.workunit.client.1.smithi117.stdout:8/6: link c1 c2 0 2024-02-16T19:28:44.223 INFO:tasks.workunit.client.1.smithi117.stdout:3/15: creat d1/f5 x:0 0 0 2024-02-16T19:28:44.224 INFO:tasks.workunit.client.1.smithi117.stdout:3/16: write d1/f2 [6778,28644] 0 2024-02-16T19:28:44.225 INFO:tasks.workunit.client.1.smithi117.stdout:8/7: mknod c3 0 2024-02-16T19:28:44.225 INFO:tasks.workunit.client.1.smithi117.stdout:8/8: fdatasync - no filename 2024-02-16T19:28:44.227 INFO:tasks.workunit.client.1.smithi117.stdout:3/17: symlink d1/l6 0 2024-02-16T19:28:44.228 INFO:tasks.workunit.client.1.smithi117.stdout:3/18: dread - d1/f5 zero size 2024-02-16T19:28:44.228 INFO:tasks.workunit.client.1.smithi117.stdout:3/19: dread - d1/f5 zero size 2024-02-16T19:28:44.229 INFO:tasks.workunit.client.1.smithi117.stdout:8/9: mkdir d4 0 2024-02-16T19:28:44.229 INFO:tasks.workunit.client.1.smithi117.stdout:3/20: dread f0 [0,4194304] 0 2024-02-16T19:28:44.231 INFO:tasks.workunit.client.1.smithi117.stdout:6/11: fsync f4 0 2024-02-16T19:28:44.231 INFO:tasks.workunit.client.1.smithi117.stdout:6/12: stat f4 0 2024-02-16T19:28:44.231 INFO:tasks.workunit.client.1.smithi117.stdout:6/13: fsync f2 0 2024-02-16T19:28:44.232 INFO:tasks.workunit.client.1.smithi117.stdout:5/16: rmdir d2/d3 39 2024-02-16T19:28:44.234 INFO:tasks.workunit.client.1.smithi117.stdout:3/21: mknod d1/c7 0 2024-02-16T19:28:44.237 INFO:tasks.workunit.client.1.smithi117.stdout:8/10: mknod d4/c5 0 2024-02-16T19:28:44.237 INFO:tasks.workunit.client.1.smithi117.stdout:8/11: truncate - no filename 2024-02-16T19:28:44.237 INFO:tasks.workunit.client.1.smithi117.stdout:8/12: readlink - no filename 2024-02-16T19:28:44.241 INFO:tasks.workunit.client.1.smithi117.stdout:3/22: creat d1/f8 x:0 0 0 2024-02-16T19:28:44.241 INFO:tasks.workunit.client.1.smithi117.stdout:6/14: dread f1 [0,4194304] 0 2024-02-16T19:28:44.242 INFO:tasks.workunit.client.1.smithi117.stdout:8/13: creat d4/f6 x:0 0 0 2024-02-16T19:28:44.246 INFO:tasks.workunit.client.1.smithi117.stdout:6/15: mkdir d5 0 2024-02-16T19:28:44.248 INFO:tasks.workunit.client.1.smithi117.stdout:5/17: dwrite d2/d3/f5 [0,4194304] 0 2024-02-16T19:28:44.249 INFO:tasks.workunit.client.1.smithi117.stdout:6/16: write f4 [2354328,25892] 0 2024-02-16T19:28:44.254 INFO:tasks.workunit.client.1.smithi117.stdout:8/14: dwrite d4/f6 [0,4194304] 0 2024-02-16T19:28:44.254 INFO:tasks.workunit.client.1.smithi117.stdout:6/17: fdatasync f1 0 2024-02-16T19:28:44.263 INFO:tasks.workunit.client.1.smithi117.stdout:8/15: unlink c3 0 2024-02-16T19:28:44.265 INFO:tasks.workunit.client.1.smithi117.stdout:8/16: mkdir d4/d7 0 2024-02-16T19:28:44.266 INFO:tasks.workunit.client.1.smithi117.stdout:6/18: getdents d5 0 2024-02-16T19:28:44.267 INFO:tasks.workunit.client.1.smithi117.stdout:6/19: write f2 [1799004,115324] 0 2024-02-16T19:28:44.269 INFO:tasks.workunit.client.1.smithi117.stdout:8/17: dread d4/f6 [0,4194304] 0 2024-02-16T19:28:44.277 INFO:tasks.workunit.client.1.smithi117.stdout:6/20: symlink d5/l6 0 2024-02-16T19:28:44.279 INFO:tasks.workunit.client.1.smithi117.stdout:8/18: symlink d4/d7/l8 0 2024-02-16T19:28:44.280 INFO:tasks.workunit.client.1.smithi117.stdout:6/21: rmdir d5 39 2024-02-16T19:28:44.280 INFO:tasks.workunit.client.1.smithi117.stdout:6/22: read f4 [1722394,12012] 0 2024-02-16T19:28:44.281 INFO:tasks.workunit.client.1.smithi117.stdout:6/23: write f2 [1283333,52258] 0 2024-02-16T19:28:44.287 INFO:tasks.workunit.client.1.smithi117.stdout:8/19: dwrite d4/f6 [0,4194304] 0 2024-02-16T19:28:44.290 INFO:tasks.workunit.client.1.smithi117.stdout:8/20: symlink d4/d7/l9 0 2024-02-16T19:28:44.296 INFO:tasks.workunit.client.1.smithi117.stdout:8/21: rename c2 to d4/d7/ca 0 2024-02-16T19:28:44.299 INFO:tasks.workunit.client.1.smithi117.stdout:8/22: mknod d4/d7/cb 0 2024-02-16T19:28:44.302 INFO:tasks.workunit.client.1.smithi117.stdout:8/23: dread d4/f6 [0,4194304] 0 2024-02-16T19:28:44.305 INFO:tasks.workunit.client.1.smithi117.stdout:8/24: creat d4/d7/fc x:0 0 0 2024-02-16T19:28:44.306 INFO:tasks.workunit.client.1.smithi117.stdout:8/25: stat d4/d7/l9 0 2024-02-16T19:28:44.308 INFO:tasks.workunit.client.1.smithi117.stdout:8/26: mknod d4/cd 0 2024-02-16T19:28:44.311 INFO:tasks.workunit.client.1.smithi117.stdout:8/27: symlink d4/le 0 2024-02-16T19:28:44.376 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:43 smithi117 ceph-mon[35309]: pgmap v253: 65 pgs: 65 active+clean; 196 MiB data, 802 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 4.5 MiB/s wr, 353 op/s 2024-02-16T19:28:44.445 INFO:tasks.workunit.client.1.smithi117.stdout:0/7: dwrite d0/f1 [0,4194304] 0 2024-02-16T19:28:44.449 INFO:tasks.workunit.client.1.smithi117.stdout:0/8: dread d0/f1 [0,4194304] 0 2024-02-16T19:28:44.450 INFO:tasks.workunit.client.1.smithi117.stdout:0/9: read d0/f1 [2332414,25581] 0 2024-02-16T19:28:44.451 INFO:tasks.workunit.client.1.smithi117.stdout:0/10: mknod d0/c3 0 2024-02-16T19:28:44.460 INFO:tasks.workunit.client.1.smithi117.stdout:3/23: getdents d1 0 2024-02-16T19:28:44.460 INFO:tasks.workunit.client.1.smithi117.stdout:3/24: write d1/f2 [548290,101304] 0 2024-02-16T19:28:44.461 INFO:tasks.workunit.client.1.smithi117.stdout:0/11: fdatasync d0/f1 0 2024-02-16T19:28:44.463 INFO:tasks.workunit.client.1.smithi117.stdout:0/12: creat d0/f4 x:0 0 0 2024-02-16T19:28:44.465 INFO:tasks.workunit.client.1.smithi117.stdout:3/25: link d1/f2 d1/f9 0 2024-02-16T19:28:44.465 INFO:tasks.workunit.client.1.smithi117.stdout:5/18: write d2/d3/f5 [4396739,14177] 0 2024-02-16T19:28:44.465 INFO:tasks.workunit.client.1.smithi117.stdout:0/13: creat d0/f5 x:0 0 0 2024-02-16T19:28:44.466 INFO:tasks.workunit.client.1.smithi117.stdout:0/14: write d0/f1 [2857953,118471] 0 2024-02-16T19:28:44.467 INFO:tasks.workunit.client.1.smithi117.stdout:0/15: stat d0/f4 0 2024-02-16T19:28:44.467 INFO:tasks.workunit.client.1.smithi117.stdout:0/16: rename d0 to d0/d6 22 2024-02-16T19:28:44.468 INFO:tasks.workunit.client.1.smithi117.stdout:3/26: creat d1/fa x:0 0 0 2024-02-16T19:28:44.468 INFO:tasks.workunit.client.1.smithi117.stdout:0/17: chown d0/f5 120 1 2024-02-16T19:28:44.468 INFO:tasks.workunit.client.1.smithi117.stdout:3/27: read - d1/f5 zero size 2024-02-16T19:28:44.468 INFO:tasks.workunit.client.1.smithi117.stdout:3/28: chown d1 4112139 1 2024-02-16T19:28:44.469 INFO:tasks.workunit.client.1.smithi117.stdout:0/18: truncate d0/f5 242969 0 2024-02-16T19:28:44.469 INFO:tasks.workunit.client.1.smithi117.stdout:5/19: creat d2/d3/f6 x:0 0 0 2024-02-16T19:28:44.469 INFO:tasks.workunit.client.1.smithi117.stdout:3/29: chown d1/fa 16166280 1 2024-02-16T19:28:44.470 INFO:tasks.workunit.client.1.smithi117.stdout:3/30: dread - d1/f5 zero size 2024-02-16T19:28:44.472 INFO:tasks.workunit.client.1.smithi117.stdout:3/31: fsync f0 0 2024-02-16T19:28:44.473 INFO:tasks.workunit.client.1.smithi117.stdout:6/24: truncate f1 861955 0 2024-02-16T19:28:44.473 INFO:tasks.workunit.client.1.smithi117.stdout:3/32: write d1/f5 [614890,45087] 0 2024-02-16T19:28:44.475 INFO:tasks.workunit.client.1.smithi117.stdout:3/33: dread - d1/f8 zero size 2024-02-16T19:28:44.475 INFO:tasks.workunit.client.1.smithi117.stdout:6/25: stat d5/l6 0 2024-02-16T19:28:44.476 INFO:tasks.workunit.client.1.smithi117.stdout:5/20: dwrite d2/d3/f5 [0,4194304] 0 2024-02-16T19:28:44.478 INFO:tasks.workunit.client.1.smithi117.stdout:3/34: mknod d1/cb 0 2024-02-16T19:28:44.478 INFO:tasks.workunit.client.1.smithi117.stdout:3/35: chown d1/c7 165 1 2024-02-16T19:28:44.479 INFO:tasks.workunit.client.1.smithi117.stdout:6/26: write f4 [2829750,109857] 0 2024-02-16T19:28:44.480 INFO:tasks.workunit.client.1.smithi117.stdout:5/21: creat d2/d3/f7 x:0 0 0 2024-02-16T19:28:44.480 INFO:tasks.workunit.client.1.smithi117.stdout:5/22: chown d2 86513 1 2024-02-16T19:28:44.481 INFO:tasks.workunit.client.1.smithi117.stdout:3/36: creat d1/fc x:0 0 0 2024-02-16T19:28:44.481 INFO:tasks.workunit.client.1.smithi117.stdout:3/37: chown d1/l4 4177549 1 2024-02-16T19:28:44.482 INFO:tasks.workunit.client.1.smithi117.stdout:6/27: unlink d5/l6 0 2024-02-16T19:28:44.486 INFO:tasks.workunit.client.1.smithi117.stdout:6/28: creat d5/f7 x:0 0 0 2024-02-16T19:28:44.488 INFO:tasks.workunit.client.1.smithi117.stdout:5/23: dwrite d2/d3/f5 [0,4194304] 0 2024-02-16T19:28:44.489 INFO:tasks.workunit.client.1.smithi117.stdout:5/24: write d2/d3/f7 [586458,23371] 0 2024-02-16T19:28:44.497 INFO:tasks.workunit.client.1.smithi117.stdout:5/25: dwrite d2/d3/f7 [0,4194304] 0 2024-02-16T19:28:44.500 INFO:tasks.workunit.client.1.smithi117.stdout:8/28: getdents d4 0 2024-02-16T19:28:44.502 INFO:tasks.workunit.client.1.smithi117.stdout:8/29: symlink d4/d7/lf 0 2024-02-16T19:28:44.503 INFO:tasks.workunit.client.1.smithi117.stdout:8/30: chown d4/f6 100983 1 2024-02-16T19:28:44.506 INFO:tasks.workunit.client.1.smithi117.stdout:8/31: dread d4/f6 [0,4194304] 0 2024-02-16T19:28:44.512 INFO:tasks.workunit.client.1.smithi117.stdout:8/32: dwrite d4/f6 [0,4194304] 0 2024-02-16T19:28:44.516 INFO:tasks.workunit.client.1.smithi117.stdout:8/33: getdents d4 0 2024-02-16T19:28:44.517 INFO:tasks.workunit.client.1.smithi117.stdout:8/34: mknod d4/c10 0 2024-02-16T19:28:44.527 INFO:tasks.workunit.client.1.smithi117.stdout:8/35: fdatasync d4/f6 0 2024-02-16T19:28:44.549 INFO:tasks.workunit.client.1.smithi117.stdout:0/19: fsync d0/f4 0 2024-02-16T19:28:44.553 INFO:tasks.workunit.client.1.smithi117.stdout:3/38: write f0 [342537,53557] 0 2024-02-16T19:28:44.556 INFO:tasks.workunit.client.1.smithi117.stdout:0/20: dwrite d0/f5 [0,4194304] 0 2024-02-16T19:28:44.556 INFO:tasks.workunit.client.1.smithi117.stdout:3/39: mkdir d1/dd 0 2024-02-16T19:28:44.559 INFO:tasks.workunit.client.1.smithi117.stdout:3/40: mkdir d1/de 0 2024-02-16T19:28:44.560 INFO:tasks.workunit.client.1.smithi117.stdout:0/21: rename d0/f4 to d0/f7 0 2024-02-16T19:28:44.561 INFO:tasks.workunit.client.1.smithi117.stdout:6/29: getdents d5 0 2024-02-16T19:28:44.563 INFO:tasks.workunit.client.1.smithi117.stdout:0/22: write d0/f7 [540116,14241] 0 2024-02-16T19:28:44.565 INFO:tasks.workunit.client.1.smithi117.stdout:6/30: creat d5/f8 x:0 0 0 2024-02-16T19:28:44.565 INFO:tasks.workunit.client.1.smithi117.stdout:5/26: write d2/d3/f7 [4813178,7748] 0 2024-02-16T19:28:44.566 INFO:tasks.workunit.client.1.smithi117.stdout:6/31: chown d5/f8 1042350 1 2024-02-16T19:28:44.566 INFO:tasks.workunit.client.1.smithi117.stdout:3/41: dread d1/f2 [0,4194304] 0 2024-02-16T19:28:44.566 INFO:tasks.workunit.client.1.smithi117.stdout:0/23: symlink d0/l8 0 2024-02-16T19:28:44.567 INFO:tasks.workunit.client.1.smithi117.stdout:5/27: read d2/d3/f5 [842238,112152] 0 2024-02-16T19:28:44.567 INFO:tasks.workunit.client.1.smithi117.stdout:8/36: rmdir d4/d7 39 2024-02-16T19:28:44.569 INFO:tasks.workunit.client.1.smithi117.stdout:8/37: write d4/f6 [761343,114487] 0 2024-02-16T19:28:44.573 INFO:tasks.workunit.client.1.smithi117.stdout:0/24: mkdir d0/d9 0 2024-02-16T19:28:44.574 INFO:tasks.workunit.client.1.smithi117.stdout:6/32: dwrite f4 [4194304,4194304] 0 2024-02-16T19:28:44.574 INFO:tasks.workunit.client.1.smithi117.stdout:5/28: dread d2/d3/f7 [0,4194304] 0 2024-02-16T19:28:44.575 INFO:tasks.workunit.client.1.smithi117.stdout:6/33: chown d5/f7 1040612279 1 2024-02-16T19:28:44.577 INFO:tasks.workunit.client.1.smithi117.stdout:0/25: dread d0/f7 [0,4194304] 0 2024-02-16T19:28:44.580 INFO:tasks.workunit.client.1.smithi117.stdout:6/34: stat f1 0 2024-02-16T19:28:44.582 INFO:tasks.workunit.client.1.smithi117.stdout:6/35: fdatasync d5/f8 0 2024-02-16T19:28:44.582 INFO:tasks.workunit.client.1.smithi117.stdout:8/38: mknod d4/d7/c11 0 2024-02-16T19:28:44.588 INFO:tasks.workunit.client.1.smithi117.stdout:0/26: dwrite d0/f5 [4194304,4194304] 0 2024-02-16T19:28:44.590 INFO:tasks.workunit.client.1.smithi117.stdout:5/29: dwrite d2/d3/f7 [4194304,4194304] 0 2024-02-16T19:28:44.595 INFO:tasks.workunit.client.1.smithi117.stdout:0/27: rmdir d0 39 2024-02-16T19:28:44.601 INFO:tasks.workunit.client.1.smithi117.stdout:5/30: dwrite d2/d3/f6 [0,4194304] 0 2024-02-16T19:28:44.602 INFO:tasks.workunit.client.1.smithi117.stdout:5/31: chown d2/d3/f7 127 1 2024-02-16T19:28:44.602 INFO:tasks.workunit.client.1.smithi117.stdout:0/28: mknod d0/d9/ca 0 2024-02-16T19:28:44.604 INFO:tasks.workunit.client.1.smithi117.stdout:8/39: truncate d4/f6 721645 0 2024-02-16T19:28:44.608 INFO:tasks.workunit.client.1.smithi117.stdout:8/40: write d4/d7/fc [539601,109295] 0 2024-02-16T19:28:44.609 INFO:tasks.workunit.client.1.smithi117.stdout:0/29: dread d0/f7 [0,4194304] 0 2024-02-16T19:28:44.611 INFO:tasks.workunit.client.1.smithi117.stdout:6/36: dwrite f1 [0,4194304] 0 2024-02-16T19:28:44.611 INFO:tasks.workunit.client.1.smithi117.stdout:8/41: write d4/d7/fc [246469,21816] 0 2024-02-16T19:28:44.612 INFO:tasks.workunit.client.1.smithi117.stdout:0/30: symlink d0/lb 0 2024-02-16T19:28:44.614 INFO:tasks.workunit.client.1.smithi117.stdout:6/37: creat d5/f9 x:0 0 0 2024-02-16T19:28:44.615 INFO:tasks.workunit.client.1.smithi117.stdout:0/31: truncate d0/f7 1520248 0 2024-02-16T19:28:44.615 INFO:tasks.workunit.client.1.smithi117.stdout:8/42: mkdir d4/d12 0 2024-02-16T19:28:44.616 INFO:tasks.workunit.client.1.smithi117.stdout:6/38: creat d5/fa x:0 0 0 2024-02-16T19:28:44.617 INFO:tasks.workunit.client.1.smithi117.stdout:0/32: mknod d0/cc 0 2024-02-16T19:28:44.618 INFO:tasks.workunit.client.1.smithi117.stdout:6/39: mkdir d5/db 0 2024-02-16T19:28:44.620 INFO:tasks.workunit.client.1.smithi117.stdout:0/33: creat d0/fd x:0 0 0 2024-02-16T19:28:44.620 INFO:tasks.workunit.client.1.smithi117.stdout:6/40: mknod d5/db/cc 0 2024-02-16T19:28:44.621 INFO:tasks.workunit.client.1.smithi117.stdout:0/34: creat d0/fe x:0 0 0 2024-02-16T19:28:44.623 INFO:tasks.workunit.client.1.smithi117.stdout:0/35: mknod d0/cf 0 2024-02-16T19:28:44.623 INFO:tasks.workunit.client.1.smithi117.stdout:0/36: dread - d0/fd zero size 2024-02-16T19:28:44.624 INFO:tasks.workunit.client.1.smithi117.stdout:0/37: chown d0/fd 9384 1 2024-02-16T19:28:44.637 INFO:tasks.workunit.client.1.smithi117.stdout:5/32: fsync d2/d3/f7 0 2024-02-16T19:28:44.637 INFO:tasks.workunit.client.1.smithi117.stdout:6/41: fdatasync f1 0 2024-02-16T19:28:44.641 INFO:tasks.workunit.client.1.smithi117.stdout:5/33: dread d2/d3/f6 [0,4194304] 0 2024-02-16T19:28:44.641 INFO:tasks.workunit.client.1.smithi117.stdout:6/42: symlink d5/ld 0 2024-02-16T19:28:44.641 INFO:tasks.workunit.client.1.smithi117.stdout:5/34: readlink l0 0 2024-02-16T19:28:44.642 INFO:tasks.workunit.client.1.smithi117.stdout:0/38: getdents d0/d9 0 2024-02-16T19:28:44.644 INFO:tasks.workunit.client.1.smithi117.stdout:5/35: dread d2/d3/f6 [0,4194304] 0 2024-02-16T19:28:44.644 INFO:tasks.workunit.client.1.smithi117.stdout:6/43: creat d5/fe x:0 0 0 2024-02-16T19:28:44.644 INFO:tasks.workunit.client.1.smithi117.stdout:8/43: write d4/f6 [738222,31138] 0 2024-02-16T19:28:44.645 INFO:tasks.workunit.client.1.smithi117.stdout:0/39: creat d0/f10 x:0 0 0 2024-02-16T19:28:44.647 INFO:tasks.workunit.client.1.smithi117.stdout:8/44: fdatasync d4/f6 0 2024-02-16T19:28:44.648 INFO:tasks.workunit.client.1.smithi117.stdout:8/45: readlink d4/d7/lf 0 2024-02-16T19:28:44.648 INFO:tasks.workunit.client.1.smithi117.stdout:6/44: creat d5/ff x:0 0 0 2024-02-16T19:28:44.649 INFO:tasks.workunit.client.1.smithi117.stdout:8/46: dread d4/f6 [0,4194304] 0 2024-02-16T19:28:44.652 INFO:tasks.workunit.client.1.smithi117.stdout:0/40: creat d0/d9/f11 x:0 0 0 2024-02-16T19:28:44.652 INFO:tasks.workunit.client.1.smithi117.stdout:0/41: dread - d0/d9/f11 zero size 2024-02-16T19:28:44.653 INFO:tasks.workunit.client.1.smithi117.stdout:6/45: symlink d5/l10 0 2024-02-16T19:28:44.685 INFO:tasks.workunit.client.1.smithi117.stdout:0/42: creat d0/d9/f12 x:0 0 0 2024-02-16T19:28:44.686 INFO:tasks.workunit.client.1.smithi117.stdout:0/43: dread - d0/d9/f12 zero size 2024-02-16T19:28:44.696 INFO:tasks.workunit.client.1.smithi117.stdout:5/36: dwrite d2/d3/f5 [4194304,4194304] 0 2024-02-16T19:28:44.697 INFO:tasks.workunit.client.1.smithi117.stdout:6/46: dwrite d5/f9 [0,4194304] 0 2024-02-16T19:28:44.699 INFO:tasks.workunit.client.1.smithi117.stdout:0/44: dwrite d0/f10 [0,4194304] 0 2024-02-16T19:28:44.699 INFO:tasks.workunit.client.1.smithi117.stdout:0/45: fsync d0/fe 0 2024-02-16T19:28:44.757 INFO:tasks.workunit.client.1.smithi117.stdout:8/47: symlink d4/d12/l13 0 2024-02-16T19:28:44.759 INFO:tasks.workunit.client.1.smithi117.stdout:5/37: rmdir d2 39 2024-02-16T19:28:44.761 INFO:tasks.workunit.client.1.smithi117.stdout:0/46: creat d0/d9/f13 x:0 0 0 2024-02-16T19:28:44.762 INFO:tasks.workunit.client.1.smithi117.stdout:8/48: symlink d4/d7/l14 0 2024-02-16T19:28:44.765 INFO:tasks.workunit.client.1.smithi117.stdout:0/47: mkdir d0/d9/d14 0 2024-02-16T19:28:44.766 INFO:tasks.workunit.client.1.smithi117.stdout:8/49: rmdir d4 39 2024-02-16T19:28:44.770 INFO:tasks.workunit.client.1.smithi117.stdout:5/38: dwrite d2/d3/f5 [4194304,4194304] 0 2024-02-16T19:28:44.772 INFO:tasks.workunit.client.1.smithi117.stdout:5/39: chown d2/d4 939775 1 2024-02-16T19:28:44.773 INFO:tasks.workunit.client.1.smithi117.stdout:0/48: link d0/fd d0/d9/d14/f15 0 2024-02-16T19:28:44.775 INFO:tasks.workunit.client.1.smithi117.stdout:5/40: mknod d2/d3/c8 0 2024-02-16T19:28:44.776 INFO:tasks.workunit.client.1.smithi117.stdout:5/41: readlink l0 0 2024-02-16T19:28:44.780 INFO:tasks.workunit.client.1.smithi117.stdout:8/50: dwrite d4/d7/fc [0,4194304] 0 2024-02-16T19:28:44.781 INFO:tasks.workunit.client.1.smithi117.stdout:5/42: write d2/d3/f6 [2083423,102455] 0 2024-02-16T19:28:44.787 INFO:tasks.workunit.client.1.smithi117.stdout:0/49: dwrite d0/fe [0,4194304] 0 2024-02-16T19:28:44.789 INFO:tasks.workunit.client.1.smithi117.stdout:0/50: write d0/d9/f11 [443755,65438] 0 2024-02-16T19:28:44.791 INFO:tasks.workunit.client.1.smithi117.stdout:0/51: dread - d0/fd zero size 2024-02-16T19:28:44.791 INFO:tasks.workunit.client.1.smithi117.stdout:5/43: dread d2/d3/f5 [0,4194304] 0 2024-02-16T19:28:44.796 INFO:tasks.workunit.client.1.smithi117.stdout:0/52: symlink d0/l16 0 2024-02-16T19:28:44.797 INFO:tasks.workunit.client.1.smithi117.stdout:5/44: mkdir d2/d9 0 2024-02-16T19:28:44.798 INFO:tasks.workunit.client.1.smithi117.stdout:0/53: rename d0/c2 to d0/c17 0 2024-02-16T19:28:44.800 INFO:tasks.workunit.client.1.smithi117.stdout:5/45: unlink d2/d3/f5 0 2024-02-16T19:28:44.800 INFO:tasks.workunit.client.1.smithi117.stdout:0/54: creat d0/f18 x:0 0 0 2024-02-16T19:28:44.803 INFO:tasks.workunit.client.1.smithi117.stdout:0/55: mknod d0/d9/d14/c19 0 2024-02-16T19:28:44.803 INFO:tasks.workunit.client.1.smithi117.stdout:5/46: dread d2/d3/f7 [4194304,4194304] 0 2024-02-16T19:28:44.804 INFO:tasks.workunit.client.1.smithi117.stdout:0/56: write d0/fe [3914745,70476] 0 2024-02-16T19:28:44.805 INFO:tasks.workunit.client.1.smithi117.stdout:0/57: dread - d0/d9/f13 zero size 2024-02-16T19:28:44.806 INFO:tasks.workunit.client.1.smithi117.stdout:0/58: dread - d0/d9/d14/f15 zero size 2024-02-16T19:28:44.807 INFO:tasks.workunit.client.1.smithi117.stdout:5/47: rename d2/d3/f7 to d2/d4/fa 0 2024-02-16T19:28:44.812 INFO:tasks.workunit.client.1.smithi117.stdout:0/59: dwrite d0/f5 [4194304,4194304] 0 2024-02-16T19:28:44.813 INFO:tasks.workunit.client.1.smithi117.stdout:5/48: dread d2/d3/f6 [0,4194304] 0 2024-02-16T19:28:44.815 INFO:tasks.workunit.client.1.smithi117.stdout:0/60: creat d0/f1a x:0 0 0 2024-02-16T19:28:44.818 INFO:tasks.workunit.client.1.smithi117.stdout:5/49: mknod d2/cb 0 2024-02-16T19:28:44.820 INFO:tasks.workunit.client.1.smithi117.stdout:5/50: write d2/d4/fa [2310274,118206] 0 2024-02-16T19:28:44.823 INFO:tasks.workunit.client.1.smithi117.stdout:5/51: dread d2/d3/f6 [0,4194304] 0 2024-02-16T19:28:44.824 INFO:tasks.workunit.client.1.smithi117.stdout:5/52: chown l0 132292809 1 2024-02-16T19:28:44.825 INFO:tasks.workunit.client.1.smithi117.stdout:5/53: chown d2/d3/f6 195745 1 2024-02-16T19:28:44.826 INFO:tasks.workunit.client.1.smithi117.stdout:5/54: symlink d2/d4/lc 0 2024-02-16T19:28:44.829 INFO:tasks.workunit.client.1.smithi117.stdout:5/55: getdents d2/d9 0 2024-02-16T19:28:44.831 INFO:tasks.workunit.client.1.smithi117.stdout:5/56: symlink d2/d3/ld 0 2024-02-16T19:28:44.831 INFO:tasks.workunit.client.1.smithi117.stdout:5/57: stat d2/d3/c8 0 2024-02-16T19:28:44.833 INFO:tasks.workunit.client.1.smithi117.stdout:5/58: mkdir d2/d9/de 0 2024-02-16T19:28:44.835 INFO:tasks.workunit.client.1.smithi117.stdout:5/59: rmdir d2/d9 39 2024-02-16T19:28:44.841 INFO:tasks.workunit.client.1.smithi117.stdout:5/60: dwrite d2/d3/f6 [4194304,4194304] 0 2024-02-16T19:28:44.862 INFO:tasks.workunit.client.1.smithi117.stdout:6/47: truncate f2 6918268 0 2024-02-16T19:28:44.862 INFO:tasks.workunit.client.1.smithi117.stdout:6/48: dread - d5/ff zero size 2024-02-16T19:28:44.862 INFO:tasks.workunit.client.1.smithi117.stdout:6/49: fdatasync d5/f8 0 2024-02-16T19:28:44.870 INFO:tasks.workunit.client.1.smithi117.stdout:8/51: truncate d4/d7/fc 1039480 0 2024-02-16T19:28:44.872 INFO:tasks.workunit.client.1.smithi117.stdout:8/52: link d4/d7/l8 d4/d12/l15 0 2024-02-16T19:28:44.875 INFO:tasks.workunit.client.1.smithi117.stdout:8/53: symlink d4/l16 0 2024-02-16T19:28:44.878 INFO:tasks.workunit.client.1.smithi117.stdout:8/54: symlink d4/l17 0 2024-02-16T19:28:44.883 INFO:tasks.workunit.client.1.smithi117.stdout:8/55: rename d4/d7/l8 to d4/d12/l18 0 2024-02-16T19:28:44.891 INFO:tasks.workunit.client.1.smithi117.stdout:0/61: dwrite d0/f7 [0,4194304] 0 2024-02-16T19:28:44.892 INFO:tasks.workunit.client.1.smithi117.stdout:5/61: truncate d2/d3/f6 7531701 0 2024-02-16T19:28:44.894 INFO:tasks.workunit.client.1.smithi117.stdout:0/62: creat d0/d9/d14/f1b x:0 0 0 2024-02-16T19:28:44.895 INFO:tasks.workunit.client.1.smithi117.stdout:0/63: dread - d0/d9/f13 zero size 2024-02-16T19:28:44.896 INFO:tasks.workunit.client.1.smithi117.stdout:0/64: truncate d0/f1a 189080 0 2024-02-16T19:28:44.896 INFO:tasks.workunit.client.1.smithi117.stdout:0/65: write d0/d9/f11 [680703,93586] 0 2024-02-16T19:28:44.897 INFO:tasks.workunit.client.1.smithi117.stdout:0/66: chown d0/d9/f11 491007182 1 2024-02-16T19:28:44.898 INFO:tasks.workunit.client.1.smithi117.stdout:0/67: dread - d0/fd zero size 2024-02-16T19:28:44.904 INFO:tasks.workunit.client.1.smithi117.stdout:5/62: getdents d2/d3 0 2024-02-16T19:28:44.906 INFO:tasks.workunit.client.1.smithi117.stdout:5/63: stat d2/cb 0 2024-02-16T19:28:44.910 INFO:tasks.workunit.client.1.smithi117.stdout:5/64: chown d2/d4/fa 20108 1 2024-02-16T19:28:44.910 INFO:tasks.workunit.client.1.smithi117.stdout:0/68: dwrite d0/d9/f13 [0,4194304] 0 2024-02-16T19:28:44.910 INFO:tasks.workunit.client.1.smithi117.stdout:6/50: dwrite f2 [4194304,4194304] 0 2024-02-16T19:28:44.912 INFO:tasks.workunit.client.1.smithi117.stdout:6/51: readlink d5/l10 0 2024-02-16T19:28:44.912 INFO:tasks.workunit.client.1.smithi117.stdout:5/65: fsync d2/d4/fa 0 2024-02-16T19:28:44.918 INFO:tasks.workunit.client.1.smithi117.stdout:6/52: unlink d5/f8 0 2024-02-16T19:28:44.918 INFO:tasks.workunit.client.1.smithi117.stdout:5/66: dread d2/d3/f6 [0,4194304] 0 2024-02-16T19:28:44.922 INFO:tasks.workunit.client.1.smithi117.stdout:8/56: link d4/d12/l18 d4/d12/l19 0 2024-02-16T19:28:44.924 INFO:tasks.workunit.client.1.smithi117.stdout:8/57: truncate d4/d7/fc 1151862 0 2024-02-16T19:28:44.924 INFO:tasks.workunit.client.1.smithi117.stdout:5/67: mknod d2/d4/cf 0 2024-02-16T19:28:44.929 INFO:tasks.workunit.client.1.smithi117.stdout:0/69: getdents d0 0 2024-02-16T19:28:44.930 INFO:tasks.workunit.client.1.smithi117.stdout:0/70: truncate d0/f1a 1055339 0 2024-02-16T19:28:44.931 INFO:tasks.workunit.client.1.smithi117.stdout:8/58: creat d4/d12/f1a x:0 0 0 2024-02-16T19:28:44.933 INFO:tasks.workunit.client.1.smithi117.stdout:0/71: creat d0/d9/f1c x:0 0 0 2024-02-16T19:28:44.937 INFO:tasks.workunit.client.1.smithi117.stdout:0/72: mkdir d0/d9/d14/d1d 0 2024-02-16T19:28:44.938 INFO:tasks.workunit.client.1.smithi117.stdout:6/53: truncate f2 788550 0 2024-02-16T19:28:44.939 INFO:tasks.workunit.client.1.smithi117.stdout:6/54: fsync d5/fa 0 2024-02-16T19:28:44.940 INFO:tasks.workunit.client.1.smithi117.stdout:0/73: mkdir d0/d9/d14/d1d/d1e 0 2024-02-16T19:28:44.941 INFO:tasks.workunit.client.1.smithi117.stdout:8/59: dwrite d4/d7/fc [0,4194304] 0 2024-02-16T19:28:44.941 INFO:tasks.workunit.client.1.smithi117.stdout:0/74: fsync d0/f1a 0 2024-02-16T19:28:44.942 INFO:tasks.workunit.client.1.smithi117.stdout:5/68: truncate d2/d3/f6 6780152 0 2024-02-16T19:28:44.942 INFO:tasks.workunit.client.1.smithi117.stdout:6/55: creat d5/f11 x:0 0 0 2024-02-16T19:28:44.945 INFO:tasks.workunit.client.1.smithi117.stdout:0/75: getdents d0 0 2024-02-16T19:28:44.946 INFO:tasks.workunit.client.1.smithi117.stdout:5/69: mknod d2/c10 0 2024-02-16T19:28:44.949 INFO:tasks.workunit.client.1.smithi117.stdout:5/70: write d2/d4/fa [2357828,109506] 0 2024-02-16T19:28:44.952 INFO:tasks.workunit.client.1.smithi117.stdout:6/56: dwrite d5/fa [0,4194304] 0 2024-02-16T19:28:44.952 INFO:tasks.workunit.client.1.smithi117.stdout:5/71: unlink l0 0 2024-02-16T19:28:44.957 INFO:tasks.workunit.client.1.smithi117.stdout:0/76: dwrite d0/f18 [0,4194304] 0 2024-02-16T19:28:44.957 INFO:tasks.workunit.client.1.smithi117.stdout:5/72: write d2/d4/fa [3548661,23833] 0 2024-02-16T19:28:44.959 INFO:tasks.workunit.client.1.smithi117.stdout:0/77: readlink d0/lb 0 2024-02-16T19:28:44.961 INFO:tasks.workunit.client.1.smithi117.stdout:0/78: dread - d0/d9/f1c zero size 2024-02-16T19:28:44.964 INFO:tasks.workunit.client.1.smithi117.stdout:6/57: dwrite d5/fa [0,4194304] 0 2024-02-16T19:28:44.966 INFO:tasks.workunit.client.1.smithi117.stdout:5/73: symlink d2/l11 0 2024-02-16T19:28:44.974 INFO:tasks.workunit.client.1.smithi117.stdout:6/58: link d5/ff d5/f12 0 2024-02-16T19:28:44.980 INFO:tasks.workunit.client.1.smithi117.stdout:6/59: dwrite d5/f11 [0,4194304] 0 2024-02-16T19:28:44.994 INFO:tasks.workunit.client.1.smithi117.stdout:6/60: truncate f2 57142 0 2024-02-16T19:28:44.996 INFO:tasks.workunit.client.1.smithi117.stdout:6/61: creat d5/f13 x:0 0 0 2024-02-16T19:28:44.998 INFO:tasks.workunit.client.1.smithi117.stdout:5/74: dread d2/d3/f6 [0,4194304] 0 2024-02-16T19:28:44.998 INFO:tasks.workunit.client.1.smithi117.stdout:6/62: symlink d5/db/l14 0 2024-02-16T19:28:44.999 INFO:tasks.workunit.client.1.smithi117.stdout:5/75: rmdir d2 39 2024-02-16T19:28:45.003 INFO:tasks.workunit.client.1.smithi117.stdout:5/76: creat d2/d3/f12 x:0 0 0 2024-02-16T19:28:45.004 INFO:tasks.workunit.client.1.smithi117.stdout:5/77: readlink d2/d4/lc 0 2024-02-16T19:28:45.005 INFO:tasks.workunit.client.1.smithi117.stdout:6/63: write d5/f12 [248182,26443] 0 2024-02-16T19:28:45.008 INFO:tasks.workunit.client.1.smithi117.stdout:5/78: creat d2/f13 x:0 0 0 2024-02-16T19:28:45.010 INFO:tasks.workunit.client.1.smithi117.stdout:6/64: creat d5/f15 x:0 0 0 2024-02-16T19:28:45.013 INFO:tasks.workunit.client.1.smithi117.stdout:6/65: creat d5/db/f16 x:0 0 0 2024-02-16T19:28:45.017 INFO:tasks.workunit.client.1.smithi117.stdout:6/66: link c3 d5/c17 0 2024-02-16T19:28:45.020 INFO:tasks.workunit.client.1.smithi117.stdout:5/79: write d2/d3/f6 [5587009,113556] 0 2024-02-16T19:28:45.023 INFO:tasks.workunit.client.1.smithi117.stdout:6/67: dwrite d5/f9 [0,4194304] 0 2024-02-16T19:28:45.025 INFO:tasks.workunit.client.1.smithi117.stdout:5/80: unlink d2/d4/fa 0 2024-02-16T19:28:45.026 INFO:tasks.workunit.client.1.smithi117.stdout:5/81: fdatasync d2/d3/f12 0 2024-02-16T19:28:45.029 INFO:tasks.workunit.client.1.smithi117.stdout:6/68: dwrite f1 [0,4194304] 0 2024-02-16T19:28:45.030 INFO:tasks.workunit.client.1.smithi117.stdout:6/69: truncate d5/f13 251267 0 2024-02-16T19:28:45.031 INFO:tasks.workunit.client.1.smithi117.stdout:6/70: fdatasync d5/fa 0 2024-02-16T19:28:45.034 INFO:tasks.workunit.client.1.smithi117.stdout:5/82: creat d2/f14 x:0 0 0 2024-02-16T19:28:45.039 INFO:tasks.workunit.client.1.smithi117.stdout:5/83: symlink d2/d9/l15 0 2024-02-16T19:28:45.043 INFO:tasks.workunit.client.1.smithi117.stdout:5/84: dread d2/d3/f6 [0,4194304] 0 2024-02-16T19:28:45.049 INFO:tasks.workunit.client.1.smithi117.stdout:5/85: dwrite d2/d3/f12 [0,4194304] 0 2024-02-16T19:28:45.064 INFO:tasks.workunit.client.1.smithi117.stdout:5/86: truncate d2/d3/f12 4033149 0 2024-02-16T19:28:45.065 INFO:tasks.workunit.client.1.smithi117.stdout:5/87: readlink d2/d9/l15 0 2024-02-16T19:28:45.067 INFO:tasks.workunit.client.1.smithi117.stdout:5/88: mkdir d2/d4/d16 0 2024-02-16T19:28:45.069 INFO:tasks.workunit.client.1.smithi117.stdout:5/89: mkdir d2/d9/de/d17 0 2024-02-16T19:28:45.070 INFO:tasks.workunit.client.1.smithi117.stdout:5/90: write d2/f14 [327873,62232] 0 2024-02-16T19:28:45.071 INFO:tasks.workunit.client.1.smithi117.stdout:5/91: rmdir d2/d9 39 2024-02-16T19:28:45.076 INFO:tasks.workunit.client.1.smithi117.stdout:5/92: creat d2/d9/f18 x:0 0 0 2024-02-16T19:28:45.077 INFO:tasks.workunit.client.1.smithi117.stdout:5/93: write d2/f14 [8679,110151] 0 2024-02-16T19:28:45.078 INFO:tasks.workunit.client.1.smithi117.stdout:5/94: symlink d2/d4/l19 0 2024-02-16T19:28:45.081 INFO:tasks.workunit.client.1.smithi117.stdout:5/95: creat d2/d4/d16/f1a x:0 0 0 2024-02-16T19:28:45.081 INFO:tasks.workunit.client.1.smithi117.stdout:5/96: dread - d2/d9/f18 zero size 2024-02-16T19:28:45.082 INFO:tasks.workunit.client.1.smithi117.stdout:5/97: dread - d2/d4/d16/f1a zero size 2024-02-16T19:28:45.091 INFO:tasks.workunit.client.1.smithi117.stdout:5/98: dread d2/d3/f12 [0,4194304] 0 2024-02-16T19:28:45.094 INFO:tasks.workunit.client.1.smithi117.stdout:5/99: symlink d2/l1b 0 2024-02-16T19:28:45.094 INFO:tasks.workunit.client.1.smithi117.stdout:5/100: fsync d2/d9/f18 0 2024-02-16T19:28:45.097 INFO:tasks.workunit.client.1.smithi117.stdout:5/101: dread d2/f14 [0,4194304] 0 2024-02-16T19:28:45.099 INFO:tasks.workunit.client.1.smithi117.stdout:5/102: mkdir d2/d9/de/d17/d1c 0 2024-02-16T19:28:45.102 INFO:tasks.workunit.client.1.smithi117.stdout:5/103: mkdir d2/d1d 0 2024-02-16T19:28:45.103 INFO:tasks.workunit.client.1.smithi117.stdout:5/104: chown d2/d9 1606 1 2024-02-16T19:28:45.105 INFO:tasks.workunit.client.1.smithi117.stdout:5/105: creat d2/d4/f1e x:0 0 0 2024-02-16T19:28:45.106 INFO:tasks.workunit.client.1.smithi117.stdout:5/106: readlink d2/d4/l19 0 2024-02-16T19:28:45.107 INFO:tasks.workunit.client.1.smithi117.stdout:5/107: write d2/d3/f12 [4711058,11093] 0 2024-02-16T19:28:45.109 INFO:tasks.workunit.client.1.smithi117.stdout:5/108: symlink d2/d4/d16/l1f 0 2024-02-16T19:28:45.110 INFO:tasks.workunit.client.1.smithi117.stdout:5/109: fdatasync d2/f14 0 2024-02-16T19:28:45.119 INFO:tasks.workunit.client.1.smithi117.stdout:5/110: getdents d2/d4/d16 0 2024-02-16T19:28:45.123 INFO:tasks.workunit.client.1.smithi117.stdout:5/111: dread d2/d3/f12 [0,4194304] 0 2024-02-16T19:28:45.124 INFO:tasks.workunit.client.1.smithi117.stdout:5/112: write d2/d9/f18 [205041,60106] 0 2024-02-16T19:28:45.124 INFO:tasks.workunit.client.1.smithi117.stdout:5/113: fsync d2/d4/f1e 0 2024-02-16T19:28:45.125 INFO:tasks.workunit.client.1.smithi117.stdout:5/114: read - d2/d4/d16/f1a zero size 2024-02-16T19:28:45.127 INFO:tasks.workunit.client.1.smithi117.stdout:5/115: rmdir d2/d9 39 2024-02-16T19:28:45.133 INFO:tasks.workunit.client.1.smithi117.stdout:5/116: creat d2/d9/de/d17/d1c/f20 x:0 0 0 2024-02-16T19:28:45.134 INFO:tasks.workunit.client.1.smithi117.stdout:5/117: write d2/d4/d16/f1a [101737,112816] 0 2024-02-16T19:28:45.135 INFO:tasks.workunit.client.1.smithi117.stdout:5/118: chown d2/d4/d16/l1f 379966835 1 2024-02-16T19:28:45.139 INFO:tasks.workunit.client.1.smithi117.stdout:5/119: dread d2/d3/f12 [0,4194304] 0 2024-02-16T19:28:45.141 INFO:tasks.workunit.client.1.smithi117.stdout:5/120: mkdir d2/d3/d21 0 2024-02-16T19:28:45.141 INFO:tasks.workunit.client.1.smithi117.stdout:5/121: fsync d2/d4/d16/f1a 0 2024-02-16T19:28:45.142 INFO:tasks.workunit.client.1.smithi117.stdout:5/122: dread - d2/d9/de/d17/d1c/f20 zero size 2024-02-16T19:28:45.143 INFO:tasks.workunit.client.1.smithi117.stdout:5/123: dread d2/f14 [0,4194304] 0 2024-02-16T19:28:45.145 INFO:tasks.workunit.client.1.smithi117.stdout:5/124: creat d2/d9/de/f22 x:0 0 0 2024-02-16T19:28:45.145 INFO:tasks.workunit.client.1.smithi117.stdout:5/125: chown d2/d4/cf 275571566 1 2024-02-16T19:28:45.147 INFO:tasks.workunit.client.1.smithi117.stdout:5/126: mkdir d2/d3/d21/d23 0 2024-02-16T19:28:45.153 INFO:tasks.workunit.client.1.smithi117.stdout:5/127: dwrite d2/d9/f18 [0,4194304] 0 2024-02-16T19:28:45.156 INFO:tasks.workunit.client.1.smithi117.stdout:5/128: mkdir d2/d4/d16/d24 0 2024-02-16T19:28:45.157 INFO:tasks.workunit.client.1.smithi117.stdout:5/129: truncate d2/f13 576396 0 2024-02-16T19:28:45.160 INFO:tasks.workunit.client.1.smithi117.stdout:5/130: symlink d2/l25 0 2024-02-16T19:28:45.160 INFO:tasks.workunit.client.1.smithi117.stdout:5/131: write d2/d4/f1e [737711,87803] 0 2024-02-16T19:28:45.163 INFO:tasks.workunit.client.1.smithi117.stdout:5/132: creat d2/d4/d16/d24/f26 x:0 0 0 2024-02-16T19:28:45.164 INFO:tasks.workunit.client.1.smithi117.stdout:5/133: fdatasync d2/d3/f6 0 2024-02-16T19:28:45.166 INFO:tasks.workunit.client.1.smithi117.stdout:5/134: creat d2/d1d/f27 x:0 0 0 2024-02-16T19:28:45.168 INFO:tasks.workunit.client.1.smithi117.stdout:5/135: symlink d2/d3/l28 0 2024-02-16T19:28:45.170 INFO:tasks.workunit.client.1.smithi117.stdout:5/136: mknod d2/d9/de/d17/c29 0 2024-02-16T19:28:45.172 INFO:tasks.workunit.client.1.smithi117.stdout:5/137: mkdir d2/d3/d21/d2a 0 2024-02-16T19:28:45.174 INFO:tasks.workunit.client.1.smithi117.stdout:5/138: symlink d2/d3/l2b 0 2024-02-16T19:28:45.177 INFO:tasks.workunit.client.1.smithi117.stdout:5/139: creat d2/d3/d21/d23/f2c x:0 0 0 2024-02-16T19:28:45.179 INFO:tasks.workunit.client.1.smithi117.stdout:5/140: unlink d2/d1d/f27 0 2024-02-16T19:28:45.187 INFO:tasks.workunit.client.1.smithi117.stdout:5/141: dwrite d2/d4/d16/d24/f26 [0,4194304] 0 2024-02-16T19:28:45.190 INFO:tasks.workunit.client.1.smithi117.stdout:5/142: unlink d2/f14 0 2024-02-16T19:28:45.196 INFO:tasks.workunit.client.1.smithi117.stdout:5/143: mkdir d2/d9/de/d17/d1c/d2d 0 2024-02-16T19:28:45.196 INFO:tasks.workunit.client.1.smithi117.stdout:5/144: fsync d2/d4/d16/f1a 0 2024-02-16T19:28:45.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:44 smithi022 ceph-mon[29221]: Standby manager daemon smithi117.cohbtv restarted 2024-02-16T19:28:45.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:44 smithi022 ceph-mon[29221]: Standby manager daemon smithi117.cohbtv started 2024-02-16T19:28:45.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:44 smithi022 ceph-mon[29221]: from='mgr.? 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi117.cohbtv/crt"}]: dispatch 2024-02-16T19:28:45.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:44 smithi022 ceph-mon[29221]: from='mgr.? 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-16T19:28:45.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:44 smithi022 ceph-mon[29221]: from='mgr.? 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi117.cohbtv/key"}]: dispatch 2024-02-16T19:28:45.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:44 smithi022 ceph-mon[29221]: from='mgr.? 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-16T19:28:45.233 INFO:tasks.workunit.client.1.smithi117.stdout:5/145: dwrite d2/d4/d16/d24/f26 [4194304,4194304] 0 2024-02-16T19:28:45.234 INFO:tasks.workunit.client.1.smithi117.stdout:5/146: chown d2/d4/f1e 885162057 1 2024-02-16T19:28:45.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:44 smithi117 ceph-mon[35309]: Standby manager daemon smithi117.cohbtv restarted 2024-02-16T19:28:45.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:44 smithi117 ceph-mon[35309]: Standby manager daemon smithi117.cohbtv started 2024-02-16T19:28:45.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:44 smithi117 ceph-mon[35309]: from='mgr.? 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi117.cohbtv/crt"}]: dispatch 2024-02-16T19:28:45.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:44 smithi117 ceph-mon[35309]: from='mgr.? 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-16T19:28:45.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:44 smithi117 ceph-mon[35309]: from='mgr.? 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi117.cohbtv/key"}]: dispatch 2024-02-16T19:28:45.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:44 smithi117 ceph-mon[35309]: from='mgr.? 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-16T19:28:46.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:45 smithi022 ceph-mon[29221]: pgmap v254: 65 pgs: 65 active+clean; 196 MiB data, 802 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 3.8 MiB/s wr, 283 op/s 2024-02-16T19:28:46.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:45 smithi022 ceph-mon[29221]: mgrmap e20: smithi022.gmdcch(active, since 7m), standbys: smithi117.cohbtv 2024-02-16T19:28:46.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:45 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:28:46.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:45 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:28:46.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:45 smithi117 ceph-mon[35309]: pgmap v254: 65 pgs: 65 active+clean; 196 MiB data, 802 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 3.8 MiB/s wr, 283 op/s 2024-02-16T19:28:46.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:45 smithi117 ceph-mon[35309]: mgrmap e20: smithi022.gmdcch(active, since 7m), standbys: smithi117.cohbtv 2024-02-16T19:28:46.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:45 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:28:46.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:45 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:28:47.464 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:47 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:28:47.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:47 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:28:47.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:47 smithi022 ceph-mon[29221]: pgmap v255: 65 pgs: 65 active+clean; 271 MiB data, 956 MiB used, 535 GiB / 536 GiB avail; 121 KiB/s rd, 13 MiB/s wr, 371 op/s 2024-02-16T19:28:47.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:47 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:28:47.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:47 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:28:47.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:47 smithi117 ceph-mon[35309]: pgmap v255: 65 pgs: 65 active+clean; 271 MiB data, 956 MiB used, 535 GiB / 536 GiB avail; 121 KiB/s rd, 13 MiB/s wr, 371 op/s 2024-02-16T19:28:48.027 INFO:tasks.workunit.client.0.smithi022.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-02-16T19:28:48.031 INFO:tasks.workunit.client.0.smithi022.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-02-16T19:28:48.031 INFO:tasks.workunit.client.0.smithi022.stderr:+ make 2024-02-16T19:28:48.524 INFO:tasks.workunit.client.0.smithi022.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-02-16T19:28:49.669 INFO:tasks.workunit.client.0.smithi022.stderr:++ readlink -f fsstress 2024-02-16T19:28:49.670 INFO:tasks.workunit.client.0.smithi022.stderr:+ BIN=/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-02-16T19:28:49.670 INFO:tasks.workunit.client.0.smithi022.stderr:+ popd 2024-02-16T19:28:49.671 INFO:tasks.workunit.client.0.smithi022.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-02-16T19:28:49.671 INFO:tasks.workunit.client.0.smithi022.stderr:+ popd 2024-02-16T19:28:49.672 INFO:tasks.workunit.client.0.smithi022.stdout:~/cephtest/mnt.0/client.0/tmp 2024-02-16T19:28:49.672 INFO:tasks.workunit.client.0.smithi022.stderr:++ mktemp -d -p . 2024-02-16T19:28:49.674 INFO:tasks.workunit.client.0.smithi022.stderr:+ T=./tmp.6udblTWqFb 2024-02-16T19:28:49.674 INFO:tasks.workunit.client.0.smithi022.stderr:+ /home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.6udblTWqFb -l 1 -n 1000 -p 10 -v 2024-02-16T19:28:49.677 INFO:tasks.workunit.client.0.smithi022.stdout:seed = 1707466803 2024-02-16T19:28:49.680 INFO:tasks.workunit.client.0.smithi022.stdout:0/0: link - no file 2024-02-16T19:28:49.680 INFO:tasks.workunit.client.0.smithi022.stdout:0/1: dread - no filename 2024-02-16T19:28:49.680 INFO:tasks.workunit.client.0.smithi022.stdout:0/2: dread - no filename 2024-02-16T19:28:49.681 INFO:tasks.workunit.client.0.smithi022.stdout:1/0: chown . 831093 1 2024-02-16T19:28:49.681 INFO:tasks.workunit.client.0.smithi022.stdout:1/1: readlink - no filename 2024-02-16T19:28:49.681 INFO:tasks.workunit.client.0.smithi022.stdout:1/2: truncate - no filename 2024-02-16T19:28:49.681 INFO:tasks.workunit.client.0.smithi022.stdout:1/3: rmdir - no directory 2024-02-16T19:28:49.681 INFO:tasks.workunit.client.0.smithi022.stdout:1/4: write - no filename 2024-02-16T19:28:49.681 INFO:tasks.workunit.client.0.smithi022.stdout:1/5: dread - no filename 2024-02-16T19:28:49.682 INFO:tasks.workunit.client.0.smithi022.stdout:1/6: fsync - no filename 2024-02-16T19:28:49.682 INFO:tasks.workunit.client.0.smithi022.stdout:1/7: rename - no filename 2024-02-16T19:28:49.682 INFO:tasks.workunit.client.0.smithi022.stdout:1/8: fsync - no filename 2024-02-16T19:28:49.682 INFO:tasks.workunit.client.0.smithi022.stdout:1/9: chown . 143 1 2024-02-16T19:28:49.682 INFO:tasks.workunit.client.0.smithi022.stdout:0/3: mknod c0 0 2024-02-16T19:28:49.682 INFO:tasks.workunit.client.0.smithi022.stdout:0/4: readlink - no filename 2024-02-16T19:28:49.683 INFO:tasks.workunit.client.0.smithi022.stdout:0/5: rename c0 to c1 0 2024-02-16T19:28:49.683 INFO:tasks.workunit.client.0.smithi022.stdout:0/6: truncate - no filename 2024-02-16T19:28:49.683 INFO:tasks.workunit.client.0.smithi022.stdout:0/7: fdatasync - no filename 2024-02-16T19:28:49.684 INFO:tasks.workunit.client.0.smithi022.stdout:3/0: rmdir - no directory 2024-02-16T19:28:49.685 INFO:tasks.workunit.client.0.smithi022.stdout:0/8: creat f2 x:0 0 0 2024-02-16T19:28:49.685 INFO:tasks.workunit.client.0.smithi022.stdout:2/0: mkdir d0 0 2024-02-16T19:28:49.685 INFO:tasks.workunit.client.0.smithi022.stdout:2/1: readlink - no filename 2024-02-16T19:28:49.686 INFO:tasks.workunit.client.0.smithi022.stdout:2/2: write - no filename 2024-02-16T19:28:49.687 INFO:tasks.workunit.client.0.smithi022.stdout:4/0: dwrite - no filename 2024-02-16T19:28:49.688 INFO:tasks.workunit.client.0.smithi022.stdout:0/9: unlink f2 0 2024-02-16T19:28:49.688 INFO:tasks.workunit.client.0.smithi022.stdout:0/10: truncate - no filename 2024-02-16T19:28:49.688 INFO:tasks.workunit.client.0.smithi022.stdout:0/11: dwrite - no filename 2024-02-16T19:28:49.688 INFO:tasks.workunit.client.0.smithi022.stdout:3/1: creat f0 x:0 0 0 2024-02-16T19:28:49.691 INFO:tasks.workunit.client.0.smithi022.stdout:3/2: symlink l1 0 2024-02-16T19:28:49.692 INFO:tasks.workunit.client.0.smithi022.stdout:6/0: read - no filename 2024-02-16T19:28:49.692 INFO:tasks.workunit.client.0.smithi022.stdout:6/1: write - no filename 2024-02-16T19:28:49.693 INFO:tasks.workunit.client.0.smithi022.stdout:4/1: creat f0 x:0 0 0 2024-02-16T19:28:49.695 INFO:tasks.workunit.client.0.smithi022.stdout:3/3: creat f2 x:0 0 0 2024-02-16T19:28:49.699 INFO:tasks.workunit.client.0.smithi022.stdout:5/0: symlink l0 0 2024-02-16T19:28:49.699 INFO:tasks.workunit.client.0.smithi022.stdout:5/1: truncate - no filename 2024-02-16T19:28:49.699 INFO:tasks.workunit.client.0.smithi022.stdout:5/2: dread - no filename 2024-02-16T19:28:49.699 INFO:tasks.workunit.client.0.smithi022.stdout:5/3: rmdir - no directory 2024-02-16T19:28:49.701 INFO:tasks.workunit.client.0.smithi022.stdout:3/4: rename l1 to l3 0 2024-02-16T19:28:49.701 INFO:tasks.workunit.client.0.smithi022.stdout:4/2: dwrite f0 [0,4194304] 0 2024-02-16T19:28:49.702 INFO:tasks.workunit.client.0.smithi022.stdout:4/3: chown f0 36 1 2024-02-16T19:28:49.703 INFO:tasks.workunit.client.0.smithi022.stdout:6/2: creat f0 x:0 0 0 2024-02-16T19:28:49.704 INFO:tasks.workunit.client.0.smithi022.stdout:6/3: write f0 [986533,38603] 0 2024-02-16T19:28:49.706 INFO:tasks.workunit.client.0.smithi022.stdout:6/4: write f0 [1112578,52412] 0 2024-02-16T19:28:49.715 INFO:tasks.workunit.client.0.smithi022.stdout:5/4: creat f1 x:0 0 0 2024-02-16T19:28:49.715 INFO:tasks.workunit.client.0.smithi022.stdout:6/5: dwrite f0 [0,4194304] 0 2024-02-16T19:28:49.716 INFO:tasks.workunit.client.0.smithi022.stdout:3/5: creat f4 x:0 0 0 2024-02-16T19:28:49.720 INFO:tasks.workunit.client.0.smithi022.stdout:4/4: rename f0 to f1 0 2024-02-16T19:28:49.720 INFO:tasks.workunit.client.0.smithi022.stdout:4/5: chown f1 3 1 2024-02-16T19:28:49.725 INFO:tasks.workunit.client.0.smithi022.stdout:9/0: chown . 1 1 2024-02-16T19:28:49.725 INFO:tasks.workunit.client.0.smithi022.stdout:9/1: chown . 14818652 1 2024-02-16T19:28:49.725 INFO:tasks.workunit.client.0.smithi022.stdout:7/0: symlink l0 0 2024-02-16T19:28:49.725 INFO:tasks.workunit.client.0.smithi022.stdout:7/1: dread - no filename 2024-02-16T19:28:49.726 INFO:tasks.workunit.client.0.smithi022.stdout:7/2: write - no filename 2024-02-16T19:28:49.726 INFO:tasks.workunit.client.0.smithi022.stdout:7/3: dwrite - no filename 2024-02-16T19:28:49.728 INFO:tasks.workunit.client.0.smithi022.stdout:3/6: rename f4 to f5 0 2024-02-16T19:28:49.729 INFO:tasks.workunit.client.0.smithi022.stdout:4/6: creat f2 x:0 0 0 2024-02-16T19:28:49.730 INFO:tasks.workunit.client.0.smithi022.stdout:8/0: mknod c0 0 2024-02-16T19:28:49.730 INFO:tasks.workunit.client.0.smithi022.stdout:6/6: fdatasync f0 0 2024-02-16T19:28:49.731 INFO:tasks.workunit.client.0.smithi022.stdout:6/7: chown f0 1 1 2024-02-16T19:28:49.731 INFO:tasks.workunit.client.0.smithi022.stdout:7/4: creat f1 x:0 0 0 2024-02-16T19:28:49.732 INFO:tasks.workunit.client.0.smithi022.stdout:7/5: chown l0 233280193 1 2024-02-16T19:28:49.732 INFO:tasks.workunit.client.0.smithi022.stdout:7/6: chown f1 8 1 2024-02-16T19:28:49.733 INFO:tasks.workunit.client.0.smithi022.stdout:7/7: write f1 [909606,119565] 0 2024-02-16T19:28:49.733 INFO:tasks.workunit.client.0.smithi022.stdout:3/7: symlink l6 0 2024-02-16T19:28:49.735 INFO:tasks.workunit.client.0.smithi022.stdout:4/7: symlink l3 0 2024-02-16T19:28:49.736 INFO:tasks.workunit.client.0.smithi022.stdout:8/1: rename c0 to c1 0 2024-02-16T19:28:49.736 INFO:tasks.workunit.client.0.smithi022.stdout:8/2: write - no filename 2024-02-16T19:28:49.736 INFO:tasks.workunit.client.0.smithi022.stdout:8/3: dread - no filename 2024-02-16T19:28:49.737 INFO:tasks.workunit.client.0.smithi022.stdout:6/8: mkdir d1 0 2024-02-16T19:28:49.739 INFO:tasks.workunit.client.0.smithi022.stdout:7/8: mknod c2 0 2024-02-16T19:28:49.740 INFO:tasks.workunit.client.0.smithi022.stdout:3/8: unlink f0 0 2024-02-16T19:28:49.744 INFO:tasks.workunit.client.0.smithi022.stdout:7/9: rename f1 to f3 0 2024-02-16T19:28:49.744 INFO:tasks.workunit.client.0.smithi022.stdout:7/10: readlink l0 0 2024-02-16T19:28:49.745 INFO:tasks.workunit.client.0.smithi022.stdout:3/9: mkdir d7 0 2024-02-16T19:28:49.746 INFO:tasks.workunit.client.0.smithi022.stdout:3/10: truncate f2 29548 0 2024-02-16T19:28:49.746 INFO:tasks.workunit.client.0.smithi022.stdout:3/11: fdatasync f2 0 2024-02-16T19:28:49.747 INFO:tasks.workunit.client.0.smithi022.stdout:6/9: creat d1/f2 x:0 0 0 2024-02-16T19:28:49.747 INFO:tasks.workunit.client.0.smithi022.stdout:3/12: truncate f2 59329 0 2024-02-16T19:28:49.748 INFO:tasks.workunit.client.0.smithi022.stdout:3/13: chown d7 161 1 2024-02-16T19:28:49.748 INFO:tasks.workunit.client.0.smithi022.stdout:3/14: truncate f2 966131 0 2024-02-16T19:28:49.752 INFO:tasks.workunit.client.0.smithi022.stdout:6/10: creat d1/f3 x:0 0 0 2024-02-16T19:28:49.753 INFO:tasks.workunit.client.0.smithi022.stdout:3/15: write f5 [64023,79855] 0 2024-02-16T19:28:49.754 INFO:tasks.workunit.client.0.smithi022.stdout:7/11: link f3 f4 0 2024-02-16T19:28:49.754 INFO:tasks.workunit.client.0.smithi022.stdout:7/12: stat l0 0 2024-02-16T19:28:49.755 INFO:tasks.workunit.client.0.smithi022.stdout:7/13: chown f4 1477 1 2024-02-16T19:28:49.756 INFO:tasks.workunit.client.0.smithi022.stdout:6/11: symlink d1/l4 0 2024-02-16T19:28:49.759 INFO:tasks.workunit.client.0.smithi022.stdout:6/12: symlink d1/l5 0 2024-02-16T19:28:49.760 INFO:tasks.workunit.client.0.smithi022.stdout:6/13: chown d1/f2 2 1 2024-02-16T19:28:49.761 INFO:tasks.workunit.client.0.smithi022.stdout:3/16: dwrite f5 [0,4194304] 0 2024-02-16T19:28:49.766 INFO:tasks.workunit.client.0.smithi022.stdout:3/17: symlink d7/l8 0 2024-02-16T19:28:49.766 INFO:tasks.workunit.client.0.smithi022.stdout:3/18: mknod d7/c9 0 2024-02-16T19:28:49.768 INFO:tasks.workunit.client.0.smithi022.stdout:3/19: mknod d7/ca 0 2024-02-16T19:28:49.775 INFO:tasks.workunit.client.0.smithi022.stdout:3/20: dwrite f5 [0,4194304] 0 2024-02-16T19:28:49.778 INFO:tasks.workunit.client.0.smithi022.stdout:3/21: mknod d7/cb 0 2024-02-16T19:28:49.779 INFO:tasks.workunit.client.0.smithi022.stdout:3/22: mkdir d7/dc 0 2024-02-16T19:28:49.782 INFO:tasks.workunit.client.0.smithi022.stdout:3/23: mkdir d7/dc/dd 0 2024-02-16T19:28:49.782 INFO:tasks.workunit.client.0.smithi022.stdout:3/24: read f5 [701640,27493] 0 2024-02-16T19:28:49.783 INFO:tasks.workunit.client.0.smithi022.stdout:3/25: write f2 [1973599,111] 0 2024-02-16T19:28:49.784 INFO:tasks.workunit.client.0.smithi022.stdout:3/26: write f2 [395981,72372] 0 2024-02-16T19:28:49.788 INFO:tasks.workunit.client.0.smithi022.stdout:3/27: creat d7/dc/dd/fe x:0 0 0 2024-02-16T19:28:49.789 INFO:tasks.workunit.client.0.smithi022.stdout:3/28: creat d7/dc/dd/ff x:0 0 0 2024-02-16T19:28:49.791 INFO:tasks.workunit.client.0.smithi022.stdout:3/29: symlink d7/dc/l10 0 2024-02-16T19:28:49.793 INFO:tasks.workunit.client.0.smithi022.stdout:3/30: rename l6 to d7/l11 0 2024-02-16T19:28:49.794 INFO:tasks.workunit.client.0.smithi022.stdout:3/31: dread - d7/dc/dd/ff zero size 2024-02-16T19:28:49.795 INFO:tasks.workunit.client.0.smithi022.stdout:3/32: unlink d7/c9 0 2024-02-16T19:28:49.797 INFO:tasks.workunit.client.0.smithi022.stdout:3/33: mkdir d7/d12 0 2024-02-16T19:28:49.798 INFO:tasks.workunit.client.0.smithi022.stdout:3/34: creat d7/dc/dd/f13 x:0 0 0 2024-02-16T19:28:49.801 INFO:tasks.workunit.client.0.smithi022.stdout:3/35: rename d7/dc/dd/fe to d7/d12/f14 0 2024-02-16T19:28:49.808 INFO:tasks.workunit.client.0.smithi022.stdout:0/12: rename c1 to c3 0 2024-02-16T19:28:49.808 INFO:tasks.workunit.client.0.smithi022.stdout:0/13: dread - no filename 2024-02-16T19:28:49.808 INFO:tasks.workunit.client.0.smithi022.stdout:0/14: write - no filename 2024-02-16T19:28:49.817 INFO:tasks.workunit.client.0.smithi022.stdout:5/5: getdents . 0 2024-02-16T19:28:49.818 INFO:tasks.workunit.client.0.smithi022.stdout:5/6: symlink l2 0 2024-02-16T19:28:49.824 INFO:tasks.workunit.client.0.smithi022.stdout:4/8: getdents . 0 2024-02-16T19:28:49.825 INFO:tasks.workunit.client.0.smithi022.stdout:5/7: dwrite f1 [0,4194304] 0 2024-02-16T19:28:49.827 INFO:tasks.workunit.client.0.smithi022.stdout:4/9: mkdir d4 0 2024-02-16T19:28:49.828 INFO:tasks.workunit.client.0.smithi022.stdout:8/4: stat c1 0 2024-02-16T19:28:49.833 INFO:tasks.workunit.client.0.smithi022.stdout:8/5: creat f2 x:0 0 0 2024-02-16T19:28:49.834 INFO:tasks.workunit.client.0.smithi022.stdout:8/6: truncate f2 468513 0 2024-02-16T19:28:49.834 INFO:tasks.workunit.client.0.smithi022.stdout:7/14: rename f3 to f5 0 2024-02-16T19:28:49.837 INFO:tasks.workunit.client.0.smithi022.stdout:4/10: dwrite f2 [0,4194304] 0 2024-02-16T19:28:49.837 INFO:tasks.workunit.client.0.smithi022.stdout:8/7: mkdir d3 0 2024-02-16T19:28:49.838 INFO:tasks.workunit.client.0.smithi022.stdout:8/8: dread f2 [0,4194304] 0 2024-02-16T19:28:49.839 INFO:tasks.workunit.client.0.smithi022.stdout:5/8: fdatasync f1 0 2024-02-16T19:28:49.840 INFO:tasks.workunit.client.0.smithi022.stdout:4/11: fdatasync f1 0 2024-02-16T19:28:49.843 INFO:tasks.workunit.client.0.smithi022.stdout:5/9: creat f3 x:0 0 0 2024-02-16T19:28:49.845 INFO:tasks.workunit.client.0.smithi022.stdout:4/12: unlink l3 0 2024-02-16T19:28:49.846 INFO:tasks.workunit.client.0.smithi022.stdout:4/13: write f1 [746248,32235] 0 2024-02-16T19:28:49.847 INFO:tasks.workunit.client.0.smithi022.stdout:8/9: unlink f2 0 2024-02-16T19:28:49.848 INFO:tasks.workunit.client.0.smithi022.stdout:8/10: readlink - no filename 2024-02-16T19:28:49.848 INFO:tasks.workunit.client.0.smithi022.stdout:8/11: dwrite - no filename 2024-02-16T19:28:49.849 INFO:tasks.workunit.client.0.smithi022.stdout:6/14: getdents d1 0 2024-02-16T19:28:49.850 INFO:tasks.workunit.client.0.smithi022.stdout:5/10: creat f4 x:0 0 0 2024-02-16T19:28:49.851 INFO:tasks.workunit.client.0.smithi022.stdout:5/11: dread - f4 zero size 2024-02-16T19:28:49.853 INFO:tasks.workunit.client.0.smithi022.stdout:4/14: symlink d4/l5 0 2024-02-16T19:28:49.854 INFO:tasks.workunit.client.0.smithi022.stdout:5/12: write f4 [567336,107246] 0 2024-02-16T19:28:49.855 INFO:tasks.workunit.client.0.smithi022.stdout:8/12: getdents d3 0 2024-02-16T19:28:49.856 INFO:tasks.workunit.client.0.smithi022.stdout:5/13: readlink l0 0 2024-02-16T19:28:49.857 INFO:tasks.workunit.client.0.smithi022.stdout:4/15: mkdir d4/d6 0 2024-02-16T19:28:49.859 INFO:tasks.workunit.client.0.smithi022.stdout:6/15: dwrite d1/f3 [0,4194304] 0 2024-02-16T19:28:49.860 INFO:tasks.workunit.client.0.smithi022.stdout:8/13: mknod d3/c4 0 2024-02-16T19:28:49.861 INFO:tasks.workunit.client.0.smithi022.stdout:8/14: rename d3 to d3/d5 22 2024-02-16T19:28:49.862 INFO:tasks.workunit.client.0.smithi022.stdout:8/15: dread - no filename 2024-02-16T19:28:49.862 INFO:tasks.workunit.client.0.smithi022.stdout:8/16: read - no filename 2024-02-16T19:28:49.862 INFO:tasks.workunit.client.0.smithi022.stdout:8/17: dwrite - no filename 2024-02-16T19:28:49.862 INFO:tasks.workunit.client.0.smithi022.stdout:4/16: mknod d4/c7 0 2024-02-16T19:28:49.865 INFO:tasks.workunit.client.0.smithi022.stdout:3/36: readlink d7/l11 0 2024-02-16T19:28:49.865 INFO:tasks.workunit.client.0.smithi022.stdout:8/18: mkdir d3/d6 0 2024-02-16T19:28:49.865 INFO:tasks.workunit.client.0.smithi022.stdout:8/19: readlink - no filename 2024-02-16T19:28:49.866 INFO:tasks.workunit.client.0.smithi022.stdout:5/14: dwrite f1 [0,4194304] 0 2024-02-16T19:28:49.868 INFO:tasks.workunit.client.0.smithi022.stdout:5/15: chown f1 31 1 2024-02-16T19:28:49.868 INFO:tasks.workunit.client.0.smithi022.stdout:5/16: chown l0 439663 1 2024-02-16T19:28:49.869 INFO:tasks.workunit.client.0.smithi022.stdout:6/16: dread f0 [0,4194304] 0 2024-02-16T19:28:49.869 INFO:tasks.workunit.client.0.smithi022.stdout:4/17: rename d4/l5 to d4/d6/l8 0 2024-02-16T19:28:49.871 INFO:tasks.workunit.client.0.smithi022.stdout:6/17: write d1/f2 [717440,31811] 0 2024-02-16T19:28:49.875 INFO:tasks.workunit.client.0.smithi022.stdout:0/15: chown c3 7 1 2024-02-16T19:28:49.875 INFO:tasks.workunit.client.0.smithi022.stdout:0/16: write - no filename 2024-02-16T19:28:49.877 INFO:tasks.workunit.client.0.smithi022.stdout:4/18: creat d4/d6/f9 x:0 0 0 2024-02-16T19:28:49.879 INFO:tasks.workunit.client.0.smithi022.stdout:8/20: creat d3/d6/f7 x:0 0 0 2024-02-16T19:28:49.880 INFO:tasks.workunit.client.0.smithi022.stdout:5/17: dwrite f1 [0,4194304] 0 2024-02-16T19:28:49.881 INFO:tasks.workunit.client.0.smithi022.stdout:0/17: mknod c4 0 2024-02-16T19:28:49.881 INFO:tasks.workunit.client.0.smithi022.stdout:6/18: fdatasync d1/f3 0 2024-02-16T19:28:49.881 INFO:tasks.workunit.client.0.smithi022.stdout:5/18: stat f4 0 2024-02-16T19:28:49.882 INFO:tasks.workunit.client.0.smithi022.stdout:6/19: rename d1 to d1/d6 22 2024-02-16T19:28:49.883 INFO:tasks.workunit.client.0.smithi022.stdout:4/19: symlink d4/d6/la 0 2024-02-16T19:28:49.886 INFO:tasks.workunit.client.0.smithi022.stdout:0/18: creat f5 x:0 0 0 2024-02-16T19:28:49.886 INFO:tasks.workunit.client.0.smithi022.stdout:0/19: rmdir - no directory 2024-02-16T19:28:49.892 INFO:tasks.workunit.client.0.smithi022.stdout:0/20: symlink l6 0 2024-02-16T19:28:49.895 INFO:tasks.workunit.client.0.smithi022.stdout:6/20: dread d1/f2 [0,4194304] 0 2024-02-16T19:28:49.898 INFO:tasks.workunit.client.0.smithi022.stdout:7/15: dwrite f5 [0,4194304] 0 2024-02-16T19:28:49.899 INFO:tasks.workunit.client.0.smithi022.stdout:0/21: chown c4 16231 1 2024-02-16T19:28:49.906 INFO:tasks.workunit.client.0.smithi022.stdout:0/22: dread - f5 zero size 2024-02-16T19:28:49.907 INFO:tasks.workunit.client.0.smithi022.stdout:5/19: dwrite f3 [0,4194304] 0 2024-02-16T19:28:49.908 INFO:tasks.workunit.client.0.smithi022.stdout:8/21: dwrite d3/d6/f7 [0,4194304] 0 2024-02-16T19:28:49.908 INFO:tasks.workunit.client.0.smithi022.stdout:4/20: dwrite f1 [0,4194304] 0 2024-02-16T19:28:49.910 INFO:tasks.workunit.client.0.smithi022.stdout:8/22: stat d3 0 2024-02-16T19:28:49.910 INFO:tasks.workunit.client.0.smithi022.stdout:8/23: stat d3 0 2024-02-16T19:28:49.910 INFO:tasks.workunit.client.0.smithi022.stdout:5/20: truncate f4 1613587 0 2024-02-16T19:28:49.910 INFO:tasks.workunit.client.0.smithi022.stdout:5/21: rmdir - no directory 2024-02-16T19:28:49.913 INFO:tasks.workunit.client.0.smithi022.stdout:0/23: rename f5 to f7 0 2024-02-16T19:28:49.913 INFO:tasks.workunit.client.0.smithi022.stdout:4/21: write d4/d6/f9 [685870,111889] 0 2024-02-16T19:28:49.914 INFO:tasks.workunit.client.0.smithi022.stdout:5/22: creat f5 x:0 0 0 2024-02-16T19:28:49.915 INFO:tasks.workunit.client.0.smithi022.stdout:6/21: dread f0 [0,4194304] 0 2024-02-16T19:28:49.917 INFO:tasks.workunit.client.0.smithi022.stdout:8/24: mknod d3/c8 0 2024-02-16T19:28:49.917 INFO:tasks.workunit.client.0.smithi022.stdout:5/23: fdatasync f4 0 2024-02-16T19:28:49.920 INFO:tasks.workunit.client.0.smithi022.stdout:4/22: unlink f2 0 2024-02-16T19:28:49.922 INFO:tasks.workunit.client.0.smithi022.stdout:6/22: creat d1/f7 x:0 0 0 2024-02-16T19:28:49.923 INFO:tasks.workunit.client.0.smithi022.stdout:5/24: creat f6 x:0 0 0 2024-02-16T19:28:49.926 INFO:tasks.workunit.client.0.smithi022.stdout:4/23: creat d4/d6/fb x:0 0 0 2024-02-16T19:28:49.927 INFO:tasks.workunit.client.0.smithi022.stdout:6/23: dread d1/f2 [0,4194304] 0 2024-02-16T19:28:49.928 INFO:tasks.workunit.client.0.smithi022.stdout:8/25: dread d3/d6/f7 [0,4194304] 0 2024-02-16T19:28:49.929 INFO:tasks.workunit.client.0.smithi022.stdout:4/24: unlink d4/c7 0 2024-02-16T19:28:49.930 INFO:tasks.workunit.client.0.smithi022.stdout:0/24: dwrite f7 [0,4194304] 0 2024-02-16T19:28:49.931 INFO:tasks.workunit.client.0.smithi022.stdout:8/26: read d3/d6/f7 [3690658,16051] 0 2024-02-16T19:28:49.931 INFO:tasks.workunit.client.0.smithi022.stdout:0/25: readlink l6 0 2024-02-16T19:28:49.934 INFO:tasks.workunit.client.0.smithi022.stdout:6/24: dread f0 [0,4194304] 0 2024-02-16T19:28:49.934 INFO:tasks.workunit.client.0.smithi022.stdout:4/25: unlink d4/d6/la 0 2024-02-16T19:28:49.935 INFO:tasks.workunit.client.0.smithi022.stdout:0/26: symlink l8 0 2024-02-16T19:28:49.936 INFO:tasks.workunit.client.0.smithi022.stdout:8/27: dread d3/d6/f7 [0,4194304] 0 2024-02-16T19:28:49.937 INFO:tasks.workunit.client.0.smithi022.stdout:4/26: mknod d4/cc 0 2024-02-16T19:28:49.939 INFO:tasks.workunit.client.0.smithi022.stdout:6/25: dread d1/f3 [0,4194304] 0 2024-02-16T19:28:49.939 INFO:tasks.workunit.client.0.smithi022.stdout:8/28: symlink d3/d6/l9 0 2024-02-16T19:28:49.940 INFO:tasks.workunit.client.0.smithi022.stdout:8/29: readlink d3/d6/l9 0 2024-02-16T19:28:49.942 INFO:tasks.workunit.client.0.smithi022.stdout:6/26: write d1/f2 [92460,83218] 0 2024-02-16T19:28:49.943 INFO:tasks.workunit.client.0.smithi022.stdout:5/25: truncate f1 1897506 0 2024-02-16T19:28:49.946 INFO:tasks.workunit.client.0.smithi022.stdout:8/30: unlink d3/c4 0 2024-02-16T19:28:49.948 INFO:tasks.workunit.client.0.smithi022.stdout:8/31: chown d3/d6/l9 3274 1 2024-02-16T19:28:49.948 INFO:tasks.workunit.client.0.smithi022.stdout:6/27: mkdir d1/d8 0 2024-02-16T19:28:49.950 INFO:tasks.workunit.client.0.smithi022.stdout:0/27: getdents . 0 2024-02-16T19:28:49.951 INFO:tasks.workunit.client.0.smithi022.stdout:5/26: dwrite f6 [0,4194304] 0 2024-02-16T19:28:49.954 INFO:tasks.workunit.client.0.smithi022.stdout:0/28: symlink l9 0 2024-02-16T19:28:49.955 INFO:tasks.workunit.client.0.smithi022.stdout:0/29: read f7 [3645575,93745] 0 2024-02-16T19:28:49.961 INFO:tasks.workunit.client.0.smithi022.stdout:6/28: dwrite f0 [4194304,4194304] 0 2024-02-16T19:28:49.962 INFO:tasks.workunit.client.0.smithi022.stdout:5/27: dread f3 [0,4194304] 0 2024-02-16T19:28:49.963 INFO:tasks.workunit.client.0.smithi022.stdout:0/30: mknod ca 0 2024-02-16T19:28:49.965 INFO:tasks.workunit.client.0.smithi022.stdout:6/29: mknod d1/c9 0 2024-02-16T19:28:49.965 INFO:tasks.workunit.client.0.smithi022.stdout:5/28: mknod c7 0 2024-02-16T19:28:49.966 INFO:tasks.workunit.client.0.smithi022.stdout:5/29: truncate f5 147412 0 2024-02-16T19:28:49.967 INFO:tasks.workunit.client.0.smithi022.stdout:5/30: write f6 [3019069,94253] 0 2024-02-16T19:28:49.967 INFO:tasks.workunit.client.0.smithi022.stdout:0/31: creat fb x:0 0 0 2024-02-16T19:28:49.968 INFO:tasks.workunit.client.0.smithi022.stdout:0/32: readlink l9 0 2024-02-16T19:28:49.971 INFO:tasks.workunit.client.0.smithi022.stdout:0/33: rename l9 to lc 0 2024-02-16T19:28:49.973 INFO:tasks.workunit.client.0.smithi022.stdout:5/31: rename f1 to f8 0 2024-02-16T19:28:49.974 INFO:tasks.workunit.client.0.smithi022.stdout:5/32: write f4 [1610858,39008] 0 2024-02-16T19:28:49.979 INFO:tasks.workunit.client.0.smithi022.stdout:0/34: write f7 [4885448,104155] 0 2024-02-16T19:28:49.980 INFO:tasks.workunit.client.0.smithi022.stdout:0/35: readlink l8 0 2024-02-16T19:28:49.982 INFO:tasks.workunit.client.0.smithi022.stdout:0/36: fsync f7 0 2024-02-16T19:28:49.982 INFO:tasks.workunit.client.0.smithi022.stdout:5/33: dwrite f3 [0,4194304] 0 2024-02-16T19:28:49.983 INFO:tasks.workunit.client.0.smithi022.stdout:0/37: write f7 [564771,95926] 0 2024-02-16T19:28:49.984 INFO:tasks.workunit.client.0.smithi022.stdout:0/38: fdatasync fb 0 2024-02-16T19:28:49.987 INFO:tasks.workunit.client.0.smithi022.stdout:0/39: mknod cd 0 2024-02-16T19:28:49.991 INFO:tasks.workunit.client.0.smithi022.stdout:5/34: dwrite f4 [0,4194304] 0 2024-02-16T19:28:49.991 INFO:tasks.workunit.client.0.smithi022.stdout:0/40: rename lc to le 0 2024-02-16T19:28:49.992 INFO:tasks.workunit.client.0.smithi022.stdout:5/35: fsync f6 0 2024-02-16T19:28:49.993 INFO:tasks.workunit.client.0.smithi022.stdout:0/41: symlink lf 0 2024-02-16T19:28:49.994 INFO:tasks.workunit.client.0.smithi022.stdout:5/36: creat f9 x:0 0 0 2024-02-16T19:28:49.996 INFO:tasks.workunit.client.0.smithi022.stdout:0/42: unlink c3 0 2024-02-16T19:28:49.997 INFO:tasks.workunit.client.0.smithi022.stdout:5/37: symlink la 0 2024-02-16T19:28:49.998 INFO:tasks.workunit.client.0.smithi022.stdout:0/43: mkdir d10 0 2024-02-16T19:28:49.999 INFO:tasks.workunit.client.0.smithi022.stdout:5/38: creat fb x:0 0 0 2024-02-16T19:28:50.000 INFO:tasks.workunit.client.0.smithi022.stdout:5/39: creat fc x:0 0 0 2024-02-16T19:28:50.012 INFO:tasks.workunit.client.0.smithi022.stdout:5/40: write f8 [1747729,68449] 0 2024-02-16T19:28:50.012 INFO:tasks.workunit.client.0.smithi022.stdout:5/41: stat f9 0 2024-02-16T19:28:50.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:49 smithi022 ceph-mon[29221]: pgmap v256: 65 pgs: 65 active+clean; 295 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 209 KiB/s rd, 14 MiB/s wr, 308 op/s 2024-02-16T19:28:50.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:49 smithi117 ceph-mon[35309]: pgmap v256: 65 pgs: 65 active+clean; 295 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 209 KiB/s rd, 14 MiB/s wr, 308 op/s 2024-02-16T19:28:50.963 INFO:tasks.workunit.client.0.smithi022.stdout:8/32: sync 2024-02-16T19:28:50.963 INFO:tasks.workunit.client.0.smithi022.stdout:1/10: sync 2024-02-16T19:28:50.963 INFO:tasks.workunit.client.0.smithi022.stdout:6/30: sync 2024-02-16T19:28:50.963 INFO:tasks.workunit.client.0.smithi022.stdout:1/11: rename - no filename 2024-02-16T19:28:50.963 INFO:tasks.workunit.client.0.smithi022.stdout:1/12: dwrite - no filename 2024-02-16T19:28:50.963 INFO:tasks.workunit.client.0.smithi022.stdout:0/44: sync 2024-02-16T19:28:50.963 INFO:tasks.workunit.client.0.smithi022.stdout:4/27: sync 2024-02-16T19:28:50.963 INFO:tasks.workunit.client.0.smithi022.stdout:3/37: sync 2024-02-16T19:28:50.963 INFO:tasks.workunit.client.0.smithi022.stdout:7/16: sync 2024-02-16T19:28:50.964 INFO:tasks.workunit.client.0.smithi022.stdout:5/42: sync 2024-02-16T19:28:50.964 INFO:tasks.workunit.client.0.smithi022.stdout:9/2: sync 2024-02-16T19:28:50.964 INFO:tasks.workunit.client.0.smithi022.stdout:9/3: rename - no filename 2024-02-16T19:28:50.964 INFO:tasks.workunit.client.0.smithi022.stdout:2/3: sync 2024-02-16T19:28:50.965 INFO:tasks.workunit.client.0.smithi022.stdout:7/17: write f4 [4293623,6771] 0 2024-02-16T19:28:50.966 INFO:tasks.workunit.client.0.smithi022.stdout:6/31: write d1/f2 [1351453,114994] 0 2024-02-16T19:28:50.971 INFO:tasks.workunit.client.0.smithi022.stdout:8/33: symlink d3/la 0 2024-02-16T19:28:50.972 INFO:tasks.workunit.client.0.smithi022.stdout:8/34: stat d3/la 0 2024-02-16T19:28:50.973 INFO:tasks.workunit.client.0.smithi022.stdout:3/38: write d7/d12/f14 [719644,21968] 0 2024-02-16T19:28:50.974 INFO:tasks.workunit.client.0.smithi022.stdout:9/4: symlink l0 0 2024-02-16T19:28:50.975 INFO:tasks.workunit.client.0.smithi022.stdout:7/18: mknod c6 0 2024-02-16T19:28:50.976 INFO:tasks.workunit.client.0.smithi022.stdout:4/28: dwrite d4/d6/f9 [0,4194304] 0 2024-02-16T19:28:50.977 INFO:tasks.workunit.client.0.smithi022.stdout:1/13: getdents . 0 2024-02-16T19:28:50.977 INFO:tasks.workunit.client.0.smithi022.stdout:1/14: fdatasync - no filename 2024-02-16T19:28:50.979 INFO:tasks.workunit.client.0.smithi022.stdout:2/4: mkdir d0/d1 0 2024-02-16T19:28:50.979 INFO:tasks.workunit.client.0.smithi022.stdout:2/5: unlink - no file 2024-02-16T19:28:50.979 INFO:tasks.workunit.client.0.smithi022.stdout:2/6: write - no filename 2024-02-16T19:28:50.979 INFO:tasks.workunit.client.0.smithi022.stdout:2/7: fdatasync - no filename 2024-02-16T19:28:50.979 INFO:tasks.workunit.client.0.smithi022.stdout:2/8: dwrite - no filename 2024-02-16T19:28:50.979 INFO:tasks.workunit.client.0.smithi022.stdout:2/9: dread - no filename 2024-02-16T19:28:50.981 INFO:tasks.workunit.client.0.smithi022.stdout:8/35: write d3/d6/f7 [568320,129508] 0 2024-02-16T19:28:50.981 INFO:tasks.workunit.client.0.smithi022.stdout:9/5: mknod c1 0 2024-02-16T19:28:50.983 INFO:tasks.workunit.client.0.smithi022.stdout:8/36: write d3/d6/f7 [3156994,48101] 0 2024-02-16T19:28:50.984 INFO:tasks.workunit.client.0.smithi022.stdout:6/32: mkdir d1/d8/da 0 2024-02-16T19:28:50.985 INFO:tasks.workunit.client.0.smithi022.stdout:8/37: write d3/d6/f7 [4025744,86987] 0 2024-02-16T19:28:50.986 INFO:tasks.workunit.client.0.smithi022.stdout:8/38: rename d3 to d3/d6/db 22 2024-02-16T19:28:50.987 INFO:tasks.workunit.client.0.smithi022.stdout:7/19: mknod c7 0 2024-02-16T19:28:50.989 INFO:tasks.workunit.client.0.smithi022.stdout:4/29: mkdir d4/d6/dd 0 2024-02-16T19:28:50.990 INFO:tasks.workunit.client.0.smithi022.stdout:4/30: rename d4/d6 to d4/d6/de 22 2024-02-16T19:28:50.991 INFO:tasks.workunit.client.0.smithi022.stdout:2/10: rmdir d0 39 2024-02-16T19:28:50.991 INFO:tasks.workunit.client.0.smithi022.stdout:2/11: dread - no filename 2024-02-16T19:28:50.994 INFO:tasks.workunit.client.0.smithi022.stdout:9/6: creat f2 x:0 0 0 2024-02-16T19:28:50.999 INFO:tasks.workunit.client.0.smithi022.stdout:8/39: mknod d3/cc 0 2024-02-16T19:28:50.999 INFO:tasks.workunit.client.0.smithi022.stdout:8/40: fdatasync d3/d6/f7 0 2024-02-16T19:28:51.000 INFO:tasks.workunit.client.0.smithi022.stdout:1/15: symlink l0 0 2024-02-16T19:28:51.000 INFO:tasks.workunit.client.0.smithi022.stdout:1/16: read - no filename 2024-02-16T19:28:51.001 INFO:tasks.workunit.client.0.smithi022.stdout:7/20: mknod c8 0 2024-02-16T19:28:51.097 INFO:tasks.workunit.client.0.smithi022.stdout:2/12: chown d0/d1 15 1 2024-02-16T19:28:51.097 INFO:tasks.workunit.client.0.smithi022.stdout:2/13: write - no filename 2024-02-16T19:28:51.097 INFO:tasks.workunit.client.0.smithi022.stdout:2/14: dwrite - no filename 2024-02-16T19:28:51.097 INFO:tasks.workunit.client.0.smithi022.stdout:2/15: dwrite - no filename 2024-02-16T19:28:51.097 INFO:tasks.workunit.client.0.smithi022.stdout:2/16: write - no filename 2024-02-16T19:28:51.097 INFO:tasks.workunit.client.0.smithi022.stdout:2/17: write - no filename 2024-02-16T19:28:51.097 INFO:tasks.workunit.client.0.smithi022.stdout:2/18: write - no filename 2024-02-16T19:28:51.098 INFO:tasks.workunit.client.0.smithi022.stdout:7/21: chown f5 870 1 2024-02-16T19:28:51.098 INFO:tasks.workunit.client.0.smithi022.stdout:5/43: getdents . 0 2024-02-16T19:28:51.098 INFO:tasks.workunit.client.0.smithi022.stdout:9/7: creat f3 x:0 0 0 2024-02-16T19:28:51.099 INFO:tasks.workunit.client.0.smithi022.stdout:1/17: unlink l0 0 2024-02-16T19:28:51.099 INFO:tasks.workunit.client.0.smithi022.stdout:1/18: rename - no filename 2024-02-16T19:28:51.100 INFO:tasks.workunit.client.0.smithi022.stdout:1/19: chown . 5 1 2024-02-16T19:28:51.100 INFO:tasks.workunit.client.0.smithi022.stdout:1/20: dread - no filename 2024-02-16T19:28:51.100 INFO:tasks.workunit.client.0.smithi022.stdout:1/21: write - no filename 2024-02-16T19:28:51.100 INFO:tasks.workunit.client.0.smithi022.stdout:1/22: write - no filename 2024-02-16T19:28:51.100 INFO:tasks.workunit.client.0.smithi022.stdout:1/23: chown . 120881174 1 2024-02-16T19:28:51.100 INFO:tasks.workunit.client.0.smithi022.stdout:1/24: rename - no filename 2024-02-16T19:28:51.100 INFO:tasks.workunit.client.0.smithi022.stdout:9/8: write f2 [830762,36970] 0 2024-02-16T19:28:51.104 INFO:tasks.workunit.client.0.smithi022.stdout:8/41: dread d3/d6/f7 [0,4194304] 0 2024-02-16T19:28:51.108 INFO:tasks.workunit.client.0.smithi022.stdout:5/44: dwrite f3 [0,4194304] 0 2024-02-16T19:28:51.108 INFO:tasks.workunit.client.0.smithi022.stdout:5/45: rmdir - no directory 2024-02-16T19:28:51.110 INFO:tasks.workunit.client.0.smithi022.stdout:4/31: mkdir d4/d6/dd/df 0 2024-02-16T19:28:51.110 INFO:tasks.workunit.client.0.smithi022.stdout:7/22: mkdir d9 0 2024-02-16T19:28:51.110 INFO:tasks.workunit.client.0.smithi022.stdout:3/39: getdents d7/d12 0 2024-02-16T19:28:51.111 INFO:tasks.workunit.client.0.smithi022.stdout:7/23: stat c7 0 2024-02-16T19:28:51.111 INFO:tasks.workunit.client.0.smithi022.stdout:2/19: rename d0/d1 to d0/d2 0 2024-02-16T19:28:51.112 INFO:tasks.workunit.client.0.smithi022.stdout:1/25: symlink l1 0 2024-02-16T19:28:51.112 INFO:tasks.workunit.client.0.smithi022.stdout:1/26: truncate - no filename 2024-02-16T19:28:51.112 INFO:tasks.workunit.client.0.smithi022.stdout:1/27: dread - no filename 2024-02-16T19:28:51.113 INFO:tasks.workunit.client.0.smithi022.stdout:9/9: symlink l4 0 2024-02-16T19:28:51.114 INFO:tasks.workunit.client.0.smithi022.stdout:8/42: mknod d3/cd 0 2024-02-16T19:28:51.116 INFO:tasks.workunit.client.0.smithi022.stdout:1/28: chown l1 99913 1 2024-02-16T19:28:51.116 INFO:tasks.workunit.client.0.smithi022.stdout:1/29: write - no filename 2024-02-16T19:28:51.116 INFO:tasks.workunit.client.0.smithi022.stdout:3/40: mknod d7/d12/c15 0 2024-02-16T19:28:51.116 INFO:tasks.workunit.client.0.smithi022.stdout:8/43: chown d3/d6 358840418 1 2024-02-16T19:28:51.117 INFO:tasks.workunit.client.0.smithi022.stdout:2/20: mknod d0/d2/c3 0 2024-02-16T19:28:51.118 INFO:tasks.workunit.client.0.smithi022.stdout:4/32: mkdir d4/d6/dd/df/d10 0 2024-02-16T19:28:51.120 INFO:tasks.workunit.client.0.smithi022.stdout:9/10: unlink c1 0 2024-02-16T19:28:51.121 INFO:tasks.workunit.client.0.smithi022.stdout:5/46: dwrite f6 [0,4194304] 0 2024-02-16T19:28:51.121 INFO:tasks.workunit.client.0.smithi022.stdout:7/24: mkdir d9/da 0 2024-02-16T19:28:51.122 INFO:tasks.workunit.client.0.smithi022.stdout:7/25: fsync f4 0 2024-02-16T19:28:51.123 INFO:tasks.workunit.client.0.smithi022.stdout:3/41: chown l3 1182631 1 2024-02-16T19:28:51.123 INFO:tasks.workunit.client.0.smithi022.stdout:2/21: mknod d0/c4 0 2024-02-16T19:28:51.123 INFO:tasks.workunit.client.0.smithi022.stdout:2/22: dwrite - no filename 2024-02-16T19:28:51.125 INFO:tasks.workunit.client.0.smithi022.stdout:3/42: rename d7 to d7/d12/d16 22 2024-02-16T19:28:51.125 INFO:tasks.workunit.client.0.smithi022.stdout:4/33: mknod d4/c11 0 2024-02-16T19:28:51.127 INFO:tasks.workunit.client.0.smithi022.stdout:9/11: creat f5 x:0 0 0 2024-02-16T19:28:51.130 INFO:tasks.workunit.client.0.smithi022.stdout:4/34: creat d4/d6/dd/f12 x:0 0 0 2024-02-16T19:28:51.134 INFO:tasks.workunit.client.0.smithi022.stdout:0/45: sync 2024-02-16T19:28:51.139 INFO:tasks.workunit.client.0.smithi022.stdout:8/44: dwrite d3/d6/f7 [0,4194304] 0 2024-02-16T19:28:51.140 INFO:tasks.workunit.client.0.smithi022.stdout:3/43: rename d7/d12/c15 to d7/c17 0 2024-02-16T19:28:51.140 INFO:tasks.workunit.client.0.smithi022.stdout:1/30: link l1 l2 0 2024-02-16T19:28:51.141 INFO:tasks.workunit.client.0.smithi022.stdout:7/26: symlink d9/da/lb 0 2024-02-16T19:28:51.142 INFO:tasks.workunit.client.0.smithi022.stdout:7/27: read f5 [2589674,1355] 0 2024-02-16T19:28:51.144 INFO:tasks.workunit.client.0.smithi022.stdout:1/31: creat f3 x:0 0 0 2024-02-16T19:28:51.144 INFO:tasks.workunit.client.0.smithi022.stdout:9/12: link l4 l6 0 2024-02-16T19:28:51.148 INFO:tasks.workunit.client.0.smithi022.stdout:8/45: rename d3/cd to d3/d6/ce 0 2024-02-16T19:28:51.148 INFO:tasks.workunit.client.0.smithi022.stdout:7/28: rename d9/da to d9/da/dc 22 2024-02-16T19:28:51.148 INFO:tasks.workunit.client.0.smithi022.stdout:8/46: readlink d3/la 0 2024-02-16T19:28:51.149 INFO:tasks.workunit.client.0.smithi022.stdout:1/32: mknod c4 0 2024-02-16T19:28:51.149 INFO:tasks.workunit.client.0.smithi022.stdout:7/29: write f5 [1744062,83210] 0 2024-02-16T19:28:51.150 INFO:tasks.workunit.client.0.smithi022.stdout:3/44: dread f2 [0,4194304] 0 2024-02-16T19:28:51.150 INFO:tasks.workunit.client.0.smithi022.stdout:4/35: mknod d4/d6/dd/df/d10/c13 0 2024-02-16T19:28:51.151 INFO:tasks.workunit.client.0.smithi022.stdout:4/36: dread - d4/d6/fb zero size 2024-02-16T19:28:51.152 INFO:tasks.workunit.client.0.smithi022.stdout:9/13: rename f2 to f7 0 2024-02-16T19:28:51.154 INFO:tasks.workunit.client.0.smithi022.stdout:4/37: truncate d4/d6/fb 679725 0 2024-02-16T19:28:51.156 INFO:tasks.workunit.client.0.smithi022.stdout:3/45: rename d7/dc to d7/d12/d18 0 2024-02-16T19:28:51.157 INFO:tasks.workunit.client.0.smithi022.stdout:9/14: dread f7 [0,4194304] 0 2024-02-16T19:28:51.158 INFO:tasks.workunit.client.0.smithi022.stdout:4/38: unlink f1 0 2024-02-16T19:28:51.159 INFO:tasks.workunit.client.0.smithi022.stdout:1/33: dwrite f3 [0,4194304] 0 2024-02-16T19:28:51.162 INFO:tasks.workunit.client.0.smithi022.stdout:9/15: chown l6 0 1 2024-02-16T19:28:51.162 INFO:tasks.workunit.client.0.smithi022.stdout:8/47: fdatasync d3/d6/f7 0 2024-02-16T19:28:51.163 INFO:tasks.workunit.client.0.smithi022.stdout:3/46: dread d7/d12/f14 [0,4194304] 0 2024-02-16T19:28:51.164 INFO:tasks.workunit.client.0.smithi022.stdout:9/16: mknod c8 0 2024-02-16T19:28:51.164 INFO:tasks.workunit.client.0.smithi022.stdout:1/34: dread f3 [0,4194304] 0 2024-02-16T19:28:51.165 INFO:tasks.workunit.client.0.smithi022.stdout:8/48: symlink d3/lf 0 2024-02-16T19:28:51.166 INFO:tasks.workunit.client.0.smithi022.stdout:9/17: symlink l9 0 2024-02-16T19:28:51.169 INFO:tasks.workunit.client.0.smithi022.stdout:8/49: creat d3/f10 x:0 0 0 2024-02-16T19:28:51.169 INFO:tasks.workunit.client.0.smithi022.stdout:8/50: dread - d3/f10 zero size 2024-02-16T19:28:51.170 INFO:tasks.workunit.client.0.smithi022.stdout:9/18: rename c8 to ca 0 2024-02-16T19:28:51.170 INFO:tasks.workunit.client.0.smithi022.stdout:8/51: chown d3/d6/l9 14213 1 2024-02-16T19:28:51.173 INFO:tasks.workunit.client.0.smithi022.stdout:8/52: creat d3/f11 x:0 0 0 2024-02-16T19:28:51.173 INFO:tasks.workunit.client.0.smithi022.stdout:8/53: chown d3/d6/f7 219547527 1 2024-02-16T19:28:51.175 INFO:tasks.workunit.client.0.smithi022.stdout:8/54: mkdir d3/d12 0 2024-02-16T19:28:51.176 INFO:tasks.workunit.client.0.smithi022.stdout:8/55: symlink d3/l13 0 2024-02-16T19:28:51.177 INFO:tasks.workunit.client.0.smithi022.stdout:1/35: fdatasync f3 0 2024-02-16T19:28:51.180 INFO:tasks.workunit.client.0.smithi022.stdout:1/36: rename l1 to l5 0 2024-02-16T19:28:51.180 INFO:tasks.workunit.client.0.smithi022.stdout:1/37: rmdir - no directory 2024-02-16T19:28:51.183 INFO:tasks.workunit.client.0.smithi022.stdout:1/38: dread f3 [0,4194304] 0 2024-02-16T19:28:51.184 INFO:tasks.workunit.client.0.smithi022.stdout:1/39: chown f3 118 1 2024-02-16T19:28:51.188 INFO:tasks.workunit.client.0.smithi022.stdout:9/19: fsync f7 0 2024-02-16T19:28:51.189 INFO:tasks.workunit.client.0.smithi022.stdout:9/20: mkdir db 0 2024-02-16T19:28:51.192 INFO:tasks.workunit.client.0.smithi022.stdout:1/40: truncate f3 3230892 0 2024-02-16T19:28:51.195 INFO:tasks.workunit.client.0.smithi022.stdout:9/21: dwrite f7 [0,4194304] 0 2024-02-16T19:28:51.200 INFO:tasks.workunit.client.0.smithi022.stdout:9/22: creat db/fc x:0 0 0 2024-02-16T19:28:51.201 INFO:tasks.workunit.client.0.smithi022.stdout:9/23: write f7 [2625782,41165] 0 2024-02-16T19:28:51.201 INFO:tasks.workunit.client.0.smithi022.stdout:1/41: write f3 [669795,119508] 0 2024-02-16T19:28:51.202 INFO:tasks.workunit.client.0.smithi022.stdout:9/24: dread - f3 zero size 2024-02-16T19:28:51.204 INFO:tasks.workunit.client.0.smithi022.stdout:9/25: mkdir db/dd 0 2024-02-16T19:28:51.205 INFO:tasks.workunit.client.0.smithi022.stdout:1/42: dread f3 [0,4194304] 0 2024-02-16T19:28:51.206 INFO:tasks.workunit.client.0.smithi022.stdout:9/26: mknod db/ce 0 2024-02-16T19:28:51.207 INFO:tasks.workunit.client.0.smithi022.stdout:9/27: readlink l6 0 2024-02-16T19:28:51.208 INFO:tasks.workunit.client.0.smithi022.stdout:9/28: truncate f3 488818 0 2024-02-16T19:28:51.210 INFO:tasks.workunit.client.0.smithi022.stdout:9/29: creat db/ff x:0 0 0 2024-02-16T19:28:51.211 INFO:tasks.workunit.client.0.smithi022.stdout:9/30: chown l6 8238 1 2024-02-16T19:28:51.212 INFO:tasks.workunit.client.0.smithi022.stdout:1/43: dwrite f3 [0,4194304] 0 2024-02-16T19:28:51.215 INFO:tasks.workunit.client.0.smithi022.stdout:1/44: getdents . 0 2024-02-16T19:28:51.221 INFO:tasks.workunit.client.0.smithi022.stdout:1/45: dwrite f3 [0,4194304] 0 2024-02-16T19:28:51.224 INFO:tasks.workunit.client.0.smithi022.stdout:1/46: mkdir d6 0 2024-02-16T19:28:51.226 INFO:tasks.workunit.client.0.smithi022.stdout:1/47: chown l2 24661912 1 2024-02-16T19:28:51.227 INFO:tasks.workunit.client.0.smithi022.stdout:1/48: unlink f3 0 2024-02-16T19:28:51.227 INFO:tasks.workunit.client.0.smithi022.stdout:1/49: write - no filename 2024-02-16T19:28:51.228 INFO:tasks.workunit.client.0.smithi022.stdout:1/50: fdatasync - no filename 2024-02-16T19:28:51.228 INFO:tasks.workunit.client.0.smithi022.stdout:1/51: write - no filename 2024-02-16T19:28:51.229 INFO:tasks.workunit.client.0.smithi022.stdout:1/52: mknod d6/c7 0 2024-02-16T19:28:51.230 INFO:tasks.workunit.client.0.smithi022.stdout:1/53: rmdir d6 39 2024-02-16T19:28:51.241 INFO:tasks.workunit.client.0.smithi022.stdout:1/54: mkdir d6/d8 0 2024-02-16T19:28:51.242 INFO:tasks.workunit.client.0.smithi022.stdout:1/55: creat d6/f9 x:0 0 0 2024-02-16T19:28:51.244 INFO:tasks.workunit.client.0.smithi022.stdout:1/56: link l2 d6/la 0 2024-02-16T19:28:51.245 INFO:tasks.workunit.client.0.smithi022.stdout:1/57: read - d6/f9 zero size 2024-02-16T19:28:51.246 INFO:tasks.workunit.client.0.smithi022.stdout:1/58: mknod d6/d8/cb 0 2024-02-16T19:28:51.247 INFO:tasks.workunit.client.0.smithi022.stdout:1/59: mknod d6/cc 0 2024-02-16T19:28:51.248 INFO:tasks.workunit.client.0.smithi022.stdout:1/60: rmdir d6/d8 39 2024-02-16T19:28:51.249 INFO:tasks.workunit.client.0.smithi022.stdout:1/61: truncate d6/f9 535240 0 2024-02-16T19:28:51.253 INFO:tasks.workunit.client.0.smithi022.stdout:1/62: mknod d6/d8/cd 0 2024-02-16T19:28:51.253 INFO:tasks.workunit.client.0.smithi022.stdout:1/63: fdatasync d6/f9 0 2024-02-16T19:28:51.254 INFO:tasks.workunit.client.0.smithi022.stdout:1/64: write d6/f9 [554398,35408] 0 2024-02-16T19:28:51.256 INFO:tasks.workunit.client.0.smithi022.stdout:1/65: read d6/f9 [213805,69407] 0 2024-02-16T19:28:51.261 INFO:tasks.workunit.client.0.smithi022.stdout:1/66: truncate d6/f9 411179 0 2024-02-16T19:28:51.290 INFO:tasks.workunit.client.0.smithi022.stdout:6/33: sync 2024-02-16T19:28:51.452 INFO:tasks.workunit.client.0.smithi022.stdout:1/67: mknod d6/ce 0 2024-02-16T19:28:51.453 INFO:tasks.workunit.client.0.smithi022.stdout:6/34: creat d1/d8/fb x:0 0 0 2024-02-16T19:28:51.454 INFO:tasks.workunit.client.0.smithi022.stdout:1/68: mkdir d6/d8/df 0 2024-02-16T19:28:51.454 INFO:tasks.workunit.client.0.smithi022.stdout:6/35: chown d1/f3 3038 1 2024-02-16T19:28:51.456 INFO:tasks.workunit.client.0.smithi022.stdout:1/69: creat d6/d8/f10 x:0 0 0 2024-02-16T19:28:51.457 INFO:tasks.workunit.client.0.smithi022.stdout:1/70: write d6/d8/f10 [34101,57782] 0 2024-02-16T19:28:51.458 INFO:tasks.workunit.client.0.smithi022.stdout:6/36: truncate d1/f3 3458069 0 2024-02-16T19:28:51.459 INFO:tasks.workunit.client.0.smithi022.stdout:1/71: creat d6/d8/df/f11 x:0 0 0 2024-02-16T19:28:51.460 INFO:tasks.workunit.client.0.smithi022.stdout:6/37: symlink d1/d8/lc 0 2024-02-16T19:28:51.461 INFO:tasks.workunit.client.0.smithi022.stdout:6/38: creat d1/fd x:0 0 0 2024-02-16T19:28:51.463 INFO:tasks.workunit.client.0.smithi022.stdout:6/39: symlink d1/d8/da/le 0 2024-02-16T19:28:51.468 INFO:tasks.workunit.client.0.smithi022.stdout:6/40: dwrite d1/f7 [0,4194304] 0 2024-02-16T19:28:51.472 INFO:tasks.workunit.client.0.smithi022.stdout:6/41: rename d1/fd to d1/d8/ff 0 2024-02-16T19:28:51.473 INFO:tasks.workunit.client.0.smithi022.stdout:6/42: write d1/f7 [1072657,79006] 0 2024-02-16T19:28:51.474 INFO:tasks.workunit.client.0.smithi022.stdout:6/43: write d1/d8/fb [303894,85441] 0 2024-02-16T19:28:51.644 INFO:tasks.workunit.client.0.smithi022.stdout:1/72: getdents d6/d8/df 0 2024-02-16T19:28:51.646 INFO:tasks.workunit.client.0.smithi022.stdout:1/73: dread d6/d8/f10 [0,4194304] 0 2024-02-16T19:28:51.647 INFO:tasks.workunit.client.0.smithi022.stdout:1/74: dread - d6/d8/df/f11 zero size 2024-02-16T19:28:51.649 INFO:tasks.workunit.client.0.smithi022.stdout:1/75: rmdir d6/d8 39 2024-02-16T19:28:51.651 INFO:tasks.workunit.client.0.smithi022.stdout:6/44: getdents d1/d8 0 2024-02-16T19:28:51.655 INFO:tasks.workunit.client.0.smithi022.stdout:1/76: creat d6/d8/f12 x:0 0 0 2024-02-16T19:28:51.658 INFO:tasks.workunit.client.0.smithi022.stdout:6/45: dwrite d1/d8/fb [0,4194304] 0 2024-02-16T19:28:51.660 INFO:tasks.workunit.client.0.smithi022.stdout:1/77: link d6/ce d6/d8/df/c13 0 2024-02-16T19:28:51.662 INFO:tasks.workunit.client.0.smithi022.stdout:0/46: sync 2024-02-16T19:28:51.662 INFO:tasks.workunit.client.0.smithi022.stdout:2/23: sync 2024-02-16T19:28:51.662 INFO:tasks.workunit.client.0.smithi022.stdout:5/47: sync 2024-02-16T19:28:51.662 INFO:tasks.workunit.client.0.smithi022.stdout:7/30: sync 2024-02-16T19:28:51.664 INFO:tasks.workunit.client.0.smithi022.stdout:5/48: mkdir dd 0 2024-02-16T19:28:51.665 INFO:tasks.workunit.client.0.smithi022.stdout:5/49: write f6 [3491394,106630] 0 2024-02-16T19:28:51.665 INFO:tasks.workunit.client.0.smithi022.stdout:6/46: dwrite d1/f7 [0,4194304] 0 2024-02-16T19:28:51.667 INFO:tasks.workunit.client.0.smithi022.stdout:2/24: unlink d0/c4 0 2024-02-16T19:28:51.667 INFO:tasks.workunit.client.0.smithi022.stdout:5/50: chown dd 10 1 2024-02-16T19:28:51.668 INFO:tasks.workunit.client.0.smithi022.stdout:5/51: chown fc 826443306 1 2024-02-16T19:28:51.668 INFO:tasks.workunit.client.0.smithi022.stdout:7/31: mkdir d9/dd 0 2024-02-16T19:28:51.671 INFO:tasks.workunit.client.0.smithi022.stdout:0/47: mkdir d10/d11 0 2024-02-16T19:28:51.672 INFO:tasks.workunit.client.0.smithi022.stdout:6/47: mknod d1/d8/da/c10 0 2024-02-16T19:28:51.673 INFO:tasks.workunit.client.0.smithi022.stdout:6/48: readlink d1/l5 0 2024-02-16T19:28:51.674 INFO:tasks.workunit.client.0.smithi022.stdout:2/25: rename d0/d2/c3 to d0/c5 0 2024-02-16T19:28:51.674 INFO:tasks.workunit.client.0.smithi022.stdout:2/26: fdatasync - no filename 2024-02-16T19:28:51.674 INFO:tasks.workunit.client.0.smithi022.stdout:2/27: dread - no filename 2024-02-16T19:28:51.674 INFO:tasks.workunit.client.0.smithi022.stdout:2/28: dread - no filename 2024-02-16T19:28:51.674 INFO:tasks.workunit.client.0.smithi022.stdout:2/29: read - no filename 2024-02-16T19:28:51.677 INFO:tasks.workunit.client.0.smithi022.stdout:7/32: unlink d9/da/lb 0 2024-02-16T19:28:51.678 INFO:tasks.workunit.client.0.smithi022.stdout:5/52: dwrite f5 [0,4194304] 0 2024-02-16T19:28:51.678 INFO:tasks.workunit.client.0.smithi022.stdout:1/78: link d6/d8/df/c13 d6/c14 0 2024-02-16T19:28:51.679 INFO:tasks.workunit.client.0.smithi022.stdout:0/48: mknod d10/c12 0 2024-02-16T19:28:51.680 INFO:tasks.workunit.client.0.smithi022.stdout:0/49: chown l6 2 1 2024-02-16T19:28:51.681 INFO:tasks.workunit.client.0.smithi022.stdout:6/49: dread d1/f2 [0,4194304] 0 2024-02-16T19:28:51.682 INFO:tasks.workunit.client.0.smithi022.stdout:2/30: mknod d0/c6 0 2024-02-16T19:28:51.682 INFO:tasks.workunit.client.0.smithi022.stdout:6/50: read f0 [6236155,87764] 0 2024-02-16T19:28:51.684 INFO:tasks.workunit.client.0.smithi022.stdout:6/51: chown d1 204 1 2024-02-16T19:28:51.686 INFO:tasks.workunit.client.0.smithi022.stdout:1/79: creat d6/d8/df/f15 x:0 0 0 2024-02-16T19:28:51.686 INFO:tasks.workunit.client.0.smithi022.stdout:7/33: mkdir d9/de 0 2024-02-16T19:28:51.686 INFO:tasks.workunit.client.0.smithi022.stdout:5/53: dread f6 [0,4194304] 0 2024-02-16T19:28:51.687 INFO:tasks.workunit.client.0.smithi022.stdout:7/34: fdatasync f5 0 2024-02-16T19:28:51.688 INFO:tasks.workunit.client.0.smithi022.stdout:2/31: mknod d0/c7 0 2024-02-16T19:28:51.690 INFO:tasks.workunit.client.0.smithi022.stdout:6/52: chown d1/f3 64268 1 2024-02-16T19:28:51.695 INFO:tasks.workunit.client.0.smithi022.stdout:1/80: dwrite d6/d8/f12 [0,4194304] 0 2024-02-16T19:28:51.695 INFO:tasks.workunit.client.0.smithi022.stdout:5/54: write f6 [647001,84812] 0 2024-02-16T19:28:51.700 INFO:tasks.workunit.client.0.smithi022.stdout:2/32: rename d0/c6 to d0/c8 0 2024-02-16T19:28:51.700 INFO:tasks.workunit.client.0.smithi022.stdout:6/53: fdatasync d1/f2 0 2024-02-16T19:28:51.704 INFO:tasks.workunit.client.0.smithi022.stdout:4/39: sync 2024-02-16T19:28:51.704 INFO:tasks.workunit.client.0.smithi022.stdout:3/47: sync 2024-02-16T19:28:51.704 INFO:tasks.workunit.client.0.smithi022.stdout:8/56: sync 2024-02-16T19:28:51.707 INFO:tasks.workunit.client.0.smithi022.stdout:2/33: chown d0/c5 2311331 1 2024-02-16T19:28:51.707 INFO:tasks.workunit.client.0.smithi022.stdout:2/34: write - no filename 2024-02-16T19:28:51.707 INFO:tasks.workunit.client.0.smithi022.stdout:2/35: fsync - no filename 2024-02-16T19:28:51.707 INFO:tasks.workunit.client.0.smithi022.stdout:2/36: fsync - no filename 2024-02-16T19:28:51.707 INFO:tasks.workunit.client.0.smithi022.stdout:2/37: fsync - no filename 2024-02-16T19:28:51.710 INFO:tasks.workunit.client.0.smithi022.stdout:6/54: unlink f0 0 2024-02-16T19:28:51.711 INFO:tasks.workunit.client.0.smithi022.stdout:5/55: symlink dd/le 0 2024-02-16T19:28:51.712 INFO:tasks.workunit.client.0.smithi022.stdout:4/40: mkdir d4/d6/d14 0 2024-02-16T19:28:51.713 INFO:tasks.workunit.client.0.smithi022.stdout:1/81: dwrite d6/d8/f10 [0,4194304] 0 2024-02-16T19:28:51.714 INFO:tasks.workunit.client.0.smithi022.stdout:1/82: chown d6/d8/df/f15 1634 1 2024-02-16T19:28:51.714 INFO:tasks.workunit.client.0.smithi022.stdout:3/48: creat d7/d12/f19 x:0 0 0 2024-02-16T19:28:51.715 INFO:tasks.workunit.client.0.smithi022.stdout:3/49: chown d7/d12/d18 56 1 2024-02-16T19:28:51.716 INFO:tasks.workunit.client.0.smithi022.stdout:6/55: creat d1/d8/f11 x:0 0 0 2024-02-16T19:28:51.718 INFO:tasks.workunit.client.0.smithi022.stdout:5/56: rename la to dd/lf 0 2024-02-16T19:28:51.719 INFO:tasks.workunit.client.0.smithi022.stdout:5/57: write fb [282830,88906] 0 2024-02-16T19:28:51.721 INFO:tasks.workunit.client.0.smithi022.stdout:4/41: mknod d4/c15 0 2024-02-16T19:28:51.722 INFO:tasks.workunit.client.0.smithi022.stdout:0/50: truncate f7 3387280 0 2024-02-16T19:28:51.723 INFO:tasks.workunit.client.0.smithi022.stdout:1/83: mkdir d6/d8/df/d16 0 2024-02-16T19:28:51.724 INFO:tasks.workunit.client.0.smithi022.stdout:7/35: truncate f4 2414290 0 2024-02-16T19:28:51.726 INFO:tasks.workunit.client.0.smithi022.stdout:6/56: mknod d1/d8/c12 0 2024-02-16T19:28:51.726 INFO:tasks.workunit.client.0.smithi022.stdout:1/84: fdatasync d6/d8/df/f11 0 2024-02-16T19:28:51.728 INFO:tasks.workunit.client.0.smithi022.stdout:5/58: dwrite f3 [0,4194304] 0 2024-02-16T19:28:51.731 INFO:tasks.workunit.client.0.smithi022.stdout:4/42: rename d4/d6/fb to d4/d6/f16 0 2024-02-16T19:28:51.733 INFO:tasks.workunit.client.0.smithi022.stdout:0/51: creat d10/f13 x:0 0 0 2024-02-16T19:28:51.736 INFO:tasks.workunit.client.0.smithi022.stdout:8/57: getdents d3/d6 0 2024-02-16T19:28:51.737 INFO:tasks.workunit.client.0.smithi022.stdout:6/57: creat d1/d8/f13 x:0 0 0 2024-02-16T19:28:51.741 INFO:tasks.workunit.client.0.smithi022.stdout:5/59: mknod dd/c10 0 2024-02-16T19:28:51.742 INFO:tasks.workunit.client.0.smithi022.stdout:4/43: readlink d4/d6/l8 0 2024-02-16T19:28:51.743 INFO:tasks.workunit.client.0.smithi022.stdout:4/44: read d4/d6/f9 [2539301,57114] 0 2024-02-16T19:28:51.744 INFO:tasks.workunit.client.0.smithi022.stdout:0/52: rename l8 to d10/l14 0 2024-02-16T19:28:51.744 INFO:tasks.workunit.client.0.smithi022.stdout:7/36: read f4 [672743,20183] 0 2024-02-16T19:28:51.745 INFO:tasks.workunit.client.0.smithi022.stdout:4/45: truncate d4/d6/dd/f12 126569 0 2024-02-16T19:28:51.747 INFO:tasks.workunit.client.0.smithi022.stdout:1/85: mkdir d6/d8/df/d16/d17 0 2024-02-16T19:28:51.750 INFO:tasks.workunit.client.0.smithi022.stdout:0/53: readlink le 0 2024-02-16T19:28:51.751 INFO:tasks.workunit.client.0.smithi022.stdout:3/50: getdents d7/d12/d18/dd 0 2024-02-16T19:28:51.753 INFO:tasks.workunit.client.0.smithi022.stdout:3/51: dread d7/d12/f14 [0,4194304] 0 2024-02-16T19:28:51.756 INFO:tasks.workunit.client.0.smithi022.stdout:5/60: link f3 dd/f11 0 2024-02-16T19:28:51.758 INFO:tasks.workunit.client.0.smithi022.stdout:7/37: mknod d9/de/cf 0 2024-02-16T19:28:51.759 INFO:tasks.workunit.client.0.smithi022.stdout:3/52: write d7/d12/f14 [1551110,102812] 0 2024-02-16T19:28:51.760 INFO:tasks.workunit.client.0.smithi022.stdout:3/53: write d7/d12/d18/dd/f13 [132742,80890] 0 2024-02-16T19:28:51.761 INFO:tasks.workunit.client.0.smithi022.stdout:5/61: fdatasync dd/f11 0 2024-02-16T19:28:51.762 INFO:tasks.workunit.client.0.smithi022.stdout:5/62: chown dd/f11 72279 1 2024-02-16T19:28:51.765 INFO:tasks.workunit.client.0.smithi022.stdout:8/58: getdents d3 0 2024-02-16T19:28:51.765 INFO:tasks.workunit.client.0.smithi022.stdout:1/86: truncate d6/d8/f12 3704266 0 2024-02-16T19:28:51.766 INFO:tasks.workunit.client.0.smithi022.stdout:7/38: mknod d9/da/c10 0 2024-02-16T19:28:51.767 INFO:tasks.workunit.client.0.smithi022.stdout:7/39: chown d9/de 4087 1 2024-02-16T19:28:51.768 INFO:tasks.workunit.client.0.smithi022.stdout:3/54: fsync f5 0 2024-02-16T19:28:51.769 INFO:tasks.workunit.client.0.smithi022.stdout:5/63: mknod dd/c12 0 2024-02-16T19:28:51.770 INFO:tasks.workunit.client.0.smithi022.stdout:1/87: rename d6/d8/cd to d6/d8/c18 0 2024-02-16T19:28:51.771 INFO:tasks.workunit.client.0.smithi022.stdout:4/46: getdents d4/d6/dd/df/d10 0 2024-02-16T19:28:51.771 INFO:tasks.workunit.client.0.smithi022.stdout:1/88: chown d6 7593990 1 2024-02-16T19:28:51.772 INFO:tasks.workunit.client.0.smithi022.stdout:7/40: dread f4 [0,4194304] 0 2024-02-16T19:28:51.773 INFO:tasks.workunit.client.0.smithi022.stdout:7/41: readlink l0 0 2024-02-16T19:28:51.773 INFO:tasks.workunit.client.0.smithi022.stdout:3/55: creat d7/d12/d18/f1a x:0 0 0 2024-02-16T19:28:51.773 INFO:tasks.workunit.client.0.smithi022.stdout:0/54: dwrite f7 [0,4194304] 0 2024-02-16T19:28:51.774 INFO:tasks.workunit.client.0.smithi022.stdout:0/55: dread - d10/f13 zero size 2024-02-16T19:28:51.775 INFO:tasks.workunit.client.0.smithi022.stdout:5/64: symlink dd/l13 0 2024-02-16T19:28:51.775 INFO:tasks.workunit.client.0.smithi022.stdout:0/56: fsync fb 0 2024-02-16T19:28:51.776 INFO:tasks.workunit.client.0.smithi022.stdout:1/89: stat d6/f9 0 2024-02-16T19:28:51.776 INFO:tasks.workunit.client.0.smithi022.stdout:5/65: fdatasync fc 0 2024-02-16T19:28:51.777 INFO:tasks.workunit.client.0.smithi022.stdout:7/42: write f5 [1000674,5650] 0 2024-02-16T19:28:51.778 INFO:tasks.workunit.client.0.smithi022.stdout:3/56: creat d7/d12/d18/f1b x:0 0 0 2024-02-16T19:28:51.781 INFO:tasks.workunit.client.0.smithi022.stdout:0/57: getdents d10/d11 0 2024-02-16T19:28:51.783 INFO:tasks.workunit.client.0.smithi022.stdout:3/57: read - d7/d12/d18/f1b zero size 2024-02-16T19:28:51.784 INFO:tasks.workunit.client.0.smithi022.stdout:0/58: dread - fb zero size 2024-02-16T19:28:51.784 INFO:tasks.workunit.client.0.smithi022.stdout:1/90: creat d6/f19 x:0 0 0 2024-02-16T19:28:51.788 INFO:tasks.workunit.client.0.smithi022.stdout:0/59: mkdir d10/d15 0 2024-02-16T19:28:51.791 INFO:tasks.workunit.client.0.smithi022.stdout:5/66: dwrite f6 [4194304,4194304] 0 2024-02-16T19:28:51.792 INFO:tasks.workunit.client.0.smithi022.stdout:4/47: dwrite d4/d6/f9 [4194304,4194304] 0 2024-02-16T19:28:51.797 INFO:tasks.workunit.client.0.smithi022.stdout:1/91: creat d6/d8/df/d16/d17/f1a x:0 0 0 2024-02-16T19:28:51.799 INFO:tasks.workunit.client.0.smithi022.stdout:3/58: link d7/d12/d18/dd/f13 d7/f1c 0 2024-02-16T19:28:51.801 INFO:tasks.workunit.client.0.smithi022.stdout:4/48: unlink d4/d6/l8 0 2024-02-16T19:28:51.802 INFO:tasks.workunit.client.0.smithi022.stdout:0/60: dwrite d10/f13 [0,4194304] 0 2024-02-16T19:28:51.803 INFO:tasks.workunit.client.0.smithi022.stdout:0/61: read - fb zero size 2024-02-16T19:28:51.804 INFO:tasks.workunit.client.0.smithi022.stdout:1/92: chown d6/c14 334024 1 2024-02-16T19:28:51.805 INFO:tasks.workunit.client.0.smithi022.stdout:7/43: truncate f5 156700 0 2024-02-16T19:28:51.806 INFO:tasks.workunit.client.0.smithi022.stdout:3/59: dread f2 [0,4194304] 0 2024-02-16T19:28:51.806 INFO:tasks.workunit.client.0.smithi022.stdout:0/62: write fb [1027077,122416] 0 2024-02-16T19:28:51.807 INFO:tasks.workunit.client.0.smithi022.stdout:7/44: chown d9/da/c10 2 1 2024-02-16T19:28:51.809 INFO:tasks.workunit.client.0.smithi022.stdout:3/60: fdatasync d7/d12/f19 0 2024-02-16T19:28:51.809 INFO:tasks.workunit.client.0.smithi022.stdout:0/63: chown d10/l14 3 1 2024-02-16T19:28:51.812 INFO:tasks.workunit.client.0.smithi022.stdout:3/61: write d7/d12/f14 [200343,55192] 0 2024-02-16T19:28:51.814 INFO:tasks.workunit.client.0.smithi022.stdout:0/64: getdents d10/d15 0 2024-02-16T19:28:51.816 INFO:tasks.workunit.client.0.smithi022.stdout:1/93: fsync d6/d8/df/d16/d17/f1a 0 2024-02-16T19:28:51.819 INFO:tasks.workunit.client.0.smithi022.stdout:4/49: dwrite d4/d6/dd/f12 [0,4194304] 0 2024-02-16T19:28:51.821 INFO:tasks.workunit.client.0.smithi022.stdout:0/65: unlink d10/l14 0 2024-02-16T19:28:51.824 INFO:tasks.workunit.client.0.smithi022.stdout:4/50: mknod d4/d6/c17 0 2024-02-16T19:28:51.826 INFO:tasks.workunit.client.0.smithi022.stdout:1/94: chown d6/d8/f12 97 1 2024-02-16T19:28:51.828 INFO:tasks.workunit.client.0.smithi022.stdout:3/62: dwrite f5 [0,4194304] 0 2024-02-16T19:28:51.829 INFO:tasks.workunit.client.0.smithi022.stdout:0/66: dread fb [0,4194304] 0 2024-02-16T19:28:51.831 INFO:tasks.workunit.client.0.smithi022.stdout:4/51: dread d4/d6/f16 [0,4194304] 0 2024-02-16T19:28:51.831 INFO:tasks.workunit.client.0.smithi022.stdout:1/95: creat d6/d8/df/f1b x:0 0 0 2024-02-16T19:28:51.833 INFO:tasks.workunit.client.0.smithi022.stdout:3/63: dread - d7/d12/d18/f1b zero size 2024-02-16T19:28:51.833 INFO:tasks.workunit.client.0.smithi022.stdout:7/45: dwrite f5 [0,4194304] 0 2024-02-16T19:28:51.836 INFO:tasks.workunit.client.0.smithi022.stdout:0/67: creat d10/d11/f16 x:0 0 0 2024-02-16T19:28:51.836 INFO:tasks.workunit.client.0.smithi022.stdout:9/31: sync 2024-02-16T19:28:51.839 INFO:tasks.workunit.client.0.smithi022.stdout:4/52: write d4/d6/f16 [122226,106771] 0 2024-02-16T19:28:51.839 INFO:tasks.workunit.client.0.smithi022.stdout:9/32: write f3 [250266,25496] 0 2024-02-16T19:28:51.839 INFO:tasks.workunit.client.0.smithi022.stdout:1/96: creat d6/f1c x:0 0 0 2024-02-16T19:28:51.840 INFO:tasks.workunit.client.0.smithi022.stdout:0/68: creat d10/f17 x:0 0 0 2024-02-16T19:28:51.842 INFO:tasks.workunit.client.0.smithi022.stdout:4/53: write d4/d6/f16 [1457437,118171] 0 2024-02-16T19:28:51.843 INFO:tasks.workunit.client.0.smithi022.stdout:7/46: dread f5 [0,4194304] 0 2024-02-16T19:28:51.843 INFO:tasks.workunit.client.0.smithi022.stdout:1/97: mkdir d6/d8/df/d16/d17/d1d 0 2024-02-16T19:28:51.845 INFO:tasks.workunit.client.0.smithi022.stdout:9/33: symlink db/dd/l10 0 2024-02-16T19:28:51.845 INFO:tasks.workunit.client.0.smithi022.stdout:1/98: dread - d6/d8/df/f1b zero size 2024-02-16T19:28:51.845 INFO:tasks.workunit.client.0.smithi022.stdout:4/54: write d4/d6/f9 [2763029,37032] 0 2024-02-16T19:28:51.846 INFO:tasks.workunit.client.0.smithi022.stdout:0/69: mkdir d10/d15/d18 0 2024-02-16T19:28:51.847 INFO:tasks.workunit.client.0.smithi022.stdout:9/34: mknod db/c11 0 2024-02-16T19:28:51.849 INFO:tasks.workunit.client.0.smithi022.stdout:4/55: dread d4/d6/f9 [4194304,4194304] 0 2024-02-16T19:28:51.862 INFO:tasks.workunit.client.0.smithi022.stdout:0/70: creat d10/f19 x:0 0 0 2024-02-16T19:28:51.864 INFO:tasks.workunit.client.0.smithi022.stdout:7/47: truncate f5 2048599 0 2024-02-16T19:28:51.865 INFO:tasks.workunit.client.0.smithi022.stdout:7/48: chown d9/da 200 1 2024-02-16T19:28:51.866 INFO:tasks.workunit.client.0.smithi022.stdout:1/99: creat d6/d8/df/d16/f1e x:0 0 0 2024-02-16T19:28:51.867 INFO:tasks.workunit.client.0.smithi022.stdout:9/35: creat db/f12 x:0 0 0 2024-02-16T19:28:51.868 INFO:tasks.workunit.client.0.smithi022.stdout:4/56: mknod d4/d6/c18 0 2024-02-16T19:28:51.868 INFO:tasks.workunit.client.0.smithi022.stdout:4/57: stat d4/c15 0 2024-02-16T19:28:51.870 INFO:tasks.workunit.client.0.smithi022.stdout:7/49: rmdir d9/da 39 2024-02-16T19:28:51.872 INFO:tasks.workunit.client.0.smithi022.stdout:1/100: symlink d6/d8/df/l1f 0 2024-02-16T19:28:51.872 INFO:tasks.workunit.client.0.smithi022.stdout:1/101: chown d6/f1c 11921 1 2024-02-16T19:28:51.873 INFO:tasks.workunit.client.0.smithi022.stdout:9/36: chown ca 13359424 1 2024-02-16T19:28:51.875 INFO:tasks.workunit.client.0.smithi022.stdout:0/71: fsync d10/f19 0 2024-02-16T19:28:51.875 INFO:tasks.workunit.client.0.smithi022.stdout:4/58: getdents d4/d6/d14 0 2024-02-16T19:28:51.875 INFO:tasks.workunit.client.0.smithi022.stdout:0/72: readlink lf 0 2024-02-16T19:28:51.876 INFO:tasks.workunit.client.0.smithi022.stdout:4/59: chown d4/c15 37534432 1 2024-02-16T19:28:51.878 INFO:tasks.workunit.client.0.smithi022.stdout:9/37: rename l4 to db/l13 0 2024-02-16T19:28:51.880 INFO:tasks.workunit.client.0.smithi022.stdout:0/73: mknod d10/d15/c1a 0 2024-02-16T19:28:51.881 INFO:tasks.workunit.client.0.smithi022.stdout:1/102: dwrite d6/d8/df/d16/d17/f1a [0,4194304] 0 2024-02-16T19:28:51.884 INFO:tasks.workunit.client.0.smithi022.stdout:7/50: dread f5 [0,4194304] 0 2024-02-16T19:28:51.885 INFO:tasks.workunit.client.0.smithi022.stdout:9/38: mkdir db/d14 0 2024-02-16T19:28:51.889 INFO:tasks.workunit.client.0.smithi022.stdout:1/103: rmdir d6/d8/df 39 2024-02-16T19:28:51.890 INFO:tasks.workunit.client.0.smithi022.stdout:4/60: dwrite d4/d6/f16 [0,4194304] 0 2024-02-16T19:28:51.892 INFO:tasks.workunit.client.0.smithi022.stdout:9/39: rmdir db/dd 39 2024-02-16T19:28:51.894 INFO:tasks.workunit.client.0.smithi022.stdout:1/104: read d6/f9 [397966,76401] 0 2024-02-16T19:28:51.894 INFO:tasks.workunit.client.0.smithi022.stdout:7/51: creat d9/dd/f11 x:0 0 0 2024-02-16T19:28:51.896 INFO:tasks.workunit.client.0.smithi022.stdout:0/74: dwrite d10/f19 [0,4194304] 0 2024-02-16T19:28:51.896 INFO:tasks.workunit.client.0.smithi022.stdout:9/40: chown f3 236965464 1 2024-02-16T19:28:51.896 INFO:tasks.workunit.client.0.smithi022.stdout:1/105: chown d6/f1c 1 1 2024-02-16T19:28:51.896 INFO:tasks.workunit.client.0.smithi022.stdout:1/106: fdatasync d6/f19 0 2024-02-16T19:28:51.898 INFO:tasks.workunit.client.0.smithi022.stdout:1/107: write d6/d8/f10 [573095,13773] 0 2024-02-16T19:28:51.898 INFO:tasks.workunit.client.0.smithi022.stdout:9/41: chown l6 29151 1 2024-02-16T19:28:51.899 INFO:tasks.workunit.client.0.smithi022.stdout:7/52: symlink d9/de/l12 0 2024-02-16T19:28:51.899 INFO:tasks.workunit.client.0.smithi022.stdout:1/108: chown d6/f19 13983654 1 2024-02-16T19:28:51.900 INFO:tasks.workunit.client.0.smithi022.stdout:0/75: mkdir d10/d15/d1b 0 2024-02-16T19:28:51.903 INFO:tasks.workunit.client.0.smithi022.stdout:1/109: fdatasync d6/d8/df/d16/d17/f1a 0 2024-02-16T19:28:51.904 INFO:tasks.workunit.client.0.smithi022.stdout:0/76: write fb [1310356,113122] 0 2024-02-16T19:28:51.904 INFO:tasks.workunit.client.0.smithi022.stdout:0/77: readlink l6 0 2024-02-16T19:28:51.906 INFO:tasks.workunit.client.0.smithi022.stdout:7/53: rename d9/dd/f11 to d9/da/f13 0 2024-02-16T19:28:51.906 INFO:tasks.workunit.client.0.smithi022.stdout:0/78: write d10/f13 [570200,100602] 0 2024-02-16T19:28:51.907 INFO:tasks.workunit.client.0.smithi022.stdout:0/79: read - d10/f17 zero size 2024-02-16T19:28:51.909 INFO:tasks.workunit.client.0.smithi022.stdout:9/42: link f3 db/dd/f15 0 2024-02-16T19:28:51.912 INFO:tasks.workunit.client.0.smithi022.stdout:9/43: rmdir db/dd 39 2024-02-16T19:28:51.916 INFO:tasks.workunit.client.0.smithi022.stdout:0/80: creat d10/d15/d1b/f1c x:0 0 0 2024-02-16T19:28:51.917 INFO:tasks.workunit.client.0.smithi022.stdout:9/44: dread f3 [0,4194304] 0 2024-02-16T19:28:51.920 INFO:tasks.workunit.client.0.smithi022.stdout:7/54: dwrite d9/da/f13 [0,4194304] 0 2024-02-16T19:28:51.920 INFO:tasks.workunit.client.0.smithi022.stdout:1/110: truncate d6/d8/f10 1822718 0 2024-02-16T19:28:51.921 INFO:tasks.workunit.client.0.smithi022.stdout:1/111: write d6/f19 [64493,53585] 0 2024-02-16T19:28:51.924 INFO:tasks.workunit.client.0.smithi022.stdout:7/55: dread f4 [0,4194304] 0 2024-02-16T19:28:51.925 INFO:tasks.workunit.client.0.smithi022.stdout:1/112: write d6/d8/df/f11 [105708,89783] 0 2024-02-16T19:28:51.925 INFO:tasks.workunit.client.0.smithi022.stdout:0/81: rmdir d10/d15/d18 0 2024-02-16T19:28:51.926 INFO:tasks.workunit.client.0.smithi022.stdout:9/45: mkdir db/d14/d16 0 2024-02-16T19:28:51.926 INFO:tasks.workunit.client.0.smithi022.stdout:0/82: fdatasync f7 0 2024-02-16T19:28:51.929 INFO:tasks.workunit.client.0.smithi022.stdout:1/113: truncate d6/d8/f12 1679715 0 2024-02-16T19:28:51.932 INFO:tasks.workunit.client.0.smithi022.stdout:1/114: stat d6/d8/df/d16/d17/f1a 0 2024-02-16T19:28:51.934 INFO:tasks.workunit.client.0.smithi022.stdout:1/115: dread - d6/d8/df/f15 zero size 2024-02-16T19:28:51.936 INFO:tasks.workunit.client.0.smithi022.stdout:9/46: dwrite db/fc [0,4194304] 0 2024-02-16T19:28:51.937 INFO:tasks.workunit.client.0.smithi022.stdout:0/83: dread d10/f19 [0,4194304] 0 2024-02-16T19:28:51.937 INFO:tasks.workunit.client.0.smithi022.stdout:9/47: readlink db/l13 0 2024-02-16T19:28:51.938 INFO:tasks.workunit.client.0.smithi022.stdout:0/84: read f7 [1512675,128913] 0 2024-02-16T19:28:51.939 INFO:tasks.workunit.client.0.smithi022.stdout:7/56: truncate f5 1412173 0 2024-02-16T19:28:51.942 INFO:tasks.workunit.client.0.smithi022.stdout:9/48: chown db/dd/l10 681 1 2024-02-16T19:28:51.943 INFO:tasks.workunit.client.0.smithi022.stdout:9/49: write db/f12 [160788,65081] 0 2024-02-16T19:28:51.946 INFO:tasks.workunit.client.0.smithi022.stdout:1/116: getdents d6 0 2024-02-16T19:28:51.948 INFO:tasks.workunit.client.0.smithi022.stdout:0/85: dwrite d10/f13 [0,4194304] 0 2024-02-16T19:28:51.948 INFO:tasks.workunit.client.0.smithi022.stdout:1/117: read - d6/d8/df/d16/f1e zero size 2024-02-16T19:28:51.950 INFO:tasks.workunit.client.0.smithi022.stdout:9/50: unlink l0 0 2024-02-16T19:28:51.952 INFO:tasks.workunit.client.0.smithi022.stdout:1/118: symlink d6/d8/df/d16/l20 0 2024-02-16T19:28:51.954 INFO:tasks.workunit.client.0.smithi022.stdout:7/57: creat d9/f14 x:0 0 0 2024-02-16T19:28:51.955 INFO:tasks.workunit.client.0.smithi022.stdout:7/58: dread - d9/f14 zero size 2024-02-16T19:28:51.956 INFO:tasks.workunit.client.0.smithi022.stdout:0/86: getdents d10/d15 0 2024-02-16T19:28:51.958 INFO:tasks.workunit.client.0.smithi022.stdout:1/119: creat d6/d8/df/d16/d17/d1d/f21 x:0 0 0 2024-02-16T19:28:51.959 INFO:tasks.workunit.client.0.smithi022.stdout:9/51: dwrite f5 [0,4194304] 0 2024-02-16T19:28:51.960 INFO:tasks.workunit.client.0.smithi022.stdout:7/59: symlink d9/da/l15 0 2024-02-16T19:28:51.961 INFO:tasks.workunit.client.0.smithi022.stdout:0/87: dread - d10/d15/d1b/f1c zero size 2024-02-16T19:28:51.962 INFO:tasks.workunit.client.0.smithi022.stdout:7/60: read f4 [278278,50497] 0 2024-02-16T19:28:51.965 INFO:tasks.workunit.client.0.smithi022.stdout:9/52: creat db/dd/f17 x:0 0 0 2024-02-16T19:28:51.966 INFO:tasks.workunit.client.0.smithi022.stdout:9/53: chown db/ff 316 1 2024-02-16T19:28:51.966 INFO:tasks.workunit.client.0.smithi022.stdout:9/54: stat f3 0 2024-02-16T19:28:51.968 INFO:tasks.workunit.client.0.smithi022.stdout:0/88: getdents d10/d15 0 2024-02-16T19:28:51.969 INFO:tasks.workunit.client.0.smithi022.stdout:7/61: creat d9/f16 x:0 0 0 2024-02-16T19:28:51.970 INFO:tasks.workunit.client.0.smithi022.stdout:9/55: mkdir db/d18 0 2024-02-16T19:28:51.971 INFO:tasks.workunit.client.0.smithi022.stdout:7/62: write d9/f16 [804125,96029] 0 2024-02-16T19:28:51.972 INFO:tasks.workunit.client.0.smithi022.stdout:9/56: dread f3 [0,4194304] 0 2024-02-16T19:28:51.972 INFO:tasks.workunit.client.0.smithi022.stdout:0/89: dread d10/f19 [0,4194304] 0 2024-02-16T19:28:51.973 INFO:tasks.workunit.client.0.smithi022.stdout:7/63: creat d9/dd/f17 x:0 0 0 2024-02-16T19:28:51.974 INFO:tasks.workunit.client.0.smithi022.stdout:7/64: chown c7 30265367 1 2024-02-16T19:28:51.974 INFO:tasks.workunit.client.0.smithi022.stdout:0/90: write d10/d11/f16 [16690,3489] 0 2024-02-16T19:28:51.975 INFO:tasks.workunit.client.0.smithi022.stdout:7/65: creat d9/de/f18 x:0 0 0 2024-02-16T19:28:51.978 INFO:tasks.workunit.client.0.smithi022.stdout:0/91: getdents d10/d11 0 2024-02-16T19:28:51.979 INFO:tasks.workunit.client.0.smithi022.stdout:0/92: write d10/d11/f16 [54818,42693] 0 2024-02-16T19:28:51.979 INFO:tasks.workunit.client.0.smithi022.stdout:0/93: fdatasync d10/f13 0 2024-02-16T19:28:51.981 INFO:tasks.workunit.client.0.smithi022.stdout:7/66: fdatasync d9/f16 0 2024-02-16T19:28:51.981 INFO:tasks.workunit.client.0.smithi022.stdout:7/67: chown d9/da 2118589722 1 2024-02-16T19:28:51.982 INFO:tasks.workunit.client.0.smithi022.stdout:7/68: dread - d9/de/f18 zero size 2024-02-16T19:28:51.982 INFO:tasks.workunit.client.0.smithi022.stdout:1/120: rename d6/d8/df to d6/d8/d22 0 2024-02-16T19:28:51.983 INFO:tasks.workunit.client.0.smithi022.stdout:0/94: dread f7 [0,4194304] 0 2024-02-16T19:28:51.983 INFO:tasks.workunit.client.0.smithi022.stdout:1/121: stat d6/d8/d22/f1b 0 2024-02-16T19:28:51.984 INFO:tasks.workunit.client.0.smithi022.stdout:0/95: truncate fb 1930295 0 2024-02-16T19:28:51.984 INFO:tasks.workunit.client.0.smithi022.stdout:0/96: readlink le 0 2024-02-16T19:28:51.986 INFO:tasks.workunit.client.0.smithi022.stdout:1/122: read d6/d8/f12 [1321555,7136] 0 2024-02-16T19:28:51.987 INFO:tasks.workunit.client.0.smithi022.stdout:9/57: rename db/dd to db/d14/d19 0 2024-02-16T19:28:51.989 INFO:tasks.workunit.client.0.smithi022.stdout:9/58: dread f3 [0,4194304] 0 2024-02-16T19:28:51.989 INFO:tasks.workunit.client.0.smithi022.stdout:0/97: creat d10/d15/f1d x:0 0 0 2024-02-16T19:28:51.990 INFO:tasks.workunit.client.0.smithi022.stdout:2/38: sync 2024-02-16T19:28:51.990 INFO:tasks.workunit.client.0.smithi022.stdout:2/39: fdatasync - no filename 2024-02-16T19:28:51.991 INFO:tasks.workunit.client.0.smithi022.stdout:1/123: rename d6/ce to d6/d8/d22/d16/d17/d1d/c23 0 2024-02-16T19:28:51.993 INFO:tasks.workunit.client.0.smithi022.stdout:9/59: unlink db/ff 0 2024-02-16T19:28:51.993 INFO:tasks.workunit.client.0.smithi022.stdout:9/60: stat db/d14 0 2024-02-16T19:28:51.993 INFO:tasks.workunit.client.0.smithi022.stdout:9/61: chown db/l13 102920723 1 2024-02-16T19:28:51.994 INFO:tasks.workunit.client.0.smithi022.stdout:9/62: write db/f12 [635495,13509] 0 2024-02-16T19:28:52.014 INFO:tasks.workunit.client.0.smithi022.stdout:9/63: rmdir db 39 2024-02-16T19:28:52.015 INFO:tasks.workunit.client.0.smithi022.stdout:0/98: truncate d10/f19 3160566 0 2024-02-16T19:28:52.016 INFO:tasks.workunit.client.0.smithi022.stdout:7/69: dwrite f4 [0,4194304] 0 2024-02-16T19:28:52.018 INFO:tasks.workunit.client.0.smithi022.stdout:0/99: truncate d10/d11/f16 139951 0 2024-02-16T19:28:52.018 INFO:tasks.workunit.client.0.smithi022.stdout:2/40: link d0/c8 d0/c9 0 2024-02-16T19:28:52.019 INFO:tasks.workunit.client.0.smithi022.stdout:7/70: write d9/dd/f17 [783962,88805] 0 2024-02-16T19:28:52.020 INFO:tasks.workunit.client.0.smithi022.stdout:7/71: readlink l0 0 2024-02-16T19:28:52.025 INFO:tasks.workunit.client.0.smithi022.stdout:2/41: mknod d0/d2/ca 0 2024-02-16T19:28:52.025 INFO:tasks.workunit.client.0.smithi022.stdout:2/42: dwrite - no filename 2024-02-16T19:28:52.025 INFO:tasks.workunit.client.0.smithi022.stdout:2/43: dread - no filename 2024-02-16T19:28:52.029 INFO:tasks.workunit.client.0.smithi022.stdout:9/64: dwrite db/d14/d19/f17 [0,4194304] 0 2024-02-16T19:28:52.030 INFO:tasks.workunit.client.0.smithi022.stdout:1/124: rename l5 to d6/d8/d22/d16/l24 0 2024-02-16T19:28:52.030 INFO:tasks.workunit.client.0.smithi022.stdout:0/100: getdents d10/d11 0 2024-02-16T19:28:52.031 INFO:tasks.workunit.client.0.smithi022.stdout:0/101: fsync d10/f17 0 2024-02-16T19:28:52.031 INFO:tasks.workunit.client.0.smithi022.stdout:1/125: dread - d6/d8/d22/f1b zero size 2024-02-16T19:28:52.033 INFO:tasks.workunit.client.0.smithi022.stdout:0/102: write d10/f17 [491330,88229] 0 2024-02-16T19:28:52.033 INFO:tasks.workunit.client.0.smithi022.stdout:2/44: creat d0/fb x:0 0 0 2024-02-16T19:28:52.035 INFO:tasks.workunit.client.0.smithi022.stdout:7/72: symlink d9/l19 0 2024-02-16T19:28:52.035 INFO:tasks.workunit.client.0.smithi022.stdout:2/45: truncate d0/fb 2823 0 2024-02-16T19:28:52.035 INFO:tasks.workunit.client.0.smithi022.stdout:7/73: read - d9/f14 zero size 2024-02-16T19:28:52.036 INFO:tasks.workunit.client.0.smithi022.stdout:7/74: truncate d9/de/f18 663438 0 2024-02-16T19:28:52.037 INFO:tasks.workunit.client.0.smithi022.stdout:9/65: mkdir db/d14/d19/d1a 0 2024-02-16T19:28:52.044 INFO:tasks.workunit.client.0.smithi022.stdout:1/126: rename d6/d8/d22/d16/l24 to d6/l25 0 2024-02-16T19:28:52.044 INFO:tasks.workunit.client.0.smithi022.stdout:2/46: getdents d0/d2 0 2024-02-16T19:28:52.047 INFO:tasks.workunit.client.0.smithi022.stdout:1/127: write d6/d8/d22/d16/f1e [993541,74370] 0 2024-02-16T19:28:52.051 INFO:tasks.workunit.client.0.smithi022.stdout:9/66: fsync db/d14/d19/f17 0 2024-02-16T19:28:52.053 INFO:tasks.workunit.client.0.smithi022.stdout:2/47: dwrite d0/fb [0,4194304] 0 2024-02-16T19:28:52.054 INFO:tasks.workunit.client.0.smithi022.stdout:9/67: rename l9 to db/d14/d19/d1a/l1b 0 2024-02-16T19:28:52.058 INFO:tasks.workunit.client.0.smithi022.stdout:1/128: write d6/d8/f10 [2739034,37873] 0 2024-02-16T19:28:52.059 INFO:tasks.workunit.client.0.smithi022.stdout:1/129: write d6/d8/f10 [947746,47316] 0 2024-02-16T19:28:52.061 INFO:tasks.workunit.client.0.smithi022.stdout:1/130: chown d6/f1c 57897468 1 2024-02-16T19:28:52.062 INFO:tasks.workunit.client.0.smithi022.stdout:9/68: truncate f7 1284550 0 2024-02-16T19:28:52.064 INFO:tasks.workunit.client.0.smithi022.stdout:1/131: mkdir d6/d8/d22/d16/d17/d26 0 2024-02-16T19:28:52.066 INFO:tasks.workunit.client.0.smithi022.stdout:9/69: creat db/d14/d16/f1c x:0 0 0 2024-02-16T19:28:52.068 INFO:tasks.workunit.client.0.smithi022.stdout:1/132: getdents d6/d8/d22/d16/d17/d1d 0 2024-02-16T19:28:52.069 INFO:tasks.workunit.client.0.smithi022.stdout:2/48: fsync d0/fb 0 2024-02-16T19:28:52.069 INFO:tasks.workunit.client.0.smithi022.stdout:1/133: write d6/d8/d22/d16/d17/f1a [2235594,6906] 0 2024-02-16T19:28:52.070 INFO:tasks.workunit.client.0.smithi022.stdout:9/70: rename f3 to db/f1d 0 2024-02-16T19:28:52.071 INFO:tasks.workunit.client.0.smithi022.stdout:1/134: chown d6/d8/d22/d16 3 1 2024-02-16T19:28:52.071 INFO:tasks.workunit.client.0.smithi022.stdout:2/49: symlink d0/lc 0 2024-02-16T19:28:52.072 INFO:tasks.workunit.client.0.smithi022.stdout:9/71: symlink db/d18/l1e 0 2024-02-16T19:28:52.072 INFO:tasks.workunit.client.0.smithi022.stdout:9/72: fdatasync db/f12 0 2024-02-16T19:28:52.073 INFO:tasks.workunit.client.0.smithi022.stdout:2/50: mknod d0/cd 0 2024-02-16T19:28:52.074 INFO:tasks.workunit.client.0.smithi022.stdout:9/73: creat db/d14/d16/f1f x:0 0 0 2024-02-16T19:28:52.076 INFO:tasks.workunit.client.0.smithi022.stdout:1/135: rename d6/d8/d22/d16/l20 to d6/l27 0 2024-02-16T19:28:52.077 INFO:tasks.workunit.client.0.smithi022.stdout:1/136: write d6/d8/f10 [2528554,62698] 0 2024-02-16T19:28:52.078 INFO:tasks.workunit.client.0.smithi022.stdout:1/137: dread - d6/f1c zero size 2024-02-16T19:28:52.079 INFO:tasks.workunit.client.0.smithi022.stdout:2/51: truncate d0/fb 1128489 0 2024-02-16T19:28:52.082 INFO:tasks.workunit.client.0.smithi022.stdout:9/74: link db/f12 db/d14/d19/f20 0 2024-02-16T19:28:52.083 INFO:tasks.workunit.client.0.smithi022.stdout:9/75: truncate db/d14/d16/f1c 309227 0 2024-02-16T19:28:52.084 INFO:tasks.workunit.client.0.smithi022.stdout:1/138: rename d6/d8/f10 to d6/d8/d22/f28 0 2024-02-16T19:28:52.087 INFO:tasks.workunit.client.0.smithi022.stdout:9/76: write db/d14/d19/f20 [35197,71417] 0 2024-02-16T19:28:52.090 INFO:tasks.workunit.client.0.smithi022.stdout:1/139: mknod d6/d8/d22/d16/c29 0 2024-02-16T19:28:52.090 INFO:tasks.workunit.client.0.smithi022.stdout:1/140: stat d6/d8/d22/f28 0 2024-02-16T19:28:52.091 INFO:tasks.workunit.client.0.smithi022.stdout:9/77: mkdir db/d14/d16/d21 0 2024-02-16T19:28:52.094 INFO:tasks.workunit.client.0.smithi022.stdout:9/78: rmdir db/d14 39 2024-02-16T19:28:52.095 INFO:tasks.workunit.client.0.smithi022.stdout:1/141: mknod d6/d8/c2a 0 2024-02-16T19:28:52.096 INFO:tasks.workunit.client.0.smithi022.stdout:1/142: stat d6/d8/d22/d16/f1e 0 2024-02-16T19:28:52.097 INFO:tasks.workunit.client.0.smithi022.stdout:1/143: fdatasync d6/d8/d22/d16/f1e 0 2024-02-16T19:28:52.098 INFO:tasks.workunit.client.0.smithi022.stdout:2/52: rename d0/fb to d0/d2/fe 0 2024-02-16T19:28:52.098 INFO:tasks.workunit.client.0.smithi022.stdout:9/79: rename db to db/d14/d16/d22 22 2024-02-16T19:28:52.100 INFO:tasks.workunit.client.0.smithi022.stdout:1/144: truncate d6/d8/d22/d16/d17/d1d/f21 615671 0 2024-02-16T19:28:52.101 INFO:tasks.workunit.client.0.smithi022.stdout:2/53: rename d0/d2/ca to d0/d2/cf 0 2024-02-16T19:28:52.102 INFO:tasks.workunit.client.0.smithi022.stdout:1/145: dread d6/d8/f12 [0,4194304] 0 2024-02-16T19:28:52.103 INFO:tasks.workunit.client.0.smithi022.stdout:9/80: mknod db/d14/d19/d1a/c23 0 2024-02-16T19:28:52.105 INFO:tasks.workunit.client.0.smithi022.stdout:1/146: readlink l2 0 2024-02-16T19:28:52.106 INFO:tasks.workunit.client.0.smithi022.stdout:1/147: write d6/d8/d22/f11 [134369,121046] 0 2024-02-16T19:28:52.107 INFO:tasks.workunit.client.0.smithi022.stdout:1/148: write d6/d8/d22/f11 [839609,44776] 0 2024-02-16T19:28:52.107 INFO:tasks.workunit.client.0.smithi022.stdout:9/81: unlink db/ce 0 2024-02-16T19:28:52.110 INFO:tasks.workunit.client.0.smithi022.stdout:1/149: chown d6/la 673129506 1 2024-02-16T19:28:52.112 INFO:tasks.workunit.client.0.smithi022.stdout:9/82: dread db/d14/d19/f15 [0,4194304] 0 2024-02-16T19:28:52.114 INFO:tasks.workunit.client.0.smithi022.stdout:2/54: dread d0/d2/fe [0,4194304] 0 2024-02-16T19:28:52.114 INFO:tasks.workunit.client.0.smithi022.stdout:9/83: creat db/d14/d19/d1a/f24 x:0 0 0 2024-02-16T19:28:52.115 INFO:tasks.workunit.client.0.smithi022.stdout:1/150: getdents d6/d8/d22/d16/d17/d1d 0 2024-02-16T19:28:52.116 INFO:tasks.workunit.client.0.smithi022.stdout:1/151: write d6/f1c [798469,115788] 0 2024-02-16T19:28:52.120 INFO:tasks.workunit.client.0.smithi022.stdout:1/152: link d6/d8/cb d6/d8/d22/d16/d17/d1d/c2b 0 2024-02-16T19:28:52.121 INFO:tasks.workunit.client.0.smithi022.stdout:9/84: rmdir db 39 2024-02-16T19:28:52.125 INFO:tasks.workunit.client.0.smithi022.stdout:9/85: rename db/d14/d19/l10 to db/l25 0 2024-02-16T19:28:52.126 INFO:tasks.workunit.client.0.smithi022.stdout:1/153: getdents d6/d8/d22/d16/d17/d1d 0 2024-02-16T19:28:52.127 INFO:tasks.workunit.client.0.smithi022.stdout:1/154: write d6/d8/d22/f1b [700658,15250] 0 2024-02-16T19:28:52.127 INFO:tasks.workunit.client.0.smithi022.stdout:9/86: creat db/d18/f26 x:0 0 0 2024-02-16T19:28:52.129 INFO:tasks.workunit.client.0.smithi022.stdout:9/87: write db/d14/d19/f17 [322906,74130] 0 2024-02-16T19:28:52.130 INFO:tasks.workunit.client.0.smithi022.stdout:1/155: fdatasync d6/f9 0 2024-02-16T19:28:52.130 INFO:tasks.workunit.client.0.smithi022.stdout:9/88: truncate db/d14/d16/f1c 536318 0 2024-02-16T19:28:52.132 INFO:tasks.workunit.client.0.smithi022.stdout:1/156: symlink d6/d8/d22/l2c 0 2024-02-16T19:28:52.137 INFO:tasks.workunit.client.0.smithi022.stdout:9/89: dwrite db/f12 [0,4194304] 0 2024-02-16T19:28:52.140 INFO:tasks.workunit.client.0.smithi022.stdout:1/157: dwrite d6/f1c [0,4194304] 0 2024-02-16T19:28:52.141 INFO:tasks.workunit.client.0.smithi022.stdout:9/90: mkdir db/d14/d16/d27 0 2024-02-16T19:28:52.145 INFO:tasks.workunit.client.0.smithi022.stdout:9/91: mkdir db/d14/d16/d28 0 2024-02-16T19:28:52.147 INFO:tasks.workunit.client.0.smithi022.stdout:1/158: creat d6/d8/f2d x:0 0 0 2024-02-16T19:28:52.148 INFO:tasks.workunit.client.0.smithi022.stdout:1/159: read d6/f9 [87601,82031] 0 2024-02-16T19:28:52.149 INFO:tasks.workunit.client.0.smithi022.stdout:9/92: getdents db/d14/d19/d1a 0 2024-02-16T19:28:52.190 INFO:tasks.workunit.client.0.smithi022.stdout:6/58: sync 2024-02-16T19:28:52.192 INFO:tasks.workunit.client.0.smithi022.stdout:6/59: creat d1/d8/da/f14 x:0 0 0 2024-02-16T19:28:52.192 INFO:tasks.workunit.client.0.smithi022.stdout:6/60: chown d1/f2 5836 1 2024-02-16T19:28:52.193 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:51 smithi117 ceph-mon[35309]: pgmap v257: 65 pgs: 65 active+clean; 323 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 251 KiB/s rd, 18 MiB/s wr, 299 op/s 2024-02-16T19:28:52.193 INFO:tasks.workunit.client.0.smithi022.stdout:6/61: readlink d1/d8/da/le 0 2024-02-16T19:28:52.193 INFO:tasks.workunit.client.0.smithi022.stdout:6/62: symlink d1/l15 0 2024-02-16T19:28:52.194 INFO:tasks.workunit.client.0.smithi022.stdout:6/63: truncate d1/d8/f11 71622 0 2024-02-16T19:28:52.194 INFO:tasks.workunit.client.0.smithi022.stdout:6/64: chown d1/d8 18 1 2024-02-16T19:28:52.200 INFO:tasks.workunit.client.0.smithi022.stdout:6/65: getdents d1/d8/da 0 2024-02-16T19:28:52.200 INFO:tasks.workunit.client.0.smithi022.stdout:6/66: write d1/d8/da/f14 [402791,24650] 0 2024-02-16T19:28:52.207 INFO:tasks.workunit.client.0.smithi022.stdout:6/67: dwrite d1/d8/da/f14 [0,4194304] 0 2024-02-16T19:28:52.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:51 smithi022 ceph-mon[29221]: pgmap v257: 65 pgs: 65 active+clean; 323 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 251 KiB/s rd, 18 MiB/s wr, 299 op/s 2024-02-16T19:28:52.288 INFO:tasks.workunit.client.0.smithi022.stdout:6/68: mknod d1/d8/c16 0 2024-02-16T19:28:52.289 INFO:tasks.workunit.client.0.smithi022.stdout:6/69: truncate d1/d8/f13 406289 0 2024-02-16T19:28:52.289 INFO:tasks.workunit.client.0.smithi022.stdout:6/70: write d1/f2 [1252846,34313] 0 2024-02-16T19:28:52.315 INFO:tasks.workunit.client.0.smithi022.stdout:8/59: sync 2024-02-16T19:28:52.438 INFO:tasks.workunit.client.0.smithi022.stdout:5/67: sync 2024-02-16T19:28:52.498 INFO:tasks.workunit.client.0.smithi022.stdout:0/103: sync 2024-02-16T19:28:52.498 INFO:tasks.workunit.client.0.smithi022.stdout:3/64: sync 2024-02-16T19:28:52.498 INFO:tasks.workunit.client.0.smithi022.stdout:4/61: sync 2024-02-16T19:28:52.501 INFO:tasks.workunit.client.0.smithi022.stdout:6/71: mkdir d1/d8/da/d17 0 2024-02-16T19:28:52.501 INFO:tasks.workunit.client.0.smithi022.stdout:8/60: mknod d3/c14 0 2024-02-16T19:28:52.502 INFO:tasks.workunit.client.0.smithi022.stdout:5/68: mkdir dd/d14 0 2024-02-16T19:28:52.503 INFO:tasks.workunit.client.0.smithi022.stdout:3/65: symlink d7/l1d 0 2024-02-16T19:28:52.504 INFO:tasks.workunit.client.0.smithi022.stdout:6/72: truncate d1/f2 1787365 0 2024-02-16T19:28:52.504 INFO:tasks.workunit.client.0.smithi022.stdout:8/61: read d3/d6/f7 [1215122,37742] 0 2024-02-16T19:28:52.507 INFO:tasks.workunit.client.0.smithi022.stdout:3/66: write d7/d12/d18/f1b [53808,36998] 0 2024-02-16T19:28:52.509 INFO:tasks.workunit.client.0.smithi022.stdout:8/62: write d3/f10 [405720,37527] 0 2024-02-16T19:28:52.511 INFO:tasks.workunit.client.0.smithi022.stdout:0/104: dwrite d10/f13 [0,4194304] 0 2024-02-16T19:28:52.512 INFO:tasks.workunit.client.0.smithi022.stdout:4/62: dwrite d4/d6/dd/f12 [0,4194304] 0 2024-02-16T19:28:52.513 INFO:tasks.workunit.client.0.smithi022.stdout:3/67: creat d7/f1e x:0 0 0 2024-02-16T19:28:52.513 INFO:tasks.workunit.client.0.smithi022.stdout:6/73: dread d1/f2 [0,4194304] 0 2024-02-16T19:28:52.515 INFO:tasks.workunit.client.0.smithi022.stdout:8/63: fdatasync d3/d6/f7 0 2024-02-16T19:28:52.517 INFO:tasks.workunit.client.0.smithi022.stdout:0/105: creat d10/d15/f1e x:0 0 0 2024-02-16T19:28:52.518 INFO:tasks.workunit.client.0.smithi022.stdout:5/69: dwrite f8 [0,4194304] 0 2024-02-16T19:28:52.518 INFO:tasks.workunit.client.0.smithi022.stdout:6/74: creat d1/f18 x:0 0 0 2024-02-16T19:28:52.519 INFO:tasks.workunit.client.0.smithi022.stdout:5/70: rename dd to dd/d15 22 2024-02-16T19:28:52.521 INFO:tasks.workunit.client.0.smithi022.stdout:3/68: rename d7/l1d to d7/l1f 0 2024-02-16T19:28:52.525 INFO:tasks.workunit.client.0.smithi022.stdout:8/64: dread d3/d6/f7 [0,4194304] 0 2024-02-16T19:28:52.526 INFO:tasks.workunit.client.0.smithi022.stdout:4/63: creat d4/d6/dd/df/d10/f19 x:0 0 0 2024-02-16T19:28:52.528 INFO:tasks.workunit.client.0.smithi022.stdout:0/106: creat d10/f1f x:0 0 0 2024-02-16T19:28:52.528 INFO:tasks.workunit.client.0.smithi022.stdout:6/75: fdatasync d1/d8/ff 0 2024-02-16T19:28:52.530 INFO:tasks.workunit.client.0.smithi022.stdout:6/76: chown d1/d8/c12 211 1 2024-02-16T19:28:52.530 INFO:tasks.workunit.client.0.smithi022.stdout:5/71: creat dd/f16 x:0 0 0 2024-02-16T19:28:52.531 INFO:tasks.workunit.client.0.smithi022.stdout:5/72: chown dd/l13 341 1 2024-02-16T19:28:52.532 INFO:tasks.workunit.client.0.smithi022.stdout:1/160: sync 2024-02-16T19:28:52.532 INFO:tasks.workunit.client.0.smithi022.stdout:2/55: sync 2024-02-16T19:28:52.532 INFO:tasks.workunit.client.0.smithi022.stdout:5/73: fsync fb 0 2024-02-16T19:28:52.533 INFO:tasks.workunit.client.0.smithi022.stdout:9/93: sync 2024-02-16T19:28:52.533 INFO:tasks.workunit.client.0.smithi022.stdout:7/75: sync 2024-02-16T19:28:52.535 INFO:tasks.workunit.client.0.smithi022.stdout:8/65: creat d3/f15 x:0 0 0 2024-02-16T19:28:52.535 INFO:tasks.workunit.client.0.smithi022.stdout:0/107: dread d10/f13 [0,4194304] 0 2024-02-16T19:28:52.536 INFO:tasks.workunit.client.0.smithi022.stdout:9/94: truncate f5 4194998 0 2024-02-16T19:28:52.536 INFO:tasks.workunit.client.0.smithi022.stdout:4/64: read d4/d6/f9 [1526893,15498] 0 2024-02-16T19:28:52.537 INFO:tasks.workunit.client.0.smithi022.stdout:8/66: write d3/f11 [177978,103530] 0 2024-02-16T19:28:52.538 INFO:tasks.workunit.client.0.smithi022.stdout:6/77: dread d1/f3 [0,4194304] 0 2024-02-16T19:28:52.540 INFO:tasks.workunit.client.0.smithi022.stdout:4/65: write d4/d6/dd/f12 [4750526,127995] 0 2024-02-16T19:28:52.542 INFO:tasks.workunit.client.0.smithi022.stdout:5/74: rename f4 to dd/f17 0 2024-02-16T19:28:52.542 INFO:tasks.workunit.client.0.smithi022.stdout:1/161: dread d6/d8/d22/d16/f1e [0,4194304] 0 2024-02-16T19:28:52.544 INFO:tasks.workunit.client.0.smithi022.stdout:1/162: fdatasync d6/d8/f2d 0 2024-02-16T19:28:52.546 INFO:tasks.workunit.client.0.smithi022.stdout:0/108: creat d10/d15/d1b/f20 x:0 0 0 2024-02-16T19:28:52.548 INFO:tasks.workunit.client.0.smithi022.stdout:0/109: write d10/d11/f16 [1054230,87214] 0 2024-02-16T19:28:52.548 INFO:tasks.workunit.client.0.smithi022.stdout:6/78: dwrite d1/d8/ff [0,4194304] 0 2024-02-16T19:28:52.549 INFO:tasks.workunit.client.0.smithi022.stdout:4/66: symlink d4/d6/dd/l1a 0 2024-02-16T19:28:52.552 INFO:tasks.workunit.client.0.smithi022.stdout:8/67: rename d3/c8 to d3/d6/c16 0 2024-02-16T19:28:52.555 INFO:tasks.workunit.client.0.smithi022.stdout:9/95: mknod db/c29 0 2024-02-16T19:28:52.556 INFO:tasks.workunit.client.0.smithi022.stdout:1/163: creat d6/d8/d22/d16/f2e x:0 0 0 2024-02-16T19:28:52.559 INFO:tasks.workunit.client.0.smithi022.stdout:0/110: symlink d10/d15/d1b/l21 0 2024-02-16T19:28:52.560 INFO:tasks.workunit.client.0.smithi022.stdout:4/67: mkdir d4/d6/dd/df/d1b 0 2024-02-16T19:28:52.563 INFO:tasks.workunit.client.0.smithi022.stdout:9/96: creat db/d18/f2a x:0 0 0 2024-02-16T19:28:52.564 INFO:tasks.workunit.client.0.smithi022.stdout:9/97: stat db/c11 0 2024-02-16T19:28:52.564 INFO:tasks.workunit.client.0.smithi022.stdout:1/164: dwrite d6/d8/d22/f11 [0,4194304] 0 2024-02-16T19:28:52.567 INFO:tasks.workunit.client.0.smithi022.stdout:8/68: mkdir d3/d12/d17 0 2024-02-16T19:28:52.571 INFO:tasks.workunit.client.0.smithi022.stdout:8/69: fdatasync d3/f10 0 2024-02-16T19:28:52.572 INFO:tasks.workunit.client.0.smithi022.stdout:1/165: symlink d6/d8/d22/d16/d17/l2f 0 2024-02-16T19:28:52.574 INFO:tasks.workunit.client.0.smithi022.stdout:4/68: dwrite d4/d6/dd/df/d10/f19 [0,4194304] 0 2024-02-16T19:28:52.575 INFO:tasks.workunit.client.0.smithi022.stdout:1/166: truncate d6/d8/d22/f15 772152 0 2024-02-16T19:28:52.576 INFO:tasks.workunit.client.0.smithi022.stdout:4/69: read d4/d6/dd/df/d10/f19 [2339075,128026] 0 2024-02-16T19:28:52.577 INFO:tasks.workunit.client.0.smithi022.stdout:9/98: dread db/fc [0,4194304] 0 2024-02-16T19:28:52.578 INFO:tasks.workunit.client.0.smithi022.stdout:4/70: stat d4/c15 0 2024-02-16T19:28:52.578 INFO:tasks.workunit.client.0.smithi022.stdout:8/70: getdents d3/d12/d17 0 2024-02-16T19:28:52.579 INFO:tasks.workunit.client.0.smithi022.stdout:9/99: write db/d14/d19/f17 [2179083,75957] 0 2024-02-16T19:28:52.580 INFO:tasks.workunit.client.0.smithi022.stdout:1/167: symlink d6/d8/d22/d16/d17/d1d/l30 0 2024-02-16T19:28:52.582 INFO:tasks.workunit.client.0.smithi022.stdout:1/168: chown d6/d8/f2d 18147 1 2024-02-16T19:28:52.584 INFO:tasks.workunit.client.0.smithi022.stdout:1/169: fdatasync d6/d8/d22/f11 0 2024-02-16T19:28:52.585 INFO:tasks.workunit.client.0.smithi022.stdout:8/71: creat d3/d12/d17/f18 x:0 0 0 2024-02-16T19:28:52.586 INFO:tasks.workunit.client.0.smithi022.stdout:4/71: dread d4/d6/f9 [4194304,4194304] 0 2024-02-16T19:28:52.587 INFO:tasks.workunit.client.0.smithi022.stdout:1/170: chown d6/d8/d22/f15 18 1 2024-02-16T19:28:52.589 INFO:tasks.workunit.client.0.smithi022.stdout:9/100: dwrite db/d14/d19/f17 [0,4194304] 0 2024-02-16T19:28:52.590 INFO:tasks.workunit.client.0.smithi022.stdout:1/171: chown d6/d8/d22/d16/d17/d26 23781 1 2024-02-16T19:28:52.592 INFO:tasks.workunit.client.0.smithi022.stdout:9/101: write db/fc [3095954,38382] 0 2024-02-16T19:28:52.594 INFO:tasks.workunit.client.0.smithi022.stdout:9/102: readlink db/d14/d19/d1a/l1b 0 2024-02-16T19:28:52.595 INFO:tasks.workunit.client.0.smithi022.stdout:1/172: dread d6/d8/d22/f11 [0,4194304] 0 2024-02-16T19:28:52.596 INFO:tasks.workunit.client.0.smithi022.stdout:1/173: read - d6/d8/d22/d16/f2e zero size 2024-02-16T19:28:52.599 INFO:tasks.workunit.client.0.smithi022.stdout:9/103: link db/c11 db/d14/d16/d28/c2b 0 2024-02-16T19:28:52.600 INFO:tasks.workunit.client.0.smithi022.stdout:1/174: mknod d6/d8/d22/d16/d17/d26/c31 0 2024-02-16T19:28:52.602 INFO:tasks.workunit.client.0.smithi022.stdout:9/104: symlink db/l2c 0 2024-02-16T19:28:52.604 INFO:tasks.workunit.client.0.smithi022.stdout:9/105: creat db/d14/f2d x:0 0 0 2024-02-16T19:28:52.605 INFO:tasks.workunit.client.0.smithi022.stdout:9/106: chown db/d14/d19/f20 421645576 1 2024-02-16T19:28:52.608 INFO:tasks.workunit.client.0.smithi022.stdout:9/107: dread db/f12 [0,4194304] 0 2024-02-16T19:28:52.610 INFO:tasks.workunit.client.0.smithi022.stdout:9/108: mkdir db/d14/d16/d27/d2e 0 2024-02-16T19:28:52.611 INFO:tasks.workunit.client.0.smithi022.stdout:9/109: write db/fc [3751070,18855] 0 2024-02-16T19:28:52.612 INFO:tasks.workunit.client.0.smithi022.stdout:9/110: dread - db/d14/f2d zero size 2024-02-16T19:28:52.685 INFO:tasks.workunit.client.0.smithi022.stdout:3/69: write d7/d12/d18/dd/f13 [661535,91744] 0 2024-02-16T19:28:52.690 INFO:tasks.workunit.client.0.smithi022.stdout:3/70: rename d7/d12/d18/dd/f13 to d7/d12/d18/dd/f20 0 2024-02-16T19:28:52.690 INFO:tasks.workunit.client.0.smithi022.stdout:2/56: dwrite d0/d2/fe [0,4194304] 0 2024-02-16T19:28:52.693 INFO:tasks.workunit.client.0.smithi022.stdout:3/71: mkdir d7/d21 0 2024-02-16T19:28:52.698 INFO:tasks.workunit.client.0.smithi022.stdout:7/76: truncate f5 270008 0 2024-02-16T19:28:52.700 INFO:tasks.workunit.client.0.smithi022.stdout:5/75: dwrite f8 [4194304,4194304] 0 2024-02-16T19:28:52.700 INFO:tasks.workunit.client.0.smithi022.stdout:5/76: stat fb 0 2024-02-16T19:28:52.701 INFO:tasks.workunit.client.0.smithi022.stdout:8/72: unlink d3/d6/c16 0 2024-02-16T19:28:52.701 INFO:tasks.workunit.client.0.smithi022.stdout:8/73: stat d3/d6/f7 0 2024-02-16T19:28:52.702 INFO:tasks.workunit.client.0.smithi022.stdout:8/74: chown d3/f10 266 1 2024-02-16T19:28:52.702 INFO:tasks.workunit.client.0.smithi022.stdout:2/57: link d0/lc d0/l10 0 2024-02-16T19:28:52.704 INFO:tasks.workunit.client.0.smithi022.stdout:3/72: creat d7/d21/f22 x:0 0 0 2024-02-16T19:28:52.707 INFO:tasks.workunit.client.0.smithi022.stdout:0/111: rename d10/d15/d1b to d10/d22 0 2024-02-16T19:28:52.709 INFO:tasks.workunit.client.0.smithi022.stdout:3/73: dread f5 [0,4194304] 0 2024-02-16T19:28:52.709 INFO:tasks.workunit.client.0.smithi022.stdout:5/77: rmdir dd 39 2024-02-16T19:28:52.711 INFO:tasks.workunit.client.0.smithi022.stdout:8/75: symlink d3/d12/d17/l19 0 2024-02-16T19:28:52.714 INFO:tasks.workunit.client.0.smithi022.stdout:2/58: unlink d0/d2/fe 0 2024-02-16T19:28:52.714 INFO:tasks.workunit.client.0.smithi022.stdout:2/59: fsync - no filename 2024-02-16T19:28:52.714 INFO:tasks.workunit.client.0.smithi022.stdout:2/60: dwrite - no filename 2024-02-16T19:28:52.715 INFO:tasks.workunit.client.0.smithi022.stdout:0/112: dwrite d10/d22/f20 [0,4194304] 0 2024-02-16T19:28:52.716 INFO:tasks.workunit.client.0.smithi022.stdout:0/113: chown d10/d22 116 1 2024-02-16T19:28:52.718 INFO:tasks.workunit.client.0.smithi022.stdout:9/111: rename db/d14/d16/f1c to db/d14/d19/f2f 0 2024-02-16T19:28:52.720 INFO:tasks.workunit.client.0.smithi022.stdout:3/74: creat d7/d21/f23 x:0 0 0 2024-02-16T19:28:52.723 INFO:tasks.workunit.client.0.smithi022.stdout:8/76: symlink d3/d6/l1a 0 2024-02-16T19:28:52.724 INFO:tasks.workunit.client.0.smithi022.stdout:6/79: sync 2024-02-16T19:28:52.725 INFO:tasks.workunit.client.0.smithi022.stdout:2/61: creat d0/f11 x:0 0 0 2024-02-16T19:28:52.726 INFO:tasks.workunit.client.0.smithi022.stdout:5/78: dwrite dd/f16 [0,4194304] 0 2024-02-16T19:28:52.728 INFO:tasks.workunit.client.0.smithi022.stdout:0/114: mkdir d10/d15/d23 0 2024-02-16T19:28:52.729 INFO:tasks.workunit.client.0.smithi022.stdout:4/72: write d4/d6/f9 [9268687,105435] 0 2024-02-16T19:28:52.744 INFO:tasks.workunit.client.0.smithi022.stdout:4/73: stat d4/d6/f9 0 2024-02-16T19:28:52.744 INFO:tasks.workunit.client.0.smithi022.stdout:8/77: mknod d3/d12/c1b 0 2024-02-16T19:28:52.744 INFO:tasks.workunit.client.0.smithi022.stdout:1/175: write d6/d8/f12 [1129012,128469] 0 2024-02-16T19:28:52.745 INFO:tasks.workunit.client.0.smithi022.stdout:2/62: mknod d0/c12 0 2024-02-16T19:28:52.746 INFO:tasks.workunit.client.0.smithi022.stdout:5/79: creat dd/f18 x:0 0 0 2024-02-16T19:28:52.748 INFO:tasks.workunit.client.0.smithi022.stdout:8/78: truncate d3/f11 1230985 0 2024-02-16T19:28:52.749 INFO:tasks.workunit.client.0.smithi022.stdout:7/77: write f5 [1108283,5952] 0 2024-02-16T19:28:52.752 INFO:tasks.workunit.client.0.smithi022.stdout:2/63: rmdir d0/d2 39 2024-02-16T19:28:52.753 INFO:tasks.workunit.client.0.smithi022.stdout:1/176: mknod d6/d8/d22/c32 0 2024-02-16T19:28:52.755 INFO:tasks.workunit.client.0.smithi022.stdout:4/74: mknod d4/d6/dd/df/c1c 0 2024-02-16T19:28:52.757 INFO:tasks.workunit.client.0.smithi022.stdout:9/112: dwrite db/f12 [4194304,4194304] 0 2024-02-16T19:28:52.758 INFO:tasks.workunit.client.0.smithi022.stdout:1/177: dread - d6/d8/f2d zero size 2024-02-16T19:28:52.759 INFO:tasks.workunit.client.0.smithi022.stdout:0/115: link l6 d10/d11/l24 0 2024-02-16T19:28:52.765 INFO:tasks.workunit.client.0.smithi022.stdout:5/80: dwrite dd/f16 [0,4194304] 0 2024-02-16T19:28:52.766 INFO:tasks.workunit.client.0.smithi022.stdout:6/80: truncate d1/f7 3114039 0 2024-02-16T19:28:52.766 INFO:tasks.workunit.client.0.smithi022.stdout:1/178: chown d6/d8/d22/d16/f1e 0 1 2024-02-16T19:28:52.766 INFO:tasks.workunit.client.0.smithi022.stdout:7/78: dwrite d9/f14 [0,4194304] 0 2024-02-16T19:28:52.767 INFO:tasks.workunit.client.0.smithi022.stdout:9/113: creat db/d14/d16/f30 x:0 0 0 2024-02-16T19:28:52.768 INFO:tasks.workunit.client.0.smithi022.stdout:5/81: stat l2 0 2024-02-16T19:28:52.768 INFO:tasks.workunit.client.0.smithi022.stdout:9/114: fdatasync db/d18/f2a 0 2024-02-16T19:28:52.770 INFO:tasks.workunit.client.0.smithi022.stdout:6/81: dread d1/d8/f11 [0,4194304] 0 2024-02-16T19:28:52.774 INFO:tasks.workunit.client.0.smithi022.stdout:7/79: mknod d9/da/c1a 0 2024-02-16T19:28:52.775 INFO:tasks.workunit.client.0.smithi022.stdout:6/82: chown d1/f3 1363451 1 2024-02-16T19:28:52.780 INFO:tasks.workunit.client.0.smithi022.stdout:5/82: fdatasync dd/f11 0 2024-02-16T19:28:52.783 INFO:tasks.workunit.client.0.smithi022.stdout:9/115: dwrite db/d14/d16/f1f [0,4194304] 0 2024-02-16T19:28:52.783 INFO:tasks.workunit.client.0.smithi022.stdout:1/179: dread d6/d8/d22/f28 [0,4194304] 0 2024-02-16T19:28:52.783 INFO:tasks.workunit.client.0.smithi022.stdout:6/83: symlink d1/d8/l19 0 2024-02-16T19:28:52.784 INFO:tasks.workunit.client.1.smithi117.stdout:9/1: sync 2024-02-16T19:28:52.785 INFO:tasks.workunit.client.0.smithi022.stdout:0/116: getdents d10 0 2024-02-16T19:28:52.787 INFO:tasks.workunit.client.0.smithi022.stdout:6/84: write d1/f18 [478031,31276] 0 2024-02-16T19:28:52.787 INFO:tasks.workunit.client.0.smithi022.stdout:7/80: mkdir d9/d1b 0 2024-02-16T19:28:52.788 INFO:tasks.workunit.client.0.smithi022.stdout:0/117: dread - d10/d15/f1e zero size 2024-02-16T19:28:52.788 INFO:tasks.workunit.client.0.smithi022.stdout:4/75: truncate d4/d6/dd/f12 3719049 0 2024-02-16T19:28:52.790 INFO:tasks.workunit.client.0.smithi022.stdout:7/81: read d9/dd/f17 [852071,120591] 0 2024-02-16T19:28:52.791 INFO:tasks.workunit.client.0.smithi022.stdout:5/83: dread dd/f17 [0,4194304] 0 2024-02-16T19:28:52.795 INFO:tasks.workunit.client.0.smithi022.stdout:6/85: rename d1/d8/lc to d1/l1a 0 2024-02-16T19:28:52.795 INFO:tasks.workunit.client.0.smithi022.stdout:7/82: symlink d9/de/l1c 0 2024-02-16T19:28:52.796 INFO:tasks.workunit.client.0.smithi022.stdout:6/86: stat d1/d8/da/f14 0 2024-02-16T19:28:52.796 INFO:tasks.workunit.client.0.smithi022.stdout:7/83: fsync d9/de/f18 0 2024-02-16T19:28:52.797 INFO:tasks.workunit.client.0.smithi022.stdout:6/87: write d1/d8/ff [3231314,39668] 0 2024-02-16T19:28:52.800 INFO:tasks.workunit.client.0.smithi022.stdout:6/88: symlink d1/l1b 0 2024-02-16T19:28:52.801 INFO:tasks.workunit.client.0.smithi022.stdout:6/89: chown d1/d8/f11 10 1 2024-02-16T19:28:52.802 INFO:tasks.workunit.client.0.smithi022.stdout:0/118: rename d10/d11/l24 to d10/d22/l25 0 2024-02-16T19:28:52.803 INFO:tasks.workunit.client.0.smithi022.stdout:0/119: truncate d10/d22/f1c 295463 0 2024-02-16T19:28:52.803 INFO:tasks.workunit.client.0.smithi022.stdout:6/90: creat d1/d8/f1c x:0 0 0 2024-02-16T19:28:52.805 INFO:tasks.workunit.client.0.smithi022.stdout:5/84: rename fb to dd/f19 0 2024-02-16T19:28:52.806 INFO:tasks.workunit.client.0.smithi022.stdout:7/84: link c6 d9/d1b/c1d 0 2024-02-16T19:28:52.806 INFO:tasks.workunit.client.0.smithi022.stdout:0/120: dread d10/f17 [0,4194304] 0 2024-02-16T19:28:52.807 INFO:tasks.workunit.client.0.smithi022.stdout:0/121: stat d10/d22/f1c 0 2024-02-16T19:28:52.807 INFO:tasks.workunit.client.0.smithi022.stdout:7/85: chown d9/da/f13 1037 1 2024-02-16T19:28:52.808 INFO:tasks.workunit.client.0.smithi022.stdout:6/91: dread d1/d8/da/f14 [0,4194304] 0 2024-02-16T19:28:52.809 INFO:tasks.workunit.client.0.smithi022.stdout:5/85: fdatasync f3 0 2024-02-16T19:28:52.810 INFO:tasks.workunit.client.0.smithi022.stdout:0/122: creat d10/f26 x:0 0 0 2024-02-16T19:28:52.812 INFO:tasks.workunit.client.0.smithi022.stdout:7/86: rename d9/de/cf to d9/dd/c1e 0 2024-02-16T19:28:52.813 INFO:tasks.workunit.client.0.smithi022.stdout:0/123: readlink l6 0 2024-02-16T19:28:52.814 INFO:tasks.workunit.client.0.smithi022.stdout:6/92: creat d1/d8/da/d17/f1d x:0 0 0 2024-02-16T19:28:52.815 INFO:tasks.workunit.client.0.smithi022.stdout:6/93: write d1/d8/da/d17/f1d [646096,111895] 0 2024-02-16T19:28:52.816 INFO:tasks.workunit.client.0.smithi022.stdout:5/86: mknod dd/d14/c1a 0 2024-02-16T19:28:52.818 INFO:tasks.workunit.client.0.smithi022.stdout:5/87: creat dd/d14/f1b x:0 0 0 2024-02-16T19:28:52.819 INFO:tasks.workunit.client.0.smithi022.stdout:0/124: getdents d10/d15 0 2024-02-16T19:28:52.820 INFO:tasks.workunit.client.0.smithi022.stdout:0/125: mkdir d10/d11/d27 0 2024-02-16T19:28:52.821 INFO:tasks.workunit.client.0.smithi022.stdout:0/126: getdents d10/d15/d23 0 2024-02-16T19:28:52.822 INFO:tasks.workunit.client.0.smithi022.stdout:0/127: write d10/d15/f1d [585291,97593] 0 2024-02-16T19:28:52.827 INFO:tasks.workunit.client.0.smithi022.stdout:0/128: dwrite f7 [0,4194304] 0 2024-02-16T19:28:52.828 INFO:tasks.workunit.client.0.smithi022.stdout:0/129: dread - d10/f1f zero size 2024-02-16T19:28:52.828 INFO:tasks.workunit.client.0.smithi022.stdout:0/130: chown d10/d22 1 1 2024-02-16T19:28:52.830 INFO:tasks.workunit.client.0.smithi022.stdout:0/131: symlink d10/d11/d27/l28 0 2024-02-16T19:28:52.831 INFO:tasks.workunit.client.0.smithi022.stdout:0/132: mkdir d10/d15/d23/d29 0 2024-02-16T19:28:52.833 INFO:tasks.workunit.client.0.smithi022.stdout:0/133: creat d10/d22/f2a x:0 0 0 2024-02-16T19:28:52.833 INFO:tasks.workunit.client.0.smithi022.stdout:0/134: fsync d10/f1f 0 2024-02-16T19:28:52.839 INFO:tasks.workunit.client.0.smithi022.stdout:0/135: dwrite fb [0,4194304] 0 2024-02-16T19:28:52.847 INFO:tasks.workunit.client.0.smithi022.stdout:0/136: dwrite d10/d15/f1e [0,4194304] 0 2024-02-16T19:28:52.907 INFO:tasks.workunit.client.0.smithi022.stdout:3/75: sync 2024-02-16T19:28:52.907 INFO:tasks.workunit.client.0.smithi022.stdout:8/79: sync 2024-02-16T19:28:52.908 INFO:tasks.workunit.client.0.smithi022.stdout:2/64: sync 2024-02-16T19:28:52.908 INFO:tasks.workunit.client.0.smithi022.stdout:2/65: read - d0/f11 zero size 2024-02-16T19:28:52.909 INFO:tasks.workunit.client.0.smithi022.stdout:3/76: truncate d7/d21/f23 622431 0 2024-02-16T19:28:52.909 INFO:tasks.workunit.client.0.smithi022.stdout:8/80: rmdir d3/d6 39 2024-02-16T19:28:52.911 INFO:tasks.workunit.client.0.smithi022.stdout:8/81: read d3/f10 [112223,117159] 0 2024-02-16T19:28:52.911 INFO:tasks.workunit.client.0.smithi022.stdout:3/77: mkdir d7/d12/d18/dd/d24 0 2024-02-16T19:28:52.914 INFO:tasks.workunit.client.0.smithi022.stdout:8/82: readlink d3/d6/l1a 0 2024-02-16T19:28:52.914 INFO:tasks.workunit.client.0.smithi022.stdout:3/78: creat d7/d12/f25 x:0 0 0 2024-02-16T19:28:52.916 INFO:tasks.workunit.client.0.smithi022.stdout:2/66: dwrite d0/f11 [0,4194304] 0 2024-02-16T19:28:52.916 INFO:tasks.workunit.client.0.smithi022.stdout:3/79: symlink d7/d12/l26 0 2024-02-16T19:28:52.923 INFO:tasks.workunit.client.0.smithi022.stdout:2/67: creat d0/d2/f13 x:0 0 0 2024-02-16T19:28:52.923 INFO:tasks.workunit.client.0.smithi022.stdout:8/83: link d3/d6/ce d3/d12/d17/c1c 0 2024-02-16T19:28:52.927 INFO:tasks.workunit.client.0.smithi022.stdout:8/84: rename d3/cc to d3/c1d 0 2024-02-16T19:28:52.928 INFO:tasks.workunit.client.0.smithi022.stdout:8/85: write d3/f10 [70722,25713] 0 2024-02-16T19:28:52.929 INFO:tasks.workunit.client.0.smithi022.stdout:8/86: rmdir d3/d6 39 2024-02-16T19:28:52.930 INFO:tasks.workunit.client.0.smithi022.stdout:8/87: creat d3/d12/d17/f1e x:0 0 0 2024-02-16T19:28:52.931 INFO:tasks.workunit.client.0.smithi022.stdout:8/88: readlink d3/lf 0 2024-02-16T19:28:52.937 INFO:tasks.workunit.client.0.smithi022.stdout:8/89: dwrite d3/f10 [0,4194304] 0 2024-02-16T19:28:52.937 INFO:tasks.workunit.client.0.smithi022.stdout:8/90: dread - d3/f15 zero size 2024-02-16T19:28:52.941 INFO:tasks.workunit.client.0.smithi022.stdout:8/91: creat d3/d6/f1f x:0 0 0 2024-02-16T19:28:52.948 INFO:tasks.workunit.client.0.smithi022.stdout:8/92: dwrite d3/d12/d17/f1e [0,4194304] 0 2024-02-16T19:28:52.955 INFO:tasks.workunit.client.0.smithi022.stdout:8/93: dwrite d3/f11 [0,4194304] 0 2024-02-16T19:28:52.969 INFO:tasks.workunit.client.0.smithi022.stdout:4/76: dread d4/d6/dd/f12 [0,4194304] 0 2024-02-16T19:28:52.970 INFO:tasks.workunit.client.0.smithi022.stdout:1/180: truncate d6/d8/d22/f15 522544 0 2024-02-16T19:28:52.972 INFO:tasks.workunit.client.0.smithi022.stdout:4/77: readlink d4/d6/dd/l1a 0 2024-02-16T19:28:52.975 INFO:tasks.workunit.client.0.smithi022.stdout:1/181: fdatasync d6/d8/d22/d16/f2e 0 2024-02-16T19:28:52.977 INFO:tasks.workunit.client.0.smithi022.stdout:1/182: readlink d6/d8/d22/d16/d17/l2f 0 2024-02-16T19:28:52.978 INFO:tasks.workunit.client.0.smithi022.stdout:1/183: chown l2 1084 1 2024-02-16T19:28:52.978 INFO:tasks.workunit.client.0.smithi022.stdout:9/116: dwrite db/d14/d19/f15 [0,4194304] 0 2024-02-16T19:28:52.983 INFO:tasks.workunit.client.0.smithi022.stdout:1/184: dread d6/f9 [0,4194304] 0 2024-02-16T19:28:52.986 INFO:tasks.workunit.client.0.smithi022.stdout:1/185: creat d6/d8/d22/d16/d17/d1d/f33 x:0 0 0 2024-02-16T19:28:52.987 INFO:tasks.workunit.client.0.smithi022.stdout:1/186: dread - d6/d8/f2d zero size 2024-02-16T19:28:52.988 INFO:tasks.workunit.client.0.smithi022.stdout:7/87: rename d9/de to d9/da/d1f 0 2024-02-16T19:28:52.993 INFO:tasks.workunit.client.0.smithi022.stdout:5/88: rename dd/f19 to dd/d14/f1c 0 2024-02-16T19:28:52.994 INFO:tasks.workunit.client.0.smithi022.stdout:1/187: dread d6/d8/d22/f1b [0,4194304] 0 2024-02-16T19:28:52.994 INFO:tasks.workunit.client.0.smithi022.stdout:5/89: write f8 [433042,97369] 0 2024-02-16T19:28:52.994 INFO:tasks.workunit.client.0.smithi022.stdout:7/88: chown d9/dd/c1e 21228035 1 2024-02-16T19:28:52.995 INFO:tasks.workunit.client.0.smithi022.stdout:5/90: read - fc zero size 2024-02-16T19:28:52.999 INFO:tasks.workunit.client.0.smithi022.stdout:2/68: rename d0/f11 to d0/d2/f14 0 2024-02-16T19:28:53.001 INFO:tasks.workunit.client.0.smithi022.stdout:7/89: dread d9/f14 [0,4194304] 0 2024-02-16T19:28:53.001 INFO:tasks.workunit.client.0.smithi022.stdout:1/188: mknod d6/d8/d22/c34 0 2024-02-16T19:28:53.001 INFO:tasks.workunit.client.0.smithi022.stdout:0/137: fsync d10/d22/f2a 0 2024-02-16T19:28:53.002 INFO:tasks.workunit.client.0.smithi022.stdout:2/69: stat d0/c5 0 2024-02-16T19:28:53.002 INFO:tasks.workunit.client.0.smithi022.stdout:5/91: creat dd/f1d x:0 0 0 2024-02-16T19:28:53.003 INFO:tasks.workunit.client.0.smithi022.stdout:1/189: readlink d6/d8/d22/l1f 0 2024-02-16T19:28:53.003 INFO:tasks.workunit.client.0.smithi022.stdout:7/90: write d9/da/f13 [2558586,126041] 0 2024-02-16T19:28:53.006 INFO:tasks.workunit.client.0.smithi022.stdout:0/138: fsync d10/f19 0 2024-02-16T19:28:53.007 INFO:tasks.workunit.client.0.smithi022.stdout:0/139: dread - d10/d22/f2a zero size 2024-02-16T19:28:53.008 INFO:tasks.workunit.client.0.smithi022.stdout:5/92: dread dd/f17 [0,4194304] 0 2024-02-16T19:28:53.009 INFO:tasks.workunit.client.0.smithi022.stdout:0/140: stat d10/d15/c1a 0 2024-02-16T19:28:53.009 INFO:tasks.workunit.client.0.smithi022.stdout:2/70: symlink d0/d2/l15 0 2024-02-16T19:28:53.010 INFO:tasks.workunit.client.0.smithi022.stdout:1/190: fsync d6/d8/d22/f28 0 2024-02-16T19:28:53.011 INFO:tasks.workunit.client.0.smithi022.stdout:0/141: stat d10/d15/d23/d29 0 2024-02-16T19:28:53.011 INFO:tasks.workunit.client.0.smithi022.stdout:7/91: symlink d9/dd/l20 0 2024-02-16T19:28:53.012 INFO:tasks.workunit.client.0.smithi022.stdout:7/92: readlink d9/l19 0 2024-02-16T19:28:53.014 INFO:tasks.workunit.client.0.smithi022.stdout:5/93: readlink dd/lf 0 2024-02-16T19:28:53.015 INFO:tasks.workunit.client.0.smithi022.stdout:3/80: dwrite f5 [0,4194304] 0 2024-02-16T19:28:53.015 INFO:tasks.workunit.client.0.smithi022.stdout:3/81: fdatasync d7/d12/f14 0 2024-02-16T19:28:53.016 INFO:tasks.workunit.client.0.smithi022.stdout:0/142: symlink d10/l2b 0 2024-02-16T19:28:53.017 INFO:tasks.workunit.client.0.smithi022.stdout:7/93: mkdir d9/dd/d21 0 2024-02-16T19:28:53.021 INFO:tasks.workunit.client.0.smithi022.stdout:8/94: truncate d3/d12/d17/f1e 1723893 0 2024-02-16T19:28:53.022 INFO:tasks.workunit.client.0.smithi022.stdout:6/94: write d1/f7 [450460,58467] 0 2024-02-16T19:28:53.022 INFO:tasks.workunit.client.0.smithi022.stdout:8/95: chown d3/f15 0 1 2024-02-16T19:28:53.023 INFO:tasks.workunit.client.0.smithi022.stdout:5/94: dwrite fc [0,4194304] 0 2024-02-16T19:28:53.023 INFO:tasks.workunit.client.0.smithi022.stdout:3/82: write f2 [2376678,33993] 0 2024-02-16T19:28:53.027 INFO:tasks.workunit.client.0.smithi022.stdout:0/143: unlink d10/d15/f1d 0 2024-02-16T19:28:53.027 INFO:tasks.workunit.client.0.smithi022.stdout:3/83: write d7/d21/f22 [757674,116848] 0 2024-02-16T19:28:53.028 INFO:tasks.workunit.client.0.smithi022.stdout:6/95: truncate d1/d8/f1c 1001704 0 2024-02-16T19:28:53.033 INFO:tasks.workunit.client.0.smithi022.stdout:9/117: truncate db/d14/d19/f2f 461595 0 2024-02-16T19:28:53.035 INFO:tasks.workunit.client.0.smithi022.stdout:5/95: dread f8 [4194304,4194304] 0 2024-02-16T19:28:53.036 INFO:tasks.workunit.client.0.smithi022.stdout:7/94: creat d9/da/d1f/f22 x:0 0 0 2024-02-16T19:28:53.038 INFO:tasks.workunit.client.0.smithi022.stdout:4/78: dwrite d4/d6/f16 [4194304,4194304] 0 2024-02-16T19:28:53.038 INFO:tasks.workunit.client.0.smithi022.stdout:5/96: dread - f9 zero size 2024-02-16T19:28:53.040 INFO:tasks.workunit.client.0.smithi022.stdout:4/79: chown d4/d6/f16 43698 1 2024-02-16T19:28:53.043 INFO:tasks.workunit.client.0.smithi022.stdout:8/96: symlink d3/d12/d17/l20 0 2024-02-16T19:28:53.047 INFO:tasks.workunit.client.0.smithi022.stdout:1/191: rename d6/d8/d22/d16/d17/d1d/l30 to d6/d8/l35 0 2024-02-16T19:28:53.049 INFO:tasks.workunit.client.0.smithi022.stdout:6/96: mknod d1/d8/da/d17/c1e 0 2024-02-16T19:28:53.050 INFO:tasks.workunit.client.0.smithi022.stdout:0/144: mkdir d10/d22/d2c 0 2024-02-16T19:28:53.052 INFO:tasks.workunit.client.0.smithi022.stdout:9/118: unlink db/d14/d19/f20 0 2024-02-16T19:28:53.053 INFO:tasks.workunit.client.0.smithi022.stdout:2/71: rename d0/l10 to d0/l16 0 2024-02-16T19:28:53.056 INFO:tasks.workunit.client.1.smithi117.stdout:1/14: sync 2024-02-16T19:28:53.056 INFO:tasks.workunit.client.1.smithi117.stdout:7/9: sync 2024-02-16T19:28:53.057 INFO:tasks.workunit.client.0.smithi022.stdout:5/97: mknod dd/c1e 0 2024-02-16T19:28:53.057 INFO:tasks.workunit.client.0.smithi022.stdout:5/98: fsync dd/d14/f1b 0 2024-02-16T19:28:53.060 INFO:tasks.workunit.client.0.smithi022.stdout:6/97: rename d1/f7 to d1/d8/f1f 0 2024-02-16T19:28:53.061 INFO:tasks.workunit.client.0.smithi022.stdout:6/98: chown d1/d8/f13 1025 1 2024-02-16T19:28:53.061 INFO:tasks.workunit.client.0.smithi022.stdout:3/84: symlink d7/d12/d18/dd/d24/l27 0 2024-02-16T19:28:53.063 INFO:tasks.workunit.client.1.smithi117.stdout:7/10: dread f0 [0,4194304] 0 2024-02-16T19:28:53.064 INFO:tasks.workunit.client.0.smithi022.stdout:3/85: readlink d7/d12/d18/dd/d24/l27 0 2024-02-16T19:28:53.065 INFO:tasks.workunit.client.0.smithi022.stdout:5/99: dwrite f6 [0,4194304] 0 2024-02-16T19:28:53.065 INFO:tasks.workunit.client.1.smithi117.stdout:1/15: rename d2/c3 to d2/d4/c5 0 2024-02-16T19:28:53.065 INFO:tasks.workunit.client.1.smithi117.stdout:1/16: chown l1 238486594 1 2024-02-16T19:28:53.067 INFO:tasks.workunit.client.0.smithi022.stdout:3/86: dread d7/d12/d18/f1b [0,4194304] 0 2024-02-16T19:28:53.068 INFO:tasks.workunit.client.0.smithi022.stdout:6/99: dread d1/d8/da/f14 [0,4194304] 0 2024-02-16T19:28:53.073 INFO:tasks.workunit.client.1.smithi117.stdout:7/11: creat d4/f6 x:0 0 0 2024-02-16T19:28:53.074 INFO:tasks.workunit.client.0.smithi022.stdout:7/95: mkdir d9/d1b/d23 0 2024-02-16T19:28:53.076 INFO:tasks.workunit.client.0.smithi022.stdout:4/80: creat d4/d6/d14/f1d x:0 0 0 2024-02-16T19:28:53.077 INFO:tasks.workunit.client.1.smithi117.stdout:7/12: creat d4/f7 x:0 0 0 2024-02-16T19:28:53.077 INFO:tasks.workunit.client.1.smithi117.stdout:7/13: chown f2 65616 1 2024-02-16T19:28:53.081 INFO:tasks.workunit.client.1.smithi117.stdout:7/14: mknod d4/c8 0 2024-02-16T19:28:53.082 INFO:tasks.workunit.client.0.smithi022.stdout:7/96: dwrite f4 [0,4194304] 0 2024-02-16T19:28:53.083 INFO:tasks.workunit.client.0.smithi022.stdout:0/145: rename d10/d15/f1e to d10/d22/f2d 0 2024-02-16T19:28:53.086 INFO:tasks.workunit.client.0.smithi022.stdout:5/100: mkdir dd/d1f 0 2024-02-16T19:28:53.087 INFO:tasks.workunit.client.0.smithi022.stdout:5/101: write f9 [928400,62954] 0 2024-02-16T19:28:53.088 INFO:tasks.workunit.client.1.smithi117.stdout:7/15: mknod d4/c9 0 2024-02-16T19:28:53.088 INFO:tasks.workunit.client.1.smithi117.stdout:7/16: chown d4/f6 2251952 1 2024-02-16T19:28:53.090 INFO:tasks.workunit.client.0.smithi022.stdout:3/87: creat d7/d12/d18/dd/f28 x:0 0 0 2024-02-16T19:28:53.093 INFO:tasks.workunit.client.0.smithi022.stdout:9/119: fsync f7 0 2024-02-16T19:28:53.093 INFO:tasks.workunit.client.1.smithi117.stdout:7/17: mknod d4/ca 0 2024-02-16T19:28:53.094 INFO:tasks.workunit.client.0.smithi022.stdout:5/102: dwrite f3 [0,4194304] 0 2024-02-16T19:28:53.094 INFO:tasks.workunit.client.0.smithi022.stdout:3/88: write d7/d12/d18/dd/f28 [652455,102168] 0 2024-02-16T19:28:53.094 INFO:tasks.workunit.client.0.smithi022.stdout:9/120: dread - db/d18/f26 zero size 2024-02-16T19:28:53.096 INFO:tasks.workunit.client.0.smithi022.stdout:4/81: creat d4/d6/dd/f1e x:0 0 0 2024-02-16T19:28:53.098 INFO:tasks.workunit.client.0.smithi022.stdout:8/97: link d3/c14 d3/d12/d17/c21 0 2024-02-16T19:28:53.098 INFO:tasks.workunit.client.0.smithi022.stdout:8/98: stat d3/d12 0 2024-02-16T19:28:53.099 INFO:tasks.workunit.client.0.smithi022.stdout:8/99: stat d3/d6/l9 0 2024-02-16T19:28:53.099 INFO:tasks.workunit.client.0.smithi022.stdout:5/103: write dd/f18 [422835,61066] 0 2024-02-16T19:28:53.100 INFO:tasks.workunit.client.1.smithi117.stdout:7/18: rename f0 to d4/fb 0 2024-02-16T19:28:53.100 INFO:tasks.workunit.client.0.smithi022.stdout:8/100: write d3/f15 [490993,4877] 0 2024-02-16T19:28:53.107 INFO:tasks.workunit.client.0.smithi022.stdout:8/101: dread d3/f15 [0,4194304] 0 2024-02-16T19:28:53.111 INFO:tasks.workunit.client.1.smithi117.stdout:1/17: link d2/d4/c5 d2/d4/c6 0 2024-02-16T19:28:53.112 INFO:tasks.workunit.client.0.smithi022.stdout:9/121: symlink db/d14/d19/l31 0 2024-02-16T19:28:53.114 INFO:tasks.workunit.client.0.smithi022.stdout:3/89: mkdir d7/d12/d18/dd/d24/d29 0 2024-02-16T19:28:53.114 INFO:tasks.workunit.client.0.smithi022.stdout:8/102: dwrite d3/d6/f1f [0,4194304] 0 2024-02-16T19:28:53.115 INFO:tasks.workunit.client.0.smithi022.stdout:9/122: chown db/d14/d19/d1a/l1b 5903287 1 2024-02-16T19:28:53.115 INFO:tasks.workunit.client.1.smithi117.stdout:1/18: creat d2/f7 x:0 0 0 2024-02-16T19:28:53.117 INFO:tasks.workunit.client.0.smithi022.stdout:4/82: unlink d4/d6/dd/f12 0 2024-02-16T19:28:53.119 INFO:tasks.workunit.client.0.smithi022.stdout:4/83: write d4/d6/f16 [3501764,20009] 0 2024-02-16T19:28:53.125 INFO:tasks.workunit.client.0.smithi022.stdout:3/90: dread d7/d12/d18/f1b [0,4194304] 0 2024-02-16T19:28:53.126 INFO:tasks.workunit.client.0.smithi022.stdout:3/91: fsync d7/d21/f23 0 2024-02-16T19:28:53.126 INFO:tasks.workunit.client.0.smithi022.stdout:8/103: creat d3/d12/f22 x:0 0 0 2024-02-16T19:28:53.127 INFO:tasks.workunit.client.0.smithi022.stdout:8/104: write d3/f10 [1461613,34399] 0 2024-02-16T19:28:53.128 INFO:tasks.workunit.client.0.smithi022.stdout:3/92: write d7/d12/d18/f1a [437894,18744] 0 2024-02-16T19:28:53.130 INFO:tasks.workunit.client.0.smithi022.stdout:4/84: creat d4/f1f x:0 0 0 2024-02-16T19:28:53.130 INFO:tasks.workunit.client.0.smithi022.stdout:4/85: dread - d4/d6/d14/f1d zero size 2024-02-16T19:28:53.131 INFO:tasks.workunit.client.0.smithi022.stdout:4/86: chown d4/d6/dd/df/d10/c13 284894 1 2024-02-16T19:28:53.132 INFO:tasks.workunit.client.0.smithi022.stdout:4/87: read - d4/d6/d14/f1d zero size 2024-02-16T19:28:53.134 INFO:tasks.workunit.client.0.smithi022.stdout:0/146: write d10/f19 [1101229,17130] 0 2024-02-16T19:28:53.134 INFO:tasks.workunit.client.1.smithi117.stdout:7/19: rename d4/fb to d4/fc 0 2024-02-16T19:28:53.136 INFO:tasks.workunit.client.0.smithi022.stdout:2/72: rename d0/d2/f13 to d0/f17 0 2024-02-16T19:28:53.140 INFO:tasks.workunit.client.1.smithi117.stdout:7/20: link d4/f7 d4/fd 0 2024-02-16T19:28:53.142 INFO:tasks.workunit.client.0.smithi022.stdout:3/93: mknod d7/d12/d18/dd/c2a 0 2024-02-16T19:28:53.143 INFO:tasks.workunit.client.0.smithi022.stdout:4/88: rmdir d4/d6/d14 39 2024-02-16T19:28:53.145 INFO:tasks.workunit.client.0.smithi022.stdout:4/89: dread - d4/d6/dd/f1e zero size 2024-02-16T19:28:53.145 INFO:tasks.workunit.client.0.smithi022.stdout:7/97: getdents d9/da/d1f 0 2024-02-16T19:28:53.146 INFO:tasks.workunit.client.0.smithi022.stdout:5/104: truncate f3 893441 0 2024-02-16T19:28:53.147 INFO:tasks.workunit.client.0.smithi022.stdout:7/98: chown c7 27737 1 2024-02-16T19:28:53.148 INFO:tasks.workunit.client.0.smithi022.stdout:0/147: creat d10/d11/f2e x:0 0 0 2024-02-16T19:28:53.149 INFO:tasks.workunit.client.0.smithi022.stdout:0/148: dread - d10/f26 zero size 2024-02-16T19:28:53.150 INFO:tasks.workunit.client.0.smithi022.stdout:3/94: dwrite d7/d12/d18/dd/ff [0,4194304] 0 2024-02-16T19:28:53.150 INFO:tasks.workunit.client.0.smithi022.stdout:7/99: truncate d9/da/d1f/f18 1338769 0 2024-02-16T19:28:53.151 INFO:tasks.workunit.client.0.smithi022.stdout:6/100: rename d1/d8/c12 to d1/c20 0 2024-02-16T19:28:53.152 INFO:tasks.workunit.client.0.smithi022.stdout:6/101: read d1/f3 [2295612,78732] 0 2024-02-16T19:28:53.153 INFO:tasks.workunit.client.0.smithi022.stdout:2/73: mkdir d0/d2/d18 0 2024-02-16T19:28:53.154 INFO:tasks.workunit.client.1.smithi117.stdout:2/21: sync 2024-02-16T19:28:53.154 INFO:tasks.workunit.client.1.smithi117.stdout:3/42: sync 2024-02-16T19:28:53.154 INFO:tasks.workunit.client.1.smithi117.stdout:4/35: sync 2024-02-16T19:28:53.154 INFO:tasks.workunit.client.1.smithi117.stdout:4/36: fsync - no filename 2024-02-16T19:28:53.154 INFO:tasks.workunit.client.1.smithi117.stdout:4/37: dread - no filename 2024-02-16T19:28:53.154 INFO:tasks.workunit.client.1.smithi117.stdout:4/38: dwrite - no filename 2024-02-16T19:28:53.154 INFO:tasks.workunit.client.0.smithi022.stdout:2/74: truncate d0/f17 230232 0 2024-02-16T19:28:53.155 INFO:tasks.workunit.client.1.smithi117.stdout:4/39: rename d0 to d0/d6/d8 22 2024-02-16T19:28:53.155 INFO:tasks.workunit.client.1.smithi117.stdout:2/22: truncate d0/d3/f7 403525 0 2024-02-16T19:28:53.157 INFO:tasks.workunit.client.1.smithi117.stdout:7/21: getdents d4 0 2024-02-16T19:28:53.158 INFO:tasks.workunit.client.0.smithi022.stdout:4/90: mknod d4/d6/dd/df/c20 0 2024-02-16T19:28:53.160 INFO:tasks.workunit.client.0.smithi022.stdout:5/105: dread f8 [0,4194304] 0 2024-02-16T19:28:53.161 INFO:tasks.workunit.client.1.smithi117.stdout:4/40: creat d0/f9 x:0 0 0 2024-02-16T19:28:53.161 INFO:tasks.workunit.client.1.smithi117.stdout:4/41: chown d0/d6/d7 22406540 1 2024-02-16T19:28:53.162 INFO:tasks.workunit.client.1.smithi117.stdout:4/42: chown d0 1940517156 1 2024-02-16T19:28:53.162 INFO:tasks.workunit.client.0.smithi022.stdout:0/149: rmdir d10/d11 39 2024-02-16T19:28:53.163 INFO:tasks.workunit.client.0.smithi022.stdout:4/91: dread d4/d6/dd/df/d10/f19 [0,4194304] 0 2024-02-16T19:28:53.165 INFO:tasks.workunit.client.0.smithi022.stdout:3/95: mknod d7/d12/c2b 0 2024-02-16T19:28:53.167 INFO:tasks.workunit.client.1.smithi117.stdout:3/43: write d1/f8 [777886,40321] 0 2024-02-16T19:28:53.167 INFO:tasks.workunit.client.1.smithi117.stdout:3/44: chown d1/dd 1 1 2024-02-16T19:28:53.170 INFO:tasks.workunit.client.0.smithi022.stdout:9/123: rename l6 to db/d14/l32 0 2024-02-16T19:28:53.170 INFO:tasks.workunit.client.0.smithi022.stdout:4/92: dwrite d4/d6/f9 [4194304,4194304] 0 2024-02-16T19:28:53.174 INFO:tasks.workunit.client.1.smithi117.stdout:2/23: link d0/d3/f6 d0/d3/f9 0 2024-02-16T19:28:53.175 INFO:tasks.workunit.client.1.smithi117.stdout:7/22: getdents d4 0 2024-02-16T19:28:53.182 INFO:tasks.workunit.client.0.smithi022.stdout:9/124: rmdir db/d18 39 2024-02-16T19:28:53.184 INFO:tasks.workunit.client.0.smithi022.stdout:4/93: fsync d4/d6/f9 0 2024-02-16T19:28:53.184 INFO:tasks.workunit.client.0.smithi022.stdout:0/150: rename d10/f13 to d10/d22/f2f 0 2024-02-16T19:28:53.185 INFO:tasks.workunit.client.0.smithi022.stdout:4/94: read d4/d6/f16 [4206588,126414] 0 2024-02-16T19:28:53.186 INFO:tasks.workunit.client.1.smithi117.stdout:3/45: write d1/f5 [502338,29723] 0 2024-02-16T19:28:53.189 INFO:tasks.workunit.client.1.smithi117.stdout:4/43: getdents d0 0 2024-02-16T19:28:53.189 INFO:tasks.workunit.client.0.smithi022.stdout:8/105: getdents d3/d6 0 2024-02-16T19:28:53.191 INFO:tasks.workunit.client.0.smithi022.stdout:3/96: creat d7/d12/d18/dd/d24/d29/f2c x:0 0 0 2024-02-16T19:28:53.191 INFO:tasks.workunit.client.1.smithi117.stdout:4/44: dread - d0/f9 zero size 2024-02-16T19:28:53.192 INFO:tasks.workunit.client.0.smithi022.stdout:3/97: truncate d7/d21/f23 682362 0 2024-02-16T19:28:53.192 INFO:tasks.workunit.client.0.smithi022.stdout:9/125: dread db/d14/d19/f2f [0,4194304] 0 2024-02-16T19:28:53.193 INFO:tasks.workunit.client.0.smithi022.stdout:9/126: chown db/d14/d19/l31 1825027 1 2024-02-16T19:28:53.194 INFO:tasks.workunit.client.0.smithi022.stdout:3/98: chown d7/d12/d18/dd/f20 2 1 2024-02-16T19:28:53.194 INFO:tasks.workunit.client.0.smithi022.stdout:3/99: stat d7/d12/f14 0 2024-02-16T19:28:53.194 INFO:tasks.workunit.client.0.smithi022.stdout:3/100: chown f5 3639 1 2024-02-16T19:28:53.194 INFO:tasks.workunit.client.1.smithi117.stdout:4/45: write d0/f9 [127445,42908] 0 2024-02-16T19:28:53.194 INFO:tasks.workunit.client.1.smithi117.stdout:2/24: dwrite d0/d3/f7 [0,4194304] 0 2024-02-16T19:28:53.195 INFO:tasks.workunit.client.0.smithi022.stdout:9/127: dread f7 [0,4194304] 0 2024-02-16T19:28:53.196 INFO:tasks.workunit.client.1.smithi117.stdout:7/23: dwrite d4/fd [0,4194304] 0 2024-02-16T19:28:53.196 INFO:tasks.workunit.client.1.smithi117.stdout:9/2: sync 2024-02-16T19:28:53.196 INFO:tasks.workunit.client.1.smithi117.stdout:9/3: truncate - no filename 2024-02-16T19:28:53.196 INFO:tasks.workunit.client.1.smithi117.stdout:9/4: dread - no filename 2024-02-16T19:28:53.196 INFO:tasks.workunit.client.1.smithi117.stdout:9/5: write - no filename 2024-02-16T19:28:53.196 INFO:tasks.workunit.client.1.smithi117.stdout:9/6: dread - no filename 2024-02-16T19:28:53.196 INFO:tasks.workunit.client.1.smithi117.stdout:9/7: stat - no entries 2024-02-16T19:28:53.196 INFO:tasks.workunit.client.1.smithi117.stdout:6/71: sync 2024-02-16T19:28:53.197 INFO:tasks.workunit.client.1.smithi117.stdout:5/147: sync 2024-02-16T19:28:53.197 INFO:tasks.workunit.client.1.smithi117.stdout:8/60: sync 2024-02-16T19:28:53.197 INFO:tasks.workunit.client.1.smithi117.stdout:0/79: sync 2024-02-16T19:28:53.197 INFO:tasks.workunit.client.1.smithi117.stdout:1/19: sync 2024-02-16T19:28:53.197 INFO:tasks.workunit.client.0.smithi022.stdout:9/128: chown db/d14/d16/d28 0 1 2024-02-16T19:28:53.197 INFO:tasks.workunit.client.1.smithi117.stdout:3/46: write d1/f2 [458610,108941] 0 2024-02-16T19:28:53.198 INFO:tasks.workunit.client.0.smithi022.stdout:9/129: write db/d14/d16/f1f [1952146,18336] 0 2024-02-16T19:28:53.201 INFO:tasks.workunit.client.1.smithi117.stdout:7/24: write d4/fd [4134174,11467] 0 2024-02-16T19:28:53.204 INFO:tasks.workunit.client.1.smithi117.stdout:2/25: unlink d0/d3/c8 0 2024-02-16T19:28:53.204 INFO:tasks.workunit.client.0.smithi022.stdout:4/95: write d4/d6/d14/f1d [501363,44985] 0 2024-02-16T19:28:53.205 INFO:tasks.workunit.client.0.smithi022.stdout:3/101: dwrite d7/d12/d18/f1a [0,4194304] 0 2024-02-16T19:28:53.205 INFO:tasks.workunit.client.0.smithi022.stdout:6/102: link d1/l4 d1/d8/da/l21 0 2024-02-16T19:28:53.206 INFO:tasks.workunit.client.1.smithi117.stdout:4/46: dwrite d0/f9 [0,4194304] 0 2024-02-16T19:28:53.210 INFO:tasks.workunit.client.1.smithi117.stdout:6/72: fdatasync d5/f7 0 2024-02-16T19:28:53.211 INFO:tasks.workunit.client.0.smithi022.stdout:8/106: mkdir d3/d6/d23 0 2024-02-16T19:28:53.213 INFO:tasks.workunit.client.0.smithi022.stdout:7/100: getdents d9/dd 0 2024-02-16T19:28:53.214 INFO:tasks.workunit.client.1.smithi117.stdout:1/20: rmdir d2/d4 39 2024-02-16T19:28:53.214 INFO:tasks.workunit.client.0.smithi022.stdout:6/103: truncate d1/d8/fb 4237481 0 2024-02-16T19:28:53.214 INFO:tasks.workunit.client.0.smithi022.stdout:5/106: dwrite f3 [0,4194304] 0 2024-02-16T19:28:53.216 INFO:tasks.workunit.client.0.smithi022.stdout:5/107: chown dd/d14 12 1 2024-02-16T19:28:53.216 INFO:tasks.workunit.client.1.smithi117.stdout:8/61: read - d4/d12/f1a zero size 2024-02-16T19:28:53.217 INFO:tasks.workunit.client.0.smithi022.stdout:5/108: write dd/f16 [2026749,8562] 0 2024-02-16T19:28:53.220 INFO:tasks.workunit.client.1.smithi117.stdout:0/80: creat d0/f1f x:0 0 0 2024-02-16T19:28:53.224 INFO:tasks.workunit.client.0.smithi022.stdout:5/109: dread dd/f16 [0,4194304] 0 2024-02-16T19:28:53.226 INFO:tasks.workunit.client.0.smithi022.stdout:9/130: write db/d18/f26 [103891,123796] 0 2024-02-16T19:28:53.226 INFO:tasks.workunit.client.1.smithi117.stdout:7/25: mknod d4/ce 0 2024-02-16T19:28:53.229 INFO:tasks.workunit.client.1.smithi117.stdout:5/148: rename d2/d3/d21/d23/f2c to d2/d9/de/d17/d1c/f2e 0 2024-02-16T19:28:53.230 INFO:tasks.workunit.client.0.smithi022.stdout:3/102: symlink d7/d12/l2d 0 2024-02-16T19:28:53.231 INFO:tasks.workunit.client.1.smithi117.stdout:2/26: creat d0/fa x:0 0 0 2024-02-16T19:28:53.232 INFO:tasks.workunit.client.1.smithi117.stdout:9/8: symlink l0 0 2024-02-16T19:28:53.232 INFO:tasks.workunit.client.1.smithi117.stdout:9/9: dwrite - no filename 2024-02-16T19:28:53.235 INFO:tasks.workunit.client.0.smithi022.stdout:6/104: symlink d1/d8/l22 0 2024-02-16T19:28:53.236 INFO:tasks.workunit.client.1.smithi117.stdout:6/73: mkdir d5/d18 0 2024-02-16T19:28:53.240 INFO:tasks.workunit.client.0.smithi022.stdout:9/131: mkdir db/d14/d33 0 2024-02-16T19:28:53.242 INFO:tasks.workunit.client.0.smithi022.stdout:9/132: dread f7 [0,4194304] 0 2024-02-16T19:28:53.243 INFO:tasks.workunit.client.0.smithi022.stdout:3/103: creat d7/d12/d18/dd/f2e x:0 0 0 2024-02-16T19:28:53.244 INFO:tasks.workunit.client.1.smithi117.stdout:7/26: mkdir d4/df 0 2024-02-16T19:28:53.245 INFO:tasks.workunit.client.1.smithi117.stdout:5/149: fsync d2/d9/de/d17/d1c/f20 0 2024-02-16T19:28:53.247 INFO:tasks.workunit.client.0.smithi022.stdout:8/107: symlink d3/l24 0 2024-02-16T19:28:53.247 INFO:tasks.workunit.client.1.smithi117.stdout:2/27: mknod d0/d3/cb 0 2024-02-16T19:28:53.249 INFO:tasks.workunit.client.0.smithi022.stdout:6/105: mkdir d1/d8/d23 0 2024-02-16T19:28:53.250 INFO:tasks.workunit.client.0.smithi022.stdout:8/108: dread d3/d6/f1f [0,4194304] 0 2024-02-16T19:28:53.250 INFO:tasks.workunit.client.0.smithi022.stdout:6/106: write d1/d8/fb [3092368,80994] 0 2024-02-16T19:28:53.252 INFO:tasks.workunit.client.0.smithi022.stdout:0/151: link d10/d15/c1a d10/d11/d27/c30 0 2024-02-16T19:28:53.253 INFO:tasks.workunit.client.0.smithi022.stdout:0/152: dread - d10/d11/f2e zero size 2024-02-16T19:28:53.253 INFO:tasks.workunit.client.1.smithi117.stdout:0/81: write d0/d9/f11 [1783005,73657] 0 2024-02-16T19:28:53.254 INFO:tasks.workunit.client.0.smithi022.stdout:9/133: symlink db/d14/d19/l34 0 2024-02-16T19:28:53.254 INFO:tasks.workunit.client.0.smithi022.stdout:9/134: readlink db/d18/l1e 0 2024-02-16T19:28:53.255 INFO:tasks.workunit.client.0.smithi022.stdout:9/135: chown db/d18/f26 6851648 1 2024-02-16T19:28:53.256 INFO:tasks.workunit.client.0.smithi022.stdout:3/104: rmdir d7/d12/d18/dd/d24/d29 39 2024-02-16T19:28:53.263 INFO:tasks.workunit.client.0.smithi022.stdout:6/107: mkdir d1/d24 0 2024-02-16T19:28:53.263 INFO:tasks.workunit.client.0.smithi022.stdout:6/108: read d1/d8/ff [641202,42902] 0 2024-02-16T19:28:53.263 INFO:tasks.workunit.client.0.smithi022.stdout:0/153: mkdir d10/d22/d31 0 2024-02-16T19:28:53.264 INFO:tasks.workunit.client.0.smithi022.stdout:6/109: rename d1/d8 to d1/d8/d23/d25 22 2024-02-16T19:28:53.264 INFO:tasks.workunit.client.0.smithi022.stdout:0/154: dread - d10/d11/f2e zero size 2024-02-16T19:28:53.266 INFO:tasks.workunit.client.0.smithi022.stdout:4/96: link d4/cc d4/d6/c21 0 2024-02-16T19:28:53.268 INFO:tasks.workunit.client.0.smithi022.stdout:8/109: mkdir d3/d25 0 2024-02-16T19:28:53.268 INFO:tasks.workunit.client.0.smithi022.stdout:8/110: fdatasync d3/d6/f7 0 2024-02-16T19:28:53.271 INFO:tasks.workunit.client.0.smithi022.stdout:4/97: unlink d4/d6/f9 0 2024-02-16T19:28:53.272 INFO:tasks.workunit.client.0.smithi022.stdout:6/110: dread d1/f2 [0,4194304] 0 2024-02-16T19:28:53.273 INFO:tasks.workunit.client.0.smithi022.stdout:3/105: creat d7/d12/d18/dd/d24/f2f x:0 0 0 2024-02-16T19:28:53.274 INFO:tasks.workunit.client.0.smithi022.stdout:0/155: symlink d10/l32 0 2024-02-16T19:28:53.277 INFO:tasks.workunit.client.0.smithi022.stdout:8/111: dwrite d3/f10 [0,4194304] 0 2024-02-16T19:28:53.277 INFO:tasks.workunit.client.0.smithi022.stdout:0/156: dread d10/d22/f1c [0,4194304] 0 2024-02-16T19:28:53.279 INFO:tasks.workunit.client.0.smithi022.stdout:0/157: chown d10/d22/l21 1 1 2024-02-16T19:28:53.280 INFO:tasks.workunit.client.0.smithi022.stdout:8/112: creat d3/d25/f26 x:0 0 0 2024-02-16T19:28:53.283 INFO:tasks.workunit.client.0.smithi022.stdout:0/158: mknod d10/d11/c33 0 2024-02-16T19:28:53.283 INFO:tasks.workunit.client.0.smithi022.stdout:3/106: dwrite f2 [0,4194304] 0 2024-02-16T19:28:53.283 INFO:tasks.workunit.client.0.smithi022.stdout:8/113: write d3/d25/f26 [353102,82467] 0 2024-02-16T19:28:53.283 INFO:tasks.workunit.client.0.smithi022.stdout:3/107: readlink l3 0 2024-02-16T19:28:53.287 INFO:tasks.workunit.client.0.smithi022.stdout:8/114: creat d3/d6/f27 x:0 0 0 2024-02-16T19:28:53.288 INFO:tasks.workunit.client.0.smithi022.stdout:0/159: mknod d10/d15/d23/c34 0 2024-02-16T19:28:53.305 INFO:tasks.workunit.client.1.smithi117.stdout:6/74: dread - d5/f15 zero size 2024-02-16T19:28:53.305 INFO:tasks.workunit.client.1.smithi117.stdout:4/47: truncate d0/f9 2638514 0 2024-02-16T19:28:53.305 INFO:tasks.workunit.client.1.smithi117.stdout:8/62: dwrite d4/d12/f1a [0,4194304] 0 2024-02-16T19:28:53.306 INFO:tasks.workunit.client.1.smithi117.stdout:3/47: link d1/l6 d1/de/lf 0 2024-02-16T19:28:53.306 INFO:tasks.workunit.client.0.smithi022.stdout:8/115: creat d3/d6/f28 x:0 0 0 2024-02-16T19:28:53.306 INFO:tasks.workunit.client.1.smithi117.stdout:5/150: mkdir d2/d3/d21/d2f 0 2024-02-16T19:28:53.307 INFO:tasks.workunit.client.1.smithi117.stdout:2/28: unlink d0/d3/f7 0 2024-02-16T19:28:53.307 INFO:tasks.workunit.client.1.smithi117.stdout:8/63: readlink d4/d12/l13 0 2024-02-16T19:28:53.308 INFO:tasks.workunit.client.0.smithi022.stdout:8/116: write d3/d6/f28 [539792,130734] 0 2024-02-16T19:28:53.309 INFO:tasks.workunit.client.1.smithi117.stdout:1/21: rename d2/f7 to d2/f8 0 2024-02-16T19:28:53.316 INFO:tasks.workunit.client.1.smithi117.stdout:0/82: write d0/f18 [4467053,79572] 0 2024-02-16T19:28:53.318 INFO:tasks.workunit.client.1.smithi117.stdout:4/48: write d0/f9 [2735777,107655] 0 2024-02-16T19:28:53.324 INFO:tasks.workunit.client.0.smithi022.stdout:4/98: write d4/d6/f16 [1715750,16254] 0 2024-02-16T19:28:53.325 INFO:tasks.workunit.client.1.smithi117.stdout:4/49: dwrite d0/f9 [0,4194304] 0 2024-02-16T19:28:53.329 INFO:tasks.workunit.client.1.smithi117.stdout:2/29: creat d0/d3/fc x:0 0 0 2024-02-16T19:28:53.332 INFO:tasks.workunit.client.1.smithi117.stdout:4/50: dwrite d0/f9 [0,4194304] 0 2024-02-16T19:28:53.333 INFO:tasks.workunit.client.1.smithi117.stdout:4/51: readlink d0/l5 0 2024-02-16T19:28:53.340 INFO:tasks.workunit.client.1.smithi117.stdout:7/27: rmdir d4/df 0 2024-02-16T19:28:53.346 INFO:tasks.workunit.client.1.smithi117.stdout:5/151: mkdir d2/d3/d21/d2f/d30 0 2024-02-16T19:28:53.350 INFO:tasks.workunit.client.1.smithi117.stdout:2/30: fsync d0/d3/f6 0 2024-02-16T19:28:53.352 INFO:tasks.workunit.client.1.smithi117.stdout:4/52: rmdir d0 39 2024-02-16T19:28:53.354 INFO:tasks.workunit.client.1.smithi117.stdout:1/22: rename d2/f8 to d2/d4/f9 0 2024-02-16T19:28:53.357 INFO:tasks.workunit.client.1.smithi117.stdout:7/28: creat d4/f10 x:0 0 0 2024-02-16T19:28:53.358 INFO:tasks.workunit.client.1.smithi117.stdout:0/83: mknod d0/d9/d14/d1d/d1e/c20 0 2024-02-16T19:28:53.359 INFO:tasks.workunit.client.1.smithi117.stdout:0/84: dread - d0/f1f zero size 2024-02-16T19:28:53.359 INFO:tasks.workunit.client.1.smithi117.stdout:3/48: write d1/fc [842790,80794] 0 2024-02-16T19:28:53.360 INFO:tasks.workunit.client.1.smithi117.stdout:5/152: rmdir d2 39 2024-02-16T19:28:53.364 INFO:tasks.workunit.client.1.smithi117.stdout:0/85: dread d0/d9/f11 [0,4194304] 0 2024-02-16T19:28:53.366 INFO:tasks.workunit.client.1.smithi117.stdout:1/23: write d2/d4/f9 [616943,20656] 0 2024-02-16T19:28:53.368 INFO:tasks.workunit.client.1.smithi117.stdout:7/29: creat d4/f11 x:0 0 0 2024-02-16T19:28:53.368 INFO:tasks.workunit.client.1.smithi117.stdout:1/24: chown l1 0 1 2024-02-16T19:28:53.372 INFO:tasks.workunit.client.1.smithi117.stdout:8/64: dwrite d4/d7/fc [0,4194304] 0 2024-02-16T19:28:53.374 INFO:tasks.workunit.client.1.smithi117.stdout:5/153: readlink d2/d3/l2b 0 2024-02-16T19:28:53.374 INFO:tasks.workunit.client.1.smithi117.stdout:1/25: dread d2/d4/f9 [0,4194304] 0 2024-02-16T19:28:53.375 INFO:tasks.workunit.client.1.smithi117.stdout:8/65: fdatasync d4/d12/f1a 0 2024-02-16T19:28:53.375 INFO:tasks.workunit.client.1.smithi117.stdout:6/75: dwrite f2 [0,4194304] 0 2024-02-16T19:28:53.375 INFO:tasks.workunit.client.1.smithi117.stdout:5/154: stat d2/d4/cf 0 2024-02-16T19:28:53.376 INFO:tasks.workunit.client.1.smithi117.stdout:0/86: dread - d0/d9/f1c zero size 2024-02-16T19:28:53.378 INFO:tasks.workunit.client.1.smithi117.stdout:5/155: readlink d2/d4/l19 0 2024-02-16T19:28:53.378 INFO:tasks.workunit.client.1.smithi117.stdout:2/31: write d0/d3/f6 [1018852,128973] 0 2024-02-16T19:28:53.380 INFO:tasks.workunit.client.1.smithi117.stdout:7/30: mknod d4/c12 0 2024-02-16T19:28:53.380 INFO:tasks.workunit.client.1.smithi117.stdout:7/31: stat d4/ce 0 2024-02-16T19:28:53.388 INFO:tasks.workunit.client.1.smithi117.stdout:7/32: dwrite d4/f6 [0,4194304] 0 2024-02-16T19:28:53.392 INFO:tasks.workunit.client.1.smithi117.stdout:8/66: rmdir d4 39 2024-02-16T19:28:53.394 INFO:tasks.workunit.client.1.smithi117.stdout:1/26: creat d2/d4/fa x:0 0 0 2024-02-16T19:28:53.400 INFO:tasks.workunit.client.1.smithi117.stdout:5/156: symlink d2/d1d/l31 0 2024-02-16T19:28:53.400 INFO:tasks.workunit.client.1.smithi117.stdout:5/157: chown d2/d9 866959 1 2024-02-16T19:28:53.401 INFO:tasks.workunit.client.1.smithi117.stdout:2/32: rename d0/fa to d0/fd 0 2024-02-16T19:28:53.405 INFO:tasks.workunit.client.1.smithi117.stdout:3/49: link d1/de/lf d1/l10 0 2024-02-16T19:28:53.407 INFO:tasks.workunit.client.1.smithi117.stdout:9/10: sync 2024-02-16T19:28:53.407 INFO:tasks.workunit.client.1.smithi117.stdout:7/33: rmdir d4 39 2024-02-16T19:28:53.410 INFO:tasks.workunit.client.1.smithi117.stdout:1/27: symlink d2/d4/lb 0 2024-02-16T19:28:53.417 INFO:tasks.workunit.client.1.smithi117.stdout:2/33: write d0/fd [186172,128924] 0 2024-02-16T19:28:53.418 INFO:tasks.workunit.client.1.smithi117.stdout:3/50: mknod d1/c11 0 2024-02-16T19:28:53.419 INFO:tasks.workunit.client.1.smithi117.stdout:4/53: truncate d0/f9 218583 0 2024-02-16T19:28:53.421 INFO:tasks.workunit.client.1.smithi117.stdout:9/11: mknod c1 0 2024-02-16T19:28:53.421 INFO:tasks.workunit.client.1.smithi117.stdout:9/12: dwrite - no filename 2024-02-16T19:28:53.423 INFO:tasks.workunit.client.1.smithi117.stdout:0/87: dwrite d0/f1a [0,4194304] 0 2024-02-16T19:28:53.423 INFO:tasks.workunit.client.1.smithi117.stdout:8/67: rmdir d4/d12 39 2024-02-16T19:28:53.425 INFO:tasks.workunit.client.1.smithi117.stdout:1/28: truncate d2/d4/f9 1333084 0 2024-02-16T19:28:53.426 INFO:tasks.workunit.client.1.smithi117.stdout:6/76: link d5/f12 d5/db/f19 0 2024-02-16T19:28:53.427 INFO:tasks.workunit.client.1.smithi117.stdout:5/158: getdents d2/d3/d21/d2f/d30 0 2024-02-16T19:28:53.428 INFO:tasks.workunit.client.1.smithi117.stdout:5/159: chown d2/d9 2622 1 2024-02-16T19:28:53.428 INFO:tasks.workunit.client.1.smithi117.stdout:7/34: dwrite d4/f10 [0,4194304] 0 2024-02-16T19:28:53.429 INFO:tasks.workunit.client.1.smithi117.stdout:2/34: creat d0/d3/fe x:0 0 0 2024-02-16T19:28:53.434 INFO:tasks.workunit.client.1.smithi117.stdout:6/77: dwrite f4 [0,4194304] 0 2024-02-16T19:28:53.437 INFO:tasks.workunit.client.1.smithi117.stdout:9/13: mknod c2 0 2024-02-16T19:28:53.437 INFO:tasks.workunit.client.1.smithi117.stdout:9/14: read - no filename 2024-02-16T19:28:53.446 INFO:tasks.workunit.client.1.smithi117.stdout:0/88: truncate d0/d9/d14/f1b 291196 0 2024-02-16T19:28:53.447 INFO:tasks.workunit.client.1.smithi117.stdout:5/160: fsync d2/d3/f12 0 2024-02-16T19:28:53.448 INFO:tasks.workunit.client.1.smithi117.stdout:5/161: chown d2/d9/de/d17 867522 1 2024-02-16T19:28:53.449 INFO:tasks.workunit.client.1.smithi117.stdout:7/35: mkdir d4/d13 0 2024-02-16T19:28:53.449 INFO:tasks.workunit.client.1.smithi117.stdout:5/162: chown d2/d3/c8 298005 1 2024-02-16T19:28:53.450 INFO:tasks.workunit.client.1.smithi117.stdout:3/51: symlink d1/dd/l12 0 2024-02-16T19:28:53.451 INFO:tasks.workunit.client.1.smithi117.stdout:2/35: creat d0/ff x:0 0 0 2024-02-16T19:28:53.451 INFO:tasks.workunit.client.1.smithi117.stdout:3/52: dread d1/f2 [0,4194304] 0 2024-02-16T19:28:53.453 INFO:tasks.workunit.client.1.smithi117.stdout:9/15: symlink l3 0 2024-02-16T19:28:53.464 INFO:tasks.workunit.client.1.smithi117.stdout:3/53: fsync f0 0 2024-02-16T19:28:53.467 INFO:tasks.workunit.client.1.smithi117.stdout:9/16: creat f4 x:0 0 0 2024-02-16T19:28:53.468 INFO:tasks.workunit.client.1.smithi117.stdout:9/17: stat f4 0 2024-02-16T19:28:53.473 INFO:tasks.workunit.client.1.smithi117.stdout:3/54: dwrite d1/f5 [0,4194304] 0 2024-02-16T19:28:53.474 INFO:tasks.workunit.client.1.smithi117.stdout:4/54: dwrite d0/f9 [0,4194304] 0 2024-02-16T19:28:53.475 INFO:tasks.workunit.client.1.smithi117.stdout:4/55: readlink d0/l5 0 2024-02-16T19:28:53.476 INFO:tasks.workunit.client.0.smithi022.stdout:1/192: sync 2024-02-16T19:28:53.477 INFO:tasks.workunit.client.1.smithi117.stdout:4/56: write d0/f9 [342501,52632] 0 2024-02-16T19:28:53.477 INFO:tasks.workunit.client.1.smithi117.stdout:4/57: chown d0/d6 32724 1 2024-02-16T19:28:53.479 INFO:tasks.workunit.client.1.smithi117.stdout:8/68: link d4/l16 d4/d12/l1b 0 2024-02-16T19:28:53.481 INFO:tasks.workunit.client.1.smithi117.stdout:9/18: dwrite f4 [0,4194304] 0 2024-02-16T19:28:53.482 INFO:tasks.workunit.client.1.smithi117.stdout:2/36: rmdir d0 39 2024-02-16T19:28:53.482 INFO:tasks.workunit.client.1.smithi117.stdout:3/55: read - d1/fa zero size 2024-02-16T19:28:53.486 INFO:tasks.workunit.client.1.smithi117.stdout:3/56: dread d1/f5 [0,4194304] 0 2024-02-16T19:28:53.493 INFO:tasks.workunit.client.1.smithi117.stdout:3/57: dread f0 [0,4194304] 0 2024-02-16T19:28:53.494 INFO:tasks.workunit.client.1.smithi117.stdout:4/58: rename d0/d6/d7 to d0/da 0 2024-02-16T19:28:53.496 INFO:tasks.workunit.client.1.smithi117.stdout:7/36: write d4/fc [4799203,42450] 0 2024-02-16T19:28:53.497 INFO:tasks.workunit.client.1.smithi117.stdout:0/89: dwrite d0/d9/f13 [0,4194304] 0 2024-02-16T19:28:53.501 INFO:tasks.workunit.client.1.smithi117.stdout:1/29: write d2/d4/f9 [858752,119495] 0 2024-02-16T19:28:53.503 INFO:tasks.workunit.client.1.smithi117.stdout:5/163: dwrite d2/f13 [0,4194304] 0 2024-02-16T19:28:53.503 INFO:tasks.workunit.client.1.smithi117.stdout:8/69: chown d4/d12/l1b 60996 1 2024-02-16T19:28:53.504 INFO:tasks.workunit.client.1.smithi117.stdout:5/164: stat d2/cb 0 2024-02-16T19:28:53.507 INFO:tasks.workunit.client.1.smithi117.stdout:3/58: mknod d1/dd/c13 0 2024-02-16T19:28:53.510 INFO:tasks.workunit.client.1.smithi117.stdout:4/59: symlink d0/da/lb 0 2024-02-16T19:28:53.514 INFO:tasks.workunit.client.1.smithi117.stdout:7/37: mkdir d4/d14 0 2024-02-16T19:28:53.515 INFO:tasks.workunit.client.1.smithi117.stdout:7/38: write d4/fc [1875548,55961] 0 2024-02-16T19:28:53.516 INFO:tasks.workunit.client.1.smithi117.stdout:0/90: creat d0/d9/d14/d1d/d1e/f21 x:0 0 0 2024-02-16T19:28:53.524 INFO:tasks.workunit.client.1.smithi117.stdout:9/19: write f4 [4470429,8710] 0 2024-02-16T19:28:53.525 INFO:tasks.workunit.client.1.smithi117.stdout:9/20: write f4 [783966,92115] 0 2024-02-16T19:28:53.525 INFO:tasks.workunit.client.1.smithi117.stdout:4/60: mknod d0/cc 0 2024-02-16T19:28:53.526 INFO:tasks.workunit.client.1.smithi117.stdout:4/61: rename d0/d6 to d0/d6/dd 22 2024-02-16T19:28:53.528 INFO:tasks.workunit.client.1.smithi117.stdout:2/37: unlink d0/f5 0 2024-02-16T19:28:53.529 INFO:tasks.workunit.client.1.smithi117.stdout:4/62: dread d0/f9 [0,4194304] 0 2024-02-16T19:28:53.530 INFO:tasks.workunit.client.1.smithi117.stdout:4/63: stat d0/f9 0 2024-02-16T19:28:53.531 INFO:tasks.workunit.client.1.smithi117.stdout:6/78: sync 2024-02-16T19:28:53.533 INFO:tasks.workunit.client.1.smithi117.stdout:0/91: mkdir d0/d9/d22 0 2024-02-16T19:28:53.535 INFO:tasks.workunit.client.1.smithi117.stdout:1/30: mknod d2/cc 0 2024-02-16T19:28:53.535 INFO:tasks.workunit.client.1.smithi117.stdout:1/31: chown d2/d4/fa 0 1 2024-02-16T19:28:53.544 INFO:tasks.workunit.client.1.smithi117.stdout:8/70: write d4/f6 [1752258,81565] 0 2024-02-16T19:28:53.546 INFO:tasks.workunit.client.1.smithi117.stdout:4/64: unlink d0/cc 0 2024-02-16T19:28:53.547 INFO:tasks.workunit.client.1.smithi117.stdout:5/165: write d2/d9/de/f22 [237049,61594] 0 2024-02-16T19:28:53.547 INFO:tasks.workunit.client.1.smithi117.stdout:7/39: symlink d4/d13/l15 0 2024-02-16T19:28:53.549 INFO:tasks.workunit.client.1.smithi117.stdout:6/79: creat d5/f1a x:0 0 0 2024-02-16T19:28:53.552 INFO:tasks.workunit.client.1.smithi117.stdout:1/32: mknod d2/d4/cd 0 2024-02-16T19:28:53.553 INFO:tasks.workunit.client.1.smithi117.stdout:3/59: link f0 d1/dd/f14 0 2024-02-16T19:28:53.559 INFO:tasks.workunit.client.1.smithi117.stdout:8/71: rmdir d4/d12 39 2024-02-16T19:28:53.561 INFO:tasks.workunit.client.1.smithi117.stdout:4/65: creat d0/d6/fe x:0 0 0 2024-02-16T19:28:53.562 INFO:tasks.workunit.client.1.smithi117.stdout:5/166: creat d2/d3/d21/f32 x:0 0 0 2024-02-16T19:28:53.564 INFO:tasks.workunit.client.1.smithi117.stdout:2/38: write d0/f1 [223154,102679] 0 2024-02-16T19:28:53.566 INFO:tasks.workunit.client.1.smithi117.stdout:0/92: fdatasync d0/d9/d14/f1b 0 2024-02-16T19:28:53.569 INFO:tasks.workunit.client.1.smithi117.stdout:3/60: chown d1/l10 10 1 2024-02-16T19:28:53.573 INFO:tasks.workunit.client.1.smithi117.stdout:5/167: creat d2/d3/d21/f33 x:0 0 0 2024-02-16T19:28:53.575 INFO:tasks.workunit.client.1.smithi117.stdout:7/40: link d4/f10 d4/d13/f16 0 2024-02-16T19:28:53.577 INFO:tasks.workunit.client.1.smithi117.stdout:2/39: mkdir d0/d3/d10 0 2024-02-16T19:28:53.579 INFO:tasks.workunit.client.1.smithi117.stdout:7/41: dread d4/d13/f16 [0,4194304] 0 2024-02-16T19:28:53.581 INFO:tasks.workunit.client.1.smithi117.stdout:0/93: mkdir d0/d9/d14/d23 0 2024-02-16T19:28:53.583 INFO:tasks.workunit.client.1.smithi117.stdout:1/33: unlink d2/d4/c5 0 2024-02-16T19:28:53.584 INFO:tasks.workunit.client.1.smithi117.stdout:2/40: dwrite d0/d3/fc [0,4194304] 0 2024-02-16T19:28:53.585 INFO:tasks.workunit.client.1.smithi117.stdout:9/21: getdents . 0 2024-02-16T19:28:53.585 INFO:tasks.workunit.client.1.smithi117.stdout:9/22: fsync f4 0 2024-02-16T19:28:53.589 INFO:tasks.workunit.client.1.smithi117.stdout:5/168: truncate d2/d4/d16/d24/f26 4543718 0 2024-02-16T19:28:53.591 INFO:tasks.workunit.client.1.smithi117.stdout:7/42: creat d4/f17 x:0 0 0 2024-02-16T19:28:53.591 INFO:tasks.workunit.client.1.smithi117.stdout:2/41: read d0/fd [96666,102976] 0 2024-02-16T19:28:53.592 INFO:tasks.workunit.client.1.smithi117.stdout:2/42: chown d0/d3/fe 1656134 1 2024-02-16T19:28:53.593 INFO:tasks.workunit.client.1.smithi117.stdout:2/43: write d0/fd [408445,49376] 0 2024-02-16T19:28:53.598 INFO:tasks.workunit.client.1.smithi117.stdout:6/80: write d5/ff [963011,12224] 0 2024-02-16T19:28:53.599 INFO:tasks.workunit.client.1.smithi117.stdout:6/81: chown d5/db/f19 15340281 1 2024-02-16T19:28:53.601 INFO:tasks.workunit.client.1.smithi117.stdout:2/44: dwrite d0/f1 [0,4194304] 0 2024-02-16T19:28:53.601 INFO:tasks.workunit.client.1.smithi117.stdout:2/45: stat d0/d3/f6 0 2024-02-16T19:28:53.601 INFO:tasks.workunit.client.1.smithi117.stdout:2/46: readlink - no filename 2024-02-16T19:28:53.603 INFO:tasks.workunit.client.1.smithi117.stdout:9/23: creat f5 x:0 0 0 2024-02-16T19:28:53.607 INFO:tasks.workunit.client.0.smithi022.stdout:7/101: sync 2024-02-16T19:28:53.607 INFO:tasks.workunit.client.0.smithi022.stdout:5/110: sync 2024-02-16T19:28:53.607 INFO:tasks.workunit.client.0.smithi022.stdout:2/75: sync 2024-02-16T19:28:53.608 INFO:tasks.workunit.client.1.smithi117.stdout:7/43: creat d4/d13/f18 x:0 0 0 2024-02-16T19:28:53.611 INFO:tasks.workunit.client.1.smithi117.stdout:2/47: dwrite d0/ff [0,4194304] 0 2024-02-16T19:28:53.615 INFO:tasks.workunit.client.0.smithi022.stdout:7/102: dwrite d9/da/d1f/f22 [0,4194304] 0 2024-02-16T19:28:53.616 INFO:tasks.workunit.client.0.smithi022.stdout:7/103: read d9/f16 [347841,64633] 0 2024-02-16T19:28:53.619 INFO:tasks.workunit.client.0.smithi022.stdout:7/104: truncate d9/da/d1f/f22 4804719 0 2024-02-16T19:28:53.620 INFO:tasks.workunit.client.0.smithi022.stdout:7/105: write d9/da/d1f/f18 [1421635,117478] 0 2024-02-16T19:28:53.624 INFO:tasks.workunit.client.1.smithi117.stdout:8/72: truncate d4/d12/f1a 2139335 0 2024-02-16T19:28:53.625 INFO:tasks.workunit.client.1.smithi117.stdout:8/73: read d4/d7/fc [168976,74283] 0 2024-02-16T19:28:53.626 INFO:tasks.workunit.client.1.smithi117.stdout:9/24: rename c2 to c6 0 2024-02-16T19:28:53.626 INFO:tasks.workunit.client.1.smithi117.stdout:9/25: fdatasync f4 0 2024-02-16T19:28:53.634 INFO:tasks.workunit.client.1.smithi117.stdout:1/34: rename l1 to d2/le 0 2024-02-16T19:28:53.638 INFO:tasks.workunit.client.1.smithi117.stdout:9/26: unlink l3 0 2024-02-16T19:28:53.639 INFO:tasks.workunit.client.1.smithi117.stdout:9/27: dread - f5 zero size 2024-02-16T19:28:53.639 INFO:tasks.workunit.client.1.smithi117.stdout:9/28: write f4 [4079376,27484] 0 2024-02-16T19:28:53.643 INFO:tasks.workunit.client.1.smithi117.stdout:6/82: write d5/fa [4428445,86394] 0 2024-02-16T19:28:53.647 INFO:tasks.workunit.client.1.smithi117.stdout:0/94: getdents d0/d9/d14/d1d/d1e 0 2024-02-16T19:28:53.649 INFO:tasks.workunit.client.0.smithi022.stdout:3/108: sync 2024-02-16T19:28:53.649 INFO:tasks.workunit.client.0.smithi022.stdout:9/136: sync 2024-02-16T19:28:53.649 INFO:tasks.workunit.client.0.smithi022.stdout:6/111: sync 2024-02-16T19:28:53.650 INFO:tasks.workunit.client.1.smithi117.stdout:6/83: dwrite d5/f12 [0,4194304] 0 2024-02-16T19:28:53.651 INFO:tasks.workunit.client.1.smithi117.stdout:2/48: rename d0/ff to d0/d3/d10/f11 0 2024-02-16T19:28:53.651 INFO:tasks.workunit.client.0.smithi022.stdout:3/109: write d7/d12/d18/dd/f2e [491440,112110] 0 2024-02-16T19:28:53.651 INFO:tasks.workunit.client.0.smithi022.stdout:9/137: dread f7 [0,4194304] 0 2024-02-16T19:28:53.652 INFO:tasks.workunit.client.0.smithi022.stdout:9/138: stat db/d18 0 2024-02-16T19:28:53.652 INFO:tasks.workunit.client.1.smithi117.stdout:2/49: write d0/d3/f6 [140912,124526] 0 2024-02-16T19:28:53.654 INFO:tasks.workunit.client.1.smithi117.stdout:2/50: read d0/d3/fc [2116720,89344] 0 2024-02-16T19:28:53.654 INFO:tasks.workunit.client.1.smithi117.stdout:2/51: write d0/d3/f6 [1124723,15775] 0 2024-02-16T19:28:53.658 INFO:tasks.workunit.client.1.smithi117.stdout:5/169: getdents d2/d9 0 2024-02-16T19:28:53.658 INFO:tasks.workunit.client.1.smithi117.stdout:5/170: chown d2/d3/d21/d2f 1481 1 2024-02-16T19:28:53.661 INFO:tasks.workunit.client.1.smithi117.stdout:9/29: mknod c7 0 2024-02-16T19:28:53.663 INFO:tasks.workunit.client.1.smithi117.stdout:7/44: rmdir d4/d14 0 2024-02-16T19:28:53.670 INFO:tasks.workunit.client.1.smithi117.stdout:0/95: dread d0/fe [0,4194304] 0 2024-02-16T19:28:53.670 INFO:tasks.workunit.client.1.smithi117.stdout:0/96: stat d0/d9/f1c 0 2024-02-16T19:28:53.673 INFO:tasks.workunit.client.1.smithi117.stdout:6/84: rename d5/fe to d5/f1b 0 2024-02-16T19:28:53.675 INFO:tasks.workunit.client.1.smithi117.stdout:8/74: dwrite d4/d12/f1a [0,4194304] 0 2024-02-16T19:28:53.675 INFO:tasks.workunit.client.1.smithi117.stdout:1/35: dread - d2/d4/fa zero size 2024-02-16T19:28:53.682 INFO:tasks.workunit.client.1.smithi117.stdout:8/75: dwrite d4/d12/f1a [0,4194304] 0 2024-02-16T19:28:53.682 INFO:tasks.workunit.client.1.smithi117.stdout:5/171: truncate d2/d9/de/d17/d1c/f20 520960 0 2024-02-16T19:28:53.685 INFO:tasks.workunit.client.0.smithi022.stdout:4/99: mkdir d4/d6/dd/d22 0 2024-02-16T19:28:53.686 INFO:tasks.workunit.client.1.smithi117.stdout:9/30: symlink l8 0 2024-02-16T19:28:53.686 INFO:tasks.workunit.client.1.smithi117.stdout:9/31: fdatasync f5 0 2024-02-16T19:28:53.688 INFO:tasks.workunit.client.1.smithi117.stdout:7/45: symlink d4/d13/l19 0 2024-02-16T19:28:53.689 INFO:tasks.workunit.client.1.smithi117.stdout:7/46: fsync d4/fd 0 2024-02-16T19:28:53.690 INFO:tasks.workunit.client.1.smithi117.stdout:5/172: dwrite d2/d9/de/f22 [0,4194304] 0 2024-02-16T19:28:53.690 INFO:tasks.workunit.client.0.smithi022.stdout:1/193: mknod d6/d8/d22/c36 0 2024-02-16T19:28:53.691 INFO:tasks.workunit.client.1.smithi117.stdout:7/47: write d4/fc [4143416,61914] 0 2024-02-16T19:28:53.691 INFO:tasks.workunit.client.1.smithi117.stdout:3/61: sync 2024-02-16T19:28:53.691 INFO:tasks.workunit.client.1.smithi117.stdout:4/66: sync 2024-02-16T19:28:53.692 INFO:tasks.workunit.client.0.smithi022.stdout:5/111: mknod dd/d14/c20 0 2024-02-16T19:28:53.700 INFO:tasks.workunit.client.0.smithi022.stdout:7/106: mknod d9/da/c24 0 2024-02-16T19:28:53.701 INFO:tasks.workunit.client.0.smithi022.stdout:7/107: read f5 [3236589,124078] 0 2024-02-16T19:28:53.702 INFO:tasks.workunit.client.0.smithi022.stdout:6/112: symlink d1/l26 0 2024-02-16T19:28:53.703 INFO:tasks.workunit.client.0.smithi022.stdout:3/110: write d7/d12/d18/dd/f20 [1277643,80636] 0 2024-02-16T19:28:53.704 INFO:tasks.workunit.client.0.smithi022.stdout:3/111: write d7/d12/f14 [2152262,42419] 0 2024-02-16T19:28:53.704 INFO:tasks.workunit.client.1.smithi117.stdout:9/32: symlink l9 0 2024-02-16T19:28:53.705 INFO:tasks.workunit.client.0.smithi022.stdout:0/160: mknod d10/d15/d23/d29/c35 0 2024-02-16T19:28:53.705 INFO:tasks.workunit.client.0.smithi022.stdout:3/112: fsync d7/d21/f22 0 2024-02-16T19:28:53.706 INFO:tasks.workunit.client.0.smithi022.stdout:8/117: mknod d3/d6/d23/c29 0 2024-02-16T19:28:53.710 INFO:tasks.workunit.client.0.smithi022.stdout:3/113: dread d7/d12/d18/dd/f28 [0,4194304] 0 2024-02-16T19:28:53.710 INFO:tasks.workunit.client.0.smithi022.stdout:1/194: fdatasync d6/f19 0 2024-02-16T19:28:53.711 INFO:tasks.workunit.client.0.smithi022.stdout:3/114: chown d7/d21/f23 1452600 1 2024-02-16T19:28:53.711 INFO:tasks.workunit.client.1.smithi117.stdout:0/97: creat d0/d9/d14/d23/f24 x:0 0 0 2024-02-16T19:28:53.713 INFO:tasks.workunit.client.0.smithi022.stdout:5/112: mkdir dd/d14/d21 0 2024-02-16T19:28:53.714 INFO:tasks.workunit.client.1.smithi117.stdout:7/48: creat d4/f1a x:0 0 0 2024-02-16T19:28:53.715 INFO:tasks.workunit.client.0.smithi022.stdout:2/76: mkdir d0/d2/d18/d19 0 2024-02-16T19:28:53.717 INFO:tasks.workunit.client.0.smithi022.stdout:1/195: dwrite d6/f19 [0,4194304] 0 2024-02-16T19:28:53.717 INFO:tasks.workunit.client.1.smithi117.stdout:4/67: unlink d0/d6/fe 0 2024-02-16T19:28:53.717 INFO:tasks.workunit.client.1.smithi117.stdout:6/85: write d5/db/f16 [973916,60692] 0 2024-02-16T19:28:53.718 INFO:tasks.workunit.client.0.smithi022.stdout:1/196: fsync d6/d8/f12 0 2024-02-16T19:28:53.718 INFO:tasks.workunit.client.1.smithi117.stdout:6/86: chown d5/f1a 0 1 2024-02-16T19:28:53.720 INFO:tasks.workunit.client.1.smithi117.stdout:3/62: rename d1/c11 to d1/dd/c15 0 2024-02-16T19:28:53.720 INFO:tasks.workunit.client.1.smithi117.stdout:3/63: chown d1/dd/c13 5493 1 2024-02-16T19:28:53.724 INFO:tasks.workunit.client.1.smithi117.stdout:1/36: read d2/d4/f9 [988617,56009] 0 2024-02-16T19:28:53.725 INFO:tasks.workunit.client.0.smithi022.stdout:1/197: dwrite d6/d8/d22/f28 [0,4194304] 0 2024-02-16T19:28:53.727 INFO:tasks.workunit.client.0.smithi022.stdout:1/198: write d6/d8/d22/d16/f1e [1332620,31325] 0 2024-02-16T19:28:53.727 INFO:tasks.workunit.client.1.smithi117.stdout:8/76: mkdir d4/d1c 0 2024-02-16T19:28:53.730 INFO:tasks.workunit.client.1.smithi117.stdout:9/33: creat fa x:0 0 0 2024-02-16T19:28:53.731 INFO:tasks.workunit.client.1.smithi117.stdout:5/173: write d2/d3/f6 [7774936,80486] 0 2024-02-16T19:28:53.732 INFO:tasks.workunit.client.1.smithi117.stdout:1/37: dwrite d2/d4/f9 [0,4194304] 0 2024-02-16T19:28:53.732 INFO:tasks.workunit.client.1.smithi117.stdout:9/34: read f4 [1283071,84376] 0 2024-02-16T19:28:53.734 INFO:tasks.workunit.client.1.smithi117.stdout:5/174: chown d2/d3/f12 2814 1 2024-02-16T19:28:53.736 INFO:tasks.workunit.client.0.smithi022.stdout:1/199: dwrite d6/d8/d22/d16/f1e [0,4194304] 0 2024-02-16T19:28:53.736 INFO:tasks.workunit.client.0.smithi022.stdout:4/100: link d4/d6/f16 d4/f23 0 2024-02-16T19:28:53.740 INFO:tasks.workunit.client.1.smithi117.stdout:0/98: unlink d0/l16 0 2024-02-16T19:28:53.744 INFO:tasks.workunit.client.1.smithi117.stdout:7/49: mkdir d4/d1b 0 2024-02-16T19:28:53.746 INFO:tasks.workunit.client.1.smithi117.stdout:4/68: creat d0/d6/ff x:0 0 0 2024-02-16T19:28:53.747 INFO:tasks.workunit.client.0.smithi022.stdout:2/77: creat d0/d2/d18/f1a x:0 0 0 2024-02-16T19:28:53.747 INFO:tasks.workunit.client.1.smithi117.stdout:6/87: creat d5/f1c x:0 0 0 2024-02-16T19:28:53.749 INFO:tasks.workunit.client.0.smithi022.stdout:6/113: dwrite d1/d8/ff [0,4194304] 0 2024-02-16T19:28:53.749 INFO:tasks.workunit.client.1.smithi117.stdout:3/64: symlink d1/l16 0 2024-02-16T19:28:53.751 INFO:tasks.workunit.client.1.smithi117.stdout:2/52: getdents d0/d3/d10 0 2024-02-16T19:28:53.759 INFO:tasks.workunit.client.1.smithi117.stdout:7/50: mkdir d4/d1c 0 2024-02-16T19:28:53.760 INFO:tasks.workunit.client.1.smithi117.stdout:7/51: chown d4/f11 1163 1 2024-02-16T19:28:53.760 INFO:tasks.workunit.client.0.smithi022.stdout:4/101: mkdir d4/d6/dd/df/d10/d24 0 2024-02-16T19:28:53.762 INFO:tasks.workunit.client.1.smithi117.stdout:4/69: unlink d0/l5 0 2024-02-16T19:28:53.762 INFO:tasks.workunit.client.1.smithi117.stdout:4/70: truncate d0/d6/ff 591793 0 2024-02-16T19:28:53.762 INFO:tasks.workunit.client.1.smithi117.stdout:4/71: chown d0/da 474 1 2024-02-16T19:28:53.763 INFO:tasks.workunit.client.1.smithi117.stdout:4/72: readlink d0/da/lb 0 2024-02-16T19:28:53.764 INFO:tasks.workunit.client.1.smithi117.stdout:6/88: mkdir d5/d1d 0 2024-02-16T19:28:53.765 INFO:tasks.workunit.client.0.smithi022.stdout:5/113: getdents dd/d14/d21 0 2024-02-16T19:28:53.765 INFO:tasks.workunit.client.1.smithi117.stdout:6/89: dread - d5/f15 zero size 2024-02-16T19:28:53.766 INFO:tasks.workunit.client.1.smithi117.stdout:6/90: chown f4 28547923 1 2024-02-16T19:28:53.767 INFO:tasks.workunit.client.1.smithi117.stdout:3/65: rename d1/f5 to d1/dd/f17 0 2024-02-16T19:28:53.770 INFO:tasks.workunit.client.1.smithi117.stdout:4/73: dwrite d0/d6/ff [0,4194304] 0 2024-02-16T19:28:53.770 INFO:tasks.workunit.client.1.smithi117.stdout:2/53: rename d0/f1 to d0/f12 0 2024-02-16T19:28:53.771 INFO:tasks.workunit.client.0.smithi022.stdout:9/139: getdents db/d14/d16/d27 0 2024-02-16T19:28:53.772 INFO:tasks.workunit.client.0.smithi022.stdout:0/161: link d10/d15/c1a d10/d22/d2c/c36 0 2024-02-16T19:28:53.774 INFO:tasks.workunit.client.1.smithi117.stdout:8/77: write d4/d7/fc [4023617,20489] 0 2024-02-16T19:28:53.775 INFO:tasks.workunit.client.0.smithi022.stdout:7/108: truncate d9/da/d1f/f22 3512177 0 2024-02-16T19:28:53.775 INFO:tasks.workunit.client.1.smithi117.stdout:2/54: dread d0/f12 [0,4194304] 0 2024-02-16T19:28:53.775 INFO:tasks.workunit.client.1.smithi117.stdout:8/78: readlink d4/d7/l14 0 2024-02-16T19:28:53.776 INFO:tasks.workunit.client.0.smithi022.stdout:4/102: fsync d4/f23 0 2024-02-16T19:28:53.777 INFO:tasks.workunit.client.1.smithi117.stdout:9/35: rename f4 to fb 0 2024-02-16T19:28:53.779 INFO:tasks.workunit.client.0.smithi022.stdout:5/114: mknod dd/c22 0 2024-02-16T19:28:53.779 INFO:tasks.workunit.client.0.smithi022.stdout:5/115: chown dd/f18 3 1 2024-02-16T19:28:53.780 INFO:tasks.workunit.client.1.smithi117.stdout:5/175: creat d2/d9/de/d17/d1c/d2d/f34 x:0 0 0 2024-02-16T19:28:53.781 INFO:tasks.workunit.client.1.smithi117.stdout:0/99: write d0/fe [2745066,98167] 0 2024-02-16T19:28:53.785 INFO:tasks.workunit.client.0.smithi022.stdout:5/116: dwrite fc [0,4194304] 0 2024-02-16T19:28:53.785 INFO:tasks.workunit.client.0.smithi022.stdout:9/140: symlink db/d14/d16/l35 0 2024-02-16T19:28:53.787 INFO:tasks.workunit.client.0.smithi022.stdout:3/115: truncate d7/d12/d18/f1a 2305367 0 2024-02-16T19:28:53.788 INFO:tasks.workunit.client.0.smithi022.stdout:3/116: dread - d7/d12/f19 zero size 2024-02-16T19:28:53.792 INFO:tasks.workunit.client.0.smithi022.stdout:8/118: getdents d3/d12 0 2024-02-16T19:28:53.793 INFO:tasks.workunit.client.1.smithi117.stdout:7/52: rename d4/f7 to d4/d1c/f1d 0 2024-02-16T19:28:53.794 INFO:tasks.workunit.client.0.smithi022.stdout:8/119: dread d3/d12/d17/f1e [0,4194304] 0 2024-02-16T19:28:53.800 INFO:tasks.workunit.client.1.smithi117.stdout:4/74: rename d0/f9 to d0/f10 0 2024-02-16T19:28:53.801 INFO:tasks.workunit.client.0.smithi022.stdout:2/78: creat d0/f1b x:0 0 0 2024-02-16T19:28:53.802 INFO:tasks.workunit.client.1.smithi117.stdout:1/38: creat d2/ff x:0 0 0 2024-02-16T19:28:53.802 INFO:tasks.workunit.client.0.smithi022.stdout:2/79: truncate d0/f1b 1030067 0 2024-02-16T19:28:53.802 INFO:tasks.workunit.client.0.smithi022.stdout:6/114: link d1/c9 d1/d8/da/d17/c27 0 2024-02-16T19:28:53.808 INFO:tasks.workunit.client.1.smithi117.stdout:6/91: dwrite f1 [0,4194304] 0 2024-02-16T19:28:53.808 INFO:tasks.workunit.client.0.smithi022.stdout:2/80: dwrite d0/d2/d18/f1a [0,4194304] 0 2024-02-16T19:28:53.809 INFO:tasks.workunit.client.0.smithi022.stdout:2/81: read d0/f17 [182308,29810] 0 2024-02-16T19:28:53.810 INFO:tasks.workunit.client.1.smithi117.stdout:9/36: symlink lc 0 2024-02-16T19:28:53.810 INFO:tasks.workunit.client.1.smithi117.stdout:9/37: stat lc 0 2024-02-16T19:28:53.811 INFO:tasks.workunit.client.0.smithi022.stdout:2/82: dread d0/f17 [0,4194304] 0 2024-02-16T19:28:53.812 INFO:tasks.workunit.client.0.smithi022.stdout:0/162: mknod d10/d22/d31/c37 0 2024-02-16T19:28:53.813 INFO:tasks.workunit.client.0.smithi022.stdout:2/83: dread d0/f1b [0,4194304] 0 2024-02-16T19:28:53.814 INFO:tasks.workunit.client.0.smithi022.stdout:2/84: dread d0/f17 [0,4194304] 0 2024-02-16T19:28:53.814 INFO:tasks.workunit.client.1.smithi117.stdout:0/100: creat d0/d9/d14/d1d/d1e/f25 x:0 0 0 2024-02-16T19:28:53.815 INFO:tasks.workunit.client.1.smithi117.stdout:9/38: dread fb [0,4194304] 0 2024-02-16T19:28:53.815 INFO:tasks.workunit.client.0.smithi022.stdout:2/85: chown d0/d2/l15 18302 1 2024-02-16T19:28:53.815 INFO:tasks.workunit.client.1.smithi117.stdout:0/101: chown d0/d9/d14/f1b 10 1 2024-02-16T19:28:53.816 INFO:tasks.workunit.client.1.smithi117.stdout:3/66: truncate f0 894534 0 2024-02-16T19:28:53.818 INFO:tasks.workunit.client.0.smithi022.stdout:8/120: dread d3/d6/f1f [0,4194304] 0 2024-02-16T19:28:53.819 INFO:tasks.workunit.client.0.smithi022.stdout:5/117: creat dd/d14/d21/f23 x:0 0 0 2024-02-16T19:28:53.819 INFO:tasks.workunit.client.0.smithi022.stdout:8/121: fdatasync d3/f10 0 2024-02-16T19:28:53.820 INFO:tasks.workunit.client.1.smithi117.stdout:7/53: creat d4/f1e x:0 0 0 2024-02-16T19:28:53.820 INFO:tasks.workunit.client.0.smithi022.stdout:5/118: truncate dd/d14/f1c 404635 0 2024-02-16T19:28:53.822 INFO:tasks.workunit.client.1.smithi117.stdout:4/75: creat d0/f11 x:0 0 0 2024-02-16T19:28:53.822 INFO:tasks.workunit.client.1.smithi117.stdout:4/76: dread - d0/f11 zero size 2024-02-16T19:28:53.822 INFO:tasks.workunit.client.1.smithi117.stdout:4/77: stat d0/d6 0 2024-02-16T19:28:53.823 INFO:tasks.workunit.client.0.smithi022.stdout:9/141: mknod db/d14/d16/d21/c36 0 2024-02-16T19:28:53.826 INFO:tasks.workunit.client.0.smithi022.stdout:5/119: dwrite f9 [0,4194304] 0 2024-02-16T19:28:53.828 INFO:tasks.workunit.client.0.smithi022.stdout:3/117: mknod d7/d12/d18/dd/d24/d29/c30 0 2024-02-16T19:28:53.828 INFO:tasks.workunit.client.1.smithi117.stdout:4/78: dwrite d0/f11 [0,4194304] 0 2024-02-16T19:28:53.830 INFO:tasks.workunit.client.0.smithi022.stdout:3/118: truncate d7/d12/d18/dd/d24/d29/f2c 825722 0 2024-02-16T19:28:53.831 INFO:tasks.workunit.client.0.smithi022.stdout:0/163: mkdir d10/d15/d23/d38 0 2024-02-16T19:28:53.831 INFO:tasks.workunit.client.0.smithi022.stdout:3/119: write d7/f1c [613569,70881] 0 2024-02-16T19:28:53.832 INFO:tasks.workunit.client.0.smithi022.stdout:3/120: dread - d7/d12/d18/dd/d24/f2f zero size 2024-02-16T19:28:53.838 INFO:tasks.workunit.client.0.smithi022.stdout:2/86: fsync d0/f17 0 2024-02-16T19:28:53.839 INFO:tasks.workunit.client.1.smithi117.stdout:3/67: symlink d1/l18 0 2024-02-16T19:28:53.842 INFO:tasks.workunit.client.0.smithi022.stdout:0/164: dwrite d10/d22/f20 [0,4194304] 0 2024-02-16T19:28:53.843 INFO:tasks.workunit.client.0.smithi022.stdout:3/121: dwrite d7/d21/f22 [0,4194304] 0 2024-02-16T19:28:53.844 INFO:tasks.workunit.client.1.smithi117.stdout:1/39: mknod d2/d4/c10 0 2024-02-16T19:28:53.845 INFO:tasks.workunit.client.1.smithi117.stdout:1/40: chown d2/d4/fa 397734 1 2024-02-16T19:28:53.846 INFO:tasks.workunit.client.1.smithi117.stdout:1/41: write d2/ff [821117,130235] 0 2024-02-16T19:28:53.851 INFO:tasks.workunit.client.1.smithi117.stdout:6/92: creat d5/d18/f1e x:0 0 0 2024-02-16T19:28:53.854 INFO:tasks.workunit.client.0.smithi022.stdout:0/165: dwrite d10/f1f [0,4194304] 0 2024-02-16T19:28:53.854 INFO:tasks.workunit.client.1.smithi117.stdout:4/79: symlink d0/da/l12 0 2024-02-16T19:28:53.855 INFO:tasks.workunit.client.0.smithi022.stdout:5/120: creat dd/d14/d21/f24 x:0 0 0 2024-02-16T19:28:53.856 INFO:tasks.workunit.client.0.smithi022.stdout:5/121: write f5 [1550326,65000] 0 2024-02-16T19:28:53.856 INFO:tasks.workunit.client.1.smithi117.stdout:8/79: truncate d4/f6 623795 0 2024-02-16T19:28:53.856 INFO:tasks.workunit.client.0.smithi022.stdout:5/122: stat dd/lf 0 2024-02-16T19:28:53.858 INFO:tasks.workunit.client.1.smithi117.stdout:2/55: truncate d0/d3/f9 2158588 0 2024-02-16T19:28:53.864 INFO:tasks.workunit.client.1.smithi117.stdout:8/80: dwrite d4/d12/f1a [0,4194304] 0 2024-02-16T19:28:53.864 INFO:tasks.workunit.client.0.smithi022.stdout:4/103: getdents d4/d6/d14 0 2024-02-16T19:28:53.866 INFO:tasks.workunit.client.0.smithi022.stdout:6/115: truncate d1/d8/fb 584112 0 2024-02-16T19:28:53.866 INFO:tasks.workunit.client.0.smithi022.stdout:6/116: stat d1/d8/l19 0 2024-02-16T19:28:53.868 INFO:tasks.workunit.client.0.smithi022.stdout:3/122: rmdir d7 39 2024-02-16T19:28:53.871 INFO:tasks.workunit.client.1.smithi117.stdout:4/80: chown d0/f10 231033 1 2024-02-16T19:28:53.871 INFO:tasks.workunit.client.1.smithi117.stdout:8/81: dread d4/d7/fc [0,4194304] 0 2024-02-16T19:28:53.871 INFO:tasks.workunit.client.1.smithi117.stdout:5/176: dwrite d2/d9/de/d17/d1c/f2e [0,4194304] 0 2024-02-16T19:28:53.873 INFO:tasks.workunit.client.0.smithi022.stdout:8/122: rename d3/d12/d17/c1c to d3/d6/d23/c2a 0 2024-02-16T19:28:53.873 INFO:tasks.workunit.client.0.smithi022.stdout:8/123: stat d3/l13 0 2024-02-16T19:28:53.874 INFO:tasks.workunit.client.0.smithi022.stdout:8/124: read d3/f10 [1389749,91373] 0 2024-02-16T19:28:53.876 INFO:tasks.workunit.client.1.smithi117.stdout:2/56: mkdir d0/d13 0 2024-02-16T19:28:53.880 INFO:tasks.workunit.client.1.smithi117.stdout:8/82: dwrite d4/d12/f1a [0,4194304] 0 2024-02-16T19:28:53.880 INFO:tasks.workunit.client.1.smithi117.stdout:7/54: link d4/f10 d4/d1c/f1f 0 2024-02-16T19:28:53.912 INFO:tasks.workunit.client.0.smithi022.stdout:5/123: creat dd/d14/d21/f25 x:0 0 0 2024-02-16T19:28:53.916 INFO:tasks.workunit.client.0.smithi022.stdout:5/124: dread dd/f11 [0,4194304] 0 2024-02-16T19:28:53.917 INFO:tasks.workunit.client.0.smithi022.stdout:4/104: fsync d4/f23 0 2024-02-16T19:28:53.919 INFO:tasks.workunit.client.1.smithi117.stdout:9/39: sync 2024-02-16T19:28:53.921 INFO:tasks.workunit.client.1.smithi117.stdout:3/68: dwrite d1/f9 [0,4194304] 0 2024-02-16T19:28:53.922 INFO:tasks.workunit.client.1.smithi117.stdout:4/81: mknod d0/da/c13 0 2024-02-16T19:28:53.923 INFO:tasks.workunit.client.0.smithi022.stdout:2/87: rename d0/f1b to d0/d2/f1c 0 2024-02-16T19:28:53.925 INFO:tasks.workunit.client.1.smithi117.stdout:5/177: truncate d2/d4/f1e 1794232 0 2024-02-16T19:28:53.927 INFO:tasks.workunit.client.0.smithi022.stdout:8/125: mkdir d3/d12/d2b 0 2024-02-16T19:28:53.928 INFO:tasks.workunit.client.0.smithi022.stdout:8/126: fsync d3/d25/f26 0 2024-02-16T19:28:53.929 INFO:tasks.workunit.client.0.smithi022.stdout:8/127: chown d3/d6/f7 284 1 2024-02-16T19:28:53.933 INFO:tasks.workunit.client.0.smithi022.stdout:9/142: dwrite db/f12 [0,4194304] 0 2024-02-16T19:28:53.936 INFO:tasks.workunit.client.0.smithi022.stdout:5/125: creat dd/d14/f26 x:0 0 0 2024-02-16T19:28:53.945 INFO:tasks.workunit.client.1.smithi117.stdout:9/40: mkdir dd 0 2024-02-16T19:28:53.946 INFO:tasks.workunit.client.0.smithi022.stdout:3/123: mkdir d7/d12/d18/dd/d31 0 2024-02-16T19:28:53.947 INFO:tasks.workunit.client.1.smithi117.stdout:9/41: write fa [185449,92060] 0 2024-02-16T19:28:53.947 INFO:tasks.workunit.client.1.smithi117.stdout:9/42: stat l0 0 2024-02-16T19:28:53.947 INFO:tasks.workunit.client.0.smithi022.stdout:3/124: fdatasync d7/d12/d18/dd/d24/d29/f2c 0 2024-02-16T19:28:53.948 INFO:tasks.workunit.client.1.smithi117.stdout:3/69: creat d1/f19 x:0 0 0 2024-02-16T19:28:53.950 INFO:tasks.workunit.client.1.smithi117.stdout:3/70: write d1/fc [1123713,126673] 0 2024-02-16T19:28:53.950 INFO:tasks.workunit.client.1.smithi117.stdout:6/93: unlink d5/f1b 0 2024-02-16T19:28:53.950 INFO:tasks.workunit.client.1.smithi117.stdout:6/94: stat d5/f11 0 2024-02-16T19:28:53.951 INFO:tasks.workunit.client.1.smithi117.stdout:3/71: write d1/f9 [1622704,40] 0 2024-02-16T19:28:53.951 INFO:tasks.workunit.client.1.smithi117.stdout:6/95: chown d5/d1d 9110 1 2024-02-16T19:28:53.951 INFO:tasks.workunit.client.1.smithi117.stdout:3/72: stat d1/c7 0 2024-02-16T19:28:53.952 INFO:tasks.workunit.client.1.smithi117.stdout:4/82: truncate d0/f10 1159371 0 2024-02-16T19:28:53.953 INFO:tasks.workunit.client.1.smithi117.stdout:4/83: write d0/f11 [1423125,111366] 0 2024-02-16T19:28:53.953 INFO:tasks.workunit.client.0.smithi022.stdout:3/125: dwrite d7/d12/d18/dd/d24/f2f [0,4194304] 0 2024-02-16T19:28:53.954 INFO:tasks.workunit.client.1.smithi117.stdout:4/84: write d0/f11 [3129154,129723] 0 2024-02-16T19:28:53.955 INFO:tasks.workunit.client.1.smithi117.stdout:6/96: dread d5/f12 [0,4194304] 0 2024-02-16T19:28:53.956 INFO:tasks.workunit.client.1.smithi117.stdout:4/85: chown d0/d6/ff 3683 1 2024-02-16T19:28:53.956 INFO:tasks.workunit.client.1.smithi117.stdout:2/57: creat d0/d13/f14 x:0 0 0 2024-02-16T19:28:53.956 INFO:tasks.workunit.client.1.smithi117.stdout:2/58: readlink - no filename 2024-02-16T19:28:53.961 INFO:tasks.workunit.client.1.smithi117.stdout:0/102: sync 2024-02-16T19:28:53.962 INFO:tasks.workunit.client.0.smithi022.stdout:9/143: mkdir db/d14/d16/d21/d37 0 2024-02-16T19:28:53.963 INFO:tasks.workunit.client.0.smithi022.stdout:9/144: chown db/d14/d19/l34 19779777 1 2024-02-16T19:28:53.963 INFO:tasks.workunit.client.0.smithi022.stdout:9/145: chown db/d14/d16/f1f 1881 1 2024-02-16T19:28:53.964 INFO:tasks.workunit.client.1.smithi117.stdout:1/42: getdents d2/d4 0 2024-02-16T19:28:53.964 INFO:tasks.workunit.client.1.smithi117.stdout:1/43: dread - d2/d4/fa zero size 2024-02-16T19:28:53.965 INFO:tasks.workunit.client.0.smithi022.stdout:9/146: chown db/f1d 2241 1 2024-02-16T19:28:53.965 INFO:tasks.workunit.client.1.smithi117.stdout:1/44: chown d2/d4/fa 14989 1 2024-02-16T19:28:53.965 INFO:tasks.workunit.client.0.smithi022.stdout:9/147: write db/d18/f2a [376519,48471] 0 2024-02-16T19:28:53.966 INFO:tasks.workunit.client.1.smithi117.stdout:1/45: write d2/d4/fa [301089,19347] 0 2024-02-16T19:28:53.966 INFO:tasks.workunit.client.1.smithi117.stdout:1/46: write d2/d4/fa [441224,21653] 0 2024-02-16T19:28:53.967 INFO:tasks.workunit.client.0.smithi022.stdout:9/148: write db/d14/d19/f17 [4642841,34262] 0 2024-02-16T19:28:53.967 INFO:tasks.workunit.client.0.smithi022.stdout:4/105: unlink d4/cc 0 2024-02-16T19:28:53.967 INFO:tasks.workunit.client.1.smithi117.stdout:1/47: chown d2/d4/fa 3 1 2024-02-16T19:28:53.968 INFO:tasks.workunit.client.1.smithi117.stdout:1/48: chown d2/ff 348073 1 2024-02-16T19:28:53.970 INFO:tasks.workunit.client.0.smithi022.stdout:2/88: mknod d0/d2/d18/d19/c1d 0 2024-02-16T19:28:53.971 INFO:tasks.workunit.client.1.smithi117.stdout:1/49: dread d2/d4/fa [0,4194304] 0 2024-02-16T19:28:53.973 INFO:tasks.workunit.client.0.smithi022.stdout:0/166: getdents d10/d22/d31 0 2024-02-16T19:28:53.977 INFO:tasks.workunit.client.0.smithi022.stdout:5/126: getdents dd/d1f 0 2024-02-16T19:28:53.980 INFO:tasks.workunit.client.0.smithi022.stdout:5/127: dread f8 [4194304,4194304] 0 2024-02-16T19:28:53.983 INFO:tasks.workunit.client.1.smithi117.stdout:2/59: creat d0/d13/f15 x:0 0 0 2024-02-16T19:28:53.986 INFO:tasks.workunit.client.1.smithi117.stdout:7/55: rmdir d4/d1b 0 2024-02-16T19:28:53.988 INFO:tasks.workunit.client.0.smithi022.stdout:0/167: unlink cd 0 2024-02-16T19:28:53.989 INFO:tasks.workunit.client.1.smithi117.stdout:0/103: fsync d0/d9/d14/f15 0 2024-02-16T19:28:53.989 INFO:tasks.workunit.client.0.smithi022.stdout:2/89: dwrite d0/d2/f1c [0,4194304] 0 2024-02-16T19:28:53.992 INFO:tasks.workunit.client.1.smithi117.stdout:9/43: creat dd/fe x:0 0 0 2024-02-16T19:28:53.993 INFO:tasks.workunit.client.0.smithi022.stdout:8/128: creat d3/f2c x:0 0 0 2024-02-16T19:28:53.994 INFO:tasks.workunit.client.0.smithi022.stdout:6/117: write d1/d8/f11 [773723,71239] 0 2024-02-16T19:28:53.996 INFO:tasks.workunit.client.0.smithi022.stdout:9/149: creat db/d14/d16/d27/d2e/f38 x:0 0 0 2024-02-16T19:28:53.996 INFO:tasks.workunit.client.0.smithi022.stdout:6/118: read d1/d8/da/d17/f1d [693032,78397] 0 2024-02-16T19:28:53.998 INFO:tasks.workunit.client.0.smithi022.stdout:5/128: creat dd/d14/f27 x:0 0 0 2024-02-16T19:28:53.999 INFO:tasks.workunit.client.1.smithi117.stdout:9/44: dwrite fa [0,4194304] 0 2024-02-16T19:28:54.000 INFO:tasks.workunit.client.1.smithi117.stdout:1/50: creat d2/d4/f11 x:0 0 0 2024-02-16T19:28:54.004 INFO:tasks.workunit.client.0.smithi022.stdout:9/150: dwrite db/d14/d16/d27/d2e/f38 [0,4194304] 0 2024-02-16T19:28:54.005 INFO:tasks.workunit.client.0.smithi022.stdout:0/168: creat d10/d15/f39 x:0 0 0 2024-02-16T19:28:54.009 INFO:tasks.workunit.client.1.smithi117.stdout:3/73: symlink d1/de/l1a 0 2024-02-16T19:28:54.010 INFO:tasks.workunit.client.0.smithi022.stdout:2/90: symlink d0/d2/d18/l1e 0 2024-02-16T19:28:54.014 INFO:tasks.workunit.client.0.smithi022.stdout:3/126: creat d7/d12/d18/dd/d24/f32 x:0 0 0 2024-02-16T19:28:54.015 INFO:tasks.workunit.client.0.smithi022.stdout:0/169: dwrite d10/d15/f39 [0,4194304] 0 2024-02-16T19:28:54.016 INFO:tasks.workunit.client.0.smithi022.stdout:8/129: truncate d3/d6/f1f 1988093 0 2024-02-16T19:28:54.018 INFO:tasks.workunit.client.0.smithi022.stdout:8/130: fsync d3/f11 0 2024-02-16T19:28:54.019 INFO:tasks.workunit.client.0.smithi022.stdout:2/91: dread d0/d2/d18/f1a [0,4194304] 0 2024-02-16T19:28:54.022 INFO:tasks.workunit.client.1.smithi117.stdout:2/60: dread d0/d3/f6 [0,4194304] 0 2024-02-16T19:28:54.026 INFO:tasks.workunit.client.1.smithi117.stdout:4/86: creat d0/f14 x:0 0 0 2024-02-16T19:28:54.027 INFO:tasks.workunit.client.0.smithi022.stdout:9/151: symlink db/d14/d19/l39 0 2024-02-16T19:28:54.030 INFO:tasks.workunit.client.1.smithi117.stdout:6/97: unlink f4 0 2024-02-16T19:28:54.033 INFO:tasks.workunit.client.1.smithi117.stdout:9/45: rename dd/fe to dd/ff 0 2024-02-16T19:28:54.033 INFO:tasks.workunit.client.1.smithi117.stdout:9/46: fsync fa 0 2024-02-16T19:28:54.037 INFO:tasks.workunit.client.0.smithi022.stdout:5/129: dread f6 [4194304,4194304] 0 2024-02-16T19:28:54.041 INFO:tasks.workunit.client.1.smithi117.stdout:3/74: mkdir d1/d1b 0 2024-02-16T19:28:54.042 INFO:tasks.workunit.client.0.smithi022.stdout:0/170: symlink d10/d11/d27/l3a 0 2024-02-16T19:28:54.042 INFO:tasks.workunit.client.1.smithi117.stdout:5/178: truncate d2/d4/f1e 2629656 0 2024-02-16T19:28:54.043 INFO:tasks.workunit.client.1.smithi117.stdout:5/179: chown d2/d3/ld 570394 1 2024-02-16T19:28:54.046 INFO:tasks.workunit.client.0.smithi022.stdout:8/131: chown d3/c14 23116 1 2024-02-16T19:28:54.048 INFO:tasks.workunit.client.1.smithi117.stdout:2/61: creat d0/d3/d10/f16 x:0 0 0 2024-02-16T19:28:54.050 INFO:tasks.workunit.client.0.smithi022.stdout:6/119: creat d1/d8/d23/f28 x:0 0 0 2024-02-16T19:28:54.050 INFO:tasks.workunit.client.0.smithi022.stdout:6/120: read d1/f3 [2405769,36040] 0 2024-02-16T19:28:54.051 INFO:tasks.workunit.client.1.smithi117.stdout:2/62: dread d0/d3/d10/f11 [0,4194304] 0 2024-02-16T19:28:54.052 INFO:tasks.workunit.client.1.smithi117.stdout:4/87: symlink d0/d6/l15 0 2024-02-16T19:28:54.053 INFO:tasks.workunit.client.0.smithi022.stdout:9/152: symlink db/d14/d16/d21/l3a 0 2024-02-16T19:28:54.056 INFO:tasks.workunit.client.0.smithi022.stdout:3/127: creat d7/d12/d18/dd/d31/f33 x:0 0 0 2024-02-16T19:28:54.058 INFO:tasks.workunit.client.0.smithi022.stdout:5/130: symlink dd/l28 0 2024-02-16T19:28:54.059 INFO:tasks.workunit.client.1.smithi117.stdout:4/88: dwrite d0/f14 [0,4194304] 0 2024-02-16T19:28:54.059 INFO:tasks.workunit.client.1.smithi117.stdout:6/98: dread d5/ff [0,4194304] 0 2024-02-16T19:28:54.060 INFO:tasks.workunit.client.1.smithi117.stdout:4/89: readlink d0/da/l12 0 2024-02-16T19:28:54.061 INFO:tasks.workunit.client.0.smithi022.stdout:1/200: sync 2024-02-16T19:28:54.063 INFO:tasks.workunit.client.0.smithi022.stdout:8/132: unlink d3/d6/l9 0 2024-02-16T19:28:54.064 INFO:tasks.workunit.client.1.smithi117.stdout:3/75: mknod d1/c1c 0 2024-02-16T19:28:54.065 INFO:tasks.workunit.client.0.smithi022.stdout:6/121: mkdir d1/d8/da/d29 0 2024-02-16T19:28:54.068 INFO:tasks.workunit.client.0.smithi022.stdout:9/153: mkdir db/d14/d19/d1a/d3b 0 2024-02-16T19:28:54.068 INFO:tasks.workunit.client.0.smithi022.stdout:9/154: read db/f12 [2072611,968] 0 2024-02-16T19:28:54.069 INFO:tasks.workunit.client.0.smithi022.stdout:6/122: dread d1/d8/da/f14 [0,4194304] 0 2024-02-16T19:28:54.071 INFO:tasks.workunit.client.1.smithi117.stdout:8/83: rmdir d4 39 2024-02-16T19:28:54.074 INFO:tasks.workunit.client.0.smithi022.stdout:3/128: mknod d7/d12/d18/dd/c34 0 2024-02-16T19:28:54.076 INFO:tasks.workunit.client.1.smithi117.stdout:2/63: rename d0/d3/f9 to d0/d3/d10/f17 0 2024-02-16T19:28:54.080 INFO:tasks.workunit.client.1.smithi117.stdout:2/64: dread d0/d3/fc [0,4194304] 0 2024-02-16T19:28:54.084 INFO:tasks.workunit.client.1.smithi117.stdout:5/180: unlink d2/d9/de/d17/d1c/f20 0 2024-02-16T19:28:54.085 INFO:tasks.workunit.client.0.smithi022.stdout:6/123: write d1/d8/f1f [173868,84983] 0 2024-02-16T19:28:54.090 INFO:tasks.workunit.client.1.smithi117.stdout:0/104: getdents d0/d9/d14/d1d 0 2024-02-16T19:28:54.091 INFO:tasks.workunit.client.1.smithi117.stdout:0/105: write d0/d9/d14/d1d/d1e/f25 [83467,7458] 0 2024-02-16T19:28:54.091 INFO:tasks.workunit.client.1.smithi117.stdout:0/106: chown d0 69125 1 2024-02-16T19:28:54.092 INFO:tasks.workunit.client.1.smithi117.stdout:0/107: stat d0/d9 0 2024-02-16T19:28:54.092 INFO:tasks.workunit.client.1.smithi117.stdout:1/51: getdents d2 0 2024-02-16T19:28:54.093 INFO:tasks.workunit.client.1.smithi117.stdout:2/65: mknod d0/c18 0 2024-02-16T19:28:54.097 INFO:tasks.workunit.client.0.smithi022.stdout:6/124: rename d1/d8/l22 to d1/d8/da/d17/l2a 0 2024-02-16T19:28:54.098 INFO:tasks.workunit.client.1.smithi117.stdout:3/76: mknod d1/d1b/c1d 0 2024-02-16T19:28:54.098 INFO:tasks.workunit.client.0.smithi022.stdout:1/201: link d6/d8/c2a d6/d8/d22/d16/c37 0 2024-02-16T19:28:54.098 INFO:tasks.workunit.client.1.smithi117.stdout:3/77: write d1/f2 [3052295,74607] 0 2024-02-16T19:28:54.099 INFO:tasks.workunit.client.1.smithi117.stdout:9/47: rmdir dd 39 2024-02-16T19:28:54.099 INFO:tasks.workunit.client.0.smithi022.stdout:1/202: stat d6/d8/d22/d16/d17/d1d/f33 0 2024-02-16T19:28:54.100 INFO:tasks.workunit.client.1.smithi117.stdout:5/181: creat d2/d3/f35 x:0 0 0 2024-02-16T19:28:54.101 INFO:tasks.workunit.client.0.smithi022.stdout:8/133: creat d3/f2d x:0 0 0 2024-02-16T19:28:54.103 INFO:tasks.workunit.client.1.smithi117.stdout:6/99: unlink f2 0 2024-02-16T19:28:54.103 INFO:tasks.workunit.client.1.smithi117.stdout:6/100: stat d5/fa 0 2024-02-16T19:28:54.106 INFO:tasks.workunit.client.0.smithi022.stdout:1/203: symlink d6/d8/d22/d16/d17/d26/l38 0 2024-02-16T19:28:54.107 INFO:tasks.workunit.client.1.smithi117.stdout:8/84: dread d4/f6 [0,4194304] 0 2024-02-16T19:28:54.108 INFO:tasks.workunit.client.1.smithi117.stdout:5/182: dwrite d2/d3/d21/f33 [0,4194304] 0 2024-02-16T19:28:54.109 INFO:tasks.workunit.client.0.smithi022.stdout:8/134: rename d3/d6/d23 to d3/d12/d2b/d2e 0 2024-02-16T19:28:54.113 INFO:tasks.workunit.client.0.smithi022.stdout:6/125: creat d1/d24/f2b x:0 0 0 2024-02-16T19:28:54.114 INFO:tasks.workunit.client.0.smithi022.stdout:6/126: write d1/d8/f1c [2030061,127593] 0 2024-02-16T19:28:54.114 INFO:tasks.workunit.client.1.smithi117.stdout:3/78: fsync d1/fa 0 2024-02-16T19:28:54.116 INFO:tasks.workunit.client.0.smithi022.stdout:1/204: dread d6/d8/d22/f15 [0,4194304] 0 2024-02-16T19:28:54.117 INFO:tasks.workunit.client.0.smithi022.stdout:1/205: write d6/d8/f2d [762434,87781] 0 2024-02-16T19:28:54.118 INFO:tasks.workunit.client.1.smithi117.stdout:7/56: write d4/d1c/f1f [4187813,90479] 0 2024-02-16T19:28:54.118 INFO:tasks.workunit.client.1.smithi117.stdout:7/57: stat d4/c12 0 2024-02-16T19:28:54.118 INFO:tasks.workunit.client.0.smithi022.stdout:1/206: readlink d6/d8/d22/l1f 0 2024-02-16T19:28:54.120 INFO:tasks.workunit.client.0.smithi022.stdout:8/135: dread d3/f15 [0,4194304] 0 2024-02-16T19:28:54.121 INFO:tasks.workunit.client.1.smithi117.stdout:7/58: dread d4/f10 [0,4194304] 0 2024-02-16T19:28:54.121 INFO:tasks.workunit.client.0.smithi022.stdout:8/136: write d3/d12/d17/f18 [167232,118776] 0 2024-02-16T19:28:54.123 INFO:tasks.workunit.client.0.smithi022.stdout:9/155: getdents db/d14/d16/d21 0 2024-02-16T19:28:54.126 INFO:tasks.workunit.client.1.smithi117.stdout:6/101: rename f1 to d5/db/f1f 0 2024-02-16T19:28:54.128 INFO:tasks.workunit.client.0.smithi022.stdout:0/171: write d10/d22/f1c [1130894,9243] 0 2024-02-16T19:28:54.131 INFO:tasks.workunit.client.0.smithi022.stdout:8/137: unlink d3/c14 0 2024-02-16T19:28:54.132 INFO:tasks.workunit.client.0.smithi022.stdout:0/172: dread d10/d22/f2d [0,4194304] 0 2024-02-16T19:28:54.132 INFO:tasks.workunit.client.0.smithi022.stdout:8/138: dread d3/f15 [0,4194304] 0 2024-02-16T19:28:54.132 INFO:tasks.workunit.client.0.smithi022.stdout:0/173: stat d10/d22/f1c 0 2024-02-16T19:28:54.133 INFO:tasks.workunit.client.0.smithi022.stdout:0/174: fsync d10/d22/f1c 0 2024-02-16T19:28:54.134 INFO:tasks.workunit.client.1.smithi117.stdout:8/85: readlink d4/d12/l15 0 2024-02-16T19:28:54.135 INFO:tasks.workunit.client.0.smithi022.stdout:0/175: write d10/d11/f2e [103367,118953] 0 2024-02-16T19:28:54.136 INFO:tasks.workunit.client.0.smithi022.stdout:0/176: fsync d10/d15/f39 0 2024-02-16T19:28:54.136 INFO:tasks.workunit.client.0.smithi022.stdout:8/139: dread d3/d6/f28 [0,4194304] 0 2024-02-16T19:28:54.143 INFO:tasks.workunit.client.0.smithi022.stdout:0/177: dwrite d10/d11/f16 [0,4194304] 0 2024-02-16T19:28:54.145 INFO:tasks.workunit.client.1.smithi117.stdout:2/66: rmdir d0 39 2024-02-16T19:28:54.145 INFO:tasks.workunit.client.1.smithi117.stdout:2/67: readlink - no filename 2024-02-16T19:28:54.146 INFO:tasks.workunit.client.0.smithi022.stdout:5/131: dwrite f9 [4194304,4194304] 0 2024-02-16T19:28:54.147 INFO:tasks.workunit.client.1.smithi117.stdout:4/90: truncate d0/d6/ff 2237332 0 2024-02-16T19:28:54.150 INFO:tasks.workunit.client.0.smithi022.stdout:5/132: dread f8 [0,4194304] 0 2024-02-16T19:28:54.153 INFO:tasks.workunit.client.1.smithi117.stdout:7/59: mknod d4/d13/c20 0 2024-02-16T19:28:54.153 INFO:tasks.workunit.client.1.smithi117.stdout:7/60: chown d4/d13 159 1 2024-02-16T19:28:54.158 INFO:tasks.workunit.client.1.smithi117.stdout:3/79: rename d1/de to d1/d1b/d1e 0 2024-02-16T19:28:54.159 INFO:tasks.workunit.client.1.smithi117.stdout:6/102: mkdir d5/d18/d20 0 2024-02-16T19:28:54.160 INFO:tasks.workunit.client.0.smithi022.stdout:8/140: creat d3/d6/f2f x:0 0 0 2024-02-16T19:28:54.161 INFO:tasks.workunit.client.0.smithi022.stdout:8/141: write d3/f2d [632309,122753] 0 2024-02-16T19:28:54.162 INFO:tasks.workunit.client.1.smithi117.stdout:1/52: mknod d2/d4/c12 0 2024-02-16T19:28:54.162 INFO:tasks.workunit.client.0.smithi022.stdout:6/127: rmdir d1 39 2024-02-16T19:28:54.164 INFO:tasks.workunit.client.1.smithi117.stdout:0/108: dwrite d0/f1 [0,4194304] 0 2024-02-16T19:28:54.164 INFO:tasks.workunit.client.1.smithi117.stdout:9/48: write fb [5179122,51478] 0 2024-02-16T19:28:54.165 INFO:tasks.workunit.client.0.smithi022.stdout:8/142: dread d3/d25/f26 [0,4194304] 0 2024-02-16T19:28:54.166 INFO:tasks.workunit.client.0.smithi022.stdout:0/178: symlink d10/d15/d23/d29/l3b 0 2024-02-16T19:28:54.167 INFO:tasks.workunit.client.1.smithi117.stdout:3/80: dwrite d1/f8 [0,4194304] 0 2024-02-16T19:28:54.168 INFO:tasks.workunit.client.1.smithi117.stdout:3/81: dread - d1/f19 zero size 2024-02-16T19:28:54.168 INFO:tasks.workunit.client.1.smithi117.stdout:2/68: write d0/d3/fe [417775,11557] 0 2024-02-16T19:28:54.174 INFO:tasks.workunit.client.1.smithi117.stdout:7/61: dread - f3 zero size 2024-02-16T19:28:54.174 INFO:tasks.workunit.client.0.smithi022.stdout:1/207: link d6/d8/d22/c34 d6/c39 0 2024-02-16T19:28:54.176 INFO:tasks.workunit.client.1.smithi117.stdout:7/62: dread d4/f6 [0,4194304] 0 2024-02-16T19:28:54.178 INFO:tasks.workunit.client.1.smithi117.stdout:6/103: write d5/db/f1f [1851184,77461] 0 2024-02-16T19:28:54.179 INFO:tasks.workunit.client.0.smithi022.stdout:9/156: getdents db/d14/d16/d27/d2e 0 2024-02-16T19:28:54.181 INFO:tasks.workunit.client.0.smithi022.stdout:9/157: write db/fc [2890549,6420] 0 2024-02-16T19:28:54.182 INFO:tasks.workunit.client.0.smithi022.stdout:8/143: symlink d3/d6/l30 0 2024-02-16T19:28:54.183 INFO:tasks.workunit.client.0.smithi022.stdout:9/158: dread - db/d14/d19/d1a/f24 zero size 2024-02-16T19:28:54.183 INFO:tasks.workunit.client.1.smithi117.stdout:8/86: mknod d4/c1d 0 2024-02-16T19:28:54.184 INFO:tasks.workunit.client.0.smithi022.stdout:7/109: sync 2024-02-16T19:28:54.184 INFO:tasks.workunit.client.0.smithi022.stdout:8/144: write d3/d25/f26 [159231,80483] 0 2024-02-16T19:28:54.184 INFO:tasks.workunit.client.1.smithi117.stdout:8/87: chown d4/f6 46 1 2024-02-16T19:28:54.185 INFO:tasks.workunit.client.0.smithi022.stdout:1/208: dwrite d6/d8/d22/d16/d17/d1d/f33 [0,4194304] 0 2024-02-16T19:28:54.186 INFO:tasks.workunit.client.0.smithi022.stdout:9/159: fdatasync db/d18/f26 0 2024-02-16T19:28:54.186 INFO:tasks.workunit.client.0.smithi022.stdout:0/179: creat d10/d15/d23/f3c x:0 0 0 2024-02-16T19:28:54.188 INFO:tasks.workunit.client.1.smithi117.stdout:5/183: rmdir d2/d3/d21/d23 0 2024-02-16T19:28:54.193 INFO:tasks.workunit.client.0.smithi022.stdout:5/133: link dd/d14/f26 dd/d14/f29 0 2024-02-16T19:28:54.194 INFO:tasks.workunit.client.1.smithi117.stdout:3/82: mknod d1/c1f 0 2024-02-16T19:28:54.195 INFO:tasks.workunit.client.1.smithi117.stdout:3/83: chown d1/dd/l12 0 1 2024-02-16T19:28:54.196 INFO:tasks.workunit.client.0.smithi022.stdout:6/128: creat d1/d8/da/d17/f2c x:0 0 0 2024-02-16T19:28:54.203 INFO:tasks.workunit.client.0.smithi022.stdout:8/145: mknod d3/d12/c31 0 2024-02-16T19:28:54.205 INFO:tasks.workunit.client.1.smithi117.stdout:6/104: fdatasync d5/f9 0 2024-02-16T19:28:54.206 INFO:tasks.workunit.client.0.smithi022.stdout:1/209: stat d6/d8/c2a 0 2024-02-16T19:28:54.208 INFO:tasks.workunit.client.0.smithi022.stdout:0/180: dread d10/f17 [0,4194304] 0 2024-02-16T19:28:54.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:53 smithi022 ceph-mon[29221]: pgmap v258: 65 pgs: 65 active+clean; 368 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 588 KiB/s rd, 24 MiB/s wr, 242 op/s 2024-02-16T19:28:54.215 INFO:tasks.workunit.client.1.smithi117.stdout:9/49: rename lc to dd/l10 0 2024-02-16T19:28:54.216 INFO:tasks.workunit.client.0.smithi022.stdout:6/129: creat d1/d8/da/d17/f2d x:0 0 0 2024-02-16T19:28:54.218 INFO:tasks.workunit.client.0.smithi022.stdout:8/146: creat d3/d6/f32 x:0 0 0 2024-02-16T19:28:54.221 INFO:tasks.workunit.client.1.smithi117.stdout:7/63: write d4/d13/f16 [4597643,2106] 0 2024-02-16T19:28:54.221 INFO:tasks.workunit.client.0.smithi022.stdout:1/210: mkdir d6/d8/d22/d3a 0 2024-02-16T19:28:54.222 INFO:tasks.workunit.client.0.smithi022.stdout:7/110: write d9/dd/f17 [538218,90703] 0 2024-02-16T19:28:54.226 INFO:tasks.workunit.client.0.smithi022.stdout:9/160: link db/fc db/d14/d19/d1a/d3b/f3c 0 2024-02-16T19:28:54.227 INFO:tasks.workunit.client.0.smithi022.stdout:9/161: fsync db/d18/f2a 0 2024-02-16T19:28:54.227 INFO:tasks.workunit.client.0.smithi022.stdout:5/134: mknod dd/d1f/c2a 0 2024-02-16T19:28:54.228 INFO:tasks.workunit.client.0.smithi022.stdout:8/147: mkdir d3/d25/d33 0 2024-02-16T19:28:54.229 INFO:tasks.workunit.client.1.smithi117.stdout:4/91: dwrite d0/d6/ff [0,4194304] 0 2024-02-16T19:28:54.229 INFO:tasks.workunit.client.1.smithi117.stdout:8/88: mknod d4/c1e 0 2024-02-16T19:28:54.230 INFO:tasks.workunit.client.0.smithi022.stdout:1/211: unlink c4 0 2024-02-16T19:28:54.231 INFO:tasks.workunit.client.1.smithi117.stdout:5/184: mkdir d2/d36 0 2024-02-16T19:28:54.231 INFO:tasks.workunit.client.1.smithi117.stdout:4/92: read d0/f14 [1037791,86842] 0 2024-02-16T19:28:54.233 INFO:tasks.workunit.client.1.smithi117.stdout:9/50: unlink c7 0 2024-02-16T19:28:54.233 INFO:tasks.workunit.client.1.smithi117.stdout:7/64: dwrite d4/f1a [0,4194304] 0 2024-02-16T19:28:54.236 INFO:tasks.workunit.client.1.smithi117.stdout:9/51: dread - dd/ff zero size 2024-02-16T19:28:54.236 INFO:tasks.workunit.client.1.smithi117.stdout:5/185: chown d2/d3/f12 15989578 1 2024-02-16T19:28:54.238 INFO:tasks.workunit.client.1.smithi117.stdout:2/69: dwrite d0/d3/d10/f17 [0,4194304] 0 2024-02-16T19:28:54.240 INFO:tasks.workunit.client.1.smithi117.stdout:9/52: truncate f5 594037 0 2024-02-16T19:28:54.241 INFO:tasks.workunit.client.0.smithi022.stdout:9/162: symlink db/d14/d16/d21/l3d 0 2024-02-16T19:28:54.241 INFO:tasks.workunit.client.0.smithi022.stdout:5/135: creat dd/d1f/f2b x:0 0 0 2024-02-16T19:28:54.242 INFO:tasks.workunit.client.0.smithi022.stdout:8/148: creat d3/d6/f34 x:0 0 0 2024-02-16T19:28:54.242 INFO:tasks.workunit.client.1.smithi117.stdout:6/105: creat d5/d18/d20/f21 x:0 0 0 2024-02-16T19:28:54.242 INFO:tasks.workunit.client.1.smithi117.stdout:5/186: write d2/d3/d21/f33 [3966227,56923] 0 2024-02-16T19:28:54.242 INFO:tasks.workunit.client.0.smithi022.stdout:5/136: readlink dd/le 0 2024-02-16T19:28:54.243 INFO:tasks.workunit.client.1.smithi117.stdout:1/53: rename d2/d4/c10 to d2/d4/c13 0 2024-02-16T19:28:54.245 INFO:tasks.workunit.client.0.smithi022.stdout:1/212: rename d6/d8/d22/c32 to d6/d8/d22/d16/d17/d26/c3b 0 2024-02-16T19:28:54.246 INFO:tasks.workunit.client.1.smithi117.stdout:1/54: write d2/ff [407370,75394] 0 2024-02-16T19:28:54.246 INFO:tasks.workunit.client.0.smithi022.stdout:9/163: dread db/d18/f2a [0,4194304] 0 2024-02-16T19:28:54.248 INFO:tasks.workunit.client.1.smithi117.stdout:4/93: dwrite d0/f14 [4194304,4194304] 0 2024-02-16T19:28:54.250 INFO:tasks.workunit.client.0.smithi022.stdout:7/111: creat d9/f25 x:0 0 0 2024-02-16T19:28:54.251 INFO:tasks.workunit.client.1.smithi117.stdout:5/187: dread d2/d3/d21/f33 [0,4194304] 0 2024-02-16T19:28:54.252 INFO:tasks.workunit.client.1.smithi117.stdout:5/188: chown d2/d3/d21 110 1 2024-02-16T19:28:54.252 INFO:tasks.workunit.client.1.smithi117.stdout:7/65: fdatasync d4/f1a 0 2024-02-16T19:28:54.254 INFO:tasks.workunit.client.0.smithi022.stdout:9/164: dwrite db/d18/f26 [0,4194304] 0 2024-02-16T19:28:54.254 INFO:tasks.workunit.client.0.smithi022.stdout:8/149: unlink d3/l24 0 2024-02-16T19:28:54.255 INFO:tasks.workunit.client.1.smithi117.stdout:5/189: read d2/d3/f6 [2520317,86045] 0 2024-02-16T19:28:54.255 INFO:tasks.workunit.client.1.smithi117.stdout:5/190: stat d2 0 2024-02-16T19:28:54.255 INFO:tasks.workunit.client.0.smithi022.stdout:8/150: dread - d3/d6/f32 zero size 2024-02-16T19:28:54.255 INFO:tasks.workunit.client.0.smithi022.stdout:9/165: chown db/d14/d19/f15 15181037 1 2024-02-16T19:28:54.256 INFO:tasks.workunit.client.0.smithi022.stdout:8/151: stat d3/d25/d33 0 2024-02-16T19:28:54.260 INFO:tasks.workunit.client.0.smithi022.stdout:5/137: unlink dd/c1e 0 2024-02-16T19:28:54.262 INFO:tasks.workunit.client.0.smithi022.stdout:0/181: write d10/d22/f2f [4523391,85377] 0 2024-02-16T19:28:54.262 INFO:tasks.workunit.client.1.smithi117.stdout:9/53: creat dd/f11 x:0 0 0 2024-02-16T19:28:54.263 INFO:tasks.workunit.client.1.smithi117.stdout:9/54: dread - dd/f11 zero size 2024-02-16T19:28:54.263 INFO:tasks.workunit.client.0.smithi022.stdout:0/182: write f7 [3301787,102901] 0 2024-02-16T19:28:54.264 INFO:tasks.workunit.client.1.smithi117.stdout:3/84: truncate d1/f9 1152685 0 2024-02-16T19:28:54.264 INFO:tasks.workunit.client.0.smithi022.stdout:0/183: readlink d10/d11/d27/l3a 0 2024-02-16T19:28:54.266 INFO:tasks.workunit.client.1.smithi117.stdout:2/70: rename d0/d13/f14 to d0/f19 0 2024-02-16T19:28:54.267 INFO:tasks.workunit.client.1.smithi117.stdout:9/55: dread fb [0,4194304] 0 2024-02-16T19:28:54.269 INFO:tasks.workunit.client.1.smithi117.stdout:3/85: dread d1/fc [0,4194304] 0 2024-02-16T19:28:54.270 INFO:tasks.workunit.client.0.smithi022.stdout:9/166: rmdir db/d14/d16/d27 39 2024-02-16T19:28:54.271 INFO:tasks.workunit.client.1.smithi117.stdout:2/71: dread d0/d3/fc [0,4194304] 0 2024-02-16T19:28:54.274 INFO:tasks.workunit.client.1.smithi117.stdout:1/55: mkdir d2/d14 0 2024-02-16T19:28:54.275 INFO:tasks.workunit.client.1.smithi117.stdout:1/56: fsync d2/d4/f11 0 2024-02-16T19:28:54.276 INFO:tasks.workunit.client.1.smithi117.stdout:4/94: symlink d0/l16 0 2024-02-16T19:28:54.277 INFO:tasks.workunit.client.1.smithi117.stdout:4/95: write d0/d6/ff [3019211,110390] 0 2024-02-16T19:28:54.278 INFO:tasks.workunit.client.1.smithi117.stdout:8/89: creat d4/d1c/f1f x:0 0 0 2024-02-16T19:28:54.279 INFO:tasks.workunit.client.0.smithi022.stdout:5/138: dwrite dd/d14/f1c [0,4194304] 0 2024-02-16T19:28:54.279 INFO:tasks.workunit.client.0.smithi022.stdout:0/184: rmdir d10/d11 39 2024-02-16T19:28:54.280 INFO:tasks.workunit.client.0.smithi022.stdout:5/139: stat dd/d14/f29 0 2024-02-16T19:28:54.281 INFO:tasks.workunit.client.0.smithi022.stdout:5/140: read dd/d14/f1c [2677609,76219] 0 2024-02-16T19:28:54.281 INFO:tasks.workunit.client.1.smithi117.stdout:7/66: unlink d4/fc 0 2024-02-16T19:28:54.283 INFO:tasks.workunit.client.0.smithi022.stdout:5/141: truncate dd/d1f/f2b 629223 0 2024-02-16T19:28:54.283 INFO:tasks.workunit.client.1.smithi117.stdout:8/90: dread d4/d12/f1a [0,4194304] 0 2024-02-16T19:28:54.284 INFO:tasks.workunit.client.0.smithi022.stdout:5/142: dread - dd/d14/f26 zero size 2024-02-16T19:28:54.284 INFO:tasks.workunit.client.0.smithi022.stdout:7/112: getdents d9/da/d1f 0 2024-02-16T19:28:54.284 INFO:tasks.workunit.client.0.smithi022.stdout:7/113: rename d9 to d9/dd/d21/d26 22 2024-02-16T19:28:54.285 INFO:tasks.workunit.client.0.smithi022.stdout:7/114: chown d9/da/c1a 1300 1 2024-02-16T19:28:54.286 INFO:tasks.workunit.client.0.smithi022.stdout:7/115: truncate d9/da/d1f/f18 1621819 0 2024-02-16T19:28:54.287 INFO:tasks.workunit.client.0.smithi022.stdout:8/152: creat d3/d25/d33/f35 x:0 0 0 2024-02-16T19:28:54.291 INFO:tasks.workunit.client.1.smithi117.stdout:5/191: dread d2/d4/d16/d24/f26 [0,4194304] 0 2024-02-16T19:28:54.291 INFO:tasks.workunit.client.0.smithi022.stdout:1/213: truncate d6/d8/d22/f11 1666507 0 2024-02-16T19:28:54.294 INFO:tasks.workunit.client.0.smithi022.stdout:0/185: mknod d10/d15/d23/c3d 0 2024-02-16T19:28:54.295 INFO:tasks.workunit.client.0.smithi022.stdout:0/186: write d10/d22/f2f [859774,84397] 0 2024-02-16T19:28:54.296 INFO:tasks.workunit.client.1.smithi117.stdout:9/56: mkdir dd/d12 0 2024-02-16T19:28:54.296 INFO:tasks.workunit.client.0.smithi022.stdout:0/187: truncate d10/f26 721869 0 2024-02-16T19:28:54.296 INFO:tasks.workunit.client.0.smithi022.stdout:0/188: chown f7 14344 1 2024-02-16T19:28:54.300 INFO:tasks.workunit.client.1.smithi117.stdout:3/86: chown d1/dd/f17 1 1 2024-02-16T19:28:54.301 INFO:tasks.workunit.client.1.smithi117.stdout:3/87: read - d1/fa zero size 2024-02-16T19:28:54.302 INFO:tasks.workunit.client.1.smithi117.stdout:3/88: dread - d1/f19 zero size 2024-02-16T19:28:54.302 INFO:tasks.workunit.client.0.smithi022.stdout:5/143: unlink f8 0 2024-02-16T19:28:54.303 INFO:tasks.workunit.client.1.smithi117.stdout:2/72: symlink d0/l1a 0 2024-02-16T19:28:54.303 INFO:tasks.workunit.client.1.smithi117.stdout:9/57: dwrite dd/f11 [0,4194304] 0 2024-02-16T19:28:54.304 INFO:tasks.workunit.client.1.smithi117.stdout:2/73: fsync d0/d3/fe 0 2024-02-16T19:28:54.306 INFO:tasks.workunit.client.1.smithi117.stdout:1/57: chown d2/d4/c6 782 1 2024-02-16T19:28:54.311 INFO:tasks.workunit.client.1.smithi117.stdout:4/96: mkdir d0/da/d17 0 2024-02-16T19:28:54.311 INFO:tasks.workunit.client.1.smithi117.stdout:4/97: read d0/f11 [1071470,110343] 0 2024-02-16T19:28:54.312 INFO:tasks.workunit.client.0.smithi022.stdout:1/214: mknod d6/d8/d22/d16/c3c 0 2024-02-16T19:28:54.313 INFO:tasks.workunit.client.0.smithi022.stdout:1/215: write d6/d8/f12 [1254371,126822] 0 2024-02-16T19:28:54.315 INFO:tasks.workunit.client.1.smithi117.stdout:9/58: fdatasync dd/f11 0 2024-02-16T19:28:54.316 INFO:tasks.workunit.client.1.smithi117.stdout:7/67: creat d4/f21 x:0 0 0 2024-02-16T19:28:54.317 INFO:tasks.workunit.client.0.smithi022.stdout:0/189: stat d10/d11 0 2024-02-16T19:28:54.323 INFO:tasks.workunit.client.0.smithi022.stdout:7/116: dwrite f4 [4194304,4194304] 0 2024-02-16T19:28:54.324 INFO:tasks.workunit.client.0.smithi022.stdout:5/144: dread f3 [0,4194304] 0 2024-02-16T19:28:54.324 INFO:tasks.workunit.client.1.smithi117.stdout:5/192: mknod d2/d4/d16/d24/c37 0 2024-02-16T19:28:54.325 INFO:tasks.workunit.client.0.smithi022.stdout:5/145: write dd/d14/d21/f23 [380724,54594] 0 2024-02-16T19:28:54.326 INFO:tasks.workunit.client.0.smithi022.stdout:5/146: fdatasync dd/d14/f1b 0 2024-02-16T19:28:54.326 INFO:tasks.workunit.client.0.smithi022.stdout:5/147: dread - dd/f1d zero size 2024-02-16T19:28:54.327 INFO:tasks.workunit.client.0.smithi022.stdout:9/167: link db/d14/d19/d1a/c23 db/d14/d16/d21/c3e 0 2024-02-16T19:28:54.327 INFO:tasks.workunit.client.0.smithi022.stdout:5/148: chown dd/l13 12424913 1 2024-02-16T19:28:54.327 INFO:tasks.workunit.client.0.smithi022.stdout:9/168: stat db/d14/d16/f30 0 2024-02-16T19:28:54.328 INFO:tasks.workunit.client.0.smithi022.stdout:5/149: fdatasync dd/d14/d21/f24 0 2024-02-16T19:28:54.328 INFO:tasks.workunit.client.0.smithi022.stdout:9/169: write db/d18/f26 [2232474,87410] 0 2024-02-16T19:28:54.330 INFO:tasks.workunit.client.0.smithi022.stdout:1/216: creat d6/d8/d22/f3d x:0 0 0 2024-02-16T19:28:54.330 INFO:tasks.workunit.client.1.smithi117.stdout:8/91: write d4/d7/fc [3571118,33311] 0 2024-02-16T19:28:54.330 INFO:tasks.workunit.client.1.smithi117.stdout:6/106: link d5/db/l14 d5/d18/d20/l22 0 2024-02-16T19:28:54.333 INFO:tasks.workunit.client.0.smithi022.stdout:0/190: rename d10/d11/f16 to d10/d22/d31/f3e 0 2024-02-16T19:28:54.334 INFO:tasks.workunit.client.0.smithi022.stdout:1/217: dread d6/d8/f2d [0,4194304] 0 2024-02-16T19:28:54.337 INFO:tasks.workunit.client.0.smithi022.stdout:7/117: creat d9/d1b/f27 x:0 0 0 2024-02-16T19:28:54.338 INFO:tasks.workunit.client.1.smithi117.stdout:2/74: mknod d0/d3/c1b 0 2024-02-16T19:28:54.339 INFO:tasks.workunit.client.0.smithi022.stdout:8/153: mknod d3/d12/d17/c36 0 2024-02-16T19:28:54.343 INFO:tasks.workunit.client.0.smithi022.stdout:4/106: sync 2024-02-16T19:28:54.343 INFO:tasks.workunit.client.0.smithi022.stdout:2/92: sync 2024-02-16T19:28:54.343 INFO:tasks.workunit.client.0.smithi022.stdout:5/150: symlink dd/d14/l2c 0 2024-02-16T19:28:54.345 INFO:tasks.workunit.client.1.smithi117.stdout:4/98: mknod d0/d6/c18 0 2024-02-16T19:28:54.348 INFO:tasks.workunit.client.1.smithi117.stdout:9/59: rmdir dd 39 2024-02-16T19:28:54.350 INFO:tasks.workunit.client.0.smithi022.stdout:7/118: dread d9/da/f13 [0,4194304] 0 2024-02-16T19:28:54.351 INFO:tasks.workunit.client.0.smithi022.stdout:5/151: dwrite dd/f1d [0,4194304] 0 2024-02-16T19:28:54.356 INFO:tasks.workunit.client.1.smithi117.stdout:7/68: rmdir d4 39 2024-02-16T19:28:54.356 INFO:tasks.workunit.client.0.smithi022.stdout:0/191: mkdir d10/d11/d27/d3f 0 2024-02-16T19:28:54.356 INFO:tasks.workunit.client.0.smithi022.stdout:0/192: chown d10/d15/d23/d29/c35 430 1 2024-02-16T19:28:54.357 INFO:tasks.workunit.client.0.smithi022.stdout:0/193: write d10/d22/f2f [1121221,42293] 0 2024-02-16T19:28:54.359 INFO:tasks.workunit.client.1.smithi117.stdout:6/107: fsync d5/f13 0 2024-02-16T19:28:54.361 INFO:tasks.workunit.client.1.smithi117.stdout:8/92: truncate d4/f6 1606928 0 2024-02-16T19:28:54.362 INFO:tasks.workunit.client.1.smithi117.stdout:3/89: fsync d1/f2 0 2024-02-16T19:28:54.363 INFO:tasks.workunit.client.0.smithi022.stdout:2/93: fsync d0/d2/d18/f1a 0 2024-02-16T19:28:54.363 INFO:tasks.workunit.client.1.smithi117.stdout:2/75: write d0/d3/fc [4177699,41228] 0 2024-02-16T19:28:54.364 INFO:tasks.workunit.client.0.smithi022.stdout:9/170: mknod db/c3f 0 2024-02-16T19:28:54.368 INFO:tasks.workunit.client.0.smithi022.stdout:5/152: unlink dd/f16 0 2024-02-16T19:28:54.370 INFO:tasks.workunit.client.0.smithi022.stdout:9/171: dwrite db/d14/d19/d1a/f24 [0,4194304] 0 2024-02-16T19:28:54.371 INFO:tasks.workunit.client.0.smithi022.stdout:7/119: mknod d9/da/d1f/c28 0 2024-02-16T19:28:54.373 INFO:tasks.workunit.client.1.smithi117.stdout:7/69: write d4/f17 [581106,66130] 0 2024-02-16T19:28:54.375 INFO:tasks.workunit.client.0.smithi022.stdout:9/172: dread db/d14/d19/f15 [0,4194304] 0 2024-02-16T19:28:54.375 INFO:tasks.workunit.client.1.smithi117.stdout:5/193: mkdir d2/d3/d21/d2f/d30/d38 0 2024-02-16T19:28:54.376 INFO:tasks.workunit.client.0.smithi022.stdout:3/129: sync 2024-02-16T19:28:54.381 INFO:tasks.workunit.client.0.smithi022.stdout:4/107: dread - d4/d6/dd/f1e zero size 2024-02-16T19:28:54.384 INFO:tasks.workunit.client.1.smithi117.stdout:2/76: unlink d0/d3/f6 0 2024-02-16T19:28:54.384 INFO:tasks.workunit.client.1.smithi117.stdout:2/77: chown d0 44993 1 2024-02-16T19:28:54.384 INFO:tasks.workunit.client.0.smithi022.stdout:3/130: dwrite f2 [4194304,4194304] 0 2024-02-16T19:28:54.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:53 smithi117 ceph-mon[35309]: pgmap v258: 65 pgs: 65 active+clean; 368 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 588 KiB/s rd, 24 MiB/s wr, 242 op/s 2024-02-16T19:28:54.387 INFO:tasks.workunit.client.1.smithi117.stdout:1/58: creat d2/d4/f15 x:0 0 0 2024-02-16T19:28:54.387 INFO:tasks.workunit.client.0.smithi022.stdout:1/218: truncate d6/d8/f12 970068 0 2024-02-16T19:28:54.389 INFO:tasks.workunit.client.0.smithi022.stdout:1/219: stat d6/d8/d22/d16/d17/d1d/f21 0 2024-02-16T19:28:54.389 INFO:tasks.workunit.client.1.smithi117.stdout:3/90: write d1/fc [2236912,45541] 0 2024-02-16T19:28:54.391 INFO:tasks.workunit.client.0.smithi022.stdout:2/94: dwrite d0/d2/d18/f1a [4194304,4194304] 0 2024-02-16T19:28:54.391 INFO:tasks.workunit.client.1.smithi117.stdout:9/60: chown dd/l10 1013 1 2024-02-16T19:28:54.393 INFO:tasks.workunit.client.1.smithi117.stdout:7/70: rename d4/ca to d4/c22 0 2024-02-16T19:28:54.394 INFO:tasks.workunit.client.0.smithi022.stdout:0/194: creat d10/d15/d23/d38/f40 x:0 0 0 2024-02-16T19:28:54.396 INFO:tasks.workunit.client.1.smithi117.stdout:5/194: dread d2/d4/d16/f1a [0,4194304] 0 2024-02-16T19:28:54.396 INFO:tasks.workunit.client.1.smithi117.stdout:5/195: dread - d2/d3/f35 zero size 2024-02-16T19:28:54.398 INFO:tasks.workunit.client.0.smithi022.stdout:8/154: rename d3/d12/d17/f18 to d3/d25/d33/f37 0 2024-02-16T19:28:54.401 INFO:tasks.workunit.client.0.smithi022.stdout:8/155: write d3/d6/f2f [257774,55055] 0 2024-02-16T19:28:54.402 INFO:tasks.workunit.client.0.smithi022.stdout:0/195: dwrite d10/d22/f1c [0,4194304] 0 2024-02-16T19:28:54.403 INFO:tasks.workunit.client.0.smithi022.stdout:0/196: chown d10/d22/f20 158069 1 2024-02-16T19:28:54.403 INFO:tasks.workunit.client.0.smithi022.stdout:8/156: write d3/d6/f27 [562377,89593] 0 2024-02-16T19:28:54.405 INFO:tasks.workunit.client.0.smithi022.stdout:8/157: dread - d3/d12/f22 zero size 2024-02-16T19:28:54.406 INFO:tasks.workunit.client.0.smithi022.stdout:4/108: creat d4/d6/dd/df/d10/f25 x:0 0 0 2024-02-16T19:28:54.406 INFO:tasks.workunit.client.0.smithi022.stdout:8/158: write d3/f11 [4821323,32596] 0 2024-02-16T19:28:54.408 INFO:tasks.workunit.client.0.smithi022.stdout:8/159: read d3/f10 [1305437,80657] 0 2024-02-16T19:28:54.408 INFO:tasks.workunit.client.0.smithi022.stdout:4/109: chown d4/d6/dd/f1e 2 1 2024-02-16T19:28:54.411 INFO:tasks.workunit.client.1.smithi117.stdout:0/109: sync 2024-02-16T19:28:54.415 INFO:tasks.workunit.client.1.smithi117.stdout:6/108: truncate d5/db/f16 1005709 0 2024-02-16T19:28:54.417 INFO:tasks.workunit.client.0.smithi022.stdout:2/95: rmdir d0/d2/d18 39 2024-02-16T19:28:54.417 INFO:tasks.workunit.client.0.smithi022.stdout:2/96: readlink d0/d2/l15 0 2024-02-16T19:28:54.419 INFO:tasks.workunit.client.1.smithi117.stdout:4/99: link d0/da/lb d0/d6/l19 0 2024-02-16T19:28:54.420 INFO:tasks.workunit.client.1.smithi117.stdout:4/100: write d0/f14 [2054859,81963] 0 2024-02-16T19:28:54.423 INFO:tasks.workunit.client.0.smithi022.stdout:0/197: dread d10/f19 [0,4194304] 0 2024-02-16T19:28:54.424 INFO:tasks.workunit.client.0.smithi022.stdout:7/120: truncate f4 3736368 0 2024-02-16T19:28:54.427 INFO:tasks.workunit.client.0.smithi022.stdout:5/153: rename f9 to dd/d14/f2d 0 2024-02-16T19:28:54.430 INFO:tasks.workunit.client.0.smithi022.stdout:1/220: mkdir d6/d8/d22/d3a/d3e 0 2024-02-16T19:28:54.431 INFO:tasks.workunit.client.0.smithi022.stdout:1/221: stat d6/d8/d22/d16/d17/f1a 0 2024-02-16T19:28:54.432 INFO:tasks.workunit.client.1.smithi117.stdout:8/93: link d4/d12/l18 d4/d7/l20 0 2024-02-16T19:28:54.432 INFO:tasks.workunit.client.0.smithi022.stdout:0/198: creat d10/d15/d23/d38/f41 x:0 0 0 2024-02-16T19:28:54.435 INFO:tasks.workunit.client.1.smithi117.stdout:6/109: creat d5/f23 x:0 0 0 2024-02-16T19:28:54.436 INFO:tasks.workunit.client.1.smithi117.stdout:6/110: chown d5/f11 1583449 1 2024-02-16T19:28:54.436 INFO:tasks.workunit.client.0.smithi022.stdout:9/173: rename db/d18/l1e to db/d14/d16/d28/l40 0 2024-02-16T19:28:54.438 INFO:tasks.workunit.client.1.smithi117.stdout:8/94: dwrite d4/d1c/f1f [0,4194304] 0 2024-02-16T19:28:54.440 INFO:tasks.workunit.client.0.smithi022.stdout:5/154: mkdir dd/d1f/d2e 0 2024-02-16T19:28:54.442 INFO:tasks.workunit.client.1.smithi117.stdout:4/101: rename d0/da/d17 to d0/d6/d1a 0 2024-02-16T19:28:54.445 INFO:tasks.workunit.client.1.smithi117.stdout:3/91: fsync f0 0 2024-02-16T19:28:54.446 INFO:tasks.workunit.client.0.smithi022.stdout:4/110: creat d4/d6/dd/df/d1b/f26 x:0 0 0 2024-02-16T19:28:54.448 INFO:tasks.workunit.client.0.smithi022.stdout:2/97: rmdir d0/d2/d18 39 2024-02-16T19:28:54.449 INFO:tasks.workunit.client.0.smithi022.stdout:2/98: fsync d0/d2/f1c 0 2024-02-16T19:28:54.450 INFO:tasks.workunit.client.1.smithi117.stdout:9/61: mkdir dd/d12/d13 0 2024-02-16T19:28:54.451 INFO:tasks.workunit.client.0.smithi022.stdout:7/121: dwrite d9/f14 [0,4194304] 0 2024-02-16T19:28:54.452 INFO:tasks.workunit.client.0.smithi022.stdout:7/122: chown c8 81187 1 2024-02-16T19:28:54.452 INFO:tasks.workunit.client.0.smithi022.stdout:7/123: chown d9/d1b/d23 47 1 2024-02-16T19:28:54.456 INFO:tasks.workunit.client.0.smithi022.stdout:0/199: unlink d10/d11/d27/l28 0 2024-02-16T19:28:54.462 INFO:tasks.workunit.client.1.smithi117.stdout:2/78: write d0/d3/d10/f17 [305151,51474] 0 2024-02-16T19:28:54.463 INFO:tasks.workunit.client.1.smithi117.stdout:0/110: mkdir d0/d9/d22/d26 0 2024-02-16T19:28:54.465 INFO:tasks.workunit.client.1.smithi117.stdout:1/59: creat d2/d4/f16 x:0 0 0 2024-02-16T19:28:54.467 INFO:tasks.workunit.client.0.smithi022.stdout:5/155: mkdir dd/d14/d21/d2f 0 2024-02-16T19:28:54.467 INFO:tasks.workunit.client.1.smithi117.stdout:7/71: truncate d4/d13/f16 3608980 0 2024-02-16T19:28:54.468 INFO:tasks.workunit.client.0.smithi022.stdout:5/156: stat dd/d1f/d2e 0 2024-02-16T19:28:54.469 INFO:tasks.workunit.client.0.smithi022.stdout:1/222: dwrite d6/f9 [0,4194304] 0 2024-02-16T19:28:54.469 INFO:tasks.workunit.client.1.smithi117.stdout:5/196: truncate d2/d3/d21/f33 2377432 0 2024-02-16T19:28:54.469 INFO:tasks.workunit.client.1.smithi117.stdout:5/197: chown d2/d9 6 1 2024-02-16T19:28:54.470 INFO:tasks.workunit.client.0.smithi022.stdout:4/111: write d4/d6/dd/df/d10/f19 [1508474,74340] 0 2024-02-16T19:28:54.470 INFO:tasks.workunit.client.1.smithi117.stdout:6/111: creat d5/f24 x:0 0 0 2024-02-16T19:28:54.474 INFO:tasks.workunit.client.1.smithi117.stdout:8/95: creat d4/d7/f21 x:0 0 0 2024-02-16T19:28:54.475 INFO:tasks.workunit.client.0.smithi022.stdout:2/99: fsync d0/d2/f14 0 2024-02-16T19:28:54.476 INFO:tasks.workunit.client.0.smithi022.stdout:2/100: fsync d0/f17 0 2024-02-16T19:28:54.479 INFO:tasks.workunit.client.0.smithi022.stdout:1/223: dwrite d6/d8/d22/d16/d17/d1d/f33 [0,4194304] 0 2024-02-16T19:28:54.480 INFO:tasks.workunit.client.0.smithi022.stdout:1/224: chown d6/d8/d22/d16/d17/f1a 160739 1 2024-02-16T19:28:54.481 INFO:tasks.workunit.client.0.smithi022.stdout:1/225: chown d6/d8/d22/d16/d17/f1a 5692872 1 2024-02-16T19:28:54.484 INFO:tasks.workunit.client.1.smithi117.stdout:3/92: mkdir d1/d1b/d20 0 2024-02-16T19:28:54.485 INFO:tasks.workunit.client.0.smithi022.stdout:1/226: dread d6/d8/d22/d16/f1e [0,4194304] 0 2024-02-16T19:28:54.488 INFO:tasks.workunit.client.0.smithi022.stdout:5/157: unlink fc 0 2024-02-16T19:28:54.488 INFO:tasks.workunit.client.0.smithi022.stdout:5/158: write dd/d14/f27 [207688,30923] 0 2024-02-16T19:28:54.490 INFO:tasks.workunit.client.1.smithi117.stdout:4/102: dwrite d0/f11 [0,4194304] 0 2024-02-16T19:28:54.491 INFO:tasks.workunit.client.1.smithi117.stdout:2/79: creat d0/d13/f1c x:0 0 0 2024-02-16T19:28:54.493 INFO:tasks.workunit.client.0.smithi022.stdout:4/112: mkdir d4/d6/dd/df/d10/d27 0 2024-02-16T19:28:54.494 INFO:tasks.workunit.client.0.smithi022.stdout:4/113: chown d4 0 1 2024-02-16T19:28:54.495 INFO:tasks.workunit.client.0.smithi022.stdout:2/101: fdatasync d0/d2/d18/f1a 0 2024-02-16T19:28:54.497 INFO:tasks.workunit.client.0.smithi022.stdout:7/124: link d9/d1b/f27 d9/da/f29 0 2024-02-16T19:28:54.498 INFO:tasks.workunit.client.1.smithi117.stdout:7/72: fdatasync d4/d1c/f1d 0 2024-02-16T19:28:54.499 INFO:tasks.workunit.client.1.smithi117.stdout:7/73: chown d4/f11 0 1 2024-02-16T19:28:54.500 INFO:tasks.workunit.client.1.smithi117.stdout:5/198: symlink d2/d9/de/d17/d1c/l39 0 2024-02-16T19:28:54.501 INFO:tasks.workunit.client.1.smithi117.stdout:5/199: readlink d2/d9/de/d17/d1c/l39 0 2024-02-16T19:28:54.501 INFO:tasks.workunit.client.0.smithi022.stdout:0/200: unlink d10/d22/d2c/c36 0 2024-02-16T19:28:54.503 INFO:tasks.workunit.client.1.smithi117.stdout:6/112: symlink d5/l25 0 2024-02-16T19:28:54.504 INFO:tasks.workunit.client.0.smithi022.stdout:0/201: dread d10/d11/f2e [0,4194304] 0 2024-02-16T19:28:54.504 INFO:tasks.workunit.client.0.smithi022.stdout:0/202: stat d10/d22/f1c 0 2024-02-16T19:28:54.504 INFO:tasks.workunit.client.0.smithi022.stdout:0/203: readlink lf 0 2024-02-16T19:28:54.505 INFO:tasks.workunit.client.1.smithi117.stdout:8/96: symlink d4/d1c/l22 0 2024-02-16T19:28:54.505 INFO:tasks.workunit.client.1.smithi117.stdout:8/97: chown d4/c1e 0 1 2024-02-16T19:28:54.506 INFO:tasks.workunit.client.0.smithi022.stdout:1/227: mkdir d6/d8/d22/d3f 0 2024-02-16T19:28:54.515 INFO:tasks.workunit.client.1.smithi117.stdout:0/111: dwrite d0/f10 [4194304,4194304] 0 2024-02-16T19:28:54.518 INFO:tasks.workunit.client.1.smithi117.stdout:3/93: creat d1/d1b/f21 x:0 0 0 2024-02-16T19:28:54.519 INFO:tasks.workunit.client.1.smithi117.stdout:3/94: write d1/f8 [2130148,28117] 0 2024-02-16T19:28:54.520 INFO:tasks.workunit.client.0.smithi022.stdout:2/102: rmdir d0 39 2024-02-16T19:28:54.527 INFO:tasks.workunit.client.0.smithi022.stdout:5/159: dwrite dd/f11 [0,4194304] 0 2024-02-16T19:28:54.528 INFO:tasks.workunit.client.0.smithi022.stdout:7/125: dread d9/da/d1f/f22 [0,4194304] 0 2024-02-16T19:28:54.529 INFO:tasks.workunit.client.0.smithi022.stdout:9/174: getdents db/d14/d19 0 2024-02-16T19:28:54.530 INFO:tasks.workunit.client.0.smithi022.stdout:0/204: rmdir d10/d15/d23/d29 39 2024-02-16T19:28:54.531 INFO:tasks.workunit.client.0.smithi022.stdout:0/205: write d10/d15/d23/f3c [829156,99968] 0 2024-02-16T19:28:54.532 INFO:tasks.workunit.client.0.smithi022.stdout:1/228: creat d6/d8/d22/d16/f40 x:0 0 0 2024-02-16T19:28:54.534 INFO:tasks.workunit.client.0.smithi022.stdout:4/114: creat d4/d6/d14/f28 x:0 0 0 2024-02-16T19:28:54.535 INFO:tasks.workunit.client.0.smithi022.stdout:4/115: dread - d4/d6/d14/f28 zero size 2024-02-16T19:28:54.536 INFO:tasks.workunit.client.0.smithi022.stdout:4/116: write d4/f1f [339729,2929] 0 2024-02-16T19:28:54.536 INFO:tasks.workunit.client.0.smithi022.stdout:2/103: chown d0/c5 535852726 1 2024-02-16T19:28:54.540 INFO:tasks.workunit.client.0.smithi022.stdout:5/160: unlink dd/d14/f27 0 2024-02-16T19:28:54.542 INFO:tasks.workunit.client.0.smithi022.stdout:7/126: unlink d9/da/l15 0 2024-02-16T19:28:54.543 INFO:tasks.workunit.client.0.smithi022.stdout:6/130: sync 2024-02-16T19:28:54.544 INFO:tasks.workunit.client.0.smithi022.stdout:6/131: readlink d1/d8/da/l21 0 2024-02-16T19:28:54.546 INFO:tasks.workunit.client.1.smithi117.stdout:6/113: dwrite d5/f9 [0,4194304] 0 2024-02-16T19:28:54.547 INFO:tasks.workunit.client.0.smithi022.stdout:0/206: mknod d10/d22/c42 0 2024-02-16T19:28:54.549 INFO:tasks.workunit.client.0.smithi022.stdout:7/127: dwrite d9/f25 [0,4194304] 0 2024-02-16T19:28:54.549 INFO:tasks.workunit.client.1.smithi117.stdout:9/62: rmdir dd/d12/d13 0 2024-02-16T19:28:54.550 INFO:tasks.workunit.client.1.smithi117.stdout:0/112: write d0/d9/f12 [745200,31516] 0 2024-02-16T19:28:54.551 INFO:tasks.workunit.client.0.smithi022.stdout:1/229: creat d6/d8/d22/d16/f41 x:0 0 0 2024-02-16T19:28:54.555 INFO:tasks.workunit.client.1.smithi117.stdout:9/63: dwrite f5 [0,4194304] 0 2024-02-16T19:28:54.556 INFO:tasks.workunit.client.0.smithi022.stdout:9/175: write db/d14/d19/f2f [1436360,113915] 0 2024-02-16T19:28:54.558 INFO:tasks.workunit.client.0.smithi022.stdout:9/176: chown db/d14/d16/d28 5010 1 2024-02-16T19:28:54.558 INFO:tasks.workunit.client.1.smithi117.stdout:3/95: write d1/dd/f17 [1167535,104425] 0 2024-02-16T19:28:54.559 INFO:tasks.workunit.client.0.smithi022.stdout:7/128: dwrite d9/f25 [0,4194304] 0 2024-02-16T19:28:54.561 INFO:tasks.workunit.client.1.smithi117.stdout:1/60: rmdir d2/d14 0 2024-02-16T19:28:54.561 INFO:tasks.workunit.client.1.smithi117.stdout:3/96: write d1/fc [2714290,129873] 0 2024-02-16T19:28:54.563 INFO:tasks.workunit.client.1.smithi117.stdout:7/74: rename d4/d1c/f1f to d4/f23 0 2024-02-16T19:28:54.565 INFO:tasks.workunit.client.0.smithi022.stdout:6/132: rename d1/d8/f13 to d1/f2e 0 2024-02-16T19:28:54.566 INFO:tasks.workunit.client.1.smithi117.stdout:9/64: dwrite fa [0,4194304] 0 2024-02-16T19:28:54.569 INFO:tasks.workunit.client.0.smithi022.stdout:1/230: symlink d6/d8/d22/d16/d17/d1d/l42 0 2024-02-16T19:28:54.571 INFO:tasks.workunit.client.1.smithi117.stdout:0/113: mkdir d0/d9/d22/d27 0 2024-02-16T19:28:54.571 INFO:tasks.workunit.client.1.smithi117.stdout:0/114: chown d0/f7 216 1 2024-02-16T19:28:54.573 INFO:tasks.workunit.client.1.smithi117.stdout:4/103: link d0/da/c13 d0/c1b 0 2024-02-16T19:28:54.575 INFO:tasks.workunit.client.0.smithi022.stdout:7/129: unlink d9/dd/l20 0 2024-02-16T19:28:54.578 INFO:tasks.workunit.client.0.smithi022.stdout:1/231: dwrite d6/d8/d22/d16/f41 [0,4194304] 0 2024-02-16T19:28:54.578 INFO:tasks.workunit.client.0.smithi022.stdout:9/177: rename db/d14/d19/f15 to db/d14/d19/f41 0 2024-02-16T19:28:54.580 INFO:tasks.workunit.client.0.smithi022.stdout:5/161: mknod dd/d1f/d2e/c30 0 2024-02-16T19:28:54.581 INFO:tasks.workunit.client.0.smithi022.stdout:5/162: truncate dd/f11 4976920 0 2024-02-16T19:28:54.583 INFO:tasks.workunit.client.0.smithi022.stdout:5/163: write dd/d14/d21/f23 [1179120,44492] 0 2024-02-16T19:28:54.584 INFO:tasks.workunit.client.0.smithi022.stdout:5/164: chown dd/lf 782 1 2024-02-16T19:28:54.584 INFO:tasks.workunit.client.1.smithi117.stdout:3/97: creat d1/d1b/d1e/f22 x:0 0 0 2024-02-16T19:28:54.585 INFO:tasks.workunit.client.0.smithi022.stdout:5/165: write dd/d14/d21/f24 [23594,9121] 0 2024-02-16T19:28:54.585 INFO:tasks.workunit.client.0.smithi022.stdout:6/133: unlink d1/d8/da/le 0 2024-02-16T19:28:54.588 INFO:tasks.workunit.client.1.smithi117.stdout:7/75: mkdir d4/d1c/d24 0 2024-02-16T19:28:54.588 INFO:tasks.workunit.client.1.smithi117.stdout:7/76: chown d4/fd 0 1 2024-02-16T19:28:54.589 INFO:tasks.workunit.client.0.smithi022.stdout:4/117: rmdir d4/d6/dd/df/d10/d27 0 2024-02-16T19:28:54.589 INFO:tasks.workunit.client.0.smithi022.stdout:2/104: truncate d0/d2/f14 423548 0 2024-02-16T19:28:54.591 INFO:tasks.workunit.client.1.smithi117.stdout:9/65: symlink dd/l14 0 2024-02-16T19:28:54.592 INFO:tasks.workunit.client.1.smithi117.stdout:6/114: truncate d5/db/f16 1992836 0 2024-02-16T19:28:54.593 INFO:tasks.workunit.client.0.smithi022.stdout:7/130: symlink d9/dd/l2a 0 2024-02-16T19:28:54.593 INFO:tasks.workunit.client.1.smithi117.stdout:0/115: symlink d0/d9/d22/l28 0 2024-02-16T19:28:54.595 INFO:tasks.workunit.client.1.smithi117.stdout:0/116: dread - d0/d9/d14/d23/f24 zero size 2024-02-16T19:28:54.595 INFO:tasks.workunit.client.1.smithi117.stdout:4/104: creat d0/d6/f1c x:0 0 0 2024-02-16T19:28:54.595 INFO:tasks.workunit.client.1.smithi117.stdout:0/117: truncate d0/f1a 4512207 0 2024-02-16T19:28:54.595 INFO:tasks.workunit.client.1.smithi117.stdout:4/105: chown d0/d6/f1c 52889437 1 2024-02-16T19:28:54.596 INFO:tasks.workunit.client.1.smithi117.stdout:4/106: dread - d0/d6/f1c zero size 2024-02-16T19:28:54.596 INFO:tasks.workunit.client.0.smithi022.stdout:4/118: dwrite d4/d6/dd/f1e [0,4194304] 0 2024-02-16T19:28:54.596 INFO:tasks.workunit.client.0.smithi022.stdout:1/232: creat d6/d8/d22/d16/d17/d26/f43 x:0 0 0 2024-02-16T19:28:54.597 INFO:tasks.workunit.client.0.smithi022.stdout:1/233: write d6/d8/d22/f3d [141798,14383] 0 2024-02-16T19:28:54.598 INFO:tasks.workunit.client.0.smithi022.stdout:5/166: creat dd/d1f/f31 x:0 0 0 2024-02-16T19:28:54.598 INFO:tasks.workunit.client.1.smithi117.stdout:3/98: rmdir d1/dd 39 2024-02-16T19:28:54.600 INFO:tasks.workunit.client.1.smithi117.stdout:5/200: getdents d2/d4/d16/d24 0 2024-02-16T19:28:54.603 INFO:tasks.workunit.client.0.smithi022.stdout:0/207: creat d10/d15/d23/f43 x:0 0 0 2024-02-16T19:28:54.603 INFO:tasks.workunit.client.0.smithi022.stdout:0/208: stat f7 0 2024-02-16T19:28:54.607 INFO:tasks.workunit.client.1.smithi117.stdout:9/66: creat dd/f15 x:0 0 0 2024-02-16T19:28:54.610 INFO:tasks.workunit.client.1.smithi117.stdout:6/115: symlink d5/db/l26 0 2024-02-16T19:28:54.611 INFO:tasks.workunit.client.0.smithi022.stdout:7/131: dread f5 [0,4194304] 0 2024-02-16T19:28:54.612 INFO:tasks.workunit.client.1.smithi117.stdout:0/118: mknod d0/d9/c29 0 2024-02-16T19:28:54.613 INFO:tasks.workunit.client.1.smithi117.stdout:8/98: write d4/f6 [1668799,66438] 0 2024-02-16T19:28:54.613 INFO:tasks.workunit.client.0.smithi022.stdout:9/178: mknod db/d14/d19/d1a/d3b/c42 0 2024-02-16T19:28:54.614 INFO:tasks.workunit.client.1.smithi117.stdout:0/119: fdatasync d0/f1 0 2024-02-16T19:28:54.614 INFO:tasks.workunit.client.0.smithi022.stdout:9/179: chown db/d14/d33 0 1 2024-02-16T19:28:54.614 INFO:tasks.workunit.client.1.smithi117.stdout:0/120: readlink d0/d9/d22/l28 0 2024-02-16T19:28:54.615 INFO:tasks.workunit.client.1.smithi117.stdout:0/121: chown d0/fe 28016 1 2024-02-16T19:28:54.615 INFO:tasks.workunit.client.1.smithi117.stdout:2/80: rename d0/d3/d10/f11 to d0/d3/d10/f1d 0 2024-02-16T19:28:54.616 INFO:tasks.workunit.client.1.smithi117.stdout:2/81: dread - d0/f19 zero size 2024-02-16T19:28:54.618 INFO:tasks.workunit.client.1.smithi117.stdout:4/107: creat d0/d6/d1a/f1d x:0 0 0 2024-02-16T19:28:54.618 INFO:tasks.workunit.client.0.smithi022.stdout:6/134: mknod d1/d8/da/d29/c2f 0 2024-02-16T19:28:54.620 INFO:tasks.workunit.client.1.smithi117.stdout:3/99: creat d1/d1b/d1e/f23 x:0 0 0 2024-02-16T19:28:54.621 INFO:tasks.workunit.client.1.smithi117.stdout:8/99: dwrite d4/f6 [0,4194304] 0 2024-02-16T19:28:54.621 INFO:tasks.workunit.client.0.smithi022.stdout:5/167: rename dd/c12 to dd/d14/c32 0 2024-02-16T19:28:54.623 INFO:tasks.workunit.client.1.smithi117.stdout:5/201: mknod d2/d4/d16/c3a 0 2024-02-16T19:28:54.623 INFO:tasks.workunit.client.1.smithi117.stdout:0/122: dwrite d0/d9/f12 [0,4194304] 0 2024-02-16T19:28:54.625 INFO:tasks.workunit.client.0.smithi022.stdout:0/209: unlink d10/d15/d23/f43 0 2024-02-16T19:28:54.627 INFO:tasks.workunit.client.1.smithi117.stdout:8/100: dread d4/d1c/f1f [0,4194304] 0 2024-02-16T19:28:54.630 INFO:tasks.workunit.client.1.smithi117.stdout:9/67: creat dd/d12/f16 x:0 0 0 2024-02-16T19:28:54.636 INFO:tasks.workunit.client.1.smithi117.stdout:1/61: dwrite d2/d4/fa [0,4194304] 0 2024-02-16T19:28:54.638 INFO:tasks.workunit.client.0.smithi022.stdout:6/135: dread d1/d8/f1f [0,4194304] 0 2024-02-16T19:28:54.643 INFO:tasks.workunit.client.0.smithi022.stdout:5/168: symlink dd/d14/d21/l33 0 2024-02-16T19:28:54.644 INFO:tasks.workunit.client.1.smithi117.stdout:2/82: write d0/fd [495614,96458] 0 2024-02-16T19:28:54.645 INFO:tasks.workunit.client.1.smithi117.stdout:2/83: chown d0/d3/c1b 37638622 1 2024-02-16T19:28:54.648 INFO:tasks.workunit.client.1.smithi117.stdout:2/84: dread d0/f12 [0,4194304] 0 2024-02-16T19:28:54.648 INFO:tasks.workunit.client.0.smithi022.stdout:2/105: truncate d0/d2/d18/f1a 7647877 0 2024-02-16T19:28:54.649 INFO:tasks.workunit.client.0.smithi022.stdout:0/210: dread d10/f19 [0,4194304] 0 2024-02-16T19:28:54.649 INFO:tasks.workunit.client.0.smithi022.stdout:0/211: fsync d10/d15/f39 0 2024-02-16T19:28:54.650 INFO:tasks.workunit.client.1.smithi117.stdout:7/77: write f3 [1000126,46287] 0 2024-02-16T19:28:54.650 INFO:tasks.workunit.client.1.smithi117.stdout:5/202: mkdir d2/d3/d21/d2f/d3b 0 2024-02-16T19:28:54.652 INFO:tasks.workunit.client.1.smithi117.stdout:5/203: dread d2/d4/d16/f1a [0,4194304] 0 2024-02-16T19:28:54.654 INFO:tasks.workunit.client.1.smithi117.stdout:8/101: symlink d4/d7/l23 0 2024-02-16T19:28:54.656 INFO:tasks.workunit.client.1.smithi117.stdout:7/78: dwrite d4/d13/f18 [0,4194304] 0 2024-02-16T19:28:54.656 INFO:tasks.workunit.client.0.smithi022.stdout:6/136: creat d1/d8/d23/f30 x:0 0 0 2024-02-16T19:28:54.656 INFO:tasks.workunit.client.0.smithi022.stdout:0/212: dwrite d10/d15/d23/f3c [0,4194304] 0 2024-02-16T19:28:54.658 INFO:tasks.workunit.client.1.smithi117.stdout:9/68: unlink dd/ff 0 2024-02-16T19:28:54.661 INFO:tasks.workunit.client.1.smithi117.stdout:6/116: creat d5/d1d/f27 x:0 0 0 2024-02-16T19:28:54.665 INFO:tasks.workunit.client.1.smithi117.stdout:9/69: dwrite f5 [0,4194304] 0 2024-02-16T19:28:54.667 INFO:tasks.workunit.client.0.smithi022.stdout:2/106: chown d0/c8 92518211 1 2024-02-16T19:28:54.672 INFO:tasks.workunit.client.1.smithi117.stdout:3/100: rename f0 to d1/dd/f24 0 2024-02-16T19:28:54.675 INFO:tasks.workunit.client.0.smithi022.stdout:7/132: dwrite f4 [0,4194304] 0 2024-02-16T19:28:54.676 INFO:tasks.workunit.client.1.smithi117.stdout:2/85: fdatasync d0/d3/d10/f17 0 2024-02-16T19:28:54.676 INFO:tasks.workunit.client.0.smithi022.stdout:1/234: dwrite d6/d8/f12 [0,4194304] 0 2024-02-16T19:28:54.677 INFO:tasks.workunit.client.0.smithi022.stdout:0/213: symlink d10/d11/d27/l44 0 2024-02-16T19:28:54.678 INFO:tasks.workunit.client.1.smithi117.stdout:3/101: dwrite d1/f19 [0,4194304] 0 2024-02-16T19:28:54.678 INFO:tasks.workunit.client.0.smithi022.stdout:7/133: chown d9/da/f13 21 1 2024-02-16T19:28:54.679 INFO:tasks.workunit.client.1.smithi117.stdout:2/86: dread d0/d3/fe [0,4194304] 0 2024-02-16T19:28:54.679 INFO:tasks.workunit.client.1.smithi117.stdout:3/102: dread - d1/d1b/d1e/f22 zero size 2024-02-16T19:28:54.680 INFO:tasks.workunit.client.0.smithi022.stdout:7/134: chown d9/dd/f17 7040253 1 2024-02-16T19:28:54.681 INFO:tasks.workunit.client.0.smithi022.stdout:7/135: fdatasync d9/dd/f17 0 2024-02-16T19:28:54.682 INFO:tasks.workunit.client.0.smithi022.stdout:5/169: unlink dd/d14/c32 0 2024-02-16T19:28:54.687 INFO:tasks.workunit.client.1.smithi117.stdout:0/123: mknod d0/d9/d22/d26/c2a 0 2024-02-16T19:28:54.688 INFO:tasks.workunit.client.0.smithi022.stdout:9/180: getdents db/d14/d16/d27 0 2024-02-16T19:28:54.689 INFO:tasks.workunit.client.0.smithi022.stdout:7/136: dread d9/f25 [0,4194304] 0 2024-02-16T19:28:54.689 INFO:tasks.workunit.client.0.smithi022.stdout:7/137: chown d9/dd 0 1 2024-02-16T19:28:54.690 INFO:tasks.workunit.client.0.smithi022.stdout:0/214: dwrite d10/d22/f2f [0,4194304] 0 2024-02-16T19:28:54.690 INFO:tasks.workunit.client.1.smithi117.stdout:2/87: dwrite d0/d3/fc [0,4194304] 0 2024-02-16T19:28:54.691 INFO:tasks.workunit.client.1.smithi117.stdout:5/204: rmdir d2/d9/de/d17/d1c/d2d 39 2024-02-16T19:28:54.691 INFO:tasks.workunit.client.1.smithi117.stdout:3/103: dwrite d1/d1b/d1e/f22 [0,4194304] 0 2024-02-16T19:28:54.696 INFO:tasks.workunit.client.0.smithi022.stdout:7/138: dread f5 [0,4194304] 0 2024-02-16T19:28:54.696 INFO:tasks.workunit.client.0.smithi022.stdout:0/215: dread d10/d22/f1c [0,4194304] 0 2024-02-16T19:28:54.697 INFO:tasks.workunit.client.0.smithi022.stdout:7/139: dread - d9/d1b/f27 zero size 2024-02-16T19:28:54.698 INFO:tasks.workunit.client.0.smithi022.stdout:1/235: creat d6/d8/d22/d3a/f44 x:0 0 0 2024-02-16T19:28:54.699 INFO:tasks.workunit.client.0.smithi022.stdout:1/236: chown d6/d8/d22/d16/f40 1330295 1 2024-02-16T19:28:54.700 INFO:tasks.workunit.client.1.smithi117.stdout:7/79: rename d4/ce to d4/c25 0 2024-02-16T19:28:54.704 INFO:tasks.workunit.client.1.smithi117.stdout:0/124: dwrite d0/fe [0,4194304] 0 2024-02-16T19:28:54.704 INFO:tasks.workunit.client.1.smithi117.stdout:7/80: chown d4/d1c/f1d 157386 1 2024-02-16T19:28:54.705 INFO:tasks.workunit.client.1.smithi117.stdout:6/117: rmdir d5 39 2024-02-16T19:28:54.706 INFO:tasks.workunit.client.0.smithi022.stdout:0/216: dwrite d10/d22/f1c [0,4194304] 0 2024-02-16T19:28:54.707 INFO:tasks.workunit.client.1.smithi117.stdout:9/70: symlink dd/d12/l17 0 2024-02-16T19:28:54.713 INFO:tasks.workunit.client.0.smithi022.stdout:6/137: write d1/d8/da/d17/f1d [150962,69724] 0 2024-02-16T19:28:54.719 INFO:tasks.workunit.client.0.smithi022.stdout:9/181: mknod db/d18/c43 0 2024-02-16T19:28:54.724 INFO:tasks.workunit.client.0.smithi022.stdout:7/140: unlink d9/da/c1a 0 2024-02-16T19:28:54.726 INFO:tasks.workunit.client.1.smithi117.stdout:3/104: creat d1/d1b/f25 x:0 0 0 2024-02-16T19:28:54.727 INFO:tasks.workunit.client.0.smithi022.stdout:1/237: mknod d6/d8/d22/d16/d17/c45 0 2024-02-16T19:28:54.728 INFO:tasks.workunit.client.0.smithi022.stdout:8/160: sync 2024-02-16T19:28:54.728 INFO:tasks.workunit.client.0.smithi022.stdout:3/131: sync 2024-02-16T19:28:54.728 INFO:tasks.workunit.client.0.smithi022.stdout:1/238: dread - d6/d8/d22/d16/d17/d26/f43 zero size 2024-02-16T19:28:54.729 INFO:tasks.workunit.client.0.smithi022.stdout:8/161: write d3/d6/f2f [101325,27182] 0 2024-02-16T19:28:54.730 INFO:tasks.workunit.client.1.smithi117.stdout:5/205: dread d2/d4/f1e [0,4194304] 0 2024-02-16T19:28:54.731 INFO:tasks.workunit.client.0.smithi022.stdout:0/217: unlink d10/d15/d23/d38/f40 0 2024-02-16T19:28:54.732 INFO:tasks.workunit.client.1.smithi117.stdout:2/88: dwrite d0/f12 [0,4194304] 0 2024-02-16T19:28:54.733 INFO:tasks.workunit.client.1.smithi117.stdout:2/89: readlink d0/l1a 0 2024-02-16T19:28:54.734 INFO:tasks.workunit.client.1.smithi117.stdout:0/125: unlink d0/d9/f11 0 2024-02-16T19:28:54.735 INFO:tasks.workunit.client.0.smithi022.stdout:5/170: mknod dd/d14/d21/d2f/c34 0 2024-02-16T19:28:54.737 INFO:tasks.workunit.client.0.smithi022.stdout:1/239: dwrite d6/d8/d22/d16/d17/d1d/f21 [0,4194304] 0 2024-02-16T19:28:54.738 INFO:tasks.workunit.client.0.smithi022.stdout:6/138: fdatasync d1/f2 0 2024-02-16T19:28:54.740 INFO:tasks.workunit.client.1.smithi117.stdout:7/81: creat d4/d1c/f26 x:0 0 0 2024-02-16T19:28:54.740 INFO:tasks.workunit.client.0.smithi022.stdout:0/218: dwrite d10/d22/f20 [0,4194304] 0 2024-02-16T19:28:54.740 INFO:tasks.workunit.client.0.smithi022.stdout:6/139: truncate d1/f2e 1248697 0 2024-02-16T19:28:54.745 INFO:tasks.workunit.client.0.smithi022.stdout:9/182: creat db/d14/d19/d1a/d3b/f44 x:0 0 0 2024-02-16T19:28:54.746 INFO:tasks.workunit.client.0.smithi022.stdout:9/183: chown ca 3467083 1 2024-02-16T19:28:54.746 INFO:tasks.workunit.client.1.smithi117.stdout:1/62: creat d2/d4/f17 x:0 0 0 2024-02-16T19:28:54.746 INFO:tasks.workunit.client.0.smithi022.stdout:2/107: write d0/d2/f14 [1185506,105012] 0 2024-02-16T19:28:54.747 INFO:tasks.workunit.client.1.smithi117.stdout:1/63: dread - d2/d4/f11 zero size 2024-02-16T19:28:54.748 INFO:tasks.workunit.client.0.smithi022.stdout:9/184: write db/d14/d16/f1f [1612357,98059] 0 2024-02-16T19:28:54.749 INFO:tasks.workunit.client.1.smithi117.stdout:4/108: getdents d0/d6/d1a 0 2024-02-16T19:28:54.753 INFO:tasks.workunit.client.1.smithi117.stdout:4/109: write d0/d6/ff [2026055,15906] 0 2024-02-16T19:28:54.755 INFO:tasks.workunit.client.0.smithi022.stdout:3/132: creat d7/f35 x:0 0 0 2024-02-16T19:28:54.755 INFO:tasks.workunit.client.0.smithi022.stdout:2/108: dwrite d0/f17 [0,4194304] 0 2024-02-16T19:28:54.756 INFO:tasks.workunit.client.1.smithi117.stdout:4/110: write d0/d6/f1c [1038142,23238] 0 2024-02-16T19:28:54.758 INFO:tasks.workunit.client.1.smithi117.stdout:1/64: dwrite d2/ff [0,4194304] 0 2024-02-16T19:28:54.760 INFO:tasks.workunit.client.0.smithi022.stdout:3/133: truncate d7/d12/d18/dd/d24/f32 81296 0 2024-02-16T19:28:54.760 INFO:tasks.workunit.client.1.smithi117.stdout:2/90: creat d0/f1e x:0 0 0 2024-02-16T19:28:54.766 INFO:tasks.workunit.client.0.smithi022.stdout:2/109: dwrite d0/d2/f1c [0,4194304] 0 2024-02-16T19:28:54.768 INFO:tasks.workunit.client.1.smithi117.stdout:8/102: truncate d4/d7/fc 2224228 0 2024-02-16T19:28:54.769 INFO:tasks.workunit.client.0.smithi022.stdout:8/162: mknod d3/d25/c38 0 2024-02-16T19:28:54.771 INFO:tasks.workunit.client.1.smithi117.stdout:6/118: fsync d5/db/f19 0 2024-02-16T19:28:54.773 INFO:tasks.workunit.client.1.smithi117.stdout:8/103: dread d4/f6 [0,4194304] 0 2024-02-16T19:28:54.774 INFO:tasks.workunit.client.1.smithi117.stdout:7/82: symlink d4/l27 0 2024-02-16T19:28:54.776 INFO:tasks.workunit.client.0.smithi022.stdout:5/171: mkdir dd/d1f/d2e/d35 0 2024-02-16T19:28:54.778 INFO:tasks.workunit.client.1.smithi117.stdout:9/71: write fb [2258158,21016] 0 2024-02-16T19:28:54.781 INFO:tasks.workunit.client.1.smithi117.stdout:3/105: symlink d1/d1b/d20/l26 0 2024-02-16T19:28:54.781 INFO:tasks.workunit.client.1.smithi117.stdout:3/106: fsync d1/dd/f17 0 2024-02-16T19:28:54.782 INFO:tasks.workunit.client.0.smithi022.stdout:6/140: symlink d1/l31 0 2024-02-16T19:28:54.783 INFO:tasks.workunit.client.1.smithi117.stdout:4/111: symlink d0/l1e 0 2024-02-16T19:28:54.784 INFO:tasks.workunit.client.0.smithi022.stdout:4/119: sync 2024-02-16T19:28:54.785 INFO:tasks.workunit.client.1.smithi117.stdout:1/65: mkdir d2/d18 0 2024-02-16T19:28:54.792 INFO:tasks.workunit.client.1.smithi117.stdout:2/91: write d0/d3/d10/f16 [949812,5881] 0 2024-02-16T19:28:54.794 INFO:tasks.workunit.client.0.smithi022.stdout:2/110: mkdir d0/d2/d18/d1f 0 2024-02-16T19:28:54.796 INFO:tasks.workunit.client.0.smithi022.stdout:8/163: mknod d3/d12/d2b/d2e/c39 0 2024-02-16T19:28:54.796 INFO:tasks.workunit.client.1.smithi117.stdout:5/206: dwrite d2/d4/d16/d24/f26 [4194304,4194304] 0 2024-02-16T19:28:54.797 INFO:tasks.workunit.client.1.smithi117.stdout:7/83: creat d4/f28 x:0 0 0 2024-02-16T19:28:54.798 INFO:tasks.workunit.client.1.smithi117.stdout:8/104: rename d4/c1d to d4/d7/c24 0 2024-02-16T19:28:54.800 INFO:tasks.workunit.client.0.smithi022.stdout:9/185: write db/d14/d19/d1a/d3b/f3c [1078705,100356] 0 2024-02-16T19:28:54.802 INFO:tasks.workunit.client.0.smithi022.stdout:9/186: fsync db/d14/d16/f1f 0 2024-02-16T19:28:54.803 INFO:tasks.workunit.client.0.smithi022.stdout:7/141: truncate d9/f14 3462663 0 2024-02-16T19:28:54.803 INFO:tasks.workunit.client.1.smithi117.stdout:4/112: dwrite d0/d6/ff [0,4194304] 0 2024-02-16T19:28:54.803 INFO:tasks.workunit.client.1.smithi117.stdout:8/105: stat d4/d7/l20 0 2024-02-16T19:28:54.805 INFO:tasks.workunit.client.1.smithi117.stdout:1/66: dwrite d2/ff [0,4194304] 0 2024-02-16T19:28:54.805 INFO:tasks.workunit.client.0.smithi022.stdout:6/141: unlink d1/l4 0 2024-02-16T19:28:54.805 INFO:tasks.workunit.client.1.smithi117.stdout:0/126: dwrite d0/f5 [0,4194304] 0 2024-02-16T19:28:54.806 INFO:tasks.workunit.client.0.smithi022.stdout:6/142: chown d1/f2e 366439 1 2024-02-16T19:28:54.808 INFO:tasks.workunit.client.0.smithi022.stdout:4/120: creat d4/d6/dd/df/f29 x:0 0 0 2024-02-16T19:28:54.809 INFO:tasks.workunit.client.0.smithi022.stdout:4/121: chown d4/d6/dd/df/d10/f19 7233 1 2024-02-16T19:28:54.810 INFO:tasks.workunit.client.0.smithi022.stdout:4/122: truncate d4/f1f 716234 0 2024-02-16T19:28:54.811 INFO:tasks.workunit.client.0.smithi022.stdout:4/123: write d4/d6/d14/f1d [285268,57623] 0 2024-02-16T19:28:54.812 INFO:tasks.workunit.client.0.smithi022.stdout:3/134: link f2 d7/d21/f36 0 2024-02-16T19:28:54.814 INFO:tasks.workunit.client.0.smithi022.stdout:4/124: truncate d4/d6/dd/df/d1b/f26 598900 0 2024-02-16T19:28:54.820 INFO:tasks.workunit.client.0.smithi022.stdout:5/172: mkdir dd/d1f/d2e/d35/d36 0 2024-02-16T19:28:54.821 INFO:tasks.workunit.client.1.smithi117.stdout:7/84: rename d4/f21 to d4/d1c/d24/f29 0 2024-02-16T19:28:54.821 INFO:tasks.workunit.client.1.smithi117.stdout:3/107: dread d1/dd/f24 [0,4194304] 0 2024-02-16T19:28:54.822 INFO:tasks.workunit.client.1.smithi117.stdout:4/113: creat d0/d6/f1f x:0 0 0 2024-02-16T19:28:54.824 INFO:tasks.workunit.client.1.smithi117.stdout:8/106: symlink d4/d1c/l25 0 2024-02-16T19:28:54.827 INFO:tasks.workunit.client.1.smithi117.stdout:0/127: symlink d0/l2b 0 2024-02-16T19:28:54.827 INFO:tasks.workunit.client.1.smithi117.stdout:0/128: write d0/f1f [425605,68697] 0 2024-02-16T19:28:54.828 INFO:tasks.workunit.client.0.smithi022.stdout:1/240: write d6/d8/d22/f11 [1604895,105599] 0 2024-02-16T19:28:54.828 INFO:tasks.workunit.client.0.smithi022.stdout:1/241: write d6/f9 [936645,93568] 0 2024-02-16T19:28:54.829 INFO:tasks.workunit.client.1.smithi117.stdout:2/92: mkdir d0/d3/d10/d1f 0 2024-02-16T19:28:54.831 INFO:tasks.workunit.client.0.smithi022.stdout:1/242: read d6/d8/d22/f1b [541642,6627] 0 2024-02-16T19:28:54.831 INFO:tasks.workunit.client.0.smithi022.stdout:6/143: creat d1/f32 x:0 0 0 2024-02-16T19:28:54.832 INFO:tasks.workunit.client.1.smithi117.stdout:2/93: dread d0/d3/d10/f1d [0,4194304] 0 2024-02-16T19:28:54.837 INFO:tasks.workunit.client.1.smithi117.stdout:2/94: dread d0/d3/d10/f1d [0,4194304] 0 2024-02-16T19:28:54.838 INFO:tasks.workunit.client.1.smithi117.stdout:2/95: write d0/d3/d10/f17 [3333489,118116] 0 2024-02-16T19:28:54.838 INFO:tasks.workunit.client.1.smithi117.stdout:2/96: chown d0/l1a 405 1 2024-02-16T19:28:54.840 INFO:tasks.workunit.client.0.smithi022.stdout:3/135: mknod d7/d12/d18/dd/d31/c37 0 2024-02-16T19:28:54.841 INFO:tasks.workunit.client.0.smithi022.stdout:3/136: truncate d7/f1c 1597898 0 2024-02-16T19:28:54.842 INFO:tasks.workunit.client.0.smithi022.stdout:8/164: rename d3/d12/d17/f1e to d3/d12/d2b/f3a 0 2024-02-16T19:28:54.843 INFO:tasks.workunit.client.1.smithi117.stdout:4/114: mknod d0/da/c20 0 2024-02-16T19:28:54.844 INFO:tasks.workunit.client.1.smithi117.stdout:4/115: truncate d0/d6/d1a/f1d 661692 0 2024-02-16T19:28:54.845 INFO:tasks.workunit.client.1.smithi117.stdout:4/116: chown d0/d6/c18 73504 1 2024-02-16T19:28:54.846 INFO:tasks.workunit.client.0.smithi022.stdout:5/173: mknod dd/d14/d21/d2f/c37 0 2024-02-16T19:28:54.847 INFO:tasks.workunit.client.1.smithi117.stdout:0/129: mkdir d0/d9/d2c 0 2024-02-16T19:28:54.847 INFO:tasks.workunit.client.0.smithi022.stdout:9/187: getdents db/d14/d16/d27/d2e 0 2024-02-16T19:28:54.848 INFO:tasks.workunit.client.1.smithi117.stdout:0/130: dread - d0/d9/d14/d1d/d1e/f21 zero size 2024-02-16T19:28:54.848 INFO:tasks.workunit.client.1.smithi117.stdout:9/72: getdents dd/d12 0 2024-02-16T19:28:54.849 INFO:tasks.workunit.client.1.smithi117.stdout:9/73: truncate dd/d12/f16 225064 0 2024-02-16T19:28:54.849 INFO:tasks.workunit.client.0.smithi022.stdout:7/142: symlink d9/dd/d21/l2b 0 2024-02-16T19:28:54.852 INFO:tasks.workunit.client.1.smithi117.stdout:8/107: write d4/d1c/f1f [3796556,114346] 0 2024-02-16T19:28:54.852 INFO:tasks.workunit.client.1.smithi117.stdout:2/97: write d0/d3/d10/f1d [1123466,101112] 0 2024-02-16T19:28:54.853 INFO:tasks.workunit.client.1.smithi117.stdout:3/108: symlink d1/l27 0 2024-02-16T19:28:54.853 INFO:tasks.workunit.client.1.smithi117.stdout:2/98: dread - d0/d13/f1c zero size 2024-02-16T19:28:54.854 INFO:tasks.workunit.client.0.smithi022.stdout:1/243: creat d6/d8/d22/d16/d17/d26/f46 x:0 0 0 2024-02-16T19:28:54.855 INFO:tasks.workunit.client.0.smithi022.stdout:1/244: stat d6/d8/d22/d16/d17/d26/l38 0 2024-02-16T19:28:54.857 INFO:tasks.workunit.client.1.smithi117.stdout:4/117: mkdir d0/da/d21 0 2024-02-16T19:28:54.858 INFO:tasks.workunit.client.0.smithi022.stdout:0/219: getdents d10/d22 0 2024-02-16T19:28:54.858 INFO:tasks.workunit.client.1.smithi117.stdout:0/131: creat d0/d9/d14/d1d/d1e/f2d x:0 0 0 2024-02-16T19:28:54.862 INFO:tasks.workunit.client.0.smithi022.stdout:2/111: creat d0/f20 x:0 0 0 2024-02-16T19:28:54.864 INFO:tasks.workunit.client.0.smithi022.stdout:3/137: creat d7/d21/f38 x:0 0 0 2024-02-16T19:28:54.867 INFO:tasks.workunit.client.1.smithi117.stdout:2/99: symlink d0/d3/l20 0 2024-02-16T19:28:54.868 INFO:tasks.workunit.client.1.smithi117.stdout:2/100: write d0/f1e [1036158,75382] 0 2024-02-16T19:28:54.869 INFO:tasks.workunit.client.1.smithi117.stdout:8/108: dread d4/d12/f1a [0,4194304] 0 2024-02-16T19:28:54.869 INFO:tasks.workunit.client.0.smithi022.stdout:5/174: unlink dd/le 0 2024-02-16T19:28:54.869 INFO:tasks.workunit.client.0.smithi022.stdout:4/125: truncate d4/d6/dd/f1e 936020 0 2024-02-16T19:28:54.870 INFO:tasks.workunit.client.0.smithi022.stdout:5/175: write dd/d14/d21/f25 [762705,102281] 0 2024-02-16T19:28:54.872 INFO:tasks.workunit.client.1.smithi117.stdout:0/132: creat d0/d9/d22/d26/f2e x:0 0 0 2024-02-16T19:28:54.872 INFO:tasks.workunit.client.0.smithi022.stdout:7/143: mkdir d9/da/d2c 0 2024-02-16T19:28:54.873 INFO:tasks.workunit.client.1.smithi117.stdout:9/74: getdents dd/d12 0 2024-02-16T19:28:54.873 INFO:tasks.workunit.client.1.smithi117.stdout:0/133: stat d0/d9/d14/d1d 0 2024-02-16T19:28:54.873 INFO:tasks.workunit.client.1.smithi117.stdout:9/75: fdatasync fa 0 2024-02-16T19:28:54.874 INFO:tasks.workunit.client.0.smithi022.stdout:1/245: fdatasync d6/d8/f2d 0 2024-02-16T19:28:54.874 INFO:tasks.workunit.client.1.smithi117.stdout:0/134: chown d0/f1 738115 1 2024-02-16T19:28:54.874 INFO:tasks.workunit.client.1.smithi117.stdout:7/85: getdents d4/d13 0 2024-02-16T19:28:54.875 INFO:tasks.workunit.client.0.smithi022.stdout:1/246: truncate d6/d8/d22/d16/d17/d26/f43 613947 0 2024-02-16T19:28:54.875 INFO:tasks.workunit.client.0.smithi022.stdout:0/220: creat d10/d22/f45 x:0 0 0 2024-02-16T19:28:54.876 INFO:tasks.workunit.client.1.smithi117.stdout:8/109: dwrite d4/d7/f21 [0,4194304] 0 2024-02-16T19:28:54.877 INFO:tasks.workunit.client.0.smithi022.stdout:1/247: write d6/d8/d22/d16/d17/d26/f43 [1615509,46058] 0 2024-02-16T19:28:54.877 INFO:tasks.workunit.client.1.smithi117.stdout:0/135: write d0/d9/d22/d26/f2e [606457,61546] 0 2024-02-16T19:28:54.882 INFO:tasks.workunit.client.1.smithi117.stdout:0/136: read - d0/d9/d14/f15 zero size 2024-02-16T19:28:54.883 INFO:tasks.workunit.client.0.smithi022.stdout:3/138: creat d7/f39 x:0 0 0 2024-02-16T19:28:54.884 INFO:tasks.workunit.client.1.smithi117.stdout:9/76: dwrite dd/d12/f16 [0,4194304] 0 2024-02-16T19:28:54.884 INFO:tasks.workunit.client.1.smithi117.stdout:7/86: dread d4/d1c/f1d [0,4194304] 0 2024-02-16T19:28:54.885 INFO:tasks.workunit.client.0.smithi022.stdout:1/248: dwrite d6/f1c [0,4194304] 0 2024-02-16T19:28:54.885 INFO:tasks.workunit.client.0.smithi022.stdout:3/139: chown d7/d12/f14 2 1 2024-02-16T19:28:54.890 INFO:tasks.workunit.client.1.smithi117.stdout:4/118: truncate d0/d6/d1a/f1d 560188 0 2024-02-16T19:28:54.893 INFO:tasks.workunit.client.1.smithi117.stdout:9/77: dwrite dd/f15 [0,4194304] 0 2024-02-16T19:28:54.894 INFO:tasks.workunit.client.1.smithi117.stdout:3/109: creat d1/f28 x:0 0 0 2024-02-16T19:28:54.895 INFO:tasks.workunit.client.1.smithi117.stdout:8/110: creat d4/d12/f26 x:0 0 0 2024-02-16T19:28:54.899 INFO:tasks.workunit.client.0.smithi022.stdout:0/221: creat d10/d22/d31/f46 x:0 0 0 2024-02-16T19:28:54.900 INFO:tasks.workunit.client.0.smithi022.stdout:0/222: write d10/d22/d31/f46 [665705,92535] 0 2024-02-16T19:28:54.901 INFO:tasks.workunit.client.0.smithi022.stdout:6/144: link d1/l31 d1/d8/da/l33 0 2024-02-16T19:28:54.906 INFO:tasks.workunit.client.1.smithi117.stdout:9/78: rename dd/d12/f16 to dd/d12/f18 0 2024-02-16T19:28:54.907 INFO:tasks.workunit.client.0.smithi022.stdout:2/112: creat d0/d2/d18/d1f/f21 x:0 0 0 2024-02-16T19:28:54.908 INFO:tasks.workunit.client.1.smithi117.stdout:3/110: mkdir d1/d1b/d1e/d29 0 2024-02-16T19:28:54.911 INFO:tasks.workunit.client.0.smithi022.stdout:8/165: rename d3/d25/f26 to d3/d12/d17/f3b 0 2024-02-16T19:28:54.912 INFO:tasks.workunit.client.0.smithi022.stdout:1/249: fsync d6/d8/d22/f1b 0 2024-02-16T19:28:54.914 INFO:tasks.workunit.client.1.smithi117.stdout:4/119: creat d0/da/d21/f22 x:0 0 0 2024-02-16T19:28:54.915 INFO:tasks.workunit.client.0.smithi022.stdout:9/188: creat db/d14/d16/f45 x:0 0 0 2024-02-16T19:28:54.919 INFO:tasks.workunit.client.1.smithi117.stdout:9/79: creat dd/f19 x:0 0 0 2024-02-16T19:28:54.921 INFO:tasks.workunit.client.1.smithi117.stdout:3/111: truncate d1/dd/f14 1581722 0 2024-02-16T19:28:54.924 INFO:tasks.workunit.client.1.smithi117.stdout:2/101: truncate d0/d3/fc 1547608 0 2024-02-16T19:28:54.925 INFO:tasks.workunit.client.1.smithi117.stdout:7/87: link d4/d13/l15 d4/d1c/d24/l2a 0 2024-02-16T19:28:54.925 INFO:tasks.workunit.client.0.smithi022.stdout:7/144: dwrite d9/d1b/f27 [0,4194304] 0 2024-02-16T19:28:54.926 INFO:tasks.workunit.client.1.smithi117.stdout:0/137: write d0/f7 [4980435,124015] 0 2024-02-16T19:28:54.929 INFO:tasks.workunit.client.0.smithi022.stdout:2/113: creat d0/d2/d18/f22 x:0 0 0 2024-02-16T19:28:54.929 INFO:tasks.workunit.client.1.smithi117.stdout:0/138: dread d0/f1f [0,4194304] 0 2024-02-16T19:28:54.930 INFO:tasks.workunit.client.0.smithi022.stdout:2/114: read d0/f17 [3073678,4403] 0 2024-02-16T19:28:54.930 INFO:tasks.workunit.client.1.smithi117.stdout:0/139: write d0/d9/d22/d26/f2e [550831,74014] 0 2024-02-16T19:28:54.931 INFO:tasks.workunit.client.1.smithi117.stdout:0/140: readlink d0/l8 0 2024-02-16T19:28:54.933 INFO:tasks.workunit.client.0.smithi022.stdout:7/145: dwrite d9/dd/f17 [0,4194304] 0 2024-02-16T19:28:54.934 INFO:tasks.workunit.client.0.smithi022.stdout:8/166: creat d3/d12/d2b/f3c x:0 0 0 2024-02-16T19:28:54.934 INFO:tasks.workunit.client.0.smithi022.stdout:7/146: chown d9/f16 5 1 2024-02-16T19:28:54.941 INFO:tasks.workunit.client.0.smithi022.stdout:1/250: mkdir d6/d8/d22/d47 0 2024-02-16T19:28:54.942 INFO:tasks.workunit.client.0.smithi022.stdout:1/251: write d6/d8/d22/d16/f41 [1269856,84409] 0 2024-02-16T19:28:54.945 INFO:tasks.workunit.client.0.smithi022.stdout:5/176: truncate dd/d14/d21/f25 398196 0 2024-02-16T19:28:54.945 INFO:tasks.workunit.client.0.smithi022.stdout:5/177: write dd/d1f/f31 [869733,53265] 0 2024-02-16T19:28:54.947 INFO:tasks.workunit.client.1.smithi117.stdout:7/88: creat d4/f2b x:0 0 0 2024-02-16T19:28:54.950 INFO:tasks.workunit.client.1.smithi117.stdout:4/120: rename d0/f10 to d0/d6/f23 0 2024-02-16T19:28:54.953 INFO:tasks.workunit.client.1.smithi117.stdout:8/111: dwrite d4/d7/fc [0,4194304] 0 2024-02-16T19:28:54.955 INFO:tasks.workunit.client.1.smithi117.stdout:0/141: mkdir d0/d9/d14/d23/d2f 0 2024-02-16T19:28:54.956 INFO:tasks.workunit.client.1.smithi117.stdout:3/112: write d1/fa [24565,5240] 0 2024-02-16T19:28:54.956 INFO:tasks.workunit.client.1.smithi117.stdout:3/113: readlink d1/l18 0 2024-02-16T19:28:54.957 INFO:tasks.workunit.client.0.smithi022.stdout:2/115: mknod d0/d2/d18/d1f/c23 0 2024-02-16T19:28:54.960 INFO:tasks.workunit.client.1.smithi117.stdout:9/80: unlink dd/f15 0 2024-02-16T19:28:54.961 INFO:tasks.workunit.client.0.smithi022.stdout:9/189: write f7 [348260,57143] 0 2024-02-16T19:28:54.965 INFO:tasks.workunit.client.0.smithi022.stdout:3/140: creat d7/d12/d18/dd/d24/f3a x:0 0 0 2024-02-16T19:28:54.965 INFO:tasks.workunit.client.0.smithi022.stdout:2/116: dwrite d0/f20 [0,4194304] 0 2024-02-16T19:28:54.965 INFO:tasks.workunit.client.1.smithi117.stdout:9/81: dwrite fa [0,4194304] 0 2024-02-16T19:28:54.966 INFO:tasks.workunit.client.1.smithi117.stdout:7/89: unlink d4/f6 0 2024-02-16T19:28:54.970 INFO:tasks.workunit.client.0.smithi022.stdout:4/126: getdents d4/d6 0 2024-02-16T19:28:54.973 INFO:tasks.workunit.client.0.smithi022.stdout:1/252: rmdir d6/d8/d22/d16/d17/d26 39 2024-02-16T19:28:54.977 INFO:tasks.workunit.client.1.smithi117.stdout:4/121: creat d0/f24 x:0 0 0 2024-02-16T19:28:54.978 INFO:tasks.workunit.client.1.smithi117.stdout:4/122: truncate d0/da/d21/f22 759926 0 2024-02-16T19:28:54.978 INFO:tasks.workunit.client.0.smithi022.stdout:0/223: creat d10/d15/d23/f47 x:0 0 0 2024-02-16T19:28:54.978 INFO:tasks.workunit.client.1.smithi117.stdout:4/123: write d0/f11 [208749,111699] 0 2024-02-16T19:28:54.979 INFO:tasks.workunit.client.1.smithi117.stdout:4/124: readlink d0/l16 0 2024-02-16T19:28:54.979 INFO:tasks.workunit.client.1.smithi117.stdout:8/112: write d4/f6 [305564,91560] 0 2024-02-16T19:28:54.980 INFO:tasks.workunit.client.1.smithi117.stdout:8/113: fdatasync d4/d7/f21 0 2024-02-16T19:28:54.980 INFO:tasks.workunit.client.1.smithi117.stdout:0/142: mkdir d0/d30 0 2024-02-16T19:28:54.981 INFO:tasks.workunit.client.1.smithi117.stdout:0/143: dread - d0/d9/d14/d1d/d1e/f2d zero size 2024-02-16T19:28:54.983 INFO:tasks.workunit.client.1.smithi117.stdout:3/114: symlink d1/d1b/l2a 0 2024-02-16T19:28:54.984 INFO:tasks.workunit.client.1.smithi117.stdout:8/114: dread d4/d7/fc [0,4194304] 0 2024-02-16T19:28:54.985 INFO:tasks.workunit.client.1.smithi117.stdout:3/115: dread d1/fa [0,4194304] 0 2024-02-16T19:28:54.986 INFO:tasks.workunit.client.1.smithi117.stdout:2/102: dread d0/d3/fc [0,4194304] 0 2024-02-16T19:28:54.986 INFO:tasks.workunit.client.1.smithi117.stdout:9/82: unlink dd/f11 0 2024-02-16T19:28:54.986 INFO:tasks.workunit.client.1.smithi117.stdout:3/116: chown d1/fc 1505151 1 2024-02-16T19:28:54.986 INFO:tasks.workunit.client.1.smithi117.stdout:2/103: stat d0/d3/c4 0 2024-02-16T19:28:54.987 INFO:tasks.workunit.client.1.smithi117.stdout:3/117: chown d1/d1b/l2a 612 1 2024-02-16T19:28:54.987 INFO:tasks.workunit.client.0.smithi022.stdout:9/190: mknod db/d14/d19/c46 0 2024-02-16T19:28:54.988 INFO:tasks.workunit.client.1.smithi117.stdout:4/125: creat d0/f25 x:0 0 0 2024-02-16T19:28:54.988 INFO:tasks.workunit.client.0.smithi022.stdout:9/191: write db/d18/f26 [5153323,67379] 0 2024-02-16T19:28:54.990 INFO:tasks.workunit.client.0.smithi022.stdout:9/192: write db/d14/d19/d1a/f24 [3202938,99252] 0 2024-02-16T19:28:54.991 INFO:tasks.workunit.client.0.smithi022.stdout:4/127: creat d4/d6/dd/f2a x:0 0 0 2024-02-16T19:28:54.991 INFO:tasks.workunit.client.0.smithi022.stdout:4/128: stat d4/c11 0 2024-02-16T19:28:54.992 INFO:tasks.workunit.client.1.smithi117.stdout:9/83: dread dd/d12/f18 [0,4194304] 0 2024-02-16T19:28:54.994 INFO:tasks.workunit.client.1.smithi117.stdout:0/144: fdatasync d0/d9/d14/f15 0 2024-02-16T19:28:54.994 INFO:tasks.workunit.client.0.smithi022.stdout:0/224: mknod d10/d15/c48 0 2024-02-16T19:28:54.995 INFO:tasks.workunit.client.1.smithi117.stdout:2/104: dwrite d0/d3/d10/f17 [0,4194304] 0 2024-02-16T19:28:54.996 INFO:tasks.workunit.client.0.smithi022.stdout:7/147: truncate d9/da/f29 3769528 0 2024-02-16T19:28:54.998 INFO:tasks.workunit.client.0.smithi022.stdout:8/167: rename d3/f2c to d3/d12/d17/f3d 0 2024-02-16T19:28:54.999 INFO:tasks.workunit.client.0.smithi022.stdout:8/168: stat d3/d12/d17/c36 0 2024-02-16T19:28:55.001 INFO:tasks.workunit.client.1.smithi117.stdout:8/115: symlink d4/d12/l27 0 2024-02-16T19:28:55.001 INFO:tasks.workunit.client.1.smithi117.stdout:8/116: read - d4/d12/f26 zero size 2024-02-16T19:28:55.002 INFO:tasks.workunit.client.1.smithi117.stdout:8/117: write d4/d1c/f1f [2830226,93864] 0 2024-02-16T19:28:55.004 INFO:tasks.workunit.client.1.smithi117.stdout:3/118: mknod d1/d1b/d1e/c2b 0 2024-02-16T19:28:55.006 INFO:tasks.workunit.client.0.smithi022.stdout:3/141: dwrite d7/d21/f36 [4194304,4194304] 0 2024-02-16T19:28:55.006 INFO:tasks.workunit.client.0.smithi022.stdout:9/193: fsync db/f1d 0 2024-02-16T19:28:55.008 INFO:tasks.workunit.client.0.smithi022.stdout:9/194: write db/d14/d19/f2f [1335419,128930] 0 2024-02-16T19:28:55.008 INFO:tasks.workunit.client.1.smithi117.stdout:4/126: mknod d0/d6/d1a/c26 0 2024-02-16T19:28:55.010 INFO:tasks.workunit.client.0.smithi022.stdout:9/195: truncate db/d14/d16/f30 483807 0 2024-02-16T19:28:55.011 INFO:tasks.workunit.client.1.smithi117.stdout:8/118: dwrite d4/d12/f26 [0,4194304] 0 2024-02-16T19:28:55.012 INFO:tasks.workunit.client.0.smithi022.stdout:1/253: symlink d6/d8/d22/d16/d17/l48 0 2024-02-16T19:28:55.014 INFO:tasks.workunit.client.0.smithi022.stdout:0/225: mkdir d10/d22/d49 0 2024-02-16T19:28:55.017 INFO:tasks.workunit.client.0.smithi022.stdout:5/178: rename dd/d14/d21/d2f/c37 to dd/d14/d21/c38 0 2024-02-16T19:28:55.019 INFO:tasks.workunit.client.0.smithi022.stdout:8/169: fdatasync d3/d12/d2b/f3a 0 2024-02-16T19:28:55.019 INFO:tasks.workunit.client.0.smithi022.stdout:1/254: dwrite d6/d8/d22/d16/d17/f1a [0,4194304] 0 2024-02-16T19:28:55.020 INFO:tasks.workunit.client.0.smithi022.stdout:1/255: readlink d6/d8/d22/l2c 0 2024-02-16T19:28:55.022 INFO:tasks.workunit.client.0.smithi022.stdout:3/142: creat d7/f3b x:0 0 0 2024-02-16T19:28:55.025 INFO:tasks.workunit.client.0.smithi022.stdout:4/129: mknod d4/d6/dd/df/d10/d24/c2b 0 2024-02-16T19:28:55.029 INFO:tasks.workunit.client.0.smithi022.stdout:9/196: unlink db/d14/d19/f41 0 2024-02-16T19:28:55.030 INFO:tasks.workunit.client.0.smithi022.stdout:3/143: dwrite d7/f1e [0,4194304] 0 2024-02-16T19:28:55.033 INFO:tasks.workunit.client.0.smithi022.stdout:0/226: truncate d10/f19 49781 0 2024-02-16T19:28:55.033 INFO:tasks.workunit.client.1.smithi117.stdout:0/145: mkdir d0/d9/d22/d27/d31 0 2024-02-16T19:28:55.034 INFO:tasks.workunit.client.1.smithi117.stdout:0/146: readlink d0/lb 0 2024-02-16T19:28:55.037 INFO:tasks.workunit.client.1.smithi117.stdout:2/105: symlink d0/d3/d10/l21 0 2024-02-16T19:28:55.037 INFO:tasks.workunit.client.0.smithi022.stdout:5/179: fdatasync dd/d14/f2d 0 2024-02-16T19:28:55.039 INFO:tasks.workunit.client.1.smithi117.stdout:4/127: fsync d0/d6/f23 0 2024-02-16T19:28:55.039 INFO:tasks.workunit.client.1.smithi117.stdout:4/128: fdatasync d0/f14 0 2024-02-16T19:28:55.041 INFO:tasks.workunit.client.0.smithi022.stdout:1/256: creat d6/d8/d22/d3a/f49 x:0 0 0 2024-02-16T19:28:55.042 INFO:tasks.workunit.client.1.smithi117.stdout:0/147: creat d0/d9/d22/d26/f32 x:0 0 0 2024-02-16T19:28:55.044 INFO:tasks.workunit.client.0.smithi022.stdout:4/130: truncate d4/f23 8168594 0 2024-02-16T19:28:55.045 INFO:tasks.workunit.client.1.smithi117.stdout:2/106: mkdir d0/d3/d22 0 2024-02-16T19:28:55.045 INFO:tasks.workunit.client.1.smithi117.stdout:2/107: truncate d0/d3/fe 652066 0 2024-02-16T19:28:55.046 INFO:tasks.workunit.client.0.smithi022.stdout:8/170: dwrite d3/d12/d17/f3b [0,4194304] 0 2024-02-16T19:28:55.047 INFO:tasks.workunit.client.0.smithi022.stdout:8/171: dread - d3/d6/f32 zero size 2024-02-16T19:28:55.050 INFO:tasks.workunit.client.0.smithi022.stdout:9/197: symlink db/d14/d19/d1a/l47 0 2024-02-16T19:28:55.052 INFO:tasks.workunit.client.1.smithi117.stdout:4/129: creat d0/da/d21/f27 x:0 0 0 2024-02-16T19:28:55.053 INFO:tasks.workunit.client.1.smithi117.stdout:2/108: dwrite d0/d3/d10/f17 [0,4194304] 0 2024-02-16T19:28:55.054 INFO:tasks.workunit.client.1.smithi117.stdout:8/119: link d4/d7/l14 d4/d1c/l28 0 2024-02-16T19:28:55.057 INFO:tasks.workunit.client.1.smithi117.stdout:0/148: dread d0/f1f [0,4194304] 0 2024-02-16T19:28:55.058 INFO:tasks.workunit.client.0.smithi022.stdout:6/145: rename d1/l1a to d1/d8/da/d17/l34 0 2024-02-16T19:28:55.058 INFO:tasks.workunit.client.1.smithi117.stdout:4/130: dread d0/d6/f1c [0,4194304] 0 2024-02-16T19:28:55.060 INFO:tasks.workunit.client.0.smithi022.stdout:6/146: dread - d1/f32 zero size 2024-02-16T19:28:55.062 INFO:tasks.workunit.client.0.smithi022.stdout:5/180: write dd/d14/f1c [3363902,104490] 0 2024-02-16T19:28:55.064 INFO:tasks.workunit.client.0.smithi022.stdout:0/227: dwrite d10/d22/f2d [0,4194304] 0 2024-02-16T19:28:55.067 INFO:tasks.workunit.client.0.smithi022.stdout:5/181: truncate dd/f18 687695 0 2024-02-16T19:28:55.069 INFO:tasks.workunit.client.0.smithi022.stdout:1/257: write d6/d8/d22/d16/d17/d26/f43 [270067,102938] 0 2024-02-16T19:28:55.070 INFO:tasks.workunit.client.0.smithi022.stdout:4/131: creat d4/d6/dd/f2c x:0 0 0 2024-02-16T19:28:55.071 INFO:tasks.workunit.client.1.smithi117.stdout:0/149: mkdir d0/d9/d14/d23/d33 0 2024-02-16T19:28:55.073 INFO:tasks.workunit.client.0.smithi022.stdout:6/147: dwrite d1/d8/f1c [0,4194304] 0 2024-02-16T19:28:55.074 INFO:tasks.workunit.client.0.smithi022.stdout:6/148: fdatasync d1/d8/da/d17/f1d 0 2024-02-16T19:28:55.074 INFO:tasks.workunit.client.0.smithi022.stdout:9/198: unlink db/d14/d19/d1a/l1b 0 2024-02-16T19:28:55.076 INFO:tasks.workunit.client.1.smithi117.stdout:4/131: mkdir d0/d6/d1a/d28 0 2024-02-16T19:28:55.077 INFO:tasks.workunit.client.1.smithi117.stdout:9/84: truncate fb 830838 0 2024-02-16T19:28:55.078 INFO:tasks.workunit.client.1.smithi117.stdout:9/85: write dd/f19 [459625,7057] 0 2024-02-16T19:28:55.080 INFO:tasks.workunit.client.0.smithi022.stdout:7/148: rename d9/f16 to d9/da/d2c/f2d 0 2024-02-16T19:28:55.081 INFO:tasks.workunit.client.0.smithi022.stdout:9/199: dwrite db/fc [0,4194304] 0 2024-02-16T19:28:55.088 INFO:tasks.workunit.client.1.smithi117.stdout:0/150: unlink d0/d9/f13 0 2024-02-16T19:28:55.088 INFO:tasks.workunit.client.0.smithi022.stdout:7/149: dread d9/da/d2c/f2d [0,4194304] 0 2024-02-16T19:28:55.089 INFO:tasks.workunit.client.1.smithi117.stdout:0/151: chown d0/c3 26175965 1 2024-02-16T19:28:55.090 INFO:tasks.workunit.client.0.smithi022.stdout:0/228: symlink d10/d22/d2c/l4a 0 2024-02-16T19:28:55.092 INFO:tasks.workunit.client.1.smithi117.stdout:4/132: creat d0/d6/d1a/f29 x:0 0 0 2024-02-16T19:28:55.093 INFO:tasks.workunit.client.1.smithi117.stdout:9/86: creat dd/d12/f1a x:0 0 0 2024-02-16T19:28:55.094 INFO:tasks.workunit.client.0.smithi022.stdout:5/182: symlink dd/d14/d21/l39 0 2024-02-16T19:28:55.094 INFO:tasks.workunit.client.0.smithi022.stdout:5/183: fdatasync f5 0 2024-02-16T19:28:55.095 INFO:tasks.workunit.client.1.smithi117.stdout:8/120: link d4/l17 d4/d7/l29 0 2024-02-16T19:28:55.099 INFO:tasks.workunit.client.1.smithi117.stdout:0/152: mkdir d0/d9/d14/d34 0 2024-02-16T19:28:55.104 INFO:tasks.workunit.client.1.smithi117.stdout:8/121: mkdir d4/d12/d2a 0 2024-02-16T19:28:55.105 INFO:tasks.workunit.client.1.smithi117.stdout:0/153: creat d0/f35 x:0 0 0 2024-02-16T19:28:55.107 INFO:tasks.workunit.client.0.smithi022.stdout:8/172: rename d3/d6/l1a to d3/d12/d2b/d2e/l3e 0 2024-02-16T19:28:55.108 INFO:tasks.workunit.client.0.smithi022.stdout:8/173: write d3/d6/f32 [90719,75558] 0 2024-02-16T19:28:55.110 INFO:tasks.workunit.client.0.smithi022.stdout:3/144: write d7/d12/d18/dd/f28 [511717,123483] 0 2024-02-16T19:28:55.111 INFO:tasks.workunit.client.0.smithi022.stdout:3/145: read d7/d12/f14 [1312975,6624] 0 2024-02-16T19:28:55.114 INFO:tasks.workunit.client.0.smithi022.stdout:5/184: mkdir dd/d14/d3a 0 2024-02-16T19:28:55.115 INFO:tasks.workunit.client.0.smithi022.stdout:5/185: fsync dd/d1f/f31 0 2024-02-16T19:28:55.116 INFO:tasks.workunit.client.0.smithi022.stdout:5/186: write dd/d14/f2d [1158603,37693] 0 2024-02-16T19:28:55.117 INFO:tasks.workunit.client.0.smithi022.stdout:5/187: read dd/d14/f2d [8330182,66196] 0 2024-02-16T19:28:55.118 INFO:tasks.workunit.client.0.smithi022.stdout:6/149: rename d1/f2e to d1/d8/da/d29/f35 0 2024-02-16T19:28:55.118 INFO:tasks.workunit.client.0.smithi022.stdout:5/188: chown dd/d14/d21/d2f/c34 2977854 1 2024-02-16T19:28:55.119 INFO:tasks.workunit.client.0.smithi022.stdout:6/150: chown d1/d8/c16 0 1 2024-02-16T19:28:55.119 INFO:tasks.workunit.client.0.smithi022.stdout:9/200: creat db/d14/d33/f48 x:0 0 0 2024-02-16T19:28:55.120 INFO:tasks.workunit.client.0.smithi022.stdout:8/174: fdatasync d3/d25/d33/f37 0 2024-02-16T19:28:55.122 INFO:tasks.workunit.client.0.smithi022.stdout:3/146: mkdir d7/d12/d18/dd/d3c 0 2024-02-16T19:28:55.124 INFO:tasks.workunit.client.0.smithi022.stdout:4/132: link d4/c15 d4/d6/d14/c2d 0 2024-02-16T19:28:55.124 INFO:tasks.workunit.client.0.smithi022.stdout:3/147: read d7/d12/d18/f1b [85098,129295] 0 2024-02-16T19:28:55.126 INFO:tasks.workunit.client.0.smithi022.stdout:4/133: chown d4/d6/dd/df/d10/d24/c2b 7 1 2024-02-16T19:28:55.128 INFO:tasks.workunit.client.0.smithi022.stdout:5/189: fdatasync dd/d14/f26 0 2024-02-16T19:28:55.130 INFO:tasks.workunit.client.0.smithi022.stdout:6/151: creat d1/d24/f36 x:0 0 0 2024-02-16T19:28:55.132 INFO:tasks.workunit.client.0.smithi022.stdout:3/148: dwrite d7/d12/d18/dd/d24/f2f [0,4194304] 0 2024-02-16T19:28:55.132 INFO:tasks.workunit.client.0.smithi022.stdout:8/175: read d3/d6/f1f [311089,34725] 0 2024-02-16T19:28:55.133 INFO:tasks.workunit.client.0.smithi022.stdout:0/229: rmdir d10 39 2024-02-16T19:28:55.133 INFO:tasks.workunit.client.0.smithi022.stdout:8/176: write d3/d12/d17/f3b [40372,35777] 0 2024-02-16T19:28:55.135 INFO:tasks.workunit.client.0.smithi022.stdout:8/177: write d3/d6/f32 [1153118,112679] 0 2024-02-16T19:28:55.137 INFO:tasks.workunit.client.0.smithi022.stdout:4/134: creat d4/d6/d14/f2e x:0 0 0 2024-02-16T19:28:55.140 INFO:tasks.workunit.client.0.smithi022.stdout:3/149: dwrite f5 [0,4194304] 0 2024-02-16T19:28:55.140 INFO:tasks.workunit.client.0.smithi022.stdout:1/258: write d6/d8/f2d [1222555,110179] 0 2024-02-16T19:28:55.141 INFO:tasks.workunit.client.0.smithi022.stdout:4/135: write d4/d6/d14/f1d [1481077,111948] 0 2024-02-16T19:28:55.143 INFO:tasks.workunit.client.0.smithi022.stdout:6/152: mknod d1/d8/da/d29/c37 0 2024-02-16T19:28:55.145 INFO:tasks.workunit.client.0.smithi022.stdout:6/153: readlink d1/l26 0 2024-02-16T19:28:55.148 INFO:tasks.workunit.client.0.smithi022.stdout:8/178: dwrite d3/f10 [0,4194304] 0 2024-02-16T19:28:55.149 INFO:tasks.workunit.client.0.smithi022.stdout:8/179: chown d3/d12/c31 6542979 1 2024-02-16T19:28:55.150 INFO:tasks.workunit.client.1.smithi117.stdout:0/154: dread - d0/d9/d14/d1d/d1e/f2d zero size 2024-02-16T19:28:55.150 INFO:tasks.workunit.client.0.smithi022.stdout:5/190: creat dd/d14/d3a/f3b x:0 0 0 2024-02-16T19:28:55.151 INFO:tasks.workunit.client.1.smithi117.stdout:9/87: dwrite dd/d12/f18 [0,4194304] 0 2024-02-16T19:28:55.153 INFO:tasks.workunit.client.1.smithi117.stdout:8/122: dread d4/d12/f1a [0,4194304] 0 2024-02-16T19:28:55.155 INFO:tasks.workunit.client.0.smithi022.stdout:8/180: read d3/d6/f7 [597525,30142] 0 2024-02-16T19:28:55.157 INFO:tasks.workunit.client.0.smithi022.stdout:4/136: mknod d4/d6/dd/df/c2f 0 2024-02-16T19:28:55.159 INFO:tasks.workunit.client.1.smithi117.stdout:9/88: rename l0 to dd/l1b 0 2024-02-16T19:28:55.159 INFO:tasks.workunit.client.1.smithi117.stdout:9/89: stat dd/d12 0 2024-02-16T19:28:55.159 INFO:tasks.workunit.client.0.smithi022.stdout:4/137: dread - d4/d6/d14/f2e zero size 2024-02-16T19:28:55.163 INFO:tasks.workunit.client.0.smithi022.stdout:4/138: stat d4/d6/dd/df/c2f 0 2024-02-16T19:28:55.163 INFO:tasks.workunit.client.0.smithi022.stdout:5/191: dwrite dd/d14/f1b [0,4194304] 0 2024-02-16T19:28:55.164 INFO:tasks.workunit.client.0.smithi022.stdout:7/150: dwrite f4 [0,4194304] 0 2024-02-16T19:28:55.165 INFO:tasks.workunit.client.0.smithi022.stdout:3/150: rename d7/l8 to d7/d12/d18/dd/d24/d29/l3d 0 2024-02-16T19:28:55.166 INFO:tasks.workunit.client.1.smithi117.stdout:4/133: getdents d0/da 0 2024-02-16T19:28:55.168 INFO:tasks.workunit.client.0.smithi022.stdout:6/154: creat d1/d24/f38 x:0 0 0 2024-02-16T19:28:55.169 INFO:tasks.workunit.client.0.smithi022.stdout:6/155: dread - d1/d24/f36 zero size 2024-02-16T19:28:55.169 INFO:tasks.workunit.client.1.smithi117.stdout:8/123: stat d4/l17 0 2024-02-16T19:28:55.175 INFO:tasks.workunit.client.0.smithi022.stdout:8/181: creat d3/d12/d2b/f3f x:0 0 0 2024-02-16T19:28:55.176 INFO:tasks.workunit.client.0.smithi022.stdout:8/182: write d3/d6/f32 [177207,96926] 0 2024-02-16T19:28:55.177 INFO:tasks.workunit.client.0.smithi022.stdout:8/183: write d3/f11 [5627994,127902] 0 2024-02-16T19:28:55.178 INFO:tasks.workunit.client.1.smithi117.stdout:8/124: unlink d4/d7/l9 0 2024-02-16T19:28:55.180 INFO:tasks.workunit.client.1.smithi117.stdout:0/155: truncate d0/f1 1474774 0 2024-02-16T19:28:55.181 INFO:tasks.workunit.client.1.smithi117.stdout:4/134: symlink d0/d6/d1a/d28/l2a 0 2024-02-16T19:28:55.182 INFO:tasks.workunit.client.1.smithi117.stdout:0/156: read d0/f18 [1385483,88375] 0 2024-02-16T19:28:55.182 INFO:tasks.workunit.client.1.smithi117.stdout:4/135: fdatasync d0/f11 0 2024-02-16T19:28:55.185 INFO:tasks.workunit.client.0.smithi022.stdout:0/230: symlink d10/d15/d23/l4b 0 2024-02-16T19:28:55.186 INFO:tasks.workunit.client.1.smithi117.stdout:0/157: creat d0/d9/d22/d26/f36 x:0 0 0 2024-02-16T19:28:55.188 INFO:tasks.workunit.client.0.smithi022.stdout:1/259: truncate d6/d8/d22/f1b 475797 0 2024-02-16T19:28:55.189 INFO:tasks.workunit.client.0.smithi022.stdout:0/231: dread d10/d15/f39 [0,4194304] 0 2024-02-16T19:28:55.191 INFO:tasks.workunit.client.1.smithi117.stdout:9/90: dread fb [0,4194304] 0 2024-02-16T19:28:55.192 INFO:tasks.workunit.client.0.smithi022.stdout:8/184: symlink d3/d6/l40 0 2024-02-16T19:28:55.194 INFO:tasks.workunit.client.0.smithi022.stdout:0/232: dwrite d10/d22/f45 [0,4194304] 0 2024-02-16T19:28:55.196 INFO:tasks.workunit.client.1.smithi117.stdout:9/91: dwrite fa [0,4194304] 0 2024-02-16T19:28:55.196 INFO:tasks.workunit.client.1.smithi117.stdout:0/158: mkdir d0/d9/d2c/d37 0 2024-02-16T19:28:55.198 INFO:tasks.workunit.client.1.smithi117.stdout:0/159: dread d0/d9/d14/f1b [0,4194304] 0 2024-02-16T19:28:55.199 INFO:tasks.workunit.client.0.smithi022.stdout:0/233: dread d10/d22/f45 [0,4194304] 0 2024-02-16T19:28:55.200 INFO:tasks.workunit.client.0.smithi022.stdout:7/151: symlink d9/d1b/d23/l2e 0 2024-02-16T19:28:55.205 INFO:tasks.workunit.client.1.smithi117.stdout:8/125: getdents d4/d7 0 2024-02-16T19:28:55.213 INFO:tasks.workunit.client.0.smithi022.stdout:8/185: creat d3/d25/d33/f41 x:0 0 0 2024-02-16T19:28:55.214 INFO:tasks.workunit.client.0.smithi022.stdout:8/186: truncate d3/d6/f2f 1029098 0 2024-02-16T19:28:55.216 INFO:tasks.workunit.client.1.smithi117.stdout:9/92: link l9 dd/d12/l1c 0 2024-02-16T19:28:55.219 INFO:tasks.workunit.client.1.smithi117.stdout:9/93: dread dd/d12/f18 [0,4194304] 0 2024-02-16T19:28:55.220 INFO:tasks.workunit.client.0.smithi022.stdout:3/151: rename d7/d21/f22 to d7/d12/d18/dd/d24/f3e 0 2024-02-16T19:28:55.222 INFO:tasks.workunit.client.0.smithi022.stdout:5/192: dwrite dd/d14/d21/f25 [0,4194304] 0 2024-02-16T19:28:55.225 INFO:tasks.workunit.client.1.smithi117.stdout:0/160: dwrite d0/fd [0,4194304] 0 2024-02-16T19:28:55.231 INFO:tasks.workunit.client.1.smithi117.stdout:9/94: rename l9 to dd/l1d 0 2024-02-16T19:28:55.231 INFO:tasks.workunit.client.0.smithi022.stdout:4/139: getdents d4 0 2024-02-16T19:28:55.232 INFO:tasks.workunit.client.1.smithi117.stdout:9/95: write fa [4766158,71297] 0 2024-02-16T19:28:55.232 INFO:tasks.workunit.client.0.smithi022.stdout:4/140: chown d4/d6/dd 369586793 1 2024-02-16T19:28:55.233 INFO:tasks.workunit.client.0.smithi022.stdout:1/260: truncate d6/d8/d22/d16/d17/d1d/f33 4026940 0 2024-02-16T19:28:55.233 INFO:tasks.workunit.client.0.smithi022.stdout:4/141: write d4/d6/d14/f1d [1355053,112300] 0 2024-02-16T19:28:55.234 INFO:tasks.workunit.client.0.smithi022.stdout:0/234: mknod d10/d22/d49/c4c 0 2024-02-16T19:28:55.236 INFO:tasks.workunit.client.1.smithi117.stdout:0/161: rename d0/d9/d14/f15 to d0/d9/d22/d27/d31/f38 0 2024-02-16T19:28:55.240 INFO:tasks.workunit.client.0.smithi022.stdout:1/261: dwrite d6/d8/f12 [0,4194304] 0 2024-02-16T19:28:55.242 INFO:tasks.workunit.client.0.smithi022.stdout:7/152: rename d9/dd to d9/da/d2c/d2f 0 2024-02-16T19:28:55.246 INFO:tasks.workunit.client.0.smithi022.stdout:6/156: getdents d1 0 2024-02-16T19:28:55.251 INFO:tasks.workunit.client.0.smithi022.stdout:1/262: dwrite d6/d8/d22/d16/d17/d26/f43 [0,4194304] 0 2024-02-16T19:28:55.253 INFO:tasks.workunit.client.1.smithi117.stdout:0/162: creat d0/d30/f39 x:0 0 0 2024-02-16T19:28:55.254 INFO:tasks.workunit.client.0.smithi022.stdout:7/153: dwrite d9/da/d2c/d2f/f17 [0,4194304] 0 2024-02-16T19:28:55.257 INFO:tasks.workunit.client.1.smithi117.stdout:0/163: mknod d0/d9/d2c/c3a 0 2024-02-16T19:28:55.259 INFO:tasks.workunit.client.1.smithi117.stdout:0/164: dread - d0/d30/f39 zero size 2024-02-16T19:28:55.260 INFO:tasks.workunit.client.1.smithi117.stdout:9/96: dwrite fb [0,4194304] 0 2024-02-16T19:28:55.260 INFO:tasks.workunit.client.0.smithi022.stdout:0/235: creat d10/d22/d49/f4d x:0 0 0 2024-02-16T19:28:55.263 INFO:tasks.workunit.client.0.smithi022.stdout:7/154: dwrite f4 [0,4194304] 0 2024-02-16T19:28:55.264 INFO:tasks.workunit.client.0.smithi022.stdout:7/155: chown d9/da/d2c/f2d 36 1 2024-02-16T19:28:55.265 INFO:tasks.workunit.client.0.smithi022.stdout:8/187: rename c1 to d3/d12/d2b/d2e/c42 0 2024-02-16T19:28:55.271 INFO:tasks.workunit.client.1.smithi117.stdout:0/165: mknod d0/c3b 0 2024-02-16T19:28:55.272 INFO:tasks.workunit.client.0.smithi022.stdout:3/152: link d7/d12/d18/dd/d24/f2f d7/d12/d18/dd/d24/d29/f3f 0 2024-02-16T19:28:55.275 INFO:tasks.workunit.client.0.smithi022.stdout:5/193: link f6 dd/f3c 0 2024-02-16T19:28:55.276 INFO:tasks.workunit.client.0.smithi022.stdout:8/188: dwrite d3/d12/f22 [0,4194304] 0 2024-02-16T19:28:55.277 INFO:tasks.workunit.client.0.smithi022.stdout:1/263: unlink d6/d8/d22/d16/d17/l48 0 2024-02-16T19:28:55.278 INFO:tasks.workunit.client.0.smithi022.stdout:1/264: chown d6/d8/d22/d3f 192 1 2024-02-16T19:28:55.279 INFO:tasks.workunit.client.0.smithi022.stdout:1/265: dread - d6/d8/d22/d3a/f44 zero size 2024-02-16T19:28:55.280 INFO:tasks.workunit.client.1.smithi117.stdout:0/166: symlink d0/d9/d22/d26/l3c 0 2024-02-16T19:28:55.283 INFO:tasks.workunit.client.0.smithi022.stdout:7/156: mknod d9/da/c30 0 2024-02-16T19:28:55.285 INFO:tasks.workunit.client.0.smithi022.stdout:4/142: rename d4/d6/dd/df/d1b to d4/d6/dd/d22/d30 0 2024-02-16T19:28:55.286 INFO:tasks.workunit.client.0.smithi022.stdout:4/143: fsync d4/d6/dd/df/f29 0 2024-02-16T19:28:55.287 INFO:tasks.workunit.client.1.smithi117.stdout:0/167: dwrite d0/f1a [0,4194304] 0 2024-02-16T19:28:55.287 INFO:tasks.workunit.client.0.smithi022.stdout:4/144: readlink d4/d6/dd/l1a 0 2024-02-16T19:28:55.287 INFO:tasks.workunit.client.0.smithi022.stdout:4/145: stat d4 0 2024-02-16T19:28:55.288 INFO:tasks.workunit.client.1.smithi117.stdout:0/168: readlink d0/d9/d22/d26/l3c 0 2024-02-16T19:28:55.289 INFO:tasks.workunit.client.0.smithi022.stdout:3/153: symlink d7/d21/l40 0 2024-02-16T19:28:55.289 INFO:tasks.workunit.client.0.smithi022.stdout:6/157: write d1/d8/f1f [2949253,16851] 0 2024-02-16T19:28:55.290 INFO:tasks.workunit.client.0.smithi022.stdout:5/194: fdatasync f6 0 2024-02-16T19:28:55.294 INFO:tasks.workunit.client.0.smithi022.stdout:2/117: sync 2024-02-16T19:28:55.295 INFO:tasks.workunit.client.0.smithi022.stdout:0/236: mkdir d10/d15/d23/d4e 0 2024-02-16T19:28:55.296 INFO:tasks.workunit.client.1.smithi117.stdout:9/97: truncate f5 29830 0 2024-02-16T19:28:55.297 INFO:tasks.workunit.client.1.smithi117.stdout:6/119: sync 2024-02-16T19:28:55.297 INFO:tasks.workunit.client.1.smithi117.stdout:1/67: sync 2024-02-16T19:28:55.297 INFO:tasks.workunit.client.1.smithi117.stdout:7/90: sync 2024-02-16T19:28:55.297 INFO:tasks.workunit.client.1.smithi117.stdout:5/207: sync 2024-02-16T19:28:55.298 INFO:tasks.workunit.client.1.smithi117.stdout:0/169: creat d0/d9/d2c/d37/f3d x:0 0 0 2024-02-16T19:28:55.299 INFO:tasks.workunit.client.1.smithi117.stdout:5/208: chown d2/d3/c8 155846 1 2024-02-16T19:28:55.299 INFO:tasks.workunit.client.0.smithi022.stdout:7/157: creat d9/da/d1f/f31 x:0 0 0 2024-02-16T19:28:55.301 INFO:tasks.workunit.client.1.smithi117.stdout:5/209: dread d2/d4/f1e [0,4194304] 0 2024-02-16T19:28:55.301 INFO:tasks.workunit.client.0.smithi022.stdout:8/189: rename d3/d12/f22 to d3/d25/f43 0 2024-02-16T19:28:55.303 INFO:tasks.workunit.client.0.smithi022.stdout:8/190: write d3/d6/f32 [689108,6347] 0 2024-02-16T19:28:55.303 INFO:tasks.workunit.client.1.smithi117.stdout:7/91: chown d4/c25 46 1 2024-02-16T19:28:55.303 INFO:tasks.workunit.client.0.smithi022.stdout:4/146: rmdir d4/d6/dd/df/d10 39 2024-02-16T19:28:55.303 INFO:tasks.workunit.client.1.smithi117.stdout:7/92: write d4/f11 [914776,67691] 0 2024-02-16T19:28:55.305 INFO:tasks.workunit.client.1.smithi117.stdout:1/68: mkdir d2/d19 0 2024-02-16T19:28:55.306 INFO:tasks.workunit.client.1.smithi117.stdout:1/69: fsync d2/d4/f17 0 2024-02-16T19:28:55.306 INFO:tasks.workunit.client.1.smithi117.stdout:6/120: mkdir d5/d1d/d28 0 2024-02-16T19:28:55.307 INFO:tasks.workunit.client.0.smithi022.stdout:3/154: mkdir d7/d12/d18/dd/d31/d41 0 2024-02-16T19:28:55.307 INFO:tasks.workunit.client.1.smithi117.stdout:7/93: dread d4/fd [0,4194304] 0 2024-02-16T19:28:55.308 INFO:tasks.workunit.client.0.smithi022.stdout:5/195: mknod dd/d14/d21/d2f/c3d 0 2024-02-16T19:28:55.308 INFO:tasks.workunit.client.1.smithi117.stdout:7/94: truncate d4/f2b 587933 0 2024-02-16T19:28:55.308 INFO:tasks.workunit.client.1.smithi117.stdout:7/95: chown d4/d1c 30 1 2024-02-16T19:28:55.309 INFO:tasks.workunit.client.1.smithi117.stdout:0/170: creat d0/d9/d2c/d37/f3e x:0 0 0 2024-02-16T19:28:55.311 INFO:tasks.workunit.client.1.smithi117.stdout:7/96: truncate d4/d1c/d24/f29 402219 0 2024-02-16T19:28:55.313 INFO:tasks.workunit.client.1.smithi117.stdout:6/121: dwrite d5/fa [0,4194304] 0 2024-02-16T19:28:55.313 INFO:tasks.workunit.client.0.smithi022.stdout:7/158: mknod d9/d1b/d23/c32 0 2024-02-16T19:28:55.314 INFO:tasks.workunit.client.1.smithi117.stdout:7/97: dread d4/d1c/d24/f29 [0,4194304] 0 2024-02-16T19:28:55.314 INFO:tasks.workunit.client.1.smithi117.stdout:5/210: truncate d2/d9/de/d17/d1c/d2d/f34 473893 0 2024-02-16T19:28:55.314 INFO:tasks.workunit.client.0.smithi022.stdout:5/196: dread dd/f1d [0,4194304] 0 2024-02-16T19:28:55.319 INFO:tasks.workunit.client.1.smithi117.stdout:7/98: dread d4/f1a [0,4194304] 0 2024-02-16T19:28:55.319 INFO:tasks.workunit.client.1.smithi117.stdout:7/99: fsync d4/d13/f18 0 2024-02-16T19:28:55.320 INFO:tasks.workunit.client.0.smithi022.stdout:8/191: rename d3/lf to d3/d25/d33/l44 0 2024-02-16T19:28:55.320 INFO:tasks.workunit.client.1.smithi117.stdout:7/100: chown d4/d1c/d24/f29 132743601 1 2024-02-16T19:28:55.320 INFO:tasks.workunit.client.1.smithi117.stdout:0/171: fsync d0/d9/d14/f1b 0 2024-02-16T19:28:55.321 INFO:tasks.workunit.client.1.smithi117.stdout:0/172: fsync d0/f10 0 2024-02-16T19:28:55.322 INFO:tasks.workunit.client.0.smithi022.stdout:4/147: mkdir d4/d6/dd/d31 0 2024-02-16T19:28:55.324 INFO:tasks.workunit.client.0.smithi022.stdout:6/158: mkdir d1/d8/da/d39 0 2024-02-16T19:28:55.325 INFO:tasks.workunit.client.0.smithi022.stdout:6/159: fdatasync d1/d8/da/d17/f1d 0 2024-02-16T19:28:55.325 INFO:tasks.workunit.client.1.smithi117.stdout:6/122: creat d5/d18/f29 x:0 0 0 2024-02-16T19:28:55.325 INFO:tasks.workunit.client.1.smithi117.stdout:7/101: dread f3 [0,4194304] 0 2024-02-16T19:28:55.326 INFO:tasks.workunit.client.1.smithi117.stdout:6/123: fdatasync d5/fa 0 2024-02-16T19:28:55.327 INFO:tasks.workunit.client.1.smithi117.stdout:7/102: truncate d4/d1c/f26 277938 0 2024-02-16T19:28:55.329 INFO:tasks.workunit.client.1.smithi117.stdout:1/70: symlink d2/d18/l1a 0 2024-02-16T19:28:55.329 INFO:tasks.workunit.client.1.smithi117.stdout:1/71: readlink d2/d18/l1a 0 2024-02-16T19:28:55.331 INFO:tasks.workunit.client.1.smithi117.stdout:5/211: rename d2/d4/d16 to d2/d9/d3c 0 2024-02-16T19:28:55.332 INFO:tasks.workunit.client.1.smithi117.stdout:5/212: write d2/f13 [4703583,931] 0 2024-02-16T19:28:55.334 INFO:tasks.workunit.client.1.smithi117.stdout:9/98: getdents dd/d12 0 2024-02-16T19:28:55.335 INFO:tasks.workunit.client.0.smithi022.stdout:0/237: creat d10/d15/d23/d29/f4f x:0 0 0 2024-02-16T19:28:55.335 INFO:tasks.workunit.client.0.smithi022.stdout:9/201: sync 2024-02-16T19:28:55.337 INFO:tasks.workunit.client.0.smithi022.stdout:0/238: dread d10/f26 [0,4194304] 0 2024-02-16T19:28:55.338 INFO:tasks.workunit.client.0.smithi022.stdout:5/197: rmdir dd/d14/d21/d2f 39 2024-02-16T19:28:55.341 INFO:tasks.workunit.client.0.smithi022.stdout:3/155: rename d7/d12/d18/dd/d31/f33 to d7/d12/d18/f42 0 2024-02-16T19:28:55.342 INFO:tasks.workunit.client.0.smithi022.stdout:5/198: dread dd/f18 [0,4194304] 0 2024-02-16T19:28:55.343 INFO:tasks.workunit.client.0.smithi022.stdout:3/156: write d7/d21/f23 [914629,125627] 0 2024-02-16T19:28:55.345 INFO:tasks.workunit.client.0.smithi022.stdout:6/160: dread d1/d8/ff [0,4194304] 0 2024-02-16T19:28:55.345 INFO:tasks.workunit.client.1.smithi117.stdout:0/173: mknod d0/d9/d22/d26/c3f 0 2024-02-16T19:28:55.347 INFO:tasks.workunit.client.0.smithi022.stdout:3/157: dread d7/d12/d18/dd/d24/d29/f2c [0,4194304] 0 2024-02-16T19:28:55.347 INFO:tasks.workunit.client.1.smithi117.stdout:0/174: dread d0/d9/d14/d1d/d1e/f25 [0,4194304] 0 2024-02-16T19:28:55.347 INFO:tasks.workunit.client.1.smithi117.stdout:0/175: stat d0/f5 0 2024-02-16T19:28:55.347 INFO:tasks.workunit.client.0.smithi022.stdout:3/158: truncate d7/f1c 2575355 0 2024-02-16T19:28:55.352 INFO:tasks.workunit.client.1.smithi117.stdout:7/103: unlink d4/d1c/f1d 0 2024-02-16T19:28:55.352 INFO:tasks.workunit.client.1.smithi117.stdout:7/104: stat d4 0 2024-02-16T19:28:55.355 INFO:tasks.workunit.client.1.smithi117.stdout:7/105: dread d4/f1a [0,4194304] 0 2024-02-16T19:28:55.355 INFO:tasks.workunit.client.0.smithi022.stdout:2/118: link d0/c9 d0/d2/d18/c24 0 2024-02-16T19:28:55.357 INFO:tasks.workunit.client.0.smithi022.stdout:4/148: dread d4/f23 [4194304,4194304] 0 2024-02-16T19:28:55.358 INFO:tasks.workunit.client.1.smithi117.stdout:6/124: write d5/db/f19 [4027986,947] 0 2024-02-16T19:28:55.359 INFO:tasks.workunit.client.0.smithi022.stdout:9/202: readlink db/d14/l32 0 2024-02-16T19:28:55.361 INFO:tasks.workunit.client.1.smithi117.stdout:9/99: creat dd/f1e x:0 0 0 2024-02-16T19:28:55.364 INFO:tasks.workunit.client.1.smithi117.stdout:6/125: dwrite d5/db/f19 [0,4194304] 0 2024-02-16T19:28:55.365 INFO:tasks.workunit.client.0.smithi022.stdout:0/239: rename d10/d22/d31 to d10/d11/d50 0 2024-02-16T19:28:55.366 INFO:tasks.workunit.client.0.smithi022.stdout:4/149: rename d4 to d4/d6/dd/d22/d30/d32 22 2024-02-16T19:28:55.366 INFO:tasks.workunit.client.1.smithi117.stdout:9/100: dread dd/d12/f18 [0,4194304] 0 2024-02-16T19:28:55.367 INFO:tasks.workunit.client.0.smithi022.stdout:0/240: write d10/d15/d23/d29/f4f [166652,97656] 0 2024-02-16T19:28:55.367 INFO:tasks.workunit.client.1.smithi117.stdout:0/176: creat d0/f40 x:0 0 0 2024-02-16T19:28:55.368 INFO:tasks.workunit.client.0.smithi022.stdout:4/150: truncate d4/d6/dd/df/f29 73314 0 2024-02-16T19:28:55.368 INFO:tasks.workunit.client.1.smithi117.stdout:9/101: dread - dd/f1e zero size 2024-02-16T19:28:55.368 INFO:tasks.workunit.client.0.smithi022.stdout:4/151: dread - d4/d6/dd/f2c zero size 2024-02-16T19:28:55.369 INFO:tasks.workunit.client.1.smithi117.stdout:0/177: chown d0/d9/d22/d26/c2a 15026432 1 2024-02-16T19:28:55.369 INFO:tasks.workunit.client.1.smithi117.stdout:9/102: write fa [3429231,117306] 0 2024-02-16T19:28:55.370 INFO:tasks.workunit.client.1.smithi117.stdout:0/178: chown d0/d9/d2c 19 1 2024-02-16T19:28:55.371 INFO:tasks.workunit.client.1.smithi117.stdout:6/126: dwrite d5/fa [0,4194304] 0 2024-02-16T19:28:55.372 INFO:tasks.workunit.client.1.smithi117.stdout:6/127: truncate d5/d1d/f27 854439 0 2024-02-16T19:28:55.374 INFO:tasks.workunit.client.0.smithi022.stdout:6/161: mknod d1/d8/da/d29/c3a 0 2024-02-16T19:28:55.374 INFO:tasks.workunit.client.0.smithi022.stdout:3/159: mknod d7/c43 0 2024-02-16T19:28:55.375 INFO:tasks.workunit.client.1.smithi117.stdout:5/213: mkdir d2/d36/d3d 0 2024-02-16T19:28:55.377 INFO:tasks.workunit.client.0.smithi022.stdout:0/241: dwrite d10/d11/d50/f46 [0,4194304] 0 2024-02-16T19:28:55.377 INFO:tasks.workunit.client.1.smithi117.stdout:3/119: sync 2024-02-16T19:28:55.377 INFO:tasks.workunit.client.0.smithi022.stdout:1/266: link d6/d8/l35 d6/l4a 0 2024-02-16T19:28:55.378 INFO:tasks.workunit.client.0.smithi022.stdout:0/242: write fb [208266,63506] 0 2024-02-16T19:28:55.380 INFO:tasks.workunit.client.1.smithi117.stdout:9/103: dread dd/f19 [0,4194304] 0 2024-02-16T19:28:55.382 INFO:tasks.workunit.client.1.smithi117.stdout:5/214: dwrite d2/d9/de/f22 [0,4194304] 0 2024-02-16T19:28:55.383 INFO:tasks.workunit.client.0.smithi022.stdout:2/119: mkdir d0/d2/d18/d19/d25 0 2024-02-16T19:28:55.385 INFO:tasks.workunit.client.0.smithi022.stdout:1/267: dread d6/d8/d22/d16/d17/d1d/f21 [0,4194304] 0 2024-02-16T19:28:55.386 INFO:tasks.workunit.client.0.smithi022.stdout:1/268: chown d6 53 1 2024-02-16T19:28:55.386 INFO:tasks.workunit.client.0.smithi022.stdout:7/159: link d9/da/c30 d9/da/d2c/c33 0 2024-02-16T19:28:55.389 INFO:tasks.workunit.client.1.smithi117.stdout:6/128: mknod d5/c2a 0 2024-02-16T19:28:55.389 INFO:tasks.workunit.client.0.smithi022.stdout:1/269: write d6/d8/d22/f11 [1965800,46556] 0 2024-02-16T19:28:55.390 INFO:tasks.workunit.client.0.smithi022.stdout:0/243: dwrite d10/d15/d23/d29/f4f [0,4194304] 0 2024-02-16T19:28:55.391 INFO:tasks.workunit.client.1.smithi117.stdout:7/106: getdents d4/d13 0 2024-02-16T19:28:55.392 INFO:tasks.workunit.client.0.smithi022.stdout:0/244: dread - d10/d22/d49/f4d zero size 2024-02-16T19:28:55.401 INFO:tasks.workunit.client.0.smithi022.stdout:6/162: creat d1/d8/f3b x:0 0 0 2024-02-16T19:28:55.401 INFO:tasks.workunit.client.0.smithi022.stdout:6/163: dread - d1/d8/d23/f30 zero size 2024-02-16T19:28:55.402 INFO:tasks.workunit.client.1.smithi117.stdout:5/215: creat d2/d3/d21/d2f/d30/f3e x:0 0 0 2024-02-16T19:28:55.403 INFO:tasks.workunit.client.0.smithi022.stdout:6/164: truncate d1/d8/da/d17/f2c 29695 0 2024-02-16T19:28:55.403 INFO:tasks.workunit.client.0.smithi022.stdout:6/165: chown d1/l1b 61473 1 2024-02-16T19:28:55.404 INFO:tasks.workunit.client.0.smithi022.stdout:3/160: creat d7/d12/f44 x:0 0 0 2024-02-16T19:28:55.413 INFO:tasks.workunit.client.1.smithi117.stdout:3/120: creat d1/d1b/d1e/d29/f2c x:0 0 0 2024-02-16T19:28:55.414 INFO:tasks.workunit.client.1.smithi117.stdout:1/72: truncate d2/d4/f9 1431518 0 2024-02-16T19:28:55.414 INFO:tasks.workunit.client.1.smithi117.stdout:3/121: write d1/d1b/d1e/f23 [961986,33780] 0 2024-02-16T19:28:55.417 INFO:tasks.workunit.client.0.smithi022.stdout:9/203: mkdir db/d14/d16/d49 0 2024-02-16T19:28:55.418 INFO:tasks.workunit.client.1.smithi117.stdout:5/216: creat d2/d1d/f3f x:0 0 0 2024-02-16T19:28:55.419 INFO:tasks.workunit.client.0.smithi022.stdout:9/204: read f5 [2227970,30054] 0 2024-02-16T19:28:55.423 INFO:tasks.workunit.client.1.smithi117.stdout:7/107: write d4/d1c/f26 [215541,89039] 0 2024-02-16T19:28:55.425 INFO:tasks.workunit.client.0.smithi022.stdout:1/270: creat d6/d8/d22/d16/f4b x:0 0 0 2024-02-16T19:28:55.425 INFO:tasks.workunit.client.1.smithi117.stdout:3/122: chown d1/dd/c15 2 1 2024-02-16T19:28:55.426 INFO:tasks.workunit.client.0.smithi022.stdout:9/205: dwrite db/d14/d16/f30 [0,4194304] 0 2024-02-16T19:28:55.428 INFO:tasks.workunit.client.0.smithi022.stdout:4/152: mknod d4/d6/dd/df/c33 0 2024-02-16T19:28:55.430 INFO:tasks.workunit.client.1.smithi117.stdout:5/217: creat d2/d9/f40 x:0 0 0 2024-02-16T19:28:55.431 INFO:tasks.workunit.client.0.smithi022.stdout:1/271: dread d6/f1c [0,4194304] 0 2024-02-16T19:28:55.431 INFO:tasks.workunit.client.1.smithi117.stdout:9/104: write f5 [787233,9219] 0 2024-02-16T19:28:55.433 INFO:tasks.workunit.client.0.smithi022.stdout:5/199: creat dd/d14/d21/f3e x:0 0 0 2024-02-16T19:28:55.433 INFO:tasks.workunit.client.1.smithi117.stdout:6/129: link d5/f11 d5/d1d/d28/f2b 0 2024-02-16T19:28:55.435 INFO:tasks.workunit.client.0.smithi022.stdout:8/192: link d3/d12/d2b/d2e/c42 d3/d12/d2b/c45 0 2024-02-16T19:28:55.437 INFO:tasks.workunit.client.1.smithi117.stdout:9/105: dwrite f5 [0,4194304] 0 2024-02-16T19:28:55.437 INFO:tasks.workunit.client.0.smithi022.stdout:6/166: mkdir d1/d8/d3c 0 2024-02-16T19:28:55.437 INFO:tasks.workunit.client.1.smithi117.stdout:9/106: chown dd/f19 12813752 1 2024-02-16T19:28:55.438 INFO:tasks.workunit.client.1.smithi117.stdout:9/107: write dd/d12/f1a [781751,46651] 0 2024-02-16T19:28:55.438 INFO:tasks.workunit.client.0.smithi022.stdout:8/193: dread d3/d25/d33/f37 [0,4194304] 0 2024-02-16T19:28:55.439 INFO:tasks.workunit.client.1.smithi117.stdout:9/108: chown dd/d12 365 1 2024-02-16T19:28:55.440 INFO:tasks.workunit.client.0.smithi022.stdout:8/194: chown d3/f11 1884244 1 2024-02-16T19:28:55.442 INFO:tasks.workunit.client.1.smithi117.stdout:5/218: rmdir d2 39 2024-02-16T19:28:55.444 INFO:tasks.workunit.client.0.smithi022.stdout:7/160: dwrite d9/da/f13 [4194304,4194304] 0 2024-02-16T19:28:55.445 INFO:tasks.workunit.client.0.smithi022.stdout:7/161: fdatasync d9/da/d1f/f31 0 2024-02-16T19:28:55.447 INFO:tasks.workunit.client.1.smithi117.stdout:6/130: creat d5/d1d/f2c x:0 0 0 2024-02-16T19:28:55.447 INFO:tasks.workunit.client.1.smithi117.stdout:7/108: dwrite d4/f28 [0,4194304] 0 2024-02-16T19:28:55.448 INFO:tasks.workunit.client.1.smithi117.stdout:9/109: dwrite fa [0,4194304] 0 2024-02-16T19:28:55.450 INFO:tasks.workunit.client.1.smithi117.stdout:3/123: rename d1/dd/f24 to d1/d1b/f2d 0 2024-02-16T19:28:55.451 INFO:tasks.workunit.client.1.smithi117.stdout:3/124: readlink d1/l10 0 2024-02-16T19:28:55.455 INFO:tasks.workunit.client.0.smithi022.stdout:0/245: mknod d10/d15/d23/c51 0 2024-02-16T19:28:55.455 INFO:tasks.workunit.client.1.smithi117.stdout:5/219: write d2/d1d/f3f [540378,34394] 0 2024-02-16T19:28:55.458 INFO:tasks.workunit.client.1.smithi117.stdout:7/109: creat d4/d1c/f2c x:0 0 0 2024-02-16T19:28:55.461 INFO:tasks.workunit.client.1.smithi117.stdout:9/110: creat dd/d12/f1f x:0 0 0 2024-02-16T19:28:55.462 INFO:tasks.workunit.client.1.smithi117.stdout:9/111: write dd/d12/f1f [493150,7851] 0 2024-02-16T19:28:55.465 INFO:tasks.workunit.client.1.smithi117.stdout:3/125: mknod d1/d1b/d1e/d29/c2e 0 2024-02-16T19:28:55.468 INFO:tasks.workunit.client.1.smithi117.stdout:7/110: symlink d4/d1c/l2d 0 2024-02-16T19:28:55.468 INFO:tasks.workunit.client.1.smithi117.stdout:3/126: dread d1/f8 [0,4194304] 0 2024-02-16T19:28:55.469 INFO:tasks.workunit.client.0.smithi022.stdout:3/161: mknod d7/d12/d18/dd/d24/c45 0 2024-02-16T19:28:55.469 INFO:tasks.workunit.client.1.smithi117.stdout:9/112: creat dd/f20 x:0 0 0 2024-02-16T19:28:55.470 INFO:tasks.workunit.client.0.smithi022.stdout:8/195: mkdir d3/d12/d2b/d46 0 2024-02-16T19:28:55.473 INFO:tasks.workunit.client.1.smithi117.stdout:5/220: mkdir d2/d9/de/d17/d1c/d41 0 2024-02-16T19:28:55.473 INFO:tasks.workunit.client.1.smithi117.stdout:3/127: dwrite d1/f8 [0,4194304] 0 2024-02-16T19:28:55.476 INFO:tasks.workunit.client.1.smithi117.stdout:9/113: stat dd/d12/l1c 0 2024-02-16T19:28:55.480 INFO:tasks.workunit.client.1.smithi117.stdout:5/221: mkdir d2/d9/d3c/d42 0 2024-02-16T19:28:55.481 INFO:tasks.workunit.client.1.smithi117.stdout:3/128: write d1/f8 [3352794,22815] 0 2024-02-16T19:28:55.483 INFO:tasks.workunit.client.0.smithi022.stdout:0/246: rename d10/d22/f1c to d10/d11/d27/f52 0 2024-02-16T19:28:55.485 INFO:tasks.workunit.client.1.smithi117.stdout:9/114: symlink dd/l21 0 2024-02-16T19:28:55.485 INFO:tasks.workunit.client.0.smithi022.stdout:4/153: creat d4/d6/dd/d31/f34 x:0 0 0 2024-02-16T19:28:55.486 INFO:tasks.workunit.client.1.smithi117.stdout:5/222: truncate d2/d3/f6 1475254 0 2024-02-16T19:28:55.487 INFO:tasks.workunit.client.1.smithi117.stdout:5/223: dread - d2/d3/d21/f32 zero size 2024-02-16T19:28:55.488 INFO:tasks.workunit.client.1.smithi117.stdout:3/129: symlink d1/d1b/l2f 0 2024-02-16T19:28:55.488 INFO:tasks.workunit.client.1.smithi117.stdout:3/130: stat d1/l6 0 2024-02-16T19:28:55.489 INFO:tasks.workunit.client.0.smithi022.stdout:5/200: creat dd/d1f/d2e/d35/d36/f3f x:0 0 0 2024-02-16T19:28:55.490 INFO:tasks.workunit.client.1.smithi117.stdout:7/111: write d4/f5 [686966,79045] 0 2024-02-16T19:28:55.490 INFO:tasks.workunit.client.0.smithi022.stdout:6/167: symlink d1/d8/da/l3d 0 2024-02-16T19:28:55.491 INFO:tasks.workunit.client.0.smithi022.stdout:3/162: fdatasync d7/d12/f14 0 2024-02-16T19:28:55.493 INFO:tasks.workunit.client.0.smithi022.stdout:8/196: readlink d3/d25/d33/l44 0 2024-02-16T19:28:55.494 INFO:tasks.workunit.client.0.smithi022.stdout:8/197: read d3/d6/f7 [2430007,40650] 0 2024-02-16T19:28:55.495 INFO:tasks.workunit.client.1.smithi117.stdout:4/136: sync 2024-02-16T19:28:55.495 INFO:tasks.workunit.client.1.smithi117.stdout:2/109: sync 2024-02-16T19:28:55.495 INFO:tasks.workunit.client.1.smithi117.stdout:8/126: sync 2024-02-16T19:28:55.495 INFO:tasks.workunit.client.0.smithi022.stdout:2/120: getdents d0 0 2024-02-16T19:28:55.496 INFO:tasks.workunit.client.1.smithi117.stdout:2/110: readlink d0/d3/l20 0 2024-02-16T19:28:55.496 INFO:tasks.workunit.client.0.smithi022.stdout:2/121: dread - d0/d2/d18/d1f/f21 zero size 2024-02-16T19:28:55.497 INFO:tasks.workunit.client.1.smithi117.stdout:8/127: write d4/d12/f26 [3048935,29366] 0 2024-02-16T19:28:55.497 INFO:tasks.workunit.client.1.smithi117.stdout:2/111: readlink d0/d3/l20 0 2024-02-16T19:28:55.497 INFO:tasks.workunit.client.1.smithi117.stdout:2/112: fsync d0/d3/d10/f1d 0 2024-02-16T19:28:55.499 INFO:tasks.workunit.client.0.smithi022.stdout:7/162: unlink c6 0 2024-02-16T19:28:55.500 INFO:tasks.workunit.client.1.smithi117.stdout:2/113: dread d0/d3/fe [0,4194304] 0 2024-02-16T19:28:55.501 INFO:tasks.workunit.client.1.smithi117.stdout:2/114: read d0/f1e [33988,125099] 0 2024-02-16T19:28:55.501 INFO:tasks.workunit.client.1.smithi117.stdout:8/128: dread d4/d12/f1a [0,4194304] 0 2024-02-16T19:28:55.502 INFO:tasks.workunit.client.1.smithi117.stdout:2/115: chown d0/d3/fe 0 1 2024-02-16T19:28:55.502 INFO:tasks.workunit.client.1.smithi117.stdout:8/129: fdatasync d4/d1c/f1f 0 2024-02-16T19:28:55.502 INFO:tasks.workunit.client.1.smithi117.stdout:5/224: creat d2/d3/d21/d2f/d3b/f43 x:0 0 0 2024-02-16T19:28:55.502 INFO:tasks.workunit.client.0.smithi022.stdout:7/163: dread f5 [0,4194304] 0 2024-02-16T19:28:55.503 INFO:tasks.workunit.client.1.smithi117.stdout:2/116: write d0/d3/d10/f16 [1437916,47706] 0 2024-02-16T19:28:55.504 INFO:tasks.workunit.client.1.smithi117.stdout:2/117: truncate d0/d13/f1c 1007331 0 2024-02-16T19:28:55.504 INFO:tasks.workunit.client.0.smithi022.stdout:0/247: rename d10/f26 to d10/d22/f53 0 2024-02-16T19:28:55.504 INFO:tasks.workunit.client.0.smithi022.stdout:0/248: stat d10/d15/d23/f47 0 2024-02-16T19:28:55.505 INFO:tasks.workunit.client.1.smithi117.stdout:3/131: truncate d1/dd/f14 291580 0 2024-02-16T19:28:55.506 INFO:tasks.workunit.client.0.smithi022.stdout:1/272: creat d6/d8/d22/d16/d17/f4c x:0 0 0 2024-02-16T19:28:55.506 INFO:tasks.workunit.client.0.smithi022.stdout:1/273: chown d6/d8/d22/d3f 68595780 1 2024-02-16T19:28:55.510 INFO:tasks.workunit.client.1.smithi117.stdout:3/132: dwrite d1/fc [0,4194304] 0 2024-02-16T19:28:55.513 INFO:tasks.workunit.client.1.smithi117.stdout:4/137: creat d0/da/d21/f2b x:0 0 0 2024-02-16T19:28:55.514 INFO:tasks.workunit.client.1.smithi117.stdout:4/138: chown d0/d6/f23 4510239 1 2024-02-16T19:28:55.515 INFO:tasks.workunit.client.1.smithi117.stdout:4/139: dread d0/da/d21/f22 [0,4194304] 0 2024-02-16T19:28:55.516 INFO:tasks.workunit.client.0.smithi022.stdout:8/198: rmdir d3/d25 39 2024-02-16T19:28:55.522 INFO:tasks.workunit.client.0.smithi022.stdout:9/206: getdents db/d14/d19/d1a 0 2024-02-16T19:28:55.523 INFO:tasks.workunit.client.1.smithi117.stdout:4/140: dwrite d0/d6/f1f [0,4194304] 0 2024-02-16T19:28:55.523 INFO:tasks.workunit.client.0.smithi022.stdout:7/164: symlink d9/da/d2c/d2f/d21/l34 0 2024-02-16T19:28:55.525 INFO:tasks.workunit.client.1.smithi117.stdout:7/112: dwrite d4/f10 [4194304,4194304] 0 2024-02-16T19:28:55.525 INFO:tasks.workunit.client.1.smithi117.stdout:4/141: write d0/d6/f1f [2088805,38555] 0 2024-02-16T19:28:55.525 INFO:tasks.workunit.client.1.smithi117.stdout:7/113: dread - d4/d1c/f2c zero size 2024-02-16T19:28:55.527 INFO:tasks.workunit.client.0.smithi022.stdout:6/168: rename d1/c20 to d1/d8/c3e 0 2024-02-16T19:28:55.529 INFO:tasks.workunit.client.0.smithi022.stdout:0/249: write d10/d22/f53 [1742881,13154] 0 2024-02-16T19:28:55.530 INFO:tasks.workunit.client.0.smithi022.stdout:0/250: read d10/d22/f2d [1278476,89889] 0 2024-02-16T19:28:55.535 INFO:tasks.workunit.client.0.smithi022.stdout:8/199: mknod d3/d12/d2b/d2e/c47 0 2024-02-16T19:28:55.541 INFO:tasks.workunit.client.1.smithi117.stdout:1/73: sync 2024-02-16T19:28:55.541 INFO:tasks.workunit.client.1.smithi117.stdout:0/179: sync 2024-02-16T19:28:55.541 INFO:tasks.workunit.client.0.smithi022.stdout:9/207: mkdir db/d14/d16/d21/d4a 0 2024-02-16T19:28:55.541 INFO:tasks.workunit.client.0.smithi022.stdout:8/200: dwrite d3/f11 [0,4194304] 0 2024-02-16T19:28:55.542 INFO:tasks.workunit.client.1.smithi117.stdout:0/180: stat d0/d9/d22/d26/f32 0 2024-02-16T19:28:55.542 INFO:tasks.workunit.client.1.smithi117.stdout:0/181: dread - d0/f40 zero size 2024-02-16T19:28:55.544 INFO:tasks.workunit.client.0.smithi022.stdout:7/165: creat d9/da/d2c/d2f/f35 x:0 0 0 2024-02-16T19:28:55.545 INFO:tasks.workunit.client.0.smithi022.stdout:7/166: chown c7 49246 1 2024-02-16T19:28:55.545 INFO:tasks.workunit.client.0.smithi022.stdout:7/167: dread - d9/da/d1f/f31 zero size 2024-02-16T19:28:55.549 INFO:tasks.workunit.client.0.smithi022.stdout:2/122: truncate d0/f17 2144009 0 2024-02-16T19:28:55.550 INFO:tasks.workunit.client.0.smithi022.stdout:6/169: creat d1/d8/da/d29/f3f x:0 0 0 2024-02-16T19:28:55.556 INFO:tasks.workunit.client.1.smithi117.stdout:9/115: getdents dd 0 2024-02-16T19:28:55.557 INFO:tasks.workunit.client.1.smithi117.stdout:9/116: chown dd/d12 166 1 2024-02-16T19:28:55.557 INFO:tasks.workunit.client.1.smithi117.stdout:9/117: chown dd/d12/f1a 11636133 1 2024-02-16T19:28:55.558 INFO:tasks.workunit.client.1.smithi117.stdout:9/118: write fa [276282,49589] 0 2024-02-16T19:28:55.564 INFO:tasks.workunit.client.0.smithi022.stdout:3/163: rename d7/c17 to d7/d12/d18/c46 0 2024-02-16T19:28:55.566 INFO:tasks.workunit.client.0.smithi022.stdout:8/201: dwrite d3/d12/d17/f3d [0,4194304] 0 2024-02-16T19:28:55.566 INFO:tasks.workunit.client.1.smithi117.stdout:4/142: unlink d0/d6/ff 0 2024-02-16T19:28:55.567 INFO:tasks.workunit.client.1.smithi117.stdout:4/143: chown d0/f11 1421543 1 2024-02-16T19:28:55.567 INFO:tasks.workunit.client.1.smithi117.stdout:4/144: stat d0/d6/f23 0 2024-02-16T19:28:55.567 INFO:tasks.workunit.client.1.smithi117.stdout:4/145: truncate d0/da/d21/f27 589092 0 2024-02-16T19:28:55.571 INFO:tasks.workunit.client.0.smithi022.stdout:0/251: mknod d10/d15/d23/d4e/c54 0 2024-02-16T19:28:55.575 INFO:tasks.workunit.client.0.smithi022.stdout:8/202: dread d3/d6/f1f [0,4194304] 0 2024-02-16T19:28:55.577 INFO:tasks.workunit.client.0.smithi022.stdout:1/274: rename d6/d8/d22/d16/d17/l2f to d6/d8/d22/d16/d17/d26/l4d 0 2024-02-16T19:28:55.579 INFO:tasks.workunit.client.0.smithi022.stdout:0/252: dwrite f7 [0,4194304] 0 2024-02-16T19:28:55.590 INFO:tasks.workunit.client.0.smithi022.stdout:5/201: rename dd/d14/d3a to dd/d14/d21/d2f/d40 0 2024-02-16T19:28:55.590 INFO:tasks.workunit.client.0.smithi022.stdout:8/203: dread d3/d25/f43 [0,4194304] 0 2024-02-16T19:28:55.590 INFO:tasks.workunit.client.0.smithi022.stdout:5/202: stat dd/c10 0 2024-02-16T19:28:55.592 INFO:tasks.workunit.client.0.smithi022.stdout:0/253: creat d10/d15/f55 x:0 0 0 2024-02-16T19:28:55.594 INFO:tasks.workunit.client.0.smithi022.stdout:0/254: chown d10/d22/f2f 282546 1 2024-02-16T19:28:55.596 INFO:tasks.workunit.client.0.smithi022.stdout:0/255: write d10/d15/d23/f47 [549790,15400] 0 2024-02-16T19:28:55.598 INFO:tasks.workunit.client.0.smithi022.stdout:5/203: dwrite f6 [8388608,4194304] 0 2024-02-16T19:28:55.599 INFO:tasks.workunit.client.0.smithi022.stdout:5/204: chown dd/f11 468 1 2024-02-16T19:28:55.601 INFO:tasks.workunit.client.0.smithi022.stdout:5/205: write dd/d14/d21/f3e [660549,27356] 0 2024-02-16T19:28:55.601 INFO:tasks.workunit.client.0.smithi022.stdout:1/275: creat d6/d8/d22/d47/f4e x:0 0 0 2024-02-16T19:28:55.602 INFO:tasks.workunit.client.0.smithi022.stdout:5/206: chown dd/d1f/c2a 5 1 2024-02-16T19:28:55.604 INFO:tasks.workunit.client.0.smithi022.stdout:2/123: getdents d0/d2/d18/d1f 0 2024-02-16T19:28:55.605 INFO:tasks.workunit.client.0.smithi022.stdout:7/168: link d9/da/d2c/c33 d9/d1b/c36 0 2024-02-16T19:28:55.605 INFO:tasks.workunit.client.1.smithi117.stdout:5/225: mknod d2/c44 0 2024-02-16T19:28:55.605 INFO:tasks.workunit.client.1.smithi117.stdout:3/133: mkdir d1/d30 0 2024-02-16T19:28:55.606 INFO:tasks.workunit.client.1.smithi117.stdout:0/182: unlink d0/d30/f39 0 2024-02-16T19:28:55.607 INFO:tasks.workunit.client.1.smithi117.stdout:9/119: creat dd/d12/f22 x:0 0 0 2024-02-16T19:28:55.608 INFO:tasks.workunit.client.1.smithi117.stdout:5/226: write d2/f13 [4064818,90115] 0 2024-02-16T19:28:55.608 INFO:tasks.workunit.client.1.smithi117.stdout:3/134: dread - d1/f28 zero size 2024-02-16T19:28:55.608 INFO:tasks.workunit.client.0.smithi022.stdout:9/208: rename db/d14/d16/d28/c2b to db/d14/d33/c4b 0 2024-02-16T19:28:55.608 INFO:tasks.workunit.client.1.smithi117.stdout:0/183: write d0/f40 [289511,71983] 0 2024-02-16T19:28:55.609 INFO:tasks.workunit.client.1.smithi117.stdout:9/120: fsync fa 0 2024-02-16T19:28:55.610 INFO:tasks.workunit.client.1.smithi117.stdout:0/184: fsync d0/fe 0 2024-02-16T19:28:55.612 INFO:tasks.workunit.client.1.smithi117.stdout:4/146: creat d0/da/f2c x:0 0 0 2024-02-16T19:28:55.613 INFO:tasks.workunit.client.0.smithi022.stdout:3/164: truncate d7/d12/d18/dd/ff 2866836 0 2024-02-16T19:28:55.614 INFO:tasks.workunit.client.0.smithi022.stdout:5/207: mknod dd/d1f/c41 0 2024-02-16T19:28:55.614 INFO:tasks.workunit.client.1.smithi117.stdout:4/147: write d0/f11 [300937,91863] 0 2024-02-16T19:28:55.614 INFO:tasks.workunit.client.1.smithi117.stdout:7/114: dwrite d4/f2b [0,4194304] 0 2024-02-16T19:28:55.615 INFO:tasks.workunit.client.0.smithi022.stdout:5/208: read - dd/d14/f26 zero size 2024-02-16T19:28:55.615 INFO:tasks.workunit.client.1.smithi117.stdout:4/148: fdatasync d0/da/d21/f2b 0 2024-02-16T19:28:55.615 INFO:tasks.workunit.client.0.smithi022.stdout:5/209: chown dd/d14/f29 2720594 1 2024-02-16T19:28:55.616 INFO:tasks.workunit.client.1.smithi117.stdout:4/149: dread d0/da/d21/f27 [0,4194304] 0 2024-02-16T19:28:55.617 INFO:tasks.workunit.client.0.smithi022.stdout:2/124: mknod d0/c26 0 2024-02-16T19:28:55.618 INFO:tasks.workunit.client.1.smithi117.stdout:4/150: read d0/da/d21/f22 [44349,7336] 0 2024-02-16T19:28:55.620 INFO:tasks.workunit.client.1.smithi117.stdout:2/118: rmdir d0/d3/d10/d1f 0 2024-02-16T19:28:55.621 INFO:tasks.workunit.client.1.smithi117.stdout:2/119: write d0/d13/f1c [848251,108163] 0 2024-02-16T19:28:55.622 INFO:tasks.workunit.client.1.smithi117.stdout:2/120: chown d0/d3 55 1 2024-02-16T19:28:55.624 INFO:tasks.workunit.client.0.smithi022.stdout:1/276: dread d6/f19 [0,4194304] 0 2024-02-16T19:28:55.625 INFO:tasks.workunit.client.0.smithi022.stdout:6/170: rename d1/d8/d23/f28 to d1/d8/f40 0 2024-02-16T19:28:55.629 INFO:tasks.workunit.client.0.smithi022.stdout:7/169: write d9/f25 [588814,100973] 0 2024-02-16T19:28:55.631 INFO:tasks.workunit.client.0.smithi022.stdout:3/165: creat d7/d12/d18/dd/d31/f47 x:0 0 0 2024-02-16T19:28:55.632 INFO:tasks.workunit.client.0.smithi022.stdout:1/277: dwrite d6/d8/d22/d47/f4e [0,4194304] 0 2024-02-16T19:28:55.632 INFO:tasks.workunit.client.1.smithi117.stdout:5/227: symlink d2/d3/l45 0 2024-02-16T19:28:55.636 INFO:tasks.workunit.client.1.smithi117.stdout:9/121: rename dd/d12 to dd/d23 0 2024-02-16T19:28:55.637 INFO:tasks.workunit.client.1.smithi117.stdout:9/122: readlink dd/l21 0 2024-02-16T19:28:55.637 INFO:tasks.workunit.client.0.smithi022.stdout:7/170: dwrite d9/f25 [0,4194304] 0 2024-02-16T19:28:55.637 INFO:tasks.workunit.client.1.smithi117.stdout:9/123: fdatasync dd/d23/f22 0 2024-02-16T19:28:55.640 INFO:tasks.workunit.client.1.smithi117.stdout:0/185: mkdir d0/d41 0 2024-02-16T19:28:55.641 INFO:tasks.workunit.client.1.smithi117.stdout:9/124: dread fb [0,4194304] 0 2024-02-16T19:28:55.642 INFO:tasks.workunit.client.1.smithi117.stdout:9/125: chown dd/d23/l17 11067109 1 2024-02-16T19:28:55.645 INFO:tasks.workunit.client.0.smithi022.stdout:8/204: rename d3/f11 to d3/d12/f48 0 2024-02-16T19:28:55.647 INFO:tasks.workunit.client.1.smithi117.stdout:7/115: creat d4/d1c/f2e x:0 0 0 2024-02-16T19:28:55.647 INFO:tasks.workunit.client.0.smithi022.stdout:8/205: read d3/d6/f27 [144068,122960] 0 2024-02-16T19:28:55.647 INFO:tasks.workunit.client.0.smithi022.stdout:8/206: fsync d3/d6/f2f 0 2024-02-16T19:28:55.651 INFO:tasks.workunit.client.1.smithi117.stdout:8/130: getdents d4/d12 0 2024-02-16T19:28:55.653 INFO:tasks.workunit.client.1.smithi117.stdout:4/151: mkdir d0/d6/d1a/d2d 0 2024-02-16T19:28:55.653 INFO:tasks.workunit.client.1.smithi117.stdout:4/152: chown d0/f25 38048455 1 2024-02-16T19:28:55.656 INFO:tasks.workunit.client.0.smithi022.stdout:2/125: dread - d0/d2/d18/d1f/f21 zero size 2024-02-16T19:28:55.656 INFO:tasks.workunit.client.1.smithi117.stdout:2/121: fdatasync d0/f19 0 2024-02-16T19:28:55.657 INFO:tasks.workunit.client.1.smithi117.stdout:6/131: sync 2024-02-16T19:28:55.658 INFO:tasks.workunit.client.1.smithi117.stdout:3/135: mkdir d1/d31 0 2024-02-16T19:28:55.658 INFO:tasks.workunit.client.1.smithi117.stdout:6/132: readlink d5/db/l14 0 2024-02-16T19:28:55.659 INFO:tasks.workunit.client.1.smithi117.stdout:4/153: dwrite d0/f25 [0,4194304] 0 2024-02-16T19:28:55.659 INFO:tasks.workunit.client.1.smithi117.stdout:5/228: mknod d2/d36/c46 0 2024-02-16T19:28:55.660 INFO:tasks.workunit.client.1.smithi117.stdout:4/154: readlink d0/d6/l15 0 2024-02-16T19:28:55.660 INFO:tasks.workunit.client.1.smithi117.stdout:5/229: chown d2/l11 7 1 2024-02-16T19:28:55.661 INFO:tasks.workunit.client.0.smithi022.stdout:7/171: mknod d9/da/d1f/c37 0 2024-02-16T19:28:55.661 INFO:tasks.workunit.client.0.smithi022.stdout:2/126: dread d0/d2/f1c [0,4194304] 0 2024-02-16T19:28:55.667 INFO:tasks.workunit.client.0.smithi022.stdout:5/210: rename dd/d14/d21/d2f/c34 to dd/d14/c42 0 2024-02-16T19:28:55.668 INFO:tasks.workunit.client.0.smithi022.stdout:1/278: dwrite d6/f1c [0,4194304] 0 2024-02-16T19:28:55.669 INFO:tasks.workunit.client.1.smithi117.stdout:9/126: mkdir dd/d24 0 2024-02-16T19:28:55.672 INFO:tasks.workunit.client.1.smithi117.stdout:7/116: mkdir d4/d1c/d24/d2f 0 2024-02-16T19:28:55.673 INFO:tasks.workunit.client.1.smithi117.stdout:0/186: write d0/f1f [1059920,17057] 0 2024-02-16T19:28:55.674 INFO:tasks.workunit.client.0.smithi022.stdout:8/207: mknod d3/d12/c49 0 2024-02-16T19:28:55.697 INFO:tasks.workunit.client.0.smithi022.stdout:9/209: getdents db/d14/d16/d27 0 2024-02-16T19:28:55.698 INFO:tasks.workunit.client.0.smithi022.stdout:9/210: write db/d14/f2d [523851,83512] 0 2024-02-16T19:28:55.699 INFO:tasks.workunit.client.1.smithi117.stdout:3/136: mkdir d1/dd/d32 0 2024-02-16T19:28:55.700 INFO:tasks.workunit.client.0.smithi022.stdout:3/166: creat d7/d12/d18/dd/d31/d41/f48 x:0 0 0 2024-02-16T19:28:55.701 INFO:tasks.workunit.client.0.smithi022.stdout:3/167: chown d7/d12/f44 99 1 2024-02-16T19:28:55.703 INFO:tasks.workunit.client.1.smithi117.stdout:3/137: dread d1/f19 [0,4194304] 0 2024-02-16T19:28:55.716 INFO:tasks.workunit.client.1.smithi117.stdout:1/74: sync 2024-02-16T19:28:55.746 INFO:tasks.workunit.client.1.smithi117.stdout:6/133: mkdir d5/d18/d20/d2d 0 2024-02-16T19:28:55.747 INFO:tasks.workunit.client.1.smithi117.stdout:6/134: truncate d5/f24 561594 0 2024-02-16T19:28:55.747 INFO:tasks.workunit.client.1.smithi117.stdout:6/135: write d5/f12 [4704244,20969] 0 2024-02-16T19:28:55.750 INFO:tasks.workunit.client.1.smithi117.stdout:4/155: symlink d0/da/l2e 0 2024-02-16T19:28:55.751 INFO:tasks.workunit.client.0.smithi022.stdout:5/211: creat dd/d14/f43 x:0 0 0 2024-02-16T19:28:55.751 INFO:tasks.workunit.client.0.smithi022.stdout:5/212: chown f6 22 1 2024-02-16T19:28:55.752 INFO:tasks.workunit.client.0.smithi022.stdout:5/213: dread - dd/d1f/d2e/d35/d36/f3f zero size 2024-02-16T19:28:55.753 INFO:tasks.workunit.client.0.smithi022.stdout:1/279: mkdir d6/d8/d22/d16/d4f 0 2024-02-16T19:28:55.754 INFO:tasks.workunit.client.0.smithi022.stdout:1/280: write d6/d8/d22/d16/d17/d26/f43 [404739,112762] 0 2024-02-16T19:28:55.755 INFO:tasks.workunit.client.1.smithi117.stdout:5/230: write d2/d9/d3c/f1a [162296,61790] 0 2024-02-16T19:28:55.756 INFO:tasks.workunit.client.1.smithi117.stdout:9/127: creat dd/d23/f25 x:0 0 0 2024-02-16T19:28:55.757 INFO:tasks.workunit.client.1.smithi117.stdout:9/128: read - dd/f1e zero size 2024-02-16T19:28:55.758 INFO:tasks.workunit.client.1.smithi117.stdout:9/129: truncate dd/d23/f22 756218 0 2024-02-16T19:28:55.758 INFO:tasks.workunit.client.1.smithi117.stdout:9/130: chown fb 3931 1 2024-02-16T19:28:55.758 INFO:tasks.workunit.client.1.smithi117.stdout:7/117: mknod d4/d1c/d24/c30 0 2024-02-16T19:28:55.759 INFO:tasks.workunit.client.1.smithi117.stdout:7/118: write d4/f2b [2645090,87127] 0 2024-02-16T19:28:55.760 INFO:tasks.workunit.client.0.smithi022.stdout:8/208: rename d3/d6/f32 to d3/d12/d2b/d2e/f4a 0 2024-02-16T19:28:55.761 INFO:tasks.workunit.client.0.smithi022.stdout:1/281: dwrite d6/f9 [0,4194304] 0 2024-02-16T19:28:55.762 INFO:tasks.workunit.client.1.smithi117.stdout:8/131: creat d4/d12/d2a/f2b x:0 0 0 2024-02-16T19:28:55.766 INFO:tasks.workunit.client.0.smithi022.stdout:8/209: dread d3/d25/f43 [0,4194304] 0 2024-02-16T19:28:55.767 INFO:tasks.workunit.client.0.smithi022.stdout:1/282: dread d6/d8/d22/d16/f41 [0,4194304] 0 2024-02-16T19:28:55.767 INFO:tasks.workunit.client.0.smithi022.stdout:9/211: creat db/d14/d16/d21/f4c x:0 0 0 2024-02-16T19:28:55.768 INFO:tasks.workunit.client.0.smithi022.stdout:1/283: write d6/d8/f2d [2233947,95999] 0 2024-02-16T19:28:55.771 INFO:tasks.workunit.client.0.smithi022.stdout:2/127: mkdir d0/d2/d18/d19/d25/d27 0 2024-02-16T19:28:55.774 INFO:tasks.workunit.client.1.smithi117.stdout:1/75: creat d2/d18/f1b x:0 0 0 2024-02-16T19:28:55.775 INFO:tasks.workunit.client.0.smithi022.stdout:9/212: dwrite db/d14/d19/f2f [0,4194304] 0 2024-02-16T19:28:55.775 INFO:tasks.workunit.client.1.smithi117.stdout:1/76: truncate d2/d4/f11 206152 0 2024-02-16T19:28:55.776 INFO:tasks.workunit.client.1.smithi117.stdout:1/77: dread - d2/d4/f17 zero size 2024-02-16T19:28:55.776 INFO:tasks.workunit.client.1.smithi117.stdout:1/78: chown d2/d4 106846663 1 2024-02-16T19:28:55.779 INFO:tasks.workunit.client.0.smithi022.stdout:6/171: getdents d1/d8/da/d17 0 2024-02-16T19:28:55.781 INFO:tasks.workunit.client.1.smithi117.stdout:6/136: mknod d5/c2e 0 2024-02-16T19:28:55.782 INFO:tasks.workunit.client.1.smithi117.stdout:6/137: write d5/d18/f29 [301166,102848] 0 2024-02-16T19:28:55.782 INFO:tasks.workunit.client.1.smithi117.stdout:6/138: dread - d5/d18/f1e zero size 2024-02-16T19:28:55.785 INFO:tasks.workunit.client.1.smithi117.stdout:9/131: symlink dd/l26 0 2024-02-16T19:28:55.785 INFO:tasks.workunit.client.1.smithi117.stdout:9/132: chown dd 481 1 2024-02-16T19:28:55.786 INFO:tasks.workunit.client.1.smithi117.stdout:6/139: dread d5/db/f19 [0,4194304] 0 2024-02-16T19:28:55.786 INFO:tasks.workunit.client.0.smithi022.stdout:3/168: creat d7/d12/d18/dd/d3c/f49 x:0 0 0 2024-02-16T19:28:55.786 INFO:tasks.workunit.client.1.smithi117.stdout:9/133: dread dd/f19 [0,4194304] 0 2024-02-16T19:28:55.787 INFO:tasks.workunit.client.1.smithi117.stdout:9/134: chown dd/d23 10 1 2024-02-16T19:28:55.787 INFO:tasks.workunit.client.1.smithi117.stdout:9/135: chown dd/f19 18748978 1 2024-02-16T19:28:55.788 INFO:tasks.workunit.client.1.smithi117.stdout:9/136: truncate dd/d23/f25 483088 0 2024-02-16T19:28:55.789 INFO:tasks.workunit.client.1.smithi117.stdout:7/119: write d4/f11 [983081,2455] 0 2024-02-16T19:28:55.791 INFO:tasks.workunit.client.0.smithi022.stdout:1/284: rmdir d6/d8/d22/d16 39 2024-02-16T19:28:55.791 INFO:tasks.workunit.client.1.smithi117.stdout:9/137: dread fb [0,4194304] 0 2024-02-16T19:28:55.791 INFO:tasks.workunit.client.0.smithi022.stdout:1/285: chown d6/d8 369960 1 2024-02-16T19:28:55.792 INFO:tasks.workunit.client.0.smithi022.stdout:1/286: stat d6/f1c 0 2024-02-16T19:28:55.792 INFO:tasks.workunit.client.1.smithi117.stdout:8/132: creat d4/d12/d2a/f2c x:0 0 0 2024-02-16T19:28:55.792 INFO:tasks.workunit.client.1.smithi117.stdout:4/156: write d0/d6/f23 [385103,117900] 0 2024-02-16T19:28:55.793 INFO:tasks.workunit.client.1.smithi117.stdout:8/133: stat d4/d1c/l25 0 2024-02-16T19:28:55.795 INFO:tasks.workunit.client.0.smithi022.stdout:2/128: creat d0/d2/d18/f28 x:0 0 0 2024-02-16T19:28:55.798 INFO:tasks.workunit.client.0.smithi022.stdout:9/213: rename db/d14/d16/f30 to db/d14/d19/f4d 0 2024-02-16T19:28:55.799 INFO:tasks.workunit.client.0.smithi022.stdout:6/172: rmdir d1 39 2024-02-16T19:28:55.801 INFO:tasks.workunit.client.1.smithi117.stdout:3/138: link d1/d1b/d1e/d29/f2c d1/d1b/d1e/d29/f33 0 2024-02-16T19:28:55.802 INFO:tasks.workunit.client.0.smithi022.stdout:8/210: symlink d3/l4b 0 2024-02-16T19:28:55.804 INFO:tasks.workunit.client.0.smithi022.stdout:1/287: stat d6/c7 0 2024-02-16T19:28:55.806 INFO:tasks.workunit.client.0.smithi022.stdout:2/129: truncate d0/d2/f1c 4264896 0 2024-02-16T19:28:55.810 INFO:tasks.workunit.client.0.smithi022.stdout:1/288: dwrite d6/f9 [0,4194304] 0 2024-02-16T19:28:55.812 INFO:tasks.workunit.client.1.smithi117.stdout:7/120: symlink d4/d1c/d24/l31 0 2024-02-16T19:28:55.813 INFO:tasks.workunit.client.0.smithi022.stdout:3/169: dread d7/d12/d18/f1a [0,4194304] 0 2024-02-16T19:28:55.816 INFO:tasks.workunit.client.1.smithi117.stdout:9/138: symlink dd/d23/l27 0 2024-02-16T19:28:55.816 INFO:tasks.workunit.client.0.smithi022.stdout:3/170: fdatasync d7/d12/d18/dd/f20 0 2024-02-16T19:28:55.817 INFO:tasks.workunit.client.1.smithi117.stdout:9/139: dread dd/d23/f22 [0,4194304] 0 2024-02-16T19:28:55.818 INFO:tasks.workunit.client.0.smithi022.stdout:8/211: creat d3/d6/f4c x:0 0 0 2024-02-16T19:28:55.819 INFO:tasks.workunit.client.1.smithi117.stdout:8/134: chown d4/l16 154410306 1 2024-02-16T19:28:55.820 INFO:tasks.workunit.client.0.smithi022.stdout:1/289: dwrite d6/f1c [4194304,4194304] 0 2024-02-16T19:28:55.820 INFO:tasks.workunit.client.1.smithi117.stdout:4/157: mkdir d0/d6/d1a/d28/d2f 0 2024-02-16T19:28:55.820 INFO:tasks.workunit.client.0.smithi022.stdout:8/212: fdatasync d3/d12/d2b/f3f 0 2024-02-16T19:28:55.821 INFO:tasks.workunit.client.1.smithi117.stdout:0/187: truncate d0/fe 3681256 0 2024-02-16T19:28:55.822 INFO:tasks.workunit.client.1.smithi117.stdout:0/188: chown d0/f1a 28574703 1 2024-02-16T19:28:55.822 INFO:tasks.workunit.client.0.smithi022.stdout:6/173: dread d1/d8/da/d17/f1d [0,4194304] 0 2024-02-16T19:28:55.822 INFO:tasks.workunit.client.0.smithi022.stdout:5/214: getdents dd/d1f/d2e/d35/d36 0 2024-02-16T19:28:55.823 INFO:tasks.workunit.client.0.smithi022.stdout:5/215: write dd/d14/f1b [2480423,125410] 0 2024-02-16T19:28:55.824 INFO:tasks.workunit.client.1.smithi117.stdout:2/122: getdents d0/d3 0 2024-02-16T19:28:55.824 INFO:tasks.workunit.client.0.smithi022.stdout:6/174: dread - d1/d8/da/d17/f2d zero size 2024-02-16T19:28:55.826 INFO:tasks.workunit.client.1.smithi117.stdout:3/139: creat d1/dd/f34 x:0 0 0 2024-02-16T19:28:55.830 INFO:tasks.workunit.client.1.smithi117.stdout:6/140: mkdir d5/d18/d20/d2d/d2f 0 2024-02-16T19:28:55.832 INFO:tasks.workunit.client.1.smithi117.stdout:7/121: creat d4/d1c/f32 x:0 0 0 2024-02-16T19:28:55.835 INFO:tasks.workunit.client.0.smithi022.stdout:1/290: write d6/d8/d22/d16/f4b [699724,119700] 0 2024-02-16T19:28:55.835 INFO:tasks.workunit.client.1.smithi117.stdout:9/140: write fb [3497062,43186] 0 2024-02-16T19:28:55.836 INFO:tasks.workunit.client.0.smithi022.stdout:1/291: rename d6/d8/d22/d16 to d6/d8/d22/d16/d4f/d50 22 2024-02-16T19:28:55.839 INFO:tasks.workunit.client.1.smithi117.stdout:8/135: unlink d4/d12/l15 0 2024-02-16T19:28:55.841 INFO:tasks.workunit.client.1.smithi117.stdout:4/158: write d0/da/d21/f22 [665891,62885] 0 2024-02-16T19:28:55.841 INFO:tasks.workunit.client.1.smithi117.stdout:4/159: readlink d0/da/l2e 0 2024-02-16T19:28:55.842 INFO:tasks.workunit.client.0.smithi022.stdout:5/216: fdatasync dd/d14/f29 0 2024-02-16T19:28:55.843 INFO:tasks.workunit.client.1.smithi117.stdout:4/160: truncate d0/d6/f1f 4972496 0 2024-02-16T19:28:55.843 INFO:tasks.workunit.client.1.smithi117.stdout:4/161: fdatasync d0/d6/f1f 0 2024-02-16T19:28:55.845 INFO:tasks.workunit.client.1.smithi117.stdout:0/189: rename d0/d9/d2c/d37/f3e to d0/d9/d22/d27/f42 0 2024-02-16T19:28:55.845 INFO:tasks.workunit.client.1.smithi117.stdout:0/190: fdatasync d0/d9/d2c/d37/f3d 0 2024-02-16T19:28:55.846 INFO:tasks.workunit.client.0.smithi022.stdout:9/214: dwrite db/f1d [0,4194304] 0 2024-02-16T19:28:55.847 INFO:tasks.workunit.client.1.smithi117.stdout:2/123: write d0/d3/fc [1242926,124350] 0 2024-02-16T19:28:55.850 INFO:tasks.workunit.client.0.smithi022.stdout:3/171: mknod d7/d12/d18/dd/d24/c4a 0 2024-02-16T19:28:55.852 INFO:tasks.workunit.client.1.smithi117.stdout:5/231: getdents d2/d3 0 2024-02-16T19:28:55.852 INFO:tasks.workunit.client.1.smithi117.stdout:5/232: chown d2/d9/de 878 1 2024-02-16T19:28:55.854 INFO:tasks.workunit.client.0.smithi022.stdout:8/213: link d3/d12/d2b/f3a d3/d6/f4d 0 2024-02-16T19:28:55.854 INFO:tasks.workunit.client.1.smithi117.stdout:6/141: read d5/f12 [433352,36728] 0 2024-02-16T19:28:55.855 INFO:tasks.workunit.client.1.smithi117.stdout:6/142: truncate d5/db/f1f 4467263 0 2024-02-16T19:28:55.856 INFO:tasks.workunit.client.1.smithi117.stdout:6/143: fsync d5/db/f1f 0 2024-02-16T19:28:55.856 INFO:tasks.workunit.client.0.smithi022.stdout:1/292: mkdir d6/d8/d22/d47/d51 0 2024-02-16T19:28:55.857 INFO:tasks.workunit.client.0.smithi022.stdout:3/172: dwrite d7/d12/d18/dd/d24/f3a [0,4194304] 0 2024-02-16T19:28:55.858 INFO:tasks.workunit.client.0.smithi022.stdout:5/217: mknod dd/c44 0 2024-02-16T19:28:55.858 INFO:tasks.workunit.client.0.smithi022.stdout:8/214: truncate d3/d12/d2b/d2e/f4a 1756750 0 2024-02-16T19:28:55.859 INFO:tasks.workunit.client.0.smithi022.stdout:3/173: dread - d7/f3b zero size 2024-02-16T19:28:55.859 INFO:tasks.workunit.client.1.smithi117.stdout:9/141: mknod dd/d23/c28 0 2024-02-16T19:28:55.860 INFO:tasks.workunit.client.0.smithi022.stdout:6/175: getdents d1/d8/da/d39 0 2024-02-16T19:28:55.861 INFO:tasks.workunit.client.0.smithi022.stdout:9/215: rmdir db/d18 39 2024-02-16T19:28:55.862 INFO:tasks.workunit.client.0.smithi022.stdout:9/216: write db/d14/d19/f17 [3572050,125792] 0 2024-02-16T19:28:55.866 INFO:tasks.workunit.client.1.smithi117.stdout:2/124: mknod d0/d3/c23 0 2024-02-16T19:28:55.867 INFO:tasks.workunit.client.0.smithi022.stdout:1/293: dread d6/d8/d22/f11 [0,4194304] 0 2024-02-16T19:28:55.868 INFO:tasks.workunit.client.1.smithi117.stdout:2/125: dread d0/d3/fc [0,4194304] 0 2024-02-16T19:28:55.869 INFO:tasks.workunit.client.1.smithi117.stdout:2/126: readlink d0/d3/d10/l21 0 2024-02-16T19:28:55.869 INFO:tasks.workunit.client.0.smithi022.stdout:3/174: mkdir d7/d12/d18/d4b 0 2024-02-16T19:28:55.870 INFO:tasks.workunit.client.1.smithi117.stdout:2/127: write d0/d13/f15 [17097,54452] 0 2024-02-16T19:28:55.874 INFO:tasks.workunit.client.1.smithi117.stdout:5/233: creat d2/d3/f47 x:0 0 0 2024-02-16T19:28:55.874 INFO:tasks.workunit.client.1.smithi117.stdout:5/234: chown d2/d36/c46 626269781 1 2024-02-16T19:28:55.876 INFO:tasks.workunit.client.1.smithi117.stdout:7/122: rename d4/d13/l19 to d4/d1c/l33 0 2024-02-16T19:28:55.877 INFO:tasks.workunit.client.1.smithi117.stdout:7/123: truncate d4/f28 5172176 0 2024-02-16T19:28:55.877 INFO:tasks.workunit.client.1.smithi117.stdout:5/235: dread d2/d9/d3c/d24/f26 [0,4194304] 0 2024-02-16T19:28:55.882 INFO:tasks.workunit.client.1.smithi117.stdout:3/140: dwrite d1/d1b/f2d [0,4194304] 0 2024-02-16T19:28:55.885 INFO:tasks.workunit.client.1.smithi117.stdout:2/128: rename d0/d3/c23 to d0/d3/d22/c24 0 2024-02-16T19:28:55.886 INFO:tasks.workunit.client.1.smithi117.stdout:7/124: rmdir d4/d13 39 2024-02-16T19:28:55.888 INFO:tasks.workunit.client.1.smithi117.stdout:5/236: creat d2/d1d/f48 x:0 0 0 2024-02-16T19:28:55.890 INFO:tasks.workunit.client.1.smithi117.stdout:5/237: dread d2/d4/f1e [0,4194304] 0 2024-02-16T19:28:55.890 INFO:tasks.workunit.client.1.smithi117.stdout:6/144: link d5/f9 d5/d1d/d28/f30 0 2024-02-16T19:28:55.891 INFO:tasks.workunit.client.1.smithi117.stdout:6/145: dread - d5/d18/d20/f21 zero size 2024-02-16T19:28:55.892 INFO:tasks.workunit.client.1.smithi117.stdout:9/142: mknod dd/d24/c29 0 2024-02-16T19:28:55.895 INFO:tasks.workunit.client.1.smithi117.stdout:8/136: rename d4/c10 to d4/c2d 0 2024-02-16T19:28:55.899 INFO:tasks.workunit.client.1.smithi117.stdout:8/137: dread d4/d7/f21 [0,4194304] 0 2024-02-16T19:28:55.899 INFO:tasks.workunit.client.1.smithi117.stdout:3/141: mknod d1/dd/c35 0 2024-02-16T19:28:55.900 INFO:tasks.workunit.client.1.smithi117.stdout:3/142: write d1/dd/f34 [948295,10141] 0 2024-02-16T19:28:55.903 INFO:tasks.workunit.client.1.smithi117.stdout:2/129: rename d0/d3/fe to d0/f25 0 2024-02-16T19:28:55.905 INFO:tasks.workunit.client.1.smithi117.stdout:8/138: dwrite d4/d12/d2a/f2c [0,4194304] 0 2024-02-16T19:28:55.905 INFO:tasks.workunit.client.1.smithi117.stdout:6/146: symlink d5/l31 0 2024-02-16T19:28:55.906 INFO:tasks.workunit.client.1.smithi117.stdout:4/162: getdents d0/d6/d1a/d28 0 2024-02-16T19:28:55.906 INFO:tasks.workunit.client.1.smithi117.stdout:6/147: truncate d5/f1c 695990 0 2024-02-16T19:28:55.909 INFO:tasks.workunit.client.1.smithi117.stdout:2/130: creat d0/d13/f26 x:0 0 0 2024-02-16T19:28:55.910 INFO:tasks.workunit.client.1.smithi117.stdout:2/131: chown d0/d3/l20 15714 1 2024-02-16T19:28:55.910 INFO:tasks.workunit.client.1.smithi117.stdout:4/163: symlink d0/da/d21/l30 0 2024-02-16T19:28:55.912 INFO:tasks.workunit.client.1.smithi117.stdout:7/125: creat d4/f34 x:0 0 0 2024-02-16T19:28:55.912 INFO:tasks.workunit.client.1.smithi117.stdout:8/139: dwrite d4/f6 [4194304,4194304] 0 2024-02-16T19:28:55.914 INFO:tasks.workunit.client.0.smithi022.stdout:4/154: sync 2024-02-16T19:28:55.914 INFO:tasks.workunit.client.0.smithi022.stdout:8/215: rename d3/d12/d2b/d2e/l3e to d3/d12/d2b/l4e 0 2024-02-16T19:28:55.915 INFO:tasks.workunit.client.1.smithi117.stdout:8/140: chown d4/d12/f1a 112 1 2024-02-16T19:28:55.915 INFO:tasks.workunit.client.0.smithi022.stdout:6/176: write d1/f2 [2125957,37580] 0 2024-02-16T19:28:55.916 INFO:tasks.workunit.client.1.smithi117.stdout:6/148: rename d5/d18/d20/d2d/d2f to d5/d1d/d28/d32 0 2024-02-16T19:28:55.917 INFO:tasks.workunit.client.1.smithi117.stdout:2/132: symlink d0/d13/l27 0 2024-02-16T19:28:55.919 INFO:tasks.workunit.client.1.smithi117.stdout:5/238: getdents d2/d9/de/d17/d1c/d2d 0 2024-02-16T19:28:55.920 INFO:tasks.workunit.client.1.smithi117.stdout:4/164: dwrite d0/f14 [0,4194304] 0 2024-02-16T19:28:55.922 INFO:tasks.workunit.client.0.smithi022.stdout:8/216: mknod d3/d12/c4f 0 2024-02-16T19:28:55.923 INFO:tasks.workunit.client.0.smithi022.stdout:6/177: truncate d1/f3 2954684 0 2024-02-16T19:28:55.924 INFO:tasks.workunit.client.1.smithi117.stdout:8/141: mknod d4/d7/c2e 0 2024-02-16T19:28:55.926 INFO:tasks.workunit.client.1.smithi117.stdout:6/149: symlink d5/d18/d20/d2d/l33 0 2024-02-16T19:28:55.927 INFO:tasks.workunit.client.0.smithi022.stdout:1/294: creat d6/d8/d22/d16/d17/f52 x:0 0 0 2024-02-16T19:28:55.928 INFO:tasks.workunit.client.1.smithi117.stdout:4/165: dwrite d0/da/d21/f22 [0,4194304] 0 2024-02-16T19:28:55.929 INFO:tasks.workunit.client.0.smithi022.stdout:4/155: creat d4/d6/dd/df/d10/f35 x:0 0 0 2024-02-16T19:28:55.930 INFO:tasks.workunit.client.1.smithi117.stdout:3/143: dread d1/dd/f17 [0,4194304] 0 2024-02-16T19:28:55.931 INFO:tasks.workunit.client.1.smithi117.stdout:7/126: creat d4/d13/f35 x:0 0 0 2024-02-16T19:28:55.938 INFO:tasks.workunit.client.1.smithi117.stdout:5/239: rename d2/d9/de/d17/d1c to d2/d3/d21/d49 0 2024-02-16T19:28:55.938 INFO:tasks.workunit.client.1.smithi117.stdout:7/127: dwrite d4/f34 [0,4194304] 0 2024-02-16T19:28:55.940 INFO:tasks.workunit.client.1.smithi117.stdout:5/240: truncate d2/d9/d3c/f1a 1019162 0 2024-02-16T19:28:55.941 INFO:tasks.workunit.client.1.smithi117.stdout:8/142: mknod d4/d12/c2f 0 2024-02-16T19:28:55.944 INFO:tasks.workunit.client.1.smithi117.stdout:2/133: link d0/d3/d10/f17 d0/d3/d22/f28 0 2024-02-16T19:28:55.947 INFO:tasks.workunit.client.1.smithi117.stdout:8/143: dwrite d4/d12/d2a/f2c [0,4194304] 0 2024-02-16T19:28:55.949 INFO:tasks.workunit.client.1.smithi117.stdout:3/144: rename d1/d1b/d1e/d29/f2c to d1/d31/f36 0 2024-02-16T19:28:55.950 INFO:tasks.workunit.client.1.smithi117.stdout:7/128: readlink d4/l27 0 2024-02-16T19:28:55.951 INFO:tasks.workunit.client.1.smithi117.stdout:5/241: mknod d2/d9/d3c/d24/c4a 0 2024-02-16T19:28:55.957 INFO:tasks.workunit.client.1.smithi117.stdout:4/166: creat d0/d6/d1a/d28/d2f/f31 x:0 0 0 2024-02-16T19:28:55.959 INFO:tasks.workunit.client.1.smithi117.stdout:8/144: symlink d4/d1c/l30 0 2024-02-16T19:28:55.959 INFO:tasks.workunit.client.1.smithi117.stdout:8/145: write d4/d12/d2a/f2c [3691112,100371] 0 2024-02-16T19:28:55.962 INFO:tasks.workunit.client.1.smithi117.stdout:7/129: rename d4/d1c/l2d to d4/d13/l36 0 2024-02-16T19:28:55.963 INFO:tasks.workunit.client.1.smithi117.stdout:6/150: write d5/db/f16 [7604,31049] 0 2024-02-16T19:28:55.964 INFO:tasks.workunit.client.1.smithi117.stdout:4/167: symlink d0/d6/d1a/d28/l32 0 2024-02-16T19:28:55.964 INFO:tasks.workunit.client.1.smithi117.stdout:6/151: fdatasync d5/d1d/f27 0 2024-02-16T19:28:55.966 INFO:tasks.workunit.client.1.smithi117.stdout:3/145: write d1/d31/f36 [600509,78473] 0 2024-02-16T19:28:55.966 INFO:tasks.workunit.client.1.smithi117.stdout:8/146: mkdir d4/d12/d31 0 2024-02-16T19:28:55.967 INFO:tasks.workunit.client.1.smithi117.stdout:5/242: creat d2/d3/d21/d49/d41/f4b x:0 0 0 2024-02-16T19:28:55.968 INFO:tasks.workunit.client.1.smithi117.stdout:6/152: creat d5/d1d/d28/f34 x:0 0 0 2024-02-16T19:28:55.969 INFO:tasks.workunit.client.1.smithi117.stdout:5/243: chown d2/d3/d21/d49/d41/f4b 20 1 2024-02-16T19:28:55.970 INFO:tasks.workunit.client.1.smithi117.stdout:5/244: read - d2/d3/d21/f32 zero size 2024-02-16T19:28:55.971 INFO:tasks.workunit.client.1.smithi117.stdout:7/130: dread d4/f1a [0,4194304] 0 2024-02-16T19:28:55.971 INFO:tasks.workunit.client.1.smithi117.stdout:8/147: mkdir d4/d7/d32 0 2024-02-16T19:28:55.973 INFO:tasks.workunit.client.1.smithi117.stdout:4/168: link d0/da/d21/f22 d0/da/f33 0 2024-02-16T19:28:55.975 INFO:tasks.workunit.client.1.smithi117.stdout:7/131: creat d4/d1c/f37 x:0 0 0 2024-02-16T19:28:55.976 INFO:tasks.workunit.client.1.smithi117.stdout:7/132: write d4/d1c/f26 [812920,16554] 0 2024-02-16T19:28:55.977 INFO:tasks.workunit.client.1.smithi117.stdout:3/146: write d1/f9 [574874,70185] 0 2024-02-16T19:28:55.978 INFO:tasks.workunit.client.1.smithi117.stdout:3/147: dread - d1/d1b/f21 zero size 2024-02-16T19:28:55.978 INFO:tasks.workunit.client.1.smithi117.stdout:5/245: mkdir d2/d4c 0 2024-02-16T19:28:55.979 INFO:tasks.workunit.client.1.smithi117.stdout:5/246: read d2/d9/de/f22 [1254252,8837] 0 2024-02-16T19:28:55.980 INFO:tasks.workunit.client.1.smithi117.stdout:7/133: mkdir d4/d13/d38 0 2024-02-16T19:28:55.980 INFO:tasks.workunit.client.1.smithi117.stdout:6/153: link d5/d1d/d28/f30 d5/d1d/f35 0 2024-02-16T19:28:55.980 INFO:tasks.workunit.client.1.smithi117.stdout:5/247: readlink d2/d1d/l31 0 2024-02-16T19:28:55.981 INFO:tasks.workunit.client.1.smithi117.stdout:8/148: rmdir d4/d7/d32 0 2024-02-16T19:28:55.982 INFO:tasks.workunit.client.1.smithi117.stdout:7/134: mkdir d4/d1c/d39 0 2024-02-16T19:28:55.983 INFO:tasks.workunit.client.1.smithi117.stdout:7/135: stat d4/d1c/f26 0 2024-02-16T19:28:55.983 INFO:tasks.workunit.client.1.smithi117.stdout:6/154: unlink d5/c2e 0 2024-02-16T19:28:55.984 INFO:tasks.workunit.client.1.smithi117.stdout:6/155: dread - d5/f23 zero size 2024-02-16T19:28:55.986 INFO:tasks.workunit.client.1.smithi117.stdout:5/248: dwrite d2/d3/d21/d49/f2e [4194304,4194304] 0 2024-02-16T19:28:55.987 INFO:tasks.workunit.client.1.smithi117.stdout:8/149: symlink d4/l33 0 2024-02-16T19:28:55.994 INFO:tasks.workunit.client.1.smithi117.stdout:8/150: mknod d4/d12/c34 0 2024-02-16T19:28:55.995 INFO:tasks.workunit.client.1.smithi117.stdout:6/156: read d5/d1d/d28/f30 [1866697,98885] 0 2024-02-16T19:28:55.996 INFO:tasks.workunit.client.1.smithi117.stdout:3/148: getdents d1/d1b/d1e/d29 0 2024-02-16T19:28:55.997 INFO:tasks.workunit.client.1.smithi117.stdout:5/249: dwrite d2/d9/de/f22 [0,4194304] 0 2024-02-16T19:28:55.998 INFO:tasks.workunit.client.1.smithi117.stdout:7/136: creat d4/f3a x:0 0 0 2024-02-16T19:28:55.999 INFO:tasks.workunit.client.1.smithi117.stdout:8/151: mknod d4/d7/c35 0 2024-02-16T19:28:56.004 INFO:tasks.workunit.client.1.smithi117.stdout:5/250: creat d2/d3/f4d x:0 0 0 2024-02-16T19:28:56.004 INFO:tasks.workunit.client.1.smithi117.stdout:5/251: chown d2/d3/d21/d2f/d30 10343319 1 2024-02-16T19:28:56.006 INFO:tasks.workunit.client.1.smithi117.stdout:8/152: chown d4/c2d 13 1 2024-02-16T19:28:56.006 INFO:tasks.workunit.client.1.smithi117.stdout:8/153: fsync d4/f6 0 2024-02-16T19:28:56.009 INFO:tasks.workunit.client.1.smithi117.stdout:7/137: symlink d4/d1c/d39/l3b 0 2024-02-16T19:28:56.010 INFO:tasks.workunit.client.1.smithi117.stdout:7/138: write d4/d1c/f26 [1074166,71103] 0 2024-02-16T19:28:56.011 INFO:tasks.workunit.client.1.smithi117.stdout:5/252: write d2/d9/d3c/d24/f26 [1028188,77757] 0 2024-02-16T19:28:56.012 INFO:tasks.workunit.client.0.smithi022.stdout:6/178: creat d1/d8/da/d17/f41 x:0 0 0 2024-02-16T19:28:56.012 INFO:tasks.workunit.client.1.smithi117.stdout:3/149: truncate d1/dd/f17 782978 0 2024-02-16T19:28:56.013 INFO:tasks.workunit.client.0.smithi022.stdout:3/175: getdents d7 0 2024-02-16T19:28:56.013 INFO:tasks.workunit.client.0.smithi022.stdout:4/156: read d4/d6/f16 [3814664,77489] 0 2024-02-16T19:28:56.014 INFO:tasks.workunit.client.1.smithi117.stdout:8/154: dwrite d4/d7/f21 [0,4194304] 0 2024-02-16T19:28:56.014 INFO:tasks.workunit.client.0.smithi022.stdout:6/179: write d1/d24/f36 [483182,54563] 0 2024-02-16T19:28:56.014 INFO:tasks.workunit.client.0.smithi022.stdout:1/295: rmdir d6/d8/d22/d16 39 2024-02-16T19:28:56.015 INFO:tasks.workunit.client.0.smithi022.stdout:3/176: chown f5 7 1 2024-02-16T19:28:56.017 INFO:tasks.workunit.client.0.smithi022.stdout:4/157: creat d4/d6/dd/f36 x:0 0 0 2024-02-16T19:28:56.018 INFO:tasks.workunit.client.1.smithi117.stdout:6/157: dwrite d5/d1d/f35 [0,4194304] 0 2024-02-16T19:28:56.019 INFO:tasks.workunit.client.1.smithi117.stdout:3/150: mkdir d1/d1b/d1e/d37 0 2024-02-16T19:28:56.021 INFO:tasks.workunit.client.1.smithi117.stdout:5/253: rename d2/d3/d21/d49/d2d to d2/d3/d21/d2f/d30/d4e 0 2024-02-16T19:28:56.022 INFO:tasks.workunit.client.1.smithi117.stdout:3/151: chown d1/l27 3670398 1 2024-02-16T19:28:56.023 INFO:tasks.workunit.client.0.smithi022.stdout:1/296: dread d6/f9 [0,4194304] 0 2024-02-16T19:28:56.024 INFO:tasks.workunit.client.1.smithi117.stdout:3/152: fsync d1/d1b/d1e/d29/f33 0 2024-02-16T19:28:56.024 INFO:tasks.workunit.client.1.smithi117.stdout:5/254: dread - d2/d3/d21/d2f/d30/f3e zero size 2024-02-16T19:28:56.024 INFO:tasks.workunit.client.1.smithi117.stdout:7/139: dwrite d4/d1c/f2e [0,4194304] 0 2024-02-16T19:28:56.025 INFO:tasks.workunit.client.1.smithi117.stdout:8/155: dread d4/d7/fc [0,4194304] 0 2024-02-16T19:28:56.025 INFO:tasks.workunit.client.1.smithi117.stdout:3/153: fsync d1/d1b/f21 0 2024-02-16T19:28:56.026 INFO:tasks.workunit.client.0.smithi022.stdout:6/180: dwrite d1/d8/f11 [0,4194304] 0 2024-02-16T19:28:56.026 INFO:tasks.workunit.client.0.smithi022.stdout:1/297: chown d6/d8/d22/c36 996 1 2024-02-16T19:28:56.026 INFO:tasks.workunit.client.1.smithi117.stdout:7/140: fsync d4/d1c/f32 0 2024-02-16T19:28:56.026 INFO:tasks.workunit.client.1.smithi117.stdout:6/158: symlink d5/d18/d20/d2d/l36 0 2024-02-16T19:28:56.027 INFO:tasks.workunit.client.1.smithi117.stdout:1/79: sync 2024-02-16T19:28:56.027 INFO:tasks.workunit.client.1.smithi117.stdout:3/154: read d1/fc [3117398,96203] 0 2024-02-16T19:28:56.028 INFO:tasks.workunit.client.1.smithi117.stdout:0/191: sync 2024-02-16T19:28:56.029 INFO:tasks.workunit.client.0.smithi022.stdout:8/217: getdents d3/d25 0 2024-02-16T19:28:56.029 INFO:tasks.workunit.client.1.smithi117.stdout:0/192: write d0/f35 [399411,126202] 0 2024-02-16T19:28:56.031 INFO:tasks.workunit.client.0.smithi022.stdout:8/218: fsync d3/f2d 0 2024-02-16T19:28:56.032 INFO:tasks.workunit.client.1.smithi117.stdout:3/155: dread d1/dd/f14 [0,4194304] 0 2024-02-16T19:28:56.033 INFO:tasks.workunit.client.0.smithi022.stdout:4/158: dwrite d4/d6/dd/f36 [0,4194304] 0 2024-02-16T19:28:56.035 INFO:tasks.workunit.client.0.smithi022.stdout:4/159: fsync d4/d6/d14/f2e 0 2024-02-16T19:28:56.037 INFO:tasks.workunit.client.0.smithi022.stdout:6/181: creat d1/f42 x:0 0 0 2024-02-16T19:28:56.038 INFO:tasks.workunit.client.0.smithi022.stdout:3/177: dwrite d7/d12/d18/dd/d24/f3e [0,4194304] 0 2024-02-16T19:28:56.041 INFO:tasks.workunit.client.0.smithi022.stdout:6/182: read - d1/d24/f38 zero size 2024-02-16T19:28:56.047 INFO:tasks.workunit.client.0.smithi022.stdout:4/160: dwrite d4/f1f [0,4194304] 0 2024-02-16T19:28:56.049 INFO:tasks.workunit.client.0.smithi022.stdout:4/161: stat d4/d6/dd/df/d10 0 2024-02-16T19:28:56.050 INFO:tasks.workunit.client.0.smithi022.stdout:4/162: fsync d4/d6/dd/df/d10/f25 0 2024-02-16T19:28:56.053 INFO:tasks.workunit.client.1.smithi117.stdout:6/159: mkdir d5/d1d/d28/d32/d37 0 2024-02-16T19:28:56.054 INFO:tasks.workunit.client.0.smithi022.stdout:4/163: fdatasync d4/d6/dd/df/f29 0 2024-02-16T19:28:56.054 INFO:tasks.workunit.client.0.smithi022.stdout:7/172: sync 2024-02-16T19:28:56.054 INFO:tasks.workunit.client.0.smithi022.stdout:0/256: sync 2024-02-16T19:28:56.057 INFO:tasks.workunit.client.0.smithi022.stdout:0/257: write d10/d11/d50/f46 [1359153,94277] 0 2024-02-16T19:28:56.061 INFO:tasks.workunit.client.0.smithi022.stdout:3/178: creat d7/d12/d18/dd/d31/d41/f4c x:0 0 0 2024-02-16T19:28:56.062 INFO:tasks.workunit.client.1.smithi117.stdout:5/255: mkdir d2/d4c/d4f 0 2024-02-16T19:28:56.063 INFO:tasks.workunit.client.0.smithi022.stdout:1/298: symlink d6/d8/d22/d47/d51/l53 0 2024-02-16T19:28:56.063 INFO:tasks.workunit.client.0.smithi022.stdout:3/179: write d7/f35 [27145,93048] 0 2024-02-16T19:28:56.063 INFO:tasks.workunit.client.1.smithi117.stdout:5/256: fdatasync d2/d9/d3c/d24/f26 0 2024-02-16T19:28:56.069 INFO:tasks.workunit.client.0.smithi022.stdout:7/173: write d9/da/d2c/f2d [1149829,7326] 0 2024-02-16T19:28:56.072 INFO:tasks.workunit.client.1.smithi117.stdout:8/156: creat d4/d12/d31/f36 x:0 0 0 2024-02-16T19:28:56.072 INFO:tasks.workunit.client.1.smithi117.stdout:5/257: dwrite d2/d3/f47 [0,4194304] 0 2024-02-16T19:28:56.072 INFO:tasks.workunit.client.0.smithi022.stdout:0/258: chown d10/d11/d27/c30 5751 1 2024-02-16T19:28:56.073 INFO:tasks.workunit.client.0.smithi022.stdout:0/259: chown d10/d15/c48 236 1 2024-02-16T19:28:56.073 INFO:tasks.workunit.client.1.smithi117.stdout:7/141: dwrite f2 [0,4194304] 0 2024-02-16T19:28:56.074 INFO:tasks.workunit.client.0.smithi022.stdout:7/174: dread d9/da/d2c/d2f/f17 [0,4194304] 0 2024-02-16T19:28:56.074 INFO:tasks.workunit.client.1.smithi117.stdout:5/258: read d2/d3/f6 [178059,10159] 0 2024-02-16T19:28:56.074 INFO:tasks.workunit.client.1.smithi117.stdout:6/160: creat d5/d18/d20/f38 x:0 0 0 2024-02-16T19:28:56.075 INFO:tasks.workunit.client.0.smithi022.stdout:6/183: creat d1/d8/d3c/f43 x:0 0 0 2024-02-16T19:28:56.076 INFO:tasks.workunit.client.0.smithi022.stdout:6/184: chown d1/f18 504046 1 2024-02-16T19:28:56.078 INFO:tasks.workunit.client.0.smithi022.stdout:2/130: sync 2024-02-16T19:28:56.078 INFO:tasks.workunit.client.0.smithi022.stdout:5/218: sync 2024-02-16T19:28:56.079 INFO:tasks.workunit.client.0.smithi022.stdout:1/299: mkdir d6/d8/d22/d47/d54 0 2024-02-16T19:28:56.079 INFO:tasks.workunit.client.0.smithi022.stdout:6/185: dread d1/d24/f36 [0,4194304] 0 2024-02-16T19:28:56.079 INFO:tasks.workunit.client.0.smithi022.stdout:5/219: chown dd/f3c 970464837 1 2024-02-16T19:28:56.081 INFO:tasks.workunit.client.0.smithi022.stdout:1/300: write d6/d8/f2d [2667341,21448] 0 2024-02-16T19:28:56.081 INFO:tasks.workunit.client.1.smithi117.stdout:9/143: sync 2024-02-16T19:28:56.083 INFO:tasks.workunit.client.1.smithi117.stdout:0/193: mknod d0/d9/d14/d23/d2f/c43 0 2024-02-16T19:28:56.083 INFO:tasks.workunit.client.1.smithi117.stdout:9/144: write dd/f1e [1046181,88865] 0 2024-02-16T19:28:56.087 INFO:tasks.workunit.client.0.smithi022.stdout:5/220: dwrite dd/d14/f1c [0,4194304] 0 2024-02-16T19:28:56.089 INFO:tasks.workunit.client.1.smithi117.stdout:6/161: dread d5/db/f16 [0,4194304] 0 2024-02-16T19:28:56.090 INFO:tasks.workunit.client.1.smithi117.stdout:3/156: creat d1/d1b/d1e/d37/f38 x:0 0 0 2024-02-16T19:28:56.091 INFO:tasks.workunit.client.1.smithi117.stdout:9/145: truncate dd/f20 925527 0 2024-02-16T19:28:56.092 INFO:tasks.workunit.client.1.smithi117.stdout:5/259: dwrite d2/d9/d3c/d24/f26 [0,4194304] 0 2024-02-16T19:28:56.093 INFO:tasks.workunit.client.1.smithi117.stdout:6/162: truncate d5/f1a 206462 0 2024-02-16T19:28:56.095 INFO:tasks.workunit.client.1.smithi117.stdout:9/146: write dd/f1e [2172354,33875] 0 2024-02-16T19:28:56.096 INFO:tasks.workunit.client.1.smithi117.stdout:8/157: mkdir d4/d7/d37 0 2024-02-16T19:28:56.098 INFO:tasks.workunit.client.1.smithi117.stdout:7/142: rmdir d4 39 2024-02-16T19:28:56.099 INFO:tasks.workunit.client.1.smithi117.stdout:8/158: write d4/d12/f26 [3114198,40654] 0 2024-02-16T19:28:56.099 INFO:tasks.workunit.client.1.smithi117.stdout:3/157: dread d1/dd/f14 [0,4194304] 0 2024-02-16T19:28:56.100 INFO:tasks.workunit.client.0.smithi022.stdout:3/180: creat d7/d12/d18/d4b/f4d x:0 0 0 2024-02-16T19:28:56.100 INFO:tasks.workunit.client.1.smithi117.stdout:3/158: fdatasync d1/f9 0 2024-02-16T19:28:56.103 INFO:tasks.workunit.client.1.smithi117.stdout:1/80: rename d2/d4/c12 to d2/d4/c1c 0 2024-02-16T19:28:56.104 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:55 smithi117 ceph-mon[35309]: pgmap v259: 65 pgs: 65 active+clean; 368 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 588 KiB/s rd, 24 MiB/s wr, 181 op/s 2024-02-16T19:28:56.105 INFO:tasks.workunit.client.1.smithi117.stdout:0/194: truncate d0/f18 5153325 0 2024-02-16T19:28:56.106 INFO:tasks.workunit.client.0.smithi022.stdout:6/186: creat d1/d8/d3c/f44 x:0 0 0 2024-02-16T19:28:56.109 INFO:tasks.workunit.client.1.smithi117.stdout:5/260: mkdir d2/d9/d3c/d24/d50 0 2024-02-16T19:28:56.110 INFO:tasks.workunit.client.0.smithi022.stdout:4/164: truncate d4/d6/dd/df/d10/f19 3931186 0 2024-02-16T19:28:56.110 INFO:tasks.workunit.client.1.smithi117.stdout:1/81: dwrite d2/ff [4194304,4194304] 0 2024-02-16T19:28:56.112 INFO:tasks.workunit.client.1.smithi117.stdout:9/147: mkdir dd/d23/d2a 0 2024-02-16T19:28:56.113 INFO:tasks.workunit.client.1.smithi117.stdout:5/261: write d2/d9/d3c/f1a [535186,50712] 0 2024-02-16T19:28:56.113 INFO:tasks.workunit.client.0.smithi022.stdout:8/219: getdents d3/d12/d2b 0 2024-02-16T19:28:56.118 INFO:tasks.workunit.client.0.smithi022.stdout:5/221: symlink dd/d14/l45 0 2024-02-16T19:28:56.121 INFO:tasks.workunit.client.1.smithi117.stdout:7/143: unlink c1 0 2024-02-16T19:28:56.121 INFO:tasks.workunit.client.1.smithi117.stdout:1/82: dwrite d2/d4/f16 [0,4194304] 0 2024-02-16T19:28:56.121 INFO:tasks.workunit.client.1.smithi117.stdout:5/262: dread d2/d9/d3c/f1a [0,4194304] 0 2024-02-16T19:28:56.122 INFO:tasks.workunit.client.1.smithi117.stdout:5/263: stat d2/cb 0 2024-02-16T19:28:56.124 INFO:tasks.workunit.client.1.smithi117.stdout:5/264: write d2/d1d/f3f [739198,15767] 0 2024-02-16T19:28:56.124 INFO:tasks.workunit.client.1.smithi117.stdout:5/265: readlink d2/d3/l45 0 2024-02-16T19:28:56.125 INFO:tasks.workunit.client.0.smithi022.stdout:3/181: creat d7/d12/d18/f4e x:0 0 0 2024-02-16T19:28:56.126 INFO:tasks.workunit.client.1.smithi117.stdout:2/134: sync 2024-02-16T19:28:56.126 INFO:tasks.workunit.client.1.smithi117.stdout:4/169: sync 2024-02-16T19:28:56.126 INFO:tasks.workunit.client.1.smithi117.stdout:3/159: write d1/f19 [2989059,72512] 0 2024-02-16T19:28:56.127 INFO:tasks.workunit.client.1.smithi117.stdout:2/135: chown d0/fd 12150681 1 2024-02-16T19:28:56.127 INFO:tasks.workunit.client.0.smithi022.stdout:9/217: sync 2024-02-16T19:28:56.130 INFO:tasks.workunit.client.0.smithi022.stdout:3/182: dread d7/d12/d18/dd/d24/f3a [0,4194304] 0 2024-02-16T19:28:56.132 INFO:tasks.workunit.client.1.smithi117.stdout:0/195: creat d0/d9/d2c/f44 x:0 0 0 2024-02-16T19:28:56.135 INFO:tasks.workunit.client.0.smithi022.stdout:9/218: dwrite db/d14/d16/d27/d2e/f38 [0,4194304] 0 2024-02-16T19:28:56.138 INFO:tasks.workunit.client.1.smithi117.stdout:9/148: creat dd/d23/f2b x:0 0 0 2024-02-16T19:28:56.138 INFO:tasks.workunit.client.0.smithi022.stdout:5/222: mkdir dd/d14/d21/d2f/d46 0 2024-02-16T19:28:56.139 INFO:tasks.workunit.client.1.smithi117.stdout:9/149: read dd/d23/f1a [295178,48843] 0 2024-02-16T19:28:56.140 INFO:tasks.workunit.client.0.smithi022.stdout:5/223: truncate dd/d1f/f2b 1024798 0 2024-02-16T19:28:56.141 INFO:tasks.workunit.client.0.smithi022.stdout:0/260: link d10/d15/d23/c3d d10/d11/d27/c56 0 2024-02-16T19:28:56.141 INFO:tasks.workunit.client.0.smithi022.stdout:0/261: dread - d10/d22/f2a zero size 2024-02-16T19:28:56.142 INFO:tasks.workunit.client.0.smithi022.stdout:7/175: truncate d9/f25 2315224 0 2024-02-16T19:28:56.143 INFO:tasks.workunit.client.0.smithi022.stdout:7/176: stat d9/da/d1f/l1c 0 2024-02-16T19:28:56.146 INFO:tasks.workunit.client.0.smithi022.stdout:6/187: creat d1/d8/da/d39/f45 x:0 0 0 2024-02-16T19:28:56.147 INFO:tasks.workunit.client.0.smithi022.stdout:2/131: dread d0/d2/d18/f1a [4194304,4194304] 0 2024-02-16T19:28:56.147 INFO:tasks.workunit.client.0.smithi022.stdout:6/188: write d1/d8/da/d17/f2d [913234,41972] 0 2024-02-16T19:28:56.148 INFO:tasks.workunit.client.0.smithi022.stdout:1/301: creat d6/f55 x:0 0 0 2024-02-16T19:28:56.149 INFO:tasks.workunit.client.0.smithi022.stdout:6/189: dread - d1/d8/da/d17/f41 zero size 2024-02-16T19:28:56.152 INFO:tasks.workunit.client.1.smithi117.stdout:1/83: mkdir d2/d1d 0 2024-02-16T19:28:56.156 INFO:tasks.workunit.client.0.smithi022.stdout:8/220: dwrite d3/d12/d2b/f3a [0,4194304] 0 2024-02-16T19:28:56.162 INFO:tasks.workunit.client.1.smithi117.stdout:4/170: write d0/da/d21/f27 [1616962,57713] 0 2024-02-16T19:28:56.164 INFO:tasks.workunit.client.1.smithi117.stdout:2/136: mkdir d0/d29 0 2024-02-16T19:28:56.165 INFO:tasks.workunit.client.0.smithi022.stdout:5/224: creat dd/d14/f47 x:0 0 0 2024-02-16T19:28:56.166 INFO:tasks.workunit.client.0.smithi022.stdout:5/225: chown dd/d14/d21/d2f/d40/f3b 62138667 1 2024-02-16T19:28:56.167 INFO:tasks.workunit.client.0.smithi022.stdout:5/226: fdatasync dd/d1f/d2e/d35/d36/f3f 0 2024-02-16T19:28:56.168 INFO:tasks.workunit.client.0.smithi022.stdout:0/262: rename d10/f1f to d10/d11/d27/d3f/f57 0 2024-02-16T19:28:56.169 INFO:tasks.workunit.client.1.smithi117.stdout:0/196: creat d0/d9/d2c/f45 x:0 0 0 2024-02-16T19:28:56.169 INFO:tasks.workunit.client.0.smithi022.stdout:0/263: write d10/d22/f53 [2110640,128845] 0 2024-02-16T19:28:56.170 INFO:tasks.workunit.client.0.smithi022.stdout:7/177: rmdir d9/da/d1f 39 2024-02-16T19:28:56.173 INFO:tasks.workunit.client.0.smithi022.stdout:4/165: truncate d4/f1f 890096 0 2024-02-16T19:28:56.174 INFO:tasks.workunit.client.0.smithi022.stdout:4/166: chown d4/d6/dd/d31/f34 32 1 2024-02-16T19:28:56.175 INFO:tasks.workunit.client.0.smithi022.stdout:4/167: write d4/d6/dd/df/d10/f35 [280344,28793] 0 2024-02-16T19:28:56.177 INFO:tasks.workunit.client.1.smithi117.stdout:6/163: link d5/l31 d5/d1d/d28/d32/d37/l39 0 2024-02-16T19:28:56.181 INFO:tasks.workunit.client.0.smithi022.stdout:3/183: unlink d7/d12/d18/dd/d24/f3a 0 2024-02-16T19:28:56.181 INFO:tasks.workunit.client.0.smithi022.stdout:1/302: dread d6/d8/d22/d16/d17/d1d/f21 [0,4194304] 0 2024-02-16T19:28:56.183 INFO:tasks.workunit.client.1.smithi117.stdout:8/159: rename d4/d7/f21 to d4/f38 0 2024-02-16T19:28:56.183 INFO:tasks.workunit.client.0.smithi022.stdout:4/168: dwrite d4/d6/dd/f2a [0,4194304] 0 2024-02-16T19:28:56.187 INFO:tasks.workunit.client.0.smithi022.stdout:3/184: write d7/d12/d18/dd/d31/d41/f48 [757950,24417] 0 2024-02-16T19:28:56.189 INFO:tasks.workunit.client.0.smithi022.stdout:6/190: dwrite d1/d8/da/d29/f35 [0,4194304] 0 2024-02-16T19:28:56.190 INFO:tasks.workunit.client.0.smithi022.stdout:6/191: write d1/f18 [234502,78640] 0 2024-02-16T19:28:56.197 INFO:tasks.workunit.client.1.smithi117.stdout:4/171: rmdir d0 39 2024-02-16T19:28:56.202 INFO:tasks.workunit.client.1.smithi117.stdout:2/137: dread d0/f1e [0,4194304] 0 2024-02-16T19:28:56.202 INFO:tasks.workunit.client.0.smithi022.stdout:5/227: rename dd/d14 to dd/d1f/d48 0 2024-02-16T19:28:56.203 INFO:tasks.workunit.client.0.smithi022.stdout:1/303: rename d6/d8/d22 to d6/d8/d22/d3a/d3e/d56 22 2024-02-16T19:28:56.203 INFO:tasks.workunit.client.0.smithi022.stdout:3/185: rename d7/d12/d18 to d7/d12/d18/dd/d31/d41/d4f 22 2024-02-16T19:28:56.204 INFO:tasks.workunit.client.0.smithi022.stdout:3/186: chown d7/d12/f14 204517 1 2024-02-16T19:28:56.205 INFO:tasks.workunit.client.1.smithi117.stdout:5/266: truncate d2/d3/d21/d49/f2e 8325596 0 2024-02-16T19:28:56.206 INFO:tasks.workunit.client.0.smithi022.stdout:3/187: chown d7/d12/d18/dd/d31/d41/f48 18 1 2024-02-16T19:28:56.207 INFO:tasks.workunit.client.0.smithi022.stdout:0/264: mkdir d10/d11/d27/d3f/d58 0 2024-02-16T19:28:56.208 INFO:tasks.workunit.client.1.smithi117.stdout:3/160: truncate d1/d1b/d1e/f23 936188 0 2024-02-16T19:28:56.209 INFO:tasks.workunit.client.1.smithi117.stdout:9/150: mkdir dd/d23/d2a/d2c 0 2024-02-16T19:28:56.209 INFO:tasks.workunit.client.1.smithi117.stdout:9/151: write fa [5314042,119754] 0 2024-02-16T19:28:56.210 INFO:tasks.workunit.client.1.smithi117.stdout:9/152: write dd/f1e [2717366,123100] 0 2024-02-16T19:28:56.212 INFO:tasks.workunit.client.1.smithi117.stdout:9/153: write fb [2014764,58206] 0 2024-02-16T19:28:56.212 INFO:tasks.workunit.client.0.smithi022.stdout:7/178: dread d9/f25 [0,4194304] 0 2024-02-16T19:28:56.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:55 smithi022 ceph-mon[29221]: pgmap v259: 65 pgs: 65 active+clean; 368 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 588 KiB/s rd, 24 MiB/s wr, 181 op/s 2024-02-16T19:28:56.215 INFO:tasks.workunit.client.1.smithi117.stdout:6/164: rename d5/f23 to d5/d18/d20/f3a 0 2024-02-16T19:28:56.215 INFO:tasks.workunit.client.1.smithi117.stdout:0/197: dwrite d0/d9/d14/f1b [0,4194304] 0 2024-02-16T19:28:56.217 INFO:tasks.workunit.client.1.smithi117.stdout:8/160: mknod d4/d12/d2a/c39 0 2024-02-16T19:28:56.222 INFO:tasks.workunit.client.1.smithi117.stdout:1/84: mknod d2/d4/c1e 0 2024-02-16T19:28:56.223 INFO:tasks.workunit.client.1.smithi117.stdout:0/198: dwrite d0/d9/d22/d26/f2e [0,4194304] 0 2024-02-16T19:28:56.224 INFO:tasks.workunit.client.1.smithi117.stdout:0/199: write d0/f10 [8117525,81548] 0 2024-02-16T19:28:56.226 INFO:tasks.workunit.client.0.smithi022.stdout:6/192: mkdir d1/d46 0 2024-02-16T19:28:56.231 INFO:tasks.workunit.client.0.smithi022.stdout:5/228: mkdir dd/d1f/d48/d21/d2f/d40/d49 0 2024-02-16T19:28:56.235 INFO:tasks.workunit.client.1.smithi117.stdout:3/161: rmdir d1 39 2024-02-16T19:28:56.237 INFO:tasks.workunit.client.0.smithi022.stdout:3/188: mknod d7/d12/d18/dd/d31/c50 0 2024-02-16T19:28:56.237 INFO:tasks.workunit.client.0.smithi022.stdout:1/304: dwrite d6/d8/d22/d16/f1e [0,4194304] 0 2024-02-16T19:28:56.245 INFO:tasks.workunit.client.0.smithi022.stdout:8/221: creat d3/d12/d17/f50 x:0 0 0 2024-02-16T19:28:56.245 INFO:tasks.workunit.client.0.smithi022.stdout:9/219: truncate db/d18/f26 1906218 0 2024-02-16T19:28:56.246 INFO:tasks.workunit.client.0.smithi022.stdout:9/220: dread - db/d14/d16/f45 zero size 2024-02-16T19:28:56.249 INFO:tasks.workunit.client.0.smithi022.stdout:6/193: rename d1/d8/da/d17/f41 to d1/d8/d3c/f47 0 2024-02-16T19:28:56.250 INFO:tasks.workunit.client.1.smithi117.stdout:2/138: dwrite d0/d3/d10/f17 [0,4194304] 0 2024-02-16T19:28:56.252 INFO:tasks.workunit.client.0.smithi022.stdout:8/222: dwrite d3/d6/f34 [0,4194304] 0 2024-02-16T19:28:56.254 INFO:tasks.workunit.client.0.smithi022.stdout:5/229: unlink l0 0 2024-02-16T19:28:56.255 INFO:tasks.workunit.client.0.smithi022.stdout:0/265: write d10/d11/f2e [831586,56466] 0 2024-02-16T19:28:56.256 INFO:tasks.workunit.client.1.smithi117.stdout:2/139: dwrite d0/f12 [0,4194304] 0 2024-02-16T19:28:56.256 INFO:tasks.workunit.client.0.smithi022.stdout:5/230: dread - dd/d1f/d48/d21/d2f/d40/f3b zero size 2024-02-16T19:28:56.256 INFO:tasks.workunit.client.0.smithi022.stdout:0/266: fsync d10/d22/d49/f4d 0 2024-02-16T19:28:56.263 INFO:tasks.workunit.client.1.smithi117.stdout:9/154: write dd/f19 [446477,82855] 0 2024-02-16T19:28:56.263 INFO:tasks.workunit.client.1.smithi117.stdout:5/267: rename d2/d36/d3d to d2/d3/d21/d2f/d30/d38/d51 0 2024-02-16T19:28:56.264 INFO:tasks.workunit.client.0.smithi022.stdout:7/179: symlink d9/da/d1f/l38 0 2024-02-16T19:28:56.269 INFO:tasks.workunit.client.1.smithi117.stdout:7/144: link d4/f23 d4/d1c/f3c 0 2024-02-16T19:28:56.270 INFO:tasks.workunit.client.1.smithi117.stdout:7/145: write d4/d1c/f2c [102911,22606] 0 2024-02-16T19:28:56.273 INFO:tasks.workunit.client.0.smithi022.stdout:6/194: unlink d1/f42 0 2024-02-16T19:28:56.273 INFO:tasks.workunit.client.1.smithi117.stdout:6/165: dwrite d5/f15 [0,4194304] 0 2024-02-16T19:28:56.273 INFO:tasks.workunit.client.1.smithi117.stdout:7/146: read d4/d1c/f26 [723963,38677] 0 2024-02-16T19:28:56.275 INFO:tasks.workunit.client.0.smithi022.stdout:8/223: rmdir d3/d12 39 2024-02-16T19:28:56.278 INFO:tasks.workunit.client.0.smithi022.stdout:1/305: mknod d6/d8/c57 0 2024-02-16T19:28:56.280 INFO:tasks.workunit.client.0.smithi022.stdout:0/267: dread d10/d11/f2e [0,4194304] 0 2024-02-16T19:28:56.281 INFO:tasks.workunit.client.0.smithi022.stdout:0/268: stat lf 0 2024-02-16T19:28:56.281 INFO:tasks.workunit.client.0.smithi022.stdout:1/306: stat d6/d8/d22/d16/d17/c45 0 2024-02-16T19:28:56.281 INFO:tasks.workunit.client.1.smithi117.stdout:3/162: chown d1/c1f 120939412 1 2024-02-16T19:28:56.282 INFO:tasks.workunit.client.1.smithi117.stdout:0/200: read d0/f18 [2869188,67840] 0 2024-02-16T19:28:56.283 INFO:tasks.workunit.client.1.smithi117.stdout:0/201: write d0/f35 [526299,73978] 0 2024-02-16T19:28:56.284 INFO:tasks.workunit.client.0.smithi022.stdout:8/224: dwrite d3/d25/d33/f35 [0,4194304] 0 2024-02-16T19:28:56.284 INFO:tasks.workunit.client.0.smithi022.stdout:5/231: rename dd/d1f/d2e/d35/d36 to dd/d1f/d4a 0 2024-02-16T19:28:56.284 INFO:tasks.workunit.client.0.smithi022.stdout:8/225: stat d3/f10 0 2024-02-16T19:28:56.285 INFO:tasks.workunit.client.0.smithi022.stdout:5/232: fdatasync dd/d1f/f31 0 2024-02-16T19:28:56.286 INFO:tasks.workunit.client.0.smithi022.stdout:1/307: read d6/d8/d22/f3d [82845,66869] 0 2024-02-16T19:28:56.286 INFO:tasks.workunit.client.1.smithi117.stdout:2/140: truncate d0/d3/fc 1530653 0 2024-02-16T19:28:56.287 INFO:tasks.workunit.client.1.smithi117.stdout:2/141: write d0/d13/f15 [711672,88657] 0 2024-02-16T19:28:56.288 INFO:tasks.workunit.client.0.smithi022.stdout:7/180: fdatasync f5 0 2024-02-16T19:28:56.289 INFO:tasks.workunit.client.0.smithi022.stdout:1/308: write d6/d8/d22/d16/f1e [2939669,9597] 0 2024-02-16T19:28:56.291 INFO:tasks.workunit.client.0.smithi022.stdout:9/221: link db/d14/d19/f2f db/d14/d16/d21/d37/f4e 0 2024-02-16T19:28:56.291 INFO:tasks.workunit.client.0.smithi022.stdout:1/309: chown d6/d8/d22/f3d 0 1 2024-02-16T19:28:56.293 INFO:tasks.workunit.client.0.smithi022.stdout:6/195: mkdir d1/d8/da/d17/d48 0 2024-02-16T19:28:56.293 INFO:tasks.workunit.client.1.smithi117.stdout:5/268: truncate d2/d3/f6 2157232 0 2024-02-16T19:28:56.294 INFO:tasks.workunit.client.1.smithi117.stdout:5/269: fsync d2/d3/d21/d2f/d30/f3e 0 2024-02-16T19:28:56.295 INFO:tasks.workunit.client.1.smithi117.stdout:8/161: mkdir d4/d7/d37/d3a 0 2024-02-16T19:28:56.304 INFO:tasks.workunit.client.1.smithi117.stdout:6/166: creat d5/f3b x:0 0 0 2024-02-16T19:28:56.305 INFO:tasks.workunit.client.0.smithi022.stdout:3/189: link d7/d12/d18/dd/f20 d7/d12/d18/dd/d24/f51 0 2024-02-16T19:28:56.308 INFO:tasks.workunit.client.0.smithi022.stdout:5/233: unlink dd/d1f/d48/f29 0 2024-02-16T19:28:56.310 INFO:tasks.workunit.client.1.smithi117.stdout:4/172: unlink d0/c1b 0 2024-02-16T19:28:56.315 INFO:tasks.workunit.client.0.smithi022.stdout:3/190: dwrite d7/d12/d18/dd/d24/f3e [4194304,4194304] 0 2024-02-16T19:28:56.315 INFO:tasks.workunit.client.1.smithi117.stdout:0/202: rename d0/c3 to d0/d9/c46 0 2024-02-16T19:28:56.316 INFO:tasks.workunit.client.0.smithi022.stdout:9/222: creat db/d14/d33/f4f x:0 0 0 2024-02-16T19:28:56.317 INFO:tasks.workunit.client.0.smithi022.stdout:9/223: read db/d14/d19/f4d [2606465,73868] 0 2024-02-16T19:28:56.323 INFO:tasks.workunit.client.1.smithi117.stdout:7/147: write f3 [1313139,34706] 0 2024-02-16T19:28:56.326 INFO:tasks.workunit.client.0.smithi022.stdout:8/226: creat d3/d12/f51 x:0 0 0 2024-02-16T19:28:56.326 INFO:tasks.workunit.client.0.smithi022.stdout:8/227: chown d3/la 350364204 1 2024-02-16T19:28:56.326 INFO:tasks.workunit.client.0.smithi022.stdout:8/228: fdatasync d3/f2d 0 2024-02-16T19:28:56.330 INFO:tasks.workunit.client.1.smithi117.stdout:3/163: mknod d1/d30/c39 0 2024-02-16T19:28:56.331 INFO:tasks.workunit.client.1.smithi117.stdout:5/270: rename d2/d4 to d2/d9/d3c/d24/d50/d52 0 2024-02-16T19:28:56.332 INFO:tasks.workunit.client.0.smithi022.stdout:7/181: chown d9/d1b/c1d 942126679 1 2024-02-16T19:28:56.332 INFO:tasks.workunit.client.1.smithi117.stdout:5/271: chown d2/d9/d3c/d24 277 1 2024-02-16T19:28:56.333 INFO:tasks.workunit.client.1.smithi117.stdout:0/203: rmdir d0/d9/d14/d23 39 2024-02-16T19:28:56.334 INFO:tasks.workunit.client.0.smithi022.stdout:3/191: creat d7/d21/f52 x:0 0 0 2024-02-16T19:28:56.336 INFO:tasks.workunit.client.0.smithi022.stdout:9/224: creat db/d14/d19/d1a/f50 x:0 0 0 2024-02-16T19:28:56.338 INFO:tasks.workunit.client.0.smithi022.stdout:1/310: truncate d6/d8/d22/d16/d17/d1d/f33 3364263 0 2024-02-16T19:28:56.340 INFO:tasks.workunit.client.0.smithi022.stdout:6/196: creat d1/d8/da/d17/d48/f49 x:0 0 0 2024-02-16T19:28:56.340 INFO:tasks.workunit.client.0.smithi022.stdout:1/311: write d6/d8/d22/d16/f4b [1413633,122626] 0 2024-02-16T19:28:56.342 INFO:tasks.workunit.client.1.smithi117.stdout:1/85: getdents d2/d18 0 2024-02-16T19:28:56.342 INFO:tasks.workunit.client.0.smithi022.stdout:9/225: dwrite db/f1d [0,4194304] 0 2024-02-16T19:28:56.344 INFO:tasks.workunit.client.0.smithi022.stdout:0/269: rename d10/d11/d27/c56 to d10/d15/d23/c59 0 2024-02-16T19:28:56.346 INFO:tasks.workunit.client.1.smithi117.stdout:7/148: mknod d4/d1c/c3d 0 2024-02-16T19:28:56.347 INFO:tasks.workunit.client.0.smithi022.stdout:8/229: mknod d3/d25/c52 0 2024-02-16T19:28:56.349 INFO:tasks.workunit.client.1.smithi117.stdout:6/167: link d5/f1c d5/d18/d20/d2d/f3c 0 2024-02-16T19:28:56.350 INFO:tasks.workunit.client.0.smithi022.stdout:5/234: mkdir dd/d1f/d48/d21/d2f/d40/d49/d4b 0 2024-02-16T19:28:56.350 INFO:tasks.workunit.client.0.smithi022.stdout:0/270: dwrite f7 [0,4194304] 0 2024-02-16T19:28:56.351 INFO:tasks.workunit.client.0.smithi022.stdout:5/235: write f3 [2076829,95716] 0 2024-02-16T19:28:56.352 INFO:tasks.workunit.client.0.smithi022.stdout:0/271: write d10/d11/d50/f46 [892181,108168] 0 2024-02-16T19:28:56.353 INFO:tasks.workunit.client.1.smithi117.stdout:3/164: rename d1/c1c to d1/d30/c3a 0 2024-02-16T19:28:56.354 INFO:tasks.workunit.client.1.smithi117.stdout:5/272: rmdir d2/d9/d3c/d24/d50/d52 39 2024-02-16T19:28:56.358 INFO:tasks.workunit.client.0.smithi022.stdout:1/312: rmdir d6/d8/d22/d16/d17/d26 39 2024-02-16T19:28:56.361 INFO:tasks.workunit.client.0.smithi022.stdout:9/226: unlink db/d14/d19/f2f 0 2024-02-16T19:28:56.361 INFO:tasks.workunit.client.1.smithi117.stdout:0/204: dwrite d0/fd [0,4194304] 0 2024-02-16T19:28:56.362 INFO:tasks.workunit.client.0.smithi022.stdout:6/197: dread d1/d8/ff [0,4194304] 0 2024-02-16T19:28:56.362 INFO:tasks.workunit.client.0.smithi022.stdout:9/227: rename db/d18 to db/d18/d51 22 2024-02-16T19:28:56.363 INFO:tasks.workunit.client.0.smithi022.stdout:9/228: stat db/d14/d16/d21/d4a 0 2024-02-16T19:28:56.367 INFO:tasks.workunit.client.1.smithi117.stdout:7/149: creat d4/d13/f3e x:0 0 0 2024-02-16T19:28:56.372 INFO:tasks.workunit.client.1.smithi117.stdout:3/165: mkdir d1/dd/d3b 0 2024-02-16T19:28:56.373 INFO:tasks.workunit.client.1.smithi117.stdout:3/166: readlink d1/dd/l12 0 2024-02-16T19:28:56.375 INFO:tasks.workunit.client.1.smithi117.stdout:5/273: rename d2/f13 to d2/d9/d3c/d24/f53 0 2024-02-16T19:28:56.376 INFO:tasks.workunit.client.1.smithi117.stdout:3/167: dread d1/d1b/d1e/d29/f33 [0,4194304] 0 2024-02-16T19:28:56.377 INFO:tasks.workunit.client.1.smithi117.stdout:3/168: write d1/f28 [59383,50571] 0 2024-02-16T19:28:56.378 INFO:tasks.workunit.client.1.smithi117.stdout:3/169: readlink d1/l18 0 2024-02-16T19:28:56.379 INFO:tasks.workunit.client.1.smithi117.stdout:2/142: dread d0/d3/fc [0,4194304] 0 2024-02-16T19:28:56.379 INFO:tasks.workunit.client.0.smithi022.stdout:5/236: rmdir dd/d1f/d2e 39 2024-02-16T19:28:56.380 INFO:tasks.workunit.client.1.smithi117.stdout:8/162: getdents d4/d12/d2a 0 2024-02-16T19:28:56.380 INFO:tasks.workunit.client.1.smithi117.stdout:8/163: write d4/d12/d2a/f2c [2081318,3730] 0 2024-02-16T19:28:56.381 INFO:tasks.workunit.client.0.smithi022.stdout:0/272: mkdir d10/d15/d23/d38/d5a 0 2024-02-16T19:28:56.382 INFO:tasks.workunit.client.1.smithi117.stdout:6/168: write d5/f7 [588929,110895] 0 2024-02-16T19:28:56.383 INFO:tasks.workunit.client.1.smithi117.stdout:1/86: mkdir d2/d1d/d1f 0 2024-02-16T19:28:56.384 INFO:tasks.workunit.client.0.smithi022.stdout:3/192: mkdir d7/d12/d18/dd/d24/d53 0 2024-02-16T19:28:56.384 INFO:tasks.workunit.client.1.smithi117.stdout:1/87: write d2/ff [3483847,119995] 0 2024-02-16T19:28:56.384 INFO:tasks.workunit.client.1.smithi117.stdout:1/88: dread - d2/d4/f17 zero size 2024-02-16T19:28:56.385 INFO:tasks.workunit.client.1.smithi117.stdout:0/205: mknod d0/d9/d2c/d37/c47 0 2024-02-16T19:28:56.385 INFO:tasks.workunit.client.0.smithi022.stdout:3/193: truncate d7/d12/d18/d4b/f4d 372768 0 2024-02-16T19:28:56.386 INFO:tasks.workunit.client.0.smithi022.stdout:1/313: stat d6/l27 0 2024-02-16T19:28:56.387 INFO:tasks.workunit.client.0.smithi022.stdout:7/182: write d9/da/f29 [4732557,85420] 0 2024-02-16T19:28:56.388 INFO:tasks.workunit.client.1.smithi117.stdout:0/206: dread d0/d9/d22/d27/d31/f38 [0,4194304] 0 2024-02-16T19:28:56.388 INFO:tasks.workunit.client.1.smithi117.stdout:0/207: chown d0/d9/d2c/f44 1 1 2024-02-16T19:28:56.389 INFO:tasks.workunit.client.0.smithi022.stdout:4/169: sync 2024-02-16T19:28:56.389 INFO:tasks.workunit.client.0.smithi022.stdout:2/132: sync 2024-02-16T19:28:56.390 INFO:tasks.workunit.client.0.smithi022.stdout:6/198: rename d1/d8/da/d17/d48/f49 to d1/d8/f4a 0 2024-02-16T19:28:56.392 INFO:tasks.workunit.client.0.smithi022.stdout:8/230: creat d3/d12/d2b/d46/f53 x:0 0 0 2024-02-16T19:28:56.394 INFO:tasks.workunit.client.0.smithi022.stdout:1/314: dwrite d6/d8/d22/d16/f40 [0,4194304] 0 2024-02-16T19:28:56.395 INFO:tasks.workunit.client.1.smithi117.stdout:5/274: rename d2/d3/d21/d2f/d3b to d2/d4c/d54 0 2024-02-16T19:28:56.396 INFO:tasks.workunit.client.1.smithi117.stdout:5/275: dread - d2/d3/d21/d49/d41/f4b zero size 2024-02-16T19:28:56.396 INFO:tasks.workunit.client.0.smithi022.stdout:1/315: dread d6/d8/d22/f15 [0,4194304] 0 2024-02-16T19:28:56.396 INFO:tasks.workunit.client.1.smithi117.stdout:3/170: rmdir d1 39 2024-02-16T19:28:56.397 INFO:tasks.workunit.client.0.smithi022.stdout:1/316: readlink d6/d8/d22/d47/d51/l53 0 2024-02-16T19:28:56.404 INFO:tasks.workunit.client.0.smithi022.stdout:9/229: mknod db/d14/d16/d21/d37/c52 0 2024-02-16T19:28:56.407 INFO:tasks.workunit.client.0.smithi022.stdout:0/273: creat d10/d11/d27/d3f/f5b x:0 0 0 2024-02-16T19:28:56.412 INFO:tasks.workunit.client.1.smithi117.stdout:1/89: mkdir d2/d18/d20 0 2024-02-16T19:28:56.413 INFO:tasks.workunit.client.1.smithi117.stdout:1/90: stat d2/cc 0 2024-02-16T19:28:56.414 INFO:tasks.workunit.client.0.smithi022.stdout:7/183: mknod d9/da/d2c/c39 0 2024-02-16T19:28:56.416 INFO:tasks.workunit.client.1.smithi117.stdout:0/208: rename d0/f18 to d0/d9/d22/d26/f48 0 2024-02-16T19:28:56.417 INFO:tasks.workunit.client.0.smithi022.stdout:4/170: creat d4/d6/dd/f37 x:0 0 0 2024-02-16T19:28:56.418 INFO:tasks.workunit.client.1.smithi117.stdout:1/91: dwrite d2/d4/f17 [0,4194304] 0 2024-02-16T19:28:56.418 INFO:tasks.workunit.client.0.smithi022.stdout:4/171: fsync d4/d6/dd/df/d10/f35 0 2024-02-16T19:28:56.419 INFO:tasks.workunit.client.1.smithi117.stdout:8/164: unlink d4/d7/ca 0 2024-02-16T19:28:56.421 INFO:tasks.workunit.client.1.smithi117.stdout:1/92: read d2/d4/fa [3591188,34822] 0 2024-02-16T19:28:56.422 INFO:tasks.workunit.client.1.smithi117.stdout:8/165: dread - d4/d12/d2a/f2b zero size 2024-02-16T19:28:56.422 INFO:tasks.workunit.client.1.smithi117.stdout:6/169: fsync d5/f11 0 2024-02-16T19:28:56.424 INFO:tasks.workunit.client.0.smithi022.stdout:8/231: mkdir d3/d12/d2b/d54 0 2024-02-16T19:28:56.424 INFO:tasks.workunit.client.1.smithi117.stdout:3/171: dwrite d1/f8 [4194304,4194304] 0 2024-02-16T19:28:56.427 INFO:tasks.workunit.client.0.smithi022.stdout:1/317: write d6/d8/d22/d16/d17/d26/f43 [374277,119308] 0 2024-02-16T19:28:56.429 INFO:tasks.workunit.client.0.smithi022.stdout:8/232: dread d3/d12/f48 [0,4194304] 0 2024-02-16T19:28:56.431 INFO:tasks.workunit.client.0.smithi022.stdout:5/237: link dd/d1f/d48/d21/f25 dd/d1f/d48/d21/d2f/d40/d49/f4c 0 2024-02-16T19:28:56.432 INFO:tasks.workunit.client.0.smithi022.stdout:5/238: chown dd/d1f/d48/f47 385 1 2024-02-16T19:28:56.432 INFO:tasks.workunit.client.1.smithi117.stdout:8/166: dwrite d4/d12/f26 [0,4194304] 0 2024-02-16T19:28:56.434 INFO:tasks.workunit.client.0.smithi022.stdout:9/230: mkdir db/d14/d16/d28/d53 0 2024-02-16T19:28:56.437 INFO:tasks.workunit.client.1.smithi117.stdout:0/209: dread d0/f5 [4194304,4194304] 0 2024-02-16T19:28:56.438 INFO:tasks.workunit.client.0.smithi022.stdout:3/194: mknod d7/d12/d18/dd/d24/c54 0 2024-02-16T19:28:56.440 INFO:tasks.workunit.client.0.smithi022.stdout:7/184: readlink d9/d1b/d23/l2e 0 2024-02-16T19:28:56.442 INFO:tasks.workunit.client.0.smithi022.stdout:2/133: symlink d0/l29 0 2024-02-16T19:28:56.446 INFO:tasks.workunit.client.0.smithi022.stdout:2/134: dread d0/d2/d18/f1a [4194304,4194304] 0 2024-02-16T19:28:56.446 INFO:tasks.workunit.client.0.smithi022.stdout:6/199: write d1/d8/da/f14 [342904,90525] 0 2024-02-16T19:28:56.447 INFO:tasks.workunit.client.1.smithi117.stdout:6/170: creat d5/d1d/f3d x:0 0 0 2024-02-16T19:28:56.448 INFO:tasks.workunit.client.1.smithi117.stdout:5/276: truncate d2/d9/d3c/d24/f26 7173728 0 2024-02-16T19:28:56.449 INFO:tasks.workunit.client.1.smithi117.stdout:6/171: truncate d5/db/f1f 5167195 0 2024-02-16T19:28:56.451 INFO:tasks.workunit.client.1.smithi117.stdout:8/167: dread d4/f6 [0,4194304] 0 2024-02-16T19:28:56.452 INFO:tasks.workunit.client.0.smithi022.stdout:1/318: symlink d6/d8/d22/d16/d17/d26/l58 0 2024-02-16T19:28:56.454 INFO:tasks.workunit.client.0.smithi022.stdout:6/200: dwrite d1/d8/da/d29/f3f [0,4194304] 0 2024-02-16T19:28:56.454 INFO:tasks.workunit.client.1.smithi117.stdout:0/210: creat d0/d9/d2c/d37/f49 x:0 0 0 2024-02-16T19:28:56.455 INFO:tasks.workunit.client.0.smithi022.stdout:5/239: rmdir dd/d1f/d48/d21 39 2024-02-16T19:28:56.455 INFO:tasks.workunit.client.0.smithi022.stdout:4/172: dread d4/d6/dd/f1e [0,4194304] 0 2024-02-16T19:28:56.455 INFO:tasks.workunit.client.1.smithi117.stdout:1/93: truncate d2/d4/f9 11215 0 2024-02-16T19:28:56.456 INFO:tasks.workunit.client.1.smithi117.stdout:1/94: fsync d2/d4/f15 0 2024-02-16T19:28:56.457 INFO:tasks.workunit.client.1.smithi117.stdout:1/95: chown d2/d4/fa 16854 1 2024-02-16T19:28:56.458 INFO:tasks.workunit.client.0.smithi022.stdout:0/274: symlink d10/d11/d27/d3f/d58/l5c 0 2024-02-16T19:28:56.458 INFO:tasks.workunit.client.1.smithi117.stdout:5/277: creat d2/d1d/f55 x:0 0 0 2024-02-16T19:28:56.460 INFO:tasks.workunit.client.0.smithi022.stdout:0/275: fsync d10/d11/d27/d3f/f5b 0 2024-02-16T19:28:56.460 INFO:tasks.workunit.client.1.smithi117.stdout:6/172: symlink d5/db/l3e 0 2024-02-16T19:28:56.461 INFO:tasks.workunit.client.1.smithi117.stdout:3/172: mkdir d1/dd/d32/d3c 0 2024-02-16T19:28:56.461 INFO:tasks.workunit.client.1.smithi117.stdout:3/173: readlink d1/l18 0 2024-02-16T19:28:56.462 INFO:tasks.workunit.client.0.smithi022.stdout:6/201: dwrite d1/d8/f1f [0,4194304] 0 2024-02-16T19:28:56.463 INFO:tasks.workunit.client.1.smithi117.stdout:8/168: symlink d4/d12/d2a/l3b 0 2024-02-16T19:28:56.464 INFO:tasks.workunit.client.1.smithi117.stdout:0/211: symlink d0/d9/d22/d27/d31/l4a 0 2024-02-16T19:28:56.466 INFO:tasks.workunit.client.1.smithi117.stdout:1/96: rmdir d2/d1d 39 2024-02-16T19:28:56.471 INFO:tasks.workunit.client.1.smithi117.stdout:6/173: rename d5/d1d/d28/f30 to d5/d1d/d28/d32/f3f 0 2024-02-16T19:28:56.474 INFO:tasks.workunit.client.0.smithi022.stdout:5/240: write dd/d1f/d48/d21/d2f/d40/f3b [475469,21070] 0 2024-02-16T19:28:56.475 INFO:tasks.workunit.client.0.smithi022.stdout:4/173: mkdir d4/d38 0 2024-02-16T19:28:56.479 INFO:tasks.workunit.client.1.smithi117.stdout:0/212: dread d0/fd [0,4194304] 0 2024-02-16T19:28:56.479 INFO:tasks.workunit.client.1.smithi117.stdout:0/213: fsync d0/f1a 0 2024-02-16T19:28:56.480 INFO:tasks.workunit.client.1.smithi117.stdout:0/214: chown d0/d9/d14/d1d/d1e 213 1 2024-02-16T19:28:56.481 INFO:tasks.workunit.client.1.smithi117.stdout:0/215: stat d0/d9/d2c/d37 0 2024-02-16T19:28:56.481 INFO:tasks.workunit.client.1.smithi117.stdout:0/216: stat d0/d9/d2c 0 2024-02-16T19:28:56.482 INFO:tasks.workunit.client.1.smithi117.stdout:0/217: write d0/f40 [415671,25324] 0 2024-02-16T19:28:56.484 INFO:tasks.workunit.client.1.smithi117.stdout:6/174: read d5/f13 [192132,55360] 0 2024-02-16T19:28:56.485 INFO:tasks.workunit.client.0.smithi022.stdout:2/135: unlink d0/c7 0 2024-02-16T19:28:56.487 INFO:tasks.workunit.client.0.smithi022.stdout:1/319: write d6/d8/d22/d16/f41 [3531345,100654] 0 2024-02-16T19:28:56.488 INFO:tasks.workunit.client.0.smithi022.stdout:6/202: creat d1/d8/da/d39/f4b x:0 0 0 2024-02-16T19:28:56.489 INFO:tasks.workunit.client.0.smithi022.stdout:0/276: dwrite d10/d11/d50/f3e [0,4194304] 0 2024-02-16T19:28:56.489 INFO:tasks.workunit.client.1.smithi117.stdout:3/174: creat d1/dd/d3b/f3d x:0 0 0 2024-02-16T19:28:56.490 INFO:tasks.workunit.client.1.smithi117.stdout:6/175: dwrite d5/d1d/f27 [0,4194304] 0 2024-02-16T19:28:56.491 INFO:tasks.workunit.client.1.smithi117.stdout:6/176: chown d5/f1c 97227 1 2024-02-16T19:28:56.492 INFO:tasks.workunit.client.0.smithi022.stdout:8/233: rename d3/d12/d2b/d2e/f4a to d3/f55 0 2024-02-16T19:28:56.492 INFO:tasks.workunit.client.0.smithi022.stdout:8/234: chown d3/d12 4 1 2024-02-16T19:28:56.493 INFO:tasks.workunit.client.0.smithi022.stdout:8/235: write d3/d12/d2b/f3c [1023836,125156] 0 2024-02-16T19:28:56.494 INFO:tasks.workunit.client.1.smithi117.stdout:1/97: mknod d2/d19/c21 0 2024-02-16T19:28:56.497 INFO:tasks.workunit.client.0.smithi022.stdout:9/231: rmdir db/d14/d16/d28/d53 0 2024-02-16T19:28:56.499 INFO:tasks.workunit.client.0.smithi022.stdout:3/195: rename d7/d12/d18/dd/d24/f51 to d7/d12/d18/dd/d24/f55 0 2024-02-16T19:28:56.501 INFO:tasks.workunit.client.0.smithi022.stdout:7/185: creat d9/da/f3a x:0 0 0 2024-02-16T19:28:56.502 INFO:tasks.workunit.client.1.smithi117.stdout:0/218: mknod d0/d9/d22/d26/c4b 0 2024-02-16T19:28:56.503 INFO:tasks.workunit.client.1.smithi117.stdout:0/219: chown d0/d9/d2c/c3a 1867624 1 2024-02-16T19:28:56.506 INFO:tasks.workunit.client.1.smithi117.stdout:3/175: read d1/dd/f17 [414617,88277] 0 2024-02-16T19:28:56.507 INFO:tasks.workunit.client.0.smithi022.stdout:6/203: mkdir d1/d8/d3c/d4c 0 2024-02-16T19:28:56.508 INFO:tasks.workunit.client.1.smithi117.stdout:0/220: dwrite d0/f1a [0,4194304] 0 2024-02-16T19:28:56.509 INFO:tasks.workunit.client.0.smithi022.stdout:6/204: write d1/d8/da/d17/f2c [852335,123094] 0 2024-02-16T19:28:56.509 INFO:tasks.workunit.client.1.smithi117.stdout:6/177: unlink d5/d1d/f35 0 2024-02-16T19:28:56.510 INFO:tasks.workunit.client.0.smithi022.stdout:0/277: rmdir d10/d22 39 2024-02-16T19:28:56.512 INFO:tasks.workunit.client.1.smithi117.stdout:8/169: creat d4/f3c x:0 0 0 2024-02-16T19:28:56.512 INFO:tasks.workunit.client.0.smithi022.stdout:0/278: truncate d10/d11/d27/d3f/f5b 547921 0 2024-02-16T19:28:56.514 INFO:tasks.workunit.client.0.smithi022.stdout:4/174: dwrite d4/f23 [0,4194304] 0 2024-02-16T19:28:56.514 INFO:tasks.workunit.client.0.smithi022.stdout:0/279: fdatasync d10/d15/d23/f47 0 2024-02-16T19:28:56.520 INFO:tasks.workunit.client.0.smithi022.stdout:9/232: symlink db/d14/d33/l54 0 2024-02-16T19:28:56.521 INFO:tasks.workunit.client.1.smithi117.stdout:5/278: getdents d2/d9/d3c/d24 0 2024-02-16T19:28:56.522 INFO:tasks.workunit.client.0.smithi022.stdout:9/233: write db/d14/d19/d1a/f24 [2634754,119794] 0 2024-02-16T19:28:56.523 INFO:tasks.workunit.client.0.smithi022.stdout:3/196: rename d7/d12/d18/dd/d24/f2f to d7/d12/d18/dd/d24/d29/f56 0 2024-02-16T19:28:56.525 INFO:tasks.workunit.client.1.smithi117.stdout:3/176: mkdir d1/dd/d3e 0 2024-02-16T19:28:56.525 INFO:tasks.workunit.client.1.smithi117.stdout:3/177: readlink d1/l4 0 2024-02-16T19:28:56.526 INFO:tasks.workunit.client.1.smithi117.stdout:0/221: mknod d0/c4c 0 2024-02-16T19:28:56.527 INFO:tasks.workunit.client.0.smithi022.stdout:2/136: creat d0/d2/d18/d19/d25/d27/f2a x:0 0 0 2024-02-16T19:28:56.528 INFO:tasks.workunit.client.1.smithi117.stdout:6/178: creat d5/d1d/d28/f40 x:0 0 0 2024-02-16T19:28:56.529 INFO:tasks.workunit.client.1.smithi117.stdout:6/179: read d5/db/f19 [189470,30022] 0 2024-02-16T19:28:56.529 INFO:tasks.workunit.client.0.smithi022.stdout:1/320: mkdir d6/d8/d22/d59 0 2024-02-16T19:28:56.530 INFO:tasks.workunit.client.0.smithi022.stdout:1/321: readlink d6/d8/d22/d16/d17/d26/l58 0 2024-02-16T19:28:56.531 INFO:tasks.workunit.client.1.smithi117.stdout:8/170: creat d4/d7/f3d x:0 0 0 2024-02-16T19:28:56.535 INFO:tasks.workunit.client.1.smithi117.stdout:1/98: rename d2/d19/c21 to d2/d1d/c22 0 2024-02-16T19:28:56.536 INFO:tasks.workunit.client.1.smithi117.stdout:1/99: write d2/ff [4193325,86236] 0 2024-02-16T19:28:56.539 INFO:tasks.workunit.client.0.smithi022.stdout:7/186: write f5 [758784,41729] 0 2024-02-16T19:28:56.540 INFO:tasks.workunit.client.0.smithi022.stdout:7/187: dread - d9/da/d1f/f31 zero size 2024-02-16T19:28:56.541 INFO:tasks.workunit.client.0.smithi022.stdout:4/175: symlink d4/d6/d14/l39 0 2024-02-16T19:28:56.541 INFO:tasks.workunit.client.0.smithi022.stdout:7/188: write d9/da/d1f/f31 [350234,704] 0 2024-02-16T19:28:56.542 INFO:tasks.workunit.client.1.smithi117.stdout:0/222: write d0/f5 [3824383,79927] 0 2024-02-16T19:28:56.545 INFO:tasks.workunit.client.1.smithi117.stdout:6/180: unlink d5/f1c 0 2024-02-16T19:28:56.546 INFO:tasks.workunit.client.1.smithi117.stdout:0/223: dread d0/d9/d22/d26/f2e [0,4194304] 0 2024-02-16T19:28:56.546 INFO:tasks.workunit.client.1.smithi117.stdout:6/181: chown d5/f15 169246340 1 2024-02-16T19:28:56.547 INFO:tasks.workunit.client.1.smithi117.stdout:0/224: write d0/d9/d22/d26/f2e [207441,76110] 0 2024-02-16T19:28:56.549 INFO:tasks.workunit.client.0.smithi022.stdout:9/234: mkdir db/d14/d16/d27/d55 0 2024-02-16T19:28:56.550 INFO:tasks.workunit.client.1.smithi117.stdout:8/171: rename d4/d7/c35 to d4/d7/d37/c3e 0 2024-02-16T19:28:56.552 INFO:tasks.workunit.client.0.smithi022.stdout:6/205: rename d1/d8/da/d39 to d1/d8/d3c/d4d 0 2024-02-16T19:28:56.553 INFO:tasks.workunit.client.0.smithi022.stdout:3/197: creat d7/d12/d18/f57 x:0 0 0 2024-02-16T19:28:56.556 INFO:tasks.workunit.client.1.smithi117.stdout:3/178: creat d1/dd/d32/d3c/f3f x:0 0 0 2024-02-16T19:28:56.557 INFO:tasks.workunit.client.1.smithi117.stdout:1/100: dwrite d2/d4/fa [0,4194304] 0 2024-02-16T19:28:56.557 INFO:tasks.workunit.client.1.smithi117.stdout:3/179: stat d1/dd/l12 0 2024-02-16T19:28:56.558 INFO:tasks.workunit.client.1.smithi117.stdout:1/101: fdatasync d2/d4/f17 0 2024-02-16T19:28:56.558 INFO:tasks.workunit.client.1.smithi117.stdout:3/180: readlink d1/d1b/l2f 0 2024-02-16T19:28:56.562 INFO:tasks.workunit.client.0.smithi022.stdout:2/137: dread d0/d2/f1c [0,4194304] 0 2024-02-16T19:28:56.562 INFO:tasks.workunit.client.0.smithi022.stdout:8/236: truncate d3/d6/f7 2640879 0 2024-02-16T19:28:56.562 INFO:tasks.workunit.client.0.smithi022.stdout:1/322: dread d6/d8/d22/f3d [0,4194304] 0 2024-02-16T19:28:56.564 INFO:tasks.workunit.client.0.smithi022.stdout:4/176: fdatasync d4/d6/dd/f1e 0 2024-02-16T19:28:56.564 INFO:tasks.workunit.client.1.smithi117.stdout:5/279: link d2/c10 d2/d3/d21/d2f/d30/d4e/c56 0 2024-02-16T19:28:56.566 INFO:tasks.workunit.client.0.smithi022.stdout:0/280: rmdir d10/d22/d49 39 2024-02-16T19:28:56.572 INFO:tasks.workunit.client.0.smithi022.stdout:8/237: dwrite d3/d12/d2b/d46/f53 [0,4194304] 0 2024-02-16T19:28:56.574 INFO:tasks.workunit.client.0.smithi022.stdout:6/206: mknod d1/d8/d3c/d4d/c4e 0 2024-02-16T19:28:56.579 INFO:tasks.workunit.client.1.smithi117.stdout:0/225: write d0/fd [1591074,39577] 0 2024-02-16T19:28:56.580 INFO:tasks.workunit.client.1.smithi117.stdout:3/181: rename d1/d1b/d1e/c2b to d1/d1b/d1e/c40 0 2024-02-16T19:28:56.580 INFO:tasks.workunit.client.0.smithi022.stdout:8/238: dwrite d3/d12/d2b/f3a [0,4194304] 0 2024-02-16T19:28:56.587 INFO:tasks.workunit.client.1.smithi117.stdout:6/182: creat d5/d1d/d28/d32/d37/f41 x:0 0 0 2024-02-16T19:28:56.587 INFO:tasks.workunit.client.1.smithi117.stdout:3/182: dwrite d1/d1b/d1e/d37/f38 [0,4194304] 0 2024-02-16T19:28:56.589 INFO:tasks.workunit.client.1.smithi117.stdout:3/183: write d1/d1b/d1e/d29/f33 [331918,77418] 0 2024-02-16T19:28:56.590 INFO:tasks.workunit.client.0.smithi022.stdout:4/177: mkdir d4/d6/d3a 0 2024-02-16T19:28:56.590 INFO:tasks.workunit.client.1.smithi117.stdout:5/280: mknod d2/d3/c57 0 2024-02-16T19:28:56.591 INFO:tasks.workunit.client.1.smithi117.stdout:6/183: dread d5/d18/f29 [0,4194304] 0 2024-02-16T19:28:56.592 INFO:tasks.workunit.client.1.smithi117.stdout:1/102: unlink d2/d4/c13 0 2024-02-16T19:28:56.593 INFO:tasks.workunit.client.1.smithi117.stdout:1/103: stat d2/d1d/d1f 0 2024-02-16T19:28:56.594 INFO:tasks.workunit.client.1.smithi117.stdout:0/226: creat d0/d9/d22/d27/f4d x:0 0 0 2024-02-16T19:28:56.594 INFO:tasks.workunit.client.1.smithi117.stdout:1/104: chown d2/d1d/d1f 88 1 2024-02-16T19:28:56.596 INFO:tasks.workunit.client.1.smithi117.stdout:1/105: read d2/d4/fa [1778310,85331] 0 2024-02-16T19:28:56.598 INFO:tasks.workunit.client.1.smithi117.stdout:3/184: dwrite d1/f8 [0,4194304] 0 2024-02-16T19:28:56.599 INFO:tasks.workunit.client.1.smithi117.stdout:3/185: write d1/f8 [5519675,46225] 0 2024-02-16T19:28:56.600 INFO:tasks.workunit.client.1.smithi117.stdout:3/186: fsync d1/d1b/d1e/d29/f33 0 2024-02-16T19:28:56.603 INFO:tasks.workunit.client.1.smithi117.stdout:0/227: dread d0/f1f [0,4194304] 0 2024-02-16T19:28:56.604 INFO:tasks.workunit.client.1.smithi117.stdout:0/228: fsync d0/fd 0 2024-02-16T19:28:56.606 INFO:tasks.workunit.client.0.smithi022.stdout:7/189: truncate d9/da/d2c/f2d 862187 0 2024-02-16T19:28:56.608 INFO:tasks.workunit.client.0.smithi022.stdout:2/138: rename d0/d2/cf to d0/d2/d18/d1f/c2b 0 2024-02-16T19:28:56.609 INFO:tasks.workunit.client.0.smithi022.stdout:1/323: rename d6/d8 to d6/d8/d22/d59/d5a 22 2024-02-16T19:28:56.611 INFO:tasks.workunit.client.0.smithi022.stdout:3/198: write d7/d12/d18/dd/ff [2776193,63752] 0 2024-02-16T19:28:56.613 INFO:tasks.workunit.client.0.smithi022.stdout:0/281: creat d10/d15/d23/d38/d5a/f5d x:0 0 0 2024-02-16T19:28:56.614 INFO:tasks.workunit.client.0.smithi022.stdout:4/178: symlink d4/d6/dd/l3b 0 2024-02-16T19:28:56.614 INFO:tasks.workunit.client.0.smithi022.stdout:0/282: readlink d10/d11/d27/d3f/d58/l5c 0 2024-02-16T19:28:56.615 INFO:tasks.workunit.client.1.smithi117.stdout:6/184: dwrite d5/d1d/d28/f2b [0,4194304] 0 2024-02-16T19:28:56.617 INFO:tasks.workunit.client.1.smithi117.stdout:3/187: creat d1/d1b/f41 x:0 0 0 2024-02-16T19:28:56.617 INFO:tasks.workunit.client.1.smithi117.stdout:3/188: chown d1 362 1 2024-02-16T19:28:56.617 INFO:tasks.workunit.client.0.smithi022.stdout:8/239: symlink d3/d12/d2b/d54/l56 0 2024-02-16T19:28:56.618 INFO:tasks.workunit.client.0.smithi022.stdout:3/199: dwrite d7/d12/f25 [0,4194304] 0 2024-02-16T19:28:56.619 INFO:tasks.workunit.client.0.smithi022.stdout:8/240: chown d3/d12/d2b/d54 105 1 2024-02-16T19:28:56.621 INFO:tasks.workunit.client.0.smithi022.stdout:4/179: dread d4/f23 [0,4194304] 0 2024-02-16T19:28:56.623 INFO:tasks.workunit.client.0.smithi022.stdout:4/180: chown d4/d6/dd/df/c20 6298 1 2024-02-16T19:28:56.624 INFO:tasks.workunit.client.1.smithi117.stdout:1/106: creat d2/d1d/d1f/f23 x:0 0 0 2024-02-16T19:28:56.624 INFO:tasks.workunit.client.1.smithi117.stdout:3/189: dwrite d1/d1b/f25 [0,4194304] 0 2024-02-16T19:28:56.625 INFO:tasks.workunit.client.1.smithi117.stdout:1/107: chown d2 0 1 2024-02-16T19:28:56.626 INFO:tasks.workunit.client.1.smithi117.stdout:6/185: creat d5/d18/d20/f42 x:0 0 0 2024-02-16T19:28:56.626 INFO:tasks.workunit.client.1.smithi117.stdout:3/190: chown d1/d1b/d1e/l1a 24226 1 2024-02-16T19:28:56.627 INFO:tasks.workunit.client.1.smithi117.stdout:6/186: fdatasync d5/d1d/d28/f40 0 2024-02-16T19:28:56.629 INFO:tasks.workunit.client.0.smithi022.stdout:0/283: creat d10/d15/f5e x:0 0 0 2024-02-16T19:28:56.633 INFO:tasks.workunit.client.0.smithi022.stdout:3/200: unlink d7/l1f 0 2024-02-16T19:28:56.634 INFO:tasks.workunit.client.1.smithi117.stdout:4/173: sync 2024-02-16T19:28:56.634 INFO:tasks.workunit.client.1.smithi117.stdout:9/155: sync 2024-02-16T19:28:56.634 INFO:tasks.workunit.client.1.smithi117.stdout:7/150: sync 2024-02-16T19:28:56.634 INFO:tasks.workunit.client.1.smithi117.stdout:2/143: sync 2024-02-16T19:28:56.635 INFO:tasks.workunit.client.1.smithi117.stdout:7/151: write d4/d1c/f32 [678799,44198] 0 2024-02-16T19:28:56.636 INFO:tasks.workunit.client.0.smithi022.stdout:7/190: mkdir d9/da/d3b 0 2024-02-16T19:28:56.637 INFO:tasks.workunit.client.0.smithi022.stdout:0/284: dwrite d10/d15/d23/d38/d5a/f5d [0,4194304] 0 2024-02-16T19:28:56.643 INFO:tasks.workunit.client.1.smithi117.stdout:6/187: creat d5/d1d/d28/d32/f43 x:0 0 0 2024-02-16T19:28:56.644 INFO:tasks.workunit.client.1.smithi117.stdout:6/188: dread - d5/d1d/d28/d32/d37/f41 zero size 2024-02-16T19:28:56.645 INFO:tasks.workunit.client.1.smithi117.stdout:9/156: mkdir dd/d2d 0 2024-02-16T19:28:56.645 INFO:tasks.workunit.client.1.smithi117.stdout:9/157: chown l8 1 1 2024-02-16T19:28:56.646 INFO:tasks.workunit.client.0.smithi022.stdout:6/207: getdents d1 0 2024-02-16T19:28:56.646 INFO:tasks.workunit.client.1.smithi117.stdout:9/158: write f5 [4114871,13526] 0 2024-02-16T19:28:56.647 INFO:tasks.workunit.client.1.smithi117.stdout:2/144: creat d0/d3/f2a x:0 0 0 2024-02-16T19:28:56.647 INFO:tasks.workunit.client.1.smithi117.stdout:9/159: chown dd/f1e 133279 1 2024-02-16T19:28:56.647 INFO:tasks.workunit.client.0.smithi022.stdout:4/181: mknod d4/d6/dd/df/d10/c3c 0 2024-02-16T19:28:56.648 INFO:tasks.workunit.client.1.smithi117.stdout:1/108: fsync d2/d4/f9 0 2024-02-16T19:28:56.654 INFO:tasks.workunit.client.1.smithi117.stdout:0/229: write d0/d9/d22/d26/f48 [4217613,7859] 0 2024-02-16T19:28:56.657 INFO:tasks.workunit.client.1.smithi117.stdout:6/189: mkdir d5/d18/d44 0 2024-02-16T19:28:56.664 INFO:tasks.workunit.client.0.smithi022.stdout:2/139: dwrite d0/f17 [0,4194304] 0 2024-02-16T19:28:56.664 INFO:tasks.workunit.client.1.smithi117.stdout:9/160: rename dd/d23/d2a to dd/d23/d2e 0 2024-02-16T19:28:56.664 INFO:tasks.workunit.client.1.smithi117.stdout:9/161: chown fb 623 1 2024-02-16T19:28:56.665 INFO:tasks.workunit.client.1.smithi117.stdout:9/162: read fa [4647924,97523] 0 2024-02-16T19:28:56.666 INFO:tasks.workunit.client.1.smithi117.stdout:9/163: truncate fb 4633856 0 2024-02-16T19:28:56.666 INFO:tasks.workunit.client.1.smithi117.stdout:9/164: fsync dd/d23/f25 0 2024-02-16T19:28:56.668 INFO:tasks.workunit.client.0.smithi022.stdout:1/324: creat d6/f5b x:0 0 0 2024-02-16T19:28:56.669 INFO:tasks.workunit.client.1.smithi117.stdout:1/109: rmdir d2/d18 39 2024-02-16T19:28:56.670 INFO:tasks.workunit.client.0.smithi022.stdout:6/208: symlink d1/d8/l4f 0 2024-02-16T19:28:56.671 INFO:tasks.workunit.client.1.smithi117.stdout:9/165: dwrite dd/f1e [0,4194304] 0 2024-02-16T19:28:56.673 INFO:tasks.workunit.client.0.smithi022.stdout:7/191: write d9/da/d1f/f22 [644276,19495] 0 2024-02-16T19:28:56.674 INFO:tasks.workunit.client.1.smithi117.stdout:7/152: mkdir d4/d13/d38/d3f 0 2024-02-16T19:28:56.675 INFO:tasks.workunit.client.0.smithi022.stdout:3/201: mkdir d7/d12/d18/dd/d24/d58 0 2024-02-16T19:28:56.677 INFO:tasks.workunit.client.0.smithi022.stdout:8/241: write d3/d6/f7 [3106464,18298] 0 2024-02-16T19:28:56.678 INFO:tasks.workunit.client.1.smithi117.stdout:0/230: mkdir d0/d9/d22/d27/d4e 0 2024-02-16T19:28:56.679 INFO:tasks.workunit.client.1.smithi117.stdout:0/231: chown d0/d9/d14/f1b 0 1 2024-02-16T19:28:56.680 INFO:tasks.workunit.client.0.smithi022.stdout:2/140: readlink d0/lc 0 2024-02-16T19:28:56.681 INFO:tasks.workunit.client.1.smithi117.stdout:2/145: mknod d0/d29/c2b 0 2024-02-16T19:28:56.681 INFO:tasks.workunit.client.1.smithi117.stdout:2/146: chown d0/fd 425383540 1 2024-02-16T19:28:56.684 INFO:tasks.workunit.client.0.smithi022.stdout:1/325: unlink d6/d8/d22/d47/d51/l53 0 2024-02-16T19:28:56.687 INFO:tasks.workunit.client.0.smithi022.stdout:4/182: link d4/d6/dd/d22/d30/f26 d4/d38/f3d 0 2024-02-16T19:28:56.688 INFO:tasks.workunit.client.0.smithi022.stdout:4/183: write d4/d6/d14/f1d [2596132,125159] 0 2024-02-16T19:28:56.689 INFO:tasks.workunit.client.0.smithi022.stdout:4/184: write d4/d6/dd/f36 [1541395,79647] 0 2024-02-16T19:28:56.690 INFO:tasks.workunit.client.0.smithi022.stdout:7/192: unlink c2 0 2024-02-16T19:28:56.693 INFO:tasks.workunit.client.0.smithi022.stdout:3/202: mkdir d7/d12/d59 0 2024-02-16T19:28:56.694 INFO:tasks.workunit.client.0.smithi022.stdout:7/193: dread d9/da/d1f/f31 [0,4194304] 0 2024-02-16T19:28:56.695 INFO:tasks.workunit.client.1.smithi117.stdout:1/110: chown d2/d18/l1a 200 1 2024-02-16T19:28:56.695 INFO:tasks.workunit.client.1.smithi117.stdout:4/174: link d0/d6/d1a/d28/l32 d0/da/d21/l34 0 2024-02-16T19:28:56.696 INFO:tasks.workunit.client.0.smithi022.stdout:8/242: symlink d3/d6/l57 0 2024-02-16T19:28:56.696 INFO:tasks.workunit.client.0.smithi022.stdout:8/243: fsync d3/d25/d33/f41 0 2024-02-16T19:28:56.696 INFO:tasks.workunit.client.0.smithi022.stdout:0/285: link d10/d15/c1a d10/d15/d23/d38/c5f 0 2024-02-16T19:28:56.696 INFO:tasks.workunit.client.1.smithi117.stdout:9/166: symlink dd/d23/d2e/l2f 0 2024-02-16T19:28:56.697 INFO:tasks.workunit.client.0.smithi022.stdout:0/286: read d10/d22/f2f [2948439,122065] 0 2024-02-16T19:28:56.698 INFO:tasks.workunit.client.1.smithi117.stdout:6/190: dwrite d5/ff [0,4194304] 0 2024-02-16T19:28:56.698 INFO:tasks.workunit.client.0.smithi022.stdout:1/326: chown d6/d8/cb 3066497 1 2024-02-16T19:28:56.699 INFO:tasks.workunit.client.0.smithi022.stdout:0/287: fsync d10/d15/d23/f47 0 2024-02-16T19:28:56.699 INFO:tasks.workunit.client.1.smithi117.stdout:3/191: getdents d1/d1b/d1e 0 2024-02-16T19:28:56.699 INFO:tasks.workunit.client.1.smithi117.stdout:7/153: write d4/f1e [151989,51370] 0 2024-02-16T19:28:56.700 INFO:tasks.workunit.client.1.smithi117.stdout:9/167: write dd/d23/f25 [800897,39597] 0 2024-02-16T19:28:56.701 INFO:tasks.workunit.client.1.smithi117.stdout:0/232: write d0/d9/d22/d27/f42 [421788,24133] 0 2024-02-16T19:28:56.702 INFO:tasks.workunit.client.0.smithi022.stdout:4/185: rename d4/d6/c17 to d4/d6/dd/c3e 0 2024-02-16T19:28:56.706 INFO:tasks.workunit.client.1.smithi117.stdout:2/147: mknod d0/d3/c2c 0 2024-02-16T19:28:56.707 INFO:tasks.workunit.client.1.smithi117.stdout:0/233: write d0/f5 [2143696,129065] 0 2024-02-16T19:28:56.708 INFO:tasks.workunit.client.1.smithi117.stdout:1/111: dwrite d2/d1d/d1f/f23 [0,4194304] 0 2024-02-16T19:28:56.710 INFO:tasks.workunit.client.0.smithi022.stdout:4/186: write d4/d6/dd/df/f29 [1111,16142] 0 2024-02-16T19:28:56.710 INFO:tasks.workunit.client.1.smithi117.stdout:0/234: write d0/d9/d22/d27/d31/f38 [4712710,70136] 0 2024-02-16T19:28:56.711 INFO:tasks.workunit.client.1.smithi117.stdout:1/112: write d2/d4/f17 [3652968,107927] 0 2024-02-16T19:28:56.712 INFO:tasks.workunit.client.1.smithi117.stdout:7/154: dread d4/f2b [0,4194304] 0 2024-02-16T19:28:56.715 INFO:tasks.workunit.client.0.smithi022.stdout:4/187: dread d4/d6/dd/f2a [0,4194304] 0 2024-02-16T19:28:56.716 INFO:tasks.workunit.client.1.smithi117.stdout:6/191: symlink d5/db/l45 0 2024-02-16T19:28:56.717 INFO:tasks.workunit.client.1.smithi117.stdout:0/235: dread d0/d9/d14/f1b [0,4194304] 0 2024-02-16T19:28:56.718 INFO:tasks.workunit.client.0.smithi022.stdout:8/244: rmdir d3/d12/d2b/d46 39 2024-02-16T19:28:56.719 INFO:tasks.workunit.client.0.smithi022.stdout:8/245: readlink d3/d6/l30 0 2024-02-16T19:28:56.722 INFO:tasks.workunit.client.0.smithi022.stdout:2/141: symlink d0/l2c 0 2024-02-16T19:28:56.726 INFO:tasks.workunit.client.0.smithi022.stdout:0/288: mkdir d10/d11/d27/d3f/d58/d60 0 2024-02-16T19:28:56.727 INFO:tasks.workunit.client.0.smithi022.stdout:0/289: dread - d10/d15/d23/d38/f41 zero size 2024-02-16T19:28:56.728 INFO:tasks.workunit.client.0.smithi022.stdout:0/290: write d10/d11/d50/f3e [4811573,48352] 0 2024-02-16T19:28:56.732 INFO:tasks.workunit.client.1.smithi117.stdout:4/175: mknod d0/d6/d1a/d2d/c35 0 2024-02-16T19:28:56.735 INFO:tasks.workunit.client.0.smithi022.stdout:7/194: dwrite d9/f25 [0,4194304] 0 2024-02-16T19:28:56.737 INFO:tasks.workunit.client.0.smithi022.stdout:7/195: write d9/da/f13 [4385644,33702] 0 2024-02-16T19:28:56.739 INFO:tasks.workunit.client.0.smithi022.stdout:4/188: chown d4/f1f 62 1 2024-02-16T19:28:56.741 INFO:tasks.workunit.client.1.smithi117.stdout:7/155: symlink d4/d13/d38/l40 0 2024-02-16T19:28:56.744 INFO:tasks.workunit.client.0.smithi022.stdout:1/327: symlink d6/l5c 0 2024-02-16T19:28:56.748 INFO:tasks.workunit.client.1.smithi117.stdout:9/168: link dd/f20 dd/d23/f30 0 2024-02-16T19:28:56.750 INFO:tasks.workunit.client.1.smithi117.stdout:4/176: creat d0/d6/d1a/d2d/f36 x:0 0 0 2024-02-16T19:28:56.750 INFO:tasks.workunit.client.0.smithi022.stdout:7/196: mknod d9/da/d2c/d2f/c3c 0 2024-02-16T19:28:56.750 INFO:tasks.workunit.client.1.smithi117.stdout:1/113: symlink d2/d18/d20/l24 0 2024-02-16T19:28:56.751 INFO:tasks.workunit.client.1.smithi117.stdout:0/236: dwrite d0/d9/d22/d26/f2e [0,4194304] 0 2024-02-16T19:28:56.757 INFO:tasks.workunit.client.0.smithi022.stdout:0/291: rename d10/f19 to d10/d15/f61 0 2024-02-16T19:28:56.763 INFO:tasks.workunit.client.0.smithi022.stdout:0/292: dread d10/d15/d23/d38/d5a/f5d [0,4194304] 0 2024-02-16T19:28:56.763 INFO:tasks.workunit.client.1.smithi117.stdout:1/114: creat d2/d1d/f25 x:0 0 0 2024-02-16T19:28:56.766 INFO:tasks.workunit.client.0.smithi022.stdout:2/142: creat d0/f2d x:0 0 0 2024-02-16T19:28:56.767 INFO:tasks.workunit.client.1.smithi117.stdout:4/177: mkdir d0/d6/d1a/d28/d2f/d37 0 2024-02-16T19:28:56.768 INFO:tasks.workunit.client.1.smithi117.stdout:4/178: fsync d0/da/f2c 0 2024-02-16T19:28:56.769 INFO:tasks.workunit.client.0.smithi022.stdout:1/328: mkdir d6/d8/d22/d16/d17/d5d 0 2024-02-16T19:28:56.774 INFO:tasks.workunit.client.1.smithi117.stdout:9/169: mkdir dd/d23/d2e/d2c/d31 0 2024-02-16T19:28:56.774 INFO:tasks.workunit.client.1.smithi117.stdout:9/170: readlink dd/l1b 0 2024-02-16T19:28:56.775 INFO:tasks.workunit.client.0.smithi022.stdout:4/189: dwrite d4/d6/dd/f1e [0,4194304] 0 2024-02-16T19:28:56.775 INFO:tasks.workunit.client.1.smithi117.stdout:9/171: stat dd/d23/f2b 0 2024-02-16T19:28:56.781 INFO:tasks.workunit.client.0.smithi022.stdout:7/197: dwrite d9/da/d1f/f31 [0,4194304] 0 2024-02-16T19:28:56.785 INFO:tasks.workunit.client.0.smithi022.stdout:0/293: dread d10/d15/d23/f3c [0,4194304] 0 2024-02-16T19:28:56.787 INFO:tasks.workunit.client.0.smithi022.stdout:0/294: chown d10/d15/d23/d38/d5a/f5d 29846 1 2024-02-16T19:28:56.789 INFO:tasks.workunit.client.1.smithi117.stdout:4/179: rename d0/d6/d1a/d28/l2a to d0/d6/d1a/d28/l38 0 2024-02-16T19:28:56.790 INFO:tasks.workunit.client.1.smithi117.stdout:4/180: chown d0/d6/d1a/d28/d2f 575145660 1 2024-02-16T19:28:56.790 INFO:tasks.workunit.client.0.smithi022.stdout:1/329: symlink d6/d8/d22/d16/d17/d26/l5e 0 2024-02-16T19:28:56.792 INFO:tasks.workunit.client.1.smithi117.stdout:7/156: rmdir d4/d13/d38/d3f 0 2024-02-16T19:28:56.793 INFO:tasks.workunit.client.1.smithi117.stdout:6/192: getdents d5/d1d 0 2024-02-16T19:28:56.795 INFO:tasks.workunit.client.1.smithi117.stdout:9/172: creat dd/d23/d2e/f32 x:0 0 0 2024-02-16T19:28:56.796 INFO:tasks.workunit.client.1.smithi117.stdout:0/237: write d0/fe [484631,35993] 0 2024-02-16T19:28:56.798 INFO:tasks.workunit.client.0.smithi022.stdout:2/143: dwrite d0/d2/f1c [0,4194304] 0 2024-02-16T19:28:56.801 INFO:tasks.workunit.client.0.smithi022.stdout:0/295: rename d10/d15/d23/d29 to d10/d11/d27/d3f/d58/d60/d62 0 2024-02-16T19:28:56.804 INFO:tasks.workunit.client.1.smithi117.stdout:7/157: unlink d4/d13/f3e 0 2024-02-16T19:28:56.807 INFO:tasks.workunit.client.1.smithi117.stdout:6/193: rename d5/db/f16 to d5/d18/f46 0 2024-02-16T19:28:56.807 INFO:tasks.workunit.client.0.smithi022.stdout:7/198: fsync d9/da/d2c/f2d 0 2024-02-16T19:28:56.813 INFO:tasks.workunit.client.1.smithi117.stdout:9/173: dwrite dd/d23/f22 [0,4194304] 0 2024-02-16T19:28:56.813 INFO:tasks.workunit.client.0.smithi022.stdout:0/296: symlink d10/d11/d27/d3f/d58/d60/d62/l63 0 2024-02-16T19:28:56.814 INFO:tasks.workunit.client.0.smithi022.stdout:0/297: truncate d10/d15/d23/f47 671239 0 2024-02-16T19:28:56.816 INFO:tasks.workunit.client.0.smithi022.stdout:1/330: unlink d6/cc 0 2024-02-16T19:28:56.817 INFO:tasks.workunit.client.1.smithi117.stdout:4/181: link d0/f25 d0/d6/d1a/d28/f39 0 2024-02-16T19:28:56.817 INFO:tasks.workunit.client.1.smithi117.stdout:4/182: truncate d0/f25 4513479 0 2024-02-16T19:28:56.818 INFO:tasks.workunit.client.1.smithi117.stdout:7/158: mknod d4/d1c/c41 0 2024-02-16T19:28:56.819 INFO:tasks.workunit.client.0.smithi022.stdout:4/190: dread d4/d6/dd/df/d10/f19 [0,4194304] 0 2024-02-16T19:28:56.820 INFO:tasks.workunit.client.0.smithi022.stdout:4/191: stat d4/d6/c18 0 2024-02-16T19:28:56.820 INFO:tasks.workunit.client.0.smithi022.stdout:7/199: read d9/f14 [2791010,96055] 0 2024-02-16T19:28:56.821 INFO:tasks.workunit.client.1.smithi117.stdout:9/174: dwrite dd/d23/f2b [0,4194304] 0 2024-02-16T19:28:56.821 INFO:tasks.workunit.client.1.smithi117.stdout:6/194: rename d5/f3b to d5/d1d/d28/d32/d37/f47 0 2024-02-16T19:28:56.821 INFO:tasks.workunit.client.1.smithi117.stdout:1/115: link d2/d1d/c22 d2/d1d/d1f/c26 0 2024-02-16T19:28:56.822 INFO:tasks.workunit.client.1.smithi117.stdout:6/195: write d5/d1d/d28/f40 [39287,41579] 0 2024-02-16T19:28:56.823 INFO:tasks.workunit.client.0.smithi022.stdout:0/298: unlink d10/d11/d50/c37 0 2024-02-16T19:28:56.823 INFO:tasks.workunit.client.0.smithi022.stdout:5/241: sync 2024-02-16T19:28:56.824 INFO:tasks.workunit.client.0.smithi022.stdout:0/299: write d10/d22/f53 [2110816,98713] 0 2024-02-16T19:28:56.826 INFO:tasks.workunit.client.1.smithi117.stdout:0/238: mknod d0/d9/d22/d27/d4e/c4f 0 2024-02-16T19:28:56.827 INFO:tasks.workunit.client.1.smithi117.stdout:0/239: chown d0/d9/d22/d26/l3c 12815 1 2024-02-16T19:28:56.828 INFO:tasks.workunit.client.1.smithi117.stdout:0/240: stat d0/l8 0 2024-02-16T19:28:56.828 INFO:tasks.workunit.client.1.smithi117.stdout:0/241: readlink d0/l2b 0 2024-02-16T19:28:56.828 INFO:tasks.workunit.client.0.smithi022.stdout:4/192: symlink d4/l3f 0 2024-02-16T19:28:56.829 INFO:tasks.workunit.client.1.smithi117.stdout:0/242: write d0/d9/d2c/d37/f3d [1022326,68425] 0 2024-02-16T19:28:56.830 INFO:tasks.workunit.client.0.smithi022.stdout:7/200: creat d9/da/d2c/d2f/f3d x:0 0 0 2024-02-16T19:28:56.834 INFO:tasks.workunit.client.0.smithi022.stdout:2/144: rename d0/d2/d18/d19/d25/d27 to d0/d2/d2e 0 2024-02-16T19:28:56.839 INFO:tasks.workunit.client.1.smithi117.stdout:9/175: symlink dd/d23/l33 0 2024-02-16T19:28:56.840 INFO:tasks.workunit.client.1.smithi117.stdout:7/159: dread d4/f10 [0,4194304] 0 2024-02-16T19:28:56.840 INFO:tasks.workunit.client.1.smithi117.stdout:9/176: readlink dd/d23/l33 0 2024-02-16T19:28:56.840 INFO:tasks.workunit.client.0.smithi022.stdout:1/331: mknod d6/d8/c5f 0 2024-02-16T19:28:56.841 INFO:tasks.workunit.client.1.smithi117.stdout:1/116: fsync d2/d4/fa 0 2024-02-16T19:28:56.843 INFO:tasks.workunit.client.1.smithi117.stdout:1/117: write d2/d4/f15 [391585,32464] 0 2024-02-16T19:28:56.844 INFO:tasks.workunit.client.1.smithi117.stdout:7/160: dread d4/d1c/f3c [0,4194304] 0 2024-02-16T19:28:56.844 INFO:tasks.workunit.client.0.smithi022.stdout:1/332: dread d6/f9 [0,4194304] 0 2024-02-16T19:28:56.845 INFO:tasks.workunit.client.0.smithi022.stdout:4/193: rename d4/f1f to d4/f40 0 2024-02-16T19:28:56.847 INFO:tasks.workunit.client.1.smithi117.stdout:6/196: chown d5/c17 16945985 1 2024-02-16T19:28:56.847 INFO:tasks.workunit.client.1.smithi117.stdout:6/197: fsync d5/f1a 0 2024-02-16T19:28:56.848 INFO:tasks.workunit.client.1.smithi117.stdout:6/198: truncate d5/d1d/f2c 176213 0 2024-02-16T19:28:56.848 INFO:tasks.workunit.client.1.smithi117.stdout:6/199: write d5/ff [2085756,897] 0 2024-02-16T19:28:56.852 INFO:tasks.workunit.client.0.smithi022.stdout:2/145: creat d0/d2/d18/d19/f2f x:0 0 0 2024-02-16T19:28:56.853 INFO:tasks.workunit.client.1.smithi117.stdout:6/200: dwrite d5/db/f1f [0,4194304] 0 2024-02-16T19:28:56.853 INFO:tasks.workunit.client.0.smithi022.stdout:2/146: dread - d0/d2/d18/f22 zero size 2024-02-16T19:28:56.854 INFO:tasks.workunit.client.1.smithi117.stdout:4/183: symlink d0/d6/d1a/d28/d2f/d37/l3a 0 2024-02-16T19:28:56.855 INFO:tasks.workunit.client.1.smithi117.stdout:4/184: write d0/da/f2c [1039769,103897] 0 2024-02-16T19:28:56.857 INFO:tasks.workunit.client.0.smithi022.stdout:7/201: symlink d9/l3e 0 2024-02-16T19:28:56.860 INFO:tasks.workunit.client.0.smithi022.stdout:0/300: rename l6 to d10/d11/d27/l64 0 2024-02-16T19:28:56.863 INFO:tasks.workunit.client.0.smithi022.stdout:4/194: creat d4/d6/d3a/f41 x:0 0 0 2024-02-16T19:28:56.868 INFO:tasks.workunit.client.1.smithi117.stdout:7/161: chown d4/d13/l36 38530914 1 2024-02-16T19:28:56.869 INFO:tasks.workunit.client.1.smithi117.stdout:0/243: dwrite d0/d9/f12 [4194304,4194304] 0 2024-02-16T19:28:56.870 INFO:tasks.workunit.client.0.smithi022.stdout:5/242: rename dd/d1f/d48/d21/l39 to dd/d1f/d48/d21/d2f/d40/l4d 0 2024-02-16T19:28:56.873 INFO:tasks.workunit.client.0.smithi022.stdout:0/301: mkdir d10/d11/d50/d65 0 2024-02-16T19:28:56.875 INFO:tasks.workunit.client.1.smithi117.stdout:8/172: sync 2024-02-16T19:28:56.876 INFO:tasks.workunit.client.1.smithi117.stdout:1/118: write d2/d4/f9 [404268,410] 0 2024-02-16T19:28:56.876 INFO:tasks.workunit.client.0.smithi022.stdout:4/195: creat d4/d6/dd/df/d10/f42 x:0 0 0 2024-02-16T19:28:56.876 INFO:tasks.workunit.client.1.smithi117.stdout:8/173: chown d4/d7/d37 8371453 1 2024-02-16T19:28:56.877 INFO:tasks.workunit.client.1.smithi117.stdout:1/119: readlink d2/d18/l1a 0 2024-02-16T19:28:56.879 INFO:tasks.workunit.client.0.smithi022.stdout:7/202: mknod d9/d1b/c3f 0 2024-02-16T19:28:56.881 INFO:tasks.workunit.client.1.smithi117.stdout:4/185: creat d0/d6/f3b x:0 0 0 2024-02-16T19:28:56.882 INFO:tasks.workunit.client.0.smithi022.stdout:0/302: mkdir d10/d15/d23/d38/d5a/d66 0 2024-02-16T19:28:56.884 INFO:tasks.workunit.client.1.smithi117.stdout:7/162: symlink d4/d1c/d24/l42 0 2024-02-16T19:28:56.884 INFO:tasks.workunit.client.1.smithi117.stdout:7/163: chown d4/d13/c20 0 1 2024-02-16T19:28:56.887 INFO:tasks.workunit.client.0.smithi022.stdout:2/147: rmdir d0/d2/d18/d19/d25 0 2024-02-16T19:28:56.887 INFO:tasks.workunit.client.0.smithi022.stdout:2/148: dread - d0/d2/d18/f22 zero size 2024-02-16T19:28:56.890 INFO:tasks.workunit.client.0.smithi022.stdout:7/203: mkdir d9/da/d1f/d40 0 2024-02-16T19:28:56.891 INFO:tasks.workunit.client.1.smithi117.stdout:8/174: readlink d4/d7/l14 0 2024-02-16T19:28:56.893 INFO:tasks.workunit.client.0.smithi022.stdout:5/243: unlink dd/d1f/d48/d21/c38 0 2024-02-16T19:28:56.896 INFO:tasks.workunit.client.1.smithi117.stdout:6/201: truncate d5/d18/f46 319826 0 2024-02-16T19:28:56.897 INFO:tasks.workunit.client.0.smithi022.stdout:4/196: mknod d4/d38/c43 0 2024-02-16T19:28:56.900 INFO:tasks.workunit.client.0.smithi022.stdout:2/149: creat d0/d2/d18/d19/f30 x:0 0 0 2024-02-16T19:28:56.900 INFO:tasks.workunit.client.0.smithi022.stdout:0/303: dwrite d10/d22/f45 [4194304,4194304] 0 2024-02-16T19:28:56.901 INFO:tasks.workunit.client.1.smithi117.stdout:7/164: fdatasync d4/f17 0 2024-02-16T19:28:56.902 INFO:tasks.workunit.client.1.smithi117.stdout:0/244: creat d0/d9/d14/d23/d2f/f50 x:0 0 0 2024-02-16T19:28:56.903 INFO:tasks.workunit.client.0.smithi022.stdout:5/244: mkdir dd/d1f/d48/d21/d2f/d40/d4e 0 2024-02-16T19:28:56.904 INFO:tasks.workunit.client.1.smithi117.stdout:8/175: truncate d4/d12/f1a 1413162 0 2024-02-16T19:28:56.905 INFO:tasks.workunit.client.0.smithi022.stdout:4/197: creat d4/d6/dd/d22/d30/f44 x:0 0 0 2024-02-16T19:28:56.906 INFO:tasks.workunit.client.0.smithi022.stdout:4/198: read - d4/d6/dd/df/d10/f42 zero size 2024-02-16T19:28:56.908 INFO:tasks.workunit.client.0.smithi022.stdout:7/204: creat d9/da/d3b/f41 x:0 0 0 2024-02-16T19:28:56.913 INFO:tasks.workunit.client.0.smithi022.stdout:0/304: write d10/d15/d23/d38/d5a/f5d [1283159,113573] 0 2024-02-16T19:28:56.915 INFO:tasks.workunit.client.0.smithi022.stdout:0/305: stat d10/d11/d27/d3f/d58/d60/d62/f4f 0 2024-02-16T19:28:56.915 INFO:tasks.workunit.client.0.smithi022.stdout:0/306: fdatasync d10/d15/f39 0 2024-02-16T19:28:56.916 INFO:tasks.workunit.client.0.smithi022.stdout:0/307: chown d10/d11/d27/d3f 14265 1 2024-02-16T19:28:56.920 INFO:tasks.workunit.client.1.smithi117.stdout:0/245: unlink d0/d9/d2c/d37/f3d 0 2024-02-16T19:28:56.929 INFO:tasks.workunit.client.1.smithi117.stdout:6/202: link d5/f1a d5/d1d/d28/d32/f48 0 2024-02-16T19:28:56.929 INFO:tasks.workunit.client.1.smithi117.stdout:6/203: readlink d5/l31 0 2024-02-16T19:28:56.930 INFO:tasks.workunit.client.1.smithi117.stdout:4/186: write d0/d6/d1a/f1d [118237,27017] 0 2024-02-16T19:28:56.931 INFO:tasks.workunit.client.1.smithi117.stdout:7/165: unlink d4/c9 0 2024-02-16T19:28:56.934 INFO:tasks.workunit.client.0.smithi022.stdout:4/199: write d4/f23 [7399395,69760] 0 2024-02-16T19:28:56.938 INFO:tasks.workunit.client.0.smithi022.stdout:5/245: dwrite dd/d1f/d48/d21/f25 [0,4194304] 0 2024-02-16T19:28:56.939 INFO:tasks.workunit.client.1.smithi117.stdout:1/120: getdents d2/d18 0 2024-02-16T19:28:56.942 INFO:tasks.workunit.client.0.smithi022.stdout:2/150: getdents d0 0 2024-02-16T19:28:56.942 INFO:tasks.workunit.client.0.smithi022.stdout:2/151: fsync d0/f2d 0 2024-02-16T19:28:56.943 INFO:tasks.workunit.client.1.smithi117.stdout:7/166: dread d4/f28 [0,4194304] 0 2024-02-16T19:28:56.944 INFO:tasks.workunit.client.1.smithi117.stdout:6/204: mkdir d5/d1d/d49 0 2024-02-16T19:28:56.944 INFO:tasks.workunit.client.1.smithi117.stdout:1/121: dread d2/d4/f15 [0,4194304] 0 2024-02-16T19:28:56.944 INFO:tasks.workunit.client.0.smithi022.stdout:7/205: dwrite d9/da/d2c/d2f/f17 [0,4194304] 0 2024-02-16T19:28:56.945 INFO:tasks.workunit.client.1.smithi117.stdout:1/122: read d2/d4/f17 [1626438,111529] 0 2024-02-16T19:28:56.945 INFO:tasks.workunit.client.1.smithi117.stdout:1/123: chown d2/ff 9 1 2024-02-16T19:28:56.946 INFO:tasks.workunit.client.1.smithi117.stdout:1/124: chown d2 523 1 2024-02-16T19:28:56.947 INFO:tasks.workunit.client.1.smithi117.stdout:4/187: symlink d0/l3c 0 2024-02-16T19:28:56.949 INFO:tasks.workunit.client.0.smithi022.stdout:0/308: link d10/d15/c1a d10/d11/d27/d3f/d58/c67 0 2024-02-16T19:28:56.952 INFO:tasks.workunit.client.0.smithi022.stdout:5/246: truncate dd/f18 1496780 0 2024-02-16T19:28:56.953 INFO:tasks.workunit.client.0.smithi022.stdout:5/247: fsync dd/d1f/d48/d21/f3e 0 2024-02-16T19:28:56.953 INFO:tasks.workunit.client.0.smithi022.stdout:2/152: rename d0/c26 to d0/d2/d18/d19/c31 0 2024-02-16T19:28:56.954 INFO:tasks.workunit.client.0.smithi022.stdout:5/248: chown dd/d1f/d48/d21/d2f/d40/d49/d4b 12450 1 2024-02-16T19:28:56.956 INFO:tasks.workunit.client.0.smithi022.stdout:5/249: read f5 [415305,53473] 0 2024-02-16T19:28:56.958 INFO:tasks.workunit.client.0.smithi022.stdout:5/250: rename dd/d1f/d48 to dd/d1f/d48/d21/d2f/d40/d49/d4f 22 2024-02-16T19:28:56.964 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:56 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:28:56.965 INFO:tasks.workunit.client.1.smithi117.stdout:7/167: symlink d4/d13/d38/l43 0 2024-02-16T19:28:56.965 INFO:tasks.workunit.client.0.smithi022.stdout:4/200: link d4/d6/dd/df/d10/c3c d4/d6/c45 0 2024-02-16T19:28:56.966 INFO:tasks.workunit.client.1.smithi117.stdout:7/168: write d4/f1e [104464,58549] 0 2024-02-16T19:28:56.967 INFO:tasks.workunit.client.1.smithi117.stdout:6/205: creat d5/db/f4a x:0 0 0 2024-02-16T19:28:56.968 INFO:tasks.workunit.client.1.smithi117.stdout:6/206: chown d5/d1d/f2c 105 1 2024-02-16T19:28:56.969 INFO:tasks.workunit.client.0.smithi022.stdout:0/309: creat d10/d11/d50/d65/f68 x:0 0 0 2024-02-16T19:28:56.970 INFO:tasks.workunit.client.0.smithi022.stdout:7/206: dwrite d9/f14 [0,4194304] 0 2024-02-16T19:28:56.970 INFO:tasks.workunit.client.1.smithi117.stdout:6/207: dread d5/f7 [0,4194304] 0 2024-02-16T19:28:56.970 INFO:tasks.workunit.client.1.smithi117.stdout:1/125: rename d2/cc to d2/d1d/c27 0 2024-02-16T19:28:56.971 INFO:tasks.workunit.client.0.smithi022.stdout:7/207: fsync d9/da/d1f/f18 0 2024-02-16T19:28:56.971 INFO:tasks.workunit.client.0.smithi022.stdout:0/310: stat d10/d11/d27/d3f/d58/d60/d62/f4f 0 2024-02-16T19:28:56.972 INFO:tasks.workunit.client.0.smithi022.stdout:7/208: write f4 [4158066,46352] 0 2024-02-16T19:28:56.973 INFO:tasks.workunit.client.1.smithi117.stdout:5/281: sync 2024-02-16T19:28:56.973 INFO:tasks.workunit.client.1.smithi117.stdout:2/148: sync 2024-02-16T19:28:56.973 INFO:tasks.workunit.client.1.smithi117.stdout:3/192: sync 2024-02-16T19:28:56.973 INFO:tasks.workunit.client.1.smithi117.stdout:1/126: dread d2/d4/f9 [0,4194304] 0 2024-02-16T19:28:56.974 INFO:tasks.workunit.client.1.smithi117.stdout:2/149: stat d0/f12 0 2024-02-16T19:28:56.974 INFO:tasks.workunit.client.1.smithi117.stdout:0/246: rmdir d0/d9/d14/d34 0 2024-02-16T19:28:56.974 INFO:tasks.workunit.client.1.smithi117.stdout:5/282: readlink d2/d3/l28 0 2024-02-16T19:28:56.976 INFO:tasks.workunit.client.1.smithi117.stdout:5/283: write d2/d3/f47 [911453,106994] 0 2024-02-16T19:28:56.977 INFO:tasks.workunit.client.0.smithi022.stdout:4/201: symlink d4/d6/d14/l46 0 2024-02-16T19:28:56.977 INFO:tasks.workunit.client.0.smithi022.stdout:2/153: dread - d0/f2d zero size 2024-02-16T19:28:56.978 INFO:tasks.workunit.client.1.smithi117.stdout:8/176: getdents d4/d1c 0 2024-02-16T19:28:56.978 INFO:tasks.workunit.client.0.smithi022.stdout:5/251: mknod dd/d1f/d48/d21/d2f/d46/c50 0 2024-02-16T19:28:56.983 INFO:tasks.workunit.client.0.smithi022.stdout:7/209: dwrite f5 [0,4194304] 0 2024-02-16T19:28:56.984 INFO:tasks.workunit.client.0.smithi022.stdout:0/311: dwrite d10/d11/d50/f3e [4194304,4194304] 0 2024-02-16T19:28:56.987 INFO:tasks.workunit.client.0.smithi022.stdout:4/202: dread d4/d6/dd/df/d10/f19 [0,4194304] 0 2024-02-16T19:28:56.987 INFO:tasks.workunit.client.1.smithi117.stdout:4/188: rename d0/d6/d1a to d0/da/d21/d3d 0 2024-02-16T19:28:56.989 INFO:tasks.workunit.client.0.smithi022.stdout:4/203: write d4/d6/dd/df/d10/f35 [28257,110373] 0 2024-02-16T19:28:56.991 INFO:tasks.workunit.client.0.smithi022.stdout:2/154: mkdir d0/d32 0 2024-02-16T19:28:56.993 INFO:tasks.workunit.client.0.smithi022.stdout:4/204: write d4/d6/dd/d31/f34 [637159,26226] 0 2024-02-16T19:28:56.996 INFO:tasks.workunit.client.1.smithi117.stdout:4/189: dwrite d0/d6/f23 [0,4194304] 0 2024-02-16T19:28:56.998 INFO:tasks.workunit.client.1.smithi117.stdout:3/193: creat d1/d1b/d1e/f42 x:0 0 0 2024-02-16T19:28:56.998 INFO:tasks.workunit.client.0.smithi022.stdout:7/210: dread d9/da/f13 [0,4194304] 0 2024-02-16T19:28:56.999 INFO:tasks.workunit.client.0.smithi022.stdout:4/205: dread d4/d6/dd/df/d10/f35 [0,4194304] 0 2024-02-16T19:28:56.999 INFO:tasks.workunit.client.0.smithi022.stdout:2/155: stat d0/d2/d18/d1f 0 2024-02-16T19:28:57.000 INFO:tasks.workunit.client.1.smithi117.stdout:2/150: creat d0/d29/f2d x:0 0 0 2024-02-16T19:28:57.000 INFO:tasks.workunit.client.0.smithi022.stdout:2/156: write d0/d2/f14 [565099,49223] 0 2024-02-16T19:28:57.002 INFO:tasks.workunit.client.0.smithi022.stdout:2/157: read - d0/d2/d2e/f2a zero size 2024-02-16T19:28:57.003 INFO:tasks.workunit.client.0.smithi022.stdout:2/158: read d0/d2/d18/f1a [5751004,83731] 0 2024-02-16T19:28:57.004 INFO:tasks.workunit.client.0.smithi022.stdout:2/159: chown d0/d2/d18/d19/c1d 671 1 2024-02-16T19:28:57.010 INFO:tasks.workunit.client.1.smithi117.stdout:5/284: symlink d2/d3/d21/d2f/d30/l58 0 2024-02-16T19:28:57.012 INFO:tasks.workunit.client.0.smithi022.stdout:4/206: fdatasync d4/d6/dd/f2a 0 2024-02-16T19:28:57.013 INFO:tasks.workunit.client.1.smithi117.stdout:7/169: mknod d4/d1c/d24/d2f/c44 0 2024-02-16T19:28:57.016 INFO:tasks.workunit.client.0.smithi022.stdout:4/207: dread d4/d6/f16 [0,4194304] 0 2024-02-16T19:28:57.016 INFO:tasks.workunit.client.0.smithi022.stdout:2/160: rename d0/d2/d18/d1f/c23 to d0/d2/d18/c33 0 2024-02-16T19:28:57.020 INFO:tasks.workunit.client.0.smithi022.stdout:7/211: symlink d9/da/l42 0 2024-02-16T19:28:57.021 INFO:tasks.workunit.client.1.smithi117.stdout:6/208: stat d5/d18/d20/d2d/l33 0 2024-02-16T19:28:57.021 INFO:tasks.workunit.client.0.smithi022.stdout:7/212: write d9/da/d3b/f41 [423100,44666] 0 2024-02-16T19:28:57.021 INFO:tasks.workunit.client.1.smithi117.stdout:6/209: dread - d5/d18/f1e zero size 2024-02-16T19:28:57.023 INFO:tasks.workunit.client.1.smithi117.stdout:1/127: symlink d2/d4/l28 0 2024-02-16T19:28:57.024 INFO:tasks.workunit.client.0.smithi022.stdout:0/312: truncate fb 3006456 0 2024-02-16T19:28:57.024 INFO:tasks.workunit.client.0.smithi022.stdout:0/313: stat d10/f17 0 2024-02-16T19:28:57.025 INFO:tasks.workunit.client.0.smithi022.stdout:4/208: stat d4/d6/c21 0 2024-02-16T19:28:57.026 INFO:tasks.workunit.client.1.smithi117.stdout:4/190: creat d0/da/d21/f3e x:0 0 0 2024-02-16T19:28:57.026 INFO:tasks.workunit.client.0.smithi022.stdout:4/209: read d4/d6/dd/df/f29 [32154,109383] 0 2024-02-16T19:28:57.027 INFO:tasks.workunit.client.1.smithi117.stdout:5/285: dread d2/d3/f12 [0,4194304] 0 2024-02-16T19:28:57.027 INFO:tasks.workunit.client.0.smithi022.stdout:2/161: write d0/f17 [2096355,127419] 0 2024-02-16T19:28:57.028 INFO:tasks.workunit.client.1.smithi117.stdout:3/194: creat d1/dd/f43 x:0 0 0 2024-02-16T19:28:57.028 INFO:tasks.workunit.client.1.smithi117.stdout:3/195: write d1/dd/f34 [1000025,7330] 0 2024-02-16T19:28:57.029 INFO:tasks.workunit.client.0.smithi022.stdout:2/162: read - d0/f2d zero size 2024-02-16T19:28:57.030 INFO:tasks.workunit.client.1.smithi117.stdout:2/151: creat d0/d29/f2e x:0 0 0 2024-02-16T19:28:57.031 INFO:tasks.workunit.client.0.smithi022.stdout:2/163: dread - d0/d2/d18/d19/f2f zero size 2024-02-16T19:28:57.031 INFO:tasks.workunit.client.1.smithi117.stdout:2/152: fdatasync d0/d13/f15 0 2024-02-16T19:28:57.032 INFO:tasks.workunit.client.1.smithi117.stdout:2/153: chown d0/d3/d10/f1d 8 1 2024-02-16T19:28:57.032 INFO:tasks.workunit.client.0.smithi022.stdout:0/314: dwrite d10/d15/d23/d38/d5a/f5d [0,4194304] 0 2024-02-16T19:28:57.033 INFO:tasks.workunit.client.1.smithi117.stdout:2/154: dread - d0/d29/f2d zero size 2024-02-16T19:28:57.033 INFO:tasks.workunit.client.1.smithi117.stdout:2/155: chown d0/f12 60835180 1 2024-02-16T19:28:57.034 INFO:tasks.workunit.client.1.smithi117.stdout:7/170: symlink d4/d1c/d39/l45 0 2024-02-16T19:28:57.037 INFO:tasks.workunit.client.1.smithi117.stdout:6/210: unlink d5/db/f19 0 2024-02-16T19:28:57.037 INFO:tasks.workunit.client.1.smithi117.stdout:2/156: dwrite d0/d3/d10/f1d [0,4194304] 0 2024-02-16T19:28:57.038 INFO:tasks.workunit.client.0.smithi022.stdout:9/235: sync 2024-02-16T19:28:57.038 INFO:tasks.workunit.client.0.smithi022.stdout:6/209: sync 2024-02-16T19:28:57.038 INFO:tasks.workunit.client.0.smithi022.stdout:9/236: stat db/f1d 0 2024-02-16T19:28:57.038 INFO:tasks.workunit.client.0.smithi022.stdout:4/210: creat d4/f47 x:0 0 0 2024-02-16T19:28:57.039 INFO:tasks.workunit.client.1.smithi117.stdout:1/128: dread d2/d4/f15 [0,4194304] 0 2024-02-16T19:28:57.039 INFO:tasks.workunit.client.0.smithi022.stdout:7/213: mknod d9/c43 0 2024-02-16T19:28:57.040 INFO:tasks.workunit.client.1.smithi117.stdout:1/129: dread d2/d4/f11 [0,4194304] 0 2024-02-16T19:28:57.042 INFO:tasks.workunit.client.1.smithi117.stdout:4/191: rename d0/f24 to d0/da/d21/f3f 0 2024-02-16T19:28:57.043 INFO:tasks.workunit.client.0.smithi022.stdout:2/164: creat d0/d2/d18/f34 x:0 0 0 2024-02-16T19:28:57.046 INFO:tasks.workunit.client.1.smithi117.stdout:5/286: dread d2/d3/f6 [0,4194304] 0 2024-02-16T19:28:57.046 INFO:tasks.workunit.client.1.smithi117.stdout:9/177: sync 2024-02-16T19:28:57.047 INFO:tasks.workunit.client.1.smithi117.stdout:9/178: chown dd/d2d 32949 1 2024-02-16T19:28:57.051 INFO:tasks.workunit.client.1.smithi117.stdout:3/196: mknod d1/d1b/d1e/d37/c44 0 2024-02-16T19:28:57.052 INFO:tasks.workunit.client.1.smithi117.stdout:9/179: dread dd/d23/f25 [0,4194304] 0 2024-02-16T19:28:57.052 INFO:tasks.workunit.client.0.smithi022.stdout:0/315: dwrite d10/d15/d23/f3c [0,4194304] 0 2024-02-16T19:28:57.054 INFO:tasks.workunit.client.0.smithi022.stdout:6/210: creat d1/d24/f50 x:0 0 0 2024-02-16T19:28:57.055 INFO:tasks.workunit.client.0.smithi022.stdout:9/237: mknod db/d14/d16/d21/c56 0 2024-02-16T19:28:57.056 INFO:tasks.workunit.client.0.smithi022.stdout:9/238: chown db/d14/d19/d1a/c23 403 1 2024-02-16T19:28:57.056 INFO:tasks.workunit.client.1.smithi117.stdout:0/247: link d0/d9/d22/d27/d31/l4a d0/d9/d14/d1d/l51 0 2024-02-16T19:28:57.061 INFO:tasks.workunit.client.0.smithi022.stdout:6/211: dwrite d1/f2 [0,4194304] 0 2024-02-16T19:28:57.062 INFO:tasks.workunit.client.1.smithi117.stdout:8/177: creat d4/f3f x:0 0 0 2024-02-16T19:28:57.063 INFO:tasks.workunit.client.0.smithi022.stdout:7/214: mkdir d9/da/d1f/d44 0 2024-02-16T19:28:57.065 INFO:tasks.workunit.client.1.smithi117.stdout:7/171: mkdir d4/d1c/d39/d46 0 2024-02-16T19:28:57.067 INFO:tasks.workunit.client.1.smithi117.stdout:7/172: write d4/d1c/f32 [1361611,79880] 0 2024-02-16T19:28:57.070 INFO:tasks.workunit.client.0.smithi022.stdout:0/316: rename d10/d15/f55 to d10/d11/d27/d3f/d58/d60/d62/f69 0 2024-02-16T19:28:57.075 INFO:tasks.workunit.client.1.smithi117.stdout:2/157: mkdir d0/d29/d2f 0 2024-02-16T19:28:57.076 INFO:tasks.workunit.client.0.smithi022.stdout:9/239: fsync f5 0 2024-02-16T19:28:57.082 INFO:tasks.workunit.client.0.smithi022.stdout:6/212: mkdir d1/d8/da/d17/d48/d51 0 2024-02-16T19:28:57.083 INFO:tasks.workunit.client.0.smithi022.stdout:6/213: readlink d1/l1b 0 2024-02-16T19:28:57.084 INFO:tasks.workunit.client.1.smithi117.stdout:1/130: creat d2/d1d/f29 x:0 0 0 2024-02-16T19:28:57.086 INFO:tasks.workunit.client.0.smithi022.stdout:2/165: mknod d0/d32/c35 0 2024-02-16T19:28:57.087 INFO:tasks.workunit.client.0.smithi022.stdout:9/240: dread db/fc [0,4194304] 0 2024-02-16T19:28:57.090 INFO:tasks.workunit.client.1.smithi117.stdout:1/131: dwrite d2/d18/f1b [0,4194304] 0 2024-02-16T19:28:57.092 INFO:tasks.workunit.client.0.smithi022.stdout:0/317: creat d10/d11/d27/d3f/d58/f6a x:0 0 0 2024-02-16T19:28:57.099 INFO:tasks.workunit.client.1.smithi117.stdout:8/178: creat d4/d12/d31/f40 x:0 0 0 2024-02-16T19:28:57.102 INFO:tasks.workunit.client.0.smithi022.stdout:4/211: dwrite d4/d6/dd/df/d10/f19 [0,4194304] 0 2024-02-16T19:28:57.102 INFO:tasks.workunit.client.0.smithi022.stdout:6/214: rename d1/d8/da/d29/c2f to d1/d8/d23/c52 0 2024-02-16T19:28:57.105 INFO:tasks.workunit.client.0.smithi022.stdout:7/215: mknod d9/c45 0 2024-02-16T19:28:57.107 INFO:tasks.workunit.client.0.smithi022.stdout:2/166: readlink d0/l2c 0 2024-02-16T19:28:57.109 INFO:tasks.workunit.client.1.smithi117.stdout:2/158: rmdir d0/d13 39 2024-02-16T19:28:57.112 INFO:tasks.workunit.client.0.smithi022.stdout:9/241: creat db/d14/d16/d28/f57 x:0 0 0 2024-02-16T19:28:57.112 INFO:tasks.workunit.client.0.smithi022.stdout:2/167: dread d0/f17 [0,4194304] 0 2024-02-16T19:28:57.113 INFO:tasks.workunit.client.0.smithi022.stdout:2/168: dread - d0/f2d zero size 2024-02-16T19:28:57.114 INFO:tasks.workunit.client.0.smithi022.stdout:0/318: symlink d10/d11/d27/l6b 0 2024-02-16T19:28:57.120 INFO:tasks.workunit.client.0.smithi022.stdout:2/169: dwrite d0/d2/d18/d1f/f21 [0,4194304] 0 2024-02-16T19:28:57.120 INFO:tasks.workunit.client.0.smithi022.stdout:4/212: mknod d4/d6/dd/d31/c48 0 2024-02-16T19:28:57.123 INFO:tasks.workunit.client.0.smithi022.stdout:6/215: rename d1/d8/d3c/d4c to d1/d8/da/d17/d53 0 2024-02-16T19:28:57.125 INFO:tasks.workunit.client.0.smithi022.stdout:7/216: mkdir d9/da/d2c/d46 0 2024-02-16T19:28:57.126 INFO:tasks.workunit.client.1.smithi117.stdout:9/180: symlink dd/d23/d2e/d2c/d31/l34 0 2024-02-16T19:28:57.128 INFO:tasks.workunit.client.1.smithi117.stdout:0/248: link d0/d9/d2c/f45 d0/d9/d22/d27/f52 0 2024-02-16T19:28:57.129 INFO:tasks.workunit.client.0.smithi022.stdout:9/242: mknod db/d14/d19/c58 0 2024-02-16T19:28:57.132 INFO:tasks.workunit.client.1.smithi117.stdout:6/211: fsync d5/f12 0 2024-02-16T19:28:57.135 INFO:tasks.workunit.client.0.smithi022.stdout:2/170: read d0/f20 [2631060,123313] 0 2024-02-16T19:28:57.135 INFO:tasks.workunit.client.1.smithi117.stdout:8/179: dread d4/d12/f1a [0,4194304] 0 2024-02-16T19:28:57.136 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:56 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:28:57.137 INFO:tasks.workunit.client.1.smithi117.stdout:5/287: rename d2/d9/de/d17 to d2/d3/d21/d49/d59 0 2024-02-16T19:28:57.139 INFO:tasks.workunit.client.1.smithi117.stdout:9/181: unlink dd/f20 0 2024-02-16T19:28:57.141 INFO:tasks.workunit.client.1.smithi117.stdout:9/182: read dd/d23/f1f [324277,34851] 0 2024-02-16T19:28:57.141 INFO:tasks.workunit.client.1.smithi117.stdout:8/180: dwrite d4/d12/d2a/f2b [0,4194304] 0 2024-02-16T19:28:57.143 INFO:tasks.workunit.client.0.smithi022.stdout:7/217: chown d9/da/d2c/c33 6673 1 2024-02-16T19:28:57.144 INFO:tasks.workunit.client.1.smithi117.stdout:8/181: read d4/d12/d2a/f2c [253093,36969] 0 2024-02-16T19:28:57.144 INFO:tasks.workunit.client.1.smithi117.stdout:0/249: creat d0/f53 x:0 0 0 2024-02-16T19:28:57.146 INFO:tasks.workunit.client.1.smithi117.stdout:8/182: stat d4/d7/l23 0 2024-02-16T19:28:57.148 INFO:tasks.workunit.client.1.smithi117.stdout:8/183: chown d4/d7/fc 264 1 2024-02-16T19:28:57.149 INFO:tasks.workunit.client.1.smithi117.stdout:9/183: dwrite dd/d23/f22 [4194304,4194304] 0 2024-02-16T19:28:57.150 INFO:tasks.workunit.client.0.smithi022.stdout:7/218: dwrite f4 [0,4194304] 0 2024-02-16T19:28:57.151 INFO:tasks.workunit.client.0.smithi022.stdout:0/319: dread d10/d22/f2d [0,4194304] 0 2024-02-16T19:28:57.151 INFO:tasks.workunit.client.1.smithi117.stdout:6/212: rename d5/d18 to d5/d4b 0 2024-02-16T19:28:57.152 INFO:tasks.workunit.client.1.smithi117.stdout:6/213: chown d5/d1d/f3d 183 1 2024-02-16T19:28:57.155 INFO:tasks.workunit.client.1.smithi117.stdout:6/214: write d5/d1d/d28/d32/f43 [425335,109200] 0 2024-02-16T19:28:57.155 INFO:tasks.workunit.client.0.smithi022.stdout:4/213: mknod d4/d6/dd/df/c49 0 2024-02-16T19:28:57.156 INFO:tasks.workunit.client.1.smithi117.stdout:4/192: getdents d0/da/d21/d3d/d28/d2f 0 2024-02-16T19:28:57.156 INFO:tasks.workunit.client.1.smithi117.stdout:8/184: dwrite d4/f3f [0,4194304] 0 2024-02-16T19:28:57.156 INFO:tasks.workunit.client.0.smithi022.stdout:9/243: dwrite db/fc [0,4194304] 0 2024-02-16T19:28:57.157 INFO:tasks.workunit.client.1.smithi117.stdout:8/185: fsync d4/d7/f3d 0 2024-02-16T19:28:57.158 INFO:tasks.workunit.client.1.smithi117.stdout:4/193: write d0/da/d21/d3d/d2d/f36 [505867,32113] 0 2024-02-16T19:28:57.160 INFO:tasks.workunit.client.0.smithi022.stdout:2/171: chown d0/lc 646804 1 2024-02-16T19:28:57.163 INFO:tasks.workunit.client.0.smithi022.stdout:2/172: dread - d0/d2/d18/d19/f30 zero size 2024-02-16T19:28:57.163 INFO:tasks.workunit.client.0.smithi022.stdout:4/214: dread d4/d6/dd/f2a [0,4194304] 0 2024-02-16T19:28:57.163 INFO:tasks.workunit.client.0.smithi022.stdout:2/173: readlink d0/lc 0 2024-02-16T19:28:57.174 INFO:tasks.workunit.client.0.smithi022.stdout:0/320: symlink d10/d11/d27/d3f/d58/d60/d62/l6c 0 2024-02-16T19:28:57.176 INFO:tasks.workunit.client.1.smithi117.stdout:7/173: getdents d4/d1c 0 2024-02-16T19:28:57.177 INFO:tasks.workunit.client.1.smithi117.stdout:7/174: write d4/f17 [892517,20305] 0 2024-02-16T19:28:57.178 INFO:tasks.workunit.client.0.smithi022.stdout:6/216: write d1/f3 [987337,74648] 0 2024-02-16T19:28:57.179 INFO:tasks.workunit.client.1.smithi117.stdout:0/250: truncate d0/d9/f1c 683163 0 2024-02-16T19:28:57.183 INFO:tasks.workunit.client.0.smithi022.stdout:9/244: write db/d14/d19/d1a/d3b/f3c [163380,96582] 0 2024-02-16T19:28:57.184 INFO:tasks.workunit.client.0.smithi022.stdout:4/215: mknod d4/d6/d3a/c4a 0 2024-02-16T19:28:57.185 INFO:tasks.workunit.client.0.smithi022.stdout:9/245: write db/fc [327444,58123] 0 2024-02-16T19:28:57.186 INFO:tasks.workunit.client.0.smithi022.stdout:4/216: fsync d4/d6/dd/f2a 0 2024-02-16T19:28:57.189 INFO:tasks.workunit.client.1.smithi117.stdout:4/194: dread d0/f25 [0,4194304] 0 2024-02-16T19:28:57.189 INFO:tasks.workunit.client.0.smithi022.stdout:2/174: mkdir d0/d2/d18/d1f/d36 0 2024-02-16T19:28:57.189 INFO:tasks.workunit.client.0.smithi022.stdout:0/321: dread d10/d15/f39 [0,4194304] 0 2024-02-16T19:28:57.193 INFO:tasks.workunit.client.0.smithi022.stdout:6/217: rename d1/d46 to d1/d8/da/d17/d53/d54 0 2024-02-16T19:28:57.196 INFO:tasks.workunit.client.0.smithi022.stdout:2/175: dwrite d0/d2/f14 [0,4194304] 0 2024-02-16T19:28:57.198 INFO:tasks.workunit.client.1.smithi117.stdout:5/288: unlink d2/d3/d21/d49/f2e 0 2024-02-16T19:28:57.203 INFO:tasks.workunit.client.0.smithi022.stdout:4/217: mkdir d4/d6/dd/df/d10/d24/d4b 0 2024-02-16T19:28:57.206 INFO:tasks.workunit.client.0.smithi022.stdout:2/176: dwrite d0/d2/d18/f34 [0,4194304] 0 2024-02-16T19:28:57.211 INFO:tasks.workunit.client.0.smithi022.stdout:7/219: link c8 d9/d1b/d23/c47 0 2024-02-16T19:28:57.213 INFO:tasks.workunit.client.0.smithi022.stdout:7/220: fsync f4 0 2024-02-16T19:28:57.213 INFO:tasks.workunit.client.0.smithi022.stdout:7/221: chown d9/l19 19116745 1 2024-02-16T19:28:57.213 INFO:tasks.workunit.client.1.smithi117.stdout:0/251: mkdir d0/d9/d14/d1d/d1e/d54 0 2024-02-16T19:28:57.214 INFO:tasks.workunit.client.1.smithi117.stdout:0/252: truncate d0/f53 117679 0 2024-02-16T19:28:57.217 INFO:tasks.workunit.client.0.smithi022.stdout:9/246: dread db/f12 [0,4194304] 0 2024-02-16T19:28:57.218 INFO:tasks.workunit.client.1.smithi117.stdout:9/184: link dd/d23/d2e/f32 dd/d2d/f35 0 2024-02-16T19:28:57.218 INFO:tasks.workunit.client.0.smithi022.stdout:9/247: chown db/d14/d19/f17 227048 1 2024-02-16T19:28:57.222 INFO:tasks.workunit.client.1.smithi117.stdout:4/195: write d0/f25 [2413276,24877] 0 2024-02-16T19:28:57.224 INFO:tasks.workunit.client.1.smithi117.stdout:3/197: rename d1/d1b/l2a to d1/l45 0 2024-02-16T19:28:57.225 INFO:tasks.workunit.client.1.smithi117.stdout:3/198: fsync d1/d1b/d1e/f22 0 2024-02-16T19:28:57.226 INFO:tasks.workunit.client.1.smithi117.stdout:8/186: symlink d4/d7/d37/d3a/l41 0 2024-02-16T19:28:57.227 INFO:tasks.workunit.client.1.smithi117.stdout:8/187: write d4/d12/d31/f40 [9275,115732] 0 2024-02-16T19:28:57.227 INFO:tasks.workunit.client.0.smithi022.stdout:7/222: creat d9/da/d2c/d2f/d21/f48 x:0 0 0 2024-02-16T19:28:57.229 INFO:tasks.workunit.client.1.smithi117.stdout:7/175: mkdir d4/d1c/d39/d46/d47 0 2024-02-16T19:28:57.232 INFO:tasks.workunit.client.0.smithi022.stdout:9/248: symlink db/d14/d16/d27/l59 0 2024-02-16T19:28:57.232 INFO:tasks.workunit.client.0.smithi022.stdout:3/203: sync 2024-02-16T19:28:57.233 INFO:tasks.workunit.client.1.smithi117.stdout:0/253: mkdir d0/d9/d22/d26/d55 0 2024-02-16T19:28:57.235 INFO:tasks.workunit.client.0.smithi022.stdout:4/218: write d4/d38/f3d [1313415,21602] 0 2024-02-16T19:28:57.236 INFO:tasks.workunit.client.0.smithi022.stdout:4/219: write d4/d6/d14/f2e [278728,119966] 0 2024-02-16T19:28:57.238 INFO:tasks.workunit.client.1.smithi117.stdout:4/196: creat d0/da/d21/f40 x:0 0 0 2024-02-16T19:28:57.240 INFO:tasks.workunit.client.0.smithi022.stdout:0/322: creat d10/f6d x:0 0 0 2024-02-16T19:28:57.241 INFO:tasks.workunit.client.1.smithi117.stdout:5/289: rename d2/d9/de to d2/d3/d21/d2f/d30/d5a 0 2024-02-16T19:28:57.242 INFO:tasks.workunit.client.0.smithi022.stdout:6/218: creat d1/d8/da/f55 x:0 0 0 2024-02-16T19:28:57.243 INFO:tasks.workunit.client.0.smithi022.stdout:6/219: fdatasync d1/d8/d3c/f43 0 2024-02-16T19:28:57.244 INFO:tasks.workunit.client.0.smithi022.stdout:6/220: write d1/d8/d3c/d4d/f4b [835618,110587] 0 2024-02-16T19:28:57.245 INFO:tasks.workunit.client.0.smithi022.stdout:7/223: rename f4 to d9/da/d1f/d44/f49 0 2024-02-16T19:28:57.246 INFO:tasks.workunit.client.1.smithi117.stdout:3/199: symlink d1/d31/l46 0 2024-02-16T19:28:57.246 INFO:tasks.workunit.client.0.smithi022.stdout:7/224: write d9/da/d2c/d2f/d21/f48 [362099,86221] 0 2024-02-16T19:28:57.247 INFO:tasks.workunit.client.1.smithi117.stdout:5/290: dwrite d2/d4c/d54/f43 [0,4194304] 0 2024-02-16T19:28:57.247 INFO:tasks.workunit.client.1.smithi117.stdout:8/188: rmdir d4/d7/d37 39 2024-02-16T19:28:57.249 INFO:tasks.workunit.client.0.smithi022.stdout:3/204: mkdir d7/d12/d18/dd/d3c/d5a 0 2024-02-16T19:28:57.250 INFO:tasks.workunit.client.0.smithi022.stdout:3/205: chown d7/d12/d18/dd/d31/d41/f4c 15 1 2024-02-16T19:28:57.251 INFO:tasks.workunit.client.1.smithi117.stdout:7/176: creat d4/d1c/d39/d46/f48 x:0 0 0 2024-02-16T19:28:57.251 INFO:tasks.workunit.client.0.smithi022.stdout:3/206: chown d7/d12/c2b 327572 1 2024-02-16T19:28:57.251 INFO:tasks.workunit.client.1.smithi117.stdout:7/177: stat d4/d1c/d24/f29 0 2024-02-16T19:28:57.255 INFO:tasks.workunit.client.1.smithi117.stdout:0/254: creat d0/d9/d14/d23/f56 x:0 0 0 2024-02-16T19:28:57.256 INFO:tasks.workunit.client.0.smithi022.stdout:4/220: symlink d4/d6/l4c 0 2024-02-16T19:28:57.256 INFO:tasks.workunit.client.1.smithi117.stdout:6/215: getdents d5/d4b/d20/d2d 0 2024-02-16T19:28:57.259 INFO:tasks.workunit.client.1.smithi117.stdout:8/189: dread d4/f38 [0,4194304] 0 2024-02-16T19:28:57.260 INFO:tasks.workunit.client.1.smithi117.stdout:3/200: unlink d1/dd/l12 0 2024-02-16T19:28:57.261 INFO:tasks.workunit.client.0.smithi022.stdout:6/221: mkdir d1/d24/d56 0 2024-02-16T19:28:57.262 INFO:tasks.workunit.client.0.smithi022.stdout:4/221: dwrite d4/d6/dd/f37 [0,4194304] 0 2024-02-16T19:28:57.263 INFO:tasks.workunit.client.0.smithi022.stdout:6/222: write d1/d8/da/d17/f1d [1582423,62832] 0 2024-02-16T19:28:57.264 INFO:tasks.workunit.client.0.smithi022.stdout:9/249: mknod db/d14/d16/d27/d55/c5a 0 2024-02-16T19:28:57.264 INFO:tasks.workunit.client.0.smithi022.stdout:4/222: chown d4/d6/f16 24499333 1 2024-02-16T19:28:57.267 INFO:tasks.workunit.client.0.smithi022.stdout:9/250: chown db/d14/d19/d1a/f50 1 1 2024-02-16T19:28:57.267 INFO:tasks.workunit.client.0.smithi022.stdout:4/223: chown d4/d6/dd/df/d10/d24/d4b 27075692 1 2024-02-16T19:28:57.269 INFO:tasks.workunit.client.0.smithi022.stdout:7/225: rename c8 to d9/da/d2c/d2f/c4a 0 2024-02-16T19:28:57.270 INFO:tasks.workunit.client.1.smithi117.stdout:0/255: symlink d0/d9/d22/d26/l57 0 2024-02-16T19:28:57.271 INFO:tasks.workunit.client.0.smithi022.stdout:3/207: mknod d7/d12/d18/dd/d3c/c5b 0 2024-02-16T19:28:57.272 INFO:tasks.workunit.client.1.smithi117.stdout:0/256: write d0/fe [4707988,120619] 0 2024-02-16T19:28:57.272 INFO:tasks.workunit.client.0.smithi022.stdout:4/224: truncate d4/d6/dd/d22/d30/f44 542455 0 2024-02-16T19:28:57.273 INFO:tasks.workunit.client.1.smithi117.stdout:6/216: symlink d5/d4b/d20/d2d/l4c 0 2024-02-16T19:28:57.274 INFO:tasks.workunit.client.0.smithi022.stdout:6/223: dwrite d1/d8/da/d17/f2d [0,4194304] 0 2024-02-16T19:28:57.274 INFO:tasks.workunit.client.0.smithi022.stdout:3/208: dread - d7/d12/d18/dd/d3c/f49 zero size 2024-02-16T19:28:57.275 INFO:tasks.workunit.client.0.smithi022.stdout:2/177: getdents d0 0 2024-02-16T19:28:57.276 INFO:tasks.workunit.client.0.smithi022.stdout:4/225: write d4/d6/dd/d31/f34 [222580,23089] 0 2024-02-16T19:28:57.277 INFO:tasks.workunit.client.0.smithi022.stdout:4/226: chown d4/d6/dd 1272 1 2024-02-16T19:28:57.277 INFO:tasks.workunit.client.1.smithi117.stdout:8/190: creat d4/d12/f42 x:0 0 0 2024-02-16T19:28:57.279 INFO:tasks.workunit.client.1.smithi117.stdout:3/201: creat d1/d1b/d1e/d29/f47 x:0 0 0 2024-02-16T19:28:57.282 INFO:tasks.workunit.client.0.smithi022.stdout:9/251: unlink db/d14/d16/d28/f57 0 2024-02-16T19:28:57.282 INFO:tasks.workunit.client.0.smithi022.stdout:3/209: dwrite d7/d12/f19 [0,4194304] 0 2024-02-16T19:28:57.284 INFO:tasks.workunit.client.0.smithi022.stdout:3/210: chown d7/d12/d18/l10 1003378428 1 2024-02-16T19:28:57.285 INFO:tasks.workunit.client.0.smithi022.stdout:7/226: rmdir d9/d1b 39 2024-02-16T19:28:57.289 INFO:tasks.workunit.client.1.smithi117.stdout:0/257: rename d0/f35 to d0/d9/d22/d27/d31/f58 0 2024-02-16T19:28:57.290 INFO:tasks.workunit.client.0.smithi022.stdout:0/323: creat d10/f6e x:0 0 0 2024-02-16T19:28:57.291 INFO:tasks.workunit.client.1.smithi117.stdout:0/258: write d0/d9/d22/d27/d31/f38 [1332183,77144] 0 2024-02-16T19:28:57.291 INFO:tasks.workunit.client.1.smithi117.stdout:6/217: creat d5/d4b/d20/f4d x:0 0 0 2024-02-16T19:28:57.292 INFO:tasks.workunit.client.0.smithi022.stdout:2/178: creat d0/f37 x:0 0 0 2024-02-16T19:28:57.292 INFO:tasks.workunit.client.1.smithi117.stdout:0/259: chown d0/d9/d14/d23/f56 17988 1 2024-02-16T19:28:57.293 INFO:tasks.workunit.client.1.smithi117.stdout:6/218: write d5/d1d/d28/f34 [65079,42232] 0 2024-02-16T19:28:57.293 INFO:tasks.workunit.client.0.smithi022.stdout:6/224: mkdir d1/d8/da/d17/d53/d54/d57 0 2024-02-16T19:28:57.295 INFO:tasks.workunit.client.1.smithi117.stdout:8/191: unlink d4/d12/l1b 0 2024-02-16T19:28:57.295 INFO:tasks.workunit.client.1.smithi117.stdout:8/192: fsync d4/d12/f42 0 2024-02-16T19:28:57.296 INFO:tasks.workunit.client.1.smithi117.stdout:3/202: creat d1/dd/f48 x:0 0 0 2024-02-16T19:28:57.296 INFO:tasks.workunit.client.1.smithi117.stdout:8/193: readlink d4/le 0 2024-02-16T19:28:57.297 INFO:tasks.workunit.client.1.smithi117.stdout:5/291: creat d2/f5b x:0 0 0 2024-02-16T19:28:57.299 INFO:tasks.workunit.client.1.smithi117.stdout:7/178: creat d4/f49 x:0 0 0 2024-02-16T19:28:57.299 INFO:tasks.workunit.client.1.smithi117.stdout:8/194: dread d4/f38 [0,4194304] 0 2024-02-16T19:28:57.302 INFO:tasks.workunit.client.1.smithi117.stdout:1/132: sync 2024-02-16T19:28:57.304 INFO:tasks.workunit.client.1.smithi117.stdout:8/195: dwrite d4/d12/d31/f36 [0,4194304] 0 2024-02-16T19:28:57.305 INFO:tasks.workunit.client.0.smithi022.stdout:1/333: sync 2024-02-16T19:28:57.305 INFO:tasks.workunit.client.0.smithi022.stdout:5/252: sync 2024-02-16T19:28:57.305 INFO:tasks.workunit.client.0.smithi022.stdout:8/246: sync 2024-02-16T19:28:57.306 INFO:tasks.workunit.client.1.smithi117.stdout:4/197: getdents d0 0 2024-02-16T19:28:57.309 INFO:tasks.workunit.client.1.smithi117.stdout:0/260: unlink d0/f1f 0 2024-02-16T19:28:57.312 INFO:tasks.workunit.client.0.smithi022.stdout:5/253: dwrite f3 [0,4194304] 0 2024-02-16T19:28:57.313 INFO:tasks.workunit.client.1.smithi117.stdout:4/198: dwrite d0/da/d21/d3d/f29 [0,4194304] 0 2024-02-16T19:28:57.318 INFO:tasks.workunit.client.1.smithi117.stdout:4/199: dwrite d0/f25 [0,4194304] 0 2024-02-16T19:28:57.319 INFO:tasks.workunit.client.1.smithi117.stdout:4/200: fdatasync d0/da/d21/d3d/d2d/f36 0 2024-02-16T19:28:57.320 INFO:tasks.workunit.client.0.smithi022.stdout:6/225: creat d1/d8/da/d17/d53/f58 x:0 0 0 2024-02-16T19:28:57.320 INFO:tasks.workunit.client.0.smithi022.stdout:5/254: write f6 [13245419,79571] 0 2024-02-16T19:28:57.321 INFO:tasks.workunit.client.1.smithi117.stdout:3/203: symlink d1/d1b/d20/l49 0 2024-02-16T19:28:57.322 INFO:tasks.workunit.client.0.smithi022.stdout:0/324: dwrite d10/d22/d49/f4d [0,4194304] 0 2024-02-16T19:28:57.323 INFO:tasks.workunit.client.1.smithi117.stdout:4/201: dread d0/da/d21/d3d/f29 [0,4194304] 0 2024-02-16T19:28:57.323 INFO:tasks.workunit.client.0.smithi022.stdout:0/325: readlink d10/d11/d27/d3f/d58/d60/d62/l63 0 2024-02-16T19:28:57.324 INFO:tasks.workunit.client.1.smithi117.stdout:2/159: sync 2024-02-16T19:28:57.324 INFO:tasks.workunit.client.1.smithi117.stdout:9/185: sync 2024-02-16T19:28:57.325 INFO:tasks.workunit.client.0.smithi022.stdout:0/326: chown d10/d22/l21 21573555 1 2024-02-16T19:28:57.325 INFO:tasks.workunit.client.1.smithi117.stdout:5/292: symlink d2/d3/d21/d2f/l5c 0 2024-02-16T19:28:57.325 INFO:tasks.workunit.client.1.smithi117.stdout:9/186: stat dd/l14 0 2024-02-16T19:28:57.330 INFO:tasks.workunit.client.0.smithi022.stdout:0/327: truncate d10/d11/d27/d3f/d58/f6a 152378 0 2024-02-16T19:28:57.331 INFO:tasks.workunit.client.0.smithi022.stdout:0/328: dread - d10/d22/f2a zero size 2024-02-16T19:28:57.333 INFO:tasks.workunit.client.0.smithi022.stdout:0/329: chown d10/d15/d23/f47 1398 1 2024-02-16T19:28:57.336 INFO:tasks.workunit.client.0.smithi022.stdout:7/227: creat d9/da/d1f/d40/f4b x:0 0 0 2024-02-16T19:28:57.337 INFO:tasks.workunit.client.0.smithi022.stdout:0/330: chown d10/d11/d50/d65 585211293 1 2024-02-16T19:28:57.338 INFO:tasks.workunit.client.1.smithi117.stdout:1/133: creat d2/d18/d20/f2a x:0 0 0 2024-02-16T19:28:57.339 INFO:tasks.workunit.client.0.smithi022.stdout:3/211: dwrite d7/f1c [0,4194304] 0 2024-02-16T19:28:57.348 INFO:tasks.workunit.client.0.smithi022.stdout:5/255: dread dd/f17 [0,4194304] 0 2024-02-16T19:28:57.349 INFO:tasks.workunit.client.0.smithi022.stdout:4/227: link d4/d6/l4c d4/d6/dd/df/d10/l4d 0 2024-02-16T19:28:57.352 INFO:tasks.workunit.client.1.smithi117.stdout:9/187: dread fb [0,4194304] 0 2024-02-16T19:28:57.353 INFO:tasks.workunit.client.0.smithi022.stdout:6/226: rmdir d1/d8/da/d17/d53 39 2024-02-16T19:28:57.356 INFO:tasks.workunit.client.0.smithi022.stdout:5/256: dwrite dd/d1f/d48/d21/d2f/d40/d49/f4c [0,4194304] 0 2024-02-16T19:28:57.356 INFO:tasks.workunit.client.1.smithi117.stdout:4/202: creat d0/da/d21/d3d/d2d/f41 x:0 0 0 2024-02-16T19:28:57.357 INFO:tasks.workunit.client.0.smithi022.stdout:9/252: rename db/d14/d16/l35 to db/d14/d16/l5b 0 2024-02-16T19:28:57.358 INFO:tasks.workunit.client.1.smithi117.stdout:8/196: dwrite d4/d12/f1a [0,4194304] 0 2024-02-16T19:28:57.361 INFO:tasks.workunit.client.1.smithi117.stdout:4/203: dread d0/da/d21/d3d/f1d [0,4194304] 0 2024-02-16T19:28:57.362 INFO:tasks.workunit.client.0.smithi022.stdout:7/228: rmdir d9/da/d2c/d2f/d21 39 2024-02-16T19:28:57.366 INFO:tasks.workunit.client.0.smithi022.stdout:1/334: mknod d6/d8/d22/d47/d54/c60 0 2024-02-16T19:28:57.367 INFO:tasks.workunit.client.0.smithi022.stdout:2/179: mkdir d0/d2/d18/d38 0 2024-02-16T19:28:57.367 INFO:tasks.workunit.client.0.smithi022.stdout:2/180: stat d0 0 2024-02-16T19:28:57.371 INFO:tasks.workunit.client.0.smithi022.stdout:4/228: mkdir d4/d6/dd/d31/d4e 0 2024-02-16T19:28:57.372 INFO:tasks.workunit.client.1.smithi117.stdout:0/261: symlink d0/d9/d14/d1d/d1e/d54/l59 0 2024-02-16T19:28:57.373 INFO:tasks.workunit.client.0.smithi022.stdout:6/227: fsync d1/d8/da/d17/d53/f58 0 2024-02-16T19:28:57.375 INFO:tasks.workunit.client.1.smithi117.stdout:6/219: symlink d5/d4b/d20/d2d/l4e 0 2024-02-16T19:28:57.375 INFO:tasks.workunit.client.0.smithi022.stdout:6/228: write d1/d8/f1f [653736,86876] 0 2024-02-16T19:28:57.376 INFO:tasks.workunit.client.0.smithi022.stdout:9/253: symlink db/d18/l5c 0 2024-02-16T19:28:57.377 INFO:tasks.workunit.client.0.smithi022.stdout:9/254: fdatasync db/fc 0 2024-02-16T19:28:57.378 INFO:tasks.workunit.client.0.smithi022.stdout:7/229: fdatasync d9/da/d2c/f2d 0 2024-02-16T19:28:57.379 INFO:tasks.workunit.client.0.smithi022.stdout:7/230: chown d9/da/d1f/d44/f49 6 1 2024-02-16T19:28:57.380 INFO:tasks.workunit.client.1.smithi117.stdout:1/134: dwrite d2/d4/f9 [0,4194304] 0 2024-02-16T19:28:57.381 INFO:tasks.workunit.client.1.smithi117.stdout:1/135: write d2/ff [6167061,122647] 0 2024-02-16T19:28:57.386 INFO:tasks.workunit.client.1.smithi117.stdout:5/293: mkdir d2/d5d 0 2024-02-16T19:28:57.388 INFO:tasks.workunit.client.0.smithi022.stdout:6/229: unlink d1/d8/f4a 0 2024-02-16T19:28:57.388 INFO:tasks.workunit.client.1.smithi117.stdout:3/204: truncate d1/d1b/f2d 1972690 0 2024-02-16T19:28:57.390 INFO:tasks.workunit.client.1.smithi117.stdout:8/197: rmdir d4 39 2024-02-16T19:28:57.392 INFO:tasks.workunit.client.0.smithi022.stdout:6/230: dread d1/d8/f1f [0,4194304] 0 2024-02-16T19:28:57.394 INFO:tasks.workunit.client.0.smithi022.stdout:6/231: write d1/d8/da/d29/f35 [433672,59318] 0 2024-02-16T19:28:57.394 INFO:tasks.workunit.client.1.smithi117.stdout:4/204: symlink d0/l42 0 2024-02-16T19:28:57.400 INFO:tasks.workunit.client.0.smithi022.stdout:5/257: write dd/f18 [1278104,57510] 0 2024-02-16T19:28:57.401 INFO:tasks.workunit.client.0.smithi022.stdout:5/258: fsync dd/d1f/d48/f1c 0 2024-02-16T19:28:57.402 INFO:tasks.workunit.client.1.smithi117.stdout:7/179: rename d4/d13/d38 to d4/d4a 0 2024-02-16T19:28:57.403 INFO:tasks.workunit.client.0.smithi022.stdout:5/259: dread dd/d1f/f2b [0,4194304] 0 2024-02-16T19:28:57.404 INFO:tasks.workunit.client.0.smithi022.stdout:7/231: rmdir d9/da/d2c 39 2024-02-16T19:28:57.405 INFO:tasks.workunit.client.1.smithi117.stdout:0/262: mknod d0/d9/d22/d27/d4e/c5a 0 2024-02-16T19:28:57.407 INFO:tasks.workunit.client.1.smithi117.stdout:0/263: truncate d0/d9/d22/d27/d31/f38 5027271 0 2024-02-16T19:28:57.407 INFO:tasks.workunit.client.0.smithi022.stdout:1/335: mknod d6/d8/d22/d16/d17/d5d/c61 0 2024-02-16T19:28:57.407 INFO:tasks.workunit.client.1.smithi117.stdout:0/264: readlink d0/d9/d14/d1d/d1e/d54/l59 0 2024-02-16T19:28:57.407 INFO:tasks.workunit.client.1.smithi117.stdout:0/265: chown d0/d9/d22/d27 75343212 1 2024-02-16T19:28:57.408 INFO:tasks.workunit.client.0.smithi022.stdout:1/336: write d6/d8/d22/d16/d17/f1a [948139,123253] 0 2024-02-16T19:28:57.410 INFO:tasks.workunit.client.0.smithi022.stdout:1/337: truncate d6/d8/d22/d16/f2e 347784 0 2024-02-16T19:28:57.411 INFO:tasks.workunit.client.0.smithi022.stdout:1/338: write d6/d8/d22/d47/f4e [3323643,42473] 0 2024-02-16T19:28:57.413 INFO:tasks.workunit.client.1.smithi117.stdout:1/136: creat d2/d18/d20/f2b x:0 0 0 2024-02-16T19:28:57.414 INFO:tasks.workunit.client.1.smithi117.stdout:1/137: write d2/d18/d20/f2a [51697,34799] 0 2024-02-16T19:28:57.416 INFO:tasks.workunit.client.0.smithi022.stdout:9/255: mkdir db/d14/d16/d5d 0 2024-02-16T19:28:57.419 INFO:tasks.workunit.client.1.smithi117.stdout:3/205: rmdir d1/dd/d3b 39 2024-02-16T19:28:57.422 INFO:tasks.workunit.client.1.smithi117.stdout:4/205: mknod d0/da/c43 0 2024-02-16T19:28:57.423 INFO:tasks.workunit.client.0.smithi022.stdout:3/212: getdents d7/d12/d18 0 2024-02-16T19:28:57.425 INFO:tasks.workunit.client.1.smithi117.stdout:2/160: link d0/d29/f2e d0/d3/d10/f30 0 2024-02-16T19:28:57.426 INFO:tasks.workunit.client.1.smithi117.stdout:8/198: dread d4/d12/f1a [0,4194304] 0 2024-02-16T19:28:57.426 INFO:tasks.workunit.client.1.smithi117.stdout:6/220: rename d5/db/l26 to d5/d1d/l4f 0 2024-02-16T19:28:57.426 INFO:tasks.workunit.client.1.smithi117.stdout:5/294: dwrite d2/d3/f12 [0,4194304] 0 2024-02-16T19:28:57.427 INFO:tasks.workunit.client.0.smithi022.stdout:7/232: read d9/da/d2c/d2f/f17 [4058184,45397] 0 2024-02-16T19:28:57.427 INFO:tasks.workunit.client.1.smithi117.stdout:4/206: truncate d0/da/d21/d3d/d28/d2f/f31 700489 0 2024-02-16T19:28:57.427 INFO:tasks.workunit.client.1.smithi117.stdout:7/180: creat d4/d13/f4b x:0 0 0 2024-02-16T19:28:57.429 INFO:tasks.workunit.client.1.smithi117.stdout:0/266: mkdir d0/d9/d22/d26/d5b 0 2024-02-16T19:28:57.430 INFO:tasks.workunit.client.1.smithi117.stdout:9/188: rmdir dd/d23/d2e 39 2024-02-16T19:28:57.431 INFO:tasks.workunit.client.0.smithi022.stdout:1/339: stat d6/d8/d22/f1b 0 2024-02-16T19:28:57.432 INFO:tasks.workunit.client.0.smithi022.stdout:1/340: dread - d6/f55 zero size 2024-02-16T19:28:57.432 INFO:tasks.workunit.client.0.smithi022.stdout:1/341: dread - d6/f55 zero size 2024-02-16T19:28:57.433 INFO:tasks.workunit.client.1.smithi117.stdout:1/138: mknod d2/d19/c2c 0 2024-02-16T19:28:57.435 INFO:tasks.workunit.client.0.smithi022.stdout:6/232: rename d1/d8/f1f to d1/d8/da/d17/d48/d51/f59 0 2024-02-16T19:28:57.435 INFO:tasks.workunit.client.0.smithi022.stdout:6/233: fdatasync d1/d8/f11 0 2024-02-16T19:28:57.438 INFO:tasks.workunit.client.0.smithi022.stdout:3/213: mknod d7/d12/d18/d4b/c5c 0 2024-02-16T19:28:57.440 INFO:tasks.workunit.client.1.smithi117.stdout:3/206: dwrite d1/fc [4194304,4194304] 0 2024-02-16T19:28:57.441 INFO:tasks.workunit.client.1.smithi117.stdout:3/207: fsync d1/f28 0 2024-02-16T19:28:57.544 INFO:tasks.workunit.client.1.smithi117.stdout:2/161: creat d0/d3/f31 x:0 0 0 2024-02-16T19:28:57.546 INFO:tasks.workunit.client.1.smithi117.stdout:6/221: rmdir d5/d4b 39 2024-02-16T19:28:57.549 INFO:tasks.workunit.client.1.smithi117.stdout:2/162: dwrite d0/f19 [0,4194304] 0 2024-02-16T19:28:57.550 INFO:tasks.workunit.client.0.smithi022.stdout:7/233: write f5 [3158342,117454] 0 2024-02-16T19:28:57.551 INFO:tasks.workunit.client.1.smithi117.stdout:7/181: creat d4/d4a/f4c x:0 0 0 2024-02-16T19:28:57.553 INFO:tasks.workunit.client.1.smithi117.stdout:5/295: chown d2/d9/d3c/d24/d50/d52 106 1 2024-02-16T19:28:57.554 INFO:tasks.workunit.client.1.smithi117.stdout:5/296: write d2/d3/d21/d2f/d30/f3e [631848,77366] 0 2024-02-16T19:28:57.555 INFO:tasks.workunit.client.0.smithi022.stdout:6/234: rename d1/d8/l4f to d1/d8/da/d17/d53/l5a 0 2024-02-16T19:28:57.556 INFO:tasks.workunit.client.1.smithi117.stdout:4/207: read d0/d6/f1c [442852,92933] 0 2024-02-16T19:28:57.556 INFO:tasks.workunit.client.0.smithi022.stdout:7/234: dread d9/da/d1f/f18 [0,4194304] 0 2024-02-16T19:28:57.556 INFO:tasks.workunit.client.0.smithi022.stdout:7/235: fdatasync d9/da/f29 0 2024-02-16T19:28:57.557 INFO:tasks.workunit.client.0.smithi022.stdout:7/236: stat d9/l3e 0 2024-02-16T19:28:57.558 INFO:tasks.workunit.client.1.smithi117.stdout:0/267: unlink d0/d9/d2c/f45 0 2024-02-16T19:28:57.559 INFO:tasks.workunit.client.0.smithi022.stdout:3/214: unlink d7/cb 0 2024-02-16T19:28:57.560 INFO:tasks.workunit.client.1.smithi117.stdout:4/208: dwrite d0/f11 [0,4194304] 0 2024-02-16T19:28:57.561 INFO:tasks.workunit.client.1.smithi117.stdout:5/297: read d2/d9/d3c/f1a [578320,48629] 0 2024-02-16T19:28:57.562 INFO:tasks.workunit.client.0.smithi022.stdout:5/260: creat dd/d1f/d48/f51 x:0 0 0 2024-02-16T19:28:57.562 INFO:tasks.workunit.client.0.smithi022.stdout:7/237: dread d9/da/d1f/d44/f49 [0,4194304] 0 2024-02-16T19:28:57.562 INFO:tasks.workunit.client.0.smithi022.stdout:7/238: chown d9/da/f3a 174 1 2024-02-16T19:28:57.563 INFO:tasks.workunit.client.1.smithi117.stdout:1/139: mknod d2/d1d/d1f/c2d 0 2024-02-16T19:28:57.566 INFO:tasks.workunit.client.1.smithi117.stdout:4/209: dwrite d0/da/d21/f2b [0,4194304] 0 2024-02-16T19:28:57.567 INFO:tasks.workunit.client.0.smithi022.stdout:6/235: mkdir d1/d24/d5b 0 2024-02-16T19:28:57.567 INFO:tasks.workunit.client.0.smithi022.stdout:7/239: dread d9/f25 [0,4194304] 0 2024-02-16T19:28:57.570 INFO:tasks.workunit.client.1.smithi117.stdout:4/210: dread d0/f25 [0,4194304] 0 2024-02-16T19:28:57.571 INFO:tasks.workunit.client.1.smithi117.stdout:3/208: symlink d1/d1b/d1e/d29/l4a 0 2024-02-16T19:28:57.571 INFO:tasks.workunit.client.0.smithi022.stdout:8/247: sync 2024-02-16T19:28:57.573 INFO:tasks.workunit.client.1.smithi117.stdout:4/211: dread d0/f11 [0,4194304] 0 2024-02-16T19:28:57.575 INFO:tasks.workunit.client.0.smithi022.stdout:5/261: mkdir dd/d1f/d48/d21/d2f/d52 0 2024-02-16T19:28:57.578 INFO:tasks.workunit.client.1.smithi117.stdout:2/163: mknod d0/d29/c32 0 2024-02-16T19:28:57.584 INFO:tasks.workunit.client.0.smithi022.stdout:6/236: creat d1/d8/da/d17/d53/f5c x:0 0 0 2024-02-16T19:28:57.587 INFO:tasks.workunit.client.1.smithi117.stdout:9/189: symlink dd/d23/l36 0 2024-02-16T19:28:57.590 INFO:tasks.workunit.client.0.smithi022.stdout:3/215: link d7/d21/f23 d7/d21/f5d 0 2024-02-16T19:28:57.590 INFO:tasks.workunit.client.0.smithi022.stdout:6/237: dwrite d1/d8/d3c/d4d/f45 [0,4194304] 0 2024-02-16T19:28:57.591 INFO:tasks.workunit.client.1.smithi117.stdout:1/140: symlink d2/d1d/l2e 0 2024-02-16T19:28:57.592 INFO:tasks.workunit.client.0.smithi022.stdout:6/238: truncate d1/f32 402391 0 2024-02-16T19:28:57.592 INFO:tasks.workunit.client.1.smithi117.stdout:1/141: chown d2/d4/c6 100 1 2024-02-16T19:28:57.595 INFO:tasks.workunit.client.0.smithi022.stdout:1/342: getdents d6/d8/d22/d16/d17 0 2024-02-16T19:28:57.598 INFO:tasks.workunit.client.0.smithi022.stdout:7/240: mknod d9/da/d2c/d2f/d21/c4c 0 2024-02-16T19:28:57.599 INFO:tasks.workunit.client.0.smithi022.stdout:1/343: dread d6/d8/d22/f1b [0,4194304] 0 2024-02-16T19:28:57.600 INFO:tasks.workunit.client.1.smithi117.stdout:3/209: fsync d1/dd/d3b/f3d 0 2024-02-16T19:28:57.602 INFO:tasks.workunit.client.0.smithi022.stdout:8/248: mkdir d3/d12/d17/d58 0 2024-02-16T19:28:57.605 INFO:tasks.workunit.client.1.smithi117.stdout:3/210: dwrite d1/f8 [4194304,4194304] 0 2024-02-16T19:28:57.606 INFO:tasks.workunit.client.0.smithi022.stdout:6/239: mknod d1/d24/c5d 0 2024-02-16T19:28:57.606 INFO:tasks.workunit.client.1.smithi117.stdout:4/212: creat d0/da/f44 x:0 0 0 2024-02-16T19:28:57.607 INFO:tasks.workunit.client.1.smithi117.stdout:3/211: dread d1/d1b/d1e/d29/f33 [0,4194304] 0 2024-02-16T19:28:57.607 INFO:tasks.workunit.client.1.smithi117.stdout:8/199: truncate d4/d1c/f1f 3112283 0 2024-02-16T19:28:57.609 INFO:tasks.workunit.client.1.smithi117.stdout:4/213: write d0/da/d21/f27 [2713886,46652] 0 2024-02-16T19:28:57.609 INFO:tasks.workunit.client.1.smithi117.stdout:3/212: chown d1/dd/d32 0 1 2024-02-16T19:28:57.609 INFO:tasks.workunit.client.1.smithi117.stdout:2/164: fdatasync d0/d3/d10/f30 0 2024-02-16T19:28:57.610 INFO:tasks.workunit.client.0.smithi022.stdout:5/262: creat dd/d1f/d48/d21/d2f/d52/f53 x:0 0 0 2024-02-16T19:28:57.613 INFO:tasks.workunit.client.1.smithi117.stdout:3/213: truncate d1/d1b/d1e/d37/f38 4547169 0 2024-02-16T19:28:57.614 INFO:tasks.workunit.client.1.smithi117.stdout:3/214: truncate d1/dd/d3b/f3d 812655 0 2024-02-16T19:28:57.615 INFO:tasks.workunit.client.1.smithi117.stdout:8/200: dwrite d4/d12/f26 [4194304,4194304] 0 2024-02-16T19:28:57.615 INFO:tasks.workunit.client.1.smithi117.stdout:3/215: dread - d1/dd/d32/d3c/f3f zero size 2024-02-16T19:28:57.616 INFO:tasks.workunit.client.1.smithi117.stdout:0/268: creat d0/d9/d14/d23/d33/f5c x:0 0 0 2024-02-16T19:28:57.623 INFO:tasks.workunit.client.0.smithi022.stdout:0/331: sync 2024-02-16T19:28:57.623 INFO:tasks.workunit.client.0.smithi022.stdout:2/181: sync 2024-02-16T19:28:57.623 INFO:tasks.workunit.client.1.smithi117.stdout:5/298: creat d2/d5d/f5e x:0 0 0 2024-02-16T19:28:57.624 INFO:tasks.workunit.client.1.smithi117.stdout:5/299: dread - d2/d3/f4d zero size 2024-02-16T19:28:57.626 INFO:tasks.workunit.client.0.smithi022.stdout:1/344: creat d6/d8/d22/d47/d54/f62 x:0 0 0 2024-02-16T19:28:57.627 INFO:tasks.workunit.client.1.smithi117.stdout:5/300: dread d2/d4c/d54/f43 [0,4194304] 0 2024-02-16T19:28:57.627 INFO:tasks.workunit.client.1.smithi117.stdout:1/142: creat d2/d19/f2f x:0 0 0 2024-02-16T19:28:57.628 INFO:tasks.workunit.client.1.smithi117.stdout:1/143: chown d2/d4/f17 20241 1 2024-02-16T19:28:57.629 INFO:tasks.workunit.client.0.smithi022.stdout:8/249: creat d3/d12/d2b/d2e/f59 x:0 0 0 2024-02-16T19:28:57.631 INFO:tasks.workunit.client.0.smithi022.stdout:6/240: chown d1/d8/da/d17/d53/l5a 14596 1 2024-02-16T19:28:57.633 INFO:tasks.workunit.client.0.smithi022.stdout:3/216: link d7/d12/d18/f57 d7/d12/d18/dd/d24/d53/f5e 0 2024-02-16T19:28:57.634 INFO:tasks.workunit.client.0.smithi022.stdout:5/263: mknod dd/d1f/d48/d21/d2f/d40/d49/c54 0 2024-02-16T19:28:57.635 INFO:tasks.workunit.client.0.smithi022.stdout:8/250: dwrite d3/d12/d2b/f3a [0,4194304] 0 2024-02-16T19:28:57.637 INFO:tasks.workunit.client.1.smithi117.stdout:7/182: sync 2024-02-16T19:28:57.638 INFO:tasks.workunit.client.0.smithi022.stdout:6/241: dread d1/d8/ff [0,4194304] 0 2024-02-16T19:28:57.638 INFO:tasks.workunit.client.0.smithi022.stdout:7/241: symlink d9/d1b/d23/l4d 0 2024-02-16T19:28:57.641 INFO:tasks.workunit.client.1.smithi117.stdout:5/301: dread d2/d9/d3c/d24/f53 [0,4194304] 0 2024-02-16T19:28:57.641 INFO:tasks.workunit.client.0.smithi022.stdout:0/332: rename d10/c12 to d10/d11/c6f 0 2024-02-16T19:28:57.642 INFO:tasks.workunit.client.0.smithi022.stdout:0/333: chown d10/d11/d27/d3f/d58/d60/d62/l3b 232802 1 2024-02-16T19:28:57.643 INFO:tasks.workunit.client.0.smithi022.stdout:0/334: fsync d10/f6e 0 2024-02-16T19:28:57.643 INFO:tasks.workunit.client.0.smithi022.stdout:1/345: mkdir d6/d8/d22/d16/d17/d63 0 2024-02-16T19:28:57.645 INFO:tasks.workunit.client.0.smithi022.stdout:1/346: write d6/d8/d22/d16/f40 [5030059,116756] 0 2024-02-16T19:28:57.645 INFO:tasks.workunit.client.1.smithi117.stdout:2/165: rename d0/d3/c1b to d0/d3/d22/c33 0 2024-02-16T19:28:57.646 INFO:tasks.workunit.client.1.smithi117.stdout:2/166: chown d0/d3/d10/f16 1691357 1 2024-02-16T19:28:57.647 INFO:tasks.workunit.client.1.smithi117.stdout:8/201: write d4/d12/f1a [4378319,107470] 0 2024-02-16T19:28:57.648 INFO:tasks.workunit.client.0.smithi022.stdout:3/217: mknod d7/d12/d18/dd/d31/d41/c5f 0 2024-02-16T19:28:57.649 INFO:tasks.workunit.client.1.smithi117.stdout:3/216: symlink d1/d1b/d1e/l4b 0 2024-02-16T19:28:57.650 INFO:tasks.workunit.client.0.smithi022.stdout:5/264: mknod dd/d1f/d48/d21/d2f/d40/c55 0 2024-02-16T19:28:57.652 INFO:tasks.workunit.client.0.smithi022.stdout:8/251: mknod d3/d12/d2b/d2e/c5a 0 2024-02-16T19:28:57.652 INFO:tasks.workunit.client.1.smithi117.stdout:9/190: mknod dd/d23/d2e/d2c/d31/c37 0 2024-02-16T19:28:57.653 INFO:tasks.workunit.client.1.smithi117.stdout:8/202: dwrite d4/d12/f26 [4194304,4194304] 0 2024-02-16T19:28:57.654 INFO:tasks.workunit.client.0.smithi022.stdout:8/252: read d3/f10 [3059871,109597] 0 2024-02-16T19:28:57.656 INFO:tasks.workunit.client.0.smithi022.stdout:7/242: unlink d9/da/d1f/f31 0 2024-02-16T19:28:57.657 INFO:tasks.workunit.client.0.smithi022.stdout:5/265: dread dd/d1f/f31 [0,4194304] 0 2024-02-16T19:28:57.660 INFO:tasks.workunit.client.1.smithi117.stdout:1/144: chown d2/d1d/c22 1782 1 2024-02-16T19:28:57.661 INFO:tasks.workunit.client.1.smithi117.stdout:1/145: chown d2/d1d/d1f/f23 62513189 1 2024-02-16T19:28:57.662 INFO:tasks.workunit.client.1.smithi117.stdout:1/146: write d2/d1d/f25 [947372,14743] 0 2024-02-16T19:28:57.662 INFO:tasks.workunit.client.0.smithi022.stdout:0/335: mknod d10/d11/c70 0 2024-02-16T19:28:57.663 INFO:tasks.workunit.client.1.smithi117.stdout:6/222: getdents d5/d1d/d28/d32/d37 0 2024-02-16T19:28:57.664 INFO:tasks.workunit.client.1.smithi117.stdout:6/223: chown d5/c17 4167092 1 2024-02-16T19:28:57.666 INFO:tasks.workunit.client.0.smithi022.stdout:0/336: dread d10/d15/d23/f3c [0,4194304] 0 2024-02-16T19:28:57.669 INFO:tasks.workunit.client.0.smithi022.stdout:3/218: rename d7/d12/d18/dd/d24/f55 to d7/d12/d59/f60 0 2024-02-16T19:28:57.669 INFO:tasks.workunit.client.0.smithi022.stdout:3/219: chown d7/d12/d18/d4b/f4d 11667 1 2024-02-16T19:28:57.670 INFO:tasks.workunit.client.0.smithi022.stdout:3/220: stat d7/d12/d18/dd/d24/c4a 0 2024-02-16T19:28:57.671 INFO:tasks.workunit.client.0.smithi022.stdout:3/221: chown d7/d12/d18/dd/d24/d53/f5e 14220817 1 2024-02-16T19:28:57.677 INFO:tasks.workunit.client.1.smithi117.stdout:4/214: rename d0/l3c to d0/da/d21/d3d/d28/d2f/l45 0 2024-02-16T19:28:57.680 INFO:tasks.workunit.client.1.smithi117.stdout:2/167: mkdir d0/d3/d34 0 2024-02-16T19:28:57.680 INFO:tasks.workunit.client.1.smithi117.stdout:2/168: stat d0/d3/d22 0 2024-02-16T19:28:57.681 INFO:tasks.workunit.client.0.smithi022.stdout:8/253: creat d3/d6/f5b x:0 0 0 2024-02-16T19:28:57.682 INFO:tasks.workunit.client.0.smithi022.stdout:8/254: chown d3/d6/l30 813870 1 2024-02-16T19:28:57.685 INFO:tasks.workunit.client.1.smithi117.stdout:5/302: write d2/d9/d3c/d24/f26 [3999951,87605] 0 2024-02-16T19:28:57.685 INFO:tasks.workunit.client.0.smithi022.stdout:6/242: write d1/d8/fb [1080927,119524] 0 2024-02-16T19:28:57.686 INFO:tasks.workunit.client.0.smithi022.stdout:2/182: creat d0/d2/f39 x:0 0 0 2024-02-16T19:28:57.695 INFO:tasks.workunit.client.0.smithi022.stdout:3/222: creat d7/f61 x:0 0 0 2024-02-16T19:28:57.698 INFO:tasks.workunit.client.1.smithi117.stdout:6/224: rename d5/f24 to d5/d1d/f50 0 2024-02-16T19:28:57.698 INFO:tasks.workunit.client.0.smithi022.stdout:3/223: dread d7/d12/f25 [0,4194304] 0 2024-02-16T19:28:57.699 INFO:tasks.workunit.client.1.smithi117.stdout:6/225: truncate d5/d1d/f27 4903526 0 2024-02-16T19:28:57.701 INFO:tasks.workunit.client.0.smithi022.stdout:8/255: rename d3/f10 to d3/d25/f5c 0 2024-02-16T19:28:57.702 INFO:tasks.workunit.client.1.smithi117.stdout:2/169: stat d0/d13/f1c 0 2024-02-16T19:28:57.703 INFO:tasks.workunit.client.0.smithi022.stdout:7/243: mkdir d9/da/d4e 0 2024-02-16T19:28:57.704 INFO:tasks.workunit.client.1.smithi117.stdout:3/217: mknod d1/dd/d3e/c4c 0 2024-02-16T19:28:57.704 INFO:tasks.workunit.client.0.smithi022.stdout:5/266: mkdir dd/d1f/d48/d21/d2f/d40/d4e/d56 0 2024-02-16T19:28:57.705 INFO:tasks.workunit.client.1.smithi117.stdout:0/269: symlink d0/d9/d22/d27/d31/l5d 0 2024-02-16T19:28:57.707 INFO:tasks.workunit.client.0.smithi022.stdout:8/256: dwrite d3/d12/d2b/f3c [0,4194304] 0 2024-02-16T19:28:57.708 INFO:tasks.workunit.client.1.smithi117.stdout:2/170: dwrite d0/f12 [0,4194304] 0 2024-02-16T19:28:57.711 INFO:tasks.workunit.client.1.smithi117.stdout:8/203: mkdir d4/d7/d37/d3a/d43 0 2024-02-16T19:28:57.717 INFO:tasks.workunit.client.1.smithi117.stdout:7/183: creat d4/f4d x:0 0 0 2024-02-16T19:28:57.722 INFO:tasks.workunit.client.0.smithi022.stdout:7/244: read d9/f25 [2541906,77387] 0 2024-02-16T19:28:57.726 INFO:tasks.workunit.client.1.smithi117.stdout:9/191: dwrite dd/d23/f1f [0,4194304] 0 2024-02-16T19:28:57.726 INFO:tasks.workunit.client.0.smithi022.stdout:2/183: symlink d0/d32/l3a 0 2024-02-16T19:28:57.726 INFO:tasks.workunit.client.0.smithi022.stdout:0/337: write d10/d15/f61 [855584,127965] 0 2024-02-16T19:28:57.727 INFO:tasks.workunit.client.1.smithi117.stdout:9/192: chown dd/l1d 252336852 1 2024-02-16T19:28:57.728 INFO:tasks.workunit.client.0.smithi022.stdout:0/338: chown d10/d11/d27/d3f/f57 112638290 1 2024-02-16T19:28:57.730 INFO:tasks.workunit.client.1.smithi117.stdout:3/218: readlink d1/l45 0 2024-02-16T19:28:57.735 INFO:tasks.workunit.client.1.smithi117.stdout:0/270: symlink d0/d9/d22/d26/l5e 0 2024-02-16T19:28:57.737 INFO:tasks.workunit.client.1.smithi117.stdout:2/171: rmdir d0/d3 39 2024-02-16T19:28:57.741 INFO:tasks.workunit.client.1.smithi117.stdout:1/147: link d2/d4/l28 d2/d1d/d1f/l30 0 2024-02-16T19:28:57.742 INFO:tasks.workunit.client.1.smithi117.stdout:4/215: link d0/da/c43 d0/da/d21/d3d/d28/d2f/c46 0 2024-02-16T19:28:57.744 INFO:tasks.workunit.client.1.smithi117.stdout:9/193: rmdir dd/d23/d2e/d2c 39 2024-02-16T19:28:57.745 INFO:tasks.workunit.client.0.smithi022.stdout:4/229: sync 2024-02-16T19:28:57.745 INFO:tasks.workunit.client.0.smithi022.stdout:9/256: sync 2024-02-16T19:28:57.747 INFO:tasks.workunit.client.0.smithi022.stdout:8/257: creat d3/d25/f5d x:0 0 0 2024-02-16T19:28:57.747 INFO:tasks.workunit.client.1.smithi117.stdout:5/303: truncate d2/d1d/f3f 662940 0 2024-02-16T19:28:57.748 INFO:tasks.workunit.client.0.smithi022.stdout:3/224: creat d7/d12/d18/dd/d24/d53/f62 x:0 0 0 2024-02-16T19:28:57.748 INFO:tasks.workunit.client.0.smithi022.stdout:2/184: rename d0/d2/d18/d1f/d36 to d0/d32/d3b 0 2024-02-16T19:28:57.748 INFO:tasks.workunit.client.1.smithi117.stdout:6/226: dwrite d5/d1d/d28/d32/f3f [0,4194304] 0 2024-02-16T19:28:57.749 INFO:tasks.workunit.client.1.smithi117.stdout:2/172: fsync d0/d3/f2a 0 2024-02-16T19:28:57.750 INFO:tasks.workunit.client.0.smithi022.stdout:6/243: link d1/d8/l19 d1/d8/da/d17/d53/l5e 0 2024-02-16T19:28:57.750 INFO:tasks.workunit.client.1.smithi117.stdout:6/227: truncate d5/d1d/f3d 546598 0 2024-02-16T19:28:57.750 INFO:tasks.workunit.client.1.smithi117.stdout:6/228: readlink d5/db/l3e 0 2024-02-16T19:28:57.751 INFO:tasks.workunit.client.1.smithi117.stdout:8/204: mkdir d4/d7/d37/d3a/d43/d44 0 2024-02-16T19:28:57.753 INFO:tasks.workunit.client.1.smithi117.stdout:2/173: dread d0/f12 [0,4194304] 0 2024-02-16T19:28:57.755 INFO:tasks.workunit.client.1.smithi117.stdout:7/184: mknod d4/d1c/d39/d46/d47/c4e 0 2024-02-16T19:28:57.755 INFO:tasks.workunit.client.1.smithi117.stdout:7/185: read d4/f23 [174251,70535] 0 2024-02-16T19:28:57.758 INFO:tasks.workunit.client.1.smithi117.stdout:1/148: chown d2/d4/c1c 48967 1 2024-02-16T19:28:57.760 INFO:tasks.workunit.client.1.smithi117.stdout:1/149: read d2/d18/d20/f2a [14125,5590] 0 2024-02-16T19:28:57.761 INFO:tasks.workunit.client.1.smithi117.stdout:1/150: chown d2/d4 37281 1 2024-02-16T19:28:57.761 INFO:tasks.workunit.client.0.smithi022.stdout:0/339: mknod d10/d11/d27/d3f/d58/d60/d62/c71 0 2024-02-16T19:28:57.761 INFO:tasks.workunit.client.1.smithi117.stdout:3/219: dwrite d1/fa [0,4194304] 0 2024-02-16T19:28:57.762 INFO:tasks.workunit.client.0.smithi022.stdout:4/230: dwrite d4/d6/dd/df/d10/f19 [0,4194304] 0 2024-02-16T19:28:57.762 INFO:tasks.workunit.client.1.smithi117.stdout:4/216: unlink d0/da/f2c 0 2024-02-16T19:28:57.763 INFO:tasks.workunit.client.0.smithi022.stdout:2/185: dwrite d0/d2/d18/f34 [0,4194304] 0 2024-02-16T19:28:57.765 INFO:tasks.workunit.client.0.smithi022.stdout:7/245: dread d9/da/d1f/f22 [0,4194304] 0 2024-02-16T19:28:57.765 INFO:tasks.workunit.client.0.smithi022.stdout:6/244: dwrite d1/f32 [0,4194304] 0 2024-02-16T19:28:57.768 INFO:tasks.workunit.client.0.smithi022.stdout:0/340: write d10/d11/d50/f3e [6999332,21138] 0 2024-02-16T19:28:57.769 INFO:tasks.workunit.client.0.smithi022.stdout:8/258: rename d3/d6/f5b to d3/d6/f5e 0 2024-02-16T19:28:57.770 INFO:tasks.workunit.client.0.smithi022.stdout:0/341: chown d10/d15/f5e 126944 1 2024-02-16T19:28:57.774 INFO:tasks.workunit.client.0.smithi022.stdout:0/342: write d10/d11/d50/f3e [8813857,78352] 0 2024-02-16T19:28:57.775 INFO:tasks.workunit.client.1.smithi117.stdout:6/229: creat d5/d1d/d28/d32/f51 x:0 0 0 2024-02-16T19:28:57.778 INFO:tasks.workunit.client.0.smithi022.stdout:4/231: dwrite d4/d6/dd/d22/d30/f44 [0,4194304] 0 2024-02-16T19:28:57.779 INFO:tasks.workunit.client.1.smithi117.stdout:3/220: dwrite d1/dd/f43 [0,4194304] 0 2024-02-16T19:28:57.779 INFO:tasks.workunit.client.0.smithi022.stdout:4/232: chown d4/d38/c43 798306 1 2024-02-16T19:28:57.779 INFO:tasks.workunit.client.1.smithi117.stdout:9/194: dwrite fb [4194304,4194304] 0 2024-02-16T19:28:57.784 INFO:tasks.workunit.client.1.smithi117.stdout:2/174: dread d0/d3/d22/f28 [0,4194304] 0 2024-02-16T19:28:57.789 INFO:tasks.workunit.client.0.smithi022.stdout:0/343: dwrite d10/d15/d23/d38/f41 [0,4194304] 0 2024-02-16T19:28:57.793 INFO:tasks.workunit.client.1.smithi117.stdout:3/221: dwrite d1/dd/d3b/f3d [0,4194304] 0 2024-02-16T19:28:57.795 INFO:tasks.workunit.client.1.smithi117.stdout:3/222: write d1/f28 [876022,44379] 0 2024-02-16T19:28:57.796 INFO:tasks.workunit.client.1.smithi117.stdout:3/223: chown d1/d1b/d20 50 1 2024-02-16T19:28:57.799 INFO:tasks.workunit.client.1.smithi117.stdout:0/271: link d0/d9/d22/d27/d31/f58 d0/d9/d22/d27/f5f 0 2024-02-16T19:28:57.800 INFO:tasks.workunit.client.0.smithi022.stdout:9/257: write db/d18/f26 [770915,113394] 0 2024-02-16T19:28:57.803 INFO:tasks.workunit.client.1.smithi117.stdout:4/217: rename d0/da/d21/d3d/d2d/f36 to d0/f47 0 2024-02-16T19:28:57.804 INFO:tasks.workunit.client.1.smithi117.stdout:4/218: chown d0/da/d21/d3d/d2d/f41 3098 1 2024-02-16T19:28:57.808 INFO:tasks.workunit.client.0.smithi022.stdout:6/245: dread d1/d24/f36 [0,4194304] 0 2024-02-16T19:28:57.811 INFO:tasks.workunit.client.1.smithi117.stdout:7/186: dread d4/fd [0,4194304] 0 2024-02-16T19:28:57.811 INFO:tasks.workunit.client.0.smithi022.stdout:4/233: unlink d4/d6/dd/df/c33 0 2024-02-16T19:28:57.812 INFO:tasks.workunit.client.1.smithi117.stdout:7/187: chown d4/d1c/l33 112113717 1 2024-02-16T19:28:57.815 INFO:tasks.workunit.client.0.smithi022.stdout:7/246: mknod d9/c4f 0 2024-02-16T19:28:57.818 INFO:tasks.workunit.client.0.smithi022.stdout:0/344: creat d10/d22/d2c/f72 x:0 0 0 2024-02-16T19:28:57.819 INFO:tasks.workunit.client.1.smithi117.stdout:5/304: write d2/d3/d21/f33 [1713215,113617] 0 2024-02-16T19:28:57.819 INFO:tasks.workunit.client.1.smithi117.stdout:5/305: dread - d2/d3/f4d zero size 2024-02-16T19:28:57.820 INFO:tasks.workunit.client.0.smithi022.stdout:3/225: creat d7/d12/d18/dd/d24/f63 x:0 0 0 2024-02-16T19:28:57.820 INFO:tasks.workunit.client.1.smithi117.stdout:5/306: stat d2/d5d 0 2024-02-16T19:28:57.820 INFO:tasks.workunit.client.1.smithi117.stdout:5/307: stat d2/d9/l15 0 2024-02-16T19:28:57.821 INFO:tasks.workunit.client.1.smithi117.stdout:5/308: read - d2/d1d/f48 zero size 2024-02-16T19:28:57.821 INFO:tasks.workunit.client.0.smithi022.stdout:8/259: symlink d3/d12/d2b/l5f 0 2024-02-16T19:28:57.823 INFO:tasks.workunit.client.0.smithi022.stdout:8/260: fdatasync d3/d12/d2b/d2e/f59 0 2024-02-16T19:28:57.824 INFO:tasks.workunit.client.1.smithi117.stdout:8/205: truncate d4/d12/f1a 3635787 0 2024-02-16T19:28:57.825 INFO:tasks.workunit.client.0.smithi022.stdout:0/345: dwrite d10/d22/f20 [0,4194304] 0 2024-02-16T19:28:57.826 INFO:tasks.workunit.client.0.smithi022.stdout:0/346: read - d10/d22/f2a zero size 2024-02-16T19:28:57.829 INFO:tasks.workunit.client.0.smithi022.stdout:6/246: creat d1/d24/f5f x:0 0 0 2024-02-16T19:28:57.830 INFO:tasks.workunit.client.0.smithi022.stdout:4/234: rmdir d4/d6/dd/d22 39 2024-02-16T19:28:57.831 INFO:tasks.workunit.client.1.smithi117.stdout:2/175: dwrite d0/d3/d22/f28 [0,4194304] 0 2024-02-16T19:28:57.833 INFO:tasks.workunit.client.0.smithi022.stdout:8/261: dwrite d3/d25/f5d [0,4194304] 0 2024-02-16T19:28:57.833 INFO:tasks.workunit.client.0.smithi022.stdout:6/247: chown d1/d8/d3c/f43 5931 1 2024-02-16T19:28:57.833 INFO:tasks.workunit.client.1.smithi117.stdout:3/224: unlink d1/d1b/d1e/f42 0 2024-02-16T19:28:57.834 INFO:tasks.workunit.client.1.smithi117.stdout:3/225: stat d1/fa 0 2024-02-16T19:28:57.838 INFO:tasks.workunit.client.0.smithi022.stdout:8/262: write d3/d12/f51 [98501,91827] 0 2024-02-16T19:28:57.839 INFO:tasks.workunit.client.0.smithi022.stdout:0/347: dwrite d10/d11/d27/d3f/d58/f6a [0,4194304] 0 2024-02-16T19:28:57.841 INFO:tasks.workunit.client.1.smithi117.stdout:0/272: rename d0/d9/d2c/d37 to d0/d9/d14/d1d/d60 0 2024-02-16T19:28:57.842 INFO:tasks.workunit.client.1.smithi117.stdout:0/273: chown d0/d9/d22/d26/l57 2 1 2024-02-16T19:28:57.844 INFO:tasks.workunit.client.0.smithi022.stdout:7/247: dread d9/da/d2c/d2f/f17 [0,4194304] 0 2024-02-16T19:28:57.851 INFO:tasks.workunit.client.1.smithi117.stdout:6/230: getdents d5/d1d/d28/d32/d37 0 2024-02-16T19:28:57.853 INFO:tasks.workunit.client.1.smithi117.stdout:9/195: mknod dd/d23/c38 0 2024-02-16T19:28:57.855 INFO:tasks.workunit.client.1.smithi117.stdout:7/188: dwrite d4/d13/f16 [4194304,4194304] 0 2024-02-16T19:28:57.858 INFO:tasks.workunit.client.0.smithi022.stdout:9/258: truncate db/d14/d19/f17 3062010 0 2024-02-16T19:28:57.859 INFO:tasks.workunit.client.0.smithi022.stdout:6/248: rmdir d1/d8/d3c/d4d 39 2024-02-16T19:28:57.863 INFO:tasks.workunit.client.1.smithi117.stdout:1/151: creat d2/f31 x:0 0 0 2024-02-16T19:28:57.865 INFO:tasks.workunit.client.0.smithi022.stdout:0/348: creat d10/d11/f73 x:0 0 0 2024-02-16T19:28:57.870 INFO:tasks.workunit.client.1.smithi117.stdout:5/309: dwrite d2/d9/d3c/d24/d50/d52/f1e [0,4194304] 0 2024-02-16T19:28:57.880 INFO:tasks.workunit.client.1.smithi117.stdout:0/274: dwrite d0/d9/d14/f1b [0,4194304] 0 2024-02-16T19:28:57.880 INFO:tasks.workunit.client.1.smithi117.stdout:2/176: dwrite d0/d29/f2e [0,4194304] 0 2024-02-16T19:28:57.882 INFO:tasks.workunit.client.1.smithi117.stdout:8/206: mknod d4/d7/c45 0 2024-02-16T19:28:57.884 INFO:tasks.workunit.client.1.smithi117.stdout:8/207: write d4/d12/d31/f40 [1125999,10488] 0 2024-02-16T19:28:57.885 INFO:tasks.workunit.client.1.smithi117.stdout:8/208: stat d4/f3c 0 2024-02-16T19:28:57.886 INFO:tasks.workunit.client.1.smithi117.stdout:8/209: chown d4/d1c/l30 36 1 2024-02-16T19:28:57.887 INFO:tasks.workunit.client.0.smithi022.stdout:9/259: link db/d14/f2d db/d14/f5e 0 2024-02-16T19:28:57.891 INFO:tasks.workunit.client.0.smithi022.stdout:6/249: creat d1/d8/da/d17/d53/d54/d57/f60 x:0 0 0 2024-02-16T19:28:57.895 INFO:tasks.workunit.client.0.smithi022.stdout:3/226: getdents d7/d12/d18/dd/d24 0 2024-02-16T19:28:57.896 INFO:tasks.workunit.client.1.smithi117.stdout:5/310: dread d2/d3/d21/f33 [0,4194304] 0 2024-02-16T19:28:57.898 INFO:tasks.workunit.client.0.smithi022.stdout:9/260: dwrite db/d14/d33/f48 [0,4194304] 0 2024-02-16T19:28:57.899 INFO:tasks.workunit.client.1.smithi117.stdout:9/196: rmdir dd/d23/d2e/d2c/d31 39 2024-02-16T19:28:57.899 INFO:tasks.workunit.client.0.smithi022.stdout:0/349: dread d10/d11/d27/d3f/d58/d60/d62/f4f [0,4194304] 0 2024-02-16T19:28:57.899 INFO:tasks.workunit.client.0.smithi022.stdout:9/261: write db/fc [586619,75793] 0 2024-02-16T19:28:57.900 INFO:tasks.workunit.client.0.smithi022.stdout:0/350: chown d10/d22/f2d 2028994981 1 2024-02-16T19:28:57.902 INFO:tasks.workunit.client.0.smithi022.stdout:9/262: write f7 [1683027,106996] 0 2024-02-16T19:28:57.902 INFO:tasks.workunit.client.1.smithi117.stdout:1/152: dwrite d2/d4/f11 [0,4194304] 0 2024-02-16T19:28:57.906 INFO:tasks.workunit.client.0.smithi022.stdout:4/235: getdents d4/d6 0 2024-02-16T19:28:57.907 INFO:tasks.workunit.client.0.smithi022.stdout:4/236: write d4/d6/dd/f1e [245975,97398] 0 2024-02-16T19:28:57.908 INFO:tasks.workunit.client.1.smithi117.stdout:3/226: write d1/d1b/f2d [1014592,51697] 0 2024-02-16T19:28:57.908 INFO:tasks.workunit.client.1.smithi117.stdout:3/227: chown d1 116152 1 2024-02-16T19:28:57.909 INFO:tasks.workunit.client.1.smithi117.stdout:3/228: chown d1/d1b/d1e 16274287 1 2024-02-16T19:28:57.911 INFO:tasks.workunit.client.1.smithi117.stdout:2/177: truncate d0/f1e 1950291 0 2024-02-16T19:28:57.914 INFO:tasks.workunit.client.0.smithi022.stdout:6/250: rename d1/d8/da/d17/d53/d54 to d1/d8/da/d29/d61 0 2024-02-16T19:28:57.917 INFO:tasks.workunit.client.0.smithi022.stdout:3/227: symlink d7/l64 0 2024-02-16T19:28:57.917 INFO:tasks.workunit.client.1.smithi117.stdout:6/231: truncate d5/d1d/d28/d32/f3f 376712 0 2024-02-16T19:28:57.918 INFO:tasks.workunit.client.0.smithi022.stdout:3/228: chown d7/d12/d18/dd/f28 3622 1 2024-02-16T19:28:57.918 INFO:tasks.workunit.client.0.smithi022.stdout:3/229: chown d7/d12/d18/dd/f2e 7056 1 2024-02-16T19:28:57.919 INFO:tasks.workunit.client.1.smithi117.stdout:7/189: truncate d4/d1c/f3c 7896283 0 2024-02-16T19:28:57.920 INFO:tasks.workunit.client.0.smithi022.stdout:8/263: link d3/d12/d17/l19 d3/d12/d2b/d54/l60 0 2024-02-16T19:28:57.921 INFO:tasks.workunit.client.1.smithi117.stdout:4/219: sync 2024-02-16T19:28:57.921 INFO:tasks.workunit.client.0.smithi022.stdout:0/351: creat d10/d11/d50/d65/f74 x:0 0 0 2024-02-16T19:28:57.923 INFO:tasks.workunit.client.0.smithi022.stdout:9/263: creat db/d14/d19/f5f x:0 0 0 2024-02-16T19:28:57.923 INFO:tasks.workunit.client.1.smithi117.stdout:5/311: creat d2/d1d/f5f x:0 0 0 2024-02-16T19:28:57.924 INFO:tasks.workunit.client.0.smithi022.stdout:7/248: truncate d9/d1b/f27 1497600 0 2024-02-16T19:28:57.924 INFO:tasks.workunit.client.1.smithi117.stdout:5/312: chown d2/d9/d3c/d24 6 1 2024-02-16T19:28:57.927 INFO:tasks.workunit.client.0.smithi022.stdout:4/237: creat d4/d6/dd/df/d10/f4f x:0 0 0 2024-02-16T19:28:57.928 INFO:tasks.workunit.client.1.smithi117.stdout:1/153: creat d2/d18/d20/f32 x:0 0 0 2024-02-16T19:28:57.930 INFO:tasks.workunit.client.1.smithi117.stdout:0/275: creat d0/d9/d22/d27/d4e/f61 x:0 0 0 2024-02-16T19:28:57.931 INFO:tasks.workunit.client.1.smithi117.stdout:0/276: write d0/d9/d22/d26/f36 [10730,36555] 0 2024-02-16T19:28:57.931 INFO:tasks.workunit.client.1.smithi117.stdout:3/229: symlink d1/d1b/l4d 0 2024-02-16T19:28:57.932 INFO:tasks.workunit.client.1.smithi117.stdout:0/277: fdatasync d0/d9/d14/d23/f24 0 2024-02-16T19:28:57.932 INFO:tasks.workunit.client.1.smithi117.stdout:3/230: fdatasync d1/d1b/f21 0 2024-02-16T19:28:57.933 INFO:tasks.workunit.client.0.smithi022.stdout:8/264: creat d3/d25/f61 x:0 0 0 2024-02-16T19:28:57.933 INFO:tasks.workunit.client.0.smithi022.stdout:1/347: sync 2024-02-16T19:28:57.934 INFO:tasks.workunit.client.0.smithi022.stdout:8/265: readlink d3/l4b 0 2024-02-16T19:28:57.935 INFO:tasks.workunit.client.0.smithi022.stdout:0/352: creat d10/d11/d27/d3f/f75 x:0 0 0 2024-02-16T19:28:57.936 INFO:tasks.workunit.client.1.smithi117.stdout:0/278: write d0/d9/d22/d27/d31/f38 [316069,16782] 0 2024-02-16T19:28:57.937 INFO:tasks.workunit.client.0.smithi022.stdout:9/264: chown db/d14/d16/l5b 6980940 1 2024-02-16T19:28:57.941 INFO:tasks.workunit.client.1.smithi117.stdout:3/231: dwrite d1/d31/f36 [0,4194304] 0 2024-02-16T19:28:57.941 INFO:tasks.workunit.client.1.smithi117.stdout:3/232: chown d1/dd/d3b 1133 1 2024-02-16T19:28:57.944 INFO:tasks.workunit.client.1.smithi117.stdout:4/220: mkdir d0/d6/d48 0 2024-02-16T19:28:57.945 INFO:tasks.workunit.client.1.smithi117.stdout:5/313: symlink d2/d3/d21/d2f/d30/d5a/l60 0 2024-02-16T19:28:57.947 INFO:tasks.workunit.client.0.smithi022.stdout:3/230: link d7/f1c d7/d12/d18/dd/d3c/f65 0 2024-02-16T19:28:57.948 INFO:tasks.workunit.client.0.smithi022.stdout:2/186: sync 2024-02-16T19:28:57.949 INFO:tasks.workunit.client.0.smithi022.stdout:5/267: sync 2024-02-16T19:28:57.949 INFO:tasks.workunit.client.0.smithi022.stdout:2/187: chown d0/d32/d3b 50 1 2024-02-16T19:28:57.949 INFO:tasks.workunit.client.1.smithi117.stdout:1/154: symlink d2/d18/d20/l33 0 2024-02-16T19:28:57.950 INFO:tasks.workunit.client.1.smithi117.stdout:9/197: dwrite dd/d2d/f35 [0,4194304] 0 2024-02-16T19:28:57.950 INFO:tasks.workunit.client.1.smithi117.stdout:1/155: dread - d2/d19/f2f zero size 2024-02-16T19:28:57.950 INFO:tasks.workunit.client.1.smithi117.stdout:9/198: read dd/d23/f1f [4087162,117251] 0 2024-02-16T19:28:57.951 INFO:tasks.workunit.client.0.smithi022.stdout:3/231: dread d7/f35 [0,4194304] 0 2024-02-16T19:28:57.951 INFO:tasks.workunit.client.0.smithi022.stdout:3/232: dread - d7/d12/d18/dd/d3c/f49 zero size 2024-02-16T19:28:57.953 INFO:tasks.workunit.client.0.smithi022.stdout:0/353: rmdir d10/d11/d27/d3f/d58 39 2024-02-16T19:28:57.954 INFO:tasks.workunit.client.0.smithi022.stdout:0/354: read d10/f17 [479156,2382] 0 2024-02-16T19:28:57.955 INFO:tasks.workunit.client.0.smithi022.stdout:0/355: chown d10/d15/d23/d38/d5a/d66 842 1 2024-02-16T19:28:57.957 INFO:tasks.workunit.client.0.smithi022.stdout:0/356: write d10/d11/d50/d65/f68 [533050,105065] 0 2024-02-16T19:28:57.960 INFO:tasks.workunit.client.0.smithi022.stdout:7/249: symlink d9/da/l50 0 2024-02-16T19:28:57.963 INFO:tasks.workunit.client.1.smithi117.stdout:7/190: mknod d4/c4f 0 2024-02-16T19:28:57.965 INFO:tasks.workunit.client.0.smithi022.stdout:4/238: dwrite d4/d6/dd/df/f29 [0,4194304] 0 2024-02-16T19:28:57.966 INFO:tasks.workunit.client.0.smithi022.stdout:6/251: rename d1/d8/da/f55 to d1/d8/da/d17/f62 0 2024-02-16T19:28:57.967 INFO:tasks.workunit.client.0.smithi022.stdout:7/250: dread d9/da/d2c/d2f/d21/f48 [0,4194304] 0 2024-02-16T19:28:57.967 INFO:tasks.workunit.client.0.smithi022.stdout:6/252: fdatasync d1/d8/f11 0 2024-02-16T19:28:57.967 INFO:tasks.workunit.client.1.smithi117.stdout:0/279: rmdir d0/d9/d22/d26 39 2024-02-16T19:28:57.968 INFO:tasks.workunit.client.0.smithi022.stdout:6/253: chown d1/d8/d23/f30 773038 1 2024-02-16T19:28:57.970 INFO:tasks.workunit.client.1.smithi117.stdout:8/210: link d4/d12/l13 d4/d7/d37/d3a/d43/d44/l46 0 2024-02-16T19:28:57.972 INFO:tasks.workunit.client.0.smithi022.stdout:8/266: mknod d3/c62 0 2024-02-16T19:28:57.973 INFO:tasks.workunit.client.0.smithi022.stdout:6/254: dread d1/f2 [0,4194304] 0 2024-02-16T19:28:57.974 INFO:tasks.workunit.client.0.smithi022.stdout:6/255: read d1/d8/ff [4115596,97369] 0 2024-02-16T19:28:57.975 INFO:tasks.workunit.client.0.smithi022.stdout:6/256: write d1/d8/fb [271975,29522] 0 2024-02-16T19:28:57.978 INFO:tasks.workunit.client.0.smithi022.stdout:3/233: mknod d7/d12/d18/dd/d24/d29/c66 0 2024-02-16T19:28:57.982 INFO:tasks.workunit.client.1.smithi117.stdout:5/314: dwrite d2/d3/d21/f33 [0,4194304] 0 2024-02-16T19:28:57.983 INFO:tasks.workunit.client.1.smithi117.stdout:9/199: creat dd/d24/f39 x:0 0 0 2024-02-16T19:28:57.986 INFO:tasks.workunit.client.1.smithi117.stdout:4/221: write d0/da/d21/f22 [5042897,49658] 0 2024-02-16T19:28:57.987 INFO:tasks.workunit.client.1.smithi117.stdout:6/232: mkdir d5/d4b/d44/d52 0 2024-02-16T19:28:57.987 INFO:tasks.workunit.client.1.smithi117.stdout:4/222: truncate d0/d6/f3b 645492 0 2024-02-16T19:28:57.992 INFO:tasks.workunit.client.1.smithi117.stdout:0/280: creat d0/d9/d22/d27/d4e/f62 x:0 0 0 2024-02-16T19:28:57.993 INFO:tasks.workunit.client.1.smithi117.stdout:8/211: mknod d4/d12/d2a/c47 0 2024-02-16T19:28:57.995 INFO:tasks.workunit.client.0.smithi022.stdout:0/357: rename d10/d15/f5e to d10/d22/d49/f76 0 2024-02-16T19:28:57.996 INFO:tasks.workunit.client.1.smithi117.stdout:3/233: mknod d1/dd/c4e 0 2024-02-16T19:28:57.996 INFO:tasks.workunit.client.0.smithi022.stdout:7/251: mknod d9/da/d1f/d44/c51 0 2024-02-16T19:28:57.998 INFO:tasks.workunit.client.0.smithi022.stdout:9/265: dread db/d14/d16/d21/d37/f4e [0,4194304] 0 2024-02-16T19:28:58.000 INFO:tasks.workunit.client.1.smithi117.stdout:5/315: mknod d2/d3/d21/d2f/d30/d38/c61 0 2024-02-16T19:28:58.000 INFO:tasks.workunit.client.0.smithi022.stdout:8/267: creat d3/d6/f63 x:0 0 0 2024-02-16T19:28:58.000 INFO:tasks.workunit.client.1.smithi117.stdout:5/316: dread - d2/d1d/f5f zero size 2024-02-16T19:28:58.003 INFO:tasks.workunit.client.0.smithi022.stdout:5/268: symlink dd/d1f/d2e/l57 0 2024-02-16T19:28:58.005 INFO:tasks.workunit.client.1.smithi117.stdout:5/317: dwrite d2/d1d/f48 [0,4194304] 0 2024-02-16T19:28:58.008 INFO:tasks.workunit.client.1.smithi117.stdout:4/223: unlink d0/c4 0 2024-02-16T19:28:58.011 INFO:tasks.workunit.client.1.smithi117.stdout:5/318: dread d2/d9/d3c/f1a [0,4194304] 0 2024-02-16T19:28:58.011 INFO:tasks.workunit.client.0.smithi022.stdout:6/257: truncate d1/d8/ff 3867779 0 2024-02-16T19:28:58.012 INFO:tasks.workunit.client.0.smithi022.stdout:6/258: chown d1/d8/da/d17/d53/l5a 1 1 2024-02-16T19:28:58.013 INFO:tasks.workunit.client.1.smithi117.stdout:0/281: creat d0/f63 x:0 0 0 2024-02-16T19:28:58.013 INFO:tasks.workunit.client.1.smithi117.stdout:1/156: truncate d2/d1d/d1f/f23 3144546 0 2024-02-16T19:28:58.016 INFO:tasks.workunit.client.1.smithi117.stdout:8/212: creat d4/d12/d2a/f48 x:0 0 0 2024-02-16T19:28:58.016 INFO:tasks.workunit.client.1.smithi117.stdout:6/233: write d5/f1a [412804,65963] 0 2024-02-16T19:28:58.017 INFO:tasks.workunit.client.0.smithi022.stdout:6/259: dread d1/d8/da/d17/d48/d51/f59 [0,4194304] 0 2024-02-16T19:28:58.017 INFO:tasks.workunit.client.1.smithi117.stdout:5/319: dwrite d2/d9/d3c/d24/f26 [4194304,4194304] 0 2024-02-16T19:28:58.017 INFO:tasks.workunit.client.0.smithi022.stdout:6/260: stat d1/d8/da/d17/d53 0 2024-02-16T19:28:58.018 INFO:tasks.workunit.client.1.smithi117.stdout:6/234: readlink d5/db/l3e 0 2024-02-16T19:28:58.018 INFO:tasks.workunit.client.0.smithi022.stdout:6/261: readlink d1/d8/da/l3d 0 2024-02-16T19:28:58.019 INFO:tasks.workunit.client.1.smithi117.stdout:5/320: truncate d2/d3/f35 74259 0 2024-02-16T19:28:58.022 INFO:tasks.workunit.client.1.smithi117.stdout:7/191: truncate f2 2883628 0 2024-02-16T19:28:58.026 INFO:tasks.workunit.client.1.smithi117.stdout:4/224: symlink d0/l49 0 2024-02-16T19:28:58.026 INFO:tasks.workunit.client.0.smithi022.stdout:3/234: write d7/d12/d18/dd/f20 [2390372,51511] 0 2024-02-16T19:28:58.027 INFO:tasks.workunit.client.1.smithi117.stdout:4/225: dread - d0/da/d21/f3e zero size 2024-02-16T19:28:58.027 INFO:tasks.workunit.client.1.smithi117.stdout:7/192: dread d4/fd [0,4194304] 0 2024-02-16T19:28:58.029 INFO:tasks.workunit.client.0.smithi022.stdout:0/358: dread d10/d15/d23/f3c [0,4194304] 0 2024-02-16T19:28:58.030 INFO:tasks.workunit.client.0.smithi022.stdout:0/359: readlink d10/l32 0 2024-02-16T19:28:58.030 INFO:tasks.workunit.client.1.smithi117.stdout:0/282: creat d0/d30/f64 x:0 0 0 2024-02-16T19:28:58.032 INFO:tasks.workunit.client.1.smithi117.stdout:8/213: chown d4/l16 7127544 1 2024-02-16T19:28:58.034 INFO:tasks.workunit.client.1.smithi117.stdout:6/235: chown d5/d4b/d20/d2d/l4e 0 1 2024-02-16T19:28:58.035 INFO:tasks.workunit.client.1.smithi117.stdout:6/236: truncate d5/d4b/d20/f4d 130442 0 2024-02-16T19:28:58.035 INFO:tasks.workunit.client.1.smithi117.stdout:6/237: chown d5/l10 12099 1 2024-02-16T19:28:58.036 INFO:tasks.workunit.client.0.smithi022.stdout:8/268: write d3/d25/f43 [1356899,14196] 0 2024-02-16T19:28:58.038 INFO:tasks.workunit.client.0.smithi022.stdout:2/188: link d0/d2/f1c d0/d2/f3c 0 2024-02-16T19:28:58.038 INFO:tasks.workunit.client.0.smithi022.stdout:0/360: dwrite d10/d22/f2a [0,4194304] 0 2024-02-16T19:28:58.040 INFO:tasks.workunit.client.1.smithi117.stdout:4/226: mkdir d0/da/d4a 0 2024-02-16T19:28:58.041 INFO:tasks.workunit.client.1.smithi117.stdout:4/227: readlink d0/d6/l19 0 2024-02-16T19:28:58.041 INFO:tasks.workunit.client.0.smithi022.stdout:0/361: dread d10/d11/d27/d3f/f5b [0,4194304] 0 2024-02-16T19:28:58.042 INFO:tasks.workunit.client.0.smithi022.stdout:7/252: dwrite d9/f25 [0,4194304] 0 2024-02-16T19:28:58.042 INFO:tasks.workunit.client.0.smithi022.stdout:2/189: truncate d0/d2/d18/f28 605920 0 2024-02-16T19:28:58.046 INFO:tasks.workunit.client.0.smithi022.stdout:7/253: write d9/da/d3b/f41 [1472197,113269] 0 2024-02-16T19:28:58.047 INFO:tasks.workunit.client.1.smithi117.stdout:0/283: mkdir d0/d9/d14/d1d/d60/d65 0 2024-02-16T19:28:58.048 INFO:tasks.workunit.client.1.smithi117.stdout:4/228: dwrite d0/da/f33 [0,4194304] 0 2024-02-16T19:28:58.050 INFO:tasks.workunit.client.1.smithi117.stdout:0/284: write d0/d9/d14/d23/d33/f5c [367141,103340] 0 2024-02-16T19:28:58.050 INFO:tasks.workunit.client.0.smithi022.stdout:7/254: truncate d9/da/d2c/d2f/f35 117930 0 2024-02-16T19:28:58.053 INFO:tasks.workunit.client.0.smithi022.stdout:2/190: dwrite d0/d2/d18/d19/f2f [0,4194304] 0 2024-02-16T19:28:58.054 INFO:tasks.workunit.client.1.smithi117.stdout:8/214: write d4/f38 [2921431,70489] 0 2024-02-16T19:28:58.059 INFO:tasks.workunit.client.1.smithi117.stdout:8/215: dread d4/d12/d31/f40 [0,4194304] 0 2024-02-16T19:28:58.060 INFO:tasks.workunit.client.1.smithi117.stdout:3/234: rename d1/dd/d32/d3c to d1/dd/d4f 0 2024-02-16T19:28:58.060 INFO:tasks.workunit.client.1.smithi117.stdout:5/321: getdents d2/d3/d21/d2f/d30/d38/d51 0 2024-02-16T19:28:58.060 INFO:tasks.workunit.client.1.smithi117.stdout:3/235: stat d1/dd/c13 0 2024-02-16T19:28:58.061 INFO:tasks.workunit.client.1.smithi117.stdout:6/238: mknod d5/d1d/c53 0 2024-02-16T19:28:58.061 INFO:tasks.workunit.client.1.smithi117.stdout:3/236: fsync d1/fc 0 2024-02-16T19:28:58.062 INFO:tasks.workunit.client.1.smithi117.stdout:6/239: fsync d5/d4b/f1e 0 2024-02-16T19:28:58.063 INFO:tasks.workunit.client.1.smithi117.stdout:6/240: truncate d5/d4b/d20/f3a 111184 0 2024-02-16T19:28:58.063 INFO:tasks.workunit.client.1.smithi117.stdout:6/241: chown d5/d4b 12384804 1 2024-02-16T19:28:58.067 INFO:tasks.workunit.client.1.smithi117.stdout:6/242: dread d5/ff [4194304,4194304] 0 2024-02-16T19:28:58.068 INFO:tasks.workunit.client.1.smithi117.stdout:6/243: write d5/f11 [3594868,55836] 0 2024-02-16T19:28:58.068 INFO:tasks.workunit.client.1.smithi117.stdout:6/244: stat d5/db/l45 0 2024-02-16T19:28:58.069 INFO:tasks.workunit.client.1.smithi117.stdout:0/285: creat d0/d30/f66 x:0 0 0 2024-02-16T19:28:58.069 INFO:tasks.workunit.client.1.smithi117.stdout:6/245: read - d5/d1d/d28/d32/f51 zero size 2024-02-16T19:28:58.072 INFO:tasks.workunit.client.1.smithi117.stdout:0/286: dread d0/d9/d14/d23/d33/f5c [0,4194304] 0 2024-02-16T19:28:58.073 INFO:tasks.workunit.client.0.smithi022.stdout:1/348: link d6/d8/d22/l1f d6/d8/d22/d16/d17/d26/l64 0 2024-02-16T19:28:58.075 INFO:tasks.workunit.client.0.smithi022.stdout:0/362: mkdir d10/d11/d27/d77 0 2024-02-16T19:28:58.076 INFO:tasks.workunit.client.0.smithi022.stdout:0/363: chown d10/d15/d23/d38 956758 1 2024-02-16T19:28:58.077 INFO:tasks.workunit.client.1.smithi117.stdout:8/216: creat d4/d1c/f49 x:0 0 0 2024-02-16T19:28:58.077 INFO:tasks.workunit.client.0.smithi022.stdout:0/364: chown d10/d15/d23/d4e 2 1 2024-02-16T19:28:58.078 INFO:tasks.workunit.client.0.smithi022.stdout:9/266: truncate db/d18/f26 1137359 0 2024-02-16T19:28:58.078 INFO:tasks.workunit.client.0.smithi022.stdout:0/365: chown d10/d15/d23/f3c 0 1 2024-02-16T19:28:58.084 INFO:tasks.workunit.client.1.smithi117.stdout:5/322: mkdir d2/d3/d21/d2f/d30/d5a/d62 0 2024-02-16T19:28:58.085 INFO:tasks.workunit.client.0.smithi022.stdout:6/262: creat d1/d24/d5b/f63 x:0 0 0 2024-02-16T19:28:58.085 INFO:tasks.workunit.client.1.smithi117.stdout:3/237: mknod d1/dd/d3e/c50 0 2024-02-16T19:28:58.086 INFO:tasks.workunit.client.0.smithi022.stdout:7/255: symlink d9/d1b/d23/l52 0 2024-02-16T19:28:58.088 INFO:tasks.workunit.client.0.smithi022.stdout:6/263: dread d1/f32 [0,4194304] 0 2024-02-16T19:28:58.089 INFO:tasks.workunit.client.0.smithi022.stdout:3/235: mkdir d7/d12/d18/dd/d3c/d5a/d67 0 2024-02-16T19:28:58.090 INFO:tasks.workunit.client.1.smithi117.stdout:4/229: dwrite d0/d6/f1c [0,4194304] 0 2024-02-16T19:28:58.090 INFO:tasks.workunit.client.0.smithi022.stdout:5/269: creat dd/d1f/f58 x:0 0 0 2024-02-16T19:28:58.090 INFO:tasks.workunit.client.1.smithi117.stdout:3/238: dread d1/fa [0,4194304] 0 2024-02-16T19:28:58.091 INFO:tasks.workunit.client.1.smithi117.stdout:1/157: getdents d2/d1d 0 2024-02-16T19:28:58.092 INFO:tasks.workunit.client.1.smithi117.stdout:1/158: chown d2/d18 178698 1 2024-02-16T19:28:58.092 INFO:tasks.workunit.client.0.smithi022.stdout:8/269: mkdir d3/d12/d17/d64 0 2024-02-16T19:28:58.096 INFO:tasks.workunit.client.1.smithi117.stdout:7/193: rename d4/f23 to d4/d4a/f50 0 2024-02-16T19:28:58.096 INFO:tasks.workunit.client.0.smithi022.stdout:5/270: dwrite dd/d1f/f31 [0,4194304] 0 2024-02-16T19:28:58.098 INFO:tasks.workunit.client.0.smithi022.stdout:1/349: fsync d6/d8/d22/f15 0 2024-02-16T19:28:58.099 INFO:tasks.workunit.client.0.smithi022.stdout:5/271: truncate dd/d1f/d48/d21/f24 177801 0 2024-02-16T19:28:58.100 INFO:tasks.workunit.client.0.smithi022.stdout:9/267: readlink db/d14/d16/d28/l40 0 2024-02-16T19:28:58.101 INFO:tasks.workunit.client.0.smithi022.stdout:9/268: stat db/d14/d16/d27/d55/c5a 0 2024-02-16T19:28:58.113 INFO:tasks.workunit.client.1.smithi117.stdout:3/239: unlink d1/c1f 0 2024-02-16T19:28:58.113 INFO:tasks.workunit.client.1.smithi117.stdout:3/240: stat d1/d1b/f41 0 2024-02-16T19:28:58.113 INFO:tasks.workunit.client.0.smithi022.stdout:7/256: read d9/da/f29 [634881,111232] 0 2024-02-16T19:28:58.113 INFO:tasks.workunit.client.1.smithi117.stdout:3/241: write d1/dd/d3b/f3d [3658643,4451] 0 2024-02-16T19:28:58.115 INFO:tasks.workunit.client.1.smithi117.stdout:1/159: symlink d2/d18/d20/l34 0 2024-02-16T19:28:58.115 INFO:tasks.workunit.client.0.smithi022.stdout:7/257: truncate d9/da/d1f/d40/f4b 1025540 0 2024-02-16T19:28:58.116 INFO:tasks.workunit.client.0.smithi022.stdout:5/272: dread f5 [0,4194304] 0 2024-02-16T19:28:58.117 INFO:tasks.workunit.client.0.smithi022.stdout:5/273: write dd/f3c [9368989,91794] 0 2024-02-16T19:28:58.118 INFO:tasks.workunit.client.1.smithi117.stdout:0/287: rename d0/f63 to d0/d9/d14/d23/d2f/f67 0 2024-02-16T19:28:58.121 INFO:tasks.workunit.client.1.smithi117.stdout:7/194: creat d4/d1c/d39/d46/d47/f51 x:0 0 0 2024-02-16T19:28:58.122 INFO:tasks.workunit.client.1.smithi117.stdout:3/242: dread d1/d1b/f2d [0,4194304] 0 2024-02-16T19:28:58.122 INFO:tasks.workunit.client.1.smithi117.stdout:3/243: write d1/d1b/f25 [3513463,9253] 0 2024-02-16T19:28:58.123 INFO:tasks.workunit.client.1.smithi117.stdout:8/217: symlink d4/d7/d37/d3a/d43/d44/l4a 0 2024-02-16T19:28:58.125 INFO:tasks.workunit.client.0.smithi022.stdout:1/350: creat d6/d8/d22/d47/d54/f65 x:0 0 0 2024-02-16T19:28:58.127 INFO:tasks.workunit.client.0.smithi022.stdout:9/269: creat db/d14/f60 x:0 0 0 2024-02-16T19:28:58.128 INFO:tasks.workunit.client.0.smithi022.stdout:9/270: stat db/d14/d16/d21/c36 0 2024-02-16T19:28:58.132 INFO:tasks.workunit.client.0.smithi022.stdout:1/351: dwrite d6/d8/d22/d3a/f44 [0,4194304] 0 2024-02-16T19:28:58.133 INFO:tasks.workunit.client.0.smithi022.stdout:2/191: creat d0/d2/f3d x:0 0 0 2024-02-16T19:28:58.136 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:57 smithi117 ceph-mon[35309]: pgmap v260: 65 pgs: 65 active+clean; 694 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 3.3 MiB/s rd, 69 MiB/s wr, 280 op/s 2024-02-16T19:28:58.139 INFO:tasks.workunit.client.1.smithi117.stdout:7/195: chown d4/d13/f16 848744 1 2024-02-16T19:28:58.144 INFO:tasks.workunit.client.1.smithi117.stdout:8/218: write d4/d7/fc [676998,55784] 0 2024-02-16T19:28:58.145 INFO:tasks.workunit.client.0.smithi022.stdout:7/258: dwrite d9/da/f13 [0,4194304] 0 2024-02-16T19:28:58.145 INFO:tasks.workunit.client.1.smithi117.stdout:8/219: write d4/f3c [851825,101434] 0 2024-02-16T19:28:58.148 INFO:tasks.workunit.client.1.smithi117.stdout:3/244: dwrite d1/fa [0,4194304] 0 2024-02-16T19:28:58.148 INFO:tasks.workunit.client.0.smithi022.stdout:1/352: write d6/f9 [4880457,29103] 0 2024-02-16T19:28:58.149 INFO:tasks.workunit.client.1.smithi117.stdout:3/245: chown d1/dd/d32 943995898 1 2024-02-16T19:28:58.150 INFO:tasks.workunit.client.0.smithi022.stdout:1/353: write d6/f55 [243660,66711] 0 2024-02-16T19:28:58.150 INFO:tasks.workunit.client.0.smithi022.stdout:2/192: mknod d0/d32/c3e 0 2024-02-16T19:28:58.151 INFO:tasks.workunit.client.0.smithi022.stdout:2/193: stat d0/d2/d18/d38 0 2024-02-16T19:28:58.153 INFO:tasks.workunit.client.0.smithi022.stdout:6/264: link d1/c9 d1/d8/da/d29/d61/c64 0 2024-02-16T19:28:58.155 INFO:tasks.workunit.client.0.smithi022.stdout:6/265: dread - d1/d24/d5b/f63 zero size 2024-02-16T19:28:58.156 INFO:tasks.workunit.client.0.smithi022.stdout:6/266: chown d1/d24/d5b 28 1 2024-02-16T19:28:58.156 INFO:tasks.workunit.client.1.smithi117.stdout:0/288: mknod d0/d9/d22/d27/c68 0 2024-02-16T19:28:58.157 INFO:tasks.workunit.client.0.smithi022.stdout:8/270: rename d3/d25/f5d to d3/d12/d2b/f65 0 2024-02-16T19:28:58.157 INFO:tasks.workunit.client.1.smithi117.stdout:5/323: getdents d2/d3/d21/d49/d59 0 2024-02-16T19:28:58.157 INFO:tasks.workunit.client.0.smithi022.stdout:6/267: chown d1/d8/da/d17/d53 1112 1 2024-02-16T19:28:58.158 INFO:tasks.workunit.client.0.smithi022.stdout:6/268: chown d1/l5 504437 1 2024-02-16T19:28:58.158 INFO:tasks.workunit.client.0.smithi022.stdout:5/274: creat dd/d1f/d48/d21/d2f/d40/d4e/d56/f59 x:0 0 0 2024-02-16T19:28:58.159 INFO:tasks.workunit.client.0.smithi022.stdout:6/269: read - d1/d8/f3b zero size 2024-02-16T19:28:58.159 INFO:tasks.workunit.client.0.smithi022.stdout:5/275: readlink dd/l28 0 2024-02-16T19:28:58.160 INFO:tasks.workunit.client.0.smithi022.stdout:5/276: dread - dd/d1f/d48/f47 zero size 2024-02-16T19:28:58.160 INFO:tasks.workunit.client.1.smithi117.stdout:8/220: creat d4/d7/d37/d3a/d43/d44/f4b x:0 0 0 2024-02-16T19:28:58.161 INFO:tasks.workunit.client.1.smithi117.stdout:3/246: creat d1/d1b/d1e/f51 x:0 0 0 2024-02-16T19:28:58.162 INFO:tasks.workunit.client.0.smithi022.stdout:7/259: dread d9/da/f29 [0,4194304] 0 2024-02-16T19:28:58.164 INFO:tasks.workunit.client.1.smithi117.stdout:0/289: mknod d0/d9/d22/d27/d4e/c69 0 2024-02-16T19:28:58.166 INFO:tasks.workunit.client.0.smithi022.stdout:8/271: symlink d3/d25/l66 0 2024-02-16T19:28:58.166 INFO:tasks.workunit.client.1.smithi117.stdout:3/247: dwrite d1/d31/f36 [0,4194304] 0 2024-02-16T19:28:58.167 INFO:tasks.workunit.client.0.smithi022.stdout:8/272: dread - d3/d12/d2b/d2e/f59 zero size 2024-02-16T19:28:58.169 INFO:tasks.workunit.client.1.smithi117.stdout:8/221: rename d4/d1c/l25 to d4/d7/d37/d3a/d43/d44/l4c 0 2024-02-16T19:28:58.170 INFO:tasks.workunit.client.0.smithi022.stdout:6/270: creat d1/d8/da/d17/d48/f65 x:0 0 0 2024-02-16T19:28:58.172 INFO:tasks.workunit.client.1.smithi117.stdout:5/324: dwrite d2/d9/d3c/d24/f53 [0,4194304] 0 2024-02-16T19:28:58.173 INFO:tasks.workunit.client.0.smithi022.stdout:1/354: mkdir d6/d66 0 2024-02-16T19:28:58.176 INFO:tasks.workunit.client.0.smithi022.stdout:5/277: creat dd/d1f/d2e/d35/f5a x:0 0 0 2024-02-16T19:28:58.177 INFO:tasks.workunit.client.0.smithi022.stdout:5/278: readlink dd/l28 0 2024-02-16T19:28:58.180 INFO:tasks.workunit.client.1.smithi117.stdout:0/290: dread d0/f10 [4194304,4194304] 0 2024-02-16T19:28:58.180 INFO:tasks.workunit.client.1.smithi117.stdout:5/325: dwrite d2/d9/d3c/d24/f26 [0,4194304] 0 2024-02-16T19:28:58.182 INFO:tasks.workunit.client.1.smithi117.stdout:3/248: unlink d1/fa 0 2024-02-16T19:28:58.183 INFO:tasks.workunit.client.1.smithi117.stdout:3/249: fsync d1/dd/f48 0 2024-02-16T19:28:58.185 INFO:tasks.workunit.client.1.smithi117.stdout:8/222: rmdir d4/d7/d37/d3a/d43 39 2024-02-16T19:28:58.186 INFO:tasks.workunit.client.1.smithi117.stdout:3/250: truncate d1/dd/f48 894115 0 2024-02-16T19:28:58.186 INFO:tasks.workunit.client.0.smithi022.stdout:0/366: rename d10/d15/d23/d38/c5f to d10/d11/d27/d3f/d58/d60/d62/c78 0 2024-02-16T19:28:58.189 INFO:tasks.workunit.client.1.smithi117.stdout:0/291: dwrite d0/d30/f66 [0,4194304] 0 2024-02-16T19:28:58.189 INFO:tasks.workunit.client.0.smithi022.stdout:8/273: fdatasync d3/d12/f48 0 2024-02-16T19:28:58.190 INFO:tasks.workunit.client.0.smithi022.stdout:8/274: write d3/d12/d2b/f3f [791773,87990] 0 2024-02-16T19:28:58.191 INFO:tasks.workunit.client.1.smithi117.stdout:7/196: getdents d4/d1c/d24/d2f 0 2024-02-16T19:28:58.191 INFO:tasks.workunit.client.0.smithi022.stdout:2/194: truncate d0/d2/d18/d19/f2f 1287612 0 2024-02-16T19:28:58.194 INFO:tasks.workunit.client.0.smithi022.stdout:2/195: dread d0/d2/d18/f28 [0,4194304] 0 2024-02-16T19:28:58.195 INFO:tasks.workunit.client.1.smithi117.stdout:3/251: dwrite d1/d1b/d1e/d29/f47 [0,4194304] 0 2024-02-16T19:28:58.197 INFO:tasks.workunit.client.0.smithi022.stdout:7/260: mknod d9/da/d2c/d2f/c53 0 2024-02-16T19:28:58.199 INFO:tasks.workunit.client.0.smithi022.stdout:9/271: rename db/d14/d16/d21/l3d to db/d18/l61 0 2024-02-16T19:28:58.200 INFO:tasks.workunit.client.0.smithi022.stdout:0/367: rmdir d10/d11/d27/d3f/d58/d60/d62 39 2024-02-16T19:28:58.202 INFO:tasks.workunit.client.0.smithi022.stdout:1/355: getdents d6/d8/d22/d3a/d3e 0 2024-02-16T19:28:58.203 INFO:tasks.workunit.client.1.smithi117.stdout:0/292: creat d0/d9/d14/d23/d2f/f6a x:0 0 0 2024-02-16T19:28:58.205 INFO:tasks.workunit.client.0.smithi022.stdout:9/272: dread db/d14/d16/d21/d37/f4e [0,4194304] 0 2024-02-16T19:28:58.205 INFO:tasks.workunit.client.1.smithi117.stdout:2/178: sync 2024-02-16T19:28:58.205 INFO:tasks.workunit.client.1.smithi117.stdout:1/160: sync 2024-02-16T19:28:58.205 INFO:tasks.workunit.client.1.smithi117.stdout:9/200: sync 2024-02-16T19:28:58.205 INFO:tasks.workunit.client.1.smithi117.stdout:6/246: sync 2024-02-16T19:28:58.206 INFO:tasks.workunit.client.1.smithi117.stdout:4/230: sync 2024-02-16T19:28:58.206 INFO:tasks.workunit.client.0.smithi022.stdout:9/273: chown db/d14/d19/d1a/d3b/f3c 1701506 1 2024-02-16T19:28:58.211 INFO:tasks.workunit.client.1.smithi117.stdout:3/252: mkdir d1/d30/d52 0 2024-02-16T19:28:58.212 INFO:tasks.workunit.client.1.smithi117.stdout:1/161: dwrite d2/d19/f2f [0,4194304] 0 2024-02-16T19:28:58.212 INFO:tasks.workunit.client.0.smithi022.stdout:7/261: read f5 [405509,59189] 0 2024-02-16T19:28:58.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:57 smithi022 ceph-mon[29221]: pgmap v260: 65 pgs: 65 active+clean; 694 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 3.3 MiB/s rd, 69 MiB/s wr, 280 op/s 2024-02-16T19:28:58.214 INFO:tasks.workunit.client.0.smithi022.stdout:6/271: rename d1/d8/d3c/f43 to d1/d8/f66 0 2024-02-16T19:28:58.216 INFO:tasks.workunit.client.1.smithi117.stdout:8/223: mkdir d4/d4d 0 2024-02-16T19:28:58.217 INFO:tasks.workunit.client.0.smithi022.stdout:5/279: truncate dd/d1f/d48/d21/f25 3523382 0 2024-02-16T19:28:58.219 INFO:tasks.workunit.client.0.smithi022.stdout:2/196: dwrite d0/d2/d18/f1a [0,4194304] 0 2024-02-16T19:28:58.219 INFO:tasks.workunit.client.0.smithi022.stdout:0/368: mkdir d10/d11/d79 0 2024-02-16T19:28:58.220 INFO:tasks.workunit.client.0.smithi022.stdout:5/280: dread - dd/d1f/d48/d21/d2f/d52/f53 zero size 2024-02-16T19:28:58.222 INFO:tasks.workunit.client.1.smithi117.stdout:8/224: dwrite d4/d12/d31/f36 [0,4194304] 0 2024-02-16T19:28:58.222 INFO:tasks.workunit.client.1.smithi117.stdout:0/293: mkdir d0/d9/d14/d1d/d1e/d6b 0 2024-02-16T19:28:58.226 INFO:tasks.workunit.client.0.smithi022.stdout:0/369: dread d10/d15/d23/d38/d5a/f5d [0,4194304] 0 2024-02-16T19:28:58.226 INFO:tasks.workunit.client.1.smithi117.stdout:4/231: rmdir d0/da 39 2024-02-16T19:28:58.229 INFO:tasks.workunit.client.1.smithi117.stdout:7/197: dread f2 [0,4194304] 0 2024-02-16T19:28:58.229 INFO:tasks.workunit.client.0.smithi022.stdout:9/274: creat db/d14/d16/d27/d55/f62 x:0 0 0 2024-02-16T19:28:58.229 INFO:tasks.workunit.client.0.smithi022.stdout:4/239: sync 2024-02-16T19:28:58.230 INFO:tasks.workunit.client.1.smithi117.stdout:6/247: mknod d5/d1d/d28/c54 0 2024-02-16T19:28:58.238 INFO:tasks.workunit.client.0.smithi022.stdout:7/262: creat d9/da/d1f/d44/f54 x:0 0 0 2024-02-16T19:28:58.240 INFO:tasks.workunit.client.1.smithi117.stdout:1/162: fdatasync d2/d4/fa 0 2024-02-16T19:28:58.241 INFO:tasks.workunit.client.0.smithi022.stdout:1/356: dread d6/d8/d22/d16/d17/d26/f43 [0,4194304] 0 2024-02-16T19:28:58.242 INFO:tasks.workunit.client.0.smithi022.stdout:1/357: fsync d6/d8/d22/d47/f4e 0 2024-02-16T19:28:58.243 INFO:tasks.workunit.client.1.smithi117.stdout:5/326: creat d2/f63 x:0 0 0 2024-02-16T19:28:58.243 INFO:tasks.workunit.client.0.smithi022.stdout:1/358: fsync d6/d8/d22/d16/f4b 0 2024-02-16T19:28:58.244 INFO:tasks.workunit.client.0.smithi022.stdout:6/272: symlink d1/d8/da/d29/d61/l67 0 2024-02-16T19:28:58.250 INFO:tasks.workunit.client.0.smithi022.stdout:5/281: symlink dd/d1f/d2e/l5b 0 2024-02-16T19:28:58.251 INFO:tasks.workunit.client.0.smithi022.stdout:5/282: stat dd/l13 0 2024-02-16T19:28:58.251 INFO:tasks.workunit.client.1.smithi117.stdout:8/225: unlink d4/d12/d2a/c47 0 2024-02-16T19:28:58.252 INFO:tasks.workunit.client.1.smithi117.stdout:3/253: dwrite d1/dd/f17 [0,4194304] 0 2024-02-16T19:28:58.253 INFO:tasks.workunit.client.0.smithi022.stdout:6/273: dwrite d1/d24/d5b/f63 [0,4194304] 0 2024-02-16T19:28:58.254 INFO:tasks.workunit.client.0.smithi022.stdout:7/263: dread d9/da/f13 [4194304,4194304] 0 2024-02-16T19:28:58.254 INFO:tasks.workunit.client.1.smithi117.stdout:2/179: mkdir d0/d35 0 2024-02-16T19:28:58.255 INFO:tasks.workunit.client.0.smithi022.stdout:5/283: write dd/d1f/d48/d21/d2f/d40/d4e/d56/f59 [501223,76331] 0 2024-02-16T19:28:58.256 INFO:tasks.workunit.client.0.smithi022.stdout:7/264: fdatasync d9/da/d2c/d2f/f35 0 2024-02-16T19:28:58.260 INFO:tasks.workunit.client.1.smithi117.stdout:9/201: mknod dd/d23/c3a 0 2024-02-16T19:28:58.260 INFO:tasks.workunit.client.1.smithi117.stdout:9/202: chown dd/l1d 19950013 1 2024-02-16T19:28:58.263 INFO:tasks.workunit.client.0.smithi022.stdout:7/265: read d9/da/d1f/f18 [988348,87317] 0 2024-02-16T19:28:58.263 INFO:tasks.workunit.client.1.smithi117.stdout:7/198: rmdir d4/d1c/d39/d46 39 2024-02-16T19:28:58.264 INFO:tasks.workunit.client.1.smithi117.stdout:9/203: dread fb [0,4194304] 0 2024-02-16T19:28:58.264 INFO:tasks.workunit.client.0.smithi022.stdout:7/266: dread - d9/da/f3a zero size 2024-02-16T19:28:58.266 INFO:tasks.workunit.client.0.smithi022.stdout:6/274: dwrite d1/d8/f3b [0,4194304] 0 2024-02-16T19:28:58.266 INFO:tasks.workunit.client.1.smithi117.stdout:6/248: creat d5/db/f55 x:0 0 0 2024-02-16T19:28:58.267 INFO:tasks.workunit.client.1.smithi117.stdout:6/249: dread d5/f13 [0,4194304] 0 2024-02-16T19:28:58.267 INFO:tasks.workunit.client.0.smithi022.stdout:7/267: chown d9/da/d1f/d40/f4b 101 1 2024-02-16T19:28:58.271 INFO:tasks.workunit.client.0.smithi022.stdout:5/284: dwrite dd/f11 [4194304,4194304] 0 2024-02-16T19:28:58.273 INFO:tasks.workunit.client.1.smithi117.stdout:1/163: creat d2/d1d/f35 x:0 0 0 2024-02-16T19:28:58.276 INFO:tasks.workunit.client.1.smithi117.stdout:5/327: creat d2/d3/d21/d2f/d30/d38/f64 x:0 0 0 2024-02-16T19:28:58.276 INFO:tasks.workunit.client.0.smithi022.stdout:2/197: dread d0/f20 [0,4194304] 0 2024-02-16T19:28:58.276 INFO:tasks.workunit.client.0.smithi022.stdout:5/285: dread - dd/d1f/d48/d21/d2f/d52/f53 zero size 2024-02-16T19:28:58.278 INFO:tasks.workunit.client.0.smithi022.stdout:7/268: dread d9/f25 [0,4194304] 0 2024-02-16T19:28:58.279 INFO:tasks.workunit.client.1.smithi117.stdout:0/294: rename d0/d9/ca to d0/d9/d22/d26/d55/c6c 0 2024-02-16T19:28:58.281 INFO:tasks.workunit.client.0.smithi022.stdout:7/269: chown d9/da 23 1 2024-02-16T19:28:58.281 INFO:tasks.workunit.client.0.smithi022.stdout:6/275: dwrite d1/d8/da/d17/d53/f5c [0,4194304] 0 2024-02-16T19:28:58.282 INFO:tasks.workunit.client.1.smithi117.stdout:8/226: mknod d4/d1c/c4e 0 2024-02-16T19:28:58.283 INFO:tasks.workunit.client.0.smithi022.stdout:7/270: chown d9/da/d1f/f18 32589766 1 2024-02-16T19:28:58.284 INFO:tasks.workunit.client.0.smithi022.stdout:7/271: dread - d9/da/d1f/d44/f54 zero size 2024-02-16T19:28:58.285 INFO:tasks.workunit.client.1.smithi117.stdout:2/180: stat d0/d3/d22/c24 0 2024-02-16T19:28:58.286 INFO:tasks.workunit.client.1.smithi117.stdout:2/181: readlink d0/d13/l27 0 2024-02-16T19:28:58.288 INFO:tasks.workunit.client.1.smithi117.stdout:4/232: symlink d0/da/l4b 0 2024-02-16T19:28:58.289 INFO:tasks.workunit.client.1.smithi117.stdout:2/182: dread d0/f19 [0,4194304] 0 2024-02-16T19:28:58.289 INFO:tasks.workunit.client.0.smithi022.stdout:7/272: write d9/da/d3b/f41 [1159642,7349] 0 2024-02-16T19:28:58.294 INFO:tasks.workunit.client.0.smithi022.stdout:5/286: dwrite dd/d1f/d48/f47 [0,4194304] 0 2024-02-16T19:28:58.294 INFO:tasks.workunit.client.0.smithi022.stdout:7/273: write d9/f25 [2040668,119496] 0 2024-02-16T19:28:58.296 INFO:tasks.workunit.client.0.smithi022.stdout:5/287: stat dd/d1f/d48/f43 0 2024-02-16T19:28:58.296 INFO:tasks.workunit.client.0.smithi022.stdout:7/274: chown d9/da/f3a 26426 1 2024-02-16T19:28:58.300 INFO:tasks.workunit.client.1.smithi117.stdout:6/250: creat d5/d4b/d44/f56 x:0 0 0 2024-02-16T19:28:58.301 INFO:tasks.workunit.client.1.smithi117.stdout:6/251: dread - d5/d4b/f1e zero size 2024-02-16T19:28:58.302 INFO:tasks.workunit.client.0.smithi022.stdout:3/236: sync 2024-02-16T19:28:58.302 INFO:tasks.workunit.client.0.smithi022.stdout:3/237: readlink d7/l11 0 2024-02-16T19:28:58.303 INFO:tasks.workunit.client.1.smithi117.stdout:1/164: creat d2/d18/d20/f36 x:0 0 0 2024-02-16T19:28:58.304 INFO:tasks.workunit.client.1.smithi117.stdout:1/165: chown d2/d1d/d1f 1388512 1 2024-02-16T19:28:58.305 INFO:tasks.workunit.client.0.smithi022.stdout:8/275: rename d3/d12/d2b/f3c to d3/d12/d17/f67 0 2024-02-16T19:28:58.306 INFO:tasks.workunit.client.0.smithi022.stdout:8/276: chown d3/d25/d33/f41 32045783 1 2024-02-16T19:28:58.307 INFO:tasks.workunit.client.1.smithi117.stdout:3/254: dwrite d1/d1b/f2d [0,4194304] 0 2024-02-16T19:28:58.308 INFO:tasks.workunit.client.1.smithi117.stdout:5/328: truncate d2/d3/f6 968710 0 2024-02-16T19:28:58.309 INFO:tasks.workunit.client.0.smithi022.stdout:2/198: chown d0/l29 154368 1 2024-02-16T19:28:58.310 INFO:tasks.workunit.client.1.smithi117.stdout:0/295: mkdir d0/d9/d22/d27/d31/d6d 0 2024-02-16T19:28:58.311 INFO:tasks.workunit.client.0.smithi022.stdout:1/359: mkdir d6/d8/d22/d67 0 2024-02-16T19:28:58.314 INFO:tasks.workunit.client.0.smithi022.stdout:6/276: creat d1/d8/d23/f68 x:0 0 0 2024-02-16T19:28:58.316 INFO:tasks.workunit.client.1.smithi117.stdout:7/199: dwrite d4/d1c/f3c [0,4194304] 0 2024-02-16T19:28:58.316 INFO:tasks.workunit.client.1.smithi117.stdout:7/200: dread - d4/f49 zero size 2024-02-16T19:28:58.317 INFO:tasks.workunit.client.0.smithi022.stdout:8/277: dread d3/f55 [0,4194304] 0 2024-02-16T19:28:58.320 INFO:tasks.workunit.client.1.smithi117.stdout:2/183: creat d0/d13/f36 x:0 0 0 2024-02-16T19:28:58.322 INFO:tasks.workunit.client.0.smithi022.stdout:0/370: creat d10/d11/d27/d3f/f7a x:0 0 0 2024-02-16T19:28:58.325 INFO:tasks.workunit.client.0.smithi022.stdout:9/275: link db/d14/d33/f4f db/d14/d16/d21/d37/f63 0 2024-02-16T19:28:58.326 INFO:tasks.workunit.client.0.smithi022.stdout:1/360: dwrite d6/d8/d22/d47/d54/f62 [0,4194304] 0 2024-02-16T19:28:58.327 INFO:tasks.workunit.client.1.smithi117.stdout:9/204: unlink dd/d23/f22 0 2024-02-16T19:28:58.330 INFO:tasks.workunit.client.0.smithi022.stdout:5/288: mkdir dd/d1f/d48/d21/d2f/d52/d5c 0 2024-02-16T19:28:58.332 INFO:tasks.workunit.client.1.smithi117.stdout:6/252: rename d5/l31 to d5/d1d/l57 0 2024-02-16T19:28:58.333 INFO:tasks.workunit.client.0.smithi022.stdout:0/371: dwrite d10/d22/d2c/f72 [0,4194304] 0 2024-02-16T19:28:58.334 INFO:tasks.workunit.client.1.smithi117.stdout:6/253: truncate d5/d1d/d28/f34 568978 0 2024-02-16T19:28:58.336 INFO:tasks.workunit.client.1.smithi117.stdout:1/166: symlink d2/d18/d20/l37 0 2024-02-16T19:28:58.341 INFO:tasks.workunit.client.1.smithi117.stdout:6/254: dwrite d5/d1d/f2c [0,4194304] 0 2024-02-16T19:28:58.342 INFO:tasks.workunit.client.1.smithi117.stdout:3/255: mknod d1/d1b/c53 0 2024-02-16T19:28:58.343 INFO:tasks.workunit.client.1.smithi117.stdout:3/256: read d1/dd/f14 [3636238,47749] 0 2024-02-16T19:28:58.344 INFO:tasks.workunit.client.1.smithi117.stdout:3/257: fsync d1/d1b/f41 0 2024-02-16T19:28:58.346 INFO:tasks.workunit.client.1.smithi117.stdout:5/329: unlink d2/d3/d21/d2f/d30/d5a/f22 0 2024-02-16T19:28:58.349 INFO:tasks.workunit.client.1.smithi117.stdout:3/258: dwrite d1/dd/f43 [0,4194304] 0 2024-02-16T19:28:58.350 INFO:tasks.workunit.client.1.smithi117.stdout:3/259: dread - d1/dd/d4f/f3f zero size 2024-02-16T19:28:58.350 INFO:tasks.workunit.client.1.smithi117.stdout:3/260: stat d1/d1b/l4d 0 2024-02-16T19:28:58.350 INFO:tasks.workunit.client.1.smithi117.stdout:3/261: fdatasync d1/dd/f17 0 2024-02-16T19:28:58.353 INFO:tasks.workunit.client.0.smithi022.stdout:7/275: dwrite f5 [4194304,4194304] 0 2024-02-16T19:28:58.354 INFO:tasks.workunit.client.1.smithi117.stdout:4/233: getdents d0/d6/d48 0 2024-02-16T19:28:58.356 INFO:tasks.workunit.client.1.smithi117.stdout:3/262: dread d1/dd/f17 [0,4194304] 0 2024-02-16T19:28:58.357 INFO:tasks.workunit.client.1.smithi117.stdout:3/263: write d1/dd/d3b/f3d [334429,88294] 0 2024-02-16T19:28:58.357 INFO:tasks.workunit.client.1.smithi117.stdout:3/264: dread - d1/d1b/f21 zero size 2024-02-16T19:28:58.358 INFO:tasks.workunit.client.1.smithi117.stdout:3/265: chown d1/l45 61419267 1 2024-02-16T19:28:58.359 INFO:tasks.workunit.client.1.smithi117.stdout:7/201: unlink d4/d1c/f32 0 2024-02-16T19:28:58.362 INFO:tasks.workunit.client.1.smithi117.stdout:2/184: mkdir d0/d3/d22/d37 0 2024-02-16T19:28:58.371 INFO:tasks.workunit.client.0.smithi022.stdout:5/289: chown dd/d1f/d48/d21/f25 12 1 2024-02-16T19:28:58.371 INFO:tasks.workunit.client.0.smithi022.stdout:2/199: dwrite d0/f20 [0,4194304] 0 2024-02-16T19:28:58.374 INFO:tasks.workunit.client.0.smithi022.stdout:0/372: write d10/d11/d27/f52 [2857704,130076] 0 2024-02-16T19:28:58.376 INFO:tasks.workunit.client.0.smithi022.stdout:0/373: write d10/f6e [382454,35480] 0 2024-02-16T19:28:58.377 INFO:tasks.workunit.client.1.smithi117.stdout:6/255: rename d5/d4b/d20/f38 to d5/d1d/d28/d32/d37/f58 0 2024-02-16T19:28:58.378 INFO:tasks.workunit.client.0.smithi022.stdout:4/240: link d4/f40 d4/d6/dd/df/d10/f50 0 2024-02-16T19:28:58.379 INFO:tasks.workunit.client.1.smithi117.stdout:8/227: sync 2024-02-16T19:28:58.380 INFO:tasks.workunit.client.0.smithi022.stdout:5/290: dwrite dd/f11 [4194304,4194304] 0 2024-02-16T19:28:58.380 INFO:tasks.workunit.client.1.smithi117.stdout:8/228: chown d4/d12/c2f 981749 1 2024-02-16T19:28:58.381 INFO:tasks.workunit.client.0.smithi022.stdout:3/238: symlink d7/d12/d18/dd/d24/d58/l68 0 2024-02-16T19:28:58.384 INFO:tasks.workunit.client.0.smithi022.stdout:6/277: symlink d1/d8/da/d17/l69 0 2024-02-16T19:28:58.385 INFO:tasks.workunit.client.1.smithi117.stdout:5/330: creat d2/d3/f65 x:0 0 0 2024-02-16T19:28:58.386 INFO:tasks.workunit.client.1.smithi117.stdout:8/229: dwrite d4/d12/d2a/f2b [4194304,4194304] 0 2024-02-16T19:28:58.387 INFO:tasks.workunit.client.1.smithi117.stdout:0/296: mknod d0/d9/d14/d1d/d60/d65/c6e 0 2024-02-16T19:28:58.387 INFO:tasks.workunit.client.1.smithi117.stdout:5/331: write d2/d3/f47 [4435256,11009] 0 2024-02-16T19:28:58.387 INFO:tasks.workunit.client.0.smithi022.stdout:7/276: creat d9/da/d1f/d44/f55 x:0 0 0 2024-02-16T19:28:58.389 INFO:tasks.workunit.client.0.smithi022.stdout:8/278: mkdir d3/d12/d2b/d46/d68 0 2024-02-16T19:28:58.390 INFO:tasks.workunit.client.0.smithi022.stdout:8/279: fsync d3/d12/d17/f50 0 2024-02-16T19:28:58.392 INFO:tasks.workunit.client.0.smithi022.stdout:1/361: creat d6/d8/d22/d16/d17/d63/f68 x:0 0 0 2024-02-16T19:28:58.393 INFO:tasks.workunit.client.0.smithi022.stdout:3/239: dread d7/d12/d18/dd/d3c/f65 [0,4194304] 0 2024-02-16T19:28:58.394 INFO:tasks.workunit.client.0.smithi022.stdout:8/280: truncate d3/d6/f7 3496248 0 2024-02-16T19:28:58.395 INFO:tasks.workunit.client.0.smithi022.stdout:5/291: dwrite dd/d1f/d48/d21/f24 [0,4194304] 0 2024-02-16T19:28:58.397 INFO:tasks.workunit.client.0.smithi022.stdout:1/362: read - d6/d8/d22/d47/d54/f65 zero size 2024-02-16T19:28:58.404 INFO:tasks.workunit.client.0.smithi022.stdout:0/374: creat d10/d22/d49/f7b x:0 0 0 2024-02-16T19:28:58.408 INFO:tasks.workunit.client.0.smithi022.stdout:0/375: dread d10/d22/f2a [0,4194304] 0 2024-02-16T19:28:58.412 INFO:tasks.workunit.client.1.smithi117.stdout:1/167: symlink d2/l38 0 2024-02-16T19:28:58.413 INFO:tasks.workunit.client.0.smithi022.stdout:9/276: link db/d14/d19/c58 db/d14/d19/d1a/d3b/c64 0 2024-02-16T19:28:58.415 INFO:tasks.workunit.client.0.smithi022.stdout:8/281: creat d3/d25/f69 x:0 0 0 2024-02-16T19:28:58.417 INFO:tasks.workunit.client.1.smithi117.stdout:1/168: dwrite d2/d19/f2f [4194304,4194304] 0 2024-02-16T19:28:58.418 INFO:tasks.workunit.client.0.smithi022.stdout:3/240: creat d7/d12/d18/dd/d3c/d5a/f69 x:0 0 0 2024-02-16T19:28:58.422 INFO:tasks.workunit.client.0.smithi022.stdout:2/200: rename d0/cd to d0/d2/d18/d1f/c3f 0 2024-02-16T19:28:58.425 INFO:tasks.workunit.client.0.smithi022.stdout:9/277: dread db/d14/d19/d1a/d3b/f3c [0,4194304] 0 2024-02-16T19:28:58.427 INFO:tasks.workunit.client.0.smithi022.stdout:1/363: dread d6/d8/d22/d16/f40 [0,4194304] 0 2024-02-16T19:28:58.429 INFO:tasks.workunit.client.0.smithi022.stdout:4/241: creat d4/d6/dd/d22/f51 x:0 0 0 2024-02-16T19:28:58.430 INFO:tasks.workunit.client.0.smithi022.stdout:4/242: truncate d4/d6/dd/df/d10/f42 609329 0 2024-02-16T19:28:58.432 INFO:tasks.workunit.client.1.smithi117.stdout:7/202: mknod d4/d1c/d39/d46/c52 0 2024-02-16T19:28:58.433 INFO:tasks.workunit.client.1.smithi117.stdout:7/203: readlink d4/d4a/l43 0 2024-02-16T19:28:58.434 INFO:tasks.workunit.client.0.smithi022.stdout:6/278: creat d1/d24/d56/f6a x:0 0 0 2024-02-16T19:28:58.436 INFO:tasks.workunit.client.0.smithi022.stdout:7/277: symlink d9/da/d2c/d46/l56 0 2024-02-16T19:28:58.437 INFO:tasks.workunit.client.0.smithi022.stdout:4/243: dwrite d4/d6/dd/df/f29 [0,4194304] 0 2024-02-16T19:28:58.438 INFO:tasks.workunit.client.1.smithi117.stdout:2/185: mkdir d0/d3/d10/d38 0 2024-02-16T19:28:58.438 INFO:tasks.workunit.client.0.smithi022.stdout:7/278: stat d9/da/d2c/d2f/c1e 0 2024-02-16T19:28:58.441 INFO:tasks.workunit.client.1.smithi117.stdout:9/205: creat dd/d23/f3b x:0 0 0 2024-02-16T19:28:58.443 INFO:tasks.workunit.client.0.smithi022.stdout:5/292: mkdir dd/d1f/d48/d21/d5d 0 2024-02-16T19:28:58.445 INFO:tasks.workunit.client.0.smithi022.stdout:8/282: mkdir d3/d12/d6a 0 2024-02-16T19:28:58.446 INFO:tasks.workunit.client.1.smithi117.stdout:3/266: truncate d1/d1b/d1e/d29/f47 1908686 0 2024-02-16T19:28:58.448 INFO:tasks.workunit.client.0.smithi022.stdout:3/241: symlink d7/d12/d18/dd/d31/l6a 0 2024-02-16T19:28:58.449 INFO:tasks.workunit.client.1.smithi117.stdout:1/169: creat d2/d1d/f39 x:0 0 0 2024-02-16T19:28:58.449 INFO:tasks.workunit.client.0.smithi022.stdout:3/242: chown d7/d12/d18/dd/d3c/d5a/f69 24 1 2024-02-16T19:28:58.453 INFO:tasks.workunit.client.1.smithi117.stdout:5/332: symlink d2/d9/d3c/d42/l66 0 2024-02-16T19:28:58.454 INFO:tasks.workunit.client.0.smithi022.stdout:0/376: rename d10/d15/d23/d4e/c54 to d10/d15/d23/d38/c7c 0 2024-02-16T19:28:58.455 INFO:tasks.workunit.client.1.smithi117.stdout:1/170: dwrite d2/d18/d20/f36 [0,4194304] 0 2024-02-16T19:28:58.455 INFO:tasks.workunit.client.1.smithi117.stdout:1/171: stat d2/d4/c1c 0 2024-02-16T19:28:58.456 INFO:tasks.workunit.client.1.smithi117.stdout:0/297: mknod d0/d9/d14/d1d/d1e/d6b/c6f 0 2024-02-16T19:28:58.456 INFO:tasks.workunit.client.0.smithi022.stdout:3/243: dread d7/d12/d18/f1a [0,4194304] 0 2024-02-16T19:28:58.457 INFO:tasks.workunit.client.1.smithi117.stdout:0/298: fdatasync d0/d9/d2c/f44 0 2024-02-16T19:28:58.457 INFO:tasks.workunit.client.1.smithi117.stdout:4/234: link d0/da/d21/d3d/d28/f39 d0/d6/d48/f4c 0 2024-02-16T19:28:58.460 INFO:tasks.workunit.client.0.smithi022.stdout:9/278: unlink db/f1d 0 2024-02-16T19:28:58.461 INFO:tasks.workunit.client.1.smithi117.stdout:4/235: dread d0/da/d21/d3d/f29 [0,4194304] 0 2024-02-16T19:28:58.462 INFO:tasks.workunit.client.1.smithi117.stdout:7/204: unlink d4/d13/l36 0 2024-02-16T19:28:58.462 INFO:tasks.workunit.client.0.smithi022.stdout:0/377: dwrite d10/d11/d27/d3f/f75 [0,4194304] 0 2024-02-16T19:28:58.464 INFO:tasks.workunit.client.0.smithi022.stdout:1/364: creat d6/d8/d22/d16/d17/d63/f69 x:0 0 0 2024-02-16T19:28:58.466 INFO:tasks.workunit.client.0.smithi022.stdout:1/365: write d6/d8/d22/d47/d54/f62 [1757667,50586] 0 2024-02-16T19:28:58.471 INFO:tasks.workunit.client.0.smithi022.stdout:6/279: symlink d1/d8/da/d17/d48/l6b 0 2024-02-16T19:28:58.471 INFO:tasks.workunit.client.0.smithi022.stdout:4/244: rmdir d4/d6 39 2024-02-16T19:28:58.473 INFO:tasks.workunit.client.0.smithi022.stdout:7/279: fdatasync d9/da/d2c/d2f/f17 0 2024-02-16T19:28:58.474 INFO:tasks.workunit.client.0.smithi022.stdout:7/280: chown d9/da/d1f/d44/f54 988968301 1 2024-02-16T19:28:58.475 INFO:tasks.workunit.client.0.smithi022.stdout:7/281: write d9/da/f3a [15887,66519] 0 2024-02-16T19:28:58.477 INFO:tasks.workunit.client.0.smithi022.stdout:5/293: mkdir dd/d1f/d48/d21/d2f/d46/d5e 0 2024-02-16T19:28:58.477 INFO:tasks.workunit.client.0.smithi022.stdout:7/282: chown d9/da/d2c/d2f/c3c 47851314 1 2024-02-16T19:28:58.479 INFO:tasks.workunit.client.0.smithi022.stdout:8/283: truncate d3/f15 361903 0 2024-02-16T19:28:58.480 INFO:tasks.workunit.client.0.smithi022.stdout:5/294: dread f5 [0,4194304] 0 2024-02-16T19:28:58.483 INFO:tasks.workunit.client.0.smithi022.stdout:8/284: dread d3/d12/d2b/f3a [0,4194304] 0 2024-02-16T19:28:58.484 INFO:tasks.workunit.client.0.smithi022.stdout:8/285: write d3/d12/f48 [5660478,31887] 0 2024-02-16T19:28:58.495 INFO:tasks.workunit.client.1.smithi117.stdout:6/256: dwrite d5/d4b/d20/d2d/f3c [0,4194304] 0 2024-02-16T19:28:58.496 INFO:tasks.workunit.client.1.smithi117.stdout:3/267: rename d1/fc to d1/d1b/d1e/f54 0 2024-02-16T19:28:58.496 INFO:tasks.workunit.client.1.smithi117.stdout:8/230: truncate d4/f6 5897107 0 2024-02-16T19:28:58.496 INFO:tasks.workunit.client.0.smithi022.stdout:1/366: mknod d6/d8/d22/d16/c6a 0 2024-02-16T19:28:58.497 INFO:tasks.workunit.client.1.smithi117.stdout:1/172: creat d2/d1d/f3a x:0 0 0 2024-02-16T19:28:58.497 INFO:tasks.workunit.client.0.smithi022.stdout:6/280: creat d1/d24/d5b/f6c x:0 0 0 2024-02-16T19:28:58.497 INFO:tasks.workunit.client.1.smithi117.stdout:4/236: symlink d0/da/d21/d3d/d28/d2f/d37/l4d 0 2024-02-16T19:28:58.498 INFO:tasks.workunit.client.1.smithi117.stdout:3/268: readlink d1/d1b/d20/l49 0 2024-02-16T19:28:58.500 INFO:tasks.workunit.client.0.smithi022.stdout:6/281: truncate d1/d8/d3c/f44 477155 0 2024-02-16T19:28:58.500 INFO:tasks.workunit.client.1.smithi117.stdout:1/173: write d2/d4/f11 [383723,31169] 0 2024-02-16T19:28:58.500 INFO:tasks.workunit.client.1.smithi117.stdout:9/206: unlink dd/d23/l27 0 2024-02-16T19:28:58.502 INFO:tasks.workunit.client.1.smithi117.stdout:9/207: read dd/d23/d2e/f32 [4067433,88534] 0 2024-02-16T19:28:58.504 INFO:tasks.workunit.client.1.smithi117.stdout:9/208: read dd/d2d/f35 [811783,71548] 0 2024-02-16T19:28:58.504 INFO:tasks.workunit.client.0.smithi022.stdout:8/286: dread d3/d6/f34 [0,4194304] 0 2024-02-16T19:28:58.504 INFO:tasks.workunit.client.1.smithi117.stdout:5/333: mkdir d2/d4c/d4f/d67 0 2024-02-16T19:28:58.506 INFO:tasks.workunit.client.1.smithi117.stdout:3/269: dwrite d1/f19 [0,4194304] 0 2024-02-16T19:28:58.507 INFO:tasks.workunit.client.0.smithi022.stdout:7/283: unlink d9/da/c10 0 2024-02-16T19:28:58.507 INFO:tasks.workunit.client.1.smithi117.stdout:1/174: dread d2/d4/f15 [0,4194304] 0 2024-02-16T19:28:58.509 INFO:tasks.workunit.client.1.smithi117.stdout:7/205: dwrite d4/f2b [0,4194304] 0 2024-02-16T19:28:58.510 INFO:tasks.workunit.client.1.smithi117.stdout:1/175: write d2/d4/f16 [3612765,88357] 0 2024-02-16T19:28:58.510 INFO:tasks.workunit.client.0.smithi022.stdout:3/244: dwrite d7/d21/f5d [0,4194304] 0 2024-02-16T19:28:58.511 INFO:tasks.workunit.client.0.smithi022.stdout:3/245: read - d7/d12/d18/f4e zero size 2024-02-16T19:28:58.515 INFO:tasks.workunit.client.0.smithi022.stdout:2/201: mkdir d0/d2/d18/d19/d40 0 2024-02-16T19:28:58.519 INFO:tasks.workunit.client.1.smithi117.stdout:7/206: dwrite d4/d1c/f37 [0,4194304] 0 2024-02-16T19:28:58.520 INFO:tasks.workunit.client.1.smithi117.stdout:7/207: write d4/d13/f16 [8833301,5343] 0 2024-02-16T19:28:58.521 INFO:tasks.workunit.client.1.smithi117.stdout:4/237: creat d0/da/d21/d3d/d28/f4e x:0 0 0 2024-02-16T19:28:58.522 INFO:tasks.workunit.client.0.smithi022.stdout:3/246: dwrite d7/d12/f44 [0,4194304] 0 2024-02-16T19:28:58.523 INFO:tasks.workunit.client.0.smithi022.stdout:3/247: dread - d7/f39 zero size 2024-02-16T19:28:58.526 INFO:tasks.workunit.client.0.smithi022.stdout:9/279: mkdir db/d14/d16/d5d/d65 0 2024-02-16T19:28:58.527 INFO:tasks.workunit.client.0.smithi022.stdout:2/202: dwrite d0/d2/d18/f1a [4194304,4194304] 0 2024-02-16T19:28:58.529 INFO:tasks.workunit.client.0.smithi022.stdout:2/203: readlink d0/d2/d18/l1e 0 2024-02-16T19:28:58.532 INFO:tasks.workunit.client.0.smithi022.stdout:0/378: mkdir d10/d7d 0 2024-02-16T19:28:58.533 INFO:tasks.workunit.client.1.smithi117.stdout:8/231: dwrite d4/d12/d31/f40 [0,4194304] 0 2024-02-16T19:28:58.533 INFO:tasks.workunit.client.0.smithi022.stdout:5/295: write dd/d1f/d48/f2d [3259938,2067] 0 2024-02-16T19:28:58.534 INFO:tasks.workunit.client.1.smithi117.stdout:5/334: creat d2/d3/d21/d2f/d30/f68 x:0 0 0 2024-02-16T19:28:58.534 INFO:tasks.workunit.client.1.smithi117.stdout:8/232: stat d4/d7/d37 0 2024-02-16T19:28:58.536 INFO:tasks.workunit.client.0.smithi022.stdout:0/379: dread - d10/d22/d49/f7b zero size 2024-02-16T19:28:58.536 INFO:tasks.workunit.client.0.smithi022.stdout:6/282: write d1/f32 [4975168,58119] 0 2024-02-16T19:28:58.537 INFO:tasks.workunit.client.0.smithi022.stdout:5/296: stat dd/f11 0 2024-02-16T19:28:58.538 INFO:tasks.workunit.client.1.smithi117.stdout:3/270: chown d1/d30/c3a 751220660 1 2024-02-16T19:28:58.538 INFO:tasks.workunit.client.0.smithi022.stdout:3/248: dwrite d7/d21/f5d [0,4194304] 0 2024-02-16T19:28:58.541 INFO:tasks.workunit.client.0.smithi022.stdout:8/287: fdatasync d3/d6/f5e 0 2024-02-16T19:28:58.543 INFO:tasks.workunit.client.0.smithi022.stdout:9/280: dwrite db/d14/d19/d1a/f50 [0,4194304] 0 2024-02-16T19:28:58.544 INFO:tasks.workunit.client.1.smithi117.stdout:2/186: sync 2024-02-16T19:28:58.545 INFO:tasks.workunit.client.1.smithi117.stdout:6/257: mkdir d5/d1d/d49/d59 0 2024-02-16T19:28:58.546 INFO:tasks.workunit.client.1.smithi117.stdout:2/187: truncate d0/d13/f36 848286 0 2024-02-16T19:28:58.546 INFO:tasks.workunit.client.0.smithi022.stdout:9/281: stat db/d14/d16/f45 0 2024-02-16T19:28:58.550 INFO:tasks.workunit.client.0.smithi022.stdout:5/297: dwrite f3 [0,4194304] 0 2024-02-16T19:28:58.551 INFO:tasks.workunit.client.0.smithi022.stdout:5/298: dread - dd/d1f/d48/f43 zero size 2024-02-16T19:28:58.554 INFO:tasks.workunit.client.1.smithi117.stdout:2/188: dread d0/f25 [0,4194304] 0 2024-02-16T19:28:58.554 INFO:tasks.workunit.client.1.smithi117.stdout:7/208: symlink d4/d4a/l53 0 2024-02-16T19:28:58.555 INFO:tasks.workunit.client.0.smithi022.stdout:9/282: dread db/d18/f2a [0,4194304] 0 2024-02-16T19:28:58.558 INFO:tasks.workunit.client.1.smithi117.stdout:4/238: rename d0/da/d21/l34 to d0/da/d21/d3d/d28/l4f 0 2024-02-16T19:28:58.565 INFO:tasks.workunit.client.1.smithi117.stdout:4/239: read d0/d6/f1f [34456,118367] 0 2024-02-16T19:28:58.568 INFO:tasks.workunit.client.1.smithi117.stdout:3/271: mkdir d1/d31/d55 0 2024-02-16T19:28:58.569 INFO:tasks.workunit.client.1.smithi117.stdout:3/272: write d1/d1b/f41 [470859,95162] 0 2024-02-16T19:28:58.570 INFO:tasks.workunit.client.1.smithi117.stdout:1/176: symlink d2/d4/l3b 0 2024-02-16T19:28:58.571 INFO:tasks.workunit.client.1.smithi117.stdout:1/177: write d2/d18/d20/f36 [4256464,89110] 0 2024-02-16T19:28:58.572 INFO:tasks.workunit.client.1.smithi117.stdout:1/178: write d2/d1d/f25 [1894020,99719] 0 2024-02-16T19:28:58.574 INFO:tasks.workunit.client.1.smithi117.stdout:1/179: write d2/d1d/f25 [1849035,88069] 0 2024-02-16T19:28:58.574 INFO:tasks.workunit.client.1.smithi117.stdout:3/273: read d1/d1b/f25 [769753,103041] 0 2024-02-16T19:28:58.574 INFO:tasks.workunit.client.1.smithi117.stdout:9/209: truncate dd/d23/f1f 847665 0 2024-02-16T19:28:58.576 INFO:tasks.workunit.client.1.smithi117.stdout:1/180: write d2/d1d/f35 [72569,22509] 0 2024-02-16T19:28:58.578 INFO:tasks.workunit.client.1.smithi117.stdout:6/258: rename d5/db/l14 to d5/d4b/d44/d52/l5a 0 2024-02-16T19:28:58.579 INFO:tasks.workunit.client.0.smithi022.stdout:8/288: mknod d3/d12/d2b/d46/c6b 0 2024-02-16T19:28:58.579 INFO:tasks.workunit.client.0.smithi022.stdout:6/283: dwrite d1/d8/f66 [0,4194304] 0 2024-02-16T19:28:58.579 INFO:tasks.workunit.client.1.smithi117.stdout:2/189: dwrite d0/f25 [0,4194304] 0 2024-02-16T19:28:58.580 INFO:tasks.workunit.client.0.smithi022.stdout:6/284: fsync d1/d24/d5b/f6c 0 2024-02-16T19:28:58.583 INFO:tasks.workunit.client.1.smithi117.stdout:8/233: mknod d4/d7/d37/d3a/d43/c4f 0 2024-02-16T19:28:58.586 INFO:tasks.workunit.client.0.smithi022.stdout:9/283: mknod db/d14/d19/d1a/c66 0 2024-02-16T19:28:58.589 INFO:tasks.workunit.client.1.smithi117.stdout:0/299: sync 2024-02-16T19:28:58.589 INFO:tasks.workunit.client.0.smithi022.stdout:2/204: symlink d0/d32/l41 0 2024-02-16T19:28:58.590 INFO:tasks.workunit.client.1.smithi117.stdout:8/234: dwrite d4/d12/f26 [4194304,4194304] 0 2024-02-16T19:28:58.592 INFO:tasks.workunit.client.0.smithi022.stdout:1/367: creat d6/f6b x:0 0 0 2024-02-16T19:28:58.593 INFO:tasks.workunit.client.1.smithi117.stdout:5/335: write d2/d1d/f3f [603138,48782] 0 2024-02-16T19:28:58.593 INFO:tasks.workunit.client.0.smithi022.stdout:3/249: dwrite d7/d12/d18/dd/d24/d53/f5e [0,4194304] 0 2024-02-16T19:28:58.595 INFO:tasks.workunit.client.0.smithi022.stdout:9/284: dread db/d14/d16/d21/d37/f4e [0,4194304] 0 2024-02-16T19:28:58.600 INFO:tasks.workunit.client.0.smithi022.stdout:0/380: unlink d10/d15/d23/c59 0 2024-02-16T19:28:58.603 INFO:tasks.workunit.client.0.smithi022.stdout:7/284: creat d9/d1b/f57 x:0 0 0 2024-02-16T19:28:58.605 INFO:tasks.workunit.client.0.smithi022.stdout:5/299: fdatasync f3 0 2024-02-16T19:28:58.608 INFO:tasks.workunit.client.0.smithi022.stdout:5/300: write dd/d1f/d48/f2d [6546650,58254] 0 2024-02-16T19:28:58.609 INFO:tasks.workunit.client.1.smithi117.stdout:9/210: unlink dd/d23/d2e/l2f 0 2024-02-16T19:28:58.609 INFO:tasks.workunit.client.0.smithi022.stdout:0/381: dwrite d10/d11/d27/d3f/f7a [0,4194304] 0 2024-02-16T19:28:58.611 INFO:tasks.workunit.client.1.smithi117.stdout:1/181: rmdir d2/d1d 39 2024-02-16T19:28:58.612 INFO:tasks.workunit.client.1.smithi117.stdout:9/211: dread fb [0,4194304] 0 2024-02-16T19:28:58.613 INFO:tasks.workunit.client.0.smithi022.stdout:6/285: symlink d1/d8/da/d29/d61/l6d 0 2024-02-16T19:28:58.617 INFO:tasks.workunit.client.1.smithi117.stdout:1/182: dwrite d2/d19/f2f [0,4194304] 0 2024-02-16T19:28:58.619 INFO:tasks.workunit.client.0.smithi022.stdout:2/205: symlink d0/d2/d18/l42 0 2024-02-16T19:28:58.619 INFO:tasks.workunit.client.1.smithi117.stdout:3/274: rename d1/dd/d4f to d1/d30/d52/d56 0 2024-02-16T19:28:58.621 INFO:tasks.workunit.client.0.smithi022.stdout:1/368: chown d6/d8/d22/d16/d17/d1d 82974 1 2024-02-16T19:28:58.622 INFO:tasks.workunit.client.1.smithi117.stdout:2/190: mknod d0/d3/c39 0 2024-02-16T19:28:58.625 INFO:tasks.workunit.client.0.smithi022.stdout:4/245: getdents d4/d6/d14 0 2024-02-16T19:28:58.626 INFO:tasks.workunit.client.0.smithi022.stdout:4/246: chown d4/d6/dd/df/d10/l4d 46894343 1 2024-02-16T19:28:58.629 INFO:tasks.workunit.client.0.smithi022.stdout:3/250: mkdir d7/d12/d18/dd/d24/d58/d6b 0 2024-02-16T19:28:58.630 INFO:tasks.workunit.client.1.smithi117.stdout:4/240: symlink d0/d6/d48/l50 0 2024-02-16T19:28:58.630 INFO:tasks.workunit.client.0.smithi022.stdout:3/251: dread - d7/d12/d18/dd/d31/d41/f4c zero size 2024-02-16T19:28:58.631 INFO:tasks.workunit.client.1.smithi117.stdout:4/241: dread d0/d6/d48/f4c [4194304,4194304] 0 2024-02-16T19:28:58.631 INFO:tasks.workunit.client.1.smithi117.stdout:8/235: stat d4/d12/f1a 0 2024-02-16T19:28:58.633 INFO:tasks.workunit.client.1.smithi117.stdout:4/242: write d0/da/d21/f22 [1350922,93797] 0 2024-02-16T19:28:58.633 INFO:tasks.workunit.client.1.smithi117.stdout:4/243: chown d0/da/l2e 13643528 1 2024-02-16T19:28:58.635 INFO:tasks.workunit.client.0.smithi022.stdout:5/301: symlink dd/d1f/d4a/l5f 0 2024-02-16T19:28:58.636 INFO:tasks.workunit.client.0.smithi022.stdout:5/302: chown dd/d1f/d48/d21/d2f/d52/f53 16598571 1 2024-02-16T19:28:58.637 INFO:tasks.workunit.client.0.smithi022.stdout:5/303: write dd/d1f/d48/f1c [22393,39487] 0 2024-02-16T19:28:58.638 INFO:tasks.workunit.client.1.smithi117.stdout:4/244: dwrite d0/d6/f23 [0,4194304] 0 2024-02-16T19:28:58.639 INFO:tasks.workunit.client.0.smithi022.stdout:9/285: write db/d14/f5e [1649934,99181] 0 2024-02-16T19:28:58.639 INFO:tasks.workunit.client.1.smithi117.stdout:4/245: write d0/d6/f1c [3471447,35276] 0 2024-02-16T19:28:58.640 INFO:tasks.workunit.client.0.smithi022.stdout:9/286: chown db/d14/d16/d28 65902487 1 2024-02-16T19:28:58.641 INFO:tasks.workunit.client.1.smithi117.stdout:7/209: creat d4/d1c/d39/f54 x:0 0 0 2024-02-16T19:28:58.646 INFO:tasks.workunit.client.0.smithi022.stdout:9/287: dread db/d14/d16/d21/d37/f4e [0,4194304] 0 2024-02-16T19:28:58.646 INFO:tasks.workunit.client.1.smithi117.stdout:7/210: dwrite d4/f10 [0,4194304] 0 2024-02-16T19:28:58.647 INFO:tasks.workunit.client.1.smithi117.stdout:7/211: truncate d4/d13/f35 559209 0 2024-02-16T19:28:58.649 INFO:tasks.workunit.client.0.smithi022.stdout:8/289: truncate d3/d25/f43 2376153 0 2024-02-16T19:28:58.651 INFO:tasks.workunit.client.0.smithi022.stdout:8/290: fdatasync d3/d12/d2b/d46/f53 0 2024-02-16T19:28:58.651 INFO:tasks.workunit.client.0.smithi022.stdout:4/247: symlink d4/d6/l52 0 2024-02-16T19:28:58.652 INFO:tasks.workunit.client.0.smithi022.stdout:9/288: dread db/d14/d19/d1a/f50 [0,4194304] 0 2024-02-16T19:28:58.654 INFO:tasks.workunit.client.0.smithi022.stdout:8/291: write d3/d25/d33/f37 [77895,14555] 0 2024-02-16T19:28:58.656 INFO:tasks.workunit.client.0.smithi022.stdout:3/252: creat d7/d12/d59/f6c x:0 0 0 2024-02-16T19:28:58.658 INFO:tasks.workunit.client.1.smithi117.stdout:3/275: creat d1/d30/f57 x:0 0 0 2024-02-16T19:28:58.659 INFO:tasks.workunit.client.1.smithi117.stdout:3/276: truncate d1/d1b/f41 1611703 0 2024-02-16T19:28:58.659 INFO:tasks.workunit.client.0.smithi022.stdout:5/304: unlink dd/c22 0 2024-02-16T19:28:58.662 INFO:tasks.workunit.client.1.smithi117.stdout:1/183: dread d2/d1d/f25 [0,4194304] 0 2024-02-16T19:28:58.662 INFO:tasks.workunit.client.0.smithi022.stdout:6/286: truncate d1/d8/ff 217575 0 2024-02-16T19:28:58.665 INFO:tasks.workunit.client.1.smithi117.stdout:2/191: dread d0/f12 [0,4194304] 0 2024-02-16T19:28:58.666 INFO:tasks.workunit.client.0.smithi022.stdout:1/369: mkdir d6/d8/d6c 0 2024-02-16T19:28:58.667 INFO:tasks.workunit.client.1.smithi117.stdout:7/212: dread d4/f34 [0,4194304] 0 2024-02-16T19:28:58.667 INFO:tasks.workunit.client.1.smithi117.stdout:7/213: readlink d4/d1c/d39/l45 0 2024-02-16T19:28:58.672 INFO:tasks.workunit.client.1.smithi117.stdout:6/259: fsync d5/d4b/d20/f3a 0 2024-02-16T19:28:58.674 INFO:tasks.workunit.client.1.smithi117.stdout:5/336: fsync d2/d3/f6 0 2024-02-16T19:28:58.675 INFO:tasks.workunit.client.0.smithi022.stdout:4/248: mknod d4/d6/d3a/c53 0 2024-02-16T19:28:58.676 INFO:tasks.workunit.client.1.smithi117.stdout:1/184: dread d2/d18/f1b [0,4194304] 0 2024-02-16T19:28:58.676 INFO:tasks.workunit.client.0.smithi022.stdout:9/289: dread db/d14/d16/f1f [0,4194304] 0 2024-02-16T19:28:58.678 INFO:tasks.workunit.client.0.smithi022.stdout:4/249: write d4/d6/dd/df/d10/f25 [29295,42825] 0 2024-02-16T19:28:58.679 INFO:tasks.workunit.client.0.smithi022.stdout:4/250: readlink d4/d6/l4c 0 2024-02-16T19:28:58.681 INFO:tasks.workunit.client.0.smithi022.stdout:3/253: symlink d7/d12/d18/dd/d31/d41/l6d 0 2024-02-16T19:28:58.683 INFO:tasks.workunit.client.1.smithi117.stdout:9/212: truncate dd/d23/f30 685469 0 2024-02-16T19:28:58.685 INFO:tasks.workunit.client.0.smithi022.stdout:0/382: rename d10/d15/c48 to d10/d11/d27/d3f/d58/d60/c7e 0 2024-02-16T19:28:58.691 INFO:tasks.workunit.client.0.smithi022.stdout:0/383: dwrite d10/d11/d27/f52 [0,4194304] 0 2024-02-16T19:28:58.691 INFO:tasks.workunit.client.0.smithi022.stdout:1/370: write d6/d8/d22/d16/d17/d1d/f21 [4841101,119731] 0 2024-02-16T19:28:58.699 INFO:tasks.workunit.client.0.smithi022.stdout:9/290: rename db/d14/d19/d1a/d3b to db/d14/d19/d1a/d67 0 2024-02-16T19:28:58.700 INFO:tasks.workunit.client.0.smithi022.stdout:0/384: dwrite d10/d11/d27/d3f/d58/f6a [4194304,4194304] 0 2024-02-16T19:28:58.701 INFO:tasks.workunit.client.0.smithi022.stdout:0/385: stat d10/l2b 0 2024-02-16T19:28:58.702 INFO:tasks.workunit.client.1.smithi117.stdout:0/300: link d0/cf d0/d9/d14/d1d/d60/d65/c70 0 2024-02-16T19:28:58.703 INFO:tasks.workunit.client.1.smithi117.stdout:0/301: dread - d0/d9/d14/d23/d2f/f6a zero size 2024-02-16T19:28:58.703 INFO:tasks.workunit.client.1.smithi117.stdout:0/302: chown d0/d9 223924894 1 2024-02-16T19:28:58.704 INFO:tasks.workunit.client.1.smithi117.stdout:0/303: stat d0/d9/d14/d23/f24 0 2024-02-16T19:28:58.705 INFO:tasks.workunit.client.1.smithi117.stdout:7/214: rename d4/d1c/d24/d2f to d4/d1c/d39/d46/d55 0 2024-02-16T19:28:58.706 INFO:tasks.workunit.client.0.smithi022.stdout:3/254: mknod d7/d12/d18/dd/d31/c6e 0 2024-02-16T19:28:58.707 INFO:tasks.workunit.client.1.smithi117.stdout:7/215: read d4/f1e [61082,58426] 0 2024-02-16T19:28:58.712 INFO:tasks.workunit.client.1.smithi117.stdout:8/236: write d4/f6 [1100435,11226] 0 2024-02-16T19:28:58.712 INFO:tasks.workunit.client.1.smithi117.stdout:7/216: fdatasync d4/d13/f35 0 2024-02-16T19:28:58.713 INFO:tasks.workunit.client.0.smithi022.stdout:3/255: dwrite d7/f1e [0,4194304] 0 2024-02-16T19:28:58.715 INFO:tasks.workunit.client.1.smithi117.stdout:0/304: dwrite d0/d30/f64 [0,4194304] 0 2024-02-16T19:28:58.715 INFO:tasks.workunit.client.0.smithi022.stdout:3/256: truncate d7/d21/f38 442068 0 2024-02-16T19:28:58.715 INFO:tasks.workunit.client.0.smithi022.stdout:2/206: link d0/lc d0/d2/d18/l43 0 2024-02-16T19:28:58.718 INFO:tasks.workunit.client.1.smithi117.stdout:1/185: symlink d2/d18/l3c 0 2024-02-16T19:28:58.718 INFO:tasks.workunit.client.0.smithi022.stdout:4/251: dread d4/d6/f16 [0,4194304] 0 2024-02-16T19:28:58.719 INFO:tasks.workunit.client.1.smithi117.stdout:1/186: fsync d2/ff 0 2024-02-16T19:28:58.719 INFO:tasks.workunit.client.0.smithi022.stdout:4/252: read - d4/d6/dd/f2c zero size 2024-02-16T19:28:58.720 INFO:tasks.workunit.client.0.smithi022.stdout:4/253: stat d4/d6/dd/f2c 0 2024-02-16T19:28:58.721 INFO:tasks.workunit.client.0.smithi022.stdout:4/254: fdatasync d4/d6/dd/d22/f51 0 2024-02-16T19:28:58.722 INFO:tasks.workunit.client.0.smithi022.stdout:4/255: chown d4/d6/d14/l46 92690473 1 2024-02-16T19:28:58.725 INFO:tasks.workunit.client.1.smithi117.stdout:6/260: dwrite d5/f7 [0,4194304] 0 2024-02-16T19:28:58.737 INFO:tasks.workunit.client.1.smithi117.stdout:8/237: mkdir d4/d7/d37/d50 0 2024-02-16T19:28:58.738 INFO:tasks.workunit.client.1.smithi117.stdout:5/337: unlink d2/c10 0 2024-02-16T19:28:58.740 INFO:tasks.workunit.client.1.smithi117.stdout:7/217: creat d4/d1c/d39/d46/d47/f56 x:0 0 0 2024-02-16T19:28:58.740 INFO:tasks.workunit.client.1.smithi117.stdout:5/338: write d2/d9/d3c/d24/f26 [2391963,77614] 0 2024-02-16T19:28:58.741 INFO:tasks.workunit.client.1.smithi117.stdout:7/218: truncate d4/d13/f4b 404492 0 2024-02-16T19:28:58.741 INFO:tasks.workunit.client.1.smithi117.stdout:0/305: unlink d0/d9/d14/d23/d2f/f6a 0 2024-02-16T19:28:58.741 INFO:tasks.workunit.client.1.smithi117.stdout:5/339: write d2/d9/d3c/d24/f26 [8069156,8947] 0 2024-02-16T19:28:58.742 INFO:tasks.workunit.client.1.smithi117.stdout:5/340: readlink d2/d3/d21/d2f/l5c 0 2024-02-16T19:28:58.743 INFO:tasks.workunit.client.1.smithi117.stdout:7/219: chown d4/d1c/d39/d46/d47/c4e 101242609 1 2024-02-16T19:28:58.743 INFO:tasks.workunit.client.1.smithi117.stdout:1/187: mknod d2/d1d/d1f/c3d 0 2024-02-16T19:28:58.743 INFO:tasks.workunit.client.1.smithi117.stdout:7/220: chown d4/d13 18520563 1 2024-02-16T19:28:58.744 INFO:tasks.workunit.client.1.smithi117.stdout:1/188: dread - d2/f31 zero size 2024-02-16T19:28:58.745 INFO:tasks.workunit.client.1.smithi117.stdout:1/189: truncate d2/d18/d20/f32 527188 0 2024-02-16T19:28:58.746 INFO:tasks.workunit.client.1.smithi117.stdout:6/261: rename d5/f1a to d5/d1d/d28/d32/f5b 0 2024-02-16T19:28:58.747 INFO:tasks.workunit.client.1.smithi117.stdout:2/192: rmdir d0/d3/d10/d38 0 2024-02-16T19:28:58.747 INFO:tasks.workunit.client.1.smithi117.stdout:2/193: chown d0/d13/f36 188521450 1 2024-02-16T19:28:58.751 INFO:tasks.workunit.client.1.smithi117.stdout:4/246: getdents d0/da 0 2024-02-16T19:28:58.753 INFO:tasks.workunit.client.1.smithi117.stdout:0/306: mknod d0/d30/c71 0 2024-02-16T19:28:58.754 INFO:tasks.workunit.client.1.smithi117.stdout:0/307: fdatasync d0/f5 0 2024-02-16T19:28:58.754 INFO:tasks.workunit.client.1.smithi117.stdout:3/277: getdents d1/d30/d52/d56 0 2024-02-16T19:28:58.755 INFO:tasks.workunit.client.1.smithi117.stdout:7/221: creat d4/d1c/d39/d46/f57 x:0 0 0 2024-02-16T19:28:58.756 INFO:tasks.workunit.client.1.smithi117.stdout:1/190: creat d2/d18/f3e x:0 0 0 2024-02-16T19:28:58.757 INFO:tasks.workunit.client.1.smithi117.stdout:2/194: unlink d0/d3/d10/f30 0 2024-02-16T19:28:58.763 INFO:tasks.workunit.client.1.smithi117.stdout:0/308: creat d0/d9/d14/d1d/d1e/d54/f72 x:0 0 0 2024-02-16T19:28:58.766 INFO:tasks.workunit.client.1.smithi117.stdout:1/191: fsync d2/d1d/f25 0 2024-02-16T19:28:58.767 INFO:tasks.workunit.client.1.smithi117.stdout:2/195: creat d0/d3/f3a x:0 0 0 2024-02-16T19:28:58.768 INFO:tasks.workunit.client.1.smithi117.stdout:4/247: link d0/da/d21/d3d/d28/d2f/f31 d0/da/d21/d3d/d28/f51 0 2024-02-16T19:28:58.769 INFO:tasks.workunit.client.1.smithi117.stdout:4/248: stat d0/da/d21/d3d/d2d/c35 0 2024-02-16T19:28:58.771 INFO:tasks.workunit.client.1.smithi117.stdout:8/238: truncate d4/f38 3659199 0 2024-02-16T19:28:58.775 INFO:tasks.workunit.client.1.smithi117.stdout:0/309: dread d0/d9/d14/d23/d33/f5c [0,4194304] 0 2024-02-16T19:28:58.776 INFO:tasks.workunit.client.1.smithi117.stdout:1/192: truncate d2/d4/f15 41178 0 2024-02-16T19:28:58.777 INFO:tasks.workunit.client.0.smithi022.stdout:1/371: truncate d6/d8/d22/f3d 167220 0 2024-02-16T19:28:58.778 INFO:tasks.workunit.client.1.smithi117.stdout:4/249: mknod d0/da/d21/d3d/d2d/c52 0 2024-02-16T19:28:58.779 INFO:tasks.workunit.client.0.smithi022.stdout:8/292: creat d3/d12/d2b/f6c x:0 0 0 2024-02-16T19:28:58.779 INFO:tasks.workunit.client.0.smithi022.stdout:1/372: chown d6/f55 1433 1 2024-02-16T19:28:58.780 INFO:tasks.workunit.client.0.smithi022.stdout:5/305: creat dd/d1f/d48/f60 x:0 0 0 2024-02-16T19:28:58.780 INFO:tasks.workunit.client.0.smithi022.stdout:3/257: creat d7/d12/d18/d4b/f6f x:0 0 0 2024-02-16T19:28:58.780 INFO:tasks.workunit.client.1.smithi117.stdout:0/310: dwrite d0/d9/d22/d26/f36 [0,4194304] 0 2024-02-16T19:28:58.780 INFO:tasks.workunit.client.1.smithi117.stdout:3/278: truncate d1/dd/f17 1042 0 2024-02-16T19:28:58.781 INFO:tasks.workunit.client.0.smithi022.stdout:3/258: chown l3 0 1 2024-02-16T19:28:58.781 INFO:tasks.workunit.client.1.smithi117.stdout:0/311: stat d0/d30/c71 0 2024-02-16T19:28:58.781 INFO:tasks.workunit.client.0.smithi022.stdout:8/293: write d3/d6/f63 [798547,4361] 0 2024-02-16T19:28:58.785 INFO:tasks.workunit.client.0.smithi022.stdout:5/306: dread f5 [0,4194304] 0 2024-02-16T19:28:58.785 INFO:tasks.workunit.client.1.smithi117.stdout:7/222: truncate d4/f2b 1595591 0 2024-02-16T19:28:58.785 INFO:tasks.workunit.client.1.smithi117.stdout:0/312: dread d0/d30/f66 [0,4194304] 0 2024-02-16T19:28:58.786 INFO:tasks.workunit.client.0.smithi022.stdout:5/307: stat dd/l13 0 2024-02-16T19:28:58.787 INFO:tasks.workunit.client.0.smithi022.stdout:5/308: chown dd/d1f/d48/d21/d2f/d40 4429 1 2024-02-16T19:28:58.790 INFO:tasks.workunit.client.0.smithi022.stdout:2/207: creat d0/d2/d18/d19/d40/f44 x:0 0 0 2024-02-16T19:28:58.792 INFO:tasks.workunit.client.1.smithi117.stdout:2/196: mkdir d0/d3b 0 2024-02-16T19:28:58.792 INFO:tasks.workunit.client.1.smithi117.stdout:2/197: fdatasync d0/d13/f15 0 2024-02-16T19:28:58.795 INFO:tasks.workunit.client.0.smithi022.stdout:5/309: dwrite dd/f18 [0,4194304] 0 2024-02-16T19:28:58.796 INFO:tasks.workunit.client.0.smithi022.stdout:5/310: write f6 [6310209,41727] 0 2024-02-16T19:28:58.797 INFO:tasks.workunit.client.0.smithi022.stdout:3/259: rmdir d7/d12/d18/d4b 39 2024-02-16T19:28:58.799 INFO:tasks.workunit.client.0.smithi022.stdout:8/294: write d3/d12/d2b/f65 [5105062,116162] 0 2024-02-16T19:28:58.799 INFO:tasks.workunit.client.0.smithi022.stdout:4/256: mkdir d4/d6/dd/d31/d4e/d54 0 2024-02-16T19:28:58.805 INFO:tasks.workunit.client.1.smithi117.stdout:8/239: rename d4/d12/l19 to d4/d7/d37/d50/l51 0 2024-02-16T19:28:58.806 INFO:tasks.workunit.client.1.smithi117.stdout:8/240: write d4/d12/d2a/f2b [1543479,110369] 0 2024-02-16T19:28:58.808 INFO:tasks.workunit.client.1.smithi117.stdout:7/223: rmdir d4/d1c/d39/d46 39 2024-02-16T19:28:58.810 INFO:tasks.workunit.client.0.smithi022.stdout:1/373: fsync d6/d8/d22/f3d 0 2024-02-16T19:28:58.811 INFO:tasks.workunit.client.1.smithi117.stdout:0/313: symlink d0/d30/l73 0 2024-02-16T19:28:58.812 INFO:tasks.workunit.client.1.smithi117.stdout:0/314: fsync d0/d9/d22/d27/d4e/f61 0 2024-02-16T19:28:58.813 INFO:tasks.workunit.client.0.smithi022.stdout:2/208: mknod d0/d2/d18/d19/d40/c45 0 2024-02-16T19:28:58.814 INFO:tasks.workunit.client.0.smithi022.stdout:9/291: dwrite db/d14/d19/f17 [0,4194304] 0 2024-02-16T19:28:58.815 INFO:tasks.workunit.client.0.smithi022.stdout:0/386: getdents d10/d11 0 2024-02-16T19:28:58.817 INFO:tasks.workunit.client.1.smithi117.stdout:2/198: creat d0/d3/d22/f3c x:0 0 0 2024-02-16T19:28:58.820 INFO:tasks.workunit.client.0.smithi022.stdout:5/311: rmdir dd/d1f/d48/d21/d2f/d40/d4e/d56 39 2024-02-16T19:28:58.821 INFO:tasks.workunit.client.1.smithi117.stdout:4/250: dwrite d0/da/d21/d3d/d28/f39 [0,4194304] 0 2024-02-16T19:28:58.822 INFO:tasks.workunit.client.1.smithi117.stdout:3/279: mknod d1/c58 0 2024-02-16T19:28:58.823 INFO:tasks.workunit.client.1.smithi117.stdout:4/251: write d0/da/d21/d3d/d28/f39 [868196,71699] 0 2024-02-16T19:28:58.823 INFO:tasks.workunit.client.0.smithi022.stdout:3/260: mkdir d7/d12/d18/dd/d24/d58/d70 0 2024-02-16T19:28:58.827 INFO:tasks.workunit.client.0.smithi022.stdout:4/257: mkdir d4/d6/dd/df/d10/d55 0 2024-02-16T19:28:58.835 INFO:tasks.workunit.client.0.smithi022.stdout:4/258: dwrite d4/d6/dd/f2c [0,4194304] 0 2024-02-16T19:28:58.836 INFO:tasks.workunit.client.0.smithi022.stdout:9/292: creat db/d14/d16/d27/f68 x:0 0 0 2024-02-16T19:28:58.837 INFO:tasks.workunit.client.1.smithi117.stdout:7/224: symlink d4/d1c/d24/l58 0 2024-02-16T19:28:58.843 INFO:tasks.workunit.client.0.smithi022.stdout:0/387: write d10/d11/f2e [349809,31621] 0 2024-02-16T19:28:58.846 INFO:tasks.workunit.client.1.smithi117.stdout:4/252: rename d0/da/d21/l30 to d0/da/l53 0 2024-02-16T19:28:58.853 INFO:tasks.workunit.client.1.smithi117.stdout:4/253: dread d0/da/d21/f27 [0,4194304] 0 2024-02-16T19:28:58.854 INFO:tasks.workunit.client.1.smithi117.stdout:4/254: chown d0/f14 130693 1 2024-02-16T19:28:58.920 INFO:tasks.workunit.client.1.smithi117.stdout:1/193: getdents d2/d1d/d1f 0 2024-02-16T19:28:58.921 INFO:tasks.workunit.client.1.smithi117.stdout:1/194: write d2/d18/d20/f36 [95781,63783] 0 2024-02-16T19:28:58.955 INFO:tasks.workunit.client.0.smithi022.stdout:5/312: mkdir dd/d1f/d48/d21/d2f/d52/d5c/d61 0 2024-02-16T19:28:59.064 INFO:tasks.workunit.client.1.smithi117.stdout:3/280: mknod d1/dd/c59 0 2024-02-16T19:28:59.064 INFO:tasks.workunit.client.1.smithi117.stdout:3/281: stat d1/f9 0 2024-02-16T19:28:59.064 INFO:tasks.workunit.client.1.smithi117.stdout:8/241: truncate d4/d7/fc 3492606 0 2024-02-16T19:28:59.064 INFO:tasks.workunit.client.1.smithi117.stdout:8/242: write d4/d12/d31/f40 [563791,93776] 0 2024-02-16T19:28:59.064 INFO:tasks.workunit.client.1.smithi117.stdout:7/225: symlink d4/d1c/d39/d46/d55/l59 0 2024-02-16T19:28:59.064 INFO:tasks.workunit.client.1.smithi117.stdout:4/255: creat d0/da/d21/d3d/d28/d2f/d37/f54 x:0 0 0 2024-02-16T19:28:59.065 INFO:tasks.workunit.client.1.smithi117.stdout:3/282: creat d1/dd/d3b/f5a x:0 0 0 2024-02-16T19:28:59.065 INFO:tasks.workunit.client.1.smithi117.stdout:9/213: sync 2024-02-16T19:28:59.065 INFO:tasks.workunit.client.1.smithi117.stdout:8/243: unlink d4/d12/l27 0 2024-02-16T19:28:59.065 INFO:tasks.workunit.client.1.smithi117.stdout:4/256: unlink d0/d6/f3b 0 2024-02-16T19:28:59.065 INFO:tasks.workunit.client.1.smithi117.stdout:4/257: dread - d0/da/d21/d3d/d2d/f41 zero size 2024-02-16T19:28:59.065 INFO:tasks.workunit.client.1.smithi117.stdout:4/258: fsync d0/f25 0 2024-02-16T19:28:59.065 INFO:tasks.workunit.client.1.smithi117.stdout:3/283: mknod d1/d1b/d20/c5b 0 2024-02-16T19:28:59.065 INFO:tasks.workunit.client.1.smithi117.stdout:4/259: dwrite d0/da/d21/f40 [0,4194304] 0 2024-02-16T19:28:59.065 INFO:tasks.workunit.client.1.smithi117.stdout:4/260: dwrite d0/d6/f23 [0,4194304] 0 2024-02-16T19:28:59.065 INFO:tasks.workunit.client.1.smithi117.stdout:7/226: creat d4/d1c/d39/f5a x:0 0 0 2024-02-16T19:28:59.065 INFO:tasks.workunit.client.1.smithi117.stdout:4/261: truncate d0/da/d21/d3d/f1d 174311 0 2024-02-16T19:28:59.065 INFO:tasks.workunit.client.1.smithi117.stdout:7/227: read d4/f17 [905319,103195] 0 2024-02-16T19:28:59.065 INFO:tasks.workunit.client.1.smithi117.stdout:3/284: mkdir d1/dd/d5c 0 2024-02-16T19:28:59.065 INFO:tasks.workunit.client.1.smithi117.stdout:9/214: creat dd/d23/d2e/d2c/f3c x:0 0 0 2024-02-16T19:28:59.065 INFO:tasks.workunit.client.1.smithi117.stdout:8/244: creat d4/f52 x:0 0 0 2024-02-16T19:28:59.066 INFO:tasks.workunit.client.1.smithi117.stdout:8/245: dread d4/d12/f26 [4194304,4194304] 0 2024-02-16T19:28:59.067 INFO:tasks.workunit.client.1.smithi117.stdout:5/341: sync 2024-02-16T19:28:59.067 INFO:tasks.workunit.client.1.smithi117.stdout:6/262: sync 2024-02-16T19:28:59.067 INFO:tasks.workunit.client.1.smithi117.stdout:0/315: sync 2024-02-16T19:28:59.067 INFO:tasks.workunit.client.0.smithi022.stdout:0/388: symlink d10/d15/d23/d38/d5a/l7f 0 2024-02-16T19:28:59.067 INFO:tasks.workunit.client.0.smithi022.stdout:2/209: link d0/d2/d18/l42 d0/d2/d2e/l46 0 2024-02-16T19:28:59.068 INFO:tasks.workunit.client.0.smithi022.stdout:4/259: symlink d4/d6/dd/d31/d4e/d54/l56 0 2024-02-16T19:28:59.068 INFO:tasks.workunit.client.0.smithi022.stdout:8/295: getdents d3/d12/d17 0 2024-02-16T19:28:59.068 INFO:tasks.workunit.client.0.smithi022.stdout:8/296: dread - d3/d6/f5e zero size 2024-02-16T19:28:59.068 INFO:tasks.workunit.client.0.smithi022.stdout:8/297: chown d3/d25/f69 368637771 1 2024-02-16T19:28:59.068 INFO:tasks.workunit.client.0.smithi022.stdout:8/298: chown d3/d25/d33 97057 1 2024-02-16T19:28:59.068 INFO:tasks.workunit.client.0.smithi022.stdout:0/389: rename d10/d11/d50/f46 to d10/d15/d23/d38/d5a/f80 0 2024-02-16T19:28:59.068 INFO:tasks.workunit.client.0.smithi022.stdout:1/374: link d6/d8/d22/d16/c37 d6/d8/d22/d16/d17/d5d/c6d 0 2024-02-16T19:28:59.068 INFO:tasks.workunit.client.0.smithi022.stdout:2/210: creat d0/d2/d18/d19/f47 x:0 0 0 2024-02-16T19:28:59.068 INFO:tasks.workunit.client.0.smithi022.stdout:4/260: mknod d4/d6/dd/d22/c57 0 2024-02-16T19:28:59.068 INFO:tasks.workunit.client.0.smithi022.stdout:5/313: mknod dd/d1f/d48/d21/d2f/d40/d49/d4b/c62 0 2024-02-16T19:28:59.068 INFO:tasks.workunit.client.0.smithi022.stdout:8/299: dread d3/d12/f48 [0,4194304] 0 2024-02-16T19:28:59.068 INFO:tasks.workunit.client.0.smithi022.stdout:4/261: dwrite d4/d6/dd/df/d10/f42 [0,4194304] 0 2024-02-16T19:28:59.068 INFO:tasks.workunit.client.0.smithi022.stdout:8/300: write d3/d12/d17/f3d [456170,79463] 0 2024-02-16T19:28:59.068 INFO:tasks.workunit.client.0.smithi022.stdout:3/261: getdents d7/d12 0 2024-02-16T19:28:59.069 INFO:tasks.workunit.client.0.smithi022.stdout:3/262: truncate f2 8998250 0 2024-02-16T19:28:59.069 INFO:tasks.workunit.client.0.smithi022.stdout:6/287: sync 2024-02-16T19:28:59.069 INFO:tasks.workunit.client.0.smithi022.stdout:7/285: sync 2024-02-16T19:28:59.069 INFO:tasks.workunit.client.0.smithi022.stdout:7/286: write d9/da/d1f/d40/f4b [1333723,24487] 0 2024-02-16T19:28:59.069 INFO:tasks.workunit.client.0.smithi022.stdout:6/288: dwrite d1/d8/f11 [0,4194304] 0 2024-02-16T19:28:59.069 INFO:tasks.workunit.client.0.smithi022.stdout:2/211: fsync d0/f2d 0 2024-02-16T19:28:59.069 INFO:tasks.workunit.client.0.smithi022.stdout:5/314: dread dd/d1f/d48/f2d [4194304,4194304] 0 2024-02-16T19:28:59.069 INFO:tasks.workunit.client.0.smithi022.stdout:4/262: mkdir d4/d6/dd/d31/d4e/d54/d58 0 2024-02-16T19:28:59.069 INFO:tasks.workunit.client.0.smithi022.stdout:9/293: creat db/d14/d16/f69 x:0 0 0 2024-02-16T19:28:59.069 INFO:tasks.workunit.client.0.smithi022.stdout:4/263: write d4/d6/dd/f37 [3884782,62289] 0 2024-02-16T19:28:59.069 INFO:tasks.workunit.client.0.smithi022.stdout:9/294: truncate db/d14/d33/f4f 999380 0 2024-02-16T19:28:59.069 INFO:tasks.workunit.client.0.smithi022.stdout:9/295: chown db/d14/d19/l34 0 1 2024-02-16T19:28:59.069 INFO:tasks.workunit.client.0.smithi022.stdout:8/301: creat d3/d12/d17/f6d x:0 0 0 2024-02-16T19:28:59.069 INFO:tasks.workunit.client.0.smithi022.stdout:8/302: chown d3/d12/d6a 5 1 2024-02-16T19:28:59.069 INFO:tasks.workunit.client.0.smithi022.stdout:4/264: dread d4/f23 [4194304,4194304] 0 2024-02-16T19:28:59.069 INFO:tasks.workunit.client.0.smithi022.stdout:3/263: creat d7/d12/d59/f71 x:0 0 0 2024-02-16T19:28:59.070 INFO:tasks.workunit.client.0.smithi022.stdout:4/265: truncate d4/f47 444365 0 2024-02-16T19:28:59.070 INFO:tasks.workunit.client.0.smithi022.stdout:9/296: dwrite db/d14/d16/f45 [0,4194304] 0 2024-02-16T19:28:59.070 INFO:tasks.workunit.client.0.smithi022.stdout:9/297: chown db/d14/d19/d1a/l47 1026526367 1 2024-02-16T19:28:59.070 INFO:tasks.workunit.client.0.smithi022.stdout:4/266: dwrite d4/d6/d14/f1d [0,4194304] 0 2024-02-16T19:28:59.072 INFO:tasks.workunit.client.1.smithi117.stdout:6/263: dwrite d5/db/f1f [4194304,4194304] 0 2024-02-16T19:28:59.074 INFO:tasks.workunit.client.1.smithi117.stdout:6/264: write d5/d4b/d20/f42 [480954,104642] 0 2024-02-16T19:28:59.075 INFO:tasks.workunit.client.0.smithi022.stdout:6/289: symlink d1/d8/da/d29/d61/d57/l6e 0 2024-02-16T19:28:59.080 INFO:tasks.workunit.client.0.smithi022.stdout:1/375: mknod d6/d8/d22/d67/c6e 0 2024-02-16T19:28:59.084 INFO:tasks.workunit.client.0.smithi022.stdout:0/390: fsync d10/d11/d27/d3f/d58/d60/d62/f69 0 2024-02-16T19:28:59.093 INFO:tasks.workunit.client.1.smithi117.stdout:5/342: mkdir d2/d4c/d69 0 2024-02-16T19:28:59.095 INFO:tasks.workunit.client.1.smithi117.stdout:0/316: mkdir d0/d30/d74 0 2024-02-16T19:28:59.101 INFO:tasks.workunit.client.0.smithi022.stdout:3/264: symlink d7/d12/d18/dd/d24/d58/l72 0 2024-02-16T19:28:59.102 INFO:tasks.workunit.client.1.smithi117.stdout:3/285: mknod d1/dd/c5d 0 2024-02-16T19:28:59.102 INFO:tasks.workunit.client.1.smithi117.stdout:3/286: chown d1/d30/c39 85938 1 2024-02-16T19:28:59.103 INFO:tasks.workunit.client.0.smithi022.stdout:9/298: stat db/d18/l61 0 2024-02-16T19:28:59.103 INFO:tasks.workunit.client.1.smithi117.stdout:3/287: chown d1/d1b/l2f 6854298 1 2024-02-16T19:28:59.104 INFO:tasks.workunit.client.1.smithi117.stdout:3/288: truncate d1/dd/f14 4525494 0 2024-02-16T19:28:59.104 INFO:tasks.workunit.client.1.smithi117.stdout:6/265: chown d5/d4b/d20/d2d/l33 118 1 2024-02-16T19:28:59.104 INFO:tasks.workunit.client.1.smithi117.stdout:3/289: dread - d1/d1b/f21 zero size 2024-02-16T19:28:59.110 INFO:tasks.workunit.client.1.smithi117.stdout:8/246: unlink d4/f38 0 2024-02-16T19:28:59.111 INFO:tasks.workunit.client.0.smithi022.stdout:1/376: dread - d6/d8/d22/d16/d17/f4c zero size 2024-02-16T19:28:59.111 INFO:tasks.workunit.client.0.smithi022.stdout:1/377: read - d6/f6b zero size 2024-02-16T19:28:59.112 INFO:tasks.workunit.client.1.smithi117.stdout:8/247: write d4/d12/d31/f36 [953414,76116] 0 2024-02-16T19:28:59.112 INFO:tasks.workunit.client.1.smithi117.stdout:4/262: link d0/f11 d0/d6/f55 0 2024-02-16T19:28:59.112 INFO:tasks.workunit.client.1.smithi117.stdout:1/195: dwrite d2/d4/f17 [4194304,4194304] 0 2024-02-16T19:28:59.118 INFO:tasks.workunit.client.0.smithi022.stdout:8/303: creat d3/d12/d6a/f6e x:0 0 0 2024-02-16T19:28:59.119 INFO:tasks.workunit.client.1.smithi117.stdout:1/196: dwrite d2/d18/d20/f32 [0,4194304] 0 2024-02-16T19:28:59.120 INFO:tasks.workunit.client.1.smithi117.stdout:1/197: chown d2/l38 860018 1 2024-02-16T19:28:59.120 INFO:tasks.workunit.client.1.smithi117.stdout:7/228: creat d4/d1c/d39/f5b x:0 0 0 2024-02-16T19:28:59.123 INFO:tasks.workunit.client.0.smithi022.stdout:9/299: mknod db/d14/d19/d1a/c6a 0 2024-02-16T19:28:59.124 INFO:tasks.workunit.client.1.smithi117.stdout:6/266: symlink d5/d1d/d49/l5c 0 2024-02-16T19:28:59.125 INFO:tasks.workunit.client.1.smithi117.stdout:3/290: mkdir d1/d1b/d1e/d5e 0 2024-02-16T19:28:59.127 INFO:tasks.workunit.client.0.smithi022.stdout:1/378: mkdir d6/d8/d22/d16/d17/d63/d6f 0 2024-02-16T19:28:59.130 INFO:tasks.workunit.client.0.smithi022.stdout:0/391: creat d10/d11/d27/d77/f81 x:0 0 0 2024-02-16T19:28:59.133 INFO:tasks.workunit.client.0.smithi022.stdout:3/265: creat d7/d12/d18/dd/d3c/d5a/d67/f73 x:0 0 0 2024-02-16T19:28:59.134 INFO:tasks.workunit.client.0.smithi022.stdout:3/266: dread - d7/f3b zero size 2024-02-16T19:28:59.134 INFO:tasks.workunit.client.0.smithi022.stdout:3/267: stat d7/c43 0 2024-02-16T19:28:59.135 INFO:tasks.workunit.client.0.smithi022.stdout:3/268: dread - d7/d12/d18/dd/d24/f63 zero size 2024-02-16T19:28:59.137 INFO:tasks.workunit.client.1.smithi117.stdout:4/263: creat d0/da/f56 x:0 0 0 2024-02-16T19:28:59.138 INFO:tasks.workunit.client.1.smithi117.stdout:4/264: chown d0/da/d21/d3d/c26 624844 1 2024-02-16T19:28:59.138 INFO:tasks.workunit.client.1.smithi117.stdout:4/265: write d0/da/f56 [858915,10184] 0 2024-02-16T19:28:59.139 INFO:tasks.workunit.client.1.smithi117.stdout:4/266: write d0/da/d21/f3e [558467,113067] 0 2024-02-16T19:28:59.141 INFO:tasks.workunit.client.0.smithi022.stdout:1/379: creat d6/d8/d22/d16/d17/d5d/f70 x:0 0 0 2024-02-16T19:28:59.142 INFO:tasks.workunit.client.0.smithi022.stdout:3/269: dwrite d7/d12/d18/dd/ff [0,4194304] 0 2024-02-16T19:28:59.142 INFO:tasks.workunit.client.1.smithi117.stdout:5/343: dread d2/d3/d21/d2f/d30/d4e/f34 [0,4194304] 0 2024-02-16T19:28:59.143 INFO:tasks.workunit.client.1.smithi117.stdout:5/344: chown d2/d9/d3c/d42 3134902 1 2024-02-16T19:28:59.144 INFO:tasks.workunit.client.0.smithi022.stdout:0/392: symlink d10/d11/d50/d65/l82 0 2024-02-16T19:28:59.146 INFO:tasks.workunit.client.0.smithi022.stdout:9/300: mkdir db/d14/d16/d5d/d65/d6b 0 2024-02-16T19:28:59.148 INFO:tasks.workunit.client.0.smithi022.stdout:9/301: write db/d14/d16/d21/f4c [216171,108075] 0 2024-02-16T19:28:59.151 INFO:tasks.workunit.client.0.smithi022.stdout:4/267: getdents d4/d6/d3a 0 2024-02-16T19:28:59.152 INFO:tasks.workunit.client.0.smithi022.stdout:0/393: dwrite d10/d15/d23/d38/f41 [0,4194304] 0 2024-02-16T19:28:59.152 INFO:tasks.workunit.client.1.smithi117.stdout:7/229: write d4/f17 [960361,74956] 0 2024-02-16T19:28:59.156 INFO:tasks.workunit.client.1.smithi117.stdout:6/267: mknod d5/d1d/c5d 0 2024-02-16T19:28:59.157 INFO:tasks.workunit.client.0.smithi022.stdout:3/270: rmdir d7/d12/d18/dd/d24 39 2024-02-16T19:28:59.157 INFO:tasks.workunit.client.0.smithi022.stdout:9/302: dwrite db/d14/d19/f17 [0,4194304] 0 2024-02-16T19:28:59.159 INFO:tasks.workunit.client.1.smithi117.stdout:3/291: write d1/dd/f14 [4116109,102782] 0 2024-02-16T19:28:59.161 INFO:tasks.workunit.client.1.smithi117.stdout:4/267: mkdir d0/da/d21/d3d/d28/d57 0 2024-02-16T19:28:59.167 INFO:tasks.workunit.client.0.smithi022.stdout:6/290: creat d1/d8/f6f x:0 0 0 2024-02-16T19:28:59.168 INFO:tasks.workunit.client.0.smithi022.stdout:1/380: creat d6/d66/f71 x:0 0 0 2024-02-16T19:28:59.171 INFO:tasks.workunit.client.0.smithi022.stdout:1/381: dread - d6/d8/d22/d16/d17/d63/f68 zero size 2024-02-16T19:28:59.171 INFO:tasks.workunit.client.1.smithi117.stdout:5/345: creat d2/d9/d3c/d24/f6a x:0 0 0 2024-02-16T19:28:59.173 INFO:tasks.workunit.client.0.smithi022.stdout:6/291: dwrite d1/d8/da/d17/d53/f5c [0,4194304] 0 2024-02-16T19:28:59.178 INFO:tasks.workunit.client.0.smithi022.stdout:6/292: dread d1/d8/da/d17/d53/f5c [0,4194304] 0 2024-02-16T19:28:59.180 INFO:tasks.workunit.client.0.smithi022.stdout:9/303: mknod db/d14/d16/d27/c6c 0 2024-02-16T19:28:59.182 INFO:tasks.workunit.client.1.smithi117.stdout:6/268: dread d5/d4b/f29 [0,4194304] 0 2024-02-16T19:28:59.182 INFO:tasks.workunit.client.1.smithi117.stdout:3/292: creat d1/d1b/f5f x:0 0 0 2024-02-16T19:28:59.183 INFO:tasks.workunit.client.1.smithi117.stdout:6/269: read - d5/d1d/d28/d32/d37/f41 zero size 2024-02-16T19:28:59.184 INFO:tasks.workunit.client.0.smithi022.stdout:9/304: truncate db/d14/d19/d1a/d67/f44 525211 0 2024-02-16T19:28:59.186 INFO:tasks.workunit.client.0.smithi022.stdout:6/293: dwrite d1/d8/d3c/f44 [0,4194304] 0 2024-02-16T19:28:59.187 INFO:tasks.workunit.client.0.smithi022.stdout:4/268: link d4/d6/dd/f36 d4/d6/dd/d22/f59 0 2024-02-16T19:28:59.188 INFO:tasks.workunit.client.1.smithi117.stdout:3/293: dwrite d1/dd/f34 [0,4194304] 0 2024-02-16T19:28:59.190 INFO:tasks.workunit.client.0.smithi022.stdout:9/305: dread db/d14/d16/d21/d37/f4e [0,4194304] 0 2024-02-16T19:28:59.191 INFO:tasks.workunit.client.1.smithi117.stdout:5/346: mknod d2/d36/c6b 0 2024-02-16T19:28:59.192 INFO:tasks.workunit.client.1.smithi117.stdout:5/347: chown d2/d9/d3c/d24 2 1 2024-02-16T19:28:59.194 INFO:tasks.workunit.client.1.smithi117.stdout:0/317: getdents d0/d9/d14/d1d 0 2024-02-16T19:28:59.194 INFO:tasks.workunit.client.0.smithi022.stdout:9/306: dread db/d14/d16/d21/f4c [0,4194304] 0 2024-02-16T19:28:59.195 INFO:tasks.workunit.client.0.smithi022.stdout:9/307: read - db/d14/f60 zero size 2024-02-16T19:28:59.199 INFO:tasks.workunit.client.1.smithi117.stdout:5/348: dwrite d2/d3/f12 [0,4194304] 0 2024-02-16T19:28:59.200 INFO:tasks.workunit.client.0.smithi022.stdout:3/271: symlink d7/d12/d18/dd/d24/d58/l74 0 2024-02-16T19:28:59.200 INFO:tasks.workunit.client.1.smithi117.stdout:0/318: dread d0/d9/d14/f1b [0,4194304] 0 2024-02-16T19:28:59.201 INFO:tasks.workunit.client.1.smithi117.stdout:5/349: write d2/d3/f12 [1418969,46929] 0 2024-02-16T19:28:59.201 INFO:tasks.workunit.client.1.smithi117.stdout:5/350: dread - d2/f63 zero size 2024-02-16T19:28:59.205 INFO:tasks.workunit.client.0.smithi022.stdout:6/294: symlink d1/l70 0 2024-02-16T19:28:59.206 INFO:tasks.workunit.client.0.smithi022.stdout:3/272: dwrite d7/d21/f36 [4194304,4194304] 0 2024-02-16T19:28:59.207 INFO:tasks.workunit.client.1.smithi117.stdout:2/199: mknod d0/c3d 0 2024-02-16T19:28:59.209 INFO:tasks.workunit.client.1.smithi117.stdout:6/270: unlink d5/d1d/d28/d32/d37/f47 0 2024-02-16T19:28:59.211 INFO:tasks.workunit.client.0.smithi022.stdout:1/382: fdatasync d6/d8/d22/d16/d17/d1d/f33 0 2024-02-16T19:28:59.214 INFO:tasks.workunit.client.1.smithi117.stdout:7/230: unlink d4/f49 0 2024-02-16T19:28:59.215 INFO:tasks.workunit.client.0.smithi022.stdout:7/287: write d9/da/d2c/f2d [1455913,87763] 0 2024-02-16T19:28:59.216 INFO:tasks.workunit.client.0.smithi022.stdout:0/394: link d10/d22/d49/c4c d10/d11/d50/d65/c83 0 2024-02-16T19:28:59.218 INFO:tasks.workunit.client.0.smithi022.stdout:0/395: truncate d10/d11/d27/d3f/f5b 1474633 0 2024-02-16T19:28:59.218 INFO:tasks.workunit.client.0.smithi022.stdout:0/396: write d10/d11/f2e [1291865,1896] 0 2024-02-16T19:28:59.219 INFO:tasks.workunit.client.0.smithi022.stdout:0/397: chown d10/d11/c70 1 1 2024-02-16T19:28:59.220 INFO:tasks.workunit.client.0.smithi022.stdout:0/398: readlink d10/d15/d23/l4b 0 2024-02-16T19:28:59.222 INFO:tasks.workunit.client.0.smithi022.stdout:9/308: creat db/d18/f6d x:0 0 0 2024-02-16T19:28:59.225 INFO:tasks.workunit.client.1.smithi117.stdout:5/351: creat d2/d4c/d4f/f6c x:0 0 0 2024-02-16T19:28:59.225 INFO:tasks.workunit.client.1.smithi117.stdout:5/352: dread - d2/d3/d21/d49/d41/f4b zero size 2024-02-16T19:28:59.226 INFO:tasks.workunit.client.1.smithi117.stdout:5/353: write d2/d9/d3c/d24/f26 [3341986,111763] 0 2024-02-16T19:28:59.227 INFO:tasks.workunit.client.1.smithi117.stdout:2/200: mknod d0/d29/c3e 0 2024-02-16T19:28:59.227 INFO:tasks.workunit.client.1.smithi117.stdout:5/354: fsync d2/d1d/f55 0 2024-02-16T19:28:59.228 INFO:tasks.workunit.client.0.smithi022.stdout:9/309: dread db/f12 [0,4194304] 0 2024-02-16T19:28:59.229 INFO:tasks.workunit.client.1.smithi117.stdout:1/198: rmdir d2/d19 39 2024-02-16T19:28:59.231 INFO:tasks.workunit.client.1.smithi117.stdout:6/271: write d5/d1d/f50 [133916,47551] 0 2024-02-16T19:28:59.232 INFO:tasks.workunit.client.0.smithi022.stdout:6/295: mknod d1/d24/d56/c71 0 2024-02-16T19:28:59.232 INFO:tasks.workunit.client.0.smithi022.stdout:3/273: creat d7/d12/d18/dd/d3c/d5a/d67/f75 x:0 0 0 2024-02-16T19:28:59.232 INFO:tasks.workunit.client.1.smithi117.stdout:6/272: chown d5/d1d/d28/d32/d37 11 1 2024-02-16T19:28:59.233 INFO:tasks.workunit.client.1.smithi117.stdout:6/273: dread d5/f12 [4194304,4194304] 0 2024-02-16T19:28:59.233 INFO:tasks.workunit.client.1.smithi117.stdout:4/268: rmdir d0/da/d21/d3d/d28/d57 0 2024-02-16T19:28:59.235 INFO:tasks.workunit.client.1.smithi117.stdout:3/294: mknod d1/dd/c60 0 2024-02-16T19:28:59.237 INFO:tasks.workunit.client.1.smithi117.stdout:7/231: unlink d4/fd 0 2024-02-16T19:28:59.238 INFO:tasks.workunit.client.1.smithi117.stdout:7/232: fsync d4/d1c/f2c 0 2024-02-16T19:28:59.238 INFO:tasks.workunit.client.1.smithi117.stdout:4/269: dwrite d0/da/f56 [0,4194304] 0 2024-02-16T19:28:59.241 INFO:tasks.workunit.client.1.smithi117.stdout:0/319: mkdir d0/d9/d14/d75 0 2024-02-16T19:28:59.242 INFO:tasks.workunit.client.1.smithi117.stdout:0/320: chown d0/d9/d22/d27/f5f 2621131 1 2024-02-16T19:28:59.248 INFO:tasks.workunit.client.0.smithi022.stdout:9/310: dread db/f12 [4194304,4194304] 0 2024-02-16T19:28:59.248 INFO:tasks.workunit.client.1.smithi117.stdout:1/199: unlink d2/f31 0 2024-02-16T19:28:59.249 INFO:tasks.workunit.client.1.smithi117.stdout:9/215: write dd/d23/f25 [338768,116191] 0 2024-02-16T19:28:59.252 INFO:tasks.workunit.client.0.smithi022.stdout:1/383: unlink d6/d8/d22/d16/d17/d26/c3b 0 2024-02-16T19:28:59.252 INFO:tasks.workunit.client.0.smithi022.stdout:1/384: stat l2 0 2024-02-16T19:28:59.254 INFO:tasks.workunit.client.0.smithi022.stdout:1/385: write d6/d8/d22/d16/f2e [131630,57608] 0 2024-02-16T19:28:59.254 INFO:tasks.workunit.client.0.smithi022.stdout:1/386: readlink d6/l5c 0 2024-02-16T19:28:59.256 INFO:tasks.workunit.client.0.smithi022.stdout:2/212: truncate d0/d2/f14 1725349 0 2024-02-16T19:28:59.261 INFO:tasks.workunit.client.0.smithi022.stdout:6/296: unlink d1/d8/da/d17/d48/l6b 0 2024-02-16T19:28:59.261 INFO:tasks.workunit.client.1.smithi117.stdout:7/233: rmdir d4/d1c/d39/d46 39 2024-02-16T19:28:59.262 INFO:tasks.workunit.client.1.smithi117.stdout:1/200: dread d2/d4/f11 [0,4194304] 0 2024-02-16T19:28:59.263 INFO:tasks.workunit.client.0.smithi022.stdout:3/274: creat d7/d12/d18/dd/d24/d53/f76 x:0 0 0 2024-02-16T19:28:59.264 INFO:tasks.workunit.client.1.smithi117.stdout:4/270: rmdir d0/da/d21/d3d/d2d 39 2024-02-16T19:28:59.264 INFO:tasks.workunit.client.0.smithi022.stdout:3/275: dread d7/d21/f38 [0,4194304] 0 2024-02-16T19:28:59.266 INFO:tasks.workunit.client.0.smithi022.stdout:9/311: mknod db/d14/d16/d27/c6e 0 2024-02-16T19:28:59.267 INFO:tasks.workunit.client.1.smithi117.stdout:2/201: truncate d0/f1e 2592016 0 2024-02-16T19:28:59.270 INFO:tasks.workunit.client.0.smithi022.stdout:1/387: creat d6/d8/d22/d67/f72 x:0 0 0 2024-02-16T19:28:59.276 INFO:tasks.workunit.client.0.smithi022.stdout:8/304: rmdir d3/d12/d17/d58 0 2024-02-16T19:28:59.277 INFO:tasks.workunit.client.1.smithi117.stdout:9/216: rmdir dd/d23/d2e/d2c 39 2024-02-16T19:28:59.280 INFO:tasks.workunit.client.1.smithi117.stdout:6/274: creat d5/d1d/d49/d59/f5e x:0 0 0 2024-02-16T19:28:59.281 INFO:tasks.workunit.client.1.smithi117.stdout:6/275: read d5/f12 [4172205,66541] 0 2024-02-16T19:28:59.282 INFO:tasks.workunit.client.0.smithi022.stdout:3/276: creat d7/d12/d18/dd/d24/d29/f77 x:0 0 0 2024-02-16T19:28:59.283 INFO:tasks.workunit.client.0.smithi022.stdout:3/277: write d7/d12/d18/dd/d31/d41/f48 [1313981,35674] 0 2024-02-16T19:28:59.284 INFO:tasks.workunit.client.0.smithi022.stdout:3/278: chown d7/d12/d18/dd/d24 65685290 1 2024-02-16T19:28:59.285 INFO:tasks.workunit.client.1.smithi117.stdout:3/295: symlink d1/dd/l61 0 2024-02-16T19:28:59.285 INFO:tasks.workunit.client.0.smithi022.stdout:3/279: fdatasync d7/d12/d18/dd/f2e 0 2024-02-16T19:28:59.286 INFO:tasks.workunit.client.1.smithi117.stdout:3/296: chown d1/d1b/d1e/f54 13173 1 2024-02-16T19:28:59.287 INFO:tasks.workunit.client.0.smithi022.stdout:3/280: readlink d7/d12/d18/dd/d24/d58/l68 0 2024-02-16T19:28:59.288 INFO:tasks.workunit.client.1.smithi117.stdout:3/297: dread d1/d1b/f2d [0,4194304] 0 2024-02-16T19:28:59.289 INFO:tasks.workunit.client.1.smithi117.stdout:3/298: write d1/d1b/f41 [672474,128070] 0 2024-02-16T19:28:59.290 INFO:tasks.workunit.client.0.smithi022.stdout:2/213: mkdir d0/d48 0 2024-02-16T19:28:59.293 INFO:tasks.workunit.client.0.smithi022.stdout:8/305: truncate d3/d12/d17/f3b 5230503 0 2024-02-16T19:28:59.299 INFO:tasks.workunit.client.1.smithi117.stdout:6/276: creat d5/d4b/d44/d52/f5f x:0 0 0 2024-02-16T19:28:59.304 INFO:tasks.workunit.client.0.smithi022.stdout:1/388: rename d6/d8/d22/d16/d17/f1a to d6/d8/d22/d3f/f73 0 2024-02-16T19:28:59.305 INFO:tasks.workunit.client.0.smithi022.stdout:1/389: fdatasync d6/d66/f71 0 2024-02-16T19:28:59.306 INFO:tasks.workunit.client.0.smithi022.stdout:1/390: fsync d6/d8/d22/d16/d17/f4c 0 2024-02-16T19:28:59.307 INFO:tasks.workunit.client.0.smithi022.stdout:8/306: symlink d3/d12/d2b/d2e/l6f 0 2024-02-16T19:28:59.309 INFO:tasks.workunit.client.0.smithi022.stdout:8/307: truncate d3/d25/d33/f41 285823 0 2024-02-16T19:28:59.311 INFO:tasks.workunit.client.0.smithi022.stdout:8/308: dread - d3/d12/d2b/d2e/f59 zero size 2024-02-16T19:28:59.312 INFO:tasks.workunit.client.1.smithi117.stdout:7/234: unlink d4/f17 0 2024-02-16T19:28:59.312 INFO:tasks.workunit.client.0.smithi022.stdout:1/391: dwrite d6/f6b [0,4194304] 0 2024-02-16T19:28:59.315 INFO:tasks.workunit.client.0.smithi022.stdout:3/281: rename d7/d12/d18/dd/d24/d29/f77 to d7/d12/d18/dd/d24/d58/d6b/f78 0 2024-02-16T19:28:59.316 INFO:tasks.workunit.client.1.smithi117.stdout:7/235: dwrite d4/d1c/f2e [0,4194304] 0 2024-02-16T19:28:59.317 INFO:tasks.workunit.client.1.smithi117.stdout:7/236: chown d4/f28 1 1 2024-02-16T19:28:59.324 INFO:tasks.workunit.client.0.smithi022.stdout:8/309: creat d3/d12/d2b/d54/f70 x:0 0 0 2024-02-16T19:28:59.326 INFO:tasks.workunit.client.0.smithi022.stdout:1/392: dread - d6/d8/d22/d16/d17/f52 zero size 2024-02-16T19:28:59.327 INFO:tasks.workunit.client.0.smithi022.stdout:1/393: chown d6/d8/d22/d16/d17/d26/f43 92 1 2024-02-16T19:28:59.332 INFO:tasks.workunit.client.1.smithi117.stdout:8/248: rename d4/d7/fc to d4/f53 0 2024-02-16T19:28:59.338 INFO:tasks.workunit.client.0.smithi022.stdout:4/269: truncate d4/f47 441730 0 2024-02-16T19:28:59.338 INFO:tasks.workunit.client.1.smithi117.stdout:3/299: creat d1/d1b/d1e/d5e/f62 x:0 0 0 2024-02-16T19:28:59.339 INFO:tasks.workunit.client.1.smithi117.stdout:3/300: fsync d1/d1b/d1e/d29/f33 0 2024-02-16T19:28:59.340 INFO:tasks.workunit.client.1.smithi117.stdout:9/217: symlink dd/d23/d2e/d2c/d31/l3d 0 2024-02-16T19:28:59.341 INFO:tasks.workunit.client.0.smithi022.stdout:4/270: dread d4/d6/d14/f2e [0,4194304] 0 2024-02-16T19:28:59.342 INFO:tasks.workunit.client.0.smithi022.stdout:1/394: unlink d6/d8/d22/f11 0 2024-02-16T19:28:59.343 INFO:tasks.workunit.client.1.smithi117.stdout:6/277: symlink d5/l60 0 2024-02-16T19:28:59.344 INFO:tasks.workunit.client.0.smithi022.stdout:5/315: sync 2024-02-16T19:28:59.344 INFO:tasks.workunit.client.0.smithi022.stdout:5/316: fsync f3 0 2024-02-16T19:28:59.345 INFO:tasks.workunit.client.0.smithi022.stdout:2/214: rename d0/d2/d18/d1f/c3f to d0/d32/d3b/c49 0 2024-02-16T19:28:59.348 INFO:tasks.workunit.client.1.smithi117.stdout:0/321: rename d0/d9/d14/d75 to d0/d9/d14/d1d/d60/d76 0 2024-02-16T19:28:59.349 INFO:tasks.workunit.client.0.smithi022.stdout:3/282: mknod d7/d12/d18/dd/d24/d58/d70/c79 0 2024-02-16T19:28:59.349 INFO:tasks.workunit.client.1.smithi117.stdout:6/278: dwrite d5/d1d/d49/d59/f5e [0,4194304] 0 2024-02-16T19:28:59.349 INFO:tasks.workunit.client.1.smithi117.stdout:6/279: chown c3 2671 1 2024-02-16T19:28:59.350 INFO:tasks.workunit.client.0.smithi022.stdout:3/283: write d7/f1e [4224177,31138] 0 2024-02-16T19:28:59.351 INFO:tasks.workunit.client.1.smithi117.stdout:8/249: chown d4/d7/d37/d50/l51 3196 1 2024-02-16T19:28:59.355 INFO:tasks.workunit.client.0.smithi022.stdout:4/271: mkdir d4/d6/dd/d31/d4e/d54/d5a 0 2024-02-16T19:28:59.357 INFO:tasks.workunit.client.1.smithi117.stdout:3/301: creat d1/dd/d3b/f63 x:0 0 0 2024-02-16T19:28:59.358 INFO:tasks.workunit.client.0.smithi022.stdout:2/215: dread d0/d2/d18/d1f/f21 [0,4194304] 0 2024-02-16T19:28:59.359 INFO:tasks.workunit.client.0.smithi022.stdout:1/395: fdatasync d6/d8/d22/d16/d17/d26/f43 0 2024-02-16T19:28:59.360 INFO:tasks.workunit.client.1.smithi117.stdout:9/218: creat dd/f3e x:0 0 0 2024-02-16T19:28:59.362 INFO:tasks.workunit.client.0.smithi022.stdout:5/317: creat dd/d1f/d48/d21/d2f/d52/d5c/f63 x:0 0 0 2024-02-16T19:28:59.362 INFO:tasks.workunit.client.1.smithi117.stdout:3/302: dwrite d1/d30/d52/d56/f3f [0,4194304] 0 2024-02-16T19:28:59.363 INFO:tasks.workunit.client.0.smithi022.stdout:5/318: dread - dd/d1f/d48/f60 zero size 2024-02-16T19:28:59.376 INFO:tasks.workunit.client.1.smithi117.stdout:5/355: truncate d2/d3/f12 3421110 0 2024-02-16T19:28:59.377 INFO:tasks.workunit.client.1.smithi117.stdout:5/356: readlink d2/d9/d3c/d42/l66 0 2024-02-16T19:28:59.377 INFO:tasks.workunit.client.1.smithi117.stdout:5/357: stat d2/d9/d3c/d24/d50/d52 0 2024-02-16T19:28:59.422 INFO:tasks.workunit.client.0.smithi022.stdout:2/216: mkdir d0/d2/d2e/d4a 0 2024-02-16T19:28:59.425 INFO:tasks.workunit.client.0.smithi022.stdout:1/396: symlink d6/d8/d22/d16/l74 0 2024-02-16T19:28:59.425 INFO:tasks.workunit.client.1.smithi117.stdout:1/201: dwrite d2/d1d/f25 [0,4194304] 0 2024-02-16T19:28:59.427 INFO:tasks.workunit.client.1.smithi117.stdout:3/303: unlink d1/d30/f57 0 2024-02-16T19:28:59.428 INFO:tasks.workunit.client.0.smithi022.stdout:9/312: truncate db/d14/d19/f4d 3343777 0 2024-02-16T19:28:59.432 INFO:tasks.workunit.client.1.smithi117.stdout:1/202: dwrite d2/d18/d20/f2b [0,4194304] 0 2024-02-16T19:28:59.433 INFO:tasks.workunit.client.1.smithi117.stdout:3/304: dread d1/d1b/d1e/d29/f33 [0,4194304] 0 2024-02-16T19:28:59.433 INFO:tasks.workunit.client.1.smithi117.stdout:1/203: write d2/d4/f17 [6377829,77881] 0 2024-02-16T19:28:59.436 INFO:tasks.workunit.client.0.smithi022.stdout:9/313: dwrite db/d14/d19/d1a/f24 [0,4194304] 0 2024-02-16T19:28:59.443 INFO:tasks.workunit.client.0.smithi022.stdout:3/284: rename d7/d12/d18/dd/d24/d53/f76 to d7/d12/d18/dd/d24/d58/d6b/f7a 0 2024-02-16T19:28:59.447 INFO:tasks.workunit.client.0.smithi022.stdout:5/319: mkdir dd/d1f/d48/d21/d64 0 2024-02-16T19:28:59.452 INFO:tasks.workunit.client.0.smithi022.stdout:9/314: symlink db/d14/d16/d27/d2e/l6f 0 2024-02-16T19:28:59.457 INFO:tasks.workunit.client.0.smithi022.stdout:1/397: rename l2 to d6/d8/d22/d47/d54/l75 0 2024-02-16T19:28:59.459 INFO:tasks.workunit.client.0.smithi022.stdout:3/285: creat d7/d12/d18/dd/d24/d58/d70/f7b x:0 0 0 2024-02-16T19:28:59.460 INFO:tasks.workunit.client.0.smithi022.stdout:3/286: fdatasync d7/d12/d18/dd/d31/f47 0 2024-02-16T19:28:59.461 INFO:tasks.workunit.client.0.smithi022.stdout:2/217: symlink d0/d2/d18/d38/l4b 0 2024-02-16T19:28:59.463 INFO:tasks.workunit.client.0.smithi022.stdout:5/320: dread - dd/d1f/d48/f26 zero size 2024-02-16T19:28:59.466 INFO:tasks.workunit.client.0.smithi022.stdout:9/315: symlink db/d14/d19/d1a/l70 0 2024-02-16T19:28:59.468 INFO:tasks.workunit.client.1.smithi117.stdout:6/280: sync 2024-02-16T19:28:59.468 INFO:tasks.workunit.client.1.smithi117.stdout:1/204: rmdir d2/d18 39 2024-02-16T19:28:59.468 INFO:tasks.workunit.client.1.smithi117.stdout:6/281: readlink d5/db/l3e 0 2024-02-16T19:28:59.472 INFO:tasks.workunit.client.0.smithi022.stdout:1/398: rename d6/d8/d22/d16/d17/d5d/f70 to d6/d8/d22/d3a/d3e/f76 0 2024-02-16T19:28:59.475 INFO:tasks.workunit.client.1.smithi117.stdout:8/250: rmdir d4/d4d 0 2024-02-16T19:28:59.476 INFO:tasks.workunit.client.1.smithi117.stdout:8/251: stat d4/d7/l29 0 2024-02-16T19:28:59.477 INFO:tasks.workunit.client.0.smithi022.stdout:3/287: truncate d7/f1c 2918494 0 2024-02-16T19:28:59.479 INFO:tasks.workunit.client.1.smithi117.stdout:9/219: creat dd/d23/d2e/d2c/f3f x:0 0 0 2024-02-16T19:28:59.479 INFO:tasks.workunit.client.0.smithi022.stdout:1/399: dwrite d6/d8/d22/d16/d17/f4c [0,4194304] 0 2024-02-16T19:28:59.481 INFO:tasks.workunit.client.1.smithi117.stdout:9/220: readlink dd/d23/d2e/d2c/d31/l3d 0 2024-02-16T19:28:59.481 INFO:tasks.workunit.client.0.smithi022.stdout:2/218: mknod d0/d2/d18/d19/d40/c4c 0 2024-02-16T19:28:59.481 INFO:tasks.workunit.client.1.smithi117.stdout:9/221: chown dd/l1d 506171 1 2024-02-16T19:28:59.482 INFO:tasks.workunit.client.0.smithi022.stdout:2/219: chown d0/d2/d18/d1f 408673786 1 2024-02-16T19:28:59.482 INFO:tasks.workunit.client.0.smithi022.stdout:3/288: dread d7/d12/d18/f1b [0,4194304] 0 2024-02-16T19:28:59.482 INFO:tasks.workunit.client.1.smithi117.stdout:9/222: chown dd/d24/f39 1 1 2024-02-16T19:28:59.483 INFO:tasks.workunit.client.0.smithi022.stdout:2/220: chown d0/d2/d18/d19 2015033 1 2024-02-16T19:28:59.484 INFO:tasks.workunit.client.1.smithi117.stdout:8/252: dwrite d4/d7/d37/d3a/d43/d44/f4b [0,4194304] 0 2024-02-16T19:28:59.485 INFO:tasks.workunit.client.1.smithi117.stdout:8/253: chown d4/d12/d2a/f2b 132127 1 2024-02-16T19:28:59.486 INFO:tasks.workunit.client.0.smithi022.stdout:3/289: chown d7/d12/d18/f1a 1206986 1 2024-02-16T19:28:59.486 INFO:tasks.workunit.client.1.smithi117.stdout:4/271: rename d0/da/d21/d3d/f1d to d0/da/d4a/f58 0 2024-02-16T19:28:59.487 INFO:tasks.workunit.client.0.smithi022.stdout:5/321: creat dd/d1f/d48/d21/d2f/d52/f65 x:0 0 0 2024-02-16T19:28:59.488 INFO:tasks.workunit.client.1.smithi117.stdout:3/305: mknod d1/c64 0 2024-02-16T19:28:59.489 INFO:tasks.workunit.client.1.smithi117.stdout:2/202: write d0/f1e [3250470,63397] 0 2024-02-16T19:28:59.489 INFO:tasks.workunit.client.0.smithi022.stdout:1/400: dwrite d6/d8/d22/d16/d17/d1d/f21 [0,4194304] 0 2024-02-16T19:28:59.490 INFO:tasks.workunit.client.1.smithi117.stdout:2/203: chown d0/c3d 85 1 2024-02-16T19:28:59.490 INFO:tasks.workunit.client.0.smithi022.stdout:9/316: creat db/d14/d16/d27/d55/f71 x:0 0 0 2024-02-16T19:28:59.491 INFO:tasks.workunit.client.0.smithi022.stdout:9/317: stat db/d14/d16/d27/d2e 0 2024-02-16T19:28:59.493 INFO:tasks.workunit.client.0.smithi022.stdout:1/401: write d6/d8/d22/d16/d17/d1d/f21 [5546579,113996] 0 2024-02-16T19:28:59.494 INFO:tasks.workunit.client.0.smithi022.stdout:1/402: fdatasync d6/d8/d22/d16/d17/d63/f68 0 2024-02-16T19:28:59.502 INFO:tasks.workunit.client.1.smithi117.stdout:8/254: unlink d4/d1c/c4e 0 2024-02-16T19:28:59.504 INFO:tasks.workunit.client.0.smithi022.stdout:2/221: creat d0/d2/d18/d1f/f4d x:0 0 0 2024-02-16T19:28:59.504 INFO:tasks.workunit.client.0.smithi022.stdout:2/222: dread - d0/d2/f3d zero size 2024-02-16T19:28:59.505 INFO:tasks.workunit.client.0.smithi022.stdout:2/223: truncate d0/d2/f39 119446 0 2024-02-16T19:28:59.506 INFO:tasks.workunit.client.0.smithi022.stdout:2/224: stat d0/f2d 0 2024-02-16T19:28:59.507 INFO:tasks.workunit.client.1.smithi117.stdout:7/237: rename d4/d13/c20 to d4/d1c/d24/c5c 0 2024-02-16T19:28:59.508 INFO:tasks.workunit.client.1.smithi117.stdout:7/238: truncate d4/d13/f16 8965627 0 2024-02-16T19:28:59.511 INFO:tasks.workunit.client.1.smithi117.stdout:0/322: write d0/f1 [471339,125817] 0 2024-02-16T19:28:59.513 INFO:tasks.workunit.client.1.smithi117.stdout:5/358: write d2/d3/f6 [429056,27252] 0 2024-02-16T19:28:59.515 INFO:tasks.workunit.client.1.smithi117.stdout:5/359: write d2/d9/d3c/d24/f6a [446981,63154] 0 2024-02-16T19:28:59.515 INFO:tasks.workunit.client.1.smithi117.stdout:3/306: creat d1/d1b/d1e/d37/f65 x:0 0 0 2024-02-16T19:28:59.518 INFO:tasks.workunit.client.1.smithi117.stdout:1/205: mknod d2/d4/c3f 0 2024-02-16T19:28:59.519 INFO:tasks.workunit.client.0.smithi022.stdout:9/318: mknod db/d14/d19/d1a/d67/c72 0 2024-02-16T19:28:59.520 INFO:tasks.workunit.client.0.smithi022.stdout:9/319: chown db/d14/d33 3971711 1 2024-02-16T19:28:59.521 INFO:tasks.workunit.client.1.smithi117.stdout:2/204: mknod d0/d3/d22/c3f 0 2024-02-16T19:28:59.522 INFO:tasks.workunit.client.1.smithi117.stdout:5/360: dwrite d2/d9/f40 [0,4194304] 0 2024-02-16T19:28:59.522 INFO:tasks.workunit.client.0.smithi022.stdout:1/403: creat d6/d8/d22/d67/f77 x:0 0 0 2024-02-16T19:28:59.523 INFO:tasks.workunit.client.1.smithi117.stdout:5/361: fsync d2/f63 0 2024-02-16T19:28:59.523 INFO:tasks.workunit.client.0.smithi022.stdout:1/404: write d6/d8/d22/d67/f77 [948127,58858] 0 2024-02-16T19:28:59.530 INFO:tasks.workunit.client.0.smithi022.stdout:3/290: rmdir d7/d12/d18/dd/d24/d58/d70 39 2024-02-16T19:28:59.531 INFO:tasks.workunit.client.1.smithi117.stdout:5/362: dwrite d2/d1d/f5f [0,4194304] 0 2024-02-16T19:28:59.532 INFO:tasks.workunit.client.1.smithi117.stdout:7/239: unlink d4/d1c/d24/f29 0 2024-02-16T19:28:59.533 INFO:tasks.workunit.client.1.smithi117.stdout:7/240: read d4/d1c/f2e [2921398,114297] 0 2024-02-16T19:28:59.534 INFO:tasks.workunit.client.1.smithi117.stdout:7/241: chown d4/d4a/f4c 2790853 1 2024-02-16T19:28:59.535 INFO:tasks.workunit.client.1.smithi117.stdout:4/272: rmdir d0/da/d21/d3d/d2d 39 2024-02-16T19:28:59.536 INFO:tasks.workunit.client.1.smithi117.stdout:4/273: write d0/da/d21/d3d/d28/d2f/d37/f54 [270368,11094] 0 2024-02-16T19:28:59.538 INFO:tasks.workunit.client.0.smithi022.stdout:3/291: dwrite d7/d12/d18/dd/d3c/d5a/d67/f73 [0,4194304] 0 2024-02-16T19:28:59.541 INFO:tasks.workunit.client.0.smithi022.stdout:7/288: sync 2024-02-16T19:28:59.542 INFO:tasks.workunit.client.0.smithi022.stdout:7/289: fsync d9/da/d1f/d44/f49 0 2024-02-16T19:28:59.543 INFO:tasks.workunit.client.1.smithi117.stdout:0/323: symlink d0/d9/d14/d23/d33/l77 0 2024-02-16T19:28:59.547 INFO:tasks.workunit.client.1.smithi117.stdout:3/307: creat d1/d1b/d1e/d5e/f66 x:0 0 0 2024-02-16T19:28:59.556 INFO:tasks.workunit.client.1.smithi117.stdout:2/205: mkdir d0/d3/d40 0 2024-02-16T19:28:59.558 INFO:tasks.workunit.client.1.smithi117.stdout:9/223: creat dd/d23/d2e/d2c/f40 x:0 0 0 2024-02-16T19:28:59.565 INFO:tasks.workunit.client.1.smithi117.stdout:7/242: mknod d4/d13/c5d 0 2024-02-16T19:28:59.567 INFO:tasks.workunit.client.0.smithi022.stdout:1/405: getdents d6/d8/d22/d16 0 2024-02-16T19:28:59.568 INFO:tasks.workunit.client.0.smithi022.stdout:5/322: write dd/d1f/d48/d21/d2f/d40/d49/f4c [456827,84882] 0 2024-02-16T19:28:59.569 INFO:tasks.workunit.client.0.smithi022.stdout:1/406: read - d6/d8/d22/d67/f72 zero size 2024-02-16T19:28:59.569 INFO:tasks.workunit.client.0.smithi022.stdout:3/292: dread d7/d12/f19 [0,4194304] 0 2024-02-16T19:28:59.571 INFO:tasks.workunit.client.0.smithi022.stdout:3/293: dread - d7/d21/f52 zero size 2024-02-16T19:28:59.575 INFO:tasks.workunit.client.0.smithi022.stdout:7/290: getdents d9/d1b 0 2024-02-16T19:28:59.575 INFO:tasks.workunit.client.0.smithi022.stdout:7/291: chown d9/da/d2c/d2f/d21/l34 411330886 1 2024-02-16T19:28:59.577 INFO:tasks.workunit.client.0.smithi022.stdout:2/225: write d0/d2/f3c [2118977,104736] 0 2024-02-16T19:28:59.579 INFO:tasks.workunit.client.0.smithi022.stdout:5/323: creat dd/d1f/d48/d21/d2f/d40/d49/d4b/f66 x:0 0 0 2024-02-16T19:28:59.581 INFO:tasks.workunit.client.1.smithi117.stdout:9/224: readlink dd/d23/l33 0 2024-02-16T19:28:59.582 INFO:tasks.workunit.client.0.smithi022.stdout:3/294: creat d7/d21/f7c x:0 0 0 2024-02-16T19:28:59.582 INFO:tasks.workunit.client.1.smithi117.stdout:8/255: dwrite d4/d12/d2a/f2c [0,4194304] 0 2024-02-16T19:28:59.583 INFO:tasks.workunit.client.1.smithi117.stdout:8/256: chown d4/l17 43553 1 2024-02-16T19:28:59.584 INFO:tasks.workunit.client.1.smithi117.stdout:8/257: readlink d4/d1c/l22 0 2024-02-16T19:28:59.586 INFO:tasks.workunit.client.0.smithi022.stdout:9/320: truncate db/d14/d33/f4f 374466 0 2024-02-16T19:28:59.588 INFO:tasks.workunit.client.0.smithi022.stdout:1/407: symlink d6/d8/d22/d16/d17/l78 0 2024-02-16T19:28:59.596 INFO:tasks.workunit.client.0.smithi022.stdout:7/292: truncate f5 7804346 0 2024-02-16T19:28:59.597 INFO:tasks.workunit.client.0.smithi022.stdout:9/321: dwrite db/d14/d16/f45 [4194304,4194304] 0 2024-02-16T19:28:59.598 INFO:tasks.workunit.client.1.smithi117.stdout:3/308: mkdir d1/d67 0 2024-02-16T19:28:59.599 INFO:tasks.workunit.client.0.smithi022.stdout:9/322: fsync db/d14/d16/d27/d55/f62 0 2024-02-16T19:28:59.602 INFO:tasks.workunit.client.1.smithi117.stdout:1/206: creat d2/d4/f40 x:0 0 0 2024-02-16T19:28:59.606 INFO:tasks.workunit.client.1.smithi117.stdout:2/206: write d0/d3/fc [1742759,87010] 0 2024-02-16T19:28:59.614 INFO:tasks.workunit.client.0.smithi022.stdout:1/408: truncate d6/d8/d22/d16/d17/d1d/f33 1346127 0 2024-02-16T19:28:59.614 INFO:tasks.workunit.client.0.smithi022.stdout:2/226: dread d0/f17 [0,4194304] 0 2024-02-16T19:28:59.615 INFO:tasks.workunit.client.0.smithi022.stdout:2/227: chown d0/d2/d18/d38 5 1 2024-02-16T19:28:59.616 INFO:tasks.workunit.client.0.smithi022.stdout:2/228: chown d0/d32/c3e 41624066 1 2024-02-16T19:28:59.618 INFO:tasks.workunit.client.1.smithi117.stdout:7/243: mknod d4/d1c/d39/d46/c5e 0 2024-02-16T19:28:59.619 INFO:tasks.workunit.client.0.smithi022.stdout:9/323: write db/d14/d16/d21/d37/f4e [680518,11010] 0 2024-02-16T19:28:59.621 INFO:tasks.workunit.client.0.smithi022.stdout:9/324: chown db/d14/d19/d1a/c6a 74304100 1 2024-02-16T19:28:59.623 INFO:tasks.workunit.client.1.smithi117.stdout:0/324: rmdir d0/d41 0 2024-02-16T19:28:59.624 INFO:tasks.workunit.client.1.smithi117.stdout:0/325: chown d0/d9/d14/d1d/d60/f49 1776916076 1 2024-02-16T19:28:59.625 INFO:tasks.workunit.client.1.smithi117.stdout:3/309: rename d1/dd/c13 to d1/d1b/d20/c68 0 2024-02-16T19:28:59.627 INFO:tasks.workunit.client.1.smithi117.stdout:1/207: mknod d2/d1d/c41 0 2024-02-16T19:28:59.630 INFO:tasks.workunit.client.0.smithi022.stdout:7/293: write d9/da/f13 [9402793,113968] 0 2024-02-16T19:28:59.631 INFO:tasks.workunit.client.0.smithi022.stdout:7/294: dread - d9/d1b/f57 zero size 2024-02-16T19:28:59.631 INFO:tasks.workunit.client.1.smithi117.stdout:2/207: rmdir d0/d13 39 2024-02-16T19:28:59.632 INFO:tasks.workunit.client.0.smithi022.stdout:7/295: write d9/da/d2c/f2d [2228533,121518] 0 2024-02-16T19:28:59.641 INFO:tasks.workunit.client.0.smithi022.stdout:3/295: getdents d7/d12/d18/dd/d31/d41 0 2024-02-16T19:28:59.641 INFO:tasks.workunit.client.0.smithi022.stdout:3/296: fsync d7/d12/f44 0 2024-02-16T19:28:59.642 INFO:tasks.workunit.client.1.smithi117.stdout:7/244: mkdir d4/d1c/d24/d5f 0 2024-02-16T19:28:59.643 INFO:tasks.workunit.client.1.smithi117.stdout:8/258: write d4/d1c/f1f [925039,24060] 0 2024-02-16T19:28:59.645 INFO:tasks.workunit.client.1.smithi117.stdout:8/259: read d4/d12/d31/f36 [712511,38765] 0 2024-02-16T19:28:59.647 INFO:tasks.workunit.client.1.smithi117.stdout:0/326: unlink d0/d9/d22/d26/c2a 0 2024-02-16T19:28:59.649 INFO:tasks.workunit.client.0.smithi022.stdout:1/409: truncate d6/d8/d22/d16/f40 2209339 0 2024-02-16T19:28:59.650 INFO:tasks.workunit.client.0.smithi022.stdout:1/410: write d6/f5b [656006,106587] 0 2024-02-16T19:28:59.651 INFO:tasks.workunit.client.1.smithi117.stdout:8/260: dwrite d4/f52 [0,4194304] 0 2024-02-16T19:28:59.651 INFO:tasks.workunit.client.0.smithi022.stdout:9/325: rename db/d18/f26 to db/d18/f73 0 2024-02-16T19:28:59.656 INFO:tasks.workunit.client.0.smithi022.stdout:3/297: unlink d7/d12/d18/dd/d3c/d5a/d67/f73 0 2024-02-16T19:28:59.657 INFO:tasks.workunit.client.1.smithi117.stdout:7/245: mkdir d4/d1c/d24/d60 0 2024-02-16T19:28:59.658 INFO:tasks.workunit.client.1.smithi117.stdout:6/282: sync 2024-02-16T19:28:59.659 INFO:tasks.workunit.client.1.smithi117.stdout:6/283: truncate d5/d4b/d44/d52/f5f 924966 0 2024-02-16T19:28:59.660 INFO:tasks.workunit.client.1.smithi117.stdout:4/274: getdents d0/da/d21/d3d/d2d 0 2024-02-16T19:28:59.660 INFO:tasks.workunit.client.1.smithi117.stdout:4/275: readlink d0/l42 0 2024-02-16T19:28:59.662 INFO:tasks.workunit.client.0.smithi022.stdout:9/326: creat db/d18/f74 x:0 0 0 2024-02-16T19:28:59.662 INFO:tasks.workunit.client.0.smithi022.stdout:9/327: fdatasync db/d18/f2a 0 2024-02-16T19:28:59.664 INFO:tasks.workunit.client.1.smithi117.stdout:4/276: dread d0/da/d21/d3d/f29 [0,4194304] 0 2024-02-16T19:28:59.668 INFO:tasks.workunit.client.1.smithi117.stdout:0/327: unlink d0/f7 0 2024-02-16T19:28:59.671 INFO:tasks.workunit.client.0.smithi022.stdout:9/328: symlink db/d14/d19/d1a/d67/l75 0 2024-02-16T19:28:59.672 INFO:tasks.workunit.client.0.smithi022.stdout:9/329: chown db/d14/d16/f1f 101593 1 2024-02-16T19:28:59.673 INFO:tasks.workunit.client.1.smithi117.stdout:3/310: symlink d1/l69 0 2024-02-16T19:28:59.674 INFO:tasks.workunit.client.0.smithi022.stdout:2/229: truncate d0/f20 1124584 0 2024-02-16T19:28:59.675 INFO:tasks.workunit.client.1.smithi117.stdout:1/208: creat d2/d18/d20/f42 x:0 0 0 2024-02-16T19:28:59.676 INFO:tasks.workunit.client.1.smithi117.stdout:1/209: chown d2/d4/f11 0 1 2024-02-16T19:28:59.676 INFO:tasks.workunit.client.0.smithi022.stdout:9/330: mknod db/d14/d16/d27/d2e/c76 0 2024-02-16T19:28:59.678 INFO:tasks.workunit.client.0.smithi022.stdout:3/298: link d7/d21/f7c d7/d12/d18/dd/f7d 0 2024-02-16T19:28:59.679 INFO:tasks.workunit.client.0.smithi022.stdout:3/299: dread - d7/d12/d18/dd/d24/d53/f62 zero size 2024-02-16T19:28:59.680 INFO:tasks.workunit.client.1.smithi117.stdout:2/208: mknod d0/d3/d10/c41 0 2024-02-16T19:28:59.681 INFO:tasks.workunit.client.0.smithi022.stdout:2/230: symlink d0/d2/d18/d1f/l4e 0 2024-02-16T19:28:59.683 INFO:tasks.workunit.client.1.smithi117.stdout:7/246: symlink d4/d1c/d24/l61 0 2024-02-16T19:28:59.684 INFO:tasks.workunit.client.1.smithi117.stdout:7/247: fdatasync d4/d1c/d39/d46/f48 0 2024-02-16T19:28:59.692 INFO:tasks.workunit.client.0.smithi022.stdout:2/231: dwrite d0/d2/d18/f28 [0,4194304] 0 2024-02-16T19:28:59.694 INFO:tasks.workunit.client.1.smithi117.stdout:8/261: mkdir d4/d7/d37/d3a/d54 0 2024-02-16T19:28:59.699 INFO:tasks.workunit.client.1.smithi117.stdout:9/225: link dd/d23/l17 dd/d23/l41 0 2024-02-16T19:28:59.700 INFO:tasks.workunit.client.0.smithi022.stdout:2/232: dwrite d0/d2/f3c [0,4194304] 0 2024-02-16T19:28:59.701 INFO:tasks.workunit.client.0.smithi022.stdout:9/331: getdents db/d14/d16/d27/d55 0 2024-02-16T19:28:59.702 INFO:tasks.workunit.client.0.smithi022.stdout:2/233: dread - d0/d2/f3d zero size 2024-02-16T19:28:59.704 INFO:tasks.workunit.client.0.smithi022.stdout:2/234: write d0/d2/f3d [842530,81376] 0 2024-02-16T19:28:59.708 INFO:tasks.workunit.client.0.smithi022.stdout:9/332: symlink db/d14/d16/d28/l77 0 2024-02-16T19:28:59.713 INFO:tasks.workunit.client.0.smithi022.stdout:3/300: dwrite d7/d12/d18/dd/d24/d58/d6b/f7a [0,4194304] 0 2024-02-16T19:28:59.715 INFO:tasks.workunit.client.1.smithi117.stdout:3/311: rename d1/d1b/d20/c68 to d1/d30/c6a 0 2024-02-16T19:28:59.717 INFO:tasks.workunit.client.1.smithi117.stdout:3/312: truncate d1/d31/f36 4340505 0 2024-02-16T19:28:59.718 INFO:tasks.workunit.client.1.smithi117.stdout:8/262: creat d4/d7/d37/d3a/d43/f55 x:0 0 0 2024-02-16T19:28:59.720 INFO:tasks.workunit.client.0.smithi022.stdout:2/235: symlink d0/d2/d2e/d4a/l4f 0 2024-02-16T19:28:59.721 INFO:tasks.workunit.client.1.smithi117.stdout:7/248: truncate d4/f2b 354009 0 2024-02-16T19:28:59.722 INFO:tasks.workunit.client.0.smithi022.stdout:3/301: unlink d7/f39 0 2024-02-16T19:28:59.723 INFO:tasks.workunit.client.0.smithi022.stdout:3/302: chown d7/d12/d18/dd/d31/d41/f48 533 1 2024-02-16T19:28:59.727 INFO:tasks.workunit.client.0.smithi022.stdout:2/236: mknod d0/d2/d18/d38/c50 0 2024-02-16T19:28:59.731 INFO:tasks.workunit.client.0.smithi022.stdout:3/303: mknod d7/d12/d18/dd/d24/d58/d6b/c7e 0 2024-02-16T19:28:59.732 INFO:tasks.workunit.client.0.smithi022.stdout:3/304: dread - d7/d12/d18/dd/d3c/f49 zero size 2024-02-16T19:28:59.737 INFO:tasks.workunit.client.1.smithi117.stdout:7/249: creat d4/d1c/d39/d46/f62 x:0 0 0 2024-02-16T19:28:59.738 INFO:tasks.workunit.client.1.smithi117.stdout:3/313: dread d1/dd/f14 [0,4194304] 0 2024-02-16T19:28:59.738 INFO:tasks.workunit.client.0.smithi022.stdout:9/333: dwrite db/f12 [0,4194304] 0 2024-02-16T19:28:59.738 INFO:tasks.workunit.client.1.smithi117.stdout:6/284: link d5/d1d/l57 d5/d4b/d44/d52/l61 0 2024-02-16T19:28:59.739 INFO:tasks.workunit.client.1.smithi117.stdout:6/285: chown d5/d1d/f27 8 1 2024-02-16T19:28:59.740 INFO:tasks.workunit.client.1.smithi117.stdout:6/286: truncate d5/d4b/d20/f42 1451164 0 2024-02-16T19:28:59.742 INFO:tasks.workunit.client.0.smithi022.stdout:3/305: write d7/d12/d18/dd/d24/d58/d70/f7b [595958,8458] 0 2024-02-16T19:28:59.742 INFO:tasks.workunit.client.1.smithi117.stdout:6/287: dread d5/d4b/d44/d52/f5f [0,4194304] 0 2024-02-16T19:28:59.745 INFO:tasks.workunit.client.1.smithi117.stdout:0/328: getdents d0/d9/d2c 0 2024-02-16T19:28:59.747 INFO:tasks.workunit.client.1.smithi117.stdout:0/329: write d0/d9/d22/d26/f36 [1139061,45426] 0 2024-02-16T19:28:59.748 INFO:tasks.workunit.client.1.smithi117.stdout:2/209: getdents d0/d3/d22 0 2024-02-16T19:28:59.749 INFO:tasks.workunit.client.1.smithi117.stdout:2/210: fsync d0/d3/d10/f16 0 2024-02-16T19:28:59.751 INFO:tasks.workunit.client.1.smithi117.stdout:9/226: creat dd/d23/d2e/d2c/f42 x:0 0 0 2024-02-16T19:28:59.752 INFO:tasks.workunit.client.0.smithi022.stdout:3/306: mkdir d7/d12/d18/dd/d31/d7f 0 2024-02-16T19:28:59.754 INFO:tasks.workunit.client.1.smithi117.stdout:8/263: link d4/f52 d4/d7/d37/f56 0 2024-02-16T19:28:59.755 INFO:tasks.workunit.client.0.smithi022.stdout:2/237: getdents d0/d32 0 2024-02-16T19:28:59.756 INFO:tasks.workunit.client.1.smithi117.stdout:7/250: mkdir d4/d13/d63 0 2024-02-16T19:28:59.757 INFO:tasks.workunit.client.0.smithi022.stdout:3/307: rmdir d7/d12 39 2024-02-16T19:28:59.757 INFO:tasks.workunit.client.1.smithi117.stdout:7/251: write d4/d1c/f3c [8254439,8644] 0 2024-02-16T19:28:59.758 INFO:tasks.workunit.client.1.smithi117.stdout:7/252: fdatasync d4/d1c/d39/f5a 0 2024-02-16T19:28:59.758 INFO:tasks.workunit.client.1.smithi117.stdout:5/363: sync 2024-02-16T19:28:59.761 INFO:tasks.workunit.client.1.smithi117.stdout:5/364: write d2/d3/d21/d49/d41/f4b [395399,128365] 0 2024-02-16T19:28:59.761 INFO:tasks.workunit.client.0.smithi022.stdout:3/308: readlink d7/d12/d18/l10 0 2024-02-16T19:28:59.765 INFO:tasks.workunit.client.0.smithi022.stdout:9/334: getdents db 0 2024-02-16T19:28:59.767 INFO:tasks.workunit.client.1.smithi117.stdout:8/264: mknod d4/d12/c57 0 2024-02-16T19:28:59.771 INFO:tasks.workunit.client.0.smithi022.stdout:0/399: sync 2024-02-16T19:28:59.771 INFO:tasks.workunit.client.0.smithi022.stdout:6/297: sync 2024-02-16T19:28:59.772 INFO:tasks.workunit.client.1.smithi117.stdout:3/314: creat d1/dd/d5c/f6b x:0 0 0 2024-02-16T19:28:59.773 INFO:tasks.workunit.client.0.smithi022.stdout:6/298: fdatasync d1/d8/da/d17/f2d 0 2024-02-16T19:28:59.774 INFO:tasks.workunit.client.0.smithi022.stdout:3/309: link d7/l64 d7/d12/d18/dd/d31/d41/l80 0 2024-02-16T19:28:59.778 INFO:tasks.workunit.client.1.smithi117.stdout:6/288: link d5/d1d/d28/f2b d5/d1d/d28/d32/f62 0 2024-02-16T19:28:59.778 INFO:tasks.workunit.client.1.smithi117.stdout:3/315: dwrite d1/d1b/d1e/d29/f33 [0,4194304] 0 2024-02-16T19:28:59.779 INFO:tasks.workunit.client.0.smithi022.stdout:6/299: creat d1/f72 x:0 0 0 2024-02-16T19:28:59.781 INFO:tasks.workunit.client.0.smithi022.stdout:9/335: dread db/d14/d19/f4d [0,4194304] 0 2024-02-16T19:28:59.781 INFO:tasks.workunit.client.0.smithi022.stdout:2/238: dwrite d0/d2/d18/d19/f2f [0,4194304] 0 2024-02-16T19:28:59.781 INFO:tasks.workunit.client.0.smithi022.stdout:6/300: read - d1/d8/f6f zero size 2024-02-16T19:28:59.784 INFO:tasks.workunit.client.0.smithi022.stdout:2/239: readlink d0/d2/d18/d38/l4b 0 2024-02-16T19:28:59.785 INFO:tasks.workunit.client.0.smithi022.stdout:6/301: fdatasync d1/d8/da/d17/d48/f65 0 2024-02-16T19:28:59.787 INFO:tasks.workunit.client.0.smithi022.stdout:6/302: stat d1/f3 0 2024-02-16T19:28:59.788 INFO:tasks.workunit.client.0.smithi022.stdout:3/310: dwrite d7/d12/d18/dd/d24/d58/d6b/f7a [0,4194304] 0 2024-02-16T19:28:59.790 INFO:tasks.workunit.client.0.smithi022.stdout:6/303: chown d1/d8/da/d17/d53/f5c 385453336 1 2024-02-16T19:28:59.791 INFO:tasks.workunit.client.0.smithi022.stdout:3/311: write d7/d12/d18/dd/d24/d58/d70/f7b [1527185,104001] 0 2024-02-16T19:28:59.793 INFO:tasks.workunit.client.0.smithi022.stdout:0/400: dread d10/d11/d27/d3f/d58/d60/d62/f4f [0,4194304] 0 2024-02-16T19:28:59.793 INFO:tasks.workunit.client.0.smithi022.stdout:0/401: chown d10/d11/d79 12276194 1 2024-02-16T19:28:59.825 INFO:tasks.workunit.client.1.smithi117.stdout:4/277: sync 2024-02-16T19:28:59.826 INFO:tasks.workunit.client.1.smithi117.stdout:4/278: dread - d0/da/d21/d3d/d28/f4e zero size 2024-02-16T19:28:59.840 INFO:tasks.workunit.client.1.smithi117.stdout:7/253: mknod d4/d1c/d39/d46/d47/c64 0 2024-02-16T19:28:59.840 INFO:tasks.workunit.client.1.smithi117.stdout:7/254: stat d4/d1c/f2e 0 2024-02-16T19:28:59.845 INFO:tasks.workunit.client.1.smithi117.stdout:5/365: creat d2/d4c/d54/f6d x:0 0 0 2024-02-16T19:28:59.846 INFO:tasks.workunit.client.0.smithi022.stdout:9/336: creat db/d14/d16/d27/f78 x:0 0 0 2024-02-16T19:28:59.847 INFO:tasks.workunit.client.1.smithi117.stdout:2/211: mkdir d0/d29/d2f/d42 0 2024-02-16T19:28:59.847 INFO:tasks.workunit.client.0.smithi022.stdout:9/337: write db/d14/d33/f48 [1405512,40118] 0 2024-02-16T19:28:59.848 INFO:tasks.workunit.client.0.smithi022.stdout:2/240: creat d0/d2/d2e/f51 x:0 0 0 2024-02-16T19:28:59.850 INFO:tasks.workunit.client.0.smithi022.stdout:9/338: write db/d14/d16/d27/f68 [453264,57219] 0 2024-02-16T19:28:59.850 INFO:tasks.workunit.client.0.smithi022.stdout:2/241: dread - d0/d2/d18/d19/f47 zero size 2024-02-16T19:28:59.851 INFO:tasks.workunit.client.1.smithi117.stdout:9/227: symlink dd/d23/l43 0 2024-02-16T19:28:59.853 INFO:tasks.workunit.client.1.smithi117.stdout:8/265: symlink d4/d12/l58 0 2024-02-16T19:28:59.853 INFO:tasks.workunit.client.1.smithi117.stdout:2/212: dwrite d0/d3/fc [0,4194304] 0 2024-02-16T19:28:59.854 INFO:tasks.workunit.client.1.smithi117.stdout:0/330: write d0/d9/d22/d27/f52 [998760,102725] 0 2024-02-16T19:28:59.856 INFO:tasks.workunit.client.0.smithi022.stdout:3/312: rename d7/f35 to d7/d12/d18/dd/d24/d58/d70/f81 0 2024-02-16T19:28:59.857 INFO:tasks.workunit.client.1.smithi117.stdout:6/289: readlink d5/d1d/l4f 0 2024-02-16T19:28:59.858 INFO:tasks.workunit.client.0.smithi022.stdout:0/402: mknod d10/d11/d50/d65/c84 0 2024-02-16T19:28:59.859 INFO:tasks.workunit.client.0.smithi022.stdout:0/403: write d10/d22/f53 [1216062,108067] 0 2024-02-16T19:28:59.860 INFO:tasks.workunit.client.1.smithi117.stdout:3/316: rename d1/d1b/d1e/d37/c44 to d1/d30/d52/d56/c6c 0 2024-02-16T19:28:59.861 INFO:tasks.workunit.client.1.smithi117.stdout:3/317: fsync d1/f19 0 2024-02-16T19:28:59.863 INFO:tasks.workunit.client.0.smithi022.stdout:9/339: creat db/f79 x:0 0 0 2024-02-16T19:28:59.865 INFO:tasks.workunit.client.0.smithi022.stdout:2/242: creat d0/d32/f52 x:0 0 0 2024-02-16T19:28:59.866 INFO:tasks.workunit.client.1.smithi117.stdout:6/290: dwrite d5/d1d/f50 [0,4194304] 0 2024-02-16T19:28:59.866 INFO:tasks.workunit.client.0.smithi022.stdout:6/304: symlink d1/d8/da/d17/l73 0 2024-02-16T19:28:59.866 INFO:tasks.workunit.client.0.smithi022.stdout:2/243: dread - d0/d2/d18/f22 zero size 2024-02-16T19:28:59.867 INFO:tasks.workunit.client.1.smithi117.stdout:5/366: rmdir d2/d3/d21/d2f 39 2024-02-16T19:28:59.869 INFO:tasks.workunit.client.1.smithi117.stdout:5/367: chown d2/d3/ld 15785 1 2024-02-16T19:28:59.871 INFO:tasks.workunit.client.1.smithi117.stdout:3/318: dwrite d1/f28 [0,4194304] 0 2024-02-16T19:28:59.872 INFO:tasks.workunit.client.0.smithi022.stdout:2/244: mknod d0/d2/d18/d1f/c53 0 2024-02-16T19:28:59.874 INFO:tasks.workunit.client.1.smithi117.stdout:0/331: unlink d0/d9/d14/d1d/d60/d65/c6e 0 2024-02-16T19:28:59.875 INFO:tasks.workunit.client.0.smithi022.stdout:6/305: creat d1/d8/da/d17/d48/f74 x:0 0 0 2024-02-16T19:28:59.876 INFO:tasks.workunit.client.1.smithi117.stdout:2/213: dread d0/d3/d22/f28 [0,4194304] 0 2024-02-16T19:28:59.876 INFO:tasks.workunit.client.1.smithi117.stdout:1/210: sync 2024-02-16T19:28:59.881 INFO:tasks.workunit.client.0.smithi022.stdout:0/404: rename d10/d15/d23/d38/c7c to d10/d11/c85 0 2024-02-16T19:28:59.883 INFO:tasks.workunit.client.1.smithi117.stdout:4/279: fdatasync d0/d6/f55 0 2024-02-16T19:28:59.887 INFO:tasks.workunit.client.1.smithi117.stdout:7/255: symlink d4/d1c/d39/l65 0 2024-02-16T19:28:59.888 INFO:tasks.workunit.client.1.smithi117.stdout:7/256: write d4/d13/f35 [1158275,68473] 0 2024-02-16T19:28:59.890 INFO:tasks.workunit.client.1.smithi117.stdout:7/257: write d4/d13/f18 [231263,109168] 0 2024-02-16T19:28:59.891 INFO:tasks.workunit.client.1.smithi117.stdout:7/258: chown d4/d1c/d39/l3b 15504576 1 2024-02-16T19:28:59.892 INFO:tasks.workunit.client.1.smithi117.stdout:9/228: creat dd/d2d/f44 x:0 0 0 2024-02-16T19:28:59.894 INFO:tasks.workunit.client.1.smithi117.stdout:8/266: mknod d4/d7/d37/d3a/d54/c59 0 2024-02-16T19:28:59.894 INFO:tasks.workunit.client.0.smithi022.stdout:3/313: write d7/d12/d18/f1a [551432,59514] 0 2024-02-16T19:28:59.895 INFO:tasks.workunit.client.1.smithi117.stdout:8/267: readlink d4/d7/d37/d3a/d43/d44/l4a 0 2024-02-16T19:28:59.897 INFO:tasks.workunit.client.0.smithi022.stdout:0/405: link d10/d22/d49/f7b d10/d11/d27/d3f/d58/d60/d62/f86 0 2024-02-16T19:28:59.900 INFO:tasks.workunit.client.0.smithi022.stdout:2/245: creat d0/d2/f54 x:0 0 0 2024-02-16T19:28:59.901 INFO:tasks.workunit.client.1.smithi117.stdout:5/368: dread - d2/d3/d21/d2f/d30/f68 zero size 2024-02-16T19:28:59.901 INFO:tasks.workunit.client.0.smithi022.stdout:2/246: chown d0/d2/d18/d19/f47 37898 1 2024-02-16T19:28:59.904 INFO:tasks.workunit.client.0.smithi022.stdout:2/247: truncate d0/d2/d18/f22 436355 0 2024-02-16T19:28:59.907 INFO:tasks.workunit.client.0.smithi022.stdout:3/314: dread d7/d12/d18/dd/d24/f3e [4194304,4194304] 0 2024-02-16T19:28:59.907 INFO:tasks.workunit.client.1.smithi117.stdout:0/332: mkdir d0/d78 0 2024-02-16T19:28:59.908 INFO:tasks.workunit.client.1.smithi117.stdout:5/369: dwrite d2/d3/d21/f32 [0,4194304] 0 2024-02-16T19:28:59.909 INFO:tasks.workunit.client.1.smithi117.stdout:5/370: dread - d2/d3/f4d zero size 2024-02-16T19:28:59.911 INFO:tasks.workunit.client.0.smithi022.stdout:0/406: fsync d10/d11/d27/d3f/f57 0 2024-02-16T19:28:59.913 INFO:tasks.workunit.client.1.smithi117.stdout:3/319: dwrite d1/d1b/d1e/f54 [4194304,4194304] 0 2024-02-16T19:28:59.913 INFO:tasks.workunit.client.1.smithi117.stdout:1/211: creat d2/d18/f43 x:0 0 0 2024-02-16T19:28:59.915 INFO:tasks.workunit.client.0.smithi022.stdout:9/340: unlink db/d14/d33/f48 0 2024-02-16T19:28:59.915 INFO:tasks.workunit.client.0.smithi022.stdout:9/341: chown db/d14/d19/f5f 728 1 2024-02-16T19:28:59.916 INFO:tasks.workunit.client.1.smithi117.stdout:6/291: write d5/f13 [578282,83654] 0 2024-02-16T19:28:59.922 INFO:tasks.workunit.client.1.smithi117.stdout:5/371: dwrite d2/d1d/f55 [0,4194304] 0 2024-02-16T19:28:59.925 INFO:tasks.workunit.client.0.smithi022.stdout:8/310: sync 2024-02-16T19:28:59.926 INFO:tasks.workunit.client.0.smithi022.stdout:4/272: sync 2024-02-16T19:28:59.926 INFO:tasks.workunit.client.1.smithi117.stdout:4/280: unlink d0/da/d21/d3d/c26 0 2024-02-16T19:28:59.926 INFO:tasks.workunit.client.0.smithi022.stdout:4/273: read d4/f23 [2962708,26248] 0 2024-02-16T19:28:59.927 INFO:tasks.workunit.client.0.smithi022.stdout:6/306: getdents d1/d8/da/d29 0 2024-02-16T19:28:59.928 INFO:tasks.workunit.client.0.smithi022.stdout:6/307: write d1/d8/d23/f68 [278859,23690] 0 2024-02-16T19:28:59.930 INFO:tasks.workunit.client.0.smithi022.stdout:6/308: read - d1/d24/f38 zero size 2024-02-16T19:28:59.931 INFO:tasks.workunit.client.0.smithi022.stdout:6/309: chown d1/d8/da/d29/d61/l67 5763125 1 2024-02-16T19:28:59.934 INFO:tasks.workunit.client.1.smithi117.stdout:7/259: rename d4/d13/f16 to d4/d4a/f66 0 2024-02-16T19:28:59.939 INFO:tasks.workunit.client.0.smithi022.stdout:6/310: dwrite d1/d8/d3c/f44 [0,4194304] 0 2024-02-16T19:28:59.940 INFO:tasks.workunit.client.0.smithi022.stdout:3/315: rmdir d7/d12/d18/dd/d24/d58/d70 39 2024-02-16T19:28:59.941 INFO:tasks.workunit.client.0.smithi022.stdout:6/311: chown d1/d8/da/d17/l2a 121 1 2024-02-16T19:28:59.942 INFO:tasks.workunit.client.0.smithi022.stdout:6/312: write d1/d8/d23/f68 [644556,44606] 0 2024-02-16T19:28:59.943 INFO:tasks.workunit.client.1.smithi117.stdout:8/268: mknod d4/d7/d37/d50/c5a 0 2024-02-16T19:28:59.943 INFO:tasks.workunit.client.0.smithi022.stdout:6/313: chown d1 218950326 1 2024-02-16T19:28:59.945 INFO:tasks.workunit.client.1.smithi117.stdout:5/372: dread d2/d3/d21/f33 [0,4194304] 0 2024-02-16T19:28:59.946 INFO:tasks.workunit.client.0.smithi022.stdout:0/407: creat d10/d15/d23/d4e/f87 x:0 0 0 2024-02-16T19:28:59.947 INFO:tasks.workunit.client.0.smithi022.stdout:0/408: dread - d10/d11/d27/d77/f81 zero size 2024-02-16T19:28:59.950 INFO:tasks.workunit.client.0.smithi022.stdout:0/409: write d10/d15/d23/d4e/f87 [478946,57496] 0 2024-02-16T19:28:59.950 INFO:tasks.workunit.client.1.smithi117.stdout:0/333: symlink d0/d9/d14/d1d/d60/d76/l79 0 2024-02-16T19:28:59.951 INFO:tasks.workunit.client.1.smithi117.stdout:0/334: chown d0/d9/d14/d23 213516942 1 2024-02-16T19:28:59.952 INFO:tasks.workunit.client.0.smithi022.stdout:0/410: chown d10/d11/d50/d65 1 1 2024-02-16T19:28:59.952 INFO:tasks.workunit.client.1.smithi117.stdout:2/214: creat d0/d13/f43 x:0 0 0 2024-02-16T19:28:59.955 INFO:tasks.workunit.client.0.smithi022.stdout:2/248: dwrite d0/f17 [0,4194304] 0 2024-02-16T19:28:59.957 INFO:tasks.workunit.client.0.smithi022.stdout:8/311: creat d3/d12/f71 x:0 0 0 2024-02-16T19:28:59.960 INFO:tasks.workunit.client.0.smithi022.stdout:8/312: fsync d3/d25/d33/f37 0 2024-02-16T19:28:59.964 INFO:tasks.workunit.client.1.smithi117.stdout:6/292: read d5/f9 [374866,77566] 0 2024-02-16T19:28:59.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:59 smithi022 ceph-mon[29221]: pgmap v261: 65 pgs: 65 active+clean; 918 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 4.5 MiB/s rd, 96 MiB/s wr, 218 op/s 2024-02-16T19:28:59.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:59 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:28:59.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:59 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:28:59.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:59 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:28:59.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:59 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:28:59.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:59 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:28:59.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:59 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:28:59.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:59 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mgr fail", "who": "smithi022.gmdcch"}]: dispatch 2024-02-16T19:28:59.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:59 smithi022 ceph-mon[29221]: osdmap e48: 6 total, 6 up, 6 in 2024-02-16T19:28:59.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:59 smithi022 ceph-mon[29221]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "mgr fail", "who": "smithi022.gmdcch"}]': finished 2024-02-16T19:28:59.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:59 smithi022 ceph-mon[29221]: mgrmap e21: smithi117.cohbtv(active, starting, since 0.00666869s) 2024-02-16T19:28:59.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:59 smithi022 ceph-mon[29221]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "mon metadata", "id": "smithi022"}]: dispatch 2024-02-16T19:28:59.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:59 smithi022 ceph-mon[29221]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "mon metadata", "id": "smithi117"}]: dispatch 2024-02-16T19:28:59.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:59 smithi022 ceph-mon[29221]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi022.btppwm"}]: dispatch 2024-02-16T19:28:59.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:59 smithi022 ceph-mon[29221]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi022.ocukvs"}]: dispatch 2024-02-16T19:28:59.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:59 smithi022 ceph-mon[29221]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi117.ibmppn"}]: dispatch 2024-02-16T19:28:59.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:59 smithi022 ceph-mon[29221]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi117.orzors"}]: dispatch 2024-02-16T19:28:59.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:59 smithi022 ceph-mon[29221]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "mgr metadata", "who": "smithi117.cohbtv", "id": "smithi117.cohbtv"}]: dispatch 2024-02-16T19:28:59.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:59 smithi022 ceph-mon[29221]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-16T19:28:59.967 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:59 smithi022 ceph-mon[29221]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-16T19:28:59.967 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:59 smithi022 ceph-mon[29221]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-16T19:28:59.967 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:59 smithi022 ceph-mon[29221]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-16T19:28:59.967 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:59 smithi022 ceph-mon[29221]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-16T19:28:59.967 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:59 smithi022 ceph-mon[29221]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-16T19:28:59.967 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:59 smithi022 ceph-mon[29221]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-16T19:28:59.967 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:59 smithi022 ceph-mon[29221]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-16T19:28:59.967 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:59 smithi022 ceph-mon[29221]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-16T19:28:59.967 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:59 smithi022 ceph-mon[29221]: Manager daemon smithi117.cohbtv is now available 2024-02-16T19:28:59.967 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:59 smithi022 ceph-mon[29221]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:28:59.967 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:59 smithi022 ceph-mon[29221]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:28:59.967 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:59 smithi022 ceph-mon[29221]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:28:59.967 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:59 smithi022 ceph-mon[29221]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi117.cohbtv/mirror_snapshot_schedule"}]: dispatch 2024-02-16T19:28:59.967 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:59 smithi022 ceph-mon[29221]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi117.cohbtv/mirror_snapshot_schedule"}]: dispatch 2024-02-16T19:28:59.968 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:59 smithi022 ceph-mon[29221]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi117.cohbtv/trash_purge_schedule"}]: dispatch 2024-02-16T19:28:59.968 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:28:59 smithi022 ceph-mon[29221]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi117.cohbtv/trash_purge_schedule"}]: dispatch 2024-02-16T19:28:59.969 INFO:tasks.workunit.client.0.smithi022.stdout:6/314: write d1/d8/f40 [150118,61844] 0 2024-02-16T19:28:59.970 INFO:tasks.workunit.client.0.smithi022.stdout:9/342: mknod db/d14/d16/d21/d4a/c7a 0 2024-02-16T19:28:59.971 INFO:tasks.workunit.client.0.smithi022.stdout:9/343: chown db/d14/d16/f69 218594 1 2024-02-16T19:28:59.973 INFO:tasks.workunit.client.0.smithi022.stdout:9/344: write db/d14/f5e [397432,88026] 0 2024-02-16T19:28:59.973 INFO:tasks.workunit.client.0.smithi022.stdout:0/411: rename d10/d11/d27/l3a to d10/d11/d27/d3f/d58/l88 0 2024-02-16T19:28:59.977 INFO:tasks.workunit.client.0.smithi022.stdout:9/345: truncate db/d14/d16/d27/d2e/f38 4965352 0 2024-02-16T19:28:59.978 INFO:tasks.workunit.client.1.smithi117.stdout:9/229: rename dd/d23/d2e/d2c to dd/d45 0 2024-02-16T19:28:59.979 INFO:tasks.workunit.client.0.smithi022.stdout:2/249: mkdir d0/d2/d18/d19/d40/d55 0 2024-02-16T19:28:59.981 INFO:tasks.workunit.client.0.smithi022.stdout:0/412: dread d10/d11/d27/d3f/d58/f6a [4194304,4194304] 0 2024-02-16T19:28:59.981 INFO:tasks.workunit.client.0.smithi022.stdout:2/250: write d0/d2/d18/d1f/f4d [822574,77318] 0 2024-02-16T19:28:59.982 INFO:tasks.workunit.client.0.smithi022.stdout:4/274: write d4/d6/dd/df/d10/f35 [289125,109177] 0 2024-02-16T19:28:59.985 INFO:tasks.workunit.client.0.smithi022.stdout:6/315: dread d1/d8/da/f14 [0,4194304] 0 2024-02-16T19:28:59.988 INFO:tasks.workunit.client.1.smithi117.stdout:2/215: creat d0/d29/d2f/f44 x:0 0 0 2024-02-16T19:28:59.989 INFO:tasks.workunit.client.1.smithi117.stdout:2/216: write d0/d29/d2f/f44 [362621,18694] 0 2024-02-16T19:28:59.992 INFO:tasks.workunit.client.0.smithi022.stdout:0/413: dwrite d10/d11/d27/f52 [0,4194304] 0 2024-02-16T19:28:59.993 INFO:tasks.workunit.client.0.smithi022.stdout:4/275: dwrite d4/d6/dd/f2a [0,4194304] 0 2024-02-16T19:28:59.994 INFO:tasks.workunit.client.1.smithi117.stdout:2/217: dwrite d0/f25 [0,4194304] 0 2024-02-16T19:28:59.994 INFO:tasks.workunit.client.1.smithi117.stdout:1/212: link d2/d4/f9 d2/d1d/d1f/f44 0 2024-02-16T19:28:59.995 INFO:tasks.workunit.client.1.smithi117.stdout:2/218: dread - d0/d13/f26 zero size 2024-02-16T19:28:59.997 INFO:tasks.workunit.client.1.smithi117.stdout:6/293: symlink d5/d4b/l63 0 2024-02-16T19:29:00.001 INFO:tasks.workunit.client.1.smithi117.stdout:3/320: link d1/d1b/d1e/d5e/f66 d1/d1b/d1e/d37/f6d 0 2024-02-16T19:29:00.002 INFO:tasks.workunit.client.0.smithi022.stdout:2/251: chown d0/c9 122 1 2024-02-16T19:29:00.003 INFO:tasks.workunit.client.1.smithi117.stdout:9/230: symlink dd/d45/d31/l46 0 2024-02-16T19:29:00.004 INFO:tasks.workunit.client.0.smithi022.stdout:0/414: dread d10/d11/f2e [0,4194304] 0 2024-02-16T19:29:00.005 INFO:tasks.workunit.client.1.smithi117.stdout:9/231: write f5 [547769,103718] 0 2024-02-16T19:29:00.005 INFO:tasks.workunit.client.0.smithi022.stdout:0/415: stat c4 0 2024-02-16T19:29:00.006 INFO:tasks.workunit.client.0.smithi022.stdout:8/313: mknod d3/d12/d17/c72 0 2024-02-16T19:29:00.007 INFO:tasks.workunit.client.1.smithi117.stdout:3/321: dread d1/d31/f36 [0,4194304] 0 2024-02-16T19:29:00.008 INFO:tasks.workunit.client.1.smithi117.stdout:7/260: mknod d4/d1c/d24/d60/c67 0 2024-02-16T19:29:00.009 INFO:tasks.workunit.client.0.smithi022.stdout:2/252: dread d0/d2/d18/d1f/f4d [0,4194304] 0 2024-02-16T19:29:00.011 INFO:tasks.workunit.client.0.smithi022.stdout:2/253: write d0/f37 [786083,7878] 0 2024-02-16T19:29:00.012 INFO:tasks.workunit.client.0.smithi022.stdout:2/254: write d0/f37 [1688440,128999] 0 2024-02-16T19:29:00.014 INFO:tasks.workunit.client.1.smithi117.stdout:7/261: dwrite d4/f3a [0,4194304] 0 2024-02-16T19:29:00.014 INFO:tasks.workunit.client.1.smithi117.stdout:5/373: symlink d2/d3/d21/d2f/d30/d5a/d62/l6e 0 2024-02-16T19:29:00.015 INFO:tasks.workunit.client.1.smithi117.stdout:7/262: truncate d4/d13/f18 5047137 0 2024-02-16T19:29:00.016 INFO:tasks.workunit.client.1.smithi117.stdout:7/263: chown d4/d4a/f50 133214945 1 2024-02-16T19:29:00.017 INFO:tasks.workunit.client.0.smithi022.stdout:6/316: truncate d1/d8/da/d17/d53/f5c 2353849 0 2024-02-16T19:29:00.018 INFO:tasks.workunit.client.0.smithi022.stdout:6/317: dread - d1/f72 zero size 2024-02-16T19:29:00.023 INFO:tasks.workunit.client.0.smithi022.stdout:2/255: dwrite d0/d2/d2e/f51 [0,4194304] 0 2024-02-16T19:29:00.026 INFO:tasks.workunit.client.0.smithi022.stdout:2/256: write d0/d2/d18/d19/f2f [2885712,125380] 0 2024-02-16T19:29:00.031 INFO:tasks.workunit.client.1.smithi117.stdout:8/269: write d4/f53 [4492291,96379] 0 2024-02-16T19:29:00.033 INFO:tasks.workunit.client.1.smithi117.stdout:2/219: rename d0/d13/l27 to d0/d3/d22/l45 0 2024-02-16T19:29:00.035 INFO:tasks.workunit.client.1.smithi117.stdout:6/294: mkdir d5/d4b/d44/d52/d64 0 2024-02-16T19:29:00.037 INFO:tasks.workunit.client.1.smithi117.stdout:2/220: dread d0/f12 [0,4194304] 0 2024-02-16T19:29:00.040 INFO:tasks.workunit.client.1.smithi117.stdout:6/295: dwrite d5/d1d/f2c [0,4194304] 0 2024-02-16T19:29:00.044 INFO:tasks.workunit.client.0.smithi022.stdout:8/314: chown d3/d25/f43 24786 1 2024-02-16T19:29:00.050 INFO:tasks.workunit.client.1.smithi117.stdout:9/232: mkdir dd/d47 0 2024-02-16T19:29:00.051 INFO:tasks.workunit.client.1.smithi117.stdout:9/233: fsync dd/d24/f39 0 2024-02-16T19:29:00.053 INFO:tasks.workunit.client.0.smithi022.stdout:4/276: dwrite d4/f23 [0,4194304] 0 2024-02-16T19:29:00.054 INFO:tasks.workunit.client.1.smithi117.stdout:3/322: unlink d1/dd/c4e 0 2024-02-16T19:29:00.055 INFO:tasks.workunit.client.1.smithi117.stdout:3/323: dread - d1/dd/d5c/f6b zero size 2024-02-16T19:29:00.056 INFO:tasks.workunit.client.0.smithi022.stdout:8/315: dwrite d3/d12/d6a/f6e [0,4194304] 0 2024-02-16T19:29:00.056 INFO:tasks.workunit.client.1.smithi117.stdout:3/324: read d1/dd/f34 [3581473,88953] 0 2024-02-16T19:29:00.056 INFO:tasks.workunit.client.1.smithi117.stdout:3/325: dread - d1/dd/d3b/f63 zero size 2024-02-16T19:29:00.061 INFO:tasks.workunit.client.0.smithi022.stdout:4/277: dread d4/f23 [0,4194304] 0 2024-02-16T19:29:00.062 INFO:tasks.workunit.client.0.smithi022.stdout:0/416: write d10/d11/d27/d3f/d58/d60/d62/f86 [521998,54494] 0 2024-02-16T19:29:00.062 INFO:tasks.workunit.client.0.smithi022.stdout:6/318: mknod d1/d24/c75 0 2024-02-16T19:29:00.064 INFO:tasks.workunit.client.0.smithi022.stdout:6/319: fsync d1/d8/fb 0 2024-02-16T19:29:00.065 INFO:tasks.workunit.client.1.smithi117.stdout:5/374: mkdir d2/d36/d6f 0 2024-02-16T19:29:00.066 INFO:tasks.workunit.client.0.smithi022.stdout:4/278: read d4/d6/dd/d22/f59 [182199,56223] 0 2024-02-16T19:29:00.066 INFO:tasks.workunit.client.1.smithi117.stdout:7/264: creat d4/d13/f68 x:0 0 0 2024-02-16T19:29:00.068 INFO:tasks.workunit.client.0.smithi022.stdout:2/257: creat d0/d2/d18/d19/f56 x:0 0 0 2024-02-16T19:29:00.069 INFO:tasks.workunit.client.1.smithi117.stdout:0/335: link d0/d9/d22/d27/d31/l5d d0/d9/d14/d23/d33/l7a 0 2024-02-16T19:29:00.070 INFO:tasks.workunit.client.1.smithi117.stdout:3/326: dread d1/f2 [0,4194304] 0 2024-02-16T19:29:00.071 INFO:tasks.workunit.client.1.smithi117.stdout:1/213: symlink d2/d19/l45 0 2024-02-16T19:29:00.073 INFO:tasks.workunit.client.0.smithi022.stdout:9/346: rename db/d14/d16/d27/d55/f71 to db/f7b 0 2024-02-16T19:29:00.073 INFO:tasks.workunit.client.0.smithi022.stdout:5/324: sync 2024-02-16T19:29:00.074 INFO:tasks.workunit.client.0.smithi022.stdout:5/325: dread - dd/d1f/d48/d21/d2f/d40/d49/d4b/f66 zero size 2024-02-16T19:29:00.075 INFO:tasks.workunit.client.1.smithi117.stdout:8/270: creat d4/d7/d37/d50/f5b x:0 0 0 2024-02-16T19:29:00.080 INFO:tasks.workunit.client.0.smithi022.stdout:3/316: link d7/d12/d18/dd/d24/d29/l3d d7/d12/d18/l82 0 2024-02-16T19:29:00.081 INFO:tasks.workunit.client.1.smithi117.stdout:4/281: getdents d0/d6/d48 0 2024-02-16T19:29:00.084 INFO:tasks.workunit.client.1.smithi117.stdout:6/296: readlink d5/d4b/d20/l22 0 2024-02-16T19:29:00.087 INFO:tasks.workunit.client.1.smithi117.stdout:4/282: dwrite d0/d6/f23 [0,4194304] 0 2024-02-16T19:29:00.089 INFO:tasks.workunit.client.1.smithi117.stdout:4/283: truncate d0/da/d21/d3d/d28/f4e 764614 0 2024-02-16T19:29:00.090 INFO:tasks.workunit.client.1.smithi117.stdout:9/234: unlink c1 0 2024-02-16T19:29:00.098 INFO:tasks.workunit.client.1.smithi117.stdout:5/375: creat d2/d4c/d4f/f70 x:0 0 0 2024-02-16T19:29:00.107 INFO:tasks.workunit.client.1.smithi117.stdout:1/214: truncate d2/d4/fa 3751727 0 2024-02-16T19:29:00.110 INFO:tasks.workunit.client.1.smithi117.stdout:2/221: mkdir d0/d29/d2f/d42/d46 0 2024-02-16T19:29:00.113 INFO:tasks.workunit.client.0.smithi022.stdout:9/347: read db/d14/d16/f45 [5961335,74795] 0 2024-02-16T19:29:00.114 INFO:tasks.workunit.client.0.smithi022.stdout:5/326: symlink dd/d1f/d48/d21/d2f/d52/d5c/l67 0 2024-02-16T19:29:00.119 INFO:tasks.workunit.client.1.smithi117.stdout:5/376: creat d2/d4c/f71 x:0 0 0 2024-02-16T19:29:00.119 INFO:tasks.workunit.client.1.smithi117.stdout:7/265: symlink d4/d13/d63/l69 0 2024-02-16T19:29:00.120 INFO:tasks.workunit.client.0.smithi022.stdout:7/296: sync 2024-02-16T19:29:00.120 INFO:tasks.workunit.client.1.smithi117.stdout:7/266: chown d4/d1c/d39/d46/f62 1654 1 2024-02-16T19:29:00.122 INFO:tasks.workunit.client.0.smithi022.stdout:4/279: getdents d4/d6/d3a 0 2024-02-16T19:29:00.129 INFO:tasks.workunit.client.0.smithi022.stdout:7/297: dwrite d9/da/d2c/d2f/f35 [0,4194304] 0 2024-02-16T19:29:00.129 INFO:tasks.workunit.client.1.smithi117.stdout:1/215: dread d2/d4/f11 [0,4194304] 0 2024-02-16T19:29:00.130 INFO:tasks.workunit.client.1.smithi117.stdout:1/216: readlink d2/d18/d20/l34 0 2024-02-16T19:29:00.130 INFO:tasks.workunit.client.0.smithi022.stdout:7/298: dread - d9/d1b/f57 zero size 2024-02-16T19:29:00.133 INFO:tasks.workunit.client.0.smithi022.stdout:0/417: truncate d10/d22/f2a 1574227 0 2024-02-16T19:29:00.136 INFO:tasks.workunit.client.1.smithi117.stdout:6/297: rename d5/d1d/d28/d32/d37/l39 to d5/d4b/d44/d52/l65 0 2024-02-16T19:29:00.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:59 smithi117 ceph-mon[35309]: pgmap v261: 65 pgs: 65 active+clean; 918 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 4.5 MiB/s rd, 96 MiB/s wr, 218 op/s 2024-02-16T19:29:00.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:59 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:29:00.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:59 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:29:00.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:59 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:29:00.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:59 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:29:00.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:59 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' 2024-02-16T19:29:00.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:59 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:29:00.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:59 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mgr fail", "who": "smithi022.gmdcch"}]: dispatch 2024-02-16T19:29:00.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:59 smithi117 ceph-mon[35309]: osdmap e48: 6 total, 6 up, 6 in 2024-02-16T19:29:00.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:59 smithi117 ceph-mon[35309]: from='mgr.14217 172.21.15.22:0/6245531' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "mgr fail", "who": "smithi022.gmdcch"}]': finished 2024-02-16T19:29:00.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:59 smithi117 ceph-mon[35309]: mgrmap e21: smithi117.cohbtv(active, starting, since 0.00666869s) 2024-02-16T19:29:00.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:59 smithi117 ceph-mon[35309]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "mon metadata", "id": "smithi022"}]: dispatch 2024-02-16T19:29:00.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:59 smithi117 ceph-mon[35309]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "mon metadata", "id": "smithi117"}]: dispatch 2024-02-16T19:29:00.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:59 smithi117 ceph-mon[35309]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi022.btppwm"}]: dispatch 2024-02-16T19:29:00.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:59 smithi117 ceph-mon[35309]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi022.ocukvs"}]: dispatch 2024-02-16T19:29:00.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:59 smithi117 ceph-mon[35309]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi117.ibmppn"}]: dispatch 2024-02-16T19:29:00.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:59 smithi117 ceph-mon[35309]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi117.orzors"}]: dispatch 2024-02-16T19:29:00.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:59 smithi117 ceph-mon[35309]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "mgr metadata", "who": "smithi117.cohbtv", "id": "smithi117.cohbtv"}]: dispatch 2024-02-16T19:29:00.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:59 smithi117 ceph-mon[35309]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-16T19:29:00.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:59 smithi117 ceph-mon[35309]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-16T19:29:00.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:59 smithi117 ceph-mon[35309]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-16T19:29:00.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:59 smithi117 ceph-mon[35309]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-16T19:29:00.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:59 smithi117 ceph-mon[35309]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-16T19:29:00.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:59 smithi117 ceph-mon[35309]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-16T19:29:00.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:59 smithi117 ceph-mon[35309]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-16T19:29:00.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:59 smithi117 ceph-mon[35309]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-16T19:29:00.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:59 smithi117 ceph-mon[35309]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-16T19:29:00.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:59 smithi117 ceph-mon[35309]: Manager daemon smithi117.cohbtv is now available 2024-02-16T19:29:00.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:59 smithi117 ceph-mon[35309]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:29:00.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:59 smithi117 ceph-mon[35309]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:29:00.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:59 smithi117 ceph-mon[35309]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:29:00.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:59 smithi117 ceph-mon[35309]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi117.cohbtv/mirror_snapshot_schedule"}]: dispatch 2024-02-16T19:29:00.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:59 smithi117 ceph-mon[35309]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi117.cohbtv/mirror_snapshot_schedule"}]: dispatch 2024-02-16T19:29:00.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:59 smithi117 ceph-mon[35309]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi117.cohbtv/trash_purge_schedule"}]: dispatch 2024-02-16T19:29:00.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:28:59 smithi117 ceph-mon[35309]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi117.cohbtv/trash_purge_schedule"}]: dispatch 2024-02-16T19:29:00.140 INFO:tasks.workunit.client.0.smithi022.stdout:2/258: truncate d0/d2/d18/f22 74253 0 2024-02-16T19:29:00.140 INFO:tasks.workunit.client.0.smithi022.stdout:8/316: creat d3/d12/d17/d64/f73 x:0 0 0 2024-02-16T19:29:00.141 INFO:tasks.workunit.client.1.smithi117.stdout:9/235: mkdir dd/d47/d48 0 2024-02-16T19:29:00.146 INFO:tasks.workunit.client.1.smithi117.stdout:7/267: mknod d4/d13/d63/c6a 0 2024-02-16T19:29:00.150 INFO:tasks.workunit.client.0.smithi022.stdout:9/348: mknod db/d14/d16/d49/c7c 0 2024-02-16T19:29:00.153 INFO:tasks.workunit.client.0.smithi022.stdout:7/299: symlink d9/da/d1f/d44/l58 0 2024-02-16T19:29:00.156 INFO:tasks.workunit.client.1.smithi117.stdout:1/217: creat d2/d1d/d1f/f46 x:0 0 0 2024-02-16T19:29:00.157 INFO:tasks.workunit.client.1.smithi117.stdout:2/222: mknod d0/d3/d22/d37/c47 0 2024-02-16T19:29:00.158 INFO:tasks.workunit.client.1.smithi117.stdout:2/223: readlink d0/d3/d10/l21 0 2024-02-16T19:29:00.159 INFO:tasks.workunit.client.0.smithi022.stdout:0/418: mknod d10/d22/d49/c89 0 2024-02-16T19:29:00.163 INFO:tasks.workunit.client.0.smithi022.stdout:3/317: rename d7/d12/d18/l82 to d7/d12/d18/dd/d24/d58/l83 0 2024-02-16T19:29:00.164 INFO:tasks.workunit.client.0.smithi022.stdout:4/280: dwrite d4/d6/f16 [4194304,4194304] 0 2024-02-16T19:29:00.166 INFO:tasks.workunit.client.0.smithi022.stdout:3/318: chown d7/d12/d18/dd/d31/d41/c5f 5 1 2024-02-16T19:29:00.169 INFO:tasks.workunit.client.0.smithi022.stdout:8/317: mkdir d3/d12/d2b/d54/d74 0 2024-02-16T19:29:00.171 INFO:tasks.workunit.client.0.smithi022.stdout:6/320: link d1/d8/d3c/d4d/c4e d1/d8/d23/c76 0 2024-02-16T19:29:00.175 INFO:tasks.workunit.client.1.smithi117.stdout:0/336: getdents d0/d9/d14/d1d/d60/d65 0 2024-02-16T19:29:00.176 INFO:tasks.workunit.client.1.smithi117.stdout:9/236: dwrite fb [4194304,4194304] 0 2024-02-16T19:29:00.176 INFO:tasks.workunit.client.0.smithi022.stdout:9/349: symlink db/d14/d16/d21/l7d 0 2024-02-16T19:29:00.178 INFO:tasks.workunit.client.1.smithi117.stdout:7/268: creat d4/d13/d63/f6b x:0 0 0 2024-02-16T19:29:00.181 INFO:tasks.workunit.client.1.smithi117.stdout:3/327: link d1/c7 d1/dd/c6e 0 2024-02-16T19:29:00.181 INFO:tasks.workunit.client.1.smithi117.stdout:3/328: dread - d1/d1b/f5f zero size 2024-02-16T19:29:00.182 INFO:tasks.workunit.client.1.smithi117.stdout:8/271: getdents d4/d7/d37/d3a/d43 0 2024-02-16T19:29:00.183 INFO:tasks.workunit.client.1.smithi117.stdout:3/329: write d1/d1b/d1e/d37/f38 [1995161,123473] 0 2024-02-16T19:29:00.184 INFO:tasks.workunit.client.1.smithi117.stdout:3/330: readlink d1/d1b/d1e/l4b 0 2024-02-16T19:29:00.184 INFO:tasks.workunit.client.0.smithi022.stdout:5/327: creat dd/d1f/d48/d21/d2f/d40/d4e/f68 x:0 0 0 2024-02-16T19:29:00.185 INFO:tasks.workunit.client.1.smithi117.stdout:3/331: truncate d1/dd/d5c/f6b 717009 0 2024-02-16T19:29:00.185 INFO:tasks.workunit.client.1.smithi117.stdout:5/377: truncate d2/d1d/f3f 377568 0 2024-02-16T19:29:00.186 INFO:tasks.workunit.client.1.smithi117.stdout:3/332: stat d1/d1b/f2d 0 2024-02-16T19:29:00.187 INFO:tasks.workunit.client.1.smithi117.stdout:2/224: rmdir d0/d3/d22 39 2024-02-16T19:29:00.189 INFO:tasks.workunit.client.1.smithi117.stdout:4/284: getdents d0 0 2024-02-16T19:29:00.189 INFO:tasks.workunit.client.1.smithi117.stdout:2/225: write d0/d3/d10/f1d [394882,73427] 0 2024-02-16T19:29:00.191 INFO:tasks.workunit.client.0.smithi022.stdout:4/281: fdatasync d4/d6/dd/df/d10/f50 0 2024-02-16T19:29:00.192 INFO:tasks.workunit.client.1.smithi117.stdout:4/285: read d0/da/d21/f3e [194767,61089] 0 2024-02-16T19:29:00.193 INFO:tasks.workunit.client.1.smithi117.stdout:5/378: dwrite d2/d3/f4d [0,4194304] 0 2024-02-16T19:29:00.194 INFO:tasks.workunit.client.1.smithi117.stdout:0/337: unlink d0/d9/d14/d1d/d60/f49 0 2024-02-16T19:29:00.199 INFO:tasks.workunit.client.0.smithi022.stdout:4/282: dwrite d4/d6/d14/f28 [0,4194304] 0 2024-02-16T19:29:00.201 INFO:tasks.workunit.client.1.smithi117.stdout:9/237: dwrite dd/d2d/f35 [0,4194304] 0 2024-02-16T19:29:00.202 INFO:tasks.workunit.client.0.smithi022.stdout:4/283: fsync d4/d6/d14/f1d 0 2024-02-16T19:29:00.203 INFO:tasks.workunit.client.1.smithi117.stdout:9/238: truncate dd/f19 1314873 0 2024-02-16T19:29:00.203 INFO:tasks.workunit.client.1.smithi117.stdout:9/239: chown dd/d23/l43 112482 1 2024-02-16T19:29:00.204 INFO:tasks.workunit.client.0.smithi022.stdout:3/319: write d7/d21/f7c [333694,99925] 0 2024-02-16T19:29:00.207 INFO:tasks.workunit.client.0.smithi022.stdout:3/320: truncate d7/d12/d18/dd/d3c/f49 745874 0 2024-02-16T19:29:00.208 INFO:tasks.workunit.client.0.smithi022.stdout:9/350: creat db/d14/d16/d5d/f7e x:0 0 0 2024-02-16T19:29:00.210 INFO:tasks.workunit.client.0.smithi022.stdout:9/351: write db/d14/d16/d27/f78 [35894,88017] 0 2024-02-16T19:29:00.217 INFO:tasks.workunit.client.0.smithi022.stdout:2/259: creat d0/d2/f57 x:0 0 0 2024-02-16T19:29:00.219 INFO:tasks.workunit.client.0.smithi022.stdout:7/300: truncate d9/da/d1f/f22 1183795 0 2024-02-16T19:29:00.224 INFO:tasks.workunit.client.1.smithi117.stdout:4/286: creat d0/da/d4a/f59 x:0 0 0 2024-02-16T19:29:00.225 INFO:tasks.workunit.client.0.smithi022.stdout:1/411: sync 2024-02-16T19:29:00.227 INFO:tasks.workunit.client.1.smithi117.stdout:0/338: mkdir d0/d9/d14/d1d/d60/d76/d7b 0 2024-02-16T19:29:00.229 INFO:tasks.workunit.client.0.smithi022.stdout:4/284: unlink d4/d6/dd/f2a 0 2024-02-16T19:29:00.229 INFO:tasks.workunit.client.1.smithi117.stdout:4/287: dwrite d0/f25 [0,4194304] 0 2024-02-16T19:29:00.231 INFO:tasks.workunit.client.1.smithi117.stdout:7/269: fsync d4/f2b 0 2024-02-16T19:29:00.233 INFO:tasks.workunit.client.1.smithi117.stdout:7/270: fdatasync d4/d1c/d39/d46/d47/f51 0 2024-02-16T19:29:00.234 INFO:tasks.workunit.client.0.smithi022.stdout:6/321: mkdir d1/d8/d3c/d77 0 2024-02-16T19:29:00.241 INFO:tasks.workunit.client.0.smithi022.stdout:5/328: symlink dd/d1f/l69 0 2024-02-16T19:29:00.244 INFO:tasks.workunit.client.1.smithi117.stdout:1/218: rename d2/d18/f1b to d2/d4/f47 0 2024-02-16T19:29:00.245 INFO:tasks.workunit.client.0.smithi022.stdout:9/352: unlink f7 0 2024-02-16T19:29:00.247 INFO:tasks.workunit.client.0.smithi022.stdout:9/353: write db/d14/d16/d5d/f7e [434976,38489] 0 2024-02-16T19:29:00.247 INFO:tasks.workunit.client.0.smithi022.stdout:0/419: creat d10/d11/d27/d3f/f8a x:0 0 0 2024-02-16T19:29:00.248 INFO:tasks.workunit.client.1.smithi117.stdout:2/226: creat d0/d3/d22/d37/f48 x:0 0 0 2024-02-16T19:29:00.250 INFO:tasks.workunit.client.1.smithi117.stdout:6/298: getdents d5/d4b/d20/d2d 0 2024-02-16T19:29:00.253 INFO:tasks.workunit.client.1.smithi117.stdout:6/299: dread d5/d1d/d28/d32/f48 [0,4194304] 0 2024-02-16T19:29:00.253 INFO:tasks.workunit.client.1.smithi117.stdout:6/300: chown d5 281719 1 2024-02-16T19:29:00.256 INFO:tasks.workunit.client.0.smithi022.stdout:7/301: mknod d9/da/d3b/c59 0 2024-02-16T19:29:00.258 INFO:tasks.workunit.client.1.smithi117.stdout:4/288: symlink d0/da/d21/d3d/d28/l5a 0 2024-02-16T19:29:00.262 INFO:tasks.workunit.client.0.smithi022.stdout:1/412: rename d6/d8/d22/d16/d17/f4c to d6/d8/d22/d47/d54/f79 0 2024-02-16T19:29:00.262 INFO:tasks.workunit.client.1.smithi117.stdout:3/333: dwrite d1/dd/f17 [0,4194304] 0 2024-02-16T19:29:00.270 INFO:tasks.workunit.client.0.smithi022.stdout:1/413: dwrite d6/d8/d22/d3a/f49 [0,4194304] 0 2024-02-16T19:29:00.270 INFO:tasks.workunit.client.0.smithi022.stdout:6/322: mkdir d1/d8/da/d29/d78 0 2024-02-16T19:29:00.272 INFO:tasks.workunit.client.0.smithi022.stdout:1/414: chown d6/d8/d22/d3a/d3e/f76 86 1 2024-02-16T19:29:00.272 INFO:tasks.workunit.client.0.smithi022.stdout:1/415: stat d6/l4a 0 2024-02-16T19:29:00.272 INFO:tasks.workunit.client.1.smithi117.stdout:5/379: rename d2/d4c/d4f/f70 to d2/d4c/d69/f72 0 2024-02-16T19:29:00.273 INFO:tasks.workunit.client.1.smithi117.stdout:5/380: chown d2/d3/l2b 25 1 2024-02-16T19:29:00.275 INFO:tasks.workunit.client.0.smithi022.stdout:4/285: write d4/d6/dd/df/d10/f50 [418125,66353] 0 2024-02-16T19:29:00.277 INFO:tasks.workunit.client.0.smithi022.stdout:4/286: chown d4/d6/d3a/c53 93011 1 2024-02-16T19:29:00.279 INFO:tasks.workunit.client.0.smithi022.stdout:5/329: unlink dd/d1f/d48/d21/f3e 0 2024-02-16T19:29:00.283 INFO:tasks.workunit.client.1.smithi117.stdout:4/289: creat d0/da/f5b x:0 0 0 2024-02-16T19:29:00.285 INFO:tasks.workunit.client.1.smithi117.stdout:6/301: dwrite d5/d4b/d44/d52/f5f [0,4194304] 0 2024-02-16T19:29:00.286 INFO:tasks.workunit.client.0.smithi022.stdout:9/354: symlink db/d14/d19/d1a/d67/l7f 0 2024-02-16T19:29:00.286 INFO:tasks.workunit.client.1.smithi117.stdout:6/302: write d5/d4b/d20/f21 [729033,51379] 0 2024-02-16T19:29:00.289 INFO:tasks.workunit.client.0.smithi022.stdout:3/321: dwrite d7/d12/d18/dd/f20 [0,4194304] 0 2024-02-16T19:29:00.291 INFO:tasks.workunit.client.1.smithi117.stdout:3/334: mknod d1/d1b/c6f 0 2024-02-16T19:29:00.292 INFO:tasks.workunit.client.0.smithi022.stdout:8/318: getdents d3/d12/d6a 0 2024-02-16T19:29:00.293 INFO:tasks.workunit.client.1.smithi117.stdout:9/240: creat dd/d45/f49 x:0 0 0 2024-02-16T19:29:00.294 INFO:tasks.workunit.client.1.smithi117.stdout:6/303: dwrite d5/d4b/f1e [0,4194304] 0 2024-02-16T19:29:00.297 INFO:tasks.workunit.client.1.smithi117.stdout:1/219: truncate d2/d4/fa 4513417 0 2024-02-16T19:29:00.301 INFO:tasks.workunit.client.0.smithi022.stdout:6/323: symlink d1/d8/d23/l79 0 2024-02-16T19:29:00.303 INFO:tasks.workunit.client.0.smithi022.stdout:6/324: read - d1/d24/f5f zero size 2024-02-16T19:29:00.304 INFO:tasks.workunit.client.1.smithi117.stdout:0/339: rename d0/d9/d14/d1d/d1e/d6b to d0/d9/d14/d23/d2f/d7c 0 2024-02-16T19:29:00.307 INFO:tasks.workunit.client.1.smithi117.stdout:5/381: rmdir d2/d9/d3c/d42 39 2024-02-16T19:29:00.308 INFO:tasks.workunit.client.1.smithi117.stdout:5/382: read d2/d1d/f48 [278151,9265] 0 2024-02-16T19:29:00.309 INFO:tasks.workunit.client.0.smithi022.stdout:7/302: dwrite d9/da/f29 [0,4194304] 0 2024-02-16T19:29:00.314 INFO:tasks.workunit.client.1.smithi117.stdout:4/290: creat d0/f5c x:0 0 0 2024-02-16T19:29:00.315 INFO:tasks.workunit.client.0.smithi022.stdout:1/416: dwrite d6/f19 [0,4194304] 0 2024-02-16T19:29:00.315 INFO:tasks.workunit.client.1.smithi117.stdout:8/272: sync 2024-02-16T19:29:00.318 INFO:tasks.workunit.client.0.smithi022.stdout:1/417: readlink d6/d8/d22/d16/l74 0 2024-02-16T19:29:00.319 INFO:tasks.workunit.client.0.smithi022.stdout:9/355: symlink db/d14/l80 0 2024-02-16T19:29:00.320 INFO:tasks.workunit.client.1.smithi117.stdout:3/335: mkdir d1/d1b/d70 0 2024-02-16T19:29:00.321 INFO:tasks.workunit.client.1.smithi117.stdout:3/336: dread - d1/d1b/d1e/d5e/f66 zero size 2024-02-16T19:29:00.321 INFO:tasks.workunit.client.0.smithi022.stdout:1/418: fdatasync d6/d8/d22/d16/d17/d26/f46 0 2024-02-16T19:29:00.321 INFO:tasks.workunit.client.0.smithi022.stdout:3/322: dread d7/d12/f14 [0,4194304] 0 2024-02-16T19:29:00.322 INFO:tasks.workunit.client.0.smithi022.stdout:9/356: fdatasync db/d14/d16/d27/f78 0 2024-02-16T19:29:00.325 INFO:tasks.workunit.client.1.smithi117.stdout:9/241: creat dd/f4a x:0 0 0 2024-02-16T19:29:00.327 INFO:tasks.workunit.client.1.smithi117.stdout:6/304: mknod d5/db/c66 0 2024-02-16T19:29:00.328 INFO:tasks.workunit.client.1.smithi117.stdout:6/305: read d5/f12 [419367,30389] 0 2024-02-16T19:29:00.329 INFO:tasks.workunit.client.0.smithi022.stdout:9/357: truncate db/d14/d16/d5d/f7e 1262283 0 2024-02-16T19:29:00.329 INFO:tasks.workunit.client.1.smithi117.stdout:6/306: stat d5/ff 0 2024-02-16T19:29:00.329 INFO:tasks.workunit.client.1.smithi117.stdout:1/220: mknod d2/d18/c48 0 2024-02-16T19:29:00.330 INFO:tasks.workunit.client.1.smithi117.stdout:2/227: dwrite d0/d29/f2e [0,4194304] 0 2024-02-16T19:29:00.332 INFO:tasks.workunit.client.0.smithi022.stdout:0/420: write fb [2698607,98269] 0 2024-02-16T19:29:00.333 INFO:tasks.workunit.client.0.smithi022.stdout:9/358: truncate db/f79 712632 0 2024-02-16T19:29:00.334 INFO:tasks.workunit.client.1.smithi117.stdout:0/340: symlink d0/d9/d22/d27/d4e/l7d 0 2024-02-16T19:29:00.335 INFO:tasks.workunit.client.0.smithi022.stdout:6/325: mknod d1/d8/da/d17/d53/c7a 0 2024-02-16T19:29:00.338 INFO:tasks.workunit.client.0.smithi022.stdout:1/419: dwrite d6/d8/d22/d47/f4e [0,4194304] 0 2024-02-16T19:29:00.340 INFO:tasks.workunit.client.0.smithi022.stdout:7/303: mkdir d9/da/d2c/d5a 0 2024-02-16T19:29:00.342 INFO:tasks.workunit.client.0.smithi022.stdout:4/287: mkdir d4/d6/dd/df/d5b 0 2024-02-16T19:29:00.344 INFO:tasks.workunit.client.0.smithi022.stdout:5/330: dwrite dd/d1f/d48/f2d [0,4194304] 0 2024-02-16T19:29:00.344 INFO:tasks.workunit.client.1.smithi117.stdout:5/383: creat d2/d3/d21/d49/d41/f73 x:0 0 0 2024-02-16T19:29:00.347 INFO:tasks.workunit.client.0.smithi022.stdout:1/420: write d6/f6b [2470385,75555] 0 2024-02-16T19:29:00.349 INFO:tasks.workunit.client.0.smithi022.stdout:4/288: write d4/d38/f3d [1778513,102212] 0 2024-02-16T19:29:00.352 INFO:tasks.workunit.client.0.smithi022.stdout:2/260: getdents d0/d2/d18/d19/d40 0 2024-02-16T19:29:00.353 INFO:tasks.workunit.client.1.smithi117.stdout:7/271: link d4/d1c/d39/d46/d47/c64 d4/c6c 0 2024-02-16T19:29:00.354 INFO:tasks.workunit.client.0.smithi022.stdout:5/331: chown dd/d1f/d48/d21/d2f/d40 30634 1 2024-02-16T19:29:00.356 INFO:tasks.workunit.client.1.smithi117.stdout:8/273: creat d4/d12/d2a/f5c x:0 0 0 2024-02-16T19:29:00.357 INFO:tasks.workunit.client.0.smithi022.stdout:3/323: read d7/d21/f38 [351843,61713] 0 2024-02-16T19:29:00.358 INFO:tasks.workunit.client.0.smithi022.stdout:9/359: dwrite db/d14/f60 [0,4194304] 0 2024-02-16T19:29:00.360 INFO:tasks.workunit.client.0.smithi022.stdout:4/289: chown d4/d6/dd/df/d10/f42 22 1 2024-02-16T19:29:00.360 INFO:tasks.workunit.client.1.smithi117.stdout:3/337: mknod d1/d1b/d1e/d29/c71 0 2024-02-16T19:29:00.360 INFO:tasks.workunit.client.1.smithi117.stdout:3/338: dread - d1/dd/d3b/f63 zero size 2024-02-16T19:29:00.361 INFO:tasks.workunit.client.0.smithi022.stdout:6/326: dwrite d1/d8/fb [0,4194304] 0 2024-02-16T19:29:00.361 INFO:tasks.workunit.client.0.smithi022.stdout:2/261: dread - d0/d2/f57 zero size 2024-02-16T19:29:00.361 INFO:tasks.workunit.client.0.smithi022.stdout:5/332: write dd/f3c [1078571,89511] 0 2024-02-16T19:29:00.362 INFO:tasks.workunit.client.1.smithi117.stdout:9/242: mknod dd/d24/c4b 0 2024-02-16T19:29:00.368 INFO:tasks.workunit.client.0.smithi022.stdout:2/262: truncate d0/d2/d18/f1a 9247139 0 2024-02-16T19:29:00.369 INFO:tasks.workunit.client.1.smithi117.stdout:6/307: rmdir d5/d4b/d44 39 2024-02-16T19:29:00.371 INFO:tasks.workunit.client.1.smithi117.stdout:0/341: symlink d0/d9/d22/d26/d55/l7e 0 2024-02-16T19:29:00.372 INFO:tasks.workunit.client.1.smithi117.stdout:1/221: read d2/d4/fa [1239965,85598] 0 2024-02-16T19:29:00.374 INFO:tasks.workunit.client.0.smithi022.stdout:0/421: unlink d10/d15/d23/d38/d5a/f80 0 2024-02-16T19:29:00.374 INFO:tasks.workunit.client.0.smithi022.stdout:4/290: dread d4/d6/dd/df/d10/f35 [0,4194304] 0 2024-02-16T19:29:00.376 INFO:tasks.workunit.client.1.smithi117.stdout:5/384: mkdir d2/d9/d3c/d24/d50/d52/d74 0 2024-02-16T19:29:00.377 INFO:tasks.workunit.client.1.smithi117.stdout:6/308: dwrite d5/d4b/d20/f21 [0,4194304] 0 2024-02-16T19:29:00.382 INFO:tasks.workunit.client.1.smithi117.stdout:7/272: symlink d4/d1c/d39/d46/d55/l6d 0 2024-02-16T19:29:00.382 INFO:tasks.workunit.client.1.smithi117.stdout:5/385: dread d2/d3/d21/f32 [0,4194304] 0 2024-02-16T19:29:00.385 INFO:tasks.workunit.client.1.smithi117.stdout:8/274: unlink d4/d7/c45 0 2024-02-16T19:29:00.386 INFO:tasks.workunit.client.0.smithi022.stdout:7/304: mkdir d9/da/d1f/d44/d5b 0 2024-02-16T19:29:00.387 INFO:tasks.workunit.client.0.smithi022.stdout:7/305: chown d9/da/d1f/l38 582595 1 2024-02-16T19:29:00.387 INFO:tasks.workunit.client.1.smithi117.stdout:3/339: mknod d1/dd/d3e/c72 0 2024-02-16T19:29:00.387 INFO:tasks.workunit.client.1.smithi117.stdout:3/340: stat d1/dd/c35 0 2024-02-16T19:29:00.388 INFO:tasks.workunit.client.0.smithi022.stdout:1/421: mkdir d6/d8/d22/d3a/d7a 0 2024-02-16T19:29:00.393 INFO:tasks.workunit.client.0.smithi022.stdout:1/422: dread d6/f6b [0,4194304] 0 2024-02-16T19:29:00.393 INFO:tasks.workunit.client.1.smithi117.stdout:0/342: creat d0/d9/f7f x:0 0 0 2024-02-16T19:29:00.395 INFO:tasks.workunit.client.1.smithi117.stdout:1/222: mkdir d2/d18/d20/d49 0 2024-02-16T19:29:00.396 INFO:tasks.workunit.client.0.smithi022.stdout:3/324: mkdir d7/d12/d18/dd/d3c/d5a/d67/d84 0 2024-02-16T19:29:00.396 INFO:tasks.workunit.client.0.smithi022.stdout:1/423: dread d6/d8/d22/d16/f2e [0,4194304] 0 2024-02-16T19:29:00.399 INFO:tasks.workunit.client.0.smithi022.stdout:9/360: rename db/d14/d16/f1f to db/d14/d19/d1a/d67/f81 0 2024-02-16T19:29:00.400 INFO:tasks.workunit.client.0.smithi022.stdout:5/333: dread dd/f1d [0,4194304] 0 2024-02-16T19:29:00.402 INFO:tasks.workunit.client.1.smithi117.stdout:4/291: creat d0/d6/d48/f5d x:0 0 0 2024-02-16T19:29:00.406 INFO:tasks.workunit.client.0.smithi022.stdout:4/291: symlink d4/d6/l5c 0 2024-02-16T19:29:00.408 INFO:tasks.workunit.client.0.smithi022.stdout:4/292: read d4/d6/dd/f2c [701721,546] 0 2024-02-16T19:29:00.410 INFO:tasks.workunit.client.1.smithi117.stdout:8/275: symlink d4/d7/d37/d3a/d43/d44/l5d 0 2024-02-16T19:29:00.411 INFO:tasks.workunit.client.1.smithi117.stdout:8/276: write d4/d12/d2a/f2c [1319731,92486] 0 2024-02-16T19:29:00.413 INFO:tasks.workunit.client.1.smithi117.stdout:8/277: truncate d4/d12/d2a/f48 72084 0 2024-02-16T19:29:00.416 INFO:tasks.workunit.client.0.smithi022.stdout:6/327: mkdir d1/d8/da/d7b 0 2024-02-16T19:29:00.417 INFO:tasks.workunit.client.0.smithi022.stdout:6/328: write d1/f72 [246048,90549] 0 2024-02-16T19:29:00.419 INFO:tasks.workunit.client.1.smithi117.stdout:8/278: dwrite d4/d7/d37/d3a/d43/d44/f4b [0,4194304] 0 2024-02-16T19:29:00.424 INFO:tasks.workunit.client.1.smithi117.stdout:1/223: rmdir d2 39 2024-02-16T19:29:00.427 INFO:tasks.workunit.client.1.smithi117.stdout:8/279: dread d4/d12/d2a/f2c [0,4194304] 0 2024-02-16T19:29:00.428 INFO:tasks.workunit.client.1.smithi117.stdout:6/309: fdatasync d5/d4b/f46 0 2024-02-16T19:29:00.429 INFO:tasks.workunit.client.1.smithi117.stdout:0/343: dwrite d0/d9/d22/d27/f4d [0,4194304] 0 2024-02-16T19:29:00.431 INFO:tasks.workunit.client.1.smithi117.stdout:8/280: truncate d4/d7/f3d 134918 0 2024-02-16T19:29:00.431 INFO:tasks.workunit.client.0.smithi022.stdout:8/319: getdents d3/d12/d2b 0 2024-02-16T19:29:00.433 INFO:tasks.workunit.client.0.smithi022.stdout:0/422: mkdir d10/d11/d79/d8b 0 2024-02-16T19:29:00.436 INFO:tasks.workunit.client.1.smithi117.stdout:7/273: dwrite f2 [0,4194304] 0 2024-02-16T19:29:00.439 INFO:tasks.workunit.client.0.smithi022.stdout:4/293: rename d4/d6/dd/d31/d4e/d54/l56 to d4/d6/dd/d31/d4e/l5d 0 2024-02-16T19:29:00.439 INFO:tasks.workunit.client.0.smithi022.stdout:4/294: fdatasync d4/d6/d3a/f41 0 2024-02-16T19:29:00.442 INFO:tasks.workunit.client.1.smithi117.stdout:9/243: link dd/d23/l33 dd/d24/l4c 0 2024-02-16T19:29:00.445 INFO:tasks.workunit.client.1.smithi117.stdout:2/228: getdents d0/d3/d22 0 2024-02-16T19:29:00.445 INFO:tasks.workunit.client.1.smithi117.stdout:5/386: truncate d2/d9/d3c/d24/f26 6493844 0 2024-02-16T19:29:00.450 INFO:tasks.workunit.client.0.smithi022.stdout:9/361: mknod db/c82 0 2024-02-16T19:29:00.451 INFO:tasks.workunit.client.0.smithi022.stdout:5/334: mkdir dd/d1f/d48/d21/d2f/d52/d5c/d61/d6a 0 2024-02-16T19:29:00.451 INFO:tasks.workunit.client.1.smithi117.stdout:6/310: stat d5/d4b/d44/d52/d64 0 2024-02-16T19:29:00.454 INFO:tasks.workunit.client.0.smithi022.stdout:7/306: creat d9/da/d2c/d2f/f5c x:0 0 0 2024-02-16T19:29:00.455 INFO:tasks.workunit.client.1.smithi117.stdout:7/274: write d4/d1c/f3c [621266,73728] 0 2024-02-16T19:29:00.456 INFO:tasks.workunit.client.0.smithi022.stdout:7/307: write d9/d1b/f27 [2118459,34200] 0 2024-02-16T19:29:00.457 INFO:tasks.workunit.client.1.smithi117.stdout:3/341: rmdir d1/d31/d55 0 2024-02-16T19:29:00.457 INFO:tasks.workunit.client.0.smithi022.stdout:7/308: write d9/f14 [1285785,15345] 0 2024-02-16T19:29:00.459 INFO:tasks.workunit.client.1.smithi117.stdout:9/244: creat dd/d2d/f4d x:0 0 0 2024-02-16T19:29:00.460 INFO:tasks.workunit.client.1.smithi117.stdout:9/245: write dd/f4a [181431,116349] 0 2024-02-16T19:29:00.462 INFO:tasks.workunit.client.0.smithi022.stdout:9/362: symlink db/d14/d19/d1a/d67/l83 0 2024-02-16T19:29:00.463 INFO:tasks.workunit.client.1.smithi117.stdout:2/229: creat d0/d29/f49 x:0 0 0 2024-02-16T19:29:00.466 INFO:tasks.workunit.client.0.smithi022.stdout:0/423: symlink d10/d15/d23/l8c 0 2024-02-16T19:29:00.468 INFO:tasks.workunit.client.0.smithi022.stdout:3/325: truncate d7/d12/d18/f1b 28482 0 2024-02-16T19:29:00.469 INFO:tasks.workunit.client.0.smithi022.stdout:3/326: readlink d7/d12/d18/l10 0 2024-02-16T19:29:00.470 INFO:tasks.workunit.client.0.smithi022.stdout:3/327: read - d7/f61 zero size 2024-02-16T19:29:00.470 INFO:tasks.workunit.client.0.smithi022.stdout:0/424: dread d10/d22/d49/f7b [0,4194304] 0 2024-02-16T19:29:00.471 INFO:tasks.workunit.client.0.smithi022.stdout:4/295: mknod d4/d6/dd/df/d10/d55/c5e 0 2024-02-16T19:29:00.473 INFO:tasks.workunit.client.0.smithi022.stdout:0/425: write d10/f6e [1146860,61326] 0 2024-02-16T19:29:00.473 INFO:tasks.workunit.client.1.smithi117.stdout:6/311: unlink c3 0 2024-02-16T19:29:00.476 INFO:tasks.workunit.client.0.smithi022.stdout:1/424: dwrite d6/d8/d22/d16/d17/d1d/f33 [0,4194304] 0 2024-02-16T19:29:00.476 INFO:tasks.workunit.client.1.smithi117.stdout:0/344: mknod d0/d30/d74/c80 0 2024-02-16T19:29:00.481 INFO:tasks.workunit.client.0.smithi022.stdout:1/425: chown d6/d8/d22/d16/c3c 0 1 2024-02-16T19:29:00.482 INFO:tasks.workunit.client.0.smithi022.stdout:3/328: dwrite d7/d12/d18/f1a [0,4194304] 0 2024-02-16T19:29:00.487 INFO:tasks.workunit.client.0.smithi022.stdout:3/329: dread d7/d12/d18/dd/d3c/f49 [0,4194304] 0 2024-02-16T19:29:00.488 INFO:tasks.workunit.client.0.smithi022.stdout:3/330: chown d7/f1c 29662 1 2024-02-16T19:29:00.488 INFO:tasks.workunit.client.1.smithi117.stdout:7/275: unlink d4/c6c 0 2024-02-16T19:29:00.489 INFO:tasks.workunit.client.0.smithi022.stdout:8/320: write d3/f15 [931484,35249] 0 2024-02-16T19:29:00.493 INFO:tasks.workunit.client.1.smithi117.stdout:3/342: rename d1/f9 to d1/dd/d32/f73 0 2024-02-16T19:29:00.493 INFO:tasks.workunit.client.0.smithi022.stdout:0/426: dread d10/d11/d27/d3f/f57 [0,4194304] 0 2024-02-16T19:29:00.493 INFO:tasks.workunit.client.1.smithi117.stdout:3/343: stat d1/d1b/d1e/d37/f6d 0 2024-02-16T19:29:00.494 INFO:tasks.workunit.client.1.smithi117.stdout:3/344: read d1/dd/f48 [594670,102324] 0 2024-02-16T19:29:00.495 INFO:tasks.workunit.client.0.smithi022.stdout:8/321: dread d3/d12/f51 [0,4194304] 0 2024-02-16T19:29:00.497 INFO:tasks.workunit.client.1.smithi117.stdout:5/387: mknod d2/d9/d3c/d24/d50/c75 0 2024-02-16T19:29:00.497 INFO:tasks.workunit.client.1.smithi117.stdout:5/388: stat d2/d3/f35 0 2024-02-16T19:29:00.501 INFO:tasks.workunit.client.0.smithi022.stdout:0/427: dwrite d10/d11/d27/f52 [0,4194304] 0 2024-02-16T19:29:00.502 INFO:tasks.workunit.client.1.smithi117.stdout:1/224: creat d2/d18/d20/d49/f4a x:0 0 0 2024-02-16T19:29:00.503 INFO:tasks.workunit.client.0.smithi022.stdout:8/322: dread d3/d12/d2b/f3f [0,4194304] 0 2024-02-16T19:29:00.503 INFO:tasks.workunit.client.1.smithi117.stdout:1/225: truncate d2/d1d/f39 7017 0 2024-02-16T19:29:00.504 INFO:tasks.workunit.client.0.smithi022.stdout:8/323: chown d3/d25/f5c 1 1 2024-02-16T19:29:00.505 INFO:tasks.workunit.client.0.smithi022.stdout:8/324: chown d3/d12/d17 59272 1 2024-02-16T19:29:00.505 INFO:tasks.workunit.client.1.smithi117.stdout:6/312: rmdir d5/d4b/d44/d52 39 2024-02-16T19:29:00.511 INFO:tasks.workunit.client.1.smithi117.stdout:0/345: symlink d0/d9/l81 0 2024-02-16T19:29:00.512 INFO:tasks.workunit.client.0.smithi022.stdout:9/363: write db/d14/d19/f4d [3632109,110701] 0 2024-02-16T19:29:00.513 INFO:tasks.workunit.client.1.smithi117.stdout:4/292: creat d0/da/d21/d3d/f5e x:0 0 0 2024-02-16T19:29:00.514 INFO:tasks.workunit.client.0.smithi022.stdout:9/364: write db/d14/d16/d27/f78 [368444,29081] 0 2024-02-16T19:29:00.516 INFO:tasks.workunit.client.0.smithi022.stdout:4/296: truncate d4/d6/dd/d22/f59 3544481 0 2024-02-16T19:29:00.517 INFO:tasks.workunit.client.1.smithi117.stdout:8/281: link d4/d7/lf d4/d7/d37/l5e 0 2024-02-16T19:29:00.518 INFO:tasks.workunit.client.0.smithi022.stdout:4/297: write d4/d6/dd/d22/d30/f26 [1426400,39471] 0 2024-02-16T19:29:00.518 INFO:tasks.workunit.client.0.smithi022.stdout:6/329: getdents d1/d8/da/d17/d48 0 2024-02-16T19:29:00.519 INFO:tasks.workunit.client.0.smithi022.stdout:6/330: stat d1/d8/da/d17/d53/c7a 0 2024-02-16T19:29:00.520 INFO:tasks.workunit.client.1.smithi117.stdout:7/276: mkdir d4/d1c/d39/d46/d6e 0 2024-02-16T19:29:00.520 INFO:tasks.workunit.client.0.smithi022.stdout:6/331: chown d1/d8/da/d29/d61/l6d 287 1 2024-02-16T19:29:00.521 INFO:tasks.workunit.client.1.smithi117.stdout:7/277: write d4/f1a [3895210,40277] 0 2024-02-16T19:29:00.522 INFO:tasks.workunit.client.0.smithi022.stdout:6/332: fsync d1/d8/da/d17/d48/f74 0 2024-02-16T19:29:00.522 INFO:tasks.workunit.client.0.smithi022.stdout:6/333: chown d1/c9 2179 1 2024-02-16T19:29:00.523 INFO:tasks.workunit.client.1.smithi117.stdout:8/282: dwrite d4/d12/f42 [0,4194304] 0 2024-02-16T19:29:00.523 INFO:tasks.workunit.client.1.smithi117.stdout:7/278: rename d4/d1c/d24/d5f to d4/d1c/d24/d5f/d6f 22 2024-02-16T19:29:00.524 INFO:tasks.workunit.client.1.smithi117.stdout:7/279: dread - d4/d1c/d39/f54 zero size 2024-02-16T19:29:00.524 INFO:tasks.workunit.client.1.smithi117.stdout:8/283: write d4/d7/f3d [415589,37032] 0 2024-02-16T19:29:00.528 INFO:tasks.workunit.client.1.smithi117.stdout:5/389: creat d2/d3/f76 x:0 0 0 2024-02-16T19:29:00.529 INFO:tasks.workunit.client.0.smithi022.stdout:3/331: creat d7/d12/d59/f85 x:0 0 0 2024-02-16T19:29:00.531 INFO:tasks.workunit.client.1.smithi117.stdout:5/390: truncate d2/d3/d21/d2f/d30/f68 215583 0 2024-02-16T19:29:00.531 INFO:tasks.workunit.client.1.smithi117.stdout:7/280: dwrite d4/f3a [4194304,4194304] 0 2024-02-16T19:29:00.531 INFO:tasks.workunit.client.1.smithi117.stdout:1/226: unlink d2/d18/f43 0 2024-02-16T19:29:00.533 INFO:tasks.workunit.client.1.smithi117.stdout:1/227: write d2/d4/f16 [1025444,52748] 0 2024-02-16T19:29:00.534 INFO:tasks.workunit.client.1.smithi117.stdout:1/228: rename d2 to d2/d18/d4b 22 2024-02-16T19:29:00.542 INFO:tasks.workunit.client.0.smithi022.stdout:5/335: link dd/c44 dd/d1f/d48/d21/d2f/d52/d5c/d61/d6a/c6b 0 2024-02-16T19:29:00.544 INFO:tasks.workunit.client.0.smithi022.stdout:0/428: symlink d10/d22/d2c/l8d 0 2024-02-16T19:29:00.548 INFO:tasks.workunit.client.1.smithi117.stdout:1/229: dread d2/d1d/d1f/f44 [0,4194304] 0 2024-02-16T19:29:00.549 INFO:tasks.workunit.client.1.smithi117.stdout:1/230: chown d2/d18/d20/f2b 26615 1 2024-02-16T19:29:00.550 INFO:tasks.workunit.client.1.smithi117.stdout:1/231: write d2/d18/d20/d49/f4a [39443,123387] 0 2024-02-16T19:29:00.551 INFO:tasks.workunit.client.1.smithi117.stdout:1/232: readlink d2/d18/d20/l33 0 2024-02-16T19:29:00.553 INFO:tasks.workunit.client.1.smithi117.stdout:3/345: fsync d1/d1b/d1e/d29/f47 0 2024-02-16T19:29:00.560 INFO:tasks.workunit.client.0.smithi022.stdout:1/426: link d6/d8/f2d d6/d8/d22/d16/d17/d5d/f7b 0 2024-02-16T19:29:00.562 INFO:tasks.workunit.client.0.smithi022.stdout:6/334: rmdir d1/d24/d5b 39 2024-02-16T19:29:00.566 INFO:tasks.workunit.client.1.smithi117.stdout:5/391: creat d2/d4c/d54/f77 x:0 0 0 2024-02-16T19:29:00.566 INFO:tasks.workunit.client.1.smithi117.stdout:4/293: dwrite d0/da/d21/f3e [0,4194304] 0 2024-02-16T19:29:00.567 INFO:tasks.workunit.client.0.smithi022.stdout:7/309: rmdir d9/da/d2c/d5a 0 2024-02-16T19:29:00.567 INFO:tasks.workunit.client.1.smithi117.stdout:3/346: dread d1/dd/f43 [0,4194304] 0 2024-02-16T19:29:00.569 INFO:tasks.workunit.client.0.smithi022.stdout:7/310: write d9/da/d2c/d2f/f35 [3719,18334] 0 2024-02-16T19:29:00.569 INFO:tasks.workunit.client.1.smithi117.stdout:3/347: write d1/dd/d3b/f5a [341676,18728] 0 2024-02-16T19:29:00.569 INFO:tasks.workunit.client.1.smithi117.stdout:7/281: mknod d4/d1c/d39/d46/d47/c70 0 2024-02-16T19:29:00.569 INFO:tasks.workunit.client.1.smithi117.stdout:3/348: fsync d1/d1b/f5f 0 2024-02-16T19:29:00.575 INFO:tasks.workunit.client.0.smithi022.stdout:8/325: dwrite d3/d6/f4d [4194304,4194304] 0 2024-02-16T19:29:00.576 INFO:tasks.workunit.client.1.smithi117.stdout:0/346: creat d0/d9/d22/d27/d31/d6d/f82 x:0 0 0 2024-02-16T19:29:00.577 INFO:tasks.workunit.client.1.smithi117.stdout:0/347: dread - d0/d9/d22/d27/d31/d6d/f82 zero size 2024-02-16T19:29:00.579 INFO:tasks.workunit.client.0.smithi022.stdout:5/336: mknod dd/d1f/d48/d21/d2f/d40/c6c 0 2024-02-16T19:29:00.580 INFO:tasks.workunit.client.0.smithi022.stdout:5/337: chown dd/d1f/d48/c20 266220152 1 2024-02-16T19:29:00.583 INFO:tasks.workunit.client.0.smithi022.stdout:0/429: unlink d10/d11/f2e 0 2024-02-16T19:29:00.584 INFO:tasks.workunit.client.0.smithi022.stdout:0/430: chown d10/d11/c6f 35295743 1 2024-02-16T19:29:00.586 INFO:tasks.workunit.client.1.smithi117.stdout:1/233: mknod d2/d19/c4c 0 2024-02-16T19:29:00.587 INFO:tasks.workunit.client.0.smithi022.stdout:4/298: creat d4/d6/dd/d31/d4e/d54/d5a/f5f x:0 0 0 2024-02-16T19:29:00.590 INFO:tasks.workunit.client.0.smithi022.stdout:6/335: unlink d1/d8/da/d29/c3a 0 2024-02-16T19:29:00.590 INFO:tasks.workunit.client.1.smithi117.stdout:6/313: link d5/c2a d5/d1d/d49/d59/c67 0 2024-02-16T19:29:00.591 INFO:tasks.workunit.client.0.smithi022.stdout:6/336: write d1/d8/d3c/f44 [2355328,123245] 0 2024-02-16T19:29:00.593 INFO:tasks.workunit.client.1.smithi117.stdout:9/246: sync 2024-02-16T19:29:00.593 INFO:tasks.workunit.client.1.smithi117.stdout:2/230: sync 2024-02-16T19:29:00.594 INFO:tasks.workunit.client.1.smithi117.stdout:5/392: fsync d2/d1d/f48 0 2024-02-16T19:29:00.594 INFO:tasks.workunit.client.1.smithi117.stdout:2/231: stat d0/d3/d22/d37/f48 0 2024-02-16T19:29:00.595 INFO:tasks.workunit.client.1.smithi117.stdout:7/282: mkdir d4/d13/d63/d71 0 2024-02-16T19:29:00.598 INFO:tasks.workunit.client.1.smithi117.stdout:3/349: creat d1/d1b/d1e/d5e/f74 x:0 0 0 2024-02-16T19:29:00.599 INFO:tasks.workunit.client.0.smithi022.stdout:8/326: mknod d3/d12/d2b/d2e/c75 0 2024-02-16T19:29:00.601 INFO:tasks.workunit.client.1.smithi117.stdout:7/283: dwrite d4/d1c/d39/d46/f57 [0,4194304] 0 2024-02-16T19:29:00.604 INFO:tasks.workunit.client.0.smithi022.stdout:0/431: creat d10/d11/d27/d77/f8e x:0 0 0 2024-02-16T19:29:00.606 INFO:tasks.workunit.client.0.smithi022.stdout:1/427: creat d6/d8/d22/d16/d17/d63/d6f/f7c x:0 0 0 2024-02-16T19:29:00.606 INFO:tasks.workunit.client.1.smithi117.stdout:7/284: dread d4/d1c/d39/d46/f57 [0,4194304] 0 2024-02-16T19:29:00.607 INFO:tasks.workunit.client.1.smithi117.stdout:4/294: dread d0/f11 [0,4194304] 0 2024-02-16T19:29:00.607 INFO:tasks.workunit.client.1.smithi117.stdout:7/285: fdatasync d4/d13/d63/f6b 0 2024-02-16T19:29:00.612 INFO:tasks.workunit.client.1.smithi117.stdout:7/286: dwrite d4/d13/f35 [0,4194304] 0 2024-02-16T19:29:00.612 INFO:tasks.workunit.client.0.smithi022.stdout:6/337: creat d1/d8/da/d17/d48/f7c x:0 0 0 2024-02-16T19:29:00.614 INFO:tasks.workunit.client.0.smithi022.stdout:7/311: mkdir d9/d5d 0 2024-02-16T19:29:00.615 INFO:tasks.workunit.client.0.smithi022.stdout:7/312: write d9/d1b/f57 [33942,27905] 0 2024-02-16T19:29:00.615 INFO:tasks.workunit.client.1.smithi117.stdout:2/232: mknod d0/d3/d22/c4a 0 2024-02-16T19:29:00.619 INFO:tasks.workunit.client.1.smithi117.stdout:3/350: read d1/d31/f36 [3409013,5412] 0 2024-02-16T19:29:00.620 INFO:tasks.workunit.client.1.smithi117.stdout:7/287: dwrite d4/d4a/f50 [4194304,4194304] 0 2024-02-16T19:29:00.623 INFO:tasks.workunit.client.0.smithi022.stdout:6/338: creat d1/d8/da/d17/d53/f7d x:0 0 0 2024-02-16T19:29:00.624 INFO:tasks.workunit.client.1.smithi117.stdout:8/284: getdents d4/d7/d37/d50 0 2024-02-16T19:29:00.625 INFO:tasks.workunit.client.0.smithi022.stdout:6/339: readlink d1/d8/da/d17/l73 0 2024-02-16T19:29:00.627 INFO:tasks.workunit.client.0.smithi022.stdout:5/338: creat dd/d1f/d48/d21/d2f/d40/d4e/f6d x:0 0 0 2024-02-16T19:29:00.629 INFO:tasks.workunit.client.1.smithi117.stdout:4/295: mknod d0/da/d21/d3d/d28/d2f/d37/c5f 0 2024-02-16T19:29:00.629 INFO:tasks.workunit.client.0.smithi022.stdout:1/428: dwrite d6/d8/d22/d3a/d3e/f76 [0,4194304] 0 2024-02-16T19:29:00.629 INFO:tasks.workunit.client.1.smithi117.stdout:4/296: fsync d0/da/d21/f40 0 2024-02-16T19:29:00.630 INFO:tasks.workunit.client.1.smithi117.stdout:6/314: chown d5/d1d/d49 86260 1 2024-02-16T19:29:00.631 INFO:tasks.workunit.client.1.smithi117.stdout:6/315: dread - d5/d1d/d28/d32/d37/f58 zero size 2024-02-16T19:29:00.644 INFO:tasks.workunit.client.0.smithi022.stdout:3/332: dwrite d7/d12/d18/dd/d24/d58/d70/f81 [0,4194304] 0 2024-02-16T19:29:00.646 INFO:tasks.workunit.client.1.smithi117.stdout:0/348: rename d0/d9/d14/d23/d2f to d0/d9/d22/d27/d83 0 2024-02-16T19:29:00.646 INFO:tasks.workunit.client.0.smithi022.stdout:8/327: dwrite d3/d12/f48 [0,4194304] 0 2024-02-16T19:29:00.648 INFO:tasks.workunit.client.0.smithi022.stdout:5/339: creat dd/d1f/d48/d21/d2f/d52/f6e x:0 0 0 2024-02-16T19:29:00.649 INFO:tasks.workunit.client.1.smithi117.stdout:7/288: mknod d4/d1c/d39/d46/d47/c72 0 2024-02-16T19:29:00.651 INFO:tasks.workunit.client.0.smithi022.stdout:4/299: rename d4/d6/dd/f36 to d4/d6/dd/f60 0 2024-02-16T19:29:00.654 INFO:tasks.workunit.client.0.smithi022.stdout:6/340: write d1/d8/da/d17/d48/d51/f59 [2677450,118615] 0 2024-02-16T19:29:00.656 INFO:tasks.workunit.client.1.smithi117.stdout:1/234: creat d2/f4d x:0 0 0 2024-02-16T19:29:00.658 INFO:tasks.workunit.client.0.smithi022.stdout:6/341: write d1/d8/da/d17/f1d [1360641,98921] 0 2024-02-16T19:29:00.660 INFO:tasks.workunit.client.0.smithi022.stdout:7/313: dwrite d9/da/d1f/f22 [0,4194304] 0 2024-02-16T19:29:00.662 INFO:tasks.workunit.client.0.smithi022.stdout:7/314: chown d9/da/d2c/d46 432484480 1 2024-02-16T19:29:00.662 INFO:tasks.workunit.client.0.smithi022.stdout:6/342: truncate d1/d8/da/d17/d53/f7d 872173 0 2024-02-16T19:29:00.663 INFO:tasks.workunit.client.0.smithi022.stdout:7/315: stat d9/d1b/c36 0 2024-02-16T19:29:00.665 INFO:tasks.workunit.client.0.smithi022.stdout:6/343: read d1/d8/d3c/d4d/f4b [108192,73933] 0 2024-02-16T19:29:00.666 INFO:tasks.workunit.client.1.smithi117.stdout:4/297: dwrite d0/da/d21/f3f [0,4194304] 0 2024-02-16T19:29:00.701 INFO:tasks.workunit.client.0.smithi022.stdout:8/328: unlink d3/d12/c1b 0 2024-02-16T19:29:00.704 INFO:tasks.workunit.client.1.smithi117.stdout:9/247: creat dd/d23/d2e/f4e x:0 0 0 2024-02-16T19:29:00.709 INFO:tasks.workunit.client.1.smithi117.stdout:8/285: dwrite d4/d12/f26 [4194304,4194304] 0 2024-02-16T19:29:00.710 INFO:tasks.workunit.client.1.smithi117.stdout:9/248: dwrite dd/d2d/f4d [0,4194304] 0 2024-02-16T19:29:00.710 INFO:tasks.workunit.client.1.smithi117.stdout:8/286: chown d4/d12/d2a/c39 999153245 1 2024-02-16T19:29:00.712 INFO:tasks.workunit.client.1.smithi117.stdout:8/287: write d4/d7/d37/d3a/d43/d44/f4b [3620723,16071] 0 2024-02-16T19:29:00.754 INFO:tasks.workunit.client.1.smithi117.stdout:2/233: getdents d0/d3b 0 2024-02-16T19:29:00.755 INFO:tasks.workunit.client.1.smithi117.stdout:2/234: chown d0/d13 174847 1 2024-02-16T19:29:00.756 INFO:tasks.workunit.client.1.smithi117.stdout:2/235: write d0/d13/f26 [713115,32707] 0 2024-02-16T19:29:00.756 INFO:tasks.workunit.client.1.smithi117.stdout:5/393: link d2/d9/d3c/d24/d50/c75 d2/d4c/d4f/c78 0 2024-02-16T19:29:00.757 INFO:tasks.workunit.client.1.smithi117.stdout:0/349: rmdir d0/d9/d22 39 2024-02-16T19:29:00.757 INFO:tasks.workunit.client.1.smithi117.stdout:3/351: sync 2024-02-16T19:29:00.759 INFO:tasks.workunit.client.1.smithi117.stdout:3/352: dread - d1/dd/d3b/f63 zero size 2024-02-16T19:29:00.759 INFO:tasks.workunit.client.0.smithi022.stdout:1/429: creat d6/d8/f7d x:0 0 0 2024-02-16T19:29:00.760 INFO:tasks.workunit.client.0.smithi022.stdout:1/430: write d6/d66/f71 [1046100,87902] 0 2024-02-16T19:29:00.760 INFO:tasks.workunit.client.1.smithi117.stdout:1/235: creat d2/d1d/d1f/f4e x:0 0 0 2024-02-16T19:29:00.761 INFO:tasks.workunit.client.1.smithi117.stdout:1/236: chown d2/d4/f17 1 1 2024-02-16T19:29:00.762 INFO:tasks.workunit.client.0.smithi022.stdout:1/431: dread - d6/d8/d22/d16/d17/d26/f46 zero size 2024-02-16T19:29:00.762 INFO:tasks.workunit.client.0.smithi022.stdout:1/432: chown d6/d8/d22/d16/d4f 1373626855 1 2024-02-16T19:29:00.763 INFO:tasks.workunit.client.1.smithi117.stdout:2/236: dwrite d0/d29/d2f/f44 [0,4194304] 0 2024-02-16T19:29:00.763 INFO:tasks.workunit.client.1.smithi117.stdout:2/237: write d0/f25 [818435,1910] 0 2024-02-16T19:29:00.765 INFO:tasks.workunit.client.1.smithi117.stdout:4/298: rmdir d0/da/d4a 39 2024-02-16T19:29:00.766 INFO:tasks.workunit.client.1.smithi117.stdout:4/299: dread - d0/da/d21/d3d/f5e zero size 2024-02-16T19:29:00.768 INFO:tasks.workunit.client.0.smithi022.stdout:8/329: creat d3/d12/d2b/d54/f76 x:0 0 0 2024-02-16T19:29:00.770 INFO:tasks.workunit.client.1.smithi117.stdout:4/300: dread d0/da/d21/d3d/f29 [0,4194304] 0 2024-02-16T19:29:00.771 INFO:tasks.workunit.client.0.smithi022.stdout:3/333: unlink d7/d12/d18/dd/d31/d41/l80 0 2024-02-16T19:29:00.771 INFO:tasks.workunit.client.1.smithi117.stdout:4/301: chown d0/da/d21/d3d/d28/f51 242262236 1 2024-02-16T19:29:00.773 INFO:tasks.workunit.client.1.smithi117.stdout:4/302: write d0/d6/d48/f4c [642509,113686] 0 2024-02-16T19:29:00.774 INFO:tasks.workunit.client.0.smithi022.stdout:5/340: mknod dd/d1f/d48/d21/d2f/d40/d4e/c6f 0 2024-02-16T19:29:00.775 INFO:tasks.workunit.client.0.smithi022.stdout:5/341: chown l2 518666532 1 2024-02-16T19:29:00.775 INFO:tasks.workunit.client.0.smithi022.stdout:8/330: dwrite d3/d12/d6a/f6e [0,4194304] 0 2024-02-16T19:29:00.778 INFO:tasks.workunit.client.1.smithi117.stdout:8/288: mkdir d4/d7/d37/d3a/d43/d44/d5f 0 2024-02-16T19:29:00.778 INFO:tasks.workunit.client.0.smithi022.stdout:4/300: mkdir d4/d6/dd/d61 0 2024-02-16T19:29:00.779 INFO:tasks.workunit.client.0.smithi022.stdout:6/344: write d1/d24/f36 [1478422,15378] 0 2024-02-16T19:29:00.780 INFO:tasks.workunit.client.0.smithi022.stdout:7/316: fsync d9/da/d1f/d44/f49 0 2024-02-16T19:29:00.782 INFO:tasks.workunit.client.0.smithi022.stdout:7/317: chown d9/da/d3b 506 1 2024-02-16T19:29:00.786 INFO:tasks.workunit.client.0.smithi022.stdout:8/331: dwrite d3/d6/f63 [0,4194304] 0 2024-02-16T19:29:00.788 INFO:tasks.workunit.client.0.smithi022.stdout:8/332: fsync d3/d12/f71 0 2024-02-16T19:29:00.791 INFO:tasks.workunit.client.0.smithi022.stdout:8/333: dread d3/d25/d33/f41 [0,4194304] 0 2024-02-16T19:29:00.793 INFO:tasks.workunit.client.1.smithi117.stdout:7/289: sync 2024-02-16T19:29:00.794 INFO:tasks.workunit.client.1.smithi117.stdout:7/290: write d4/d1c/f3c [9570684,13140] 0 2024-02-16T19:29:00.795 INFO:tasks.workunit.client.0.smithi022.stdout:7/318: dread d9/da/d1f/f18 [0,4194304] 0 2024-02-16T19:29:00.795 INFO:tasks.workunit.client.0.smithi022.stdout:4/301: dwrite d4/d6/dd/df/d10/f25 [0,4194304] 0 2024-02-16T19:29:00.795 INFO:tasks.workunit.client.1.smithi117.stdout:7/291: fsync d4/d1c/d39/f5b 0 2024-02-16T19:29:00.798 INFO:tasks.workunit.client.0.smithi022.stdout:3/334: truncate d7/d12/d18/d4b/f6f 869535 0 2024-02-16T19:29:00.802 INFO:tasks.workunit.client.1.smithi117.stdout:0/350: mknod d0/d9/d14/d23/d33/c84 0 2024-02-16T19:29:00.803 INFO:tasks.workunit.client.0.smithi022.stdout:6/345: write d1/d8/d3c/f47 [703957,56880] 0 2024-02-16T19:29:00.804 INFO:tasks.workunit.client.1.smithi117.stdout:3/353: creat d1/dd/d32/f75 x:0 0 0 2024-02-16T19:29:00.805 INFO:tasks.workunit.client.0.smithi022.stdout:7/319: read d9/d1b/f57 [26883,56840] 0 2024-02-16T19:29:00.806 INFO:tasks.workunit.client.1.smithi117.stdout:1/237: creat d2/d1d/f4f x:0 0 0 2024-02-16T19:29:00.809 INFO:tasks.workunit.client.1.smithi117.stdout:2/238: mknod d0/d13/c4b 0 2024-02-16T19:29:00.811 INFO:tasks.workunit.client.0.smithi022.stdout:4/302: rename d4/d6/d3a/c4a to d4/d6/dd/df/d10/d24/c62 0 2024-02-16T19:29:00.812 INFO:tasks.workunit.client.1.smithi117.stdout:6/316: mknod d5/d4b/c68 0 2024-02-16T19:29:00.813 INFO:tasks.workunit.client.0.smithi022.stdout:6/346: dwrite d1/d24/f38 [0,4194304] 0 2024-02-16T19:29:00.813 INFO:tasks.workunit.client.1.smithi117.stdout:2/239: dread d0/d3/d22/f28 [0,4194304] 0 2024-02-16T19:29:00.816 INFO:tasks.workunit.client.0.smithi022.stdout:7/320: dwrite d9/da/f29 [0,4194304] 0 2024-02-16T19:29:00.816 INFO:tasks.workunit.client.0.smithi022.stdout:4/303: chown d4/d6/dd/df/d10/c13 11276 1 2024-02-16T19:29:00.816 INFO:tasks.workunit.client.0.smithi022.stdout:5/342: link dd/d1f/d48/d21/d2f/d40/d49/f4c dd/d1f/d48/d21/d2f/d52/d5c/d61/d6a/f70 0 2024-02-16T19:29:00.817 INFO:tasks.workunit.client.0.smithi022.stdout:3/335: symlink d7/d12/d18/dd/d24/d58/d70/l86 0 2024-02-16T19:29:00.818 INFO:tasks.workunit.client.1.smithi117.stdout:2/240: dwrite d0/f1e [0,4194304] 0 2024-02-16T19:29:00.819 INFO:tasks.workunit.client.0.smithi022.stdout:4/304: write d4/d6/dd/df/d10/f4f [875105,6833] 0 2024-02-16T19:29:00.821 INFO:tasks.workunit.client.1.smithi117.stdout:4/303: symlink d0/da/d21/d3d/d28/d2f/l60 0 2024-02-16T19:29:00.822 INFO:tasks.workunit.client.1.smithi117.stdout:4/304: chown d0/da/d21/d3d/d28 807 1 2024-02-16T19:29:00.823 INFO:tasks.workunit.client.0.smithi022.stdout:1/433: dread d6/d8/d22/f28 [0,4194304] 0 2024-02-16T19:29:00.823 INFO:tasks.workunit.client.1.smithi117.stdout:4/305: chown d0/da/d21/d3d/d28/l5a 950134 1 2024-02-16T19:29:00.825 INFO:tasks.workunit.client.1.smithi117.stdout:4/306: readlink d0/da/d21/d3d/d28/d2f/d37/l4d 0 2024-02-16T19:29:00.826 INFO:tasks.workunit.client.1.smithi117.stdout:7/292: creat d4/d1c/d39/d46/d47/f73 x:0 0 0 2024-02-16T19:29:00.826 INFO:tasks.workunit.client.0.smithi022.stdout:5/343: dwrite dd/f3c [0,4194304] 0 2024-02-16T19:29:00.827 INFO:tasks.workunit.client.1.smithi117.stdout:4/307: dread - d0/da/d21/d3d/d2d/f41 zero size 2024-02-16T19:29:00.829 INFO:tasks.workunit.client.0.smithi022.stdout:1/434: read d6/d8/d22/d47/f4e [1786640,9279] 0 2024-02-16T19:29:00.831 INFO:tasks.workunit.client.0.smithi022.stdout:6/347: rename d1/d8/da/d17/f1d to d1/d8/d3c/d4d/f7e 0 2024-02-16T19:29:00.831 INFO:tasks.workunit.client.0.smithi022.stdout:3/336: rmdir d7/d12 39 2024-02-16T19:29:00.831 INFO:tasks.workunit.client.0.smithi022.stdout:1/435: dread - d6/d8/d22/d47/d54/f65 zero size 2024-02-16T19:29:00.832 INFO:tasks.workunit.client.1.smithi117.stdout:3/354: symlink d1/d1b/d1e/d29/l76 0 2024-02-16T19:29:00.834 INFO:tasks.workunit.client.0.smithi022.stdout:7/321: creat d9/da/d1f/d40/f5e x:0 0 0 2024-02-16T19:29:00.835 INFO:tasks.workunit.client.1.smithi117.stdout:6/317: mkdir d5/d4b/d20/d69 0 2024-02-16T19:29:00.836 INFO:tasks.workunit.client.0.smithi022.stdout:3/337: truncate d7/d21/f52 299425 0 2024-02-16T19:29:00.837 INFO:tasks.workunit.client.0.smithi022.stdout:4/305: dwrite d4/d6/d14/f1d [0,4194304] 0 2024-02-16T19:29:00.839 INFO:tasks.workunit.client.1.smithi117.stdout:3/355: dwrite d1/d1b/d1e/f51 [0,4194304] 0 2024-02-16T19:29:00.842 INFO:tasks.workunit.client.0.smithi022.stdout:1/436: mkdir d6/d8/d22/d67/d7e 0 2024-02-16T19:29:00.842 INFO:tasks.workunit.client.0.smithi022.stdout:6/348: mkdir d1/d8/da/d17/d53/d7f 0 2024-02-16T19:29:00.842 INFO:tasks.workunit.client.1.smithi117.stdout:6/318: dwrite d5/d1d/f50 [0,4194304] 0 2024-02-16T19:29:00.843 INFO:tasks.workunit.client.0.smithi022.stdout:1/437: chown d6/d8/f7d 111585764 1 2024-02-16T19:29:00.846 INFO:tasks.workunit.client.0.smithi022.stdout:6/349: chown d1/d8/da/d29/d61/l6d 3 1 2024-02-16T19:29:00.846 INFO:tasks.workunit.client.0.smithi022.stdout:6/350: chown d1/d8/da/d17/l2a 1864450 1 2024-02-16T19:29:00.847 INFO:tasks.workunit.client.1.smithi117.stdout:2/241: dwrite d0/f19 [0,4194304] 0 2024-02-16T19:29:00.848 INFO:tasks.workunit.client.0.smithi022.stdout:6/351: dread - d1/d8/da/d29/d61/d57/f60 zero size 2024-02-16T19:29:00.850 INFO:tasks.workunit.client.1.smithi117.stdout:7/293: symlink d4/d1c/d39/d46/d47/l74 0 2024-02-16T19:29:00.851 INFO:tasks.workunit.client.1.smithi117.stdout:4/308: symlink d0/d6/l61 0 2024-02-16T19:29:00.851 INFO:tasks.workunit.client.1.smithi117.stdout:4/309: fdatasync d0/f5c 0 2024-02-16T19:29:00.852 INFO:tasks.workunit.client.0.smithi022.stdout:7/322: creat d9/d1b/d23/f5f x:0 0 0 2024-02-16T19:29:00.857 INFO:tasks.workunit.client.0.smithi022.stdout:4/306: read d4/d6/d14/f2e [302554,130744] 0 2024-02-16T19:29:00.861 INFO:tasks.workunit.client.1.smithi117.stdout:9/249: rename dd/d23/f2b to dd/d45/f4f 0 2024-02-16T19:29:00.863 INFO:tasks.workunit.client.0.smithi022.stdout:5/344: truncate dd/f11 5966527 0 2024-02-16T19:29:00.864 INFO:tasks.workunit.client.1.smithi117.stdout:6/319: fsync d5/d1d/d28/d32/f5b 0 2024-02-16T19:29:00.864 INFO:tasks.workunit.client.0.smithi022.stdout:5/345: dread - dd/d1f/d48/d21/d2f/d40/d4e/f6d zero size 2024-02-16T19:29:00.868 INFO:tasks.workunit.client.1.smithi117.stdout:5/394: link d2/d9/d3c/d42/l66 d2/d3/l79 0 2024-02-16T19:29:00.871 INFO:tasks.workunit.client.1.smithi117.stdout:0/351: mknod d0/d9/d22/d27/c85 0 2024-02-16T19:29:00.875 INFO:tasks.workunit.client.1.smithi117.stdout:4/310: creat d0/da/d21/d3d/d28/d2f/f62 x:0 0 0 2024-02-16T19:29:00.876 INFO:tasks.workunit.client.1.smithi117.stdout:4/311: read d0/da/d21/d3d/d28/d2f/f31 [459600,26357] 0 2024-02-16T19:29:00.876 INFO:tasks.workunit.client.1.smithi117.stdout:8/289: write d4/d12/f1a [3517407,78094] 0 2024-02-16T19:29:00.877 INFO:tasks.workunit.client.0.smithi022.stdout:4/307: mkdir d4/d6/dd/d31/d4e/d54/d5a/d63 0 2024-02-16T19:29:00.880 INFO:tasks.workunit.client.1.smithi117.stdout:1/238: rename d2/d4/fa to d2/d18/d20/d49/f50 0 2024-02-16T19:29:00.881 INFO:tasks.workunit.client.0.smithi022.stdout:6/352: creat d1/d8/d3c/d77/f80 x:0 0 0 2024-02-16T19:29:00.882 INFO:tasks.workunit.client.0.smithi022.stdout:3/338: dread d7/d12/d18/dd/d24/f3e [0,4194304] 0 2024-02-16T19:29:00.883 INFO:tasks.workunit.client.0.smithi022.stdout:3/339: write d7/d12/d18/d4b/f4d [859237,27249] 0 2024-02-16T19:29:00.884 INFO:tasks.workunit.client.1.smithi117.stdout:8/290: dwrite d4/d12/f26 [4194304,4194304] 0 2024-02-16T19:29:00.884 INFO:tasks.workunit.client.1.smithi117.stdout:4/312: dwrite d0/da/d21/f22 [4194304,4194304] 0 2024-02-16T19:29:00.884 INFO:tasks.workunit.client.0.smithi022.stdout:5/346: mkdir dd/d1f/d48/d21/d2f/d40/d71 0 2024-02-16T19:29:00.889 INFO:tasks.workunit.client.1.smithi117.stdout:5/395: write d2/d4c/d69/f72 [168687,7637] 0 2024-02-16T19:29:00.891 INFO:tasks.workunit.client.1.smithi117.stdout:0/352: creat d0/d30/f86 x:0 0 0 2024-02-16T19:29:00.892 INFO:tasks.workunit.client.1.smithi117.stdout:5/396: dread d2/d3/f35 [0,4194304] 0 2024-02-16T19:29:00.893 INFO:tasks.workunit.client.0.smithi022.stdout:6/353: chown d1/d24/d5b/f63 7432 1 2024-02-16T19:29:00.894 INFO:tasks.workunit.client.1.smithi117.stdout:4/313: dwrite d0/d6/d48/f5d [0,4194304] 0 2024-02-16T19:29:00.894 INFO:tasks.workunit.client.0.smithi022.stdout:6/354: fdatasync d1/d24/f50 0 2024-02-16T19:29:00.897 INFO:tasks.workunit.client.0.smithi022.stdout:3/340: symlink d7/d12/d18/dd/d24/d29/l87 0 2024-02-16T19:29:00.900 INFO:tasks.workunit.client.1.smithi117.stdout:3/356: rename d1/dd/d32/f73 to d1/d30/f77 0 2024-02-16T19:29:00.902 INFO:tasks.workunit.client.0.smithi022.stdout:5/347: mknod dd/d1f/d2e/d35/c72 0 2024-02-16T19:29:00.903 INFO:tasks.workunit.client.0.smithi022.stdout:5/348: fsync dd/d1f/d48/f1b 0 2024-02-16T19:29:00.904 INFO:tasks.workunit.client.0.smithi022.stdout:5/349: dread - dd/d1f/d48/d21/d2f/d52/f65 zero size 2024-02-16T19:29:00.906 INFO:tasks.workunit.client.0.smithi022.stdout:7/323: rename d9/da/d1f/d40/f4b to d9/da/f60 0 2024-02-16T19:29:00.908 INFO:tasks.workunit.client.1.smithi117.stdout:9/250: creat dd/d47/d48/f50 x:0 0 0 2024-02-16T19:29:00.910 INFO:tasks.workunit.client.1.smithi117.stdout:8/291: unlink d4/d12/c57 0 2024-02-16T19:29:00.912 INFO:tasks.workunit.client.0.smithi022.stdout:6/355: mkdir d1/d8/d81 0 2024-02-16T19:29:00.913 INFO:tasks.workunit.client.1.smithi117.stdout:2/242: link d0/d3/d22/f3c d0/f4c 0 2024-02-16T19:29:00.914 INFO:tasks.workunit.client.1.smithi117.stdout:2/243: chown d0/d3b 21 1 2024-02-16T19:29:00.916 INFO:tasks.workunit.client.0.smithi022.stdout:1/438: truncate d6/d8/d22/d16/f41 2515711 0 2024-02-16T19:29:00.919 INFO:tasks.workunit.client.1.smithi117.stdout:0/353: symlink d0/d30/d74/l87 0 2024-02-16T19:29:00.921 INFO:tasks.workunit.client.0.smithi022.stdout:5/350: mkdir dd/d1f/d48/d21/d2f/d46/d73 0 2024-02-16T19:29:00.922 INFO:tasks.workunit.client.1.smithi117.stdout:5/397: mknod d2/d4c/d69/c7a 0 2024-02-16T19:29:00.923 INFO:tasks.workunit.client.1.smithi117.stdout:5/398: write d2/d3/d21/d2f/d30/f3e [501067,110662] 0 2024-02-16T19:29:00.924 INFO:tasks.workunit.client.0.smithi022.stdout:7/324: rename d9/d1b/d23/f5f to d9/da/d1f/d40/f61 0 2024-02-16T19:29:00.926 INFO:tasks.workunit.client.1.smithi117.stdout:4/314: creat d0/da/d21/d3d/d28/f63 x:0 0 0 2024-02-16T19:29:00.927 INFO:tasks.workunit.client.0.smithi022.stdout:6/356: mknod d1/c82 0 2024-02-16T19:29:00.929 INFO:tasks.workunit.client.1.smithi117.stdout:7/294: rename d4/d4a/f4c to d4/d1c/d39/d46/f75 0 2024-02-16T19:29:00.930 INFO:tasks.workunit.client.1.smithi117.stdout:7/295: chown d4/d13/d63/l69 0 1 2024-02-16T19:29:00.930 INFO:tasks.workunit.client.1.smithi117.stdout:7/296: readlink d4/d1c/d39/l45 0 2024-02-16T19:29:00.931 INFO:tasks.workunit.client.1.smithi117.stdout:3/357: mknod d1/d1b/d1e/d5e/c78 0 2024-02-16T19:29:00.933 INFO:tasks.workunit.client.1.smithi117.stdout:0/354: dread d0/f1a [0,4194304] 0 2024-02-16T19:29:00.933 INFO:tasks.workunit.client.1.smithi117.stdout:1/239: write d2/d4/f9 [3539596,84344] 0 2024-02-16T19:29:00.936 INFO:tasks.workunit.client.1.smithi117.stdout:2/244: mkdir d0/d13/d4d 0 2024-02-16T19:29:00.937 INFO:tasks.workunit.client.0.smithi022.stdout:4/308: truncate d4/d38/f3d 1742768 0 2024-02-16T19:29:00.937 INFO:tasks.workunit.client.1.smithi117.stdout:2/245: chown d0/d13/f43 173866 1 2024-02-16T19:29:00.939 INFO:tasks.workunit.client.1.smithi117.stdout:5/399: dread d2/d9/f40 [0,4194304] 0 2024-02-16T19:29:00.940 INFO:tasks.workunit.client.1.smithi117.stdout:2/246: dread d0/f1e [0,4194304] 0 2024-02-16T19:29:00.941 INFO:tasks.workunit.client.1.smithi117.stdout:2/247: chown d0/d3/d10/f16 1503241 1 2024-02-16T19:29:00.942 INFO:tasks.workunit.client.1.smithi117.stdout:2/248: write d0/d3/d22/d37/f48 [392098,74343] 0 2024-02-16T19:29:00.944 INFO:tasks.workunit.client.0.smithi022.stdout:4/309: dwrite d4/d6/dd/d31/f34 [0,4194304] 0 2024-02-16T19:29:00.944 INFO:tasks.workunit.client.0.smithi022.stdout:4/310: stat d4/d38 0 2024-02-16T19:29:00.946 INFO:tasks.workunit.client.0.smithi022.stdout:3/341: creat d7/d12/d18/f88 x:0 0 0 2024-02-16T19:29:00.947 INFO:tasks.workunit.client.1.smithi117.stdout:4/315: truncate d0/d6/f55 4389491 0 2024-02-16T19:29:00.947 INFO:tasks.workunit.client.0.smithi022.stdout:1/439: creat d6/d8/d22/d67/d7e/f7f x:0 0 0 2024-02-16T19:29:00.950 INFO:tasks.workunit.client.0.smithi022.stdout:4/311: dread d4/d6/f16 [0,4194304] 0 2024-02-16T19:29:00.951 INFO:tasks.workunit.client.1.smithi117.stdout:7/297: symlink d4/d1c/d24/l76 0 2024-02-16T19:29:00.952 INFO:tasks.workunit.client.1.smithi117.stdout:7/298: chown d4/d1c/d39/d46/d47/c4e 6 1 2024-02-16T19:29:00.956 INFO:tasks.workunit.client.0.smithi022.stdout:6/357: symlink d1/d8/da/d17/d53/d7f/l83 0 2024-02-16T19:29:00.959 INFO:tasks.workunit.client.1.smithi117.stdout:6/320: sync 2024-02-16T19:29:00.960 INFO:tasks.workunit.client.1.smithi117.stdout:1/240: write d2/d4/f47 [1230278,128823] 0 2024-02-16T19:29:00.960 INFO:tasks.workunit.client.1.smithi117.stdout:1/241: fdatasync d2/d1d/d1f/f44 0 2024-02-16T19:29:00.961 INFO:tasks.workunit.client.1.smithi117.stdout:1/242: write d2/d4/f17 [8400162,21097] 0 2024-02-16T19:29:00.962 INFO:tasks.workunit.client.1.smithi117.stdout:1/243: readlink d2/d4/l28 0 2024-02-16T19:29:00.962 INFO:tasks.workunit.client.1.smithi117.stdout:1/244: stat d2/d1d/f4f 0 2024-02-16T19:29:00.964 INFO:tasks.workunit.client.0.smithi022.stdout:1/440: mkdir d6/d8/d22/d3a/d80 0 2024-02-16T19:29:00.967 INFO:tasks.workunit.client.1.smithi117.stdout:5/400: mknod d2/d9/c7b 0 2024-02-16T19:29:00.968 INFO:tasks.workunit.client.0.smithi022.stdout:5/351: creat dd/d1f/d48/f74 x:0 0 0 2024-02-16T19:29:00.970 INFO:tasks.workunit.client.0.smithi022.stdout:4/312: rmdir d4/d6/dd/df/d10/d24 39 2024-02-16T19:29:00.971 INFO:tasks.workunit.client.0.smithi022.stdout:4/313: readlink d4/d6/d14/l46 0 2024-02-16T19:29:00.974 INFO:tasks.workunit.client.0.smithi022.stdout:7/325: creat d9/da/f62 x:0 0 0 2024-02-16T19:29:00.974 INFO:tasks.workunit.client.0.smithi022.stdout:7/326: readlink d9/da/d1f/l1c 0 2024-02-16T19:29:00.975 INFO:tasks.workunit.client.1.smithi117.stdout:4/316: dread - d0/da/d4a/f59 zero size 2024-02-16T19:29:00.976 INFO:tasks.workunit.client.0.smithi022.stdout:6/358: creat d1/d8/d23/f84 x:0 0 0 2024-02-16T19:29:00.977 INFO:tasks.workunit.client.1.smithi117.stdout:7/299: creat d4/d1c/d39/d46/d47/f77 x:0 0 0 2024-02-16T19:29:00.980 INFO:tasks.workunit.client.1.smithi117.stdout:3/358: write d1/d1b/d1e/d29/f33 [4847510,70912] 0 2024-02-16T19:29:00.981 INFO:tasks.workunit.client.1.smithi117.stdout:3/359: chown d1/d1b/c6f 10 1 2024-02-16T19:29:00.984 INFO:tasks.workunit.client.0.smithi022.stdout:4/314: dread d4/d6/dd/df/f29 [0,4194304] 0 2024-02-16T19:29:00.987 INFO:tasks.workunit.client.1.smithi117.stdout:6/321: symlink d5/d1d/d28/l6a 0 2024-02-16T19:29:00.993 INFO:tasks.workunit.client.1.smithi117.stdout:5/401: creat d2/d9/f7c x:0 0 0 2024-02-16T19:29:00.993 INFO:tasks.workunit.client.0.smithi022.stdout:5/352: mknod dd/d1f/d48/d21/d5d/c75 0 2024-02-16T19:29:00.996 INFO:tasks.workunit.client.1.smithi117.stdout:4/317: rename d0/d6 to d0/da/d21/d3d/d28/d64 0 2024-02-16T19:29:00.999 INFO:tasks.workunit.client.0.smithi022.stdout:4/315: rename d4/d6/dd/d31 to d4/d6/d14/d64 0 2024-02-16T19:29:01.000 INFO:tasks.workunit.client.0.smithi022.stdout:4/316: chown d4/d6/dd/df/d10/f50 0 1 2024-02-16T19:29:01.002 INFO:tasks.workunit.client.1.smithi117.stdout:9/251: getdents dd/d23/d2e 0 2024-02-16T19:29:01.003 INFO:tasks.workunit.client.0.smithi022.stdout:7/327: symlink d9/da/d1f/d44/d5b/l63 0 2024-02-16T19:29:01.004 INFO:tasks.workunit.client.1.smithi117.stdout:3/360: creat d1/d1b/d1e/d37/f79 x:0 0 0 2024-02-16T19:29:01.005 INFO:tasks.workunit.client.0.smithi022.stdout:1/441: dwrite d6/d8/f2d [0,4194304] 0 2024-02-16T19:29:01.005 INFO:tasks.workunit.client.0.smithi022.stdout:7/328: chown d9/l3e 2380158 1 2024-02-16T19:29:01.005 INFO:tasks.workunit.client.0.smithi022.stdout:6/359: getdents d1/d8/da/d7b 0 2024-02-16T19:29:01.008 INFO:tasks.workunit.client.1.smithi117.stdout:9/252: dwrite dd/d23/f25 [0,4194304] 0 2024-02-16T19:29:01.009 INFO:tasks.workunit.client.1.smithi117.stdout:9/253: chown dd/d24/f39 27 1 2024-02-16T19:29:01.010 INFO:tasks.workunit.client.0.smithi022.stdout:4/317: dwrite d4/d6/dd/df/d10/f50 [0,4194304] 0 2024-02-16T19:29:01.020 INFO:tasks.workunit.client.1.smithi117.stdout:7/300: rename d4/d1c/d24/l31 to d4/d13/d63/l78 0 2024-02-16T19:29:01.022 INFO:tasks.workunit.client.0.smithi022.stdout:2/263: sync 2024-02-16T19:29:01.023 INFO:tasks.workunit.client.1.smithi117.stdout:0/355: getdents d0/d9/d14 0 2024-02-16T19:29:01.024 INFO:tasks.workunit.client.0.smithi022.stdout:2/264: chown d0/d2/d18/l1e 109139 1 2024-02-16T19:29:01.025 INFO:tasks.workunit.client.0.smithi022.stdout:7/329: mknod d9/c64 0 2024-02-16T19:29:01.026 INFO:tasks.workunit.client.0.smithi022.stdout:5/353: dwrite dd/d1f/d48/d21/f25 [0,4194304] 0 2024-02-16T19:29:01.027 INFO:tasks.workunit.client.0.smithi022.stdout:4/318: symlink d4/d6/d14/d64/d4e/d54/d5a/d63/l65 0 2024-02-16T19:29:01.029 INFO:tasks.workunit.client.0.smithi022.stdout:7/330: dread - d9/da/d1f/d40/f61 zero size 2024-02-16T19:29:01.029 INFO:tasks.workunit.client.1.smithi117.stdout:5/402: link d2/d9/f40 d2/d5d/f7d 0 2024-02-16T19:29:01.034 INFO:tasks.workunit.client.0.smithi022.stdout:7/331: read d9/f25 [181366,70197] 0 2024-02-16T19:29:01.035 INFO:tasks.workunit.client.0.smithi022.stdout:6/360: rename d1/d24 to d1/d8/da/d17/d85 0 2024-02-16T19:29:01.037 INFO:tasks.workunit.client.1.smithi117.stdout:4/318: chown d0/da/d21/d3d/d28/d64/d48 46164 1 2024-02-16T19:29:01.038 INFO:tasks.workunit.client.0.smithi022.stdout:4/319: dread d4/f40 [0,4194304] 0 2024-02-16T19:29:01.038 INFO:tasks.workunit.client.0.smithi022.stdout:5/354: dread dd/d1f/d48/f2d [0,4194304] 0 2024-02-16T19:29:01.038 INFO:tasks.workunit.client.0.smithi022.stdout:2/265: dwrite d0/d2/d18/d19/f2f [0,4194304] 0 2024-02-16T19:29:01.038 INFO:tasks.workunit.client.1.smithi117.stdout:3/361: rename d1/d1b/l2f to d1/d30/l7a 0 2024-02-16T19:29:01.039 INFO:tasks.workunit.client.1.smithi117.stdout:3/362: readlink d1/d1b/d1e/l4b 0 2024-02-16T19:29:01.039 INFO:tasks.workunit.client.1.smithi117.stdout:3/363: dread - d1/d1b/f5f zero size 2024-02-16T19:29:01.040 INFO:tasks.workunit.client.1.smithi117.stdout:0/356: dread - d0/d9/d22/d26/f32 zero size 2024-02-16T19:29:01.041 INFO:tasks.workunit.client.1.smithi117.stdout:0/357: chown d0/d9/d22/d27/f52 1042010 1 2024-02-16T19:29:01.042 INFO:tasks.workunit.client.1.smithi117.stdout:8/292: sync 2024-02-16T19:29:01.044 INFO:tasks.workunit.client.1.smithi117.stdout:9/254: getdents dd/d23/d2e 0 2024-02-16T19:29:01.045 INFO:tasks.workunit.client.1.smithi117.stdout:3/364: dwrite d1/f19 [0,4194304] 0 2024-02-16T19:29:01.045 INFO:tasks.workunit.client.1.smithi117.stdout:8/293: read d4/d12/f26 [1938947,33631] 0 2024-02-16T19:29:01.046 INFO:tasks.workunit.client.0.smithi022.stdout:7/332: mkdir d9/da/d2c/d65 0 2024-02-16T19:29:01.047 INFO:tasks.workunit.client.1.smithi117.stdout:9/255: write dd/f3e [494880,72093] 0 2024-02-16T19:29:01.047 INFO:tasks.workunit.client.1.smithi117.stdout:3/365: chown d1/d1b/f41 0 1 2024-02-16T19:29:01.048 INFO:tasks.workunit.client.0.smithi022.stdout:7/333: chown d9/d1b/c3f 2 1 2024-02-16T19:29:01.048 INFO:tasks.workunit.client.1.smithi117.stdout:3/366: fsync d1/d1b/f5f 0 2024-02-16T19:29:01.050 INFO:tasks.workunit.client.1.smithi117.stdout:3/367: chown d1/dd/l61 38 1 2024-02-16T19:29:01.052 INFO:tasks.workunit.client.1.smithi117.stdout:5/403: creat d2/d3/d21/d2f/d30/d38/d51/f7e x:0 0 0 2024-02-16T19:29:01.054 INFO:tasks.workunit.client.1.smithi117.stdout:8/294: dwrite d4/d7/d37/d50/f5b [0,4194304] 0 2024-02-16T19:29:01.056 INFO:tasks.workunit.client.1.smithi117.stdout:8/295: chown d4/d12/d2a/f48 1713 1 2024-02-16T19:29:01.056 INFO:tasks.workunit.client.1.smithi117.stdout:8/296: chown d4/d7/d37/d50/c5a 176820971 1 2024-02-16T19:29:01.060 INFO:tasks.workunit.client.0.smithi022.stdout:5/355: dwrite dd/f17 [0,4194304] 0 2024-02-16T19:29:01.060 INFO:tasks.workunit.client.1.smithi117.stdout:8/297: dread d4/d12/f26 [4194304,4194304] 0 2024-02-16T19:29:01.061 INFO:tasks.workunit.client.0.smithi022.stdout:5/356: readlink dd/d1f/d2e/l57 0 2024-02-16T19:29:01.066 INFO:tasks.workunit.client.0.smithi022.stdout:4/320: unlink d4/d6/d3a/c53 0 2024-02-16T19:29:01.069 INFO:tasks.workunit.client.0.smithi022.stdout:2/266: creat d0/d32/f58 x:0 0 0 2024-02-16T19:29:01.070 INFO:tasks.workunit.client.1.smithi117.stdout:9/256: mknod dd/d2d/c51 0 2024-02-16T19:29:01.071 INFO:tasks.workunit.client.1.smithi117.stdout:9/257: dread - dd/d23/d2e/f4e zero size 2024-02-16T19:29:01.071 INFO:tasks.workunit.client.1.smithi117.stdout:6/322: link d5/d1d/l57 d5/d4b/d20/d69/l6b 0 2024-02-16T19:29:01.075 INFO:tasks.workunit.client.1.smithi117.stdout:3/368: unlink d1/dd/d3b/f63 0 2024-02-16T19:29:01.076 INFO:tasks.workunit.client.0.smithi022.stdout:2/267: dwrite d0/f2d [0,4194304] 0 2024-02-16T19:29:01.077 INFO:tasks.workunit.client.1.smithi117.stdout:3/369: read d1/dd/d3b/f5a [285299,110309] 0 2024-02-16T19:29:01.077 INFO:tasks.workunit.client.0.smithi022.stdout:6/361: mknod d1/d8/da/d17/d85/c86 0 2024-02-16T19:29:01.078 INFO:tasks.workunit.client.1.smithi117.stdout:3/370: read d1/dd/d5c/f6b [27417,113183] 0 2024-02-16T19:29:01.078 INFO:tasks.workunit.client.0.smithi022.stdout:6/362: write d1/f18 [1114196,121630] 0 2024-02-16T19:29:01.079 INFO:tasks.workunit.client.1.smithi117.stdout:3/371: write d1/d1b/d1e/f22 [1338928,120444] 0 2024-02-16T19:29:01.081 INFO:tasks.workunit.client.1.smithi117.stdout:3/372: chown d1/d30/c39 1161 1 2024-02-16T19:29:01.082 INFO:tasks.workunit.client.0.smithi022.stdout:4/321: chown d4/d6/dd/f60 662457 1 2024-02-16T19:29:01.082 INFO:tasks.workunit.client.1.smithi117.stdout:3/373: write d1/d1b/d1e/d29/f33 [144789,129851] 0 2024-02-16T19:29:01.085 INFO:tasks.workunit.client.1.smithi117.stdout:4/319: rename d0/f11 to d0/da/d21/f65 0 2024-02-16T19:29:01.085 INFO:tasks.workunit.client.1.smithi117.stdout:2/249: sync 2024-02-16T19:29:01.086 INFO:tasks.workunit.client.1.smithi117.stdout:1/245: sync 2024-02-16T19:29:01.086 INFO:tasks.workunit.client.1.smithi117.stdout:4/320: chown d0/da/d21/d3d/d28/d2f/d37 23 1 2024-02-16T19:29:01.086 INFO:tasks.workunit.client.1.smithi117.stdout:3/374: dread d1/d31/f36 [4194304,4194304] 0 2024-02-16T19:29:01.087 INFO:tasks.workunit.client.1.smithi117.stdout:0/358: unlink d0/d9/d22/d26/d55/c6c 0 2024-02-16T19:29:01.087 INFO:tasks.workunit.client.1.smithi117.stdout:8/298: dread d4/f52 [0,4194304] 0 2024-02-16T19:29:01.089 INFO:tasks.workunit.client.0.smithi022.stdout:4/322: dwrite d4/d6/dd/d22/d30/f44 [0,4194304] 0 2024-02-16T19:29:01.090 INFO:tasks.workunit.client.1.smithi117.stdout:0/359: dread - d0/d9/d22/d27/d4e/f61 zero size 2024-02-16T19:29:01.090 INFO:tasks.workunit.client.1.smithi117.stdout:0/360: fdatasync d0/f40 0 2024-02-16T19:29:01.091 INFO:tasks.workunit.client.0.smithi022.stdout:4/323: chown d4/d6/dd/df/d10/f4f 30 1 2024-02-16T19:29:01.093 INFO:tasks.workunit.client.1.smithi117.stdout:2/250: dwrite d0/d29/f2d [0,4194304] 0 2024-02-16T19:29:01.093 INFO:tasks.workunit.client.1.smithi117.stdout:2/251: fsync d0/d13/f15 0 2024-02-16T19:29:01.096 INFO:tasks.workunit.client.0.smithi022.stdout:4/324: dread d4/d6/dd/df/d10/f4f [0,4194304] 0 2024-02-16T19:29:01.101 INFO:tasks.workunit.client.1.smithi117.stdout:2/252: dwrite d0/d29/f2e [0,4194304] 0 2024-02-16T19:29:01.102 INFO:tasks.workunit.client.1.smithi117.stdout:9/258: creat dd/d23/d2e/f52 x:0 0 0 2024-02-16T19:29:01.102 INFO:tasks.workunit.client.1.smithi117.stdout:7/301: sync 2024-02-16T19:29:01.107 INFO:tasks.workunit.client.1.smithi117.stdout:2/253: dread d0/d13/f26 [0,4194304] 0 2024-02-16T19:29:01.109 INFO:tasks.workunit.client.0.smithi022.stdout:7/334: link d9/da/c30 d9/da/d3b/c66 0 2024-02-16T19:29:01.119 INFO:tasks.workunit.client.1.smithi117.stdout:5/404: creat d2/d4c/d4f/d67/f7f x:0 0 0 2024-02-16T19:29:01.120 INFO:tasks.workunit.client.0.smithi022.stdout:5/357: dwrite dd/d1f/d48/f26 [0,4194304] 0 2024-02-16T19:29:01.131 INFO:tasks.workunit.client.1.smithi117.stdout:8/299: creat d4/d1c/f60 x:0 0 0 2024-02-16T19:29:01.131 INFO:tasks.workunit.client.1.smithi117.stdout:3/375: dread d1/dd/f43 [0,4194304] 0 2024-02-16T19:29:01.132 INFO:tasks.workunit.client.0.smithi022.stdout:5/358: fsync dd/f1d 0 2024-02-16T19:29:01.133 INFO:tasks.workunit.client.1.smithi117.stdout:4/321: mkdir d0/da/d21/d3d/d28/d66 0 2024-02-16T19:29:01.135 INFO:tasks.workunit.client.0.smithi022.stdout:6/363: getdents d1/d8 0 2024-02-16T19:29:01.139 INFO:tasks.workunit.client.1.smithi117.stdout:0/361: creat d0/d9/d14/d1d/f88 x:0 0 0 2024-02-16T19:29:01.139 INFO:tasks.workunit.client.1.smithi117.stdout:0/362: chown d0/f1a 251792 1 2024-02-16T19:29:01.139 INFO:tasks.workunit.client.1.smithi117.stdout:0/363: read - d0/d30/f86 zero size 2024-02-16T19:29:01.140 INFO:tasks.workunit.client.0.smithi022.stdout:5/359: mkdir dd/d1f/d76 0 2024-02-16T19:29:01.140 INFO:tasks.workunit.client.1.smithi117.stdout:6/323: truncate d5/f9 507866 0 2024-02-16T19:29:01.141 INFO:tasks.workunit.client.1.smithi117.stdout:9/259: mknod dd/d45/d31/c53 0 2024-02-16T19:29:01.142 INFO:tasks.workunit.client.1.smithi117.stdout:7/302: mkdir d4/d1c/d24/d60/d79 0 2024-02-16T19:29:01.142 INFO:tasks.workunit.client.1.smithi117.stdout:0/364: dread d0/d9/d22/d27/d31/f58 [0,4194304] 0 2024-02-16T19:29:01.143 INFO:tasks.workunit.client.1.smithi117.stdout:2/254: write d0/d3/d22/f28 [522411,39248] 0 2024-02-16T19:29:01.145 INFO:tasks.workunit.client.1.smithi117.stdout:5/405: mknod d2/d3/d21/d49/d59/c80 0 2024-02-16T19:29:01.146 INFO:tasks.workunit.client.1.smithi117.stdout:9/260: dread dd/d23/f25 [0,4194304] 0 2024-02-16T19:29:01.147 INFO:tasks.workunit.client.1.smithi117.stdout:1/246: mkdir d2/d51 0 2024-02-16T19:29:01.148 INFO:tasks.workunit.client.0.smithi022.stdout:4/325: dwrite d4/d6/dd/d22/f59 [0,4194304] 0 2024-02-16T19:29:01.152 INFO:tasks.workunit.client.0.smithi022.stdout:7/335: dwrite d9/da/d2c/d2f/f17 [0,4194304] 0 2024-02-16T19:29:01.152 INFO:tasks.workunit.client.1.smithi117.stdout:3/376: write d1/d1b/d1e/d29/f33 [2055240,45429] 0 2024-02-16T19:29:01.154 INFO:tasks.workunit.client.1.smithi117.stdout:4/322: creat d0/da/d21/d3d/d28/d64/f67 x:0 0 0 2024-02-16T19:29:01.157 INFO:tasks.workunit.client.1.smithi117.stdout:3/377: dwrite d1/d1b/d1e/d5e/f62 [0,4194304] 0 2024-02-16T19:29:01.158 INFO:tasks.workunit.client.0.smithi022.stdout:4/326: mkdir d4/d6/d14/d64/d66 0 2024-02-16T19:29:01.163 INFO:tasks.workunit.client.0.smithi022.stdout:4/327: dread d4/d6/d14/f1d [0,4194304] 0 2024-02-16T19:29:01.164 INFO:tasks.workunit.client.0.smithi022.stdout:7/336: mkdir d9/da/d1f/d67 0 2024-02-16T19:29:01.165 INFO:tasks.workunit.client.0.smithi022.stdout:7/337: fsync d9/da/d1f/d40/f5e 0 2024-02-16T19:29:01.167 INFO:tasks.workunit.client.0.smithi022.stdout:7/338: chown d9/l19 13019 1 2024-02-16T19:29:01.169 INFO:tasks.workunit.client.0.smithi022.stdout:7/339: write d9/da/f3a [381942,71815] 0 2024-02-16T19:29:01.170 INFO:tasks.workunit.client.0.smithi022.stdout:7/340: dread - d9/da/d1f/d40/f61 zero size 2024-02-16T19:29:01.171 INFO:tasks.workunit.client.0.smithi022.stdout:7/341: dread - d9/da/d2c/d2f/f3d zero size 2024-02-16T19:29:01.172 INFO:tasks.workunit.client.0.smithi022.stdout:7/342: readlink d9/da/d1f/l38 0 2024-02-16T19:29:01.173 INFO:tasks.workunit.client.0.smithi022.stdout:7/343: chown d9/da/d2c/d2f/f5c 0 1 2024-02-16T19:29:01.174 INFO:tasks.workunit.client.0.smithi022.stdout:7/344: dread - d9/da/d2c/d2f/f5c zero size 2024-02-16T19:29:01.175 INFO:tasks.workunit.client.1.smithi117.stdout:8/300: write d4/d12/d31/f36 [992785,47072] 0 2024-02-16T19:29:01.176 INFO:tasks.workunit.client.1.smithi117.stdout:2/255: read d0/d13/f26 [189458,73523] 0 2024-02-16T19:29:01.176 INFO:tasks.workunit.client.1.smithi117.stdout:8/301: dread - d4/d1c/f60 zero size 2024-02-16T19:29:01.177 INFO:tasks.workunit.client.1.smithi117.stdout:8/302: chown d4/d7/f3d 1246452161 1 2024-02-16T19:29:01.179 INFO:tasks.workunit.client.1.smithi117.stdout:3/378: dread d1/d1b/d1e/f54 [0,4194304] 0 2024-02-16T19:29:01.204 INFO:tasks.workunit.client.0.smithi022.stdout:0/432: sync 2024-02-16T19:29:01.204 INFO:tasks.workunit.client.0.smithi022.stdout:8/334: sync 2024-02-16T19:29:01.204 INFO:tasks.workunit.client.0.smithi022.stdout:9/365: sync 2024-02-16T19:29:01.206 INFO:tasks.workunit.client.0.smithi022.stdout:9/366: dread db/d14/d19/d1a/d67/f44 [0,4194304] 0 2024-02-16T19:29:01.262 INFO:tasks.workunit.client.1.smithi117.stdout:8/303: sync 2024-02-16T19:29:01.263 INFO:tasks.workunit.client.1.smithi117.stdout:8/304: read d4/d12/d2a/f48 [22168,41788] 0 2024-02-16T19:29:01.264 INFO:tasks.workunit.client.1.smithi117.stdout:8/305: write d4/d7/d37/d50/f5b [2055257,110092] 0 2024-02-16T19:29:01.268 INFO:tasks.workunit.client.1.smithi117.stdout:0/365: rename d0/d30/d74/c80 to d0/d30/c89 0 2024-02-16T19:29:01.271 INFO:tasks.workunit.client.1.smithi117.stdout:5/406: creat d2/d4c/d4f/f81 x:0 0 0 2024-02-16T19:29:01.272 INFO:tasks.workunit.client.0.smithi022.stdout:7/345: fsync d9/da/f60 0 2024-02-16T19:29:01.273 INFO:tasks.workunit.client.1.smithi117.stdout:9/261: symlink dd/d23/d2e/l54 0 2024-02-16T19:29:01.278 INFO:tasks.workunit.client.0.smithi022.stdout:4/328: write d4/d6/dd/d22/d30/f26 [2293643,30918] 0 2024-02-16T19:29:01.279 INFO:tasks.workunit.client.0.smithi022.stdout:4/329: fdatasync d4/d6/d14/d64/d4e/d54/d5a/f5f 0 2024-02-16T19:29:01.281 INFO:tasks.workunit.client.0.smithi022.stdout:7/346: rename d9/da/c24 to d9/d1b/c68 0 2024-02-16T19:29:01.283 INFO:tasks.workunit.client.0.smithi022.stdout:7/347: write d9/da/d1f/d44/f54 [233457,24928] 0 2024-02-16T19:29:01.283 INFO:tasks.workunit.client.1.smithi117.stdout:6/324: creat d5/d4b/d44/d52/f6c x:0 0 0 2024-02-16T19:29:01.284 INFO:tasks.workunit.client.1.smithi117.stdout:4/323: dread d0/da/d21/f27 [0,4194304] 0 2024-02-16T19:29:01.285 INFO:tasks.workunit.client.1.smithi117.stdout:4/324: truncate d0/da/f44 535094 0 2024-02-16T19:29:01.285 INFO:tasks.workunit.client.0.smithi022.stdout:0/433: mkdir d10/d15/d23/d8f 0 2024-02-16T19:29:01.288 INFO:tasks.workunit.client.1.smithi117.stdout:4/325: dread d0/da/d21/f3f [0,4194304] 0 2024-02-16T19:29:01.289 INFO:tasks.workunit.client.0.smithi022.stdout:4/330: symlink d4/d6/d14/d64/l67 0 2024-02-16T19:29:01.291 INFO:tasks.workunit.client.0.smithi022.stdout:7/348: creat d9/da/d2c/f69 x:0 0 0 2024-02-16T19:29:01.293 INFO:tasks.workunit.client.1.smithi117.stdout:3/379: creat d1/d1b/d20/f7b x:0 0 0 2024-02-16T19:29:01.295 INFO:tasks.workunit.client.0.smithi022.stdout:8/335: truncate d3/d25/d33/f35 3025452 0 2024-02-16T19:29:01.297 INFO:tasks.workunit.client.0.smithi022.stdout:9/367: link db/d14/d19/d1a/d67/f44 db/f84 0 2024-02-16T19:29:01.299 INFO:tasks.workunit.client.1.smithi117.stdout:3/380: dwrite d1/d1b/d1e/d5e/f62 [4194304,4194304] 0 2024-02-16T19:29:01.300 INFO:tasks.workunit.client.0.smithi022.stdout:7/349: rename d9/da/d2c/d2f/f17 to d9/da/d2c/d46/f6a 0 2024-02-16T19:29:01.301 INFO:tasks.workunit.client.1.smithi117.stdout:8/306: unlink d4/d7/cb 0 2024-02-16T19:29:01.302 INFO:tasks.workunit.client.1.smithi117.stdout:2/256: dwrite d0/d13/f26 [0,4194304] 0 2024-02-16T19:29:01.305 INFO:tasks.workunit.client.0.smithi022.stdout:8/336: mknod d3/d12/d6a/c77 0 2024-02-16T19:29:01.308 INFO:tasks.workunit.client.0.smithi022.stdout:7/350: dwrite d9/da/f3a [0,4194304] 0 2024-02-16T19:29:01.311 INFO:tasks.workunit.client.1.smithi117.stdout:9/262: creat dd/d47/d48/f55 x:0 0 0 2024-02-16T19:29:01.314 INFO:tasks.workunit.client.1.smithi117.stdout:5/407: dwrite d2/d3/d21/f33 [0,4194304] 0 2024-02-16T19:29:01.317 INFO:tasks.workunit.client.0.smithi022.stdout:0/434: truncate f7 1923793 0 2024-02-16T19:29:01.320 INFO:tasks.workunit.client.0.smithi022.stdout:9/368: symlink db/d14/d16/l85 0 2024-02-16T19:29:01.326 INFO:tasks.workunit.client.1.smithi117.stdout:6/325: dwrite d5/db/f55 [0,4194304] 0 2024-02-16T19:29:01.327 INFO:tasks.workunit.client.0.smithi022.stdout:7/351: creat d9/da/d1f/d67/f6b x:0 0 0 2024-02-16T19:29:01.336 INFO:tasks.workunit.client.1.smithi117.stdout:1/247: link d2/d18/l1a d2/d19/l52 0 2024-02-16T19:29:01.339 INFO:tasks.workunit.client.1.smithi117.stdout:9/263: rename dd/d45/f40 to dd/d47/f56 0 2024-02-16T19:29:01.344 INFO:tasks.workunit.client.1.smithi117.stdout:7/303: getdents d4/d1c/d24/d60 0 2024-02-16T19:29:01.350 INFO:tasks.workunit.client.1.smithi117.stdout:2/257: symlink d0/d13/d4d/l4e 0 2024-02-16T19:29:01.351 INFO:tasks.workunit.client.0.smithi022.stdout:7/352: mknod d9/da/d2c/d46/c6c 0 2024-02-16T19:29:01.352 INFO:tasks.workunit.client.0.smithi022.stdout:0/435: getdents d10/d11/d27/d3f/d58/d60 0 2024-02-16T19:29:01.354 INFO:tasks.workunit.client.0.smithi022.stdout:9/369: getdents db 0 2024-02-16T19:29:01.355 INFO:tasks.workunit.client.1.smithi117.stdout:9/264: dread dd/f1e [0,4194304] 0 2024-02-16T19:29:01.355 INFO:tasks.workunit.client.1.smithi117.stdout:9/265: chown dd/d45/f49 5679 1 2024-02-16T19:29:01.357 INFO:tasks.workunit.client.0.smithi022.stdout:8/337: truncate d3/d12/d17/f3d 2138832 0 2024-02-16T19:29:01.358 INFO:tasks.workunit.client.0.smithi022.stdout:7/353: mkdir d9/da/d2c/d6d 0 2024-02-16T19:29:01.358 INFO:tasks.workunit.client.1.smithi117.stdout:3/381: write d1/d1b/d1e/d37/f6d [120639,44834] 0 2024-02-16T19:29:01.358 INFO:tasks.workunit.client.0.smithi022.stdout:8/338: write d3/d12/d2b/d2e/f59 [316672,67492] 0 2024-02-16T19:29:01.359 INFO:tasks.workunit.client.1.smithi117.stdout:3/382: dread - d1/d1b/d1e/d37/f65 zero size 2024-02-16T19:29:01.359 INFO:tasks.workunit.client.0.smithi022.stdout:7/354: fsync d9/da/f13 0 2024-02-16T19:29:01.360 INFO:tasks.workunit.client.1.smithi117.stdout:3/383: write d1/d1b/d20/f7b [739019,22546] 0 2024-02-16T19:29:01.360 INFO:tasks.workunit.client.1.smithi117.stdout:3/384: fsync d1/d1b/d1e/f51 0 2024-02-16T19:29:01.363 INFO:tasks.workunit.client.1.smithi117.stdout:0/366: truncate d0/d9/d22/d27/f52 488682 0 2024-02-16T19:29:01.365 INFO:tasks.workunit.client.1.smithi117.stdout:7/304: write d4/d1c/d39/d46/f75 [658509,91365] 0 2024-02-16T19:29:01.365 INFO:tasks.workunit.client.1.smithi117.stdout:7/305: readlink d4/d4a/l53 0 2024-02-16T19:29:01.368 INFO:tasks.workunit.client.1.smithi117.stdout:4/326: creat d0/da/d21/d3d/f68 x:0 0 0 2024-02-16T19:29:01.369 INFO:tasks.workunit.client.1.smithi117.stdout:4/327: chown d0/da/d21/d3d/d28/d2f/f62 7296 1 2024-02-16T19:29:01.371 INFO:tasks.workunit.client.0.smithi022.stdout:9/370: dread db/d14/d16/d27/d2e/f38 [0,4194304] 0 2024-02-16T19:29:01.371 INFO:tasks.workunit.client.1.smithi117.stdout:2/258: truncate d0/f12 3958202 0 2024-02-16T19:29:01.372 INFO:tasks.workunit.client.0.smithi022.stdout:0/436: symlink d10/d11/d79/d8b/l90 0 2024-02-16T19:29:01.372 INFO:tasks.workunit.client.0.smithi022.stdout:9/371: chown db/d14/d16/d5d 65 1 2024-02-16T19:29:01.375 INFO:tasks.workunit.client.1.smithi117.stdout:4/328: dwrite d0/da/d21/d3d/d28/d64/f1c [4194304,4194304] 0 2024-02-16T19:29:01.375 INFO:tasks.workunit.client.1.smithi117.stdout:8/307: link d4/f3c d4/d7/d37/d3a/f61 0 2024-02-16T19:29:01.375 INFO:tasks.workunit.client.0.smithi022.stdout:8/339: dwrite d3/d12/d17/f50 [0,4194304] 0 2024-02-16T19:29:01.375 INFO:tasks.workunit.client.1.smithi117.stdout:6/326: write d5/d4b/f46 [240023,129658] 0 2024-02-16T19:29:01.377 INFO:tasks.workunit.client.1.smithi117.stdout:4/329: chown d0/da/d21/d3d/d28/f39 535453912 1 2024-02-16T19:29:01.378 INFO:tasks.workunit.client.1.smithi117.stdout:6/327: write d5/d1d/d28/d32/d37/f41 [198166,60256] 0 2024-02-16T19:29:01.378 INFO:tasks.workunit.client.1.smithi117.stdout:9/266: fdatasync dd/d23/f25 0 2024-02-16T19:29:01.381 INFO:tasks.workunit.client.0.smithi022.stdout:3/342: sync 2024-02-16T19:29:01.389 INFO:tasks.workunit.client.1.smithi117.stdout:5/408: creat d2/d3/d21/d49/f82 x:0 0 0 2024-02-16T19:29:01.391 INFO:tasks.workunit.client.0.smithi022.stdout:7/355: rename d9/da/d1f/d44/f55 to d9/f6e 0 2024-02-16T19:29:01.394 INFO:tasks.workunit.client.1.smithi117.stdout:7/306: read d4/f1e [13361,32094] 0 2024-02-16T19:29:01.395 INFO:tasks.workunit.client.0.smithi022.stdout:0/437: symlink d10/d11/d27/d3f/l91 0 2024-02-16T19:29:01.395 INFO:tasks.workunit.client.1.smithi117.stdout:5/409: dwrite d2/d3/d21/d49/f82 [0,4194304] 0 2024-02-16T19:29:01.400 INFO:tasks.workunit.client.0.smithi022.stdout:8/340: creat d3/d12/d2b/d46/d68/f78 x:0 0 0 2024-02-16T19:29:01.402 INFO:tasks.workunit.client.1.smithi117.stdout:1/248: creat d2/f53 x:0 0 0 2024-02-16T19:29:01.404 INFO:tasks.workunit.client.1.smithi117.stdout:1/249: write d2/d18/d20/f42 [611440,102989] 0 2024-02-16T19:29:01.405 INFO:tasks.workunit.client.1.smithi117.stdout:8/308: creat d4/d7/d37/d3a/d43/d44/f62 x:0 0 0 2024-02-16T19:29:01.405 INFO:tasks.workunit.client.1.smithi117.stdout:1/250: write d2/d1d/d1f/f46 [953898,77230] 0 2024-02-16T19:29:01.406 INFO:tasks.workunit.client.1.smithi117.stdout:1/251: dread - d2/d18/f3e zero size 2024-02-16T19:29:01.406 INFO:tasks.workunit.client.1.smithi117.stdout:1/252: readlink d2/le 0 2024-02-16T19:29:01.407 INFO:tasks.workunit.client.0.smithi022.stdout:7/356: creat d9/da/d2c/d46/f6f x:0 0 0 2024-02-16T19:29:01.407 INFO:tasks.workunit.client.1.smithi117.stdout:1/253: chown d2/d1d/d1f/f44 173125 1 2024-02-16T19:29:01.408 INFO:tasks.workunit.client.1.smithi117.stdout:5/410: dread d2/d3/f6 [0,4194304] 0 2024-02-16T19:29:01.408 INFO:tasks.workunit.client.1.smithi117.stdout:3/385: write d1/dd/f14 [3311131,16497] 0 2024-02-16T19:29:01.409 INFO:tasks.workunit.client.1.smithi117.stdout:5/411: truncate d2/d9/f7c 568221 0 2024-02-16T19:29:01.412 INFO:tasks.workunit.client.0.smithi022.stdout:8/341: creat d3/d25/f79 x:0 0 0 2024-02-16T19:29:01.418 INFO:tasks.workunit.client.0.smithi022.stdout:9/372: truncate db/d14/d19/d1a/f50 2200343 0 2024-02-16T19:29:01.419 INFO:tasks.workunit.client.0.smithi022.stdout:8/342: symlink d3/d12/d2b/d54/l7a 0 2024-02-16T19:29:01.421 INFO:tasks.workunit.client.1.smithi117.stdout:7/307: truncate d4/f2b 1222842 0 2024-02-16T19:29:01.422 INFO:tasks.workunit.client.1.smithi117.stdout:7/308: chown d4/d1c/d39/d46/f48 2048050 1 2024-02-16T19:29:01.423 INFO:tasks.workunit.client.0.smithi022.stdout:7/357: symlink d9/da/d2c/d2f/l70 0 2024-02-16T19:29:01.424 INFO:tasks.workunit.client.0.smithi022.stdout:9/373: dwrite db/d18/f2a [0,4194304] 0 2024-02-16T19:29:01.428 INFO:tasks.workunit.client.0.smithi022.stdout:8/343: symlink d3/d12/d2b/l7b 0 2024-02-16T19:29:01.428 INFO:tasks.workunit.client.0.smithi022.stdout:8/344: readlink d3/d6/l57 0 2024-02-16T19:29:01.434 INFO:tasks.workunit.client.1.smithi117.stdout:1/254: creat d2/d19/f54 x:0 0 0 2024-02-16T19:29:01.437 INFO:tasks.workunit.client.0.smithi022.stdout:9/374: unlink db/c82 0 2024-02-16T19:29:01.440 INFO:tasks.workunit.client.0.smithi022.stdout:0/438: link d10/d11/d27/l64 d10/d15/d23/d38/d5a/l92 0 2024-02-16T19:29:01.440 INFO:tasks.workunit.client.1.smithi117.stdout:1/255: dwrite d2/d1d/d1f/f44 [0,4194304] 0 2024-02-16T19:29:01.441 INFO:tasks.workunit.client.1.smithi117.stdout:1/256: dread - d2/d1d/d1f/f4e zero size 2024-02-16T19:29:01.441 INFO:tasks.workunit.client.1.smithi117.stdout:5/412: mknod d2/d3/d21/d2f/d30/c83 0 2024-02-16T19:29:01.442 INFO:tasks.workunit.client.0.smithi022.stdout:8/345: chown d3/d12/d17/f3d 7932182 1 2024-02-16T19:29:01.445 INFO:tasks.workunit.client.1.smithi117.stdout:8/309: write d4/d12/f26 [7412219,113976] 0 2024-02-16T19:29:01.445 INFO:tasks.workunit.client.1.smithi117.stdout:6/328: sync 2024-02-16T19:29:01.445 INFO:tasks.workunit.client.1.smithi117.stdout:9/267: sync 2024-02-16T19:29:01.446 INFO:tasks.workunit.client.1.smithi117.stdout:6/329: chown d5/d4b/d44/f56 16115 1 2024-02-16T19:29:01.450 INFO:tasks.workunit.client.0.smithi022.stdout:8/346: truncate d3/d12/f51 523280 0 2024-02-16T19:29:01.453 INFO:tasks.workunit.client.0.smithi022.stdout:0/439: creat d10/d7d/f93 x:0 0 0 2024-02-16T19:29:01.454 INFO:tasks.workunit.client.0.smithi022.stdout:0/440: write fb [3880029,108303] 0 2024-02-16T19:29:01.456 INFO:tasks.workunit.client.0.smithi022.stdout:0/441: truncate d10/d11/d27/f52 4952352 0 2024-02-16T19:29:01.456 INFO:tasks.workunit.client.1.smithi117.stdout:4/330: rename d0/da/d4a/f59 to d0/da/f69 0 2024-02-16T19:29:01.456 INFO:tasks.workunit.client.1.smithi117.stdout:4/331: rename d0/da to d0/da/d21/d3d/d28/d6a 22 2024-02-16T19:29:01.457 INFO:tasks.workunit.client.0.smithi022.stdout:8/347: creat d3/d12/d17/d64/f7c x:0 0 0 2024-02-16T19:29:01.461 INFO:tasks.workunit.client.1.smithi117.stdout:1/257: mkdir d2/d1d/d1f/d55 0 2024-02-16T19:29:01.463 INFO:tasks.workunit.client.1.smithi117.stdout:5/413: unlink d2/d3/l28 0 2024-02-16T19:29:01.463 INFO:tasks.workunit.client.1.smithi117.stdout:2/259: dread d0/f12 [0,4194304] 0 2024-02-16T19:29:01.463 INFO:tasks.workunit.client.0.smithi022.stdout:8/348: creat d3/d25/f7d x:0 0 0 2024-02-16T19:29:01.464 INFO:tasks.workunit.client.1.smithi117.stdout:5/414: fsync d2/d3/d21/d49/d41/f4b 0 2024-02-16T19:29:01.464 INFO:tasks.workunit.client.1.smithi117.stdout:0/367: getdents d0/d9/d14/d1d/d60/d76 0 2024-02-16T19:29:01.464 INFO:tasks.workunit.client.1.smithi117.stdout:2/260: dread - d0/d3/f31 zero size 2024-02-16T19:29:01.464 INFO:tasks.workunit.client.0.smithi022.stdout:9/375: stat db/d14/d16/d49 0 2024-02-16T19:29:01.465 INFO:tasks.workunit.client.1.smithi117.stdout:9/268: rmdir dd/d47/d48 39 2024-02-16T19:29:01.466 INFO:tasks.workunit.client.1.smithi117.stdout:9/269: write dd/d45/f3f [197911,15981] 0 2024-02-16T19:29:01.468 INFO:tasks.workunit.client.1.smithi117.stdout:8/310: creat d4/d7/d37/f63 x:0 0 0 2024-02-16T19:29:01.468 INFO:tasks.workunit.client.0.smithi022.stdout:0/442: mkdir d10/d11/d27/d3f/d94 0 2024-02-16T19:29:01.469 INFO:tasks.workunit.client.1.smithi117.stdout:8/311: write d4/d7/d37/d3a/d43/d44/f62 [485299,37589] 0 2024-02-16T19:29:01.470 INFO:tasks.workunit.client.1.smithi117.stdout:8/312: stat d4/d7/d37/d3a/d43/f55 0 2024-02-16T19:29:01.471 INFO:tasks.workunit.client.1.smithi117.stdout:8/313: write d4/d7/d37/d3a/d43/d44/f62 [641245,31167] 0 2024-02-16T19:29:01.471 INFO:tasks.workunit.client.1.smithi117.stdout:7/309: creat d4/d1c/d24/d60/d79/f7a x:0 0 0 2024-02-16T19:29:01.471 INFO:tasks.workunit.client.1.smithi117.stdout:8/314: stat d4/f3f 0 2024-02-16T19:29:01.473 INFO:tasks.workunit.client.1.smithi117.stdout:7/310: write d4/d1c/f3c [10311808,100524] 0 2024-02-16T19:29:01.473 INFO:tasks.workunit.client.0.smithi022.stdout:9/376: read db/f12 [3038944,82825] 0 2024-02-16T19:29:01.473 INFO:tasks.workunit.client.0.smithi022.stdout:7/358: link d9/d1b/d23/c32 d9/da/d2c/d2f/c71 0 2024-02-16T19:29:01.475 INFO:tasks.workunit.client.1.smithi117.stdout:8/315: dread d4/f52 [0,4194304] 0 2024-02-16T19:29:01.476 INFO:tasks.workunit.client.1.smithi117.stdout:8/316: write d4/f3f [3136289,85358] 0 2024-02-16T19:29:01.477 INFO:tasks.workunit.client.1.smithi117.stdout:8/317: fsync d4/d12/d31/f40 0 2024-02-16T19:29:01.478 INFO:tasks.workunit.client.1.smithi117.stdout:3/386: rmdir d1/d1b/d70 0 2024-02-16T19:29:01.480 INFO:tasks.workunit.client.1.smithi117.stdout:1/258: creat d2/d1d/d1f/f56 x:0 0 0 2024-02-16T19:29:01.482 INFO:tasks.workunit.client.1.smithi117.stdout:0/368: symlink d0/d30/d74/l8a 0 2024-02-16T19:29:01.484 INFO:tasks.workunit.client.0.smithi022.stdout:9/377: rmdir db/d14/d16/d49 39 2024-02-16T19:29:01.486 INFO:tasks.workunit.client.0.smithi022.stdout:0/443: dread f7 [0,4194304] 0 2024-02-16T19:29:01.488 INFO:tasks.workunit.client.1.smithi117.stdout:6/330: mknod d5/d4b/d44/c6d 0 2024-02-16T19:29:01.490 INFO:tasks.workunit.client.0.smithi022.stdout:8/349: dwrite d3/d12/d2b/f3a [8388608,4194304] 0 2024-02-16T19:29:01.491 INFO:tasks.workunit.client.1.smithi117.stdout:8/318: dread d4/d12/f1a [0,4194304] 0 2024-02-16T19:29:01.492 INFO:tasks.workunit.client.1.smithi117.stdout:4/332: getdents d0/da/d21/d3d/d2d 0 2024-02-16T19:29:01.492 INFO:tasks.workunit.client.1.smithi117.stdout:8/319: stat d4/d1c/l30 0 2024-02-16T19:29:01.493 INFO:tasks.workunit.client.0.smithi022.stdout:7/359: dwrite d9/d1b/f27 [0,4194304] 0 2024-02-16T19:29:01.494 INFO:tasks.workunit.client.1.smithi117.stdout:6/331: dwrite d5/d4b/d20/f21 [0,4194304] 0 2024-02-16T19:29:01.495 INFO:tasks.workunit.client.1.smithi117.stdout:8/320: fdatasync d4/d7/d37/d3a/d43/d44/f62 0 2024-02-16T19:29:01.497 INFO:tasks.workunit.client.0.smithi022.stdout:7/360: fsync d9/f6e 0 2024-02-16T19:29:01.498 INFO:tasks.workunit.client.1.smithi117.stdout:6/332: write d5/d1d/d49/d59/f5e [3320690,107788] 0 2024-02-16T19:29:01.499 INFO:tasks.workunit.client.1.smithi117.stdout:3/387: symlink d1/dd/d5c/l7c 0 2024-02-16T19:29:01.500 INFO:tasks.workunit.client.0.smithi022.stdout:9/378: getdents db/d14/d16/d27/d2e 0 2024-02-16T19:29:01.501 INFO:tasks.workunit.client.0.smithi022.stdout:9/379: dread - db/d14/d19/f5f zero size 2024-02-16T19:29:01.503 INFO:tasks.workunit.client.1.smithi117.stdout:5/415: creat d2/d3/d21/d2a/f84 x:0 0 0 2024-02-16T19:29:01.504 INFO:tasks.workunit.client.1.smithi117.stdout:4/333: dwrite d0/da/f56 [0,4194304] 0 2024-02-16T19:29:01.505 INFO:tasks.workunit.client.1.smithi117.stdout:5/416: chown d2/d3/d21/d49/f82 7755610 1 2024-02-16T19:29:01.514 INFO:tasks.workunit.client.1.smithi117.stdout:0/369: rename d0/d30/d74 to d0/d9/d2c/d8b 0 2024-02-16T19:29:01.517 INFO:tasks.workunit.client.1.smithi117.stdout:5/417: dwrite d2/d3/d21/d49/d41/f4b [0,4194304] 0 2024-02-16T19:29:01.518 INFO:tasks.workunit.client.1.smithi117.stdout:8/321: mkdir d4/d12/d2a/d64 0 2024-02-16T19:29:01.519 INFO:tasks.workunit.client.1.smithi117.stdout:8/322: chown d4/d7/d37/d3a/d54 479 1 2024-02-16T19:29:01.520 INFO:tasks.workunit.client.1.smithi117.stdout:8/323: chown d4/d7/d37/d3a/d43/d44/f62 324 1 2024-02-16T19:29:01.520 INFO:tasks.workunit.client.1.smithi117.stdout:8/324: read - d4/d1c/f60 zero size 2024-02-16T19:29:01.524 INFO:tasks.workunit.client.0.smithi022.stdout:0/444: fsync d10/d11/d27/d3f/f5b 0 2024-02-16T19:29:01.525 INFO:tasks.workunit.client.0.smithi022.stdout:0/445: fdatasync d10/d22/f45 0 2024-02-16T19:29:01.526 INFO:tasks.workunit.client.1.smithi117.stdout:9/270: creat dd/d47/f57 x:0 0 0 2024-02-16T19:29:01.526 INFO:tasks.workunit.client.0.smithi022.stdout:0/446: chown d10/d11/d27/d3f/f5b 104 1 2024-02-16T19:29:01.528 INFO:tasks.workunit.client.0.smithi022.stdout:0/447: write d10/d11/d50/d65/f68 [1393891,24189] 0 2024-02-16T19:29:01.528 INFO:tasks.workunit.client.0.smithi022.stdout:9/380: dread db/d14/d16/d21/f4c [0,4194304] 0 2024-02-16T19:29:01.529 INFO:tasks.workunit.client.0.smithi022.stdout:9/381: chown db/d14/d19/f17 277044541 1 2024-02-16T19:29:01.532 INFO:tasks.workunit.client.1.smithi117.stdout:5/418: creat d2/d3/d21/d2f/d30/d38/f85 x:0 0 0 2024-02-16T19:29:01.533 INFO:tasks.workunit.client.1.smithi117.stdout:8/325: chown c1 16264592 1 2024-02-16T19:29:01.536 INFO:tasks.workunit.client.1.smithi117.stdout:1/259: creat d2/f57 x:0 0 0 2024-02-16T19:29:01.536 INFO:tasks.workunit.client.0.smithi022.stdout:9/382: dwrite db/d14/f5e [0,4194304] 0 2024-02-16T19:29:01.539 INFO:tasks.workunit.client.0.smithi022.stdout:0/448: creat d10/d15/d23/d8f/f95 x:0 0 0 2024-02-16T19:29:01.540 INFO:tasks.workunit.client.1.smithi117.stdout:2/261: getdents d0/d3/d22 0 2024-02-16T19:29:01.542 INFO:tasks.workunit.client.1.smithi117.stdout:7/311: getdents d4/d1c/d39/d46/d47 0 2024-02-16T19:29:01.542 INFO:tasks.workunit.client.1.smithi117.stdout:1/260: dread d2/d1d/d1f/f44 [0,4194304] 0 2024-02-16T19:29:01.543 INFO:tasks.workunit.client.1.smithi117.stdout:9/271: write dd/d47/f56 [658078,117029] 0 2024-02-16T19:29:01.544 INFO:tasks.workunit.client.0.smithi022.stdout:0/449: truncate d10/d15/d23/d38/d5a/f5d 802064 0 2024-02-16T19:29:01.548 INFO:tasks.workunit.client.0.smithi022.stdout:7/361: truncate d9/da/d2c/d2f/f35 1643157 0 2024-02-16T19:29:01.548 INFO:tasks.workunit.client.0.smithi022.stdout:1/442: sync 2024-02-16T19:29:01.548 INFO:tasks.workunit.client.1.smithi117.stdout:5/419: mknod d2/d36/c86 0 2024-02-16T19:29:01.548 INFO:tasks.workunit.client.1.smithi117.stdout:9/272: chown dd/d2d/f35 452801344 1 2024-02-16T19:29:01.549 INFO:tasks.workunit.client.1.smithi117.stdout:5/420: chown d2/d3/d21/d2f/d30/d5a 45108412 1 2024-02-16T19:29:01.549 INFO:tasks.workunit.client.1.smithi117.stdout:0/370: dread d0/f5 [0,4194304] 0 2024-02-16T19:29:01.550 INFO:tasks.workunit.client.0.smithi022.stdout:9/383: getdents db/d14/d19 0 2024-02-16T19:29:01.551 INFO:tasks.workunit.client.1.smithi117.stdout:8/326: mkdir d4/d7/d37/d3a/d65 0 2024-02-16T19:29:01.555 INFO:tasks.workunit.client.0.smithi022.stdout:7/362: symlink d9/da/d1f/d40/l72 0 2024-02-16T19:29:01.555 INFO:tasks.workunit.client.1.smithi117.stdout:6/333: dread d5/d4b/f1e [0,4194304] 0 2024-02-16T19:29:01.557 INFO:tasks.workunit.client.1.smithi117.stdout:7/312: dwrite d4/d1c/d39/f54 [0,4194304] 0 2024-02-16T19:29:01.557 INFO:tasks.workunit.client.1.smithi117.stdout:8/327: chown d4/d7/d37/d3a/d43/d44/d5f 0 1 2024-02-16T19:29:01.558 INFO:tasks.workunit.client.1.smithi117.stdout:1/261: dwrite d2/d1d/d1f/f56 [0,4194304] 0 2024-02-16T19:29:01.559 INFO:tasks.workunit.client.1.smithi117.stdout:1/262: dread - d2/d1d/f3a zero size 2024-02-16T19:29:01.560 INFO:tasks.workunit.client.0.smithi022.stdout:1/443: symlink d6/l81 0 2024-02-16T19:29:01.562 INFO:tasks.workunit.client.1.smithi117.stdout:0/371: creat d0/d30/f8c x:0 0 0 2024-02-16T19:29:01.563 INFO:tasks.workunit.client.1.smithi117.stdout:3/388: truncate d1/dd/f17 1879486 0 2024-02-16T19:29:01.568 INFO:tasks.workunit.client.0.smithi022.stdout:9/384: rename db/d14/d19/c58 to db/d18/c86 0 2024-02-16T19:29:01.570 INFO:tasks.workunit.client.0.smithi022.stdout:9/385: write db/d14/d19/f4d [1619522,59987] 0 2024-02-16T19:29:01.574 INFO:tasks.workunit.client.0.smithi022.stdout:0/450: write f7 [2962250,67921] 0 2024-02-16T19:29:01.578 INFO:tasks.workunit.client.0.smithi022.stdout:0/451: dread d10/d11/d27/d3f/f57 [0,4194304] 0 2024-02-16T19:29:01.579 INFO:tasks.workunit.client.1.smithi117.stdout:7/313: rmdir d4/d13 39 2024-02-16T19:29:01.579 INFO:tasks.workunit.client.0.smithi022.stdout:0/452: chown le 112437105 1 2024-02-16T19:29:01.580 INFO:tasks.workunit.client.1.smithi117.stdout:7/314: dread - d4/d1c/d39/d46/d47/f73 zero size 2024-02-16T19:29:01.582 INFO:tasks.workunit.client.1.smithi117.stdout:2/262: creat d0/f4f x:0 0 0 2024-02-16T19:29:01.582 INFO:tasks.workunit.client.1.smithi117.stdout:2/263: chown d0/d13/f1c 460 1 2024-02-16T19:29:01.583 INFO:tasks.workunit.client.1.smithi117.stdout:2/264: truncate d0/d3/f31 772190 0 2024-02-16T19:29:01.587 INFO:tasks.workunit.client.0.smithi022.stdout:9/386: mkdir db/d14/d16/d87 0 2024-02-16T19:29:01.589 INFO:tasks.workunit.client.1.smithi117.stdout:3/389: mkdir d1/dd/d3e/d7d 0 2024-02-16T19:29:01.592 INFO:tasks.workunit.client.0.smithi022.stdout:1/444: creat d6/d8/d22/f82 x:0 0 0 2024-02-16T19:29:01.593 INFO:tasks.workunit.client.0.smithi022.stdout:7/363: dwrite f5 [4194304,4194304] 0 2024-02-16T19:29:01.597 INFO:tasks.workunit.client.1.smithi117.stdout:0/372: rename d0/d9/d22/d26 to d0/d9/d14/d1d/d60/d76/d7b/d8d 0 2024-02-16T19:29:01.601 INFO:tasks.workunit.client.0.smithi022.stdout:1/445: dwrite d6/d8/d22/d16/d17/d1d/f21 [0,4194304] 0 2024-02-16T19:29:01.602 INFO:tasks.workunit.client.1.smithi117.stdout:5/421: symlink d2/d9/d3c/d24/d50/l87 0 2024-02-16T19:29:01.604 INFO:tasks.workunit.client.0.smithi022.stdout:1/446: read d6/d8/d22/d16/d17/d1d/f33 [2495241,92954] 0 2024-02-16T19:29:01.606 INFO:tasks.workunit.client.0.smithi022.stdout:9/387: dread db/d14/d19/f17 [0,4194304] 0 2024-02-16T19:29:01.607 INFO:tasks.workunit.client.1.smithi117.stdout:6/334: truncate d5/d1d/d28/d32/f3f 3775 0 2024-02-16T19:29:01.607 INFO:tasks.workunit.client.0.smithi022.stdout:1/447: stat d6/d8/cb 0 2024-02-16T19:29:01.609 INFO:tasks.workunit.client.0.smithi022.stdout:0/453: dread d10/d22/f2d [0,4194304] 0 2024-02-16T19:29:01.612 INFO:tasks.workunit.client.1.smithi117.stdout:9/273: creat dd/d45/f58 x:0 0 0 2024-02-16T19:29:01.613 INFO:tasks.workunit.client.1.smithi117.stdout:4/334: sync 2024-02-16T19:29:01.614 INFO:tasks.workunit.client.0.smithi022.stdout:9/388: dwrite db/d14/f60 [4194304,4194304] 0 2024-02-16T19:29:01.615 INFO:tasks.workunit.client.1.smithi117.stdout:4/335: truncate d0/da/d21/d3d/d28/d64/d48/f5d 4442721 0 2024-02-16T19:29:01.617 INFO:tasks.workunit.client.1.smithi117.stdout:2/265: creat d0/d13/d4d/f50 x:0 0 0 2024-02-16T19:29:01.624 INFO:tasks.workunit.client.1.smithi117.stdout:2/266: dwrite d0/d3/fc [0,4194304] 0 2024-02-16T19:29:01.625 INFO:tasks.workunit.client.1.smithi117.stdout:2/267: write d0/d3/f3a [4191,94304] 0 2024-02-16T19:29:01.625 INFO:tasks.workunit.client.0.smithi022.stdout:7/364: write d9/da/d2c/d2f/d21/f48 [1136721,8381] 0 2024-02-16T19:29:01.626 INFO:tasks.workunit.client.1.smithi117.stdout:8/328: mknod d4/d7/d37/d3a/d65/c66 0 2024-02-16T19:29:01.626 INFO:tasks.workunit.client.1.smithi117.stdout:2/268: stat d0/d3/c39 0 2024-02-16T19:29:01.628 INFO:tasks.workunit.client.1.smithi117.stdout:3/390: creat d1/d31/f7e x:0 0 0 2024-02-16T19:29:01.629 INFO:tasks.workunit.client.0.smithi022.stdout:1/448: dread d6/f6b [0,4194304] 0 2024-02-16T19:29:01.630 INFO:tasks.workunit.client.0.smithi022.stdout:0/454: truncate d10/d22/f2f 3068766 0 2024-02-16T19:29:01.630 INFO:tasks.workunit.client.0.smithi022.stdout:1/449: read d6/d8/d22/f28 [2248610,46708] 0 2024-02-16T19:29:01.633 INFO:tasks.workunit.client.0.smithi022.stdout:9/389: creat db/d14/d33/f88 x:0 0 0 2024-02-16T19:29:01.634 INFO:tasks.workunit.client.0.smithi022.stdout:9/390: chown db/d14/d16/d5d/d65 0 1 2024-02-16T19:29:01.634 INFO:tasks.workunit.client.1.smithi117.stdout:0/373: rename d0/d9/d14/d1d/d60/d76/d7b/d8d/c3f to d0/d9/d14/d1d/d60/d76/d7b/d8d/c8e 0 2024-02-16T19:29:01.634 INFO:tasks.workunit.client.1.smithi117.stdout:2/269: dwrite d0/d3/d22/f28 [4194304,4194304] 0 2024-02-16T19:29:01.636 INFO:tasks.workunit.client.1.smithi117.stdout:5/422: mknod d2/d3/d21/d49/d59/c88 0 2024-02-16T19:29:01.636 INFO:tasks.workunit.client.1.smithi117.stdout:0/374: readlink d0/d9/d14/d1d/d60/d76/d7b/d8d/l3c 0 2024-02-16T19:29:01.637 INFO:tasks.workunit.client.0.smithi022.stdout:7/365: dread d9/da/d3b/f41 [0,4194304] 0 2024-02-16T19:29:01.638 INFO:tasks.workunit.client.0.smithi022.stdout:7/366: chown d9/da/f29 14553 1 2024-02-16T19:29:01.640 INFO:tasks.workunit.client.1.smithi117.stdout:1/263: dwrite d2/d4/f15 [0,4194304] 0 2024-02-16T19:29:01.641 INFO:tasks.workunit.client.0.smithi022.stdout:0/455: mknod d10/d11/d27/d3f/d58/d60/d62/c96 0 2024-02-16T19:29:01.645 INFO:tasks.workunit.client.1.smithi117.stdout:7/315: creat d4/d1c/d24/d5f/f7b x:0 0 0 2024-02-16T19:29:01.645 INFO:tasks.workunit.client.1.smithi117.stdout:0/375: dwrite d0/d9/f12 [4194304,4194304] 0 2024-02-16T19:29:01.646 INFO:tasks.workunit.client.0.smithi022.stdout:9/391: truncate f5 2947248 0 2024-02-16T19:29:01.650 INFO:tasks.workunit.client.0.smithi022.stdout:0/456: rename d10/d11/d27/l6b to d10/d15/l97 0 2024-02-16T19:29:01.650 INFO:tasks.workunit.client.0.smithi022.stdout:7/367: dwrite d9/da/d1f/d67/f6b [0,4194304] 0 2024-02-16T19:29:01.654 INFO:tasks.workunit.client.1.smithi117.stdout:3/391: symlink d1/d1b/d1e/l7f 0 2024-02-16T19:29:01.656 INFO:tasks.workunit.client.0.smithi022.stdout:0/457: mknod d10/d11/c98 0 2024-02-16T19:29:01.660 INFO:tasks.workunit.client.0.smithi022.stdout:7/368: mkdir d9/da/d2c/d46/d73 0 2024-02-16T19:29:01.660 INFO:tasks.workunit.client.1.smithi117.stdout:5/423: symlink d2/d9/l89 0 2024-02-16T19:29:01.661 INFO:tasks.workunit.client.1.smithi117.stdout:6/335: symlink d5/d4b/d20/d69/l6e 0 2024-02-16T19:29:01.662 INFO:tasks.workunit.client.1.smithi117.stdout:6/336: write d5/d4b/d20/d2d/f3c [750083,1099] 0 2024-02-16T19:29:01.664 INFO:tasks.workunit.client.0.smithi022.stdout:1/450: dread d6/d8/d22/d16/f41 [0,4194304] 0 2024-02-16T19:29:01.664 INFO:tasks.workunit.client.1.smithi117.stdout:9/274: mknod dd/d45/c59 0 2024-02-16T19:29:01.666 INFO:tasks.workunit.client.1.smithi117.stdout:0/376: symlink d0/d9/d14/l8f 0 2024-02-16T19:29:01.666 INFO:tasks.workunit.client.1.smithi117.stdout:7/316: chown d4/c22 9010 1 2024-02-16T19:29:01.666 INFO:tasks.workunit.client.0.smithi022.stdout:0/458: read d10/d22/f2a [830789,20519] 0 2024-02-16T19:29:01.668 INFO:tasks.workunit.client.0.smithi022.stdout:0/459: truncate d10/d22/d49/f76 241173 0 2024-02-16T19:29:01.671 INFO:tasks.workunit.client.1.smithi117.stdout:6/337: dwrite d5/d1d/d28/d32/d37/f41 [0,4194304] 0 2024-02-16T19:29:01.672 INFO:tasks.workunit.client.0.smithi022.stdout:7/369: mkdir d9/da/d74 0 2024-02-16T19:29:01.674 INFO:tasks.workunit.client.1.smithi117.stdout:9/275: dwrite dd/d23/f3b [0,4194304] 0 2024-02-16T19:29:01.677 INFO:tasks.workunit.client.1.smithi117.stdout:2/270: creat d0/d3b/f51 x:0 0 0 2024-02-16T19:29:01.677 INFO:tasks.workunit.client.0.smithi022.stdout:2/268: sync 2024-02-16T19:29:01.677 INFO:tasks.workunit.client.0.smithi022.stdout:3/343: sync 2024-02-16T19:29:01.677 INFO:tasks.workunit.client.0.smithi022.stdout:5/360: sync 2024-02-16T19:29:01.677 INFO:tasks.workunit.client.0.smithi022.stdout:4/331: sync 2024-02-16T19:29:01.678 INFO:tasks.workunit.client.0.smithi022.stdout:6/364: sync 2024-02-16T19:29:01.678 INFO:tasks.workunit.client.0.smithi022.stdout:1/451: mkdir d6/d8/d22/d83 0 2024-02-16T19:29:01.685 INFO:tasks.workunit.client.0.smithi022.stdout:9/392: write db/d14/d19/d1a/d67/f44 [37170,96542] 0 2024-02-16T19:29:01.687 INFO:tasks.workunit.client.0.smithi022.stdout:7/370: rename d9/da/d1f/d44/l58 to d9/da/d1f/d67/l75 0 2024-02-16T19:29:01.688 INFO:tasks.workunit.client.0.smithi022.stdout:9/393: stat db/d14/d16/d21/l7d 0 2024-02-16T19:29:01.689 INFO:tasks.workunit.client.0.smithi022.stdout:7/371: chown d9/da/d2c/d2f/l2a 472836 1 2024-02-16T19:29:01.690 INFO:tasks.workunit.client.0.smithi022.stdout:9/394: chown db/d14/d16/l5b 4377 1 2024-02-16T19:29:01.691 INFO:tasks.workunit.client.1.smithi117.stdout:5/424: rename d2/d9/d3c/d24/d50/d52/cf to d2/d4c/d4f/c8a 0 2024-02-16T19:29:01.691 INFO:tasks.workunit.client.0.smithi022.stdout:1/452: dwrite d6/d8/d22/d16/d17/d5d/f7b [0,4194304] 0 2024-02-16T19:29:01.693 INFO:tasks.workunit.client.0.smithi022.stdout:9/395: write db/d14/d16/d27/f68 [856999,101144] 0 2024-02-16T19:29:01.696 INFO:tasks.workunit.client.0.smithi022.stdout:4/332: creat d4/d6/dd/d22/f68 x:0 0 0 2024-02-16T19:29:01.697 INFO:tasks.workunit.client.1.smithi117.stdout:1/264: mknod d2/c58 0 2024-02-16T19:29:01.698 INFO:tasks.workunit.client.1.smithi117.stdout:3/392: write d1/d1b/d1e/f23 [690350,8745] 0 2024-02-16T19:29:01.701 INFO:tasks.workunit.client.1.smithi117.stdout:5/425: dwrite d2/d4c/d54/f77 [0,4194304] 0 2024-02-16T19:29:01.701 INFO:tasks.workunit.client.0.smithi022.stdout:5/361: mkdir dd/d1f/d48/d21/d2f/d52/d77 0 2024-02-16T19:29:01.702 INFO:tasks.workunit.client.1.smithi117.stdout:3/393: dread - d1/d1b/f21 zero size 2024-02-16T19:29:01.705 INFO:tasks.workunit.client.1.smithi117.stdout:4/336: dwrite d0/da/d21/d3d/d28/d64/f55 [4194304,4194304] 0 2024-02-16T19:29:01.709 INFO:tasks.workunit.client.0.smithi022.stdout:3/344: creat d7/d12/d18/dd/d24/d58/d6b/f89 x:0 0 0 2024-02-16T19:29:01.709 INFO:tasks.workunit.client.1.smithi117.stdout:1/265: dread d2/d1d/d1f/f46 [0,4194304] 0 2024-02-16T19:29:01.714 INFO:tasks.workunit.client.0.smithi022.stdout:0/460: truncate d10/d22/f45 5516138 0 2024-02-16T19:29:01.715 INFO:tasks.workunit.client.1.smithi117.stdout:6/338: write d5/f15 [812657,12421] 0 2024-02-16T19:29:01.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:01 smithi022 ceph-mon[29221]: [16/Feb/2024:19:28:59] ENGINE Bus STARTING 2024-02-16T19:29:01.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:01 smithi022 ceph-mon[29221]: Deploying cephadm binary to smithi117 2024-02-16T19:29:01.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:01 smithi022 ceph-mon[29221]: [16/Feb/2024:19:29:00] ENGINE Serving on https://172.21.15.117:7150 2024-02-16T19:29:01.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:01 smithi022 ceph-mon[29221]: Deploying cephadm binary to smithi022 2024-02-16T19:29:01.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:01 smithi022 ceph-mon[29221]: [16/Feb/2024:19:29:00] ENGINE Serving on http://172.21.15.117:8765 2024-02-16T19:29:01.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:01 smithi022 ceph-mon[29221]: [16/Feb/2024:19:29:00] ENGINE Bus STARTED 2024-02-16T19:29:01.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:01 smithi022 ceph-mon[29221]: mgrmap e22: smithi117.cohbtv(active, since 1.00844s) 2024-02-16T19:29:01.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:01 smithi022 ceph-mon[29221]: pgmap v3: 65 pgs: 65 active+clean; 1.4 GiB data, 4.2 GiB used, 532 GiB / 536 GiB avail 2024-02-16T19:29:01.715 INFO:tasks.workunit.client.1.smithi117.stdout:3/394: dwrite d1/dd/d32/f75 [0,4194304] 0 2024-02-16T19:29:01.716 INFO:tasks.workunit.client.0.smithi022.stdout:1/453: creat d6/d8/d22/d3f/f84 x:0 0 0 2024-02-16T19:29:01.717 INFO:tasks.workunit.client.1.smithi117.stdout:2/271: mkdir d0/d29/d52 0 2024-02-16T19:29:01.717 INFO:tasks.workunit.client.0.smithi022.stdout:1/454: chown d6/d8/d22/d16/d17/d63/f68 26 1 2024-02-16T19:29:01.721 INFO:tasks.workunit.client.0.smithi022.stdout:4/333: rename d4/l3f to d4/d6/dd/d22/d30/l69 0 2024-02-16T19:29:01.723 INFO:tasks.workunit.client.1.smithi117.stdout:7/317: mknod d4/d13/d63/c7c 0 2024-02-16T19:29:01.723 INFO:tasks.workunit.client.1.smithi117.stdout:7/318: dread - d4/d1c/d39/d46/f48 zero size 2024-02-16T19:29:01.724 INFO:tasks.workunit.client.1.smithi117.stdout:7/319: write d4/d13/f18 [1346541,42411] 0 2024-02-16T19:29:01.725 INFO:tasks.workunit.client.0.smithi022.stdout:9/396: dwrite db/f84 [0,4194304] 0 2024-02-16T19:29:01.729 INFO:tasks.workunit.client.1.smithi117.stdout:7/320: read f2 [3657327,114762] 0 2024-02-16T19:29:01.730 INFO:tasks.workunit.client.0.smithi022.stdout:5/362: mknod dd/c78 0 2024-02-16T19:29:01.730 INFO:tasks.workunit.client.0.smithi022.stdout:9/397: readlink db/d14/d16/d28/l77 0 2024-02-16T19:29:01.732 INFO:tasks.workunit.client.0.smithi022.stdout:8/350: sync 2024-02-16T19:29:01.732 INFO:tasks.workunit.client.1.smithi117.stdout:5/426: rename d2/c44 to d2/d3/d21/d2a/c8b 0 2024-02-16T19:29:01.735 INFO:tasks.workunit.client.1.smithi117.stdout:4/337: mknod d0/da/d21/d3d/d28/d64/d48/c6b 0 2024-02-16T19:29:01.736 INFO:tasks.workunit.client.1.smithi117.stdout:4/338: read - d0/da/d21/d3d/d2d/f41 zero size 2024-02-16T19:29:01.737 INFO:tasks.workunit.client.1.smithi117.stdout:4/339: dread - d0/da/d21/d3d/d28/d2f/f62 zero size 2024-02-16T19:29:01.737 INFO:tasks.workunit.client.0.smithi022.stdout:2/269: mkdir d0/d2/d18/d19/d40/d55/d59 0 2024-02-16T19:29:01.738 INFO:tasks.workunit.client.1.smithi117.stdout:8/329: getdents d4/d1c 0 2024-02-16T19:29:01.743 INFO:tasks.workunit.client.1.smithi117.stdout:1/266: readlink d2/d19/l52 0 2024-02-16T19:29:01.748 INFO:tasks.workunit.client.0.smithi022.stdout:6/365: write d1/d8/da/d17/d53/f5c [316676,93174] 0 2024-02-16T19:29:01.748 INFO:tasks.workunit.client.1.smithi117.stdout:1/267: dwrite d2/d4/f47 [0,4194304] 0 2024-02-16T19:29:01.751 INFO:tasks.workunit.client.0.smithi022.stdout:1/455: creat d6/d8/d22/d47/d51/f85 x:0 0 0 2024-02-16T19:29:01.751 INFO:tasks.workunit.client.0.smithi022.stdout:1/456: chown d6 2719915 1 2024-02-16T19:29:01.752 INFO:tasks.workunit.client.1.smithi117.stdout:0/377: write d0/d9/d22/d27/f52 [1037947,116053] 0 2024-02-16T19:29:01.764 INFO:tasks.workunit.client.0.smithi022.stdout:3/345: dwrite f5 [0,4194304] 0 2024-02-16T19:29:01.766 INFO:tasks.workunit.client.0.smithi022.stdout:4/334: write d4/f23 [4119211,33325] 0 2024-02-16T19:29:01.767 INFO:tasks.workunit.client.1.smithi117.stdout:9/276: dwrite dd/d23/f18 [0,4194304] 0 2024-02-16T19:29:01.768 INFO:tasks.workunit.client.0.smithi022.stdout:7/372: creat d9/da/d2c/d2f/f76 x:0 0 0 2024-02-16T19:29:01.768 INFO:tasks.workunit.client.0.smithi022.stdout:4/335: fsync d4/d6/dd/d22/f68 0 2024-02-16T19:29:01.769 INFO:tasks.workunit.client.1.smithi117.stdout:9/277: truncate dd/d45/f42 1021374 0 2024-02-16T19:29:01.770 INFO:tasks.workunit.client.0.smithi022.stdout:4/336: write d4/d6/d14/d64/f34 [2388863,63341] 0 2024-02-16T19:29:01.772 INFO:tasks.workunit.client.0.smithi022.stdout:4/337: chown d4/d6/d14/d64 705231 1 2024-02-16T19:29:01.773 INFO:tasks.workunit.client.0.smithi022.stdout:6/366: rename d1/d8/da/d17/c1e to d1/d8/da/d29/d61/d57/c87 0 2024-02-16T19:29:01.775 INFO:tasks.workunit.client.1.smithi117.stdout:6/339: rename d5/d1d/f3d to d5/db/f6f 0 2024-02-16T19:29:01.779 INFO:tasks.workunit.client.1.smithi117.stdout:4/340: creat d0/da/d21/d3d/d28/d64/f6c x:0 0 0 2024-02-16T19:29:01.780 INFO:tasks.workunit.client.1.smithi117.stdout:4/341: chown d0/da/d21/d3d/f68 52425 1 2024-02-16T19:29:01.783 INFO:tasks.workunit.client.0.smithi022.stdout:9/398: mkdir db/d14/d16/d5d/d65/d6b/d89 0 2024-02-16T19:29:01.783 INFO:tasks.workunit.client.0.smithi022.stdout:9/399: read db/f79 [662196,49993] 0 2024-02-16T19:29:01.788 INFO:tasks.workunit.client.0.smithi022.stdout:2/270: creat d0/d48/f5a x:0 0 0 2024-02-16T19:29:01.788 INFO:tasks.workunit.client.0.smithi022.stdout:3/346: symlink d7/d12/d18/dd/d31/l8a 0 2024-02-16T19:29:01.789 INFO:tasks.workunit.client.0.smithi022.stdout:0/461: dwrite d10/d22/f2f [0,4194304] 0 2024-02-16T19:29:01.790 INFO:tasks.workunit.client.0.smithi022.stdout:2/271: dread - d0/d32/f58 zero size 2024-02-16T19:29:01.792 INFO:tasks.workunit.client.0.smithi022.stdout:3/347: fdatasync d7/d12/d18/dd/d31/d41/f48 0 2024-02-16T19:29:01.792 INFO:tasks.workunit.client.0.smithi022.stdout:1/457: creat d6/d8/d22/d3a/d80/f86 x:0 0 0 2024-02-16T19:29:01.793 INFO:tasks.workunit.client.0.smithi022.stdout:2/272: fsync d0/d2/d18/f34 0 2024-02-16T19:29:01.796 INFO:tasks.workunit.client.1.smithi117.stdout:9/278: creat dd/d24/f5a x:0 0 0 2024-02-16T19:29:01.796 INFO:tasks.workunit.client.0.smithi022.stdout:1/458: fsync d6/d8/d22/d16/d17/d63/f69 0 2024-02-16T19:29:01.800 INFO:tasks.workunit.client.0.smithi022.stdout:4/338: mkdir d4/d38/d6a 0 2024-02-16T19:29:01.802 INFO:tasks.workunit.client.1.smithi117.stdout:6/340: readlink d5/l60 0 2024-02-16T19:29:01.803 INFO:tasks.workunit.client.0.smithi022.stdout:5/363: dwrite dd/f11 [0,4194304] 0 2024-02-16T19:29:01.804 INFO:tasks.workunit.client.0.smithi022.stdout:5/364: dread - dd/d1f/d48/f43 zero size 2024-02-16T19:29:01.805 INFO:tasks.workunit.client.0.smithi022.stdout:5/365: fdatasync dd/d1f/d48/d21/d2f/d52/d5c/f63 0 2024-02-16T19:29:01.808 INFO:tasks.workunit.client.0.smithi022.stdout:6/367: dread d1/d8/f11 [0,4194304] 0 2024-02-16T19:29:01.812 INFO:tasks.workunit.client.0.smithi022.stdout:7/373: rename d9/da/d1f/c28 to d9/da/d1f/d44/d5b/c77 0 2024-02-16T19:29:01.813 INFO:tasks.workunit.client.1.smithi117.stdout:7/321: rename d4/d4a/l40 to d4/d13/d63/d71/l7d 0 2024-02-16T19:29:01.814 INFO:tasks.workunit.client.1.smithi117.stdout:2/272: sync 2024-02-16T19:29:01.814 INFO:tasks.workunit.client.1.smithi117.stdout:2/273: readlink d0/d13/d4d/l4e 0 2024-02-16T19:29:01.815 INFO:tasks.workunit.client.1.smithi117.stdout:2/274: dread - d0/d3b/f51 zero size 2024-02-16T19:29:01.815 INFO:tasks.workunit.client.1.smithi117.stdout:3/395: truncate d1/dd/d32/f75 1753361 0 2024-02-16T19:29:01.816 INFO:tasks.workunit.client.1.smithi117.stdout:3/396: chown d1/d1b/d1e/d29/f33 43 1 2024-02-16T19:29:01.819 INFO:tasks.workunit.client.1.smithi117.stdout:4/342: mkdir d0/da/d21/d3d/d2d/d6d 0 2024-02-16T19:29:01.821 INFO:tasks.workunit.client.1.smithi117.stdout:3/397: read d1/d31/f36 [2035009,25852] 0 2024-02-16T19:29:01.823 INFO:tasks.workunit.client.0.smithi022.stdout:8/351: creat d3/d12/d17/f7e x:0 0 0 2024-02-16T19:29:01.824 INFO:tasks.workunit.client.1.smithi117.stdout:0/378: write d0/d9/d14/d1d/d60/d76/d7b/d8d/f2e [4792929,84253] 0 2024-02-16T19:29:01.827 INFO:tasks.workunit.client.1.smithi117.stdout:6/341: mknod d5/d4b/d20/c70 0 2024-02-16T19:29:01.828 INFO:tasks.workunit.client.1.smithi117.stdout:6/342: stat d5/d1d/d28/d32 0 2024-02-16T19:29:01.829 INFO:tasks.workunit.client.0.smithi022.stdout:8/352: dwrite d3/d6/f7 [0,4194304] 0 2024-02-16T19:29:01.835 INFO:tasks.workunit.client.1.smithi117.stdout:2/275: mkdir d0/d13/d4d/d53 0 2024-02-16T19:29:01.844 INFO:tasks.workunit.client.0.smithi022.stdout:6/368: creat d1/d8/da/d17/d85/d5b/f88 x:0 0 0 2024-02-16T19:29:01.848 INFO:tasks.workunit.client.1.smithi117.stdout:3/398: mkdir d1/d30/d80 0 2024-02-16T19:29:01.853 INFO:tasks.workunit.client.1.smithi117.stdout:0/379: rmdir d0/d9/d14/d1d 39 2024-02-16T19:29:01.855 INFO:tasks.workunit.client.0.smithi022.stdout:2/273: write d0/d2/d18/d1f/f4d [886876,44521] 0 2024-02-16T19:29:01.856 INFO:tasks.workunit.client.0.smithi022.stdout:2/274: write d0/d32/f58 [492217,72846] 0 2024-02-16T19:29:01.857 INFO:tasks.workunit.client.0.smithi022.stdout:6/369: dread d1/f2 [0,4194304] 0 2024-02-16T19:29:01.858 INFO:tasks.workunit.client.0.smithi022.stdout:9/400: fdatasync db/d14/d16/d21/d37/f63 0 2024-02-16T19:29:01.860 INFO:tasks.workunit.client.0.smithi022.stdout:9/401: write db/d14/d16/d21/d37/f4e [2209975,87834] 0 2024-02-16T19:29:01.861 INFO:tasks.workunit.client.0.smithi022.stdout:9/402: stat db/d14/d33/l54 0 2024-02-16T19:29:01.864 INFO:tasks.workunit.client.1.smithi117.stdout:8/330: rename d4/d7/d37/c3e to d4/d7/d37/d3a/d54/c67 0 2024-02-16T19:29:01.867 INFO:tasks.workunit.client.1.smithi117.stdout:2/276: mknod d0/d29/c54 0 2024-02-16T19:29:01.867 INFO:tasks.workunit.client.1.smithi117.stdout:2/277: fsync d0/f4f 0 2024-02-16T19:29:01.867 INFO:tasks.workunit.client.1.smithi117.stdout:9/279: truncate dd/d23/f18 674803 0 2024-02-16T19:29:01.868 INFO:tasks.workunit.client.1.smithi117.stdout:9/280: dread - dd/d45/f3c zero size 2024-02-16T19:29:01.868 INFO:tasks.workunit.client.1.smithi117.stdout:2/278: dread - d0/d13/d4d/f50 zero size 2024-02-16T19:29:01.869 INFO:tasks.workunit.client.1.smithi117.stdout:2/279: stat d0/fd 0 2024-02-16T19:29:01.870 INFO:tasks.workunit.client.0.smithi022.stdout:0/462: write d10/d15/d23/d38/d5a/f5d [1240414,127059] 0 2024-02-16T19:29:01.872 INFO:tasks.workunit.client.1.smithi117.stdout:5/427: getdents d2/d3/d21/d49 0 2024-02-16T19:29:01.873 INFO:tasks.workunit.client.1.smithi117.stdout:5/428: fdatasync d2/d5d/f5e 0 2024-02-16T19:29:01.873 INFO:tasks.workunit.client.0.smithi022.stdout:4/339: rename d4/d6/dd/df/d5b to d4/d6/dd/df/d10/d24/d6b 0 2024-02-16T19:29:01.874 INFO:tasks.workunit.client.0.smithi022.stdout:4/340: fdatasync d4/d6/dd/d22/d30/f26 0 2024-02-16T19:29:01.878 INFO:tasks.workunit.client.1.smithi117.stdout:1/268: getdents d2/d18/d20 0 2024-02-16T19:29:01.879 INFO:tasks.workunit.client.1.smithi117.stdout:5/429: dwrite d2/d3/d21/d2f/d30/d38/d51/f7e [0,4194304] 0 2024-02-16T19:29:01.879 INFO:tasks.workunit.client.0.smithi022.stdout:2/275: symlink d0/d32/l5b 0 2024-02-16T19:29:01.880 INFO:tasks.workunit.client.0.smithi022.stdout:1/459: write d6/d8/d22/d16/f40 [1377771,10016] 0 2024-02-16T19:29:01.882 INFO:tasks.workunit.client.0.smithi022.stdout:1/460: dread - d6/d8/d22/d16/d17/f52 zero size 2024-02-16T19:29:01.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:01 smithi117 ceph-mon[35309]: [16/Feb/2024:19:28:59] ENGINE Bus STARTING 2024-02-16T19:29:01.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:01 smithi117 ceph-mon[35309]: Deploying cephadm binary to smithi117 2024-02-16T19:29:01.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:01 smithi117 ceph-mon[35309]: [16/Feb/2024:19:29:00] ENGINE Serving on https://172.21.15.117:7150 2024-02-16T19:29:01.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:01 smithi117 ceph-mon[35309]: Deploying cephadm binary to smithi022 2024-02-16T19:29:01.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:01 smithi117 ceph-mon[35309]: [16/Feb/2024:19:29:00] ENGINE Serving on http://172.21.15.117:8765 2024-02-16T19:29:01.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:01 smithi117 ceph-mon[35309]: [16/Feb/2024:19:29:00] ENGINE Bus STARTED 2024-02-16T19:29:01.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:01 smithi117 ceph-mon[35309]: mgrmap e22: smithi117.cohbtv(active, since 1.00844s) 2024-02-16T19:29:01.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:01 smithi117 ceph-mon[35309]: pgmap v3: 65 pgs: 65 active+clean; 1.4 GiB data, 4.2 GiB used, 532 GiB / 536 GiB avail 2024-02-16T19:29:01.889 INFO:tasks.workunit.client.0.smithi022.stdout:5/366: truncate dd/d1f/d48/d21/d2f/d52/d5c/d61/d6a/f70 2872782 0 2024-02-16T19:29:01.890 INFO:tasks.workunit.client.1.smithi117.stdout:8/331: symlink d4/d7/d37/d50/l68 0 2024-02-16T19:29:01.890 INFO:tasks.workunit.client.1.smithi117.stdout:8/332: read - d4/d1c/f60 zero size 2024-02-16T19:29:01.893 INFO:tasks.workunit.client.0.smithi022.stdout:1/461: dread d6/d8/d22/f1b [0,4194304] 0 2024-02-16T19:29:01.893 INFO:tasks.workunit.client.0.smithi022.stdout:2/276: dwrite d0/d2/d18/d19/d40/f44 [0,4194304] 0 2024-02-16T19:29:01.895 INFO:tasks.workunit.client.0.smithi022.stdout:2/277: fsync d0/d2/d18/d19/f47 0 2024-02-16T19:29:01.895 INFO:tasks.workunit.client.0.smithi022.stdout:4/341: dread d4/d6/dd/f1e [0,4194304] 0 2024-02-16T19:29:01.898 INFO:tasks.workunit.client.1.smithi117.stdout:6/343: truncate d5/d4b/d20/d2d/f3c 957916 0 2024-02-16T19:29:01.898 INFO:tasks.workunit.client.1.smithi117.stdout:6/344: dread - d5/db/f4a zero size 2024-02-16T19:29:01.900 INFO:tasks.workunit.client.0.smithi022.stdout:8/353: unlink d3/d12/d2b/d54/l7a 0 2024-02-16T19:29:01.903 INFO:tasks.workunit.client.0.smithi022.stdout:3/348: creat d7/d12/d18/dd/d24/d58/f8b x:0 0 0 2024-02-16T19:29:01.904 INFO:tasks.workunit.client.1.smithi117.stdout:1/269: mkdir d2/d19/d59 0 2024-02-16T19:29:01.906 INFO:tasks.workunit.client.1.smithi117.stdout:3/399: symlink d1/d67/l81 0 2024-02-16T19:29:01.908 INFO:tasks.workunit.client.0.smithi022.stdout:6/370: dwrite d1/d8/da/f14 [0,4194304] 0 2024-02-16T19:29:01.909 INFO:tasks.workunit.client.0.smithi022.stdout:0/463: rename d10/d11/d50/d65/c84 to d10/d15/d23/d38/d5a/d66/c99 0 2024-02-16T19:29:01.910 INFO:tasks.workunit.client.0.smithi022.stdout:4/342: dwrite d4/d6/dd/d22/f51 [0,4194304] 0 2024-02-16T19:29:01.910 INFO:tasks.workunit.client.0.smithi022.stdout:0/464: chown d10/d22/l21 12067 1 2024-02-16T19:29:01.911 INFO:tasks.workunit.client.1.smithi117.stdout:4/343: sync 2024-02-16T19:29:01.913 INFO:tasks.workunit.client.1.smithi117.stdout:5/430: rename d2/d3/d21/d49/d41 to d2/d9/d3c/d24/d50/d52/d74/d8c 0 2024-02-16T19:29:01.915 INFO:tasks.workunit.client.0.smithi022.stdout:0/465: read d10/d22/d2c/f72 [1610697,128053] 0 2024-02-16T19:29:01.917 INFO:tasks.workunit.client.0.smithi022.stdout:0/466: truncate d10/d11/d27/d3f/f8a 366273 0 2024-02-16T19:29:01.917 INFO:tasks.workunit.client.0.smithi022.stdout:0/467: chown f7 1 1 2024-02-16T19:29:01.920 INFO:tasks.workunit.client.1.smithi117.stdout:7/322: link d4/f2b d4/d1c/d24/d60/d79/f7e 0 2024-02-16T19:29:01.921 INFO:tasks.workunit.client.1.smithi117.stdout:7/323: chown d4/d1c/d24/d60/d79/f7a 63358 1 2024-02-16T19:29:01.922 INFO:tasks.workunit.client.1.smithi117.stdout:9/281: mkdir dd/d47/d5b 0 2024-02-16T19:29:01.924 INFO:tasks.workunit.client.1.smithi117.stdout:2/280: creat d0/d3/d40/f55 x:0 0 0 2024-02-16T19:29:01.925 INFO:tasks.workunit.client.1.smithi117.stdout:2/281: write d0/f19 [1074957,115359] 0 2024-02-16T19:29:01.929 INFO:tasks.workunit.client.1.smithi117.stdout:2/282: dread d0/d3/d22/f28 [4194304,4194304] 0 2024-02-16T19:29:01.930 INFO:tasks.workunit.client.1.smithi117.stdout:2/283: write d0/d13/f15 [1467677,27269] 0 2024-02-16T19:29:01.933 INFO:tasks.workunit.client.1.smithi117.stdout:3/400: creat d1/d1b/d1e/d29/f82 x:0 0 0 2024-02-16T19:29:01.933 INFO:tasks.workunit.client.1.smithi117.stdout:2/284: dread d0/f1e [0,4194304] 0 2024-02-16T19:29:01.934 INFO:tasks.workunit.client.0.smithi022.stdout:9/403: symlink db/d14/d16/d5d/d65/d6b/d89/l8a 0 2024-02-16T19:29:01.934 INFO:tasks.workunit.client.0.smithi022.stdout:9/404: fsync db/f84 0 2024-02-16T19:29:01.936 INFO:tasks.workunit.client.1.smithi117.stdout:4/344: rename d0/da/d21/d3d/d28/d2f/d37/l3a to d0/da/d21/d3d/d28/d64/l6e 0 2024-02-16T19:29:01.939 INFO:tasks.workunit.client.0.smithi022.stdout:2/278: unlink d0/d2/d18/f1a 0 2024-02-16T19:29:01.939 INFO:tasks.workunit.client.1.smithi117.stdout:0/380: sync 2024-02-16T19:29:01.939 INFO:tasks.workunit.client.1.smithi117.stdout:0/381: chown d0/f1 3713253 1 2024-02-16T19:29:01.941 INFO:tasks.workunit.client.0.smithi022.stdout:1/462: mknod d6/d8/d22/d16/d17/d26/c87 0 2024-02-16T19:29:01.948 INFO:tasks.workunit.client.0.smithi022.stdout:3/349: mknod d7/d12/d18/dd/d3c/c8c 0 2024-02-16T19:29:01.948 INFO:tasks.workunit.client.0.smithi022.stdout:3/350: read - d7/d12/d18/f42 zero size 2024-02-16T19:29:01.953 INFO:tasks.workunit.client.1.smithi117.stdout:6/345: mkdir d5/d4b/d71 0 2024-02-16T19:29:01.954 INFO:tasks.workunit.client.1.smithi117.stdout:6/346: write d5/d4b/d44/d52/f6c [823937,62441] 0 2024-02-16T19:29:01.955 INFO:tasks.workunit.client.0.smithi022.stdout:6/371: symlink d1/d8/da/d29/l89 0 2024-02-16T19:29:01.955 INFO:tasks.workunit.client.0.smithi022.stdout:6/372: stat d1/d8/d23/l79 0 2024-02-16T19:29:01.956 INFO:tasks.workunit.client.1.smithi117.stdout:1/270: symlink d2/d19/d59/l5a 0 2024-02-16T19:29:01.956 INFO:tasks.workunit.client.1.smithi117.stdout:1/271: fsync d2/d4/f47 0 2024-02-16T19:29:01.957 INFO:tasks.workunit.client.1.smithi117.stdout:1/272: write d2/d18/d20/f36 [3668499,7414] 0 2024-02-16T19:29:01.958 INFO:tasks.workunit.client.1.smithi117.stdout:1/273: chown d2/d18/d20/l24 2070464 1 2024-02-16T19:29:01.963 INFO:tasks.workunit.client.1.smithi117.stdout:9/282: rename dd/d45/c59 to dd/d24/c5c 0 2024-02-16T19:29:01.968 INFO:tasks.workunit.client.0.smithi022.stdout:5/367: dread dd/d1f/d48/d21/d2f/d40/d49/f4c [0,4194304] 0 2024-02-16T19:29:01.970 INFO:tasks.workunit.client.1.smithi117.stdout:0/382: mknod d0/c90 0 2024-02-16T19:29:01.970 INFO:tasks.workunit.client.0.smithi022.stdout:9/405: truncate db/d14/d33/f4f 1207474 0 2024-02-16T19:29:01.971 INFO:tasks.workunit.client.1.smithi117.stdout:8/333: link d4/d1c/l28 d4/d7/d37/d3a/d65/l69 0 2024-02-16T19:29:01.971 INFO:tasks.workunit.client.1.smithi117.stdout:8/334: dread - d4/d7/d37/d3a/d43/f55 zero size 2024-02-16T19:29:01.975 INFO:tasks.workunit.client.0.smithi022.stdout:2/279: rmdir d0/d2/d18/d1f 39 2024-02-16T19:29:01.978 INFO:tasks.workunit.client.1.smithi117.stdout:6/347: mkdir d5/d1d/d72 0 2024-02-16T19:29:01.979 INFO:tasks.workunit.client.0.smithi022.stdout:1/463: dread d6/d8/d22/f3d [0,4194304] 0 2024-02-16T19:29:01.979 INFO:tasks.workunit.client.1.smithi117.stdout:7/324: truncate d4/d1c/d39/f54 2779083 0 2024-02-16T19:29:01.980 INFO:tasks.workunit.client.0.smithi022.stdout:8/354: symlink d3/d12/d2b/d46/l7f 0 2024-02-16T19:29:01.981 INFO:tasks.workunit.client.0.smithi022.stdout:3/351: fdatasync d7/d12/d18/dd/d3c/f49 0 2024-02-16T19:29:01.982 INFO:tasks.workunit.client.0.smithi022.stdout:3/352: write d7/d21/f52 [962329,34867] 0 2024-02-16T19:29:01.983 INFO:tasks.workunit.client.1.smithi117.stdout:7/325: dread d4/f2b [0,4194304] 0 2024-02-16T19:29:01.983 INFO:tasks.workunit.client.0.smithi022.stdout:4/343: mknod d4/d6/dd/df/d10/d24/d4b/c6c 0 2024-02-16T19:29:01.984 INFO:tasks.workunit.client.1.smithi117.stdout:7/326: readlink d4/d1c/d24/l76 0 2024-02-16T19:29:01.984 INFO:tasks.workunit.client.1.smithi117.stdout:3/401: mkdir d1/d83 0 2024-02-16T19:29:01.986 INFO:tasks.workunit.client.0.smithi022.stdout:0/468: rename d10/d15/d23/c3d to d10/d11/d27/d3f/d58/c9a 0 2024-02-16T19:29:01.987 INFO:tasks.workunit.client.0.smithi022.stdout:5/368: fdatasync dd/d1f/d48/d21/d2f/d52/d5c/d61/d6a/f70 0 2024-02-16T19:29:01.988 INFO:tasks.workunit.client.1.smithi117.stdout:9/283: symlink dd/d45/d31/l5d 0 2024-02-16T19:29:01.990 INFO:tasks.workunit.client.0.smithi022.stdout:9/406: mkdir db/d14/d16/d27/d8b 0 2024-02-16T19:29:01.991 INFO:tasks.workunit.client.1.smithi117.stdout:4/345: symlink d0/da/d21/d3d/d2d/d6d/l6f 0 2024-02-16T19:29:01.993 INFO:tasks.workunit.client.0.smithi022.stdout:2/280: mknod d0/d32/c5c 0 2024-02-16T19:29:01.994 INFO:tasks.workunit.client.0.smithi022.stdout:0/469: dwrite d10/d15/d23/d8f/f95 [0,4194304] 0 2024-02-16T19:29:01.994 INFO:tasks.workunit.client.0.smithi022.stdout:2/281: stat d0/d32/d3b 0 2024-02-16T19:29:01.996 INFO:tasks.workunit.client.1.smithi117.stdout:5/431: creat d2/d9/d3c/f8d x:0 0 0 2024-02-16T19:29:01.997 INFO:tasks.workunit.client.1.smithi117.stdout:4/346: dwrite d0/da/d21/d3d/d28/f63 [0,4194304] 0 2024-02-16T19:29:01.998 INFO:tasks.workunit.client.1.smithi117.stdout:0/383: chown d0/d9/d14/f1b 2 1 2024-02-16T19:29:01.999 INFO:tasks.workunit.client.0.smithi022.stdout:1/464: symlink d6/d8/d22/d16/d17/d63/d6f/l88 0 2024-02-16T19:29:01.999 INFO:tasks.workunit.client.1.smithi117.stdout:4/347: read - d0/f5c zero size 2024-02-16T19:29:02.003 INFO:tasks.workunit.client.1.smithi117.stdout:4/348: read d0/da/f33 [6399006,108802] 0 2024-02-16T19:29:02.005 INFO:tasks.workunit.client.1.smithi117.stdout:6/348: creat d5/d1d/d28/d32/f73 x:0 0 0 2024-02-16T19:29:02.007 INFO:tasks.workunit.client.1.smithi117.stdout:5/432: dwrite d2/d9/f7c [0,4194304] 0 2024-02-16T19:29:02.012 INFO:tasks.workunit.client.1.smithi117.stdout:7/327: symlink d4/d1c/d24/d60/l7f 0 2024-02-16T19:29:02.012 INFO:tasks.workunit.client.0.smithi022.stdout:0/470: dread d10/d22/f2a [0,4194304] 0 2024-02-16T19:29:02.013 INFO:tasks.workunit.client.0.smithi022.stdout:0/471: truncate d10/d11/d27/d3f/f5b 2520356 0 2024-02-16T19:29:02.014 INFO:tasks.workunit.client.1.smithi117.stdout:3/402: symlink d1/dd/d32/l84 0 2024-02-16T19:29:02.014 INFO:tasks.workunit.client.1.smithi117.stdout:3/403: chown d1/dd/c15 1 1 2024-02-16T19:29:02.022 INFO:tasks.workunit.client.0.smithi022.stdout:0/472: dwrite d10/d15/d23/d8f/f95 [0,4194304] 0 2024-02-16T19:29:02.024 INFO:tasks.workunit.client.0.smithi022.stdout:0/473: write d10/d11/d50/d65/f68 [1938963,12859] 0 2024-02-16T19:29:02.026 INFO:tasks.workunit.client.0.smithi022.stdout:9/407: symlink db/d14/d16/d27/d2e/l8c 0 2024-02-16T19:29:02.026 INFO:tasks.workunit.client.0.smithi022.stdout:3/353: write d7/d12/f14 [936443,126999] 0 2024-02-16T19:29:02.027 INFO:tasks.workunit.client.0.smithi022.stdout:0/474: write d10/d15/d23/d8f/f95 [1303028,49095] 0 2024-02-16T19:29:02.028 INFO:tasks.workunit.client.1.smithi117.stdout:8/335: dwrite d4/f3c [0,4194304] 0 2024-02-16T19:29:02.035 INFO:tasks.workunit.client.1.smithi117.stdout:0/384: dread d0/d9/d14/d23/d33/f5c [0,4194304] 0 2024-02-16T19:29:02.043 INFO:tasks.workunit.client.1.smithi117.stdout:6/349: symlink d5/d1d/d28/l74 0 2024-02-16T19:29:02.046 INFO:tasks.workunit.client.1.smithi117.stdout:1/274: creat d2/f5b x:0 0 0 2024-02-16T19:29:02.046 INFO:tasks.workunit.client.1.smithi117.stdout:1/275: chown d2/f53 21737027 1 2024-02-16T19:29:02.046 INFO:tasks.workunit.client.0.smithi022.stdout:0/475: dread d10/d11/d27/d3f/f7a [0,4194304] 0 2024-02-16T19:29:02.047 INFO:tasks.workunit.client.1.smithi117.stdout:1/276: chown d2/d4/c1c 13729274 1 2024-02-16T19:29:02.047 INFO:tasks.workunit.client.0.smithi022.stdout:4/344: truncate d4/f23 500077 0 2024-02-16T19:29:02.047 INFO:tasks.workunit.client.1.smithi117.stdout:1/277: write d2/f53 [949425,53216] 0 2024-02-16T19:29:02.048 INFO:tasks.workunit.client.1.smithi117.stdout:1/278: chown d2/d1d/d1f/c3d 43835 1 2024-02-16T19:29:02.053 INFO:tasks.workunit.client.1.smithi117.stdout:5/433: mkdir d2/d5d/d8e 0 2024-02-16T19:29:02.053 INFO:tasks.workunit.client.1.smithi117.stdout:9/284: dread fa [0,4194304] 0 2024-02-16T19:29:02.053 INFO:tasks.workunit.client.0.smithi022.stdout:1/465: mknod d6/d8/d22/d16/d17/c89 0 2024-02-16T19:29:02.054 INFO:tasks.workunit.client.1.smithi117.stdout:9/285: fdatasync dd/d23/d2e/f4e 0 2024-02-16T19:29:02.054 INFO:tasks.workunit.client.1.smithi117.stdout:3/404: mkdir d1/dd/d5c/d85 0 2024-02-16T19:29:02.055 INFO:tasks.workunit.client.0.smithi022.stdout:9/408: mknod db/d14/d16/d5d/d65/c8d 0 2024-02-16T19:29:02.056 INFO:tasks.workunit.client.0.smithi022.stdout:9/409: chown db/d14/d16/l5b 127596 1 2024-02-16T19:29:02.057 INFO:tasks.workunit.client.1.smithi117.stdout:2/285: getdents d0/d13 0 2024-02-16T19:29:02.062 INFO:tasks.workunit.client.0.smithi022.stdout:2/282: truncate d0/d2/d2e/f51 1666708 0 2024-02-16T19:29:02.063 INFO:tasks.workunit.client.0.smithi022.stdout:5/369: link dd/d1f/c2a dd/d1f/d48/d21/d2f/d52/d5c/c79 0 2024-02-16T19:29:02.063 INFO:tasks.workunit.client.0.smithi022.stdout:2/283: chown d0/d2/d18/d19/f2f 0 1 2024-02-16T19:29:02.064 INFO:tasks.workunit.client.0.smithi022.stdout:5/370: write dd/d1f/d48/f1c [1907496,11036] 0 2024-02-16T19:29:02.069 INFO:tasks.workunit.client.1.smithi117.stdout:3/405: dread d1/d30/d52/d56/f3f [0,4194304] 0 2024-02-16T19:29:02.069 INFO:tasks.workunit.client.0.smithi022.stdout:1/466: creat d6/d8/d22/d47/d54/f8a x:0 0 0 2024-02-16T19:29:02.070 INFO:tasks.workunit.client.0.smithi022.stdout:1/467: write d6/d8/f2d [3622259,35155] 0 2024-02-16T19:29:02.071 INFO:tasks.workunit.client.1.smithi117.stdout:0/385: creat d0/d9/d2c/f91 x:0 0 0 2024-02-16T19:29:02.073 INFO:tasks.workunit.client.1.smithi117.stdout:1/279: mknod d2/d18/d20/c5c 0 2024-02-16T19:29:02.074 INFO:tasks.workunit.client.1.smithi117.stdout:1/280: dread d2/d1d/f39 [0,4194304] 0 2024-02-16T19:29:02.076 INFO:tasks.workunit.client.1.smithi117.stdout:5/434: rmdir d2/d5d 39 2024-02-16T19:29:02.081 INFO:tasks.workunit.client.0.smithi022.stdout:5/371: mkdir dd/d1f/d4a/d7a 0 2024-02-16T19:29:02.083 INFO:tasks.workunit.client.0.smithi022.stdout:5/372: write dd/d1f/d48/f1c [4545615,20543] 0 2024-02-16T19:29:02.085 INFO:tasks.workunit.client.0.smithi022.stdout:5/373: write dd/d1f/d48/f1c [2687861,11949] 0 2024-02-16T19:29:02.086 INFO:tasks.workunit.client.1.smithi117.stdout:2/286: rename d0/d29/d52 to d0/d29/d56 0 2024-02-16T19:29:02.087 INFO:tasks.workunit.client.0.smithi022.stdout:5/374: dread dd/d1f/f2b [0,4194304] 0 2024-02-16T19:29:02.088 INFO:tasks.workunit.client.1.smithi117.stdout:4/349: truncate d0/da/d21/d3d/d28/d64/f23 2966306 0 2024-02-16T19:29:02.088 INFO:tasks.workunit.client.0.smithi022.stdout:5/375: write dd/f11 [339448,120393] 0 2024-02-16T19:29:02.089 INFO:tasks.workunit.client.1.smithi117.stdout:4/350: fdatasync d0/da/d21/d3d/f5e 0 2024-02-16T19:29:02.092 INFO:tasks.workunit.client.0.smithi022.stdout:4/345: mkdir d4/d6/d14/d64/d66/d6d 0 2024-02-16T19:29:02.093 INFO:tasks.workunit.client.0.smithi022.stdout:4/346: readlink d4/d6/dd/l1a 0 2024-02-16T19:29:02.095 INFO:tasks.workunit.client.1.smithi117.stdout:3/406: creat d1/d1b/d1e/d37/f86 x:0 0 0 2024-02-16T19:29:02.100 INFO:tasks.workunit.client.1.smithi117.stdout:1/281: mknod d2/d1d/c5d 0 2024-02-16T19:29:02.102 INFO:tasks.workunit.client.0.smithi022.stdout:2/284: fsync d0/f20 0 2024-02-16T19:29:02.102 INFO:tasks.workunit.client.0.smithi022.stdout:2/285: stat d0/d2/d2e/l46 0 2024-02-16T19:29:02.103 INFO:tasks.workunit.client.1.smithi117.stdout:8/336: truncate d4/d12/f26 5655813 0 2024-02-16T19:29:02.103 INFO:tasks.workunit.client.0.smithi022.stdout:2/286: chown d0/d32/c35 220858127 1 2024-02-16T19:29:02.104 INFO:tasks.workunit.client.1.smithi117.stdout:5/435: mkdir d2/d9/d3c/d24/d50/d52/d74/d8f 0 2024-02-16T19:29:02.106 INFO:tasks.workunit.client.1.smithi117.stdout:9/286: mkdir dd/d47/d5e 0 2024-02-16T19:29:02.106 INFO:tasks.workunit.client.1.smithi117.stdout:1/282: dwrite d2/f57 [0,4194304] 0 2024-02-16T19:29:02.108 INFO:tasks.workunit.client.0.smithi022.stdout:9/410: truncate db/d14/d19/d1a/d67/f81 3397277 0 2024-02-16T19:29:02.111 INFO:tasks.workunit.client.1.smithi117.stdout:2/287: unlink d0/d3/l20 0 2024-02-16T19:29:02.114 INFO:tasks.workunit.client.1.smithi117.stdout:5/436: dread d2/d9/f40 [0,4194304] 0 2024-02-16T19:29:02.114 INFO:tasks.workunit.client.1.smithi117.stdout:6/350: creat d5/d4b/d44/f75 x:0 0 0 2024-02-16T19:29:02.115 INFO:tasks.workunit.client.1.smithi117.stdout:7/328: sync 2024-02-16T19:29:02.116 INFO:tasks.workunit.client.1.smithi117.stdout:3/407: rename d1/d1b/d1e/d5e/c78 to d1/d1b/d1e/d37/c87 0 2024-02-16T19:29:02.121 INFO:tasks.workunit.client.1.smithi117.stdout:6/351: dwrite d5/d4b/d20/f21 [0,4194304] 0 2024-02-16T19:29:02.124 INFO:tasks.workunit.client.0.smithi022.stdout:0/476: getdents d10/d22 0 2024-02-16T19:29:02.124 INFO:tasks.workunit.client.1.smithi117.stdout:3/408: dwrite d1/d1b/d1e/f23 [0,4194304] 0 2024-02-16T19:29:02.126 INFO:tasks.workunit.client.0.smithi022.stdout:5/376: getdents dd/d1f/d48/d21/d2f/d52/d77 0 2024-02-16T19:29:02.129 INFO:tasks.workunit.client.1.smithi117.stdout:4/351: write d0/da/d21/f27 [2167765,4382] 0 2024-02-16T19:29:02.131 INFO:tasks.workunit.client.1.smithi117.stdout:1/283: fsync d2/d1d/d1f/f44 0 2024-02-16T19:29:02.132 INFO:tasks.workunit.client.1.smithi117.stdout:8/337: dread d4/d12/d31/f36 [0,4194304] 0 2024-02-16T19:29:02.133 INFO:tasks.workunit.client.1.smithi117.stdout:6/352: dwrite d5/d1d/d28/d32/f73 [0,4194304] 0 2024-02-16T19:29:02.133 INFO:tasks.workunit.client.1.smithi117.stdout:3/409: dread d1/d1b/d1e/d5e/f66 [0,4194304] 0 2024-02-16T19:29:02.134 INFO:tasks.workunit.client.1.smithi117.stdout:1/284: write d2/d1d/f25 [2180416,14402] 0 2024-02-16T19:29:02.135 INFO:tasks.workunit.client.0.smithi022.stdout:1/468: truncate d6/d8/d22/d3a/f44 3179081 0 2024-02-16T19:29:02.136 INFO:tasks.workunit.client.0.smithi022.stdout:1/469: chown d6/f9 6896 1 2024-02-16T19:29:02.136 INFO:tasks.workunit.client.0.smithi022.stdout:3/354: getdents d7/d12/d18/dd/d24 0 2024-02-16T19:29:02.136 INFO:tasks.workunit.client.1.smithi117.stdout:2/288: unlink d0/d3/d22/c33 0 2024-02-16T19:29:02.141 INFO:tasks.workunit.client.1.smithi117.stdout:0/386: creat d0/d9/d14/f92 x:0 0 0 2024-02-16T19:29:02.141 INFO:tasks.workunit.client.0.smithi022.stdout:2/287: mkdir d0/d5d 0 2024-02-16T19:29:02.141 INFO:tasks.workunit.client.0.smithi022.stdout:9/411: dread db/d18/f73 [0,4194304] 0 2024-02-16T19:29:02.142 INFO:tasks.workunit.client.0.smithi022.stdout:2/288: chown d0/d2/d18/f34 4278 1 2024-02-16T19:29:02.143 INFO:tasks.workunit.client.1.smithi117.stdout:8/338: dwrite d4/d7/d37/d3a/f61 [0,4194304] 0 2024-02-16T19:29:02.143 INFO:tasks.workunit.client.0.smithi022.stdout:0/477: rmdir d10/d7d 39 2024-02-16T19:29:02.144 INFO:tasks.workunit.client.0.smithi022.stdout:0/478: chown f7 1067 1 2024-02-16T19:29:02.146 INFO:tasks.workunit.client.1.smithi117.stdout:5/437: mkdir d2/d9/d3c/d24/d90 0 2024-02-16T19:29:02.147 INFO:tasks.workunit.client.0.smithi022.stdout:7/374: sync 2024-02-16T19:29:02.150 INFO:tasks.workunit.client.0.smithi022.stdout:4/347: rmdir d4/d6/d14/d64/d4e/d54/d58 0 2024-02-16T19:29:02.152 INFO:tasks.workunit.client.0.smithi022.stdout:1/470: mkdir d6/d8/d22/d16/d17/d63/d6f/d8b 0 2024-02-16T19:29:02.155 INFO:tasks.workunit.client.1.smithi117.stdout:7/329: rename d4/d4a/l43 to d4/d1c/d24/d60/l80 0 2024-02-16T19:29:02.156 INFO:tasks.workunit.client.0.smithi022.stdout:3/355: mknod d7/d12/d18/dd/d24/d58/d6b/c8d 0 2024-02-16T19:29:02.159 INFO:tasks.workunit.client.0.smithi022.stdout:9/412: dread - db/f7b zero size 2024-02-16T19:29:02.163 INFO:tasks.workunit.client.0.smithi022.stdout:0/479: symlink d10/d11/l9b 0 2024-02-16T19:29:02.164 INFO:tasks.workunit.client.1.smithi117.stdout:9/287: mkdir dd/d47/d5b/d5f 0 2024-02-16T19:29:02.164 INFO:tasks.workunit.client.0.smithi022.stdout:0/480: chown d10/d11/d27/d3f/d58/d60/d62/c35 62912 1 2024-02-16T19:29:02.165 INFO:tasks.workunit.client.0.smithi022.stdout:5/377: creat dd/d1f/d48/d21/d2f/d46/d5e/f7b x:0 0 0 2024-02-16T19:29:02.171 INFO:tasks.workunit.client.0.smithi022.stdout:0/481: dwrite d10/d15/d23/d8f/f95 [0,4194304] 0 2024-02-16T19:29:02.171 INFO:tasks.workunit.client.1.smithi117.stdout:6/353: symlink d5/db/l76 0 2024-02-16T19:29:02.172 INFO:tasks.workunit.client.0.smithi022.stdout:0/482: fdatasync d10/d11/d50/d65/f74 0 2024-02-16T19:29:02.173 INFO:tasks.workunit.client.0.smithi022.stdout:4/348: fdatasync d4/f40 0 2024-02-16T19:29:02.178 INFO:tasks.workunit.client.1.smithi117.stdout:1/285: creat d2/d1d/d1f/f5e x:0 0 0 2024-02-16T19:29:02.180 INFO:tasks.workunit.client.1.smithi117.stdout:4/352: mknod d0/da/d21/d3d/d2d/c70 0 2024-02-16T19:29:02.180 INFO:tasks.workunit.client.0.smithi022.stdout:4/349: dwrite d4/d6/dd/d22/f59 [4194304,4194304] 0 2024-02-16T19:29:02.182 INFO:tasks.workunit.client.0.smithi022.stdout:9/413: creat db/d14/d16/d27/d2e/f8e x:0 0 0 2024-02-16T19:29:02.183 INFO:tasks.workunit.client.0.smithi022.stdout:9/414: dread - db/d14/d33/f88 zero size 2024-02-16T19:29:02.184 INFO:tasks.workunit.client.1.smithi117.stdout:2/289: write d0/f12 [1979668,69370] 0 2024-02-16T19:29:02.185 INFO:tasks.workunit.client.1.smithi117.stdout:2/290: write d0/d13/d4d/f50 [131275,106177] 0 2024-02-16T19:29:02.187 INFO:tasks.workunit.client.1.smithi117.stdout:2/291: truncate d0/d3/d40/f55 463030 0 2024-02-16T19:29:02.192 INFO:tasks.workunit.client.0.smithi022.stdout:9/415: dwrite db/d14/d33/f88 [0,4194304] 0 2024-02-16T19:29:02.192 INFO:tasks.workunit.client.1.smithi117.stdout:2/292: dwrite d0/d29/f2d [0,4194304] 0 2024-02-16T19:29:02.194 INFO:tasks.workunit.client.0.smithi022.stdout:5/378: mknod dd/d1f/d2e/c7c 0 2024-02-16T19:29:02.195 INFO:tasks.workunit.client.0.smithi022.stdout:9/416: write db/d14/d16/d27/f68 [968640,36560] 0 2024-02-16T19:29:02.196 INFO:tasks.workunit.client.1.smithi117.stdout:5/438: write d2/d4c/d54/f43 [1375039,129389] 0 2024-02-16T19:29:02.197 INFO:tasks.workunit.client.0.smithi022.stdout:5/379: dread dd/d1f/f2b [0,4194304] 0 2024-02-16T19:29:02.198 INFO:tasks.workunit.client.0.smithi022.stdout:6/373: sync 2024-02-16T19:29:02.198 INFO:tasks.workunit.client.0.smithi022.stdout:8/355: sync 2024-02-16T19:29:02.198 INFO:tasks.workunit.client.1.smithi117.stdout:5/439: write d2/d4c/d54/f6d [213779,64377] 0 2024-02-16T19:29:02.199 INFO:tasks.workunit.client.0.smithi022.stdout:5/380: truncate dd/f18 4573950 0 2024-02-16T19:29:02.200 INFO:tasks.workunit.client.0.smithi022.stdout:5/381: write dd/f1d [4351294,101321] 0 2024-02-16T19:29:02.201 INFO:tasks.workunit.client.1.smithi117.stdout:3/410: rename d1/d1b/d20/c5b to d1/dd/d5c/d85/c88 0 2024-02-16T19:29:02.201 INFO:tasks.workunit.client.0.smithi022.stdout:7/375: mknod d9/da/d4e/c78 0 2024-02-16T19:29:02.202 INFO:tasks.workunit.client.0.smithi022.stdout:0/483: symlink d10/d22/d2c/l9c 0 2024-02-16T19:29:02.202 INFO:tasks.workunit.client.1.smithi117.stdout:5/440: chown d2/d9/d3c/d24/d50/d52/l19 8428 1 2024-02-16T19:29:02.203 INFO:tasks.workunit.client.0.smithi022.stdout:9/417: dread db/d14/d16/d5d/f7e [0,4194304] 0 2024-02-16T19:29:02.203 INFO:tasks.workunit.client.0.smithi022.stdout:0/484: read - d10/d11/d27/d77/f8e zero size 2024-02-16T19:29:02.203 INFO:tasks.workunit.client.1.smithi117.stdout:5/441: write d2/d3/f65 [713727,69176] 0 2024-02-16T19:29:02.206 INFO:tasks.workunit.client.0.smithi022.stdout:3/356: link d7/d12/d18/dd/d24/f63 d7/d12/d18/dd/d24/d58/d70/f8e 0 2024-02-16T19:29:02.206 INFO:tasks.workunit.client.0.smithi022.stdout:1/471: creat d6/d8/d22/d16/d4f/f8c x:0 0 0 2024-02-16T19:29:02.208 INFO:tasks.workunit.client.0.smithi022.stdout:0/485: write d10/d15/f39 [4033144,98899] 0 2024-02-16T19:29:02.210 INFO:tasks.workunit.client.0.smithi022.stdout:1/472: write d6/d8/d22/d16/d17/d1d/f21 [4299033,32519] 0 2024-02-16T19:29:02.210 INFO:tasks.workunit.client.0.smithi022.stdout:7/376: write d9/da/f62 [941527,28777] 0 2024-02-16T19:29:02.213 INFO:tasks.workunit.client.0.smithi022.stdout:2/289: unlink d0/d2/d18/d1f/c2b 0 2024-02-16T19:29:02.213 INFO:tasks.workunit.client.0.smithi022.stdout:4/350: chown d4/d6/f16 101374 1 2024-02-16T19:29:02.213 INFO:tasks.workunit.client.1.smithi117.stdout:4/353: symlink d0/da/d21/l71 0 2024-02-16T19:29:02.214 INFO:tasks.workunit.client.1.smithi117.stdout:4/354: dread - d0/da/d21/d3d/d28/d2f/f62 zero size 2024-02-16T19:29:02.217 INFO:tasks.workunit.client.0.smithi022.stdout:7/377: dread d9/da/d2c/d46/f6a [0,4194304] 0 2024-02-16T19:29:02.218 INFO:tasks.workunit.client.1.smithi117.stdout:0/387: mknod d0/d9/d14/c93 0 2024-02-16T19:29:02.221 INFO:tasks.workunit.client.1.smithi117.stdout:8/339: mkdir d4/d7/d6a 0 2024-02-16T19:29:02.222 INFO:tasks.workunit.client.1.smithi117.stdout:2/293: fsync d0/d3/d10/f17 0 2024-02-16T19:29:02.223 INFO:tasks.workunit.client.0.smithi022.stdout:6/374: creat d1/d8/da/d17/d85/d5b/f8a x:0 0 0 2024-02-16T19:29:02.224 INFO:tasks.workunit.client.1.smithi117.stdout:0/388: dwrite d0/d9/d22/d27/d31/f38 [0,4194304] 0 2024-02-16T19:29:02.226 INFO:tasks.workunit.client.1.smithi117.stdout:1/286: rename d2/d18/d20/f2b to d2/d18/f5f 0 2024-02-16T19:29:02.227 INFO:tasks.workunit.client.0.smithi022.stdout:5/382: creat dd/d1f/d48/d21/d2f/d52/d5c/f7d x:0 0 0 2024-02-16T19:29:02.230 INFO:tasks.workunit.client.1.smithi117.stdout:2/294: dwrite d0/d13/f15 [0,4194304] 0 2024-02-16T19:29:02.230 INFO:tasks.workunit.client.1.smithi117.stdout:6/354: write d5/d1d/d28/f2b [3579881,100666] 0 2024-02-16T19:29:02.232 INFO:tasks.workunit.client.1.smithi117.stdout:2/295: truncate d0/d3/f31 1231259 0 2024-02-16T19:29:02.232 INFO:tasks.workunit.client.1.smithi117.stdout:5/442: creat d2/d9/d3c/d24/d50/d52/f91 x:0 0 0 2024-02-16T19:29:02.233 INFO:tasks.workunit.client.1.smithi117.stdout:6/355: write d5/d4b/d44/f75 [991937,44281] 0 2024-02-16T19:29:02.234 INFO:tasks.workunit.client.0.smithi022.stdout:1/473: fsync d6/f6b 0 2024-02-16T19:29:02.235 INFO:tasks.workunit.client.0.smithi022.stdout:1/474: chown d6/l27 252635690 1 2024-02-16T19:29:02.238 INFO:tasks.workunit.client.0.smithi022.stdout:0/486: creat d10/d15/d23/d38/d5a/d66/f9d x:0 0 0 2024-02-16T19:29:02.238 INFO:tasks.workunit.client.0.smithi022.stdout:2/290: creat d0/d2/d18/d1f/f5e x:0 0 0 2024-02-16T19:29:02.239 INFO:tasks.workunit.client.1.smithi117.stdout:4/355: chown d0/da/d21/d3d/d28/l32 13 1 2024-02-16T19:29:02.240 INFO:tasks.workunit.client.1.smithi117.stdout:2/296: dwrite d0/d3/f31 [0,4194304] 0 2024-02-16T19:29:02.242 INFO:tasks.workunit.client.0.smithi022.stdout:4/351: symlink d4/d6/d14/d64/d4e/d54/d5a/d63/l6e 0 2024-02-16T19:29:02.247 INFO:tasks.workunit.client.1.smithi117.stdout:8/340: creat d4/d7/d37/d3a/d43/f6b x:0 0 0 2024-02-16T19:29:02.247 INFO:tasks.workunit.client.0.smithi022.stdout:7/378: symlink d9/da/d1f/d40/l79 0 2024-02-16T19:29:02.251 INFO:tasks.workunit.client.0.smithi022.stdout:8/356: truncate d3/d12/f51 1040207 0 2024-02-16T19:29:02.252 INFO:tasks.workunit.client.0.smithi022.stdout:0/487: dwrite d10/d15/d23/d38/d5a/d66/f9d [0,4194304] 0 2024-02-16T19:29:02.256 INFO:tasks.workunit.client.0.smithi022.stdout:2/291: dwrite d0/d2/d18/d19/d40/f44 [0,4194304] 0 2024-02-16T19:29:02.256 INFO:tasks.workunit.client.0.smithi022.stdout:9/418: creat db/d14/d16/d87/f8f x:0 0 0 2024-02-16T19:29:02.259 INFO:tasks.workunit.client.1.smithi117.stdout:1/287: truncate d2/d1d/d1f/f23 4064921 0 2024-02-16T19:29:02.259 INFO:tasks.workunit.client.1.smithi117.stdout:7/330: link d4/d1c/f2e d4/d1c/d24/d5f/f81 0 2024-02-16T19:29:02.260 INFO:tasks.workunit.client.0.smithi022.stdout:0/488: write d10/d11/d27/d3f/f57 [2763369,49476] 0 2024-02-16T19:29:02.261 INFO:tasks.workunit.client.0.smithi022.stdout:2/292: read - d0/d32/f52 zero size 2024-02-16T19:29:02.264 INFO:tasks.workunit.client.0.smithi022.stdout:7/379: dwrite d9/d1b/f27 [4194304,4194304] 0 2024-02-16T19:29:02.265 INFO:tasks.workunit.client.0.smithi022.stdout:7/380: stat d9/da/f62 0 2024-02-16T19:29:02.267 INFO:tasks.workunit.client.1.smithi117.stdout:7/331: dwrite d4/d1c/d39/d46/d47/f73 [0,4194304] 0 2024-02-16T19:29:02.268 INFO:tasks.workunit.client.0.smithi022.stdout:5/383: dread dd/d1f/d48/f1b [0,4194304] 0 2024-02-16T19:29:02.269 INFO:tasks.workunit.client.1.smithi117.stdout:9/288: creat dd/d47/f60 x:0 0 0 2024-02-16T19:29:02.270 INFO:tasks.workunit.client.1.smithi117.stdout:9/289: readlink dd/d45/d31/l34 0 2024-02-16T19:29:02.270 INFO:tasks.workunit.client.0.smithi022.stdout:5/384: dread - dd/d1f/d48/f43 zero size 2024-02-16T19:29:02.271 INFO:tasks.workunit.client.1.smithi117.stdout:6/356: readlink d5/d4b/d20/d69/l6b 0 2024-02-16T19:29:02.277 INFO:tasks.workunit.client.0.smithi022.stdout:9/419: dread db/d14/d16/d27/f78 [0,4194304] 0 2024-02-16T19:29:02.282 INFO:tasks.workunit.client.1.smithi117.stdout:3/411: sync 2024-02-16T19:29:02.283 INFO:tasks.workunit.client.1.smithi117.stdout:3/412: readlink d1/d67/l81 0 2024-02-16T19:29:02.283 INFO:tasks.workunit.client.0.smithi022.stdout:6/375: mkdir d1/d8/da/d17/d8b 0 2024-02-16T19:29:02.284 INFO:tasks.workunit.client.1.smithi117.stdout:3/413: fdatasync d1/d1b/f2d 0 2024-02-16T19:29:02.284 INFO:tasks.workunit.client.0.smithi022.stdout:5/385: dwrite dd/d1f/d48/d21/d2f/d40/f3b [0,4194304] 0 2024-02-16T19:29:02.285 INFO:tasks.workunit.client.0.smithi022.stdout:8/357: creat d3/d12/d17/d64/f80 x:0 0 0 2024-02-16T19:29:02.287 INFO:tasks.workunit.client.0.smithi022.stdout:0/489: dwrite d10/d15/d23/d38/d5a/f5d [0,4194304] 0 2024-02-16T19:29:02.288 INFO:tasks.workunit.client.1.smithi117.stdout:1/288: rename d2/d18 to d2/d1d/d60 0 2024-02-16T19:29:02.289 INFO:tasks.workunit.client.0.smithi022.stdout:1/475: dread d6/f19 [0,4194304] 0 2024-02-16T19:29:02.290 INFO:tasks.workunit.client.1.smithi117.stdout:1/289: truncate d2/d1d/d60/d20/d49/f4a 224634 0 2024-02-16T19:29:02.295 INFO:tasks.workunit.client.0.smithi022.stdout:6/376: truncate d1/d8/d23/f68 1110026 0 2024-02-16T19:29:02.297 INFO:tasks.workunit.client.0.smithi022.stdout:7/381: mkdir d9/da/d4e/d7a 0 2024-02-16T19:29:02.298 INFO:tasks.workunit.client.1.smithi117.stdout:1/290: dwrite d2/f57 [0,4194304] 0 2024-02-16T19:29:02.300 INFO:tasks.workunit.client.1.smithi117.stdout:1/291: truncate d2/d1d/f3a 224813 0 2024-02-16T19:29:02.301 INFO:tasks.workunit.client.0.smithi022.stdout:4/352: symlink d4/d6/dd/df/d10/l6f 0 2024-02-16T19:29:02.302 INFO:tasks.workunit.client.0.smithi022.stdout:9/420: dwrite db/d14/d16/d87/f8f [0,4194304] 0 2024-02-16T19:29:02.306 INFO:tasks.workunit.client.0.smithi022.stdout:5/386: dwrite dd/d1f/d48/d21/f24 [0,4194304] 0 2024-02-16T19:29:02.309 INFO:tasks.workunit.client.0.smithi022.stdout:1/476: dread d6/d8/d22/d16/d17/d1d/f33 [0,4194304] 0 2024-02-16T19:29:02.309 INFO:tasks.workunit.client.0.smithi022.stdout:5/387: chown dd/d1f/d48/d21/d2f 17299 1 2024-02-16T19:29:02.311 INFO:tasks.workunit.client.1.smithi117.stdout:7/332: dread d4/d1c/f3c [0,4194304] 0 2024-02-16T19:29:02.311 INFO:tasks.workunit.client.0.smithi022.stdout:8/358: fdatasync d3/f55 0 2024-02-16T19:29:02.313 INFO:tasks.workunit.client.0.smithi022.stdout:8/359: dread - d3/d12/d17/d64/f73 zero size 2024-02-16T19:29:02.313 INFO:tasks.workunit.client.1.smithi117.stdout:0/389: write d0/d9/d14/d1d/d60/d76/d7b/d8d/f32 [118751,116338] 0 2024-02-16T19:29:02.314 INFO:tasks.workunit.client.0.smithi022.stdout:8/360: fdatasync d3/d12/d17/d64/f80 0 2024-02-16T19:29:02.316 INFO:tasks.workunit.client.1.smithi117.stdout:8/341: truncate d4/d12/f26 3977290 0 2024-02-16T19:29:02.316 INFO:tasks.workunit.client.1.smithi117.stdout:0/390: read d0/f53 [35622,61737] 0 2024-02-16T19:29:02.320 INFO:tasks.workunit.client.1.smithi117.stdout:9/290: dwrite dd/d47/d48/f55 [0,4194304] 0 2024-02-16T19:29:02.320 INFO:tasks.workunit.client.1.smithi117.stdout:3/414: mkdir d1/d31/d89 0 2024-02-16T19:29:02.322 INFO:tasks.workunit.client.0.smithi022.stdout:0/490: rename d10/d22/d49/f7b to d10/d15/d23/d38/f9e 0 2024-02-16T19:29:02.323 INFO:tasks.workunit.client.1.smithi117.stdout:2/297: dwrite d0/fd [0,4194304] 0 2024-02-16T19:29:02.323 INFO:tasks.workunit.client.1.smithi117.stdout:2/298: chown d0 2234488 1 2024-02-16T19:29:02.325 INFO:tasks.workunit.client.1.smithi117.stdout:5/443: creat d2/d3/d21/f92 x:0 0 0 2024-02-16T19:29:02.325 INFO:tasks.workunit.client.1.smithi117.stdout:8/342: write d4/d7/d37/d3a/d43/d44/f4b [4485682,57816] 0 2024-02-16T19:29:02.330 INFO:tasks.workunit.client.0.smithi022.stdout:7/382: mknod d9/da/d1f/d40/c7b 0 2024-02-16T19:29:02.333 INFO:tasks.workunit.client.1.smithi117.stdout:1/292: rmdir d2/d19/d59 39 2024-02-16T19:29:02.334 INFO:tasks.workunit.client.0.smithi022.stdout:4/353: rmdir d4/d38 39 2024-02-16T19:29:02.336 INFO:tasks.workunit.client.0.smithi022.stdout:4/354: chown d4/d6/d14/d64/f34 140 1 2024-02-16T19:29:02.337 INFO:tasks.workunit.client.0.smithi022.stdout:1/477: rmdir d6/d8/d22/d16 39 2024-02-16T19:29:02.339 INFO:tasks.workunit.client.1.smithi117.stdout:6/357: link d5/d4b/d20/f42 d5/d1d/d72/f77 0 2024-02-16T19:29:02.341 INFO:tasks.workunit.client.0.smithi022.stdout:5/388: creat dd/d1f/d48/d21/d2f/d40/d49/d4b/f7e x:0 0 0 2024-02-16T19:29:02.341 INFO:tasks.workunit.client.0.smithi022.stdout:9/421: dwrite db/d14/d16/f45 [4194304,4194304] 0 2024-02-16T19:29:02.343 INFO:tasks.workunit.client.1.smithi117.stdout:4/356: creat d0/f72 x:0 0 0 2024-02-16T19:29:02.344 INFO:tasks.workunit.client.1.smithi117.stdout:4/357: chown d0/da/d4a 43522 1 2024-02-16T19:29:02.345 INFO:tasks.workunit.client.0.smithi022.stdout:9/422: write db/d14/d16/d21/d37/f4e [456559,3063] 0 2024-02-16T19:29:02.346 INFO:tasks.workunit.client.0.smithi022.stdout:5/389: dread - dd/d1f/d48/d21/d2f/d40/d49/d4b/f66 zero size 2024-02-16T19:29:02.347 INFO:tasks.workunit.client.1.smithi117.stdout:7/333: creat d4/d1c/d39/d46/d47/f82 x:0 0 0 2024-02-16T19:29:02.348 INFO:tasks.workunit.client.0.smithi022.stdout:2/293: creat d0/f5f x:0 0 0 2024-02-16T19:29:02.348 INFO:tasks.workunit.client.0.smithi022.stdout:8/361: creat d3/d12/d2b/d46/d68/f81 x:0 0 0 2024-02-16T19:29:02.351 INFO:tasks.workunit.client.0.smithi022.stdout:6/377: creat d1/d8/d81/f8c x:0 0 0 2024-02-16T19:29:02.353 INFO:tasks.workunit.client.0.smithi022.stdout:6/378: stat d1/f3 0 2024-02-16T19:29:02.354 INFO:tasks.workunit.client.1.smithi117.stdout:0/391: mkdir d0/d9/d22/d27/d31/d94 0 2024-02-16T19:29:02.354 INFO:tasks.workunit.client.0.smithi022.stdout:7/383: symlink d9/da/d2c/l7c 0 2024-02-16T19:29:02.355 INFO:tasks.workunit.client.0.smithi022.stdout:5/390: dwrite dd/f3c [0,4194304] 0 2024-02-16T19:29:02.360 INFO:tasks.workunit.client.1.smithi117.stdout:9/291: creat dd/d24/f61 x:0 0 0 2024-02-16T19:29:02.365 INFO:tasks.workunit.client.1.smithi117.stdout:9/292: dwrite dd/d23/d2e/f52 [0,4194304] 0 2024-02-16T19:29:02.365 INFO:tasks.workunit.client.1.smithi117.stdout:2/299: unlink d0/d13/f26 0 2024-02-16T19:29:02.368 INFO:tasks.workunit.client.1.smithi117.stdout:5/444: creat d2/d3/d21/d2a/f93 x:0 0 0 2024-02-16T19:29:02.371 INFO:tasks.workunit.client.1.smithi117.stdout:2/300: dwrite d0/d3b/f51 [0,4194304] 0 2024-02-16T19:29:02.373 INFO:tasks.workunit.client.1.smithi117.stdout:6/358: creat d5/d4b/d20/f78 x:0 0 0 2024-02-16T19:29:02.375 INFO:tasks.workunit.client.0.smithi022.stdout:4/355: readlink d4/d6/d14/d64/d4e/l5d 0 2024-02-16T19:29:02.376 INFO:tasks.workunit.client.1.smithi117.stdout:4/358: mkdir d0/da/d21/d3d/d28/d64/d73 0 2024-02-16T19:29:02.378 INFO:tasks.workunit.client.1.smithi117.stdout:4/359: write d0/da/d21/d3d/d28/d64/d48/f4c [857254,63650] 0 2024-02-16T19:29:02.378 INFO:tasks.workunit.client.1.smithi117.stdout:7/334: symlink d4/d1c/l83 0 2024-02-16T19:29:02.380 INFO:tasks.workunit.client.1.smithi117.stdout:7/335: dread - d4/d1c/d39/d46/d47/f82 zero size 2024-02-16T19:29:02.381 INFO:tasks.workunit.client.1.smithi117.stdout:0/392: creat d0/d9/d22/d27/d83/f95 x:0 0 0 2024-02-16T19:29:02.382 INFO:tasks.workunit.client.0.smithi022.stdout:2/294: fdatasync d0/d2/d18/d1f/f21 0 2024-02-16T19:29:02.383 INFO:tasks.workunit.client.1.smithi117.stdout:3/415: mknod d1/c8a 0 2024-02-16T19:29:02.387 INFO:tasks.workunit.client.1.smithi117.stdout:5/445: dread d2/d9/f18 [0,4194304] 0 2024-02-16T19:29:02.389 INFO:tasks.workunit.client.1.smithi117.stdout:5/446: write d2/d3/f47 [4845895,41599] 0 2024-02-16T19:29:02.389 INFO:tasks.workunit.client.0.smithi022.stdout:2/295: dwrite d0/f17 [0,4194304] 0 2024-02-16T19:29:02.390 INFO:tasks.workunit.client.0.smithi022.stdout:7/384: mknod d9/da/d4e/c7d 0 2024-02-16T19:29:02.390 INFO:tasks.workunit.client.0.smithi022.stdout:2/296: write d0/d2/d18/f28 [570839,59965] 0 2024-02-16T19:29:02.397 INFO:tasks.workunit.client.0.smithi022.stdout:5/391: rename dd/d1f/d48/d21/d2f/d52/f6e to dd/d1f/d48/d21/d2f/d52/d5c/d61/d6a/f7f 0 2024-02-16T19:29:02.400 INFO:tasks.workunit.client.0.smithi022.stdout:2/297: dwrite d0/f17 [0,4194304] 0 2024-02-16T19:29:02.402 INFO:tasks.workunit.client.0.smithi022.stdout:2/298: read - d0/d2/f57 zero size 2024-02-16T19:29:02.403 INFO:tasks.workunit.client.0.smithi022.stdout:2/299: chown d0/d2/d18/d19/d40/f44 820039236 1 2024-02-16T19:29:02.404 INFO:tasks.workunit.client.0.smithi022.stdout:2/300: chown d0/d2 1203 1 2024-02-16T19:29:02.410 INFO:tasks.workunit.client.1.smithi117.stdout:6/359: truncate d5/d4b/d20/f42 1911138 0 2024-02-16T19:29:02.410 INFO:tasks.workunit.client.0.smithi022.stdout:2/301: dread d0/d2/d18/d19/d40/f44 [0,4194304] 0 2024-02-16T19:29:02.413 INFO:tasks.workunit.client.0.smithi022.stdout:8/362: creat d3/d12/d2b/d54/d74/f82 x:0 0 0 2024-02-16T19:29:02.414 INFO:tasks.workunit.client.0.smithi022.stdout:8/363: read - d3/d25/f79 zero size 2024-02-16T19:29:02.414 INFO:tasks.workunit.client.1.smithi117.stdout:4/360: mkdir d0/da/d21/d3d/d28/d64/d74 0 2024-02-16T19:29:02.415 INFO:tasks.workunit.client.1.smithi117.stdout:4/361: chown d0/da/d21/f3e 827642 1 2024-02-16T19:29:02.415 INFO:tasks.workunit.client.0.smithi022.stdout:8/364: chown d3/d12/d2b/d46/d68/f81 2713579 1 2024-02-16T19:29:02.416 INFO:tasks.workunit.client.0.smithi022.stdout:6/379: symlink d1/d8/da/l8d 0 2024-02-16T19:29:02.417 INFO:tasks.workunit.client.1.smithi117.stdout:7/336: creat d4/d1c/d24/d5f/f84 x:0 0 0 2024-02-16T19:29:02.418 INFO:tasks.workunit.client.0.smithi022.stdout:6/380: chown d1/d8/da/d29/d61/d57/l6e 536067 1 2024-02-16T19:29:02.419 INFO:tasks.workunit.client.1.smithi117.stdout:0/393: rmdir d0/d9/d14/d1d 39 2024-02-16T19:29:02.420 INFO:tasks.workunit.client.0.smithi022.stdout:1/478: dread d6/d8/d22/d16/f1e [0,4194304] 0 2024-02-16T19:29:02.422 INFO:tasks.workunit.client.1.smithi117.stdout:3/416: creat d1/d1b/d1e/d29/f8b x:0 0 0 2024-02-16T19:29:02.424 INFO:tasks.workunit.client.0.smithi022.stdout:7/385: unlink d9/da/d2c/d2f/d21/l2b 0 2024-02-16T19:29:02.427 INFO:tasks.workunit.client.0.smithi022.stdout:4/356: write d4/f23 [1345002,90823] 0 2024-02-16T19:29:02.429 INFO:tasks.workunit.client.1.smithi117.stdout:5/447: unlink d2/d36/c6b 0 2024-02-16T19:29:02.429 INFO:tasks.workunit.client.0.smithi022.stdout:4/357: read d4/f23 [587389,101063] 0 2024-02-16T19:29:02.430 INFO:tasks.workunit.client.1.smithi117.stdout:5/448: stat d2/d3/d21/d2f/d30/d5a/d62/l6e 0 2024-02-16T19:29:02.435 INFO:tasks.workunit.client.1.smithi117.stdout:1/293: rename d2/d1d/d60/d20/l24 to d2/d19/l61 0 2024-02-16T19:29:02.436 INFO:tasks.workunit.client.0.smithi022.stdout:4/358: dwrite d4/d6/dd/df/d10/f42 [0,4194304] 0 2024-02-16T19:29:02.438 INFO:tasks.workunit.client.0.smithi022.stdout:0/491: getdents d10 0 2024-02-16T19:29:02.440 INFO:tasks.workunit.client.1.smithi117.stdout:6/360: mkdir d5/d1d/d28/d32/d79 0 2024-02-16T19:29:02.447 INFO:tasks.workunit.client.1.smithi117.stdout:0/394: symlink d0/d9/d22/d27/d31/l96 0 2024-02-16T19:29:02.448 INFO:tasks.workunit.client.1.smithi117.stdout:4/362: dwrite d0/f25 [4194304,4194304] 0 2024-02-16T19:29:02.448 INFO:tasks.workunit.client.1.smithi117.stdout:0/395: chown d0/d9/d22/d27/f5f 300536389 1 2024-02-16T19:29:02.448 INFO:tasks.workunit.client.0.smithi022.stdout:1/479: mkdir d6/d8/d22/d47/d51/d8d 0 2024-02-16T19:29:02.449 INFO:tasks.workunit.client.1.smithi117.stdout:0/396: fdatasync d0/d9/d14/f92 0 2024-02-16T19:29:02.451 INFO:tasks.workunit.client.1.smithi117.stdout:0/397: read d0/d9/d22/d27/f5f [32132,95509] 0 2024-02-16T19:29:02.453 INFO:tasks.workunit.client.1.smithi117.stdout:5/449: creat d2/d4c/d4f/d67/f94 x:0 0 0 2024-02-16T19:29:02.454 INFO:tasks.workunit.client.0.smithi022.stdout:5/392: dwrite dd/d1f/d48/d21/d2f/d52/d5c/d61/d6a/f70 [0,4194304] 0 2024-02-16T19:29:02.454 INFO:tasks.workunit.client.1.smithi117.stdout:8/343: getdents d4/d7/d37/d3a/d43/d44 0 2024-02-16T19:29:02.455 INFO:tasks.workunit.client.1.smithi117.stdout:5/450: write d2/d3/d21/d2f/d30/d38/d51/f7e [2721264,129513] 0 2024-02-16T19:29:02.459 INFO:tasks.workunit.client.1.smithi117.stdout:5/451: dread d2/d9/f18 [0,4194304] 0 2024-02-16T19:29:02.460 INFO:tasks.workunit.client.1.smithi117.stdout:0/398: dread d0/d9/d22/d27/f5f [0,4194304] 0 2024-02-16T19:29:02.462 INFO:tasks.workunit.client.0.smithi022.stdout:4/359: mkdir d4/d6/d70 0 2024-02-16T19:29:02.462 INFO:tasks.workunit.client.1.smithi117.stdout:2/301: creat d0/d3/f57 x:0 0 0 2024-02-16T19:29:02.464 INFO:tasks.workunit.client.0.smithi022.stdout:4/360: fdatasync d4/d6/dd/d22/f51 0 2024-02-16T19:29:02.467 INFO:tasks.workunit.client.1.smithi117.stdout:6/361: unlink d5/d4b/d20/f4d 0 2024-02-16T19:29:02.468 INFO:tasks.workunit.client.1.smithi117.stdout:7/337: mkdir d4/d85 0 2024-02-16T19:29:02.469 INFO:tasks.workunit.client.0.smithi022.stdout:0/492: creat d10/d15/f9f x:0 0 0 2024-02-16T19:29:02.469 INFO:tasks.workunit.client.1.smithi117.stdout:7/338: fsync d4/d1c/d24/d5f/f7b 0 2024-02-16T19:29:02.469 INFO:tasks.workunit.client.0.smithi022.stdout:2/302: mkdir d0/d60 0 2024-02-16T19:29:02.470 INFO:tasks.workunit.client.0.smithi022.stdout:5/393: dwrite dd/d1f/d48/f26 [4194304,4194304] 0 2024-02-16T19:29:02.472 INFO:tasks.workunit.client.0.smithi022.stdout:0/493: readlink d10/d15/d23/d38/d5a/l7f 0 2024-02-16T19:29:02.472 INFO:tasks.workunit.client.0.smithi022.stdout:6/381: mknod d1/d8/da/d17/c8e 0 2024-02-16T19:29:02.473 INFO:tasks.workunit.client.1.smithi117.stdout:3/417: link d1/f2 d1/d31/f8c 0 2024-02-16T19:29:02.476 INFO:tasks.workunit.client.1.smithi117.stdout:4/363: dwrite d0/da/d21/d3d/f29 [0,4194304] 0 2024-02-16T19:29:02.476 INFO:tasks.workunit.client.1.smithi117.stdout:7/339: dwrite d4/d1c/d39/d46/d47/f56 [0,4194304] 0 2024-02-16T19:29:02.477 INFO:tasks.workunit.client.1.smithi117.stdout:3/418: truncate d1/d1b/d1e/d37/f65 334163 0 2024-02-16T19:29:02.478 INFO:tasks.workunit.client.1.smithi117.stdout:7/340: fsync d4/d13/f68 0 2024-02-16T19:29:02.479 INFO:tasks.workunit.client.1.smithi117.stdout:3/419: fsync d1/d1b/d1e/d5e/f74 0 2024-02-16T19:29:02.481 INFO:tasks.workunit.client.0.smithi022.stdout:8/365: rename d3/d6/f63 to d3/d12/d17/f83 0 2024-02-16T19:29:02.481 INFO:tasks.workunit.client.0.smithi022.stdout:6/382: write d1/d8/da/d17/d85/d5b/f63 [3886591,82284] 0 2024-02-16T19:29:02.482 INFO:tasks.workunit.client.1.smithi117.stdout:8/344: mkdir d4/d7/d37/d3a/d65/d6c 0 2024-02-16T19:29:02.482 INFO:tasks.workunit.client.0.smithi022.stdout:1/480: dread d6/d8/d22/d16/d17/d26/f43 [0,4194304] 0 2024-02-16T19:29:02.485 INFO:tasks.workunit.client.0.smithi022.stdout:0/494: rmdir d10/d15/d23/d8f 39 2024-02-16T19:29:02.485 INFO:tasks.workunit.client.1.smithi117.stdout:8/345: dread d4/f3c [0,4194304] 0 2024-02-16T19:29:02.488 INFO:tasks.workunit.client.0.smithi022.stdout:6/383: chown d1/f3 659 1 2024-02-16T19:29:02.489 INFO:tasks.workunit.client.0.smithi022.stdout:2/303: symlink d0/d2/d18/d19/d40/d55/l61 0 2024-02-16T19:29:02.490 INFO:tasks.workunit.client.0.smithi022.stdout:5/394: dwrite dd/d1f/d48/f1b [4194304,4194304] 0 2024-02-16T19:29:02.490 INFO:tasks.workunit.client.0.smithi022.stdout:3/357: sync 2024-02-16T19:29:02.495 INFO:tasks.workunit.client.0.smithi022.stdout:3/358: dread - d7/d12/d18/f4e zero size 2024-02-16T19:29:02.496 INFO:tasks.workunit.client.0.smithi022.stdout:4/361: mknod d4/d6/dd/d61/c71 0 2024-02-16T19:29:02.498 INFO:tasks.workunit.client.1.smithi117.stdout:5/452: mkdir d2/d3/d21/d2f/d30/d95 0 2024-02-16T19:29:02.500 INFO:tasks.workunit.client.1.smithi117.stdout:3/420: dread d1/d1b/d20/f7b [0,4194304] 0 2024-02-16T19:29:02.501 INFO:tasks.workunit.client.1.smithi117.stdout:1/294: symlink d2/d51/l62 0 2024-02-16T19:29:02.503 INFO:tasks.workunit.client.1.smithi117.stdout:1/295: fsync d2/d1d/f35 0 2024-02-16T19:29:02.505 INFO:tasks.workunit.client.0.smithi022.stdout:8/366: chown d3/d12/d17/l19 49 1 2024-02-16T19:29:02.505 INFO:tasks.workunit.client.1.smithi117.stdout:5/453: dwrite d2/d4c/d54/f6d [0,4194304] 0 2024-02-16T19:29:02.520 INFO:tasks.workunit.client.0.smithi022.stdout:9/423: sync 2024-02-16T19:29:02.520 INFO:tasks.workunit.client.0.smithi022.stdout:7/386: dwrite d9/da/d2c/d2f/f35 [0,4194304] 0 2024-02-16T19:29:02.520 INFO:tasks.workunit.client.1.smithi117.stdout:4/364: symlink d0/da/d21/d3d/d28/d64/l75 0 2024-02-16T19:29:02.522 INFO:tasks.workunit.client.0.smithi022.stdout:7/387: chown d9/da/d1f/d40/f61 882448923 1 2024-02-16T19:29:02.522 INFO:tasks.workunit.client.0.smithi022.stdout:9/424: chown db/d14/d16/d27/d2e 3002104 1 2024-02-16T19:29:02.522 INFO:tasks.workunit.client.0.smithi022.stdout:5/395: symlink dd/d1f/d48/d21/d2f/d46/d5e/l80 0 2024-02-16T19:29:02.524 INFO:tasks.workunit.client.0.smithi022.stdout:5/396: fdatasync dd/d1f/d48/d21/d2f/d46/d5e/f7b 0 2024-02-16T19:29:02.525 INFO:tasks.workunit.client.0.smithi022.stdout:5/397: write dd/f1d [123345,122807] 0 2024-02-16T19:29:02.526 INFO:tasks.workunit.client.0.smithi022.stdout:5/398: chown dd/d1f/d2e/c30 1182210 1 2024-02-16T19:29:02.528 INFO:tasks.workunit.client.0.smithi022.stdout:5/399: truncate dd/d1f/d48/f60 179684 0 2024-02-16T19:29:02.532 INFO:tasks.workunit.client.0.smithi022.stdout:4/362: dread d4/d6/d14/d64/f34 [0,4194304] 0 2024-02-16T19:29:02.535 INFO:tasks.workunit.client.0.smithi022.stdout:8/367: creat d3/d12/d2b/d54/d74/f84 x:0 0 0 2024-02-16T19:29:02.536 INFO:tasks.workunit.client.0.smithi022.stdout:8/368: write d3/d12/d2b/d54/f76 [953525,2411] 0 2024-02-16T19:29:02.538 INFO:tasks.workunit.client.1.smithi117.stdout:9/293: rename dd/d23/d2e to dd/d45/d62 0 2024-02-16T19:29:02.539 INFO:tasks.workunit.client.1.smithi117.stdout:2/302: write d0/d3/d22/f3c [912022,18146] 0 2024-02-16T19:29:02.540 INFO:tasks.workunit.client.1.smithi117.stdout:2/303: fdatasync d0/f4f 0 2024-02-16T19:29:02.541 INFO:tasks.workunit.client.1.smithi117.stdout:2/304: chown d0/d3/c4 2 1 2024-02-16T19:29:02.543 INFO:tasks.workunit.client.1.smithi117.stdout:3/421: symlink d1/d1b/d1e/d29/l8d 0 2024-02-16T19:29:02.545 INFO:tasks.workunit.client.1.smithi117.stdout:9/294: dwrite dd/d24/f5a [0,4194304] 0 2024-02-16T19:29:02.546 INFO:tasks.workunit.client.1.smithi117.stdout:3/422: fsync d1/d1b/d1e/d5e/f62 0 2024-02-16T19:29:02.547 INFO:tasks.workunit.client.1.smithi117.stdout:7/341: write d4/f11 [261526,103727] 0 2024-02-16T19:29:02.547 INFO:tasks.workunit.client.1.smithi117.stdout:1/296: symlink d2/d1d/d60/d20/l63 0 2024-02-16T19:29:02.548 INFO:tasks.workunit.client.0.smithi022.stdout:2/304: dwrite d0/d2/d18/d1f/f21 [0,4194304] 0 2024-02-16T19:29:02.549 INFO:tasks.workunit.client.1.smithi117.stdout:3/423: chown d1/d1b/f41 53 1 2024-02-16T19:29:02.550 INFO:tasks.workunit.client.1.smithi117.stdout:2/305: dwrite d0/d13/f36 [0,4194304] 0 2024-02-16T19:29:02.552 INFO:tasks.workunit.client.1.smithi117.stdout:2/306: write d0/d29/f49 [989950,52023] 0 2024-02-16T19:29:02.557 INFO:tasks.workunit.client.1.smithi117.stdout:0/399: creat d0/d9/d14/d1d/d60/d76/d7b/d8d/f97 x:0 0 0 2024-02-16T19:29:02.558 INFO:tasks.workunit.client.1.smithi117.stdout:0/400: write d0/d9/d14/d1d/d60/d76/d7b/d8d/f2e [2494923,11639] 0 2024-02-16T19:29:02.562 INFO:tasks.workunit.client.1.smithi117.stdout:1/297: dread d2/d1d/d60/d20/d49/f4a [0,4194304] 0 2024-02-16T19:29:02.563 INFO:tasks.workunit.client.1.smithi117.stdout:1/298: dread d2/d1d/f3a [0,4194304] 0 2024-02-16T19:29:02.564 INFO:tasks.workunit.client.1.smithi117.stdout:4/365: symlink d0/da/d21/d3d/d28/d64/l76 0 2024-02-16T19:29:02.564 INFO:tasks.workunit.client.0.smithi022.stdout:0/495: dwrite d10/d11/d27/d3f/d58/d60/d62/f86 [0,4194304] 0 2024-02-16T19:29:02.572 INFO:tasks.workunit.client.0.smithi022.stdout:1/481: rename d6/d8/d22/d16/d17/d26/f46 to d6/f8e 0 2024-02-16T19:29:02.573 INFO:tasks.workunit.client.0.smithi022.stdout:1/482: stat d6/d8/d22/d67 0 2024-02-16T19:29:02.577 INFO:tasks.workunit.client.0.smithi022.stdout:6/384: truncate d1/d8/da/d17/d53/f5c 2296920 0 2024-02-16T19:29:02.580 INFO:tasks.workunit.client.0.smithi022.stdout:9/425: dwrite db/d18/f73 [0,4194304] 0 2024-02-16T19:29:02.581 INFO:tasks.workunit.client.0.smithi022.stdout:7/388: creat d9/da/d2c/d6d/f7e x:0 0 0 2024-02-16T19:29:02.583 INFO:tasks.workunit.client.0.smithi022.stdout:9/426: write db/d18/f2a [261095,31211] 0 2024-02-16T19:29:02.584 INFO:tasks.workunit.client.0.smithi022.stdout:7/389: chown d9/da/d2c/d6d/f7e 482 1 2024-02-16T19:29:02.585 INFO:tasks.workunit.client.0.smithi022.stdout:7/390: write d9/da/f62 [220721,69079] 0 2024-02-16T19:29:02.586 INFO:tasks.workunit.client.0.smithi022.stdout:9/427: chown db/d14/d16/d28/l77 55 1 2024-02-16T19:29:02.587 INFO:tasks.workunit.client.1.smithi117.stdout:9/295: dread dd/d23/f1a [0,4194304] 0 2024-02-16T19:29:02.588 INFO:tasks.workunit.client.1.smithi117.stdout:9/296: stat dd/d24/c29 0 2024-02-16T19:29:02.589 INFO:tasks.workunit.client.1.smithi117.stdout:9/297: dread - dd/d45/d62/f4e zero size 2024-02-16T19:29:02.591 INFO:tasks.workunit.client.0.smithi022.stdout:0/496: chown d10/d11/d27/d3f/d58/c67 785456603 1 2024-02-16T19:29:02.592 INFO:tasks.workunit.client.0.smithi022.stdout:3/359: link d7/d12/d18/dd/d24/d29/f56 d7/d12/d18/dd/d24/d58/d6b/f8f 0 2024-02-16T19:29:02.593 INFO:tasks.workunit.client.1.smithi117.stdout:5/454: mkdir d2/d9/d3c/d24/d50/d96 0 2024-02-16T19:29:02.594 INFO:tasks.workunit.client.0.smithi022.stdout:6/385: dwrite d1/d8/da/d17/d48/d51/f59 [0,4194304] 0 2024-02-16T19:29:02.595 INFO:tasks.workunit.client.1.smithi117.stdout:9/298: dwrite dd/d45/f3f [0,4194304] 0 2024-02-16T19:29:02.595 INFO:tasks.workunit.client.1.smithi117.stdout:5/455: chown d2/d3/d21/d2f/d30/f68 16797528 1 2024-02-16T19:29:02.598 INFO:tasks.workunit.client.0.smithi022.stdout:7/391: dwrite d9/da/f13 [0,4194304] 0 2024-02-16T19:29:02.601 INFO:tasks.workunit.client.0.smithi022.stdout:8/369: symlink d3/d12/d2b/d2e/l85 0 2024-02-16T19:29:02.604 INFO:tasks.workunit.client.0.smithi022.stdout:8/370: rename d3/d12/d2b to d3/d12/d2b/d46/d86 22 2024-02-16T19:29:02.607 INFO:tasks.workunit.client.1.smithi117.stdout:3/424: dwrite d1/f2 [0,4194304] 0 2024-02-16T19:29:02.613 INFO:tasks.workunit.client.0.smithi022.stdout:1/483: read d6/d8/d22/d16/f2e [68783,99860] 0 2024-02-16T19:29:02.614 INFO:tasks.workunit.client.1.smithi117.stdout:2/307: mkdir d0/d13/d4d/d53/d58 0 2024-02-16T19:29:02.615 INFO:tasks.workunit.client.1.smithi117.stdout:2/308: readlink d0/d13/d4d/l4e 0 2024-02-16T19:29:02.616 INFO:tasks.workunit.client.1.smithi117.stdout:2/309: write d0/d3/f3a [727063,93750] 0 2024-02-16T19:29:02.617 INFO:tasks.workunit.client.1.smithi117.stdout:2/310: chown d0/d29/d2f/d42 211032 1 2024-02-16T19:29:02.617 INFO:tasks.workunit.client.0.smithi022.stdout:6/386: creat d1/d8/f8f x:0 0 0 2024-02-16T19:29:02.618 INFO:tasks.workunit.client.1.smithi117.stdout:5/456: write d2/d5d/f5e [742809,92499] 0 2024-02-16T19:29:02.620 INFO:tasks.workunit.client.0.smithi022.stdout:3/360: unlink d7/d12/d18/dd/d24/d29/f56 0 2024-02-16T19:29:02.622 INFO:tasks.workunit.client.0.smithi022.stdout:3/361: write d7/d12/d18/dd/d31/d41/f48 [2175772,85745] 0 2024-02-16T19:29:02.622 INFO:tasks.workunit.client.0.smithi022.stdout:4/363: link d4/d6/dd/df/d10/d24/d4b/c6c d4/d6/d14/d64/d4e/d54/d5a/c72 0 2024-02-16T19:29:02.625 INFO:tasks.workunit.client.0.smithi022.stdout:0/497: truncate d10/f6d 131559 0 2024-02-16T19:29:02.627 INFO:tasks.workunit.client.0.smithi022.stdout:7/392: write d9/da/d1f/d40/f61 [229099,80754] 0 2024-02-16T19:29:02.628 INFO:tasks.workunit.client.0.smithi022.stdout:7/393: read - d9/da/d2c/d2f/f76 zero size 2024-02-16T19:29:02.629 INFO:tasks.workunit.client.1.smithi117.stdout:8/346: getdents d4/d12 0 2024-02-16T19:29:02.631 INFO:tasks.workunit.client.1.smithi117.stdout:8/347: read d4/f6 [3631421,117007] 0 2024-02-16T19:29:02.632 INFO:tasks.workunit.client.0.smithi022.stdout:5/400: link dd/d1f/d48/d21/d2f/d40/l4d dd/d1f/d2e/d35/l81 0 2024-02-16T19:29:02.633 INFO:tasks.workunit.client.0.smithi022.stdout:0/498: dread d10/f6e [0,4194304] 0 2024-02-16T19:29:02.634 INFO:tasks.workunit.client.1.smithi117.stdout:6/362: rename d5/d1d/d28/f34 to d5/d4b/f7a 0 2024-02-16T19:29:02.636 INFO:tasks.workunit.client.1.smithi117.stdout:8/348: dwrite d4/d7/d37/d3a/d43/f6b [0,4194304] 0 2024-02-16T19:29:02.639 INFO:tasks.workunit.client.1.smithi117.stdout:3/425: truncate d1/d31/f36 1312782 0 2024-02-16T19:29:02.640 INFO:tasks.workunit.client.0.smithi022.stdout:1/484: fsync d6/f19 0 2024-02-16T19:29:02.642 INFO:tasks.workunit.client.0.smithi022.stdout:6/387: rmdir d1/d8/d81 39 2024-02-16T19:29:02.645 INFO:tasks.workunit.client.0.smithi022.stdout:3/362: symlink d7/d12/d18/dd/d31/d41/l90 0 2024-02-16T19:29:02.650 INFO:tasks.workunit.client.1.smithi117.stdout:9/299: getdents dd/d47/d5e 0 2024-02-16T19:29:02.656 INFO:tasks.workunit.client.0.smithi022.stdout:8/371: link d3/d12/d17/f83 d3/d12/d2b/d2e/f87 0 2024-02-16T19:29:02.657 INFO:tasks.workunit.client.1.smithi117.stdout:4/366: creat d0/f77 x:0 0 0 2024-02-16T19:29:02.658 INFO:tasks.workunit.client.0.smithi022.stdout:7/394: creat d9/da/d1f/d40/f7f x:0 0 0 2024-02-16T19:29:02.659 INFO:tasks.workunit.client.0.smithi022.stdout:2/305: getdents d0/d2 0 2024-02-16T19:29:02.660 INFO:tasks.workunit.client.0.smithi022.stdout:2/306: fdatasync d0/d2/d18/d1f/f5e 0 2024-02-16T19:29:02.661 INFO:tasks.workunit.client.0.smithi022.stdout:5/401: mknod dd/d1f/d48/d21/d2f/d40/d49/c82 0 2024-02-16T19:29:02.662 INFO:tasks.workunit.client.1.smithi117.stdout:4/367: dwrite d0/f5c [0,4194304] 0 2024-02-16T19:29:02.664 INFO:tasks.workunit.client.0.smithi022.stdout:0/499: rename d10/d11/d27/d3f/d58/d60/d62/c35 to d10/d11/d50/d65/ca0 0 2024-02-16T19:29:02.667 INFO:tasks.workunit.client.0.smithi022.stdout:9/428: truncate db/f84 2073098 0 2024-02-16T19:29:02.669 INFO:tasks.workunit.client.0.smithi022.stdout:9/429: fdatasync db/d14/d19/d1a/f24 0 2024-02-16T19:29:02.670 INFO:tasks.workunit.client.0.smithi022.stdout:0/500: dread d10/d15/d23/d38/d5a/f5d [0,4194304] 0 2024-02-16T19:29:02.671 INFO:tasks.workunit.client.0.smithi022.stdout:1/485: readlink d6/l25 0 2024-02-16T19:29:02.672 INFO:tasks.workunit.client.0.smithi022.stdout:0/501: read d10/d11/d27/d3f/f7a [619152,25116] 0 2024-02-16T19:29:02.674 INFO:tasks.workunit.client.1.smithi117.stdout:2/311: getdents d0/d29/d2f/d42/d46 0 2024-02-16T19:29:02.677 INFO:tasks.workunit.client.1.smithi117.stdout:2/312: dread d0/d13/f36 [0,4194304] 0 2024-02-16T19:29:02.678 INFO:tasks.workunit.client.1.smithi117.stdout:9/300: mkdir dd/d45/d31/d63 0 2024-02-16T19:29:02.680 INFO:tasks.workunit.client.0.smithi022.stdout:6/388: dread d1/d8/da/d17/d53/f5c [0,4194304] 0 2024-02-16T19:29:02.682 INFO:tasks.workunit.client.1.smithi117.stdout:1/299: getdents d2/d1d/d60/d20 0 2024-02-16T19:29:02.683 INFO:tasks.workunit.client.0.smithi022.stdout:4/364: creat d4/d6/d14/d64/d66/d6d/f73 x:0 0 0 2024-02-16T19:29:02.688 INFO:tasks.workunit.client.0.smithi022.stdout:8/372: mknod d3/d12/d2b/d54/c88 0 2024-02-16T19:29:02.695 INFO:tasks.workunit.client.1.smithi117.stdout:7/342: rename d4/d1c/d24/c5c to d4/d85/c86 0 2024-02-16T19:29:02.697 INFO:tasks.workunit.client.0.smithi022.stdout:2/307: creat d0/d32/f62 x:0 0 0 2024-02-16T19:29:02.698 INFO:tasks.workunit.client.0.smithi022.stdout:2/308: write d0/d2/d18/d19/f2f [806266,49521] 0 2024-02-16T19:29:02.699 INFO:tasks.workunit.client.0.smithi022.stdout:2/309: dread - d0/d32/f52 zero size 2024-02-16T19:29:02.700 INFO:tasks.workunit.client.0.smithi022.stdout:5/402: creat dd/d1f/d4a/f83 x:0 0 0 2024-02-16T19:29:02.700 INFO:tasks.workunit.client.0.smithi022.stdout:2/310: readlink d0/d2/d18/d1f/l4e 0 2024-02-16T19:29:02.701 INFO:tasks.workunit.client.0.smithi022.stdout:5/403: stat dd/d1f/d48/d21/d2f 0 2024-02-16T19:29:02.701 INFO:tasks.workunit.client.1.smithi117.stdout:3/426: mkdir d1/dd/d8e 0 2024-02-16T19:29:02.704 INFO:tasks.workunit.client.1.smithi117.stdout:5/457: mkdir d2/d3/d21/d49/d59/d97 0 2024-02-16T19:29:02.704 INFO:tasks.workunit.client.0.smithi022.stdout:5/404: read dd/f17 [58433,126758] 0 2024-02-16T19:29:02.707 INFO:tasks.workunit.client.0.smithi022.stdout:5/405: truncate dd/d1f/d48/f74 803070 0 2024-02-16T19:29:02.708 INFO:tasks.workunit.client.1.smithi117.stdout:1/300: creat d2/d1d/d60/d20/f64 x:0 0 0 2024-02-16T19:29:02.708 INFO:tasks.workunit.client.0.smithi022.stdout:2/311: dwrite d0/d2/f39 [0,4194304] 0 2024-02-16T19:29:02.713 INFO:tasks.workunit.client.0.smithi022.stdout:6/389: rename d1/d8/fb to d1/d8/da/d17/d85/d56/f90 0 2024-02-16T19:29:02.714 INFO:tasks.workunit.client.1.smithi117.stdout:1/301: dwrite d2/d1d/d60/d20/f42 [0,4194304] 0 2024-02-16T19:29:02.714 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:02 smithi022 ceph-mon[29221]: pgmap v4: 65 pgs: 65 active+clean; 1.4 GiB data, 4.2 GiB used, 532 GiB / 536 GiB avail 2024-02-16T19:29:02.714 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:02 smithi022 ceph-mon[29221]: mgrmap e23: smithi117.cohbtv(active, since 2s) 2024-02-16T19:29:02.715 INFO:tasks.workunit.client.1.smithi117.stdout:1/302: dread - d2/d1d/d1f/f5e zero size 2024-02-16T19:29:02.716 INFO:tasks.workunit.client.0.smithi022.stdout:5/406: dwrite dd/f1d [0,4194304] 0 2024-02-16T19:29:02.716 INFO:tasks.workunit.client.1.smithi117.stdout:1/303: write d2/d1d/d60/d20/f64 [209525,116949] 0 2024-02-16T19:29:02.717 INFO:tasks.workunit.client.1.smithi117.stdout:0/401: rename d0/cf to d0/d78/c98 0 2024-02-16T19:29:02.719 INFO:tasks.workunit.client.1.smithi117.stdout:7/343: readlink d4/d1c/d24/d60/l80 0 2024-02-16T19:29:02.721 INFO:tasks.workunit.client.0.smithi022.stdout:7/395: creat d9/da/d74/f80 x:0 0 0 2024-02-16T19:29:02.726 INFO:tasks.workunit.client.1.smithi117.stdout:3/427: unlink d1/d1b/d1e/f23 0 2024-02-16T19:29:02.729 INFO:tasks.workunit.client.1.smithi117.stdout:0/402: dwrite d0/d9/d22/d27/d31/d6d/f82 [0,4194304] 0 2024-02-16T19:29:02.733 INFO:tasks.workunit.client.1.smithi117.stdout:2/313: creat d0/d35/f59 x:0 0 0 2024-02-16T19:29:02.734 INFO:tasks.workunit.client.1.smithi117.stdout:2/314: read d0/d3/d10/f16 [678644,13855] 0 2024-02-16T19:29:02.737 INFO:tasks.workunit.client.0.smithi022.stdout:8/373: dwrite d3/d6/f1f [0,4194304] 0 2024-02-16T19:29:02.742 INFO:tasks.workunit.client.1.smithi117.stdout:5/458: dwrite d2/d9/f18 [4194304,4194304] 0 2024-02-16T19:29:02.744 INFO:tasks.workunit.client.1.smithi117.stdout:9/301: truncate f5 3379124 0 2024-02-16T19:29:02.745 INFO:tasks.workunit.client.1.smithi117.stdout:4/368: rename d0/da/f56 to d0/da/d4a/f78 0 2024-02-16T19:29:02.745 INFO:tasks.workunit.client.0.smithi022.stdout:1/486: symlink d6/d8/d6c/l8f 0 2024-02-16T19:29:02.746 INFO:tasks.workunit.client.1.smithi117.stdout:8/349: sync 2024-02-16T19:29:02.746 INFO:tasks.workunit.client.1.smithi117.stdout:1/304: truncate d2/d1d/d1f/f46 716267 0 2024-02-16T19:29:02.749 INFO:tasks.workunit.client.1.smithi117.stdout:4/369: truncate d0/da/d21/d3d/d28/d64/f55 8506704 0 2024-02-16T19:29:02.749 INFO:tasks.workunit.client.1.smithi117.stdout:7/344: unlink d4/d1c/d39/d46/d47/c4e 0 2024-02-16T19:29:02.750 INFO:tasks.workunit.client.0.smithi022.stdout:1/487: dread d6/f55 [0,4194304] 0 2024-02-16T19:29:02.751 INFO:tasks.workunit.client.1.smithi117.stdout:6/363: getdents d5/d1d/d72 0 2024-02-16T19:29:02.752 INFO:tasks.workunit.client.0.smithi022.stdout:2/312: mknod d0/d32/c63 0 2024-02-16T19:29:02.754 INFO:tasks.workunit.client.0.smithi022.stdout:3/363: creat d7/d12/d18/f91 x:0 0 0 2024-02-16T19:29:02.758 INFO:tasks.workunit.client.0.smithi022.stdout:6/390: rename d1/d8/da/d17/d48/f65 to d1/d8/da/d17/d53/f91 0 2024-02-16T19:29:02.765 INFO:tasks.workunit.client.0.smithi022.stdout:7/396: write d9/da/d2c/d46/f6a [959690,27984] 0 2024-02-16T19:29:02.771 INFO:tasks.workunit.client.0.smithi022.stdout:8/374: symlink d3/d12/l89 0 2024-02-16T19:29:02.777 INFO:tasks.workunit.client.0.smithi022.stdout:0/502: link d10/d22/f45 d10/d22/fa1 0 2024-02-16T19:29:02.781 INFO:tasks.workunit.client.1.smithi117.stdout:3/428: mknod d1/d1b/d1e/d29/c8f 0 2024-02-16T19:29:02.784 INFO:tasks.workunit.client.0.smithi022.stdout:5/407: rename dd/d1f/d2e/d35 to dd/d1f/d48/d21/d2f/d46/d5e/d84 0 2024-02-16T19:29:02.790 INFO:tasks.workunit.client.0.smithi022.stdout:7/397: creat d9/da/d1f/f81 x:0 0 0 2024-02-16T19:29:02.790 INFO:tasks.workunit.client.0.smithi022.stdout:7/398: readlink l0 0 2024-02-16T19:29:02.791 INFO:tasks.workunit.client.0.smithi022.stdout:5/408: dwrite dd/d1f/d48/d21/d2f/d52/f53 [0,4194304] 0 2024-02-16T19:29:02.792 INFO:tasks.workunit.client.0.smithi022.stdout:8/375: mknod d3/d25/c8a 0 2024-02-16T19:29:02.795 INFO:tasks.workunit.client.0.smithi022.stdout:9/430: getdents db/d14/d19/d1a/d67 0 2024-02-16T19:29:02.796 INFO:tasks.workunit.client.0.smithi022.stdout:9/431: chown db/d18/l61 3896245 1 2024-02-16T19:29:02.797 INFO:tasks.workunit.client.0.smithi022.stdout:3/364: write d7/d12/d18/dd/d24/d58/d70/f8e [327046,2493] 0 2024-02-16T19:29:02.799 INFO:tasks.workunit.client.0.smithi022.stdout:3/365: fdatasync d7/d12/d18/f88 0 2024-02-16T19:29:02.799 INFO:tasks.workunit.client.0.smithi022.stdout:9/432: chown db/d14/d16/d21/d37/c52 69720861 1 2024-02-16T19:29:02.802 INFO:tasks.workunit.client.0.smithi022.stdout:5/409: dwrite dd/d1f/d48/f60 [0,4194304] 0 2024-02-16T19:29:02.803 INFO:tasks.workunit.client.1.smithi117.stdout:4/370: mknod d0/da/d21/c79 0 2024-02-16T19:29:02.804 INFO:tasks.workunit.client.0.smithi022.stdout:6/391: write d1/d8/f1c [4196325,110015] 0 2024-02-16T19:29:02.806 INFO:tasks.workunit.client.0.smithi022.stdout:6/392: chown d1/d8/da/d17/d53/d7f 1390489758 1 2024-02-16T19:29:02.811 INFO:tasks.workunit.client.0.smithi022.stdout:4/365: getdents d4/d6/dd 0 2024-02-16T19:29:02.815 INFO:tasks.workunit.client.1.smithi117.stdout:2/315: sync 2024-02-16T19:29:02.817 INFO:tasks.workunit.client.1.smithi117.stdout:3/429: creat d1/d30/f90 x:0 0 0 2024-02-16T19:29:02.818 INFO:tasks.workunit.client.1.smithi117.stdout:0/403: symlink d0/d9/d22/d27/d31/d94/l99 0 2024-02-16T19:29:02.818 INFO:tasks.workunit.client.1.smithi117.stdout:8/350: dwrite d4/d12/f26 [0,4194304] 0 2024-02-16T19:29:02.819 INFO:tasks.workunit.client.0.smithi022.stdout:8/376: creat d3/d12/f8b x:0 0 0 2024-02-16T19:29:02.820 INFO:tasks.workunit.client.1.smithi117.stdout:3/430: chown d1/d1b/d1e/l4b 49503 1 2024-02-16T19:29:02.821 INFO:tasks.workunit.client.1.smithi117.stdout:3/431: chown d1/d1b/f5f 184 1 2024-02-16T19:29:02.821 INFO:tasks.workunit.client.1.smithi117.stdout:6/364: dwrite d5/d4b/f29 [0,4194304] 0 2024-02-16T19:29:02.823 INFO:tasks.workunit.client.1.smithi117.stdout:3/432: readlink d1/d1b/d1e/l7f 0 2024-02-16T19:29:02.823 INFO:tasks.workunit.client.0.smithi022.stdout:9/433: rmdir db 39 2024-02-16T19:29:02.825 INFO:tasks.workunit.client.0.smithi022.stdout:3/366: creat d7/d12/d18/dd/d3c/f92 x:0 0 0 2024-02-16T19:29:02.828 INFO:tasks.workunit.client.0.smithi022.stdout:1/488: rename d6/d8/d22/c13 to d6/d8/d22/d3a/d7a/c90 0 2024-02-16T19:29:02.829 INFO:tasks.workunit.client.0.smithi022.stdout:0/503: rename d10/d15 to d10/d15/d23/d38/d5a/d66/da2 22 2024-02-16T19:29:02.833 INFO:tasks.workunit.client.0.smithi022.stdout:7/399: dread d9/da/d1f/d44/f49 [4194304,4194304] 0 2024-02-16T19:29:02.833 INFO:tasks.workunit.client.1.smithi117.stdout:7/345: truncate d4/d1c/d39/f54 3817661 0 2024-02-16T19:29:02.836 INFO:tasks.workunit.client.0.smithi022.stdout:6/393: creat d1/d8/d3c/d77/f92 x:0 0 0 2024-02-16T19:29:02.838 INFO:tasks.workunit.client.0.smithi022.stdout:1/489: dwrite d6/d8/d22/d16/d17/d1d/f21 [0,4194304] 0 2024-02-16T19:29:02.839 INFO:tasks.workunit.client.1.smithi117.stdout:5/459: rename d2/f5b to d2/d9/d3c/f98 0 2024-02-16T19:29:02.840 INFO:tasks.workunit.client.0.smithi022.stdout:8/377: dread d3/f55 [0,4194304] 0 2024-02-16T19:29:02.840 INFO:tasks.workunit.client.1.smithi117.stdout:1/305: dwrite d2/d1d/d60/d20/d49/f50 [4194304,4194304] 0 2024-02-16T19:29:02.842 INFO:tasks.workunit.client.0.smithi022.stdout:9/434: fdatasync db/d18/f2a 0 2024-02-16T19:29:02.844 INFO:tasks.workunit.client.1.smithi117.stdout:0/404: readlink d0/d9/d14/d1d/l51 0 2024-02-16T19:29:02.846 INFO:tasks.workunit.client.1.smithi117.stdout:8/351: mkdir d4/d12/d2a/d6d 0 2024-02-16T19:29:02.847 INFO:tasks.workunit.client.0.smithi022.stdout:0/504: stat d10/f6d 0 2024-02-16T19:29:02.848 INFO:tasks.workunit.client.1.smithi117.stdout:6/365: symlink d5/d1d/d28/d32/l7b 0 2024-02-16T19:29:02.849 INFO:tasks.workunit.client.0.smithi022.stdout:0/505: write d10/d15/d23/d38/d5a/d66/f9d [4136249,67675] 0 2024-02-16T19:29:02.849 INFO:tasks.workunit.client.0.smithi022.stdout:4/366: dread d4/d6/dd/df/d10/f35 [0,4194304] 0 2024-02-16T19:29:02.851 INFO:tasks.workunit.client.0.smithi022.stdout:3/367: rename d7/d12/d18/dd/d3c/d5a/d67/d84 to d7/d12/d18/dd/d24/d93 0 2024-02-16T19:29:02.851 INFO:tasks.workunit.client.0.smithi022.stdout:4/367: chown d4/d6/dd/df/d10/d24/c2b 1 1 2024-02-16T19:29:02.852 INFO:tasks.workunit.client.1.smithi117.stdout:3/433: dread d1/d1b/d1e/d5e/f66 [0,4194304] 0 2024-02-16T19:29:02.854 INFO:tasks.workunit.client.1.smithi117.stdout:8/352: dwrite d4/d7/d37/d3a/d43/d44/f62 [0,4194304] 0 2024-02-16T19:29:02.854 INFO:tasks.workunit.client.0.smithi022.stdout:7/400: symlink d9/da/d2c/d6d/l82 0 2024-02-16T19:29:02.855 INFO:tasks.workunit.client.1.smithi117.stdout:8/353: chown d4/d7/d37/d3a/d54 114 1 2024-02-16T19:29:02.856 INFO:tasks.workunit.client.0.smithi022.stdout:7/401: chown d9/da/d1f/d40 724923 1 2024-02-16T19:29:02.858 INFO:tasks.workunit.client.0.smithi022.stdout:0/506: dwrite d10/d15/f39 [4194304,4194304] 0 2024-02-16T19:29:02.862 INFO:tasks.workunit.client.0.smithi022.stdout:8/378: creat d3/d6/f8c x:0 0 0 2024-02-16T19:29:02.862 INFO:tasks.workunit.client.0.smithi022.stdout:1/490: creat d6/d8/d22/d16/d17/d26/f91 x:0 0 0 2024-02-16T19:29:02.866 INFO:tasks.workunit.client.1.smithi117.stdout:1/306: symlink d2/d1d/d60/l65 0 2024-02-16T19:29:02.872 INFO:tasks.workunit.client.0.smithi022.stdout:5/410: link dd/d1f/d4a/f3f dd/d1f/d48/d21/d2f/d52/d5c/f85 0 2024-02-16T19:29:02.874 INFO:tasks.workunit.client.0.smithi022.stdout:3/368: creat d7/d12/d18/dd/d24/d58/d6b/f94 x:0 0 0 2024-02-16T19:29:02.877 INFO:tasks.workunit.client.0.smithi022.stdout:6/394: write d1/d8/da/d17/d53/f5c [42778,40203] 0 2024-02-16T19:29:02.880 INFO:tasks.workunit.client.0.smithi022.stdout:7/402: mkdir d9/da/d4e/d83 0 2024-02-16T19:29:02.882 INFO:tasks.workunit.client.0.smithi022.stdout:7/403: truncate d9/da/d1f/d40/f61 1131983 0 2024-02-16T19:29:02.883 INFO:tasks.workunit.client.1.smithi117.stdout:8/354: write d4/d12/d31/f36 [541515,48460] 0 2024-02-16T19:29:02.883 INFO:tasks.workunit.client.0.smithi022.stdout:7/404: truncate f5 8617865 0 2024-02-16T19:29:02.884 INFO:tasks.workunit.client.0.smithi022.stdout:0/507: creat d10/d22/d49/fa3 x:0 0 0 2024-02-16T19:29:02.885 INFO:tasks.workunit.client.1.smithi117.stdout:2/316: truncate d0/d3/d10/f16 1364970 0 2024-02-16T19:29:02.886 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:02 smithi117 ceph-mon[35309]: pgmap v4: 65 pgs: 65 active+clean; 1.4 GiB data, 4.2 GiB used, 532 GiB / 536 GiB avail 2024-02-16T19:29:02.886 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:02 smithi117 ceph-mon[35309]: mgrmap e23: smithi117.cohbtv(active, since 2s) 2024-02-16T19:29:02.889 INFO:tasks.workunit.client.0.smithi022.stdout:1/491: rename d6/d8/d22/d16/d17/d1d/l42 to d6/d8/d22/d16/d4f/l92 0 2024-02-16T19:29:02.890 INFO:tasks.workunit.client.0.smithi022.stdout:4/368: write d4/d6/dd/df/d10/f35 [1002426,11038] 0 2024-02-16T19:29:02.891 INFO:tasks.workunit.client.0.smithi022.stdout:0/508: dwrite d10/d11/d27/d3f/f8a [0,4194304] 0 2024-02-16T19:29:02.892 INFO:tasks.workunit.client.0.smithi022.stdout:1/492: write d6/d8/d22/d16/d17/d63/d6f/f7c [801215,63338] 0 2024-02-16T19:29:02.893 INFO:tasks.workunit.client.0.smithi022.stdout:9/435: mkdir db/d14/d16/d49/d90 0 2024-02-16T19:29:02.894 INFO:tasks.workunit.client.1.smithi117.stdout:7/346: truncate d4/d13/f18 7390 0 2024-02-16T19:29:02.897 INFO:tasks.workunit.client.1.smithi117.stdout:2/317: dread d0/f19 [0,4194304] 0 2024-02-16T19:29:02.898 INFO:tasks.workunit.client.0.smithi022.stdout:3/369: truncate d7/d12/d18/dd/d24/f32 345004 0 2024-02-16T19:29:02.898 INFO:tasks.workunit.client.0.smithi022.stdout:5/411: read dd/d1f/d48/d21/f23 [485979,96282] 0 2024-02-16T19:29:02.899 INFO:tasks.workunit.client.1.smithi117.stdout:6/366: mkdir d5/d4b/d44/d7c 0 2024-02-16T19:29:02.900 INFO:tasks.workunit.client.1.smithi117.stdout:6/367: dread - d5/d1d/d28/d32/f51 zero size 2024-02-16T19:29:02.901 INFO:tasks.workunit.client.1.smithi117.stdout:6/368: write d5/d4b/d44/d52/f5f [839371,127917] 0 2024-02-16T19:29:02.902 INFO:tasks.workunit.client.1.smithi117.stdout:6/369: fsync d5/d4b/f29 0 2024-02-16T19:29:02.903 INFO:tasks.workunit.client.0.smithi022.stdout:0/509: dwrite d10/d11/d27/f52 [0,4194304] 0 2024-02-16T19:29:02.904 INFO:tasks.workunit.client.0.smithi022.stdout:1/493: dwrite d6/d8/d22/d16/d17/d1d/f21 [4194304,4194304] 0 2024-02-16T19:29:02.904 INFO:tasks.workunit.client.1.smithi117.stdout:8/355: mkdir d4/d7/d37/d3a/d43/d44/d6e 0 2024-02-16T19:29:02.906 INFO:tasks.workunit.client.0.smithi022.stdout:1/494: write d6/d8/f7d [736623,101836] 0 2024-02-16T19:29:02.909 INFO:tasks.workunit.client.1.smithi117.stdout:9/302: rename dd/d24 to dd/d23/d64 0 2024-02-16T19:29:02.914 INFO:tasks.workunit.client.0.smithi022.stdout:4/369: mknod d4/d6/d14/d64/d4e/d54/d5a/d63/c74 0 2024-02-16T19:29:02.915 INFO:tasks.workunit.client.1.smithi117.stdout:5/460: link d2/l11 d2/d9/d3c/d24/d50/d96/l99 0 2024-02-16T19:29:02.916 INFO:tasks.workunit.client.1.smithi117.stdout:0/405: link d0/d9/d14/d1d/d60/d76/l79 d0/d9/d22/d27/d31/d6d/l9a 0 2024-02-16T19:29:02.916 INFO:tasks.workunit.client.1.smithi117.stdout:1/307: dread d2/d1d/d1f/f23 [0,4194304] 0 2024-02-16T19:29:02.917 INFO:tasks.workunit.client.1.smithi117.stdout:1/308: dread - d2/f5b zero size 2024-02-16T19:29:02.917 INFO:tasks.workunit.client.0.smithi022.stdout:4/370: write d4/d6/dd/d22/d30/f26 [1424690,52494] 0 2024-02-16T19:29:02.918 INFO:tasks.workunit.client.1.smithi117.stdout:0/406: dread d0/d9/d14/d23/d33/f5c [0,4194304] 0 2024-02-16T19:29:02.918 INFO:tasks.workunit.client.1.smithi117.stdout:1/309: truncate d2/d19/f2f 9095185 0 2024-02-16T19:29:02.919 INFO:tasks.workunit.client.1.smithi117.stdout:2/318: creat d0/d3/d22/f5a x:0 0 0 2024-02-16T19:29:02.920 INFO:tasks.workunit.client.1.smithi117.stdout:1/310: write d2/d1d/f25 [1062628,40217] 0 2024-02-16T19:29:02.920 INFO:tasks.workunit.client.1.smithi117.stdout:4/371: link d0/da/d21/d3d/d28/d64/l6e d0/da/d21/d3d/l7a 0 2024-02-16T19:29:02.921 INFO:tasks.workunit.client.0.smithi022.stdout:4/371: dread d4/d6/d14/d64/f34 [0,4194304] 0 2024-02-16T19:29:02.922 INFO:tasks.workunit.client.0.smithi022.stdout:3/370: unlink d7/d21/f52 0 2024-02-16T19:29:02.923 INFO:tasks.workunit.client.1.smithi117.stdout:1/311: truncate d2/f5b 391194 0 2024-02-16T19:29:02.923 INFO:tasks.workunit.client.1.smithi117.stdout:2/319: truncate d0/d3/d22/f28 9340423 0 2024-02-16T19:29:02.924 INFO:tasks.workunit.client.0.smithi022.stdout:3/371: chown d7/d12/d18/dd/d24/d58/d6b/f89 137 1 2024-02-16T19:29:02.925 INFO:tasks.workunit.client.1.smithi117.stdout:4/372: truncate d0/da/d21/d3d/d28/f39 8918431 0 2024-02-16T19:29:02.926 INFO:tasks.workunit.client.0.smithi022.stdout:3/372: write d7/d12/d59/f85 [396788,81042] 0 2024-02-16T19:29:02.926 INFO:tasks.workunit.client.1.smithi117.stdout:0/407: dwrite d0/d30/f64 [0,4194304] 0 2024-02-16T19:29:02.926 INFO:tasks.workunit.client.1.smithi117.stdout:4/373: dread - d0/f72 zero size 2024-02-16T19:29:02.930 INFO:tasks.workunit.client.0.smithi022.stdout:6/395: mknod d1/d8/da/d17/d8b/c93 0 2024-02-16T19:29:02.932 INFO:tasks.workunit.client.1.smithi117.stdout:1/312: dwrite d2/d1d/d60/d20/f64 [0,4194304] 0 2024-02-16T19:29:02.932 INFO:tasks.workunit.client.1.smithi117.stdout:3/434: truncate d1/d1b/f2d 208341 0 2024-02-16T19:29:02.933 INFO:tasks.workunit.client.1.smithi117.stdout:3/435: chown d1/d1b/c6f 23 1 2024-02-16T19:29:02.941 INFO:tasks.workunit.client.0.smithi022.stdout:0/510: creat d10/d11/d27/fa4 x:0 0 0 2024-02-16T19:29:02.981 INFO:tasks.workunit.client.1.smithi117.stdout:6/370: rename d5/d1d/d28 to d5/db/d7d 0 2024-02-16T19:29:02.982 INFO:tasks.workunit.client.0.smithi022.stdout:8/379: creat d3/f8d x:0 0 0 2024-02-16T19:29:03.018 INFO:tasks.workunit.client.0.smithi022.stdout:4/372: symlink d4/d6/d3a/l75 0 2024-02-16T19:29:03.018 INFO:tasks.workunit.client.1.smithi117.stdout:5/461: rmdir d2/d3/d21/d49 39 2024-02-16T19:29:03.019 INFO:tasks.workunit.client.1.smithi117.stdout:5/462: read d2/d3/d21/d2f/d30/d38/d51/f7e [2050023,72006] 0 2024-02-16T19:29:03.020 INFO:tasks.workunit.client.0.smithi022.stdout:5/412: mknod dd/d1f/d76/c86 0 2024-02-16T19:29:03.020 INFO:tasks.workunit.client.1.smithi117.stdout:5/463: read d2/d9/f40 [1303645,16216] 0 2024-02-16T19:29:03.021 INFO:tasks.workunit.client.0.smithi022.stdout:5/413: readlink dd/d1f/d48/d21/d2f/d52/d5c/l67 0 2024-02-16T19:29:03.024 INFO:tasks.workunit.client.0.smithi022.stdout:6/396: creat d1/d8/da/d17/d53/d7f/f94 x:0 0 0 2024-02-16T19:29:03.026 INFO:tasks.workunit.client.0.smithi022.stdout:7/405: link d9/da/d2c/d2f/l70 d9/da/d2c/d46/d73/l84 0 2024-02-16T19:29:03.027 INFO:tasks.workunit.client.0.smithi022.stdout:7/406: chown d9/da/d2c/d6d/f7e 113272 1 2024-02-16T19:29:03.031 INFO:tasks.workunit.client.0.smithi022.stdout:6/397: dwrite d1/d8/f1c [0,4194304] 0 2024-02-16T19:29:03.031 INFO:tasks.workunit.client.1.smithi117.stdout:2/320: dread d0/f19 [0,4194304] 0 2024-02-16T19:29:03.033 INFO:tasks.workunit.client.0.smithi022.stdout:6/398: fdatasync d1/d8/da/d17/d53/f58 0 2024-02-16T19:29:03.033 INFO:tasks.workunit.client.0.smithi022.stdout:0/511: creat d10/d11/d79/d8b/fa5 x:0 0 0 2024-02-16T19:29:03.034 INFO:tasks.workunit.client.0.smithi022.stdout:6/399: dread - d1/d8/d3c/d77/f80 zero size 2024-02-16T19:29:03.035 INFO:tasks.workunit.client.1.smithi117.stdout:5/464: dread d2/d4c/d54/f43 [0,4194304] 0 2024-02-16T19:29:03.038 INFO:tasks.workunit.client.1.smithi117.stdout:0/408: mknod d0/d9/d14/d23/c9b 0 2024-02-16T19:29:03.040 INFO:tasks.workunit.client.1.smithi117.stdout:4/374: mknod d0/da/d21/d3d/d28/d2f/c7b 0 2024-02-16T19:29:03.044 INFO:tasks.workunit.client.1.smithi117.stdout:3/436: symlink d1/dd/d5c/d85/l91 0 2024-02-16T19:29:03.047 INFO:tasks.workunit.client.1.smithi117.stdout:8/356: mkdir d4/d7/d6a/d6f 0 2024-02-16T19:29:03.047 INFO:tasks.workunit.client.0.smithi022.stdout:4/373: dread d4/d6/f16 [0,4194304] 0 2024-02-16T19:29:03.049 INFO:tasks.workunit.client.1.smithi117.stdout:1/313: rename d2/d1d/d1f/f23 to d2/d1d/d60/f66 0 2024-02-16T19:29:03.051 INFO:tasks.workunit.client.1.smithi117.stdout:6/371: creat d5/d4b/d44/d52/f7e x:0 0 0 2024-02-16T19:29:03.052 INFO:tasks.workunit.client.0.smithi022.stdout:0/512: symlink d10/d11/d79/d8b/la6 0 2024-02-16T19:29:03.052 INFO:tasks.workunit.client.0.smithi022.stdout:1/495: write d6/d8/d22/d3a/f44 [634019,108634] 0 2024-02-16T19:29:03.053 INFO:tasks.workunit.client.0.smithi022.stdout:1/496: fsync d6/f5b 0 2024-02-16T19:29:03.053 INFO:tasks.workunit.client.1.smithi117.stdout:9/303: creat dd/d47/d5b/d5f/f65 x:0 0 0 2024-02-16T19:29:03.054 INFO:tasks.workunit.client.0.smithi022.stdout:6/400: dread - d1/d8/da/d17/d85/f2b zero size 2024-02-16T19:29:03.056 INFO:tasks.workunit.client.0.smithi022.stdout:6/401: chown d1/d8/da/d17/d53/f7d 12 1 2024-02-16T19:29:03.060 INFO:tasks.workunit.client.0.smithi022.stdout:3/373: rename d7/d12/d18/dd/f2e to d7/d12/d18/dd/d24/d58/f95 0 2024-02-16T19:29:03.065 INFO:tasks.workunit.client.0.smithi022.stdout:4/374: symlink d4/d6/d14/d64/d66/l76 0 2024-02-16T19:29:03.069 INFO:tasks.workunit.client.1.smithi117.stdout:2/321: creat d0/d3b/f5b x:0 0 0 2024-02-16T19:29:03.069 INFO:tasks.workunit.client.0.smithi022.stdout:7/407: getdents d9/da/d4e/d83 0 2024-02-16T19:29:03.071 INFO:tasks.workunit.client.1.smithi117.stdout:5/465: read d2/d3/d21/d2f/d30/d4e/f34 [281408,33548] 0 2024-02-16T19:29:03.072 INFO:tasks.workunit.client.0.smithi022.stdout:4/375: dread d4/d6/dd/d22/d30/f26 [0,4194304] 0 2024-02-16T19:29:03.072 INFO:tasks.workunit.client.0.smithi022.stdout:4/376: readlink d4/d6/l52 0 2024-02-16T19:29:03.074 INFO:tasks.workunit.client.0.smithi022.stdout:4/377: dread - d4/d6/d14/d64/d4e/d54/d5a/f5f zero size 2024-02-16T19:29:03.075 INFO:tasks.workunit.client.0.smithi022.stdout:4/378: write d4/d6/d14/d64/d4e/d54/d5a/f5f [980701,88379] 0 2024-02-16T19:29:03.078 INFO:tasks.workunit.client.0.smithi022.stdout:1/497: symlink d6/d8/d22/d16/d17/d63/d6f/l93 0 2024-02-16T19:29:03.079 INFO:tasks.workunit.client.0.smithi022.stdout:1/498: read - d6/d8/d22/d16/d17/d63/f69 zero size 2024-02-16T19:29:03.080 INFO:tasks.workunit.client.1.smithi117.stdout:4/375: dwrite d0/da/d4a/f78 [0,4194304] 0 2024-02-16T19:29:03.081 INFO:tasks.workunit.client.0.smithi022.stdout:4/379: dread d4/d6/d14/d64/d4e/d54/d5a/f5f [0,4194304] 0 2024-02-16T19:29:03.081 INFO:tasks.workunit.client.0.smithi022.stdout:1/499: chown d6/d8/d22/d16/d17/d26/l38 2 1 2024-02-16T19:29:03.082 INFO:tasks.workunit.client.0.smithi022.stdout:1/500: write d6/d66/f71 [775385,22062] 0 2024-02-16T19:29:03.082 INFO:tasks.workunit.client.1.smithi117.stdout:8/357: mkdir d4/d7/d37/d3a/d54/d70 0 2024-02-16T19:29:03.086 INFO:tasks.workunit.client.1.smithi117.stdout:1/314: creat d2/d1d/d60/d20/d49/f67 x:0 0 0 2024-02-16T19:29:03.087 INFO:tasks.workunit.client.1.smithi117.stdout:6/372: write d5/d4b/f1e [4076120,9442] 0 2024-02-16T19:29:03.089 INFO:tasks.workunit.client.0.smithi022.stdout:5/414: creat dd/d1f/d48/d21/d2f/d40/d4e/f87 x:0 0 0 2024-02-16T19:29:03.092 INFO:tasks.workunit.client.1.smithi117.stdout:7/347: getdents d4/d1c/d39 0 2024-02-16T19:29:03.094 INFO:tasks.workunit.client.0.smithi022.stdout:7/408: mknod d9/da/d2c/c85 0 2024-02-16T19:29:03.099 INFO:tasks.workunit.client.0.smithi022.stdout:8/380: getdents d3/d12/d17 0 2024-02-16T19:29:03.099 INFO:tasks.workunit.client.1.smithi117.stdout:5/466: mknod d2/d9/d3c/d24/d50/d52/d74/c9a 0 2024-02-16T19:29:03.100 INFO:tasks.workunit.client.0.smithi022.stdout:7/409: dread d9/da/d1f/d40/f61 [0,4194304] 0 2024-02-16T19:29:03.104 INFO:tasks.workunit.client.1.smithi117.stdout:4/376: rmdir d0/da/d21/d3d/d2d 39 2024-02-16T19:29:03.106 INFO:tasks.workunit.client.1.smithi117.stdout:4/377: dread d0/da/d21/d3d/d28/d2f/f31 [0,4194304] 0 2024-02-16T19:29:03.111 INFO:tasks.workunit.client.0.smithi022.stdout:5/415: creat dd/d1f/d2e/f88 x:0 0 0 2024-02-16T19:29:03.113 INFO:tasks.workunit.client.1.smithi117.stdout:9/304: link dd/f19 dd/d47/d48/f66 0 2024-02-16T19:29:03.114 INFO:tasks.workunit.client.0.smithi022.stdout:0/513: link d10/f6d d10/d11/d27/d3f/d58/d60/fa7 0 2024-02-16T19:29:03.115 INFO:tasks.workunit.client.1.smithi117.stdout:7/348: truncate d4/d1c/d39/f54 956654 0 2024-02-16T19:29:03.115 INFO:tasks.workunit.client.0.smithi022.stdout:0/514: stat d10/d11/d50/d65 0 2024-02-16T19:29:03.118 INFO:tasks.workunit.client.1.smithi117.stdout:2/322: mknod d0/d3/d10/c5c 0 2024-02-16T19:29:03.119 INFO:tasks.workunit.client.0.smithi022.stdout:1/501: dread d6/d8/d22/d47/d54/f62 [0,4194304] 0 2024-02-16T19:29:03.119 INFO:tasks.workunit.client.0.smithi022.stdout:5/416: dwrite dd/d1f/d48/d21/f24 [4194304,4194304] 0 2024-02-16T19:29:03.120 INFO:tasks.workunit.client.0.smithi022.stdout:1/502: chown d6/d8/d22/d47/d54/f8a 1091275560 1 2024-02-16T19:29:03.120 INFO:tasks.workunit.client.0.smithi022.stdout:8/381: mknod d3/d25/d33/c8e 0 2024-02-16T19:29:03.121 INFO:tasks.workunit.client.1.smithi117.stdout:3/437: link d1/dd/c6e d1/d30/d52/d56/c92 0 2024-02-16T19:29:03.122 INFO:tasks.workunit.client.1.smithi117.stdout:3/438: readlink d1/d1b/d1e/d29/l4a 0 2024-02-16T19:29:03.122 INFO:tasks.workunit.client.1.smithi117.stdout:0/409: truncate d0/d9/d14/d1d/d60/d76/d7b/d8d/f2e 3073093 0 2024-02-16T19:29:03.122 INFO:tasks.workunit.client.1.smithi117.stdout:3/439: dread - d1/d30/f90 zero size 2024-02-16T19:29:03.123 INFO:tasks.workunit.client.0.smithi022.stdout:6/402: truncate d1/d8/da/d17/d85/d56/f90 1630489 0 2024-02-16T19:29:03.123 INFO:tasks.workunit.client.1.smithi117.stdout:3/440: readlink d1/dd/d5c/l7c 0 2024-02-16T19:29:03.125 INFO:tasks.workunit.client.1.smithi117.stdout:8/358: link d4/d7/f3d d4/d12/d2a/f71 0 2024-02-16T19:29:03.128 INFO:tasks.workunit.client.0.smithi022.stdout:8/382: dwrite d3/d12/d17/d64/f7c [0,4194304] 0 2024-02-16T19:29:03.130 INFO:tasks.workunit.client.0.smithi022.stdout:7/410: symlink d9/da/d1f/d67/l86 0 2024-02-16T19:29:03.131 INFO:tasks.workunit.client.0.smithi022.stdout:3/374: truncate d7/d12/d18/dd/d3c/f65 447016 0 2024-02-16T19:29:03.133 INFO:tasks.workunit.client.0.smithi022.stdout:3/375: dread - d7/d12/d18/dd/d24/d58/f8b zero size 2024-02-16T19:29:03.135 INFO:tasks.workunit.client.1.smithi117.stdout:1/315: fsync d2/d1d/d1f/f46 0 2024-02-16T19:29:03.135 INFO:tasks.workunit.client.1.smithi117.stdout:6/373: symlink d5/d4b/d71/l7f 0 2024-02-16T19:29:03.135 INFO:tasks.workunit.client.0.smithi022.stdout:3/376: read d7/d21/f7c [419406,99347] 0 2024-02-16T19:29:03.136 INFO:tasks.workunit.client.1.smithi117.stdout:6/374: write d5/d4b/f46 [1083420,14002] 0 2024-02-16T19:29:03.136 INFO:tasks.workunit.client.1.smithi117.stdout:9/305: symlink dd/l67 0 2024-02-16T19:29:03.140 INFO:tasks.workunit.client.1.smithi117.stdout:7/349: rename d4/d4a to d4/d1c/d87 0 2024-02-16T19:29:03.142 INFO:tasks.workunit.client.1.smithi117.stdout:9/306: dread dd/d45/d62/f52 [0,4194304] 0 2024-02-16T19:29:03.143 INFO:tasks.workunit.client.1.smithi117.stdout:6/375: dwrite d5/d4b/d20/f21 [0,4194304] 0 2024-02-16T19:29:03.143 INFO:tasks.workunit.client.1.smithi117.stdout:9/307: chown dd/d23/f25 0 1 2024-02-16T19:29:03.145 INFO:tasks.workunit.client.1.smithi117.stdout:0/410: creat d0/d9/d14/d1d/d60/d76/d7b/f9c x:0 0 0 2024-02-16T19:29:03.149 INFO:tasks.workunit.client.0.smithi022.stdout:8/383: dread d3/d6/f27 [0,4194304] 0 2024-02-16T19:29:03.152 INFO:tasks.workunit.client.0.smithi022.stdout:0/515: getdents d10/d11/d27/d3f/d94 0 2024-02-16T19:29:03.154 INFO:tasks.workunit.client.1.smithi117.stdout:8/359: dread d4/d7/d37/d3a/f61 [0,4194304] 0 2024-02-16T19:29:03.154 INFO:tasks.workunit.client.1.smithi117.stdout:4/378: symlink d0/da/d21/d3d/d2d/l7c 0 2024-02-16T19:29:03.155 INFO:tasks.workunit.client.1.smithi117.stdout:4/379: chown d0/f77 2045197 1 2024-02-16T19:29:03.155 INFO:tasks.workunit.client.0.smithi022.stdout:3/377: creat d7/d21/f96 x:0 0 0 2024-02-16T19:29:03.158 INFO:tasks.workunit.client.1.smithi117.stdout:1/316: mkdir d2/d1d/d60/d68 0 2024-02-16T19:29:03.166 INFO:tasks.workunit.client.1.smithi117.stdout:7/350: creat d4/d13/d63/f88 x:0 0 0 2024-02-16T19:29:03.168 INFO:tasks.workunit.client.0.smithi022.stdout:8/384: mkdir d3/d12/d17/d64/d8f 0 2024-02-16T19:29:03.169 INFO:tasks.workunit.client.0.smithi022.stdout:8/385: fsync d3/d12/d2b/d46/f53 0 2024-02-16T19:29:03.170 INFO:tasks.workunit.client.1.smithi117.stdout:2/323: link d0/f1e d0/d3/d22/f5d 0 2024-02-16T19:29:03.171 INFO:tasks.workunit.client.0.smithi022.stdout:8/386: write d3/d12/d2b/d54/d74/f82 [223542,113441] 0 2024-02-16T19:29:03.172 INFO:tasks.workunit.client.0.smithi022.stdout:0/516: symlink d10/d11/d27/la8 0 2024-02-16T19:29:03.178 INFO:tasks.workunit.client.1.smithi117.stdout:0/411: unlink d0/d9/d22/d27/d31/l5d 0 2024-02-16T19:29:03.179 INFO:tasks.workunit.client.0.smithi022.stdout:3/378: dwrite d7/d12/d18/dd/d24/d58/d6b/f78 [0,4194304] 0 2024-02-16T19:29:03.180 INFO:tasks.workunit.client.0.smithi022.stdout:3/379: chown d7/d12/d18/f88 3756055 1 2024-02-16T19:29:03.182 INFO:tasks.workunit.client.0.smithi022.stdout:3/380: fdatasync d7/d12/d59/f85 0 2024-02-16T19:29:03.182 INFO:tasks.workunit.client.1.smithi117.stdout:0/412: dread d0/f40 [0,4194304] 0 2024-02-16T19:29:03.182 INFO:tasks.workunit.client.1.smithi117.stdout:0/413: chown d0/l8 113714545 1 2024-02-16T19:29:03.183 INFO:tasks.workunit.client.1.smithi117.stdout:0/414: dread - d0/d9/d14/d1d/d60/d76/d7b/d8d/f97 zero size 2024-02-16T19:29:03.183 INFO:tasks.workunit.client.0.smithi022.stdout:6/403: rmdir d1/d8/da/d29/d78 0 2024-02-16T19:29:03.185 INFO:tasks.workunit.client.0.smithi022.stdout:6/404: fsync d1/d8/da/d17/d85/f36 0 2024-02-16T19:29:03.187 INFO:tasks.workunit.client.1.smithi117.stdout:8/360: rename d4/d7/d37/d3a/d43/d44/d6e to d4/d7/d6a/d72 0 2024-02-16T19:29:03.189 INFO:tasks.workunit.client.1.smithi117.stdout:8/361: write d4/d7/d37/f63 [528033,120259] 0 2024-02-16T19:29:03.189 INFO:tasks.workunit.client.0.smithi022.stdout:0/517: creat d10/d11/d50/fa9 x:0 0 0 2024-02-16T19:29:03.190 INFO:tasks.workunit.client.1.smithi117.stdout:1/317: mkdir d2/d1d/d60/d20/d69 0 2024-02-16T19:29:03.190 INFO:tasks.workunit.client.0.smithi022.stdout:0/518: readlink d10/d11/d50/d65/l82 0 2024-02-16T19:29:03.196 INFO:tasks.workunit.client.1.smithi117.stdout:3/441: link d1/c3 d1/d83/c93 0 2024-02-16T19:29:03.199 INFO:tasks.workunit.client.0.smithi022.stdout:6/405: mkdir d1/d8/d23/d95 0 2024-02-16T19:29:03.206 INFO:tasks.workunit.client.0.smithi022.stdout:1/503: link d6/d8/d22/d16/d4f/l92 d6/d8/d22/d16/d17/l94 0 2024-02-16T19:29:03.207 INFO:tasks.workunit.client.0.smithi022.stdout:1/504: chown d6/d8/d22 2387 1 2024-02-16T19:29:03.208 INFO:tasks.workunit.client.0.smithi022.stdout:1/505: write d6/d8/f7d [1038727,123417] 0 2024-02-16T19:29:03.209 INFO:tasks.workunit.client.0.smithi022.stdout:1/506: truncate d6/f8e 964076 0 2024-02-16T19:29:03.215 INFO:tasks.workunit.client.1.smithi117.stdout:9/308: creat dd/d23/f68 x:0 0 0 2024-02-16T19:29:03.216 INFO:tasks.workunit.client.1.smithi117.stdout:1/318: rmdir d2/d1d 39 2024-02-16T19:29:03.216 INFO:tasks.workunit.client.1.smithi117.stdout:9/309: truncate dd/d45/f3c 868355 0 2024-02-16T19:29:03.217 INFO:tasks.workunit.client.1.smithi117.stdout:3/442: rmdir d1/dd/d3e 39 2024-02-16T19:29:03.217 INFO:tasks.workunit.client.1.smithi117.stdout:9/310: readlink l8 0 2024-02-16T19:29:03.217 INFO:tasks.workunit.client.1.smithi117.stdout:3/443: chown d1/dd/c35 45713802 1 2024-02-16T19:29:03.217 INFO:tasks.workunit.client.1.smithi117.stdout:9/311: read - dd/d47/f57 zero size 2024-02-16T19:29:03.218 INFO:tasks.workunit.client.1.smithi117.stdout:3/444: dread - d1/d1b/f21 zero size 2024-02-16T19:29:03.220 INFO:tasks.workunit.client.0.smithi022.stdout:1/507: unlink d6/d8/d22/d16/d17/d63/d6f/l88 0 2024-02-16T19:29:03.223 INFO:tasks.workunit.client.1.smithi117.stdout:5/467: sync 2024-02-16T19:29:03.223 INFO:tasks.workunit.client.1.smithi117.stdout:5/468: truncate d2/d3/f65 849446 0 2024-02-16T19:29:03.224 INFO:tasks.workunit.client.1.smithi117.stdout:8/362: mkdir d4/d73 0 2024-02-16T19:29:03.225 INFO:tasks.workunit.client.0.smithi022.stdout:8/387: truncate d3/f15 164971 0 2024-02-16T19:29:03.226 INFO:tasks.workunit.client.0.smithi022.stdout:0/519: write d10/f6e [1127237,14439] 0 2024-02-16T19:29:03.230 INFO:tasks.workunit.client.1.smithi117.stdout:2/324: creat d0/f5e x:0 0 0 2024-02-16T19:29:03.230 INFO:tasks.workunit.client.1.smithi117.stdout:2/325: readlink d0/d3/d22/l45 0 2024-02-16T19:29:03.233 INFO:tasks.workunit.client.0.smithi022.stdout:6/406: dwrite d1/d8/da/d17/d85/f2b [0,4194304] 0 2024-02-16T19:29:03.234 INFO:tasks.workunit.client.0.smithi022.stdout:6/407: stat d1/d8/da/d17/d48 0 2024-02-16T19:29:03.235 INFO:tasks.workunit.client.0.smithi022.stdout:8/388: mknod d3/d12/d2b/d2e/c90 0 2024-02-16T19:29:03.237 INFO:tasks.workunit.client.0.smithi022.stdout:6/408: chown d1/d8/da/d17/d85/d56/c71 3428 1 2024-02-16T19:29:03.238 INFO:tasks.workunit.client.1.smithi117.stdout:0/415: link d0/cc d0/d9/d14/d23/d33/c9d 0 2024-02-16T19:29:03.242 INFO:tasks.workunit.client.0.smithi022.stdout:1/508: link d6/d8/d22/d16/d4f/f8c d6/d8/d22/d16/d17/d5d/f95 0 2024-02-16T19:29:03.245 INFO:tasks.workunit.client.0.smithi022.stdout:3/381: dwrite d7/d12/d18/dd/d24/d29/f2c [0,4194304] 0 2024-02-16T19:29:03.250 INFO:tasks.workunit.client.1.smithi117.stdout:4/380: rename d0/da/d21/d3d/d28/d64/d48/c6b to d0/da/d21/d3d/c7d 0 2024-02-16T19:29:03.252 INFO:tasks.workunit.client.0.smithi022.stdout:8/389: creat d3/d12/d2b/d46/f91 x:0 0 0 2024-02-16T19:29:03.260 INFO:tasks.workunit.client.1.smithi117.stdout:0/416: unlink d0/d9/d22/d27/d83/c43 0 2024-02-16T19:29:03.262 INFO:tasks.workunit.client.1.smithi117.stdout:3/445: creat d1/dd/d8e/f94 x:0 0 0 2024-02-16T19:29:03.266 INFO:tasks.workunit.client.1.smithi117.stdout:7/351: rename d4/d1c/d24 to d4/d1c/d87/d89 0 2024-02-16T19:29:03.268 INFO:tasks.workunit.client.0.smithi022.stdout:8/390: mkdir d3/d12/d92 0 2024-02-16T19:29:03.268 INFO:tasks.workunit.client.0.smithi022.stdout:1/509: dwrite d6/d8/d22/d16/d17/d1d/f33 [4194304,4194304] 0 2024-02-16T19:29:03.269 INFO:tasks.workunit.client.1.smithi117.stdout:8/363: getdents d4/d73 0 2024-02-16T19:29:03.271 INFO:tasks.workunit.client.1.smithi117.stdout:2/326: mknod d0/d3/c5f 0 2024-02-16T19:29:03.272 INFO:tasks.workunit.client.0.smithi022.stdout:8/391: write d3/d12/d17/d64/f7c [1074284,30690] 0 2024-02-16T19:29:03.272 INFO:tasks.workunit.client.1.smithi117.stdout:0/417: dread d0/d9/d22/d27/f4d [0,4194304] 0 2024-02-16T19:29:03.273 INFO:tasks.workunit.client.1.smithi117.stdout:0/418: read d0/f1 [491399,31552] 0 2024-02-16T19:29:03.275 INFO:tasks.workunit.client.1.smithi117.stdout:1/319: mknod d2/d1d/d60/d68/c6a 0 2024-02-16T19:29:03.276 INFO:tasks.workunit.client.1.smithi117.stdout:2/327: dread d0/d13/f36 [0,4194304] 0 2024-02-16T19:29:03.277 INFO:tasks.workunit.client.1.smithi117.stdout:0/419: truncate d0/d9/d22/d27/d31/f38 5445364 0 2024-02-16T19:29:03.278 INFO:tasks.workunit.client.0.smithi022.stdout:6/409: creat d1/d8/da/d17/f96 x:0 0 0 2024-02-16T19:29:03.279 INFO:tasks.workunit.client.0.smithi022.stdout:6/410: chown d1/d8/d23/d95 112 1 2024-02-16T19:29:03.282 INFO:tasks.workunit.client.1.smithi117.stdout:9/312: rename dd/d47/f56 to dd/d47/d5b/f69 0 2024-02-16T19:29:03.283 INFO:tasks.workunit.client.0.smithi022.stdout:8/392: truncate d3/d25/f43 368884 0 2024-02-16T19:29:03.284 INFO:tasks.workunit.client.1.smithi117.stdout:6/376: sync 2024-02-16T19:29:03.284 INFO:tasks.workunit.client.1.smithi117.stdout:7/352: symlink d4/d13/l8a 0 2024-02-16T19:29:03.285 INFO:tasks.workunit.client.1.smithi117.stdout:7/353: chown d4/d1c/c41 799 1 2024-02-16T19:29:03.285 INFO:tasks.workunit.client.1.smithi117.stdout:6/377: readlink d5/d4b/d71/l7f 0 2024-02-16T19:29:03.286 INFO:tasks.workunit.client.0.smithi022.stdout:1/510: fdatasync d6/d8/d22/d16/f2e 0 2024-02-16T19:29:03.287 INFO:tasks.workunit.client.1.smithi117.stdout:8/364: dread d4/d7/d37/d50/f5b [0,4194304] 0 2024-02-16T19:29:03.288 INFO:tasks.workunit.client.0.smithi022.stdout:3/382: truncate d7/d12/f25 3520955 0 2024-02-16T19:29:03.289 INFO:tasks.workunit.client.0.smithi022.stdout:1/511: dread d6/d8/d22/d16/f2e [0,4194304] 0 2024-02-16T19:29:03.289 INFO:tasks.workunit.client.0.smithi022.stdout:3/383: write d7/d12/d18/dd/d3c/f92 [640570,121178] 0 2024-02-16T19:29:03.290 INFO:tasks.workunit.client.0.smithi022.stdout:6/411: rename d1/d8/da/d29/d61/d57/l6e to d1/d8/da/d17/d8b/l97 0 2024-02-16T19:29:03.291 INFO:tasks.workunit.client.1.smithi117.stdout:3/446: dread d1/d31/f36 [0,4194304] 0 2024-02-16T19:29:03.291 INFO:tasks.workunit.client.0.smithi022.stdout:6/412: fdatasync d1/d8/da/d17/d48/f74 0 2024-02-16T19:29:03.295 INFO:tasks.workunit.client.1.smithi117.stdout:1/320: unlink d2/d1d/d1f/f46 0 2024-02-16T19:29:03.296 INFO:tasks.workunit.client.1.smithi117.stdout:1/321: write d2/d4/f16 [634037,50545] 0 2024-02-16T19:29:03.298 INFO:tasks.workunit.client.0.smithi022.stdout:1/512: mkdir d6/d8/d6c/d96 0 2024-02-16T19:29:03.299 INFO:tasks.workunit.client.0.smithi022.stdout:1/513: chown d6/d8/d22/d16/d17/d26/c87 1 1 2024-02-16T19:29:03.300 INFO:tasks.workunit.client.1.smithi117.stdout:6/378: dread d5/db/d7d/d32/f62 [0,4194304] 0 2024-02-16T19:29:03.300 INFO:tasks.workunit.client.0.smithi022.stdout:3/384: mknod d7/d21/c97 0 2024-02-16T19:29:03.367 INFO:tasks.workunit.client.1.smithi117.stdout:5/469: sync 2024-02-16T19:29:03.372 INFO:tasks.workunit.client.1.smithi117.stdout:0/420: creat d0/d9/d14/d1d/f9e x:0 0 0 2024-02-16T19:29:03.373 INFO:tasks.workunit.client.0.smithi022.stdout:2/313: sync 2024-02-16T19:29:03.373 INFO:tasks.workunit.client.0.smithi022.stdout:9/436: sync 2024-02-16T19:29:03.374 INFO:tasks.workunit.client.0.smithi022.stdout:2/314: chown d0/d2/f54 12479092 1 2024-02-16T19:29:03.376 INFO:tasks.workunit.client.0.smithi022.stdout:8/393: mknod d3/d12/d2b/c93 0 2024-02-16T19:29:03.377 INFO:tasks.workunit.client.0.smithi022.stdout:8/394: chown d3/d12/d17/d64/f80 96255627 1 2024-02-16T19:29:03.379 INFO:tasks.workunit.client.0.smithi022.stdout:2/315: dwrite d0/f17 [0,4194304] 0 2024-02-16T19:29:03.398 INFO:tasks.workunit.client.0.smithi022.stdout:2/316: dwrite d0/d2/d18/f34 [0,4194304] 0 2024-02-16T19:29:03.399 INFO:tasks.workunit.client.0.smithi022.stdout:1/514: symlink d6/d8/d22/d3f/l97 0 2024-02-16T19:29:03.401 INFO:tasks.workunit.client.1.smithi117.stdout:7/354: mknod d4/d1c/d87/d89/d60/c8b 0 2024-02-16T19:29:03.401 INFO:tasks.workunit.client.0.smithi022.stdout:1/515: write d6/d8/d22/d67/f77 [865232,95916] 0 2024-02-16T19:29:03.409 INFO:tasks.workunit.client.0.smithi022.stdout:1/516: dwrite d6/d8/d22/d16/f40 [0,4194304] 0 2024-02-16T19:29:03.410 INFO:tasks.workunit.client.1.smithi117.stdout:8/365: mknod d4/d7/d37/d3a/d54/c74 0 2024-02-16T19:29:03.412 INFO:tasks.workunit.client.1.smithi117.stdout:3/447: mknod d1/dd/d3b/c95 0 2024-02-16T19:29:03.418 INFO:tasks.workunit.client.1.smithi117.stdout:6/379: mknod d5/db/d7d/d32/c80 0 2024-02-16T19:29:03.420 INFO:tasks.workunit.client.1.smithi117.stdout:5/470: mkdir d2/d4c/d54/d9b 0 2024-02-16T19:29:03.421 INFO:tasks.workunit.client.1.smithi117.stdout:5/471: dread - d2/d3/d21/d2f/d30/d38/f64 zero size 2024-02-16T19:29:03.422 INFO:tasks.workunit.client.1.smithi117.stdout:5/472: dread - d2/d9/d3c/d24/d50/d52/d74/d8c/f73 zero size 2024-02-16T19:29:03.430 INFO:tasks.workunit.client.1.smithi117.stdout:1/322: symlink d2/d1d/d60/d20/d69/l6b 0 2024-02-16T19:29:03.434 INFO:tasks.workunit.client.1.smithi117.stdout:0/421: readlink d0/d9/d14/d23/d33/l7a 0 2024-02-16T19:29:03.435 INFO:tasks.workunit.client.1.smithi117.stdout:0/422: stat d0/d9/d22/d27/d83 0 2024-02-16T19:29:03.435 INFO:tasks.workunit.client.1.smithi117.stdout:0/423: chown d0/d9/d22/d27/d31/f38 7858651 1 2024-02-16T19:29:03.441 INFO:tasks.workunit.client.1.smithi117.stdout:3/448: mkdir d1/d83/d96 0 2024-02-16T19:29:03.443 INFO:tasks.workunit.client.1.smithi117.stdout:2/328: getdents d0/d29/d2f 0 2024-02-16T19:29:03.447 INFO:tasks.workunit.client.1.smithi117.stdout:6/380: fdatasync d5/f12 0 2024-02-16T19:29:03.447 INFO:tasks.workunit.client.1.smithi117.stdout:2/329: dread d0/d13/f1c [0,4194304] 0 2024-02-16T19:29:03.449 INFO:tasks.workunit.client.1.smithi117.stdout:0/424: creat d0/d9/d14/d1d/d60/d76/f9f x:0 0 0 2024-02-16T19:29:03.450 INFO:tasks.workunit.client.1.smithi117.stdout:0/425: stat d0/d9/d22/d27/d4e/f61 0 2024-02-16T19:29:03.451 INFO:tasks.workunit.client.1.smithi117.stdout:5/473: mkdir d2/d36/d6f/d9c 0 2024-02-16T19:29:03.452 INFO:tasks.workunit.client.1.smithi117.stdout:5/474: chown d2/d4c/d4f 337927496 1 2024-02-16T19:29:03.452 INFO:tasks.workunit.client.1.smithi117.stdout:9/313: getdents dd/d45/d31 0 2024-02-16T19:29:03.454 INFO:tasks.workunit.client.1.smithi117.stdout:2/330: mkdir d0/d29/d56/d60 0 2024-02-16T19:29:03.455 INFO:tasks.workunit.client.1.smithi117.stdout:6/381: mkdir d5/d4b/d44/d52/d81 0 2024-02-16T19:29:03.455 INFO:tasks.workunit.client.1.smithi117.stdout:8/366: write d4/d7/d37/d3a/f61 [3416723,127067] 0 2024-02-16T19:29:03.456 INFO:tasks.workunit.client.1.smithi117.stdout:0/426: truncate d0/d9/d22/d27/d83/f50 536938 0 2024-02-16T19:29:03.456 INFO:tasks.workunit.client.1.smithi117.stdout:1/323: dread d2/d4/f9 [0,4194304] 0 2024-02-16T19:29:03.457 INFO:tasks.workunit.client.1.smithi117.stdout:3/449: creat d1/d31/d89/f97 x:0 0 0 2024-02-16T19:29:03.458 INFO:tasks.workunit.client.1.smithi117.stdout:9/314: fsync dd/f19 0 2024-02-16T19:29:03.459 INFO:tasks.workunit.client.1.smithi117.stdout:3/450: fdatasync d1/d1b/d1e/d37/f86 0 2024-02-16T19:29:03.460 INFO:tasks.workunit.client.1.smithi117.stdout:0/427: readlink d0/d9/d14/d1d/d60/d76/d7b/d8d/d55/l7e 0 2024-02-16T19:29:03.461 INFO:tasks.workunit.client.1.smithi117.stdout:2/331: fdatasync d0/d3/d22/f5d 0 2024-02-16T19:29:03.462 INFO:tasks.workunit.client.1.smithi117.stdout:2/332: readlink d0/d13/d4d/l4e 0 2024-02-16T19:29:03.464 INFO:tasks.workunit.client.1.smithi117.stdout:2/333: write d0/d3/d22/f5a [410463,67748] 0 2024-02-16T19:29:03.464 INFO:tasks.workunit.client.1.smithi117.stdout:8/367: dwrite d4/d12/f26 [0,4194304] 0 2024-02-16T19:29:03.466 INFO:tasks.workunit.client.1.smithi117.stdout:2/334: write d0/d3/fc [4422174,107709] 0 2024-02-16T19:29:03.467 INFO:tasks.workunit.client.1.smithi117.stdout:8/368: write d4/d7/d37/f63 [408865,107193] 0 2024-02-16T19:29:03.467 INFO:tasks.workunit.client.1.smithi117.stdout:1/324: symlink d2/d1d/d60/d20/d49/l6c 0 2024-02-16T19:29:03.471 INFO:tasks.workunit.client.1.smithi117.stdout:5/475: link d2/d9/d3c/d24/d50/d52/d74/d8c/f73 d2/d5d/f9d 0 2024-02-16T19:29:03.471 INFO:tasks.workunit.client.1.smithi117.stdout:9/315: mknod dd/d45/d62/c6a 0 2024-02-16T19:29:03.475 INFO:tasks.workunit.client.1.smithi117.stdout:0/428: mknod d0/d9/d22/d27/d31/d94/ca0 0 2024-02-16T19:29:03.476 INFO:tasks.workunit.client.1.smithi117.stdout:8/369: rmdir d4/d12 39 2024-02-16T19:29:03.477 INFO:tasks.workunit.client.1.smithi117.stdout:6/382: dread d5/db/f55 [0,4194304] 0 2024-02-16T19:29:03.477 INFO:tasks.workunit.client.1.smithi117.stdout:9/316: dwrite dd/d45/f3f [0,4194304] 0 2024-02-16T19:29:03.478 INFO:tasks.workunit.client.1.smithi117.stdout:8/370: readlink d4/d7/l20 0 2024-02-16T19:29:03.478 INFO:tasks.workunit.client.1.smithi117.stdout:1/325: rename d2/d1d/d60/d20/f36 to d2/d1d/f6d 0 2024-02-16T19:29:03.479 INFO:tasks.workunit.client.1.smithi117.stdout:6/383: chown d5/d1d/l57 2 1 2024-02-16T19:29:03.480 INFO:tasks.workunit.client.1.smithi117.stdout:3/451: mkdir d1/d98 0 2024-02-16T19:29:03.481 INFO:tasks.workunit.client.1.smithi117.stdout:3/452: readlink d1/d31/l46 0 2024-02-16T19:29:03.486 INFO:tasks.workunit.client.1.smithi117.stdout:5/476: fsync d2/d1d/f3f 0 2024-02-16T19:29:03.488 INFO:tasks.workunit.client.1.smithi117.stdout:4/381: sync 2024-02-16T19:29:03.489 INFO:tasks.workunit.client.1.smithi117.stdout:1/326: symlink d2/d1d/d60/d20/l6e 0 2024-02-16T19:29:03.491 INFO:tasks.workunit.client.1.smithi117.stdout:6/384: mknod d5/db/d7d/d32/c82 0 2024-02-16T19:29:03.492 INFO:tasks.workunit.client.1.smithi117.stdout:6/385: chown d5/db/f6f 318 1 2024-02-16T19:29:03.492 INFO:tasks.workunit.client.1.smithi117.stdout:2/335: creat d0/f61 x:0 0 0 2024-02-16T19:29:03.492 INFO:tasks.workunit.client.1.smithi117.stdout:2/336: fsync d0/d29/f49 0 2024-02-16T19:29:03.493 INFO:tasks.workunit.client.1.smithi117.stdout:4/382: read d0/da/d21/f65 [5991781,38663] 0 2024-02-16T19:29:03.494 INFO:tasks.workunit.client.1.smithi117.stdout:1/327: dread d2/f53 [0,4194304] 0 2024-02-16T19:29:03.495 INFO:tasks.workunit.client.1.smithi117.stdout:8/371: creat d4/d7/d37/d3a/d65/d6c/f75 x:0 0 0 2024-02-16T19:29:03.497 INFO:tasks.workunit.client.1.smithi117.stdout:0/429: write d0/d9/d14/f1b [2319673,95684] 0 2024-02-16T19:29:03.498 INFO:tasks.workunit.client.1.smithi117.stdout:0/430: chown d0/d9/d22 149869 1 2024-02-16T19:29:03.498 INFO:tasks.workunit.client.1.smithi117.stdout:3/453: symlink d1/d98/l99 0 2024-02-16T19:29:03.501 INFO:tasks.workunit.client.1.smithi117.stdout:2/337: mknod d0/d29/c62 0 2024-02-16T19:29:03.502 INFO:tasks.workunit.client.1.smithi117.stdout:9/317: link l8 dd/l6b 0 2024-02-16T19:29:03.503 INFO:tasks.workunit.client.1.smithi117.stdout:1/328: write d2/d1d/f39 [341970,82606] 0 2024-02-16T19:29:03.508 INFO:tasks.workunit.client.1.smithi117.stdout:3/454: stat d1/dd/d5c/d85/c88 0 2024-02-16T19:29:03.510 INFO:tasks.workunit.client.1.smithi117.stdout:0/431: creat d0/d9/d14/d1d/d60/d76/d7b/d8d/fa1 x:0 0 0 2024-02-16T19:29:03.511 INFO:tasks.workunit.client.1.smithi117.stdout:2/338: creat d0/d3/d22/d37/f63 x:0 0 0 2024-02-16T19:29:03.516 INFO:tasks.workunit.client.0.smithi022.stdout:6/413: creat d1/d8/d81/f98 x:0 0 0 2024-02-16T19:29:03.516 INFO:tasks.workunit.client.0.smithi022.stdout:9/437: symlink db/d14/d16/d87/l91 0 2024-02-16T19:29:03.516 INFO:tasks.workunit.client.1.smithi117.stdout:4/383: link d0/da/d21/d3d/d28/d64/d48/f5d d0/da/d21/d3d/d2d/d6d/f7e 0 2024-02-16T19:29:03.516 INFO:tasks.workunit.client.0.smithi022.stdout:3/385: mkdir d7/d12/d18/dd/d98 0 2024-02-16T19:29:03.517 INFO:tasks.workunit.client.0.smithi022.stdout:1/517: stat d6/d8/d22/c34 0 2024-02-16T19:29:03.518 INFO:tasks.workunit.client.1.smithi117.stdout:8/372: mkdir d4/d7/d37/d3a/d54/d70/d76 0 2024-02-16T19:29:03.520 INFO:tasks.workunit.client.0.smithi022.stdout:2/317: mknod d0/d2/d18/c64 0 2024-02-16T19:29:03.521 INFO:tasks.workunit.client.0.smithi022.stdout:9/438: mkdir db/d14/d16/d27/d92 0 2024-02-16T19:29:03.524 INFO:tasks.workunit.client.1.smithi117.stdout:1/329: write d2/d4/f9 [4127319,69834] 0 2024-02-16T19:29:03.525 INFO:tasks.workunit.client.1.smithi117.stdout:6/386: dwrite d5/f9 [0,4194304] 0 2024-02-16T19:29:03.526 INFO:tasks.workunit.client.1.smithi117.stdout:1/330: readlink d2/d1d/d60/l65 0 2024-02-16T19:29:03.526 INFO:tasks.workunit.client.0.smithi022.stdout:1/518: dwrite d6/d8/d22/d47/d54/f65 [0,4194304] 0 2024-02-16T19:29:03.528 INFO:tasks.workunit.client.1.smithi117.stdout:2/339: rename d0/d29/d2f/d42 to d0/d13/d4d/d64 0 2024-02-16T19:29:03.528 INFO:tasks.workunit.client.1.smithi117.stdout:2/340: chown d0/f5e 823000562 1 2024-02-16T19:29:03.530 INFO:tasks.workunit.client.0.smithi022.stdout:9/439: creat db/d18/f93 x:0 0 0 2024-02-16T19:29:03.532 INFO:tasks.workunit.client.0.smithi022.stdout:2/318: dwrite d0/d48/f5a [0,4194304] 0 2024-02-16T19:29:03.537 INFO:tasks.workunit.client.0.smithi022.stdout:2/319: truncate d0/d32/f62 176408 0 2024-02-16T19:29:03.539 INFO:tasks.workunit.client.0.smithi022.stdout:3/386: dwrite d7/d12/d18/f42 [0,4194304] 0 2024-02-16T19:29:03.542 INFO:tasks.workunit.client.0.smithi022.stdout:1/519: dwrite d6/d66/f71 [0,4194304] 0 2024-02-16T19:29:03.543 INFO:tasks.workunit.client.1.smithi117.stdout:9/318: link dd/d23/d64/c4b dd/d47/d48/c6c 0 2024-02-16T19:29:03.543 INFO:tasks.workunit.client.0.smithi022.stdout:3/387: fdatasync d7/d12/d18/dd/d24/d58/f8b 0 2024-02-16T19:29:03.549 INFO:tasks.workunit.client.0.smithi022.stdout:6/414: creat d1/d8/da/d17/f99 x:0 0 0 2024-02-16T19:29:03.549 INFO:tasks.workunit.client.0.smithi022.stdout:9/440: mknod db/d14/d33/c94 0 2024-02-16T19:29:03.551 INFO:tasks.workunit.client.1.smithi117.stdout:3/455: mknod d1/dd/d3e/d7d/c9a 0 2024-02-16T19:29:03.551 INFO:tasks.workunit.client.1.smithi117.stdout:8/373: creat d4/d12/d31/f77 x:0 0 0 2024-02-16T19:29:03.557 INFO:tasks.workunit.client.0.smithi022.stdout:3/388: fsync d7/d12/d18/dd/d24/f3e 0 2024-02-16T19:29:03.561 INFO:tasks.workunit.client.1.smithi117.stdout:8/374: creat d4/d7/d37/d3a/d54/f78 x:0 0 0 2024-02-16T19:29:03.562 INFO:tasks.workunit.client.1.smithi117.stdout:1/331: dread d2/d4/f11 [0,4194304] 0 2024-02-16T19:29:03.562 INFO:tasks.workunit.client.1.smithi117.stdout:8/375: write d4/d12/d31/f36 [4074331,117339] 0 2024-02-16T19:29:03.563 INFO:tasks.workunit.client.1.smithi117.stdout:6/387: creat d5/f83 x:0 0 0 2024-02-16T19:29:03.570 INFO:tasks.workunit.client.0.smithi022.stdout:4/380: sync 2024-02-16T19:29:03.572 INFO:tasks.workunit.client.0.smithi022.stdout:4/381: write d4/d6/dd/df/d10/f35 [242395,129624] 0 2024-02-16T19:29:03.578 INFO:tasks.workunit.client.1.smithi117.stdout:7/355: sync 2024-02-16T19:29:03.578 INFO:tasks.workunit.client.1.smithi117.stdout:0/432: truncate d0/f10 5582233 0 2024-02-16T19:29:03.578 INFO:tasks.workunit.client.1.smithi117.stdout:6/388: read d5/d1d/d72/f77 [1542983,877] 0 2024-02-16T19:29:03.579 INFO:tasks.workunit.client.0.smithi022.stdout:2/320: rmdir d0/d60 0 2024-02-16T19:29:03.580 INFO:tasks.workunit.client.1.smithi117.stdout:0/433: chown d0/d9/d22/d27/d4e 116 1 2024-02-16T19:29:03.580 INFO:tasks.workunit.client.1.smithi117.stdout:8/376: dread d4/f53 [0,4194304] 0 2024-02-16T19:29:03.581 INFO:tasks.workunit.client.1.smithi117.stdout:3/456: link d1/dd/c5d d1/d30/d52/d56/c9b 0 2024-02-16T19:29:03.582 INFO:tasks.workunit.client.1.smithi117.stdout:3/457: fsync d1/d1b/f5f 0 2024-02-16T19:29:03.582 INFO:tasks.workunit.client.1.smithi117.stdout:3/458: dread - d1/d1b/d1e/d37/f86 zero size 2024-02-16T19:29:03.583 INFO:tasks.workunit.client.0.smithi022.stdout:1/520: creat d6/d8/d22/d16/d17/f98 x:0 0 0 2024-02-16T19:29:03.583 INFO:tasks.workunit.client.1.smithi117.stdout:3/459: write d1/d1b/f41 [708861,34545] 0 2024-02-16T19:29:03.584 INFO:tasks.workunit.client.0.smithi022.stdout:1/521: write d6/d8/d22/d16/d17/f52 [725932,79469] 0 2024-02-16T19:29:03.586 INFO:tasks.workunit.client.1.smithi117.stdout:2/341: truncate d0/d29/f2e 2297623 0 2024-02-16T19:29:03.586 INFO:tasks.workunit.client.0.smithi022.stdout:1/522: stat d6/d8/d22/f82 0 2024-02-16T19:29:03.586 INFO:tasks.workunit.client.1.smithi117.stdout:9/319: write dd/d23/f30 [1710514,49813] 0 2024-02-16T19:29:03.587 INFO:tasks.workunit.client.0.smithi022.stdout:9/441: mknod db/d14/d16/d27/d8b/c95 0 2024-02-16T19:29:03.588 INFO:tasks.workunit.client.1.smithi117.stdout:9/320: dread dd/d23/f1a [0,4194304] 0 2024-02-16T19:29:03.588 INFO:tasks.workunit.client.0.smithi022.stdout:9/442: chown db/d14/d16/d27/f68 381913 1 2024-02-16T19:29:03.589 INFO:tasks.workunit.client.1.smithi117.stdout:9/321: dread - dd/d23/d64/f39 zero size 2024-02-16T19:29:03.592 INFO:tasks.workunit.client.1.smithi117.stdout:7/356: rename d4/d13/c5d to d4/d1c/d39/d46/d55/c8c 0 2024-02-16T19:29:03.595 INFO:tasks.workunit.client.1.smithi117.stdout:8/377: fsync d4/f52 0 2024-02-16T19:29:03.596 INFO:tasks.workunit.client.0.smithi022.stdout:1/523: mkdir d6/d8/d22/d3f/d99 0 2024-02-16T19:29:03.597 INFO:tasks.workunit.client.1.smithi117.stdout:0/434: creat d0/d9/d22/d27/d31/d6d/fa2 x:0 0 0 2024-02-16T19:29:03.597 INFO:tasks.workunit.client.1.smithi117.stdout:0/435: chown d0/d9/d22/d27/d83/d7c 1790609 1 2024-02-16T19:29:03.599 INFO:tasks.workunit.client.0.smithi022.stdout:3/389: dwrite d7/d21/f7c [0,4194304] 0 2024-02-16T19:29:03.601 INFO:tasks.workunit.client.0.smithi022.stdout:9/443: rmdir db/d14/d19 39 2024-02-16T19:29:03.602 INFO:tasks.workunit.client.0.smithi022.stdout:9/444: chown db/d14/d16/d27/d8b 10472 1 2024-02-16T19:29:03.604 INFO:tasks.workunit.client.1.smithi117.stdout:2/342: symlink d0/d35/l65 0 2024-02-16T19:29:03.609 INFO:tasks.workunit.client.0.smithi022.stdout:6/415: getdents d1/d8 0 2024-02-16T19:29:03.610 INFO:tasks.workunit.client.0.smithi022.stdout:6/416: dread - d1/d8/da/d17/d85/d5b/f88 zero size 2024-02-16T19:29:03.610 INFO:tasks.workunit.client.0.smithi022.stdout:6/417: chown d1/d8/d3c 123729 1 2024-02-16T19:29:03.615 INFO:tasks.workunit.client.0.smithi022.stdout:4/382: truncate d4/f40 2440760 0 2024-02-16T19:29:03.618 INFO:tasks.workunit.client.0.smithi022.stdout:5/417: sync 2024-02-16T19:29:03.618 INFO:tasks.workunit.client.1.smithi117.stdout:3/460: rename d1/f2 to d1/d1b/d1e/d37/f9c 0 2024-02-16T19:29:03.618 INFO:tasks.workunit.client.0.smithi022.stdout:6/418: dwrite d1/d8/da/d17/d53/d7f/f94 [0,4194304] 0 2024-02-16T19:29:03.621 INFO:tasks.workunit.client.1.smithi117.stdout:7/357: unlink d4/d13/f68 0 2024-02-16T19:29:03.625 INFO:tasks.workunit.client.0.smithi022.stdout:2/321: creat d0/d2/d18/f65 x:0 0 0 2024-02-16T19:29:03.628 INFO:tasks.workunit.client.1.smithi117.stdout:2/343: write d0/f19 [3644136,44570] 0 2024-02-16T19:29:03.629 INFO:tasks.workunit.client.0.smithi022.stdout:3/390: symlink d7/d12/d18/dd/d31/d7f/l99 0 2024-02-16T19:29:03.631 INFO:tasks.workunit.client.0.smithi022.stdout:9/445: dread db/d14/d19/f17 [0,4194304] 0 2024-02-16T19:29:03.632 INFO:tasks.workunit.client.0.smithi022.stdout:4/383: truncate d4/d6/dd/d22/d30/f26 1642966 0 2024-02-16T19:29:03.633 INFO:tasks.workunit.client.0.smithi022.stdout:6/419: mknod d1/d8/d81/c9a 0 2024-02-16T19:29:03.638 INFO:tasks.workunit.client.0.smithi022.stdout:1/524: creat d6/d8/d22/f9a x:0 0 0 2024-02-16T19:29:03.641 INFO:tasks.workunit.client.1.smithi117.stdout:0/436: rename d0/d9/d22/d27/d31/f58 to d0/d9/d14/d1d/d60/d65/fa3 0 2024-02-16T19:29:03.641 INFO:tasks.workunit.client.0.smithi022.stdout:2/322: symlink d0/d48/l66 0 2024-02-16T19:29:03.642 INFO:tasks.workunit.client.1.smithi117.stdout:0/437: chown d0/d9/d14/d23/f24 5 1 2024-02-16T19:29:03.643 INFO:tasks.workunit.client.0.smithi022.stdout:3/391: mknod d7/d12/d18/dd/d24/c9a 0 2024-02-16T19:29:03.644 INFO:tasks.workunit.client.1.smithi117.stdout:3/461: read d1/d1b/f2d [21038,82421] 0 2024-02-16T19:29:03.650 INFO:tasks.workunit.client.1.smithi117.stdout:7/358: creat d4/d13/f8d x:0 0 0 2024-02-16T19:29:03.652 INFO:tasks.workunit.client.1.smithi117.stdout:1/332: getdents d2/d1d/d60 0 2024-02-16T19:29:03.653 INFO:tasks.workunit.client.0.smithi022.stdout:5/418: link f3 dd/d1f/d48/d21/d2f/d46/d5e/f89 0 2024-02-16T19:29:03.657 INFO:tasks.workunit.client.0.smithi022.stdout:2/323: symlink d0/d32/d3b/l67 0 2024-02-16T19:29:03.657 INFO:tasks.workunit.client.1.smithi117.stdout:2/344: creat d0/d3/d40/f66 x:0 0 0 2024-02-16T19:29:03.658 INFO:tasks.workunit.client.1.smithi117.stdout:2/345: readlink d0/l1a 0 2024-02-16T19:29:03.658 INFO:tasks.workunit.client.1.smithi117.stdout:9/322: creat dd/d47/f6d x:0 0 0 2024-02-16T19:29:03.659 INFO:tasks.workunit.client.1.smithi117.stdout:6/389: getdents d5/db/d7d 0 2024-02-16T19:29:03.661 INFO:tasks.workunit.client.1.smithi117.stdout:0/438: unlink d0/d9/d22/d27/c68 0 2024-02-16T19:29:03.661 INFO:tasks.workunit.client.1.smithi117.stdout:3/462: rmdir d1/dd/d3e/d7d 39 2024-02-16T19:29:03.663 INFO:tasks.workunit.client.0.smithi022.stdout:7/411: sync 2024-02-16T19:29:03.663 INFO:tasks.workunit.client.0.smithi022.stdout:8/395: sync 2024-02-16T19:29:03.663 INFO:tasks.workunit.client.0.smithi022.stdout:0/520: sync 2024-02-16T19:29:03.664 INFO:tasks.workunit.client.1.smithi117.stdout:1/333: unlink d2/d4/c1c 0 2024-02-16T19:29:03.665 INFO:tasks.workunit.client.1.smithi117.stdout:8/378: link d4/d12/d31/f40 d4/f79 0 2024-02-16T19:29:03.667 INFO:tasks.workunit.client.1.smithi117.stdout:0/439: symlink d0/d9/d14/d1d/d1e/d54/la4 0 2024-02-16T19:29:03.667 INFO:tasks.workunit.client.1.smithi117.stdout:3/463: mkdir d1/dd/d3b/d9d 0 2024-02-16T19:29:03.670 INFO:tasks.workunit.client.1.smithi117.stdout:0/440: truncate d0/d9/d14/d1d/d60/d76/d7b/d8d/fa1 525207 0 2024-02-16T19:29:03.670 INFO:tasks.workunit.client.1.smithi117.stdout:9/323: creat dd/d47/d5e/f6e x:0 0 0 2024-02-16T19:29:03.671 INFO:tasks.workunit.client.1.smithi117.stdout:8/379: fdatasync d4/f52 0 2024-02-16T19:29:03.672 INFO:tasks.workunit.client.1.smithi117.stdout:2/346: symlink d0/d29/d56/d60/l67 0 2024-02-16T19:29:03.672 INFO:tasks.workunit.client.1.smithi117.stdout:0/441: write d0/d9/d14/d1d/d60/d76/d7b/d8d/fa1 [1421829,103974] 0 2024-02-16T19:29:03.672 INFO:tasks.workunit.client.1.smithi117.stdout:6/390: mknod d5/d4b/d44/d52/d64/c84 0 2024-02-16T19:29:03.673 INFO:tasks.workunit.client.1.smithi117.stdout:6/391: stat d5/d4b/d20/d69/l6b 0 2024-02-16T19:29:03.673 INFO:tasks.workunit.client.1.smithi117.stdout:3/464: symlink d1/d67/l9e 0 2024-02-16T19:29:03.674 INFO:tasks.workunit.client.1.smithi117.stdout:7/359: link d4/c4f d4/d1c/c8e 0 2024-02-16T19:29:03.676 INFO:tasks.workunit.client.1.smithi117.stdout:8/380: truncate d4/d12/d2a/f2c 1750904 0 2024-02-16T19:29:03.681 INFO:tasks.workunit.client.0.smithi022.stdout:9/446: link db/d14/d16/d27/d2e/f8e db/d14/d16/d5d/d65/f96 0 2024-02-16T19:29:03.681 INFO:tasks.workunit.client.1.smithi117.stdout:4/384: sync 2024-02-16T19:29:03.681 INFO:tasks.workunit.client.1.smithi117.stdout:5/477: sync 2024-02-16T19:29:03.681 INFO:tasks.workunit.client.0.smithi022.stdout:1/525: link d6/d8/d22/d47/d54/f79 d6/d8/d22/d3f/d99/f9b 0 2024-02-16T19:29:03.682 INFO:tasks.workunit.client.1.smithi117.stdout:3/465: mkdir d1/d30/d9f 0 2024-02-16T19:29:03.682 INFO:tasks.workunit.client.1.smithi117.stdout:8/381: read d4/d7/d37/d3a/d43/d44/f62 [3637357,42360] 0 2024-02-16T19:29:03.683 INFO:tasks.workunit.client.1.smithi117.stdout:7/360: symlink d4/d1c/d87/d89/d60/l8f 0 2024-02-16T19:29:03.684 INFO:tasks.workunit.client.1.smithi117.stdout:2/347: mkdir d0/d13/d4d/d64/d46/d68 0 2024-02-16T19:29:03.684 INFO:tasks.workunit.client.1.smithi117.stdout:4/385: chown d0/da/d21/d3d/d2d/c52 1 1 2024-02-16T19:29:03.685 INFO:tasks.workunit.client.1.smithi117.stdout:9/324: dwrite dd/d47/d5b/f69 [0,4194304] 0 2024-02-16T19:29:03.685 INFO:tasks.workunit.client.0.smithi022.stdout:4/384: dwrite d4/d6/d14/f2e [0,4194304] 0 2024-02-16T19:29:03.685 INFO:tasks.workunit.client.0.smithi022.stdout:2/324: symlink d0/d2/d18/d19/l68 0 2024-02-16T19:29:03.690 INFO:tasks.workunit.client.1.smithi117.stdout:8/382: write d4/d7/d37/d3a/d65/d6c/f75 [201446,113128] 0 2024-02-16T19:29:03.691 INFO:tasks.workunit.client.1.smithi117.stdout:9/325: dread - dd/d45/d62/f4e zero size 2024-02-16T19:29:03.694 INFO:tasks.workunit.client.1.smithi117.stdout:2/348: dread d0/d13/f15 [0,4194304] 0 2024-02-16T19:29:03.696 INFO:tasks.workunit.client.1.smithi117.stdout:1/334: rmdir d2/d4 39 2024-02-16T19:29:03.700 INFO:tasks.workunit.client.1.smithi117.stdout:1/335: read d2/d1d/d60/d20/f2a [74439,13084] 0 2024-02-16T19:29:03.700 INFO:tasks.workunit.client.0.smithi022.stdout:1/526: creat d6/d66/f9c x:0 0 0 2024-02-16T19:29:03.701 INFO:tasks.workunit.client.1.smithi117.stdout:1/336: chown d2/d1d/d1f/c2d 313575 1 2024-02-16T19:29:03.707 INFO:tasks.workunit.client.1.smithi117.stdout:7/361: rename f2 to d4/d13/d63/d71/f90 0 2024-02-16T19:29:03.707 INFO:tasks.workunit.client.0.smithi022.stdout:8/396: dread d3/d6/f2f [0,4194304] 0 2024-02-16T19:29:03.711 INFO:tasks.workunit.client.0.smithi022.stdout:4/385: mknod d4/d6/d14/d64/d4e/d54/d5a/d63/c77 0 2024-02-16T19:29:03.711 INFO:tasks.workunit.client.0.smithi022.stdout:2/325: dread d0/d2/d18/d19/d40/f44 [0,4194304] 0 2024-02-16T19:29:03.713 INFO:tasks.workunit.client.0.smithi022.stdout:2/326: write d0/d2/d18/d1f/f21 [1520431,112217] 0 2024-02-16T19:29:03.714 INFO:tasks.workunit.client.1.smithi117.stdout:7/362: dwrite d4/d1c/d87/d89/d5f/f7b [0,4194304] 0 2024-02-16T19:29:03.714 INFO:tasks.workunit.client.1.smithi117.stdout:3/466: dread d1/dd/f43 [0,4194304] 0 2024-02-16T19:29:03.714 INFO:tasks.workunit.client.0.smithi022.stdout:2/327: read d0/d2/d18/f34 [2943578,38866] 0 2024-02-16T19:29:03.715 INFO:tasks.workunit.client.1.smithi117.stdout:7/363: fsync d4/f3a 0 2024-02-16T19:29:03.716 INFO:tasks.workunit.client.0.smithi022.stdout:2/328: readlink d0/d2/d18/d38/l4b 0 2024-02-16T19:29:03.716 INFO:tasks.workunit.client.1.smithi117.stdout:7/364: fdatasync d4/d1c/d39/d46/d47/f73 0 2024-02-16T19:29:03.718 INFO:tasks.workunit.client.1.smithi117.stdout:7/365: read - d4/d1c/d87/d89/d60/d79/f7a zero size 2024-02-16T19:29:03.719 INFO:tasks.workunit.client.1.smithi117.stdout:7/366: readlink d4/d1c/d87/d89/d60/l7f 0 2024-02-16T19:29:03.721 INFO:tasks.workunit.client.0.smithi022.stdout:7/412: dread d9/da/d1f/d67/f6b [0,4194304] 0 2024-02-16T19:29:03.723 INFO:tasks.workunit.client.0.smithi022.stdout:1/527: unlink d6/d8/d22/d16/d17/d63/d6f/f7c 0 2024-02-16T19:29:03.724 INFO:tasks.workunit.client.0.smithi022.stdout:1/528: fdatasync d6/d8/d22/d16/d17/d1d/f33 0 2024-02-16T19:29:03.725 INFO:tasks.workunit.client.0.smithi022.stdout:0/521: creat d10/d11/d27/d3f/d94/faa x:0 0 0 2024-02-16T19:29:03.729 INFO:tasks.workunit.client.1.smithi117.stdout:6/392: write d5/ff [145888,49003] 0 2024-02-16T19:29:03.730 INFO:tasks.workunit.client.1.smithi117.stdout:1/337: mknod d2/d1d/c6f 0 2024-02-16T19:29:03.731 INFO:tasks.workunit.client.0.smithi022.stdout:4/386: rmdir d4/d6/d14/d64/d4e 39 2024-02-16T19:29:03.734 INFO:tasks.workunit.client.1.smithi117.stdout:5/478: dwrite d2/d3/d21/d2f/d30/d38/d51/f7e [0,4194304] 0 2024-02-16T19:29:03.734 INFO:tasks.workunit.client.0.smithi022.stdout:8/397: dwrite d3/d12/d2b/f65 [4194304,4194304] 0 2024-02-16T19:29:03.737 INFO:tasks.workunit.client.1.smithi117.stdout:4/386: rename d0/da/d21/d3d/d28/d64/f1c to d0/da/d21/d3d/d28/d66/f7f 0 2024-02-16T19:29:03.738 INFO:tasks.workunit.client.1.smithi117.stdout:1/338: dwrite d2/d1d/d60/d20/f32 [4194304,4194304] 0 2024-02-16T19:29:03.740 INFO:tasks.workunit.client.1.smithi117.stdout:1/339: chown d2/d1d/d1f/f5e 903 1 2024-02-16T19:29:03.742 INFO:tasks.workunit.client.1.smithi117.stdout:1/340: read d2/d1d/d60/d20/d49/f50 [1410523,35562] 0 2024-02-16T19:29:03.745 INFO:tasks.workunit.client.1.smithi117.stdout:1/341: dread d2/d1d/d60/d20/f2a [0,4194304] 0 2024-02-16T19:29:03.746 INFO:tasks.workunit.client.0.smithi022.stdout:2/329: unlink d0/d2/d18/d19/f47 0 2024-02-16T19:29:03.746 INFO:tasks.workunit.client.0.smithi022.stdout:9/447: dwrite f5 [0,4194304] 0 2024-02-16T19:29:03.748 INFO:tasks.workunit.client.0.smithi022.stdout:2/330: write d0/f5f [1003723,116048] 0 2024-02-16T19:29:03.749 INFO:tasks.workunit.client.0.smithi022.stdout:2/331: readlink d0/d2/d2e/d4a/l4f 0 2024-02-16T19:29:03.760 INFO:tasks.workunit.client.1.smithi117.stdout:8/383: link d4/d7/d37/d3a/f61 d4/d7/d6a/d6f/f7a 0 2024-02-16T19:29:03.760 INFO:tasks.workunit.client.0.smithi022.stdout:1/529: fdatasync d6/d8/d22/d3f/d99/f9b 0 2024-02-16T19:29:03.762 INFO:tasks.workunit.client.1.smithi117.stdout:7/367: creat d4/d1c/d39/f91 x:0 0 0 2024-02-16T19:29:03.767 INFO:tasks.workunit.client.0.smithi022.stdout:4/387: fdatasync d4/d6/dd/f1e 0 2024-02-16T19:29:03.772 INFO:tasks.workunit.client.0.smithi022.stdout:8/398: mknod d3/d12/d2b/d46/d68/c94 0 2024-02-16T19:29:03.773 INFO:tasks.workunit.client.1.smithi117.stdout:8/384: dread d4/d12/f1a [0,4194304] 0 2024-02-16T19:29:03.780 INFO:tasks.workunit.client.0.smithi022.stdout:7/413: link d9/da/d2c/d2f/f5c d9/da/d1f/d40/f87 0 2024-02-16T19:29:03.781 INFO:tasks.workunit.client.0.smithi022.stdout:1/530: fsync d6/d8/d22/d3f/f73 0 2024-02-16T19:29:03.782 INFO:tasks.workunit.client.0.smithi022.stdout:7/414: dread - d9/da/d1f/d40/f87 zero size 2024-02-16T19:29:03.783 INFO:tasks.workunit.client.0.smithi022.stdout:7/415: stat d9/da/d2c/f2d 0 2024-02-16T19:29:03.784 INFO:tasks.workunit.client.0.smithi022.stdout:0/522: dread d10/d11/d27/d3f/d58/d60/fa7 [0,4194304] 0 2024-02-16T19:29:03.785 INFO:tasks.workunit.client.0.smithi022.stdout:7/416: fsync d9/da/d2c/f69 0 2024-02-16T19:29:03.785 INFO:tasks.workunit.client.0.smithi022.stdout:9/448: dread db/d14/d19/d1a/f24 [0,4194304] 0 2024-02-16T19:29:03.786 INFO:tasks.workunit.client.0.smithi022.stdout:2/332: dread d0/d2/d18/f28 [0,4194304] 0 2024-02-16T19:29:03.787 INFO:tasks.workunit.client.0.smithi022.stdout:0/523: truncate d10/d11/d27/fa4 270015 0 2024-02-16T19:29:03.792 INFO:tasks.workunit.client.1.smithi117.stdout:8/385: dread d4/f79 [0,4194304] 0 2024-02-16T19:29:03.793 INFO:tasks.workunit.client.1.smithi117.stdout:8/386: chown d4/d1c/f1f 25982515 1 2024-02-16T19:29:03.795 INFO:tasks.workunit.client.1.smithi117.stdout:6/393: dwrite d5/d4b/d44/f56 [0,4194304] 0 2024-02-16T19:29:03.795 INFO:tasks.workunit.client.0.smithi022.stdout:1/531: symlink d6/d8/d22/d47/l9d 0 2024-02-16T19:29:03.800 INFO:tasks.workunit.client.0.smithi022.stdout:0/524: dwrite d10/d11/d27/d3f/f8a [0,4194304] 0 2024-02-16T19:29:03.802 INFO:tasks.workunit.client.0.smithi022.stdout:2/333: dwrite d0/d2/f3c [0,4194304] 0 2024-02-16T19:29:03.806 INFO:tasks.workunit.client.0.smithi022.stdout:1/532: dread d6/d8/d22/d16/d17/d1d/f33 [4194304,4194304] 0 2024-02-16T19:29:03.807 INFO:tasks.workunit.client.1.smithi117.stdout:2/349: rename d0/d3/c5f to d0/d13/d4d/c69 0 2024-02-16T19:29:03.807 INFO:tasks.workunit.client.0.smithi022.stdout:8/399: dwrite d3/d6/f34 [0,4194304] 0 2024-02-16T19:29:03.813 INFO:tasks.workunit.client.0.smithi022.stdout:9/449: rename db/d18/l61 to db/d14/d16/d5d/l97 0 2024-02-16T19:29:03.816 INFO:tasks.workunit.client.0.smithi022.stdout:4/388: link d4/d6/dd/df/f29 d4/d6/dd/d61/f78 0 2024-02-16T19:29:03.817 INFO:tasks.workunit.client.0.smithi022.stdout:4/389: readlink d4/d6/d14/d64/d66/l76 0 2024-02-16T19:29:03.817 INFO:tasks.workunit.client.1.smithi117.stdout:9/326: link dd/d47/d48/f66 dd/d47/d5b/d5f/f6f 0 2024-02-16T19:29:03.823 INFO:tasks.workunit.client.0.smithi022.stdout:7/417: dread d9/da/f29 [0,4194304] 0 2024-02-16T19:29:03.824 INFO:tasks.workunit.client.0.smithi022.stdout:7/418: readlink d9/da/l42 0 2024-02-16T19:29:03.825 INFO:tasks.workunit.client.0.smithi022.stdout:2/334: dread d0/d2/f3d [0,4194304] 0 2024-02-16T19:29:03.830 INFO:tasks.workunit.client.1.smithi117.stdout:0/442: sync 2024-02-16T19:29:03.837 INFO:tasks.workunit.client.0.smithi022.stdout:0/525: mkdir d10/d11/d27/d3f/d58/dab 0 2024-02-16T19:29:03.839 INFO:tasks.workunit.client.1.smithi117.stdout:7/368: fsync d4/d1c/d87/f50 0 2024-02-16T19:29:03.842 INFO:tasks.workunit.client.1.smithi117.stdout:7/369: dread d4/f11 [0,4194304] 0 2024-02-16T19:29:03.843 INFO:tasks.workunit.client.0.smithi022.stdout:1/533: mknod d6/d8/d22/d16/d17/d5d/c9e 0 2024-02-16T19:29:03.851 INFO:tasks.workunit.client.1.smithi117.stdout:8/387: truncate d4/d1c/f49 163541 0 2024-02-16T19:29:03.851 INFO:tasks.workunit.client.0.smithi022.stdout:0/526: dread d10/d22/f2d [0,4194304] 0 2024-02-16T19:29:03.854 INFO:tasks.workunit.client.0.smithi022.stdout:7/419: mkdir d9/da/d2c/d88 0 2024-02-16T19:29:03.855 INFO:tasks.workunit.client.1.smithi117.stdout:6/394: mkdir d5/d4b/d44/d52/d64/d85 0 2024-02-16T19:29:03.855 INFO:tasks.workunit.client.1.smithi117.stdout:1/342: sync 2024-02-16T19:29:03.855 INFO:tasks.workunit.client.1.smithi117.stdout:1/343: readlink d2/d19/l45 0 2024-02-16T19:29:03.856 INFO:tasks.workunit.client.1.smithi117.stdout:1/344: readlink d2/d1d/d60/d20/l34 0 2024-02-16T19:29:03.857 INFO:tasks.workunit.client.0.smithi022.stdout:2/335: symlink d0/d2/d18/d19/d40/d55/l69 0 2024-02-16T19:29:03.857 INFO:tasks.workunit.client.0.smithi022.stdout:2/336: chown d0/f2d 891 1 2024-02-16T19:29:03.859 INFO:tasks.workunit.client.1.smithi117.stdout:5/479: creat d2/d3/d21/d2f/d30/d95/f9e x:0 0 0 2024-02-16T19:29:03.862 INFO:tasks.workunit.client.1.smithi117.stdout:4/387: symlink d0/da/l80 0 2024-02-16T19:29:03.864 INFO:tasks.workunit.client.1.smithi117.stdout:5/480: dwrite d2/d1d/f5f [0,4194304] 0 2024-02-16T19:29:03.865 INFO:tasks.workunit.client.1.smithi117.stdout:2/350: mknod d0/d3/d40/c6a 0 2024-02-16T19:29:03.866 INFO:tasks.workunit.client.0.smithi022.stdout:1/534: fsync d6/d8/d22/d3f/d99/f9b 0 2024-02-16T19:29:03.867 INFO:tasks.workunit.client.1.smithi117.stdout:2/351: truncate d0/d35/f59 159037 0 2024-02-16T19:29:03.868 INFO:tasks.workunit.client.0.smithi022.stdout:1/535: write d6/d8/d22/d47/d54/f65 [4884771,35352] 0 2024-02-16T19:29:03.872 INFO:tasks.workunit.client.0.smithi022.stdout:7/420: dread d9/da/f13 [4194304,4194304] 0 2024-02-16T19:29:03.873 INFO:tasks.workunit.client.0.smithi022.stdout:7/421: stat d9/da/d1f/d40/f5e 0 2024-02-16T19:29:03.874 INFO:tasks.workunit.client.0.smithi022.stdout:4/390: creat d4/d38/f79 x:0 0 0 2024-02-16T19:29:03.875 INFO:tasks.workunit.client.0.smithi022.stdout:7/422: write d9/da/d1f/f22 [3432155,51018] 0 2024-02-16T19:29:03.875 INFO:tasks.workunit.client.1.smithi117.stdout:0/443: creat d0/d9/d22/d27/d4e/fa5 x:0 0 0 2024-02-16T19:29:03.876 INFO:tasks.workunit.client.0.smithi022.stdout:4/391: chown d4/d6/dd/df/d10/d55 1846822 1 2024-02-16T19:29:03.876 INFO:tasks.workunit.client.1.smithi117.stdout:0/444: write d0/d9/d14/d1d/d1e/d54/f72 [742496,67436] 0 2024-02-16T19:29:03.877 INFO:tasks.workunit.client.0.smithi022.stdout:0/527: mkdir d10/d7d/dac 0 2024-02-16T19:29:03.877 INFO:tasks.workunit.client.1.smithi117.stdout:5/481: dread d2/d3/d21/f32 [0,4194304] 0 2024-02-16T19:29:03.878 INFO:tasks.workunit.client.0.smithi022.stdout:0/528: chown d10/d22/d49 0 1 2024-02-16T19:29:03.880 INFO:tasks.workunit.client.0.smithi022.stdout:0/529: truncate d10/d22/d49/f76 952756 0 2024-02-16T19:29:03.883 INFO:tasks.workunit.client.1.smithi117.stdout:1/345: dread d2/d1d/f25 [0,4194304] 0 2024-02-16T19:29:03.985 INFO:tasks.workunit.client.0.smithi022.stdout:6/420: sync 2024-02-16T19:29:03.985 INFO:tasks.workunit.client.0.smithi022.stdout:3/392: sync 2024-02-16T19:29:03.985 INFO:tasks.workunit.client.0.smithi022.stdout:5/419: sync 2024-02-16T19:29:03.997 INFO:tasks.workunit.client.0.smithi022.stdout:1/536: creat d6/d8/d22/d67/f9f x:0 0 0 2024-02-16T19:29:03.999 INFO:tasks.workunit.client.1.smithi117.stdout:4/388: mknod d0/da/d21/d3d/d28/d2f/d37/c81 0 2024-02-16T19:29:04.001 INFO:tasks.workunit.client.0.smithi022.stdout:1/537: dread d6/d66/f71 [0,4194304] 0 2024-02-16T19:29:04.007 INFO:tasks.workunit.client.1.smithi117.stdout:9/327: link dd/d23/f30 dd/d23/d64/f70 0 2024-02-16T19:29:04.014 INFO:tasks.workunit.client.1.smithi117.stdout:3/467: getdents d1 0 2024-02-16T19:29:04.014 INFO:tasks.workunit.client.1.smithi117.stdout:3/468: chown d1/d1b/c6f 0 1 2024-02-16T19:29:04.023 INFO:tasks.workunit.client.1.smithi117.stdout:1/346: symlink d2/d1d/d60/d20/d69/l70 0 2024-02-16T19:29:04.024 INFO:tasks.workunit.client.0.smithi022.stdout:8/400: getdents d3/d12/d2b 0 2024-02-16T19:29:04.029 INFO:tasks.workunit.client.1.smithi117.stdout:7/370: rmdir d4/d1c/d87/d89/d60 39 2024-02-16T19:29:04.033 INFO:tasks.workunit.client.0.smithi022.stdout:9/450: getdents db 0 2024-02-16T19:29:04.035 INFO:tasks.workunit.client.1.smithi117.stdout:4/389: rename d0/da/d21/d3d/d28/d64/d48/l50 to d0/da/d21/d3d/d28/d64/d48/l82 0 2024-02-16T19:29:04.036 INFO:tasks.workunit.client.1.smithi117.stdout:4/390: chown d0/da/d21/d3d/d28/d2f/d37/c81 0 1 2024-02-16T19:29:04.037 INFO:tasks.workunit.client.1.smithi117.stdout:4/391: truncate d0/da/f5b 1024601 0 2024-02-16T19:29:04.040 INFO:tasks.workunit.client.0.smithi022.stdout:7/423: link f5 d9/da/d4e/f89 0 2024-02-16T19:29:04.050 INFO:tasks.workunit.client.0.smithi022.stdout:2/337: link d0/d32/c63 d0/d2/d18/d19/d40/c6a 0 2024-02-16T19:29:04.052 INFO:tasks.workunit.client.1.smithi117.stdout:1/347: write d2/d4/f16 [2023746,35217] 0 2024-02-16T19:29:04.055 INFO:tasks.workunit.client.1.smithi117.stdout:1/348: dread d2/d1d/d60/d20/f42 [0,4194304] 0 2024-02-16T19:29:04.060 INFO:tasks.workunit.client.0.smithi022.stdout:8/401: rename d3/d25/d33/f41 to d3/d12/d2b/d54/f95 0 2024-02-16T19:29:04.061 INFO:tasks.workunit.client.1.smithi117.stdout:1/349: dwrite d2/f5b [0,4194304] 0 2024-02-16T19:29:04.067 INFO:tasks.workunit.client.1.smithi117.stdout:2/352: link d0/d3/d22/c4a d0/d3/d40/c6b 0 2024-02-16T19:29:04.068 INFO:tasks.workunit.client.0.smithi022.stdout:9/451: mknod db/d18/c98 0 2024-02-16T19:29:04.075 INFO:tasks.workunit.client.1.smithi117.stdout:3/469: creat d1/d83/d96/fa0 x:0 0 0 2024-02-16T19:29:04.077 INFO:tasks.workunit.client.0.smithi022.stdout:7/424: chown d9/da/d1f/d67/l75 77717935 1 2024-02-16T19:29:04.078 INFO:tasks.workunit.client.0.smithi022.stdout:7/425: read - d9/da/d1f/d40/f5e zero size 2024-02-16T19:29:04.079 INFO:tasks.workunit.client.0.smithi022.stdout:7/426: dread - d9/da/d2c/d46/f6f zero size 2024-02-16T19:29:04.087 INFO:tasks.workunit.client.0.smithi022.stdout:2/338: dread - d0/d2/d18/d19/f30 zero size 2024-02-16T19:29:04.096 INFO:tasks.workunit.client.0.smithi022.stdout:8/402: mknod d3/d12/d17/d64/c96 0 2024-02-16T19:29:04.098 INFO:tasks.workunit.client.0.smithi022.stdout:9/452: fdatasync db/f12 0 2024-02-16T19:29:04.106 INFO:tasks.workunit.client.1.smithi117.stdout:3/470: creat d1/dd/d5c/fa1 x:0 0 0 2024-02-16T19:29:04.108 INFO:tasks.workunit.client.0.smithi022.stdout:0/530: getdents d10/d11 0 2024-02-16T19:29:04.111 INFO:tasks.workunit.client.1.smithi117.stdout:3/471: dwrite d1/d1b/d1e/d29/f8b [0,4194304] 0 2024-02-16T19:29:04.111 INFO:tasks.workunit.client.0.smithi022.stdout:2/339: mkdir d0/d2/d18/d38/d6b 0 2024-02-16T19:29:04.112 INFO:tasks.workunit.client.0.smithi022.stdout:2/340: write d0/d32/f62 [1118410,80911] 0 2024-02-16T19:29:04.115 INFO:tasks.workunit.client.1.smithi117.stdout:7/371: mknod d4/d1c/d39/d46/d55/c92 0 2024-02-16T19:29:04.118 INFO:tasks.workunit.client.0.smithi022.stdout:8/403: mkdir d3/d12/d2b/d54/d97 0 2024-02-16T19:29:04.119 INFO:tasks.workunit.client.0.smithi022.stdout:8/404: dread d3/d6/f27 [0,4194304] 0 2024-02-16T19:29:04.124 INFO:tasks.workunit.client.0.smithi022.stdout:9/453: creat db/d14/d16/d27/d55/f99 x:0 0 0 2024-02-16T19:29:04.125 INFO:tasks.workunit.client.1.smithi117.stdout:3/472: creat d1/d1b/d20/fa2 x:0 0 0 2024-02-16T19:29:04.126 INFO:tasks.workunit.client.0.smithi022.stdout:7/427: getdents d9/da/d4e/d7a 0 2024-02-16T19:29:04.128 INFO:tasks.workunit.client.0.smithi022.stdout:4/392: link d4/d6/d14/d64/d4e/l5d d4/l7a 0 2024-02-16T19:29:04.129 INFO:tasks.workunit.client.1.smithi117.stdout:2/353: creat d0/f6c x:0 0 0 2024-02-16T19:29:04.131 INFO:tasks.workunit.client.1.smithi117.stdout:2/354: write d0/d3/d40/f55 [1266643,73300] 0 2024-02-16T19:29:04.132 INFO:tasks.workunit.client.0.smithi022.stdout:9/454: dwrite db/d18/f73 [4194304,4194304] 0 2024-02-16T19:29:04.133 INFO:tasks.workunit.client.0.smithi022.stdout:0/531: read d10/d11/d27/d3f/d58/d60/d62/f4f [2170440,65703] 0 2024-02-16T19:29:04.135 INFO:tasks.workunit.client.1.smithi117.stdout:2/355: dwrite d0/fd [0,4194304] 0 2024-02-16T19:29:04.137 INFO:tasks.workunit.client.1.smithi117.stdout:3/473: mkdir d1/dd/d3b/da3 0 2024-02-16T19:29:04.139 INFO:tasks.workunit.client.1.smithi117.stdout:3/474: write d1/d83/d96/fa0 [719067,30027] 0 2024-02-16T19:29:04.139 INFO:tasks.workunit.client.0.smithi022.stdout:2/341: mkdir d0/d2/d18/d1f/d6c 0 2024-02-16T19:29:04.140 INFO:tasks.workunit.client.0.smithi022.stdout:2/342: chown d0/d2/f1c 52434 1 2024-02-16T19:29:04.145 INFO:tasks.workunit.client.1.smithi117.stdout:2/356: creat d0/d13/d4d/d64/f6d x:0 0 0 2024-02-16T19:29:04.145 INFO:tasks.workunit.client.1.smithi117.stdout:2/357: chown d0/d29/d2f 35492928 1 2024-02-16T19:29:04.146 INFO:tasks.workunit.client.1.smithi117.stdout:2/358: fdatasync d0/f61 0 2024-02-16T19:29:04.148 INFO:tasks.workunit.client.0.smithi022.stdout:8/405: rename d3/d6 to d3/d12/d6a/d98 0 2024-02-16T19:29:04.149 INFO:tasks.workunit.client.0.smithi022.stdout:8/406: fdatasync d3/d12/d6a/d98/f5e 0 2024-02-16T19:29:04.150 INFO:tasks.workunit.client.1.smithi117.stdout:3/475: creat d1/d30/d52/fa4 x:0 0 0 2024-02-16T19:29:04.154 INFO:tasks.workunit.client.1.smithi117.stdout:2/359: mkdir d0/d3b/d6e 0 2024-02-16T19:29:04.156 INFO:tasks.workunit.client.0.smithi022.stdout:7/428: unlink d9/da/d1f/d40/f7f 0 2024-02-16T19:29:04.157 INFO:tasks.workunit.client.0.smithi022.stdout:4/393: write d4/d6/d14/d64/f34 [4031995,91061] 0 2024-02-16T19:29:04.158 INFO:tasks.workunit.client.1.smithi117.stdout:2/360: creat d0/d13/d4d/d53/f6f x:0 0 0 2024-02-16T19:29:04.159 INFO:tasks.workunit.client.1.smithi117.stdout:8/388: dwrite d4/d7/f3d [0,4194304] 0 2024-02-16T19:29:04.159 INFO:tasks.workunit.client.0.smithi022.stdout:4/394: read d4/d6/dd/df/d10/f35 [943109,31141] 0 2024-02-16T19:29:04.160 INFO:tasks.workunit.client.0.smithi022.stdout:0/532: mkdir d10/d22/dad 0 2024-02-16T19:29:04.163 INFO:tasks.workunit.client.1.smithi117.stdout:8/389: dread d4/d7/d6a/d6f/f7a [0,4194304] 0 2024-02-16T19:29:04.166 INFO:tasks.workunit.client.0.smithi022.stdout:8/407: creat d3/d12/d2b/d46/f99 x:0 0 0 2024-02-16T19:29:04.168 INFO:tasks.workunit.client.0.smithi022.stdout:1/538: sync 2024-02-16T19:29:04.173 INFO:tasks.workunit.client.1.smithi117.stdout:8/390: creat d4/d7/d6a/f7b x:0 0 0 2024-02-16T19:29:04.175 INFO:tasks.workunit.client.1.smithi117.stdout:2/361: dread d0/d29/f2e [0,4194304] 0 2024-02-16T19:29:04.177 INFO:tasks.workunit.client.0.smithi022.stdout:4/395: fsync d4/d6/dd/df/d10/f19 0 2024-02-16T19:29:04.178 INFO:tasks.workunit.client.1.smithi117.stdout:8/391: mkdir d4/d7/d37/d3a/d65/d7c 0 2024-02-16T19:29:04.178 INFO:tasks.workunit.client.1.smithi117.stdout:0/445: write d0/fe [2998636,33953] 0 2024-02-16T19:29:04.179 INFO:tasks.workunit.client.0.smithi022.stdout:0/533: dread d10/d15/d23/f47 [0,4194304] 0 2024-02-16T19:29:04.181 INFO:tasks.workunit.client.1.smithi117.stdout:0/446: truncate d0/d9/d22/d27/d31/d6d/fa2 116988 0 2024-02-16T19:29:04.181 INFO:tasks.workunit.client.1.smithi117.stdout:0/447: fsync d0/d9/d22/d27/f52 0 2024-02-16T19:29:04.183 INFO:tasks.workunit.client.1.smithi117.stdout:2/362: symlink d0/d3/d22/l70 0 2024-02-16T19:29:04.186 INFO:tasks.workunit.client.1.smithi117.stdout:1/350: mknod d2/d1d/c71 0 2024-02-16T19:29:04.186 INFO:tasks.workunit.client.1.smithi117.stdout:1/351: chown d2/d19/f54 136598855 1 2024-02-16T19:29:04.188 INFO:tasks.workunit.client.0.smithi022.stdout:1/539: truncate d6/f6b 3479558 0 2024-02-16T19:29:04.188 INFO:tasks.workunit.client.0.smithi022.stdout:8/408: dread d3/d25/f43 [0,4194304] 0 2024-02-16T19:29:04.189 INFO:tasks.workunit.client.1.smithi117.stdout:0/448: truncate d0/d9/d14/d23/f56 79368 0 2024-02-16T19:29:04.189 INFO:tasks.workunit.client.1.smithi117.stdout:1/352: dread d2/d4/f11 [0,4194304] 0 2024-02-16T19:29:04.189 INFO:tasks.workunit.client.0.smithi022.stdout:7/429: mknod d9/da/d2c/d88/c8a 0 2024-02-16T19:29:04.190 INFO:tasks.workunit.client.1.smithi117.stdout:0/449: truncate d0/d9/d2c/f91 506834 0 2024-02-16T19:29:04.191 INFO:tasks.workunit.client.1.smithi117.stdout:8/392: fdatasync d4/d1c/f49 0 2024-02-16T19:29:04.194 INFO:tasks.workunit.client.1.smithi117.stdout:1/353: creat d2/d1d/d60/d20/d69/f72 x:0 0 0 2024-02-16T19:29:04.194 INFO:tasks.workunit.client.0.smithi022.stdout:6/421: dwrite d1/d8/f3b [0,4194304] 0 2024-02-16T19:29:04.196 INFO:tasks.workunit.client.1.smithi117.stdout:0/450: fsync d0/d9/d2c/f44 0 2024-02-16T19:29:04.199 INFO:tasks.workunit.client.1.smithi117.stdout:0/451: read d0/fd [4170473,48727] 0 2024-02-16T19:29:04.205 INFO:tasks.workunit.client.0.smithi022.stdout:1/540: unlink d6/d8/d22/d16/f41 0 2024-02-16T19:29:04.208 INFO:tasks.workunit.client.1.smithi117.stdout:4/392: unlink d0/f77 0 2024-02-16T19:29:04.210 INFO:tasks.workunit.client.1.smithi117.stdout:5/482: rename d2/d9/f40 to d2/d1d/f9f 0 2024-02-16T19:29:04.213 INFO:tasks.workunit.client.0.smithi022.stdout:7/430: unlink d9/f14 0 2024-02-16T19:29:04.213 INFO:tasks.workunit.client.1.smithi117.stdout:0/452: dread - d0/d9/d14/d1d/d1e/f2d zero size 2024-02-16T19:29:04.214 INFO:tasks.workunit.client.1.smithi117.stdout:0/453: chown d0/d9/d22/d27/c85 2014166 1 2024-02-16T19:29:04.215 INFO:tasks.workunit.client.0.smithi022.stdout:6/422: rmdir d1/d8/da 39 2024-02-16T19:29:04.215 INFO:tasks.workunit.client.1.smithi117.stdout:0/454: dread - d0/d9/d22/d27/d83/f95 zero size 2024-02-16T19:29:04.216 INFO:tasks.workunit.client.1.smithi117.stdout:8/393: creat d4/d12/d2a/d64/f7d x:0 0 0 2024-02-16T19:29:04.217 INFO:tasks.workunit.client.1.smithi117.stdout:6/395: creat d5/f86 x:0 0 0 2024-02-16T19:29:04.218 INFO:tasks.workunit.client.0.smithi022.stdout:5/420: truncate dd/d1f/d48/d21/d2f/d52/f53 519503 0 2024-02-16T19:29:04.221 INFO:tasks.workunit.client.0.smithi022.stdout:3/393: dwrite d7/d12/d18/dd/d24/f32 [0,4194304] 0 2024-02-16T19:29:04.229 INFO:tasks.workunit.client.0.smithi022.stdout:3/394: dwrite d7/d12/d18/dd/d24/d53/f5e [4194304,4194304] 0 2024-02-16T19:29:04.229 INFO:tasks.workunit.client.0.smithi022.stdout:7/431: creat d9/da/d2c/d6d/f8b x:0 0 0 2024-02-16T19:29:04.230 INFO:tasks.workunit.client.0.smithi022.stdout:3/395: readlink d7/d12/d18/dd/d31/l6a 0 2024-02-16T19:29:04.231 INFO:tasks.workunit.client.1.smithi117.stdout:3/476: rename d1/d1b/f2d to d1/d98/fa5 0 2024-02-16T19:29:04.232 INFO:tasks.workunit.client.1.smithi117.stdout:0/455: mknod d0/ca6 0 2024-02-16T19:29:04.234 INFO:tasks.workunit.client.1.smithi117.stdout:8/394: symlink d4/d12/d31/l7e 0 2024-02-16T19:29:04.236 INFO:tasks.workunit.client.1.smithi117.stdout:8/395: stat d4/d7/d37/d3a/d65/d6c/f75 0 2024-02-16T19:29:04.240 INFO:tasks.workunit.client.1.smithi117.stdout:3/477: fsync d1/d1b/d1e/d5e/f66 0 2024-02-16T19:29:04.241 INFO:tasks.workunit.client.1.smithi117.stdout:3/478: chown d1/f8 948529032 1 2024-02-16T19:29:04.242 INFO:tasks.workunit.client.1.smithi117.stdout:8/396: dwrite d4/d12/d2a/f2b [8388608,4194304] 0 2024-02-16T19:29:04.242 INFO:tasks.workunit.client.1.smithi117.stdout:3/479: chown d1/d1b/d1e/lf 6444 1 2024-02-16T19:29:04.244 INFO:tasks.workunit.client.1.smithi117.stdout:0/456: symlink d0/d9/d14/d23/la7 0 2024-02-16T19:29:04.245 INFO:tasks.workunit.client.1.smithi117.stdout:0/457: dread - d0/d9/d14/d1d/d1e/f2d zero size 2024-02-16T19:29:04.246 INFO:tasks.workunit.client.0.smithi022.stdout:1/541: mknod d6/d8/d22/d16/d17/ca0 0 2024-02-16T19:29:04.248 INFO:tasks.workunit.client.1.smithi117.stdout:1/354: getdents d2/d1d/d60/d20 0 2024-02-16T19:29:04.258 INFO:tasks.workunit.client.1.smithi117.stdout:8/397: fdatasync d4/f3c 0 2024-02-16T19:29:04.259 INFO:tasks.workunit.client.0.smithi022.stdout:1/542: creat d6/d8/d22/d3a/d3e/fa1 x:0 0 0 2024-02-16T19:29:04.261 INFO:tasks.workunit.client.1.smithi117.stdout:3/480: mkdir d1/d1b/d1e/da6 0 2024-02-16T19:29:04.264 INFO:tasks.workunit.client.1.smithi117.stdout:0/458: dread d0/d9/f12 [4194304,4194304] 0 2024-02-16T19:29:04.266 INFO:tasks.workunit.client.1.smithi117.stdout:0/459: truncate d0/d9/d22/d27/d31/d6d/fa2 859956 0 2024-02-16T19:29:04.267 INFO:tasks.workunit.client.0.smithi022.stdout:1/543: fsync d6/d8/d22/d16/f4b 0 2024-02-16T19:29:04.268 INFO:tasks.workunit.client.1.smithi117.stdout:5/483: getdents d2/d3/d21/d2f/d30/d5a 0 2024-02-16T19:29:04.270 INFO:tasks.workunit.client.1.smithi117.stdout:5/484: dread d2/d1d/f5f [0,4194304] 0 2024-02-16T19:29:04.272 INFO:tasks.workunit.client.1.smithi117.stdout:9/328: symlink dd/d23/l71 0 2024-02-16T19:29:04.273 INFO:tasks.workunit.client.0.smithi022.stdout:6/423: link d1/d8/da/f14 d1/d8/da/d17/d48/d51/f9b 0 2024-02-16T19:29:04.274 INFO:tasks.workunit.client.1.smithi117.stdout:9/329: dread dd/d45/f3f [0,4194304] 0 2024-02-16T19:29:04.274 INFO:tasks.workunit.client.1.smithi117.stdout:8/398: creat d4/d1c/f7f x:0 0 0 2024-02-16T19:29:04.277 INFO:tasks.workunit.client.1.smithi117.stdout:3/481: symlink d1/d83/la7 0 2024-02-16T19:29:04.277 INFO:tasks.workunit.client.1.smithi117.stdout:3/482: dread - d1/d1b/f21 zero size 2024-02-16T19:29:04.279 INFO:tasks.workunit.client.1.smithi117.stdout:1/355: symlink d2/d4/l73 0 2024-02-16T19:29:04.280 INFO:tasks.workunit.client.0.smithi022.stdout:6/424: dwrite d1/d8/da/d17/f96 [0,4194304] 0 2024-02-16T19:29:04.281 INFO:tasks.workunit.client.0.smithi022.stdout:6/425: fdatasync d1/d8/da/d17/d85/f50 0 2024-02-16T19:29:04.283 INFO:tasks.workunit.client.1.smithi117.stdout:2/363: sync 2024-02-16T19:29:04.283 INFO:tasks.workunit.client.1.smithi117.stdout:2/364: fdatasync d0/d3/f31 0 2024-02-16T19:29:04.286 INFO:tasks.workunit.client.1.smithi117.stdout:5/485: rename d2/d9/d3c/d24/d50/d52/d74/d8c/f73 to d2/d1d/fa0 0 2024-02-16T19:29:04.287 INFO:tasks.workunit.client.0.smithi022.stdout:0/534: rename d10/d11/d50/d65/ca0 to d10/d11/d27/d3f/cae 0 2024-02-16T19:29:04.289 INFO:tasks.workunit.client.0.smithi022.stdout:0/535: fdatasync d10/d11/d27/d77/f8e 0 2024-02-16T19:29:04.291 INFO:tasks.workunit.client.1.smithi117.stdout:3/483: mknod d1/d83/d96/ca8 0 2024-02-16T19:29:04.292 INFO:tasks.workunit.client.0.smithi022.stdout:5/421: rename dd/d1f/d2e/c7c to dd/c8a 0 2024-02-16T19:29:04.294 INFO:tasks.workunit.client.0.smithi022.stdout:6/426: write d1/d8/d3c/d4d/f7e [835438,51977] 0 2024-02-16T19:29:04.295 INFO:tasks.workunit.client.0.smithi022.stdout:6/427: fdatasync d1/d8/d3c/f44 0 2024-02-16T19:29:04.297 INFO:tasks.workunit.client.0.smithi022.stdout:0/536: creat d10/d11/d27/d3f/d94/faf x:0 0 0 2024-02-16T19:29:04.298 INFO:tasks.workunit.client.1.smithi117.stdout:7/372: mknod d4/d1c/d39/c93 0 2024-02-16T19:29:04.299 INFO:tasks.workunit.client.1.smithi117.stdout:7/373: truncate d4/d1c/d39/d46/f48 569072 0 2024-02-16T19:29:04.300 INFO:tasks.workunit.client.0.smithi022.stdout:7/432: rename d9/c43 to d9/da/d2c/d46/d73/c8c 0 2024-02-16T19:29:04.303 INFO:tasks.workunit.client.0.smithi022.stdout:5/422: mkdir dd/d1f/d48/d21/d2f/d46/d5e/d84/d8b 0 2024-02-16T19:29:04.311 INFO:tasks.workunit.client.0.smithi022.stdout:0/537: symlink d10/d11/d27/d3f/d94/lb0 0 2024-02-16T19:29:04.314 INFO:tasks.workunit.client.1.smithi117.stdout:8/399: fsync d4/d12/d2a/f2c 0 2024-02-16T19:29:04.318 INFO:tasks.workunit.client.0.smithi022.stdout:1/544: rename d6/d8/d22/d67/d7e/f7f to d6/d8/d22/d47/d51/fa2 0 2024-02-16T19:29:04.319 INFO:tasks.workunit.client.1.smithi117.stdout:1/356: symlink d2/l74 0 2024-02-16T19:29:04.319 INFO:tasks.workunit.client.1.smithi117.stdout:5/486: dread d2/d3/f12 [0,4194304] 0 2024-02-16T19:29:04.320 INFO:tasks.workunit.client.1.smithi117.stdout:5/487: read - d2/d4c/d4f/d67/f7f zero size 2024-02-16T19:29:04.320 INFO:tasks.workunit.client.0.smithi022.stdout:7/433: creat d9/da/d2c/f8d x:0 0 0 2024-02-16T19:29:04.324 INFO:tasks.workunit.client.1.smithi117.stdout:2/365: rmdir d0/d13 39 2024-02-16T19:29:04.325 INFO:tasks.workunit.client.1.smithi117.stdout:5/488: dwrite d2/d4c/d4f/d67/f7f [0,4194304] 0 2024-02-16T19:29:04.327 INFO:tasks.workunit.client.1.smithi117.stdout:9/330: rename dd/l6b to dd/d47/d48/l72 0 2024-02-16T19:29:04.327 INFO:tasks.workunit.client.0.smithi022.stdout:2/343: write d0/f20 [1783229,112599] 0 2024-02-16T19:29:04.331 INFO:tasks.workunit.client.1.smithi117.stdout:8/400: unlink d4/d12/d31/f36 0 2024-02-16T19:29:04.335 INFO:tasks.workunit.client.0.smithi022.stdout:7/434: dread d9/da/d2c/d46/f6a [0,4194304] 0 2024-02-16T19:29:04.335 INFO:tasks.workunit.client.1.smithi117.stdout:3/484: mknod d1/dd/d3e/d7d/ca9 0 2024-02-16T19:29:04.336 INFO:tasks.workunit.client.0.smithi022.stdout:7/435: chown d9/da/d2c/d46 26674 1 2024-02-16T19:29:04.336 INFO:tasks.workunit.client.0.smithi022.stdout:7/436: dread - d9/f6e zero size 2024-02-16T19:29:04.343 INFO:tasks.workunit.client.0.smithi022.stdout:9/455: truncate db/d14/f5e 1677885 0 2024-02-16T19:29:04.344 INFO:tasks.workunit.client.1.smithi117.stdout:2/366: unlink d0/f61 0 2024-02-16T19:29:04.345 INFO:tasks.workunit.client.0.smithi022.stdout:2/344: creat d0/d2/d18/d19/d40/f6d x:0 0 0 2024-02-16T19:29:04.347 INFO:tasks.workunit.client.1.smithi117.stdout:5/489: creat d2/d3/d21/d2f/d30/d5a/fa1 x:0 0 0 2024-02-16T19:29:04.348 INFO:tasks.workunit.client.1.smithi117.stdout:5/490: fsync d2/d3/d21/d2a/f84 0 2024-02-16T19:29:04.349 INFO:tasks.workunit.client.0.smithi022.stdout:1/545: creat d6/d8/d22/d16/d17/d63/d6f/d8b/fa3 x:0 0 0 2024-02-16T19:29:04.350 INFO:tasks.workunit.client.0.smithi022.stdout:1/546: chown d6/d8/d22/d16/d17/d26/l64 294766000 1 2024-02-16T19:29:04.352 INFO:tasks.workunit.client.0.smithi022.stdout:1/547: write d6/d8/d22/d47/d51/f85 [1006773,70147] 0 2024-02-16T19:29:04.355 INFO:tasks.workunit.client.1.smithi117.stdout:1/357: rename d2/d1d/d60/d20/d49/f67 to d2/d51/f75 0 2024-02-16T19:29:04.357 INFO:tasks.workunit.client.0.smithi022.stdout:7/437: creat d9/da/d2c/d46/d73/f8e x:0 0 0 2024-02-16T19:29:04.358 INFO:tasks.workunit.client.0.smithi022.stdout:7/438: write d9/da/d2c/d46/d73/f8e [271657,66163] 0 2024-02-16T19:29:04.359 INFO:tasks.workunit.client.1.smithi117.stdout:8/401: mknod d4/d12/d2a/c80 0 2024-02-16T19:29:04.360 INFO:tasks.workunit.client.1.smithi117.stdout:8/402: stat d4/d7/d37/d3a/d43/d44/l46 0 2024-02-16T19:29:04.360 INFO:tasks.workunit.client.0.smithi022.stdout:4/396: write d4/d6/dd/f2c [2236643,95308] 0 2024-02-16T19:29:04.361 INFO:tasks.workunit.client.0.smithi022.stdout:2/345: dread d0/f2d [0,4194304] 0 2024-02-16T19:29:04.366 INFO:tasks.workunit.client.1.smithi117.stdout:4/393: sync 2024-02-16T19:29:04.366 INFO:tasks.workunit.client.1.smithi117.stdout:3/485: dwrite d1/d1b/d1e/d37/f9c [0,4194304] 0 2024-02-16T19:29:04.369 INFO:tasks.workunit.client.1.smithi117.stdout:5/491: truncate d2/d3/d21/d2f/d30/d4e/f34 1039429 0 2024-02-16T19:29:04.373 INFO:tasks.workunit.client.1.smithi117.stdout:3/486: dwrite d1/d1b/d1e/d37/f79 [0,4194304] 0 2024-02-16T19:29:04.373 INFO:tasks.workunit.client.1.smithi117.stdout:9/331: mkdir dd/d45/d73 0 2024-02-16T19:29:04.375 INFO:tasks.workunit.client.1.smithi117.stdout:8/403: dread d4/d12/f42 [0,4194304] 0 2024-02-16T19:29:04.385 INFO:tasks.workunit.client.1.smithi117.stdout:2/367: creat d0/d3/d34/f71 x:0 0 0 2024-02-16T19:29:04.389 INFO:tasks.workunit.client.1.smithi117.stdout:4/394: dread d0/da/d21/d3d/d28/f39 [4194304,4194304] 0 2024-02-16T19:29:04.390 INFO:tasks.workunit.client.1.smithi117.stdout:4/395: dread - d0/da/d21/d3d/f5e zero size 2024-02-16T19:29:04.393 INFO:tasks.workunit.client.0.smithi022.stdout:8/409: dwrite d3/d12/d2b/d54/f95 [0,4194304] 0 2024-02-16T19:29:04.398 INFO:tasks.workunit.client.0.smithi022.stdout:3/396: write d7/d12/d18/f1b [162379,108925] 0 2024-02-16T19:29:04.401 INFO:tasks.workunit.client.1.smithi117.stdout:1/358: mkdir d2/d19/d59/d76 0 2024-02-16T19:29:04.403 INFO:tasks.workunit.client.1.smithi117.stdout:1/359: dread - d2/d1d/f4f zero size 2024-02-16T19:29:04.403 INFO:tasks.workunit.client.0.smithi022.stdout:8/410: dwrite d3/d12/d6a/d98/f8c [0,4194304] 0 2024-02-16T19:29:04.409 INFO:tasks.workunit.client.1.smithi117.stdout:5/492: dread d2/d9/d3c/d24/f26 [0,4194304] 0 2024-02-16T19:29:04.410 INFO:tasks.workunit.client.0.smithi022.stdout:8/411: dwrite d3/f8d [0,4194304] 0 2024-02-16T19:29:04.413 INFO:tasks.workunit.client.1.smithi117.stdout:4/396: unlink d0/da/d21/d3d/d28/d64/l61 0 2024-02-16T19:29:04.414 INFO:tasks.workunit.client.1.smithi117.stdout:4/397: fdatasync d0/da/d21/d3d/d28/d64/f6c 0 2024-02-16T19:29:04.415 INFO:tasks.workunit.client.1.smithi117.stdout:3/487: mknod d1/dd/caa 0 2024-02-16T19:29:04.420 INFO:tasks.workunit.client.1.smithi117.stdout:1/360: unlink d2/d1d/f35 0 2024-02-16T19:29:04.427 INFO:tasks.workunit.client.0.smithi022.stdout:5/423: rmdir dd/d1f/d48/d21/d2f/d46/d73 0 2024-02-16T19:29:04.428 INFO:tasks.workunit.client.1.smithi117.stdout:4/398: rmdir d0/da 39 2024-02-16T19:29:04.431 INFO:tasks.workunit.client.1.smithi117.stdout:3/488: fdatasync d1/dd/d5c/f6b 0 2024-02-16T19:29:04.432 INFO:tasks.workunit.client.1.smithi117.stdout:3/489: dread - d1/d1b/f21 zero size 2024-02-16T19:29:04.433 INFO:tasks.workunit.client.1.smithi117.stdout:0/460: sync 2024-02-16T19:29:04.433 INFO:tasks.workunit.client.1.smithi117.stdout:6/396: sync 2024-02-16T19:29:04.436 INFO:tasks.workunit.client.1.smithi117.stdout:1/361: mkdir d2/d1d/d77 0 2024-02-16T19:29:04.438 INFO:tasks.workunit.client.1.smithi117.stdout:5/493: mknod d2/d4c/d54/d9b/ca2 0 2024-02-16T19:29:04.440 INFO:tasks.workunit.client.1.smithi117.stdout:5/494: truncate d2/d9/d3c/d24/d50/d52/f91 755047 0 2024-02-16T19:29:04.442 INFO:tasks.workunit.client.0.smithi022.stdout:6/428: dwrite d1/d8/d3c/d4d/f4b [0,4194304] 0 2024-02-16T19:29:04.445 INFO:tasks.workunit.client.0.smithi022.stdout:2/346: creat d0/d32/d3b/f6e x:0 0 0 2024-02-16T19:29:04.445 INFO:tasks.workunit.client.1.smithi117.stdout:3/490: rmdir d1/d1b/d20 39 2024-02-16T19:29:04.447 INFO:tasks.workunit.client.1.smithi117.stdout:7/374: write d4/d1c/f37 [1328146,32888] 0 2024-02-16T19:29:04.448 INFO:tasks.workunit.client.1.smithi117.stdout:6/397: dread d5/db/d7d/d32/d37/f41 [0,4194304] 0 2024-02-16T19:29:04.448 INFO:tasks.workunit.client.1.smithi117.stdout:3/491: chown d1/d1b/d1e/d37/f9c 170 1 2024-02-16T19:29:04.450 INFO:tasks.workunit.client.0.smithi022.stdout:6/429: dwrite d1/d8/d3c/d77/f80 [0,4194304] 0 2024-02-16T19:29:04.450 INFO:tasks.workunit.client.1.smithi117.stdout:7/375: read d4/d13/d63/d71/f90 [4094129,85970] 0 2024-02-16T19:29:04.456 INFO:tasks.workunit.client.1.smithi117.stdout:6/398: dwrite d5/db/f4a [0,4194304] 0 2024-02-16T19:29:04.458 INFO:tasks.workunit.client.1.smithi117.stdout:8/404: sync 2024-02-16T19:29:04.463 INFO:tasks.workunit.client.1.smithi117.stdout:5/495: dread d2/d4c/d54/f6d [0,4194304] 0 2024-02-16T19:29:04.466 INFO:tasks.workunit.client.1.smithi117.stdout:4/399: rename d0/da/d21/d3d/d28/d64/f6c to d0/da/d21/d3d/d2d/d6d/f83 0 2024-02-16T19:29:04.468 INFO:tasks.workunit.client.0.smithi022.stdout:3/397: dread d7/f1c [0,4194304] 0 2024-02-16T19:29:04.471 INFO:tasks.workunit.client.1.smithi117.stdout:3/492: mknod d1/dd/d32/cab 0 2024-02-16T19:29:04.481 INFO:tasks.workunit.client.0.smithi022.stdout:7/439: rename d9/da/d2c/d46/d73/c8c to d9/da/d1f/d40/c8f 0 2024-02-16T19:29:04.485 INFO:tasks.workunit.client.1.smithi117.stdout:6/399: creat d5/d1d/f87 x:0 0 0 2024-02-16T19:29:04.486 INFO:tasks.workunit.client.1.smithi117.stdout:8/405: write d4/f53 [1403936,26383] 0 2024-02-16T19:29:04.491 INFO:tasks.workunit.client.0.smithi022.stdout:6/430: unlink d1/d8/d3c/d77/f92 0 2024-02-16T19:29:04.492 INFO:tasks.workunit.client.1.smithi117.stdout:5/496: rmdir d2/d3/d21 39 2024-02-16T19:29:04.492 INFO:tasks.workunit.client.1.smithi117.stdout:5/497: chown d2/d9/d3c/f8d 114036 1 2024-02-16T19:29:04.494 INFO:tasks.workunit.client.0.smithi022.stdout:8/412: creat d3/d12/d17/d64/d8f/f9a x:0 0 0 2024-02-16T19:29:04.499 INFO:tasks.workunit.client.1.smithi117.stdout:4/400: dwrite d0/f25 [4194304,4194304] 0 2024-02-16T19:29:04.499 INFO:tasks.workunit.client.1.smithi117.stdout:3/493: dread d1/dd/d3b/f5a [0,4194304] 0 2024-02-16T19:29:04.501 INFO:tasks.workunit.client.1.smithi117.stdout:1/362: creat d2/f78 x:0 0 0 2024-02-16T19:29:04.501 INFO:tasks.workunit.client.1.smithi117.stdout:9/332: write dd/d47/d48/f66 [489737,106876] 0 2024-02-16T19:29:04.501 INFO:tasks.workunit.client.1.smithi117.stdout:1/363: chown d2/d4/f40 4 1 2024-02-16T19:29:04.502 INFO:tasks.workunit.client.1.smithi117.stdout:9/333: chown dd/d45/d31/l46 85 1 2024-02-16T19:29:04.504 INFO:tasks.workunit.client.0.smithi022.stdout:2/347: rename d0/d48 to d0/d32/d3b/d6f 0 2024-02-16T19:29:04.509 INFO:tasks.workunit.client.1.smithi117.stdout:6/400: creat d5/d1d/d49/f88 x:0 0 0 2024-02-16T19:29:04.511 INFO:tasks.workunit.client.0.smithi022.stdout:6/431: stat d1/d8/da/l21 0 2024-02-16T19:29:04.512 INFO:tasks.workunit.client.1.smithi117.stdout:5/498: dread d2/d1d/f55 [0,4194304] 0 2024-02-16T19:29:04.514 INFO:tasks.workunit.client.0.smithi022.stdout:3/398: mknod d7/d12/d18/c9b 0 2024-02-16T19:29:04.516 INFO:tasks.workunit.client.1.smithi117.stdout:0/461: getdents d0/d9/d22/d27/d31/d94 0 2024-02-16T19:29:04.517 INFO:tasks.workunit.client.1.smithi117.stdout:0/462: readlink d0/l8 0 2024-02-16T19:29:04.518 INFO:tasks.workunit.client.1.smithi117.stdout:4/401: readlink d0/l42 0 2024-02-16T19:29:04.518 INFO:tasks.workunit.client.1.smithi117.stdout:5/499: dwrite d2/d5d/f5e [0,4194304] 0 2024-02-16T19:29:04.519 INFO:tasks.workunit.client.0.smithi022.stdout:7/440: mknod d9/da/c90 0 2024-02-16T19:29:04.520 INFO:tasks.workunit.client.0.smithi022.stdout:2/348: write d0/f2d [3285029,95111] 0 2024-02-16T19:29:04.522 INFO:tasks.workunit.client.0.smithi022.stdout:2/349: truncate d0/d2/d18/f65 263515 0 2024-02-16T19:29:04.524 INFO:tasks.workunit.client.1.smithi117.stdout:0/463: write d0/d30/f8c [1563,104048] 0 2024-02-16T19:29:04.526 INFO:tasks.workunit.client.1.smithi117.stdout:0/464: dread d0/d9/d22/d27/d31/d6d/fa2 [0,4194304] 0 2024-02-16T19:29:04.527 INFO:tasks.workunit.client.1.smithi117.stdout:9/334: mkdir dd/d74 0 2024-02-16T19:29:04.527 INFO:tasks.workunit.client.1.smithi117.stdout:9/335: stat dd/f1e 0 2024-02-16T19:29:04.527 INFO:tasks.workunit.client.1.smithi117.stdout:2/368: truncate d0/d3/fc 1060707 0 2024-02-16T19:29:04.528 INFO:tasks.workunit.client.0.smithi022.stdout:2/350: dwrite d0/f17 [0,4194304] 0 2024-02-16T19:29:04.537 INFO:tasks.workunit.client.1.smithi117.stdout:8/406: mkdir d4/d7/d37/d3a/d43/d44/d5f/d81 0 2024-02-16T19:29:04.538 INFO:tasks.workunit.client.0.smithi022.stdout:9/456: dwrite db/d14/d16/d5d/d65/f96 [0,4194304] 0 2024-02-16T19:29:04.544 INFO:tasks.workunit.client.0.smithi022.stdout:6/432: mkdir d1/d8/da/d17/d85/d9c 0 2024-02-16T19:29:04.546 INFO:tasks.workunit.client.0.smithi022.stdout:6/433: write d1/d8/da/d17/d48/f7c [425800,80535] 0 2024-02-16T19:29:04.547 INFO:tasks.workunit.client.1.smithi117.stdout:5/500: truncate d2/d3/d21/f33 5148786 0 2024-02-16T19:29:04.549 INFO:tasks.workunit.client.0.smithi022.stdout:2/351: rename d0/d32/d3b/d6f/l66 to d0/d32/d3b/l70 0 2024-02-16T19:29:04.551 INFO:tasks.workunit.client.0.smithi022.stdout:8/413: link d3/d12/d2b/d54/l60 d3/d12/d2b/d2e/l9b 0 2024-02-16T19:29:04.551 INFO:tasks.workunit.client.0.smithi022.stdout:8/414: readlink d3/l13 0 2024-02-16T19:29:04.554 INFO:tasks.workunit.client.0.smithi022.stdout:5/424: write dd/d1f/d48/d21/d2f/d46/d5e/f89 [5844718,71327] 0 2024-02-16T19:29:04.555 INFO:tasks.workunit.client.1.smithi117.stdout:4/402: read d0/da/d21/f65 [7271918,90366] 0 2024-02-16T19:29:04.555 INFO:tasks.workunit.client.0.smithi022.stdout:7/441: dread d9/da/f13 [0,4194304] 0 2024-02-16T19:29:04.557 INFO:tasks.workunit.client.0.smithi022.stdout:7/442: write d9/da/d2c/f8d [386142,32488] 0 2024-02-16T19:29:04.558 INFO:tasks.workunit.client.0.smithi022.stdout:9/457: creat db/d14/d16/d28/f9a x:0 0 0 2024-02-16T19:29:04.560 INFO:tasks.workunit.client.0.smithi022.stdout:1/548: truncate d6/f8e 942734 0 2024-02-16T19:29:04.561 INFO:tasks.workunit.client.0.smithi022.stdout:3/399: link d7/d12/d18/dd/d24/c54 d7/d21/c9c 0 2024-02-16T19:29:04.562 INFO:tasks.workunit.client.1.smithi117.stdout:7/376: dwrite d4/f11 [0,4194304] 0 2024-02-16T19:29:04.562 INFO:tasks.workunit.client.1.smithi117.stdout:0/465: symlink d0/d9/d14/d1d/d60/d76/d7b/d8d/la8 0 2024-02-16T19:29:04.563 INFO:tasks.workunit.client.1.smithi117.stdout:4/403: dwrite d0/da/d21/d3d/d28/f4e [0,4194304] 0 2024-02-16T19:29:04.564 INFO:tasks.workunit.client.0.smithi022.stdout:5/425: dwrite dd/d1f/d48/d21/f25 [0,4194304] 0 2024-02-16T19:29:04.566 INFO:tasks.workunit.client.0.smithi022.stdout:7/443: dread d9/d1b/f27 [0,4194304] 0 2024-02-16T19:29:04.566 INFO:tasks.workunit.client.1.smithi117.stdout:9/336: creat dd/d45/d31/f75 x:0 0 0 2024-02-16T19:29:04.567 INFO:tasks.workunit.client.1.smithi117.stdout:9/337: truncate dd/d47/f60 646522 0 2024-02-16T19:29:04.569 INFO:tasks.workunit.client.0.smithi022.stdout:7/444: write d9/da/d1f/f81 [197749,45956] 0 2024-02-16T19:29:04.572 INFO:tasks.workunit.client.0.smithi022.stdout:4/397: dwrite d4/d38/f3d [0,4194304] 0 2024-02-16T19:29:04.577 INFO:tasks.workunit.client.0.smithi022.stdout:8/415: truncate d3/d12/d6a/d98/f4c 393785 0 2024-02-16T19:29:04.582 INFO:tasks.workunit.client.1.smithi117.stdout:8/407: rmdir d4/d12/d2a/d64 39 2024-02-16T19:29:04.585 INFO:tasks.workunit.client.1.smithi117.stdout:5/501: mknod d2/d3/d21/d2f/d30/d38/d51/ca3 0 2024-02-16T19:29:04.586 INFO:tasks.workunit.client.1.smithi117.stdout:5/502: readlink d2/d9/d3c/d24/d50/l87 0 2024-02-16T19:29:04.587 INFO:tasks.workunit.client.0.smithi022.stdout:3/400: chown d7/d12/d18/dd/d24/d29/l3d 0 1 2024-02-16T19:29:04.589 INFO:tasks.workunit.client.0.smithi022.stdout:1/549: creat d6/d8/d22/d67/d7e/fa4 x:0 0 0 2024-02-16T19:29:04.590 INFO:tasks.workunit.client.0.smithi022.stdout:1/550: dread - d6/d8/d22/f82 zero size 2024-02-16T19:29:04.592 INFO:tasks.workunit.client.1.smithi117.stdout:6/401: rmdir d5/d1d/d49 39 2024-02-16T19:29:04.595 INFO:tasks.workunit.client.0.smithi022.stdout:4/398: dread d4/d6/dd/d22/d30/f44 [0,4194304] 0 2024-02-16T19:29:04.596 INFO:tasks.workunit.client.0.smithi022.stdout:4/399: chown d4/d6/dd/df/c20 521510 1 2024-02-16T19:29:04.598 INFO:tasks.workunit.client.0.smithi022.stdout:2/352: symlink d0/d2/d18/d38/d6b/l71 0 2024-02-16T19:29:04.598 INFO:tasks.workunit.client.0.smithi022.stdout:2/353: fdatasync d0/d2/f57 0 2024-02-16T19:29:04.601 INFO:tasks.workunit.client.1.smithi117.stdout:2/369: rename d0/d13/f43 to d0/d29/d56/f72 0 2024-02-16T19:29:04.601 INFO:tasks.workunit.client.1.smithi117.stdout:2/370: chown d0/d3/d40/f66 7401 1 2024-02-16T19:29:04.602 INFO:tasks.workunit.client.1.smithi117.stdout:3/494: write d1/d1b/d1e/d29/f33 [587997,8140] 0 2024-02-16T19:29:04.603 INFO:tasks.workunit.client.1.smithi117.stdout:3/495: write d1/d1b/f5f [673336,24596] 0 2024-02-16T19:29:04.604 INFO:tasks.workunit.client.1.smithi117.stdout:0/466: mkdir d0/d9/d2c/da9 0 2024-02-16T19:29:04.607 INFO:tasks.workunit.client.0.smithi022.stdout:2/354: dwrite d0/d32/f52 [0,4194304] 0 2024-02-16T19:29:04.607 INFO:tasks.workunit.client.0.smithi022.stdout:7/445: fdatasync d9/da/d1f/d40/f87 0 2024-02-16T19:29:04.611 INFO:tasks.workunit.client.1.smithi117.stdout:7/377: creat d4/d13/d63/f94 x:0 0 0 2024-02-16T19:29:04.613 INFO:tasks.workunit.client.1.smithi117.stdout:4/404: mknod d0/da/d21/d3d/d2d/d6d/c84 0 2024-02-16T19:29:04.617 INFO:tasks.workunit.client.0.smithi022.stdout:1/551: creat d6/d8/d22/d67/d7e/fa5 x:0 0 0 2024-02-16T19:29:04.619 INFO:tasks.workunit.client.1.smithi117.stdout:8/408: truncate d4/f6 4505425 0 2024-02-16T19:29:04.622 INFO:tasks.workunit.client.0.smithi022.stdout:6/434: rename d1/d8/da/d17/d53/d7f/f94 to d1/d8/d3c/f9d 0 2024-02-16T19:29:04.622 INFO:tasks.workunit.client.0.smithi022.stdout:6/435: readlink d1/l5 0 2024-02-16T19:29:04.623 INFO:tasks.workunit.client.0.smithi022.stdout:8/416: dread d3/d25/d33/f35 [0,4194304] 0 2024-02-16T19:29:04.624 INFO:tasks.workunit.client.1.smithi117.stdout:6/402: symlink d5/d4b/d71/l89 0 2024-02-16T19:29:04.626 INFO:tasks.workunit.client.0.smithi022.stdout:4/400: creat d4/d6/dd/df/d10/d55/f7b x:0 0 0 2024-02-16T19:29:04.627 INFO:tasks.workunit.client.0.smithi022.stdout:4/401: chown d4/d6/d3a 403 1 2024-02-16T19:29:04.632 INFO:tasks.workunit.client.1.smithi117.stdout:2/371: unlink d0/d3/d22/l45 0 2024-02-16T19:29:04.632 INFO:tasks.workunit.client.1.smithi117.stdout:1/364: truncate d2/d1d/d60/d20/f32 966493 0 2024-02-16T19:29:04.634 INFO:tasks.workunit.client.1.smithi117.stdout:3/496: unlink d1/d1b/d1e/lf 0 2024-02-16T19:29:04.636 INFO:tasks.workunit.client.1.smithi117.stdout:0/467: rmdir d0/d9/d22/d27/d83/d7c 39 2024-02-16T19:29:04.637 INFO:tasks.workunit.client.1.smithi117.stdout:0/468: truncate d0/d30/f64 4194950 0 2024-02-16T19:29:04.642 INFO:tasks.workunit.client.1.smithi117.stdout:7/378: truncate d4/f28 5856841 0 2024-02-16T19:29:04.651 INFO:tasks.workunit.client.0.smithi022.stdout:9/458: rename db/fc to db/d14/d16/d5d/d65/d6b/f9b 0 2024-02-16T19:29:04.651 INFO:tasks.workunit.client.0.smithi022.stdout:7/446: write d9/da/d1f/d40/f87 [169170,69938] 0 2024-02-16T19:29:04.653 INFO:tasks.workunit.client.0.smithi022.stdout:0/538: sync 2024-02-16T19:29:04.657 INFO:tasks.workunit.client.1.smithi117.stdout:5/503: rename d2/d4c/d4f/c8a to d2/d5d/ca4 0 2024-02-16T19:29:04.658 INFO:tasks.workunit.client.1.smithi117.stdout:6/403: dread d5/f13 [0,4194304] 0 2024-02-16T19:29:04.659 INFO:tasks.workunit.client.1.smithi117.stdout:6/404: chown d5/db/d7d/f2b 1506 1 2024-02-16T19:29:04.660 INFO:tasks.workunit.client.1.smithi117.stdout:1/365: truncate d2/d1d/f3a 570398 0 2024-02-16T19:29:04.660 INFO:tasks.workunit.client.1.smithi117.stdout:1/366: readlink d2/d19/l52 0 2024-02-16T19:29:04.660 INFO:tasks.workunit.client.0.smithi022.stdout:4/402: chown d4/l7a 312232 1 2024-02-16T19:29:04.663 INFO:tasks.workunit.client.1.smithi117.stdout:9/338: dwrite dd/d23/f30 [0,4194304] 0 2024-02-16T19:29:04.663 INFO:tasks.workunit.client.1.smithi117.stdout:3/497: fdatasync d1/d1b/d1e/d29/f47 0 2024-02-16T19:29:04.664 INFO:tasks.workunit.client.1.smithi117.stdout:9/339: readlink dd/d45/d31/l46 0 2024-02-16T19:29:04.669 INFO:tasks.workunit.client.0.smithi022.stdout:2/355: link d0/d2/d18/d1f/f5e d0/d2/d2e/f72 0 2024-02-16T19:29:04.681 INFO:tasks.workunit.client.0.smithi022.stdout:3/401: rename d7/d12/d18/dd/d24/d53/f62 to d7/d12/d18/dd/d31/d41/f9d 0 2024-02-16T19:29:04.684 INFO:tasks.workunit.client.1.smithi117.stdout:2/372: rename d0/d13/f1c to d0/d29/f73 0 2024-02-16T19:29:04.684 INFO:tasks.workunit.client.1.smithi117.stdout:0/469: dread d0/f1a [0,4194304] 0 2024-02-16T19:29:04.685 INFO:tasks.workunit.client.0.smithi022.stdout:1/552: write d6/d8/d22/d16/f4b [2220544,47874] 0 2024-02-16T19:29:04.686 INFO:tasks.workunit.client.1.smithi117.stdout:2/373: write d0/d29/f2d [4509199,78343] 0 2024-02-16T19:29:04.686 INFO:tasks.workunit.client.1.smithi117.stdout:0/470: chown d0/d9/l81 1 1 2024-02-16T19:29:04.688 INFO:tasks.workunit.client.0.smithi022.stdout:0/539: creat d10/d15/d23/d38/fb1 x:0 0 0 2024-02-16T19:29:04.692 INFO:tasks.workunit.client.1.smithi117.stdout:2/374: dwrite d0/d29/f49 [0,4194304] 0 2024-02-16T19:29:04.692 INFO:tasks.workunit.client.1.smithi117.stdout:2/375: readlink d0/d35/l65 0 2024-02-16T19:29:04.694 INFO:tasks.workunit.client.1.smithi117.stdout:2/376: chown d0/d3/d22/f28 112321 1 2024-02-16T19:29:04.694 INFO:tasks.workunit.client.0.smithi022.stdout:6/436: link d1/d8/da/d17/d85/d5b/f88 d1/d8/da/d29/d61/f9e 0 2024-02-16T19:29:04.695 INFO:tasks.workunit.client.1.smithi117.stdout:2/377: fsync d0/d3/f31 0 2024-02-16T19:29:04.696 INFO:tasks.workunit.client.0.smithi022.stdout:5/426: getdents dd/d1f/d4a 0 2024-02-16T19:29:04.697 INFO:tasks.workunit.client.1.smithi117.stdout:8/409: dwrite d4/f3c [0,4194304] 0 2024-02-16T19:29:04.699 INFO:tasks.workunit.client.0.smithi022.stdout:4/403: symlink d4/d6/d3a/l7c 0 2024-02-16T19:29:04.699 INFO:tasks.workunit.client.1.smithi117.stdout:9/340: rmdir dd/d47/d48 39 2024-02-16T19:29:04.701 INFO:tasks.workunit.client.1.smithi117.stdout:4/405: link d0/da/d21/d3d/d28/d64/d48/f5d d0/da/d21/d3d/d28/d64/d74/f85 0 2024-02-16T19:29:04.711 INFO:tasks.workunit.client.0.smithi022.stdout:7/447: creat d9/d5d/f91 x:0 0 0 2024-02-16T19:29:04.713 INFO:tasks.workunit.client.0.smithi022.stdout:7/448: truncate d9/da/d2c/d2f/f76 916974 0 2024-02-16T19:29:04.714 INFO:tasks.workunit.client.1.smithi117.stdout:5/504: rename d2/d3/d21/f32 to d2/d9/d3c/d24/d50/d52/fa5 0 2024-02-16T19:29:04.714 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:04 smithi022 ceph-mon[29221]: pgmap v5: 65 pgs: 65 active+clean; 1.4 GiB data, 4.2 GiB used, 532 GiB / 536 GiB avail 2024-02-16T19:29:04.714 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:04 smithi022 ceph-mon[29221]: mgrmap e24: smithi117.cohbtv(active, since 4s) 2024-02-16T19:29:04.716 INFO:tasks.workunit.client.0.smithi022.stdout:1/553: symlink d6/d8/d22/d16/d17/d1d/la6 0 2024-02-16T19:29:04.718 INFO:tasks.workunit.client.0.smithi022.stdout:0/540: write d10/d22/d2c/f72 [1458670,126934] 0 2024-02-16T19:29:04.719 INFO:tasks.workunit.client.1.smithi117.stdout:1/367: mknod d2/d19/c79 0 2024-02-16T19:29:04.720 INFO:tasks.workunit.client.1.smithi117.stdout:8/410: dread d4/d7/d37/f56 [0,4194304] 0 2024-02-16T19:29:04.720 INFO:tasks.workunit.client.0.smithi022.stdout:8/417: dwrite d3/d12/f51 [0,4194304] 0 2024-02-16T19:29:04.723 INFO:tasks.workunit.client.0.smithi022.stdout:8/418: read d3/d12/d2b/f3a [10929896,89847] 0 2024-02-16T19:29:04.725 INFO:tasks.workunit.client.1.smithi117.stdout:0/471: fsync d0/f1 0 2024-02-16T19:29:04.727 INFO:tasks.workunit.client.0.smithi022.stdout:9/459: dread db/d14/d19/d1a/f24 [0,4194304] 0 2024-02-16T19:29:04.733 INFO:tasks.workunit.client.1.smithi117.stdout:2/378: symlink d0/d3/d40/l74 0 2024-02-16T19:29:04.735 INFO:tasks.workunit.client.1.smithi117.stdout:8/411: dread d4/d7/d37/d3a/d43/d44/f4b [0,4194304] 0 2024-02-16T19:29:04.735 INFO:tasks.workunit.client.0.smithi022.stdout:3/402: dwrite d7/d21/f23 [0,4194304] 0 2024-02-16T19:29:04.736 INFO:tasks.workunit.client.1.smithi117.stdout:2/379: dread d0/d29/f2e [0,4194304] 0 2024-02-16T19:29:04.739 INFO:tasks.workunit.client.1.smithi117.stdout:7/379: creat d4/f95 x:0 0 0 2024-02-16T19:29:04.740 INFO:tasks.workunit.client.0.smithi022.stdout:7/449: creat d9/da/d1f/d40/f92 x:0 0 0 2024-02-16T19:29:04.741 INFO:tasks.workunit.client.0.smithi022.stdout:7/450: dread - d9/da/d74/f80 zero size 2024-02-16T19:29:04.743 INFO:tasks.workunit.client.1.smithi117.stdout:4/406: creat d0/da/d21/d3d/d28/f86 x:0 0 0 2024-02-16T19:29:04.743 INFO:tasks.workunit.client.1.smithi117.stdout:7/380: dread d4/f1e [0,4194304] 0 2024-02-16T19:29:04.745 INFO:tasks.workunit.client.0.smithi022.stdout:0/541: symlink d10/d7d/lb2 0 2024-02-16T19:29:04.747 INFO:tasks.workunit.client.1.smithi117.stdout:3/498: dwrite d1/dd/d32/f75 [0,4194304] 0 2024-02-16T19:29:04.750 INFO:tasks.workunit.client.0.smithi022.stdout:8/419: creat d3/d12/f9c x:0 0 0 2024-02-16T19:29:04.752 INFO:tasks.workunit.client.0.smithi022.stdout:8/420: truncate d3/d12/d2b/d46/d68/f81 173849 0 2024-02-16T19:29:04.753 INFO:tasks.workunit.client.1.smithi117.stdout:1/368: dwrite d2/d51/f75 [0,4194304] 0 2024-02-16T19:29:04.753 INFO:tasks.workunit.client.1.smithi117.stdout:0/472: write d0/d9/d22/d27/d83/f67 [307080,49655] 0 2024-02-16T19:29:04.755 INFO:tasks.workunit.client.1.smithi117.stdout:1/369: chown d2/d1d/d60/d20/f2a 0 1 2024-02-16T19:29:04.758 INFO:tasks.workunit.client.1.smithi117.stdout:6/405: sync 2024-02-16T19:29:04.759 INFO:tasks.workunit.client.0.smithi022.stdout:9/460: dwrite db/f79 [0,4194304] 0 2024-02-16T19:29:04.759 INFO:tasks.workunit.client.1.smithi117.stdout:6/406: chown d5/f13 493 1 2024-02-16T19:29:04.759 INFO:tasks.workunit.client.0.smithi022.stdout:5/427: unlink dd/d1f/d48/d21/d2f/d52/f53 0 2024-02-16T19:29:04.764 INFO:tasks.workunit.client.0.smithi022.stdout:6/437: truncate d1/d8/d3c/d77/f80 1857887 0 2024-02-16T19:29:04.769 INFO:tasks.workunit.client.0.smithi022.stdout:3/403: rename d7/d12/d18/dd/f20 to d7/d12/d18/d4b/f9e 0 2024-02-16T19:29:04.770 INFO:tasks.workunit.client.0.smithi022.stdout:3/404: chown d7/d12/d18/dd/d24/d58/l68 2 1 2024-02-16T19:29:04.773 INFO:tasks.workunit.client.0.smithi022.stdout:1/554: mkdir d6/d8/d22/d59/da7 0 2024-02-16T19:29:04.775 INFO:tasks.workunit.client.0.smithi022.stdout:1/555: write d6/d8/d22/d67/d7e/fa4 [773401,27512] 0 2024-02-16T19:29:04.776 INFO:tasks.workunit.client.0.smithi022.stdout:7/451: mkdir d9/da/d1f/d93 0 2024-02-16T19:29:04.779 INFO:tasks.workunit.client.0.smithi022.stdout:0/542: unlink d10/d15/d23/c34 0 2024-02-16T19:29:04.779 INFO:tasks.workunit.client.0.smithi022.stdout:4/404: dwrite d4/d6/dd/df/f29 [0,4194304] 0 2024-02-16T19:29:04.782 INFO:tasks.workunit.client.0.smithi022.stdout:8/421: creat d3/d12/d17/f9d x:0 0 0 2024-02-16T19:29:04.786 INFO:tasks.workunit.client.0.smithi022.stdout:5/428: dread dd/d1f/d48/f26 [0,4194304] 0 2024-02-16T19:29:04.787 INFO:tasks.workunit.client.0.smithi022.stdout:5/429: readlink dd/d1f/l69 0 2024-02-16T19:29:04.791 INFO:tasks.workunit.client.0.smithi022.stdout:9/461: mknod db/d14/d16/d5d/d65/d6b/c9c 0 2024-02-16T19:29:04.791 INFO:tasks.workunit.client.1.smithi117.stdout:8/412: read d4/d7/d37/d3a/d43/d44/f62 [2980858,34484] 0 2024-02-16T19:29:04.793 INFO:tasks.workunit.client.0.smithi022.stdout:2/356: getdents d0/d2/d2e 0 2024-02-16T19:29:04.795 INFO:tasks.workunit.client.0.smithi022.stdout:3/405: creat d7/d12/d18/dd/d24/f9f x:0 0 0 2024-02-16T19:29:04.797 INFO:tasks.workunit.client.0.smithi022.stdout:3/406: chown d7/d12/d18/dd/d3c/d5a/d67/f75 96 1 2024-02-16T19:29:04.797 INFO:tasks.workunit.client.1.smithi117.stdout:9/341: mknod dd/d23/c76 0 2024-02-16T19:29:04.797 INFO:tasks.workunit.client.1.smithi117.stdout:8/413: dwrite d4/d7/d37/f63 [0,4194304] 0 2024-02-16T19:29:04.801 INFO:tasks.workunit.client.1.smithi117.stdout:8/414: dread d4/d7/d6a/d6f/f7a [0,4194304] 0 2024-02-16T19:29:04.801 INFO:tasks.workunit.client.0.smithi022.stdout:4/405: dread d4/d6/dd/df/d10/f42 [0,4194304] 0 2024-02-16T19:29:04.805 INFO:tasks.workunit.client.1.smithi117.stdout:4/407: symlink d0/da/d21/d3d/d28/d66/l87 0 2024-02-16T19:29:04.808 INFO:tasks.workunit.client.0.smithi022.stdout:6/438: dread d1/f3 [0,4194304] 0 2024-02-16T19:29:04.812 INFO:tasks.workunit.client.1.smithi117.stdout:7/381: creat d4/d1c/d39/d46/f96 x:0 0 0 2024-02-16T19:29:04.814 INFO:tasks.workunit.client.1.smithi117.stdout:3/499: symlink d1/dd/d3b/lac 0 2024-02-16T19:29:04.815 INFO:tasks.workunit.client.1.smithi117.stdout:3/500: write d1/d30/d52/fa4 [357719,56113] 0 2024-02-16T19:29:04.815 INFO:tasks.workunit.client.0.smithi022.stdout:0/543: dwrite d10/d11/d27/d3f/d58/d60/d62/f4f [0,4194304] 0 2024-02-16T19:29:04.824 INFO:tasks.workunit.client.0.smithi022.stdout:0/544: dwrite d10/d11/d50/d65/f68 [0,4194304] 0 2024-02-16T19:29:04.824 INFO:tasks.workunit.client.0.smithi022.stdout:7/452: write d9/da/d1f/d44/f49 [4848973,78854] 0 2024-02-16T19:29:04.828 INFO:tasks.workunit.client.0.smithi022.stdout:1/556: creat d6/d8/d22/d59/da7/fa8 x:0 0 0 2024-02-16T19:29:04.829 INFO:tasks.workunit.client.1.smithi117.stdout:6/407: symlink d5/db/d7d/d32/l8a 0 2024-02-16T19:29:04.830 INFO:tasks.workunit.client.0.smithi022.stdout:7/453: dread d9/da/d2c/d46/d73/f8e [0,4194304] 0 2024-02-16T19:29:04.830 INFO:tasks.workunit.client.1.smithi117.stdout:0/473: dread d0/fd [4194304,4194304] 0 2024-02-16T19:29:04.832 INFO:tasks.workunit.client.0.smithi022.stdout:4/406: creat d4/d6/d14/d64/d66/d6d/f7d x:0 0 0 2024-02-16T19:29:04.836 INFO:tasks.workunit.client.0.smithi022.stdout:6/439: read - d1/d8/d81/f8c zero size 2024-02-16T19:29:04.837 INFO:tasks.workunit.client.0.smithi022.stdout:6/440: chown d1/f3 12607396 1 2024-02-16T19:29:04.839 INFO:tasks.workunit.client.0.smithi022.stdout:6/441: fsync d1/d8/da/d17/f96 0 2024-02-16T19:29:04.840 INFO:tasks.workunit.client.0.smithi022.stdout:8/422: link d3/d12/d2b/d54/f76 d3/d12/f9e 0 2024-02-16T19:29:04.841 INFO:tasks.workunit.client.0.smithi022.stdout:2/357: write d0/d2/d18/d19/f30 [256631,108323] 0 2024-02-16T19:29:04.842 INFO:tasks.workunit.client.0.smithi022.stdout:6/442: stat d1/d8/da/d17/d53/f5c 0 2024-02-16T19:29:04.845 INFO:tasks.workunit.client.0.smithi022.stdout:9/462: dwrite db/d14/d16/d27/f78 [0,4194304] 0 2024-02-16T19:29:04.846 INFO:tasks.workunit.client.0.smithi022.stdout:6/443: chown d1/d8/da/d17/d53/l5a 19380 1 2024-02-16T19:29:04.846 INFO:tasks.workunit.client.0.smithi022.stdout:9/463: dread - db/d14/d19/f5f zero size 2024-02-16T19:29:04.849 INFO:tasks.workunit.client.1.smithi117.stdout:4/408: mknod d0/da/d21/d3d/d28/d2f/d37/c88 0 2024-02-16T19:29:04.849 INFO:tasks.workunit.client.0.smithi022.stdout:9/464: write db/d14/d33/f88 [503116,119828] 0 2024-02-16T19:29:04.852 INFO:tasks.workunit.client.1.smithi117.stdout:4/409: truncate d0/da/d21/d3d/d28/d64/f67 498697 0 2024-02-16T19:29:04.853 INFO:tasks.workunit.client.1.smithi117.stdout:4/410: readlink d0/da/l2e 0 2024-02-16T19:29:04.855 INFO:tasks.workunit.client.1.smithi117.stdout:9/342: dwrite dd/f1e [0,4194304] 0 2024-02-16T19:29:04.858 INFO:tasks.workunit.client.1.smithi117.stdout:1/370: link d2/d1d/d1f/f56 d2/d19/d59/d76/f7a 0 2024-02-16T19:29:04.859 INFO:tasks.workunit.client.0.smithi022.stdout:5/430: dwrite dd/d1f/d48/d21/d2f/d52/d5c/f85 [0,4194304] 0 2024-02-16T19:29:04.861 INFO:tasks.workunit.client.0.smithi022.stdout:1/557: creat d6/d8/d22/d67/d7e/fa9 x:0 0 0 2024-02-16T19:29:04.861 INFO:tasks.workunit.client.1.smithi117.stdout:6/408: read d5/fa [3904008,56536] 0 2024-02-16T19:29:04.861 INFO:tasks.workunit.client.0.smithi022.stdout:7/454: symlink d9/da/d4e/l94 0 2024-02-16T19:29:04.864 INFO:tasks.workunit.client.1.smithi117.stdout:2/380: truncate d0/d3/f31 663800 0 2024-02-16T19:29:04.870 INFO:tasks.workunit.client.1.smithi117.stdout:8/415: creat d4/d73/f82 x:0 0 0 2024-02-16T19:29:04.871 INFO:tasks.workunit.client.0.smithi022.stdout:2/358: mkdir d0/d2/d2e/d73 0 2024-02-16T19:29:04.871 INFO:tasks.workunit.client.1.smithi117.stdout:8/416: write d4/d7/d37/d3a/d43/f55 [829953,1857] 0 2024-02-16T19:29:04.872 INFO:tasks.workunit.client.1.smithi117.stdout:5/505: getdents d2/d3/d21/d2f/d30 0 2024-02-16T19:29:04.875 INFO:tasks.workunit.client.1.smithi117.stdout:7/382: symlink d4/l97 0 2024-02-16T19:29:04.877 INFO:tasks.workunit.client.1.smithi117.stdout:4/411: dread d0/da/d21/d3d/d28/d66/f7f [0,4194304] 0 2024-02-16T19:29:04.877 INFO:tasks.workunit.client.0.smithi022.stdout:6/444: symlink d1/d8/da/d17/d53/l9f 0 2024-02-16T19:29:04.878 INFO:tasks.workunit.client.0.smithi022.stdout:2/359: dwrite d0/f37 [0,4194304] 0 2024-02-16T19:29:04.884 INFO:tasks.workunit.client.1.smithi117.stdout:9/343: mkdir dd/d47/d5b/d5f/d77 0 2024-02-16T19:29:04.884 INFO:tasks.workunit.client.0.smithi022.stdout:4/407: write d4/f23 [663791,113548] 0 2024-02-16T19:29:04.885 INFO:tasks.workunit.client.0.smithi022.stdout:2/360: dwrite d0/d2/d18/d1f/f4d [0,4194304] 0 2024-02-16T19:29:04.886 INFO:tasks.workunit.client.0.smithi022.stdout:4/408: stat d4/d6/dd/d22 0 2024-02-16T19:29:04.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:04 smithi117 ceph-mon[35309]: pgmap v5: 65 pgs: 65 active+clean; 1.4 GiB data, 4.2 GiB used, 532 GiB / 536 GiB avail 2024-02-16T19:29:04.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:04 smithi117 ceph-mon[35309]: mgrmap e24: smithi117.cohbtv(active, since 4s) 2024-02-16T19:29:04.887 INFO:tasks.workunit.client.0.smithi022.stdout:0/545: dread d10/d15/d23/d8f/f95 [0,4194304] 0 2024-02-16T19:29:04.890 INFO:tasks.workunit.client.1.smithi117.stdout:1/371: write d2/d1d/d60/d20/d49/f50 [1318806,42295] 0 2024-02-16T19:29:04.890 INFO:tasks.workunit.client.0.smithi022.stdout:9/465: unlink db/d14/d16/d27/d55/f62 0 2024-02-16T19:29:04.893 INFO:tasks.workunit.client.1.smithi117.stdout:0/474: dread d0/f5 [4194304,4194304] 0 2024-02-16T19:29:04.894 INFO:tasks.workunit.client.0.smithi022.stdout:0/546: dread d10/d15/d23/d8f/f95 [0,4194304] 0 2024-02-16T19:29:04.894 INFO:tasks.workunit.client.1.smithi117.stdout:6/409: rename d5/d1d/d72/f77 to d5/d4b/d44/d52/d64/d85/f8b 0 2024-02-16T19:29:04.895 INFO:tasks.workunit.client.1.smithi117.stdout:6/410: chown d5/d1d/d72 57 1 2024-02-16T19:29:04.896 INFO:tasks.workunit.client.1.smithi117.stdout:6/411: chown d5/d4b/d20/f78 18800902 1 2024-02-16T19:29:04.899 INFO:tasks.workunit.client.1.smithi117.stdout:8/417: unlink d4/d12/d2a/c39 0 2024-02-16T19:29:04.900 INFO:tasks.workunit.client.1.smithi117.stdout:5/506: dread d2/d3/f47 [0,4194304] 0 2024-02-16T19:29:04.901 INFO:tasks.workunit.client.1.smithi117.stdout:5/507: truncate d2/d3/f76 188197 0 2024-02-16T19:29:04.905 INFO:tasks.workunit.client.1.smithi117.stdout:7/383: mkdir d4/d1c/d98 0 2024-02-16T19:29:04.906 INFO:tasks.workunit.client.0.smithi022.stdout:7/455: mkdir d9/da/d1f/d44/d5b/d95 0 2024-02-16T19:29:04.906 INFO:tasks.workunit.client.1.smithi117.stdout:5/508: dwrite d2/d4c/d4f/f81 [0,4194304] 0 2024-02-16T19:29:04.907 INFO:tasks.workunit.client.0.smithi022.stdout:7/456: fdatasync d9/da/d1f/d44/f49 0 2024-02-16T19:29:04.908 INFO:tasks.workunit.client.0.smithi022.stdout:1/558: symlink d6/d8/d22/d16/d17/d63/laa 0 2024-02-16T19:29:04.912 INFO:tasks.workunit.client.1.smithi117.stdout:5/509: dread d2/d5d/f5e [0,4194304] 0 2024-02-16T19:29:04.913 INFO:tasks.workunit.client.0.smithi022.stdout:1/559: dread d6/d8/d22/d16/d17/d1d/f33 [4194304,4194304] 0 2024-02-16T19:29:04.914 INFO:tasks.workunit.client.1.smithi117.stdout:4/412: dwrite d0/da/f69 [0,4194304] 0 2024-02-16T19:29:04.917 INFO:tasks.workunit.client.1.smithi117.stdout:1/372: mkdir d2/d1d/d60/d20/d7b 0 2024-02-16T19:29:04.919 INFO:tasks.workunit.client.1.smithi117.stdout:1/373: write d2/d19/f2f [7933370,1915] 0 2024-02-16T19:29:04.920 INFO:tasks.workunit.client.0.smithi022.stdout:6/445: creat d1/d8/d81/fa0 x:0 0 0 2024-02-16T19:29:04.922 INFO:tasks.workunit.client.1.smithi117.stdout:9/344: dwrite dd/d47/d48/f66 [0,4194304] 0 2024-02-16T19:29:04.922 INFO:tasks.workunit.client.1.smithi117.stdout:9/345: readlink dd/l10 0 2024-02-16T19:29:04.923 INFO:tasks.workunit.client.1.smithi117.stdout:9/346: chown dd/d47/f60 62866 1 2024-02-16T19:29:04.926 INFO:tasks.workunit.client.0.smithi022.stdout:9/466: dread db/d14/d33/f88 [0,4194304] 0 2024-02-16T19:29:04.927 INFO:tasks.workunit.client.1.smithi117.stdout:2/381: rename d0/d35 to d0/d13/d75 0 2024-02-16T19:29:04.928 INFO:tasks.workunit.client.1.smithi117.stdout:8/418: mkdir d4/d7/d6a/d72/d83 0 2024-02-16T19:29:04.928 INFO:tasks.workunit.client.0.smithi022.stdout:9/467: read db/d14/f60 [3737894,56281] 0 2024-02-16T19:29:04.934 INFO:tasks.workunit.client.1.smithi117.stdout:8/419: dwrite d4/d12/d2a/f2b [0,4194304] 0 2024-02-16T19:29:04.934 INFO:tasks.workunit.client.0.smithi022.stdout:0/547: mknod d10/d11/d27/cb3 0 2024-02-16T19:29:04.941 INFO:tasks.workunit.client.1.smithi117.stdout:8/420: dwrite d4/d12/f26 [0,4194304] 0 2024-02-16T19:29:04.942 INFO:tasks.workunit.client.1.smithi117.stdout:3/501: link d1/d30/c6a d1/d83/d96/cad 0 2024-02-16T19:29:04.943 INFO:tasks.workunit.client.1.smithi117.stdout:3/502: stat d1/dd/d8e/f94 0 2024-02-16T19:29:04.944 INFO:tasks.workunit.client.0.smithi022.stdout:1/560: unlink d6/d8/d22/d47/d54/l75 0 2024-02-16T19:29:04.946 INFO:tasks.workunit.client.0.smithi022.stdout:1/561: write d6/d8/d22/d47/d51/f85 [1928245,72293] 0 2024-02-16T19:29:04.947 INFO:tasks.workunit.client.0.smithi022.stdout:1/562: chown d6/d8/d22/d3f/d99 217718247 1 2024-02-16T19:29:04.950 INFO:tasks.workunit.client.0.smithi022.stdout:8/423: fdatasync d3/d12/d17/f3d 0 2024-02-16T19:29:04.951 INFO:tasks.workunit.client.0.smithi022.stdout:8/424: truncate d3/d25/f79 199685 0 2024-02-16T19:29:04.951 INFO:tasks.workunit.client.1.smithi117.stdout:5/510: creat d2/d3/d21/d2a/fa6 x:0 0 0 2024-02-16T19:29:04.952 INFO:tasks.workunit.client.0.smithi022.stdout:8/425: write d3/f55 [799904,24208] 0 2024-02-16T19:29:04.952 INFO:tasks.workunit.client.0.smithi022.stdout:6/446: fsync d1/d8/f40 0 2024-02-16T19:29:04.954 INFO:tasks.workunit.client.0.smithi022.stdout:8/426: write d3/f8d [4672221,20954] 0 2024-02-16T19:29:04.955 INFO:tasks.workunit.client.0.smithi022.stdout:2/361: creat d0/d2/d18/d19/d40/d55/d59/f74 x:0 0 0 2024-02-16T19:29:04.959 INFO:tasks.workunit.client.0.smithi022.stdout:5/431: write dd/d1f/d48/d21/d2f/d40/d4e/d56/f59 [933104,130113] 0 2024-02-16T19:29:04.960 INFO:tasks.workunit.client.1.smithi117.stdout:7/384: dwrite d4/f2b [0,4194304] 0 2024-02-16T19:29:04.962 INFO:tasks.workunit.client.1.smithi117.stdout:0/475: creat d0/d9/d2c/da9/faa x:0 0 0 2024-02-16T19:29:04.964 INFO:tasks.workunit.client.1.smithi117.stdout:7/385: dread d4/f2b [0,4194304] 0 2024-02-16T19:29:04.965 INFO:tasks.workunit.client.1.smithi117.stdout:7/386: stat d4/f2b 0 2024-02-16T19:29:04.965 INFO:tasks.workunit.client.0.smithi022.stdout:9/468: rmdir db/d14/d16 39 2024-02-16T19:29:04.967 INFO:tasks.workunit.client.1.smithi117.stdout:1/374: mkdir d2/d1d/d60/d7c 0 2024-02-16T19:29:04.970 INFO:tasks.workunit.client.1.smithi117.stdout:6/412: mknod d5/db/d7d/d32/d79/c8c 0 2024-02-16T19:29:04.980 INFO:tasks.workunit.client.0.smithi022.stdout:8/427: chown d3/d12/d17/f7e 2242 1 2024-02-16T19:29:04.982 INFO:tasks.workunit.client.0.smithi022.stdout:6/447: truncate d1/d8/da/d17/d85/d56/f6a 749160 0 2024-02-16T19:29:04.983 INFO:tasks.workunit.client.1.smithi117.stdout:2/382: dread d0/d3/d10/f16 [0,4194304] 0 2024-02-16T19:29:04.984 INFO:tasks.workunit.client.0.smithi022.stdout:2/362: fdatasync d0/d2/d18/d1f/f5e 0 2024-02-16T19:29:04.988 INFO:tasks.workunit.client.1.smithi117.stdout:4/413: creat d0/da/d21/d3d/d28/d64/d74/f89 x:0 0 0 2024-02-16T19:29:04.988 INFO:tasks.workunit.client.0.smithi022.stdout:1/563: dread d6/f9 [0,4194304] 0 2024-02-16T19:29:04.990 INFO:tasks.workunit.client.0.smithi022.stdout:2/363: dread d0/d32/f52 [0,4194304] 0 2024-02-16T19:29:04.990 INFO:tasks.workunit.client.0.smithi022.stdout:4/409: link d4/d6/d14/f1d d4/d6/d14/d64/d66/d6d/f7e 0 2024-02-16T19:29:04.991 INFO:tasks.workunit.client.0.smithi022.stdout:2/364: dread - d0/d2/d2e/f72 zero size 2024-02-16T19:29:04.992 INFO:tasks.workunit.client.0.smithi022.stdout:2/365: readlink d0/d2/d18/d19/l68 0 2024-02-16T19:29:04.995 INFO:tasks.workunit.client.1.smithi117.stdout:7/387: rename d4/d1c/d87/d89/d5f/f84 to d4/d1c/d39/d46/d6e/f99 0 2024-02-16T19:29:04.999 INFO:tasks.workunit.client.1.smithi117.stdout:1/375: mkdir d2/d1d/d60/d20/d7d 0 2024-02-16T19:29:05.000 INFO:tasks.workunit.client.0.smithi022.stdout:2/366: dwrite d0/d2/f3c [0,4194304] 0 2024-02-16T19:29:05.002 INFO:tasks.workunit.client.0.smithi022.stdout:2/367: dread - d0/d2/d18/d19/d40/f6d zero size 2024-02-16T19:29:05.003 INFO:tasks.workunit.client.1.smithi117.stdout:9/347: creat dd/d45/d73/f78 x:0 0 0 2024-02-16T19:29:05.004 INFO:tasks.workunit.client.0.smithi022.stdout:2/368: write d0/d32/f62 [116177,86773] 0 2024-02-16T19:29:05.005 INFO:tasks.workunit.client.1.smithi117.stdout:7/388: dread d4/d1c/d87/f50 [4194304,4194304] 0 2024-02-16T19:29:05.006 INFO:tasks.workunit.client.1.smithi117.stdout:9/348: write dd/d23/d64/f70 [2821872,3255] 0 2024-02-16T19:29:05.007 INFO:tasks.workunit.client.1.smithi117.stdout:7/389: dread - d4/d1c/d39/d46/d47/f51 zero size 2024-02-16T19:29:05.008 INFO:tasks.workunit.client.0.smithi022.stdout:6/448: fdatasync d1/f3 0 2024-02-16T19:29:05.009 INFO:tasks.workunit.client.0.smithi022.stdout:4/410: dread d4/d6/d14/f2e [0,4194304] 0 2024-02-16T19:29:05.011 INFO:tasks.workunit.client.1.smithi117.stdout:3/503: symlink d1/d1b/d1e/da6/lae 0 2024-02-16T19:29:05.016 INFO:tasks.workunit.client.0.smithi022.stdout:5/432: symlink dd/d1f/d48/d21/l8c 0 2024-02-16T19:29:05.018 INFO:tasks.workunit.client.1.smithi117.stdout:3/504: truncate d1/d30/f77 4401550 0 2024-02-16T19:29:05.019 INFO:tasks.workunit.client.1.smithi117.stdout:8/421: dwrite d4/d12/d2a/f2c [0,4194304] 0 2024-02-16T19:29:05.021 INFO:tasks.workunit.client.1.smithi117.stdout:7/390: dwrite d4/d1c/d39/d46/f48 [0,4194304] 0 2024-02-16T19:29:05.021 INFO:tasks.workunit.client.1.smithi117.stdout:5/511: dwrite d2/d1d/f5f [0,4194304] 0 2024-02-16T19:29:05.022 INFO:tasks.workunit.client.1.smithi117.stdout:3/505: chown d1/d1b/d1e/d5e 32568 1 2024-02-16T19:29:05.023 INFO:tasks.workunit.client.1.smithi117.stdout:7/391: dread - d4/f95 zero size 2024-02-16T19:29:05.024 INFO:tasks.workunit.client.1.smithi117.stdout:4/414: rmdir d0/da/d21/d3d/d2d/d6d 39 2024-02-16T19:29:05.025 INFO:tasks.workunit.client.1.smithi117.stdout:8/422: write d4/d1c/f7f [272288,8660] 0 2024-02-16T19:29:05.025 INFO:tasks.workunit.client.1.smithi117.stdout:1/376: dread d2/d4/f15 [0,4194304] 0 2024-02-16T19:29:05.026 INFO:tasks.workunit.client.1.smithi117.stdout:0/476: dwrite d0/f5 [0,4194304] 0 2024-02-16T19:29:05.026 INFO:tasks.workunit.client.0.smithi022.stdout:9/469: read db/d14/d16/d21/d37/f63 [808941,44093] 0 2024-02-16T19:29:05.027 INFO:tasks.workunit.client.1.smithi117.stdout:3/506: write d1/d1b/d1e/d5e/f62 [1330742,83030] 0 2024-02-16T19:29:05.031 INFO:tasks.workunit.client.0.smithi022.stdout:0/548: link ca d10/d15/d23/d38/d5a/cb4 0 2024-02-16T19:29:05.033 INFO:tasks.workunit.client.0.smithi022.stdout:0/549: chown d10/d15/d23/d38/d5a/d66 42 1 2024-02-16T19:29:05.035 INFO:tasks.workunit.client.1.smithi117.stdout:7/392: dwrite d4/d1c/d39/d46/d47/f82 [0,4194304] 0 2024-02-16T19:29:05.036 INFO:tasks.workunit.client.1.smithi117.stdout:7/393: chown d4/d1c/d39/d46/d55/c44 259113327 1 2024-02-16T19:29:05.039 INFO:tasks.workunit.client.0.smithi022.stdout:8/428: mknod d3/d12/d2b/d54/d97/c9f 0 2024-02-16T19:29:05.040 INFO:tasks.workunit.client.0.smithi022.stdout:8/429: write d3/d12/f51 [2542504,114410] 0 2024-02-16T19:29:05.042 INFO:tasks.workunit.client.0.smithi022.stdout:2/369: truncate d0/d2/d2e/f72 736027 0 2024-02-16T19:29:05.043 INFO:tasks.workunit.client.0.smithi022.stdout:2/370: readlink d0/d2/d18/d38/l4b 0 2024-02-16T19:29:05.045 INFO:tasks.workunit.client.0.smithi022.stdout:2/371: write d0/d2/d18/d19/d40/f6d [600418,104382] 0 2024-02-16T19:29:05.048 INFO:tasks.workunit.client.0.smithi022.stdout:4/411: creat d4/d6/f7f x:0 0 0 2024-02-16T19:29:05.052 INFO:tasks.workunit.client.0.smithi022.stdout:4/412: write d4/d6/d3a/f41 [645812,128431] 0 2024-02-16T19:29:05.056 INFO:tasks.workunit.client.1.smithi117.stdout:2/383: mkdir d0/d13/d4d/d64/d46/d68/d76 0 2024-02-16T19:29:05.056 INFO:tasks.workunit.client.0.smithi022.stdout:2/372: dwrite d0/d32/d3b/d6f/f5a [0,4194304] 0 2024-02-16T19:29:05.059 INFO:tasks.workunit.client.1.smithi117.stdout:8/423: fsync d4/d1c/f1f 0 2024-02-16T19:29:05.060 INFO:tasks.workunit.client.0.smithi022.stdout:0/550: symlink d10/d11/d50/d65/lb5 0 2024-02-16T19:29:05.061 INFO:tasks.workunit.client.0.smithi022.stdout:0/551: chown d10/d11/d27/l44 494239 1 2024-02-16T19:29:05.062 INFO:tasks.workunit.client.1.smithi117.stdout:1/377: fdatasync d2/d1d/d60/d20/f2a 0 2024-02-16T19:29:05.064 INFO:tasks.workunit.client.0.smithi022.stdout:8/430: creat d3/d25/d33/fa0 x:0 0 0 2024-02-16T19:29:05.066 INFO:tasks.workunit.client.0.smithi022.stdout:2/373: dwrite d0/d2/f1c [0,4194304] 0 2024-02-16T19:29:05.070 INFO:tasks.workunit.client.0.smithi022.stdout:0/552: read d10/d11/d27/f52 [3348661,46553] 0 2024-02-16T19:29:05.072 INFO:tasks.workunit.client.0.smithi022.stdout:0/553: stat d10/d11/d27/d3f/d58/d60/d62/l63 0 2024-02-16T19:29:05.074 INFO:tasks.workunit.client.0.smithi022.stdout:2/374: dwrite d0/d2/d18/d1f/f21 [0,4194304] 0 2024-02-16T19:29:05.074 INFO:tasks.workunit.client.1.smithi117.stdout:0/477: write d0/d9/d22/d27/f4d [2211131,542] 0 2024-02-16T19:29:05.074 INFO:tasks.workunit.client.1.smithi117.stdout:3/507: dwrite d1/dd/f34 [0,4194304] 0 2024-02-16T19:29:05.079 INFO:tasks.workunit.client.1.smithi117.stdout:3/508: dread d1/d1b/d1e/d37/f79 [0,4194304] 0 2024-02-16T19:29:05.085 INFO:tasks.workunit.client.1.smithi117.stdout:7/394: chown d4/d1c/d87/d89/d60/l80 118813 1 2024-02-16T19:29:05.086 INFO:tasks.workunit.client.0.smithi022.stdout:4/413: dread d4/d6/dd/df/d10/f35 [0,4194304] 0 2024-02-16T19:29:05.086 INFO:tasks.workunit.client.1.smithi117.stdout:3/509: dwrite d1/d31/f7e [0,4194304] 0 2024-02-16T19:29:05.086 INFO:tasks.workunit.client.1.smithi117.stdout:7/395: stat d4/d1c/d39/d46 0 2024-02-16T19:29:05.087 INFO:tasks.workunit.client.0.smithi022.stdout:5/433: symlink dd/d1f/d48/l8d 0 2024-02-16T19:29:05.088 INFO:tasks.workunit.client.1.smithi117.stdout:6/413: truncate d5/db/f4a 2257055 0 2024-02-16T19:29:05.090 INFO:tasks.workunit.client.0.smithi022.stdout:9/470: mknod db/d14/d16/c9d 0 2024-02-16T19:29:05.092 INFO:tasks.workunit.client.1.smithi117.stdout:6/414: read d5/db/f6f [475830,61794] 0 2024-02-16T19:29:05.093 INFO:tasks.workunit.client.0.smithi022.stdout:9/471: readlink db/d14/d16/d5d/d65/d6b/d89/l8a 0 2024-02-16T19:29:05.094 INFO:tasks.workunit.client.1.smithi117.stdout:2/384: rename d0/d3/d40/c6a to d0/d3/d34/c77 0 2024-02-16T19:29:05.096 INFO:tasks.workunit.client.1.smithi117.stdout:3/510: dwrite d1/d30/f90 [0,4194304] 0 2024-02-16T19:29:05.096 INFO:tasks.workunit.client.1.smithi117.stdout:5/512: write d2/d3/f35 [1119547,108377] 0 2024-02-16T19:29:05.099 INFO:tasks.workunit.client.0.smithi022.stdout:5/434: dwrite dd/d1f/d48/f74 [0,4194304] 0 2024-02-16T19:29:05.100 INFO:tasks.workunit.client.0.smithi022.stdout:1/564: dwrite d6/f8e [0,4194304] 0 2024-02-16T19:29:05.111 INFO:tasks.workunit.client.0.smithi022.stdout:2/375: rename d0/d32/l41 to d0/d2/d18/d38/d6b/l75 0 2024-02-16T19:29:05.112 INFO:tasks.workunit.client.0.smithi022.stdout:2/376: truncate d0/d32/f58 1530018 0 2024-02-16T19:29:05.117 INFO:tasks.workunit.client.0.smithi022.stdout:5/435: read dd/d1f/d48/d21/f24 [2675565,96473] 0 2024-02-16T19:29:05.117 INFO:tasks.workunit.client.1.smithi117.stdout:4/415: fdatasync d0/da/d21/d3d/d28/d64/f23 0 2024-02-16T19:29:05.118 INFO:tasks.workunit.client.0.smithi022.stdout:5/436: readlink dd/d1f/d48/d21/l8c 0 2024-02-16T19:29:05.122 INFO:tasks.workunit.client.1.smithi117.stdout:5/513: dread d2/d3/d21/d2f/d30/d38/d51/f7e [0,4194304] 0 2024-02-16T19:29:05.126 INFO:tasks.workunit.client.1.smithi117.stdout:0/478: creat d0/d9/d14/d1d/d60/d65/fab x:0 0 0 2024-02-16T19:29:05.128 INFO:tasks.workunit.client.0.smithi022.stdout:9/472: write db/d18/f74 [415654,62223] 0 2024-02-16T19:29:05.133 INFO:tasks.workunit.client.0.smithi022.stdout:9/473: dread db/d14/d16/d27/f78 [0,4194304] 0 2024-02-16T19:29:05.134 INFO:tasks.workunit.client.1.smithi117.stdout:9/349: creat dd/d47/f79 x:0 0 0 2024-02-16T19:29:05.135 INFO:tasks.workunit.client.1.smithi117.stdout:9/350: stat dd/d23/f68 0 2024-02-16T19:29:05.135 INFO:tasks.workunit.client.1.smithi117.stdout:6/415: rmdir d5/d4b 39 2024-02-16T19:29:05.136 INFO:tasks.workunit.client.1.smithi117.stdout:7/396: dwrite d4/d1c/d39/f5a [0,4194304] 0 2024-02-16T19:29:05.136 INFO:tasks.workunit.client.1.smithi117.stdout:6/416: chown d5/db/d7d/d32/c80 1538733 1 2024-02-16T19:29:05.137 INFO:tasks.workunit.client.0.smithi022.stdout:8/431: symlink d3/d12/d17/d64/la1 0 2024-02-16T19:29:05.140 INFO:tasks.workunit.client.1.smithi117.stdout:2/385: rmdir d0/d13/d4d 39 2024-02-16T19:29:05.142 INFO:tasks.workunit.client.0.smithi022.stdout:1/565: symlink d6/d66/lab 0 2024-02-16T19:29:05.144 INFO:tasks.workunit.client.1.smithi117.stdout:8/424: dwrite d4/d12/f42 [0,4194304] 0 2024-02-16T19:29:05.144 INFO:tasks.workunit.client.1.smithi117.stdout:3/511: mkdir d1/d1b/d1e/d29/daf 0 2024-02-16T19:29:05.145 INFO:tasks.workunit.client.1.smithi117.stdout:7/397: dread d4/d1c/f26 [0,4194304] 0 2024-02-16T19:29:05.146 INFO:tasks.workunit.client.0.smithi022.stdout:0/554: symlink d10/d15/d23/lb6 0 2024-02-16T19:29:05.147 INFO:tasks.workunit.client.1.smithi117.stdout:7/398: truncate d4/d1c/d87/d89/d60/d79/f7e 4469910 0 2024-02-16T19:29:05.154 INFO:tasks.workunit.client.1.smithi117.stdout:1/378: creat d2/d1d/d60/d20/d7b/f7e x:0 0 0 2024-02-16T19:29:05.154 INFO:tasks.workunit.client.1.smithi117.stdout:1/379: fsync d2/f5b 0 2024-02-16T19:29:05.155 INFO:tasks.workunit.client.1.smithi117.stdout:1/380: fdatasync d2/d4/f47 0 2024-02-16T19:29:05.155 INFO:tasks.workunit.client.1.smithi117.stdout:1/381: read - d2/d1d/f4f zero size 2024-02-16T19:29:05.159 INFO:tasks.workunit.client.0.smithi022.stdout:9/474: symlink db/d14/d33/l9e 0 2024-02-16T19:29:05.160 INFO:tasks.workunit.client.1.smithi117.stdout:4/416: mkdir d0/da/d21/d3d/d28/d64/d8a 0 2024-02-16T19:29:05.161 INFO:tasks.workunit.client.1.smithi117.stdout:4/417: write d0/da/d21/d3d/d28/f4e [818429,32419] 0 2024-02-16T19:29:05.166 INFO:tasks.workunit.client.0.smithi022.stdout:0/555: symlink d10/d15/d23/d8f/lb7 0 2024-02-16T19:29:05.169 INFO:tasks.workunit.client.0.smithi022.stdout:3/407: sync 2024-02-16T19:29:05.170 INFO:tasks.workunit.client.0.smithi022.stdout:3/408: write f5 [294092,55519] 0 2024-02-16T19:29:05.171 INFO:tasks.workunit.client.1.smithi117.stdout:9/351: mknod dd/d47/d5e/c7a 0 2024-02-16T19:29:05.172 INFO:tasks.workunit.client.0.smithi022.stdout:5/437: creat dd/d1f/d48/d21/d2f/d46/d5e/d84/d8b/f8e x:0 0 0 2024-02-16T19:29:05.172 INFO:tasks.workunit.client.1.smithi117.stdout:0/479: dread d0/d9/d22/d27/f5f [0,4194304] 0 2024-02-16T19:29:05.173 INFO:tasks.workunit.client.0.smithi022.stdout:8/432: write d3/d12/d2b/d54/f70 [664381,48091] 0 2024-02-16T19:29:05.179 INFO:tasks.workunit.client.1.smithi117.stdout:8/425: unlink d4/d1c/f1f 0 2024-02-16T19:29:05.180 INFO:tasks.workunit.client.0.smithi022.stdout:1/566: dwrite d6/d8/d22/f3d [0,4194304] 0 2024-02-16T19:29:05.181 INFO:tasks.workunit.client.0.smithi022.stdout:0/556: stat d10/d11/d50/d65/c83 0 2024-02-16T19:29:05.184 INFO:tasks.workunit.client.1.smithi117.stdout:7/399: rename d4/d1c to d4/d1c/d87/d89/d9a 22 2024-02-16T19:29:05.187 INFO:tasks.workunit.client.0.smithi022.stdout:3/409: unlink d7/d12/d18/c9b 0 2024-02-16T19:29:05.188 INFO:tasks.workunit.client.0.smithi022.stdout:3/410: chown d7/d12/d18/dd/d31/c37 65 1 2024-02-16T19:29:05.189 INFO:tasks.workunit.client.0.smithi022.stdout:3/411: write d7/f1e [2783461,25581] 0 2024-02-16T19:29:05.190 INFO:tasks.workunit.client.0.smithi022.stdout:5/438: mkdir dd/d1f/d48/d21/d2f/d52/d5c/d61/d8f 0 2024-02-16T19:29:05.192 INFO:tasks.workunit.client.0.smithi022.stdout:8/433: mkdir d3/d12/d2b/d54/d74/da2 0 2024-02-16T19:29:05.196 INFO:tasks.workunit.client.0.smithi022.stdout:3/412: dwrite d7/d12/d18/dd/d24/f63 [0,4194304] 0 2024-02-16T19:29:05.201 INFO:tasks.workunit.client.1.smithi117.stdout:5/514: creat d2/d5d/d8e/fa7 x:0 0 0 2024-02-16T19:29:05.205 INFO:tasks.workunit.client.0.smithi022.stdout:6/449: sync 2024-02-16T19:29:05.205 INFO:tasks.workunit.client.0.smithi022.stdout:7/457: sync 2024-02-16T19:29:05.207 INFO:tasks.workunit.client.0.smithi022.stdout:6/450: write d1/d8/da/d17/d85/d5b/f63 [2550603,13207] 0 2024-02-16T19:29:05.208 INFO:tasks.workunit.client.0.smithi022.stdout:4/414: rename d4/d6/dd/df/d10/d24/c62 to d4/d6/dd/df/c80 0 2024-02-16T19:29:05.210 INFO:tasks.workunit.client.0.smithi022.stdout:6/451: chown d1/d8/da/d17/f96 9103 1 2024-02-16T19:29:05.212 INFO:tasks.workunit.client.0.smithi022.stdout:0/557: creat d10/d11/d50/d65/fb8 x:0 0 0 2024-02-16T19:29:05.213 INFO:tasks.workunit.client.0.smithi022.stdout:0/558: stat d10/l32 0 2024-02-16T19:29:05.213 INFO:tasks.workunit.client.0.smithi022.stdout:9/475: write db/d14/f2d [1268519,103499] 0 2024-02-16T19:29:05.215 INFO:tasks.workunit.client.0.smithi022.stdout:1/567: write d6/d8/d22/d47/d54/f79 [3550312,22807] 0 2024-02-16T19:29:05.215 INFO:tasks.workunit.client.0.smithi022.stdout:2/377: getdents d0/d32/d3b/d6f 0 2024-02-16T19:29:05.216 INFO:tasks.workunit.client.1.smithi117.stdout:0/480: fsync d0/fd 0 2024-02-16T19:29:05.216 INFO:tasks.workunit.client.0.smithi022.stdout:7/458: dwrite d9/da/d2c/d6d/f7e [0,4194304] 0 2024-02-16T19:29:05.217 INFO:tasks.workunit.client.1.smithi117.stdout:0/481: chown d0/d9/d14/d23 115964 1 2024-02-16T19:29:05.219 INFO:tasks.workunit.client.0.smithi022.stdout:7/459: stat d9/da/d1f/d67/l86 0 2024-02-16T19:29:05.219 INFO:tasks.workunit.client.0.smithi022.stdout:2/378: chown d0/d2/d18/d1f/c53 2437171 1 2024-02-16T19:29:05.225 INFO:tasks.workunit.client.1.smithi117.stdout:6/417: rename d5/db/d7d/c54 to d5/d1d/d49/c8d 0 2024-02-16T19:29:05.226 INFO:tasks.workunit.client.0.smithi022.stdout:7/460: dwrite d9/da/d1f/f22 [4194304,4194304] 0 2024-02-16T19:29:05.228 INFO:tasks.workunit.client.1.smithi117.stdout:7/400: rmdir d4/d1c/d39/d46 39 2024-02-16T19:29:05.231 INFO:tasks.workunit.client.1.smithi117.stdout:1/382: mkdir d2/d1d/d60/d20/d7d/d7f 0 2024-02-16T19:29:05.232 INFO:tasks.workunit.client.1.smithi117.stdout:1/383: readlink d2/d4/l3b 0 2024-02-16T19:29:05.239 INFO:tasks.workunit.client.0.smithi022.stdout:7/461: dwrite d9/da/d1f/d40/f5e [0,4194304] 0 2024-02-16T19:29:05.239 INFO:tasks.workunit.client.0.smithi022.stdout:1/568: fsync d6/f9 0 2024-02-16T19:29:05.239 INFO:tasks.workunit.client.1.smithi117.stdout:2/386: creat d0/d13/d4d/d53/d58/f78 x:0 0 0 2024-02-16T19:29:05.239 INFO:tasks.workunit.client.1.smithi117.stdout:2/387: stat d0/d29/d56/d60/l67 0 2024-02-16T19:29:05.240 INFO:tasks.workunit.client.1.smithi117.stdout:2/388: chown d0/d29/c32 63251 1 2024-02-16T19:29:05.242 INFO:tasks.workunit.client.0.smithi022.stdout:0/559: rename d10/d11/d27/d3f/d58/d60/d62/f86 to d10/d11/d79/d8b/fb9 0 2024-02-16T19:29:05.245 INFO:tasks.workunit.client.0.smithi022.stdout:9/476: creat db/d14/d16/d49/f9f x:0 0 0 2024-02-16T19:29:05.248 INFO:tasks.workunit.client.0.smithi022.stdout:3/413: write d7/d12/d18/dd/d24/f3e [8478929,11335] 0 2024-02-16T19:29:05.248 INFO:tasks.workunit.client.0.smithi022.stdout:6/452: dread d1/d8/d23/f68 [0,4194304] 0 2024-02-16T19:29:05.251 INFO:tasks.workunit.client.1.smithi117.stdout:4/418: truncate d0/da/f69 2781369 0 2024-02-16T19:29:05.251 INFO:tasks.workunit.client.0.smithi022.stdout:6/453: write d1/d8/da/d17/d85/f5f [571707,114851] 0 2024-02-16T19:29:05.253 INFO:tasks.workunit.client.0.smithi022.stdout:2/379: creat d0/d2/d18/d38/d6b/f76 x:0 0 0 2024-02-16T19:29:05.254 INFO:tasks.workunit.client.0.smithi022.stdout:5/439: dread dd/d1f/d48/f47 [0,4194304] 0 2024-02-16T19:29:05.255 INFO:tasks.workunit.client.1.smithi117.stdout:6/418: chown d5/d4b/d44/d52/f5f 3 1 2024-02-16T19:29:05.255 INFO:tasks.workunit.client.1.smithi117.stdout:6/419: stat d5/d4b/d44/f56 0 2024-02-16T19:29:05.256 INFO:tasks.workunit.client.1.smithi117.stdout:6/420: stat d5/d1d/l4f 0 2024-02-16T19:29:05.256 INFO:tasks.workunit.client.1.smithi117.stdout:6/421: truncate d5/f86 573476 0 2024-02-16T19:29:05.256 INFO:tasks.workunit.client.0.smithi022.stdout:8/434: link d3/d12/d2b/d54/f70 d3/d12/d2b/fa3 0 2024-02-16T19:29:05.257 INFO:tasks.workunit.client.1.smithi117.stdout:7/401: mkdir d4/d1c/d87/d89/d60/d9b 0 2024-02-16T19:29:05.260 INFO:tasks.workunit.client.1.smithi117.stdout:1/384: rename d2/d1d/d1f/d55 to d2/d1d/d60/d20/d69/d80 0 2024-02-16T19:29:05.262 INFO:tasks.workunit.client.1.smithi117.stdout:8/426: truncate d4/d7/f3d 3640757 0 2024-02-16T19:29:05.263 INFO:tasks.workunit.client.0.smithi022.stdout:4/415: mknod d4/c81 0 2024-02-16T19:29:05.263 INFO:tasks.workunit.client.0.smithi022.stdout:4/416: readlink d4/d6/dd/df/d10/l6f 0 2024-02-16T19:29:05.264 INFO:tasks.workunit.client.1.smithi117.stdout:9/352: rmdir dd/d74 0 2024-02-16T19:29:05.264 INFO:tasks.workunit.client.0.smithi022.stdout:4/417: readlink d4/d6/d14/d64/d4e/l5d 0 2024-02-16T19:29:05.266 INFO:tasks.workunit.client.1.smithi117.stdout:1/385: dwrite d2/d1d/f4f [0,4194304] 0 2024-02-16T19:29:05.270 INFO:tasks.workunit.client.1.smithi117.stdout:2/389: creat d0/d3b/f79 x:0 0 0 2024-02-16T19:29:05.275 INFO:tasks.workunit.client.1.smithi117.stdout:0/482: link d0/d30/f66 d0/d9/d22/d27/d4e/fac 0 2024-02-16T19:29:05.277 INFO:tasks.workunit.client.1.smithi117.stdout:3/512: getdents d1/d1b/d1e/d5e 0 2024-02-16T19:29:05.277 INFO:tasks.workunit.client.1.smithi117.stdout:3/513: readlink d1/d67/l9e 0 2024-02-16T19:29:05.279 INFO:tasks.workunit.client.0.smithi022.stdout:1/569: mknod d6/d8/d22/d3a/d80/cac 0 2024-02-16T19:29:05.281 INFO:tasks.workunit.client.1.smithi117.stdout:4/419: dread d0/da/d21/d3d/d28/d64/f1f [4194304,4194304] 0 2024-02-16T19:29:05.282 INFO:tasks.workunit.client.1.smithi117.stdout:6/422: write d5/db/d7d/d32/d37/f58 [1038113,66800] 0 2024-02-16T19:29:05.284 INFO:tasks.workunit.client.0.smithi022.stdout:9/477: mknod db/d14/d16/d21/ca0 0 2024-02-16T19:29:05.285 INFO:tasks.workunit.client.1.smithi117.stdout:7/402: creat d4/d13/d63/d71/f9c x:0 0 0 2024-02-16T19:29:05.287 INFO:tasks.workunit.client.0.smithi022.stdout:6/454: mkdir d1/d8/da/d29/da1 0 2024-02-16T19:29:05.290 INFO:tasks.workunit.client.1.smithi117.stdout:5/515: link d2/d4c/d4f/f81 d2/d3/fa8 0 2024-02-16T19:29:05.290 INFO:tasks.workunit.client.1.smithi117.stdout:5/516: stat d2/d9 0 2024-02-16T19:29:05.290 INFO:tasks.workunit.client.0.smithi022.stdout:2/380: symlink d0/d2/d18/d19/d40/l77 0 2024-02-16T19:29:05.292 INFO:tasks.workunit.client.0.smithi022.stdout:5/440: readlink dd/d1f/d48/d21/d2f/d46/d5e/d84/l81 0 2024-02-16T19:29:05.293 INFO:tasks.workunit.client.1.smithi117.stdout:8/427: mkdir d4/d7/d6a/d72/d84 0 2024-02-16T19:29:05.294 INFO:tasks.workunit.client.0.smithi022.stdout:8/435: creat d3/d12/d6a/fa4 x:0 0 0 2024-02-16T19:29:05.296 INFO:tasks.workunit.client.1.smithi117.stdout:6/423: dread d5/f12 [0,4194304] 0 2024-02-16T19:29:05.301 INFO:tasks.workunit.client.1.smithi117.stdout:6/424: dwrite d5/f12 [4194304,4194304] 0 2024-02-16T19:29:05.301 INFO:tasks.workunit.client.1.smithi117.stdout:2/390: rename d0/f1e to d0/d3/d34/f7a 0 2024-02-16T19:29:05.303 INFO:tasks.workunit.client.1.smithi117.stdout:0/483: creat d0/d9/d14/d1d/d60/d65/fad x:0 0 0 2024-02-16T19:29:05.309 INFO:tasks.workunit.client.1.smithi117.stdout:4/420: rmdir d0/da/d21/d3d/d2d 39 2024-02-16T19:29:05.314 INFO:tasks.workunit.client.0.smithi022.stdout:9/478: symlink db/d14/d16/d5d/d65/d6b/la1 0 2024-02-16T19:29:05.314 INFO:tasks.workunit.client.1.smithi117.stdout:2/391: dread d0/f25 [0,4194304] 0 2024-02-16T19:29:05.315 INFO:tasks.workunit.client.1.smithi117.stdout:7/403: truncate f3 10747 0 2024-02-16T19:29:05.315 INFO:tasks.workunit.client.1.smithi117.stdout:0/484: dread d0/d9/d14/d1d/d60/d76/d7b/d8d/fa1 [0,4194304] 0 2024-02-16T19:29:05.316 INFO:tasks.workunit.client.1.smithi117.stdout:7/404: write d4/d13/f35 [3890573,112868] 0 2024-02-16T19:29:05.316 INFO:tasks.workunit.client.0.smithi022.stdout:3/414: link d7/d12/d18/f42 d7/d12/d18/dd/d31/d7f/fa0 0 2024-02-16T19:29:05.317 INFO:tasks.workunit.client.0.smithi022.stdout:4/418: write d4/d6/d14/d64/d4e/d54/d5a/f5f [1899436,128733] 0 2024-02-16T19:29:05.319 INFO:tasks.workunit.client.1.smithi117.stdout:5/517: creat d2/d5d/fa9 x:0 0 0 2024-02-16T19:29:05.320 INFO:tasks.workunit.client.1.smithi117.stdout:5/518: dread - d2/d5d/f9d zero size 2024-02-16T19:29:05.325 INFO:tasks.workunit.client.0.smithi022.stdout:5/441: symlink dd/d1f/d48/d21/d2f/d40/l90 0 2024-02-16T19:29:05.328 INFO:tasks.workunit.client.0.smithi022.stdout:1/570: dwrite d6/d8/d22/d16/f1e [4194304,4194304] 0 2024-02-16T19:29:05.335 INFO:tasks.workunit.client.1.smithi117.stdout:2/392: creat d0/d3/d22/f7b x:0 0 0 2024-02-16T19:29:05.340 INFO:tasks.workunit.client.1.smithi117.stdout:2/393: dwrite d0/d3/d10/f1d [0,4194304] 0 2024-02-16T19:29:05.342 INFO:tasks.workunit.client.1.smithi117.stdout:7/405: mknod d4/d1c/d87/d89/d60/d79/c9d 0 2024-02-16T19:29:05.345 INFO:tasks.workunit.client.1.smithi117.stdout:0/485: truncate d0/d9/d14/d1d/d1e/f21 386159 0 2024-02-16T19:29:05.346 INFO:tasks.workunit.client.1.smithi117.stdout:0/486: fsync d0/f40 0 2024-02-16T19:29:05.352 INFO:tasks.workunit.client.0.smithi022.stdout:5/442: creat dd/d1f/d48/d21/d2f/d40/f91 x:0 0 0 2024-02-16T19:29:05.354 INFO:tasks.workunit.client.0.smithi022.stdout:5/443: chown dd/d1f/d48/d21/d2f/d46/d5e/l80 1550473 1 2024-02-16T19:29:05.355 INFO:tasks.workunit.client.0.smithi022.stdout:5/444: fsync dd/d1f/d48/f1b 0 2024-02-16T19:29:05.355 INFO:tasks.workunit.client.1.smithi117.stdout:8/428: creat d4/d7/d37/d3a/d54/d70/d76/f85 x:0 0 0 2024-02-16T19:29:05.356 INFO:tasks.workunit.client.1.smithi117.stdout:9/353: creat dd/d47/f7b x:0 0 0 2024-02-16T19:29:05.358 INFO:tasks.workunit.client.0.smithi022.stdout:3/415: write d7/d21/f38 [1265663,91248] 0 2024-02-16T19:29:05.359 INFO:tasks.workunit.client.1.smithi117.stdout:3/514: rename d1/d30/d52/d56/c6c to d1/dd/d3b/d9d/cb0 0 2024-02-16T19:29:05.361 INFO:tasks.workunit.client.0.smithi022.stdout:1/571: mknod d6/d8/d22/d47/d51/cad 0 2024-02-16T19:29:05.364 INFO:tasks.workunit.client.0.smithi022.stdout:7/462: getdents d9/d1b 0 2024-02-16T19:29:05.364 INFO:tasks.workunit.client.0.smithi022.stdout:3/416: dwrite d7/d12/d18/dd/d24/f32 [0,4194304] 0 2024-02-16T19:29:05.366 INFO:tasks.workunit.client.0.smithi022.stdout:0/560: link d10/d15/d23/d38/d5a/l92 d10/d11/d79/d8b/lba 0 2024-02-16T19:29:05.366 INFO:tasks.workunit.client.0.smithi022.stdout:0/561: chown d10/d11/c70 7 1 2024-02-16T19:29:05.368 INFO:tasks.workunit.client.1.smithi117.stdout:4/421: link d0/da/d4a/f78 d0/da/d21/d3d/d28/d64/f8b 0 2024-02-16T19:29:05.368 INFO:tasks.workunit.client.0.smithi022.stdout:0/562: chown d10/d22/d49/c89 5408 1 2024-02-16T19:29:05.370 INFO:tasks.workunit.client.0.smithi022.stdout:7/463: dread d9/da/f60 [0,4194304] 0 2024-02-16T19:29:05.371 INFO:tasks.workunit.client.0.smithi022.stdout:9/479: rename db/f84 to db/d14/d19/fa2 0 2024-02-16T19:29:05.383 INFO:tasks.workunit.client.1.smithi117.stdout:7/406: mknod d4/d1c/d39/d46/d55/c9e 0 2024-02-16T19:29:05.383 INFO:tasks.workunit.client.1.smithi117.stdout:0/487: dread d0/f10 [4194304,4194304] 0 2024-02-16T19:29:05.384 INFO:tasks.workunit.client.1.smithi117.stdout:9/354: mkdir dd/d47/d5b/d5f/d77/d7c 0 2024-02-16T19:29:05.386 INFO:tasks.workunit.client.1.smithi117.stdout:3/515: fdatasync d1/dd/f17 0 2024-02-16T19:29:05.387 INFO:tasks.workunit.client.1.smithi117.stdout:2/394: creat d0/d3/f7c x:0 0 0 2024-02-16T19:29:05.387 INFO:tasks.workunit.client.1.smithi117.stdout:4/422: dread d0/da/d4a/f78 [0,4194304] 0 2024-02-16T19:29:05.389 INFO:tasks.workunit.client.1.smithi117.stdout:4/423: truncate d0/f5c 4273645 0 2024-02-16T19:29:05.390 INFO:tasks.workunit.client.1.smithi117.stdout:9/355: dwrite dd/d47/d48/f55 [0,4194304] 0 2024-02-16T19:29:05.395 INFO:tasks.workunit.client.1.smithi117.stdout:0/488: mknod d0/d30/cae 0 2024-02-16T19:29:05.397 INFO:tasks.workunit.client.1.smithi117.stdout:7/407: dread d4/d1c/d39/f54 [0,4194304] 0 2024-02-16T19:29:05.403 INFO:tasks.workunit.client.0.smithi022.stdout:6/455: truncate d1/d8/da/d17/f2d 3707055 0 2024-02-16T19:29:05.405 INFO:tasks.workunit.client.0.smithi022.stdout:2/381: creat d0/f78 x:0 0 0 2024-02-16T19:29:05.406 INFO:tasks.workunit.client.1.smithi117.stdout:8/429: getdents d4/d7/d37/d3a/d54 0 2024-02-16T19:29:05.407 INFO:tasks.workunit.client.1.smithi117.stdout:3/516: dwrite d1/d1b/d20/fa2 [0,4194304] 0 2024-02-16T19:29:05.408 INFO:tasks.workunit.client.1.smithi117.stdout:3/517: dread - d1/d1b/f21 zero size 2024-02-16T19:29:05.410 INFO:tasks.workunit.client.1.smithi117.stdout:4/424: creat d0/da/d21/f8c x:0 0 0 2024-02-16T19:29:05.412 INFO:tasks.workunit.client.1.smithi117.stdout:9/356: mknod dd/d45/d31/c7d 0 2024-02-16T19:29:05.414 INFO:tasks.workunit.client.1.smithi117.stdout:9/357: read - dd/d47/d5e/f6e zero size 2024-02-16T19:29:05.415 INFO:tasks.workunit.client.0.smithi022.stdout:1/572: rmdir d6/d8/d6c 39 2024-02-16T19:29:05.416 INFO:tasks.workunit.client.1.smithi117.stdout:8/430: dwrite d4/d7/d37/d3a/d65/d6c/f75 [0,4194304] 0 2024-02-16T19:29:05.418 INFO:tasks.workunit.client.0.smithi022.stdout:3/417: mknod d7/d12/d18/dd/d31/ca1 0 2024-02-16T19:29:05.422 INFO:tasks.workunit.client.1.smithi117.stdout:8/431: dwrite d4/d7/d37/d3a/d43/f6b [0,4194304] 0 2024-02-16T19:29:05.422 INFO:tasks.workunit.client.0.smithi022.stdout:0/563: creat d10/d11/d27/fbb x:0 0 0 2024-02-16T19:29:05.423 INFO:tasks.workunit.client.0.smithi022.stdout:1/573: dwrite d6/d8/d22/d3a/d3e/f76 [0,4194304] 0 2024-02-16T19:29:05.423 INFO:tasks.workunit.client.1.smithi117.stdout:2/395: truncate d0/d3/d34/f7a 803416 0 2024-02-16T19:29:05.426 INFO:tasks.workunit.client.0.smithi022.stdout:7/464: creat d9/d1b/d23/f96 x:0 0 0 2024-02-16T19:29:05.431 INFO:tasks.workunit.client.1.smithi117.stdout:4/425: symlink d0/da/d21/d3d/d28/l8d 0 2024-02-16T19:29:05.433 INFO:tasks.workunit.client.0.smithi022.stdout:5/445: rename f3 to dd/d1f/d48/d21/d2f/d52/d5c/d61/f92 0 2024-02-16T19:29:05.434 INFO:tasks.workunit.client.1.smithi117.stdout:4/426: chown d0/da/d21/f2b 0 1 2024-02-16T19:29:05.434 INFO:tasks.workunit.client.1.smithi117.stdout:2/396: dwrite d0/d3/d22/f5a [0,4194304] 0 2024-02-16T19:29:05.435 INFO:tasks.workunit.client.0.smithi022.stdout:4/419: write d4/f40 [528262,28871] 0 2024-02-16T19:29:05.435 INFO:tasks.workunit.client.1.smithi117.stdout:4/427: chown d0/da/d21/f3f 921 1 2024-02-16T19:29:05.436 INFO:tasks.workunit.client.0.smithi022.stdout:4/420: dread - d4/d6/d14/d64/d66/d6d/f7d zero size 2024-02-16T19:29:05.438 INFO:tasks.workunit.client.0.smithi022.stdout:4/421: write d4/d6/dd/df/d10/d55/f7b [284107,93327] 0 2024-02-16T19:29:05.439 INFO:tasks.workunit.client.1.smithi117.stdout:3/518: dread d1/d1b/d1e/f51 [0,4194304] 0 2024-02-16T19:29:05.441 INFO:tasks.workunit.client.1.smithi117.stdout:6/425: sync 2024-02-16T19:29:05.442 INFO:tasks.workunit.client.1.smithi117.stdout:5/519: sync 2024-02-16T19:29:05.442 INFO:tasks.workunit.client.1.smithi117.stdout:1/386: sync 2024-02-16T19:29:05.443 INFO:tasks.workunit.client.1.smithi117.stdout:6/426: chown d5/d4b/d20/d69/l6e 2 1 2024-02-16T19:29:05.443 INFO:tasks.workunit.client.1.smithi117.stdout:7/408: mkdir d4/d1c/d39/d9f 0 2024-02-16T19:29:05.446 INFO:tasks.workunit.client.0.smithi022.stdout:8/436: creat d3/d12/d2b/fa5 x:0 0 0 2024-02-16T19:29:05.448 INFO:tasks.workunit.client.1.smithi117.stdout:8/432: rename d4/d12/l13 to d4/d7/d37/d3a/d54/d70/l86 0 2024-02-16T19:29:05.452 INFO:tasks.workunit.client.1.smithi117.stdout:2/397: unlink d0/d3/d10/c5c 0 2024-02-16T19:29:05.457 INFO:tasks.workunit.client.1.smithi117.stdout:0/489: link d0/d9/d14/d1d/f88 d0/d9/d22/d27/d83/faf 0 2024-02-16T19:29:05.459 INFO:tasks.workunit.client.1.smithi117.stdout:2/398: dwrite d0/d3b/f5b [0,4194304] 0 2024-02-16T19:29:05.459 INFO:tasks.workunit.client.1.smithi117.stdout:2/399: chown d0/d3/d22/d37/f63 0 1 2024-02-16T19:29:05.460 INFO:tasks.workunit.client.1.smithi117.stdout:3/519: symlink d1/d30/lb1 0 2024-02-16T19:29:05.463 INFO:tasks.workunit.client.1.smithi117.stdout:1/387: creat d2/d1d/d60/d68/f81 x:0 0 0 2024-02-16T19:29:05.466 INFO:tasks.workunit.client.1.smithi117.stdout:7/409: symlink d4/d1c/d39/d46/d47/la0 0 2024-02-16T19:29:05.467 INFO:tasks.workunit.client.0.smithi022.stdout:5/446: unlink dd/f18 0 2024-02-16T19:29:05.469 INFO:tasks.workunit.client.1.smithi117.stdout:1/388: dwrite d2/d1d/d60/d20/d7b/f7e [0,4194304] 0 2024-02-16T19:29:05.469 INFO:tasks.workunit.client.1.smithi117.stdout:4/428: rename d0/da/l2e to d0/da/d21/d3d/d28/d64/d48/l8e 0 2024-02-16T19:29:05.470 INFO:tasks.workunit.client.0.smithi022.stdout:6/456: mknod d1/d8/da/d17/ca2 0 2024-02-16T19:29:05.471 INFO:tasks.workunit.client.1.smithi117.stdout:1/389: fdatasync d2/d1d/f39 0 2024-02-16T19:29:05.471 INFO:tasks.workunit.client.0.smithi022.stdout:6/457: dread - d1/d8/da/d17/d53/f91 zero size 2024-02-16T19:29:05.472 INFO:tasks.workunit.client.0.smithi022.stdout:6/458: readlink d1/d8/da/d17/d53/l9f 0 2024-02-16T19:29:05.472 INFO:tasks.workunit.client.1.smithi117.stdout:8/433: write d4/d7/d37/d50/f5b [1761066,125055] 0 2024-02-16T19:29:05.473 INFO:tasks.workunit.client.0.smithi022.stdout:4/422: creat d4/d6/d14/d64/d66/f82 x:0 0 0 2024-02-16T19:29:05.475 INFO:tasks.workunit.client.0.smithi022.stdout:0/564: write d10/d22/f53 [1948774,63926] 0 2024-02-16T19:29:05.475 INFO:tasks.workunit.client.0.smithi022.stdout:2/382: getdents d0/d5d 0 2024-02-16T19:29:05.476 INFO:tasks.workunit.client.0.smithi022.stdout:0/565: readlink d10/d11/d50/d65/lb5 0 2024-02-16T19:29:05.477 INFO:tasks.workunit.client.1.smithi117.stdout:0/490: mknod d0/cb0 0 2024-02-16T19:29:05.479 INFO:tasks.workunit.client.0.smithi022.stdout:8/437: rename d3/d12/d6a/d98 to d3/d12/da6 0 2024-02-16T19:29:05.480 INFO:tasks.workunit.client.1.smithi117.stdout:9/358: creat dd/d47/f7e x:0 0 0 2024-02-16T19:29:05.483 INFO:tasks.workunit.client.1.smithi117.stdout:3/520: symlink d1/d1b/d1e/lb2 0 2024-02-16T19:29:05.483 INFO:tasks.workunit.client.1.smithi117.stdout:3/521: readlink d1/dd/l61 0 2024-02-16T19:29:05.484 INFO:tasks.workunit.client.0.smithi022.stdout:1/574: mknod d6/d8/d22/d47/d51/d8d/cae 0 2024-02-16T19:29:05.486 INFO:tasks.workunit.client.1.smithi117.stdout:7/410: creat d4/d13/fa1 x:0 0 0 2024-02-16T19:29:05.486 INFO:tasks.workunit.client.1.smithi117.stdout:3/522: dread d1/d1b/d1e/d5e/f66 [0,4194304] 0 2024-02-16T19:29:05.487 INFO:tasks.workunit.client.1.smithi117.stdout:3/523: write d1/d83/d96/fa0 [1095172,97512] 0 2024-02-16T19:29:05.491 INFO:tasks.workunit.client.1.smithi117.stdout:8/434: creat d4/d7/d37/d3a/d54/d70/f87 x:0 0 0 2024-02-16T19:29:05.493 INFO:tasks.workunit.client.0.smithi022.stdout:4/423: mkdir d4/d6/dd/d22/d83 0 2024-02-16T19:29:05.494 INFO:tasks.workunit.client.0.smithi022.stdout:6/459: read d1/f32 [2075452,116568] 0 2024-02-16T19:29:05.495 INFO:tasks.workunit.client.1.smithi117.stdout:0/491: creat d0/d9/d14/d1d/d60/d76/d7b/fb1 x:0 0 0 2024-02-16T19:29:05.499 INFO:tasks.workunit.client.1.smithi117.stdout:2/400: truncate d0/d29/f73 1311049 0 2024-02-16T19:29:05.499 INFO:tasks.workunit.client.1.smithi117.stdout:4/429: dread d0/da/d21/d3d/d28/f4e [0,4194304] 0 2024-02-16T19:29:05.500 INFO:tasks.workunit.client.0.smithi022.stdout:8/438: rmdir d3/d12/d17/d64/d8f 39 2024-02-16T19:29:05.500 INFO:tasks.workunit.client.1.smithi117.stdout:9/359: dread dd/d45/d62/f32 [0,4194304] 0 2024-02-16T19:29:05.504 INFO:tasks.workunit.client.0.smithi022.stdout:1/575: read d6/f55 [53916,69791] 0 2024-02-16T19:29:05.505 INFO:tasks.workunit.client.1.smithi117.stdout:7/411: dread d4/d1c/f2c [0,4194304] 0 2024-02-16T19:29:05.505 INFO:tasks.workunit.client.0.smithi022.stdout:1/576: read d6/d8/d22/d67/d7e/fa4 [684400,53822] 0 2024-02-16T19:29:05.508 INFO:tasks.workunit.client.0.smithi022.stdout:5/447: symlink dd/d1f/d4a/d7a/l93 0 2024-02-16T19:29:05.508 INFO:tasks.workunit.client.0.smithi022.stdout:5/448: stat dd/d1f/d48/d21/d2f/d46 0 2024-02-16T19:29:05.509 INFO:tasks.workunit.client.1.smithi117.stdout:3/524: stat d1/l10 0 2024-02-16T19:29:05.509 INFO:tasks.workunit.client.1.smithi117.stdout:3/525: chown d1/dd/d5c/fa1 37581 1 2024-02-16T19:29:05.512 INFO:tasks.workunit.client.0.smithi022.stdout:6/460: truncate d1/f32 2738857 0 2024-02-16T19:29:05.513 INFO:tasks.workunit.client.1.smithi117.stdout:1/390: truncate d2/d1d/d1f/f56 1079332 0 2024-02-16T19:29:05.514 INFO:tasks.workunit.client.0.smithi022.stdout:7/465: truncate d9/da/d2c/d2f/f35 2090495 0 2024-02-16T19:29:05.515 INFO:tasks.workunit.client.0.smithi022.stdout:3/418: dwrite d7/d12/d18/dd/d3c/f65 [0,4194304] 0 2024-02-16T19:29:05.516 INFO:tasks.workunit.client.1.smithi117.stdout:8/435: rmdir d4/d7/d6a/d72 39 2024-02-16T19:29:05.516 INFO:tasks.workunit.client.0.smithi022.stdout:3/419: dread - d7/d12/d18/dd/d24/f9f zero size 2024-02-16T19:29:05.517 INFO:tasks.workunit.client.0.smithi022.stdout:3/420: stat d7/d12/d18/f1a 0 2024-02-16T19:29:05.518 INFO:tasks.workunit.client.0.smithi022.stdout:0/566: mkdir d10/d11/d27/d3f/d58/d60/dbc 0 2024-02-16T19:29:05.521 INFO:tasks.workunit.client.1.smithi117.stdout:0/492: read d0/d9/d22/d27/d83/f50 [80945,120804] 0 2024-02-16T19:29:05.523 INFO:tasks.workunit.client.1.smithi117.stdout:8/436: dwrite d4/d12/f26 [4194304,4194304] 0 2024-02-16T19:29:05.525 INFO:tasks.workunit.client.0.smithi022.stdout:6/461: dread d1/f2 [0,4194304] 0 2024-02-16T19:29:05.529 INFO:tasks.workunit.client.1.smithi117.stdout:4/430: creat d0/da/d21/d3d/d28/d2f/f8f x:0 0 0 2024-02-16T19:29:05.530 INFO:tasks.workunit.client.0.smithi022.stdout:2/383: write d0/d2/f14 [1648220,29940] 0 2024-02-16T19:29:05.533 INFO:tasks.workunit.client.0.smithi022.stdout:4/424: fsync d4/f47 0 2024-02-16T19:29:05.535 INFO:tasks.workunit.client.0.smithi022.stdout:9/480: truncate db/d14/f60 6195391 0 2024-02-16T19:29:05.536 INFO:tasks.workunit.client.1.smithi117.stdout:7/412: creat d4/d13/d63/fa2 x:0 0 0 2024-02-16T19:29:05.537 INFO:tasks.workunit.client.1.smithi117.stdout:3/526: mknod d1/cb3 0 2024-02-16T19:29:05.542 INFO:tasks.workunit.client.1.smithi117.stdout:1/391: unlink d2/le 0 2024-02-16T19:29:05.545 INFO:tasks.workunit.client.0.smithi022.stdout:7/466: unlink c7 0 2024-02-16T19:29:05.547 INFO:tasks.workunit.client.0.smithi022.stdout:3/421: creat d7/d12/d18/dd/d3c/fa2 x:0 0 0 2024-02-16T19:29:05.548 INFO:tasks.workunit.client.1.smithi117.stdout:1/392: dwrite d2/d1d/f39 [0,4194304] 0 2024-02-16T19:29:05.549 INFO:tasks.workunit.client.1.smithi117.stdout:0/493: creat d0/d9/d22/d27/d31/fb2 x:0 0 0 2024-02-16T19:29:05.551 INFO:tasks.workunit.client.1.smithi117.stdout:8/437: write d4/d12/d2a/f48 [78939,5201] 0 2024-02-16T19:29:05.553 INFO:tasks.workunit.client.1.smithi117.stdout:2/401: symlink d0/d13/l7d 0 2024-02-16T19:29:05.558 INFO:tasks.workunit.client.1.smithi117.stdout:4/431: mkdir d0/da/d21/d3d/d28/d64/d90 0 2024-02-16T19:29:05.560 INFO:tasks.workunit.client.0.smithi022.stdout:6/462: mkdir d1/d8/da/d17/d53/d7f/da3 0 2024-02-16T19:29:05.561 INFO:tasks.workunit.client.0.smithi022.stdout:2/384: mknod d0/d32/d3b/d6f/c79 0 2024-02-16T19:29:05.563 INFO:tasks.workunit.client.1.smithi117.stdout:3/527: mknod d1/cb4 0 2024-02-16T19:29:05.563 INFO:tasks.workunit.client.1.smithi117.stdout:3/528: rename d1/dd to d1/dd/d3b/db5 22 2024-02-16T19:29:05.566 INFO:tasks.workunit.client.0.smithi022.stdout:4/425: symlink d4/d6/d14/d64/d66/d6d/l84 0 2024-02-16T19:29:05.569 INFO:tasks.workunit.client.0.smithi022.stdout:9/481: creat db/d14/d16/d27/fa3 x:0 0 0 2024-02-16T19:29:05.571 INFO:tasks.workunit.client.0.smithi022.stdout:5/449: creat dd/d1f/d48/d21/d2f/d52/d5c/d61/d8f/f94 x:0 0 0 2024-02-16T19:29:05.572 INFO:tasks.workunit.client.1.smithi117.stdout:2/402: mknod d0/d13/d4d/d64/d46/c7e 0 2024-02-16T19:29:05.576 INFO:tasks.workunit.client.0.smithi022.stdout:0/567: write d10/d11/d27/d77/f81 [987729,1973] 0 2024-02-16T19:29:05.576 INFO:tasks.workunit.client.0.smithi022.stdout:7/467: creat d9/d5d/f97 x:0 0 0 2024-02-16T19:29:05.578 INFO:tasks.workunit.client.0.smithi022.stdout:0/568: truncate d10/d11/d79/d8b/fa5 784086 0 2024-02-16T19:29:05.578 INFO:tasks.workunit.client.1.smithi117.stdout:3/529: creat d1/d83/d96/fb6 x:0 0 0 2024-02-16T19:29:05.583 INFO:tasks.workunit.client.1.smithi117.stdout:1/393: write d2/d1d/f25 [1250837,96372] 0 2024-02-16T19:29:05.583 INFO:tasks.workunit.client.0.smithi022.stdout:6/463: creat d1/d8/da/d17/d85/d5b/fa4 x:0 0 0 2024-02-16T19:29:05.586 INFO:tasks.workunit.client.1.smithi117.stdout:3/530: symlink d1/d30/d52/d56/lb7 0 2024-02-16T19:29:05.588 INFO:tasks.workunit.client.1.smithi117.stdout:0/494: creat d0/d9/d14/d23/d33/fb3 x:0 0 0 2024-02-16T19:29:05.589 INFO:tasks.workunit.client.1.smithi117.stdout:0/495: fdatasync d0/d9/d14/d1d/d1e/d54/f72 0 2024-02-16T19:29:05.590 INFO:tasks.workunit.client.1.smithi117.stdout:8/438: creat d4/d7/f88 x:0 0 0 2024-02-16T19:29:05.591 INFO:tasks.workunit.client.1.smithi117.stdout:8/439: write d4/d73/f82 [706396,123316] 0 2024-02-16T19:29:05.591 INFO:tasks.workunit.client.0.smithi022.stdout:9/482: creat db/d14/d16/d21/d4a/fa4 x:0 0 0 2024-02-16T19:29:05.593 INFO:tasks.workunit.client.1.smithi117.stdout:1/394: symlink d2/d1d/l82 0 2024-02-16T19:29:05.594 INFO:tasks.workunit.client.1.smithi117.stdout:0/496: dwrite d0/fe [0,4194304] 0 2024-02-16T19:29:05.595 INFO:tasks.workunit.client.0.smithi022.stdout:7/468: unlink d9/c45 0 2024-02-16T19:29:05.596 INFO:tasks.workunit.client.0.smithi022.stdout:6/464: dread d1/d8/da/f14 [0,4194304] 0 2024-02-16T19:29:05.600 INFO:tasks.workunit.client.0.smithi022.stdout:8/439: link d3/d12/da6/f8c d3/d12/d2b/d54/d74/fa7 0 2024-02-16T19:29:05.602 INFO:tasks.workunit.client.0.smithi022.stdout:4/426: dread d4/d6/d14/f28 [0,4194304] 0 2024-02-16T19:29:05.603 INFO:tasks.workunit.client.0.smithi022.stdout:2/385: creat d0/d5d/f7a x:0 0 0 2024-02-16T19:29:05.605 INFO:tasks.workunit.client.0.smithi022.stdout:1/577: link d6/d8/d22/d16/d17/c45 d6/d8/d22/d3a/d7a/caf 0 2024-02-16T19:29:05.606 INFO:tasks.workunit.client.0.smithi022.stdout:1/578: stat d6/d8/d22/d47/f4e 0 2024-02-16T19:29:05.606 INFO:tasks.workunit.client.0.smithi022.stdout:1/579: stat d6/d66 0 2024-02-16T19:29:05.606 INFO:tasks.workunit.client.0.smithi022.stdout:5/450: mkdir dd/d1f/d48/d21/d2f/d52/d77/d95 0 2024-02-16T19:29:05.609 INFO:tasks.workunit.client.0.smithi022.stdout:5/451: chown dd/d1f/d48/d21/d2f/d52/f65 277116247 1 2024-02-16T19:29:05.611 INFO:tasks.workunit.client.1.smithi117.stdout:2/403: getdents d0/d29/d56 0 2024-02-16T19:29:05.614 INFO:tasks.workunit.client.0.smithi022.stdout:3/422: creat d7/d12/d18/dd/d24/d29/fa3 x:0 0 0 2024-02-16T19:29:05.614 INFO:tasks.workunit.client.0.smithi022.stdout:5/452: dread dd/d1f/d48/d21/d2f/d40/d4e/d56/f59 [0,4194304] 0 2024-02-16T19:29:05.616 INFO:tasks.workunit.client.1.smithi117.stdout:8/440: symlink d4/l89 0 2024-02-16T19:29:05.616 INFO:tasks.workunit.client.1.smithi117.stdout:8/441: readlink d4/d12/d2a/l3b 0 2024-02-16T19:29:05.618 INFO:tasks.workunit.client.1.smithi117.stdout:4/432: getdents d0/da/d21/d3d/d2d 0 2024-02-16T19:29:05.620 INFO:tasks.workunit.client.0.smithi022.stdout:0/569: creat d10/d11/d27/d3f/d58/d60/dbc/fbd x:0 0 0 2024-02-16T19:29:05.621 INFO:tasks.workunit.client.0.smithi022.stdout:0/570: write d10/d22/d2c/f72 [2156914,100165] 0 2024-02-16T19:29:05.622 INFO:tasks.workunit.client.1.smithi117.stdout:8/442: dwrite d4/d7/d37/f56 [0,4194304] 0 2024-02-16T19:29:05.622 INFO:tasks.workunit.client.0.smithi022.stdout:6/465: creat d1/d8/da/d17/d85/d56/fa5 x:0 0 0 2024-02-16T19:29:05.623 INFO:tasks.workunit.client.1.smithi117.stdout:3/531: link d1/cb3 d1/d30/cb8 0 2024-02-16T19:29:05.624 INFO:tasks.workunit.client.0.smithi022.stdout:6/466: chown d1/d8/d3c/f44 0 1 2024-02-16T19:29:05.625 INFO:tasks.workunit.client.0.smithi022.stdout:8/440: creat d3/d12/d6a/fa8 x:0 0 0 2024-02-16T19:29:05.626 INFO:tasks.workunit.client.0.smithi022.stdout:8/441: chown d3/d12/c49 0 1 2024-02-16T19:29:05.627 INFO:tasks.workunit.client.1.smithi117.stdout:2/404: mknod d0/d13/d4d/d64/c7f 0 2024-02-16T19:29:05.627 INFO:tasks.workunit.client.1.smithi117.stdout:4/433: dwrite d0/da/d21/f27 [0,4194304] 0 2024-02-16T19:29:05.628 INFO:tasks.workunit.client.0.smithi022.stdout:4/427: creat d4/d6/dd/f85 x:0 0 0 2024-02-16T19:29:05.630 INFO:tasks.workunit.client.0.smithi022.stdout:9/483: dread db/d18/f73 [0,4194304] 0 2024-02-16T19:29:05.632 INFO:tasks.workunit.client.0.smithi022.stdout:2/386: creat d0/d2/d18/d1f/f7b x:0 0 0 2024-02-16T19:29:05.634 INFO:tasks.workunit.client.0.smithi022.stdout:2/387: readlink d0/d2/d18/d19/d40/l77 0 2024-02-16T19:29:05.635 INFO:tasks.workunit.client.1.smithi117.stdout:2/405: dwrite d0/d3/d22/f28 [8388608,4194304] 0 2024-02-16T19:29:05.636 INFO:tasks.workunit.client.1.smithi117.stdout:2/406: stat d0/d3/d34/f71 0 2024-02-16T19:29:05.637 INFO:tasks.workunit.client.0.smithi022.stdout:2/388: write d0/d2/d18/d1f/f21 [4045196,129832] 0 2024-02-16T19:29:05.637 INFO:tasks.workunit.client.0.smithi022.stdout:4/428: dwrite d4/f40 [0,4194304] 0 2024-02-16T19:29:05.640 INFO:tasks.workunit.client.1.smithi117.stdout:5/520: sync 2024-02-16T19:29:05.640 INFO:tasks.workunit.client.1.smithi117.stdout:6/427: sync 2024-02-16T19:29:05.642 INFO:tasks.workunit.client.1.smithi117.stdout:8/443: dread d4/d73/f82 [0,4194304] 0 2024-02-16T19:29:05.642 INFO:tasks.workunit.client.0.smithi022.stdout:7/469: symlink d9/l98 0 2024-02-16T19:29:05.643 INFO:tasks.workunit.client.1.smithi117.stdout:3/532: rmdir d1 39 2024-02-16T19:29:05.645 INFO:tasks.workunit.client.1.smithi117.stdout:8/444: stat d4/d7/d37/d3a/d54/f78 0 2024-02-16T19:29:05.645 INFO:tasks.workunit.client.1.smithi117.stdout:2/407: dwrite d0/d3b/f51 [0,4194304] 0 2024-02-16T19:29:05.646 INFO:tasks.workunit.client.0.smithi022.stdout:7/470: dread d9/da/d2c/f8d [0,4194304] 0 2024-02-16T19:29:05.649 INFO:tasks.workunit.client.0.smithi022.stdout:5/453: creat dd/d1f/d76/f96 x:0 0 0 2024-02-16T19:29:05.649 INFO:tasks.workunit.client.1.smithi117.stdout:4/434: rmdir d0/da/d21/d3d/d28/d64/d48 39 2024-02-16T19:29:05.654 INFO:tasks.workunit.client.1.smithi117.stdout:0/497: getdents d0/d9/d14/d1d/d1e 0 2024-02-16T19:29:05.655 INFO:tasks.workunit.client.0.smithi022.stdout:6/467: mknod d1/d8/da/d29/ca6 0 2024-02-16T19:29:05.659 INFO:tasks.workunit.client.0.smithi022.stdout:0/571: dread d10/d22/fa1 [0,4194304] 0 2024-02-16T19:29:05.661 INFO:tasks.workunit.client.0.smithi022.stdout:0/572: write d10/d11/d27/d3f/f5b [2267130,53997] 0 2024-02-16T19:29:05.664 INFO:tasks.workunit.client.1.smithi117.stdout:6/428: rmdir d5/d4b/d44/d52 39 2024-02-16T19:29:05.666 INFO:tasks.workunit.client.1.smithi117.stdout:6/429: stat d5/d1d/d49/f88 0 2024-02-16T19:29:05.667 INFO:tasks.workunit.client.0.smithi022.stdout:1/580: creat d6/d8/d22/d3f/d99/fb0 x:0 0 0 2024-02-16T19:29:05.668 INFO:tasks.workunit.client.1.smithi117.stdout:1/395: dwrite d2/d1d/d1f/f56 [0,4194304] 0 2024-02-16T19:29:05.668 INFO:tasks.workunit.client.1.smithi117.stdout:2/408: mkdir d0/d13/d4d/d80 0 2024-02-16T19:29:05.669 INFO:tasks.workunit.client.1.smithi117.stdout:6/430: write d5/d4b/d44/f56 [3811334,73912] 0 2024-02-16T19:29:05.669 INFO:tasks.workunit.client.1.smithi117.stdout:8/445: truncate d4/d7/d6a/d6f/f7a 2680153 0 2024-02-16T19:29:05.671 INFO:tasks.workunit.client.0.smithi022.stdout:4/429: mkdir d4/d6/d14/d64/d66/d86 0 2024-02-16T19:29:05.672 INFO:tasks.workunit.client.1.smithi117.stdout:1/396: chown d2/d1d/d60/d20/d49/f50 7 1 2024-02-16T19:29:05.675 INFO:tasks.workunit.client.1.smithi117.stdout:5/521: mkdir d2/d3/d21/d49/d59/daa 0 2024-02-16T19:29:05.678 INFO:tasks.workunit.client.0.smithi022.stdout:7/471: creat d9/da/d74/f99 x:0 0 0 2024-02-16T19:29:05.678 INFO:tasks.workunit.client.1.smithi117.stdout:7/413: sync 2024-02-16T19:29:05.678 INFO:tasks.workunit.client.1.smithi117.stdout:9/360: sync 2024-02-16T19:29:05.679 INFO:tasks.workunit.client.1.smithi117.stdout:9/361: write dd/d47/d48/f66 [363231,130966] 0 2024-02-16T19:29:05.680 INFO:tasks.workunit.client.0.smithi022.stdout:5/454: mknod dd/d1f/d48/d21/d2f/d46/d5e/c97 0 2024-02-16T19:29:05.681 INFO:tasks.workunit.client.0.smithi022.stdout:5/455: dread - dd/d1f/d48/f43 zero size 2024-02-16T19:29:05.681 INFO:tasks.workunit.client.1.smithi117.stdout:3/533: symlink d1/d83/d96/lb9 0 2024-02-16T19:29:05.685 INFO:tasks.workunit.client.0.smithi022.stdout:2/389: dread d0/f2d [0,4194304] 0 2024-02-16T19:29:05.685 INFO:tasks.workunit.client.1.smithi117.stdout:8/446: chown d4/d7/f3d 14761 1 2024-02-16T19:29:05.685 INFO:tasks.workunit.client.1.smithi117.stdout:8/447: chown d4/f52 1234 1 2024-02-16T19:29:05.686 INFO:tasks.workunit.client.1.smithi117.stdout:8/448: stat d4/d7/d37/d3a/d54/d70/d76 0 2024-02-16T19:29:05.686 INFO:tasks.workunit.client.0.smithi022.stdout:8/442: mkdir d3/d12/d2b/da9 0 2024-02-16T19:29:05.687 INFO:tasks.workunit.client.1.smithi117.stdout:2/409: symlink d0/d13/d4d/d53/l81 0 2024-02-16T19:29:05.687 INFO:tasks.workunit.client.1.smithi117.stdout:2/410: read - d0/d3/f7c zero size 2024-02-16T19:29:05.689 INFO:tasks.workunit.client.1.smithi117.stdout:2/411: chown d0/d13/d4d/d53/l81 375197 1 2024-02-16T19:29:05.690 INFO:tasks.workunit.client.0.smithi022.stdout:0/573: fsync d10/d11/d27/d3f/d58/d60/d62/f69 0 2024-02-16T19:29:05.691 INFO:tasks.workunit.client.0.smithi022.stdout:0/574: truncate d10/d11/d50/d65/fb8 10601 0 2024-02-16T19:29:05.692 INFO:tasks.workunit.client.0.smithi022.stdout:9/484: symlink db/d14/d16/d21/d37/la5 0 2024-02-16T19:29:05.692 INFO:tasks.workunit.client.0.smithi022.stdout:0/575: fdatasync d10/d22/f2f 0 2024-02-16T19:29:05.695 INFO:tasks.workunit.client.1.smithi117.stdout:5/522: creat d2/d9/d3c/d24/d50/d52/d74/fab x:0 0 0 2024-02-16T19:29:05.703 INFO:tasks.workunit.client.0.smithi022.stdout:7/472: creat d9/da/d2c/d46/f9a x:0 0 0 2024-02-16T19:29:05.706 INFO:tasks.workunit.client.1.smithi117.stdout:0/498: write d0/d9/d14/d1d/d1e/f21 [451788,106981] 0 2024-02-16T19:29:05.711 INFO:tasks.workunit.client.1.smithi117.stdout:8/449: rmdir d4/d12/d2a 39 2024-02-16T19:29:05.711 INFO:tasks.workunit.client.1.smithi117.stdout:8/450: fsync d4/d7/d37/d3a/d54/d70/f87 0 2024-02-16T19:29:05.712 INFO:tasks.workunit.client.0.smithi022.stdout:5/456: dwrite dd/d1f/d48/d21/d2f/d52/d5c/d61/d6a/f7f [0,4194304] 0 2024-02-16T19:29:05.717 INFO:tasks.workunit.client.0.smithi022.stdout:7/473: dread d9/da/d1f/d44/f49 [0,4194304] 0 2024-02-16T19:29:05.719 INFO:tasks.workunit.client.0.smithi022.stdout:7/474: chown d9/da/d1f/l1c 4 1 2024-02-16T19:29:05.725 INFO:tasks.workunit.client.1.smithi117.stdout:5/523: mkdir d2/d9/d3c/d24/d50/d52/d74/d8c/dac 0 2024-02-16T19:29:05.725 INFO:tasks.workunit.client.0.smithi022.stdout:6/468: dwrite d1/d8/da/f14 [0,4194304] 0 2024-02-16T19:29:05.727 INFO:tasks.workunit.client.1.smithi117.stdout:4/435: write d0/da/d21/d3d/d2d/d6d/f7e [2224031,65013] 0 2024-02-16T19:29:05.729 INFO:tasks.workunit.client.0.smithi022.stdout:0/576: creat d10/d11/d50/d65/fbe x:0 0 0 2024-02-16T19:29:05.730 INFO:tasks.workunit.client.1.smithi117.stdout:3/534: fdatasync d1/dd/f14 0 2024-02-16T19:29:05.735 INFO:tasks.workunit.client.0.smithi022.stdout:9/485: rename db/d14/d19/f4d to db/d14/d16/d27/d55/fa6 0 2024-02-16T19:29:05.737 INFO:tasks.workunit.client.0.smithi022.stdout:1/581: fsync d6/f6b 0 2024-02-16T19:29:05.739 INFO:tasks.workunit.client.0.smithi022.stdout:4/430: readlink d4/d6/dd/l3b 0 2024-02-16T19:29:05.742 INFO:tasks.workunit.client.1.smithi117.stdout:8/451: unlink d4/d7/d37/d50/f5b 0 2024-02-16T19:29:05.742 INFO:tasks.workunit.client.1.smithi117.stdout:6/431: write d5/f11 [3987421,34098] 0 2024-02-16T19:29:05.742 INFO:tasks.workunit.client.0.smithi022.stdout:3/423: link d7/d12/d18/dd/d31/l8a d7/d12/d18/dd/la4 0 2024-02-16T19:29:05.745 INFO:tasks.workunit.client.1.smithi117.stdout:5/524: fsync d2/d4c/d54/f43 0 2024-02-16T19:29:05.746 INFO:tasks.workunit.client.1.smithi117.stdout:1/397: truncate d2/d1d/d60/d20/f64 1329737 0 2024-02-16T19:29:05.746 INFO:tasks.workunit.client.1.smithi117.stdout:1/398: stat d2/d4 0 2024-02-16T19:29:05.748 INFO:tasks.workunit.client.1.smithi117.stdout:7/414: truncate d4/f1a 3669587 0 2024-02-16T19:29:05.748 INFO:tasks.workunit.client.1.smithi117.stdout:7/415: write d4/d1c/d39/d46/f96 [1021196,27660] 0 2024-02-16T19:29:05.752 INFO:tasks.workunit.client.1.smithi117.stdout:4/436: rename d0/da/d21/d3d/d28/d2f to d0/da/d21/d3d/d28/d64/d73/d91 0 2024-02-16T19:29:05.755 INFO:tasks.workunit.client.0.smithi022.stdout:8/443: symlink d3/d12/d17/d64/d8f/laa 0 2024-02-16T19:29:05.756 INFO:tasks.workunit.client.1.smithi117.stdout:3/535: mkdir d1/dd/d3b/dba 0 2024-02-16T19:29:05.763 INFO:tasks.workunit.client.0.smithi022.stdout:8/444: dwrite d3/d12/d17/d64/f73 [0,4194304] 0 2024-02-16T19:29:05.765 INFO:tasks.workunit.client.0.smithi022.stdout:0/577: fsync d10/d15/d23/f47 0 2024-02-16T19:29:05.770 INFO:tasks.workunit.client.0.smithi022.stdout:8/445: dwrite d3/d12/f8b [0,4194304] 0 2024-02-16T19:29:05.770 INFO:tasks.workunit.client.0.smithi022.stdout:2/390: truncate d0/d2/f14 1057483 0 2024-02-16T19:29:05.772 INFO:tasks.workunit.client.0.smithi022.stdout:9/486: stat db/c11 0 2024-02-16T19:29:05.778 INFO:tasks.workunit.client.0.smithi022.stdout:7/475: write d9/d1b/f57 [677485,58510] 0 2024-02-16T19:29:05.782 INFO:tasks.workunit.client.0.smithi022.stdout:3/424: rename d7/d12/d18/dd/d3c/d5a/d67/f75 to d7/d12/d18/dd/d31/d7f/fa5 0 2024-02-16T19:29:05.785 INFO:tasks.workunit.client.0.smithi022.stdout:6/469: dwrite d1/f3 [0,4194304] 0 2024-02-16T19:29:05.785 INFO:tasks.workunit.client.1.smithi117.stdout:2/412: sync 2024-02-16T19:29:05.790 INFO:tasks.workunit.client.1.smithi117.stdout:2/413: dwrite d0/d3/d10/f17 [0,4194304] 0 2024-02-16T19:29:05.792 INFO:tasks.workunit.client.0.smithi022.stdout:8/446: creat d3/d25/d33/fab x:0 0 0 2024-02-16T19:29:05.795 INFO:tasks.workunit.client.0.smithi022.stdout:2/391: creat d0/d2/d18/d19/d40/d55/f7c x:0 0 0 2024-02-16T19:29:05.805 INFO:tasks.workunit.client.0.smithi022.stdout:2/392: dwrite d0/d2/d18/d19/f30 [0,4194304] 0 2024-02-16T19:29:05.812 INFO:tasks.workunit.client.0.smithi022.stdout:5/457: creat dd/d1f/d48/d21/d2f/d40/d4e/f98 x:0 0 0 2024-02-16T19:29:05.813 INFO:tasks.workunit.client.0.smithi022.stdout:5/458: readlink dd/d1f/d48/d21/d2f/d40/l4d 0 2024-02-16T19:29:05.815 INFO:tasks.workunit.client.0.smithi022.stdout:6/470: symlink d1/d8/da/d29/d61/la7 0 2024-02-16T19:29:05.816 INFO:tasks.workunit.client.1.smithi117.stdout:8/452: mknod d4/d7/d37/d3a/d43/d44/c8a 0 2024-02-16T19:29:05.820 INFO:tasks.workunit.client.0.smithi022.stdout:0/578: mknod d10/d7d/dac/cbf 0 2024-02-16T19:29:05.821 INFO:tasks.workunit.client.1.smithi117.stdout:5/525: symlink d2/d3/d21/d2f/d30/d95/lad 0 2024-02-16T19:29:05.824 INFO:tasks.workunit.client.1.smithi117.stdout:9/362: getdents dd/d47/d5b 0 2024-02-16T19:29:05.827 INFO:tasks.workunit.client.1.smithi117.stdout:7/416: mkdir d4/d13/da3 0 2024-02-16T19:29:05.831 INFO:tasks.workunit.client.0.smithi022.stdout:9/487: link db/d14/d16/d5d/f7e db/d14/d33/fa7 0 2024-02-16T19:29:05.831 INFO:tasks.workunit.client.0.smithi022.stdout:4/431: dwrite d4/d6/dd/f1e [0,4194304] 0 2024-02-16T19:29:05.831 INFO:tasks.workunit.client.0.smithi022.stdout:1/582: symlink d6/d8/d6c/d96/lb1 0 2024-02-16T19:29:05.833 INFO:tasks.workunit.client.0.smithi022.stdout:4/432: chown d4/d6/d14/d64/d4e/d54/d5a/d63/l6e 5217625 1 2024-02-16T19:29:05.835 INFO:tasks.workunit.client.1.smithi117.stdout:4/437: rename d0/da/d21/d3d/d28/d64/d74 to d0/da/d21/d3d/d28/d64/d8a/d92 0 2024-02-16T19:29:05.835 INFO:tasks.workunit.client.1.smithi117.stdout:3/536: symlink d1/d1b/d20/lbb 0 2024-02-16T19:29:05.837 INFO:tasks.workunit.client.1.smithi117.stdout:3/537: truncate d1/dd/d8e/f94 80728 0 2024-02-16T19:29:05.837 INFO:tasks.workunit.client.0.smithi022.stdout:7/476: symlink d9/da/d4e/d7a/l9b 0 2024-02-16T19:29:05.844 INFO:tasks.workunit.client.0.smithi022.stdout:7/477: dwrite d9/da/d1f/f22 [4194304,4194304] 0 2024-02-16T19:29:05.846 INFO:tasks.workunit.client.0.smithi022.stdout:7/478: write d9/da/f3a [1866545,41614] 0 2024-02-16T19:29:05.847 INFO:tasks.workunit.client.0.smithi022.stdout:5/459: rename dd/d1f/d48/d21/d2f/d46/d5e/l80 to dd/d1f/d48/d21/d2f/d46/l99 0 2024-02-16T19:29:05.848 INFO:tasks.workunit.client.0.smithi022.stdout:7/479: read - d9/d5d/f97 zero size 2024-02-16T19:29:05.857 INFO:tasks.workunit.client.0.smithi022.stdout:4/433: truncate d4/d6/dd/d22/f68 834589 0 2024-02-16T19:29:05.858 INFO:tasks.workunit.client.0.smithi022.stdout:4/434: chown d4/d6/d14/f2e 82190717 1 2024-02-16T19:29:05.859 INFO:tasks.workunit.client.0.smithi022.stdout:4/435: dread - d4/d6/d14/d64/d66/d6d/f7d zero size 2024-02-16T19:29:05.859 INFO:tasks.workunit.client.0.smithi022.stdout:2/393: fsync d0/d2/d18/d1f/f5e 0 2024-02-16T19:29:05.864 INFO:tasks.workunit.client.1.smithi117.stdout:1/399: symlink d2/d19/l83 0 2024-02-16T19:29:05.866 INFO:tasks.workunit.client.0.smithi022.stdout:6/471: dread d1/d8/f66 [0,4194304] 0 2024-02-16T19:29:05.867 INFO:tasks.workunit.client.1.smithi117.stdout:7/417: mkdir d4/d1c/d39/d46/da4 0 2024-02-16T19:29:05.867 INFO:tasks.workunit.client.1.smithi117.stdout:8/453: dread d4/d7/d37/d3a/d43/d44/f4b [0,4194304] 0 2024-02-16T19:29:05.870 INFO:tasks.workunit.client.0.smithi022.stdout:9/488: rename db/d14/d19/d1a/d67 to db/d14/d19/da8 0 2024-02-16T19:29:05.871 INFO:tasks.workunit.client.0.smithi022.stdout:3/425: dwrite d7/d12/f25 [0,4194304] 0 2024-02-16T19:29:05.872 INFO:tasks.workunit.client.1.smithi117.stdout:2/414: dwrite d0/d3/f2a [0,4194304] 0 2024-02-16T19:29:05.873 INFO:tasks.workunit.client.1.smithi117.stdout:9/363: rename fb to dd/d47/d5e/f7f 0 2024-02-16T19:29:05.873 INFO:tasks.workunit.client.1.smithi117.stdout:7/418: rename d4/d1c to d4/d1c/d39/d46/d6e/da5 22 2024-02-16T19:29:05.874 INFO:tasks.workunit.client.0.smithi022.stdout:0/579: write d10/d22/f2d [2279565,88520] 0 2024-02-16T19:29:05.874 INFO:tasks.workunit.client.1.smithi117.stdout:7/419: truncate d4/f2b 4532907 0 2024-02-16T19:29:05.875 INFO:tasks.workunit.client.1.smithi117.stdout:4/438: truncate d0/da/d21/d3d/d28/f51 1425111 0 2024-02-16T19:29:05.876 INFO:tasks.workunit.client.1.smithi117.stdout:5/526: write d2/d1d/f48 [2062711,99488] 0 2024-02-16T19:29:05.878 INFO:tasks.workunit.client.1.smithi117.stdout:3/538: unlink d1/dd/d3b/f5a 0 2024-02-16T19:29:05.884 INFO:tasks.workunit.client.1.smithi117.stdout:0/499: getdents d0/d9 0 2024-02-16T19:29:05.885 INFO:tasks.workunit.client.0.smithi022.stdout:4/436: creat d4/d6/f87 x:0 0 0 2024-02-16T19:29:05.889 INFO:tasks.workunit.client.1.smithi117.stdout:1/400: truncate d2/d4/f11 4661792 0 2024-02-16T19:29:05.892 INFO:tasks.workunit.client.0.smithi022.stdout:3/426: creat d7/d21/fa6 x:0 0 0 2024-02-16T19:29:05.894 INFO:tasks.workunit.client.0.smithi022.stdout:9/489: creat db/d14/d16/d87/fa9 x:0 0 0 2024-02-16T19:29:05.896 INFO:tasks.workunit.client.1.smithi117.stdout:2/415: mkdir d0/d13/d4d/d64/d82 0 2024-02-16T19:29:05.899 INFO:tasks.workunit.client.1.smithi117.stdout:9/364: unlink dd/l21 0 2024-02-16T19:29:05.903 INFO:tasks.workunit.client.1.smithi117.stdout:4/439: creat d0/da/d21/d3d/d2d/d6d/f93 x:0 0 0 2024-02-16T19:29:05.905 INFO:tasks.workunit.client.0.smithi022.stdout:8/447: getdents d3/d12 0 2024-02-16T19:29:05.906 INFO:tasks.workunit.client.1.smithi117.stdout:5/527: truncate d2/d3/f4d 5180474 0 2024-02-16T19:29:05.907 INFO:tasks.workunit.client.0.smithi022.stdout:0/580: write d10/d11/d27/f52 [452272,101431] 0 2024-02-16T19:29:05.909 INFO:tasks.workunit.client.1.smithi117.stdout:6/432: getdents d5/d1d/d49 0 2024-02-16T19:29:05.910 INFO:tasks.workunit.client.0.smithi022.stdout:2/394: creat d0/d2/d2e/d73/f7d x:0 0 0 2024-02-16T19:29:05.912 INFO:tasks.workunit.client.0.smithi022.stdout:4/437: dread d4/d6/dd/df/d10/d55/f7b [0,4194304] 0 2024-02-16T19:29:05.912 INFO:tasks.workunit.client.1.smithi117.stdout:0/500: creat d0/d9/d14/d1d/d1e/d54/fb4 x:0 0 0 2024-02-16T19:29:05.913 INFO:tasks.workunit.client.0.smithi022.stdout:6/472: mknod d1/d8/da/ca8 0 2024-02-16T19:29:05.915 INFO:tasks.workunit.client.0.smithi022.stdout:3/427: creat d7/d12/d18/dd/d24/d58/d70/fa7 x:0 0 0 2024-02-16T19:29:05.916 INFO:tasks.workunit.client.0.smithi022.stdout:3/428: write d7/d12/d18/dd/d24/f9f [408331,94168] 0 2024-02-16T19:29:05.917 INFO:tasks.workunit.client.0.smithi022.stdout:9/490: mkdir db/d14/d16/d28/daa 0 2024-02-16T19:29:05.917 INFO:tasks.workunit.client.0.smithi022.stdout:7/480: link d9/da/d2c/d2f/c53 d9/da/d2c/d65/c9c 0 2024-02-16T19:29:05.919 INFO:tasks.workunit.client.0.smithi022.stdout:5/460: link dd/d1f/d48/f51 dd/d1f/d48/d21/d2f/d46/d5e/d84/d8b/f9a 0 2024-02-16T19:29:05.922 INFO:tasks.workunit.client.0.smithi022.stdout:8/448: write d3/d12/d2b/d54/d74/f82 [107173,48801] 0 2024-02-16T19:29:05.922 INFO:tasks.workunit.client.1.smithi117.stdout:7/420: creat d4/d1c/d98/fa6 x:0 0 0 2024-02-16T19:29:05.923 INFO:tasks.workunit.client.1.smithi117.stdout:2/416: dwrite d0/d13/f15 [0,4194304] 0 2024-02-16T19:29:05.923 INFO:tasks.workunit.client.0.smithi022.stdout:0/581: creat d10/d11/d79/d8b/fc0 x:0 0 0 2024-02-16T19:29:05.923 INFO:tasks.workunit.client.1.smithi117.stdout:7/421: chown d4/f3a 3 1 2024-02-16T19:29:05.924 INFO:tasks.workunit.client.0.smithi022.stdout:9/491: dwrite db/d14/d16/d87/fa9 [0,4194304] 0 2024-02-16T19:29:05.925 INFO:tasks.workunit.client.1.smithi117.stdout:7/422: chown d4/d1c/d87/d89/l76 439523 1 2024-02-16T19:29:05.927 INFO:tasks.workunit.client.1.smithi117.stdout:5/528: rename d2/d5d/d8e/fa7 to d2/d4c/fae 0 2024-02-16T19:29:05.929 INFO:tasks.workunit.client.1.smithi117.stdout:3/539: mkdir d1/d30/d9f/dbc 0 2024-02-16T19:29:05.929 INFO:tasks.workunit.client.1.smithi117.stdout:0/501: dread d0/d9/d22/d27/f4d [0,4194304] 0 2024-02-16T19:29:05.932 INFO:tasks.workunit.client.1.smithi117.stdout:0/502: chown d0/d30/f64 76 1 2024-02-16T19:29:05.933 INFO:tasks.workunit.client.1.smithi117.stdout:2/417: dwrite d0/d3b/f79 [0,4194304] 0 2024-02-16T19:29:05.936 INFO:tasks.workunit.client.1.smithi117.stdout:1/401: creat d2/d1d/d60/d20/d7d/d7f/f84 x:0 0 0 2024-02-16T19:29:05.938 INFO:tasks.workunit.client.0.smithi022.stdout:5/461: mknod dd/d1f/d4a/c9b 0 2024-02-16T19:29:05.939 INFO:tasks.workunit.client.1.smithi117.stdout:6/433: dwrite d5/ff [4194304,4194304] 0 2024-02-16T19:29:05.940 INFO:tasks.workunit.client.1.smithi117.stdout:9/365: mknod dd/d47/c80 0 2024-02-16T19:29:05.947 INFO:tasks.workunit.client.1.smithi117.stdout:3/540: dread d1/d1b/d1e/d29/f47 [0,4194304] 0 2024-02-16T19:29:05.947 INFO:tasks.workunit.client.1.smithi117.stdout:3/541: fsync d1/d1b/d20/fa2 0 2024-02-16T19:29:05.951 INFO:tasks.workunit.client.0.smithi022.stdout:6/473: dread d1/d8/d3c/d77/f80 [0,4194304] 0 2024-02-16T19:29:05.954 INFO:tasks.workunit.client.1.smithi117.stdout:3/542: dwrite d1/d83/d96/fa0 [0,4194304] 0 2024-02-16T19:29:05.954 INFO:tasks.workunit.client.1.smithi117.stdout:3/543: fsync d1/f19 0 2024-02-16T19:29:05.955 INFO:tasks.workunit.client.0.smithi022.stdout:0/582: unlink d10/d11/d50/d65/f74 0 2024-02-16T19:29:05.956 INFO:tasks.workunit.client.1.smithi117.stdout:3/544: write d1/d1b/d1e/d5e/f74 [244838,97806] 0 2024-02-16T19:29:05.956 INFO:tasks.workunit.client.1.smithi117.stdout:3/545: fsync d1/f19 0 2024-02-16T19:29:05.963 INFO:tasks.workunit.client.0.smithi022.stdout:2/395: rename d0/c8 to d0/d32/d3b/c7e 0 2024-02-16T19:29:05.965 INFO:tasks.workunit.client.0.smithi022.stdout:0/583: dwrite d10/d15/d23/d38/fb1 [0,4194304] 0 2024-02-16T19:29:05.965 INFO:tasks.workunit.client.0.smithi022.stdout:2/396: stat d0/d2/d18/d19/d40/c45 0 2024-02-16T19:29:05.970 INFO:tasks.workunit.client.0.smithi022.stdout:3/429: truncate d7/d12/d18/dd/d31/d7f/fa5 70143 0 2024-02-16T19:29:05.972 INFO:tasks.workunit.client.1.smithi117.stdout:8/454: getdents d4/d7 0 2024-02-16T19:29:05.974 INFO:tasks.workunit.client.0.smithi022.stdout:7/481: mknod d9/c9d 0 2024-02-16T19:29:05.975 INFO:tasks.workunit.client.1.smithi117.stdout:6/434: mknod d5/db/c8e 0 2024-02-16T19:29:05.976 INFO:tasks.workunit.client.0.smithi022.stdout:4/438: dwrite d4/d6/dd/df/d10/f42 [0,4194304] 0 2024-02-16T19:29:05.976 INFO:tasks.workunit.client.1.smithi117.stdout:6/435: fsync d5/f12 0 2024-02-16T19:29:05.978 INFO:tasks.workunit.client.0.smithi022.stdout:5/462: unlink dd/d1f/d48/f26 0 2024-02-16T19:29:05.978 INFO:tasks.workunit.client.0.smithi022.stdout:7/482: truncate d9/da/d1f/f18 1790255 0 2024-02-16T19:29:05.979 INFO:tasks.workunit.client.1.smithi117.stdout:9/366: symlink dd/d47/d48/l81 0 2024-02-16T19:29:05.982 INFO:tasks.workunit.client.0.smithi022.stdout:3/430: dwrite d7/d12/d18/dd/d24/d58/d6b/f78 [0,4194304] 0 2024-02-16T19:29:05.984 INFO:tasks.workunit.client.1.smithi117.stdout:4/440: dwrite d0/da/f69 [0,4194304] 0 2024-02-16T19:29:05.986 INFO:tasks.workunit.client.0.smithi022.stdout:8/449: creat d3/d12/d2b/d54/d74/da2/fac x:0 0 0 2024-02-16T19:29:05.987 INFO:tasks.workunit.client.0.smithi022.stdout:8/450: chown d3/d12/d2b/d54 1 1 2024-02-16T19:29:05.992 INFO:tasks.workunit.client.0.smithi022.stdout:0/584: rename d10/d11/d27/d3f/d58/d60/d62 to d10/d15/d23/d8f/dc1 0 2024-02-16T19:29:05.995 INFO:tasks.workunit.client.1.smithi117.stdout:5/529: link d2/d1d/f5f d2/d5d/faf 0 2024-02-16T19:29:05.996 INFO:tasks.workunit.client.0.smithi022.stdout:4/439: dread d4/d6/d14/d64/d4e/d54/d5a/f5f [0,4194304] 0 2024-02-16T19:29:05.998 INFO:tasks.workunit.client.1.smithi117.stdout:3/546: mkdir d1/d31/d89/dbd 0 2024-02-16T19:29:05.999 INFO:tasks.workunit.client.0.smithi022.stdout:4/440: dread d4/d6/dd/d22/f68 [0,4194304] 0 2024-02-16T19:29:06.001 INFO:tasks.workunit.client.0.smithi022.stdout:4/441: dread d4/d6/dd/df/d10/f35 [0,4194304] 0 2024-02-16T19:29:06.002 INFO:tasks.workunit.client.1.smithi117.stdout:2/418: truncate d0/d3/f31 555784 0 2024-02-16T19:29:06.005 INFO:tasks.workunit.client.1.smithi117.stdout:3/547: dwrite d1/d1b/d1e/f51 [0,4194304] 0 2024-02-16T19:29:06.005 INFO:tasks.workunit.client.1.smithi117.stdout:0/503: rename d0/d9/d14/d1d/d60/d65/c70 to d0/cb5 0 2024-02-16T19:29:06.008 INFO:tasks.workunit.client.0.smithi022.stdout:3/431: creat d7/d12/d18/dd/d3c/d5a/d67/fa8 x:0 0 0 2024-02-16T19:29:06.010 INFO:tasks.workunit.client.0.smithi022.stdout:8/451: readlink d3/d12/d2b/d46/l7f 0 2024-02-16T19:29:06.015 INFO:tasks.workunit.client.0.smithi022.stdout:9/492: write db/d14/d16/d5d/d65/d6b/f9b [4160403,97306] 0 2024-02-16T19:29:06.018 INFO:tasks.workunit.client.1.smithi117.stdout:4/441: creat d0/da/d21/d3d/d2d/f94 x:0 0 0 2024-02-16T19:29:06.021 INFO:tasks.workunit.client.1.smithi117.stdout:5/530: dread - d2/d9/d3c/f98 zero size 2024-02-16T19:29:06.023 INFO:tasks.workunit.client.1.smithi117.stdout:2/419: write d0/d29/d56/f72 [121334,12817] 0 2024-02-16T19:29:06.023 INFO:tasks.workunit.client.1.smithi117.stdout:2/420: stat d0 0 2024-02-16T19:29:06.025 INFO:tasks.workunit.client.1.smithi117.stdout:1/402: rename d2/d1d/d60/d20/d69/f72 to d2/d1d/d60/d20/d7d/d7f/f85 0 2024-02-16T19:29:06.027 INFO:tasks.workunit.client.1.smithi117.stdout:3/548: creat d1/dd/d3e/d7d/fbe x:0 0 0 2024-02-16T19:29:06.028 INFO:tasks.workunit.client.0.smithi022.stdout:7/483: write d9/da/d2c/d46/d73/f8e [518052,98069] 0 2024-02-16T19:29:06.029 INFO:tasks.workunit.client.0.smithi022.stdout:4/442: creat d4/d6/dd/df/d10/d24/f88 x:0 0 0 2024-02-16T19:29:06.030 INFO:tasks.workunit.client.1.smithi117.stdout:8/455: mknod d4/d7/d6a/d6f/c8b 0 2024-02-16T19:29:06.031 INFO:tasks.workunit.client.0.smithi022.stdout:3/432: rmdir d7/d12 39 2024-02-16T19:29:06.035 INFO:tasks.workunit.client.0.smithi022.stdout:2/397: link d0/d2/l15 d0/d2/d18/d19/d40/l7f 0 2024-02-16T19:29:06.035 INFO:tasks.workunit.client.1.smithi117.stdout:7/423: getdents d4/d1c/d39 0 2024-02-16T19:29:06.036 INFO:tasks.workunit.client.0.smithi022.stdout:0/585: creat d10/d22/dad/fc2 x:0 0 0 2024-02-16T19:29:06.036 INFO:tasks.workunit.client.0.smithi022.stdout:2/398: read d0/f5f [883090,110848] 0 2024-02-16T19:29:06.038 INFO:tasks.workunit.client.0.smithi022.stdout:4/443: dwrite d4/d6/dd/d22/d30/f26 [0,4194304] 0 2024-02-16T19:29:06.039 INFO:tasks.workunit.client.0.smithi022.stdout:9/493: dread db/d14/d19/d1a/f24 [0,4194304] 0 2024-02-16T19:29:06.039 INFO:tasks.workunit.client.1.smithi117.stdout:1/403: dread d2/f57 [0,4194304] 0 2024-02-16T19:29:06.040 INFO:tasks.workunit.client.0.smithi022.stdout:0/586: write d10/d15/d23/d38/d5a/d66/f9d [2791511,3224] 0 2024-02-16T19:29:06.040 INFO:tasks.workunit.client.0.smithi022.stdout:9/494: readlink db/d14/d19/da8/l7f 0 2024-02-16T19:29:06.041 INFO:tasks.workunit.client.0.smithi022.stdout:0/587: fdatasync d10/d22/f2f 0 2024-02-16T19:29:06.041 INFO:tasks.workunit.client.1.smithi117.stdout:7/424: dwrite d4/d1c/d39/f5b [0,4194304] 0 2024-02-16T19:29:06.043 INFO:tasks.workunit.client.0.smithi022.stdout:4/444: write d4/d6/d14/d64/f34 [3612115,124555] 0 2024-02-16T19:29:06.043 INFO:tasks.workunit.client.1.smithi117.stdout:7/425: write d4/d1c/d39/d46/d47/f51 [710796,75438] 0 2024-02-16T19:29:06.045 INFO:tasks.workunit.client.0.smithi022.stdout:9/495: truncate db/d14/d16/d28/f9a 607943 0 2024-02-16T19:29:06.047 INFO:tasks.workunit.client.1.smithi117.stdout:2/421: creat d0/d13/d4d/d64/d46/f83 x:0 0 0 2024-02-16T19:29:06.048 INFO:tasks.workunit.client.1.smithi117.stdout:5/531: rename d2/d4c/d54 to d2/d1d/db0 0 2024-02-16T19:29:06.048 INFO:tasks.workunit.client.1.smithi117.stdout:2/422: read d0/d3/d22/f28 [1789550,57534] 0 2024-02-16T19:29:06.049 INFO:tasks.workunit.client.1.smithi117.stdout:5/532: dread - d2/d4c/d4f/d67/f94 zero size 2024-02-16T19:29:06.050 INFO:tasks.workunit.client.0.smithi022.stdout:7/484: unlink d9/da/d2c/f69 0 2024-02-16T19:29:06.052 INFO:tasks.workunit.client.0.smithi022.stdout:7/485: write d9/da/d2c/d6d/f8b [128728,75650] 0 2024-02-16T19:29:06.054 INFO:tasks.workunit.client.0.smithi022.stdout:7/486: write f5 [1451581,24759] 0 2024-02-16T19:29:06.060 INFO:tasks.workunit.client.0.smithi022.stdout:8/452: creat d3/d12/d92/fad x:0 0 0 2024-02-16T19:29:06.061 INFO:tasks.workunit.client.1.smithi117.stdout:6/436: creat d5/d4b/f8f x:0 0 0 2024-02-16T19:29:06.062 INFO:tasks.workunit.client.0.smithi022.stdout:8/453: write d3/d25/d33/fa0 [654926,124737] 0 2024-02-16T19:29:06.064 INFO:tasks.workunit.client.1.smithi117.stdout:9/367: dwrite dd/d47/d5e/f7f [4194304,4194304] 0 2024-02-16T19:29:06.064 INFO:tasks.workunit.client.0.smithi022.stdout:3/433: dwrite d7/d12/d18/dd/d24/d58/f8b [0,4194304] 0 2024-02-16T19:29:06.069 INFO:tasks.workunit.client.1.smithi117.stdout:2/423: dread d0/fd [0,4194304] 0 2024-02-16T19:29:06.070 INFO:tasks.workunit.client.0.smithi022.stdout:2/399: symlink d0/d2/d18/d19/d40/d55/d59/l80 0 2024-02-16T19:29:06.070 INFO:tasks.workunit.client.1.smithi117.stdout:2/424: write d0/d3/d22/d37/f63 [430299,23973] 0 2024-02-16T19:29:06.072 INFO:tasks.workunit.client.0.smithi022.stdout:5/463: getdents dd/d1f/d48/d21/d5d 0 2024-02-16T19:29:06.075 INFO:tasks.workunit.client.1.smithi117.stdout:2/425: dread d0/d3b/f51 [0,4194304] 0 2024-02-16T19:29:06.075 INFO:tasks.workunit.client.0.smithi022.stdout:0/588: write d10/d11/d79/d8b/fb9 [2050095,43083] 0 2024-02-16T19:29:06.076 INFO:tasks.workunit.client.1.smithi117.stdout:7/426: truncate d4/f1e 1114381 0 2024-02-16T19:29:06.078 INFO:tasks.workunit.client.0.smithi022.stdout:0/589: read d10/d11/d27/d3f/f7a [446054,74808] 0 2024-02-16T19:29:06.079 INFO:tasks.workunit.client.0.smithi022.stdout:0/590: fsync d10/d11/d27/f52 0 2024-02-16T19:29:06.082 INFO:tasks.workunit.client.0.smithi022.stdout:9/496: creat db/d14/d19/da8/fab x:0 0 0 2024-02-16T19:29:06.084 INFO:tasks.workunit.client.1.smithi117.stdout:5/533: creat d2/d3/d21/d2f/fb1 x:0 0 0 2024-02-16T19:29:06.085 INFO:tasks.workunit.client.0.smithi022.stdout:7/487: chown d9/da/c30 3 1 2024-02-16T19:29:06.088 INFO:tasks.workunit.client.1.smithi117.stdout:0/504: creat d0/d9/d22/d27/d83/fb6 x:0 0 0 2024-02-16T19:29:06.091 INFO:tasks.workunit.client.0.smithi022.stdout:8/454: creat d3/d12/d2b/d54/d97/fae x:0 0 0 2024-02-16T19:29:06.092 INFO:tasks.workunit.client.0.smithi022.stdout:8/455: chown d3/d12/f9e 48422 1 2024-02-16T19:29:06.093 INFO:tasks.workunit.client.1.smithi117.stdout:6/437: creat d5/d1d/d72/f90 x:0 0 0 2024-02-16T19:29:06.093 INFO:tasks.workunit.client.0.smithi022.stdout:8/456: write d3/d25/d33/fab [370609,109565] 0 2024-02-16T19:29:06.097 INFO:tasks.workunit.client.1.smithi117.stdout:1/404: write d2/d1d/d60/f66 [4779510,9675] 0 2024-02-16T19:29:06.098 INFO:tasks.workunit.client.1.smithi117.stdout:9/368: creat dd/d45/d73/f82 x:0 0 0 2024-02-16T19:29:06.101 INFO:tasks.workunit.client.0.smithi022.stdout:5/464: rename dd/d1f/d48/d21/d2f/d40/l4d to dd/d1f/d4a/d7a/l9c 0 2024-02-16T19:29:06.102 INFO:tasks.workunit.client.0.smithi022.stdout:4/445: write d4/d6/dd/d22/f68 [1827495,130221] 0 2024-02-16T19:29:06.103 INFO:tasks.workunit.client.1.smithi117.stdout:1/405: dwrite d2/d51/f75 [0,4194304] 0 2024-02-16T19:29:06.108 INFO:tasks.workunit.client.0.smithi022.stdout:9/497: dread db/d18/f2a [0,4194304] 0 2024-02-16T19:29:06.110 INFO:tasks.workunit.client.0.smithi022.stdout:0/591: creat d10/d11/d50/fc3 x:0 0 0 2024-02-16T19:29:06.111 INFO:tasks.workunit.client.1.smithi117.stdout:2/426: creat d0/d3/d22/f84 x:0 0 0 2024-02-16T19:29:06.112 INFO:tasks.workunit.client.0.smithi022.stdout:4/446: dwrite d4/d6/f16 [0,4194304] 0 2024-02-16T19:29:06.119 INFO:tasks.workunit.client.1.smithi117.stdout:3/549: truncate d1/d1b/d1e/d5e/f62 7027781 0 2024-02-16T19:29:06.122 INFO:tasks.workunit.client.0.smithi022.stdout:5/465: mkdir dd/d1f/d48/d21/d2f/d40/d4e/d56/d9d 0 2024-02-16T19:29:06.123 INFO:tasks.workunit.client.0.smithi022.stdout:9/498: write db/d14/d19/f17 [2887638,78281] 0 2024-02-16T19:29:06.125 INFO:tasks.workunit.client.1.smithi117.stdout:6/438: creat d5/d4b/d20/d69/f91 x:0 0 0 2024-02-16T19:29:06.126 INFO:tasks.workunit.client.1.smithi117.stdout:6/439: truncate d5/db/d7d/d32/f51 988755 0 2024-02-16T19:29:06.126 INFO:tasks.workunit.client.0.smithi022.stdout:0/592: symlink d10/d11/d27/lc4 0 2024-02-16T19:29:06.127 INFO:tasks.workunit.client.0.smithi022.stdout:0/593: dread - d10/d11/d50/fc3 zero size 2024-02-16T19:29:06.128 INFO:tasks.workunit.client.0.smithi022.stdout:0/594: readlink d10/d11/d79/d8b/lba 0 2024-02-16T19:29:06.130 INFO:tasks.workunit.client.0.smithi022.stdout:0/595: stat d10/d11/d27/d3f/d58/d60 0 2024-02-16T19:29:06.132 INFO:tasks.workunit.client.0.smithi022.stdout:4/447: creat d4/d6/d14/d64/d66/d6d/f89 x:0 0 0 2024-02-16T19:29:06.134 INFO:tasks.workunit.client.0.smithi022.stdout:3/434: truncate d7/d12/d18/dd/d31/d41/f48 870512 0 2024-02-16T19:29:06.135 INFO:tasks.workunit.client.1.smithi117.stdout:9/369: rmdir dd/d47 39 2024-02-16T19:29:06.137 INFO:tasks.workunit.client.1.smithi117.stdout:1/406: mkdir d2/d1d/d60/d20/d7b/d86 0 2024-02-16T19:29:06.139 INFO:tasks.workunit.client.0.smithi022.stdout:9/499: write db/d18/f73 [8328018,23793] 0 2024-02-16T19:29:06.140 INFO:tasks.workunit.client.1.smithi117.stdout:4/442: getdents d0 0 2024-02-16T19:29:06.141 INFO:tasks.workunit.client.0.smithi022.stdout:2/400: dwrite d0/d2/d18/f22 [0,4194304] 0 2024-02-16T19:29:06.142 INFO:tasks.workunit.client.0.smithi022.stdout:5/466: truncate dd/d1f/d48/d21/d2f/d40/d4e/f68 86997 0 2024-02-16T19:29:06.145 INFO:tasks.workunit.client.0.smithi022.stdout:7/488: creat d9/da/d2c/d2f/f9e x:0 0 0 2024-02-16T19:29:06.146 INFO:tasks.workunit.client.1.smithi117.stdout:2/427: mknod d0/d3/d34/c85 0 2024-02-16T19:29:06.147 INFO:tasks.workunit.client.0.smithi022.stdout:0/596: unlink d10/d11/d50/d65/l82 0 2024-02-16T19:29:06.149 INFO:tasks.workunit.client.1.smithi117.stdout:7/427: creat d4/d13/da3/fa7 x:0 0 0 2024-02-16T19:29:06.150 INFO:tasks.workunit.client.0.smithi022.stdout:4/448: mkdir d4/d6/d14/d64/d4e/d54/d8a 0 2024-02-16T19:29:06.152 INFO:tasks.workunit.client.1.smithi117.stdout:3/550: fsync d1/dd/d3b/f3d 0 2024-02-16T19:29:06.156 INFO:tasks.workunit.client.0.smithi022.stdout:3/435: creat d7/d12/d18/dd/d3c/fa9 x:0 0 0 2024-02-16T19:29:06.157 INFO:tasks.workunit.client.1.smithi117.stdout:0/505: mkdir d0/d9/d22/db7 0 2024-02-16T19:29:06.158 INFO:tasks.workunit.client.0.smithi022.stdout:3/436: stat d7/d12/d18/dd/d24/d53/f5e 0 2024-02-16T19:29:06.159 INFO:tasks.workunit.client.0.smithi022.stdout:3/437: write d7/d12/d18/dd/d3c/f92 [941893,91473] 0 2024-02-16T19:29:06.161 INFO:tasks.workunit.client.1.smithi117.stdout:6/440: unlink d5/d4b/f8f 0 2024-02-16T19:29:06.163 INFO:tasks.workunit.client.1.smithi117.stdout:8/456: getdents d4/d7/d37/d3a/d54/d70/d76 0 2024-02-16T19:29:06.163 INFO:tasks.workunit.client.0.smithi022.stdout:9/500: read db/f12 [229694,88130] 0 2024-02-16T19:29:06.164 INFO:tasks.workunit.client.1.smithi117.stdout:8/457: truncate d4/d7/d6a/f7b 357754 0 2024-02-16T19:29:06.164 INFO:tasks.workunit.client.0.smithi022.stdout:9/501: chown db/d14/d16/d21/d37/f4e 8916145 1 2024-02-16T19:29:06.165 INFO:tasks.workunit.client.0.smithi022.stdout:2/401: symlink d0/d32/d3b/d6f/l81 0 2024-02-16T19:29:06.165 INFO:tasks.workunit.client.1.smithi117.stdout:7/428: dread d4/d1c/d87/d89/d5f/f81 [0,4194304] 0 2024-02-16T19:29:06.166 INFO:tasks.workunit.client.0.smithi022.stdout:9/502: truncate db/d14/d19/da8/fab 929837 0 2024-02-16T19:29:06.167 INFO:tasks.workunit.client.1.smithi117.stdout:7/429: write d4/d1c/d39/d46/d47/f56 [3657839,19557] 0 2024-02-16T19:29:06.167 INFO:tasks.workunit.client.1.smithi117.stdout:9/370: truncate dd/d47/f60 1280839 0 2024-02-16T19:29:06.168 INFO:tasks.workunit.client.0.smithi022.stdout:9/503: write db/d14/d16/d27/fa3 [293391,19044] 0 2024-02-16T19:29:06.169 INFO:tasks.workunit.client.0.smithi022.stdout:5/467: rename dd/d1f/d48/d21/d2f/d52/d5c/c79 to dd/d1f/d48/d21/d2f/d46/d5e/d84/c9e 0 2024-02-16T19:29:06.171 INFO:tasks.workunit.client.1.smithi117.stdout:9/371: dread dd/d2d/f35 [0,4194304] 0 2024-02-16T19:29:06.171 INFO:tasks.workunit.client.0.smithi022.stdout:7/489: creat d9/da/d3b/f9f x:0 0 0 2024-02-16T19:29:06.172 INFO:tasks.workunit.client.0.smithi022.stdout:7/490: chown d9/da/d1f/d44 58182050 1 2024-02-16T19:29:06.180 INFO:tasks.workunit.client.0.smithi022.stdout:3/438: dread d7/d12/d18/f1a [0,4194304] 0 2024-02-16T19:29:06.180 INFO:tasks.workunit.client.0.smithi022.stdout:8/457: link d3/d12/d17/c72 d3/d12/caf 0 2024-02-16T19:29:06.181 INFO:tasks.workunit.client.1.smithi117.stdout:3/551: write d1/d1b/d1e/d37/f79 [2037857,38654] 0 2024-02-16T19:29:06.183 INFO:tasks.workunit.client.1.smithi117.stdout:5/534: link d2/d9/d3c/d24/f26 d2/d36/fb2 0 2024-02-16T19:29:06.188 INFO:tasks.workunit.client.0.smithi022.stdout:4/449: write d4/d6/dd/df/d10/f4f [1790731,55941] 0 2024-02-16T19:29:06.194 INFO:tasks.workunit.client.0.smithi022.stdout:0/597: symlink d10/d11/d27/d3f/d58/dab/lc5 0 2024-02-16T19:29:06.194 INFO:tasks.workunit.client.1.smithi117.stdout:0/506: dwrite d0/d9/d22/d27/d4e/fac [0,4194304] 0 2024-02-16T19:29:06.194 INFO:tasks.workunit.client.1.smithi117.stdout:1/407: mknod d2/c87 0 2024-02-16T19:29:06.195 INFO:tasks.workunit.client.0.smithi022.stdout:5/468: write dd/d1f/f58 [118255,98726] 0 2024-02-16T19:29:06.196 INFO:tasks.workunit.client.1.smithi117.stdout:1/408: chown d2/d1d/d60/d20/d7b 85 1 2024-02-16T19:29:06.197 INFO:tasks.workunit.client.1.smithi117.stdout:8/458: dwrite d4/d12/d2a/f2b [4194304,4194304] 0 2024-02-16T19:29:06.200 INFO:tasks.workunit.client.0.smithi022.stdout:8/458: unlink d3/d12/d17/d64/la1 0 2024-02-16T19:29:06.202 INFO:tasks.workunit.client.1.smithi117.stdout:4/443: truncate d0/da/f69 3205993 0 2024-02-16T19:29:06.202 INFO:tasks.workunit.client.1.smithi117.stdout:8/459: dread d4/d7/d37/d3a/d43/f6b [0,4194304] 0 2024-02-16T19:29:06.207 INFO:tasks.workunit.client.1.smithi117.stdout:3/552: rename d1/d1b/f5f to d1/dd/d3e/d7d/fbf 0 2024-02-16T19:29:06.210 INFO:tasks.workunit.client.1.smithi117.stdout:6/441: unlink d5/d4b/d44/d52/f5f 0 2024-02-16T19:29:06.212 INFO:tasks.workunit.client.1.smithi117.stdout:7/430: creat d4/d1c/d87/d89/d60/d9b/fa8 x:0 0 0 2024-02-16T19:29:06.215 INFO:tasks.workunit.client.0.smithi022.stdout:8/459: rename d3/d12/d2b/d46/d68 to d3/d12/db0 0 2024-02-16T19:29:06.221 INFO:tasks.workunit.client.0.smithi022.stdout:9/504: link db/d14/l32 db/d14/d16/d5d/lac 0 2024-02-16T19:29:06.225 INFO:tasks.workunit.client.0.smithi022.stdout:4/450: link d4/d6/f7f d4/d6/d14/d64/d4e/f8b 0 2024-02-16T19:29:06.227 INFO:tasks.workunit.client.1.smithi117.stdout:2/428: rmdir d0/d3b/d6e 0 2024-02-16T19:29:06.227 INFO:tasks.workunit.client.1.smithi117.stdout:7/431: dread d4/d1c/d39/d46/f48 [0,4194304] 0 2024-02-16T19:29:06.227 INFO:tasks.workunit.client.1.smithi117.stdout:2/429: chown d0/d3/d22/c24 0 1 2024-02-16T19:29:06.228 INFO:tasks.workunit.client.0.smithi022.stdout:7/491: link d9/d1b/d23/l52 d9/d1b/d23/la0 0 2024-02-16T19:29:06.231 INFO:tasks.workunit.client.0.smithi022.stdout:7/492: write d9/da/d74/f99 [36399,127641] 0 2024-02-16T19:29:06.233 INFO:tasks.workunit.client.1.smithi117.stdout:7/432: dwrite d4/d1c/d39/f5b [0,4194304] 0 2024-02-16T19:29:06.234 INFO:tasks.workunit.client.1.smithi117.stdout:7/433: read - d4/d13/d63/fa2 zero size 2024-02-16T19:29:06.234 INFO:tasks.workunit.client.1.smithi117.stdout:8/460: mkdir d4/d7/d37/d3a/d54/d70/d76/d8c 0 2024-02-16T19:29:06.235 INFO:tasks.workunit.client.0.smithi022.stdout:4/451: dwrite d4/d6/d3a/f41 [0,4194304] 0 2024-02-16T19:29:06.239 INFO:tasks.workunit.client.1.smithi117.stdout:1/409: write d2/ff [7943175,61314] 0 2024-02-16T19:29:06.239 INFO:tasks.workunit.client.1.smithi117.stdout:9/372: dwrite dd/d23/f1f [0,4194304] 0 2024-02-16T19:29:06.242 INFO:tasks.workunit.client.1.smithi117.stdout:3/553: creat d1/d30/d52/fc0 x:0 0 0 2024-02-16T19:29:06.243 INFO:tasks.workunit.client.0.smithi022.stdout:4/452: dread d4/d38/f3d [0,4194304] 0 2024-02-16T19:29:06.245 INFO:tasks.workunit.client.0.smithi022.stdout:8/460: stat d3/d12/d2b/c93 0 2024-02-16T19:29:06.247 INFO:tasks.workunit.client.0.smithi022.stdout:2/402: getdents d0/d2 0 2024-02-16T19:29:06.251 INFO:tasks.workunit.client.1.smithi117.stdout:6/442: creat d5/db/d7d/d32/f92 x:0 0 0 2024-02-16T19:29:06.254 INFO:tasks.workunit.client.0.smithi022.stdout:0/598: truncate d10/d11/d27/f52 3506089 0 2024-02-16T19:29:06.254 INFO:tasks.workunit.client.0.smithi022.stdout:2/403: truncate d0/d2/d18/d38/d6b/f76 766096 0 2024-02-16T19:29:06.255 INFO:tasks.workunit.client.0.smithi022.stdout:7/493: mknod d9/da/d1f/d40/ca1 0 2024-02-16T19:29:06.256 INFO:tasks.workunit.client.1.smithi117.stdout:9/373: dread dd/f1e [0,4194304] 0 2024-02-16T19:29:06.257 INFO:tasks.workunit.client.1.smithi117.stdout:9/374: dread - dd/d45/d62/f4e zero size 2024-02-16T19:29:06.259 INFO:tasks.workunit.client.0.smithi022.stdout:5/469: creat dd/d1f/f9f x:0 0 0 2024-02-16T19:29:06.259 INFO:tasks.workunit.client.0.smithi022.stdout:4/453: dwrite d4/d6/d14/d64/d66/f82 [0,4194304] 0 2024-02-16T19:29:06.261 INFO:tasks.workunit.client.0.smithi022.stdout:3/439: getdents d7/d12/d18/dd 0 2024-02-16T19:29:06.263 INFO:tasks.workunit.client.0.smithi022.stdout:8/461: rmdir d3/d12/d17/d64/d8f 39 2024-02-16T19:29:06.263 INFO:tasks.workunit.client.0.smithi022.stdout:9/505: creat db/d14/d16/d28/daa/fad x:0 0 0 2024-02-16T19:29:06.265 INFO:tasks.workunit.client.0.smithi022.stdout:3/440: stat d7/d12/d18/dd/d24/d58/f8b 0 2024-02-16T19:29:06.266 INFO:tasks.workunit.client.0.smithi022.stdout:9/506: chown db/d14/d16/f69 973 1 2024-02-16T19:29:06.266 INFO:tasks.workunit.client.1.smithi117.stdout:2/430: rename d0/d3/d22/l70 to d0/d13/d4d/d64/l86 0 2024-02-16T19:29:06.270 INFO:tasks.workunit.client.0.smithi022.stdout:2/404: mkdir d0/d2/d2e/d82 0 2024-02-16T19:29:06.276 INFO:tasks.workunit.client.0.smithi022.stdout:9/507: dread db/d18/f2a [0,4194304] 0 2024-02-16T19:29:06.277 INFO:tasks.workunit.client.0.smithi022.stdout:3/441: dwrite d7/d12/d18/dd/d24/d58/d6b/f89 [0,4194304] 0 2024-02-16T19:29:06.282 INFO:tasks.workunit.client.0.smithi022.stdout:2/405: dread d0/f2d [0,4194304] 0 2024-02-16T19:29:06.282 INFO:tasks.workunit.client.0.smithi022.stdout:5/470: dread dd/d1f/d48/d21/d2f/d52/d5c/d61/f92 [0,4194304] 0 2024-02-16T19:29:06.283 INFO:tasks.workunit.client.1.smithi117.stdout:1/410: rmdir d2/d1d 39 2024-02-16T19:29:06.284 INFO:tasks.workunit.client.0.smithi022.stdout:2/406: chown d0/d2/d18/d1f/f4d 4946 1 2024-02-16T19:29:06.290 INFO:tasks.workunit.client.0.smithi022.stdout:8/462: rmdir d3/d12/d2b/d54/d74 39 2024-02-16T19:29:06.291 INFO:tasks.workunit.client.0.smithi022.stdout:8/463: dread d3/d25/f79 [0,4194304] 0 2024-02-16T19:29:06.292 INFO:tasks.workunit.client.1.smithi117.stdout:3/554: creat d1/dd/d5c/fc1 x:0 0 0 2024-02-16T19:29:06.295 INFO:tasks.workunit.client.1.smithi117.stdout:5/535: rename d2/d1d to d2/db3 0 2024-02-16T19:29:06.295 INFO:tasks.workunit.client.1.smithi117.stdout:5/536: chown d2/d9/c7b 53859 1 2024-02-16T19:29:06.295 INFO:tasks.workunit.client.1.smithi117.stdout:5/537: readlink d2/d9/d3c/l1f 0 2024-02-16T19:29:06.297 INFO:tasks.workunit.client.1.smithi117.stdout:0/507: creat d0/d9/d22/fb8 x:0 0 0 2024-02-16T19:29:06.299 INFO:tasks.workunit.client.0.smithi022.stdout:9/508: mknod db/d14/d16/d27/d55/cae 0 2024-02-16T19:29:06.301 INFO:tasks.workunit.client.0.smithi022.stdout:7/494: creat d9/da/d1f/d93/fa2 x:0 0 0 2024-02-16T19:29:06.302 INFO:tasks.workunit.client.1.smithi117.stdout:7/434: write d4/d1c/f2e [5024405,31669] 0 2024-02-16T19:29:06.305 INFO:tasks.workunit.client.1.smithi117.stdout:9/375: rename dd/l26 to dd/d47/d5b/d5f/d77/l83 0 2024-02-16T19:29:06.306 INFO:tasks.workunit.client.0.smithi022.stdout:5/471: symlink dd/d1f/d48/d21/d2f/d40/d49/la0 0 2024-02-16T19:29:06.309 INFO:tasks.workunit.client.1.smithi117.stdout:4/444: creat d0/da/d21/d3d/d2d/d6d/f95 x:0 0 0 2024-02-16T19:29:06.309 INFO:tasks.workunit.client.1.smithi117.stdout:4/445: chown d0/da/d21/f3f 1 1 2024-02-16T19:29:06.310 INFO:tasks.workunit.client.0.smithi022.stdout:2/407: symlink d0/d2/d18/d19/d40/d55/d59/l83 0 2024-02-16T19:29:06.313 INFO:tasks.workunit.client.1.smithi117.stdout:8/461: mknod d4/d12/d2a/d64/c8d 0 2024-02-16T19:29:06.314 INFO:tasks.workunit.client.0.smithi022.stdout:0/599: truncate d10/d22/d49/f76 355457 0 2024-02-16T19:29:06.314 INFO:tasks.workunit.client.1.smithi117.stdout:8/462: dread d4/d73/f82 [0,4194304] 0 2024-02-16T19:29:06.317 INFO:tasks.workunit.client.1.smithi117.stdout:1/411: unlink d2/f78 0 2024-02-16T19:29:06.321 INFO:tasks.workunit.client.0.smithi022.stdout:9/509: creat db/d14/d16/d5d/d65/d6b/faf x:0 0 0 2024-02-16T19:29:06.322 INFO:tasks.workunit.client.0.smithi022.stdout:7/495: symlink d9/da/d2c/d46/la3 0 2024-02-16T19:29:06.324 INFO:tasks.workunit.client.0.smithi022.stdout:0/600: dwrite d10/d15/d23/d38/f9e [0,4194304] 0 2024-02-16T19:29:06.324 INFO:tasks.workunit.client.0.smithi022.stdout:7/496: chown d9/da/d1f/d67/l86 3171 1 2024-02-16T19:29:06.325 INFO:tasks.workunit.client.0.smithi022.stdout:2/408: dread d0/d2/d18/d19/d40/f44 [0,4194304] 0 2024-02-16T19:29:06.327 INFO:tasks.workunit.client.0.smithi022.stdout:3/442: link d7/d21/f38 d7/d12/d18/dd/d31/d41/faa 0 2024-02-16T19:29:06.328 INFO:tasks.workunit.client.1.smithi117.stdout:0/508: symlink d0/d9/d14/d1d/d60/d65/lb9 0 2024-02-16T19:29:06.330 INFO:tasks.workunit.client.1.smithi117.stdout:0/509: write d0/d9/d14/d1d/d1e/d54/f72 [1614131,125283] 0 2024-02-16T19:29:06.330 INFO:tasks.workunit.client.1.smithi117.stdout:6/443: mkdir d5/d4b/d44/d93 0 2024-02-16T19:29:06.332 INFO:tasks.workunit.client.1.smithi117.stdout:0/510: dread - d0/d9/d14/d23/d33/fb3 zero size 2024-02-16T19:29:06.332 INFO:tasks.workunit.client.1.smithi117.stdout:0/511: chown d0/d9/d14 477 1 2024-02-16T19:29:06.334 INFO:tasks.workunit.client.1.smithi117.stdout:0/512: fsync d0/d9/d14/d1d/d1e/d54/f72 0 2024-02-16T19:29:06.373 INFO:tasks.workunit.client.1.smithi117.stdout:7/435: dread - d4/f4d zero size 2024-02-16T19:29:06.374 INFO:tasks.workunit.client.0.smithi022.stdout:4/454: link d4/d6/dd/f2c d4/d6/dd/df/d10/d24/f8c 0 2024-02-16T19:29:06.376 INFO:tasks.workunit.client.1.smithi117.stdout:9/376: rmdir dd/d45/d31 39 2024-02-16T19:29:06.384 INFO:tasks.workunit.client.1.smithi117.stdout:2/431: sync 2024-02-16T19:29:06.388 INFO:tasks.workunit.client.1.smithi117.stdout:7/436: dread d4/d1c/d39/d46/d47/f82 [0,4194304] 0 2024-02-16T19:29:06.388 INFO:tasks.workunit.client.1.smithi117.stdout:2/432: dread d0/d3/d10/f1d [0,4194304] 0 2024-02-16T19:29:06.396 INFO:tasks.workunit.client.1.smithi117.stdout:4/446: mknod d0/da/d21/d3d/d28/d64/d8a/d92/c96 0 2024-02-16T19:29:06.396 INFO:tasks.workunit.client.1.smithi117.stdout:4/447: stat d0/da/l12 0 2024-02-16T19:29:06.399 INFO:tasks.workunit.client.1.smithi117.stdout:8/463: readlink d4/d7/d37/d3a/d54/d70/l86 0 2024-02-16T19:29:06.399 INFO:tasks.workunit.client.1.smithi117.stdout:7/437: dread d4/f11 [0,4194304] 0 2024-02-16T19:29:06.402 INFO:tasks.workunit.client.0.smithi022.stdout:9/510: fsync db/f12 0 2024-02-16T19:29:06.402 INFO:tasks.workunit.client.1.smithi117.stdout:4/448: dwrite d0/f5c [0,4194304] 0 2024-02-16T19:29:06.403 INFO:tasks.workunit.client.0.smithi022.stdout:8/464: dwrite d3/d12/da6/f8c [0,4194304] 0 2024-02-16T19:29:06.406 INFO:tasks.workunit.client.0.smithi022.stdout:8/465: fsync d3/d12/f8b 0 2024-02-16T19:29:06.406 INFO:tasks.workunit.client.0.smithi022.stdout:7/497: mknod d9/da/d3b/ca4 0 2024-02-16T19:29:06.411 INFO:tasks.workunit.client.0.smithi022.stdout:7/498: chown d9/da/d1f/d40 148234 1 2024-02-16T19:29:06.413 INFO:tasks.workunit.client.1.smithi117.stdout:3/555: mknod d1/d30/d80/cc2 0 2024-02-16T19:29:06.413 INFO:tasks.workunit.client.1.smithi117.stdout:8/464: dread d4/d12/f26 [0,4194304] 0 2024-02-16T19:29:06.414 INFO:tasks.workunit.client.1.smithi117.stdout:3/556: fsync d1/d1b/d1e/d29/f33 0 2024-02-16T19:29:06.416 INFO:tasks.workunit.client.1.smithi117.stdout:8/465: truncate d4/d7/d37/d3a/d54/d70/f87 194277 0 2024-02-16T19:29:06.417 INFO:tasks.workunit.client.0.smithi022.stdout:2/409: chown d0/d2/d18/d19/c31 3731 1 2024-02-16T19:29:06.419 INFO:tasks.workunit.client.0.smithi022.stdout:8/466: dwrite d3/d12/da6/f5e [0,4194304] 0 2024-02-16T19:29:06.419 INFO:tasks.workunit.client.1.smithi117.stdout:6/444: rmdir d5/d4b/d71 39 2024-02-16T19:29:06.421 INFO:tasks.workunit.client.0.smithi022.stdout:9/511: dwrite db/d14/d16/d28/daa/fad [0,4194304] 0 2024-02-16T19:29:06.423 INFO:tasks.workunit.client.0.smithi022.stdout:9/512: stat db 0 2024-02-16T19:29:06.426 INFO:tasks.workunit.client.0.smithi022.stdout:5/472: dwrite dd/d1f/f31 [0,4194304] 0 2024-02-16T19:29:06.434 INFO:tasks.workunit.client.0.smithi022.stdout:5/473: dread dd/d1f/d48/d21/d2f/d52/d5c/d61/d6a/f7f [0,4194304] 0 2024-02-16T19:29:06.436 INFO:tasks.workunit.client.0.smithi022.stdout:3/443: write d7/d12/f19 [4329768,77484] 0 2024-02-16T19:29:06.442 INFO:tasks.workunit.client.0.smithi022.stdout:3/444: read d7/d12/d18/dd/d24/d29/f2c [2081757,2725] 0 2024-02-16T19:29:06.445 INFO:tasks.workunit.client.0.smithi022.stdout:5/474: dwrite dd/d1f/d48/f1c [0,4194304] 0 2024-02-16T19:29:06.446 INFO:tasks.workunit.client.1.smithi117.stdout:7/438: fsync d4/f4d 0 2024-02-16T19:29:06.452 INFO:tasks.workunit.client.1.smithi117.stdout:1/412: rename d2/d1d/d1f/c26 to d2/d1d/d60/d68/c88 0 2024-02-16T19:29:06.453 INFO:tasks.workunit.client.1.smithi117.stdout:1/413: readlink d2/d19/l45 0 2024-02-16T19:29:06.459 INFO:tasks.workunit.client.1.smithi117.stdout:6/445: creat d5/db/d7d/d32/d79/f94 x:0 0 0 2024-02-16T19:29:06.460 INFO:tasks.workunit.client.1.smithi117.stdout:6/446: fdatasync d5/db/d7d/d32/f51 0 2024-02-16T19:29:06.461 INFO:tasks.workunit.client.1.smithi117.stdout:6/447: chown d5/d4b/d44/d52/f6c 95 1 2024-02-16T19:29:06.463 INFO:tasks.workunit.client.1.smithi117.stdout:0/513: symlink d0/d9/d14/d23/d33/lba 0 2024-02-16T19:29:06.466 INFO:tasks.workunit.client.1.smithi117.stdout:4/449: dread d0/da/d21/d3d/d28/d64/d48/f4c [4194304,4194304] 0 2024-02-16T19:29:06.471 INFO:tasks.workunit.client.1.smithi117.stdout:2/433: symlink d0/d3/l87 0 2024-02-16T19:29:06.472 INFO:tasks.workunit.client.1.smithi117.stdout:2/434: stat d0/d13/d4d/d64/d46/f83 0 2024-02-16T19:29:06.474 INFO:tasks.workunit.client.0.smithi022.stdout:4/455: mkdir d4/d38/d8d 0 2024-02-16T19:29:06.477 INFO:tasks.workunit.client.1.smithi117.stdout:5/538: truncate d2/d3/d21/d49/f82 3216964 0 2024-02-16T19:29:06.478 INFO:tasks.workunit.client.1.smithi117.stdout:5/539: write d2/d3/d21/f92 [153353,96127] 0 2024-02-16T19:29:06.479 INFO:tasks.workunit.client.1.smithi117.stdout:7/439: symlink d4/d13/la9 0 2024-02-16T19:29:06.479 INFO:tasks.workunit.client.1.smithi117.stdout:5/540: read - d2/db3/fa0 zero size 2024-02-16T19:29:06.480 INFO:tasks.workunit.client.0.smithi022.stdout:7/499: write d9/d1b/f27 [1360067,89446] 0 2024-02-16T19:29:06.484 INFO:tasks.workunit.client.1.smithi117.stdout:3/557: rename d1/d1b/d1e/f54 to d1/dd/d32/fc3 0 2024-02-16T19:29:06.486 INFO:tasks.workunit.client.0.smithi022.stdout:7/500: dread d9/da/d3b/f41 [0,4194304] 0 2024-02-16T19:29:06.487 INFO:tasks.workunit.client.0.smithi022.stdout:7/501: fdatasync d9/da/d1f/d40/f5e 0 2024-02-16T19:29:06.490 INFO:tasks.workunit.client.1.smithi117.stdout:9/377: truncate dd/d47/d48/f66 3994255 0 2024-02-16T19:29:06.493 INFO:tasks.workunit.client.1.smithi117.stdout:6/448: truncate d5/db/d7d/d32/f43 1289335 0 2024-02-16T19:29:06.496 INFO:tasks.workunit.client.1.smithi117.stdout:0/514: symlink d0/d9/d14/d1d/d60/d76/d7b/lbb 0 2024-02-16T19:29:06.498 INFO:tasks.workunit.client.1.smithi117.stdout:4/450: readlink d0/da/l53 0 2024-02-16T19:29:06.500 INFO:tasks.workunit.client.1.smithi117.stdout:6/449: dwrite d5/d1d/f87 [0,4194304] 0 2024-02-16T19:29:06.504 INFO:tasks.workunit.client.1.smithi117.stdout:0/515: dwrite d0/d9/d14/f1b [0,4194304] 0 2024-02-16T19:29:06.506 INFO:tasks.workunit.client.1.smithi117.stdout:7/440: fdatasync d4/d1c/d39/d46/f57 0 2024-02-16T19:29:06.510 INFO:tasks.workunit.client.1.smithi117.stdout:6/450: dwrite d5/f15 [0,4194304] 0 2024-02-16T19:29:06.510 INFO:tasks.workunit.client.1.smithi117.stdout:8/466: truncate d4/d7/d37/d3a/d65/d6c/f75 3493672 0 2024-02-16T19:29:06.517 INFO:tasks.workunit.client.1.smithi117.stdout:1/414: rename d2/d1d/d60/d20 to d2/d19/d59/d89 0 2024-02-16T19:29:06.518 INFO:tasks.workunit.client.1.smithi117.stdout:8/467: dwrite d4/f52 [0,4194304] 0 2024-02-16T19:29:06.518 INFO:tasks.workunit.client.0.smithi022.stdout:4/456: fsync d4/d6/d14/f2e 0 2024-02-16T19:29:06.524 INFO:tasks.workunit.client.1.smithi117.stdout:9/378: creat dd/d2d/f84 x:0 0 0 2024-02-16T19:29:06.527 INFO:tasks.workunit.client.1.smithi117.stdout:8/468: dwrite d4/f53 [4194304,4194304] 0 2024-02-16T19:29:06.528 INFO:tasks.workunit.client.0.smithi022.stdout:7/502: unlink d9/da/d2c/d6d/f7e 0 2024-02-16T19:29:06.529 INFO:tasks.workunit.client.0.smithi022.stdout:7/503: fdatasync d9/da/d2c/d46/f6f 0 2024-02-16T19:29:06.533 INFO:tasks.workunit.client.0.smithi022.stdout:2/410: mknod d0/d2/d18/c84 0 2024-02-16T19:29:06.535 INFO:tasks.workunit.client.1.smithi117.stdout:4/451: chown d0/da/d21/d3d/c7d 3 1 2024-02-16T19:29:06.535 INFO:tasks.workunit.client.0.smithi022.stdout:2/411: write d0/d2/d18/d19/f30 [1658157,54511] 0 2024-02-16T19:29:06.540 INFO:tasks.workunit.client.0.smithi022.stdout:7/504: dwrite d9/da/d2c/d6d/f8b [0,4194304] 0 2024-02-16T19:29:06.542 INFO:tasks.workunit.client.0.smithi022.stdout:7/505: truncate d9/d5d/f91 773894 0 2024-02-16T19:29:06.544 INFO:tasks.workunit.client.1.smithi117.stdout:3/558: dread d1/dd/d32/fc3 [4194304,4194304] 0 2024-02-16T19:29:06.558 INFO:tasks.workunit.client.1.smithi117.stdout:2/435: write d0/d3/f31 [394852,76552] 0 2024-02-16T19:29:06.558 INFO:tasks.workunit.client.0.smithi022.stdout:8/467: rename d3/la to d3/d12/d17/lb1 0 2024-02-16T19:29:06.560 INFO:tasks.workunit.client.0.smithi022.stdout:9/513: dwrite db/d14/d19/fa2 [0,4194304] 0 2024-02-16T19:29:06.562 INFO:tasks.workunit.client.0.smithi022.stdout:8/468: dread d3/d25/f43 [0,4194304] 0 2024-02-16T19:29:06.562 INFO:tasks.workunit.client.1.smithi117.stdout:6/451: rename d5/d1d to d5/db/d7d/d32/d95 0 2024-02-16T19:29:06.563 INFO:tasks.workunit.client.1.smithi117.stdout:6/452: stat d5/d4b/d44/d52/l65 0 2024-02-16T19:29:06.565 INFO:tasks.workunit.client.0.smithi022.stdout:5/475: truncate dd/d1f/d48/d21/d2f/d52/d5c/f85 557964 0 2024-02-16T19:29:06.566 INFO:tasks.workunit.client.0.smithi022.stdout:2/412: creat d0/d2/f85 x:0 0 0 2024-02-16T19:29:06.568 INFO:tasks.workunit.client.1.smithi117.stdout:1/415: dread - d2/d4/f40 zero size 2024-02-16T19:29:06.568 INFO:tasks.workunit.client.0.smithi022.stdout:2/413: write d0/d2/d18/d19/f2f [4299005,191] 0 2024-02-16T19:29:06.568 INFO:tasks.workunit.client.1.smithi117.stdout:1/416: chown d2/d1d/c6f 248932 1 2024-02-16T19:29:06.569 INFO:tasks.workunit.client.0.smithi022.stdout:2/414: stat d0/d32/d3b/d6f/l81 0 2024-02-16T19:29:06.570 INFO:tasks.workunit.client.0.smithi022.stdout:3/445: link d7/d12/c2b d7/d12/d18/dd/d3c/d5a/cab 0 2024-02-16T19:29:06.572 INFO:tasks.workunit.client.1.smithi117.stdout:9/379: mknod dd/d2d/c85 0 2024-02-16T19:29:06.573 INFO:tasks.workunit.client.1.smithi117.stdout:9/380: dread - dd/d47/f6d zero size 2024-02-16T19:29:06.574 INFO:tasks.workunit.client.1.smithi117.stdout:9/381: write dd/d47/f60 [1974876,22862] 0 2024-02-16T19:29:06.575 INFO:tasks.workunit.client.1.smithi117.stdout:4/452: readlink d0/da/d21/d3d/d28/l38 0 2024-02-16T19:29:06.576 INFO:tasks.workunit.client.0.smithi022.stdout:9/514: creat db/d14/d16/d87/fb0 x:0 0 0 2024-02-16T19:29:06.581 INFO:tasks.workunit.client.1.smithi117.stdout:0/516: mkdir d0/d9/d14/dbc 0 2024-02-16T19:29:06.583 INFO:tasks.workunit.client.1.smithi117.stdout:3/559: mknod d1/dd/d3b/cc4 0 2024-02-16T19:29:06.585 INFO:tasks.workunit.client.0.smithi022.stdout:3/446: fsync f2 0 2024-02-16T19:29:06.590 INFO:tasks.workunit.client.1.smithi117.stdout:8/469: rename d4/d7/d37/d3a/d43/d44/d5f/d81 to d4/d12/d31/d8e 0 2024-02-16T19:29:06.591 INFO:tasks.workunit.client.1.smithi117.stdout:6/453: creat d5/d4b/d20/d2d/f96 x:0 0 0 2024-02-16T19:29:06.596 INFO:tasks.workunit.client.0.smithi022.stdout:2/415: rename d0/l2c to d0/d32/d3b/d6f/l86 0 2024-02-16T19:29:06.597 INFO:tasks.workunit.client.0.smithi022.stdout:2/416: fdatasync d0/f37 0 2024-02-16T19:29:06.601 INFO:tasks.workunit.client.0.smithi022.stdout:8/469: truncate d3/d12/d17/f83 575424 0 2024-02-16T19:29:06.602 INFO:tasks.workunit.client.1.smithi117.stdout:7/441: creat d4/d1c/d87/d89/faa x:0 0 0 2024-02-16T19:29:06.603 INFO:tasks.workunit.client.1.smithi117.stdout:5/541: link d2/d9/d3c/d42/l66 d2/d3/d21/d2f/d30/lb4 0 2024-02-16T19:29:06.604 INFO:tasks.workunit.client.1.smithi117.stdout:1/417: dwrite d2/f53 [0,4194304] 0 2024-02-16T19:29:06.605 INFO:tasks.workunit.client.1.smithi117.stdout:2/436: link d0/d29/f2d d0/d13/d4d/d80/f88 0 2024-02-16T19:29:06.607 INFO:tasks.workunit.client.0.smithi022.stdout:2/417: unlink d0/d32/c35 0 2024-02-16T19:29:06.609 INFO:tasks.workunit.client.0.smithi022.stdout:2/418: fdatasync d0/d2/d18/d19/f30 0 2024-02-16T19:29:06.611 INFO:tasks.workunit.client.1.smithi117.stdout:9/382: rename dd/d47/f60 to dd/d47/d48/f86 0 2024-02-16T19:29:06.611 INFO:tasks.workunit.client.0.smithi022.stdout:2/419: write d0/d32/d3b/d6f/f5a [2693836,86863] 0 2024-02-16T19:29:06.611 INFO:tasks.workunit.client.1.smithi117.stdout:9/383: dread - dd/d47/f79 zero size 2024-02-16T19:29:06.612 INFO:tasks.workunit.client.1.smithi117.stdout:9/384: write dd/f3e [296639,107551] 0 2024-02-16T19:29:06.619 INFO:tasks.workunit.client.0.smithi022.stdout:9/515: creat db/d14/d16/d21/d37/fb1 x:0 0 0 2024-02-16T19:29:06.620 INFO:tasks.workunit.client.0.smithi022.stdout:9/516: chown db/d14/d19/d1a/l47 0 1 2024-02-16T19:29:06.620 INFO:tasks.workunit.client.1.smithi117.stdout:6/454: symlink d5/d4b/d20/l97 0 2024-02-16T19:29:06.621 INFO:tasks.workunit.client.0.smithi022.stdout:9/517: fdatasync db/d14/d19/da8/f3c 0 2024-02-16T19:29:06.621 INFO:tasks.workunit.client.1.smithi117.stdout:6/455: write d5/d4b/f29 [3999723,16562] 0 2024-02-16T19:29:06.622 INFO:tasks.workunit.client.0.smithi022.stdout:8/470: creat d3/d12/d6a/fb2 x:0 0 0 2024-02-16T19:29:06.625 INFO:tasks.workunit.client.0.smithi022.stdout:5/476: getdents dd/d1f/d48/d21/d2f/d52/d5c/d61 0 2024-02-16T19:29:06.626 INFO:tasks.workunit.client.1.smithi117.stdout:0/517: mkdir d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/dbd 0 2024-02-16T19:29:06.626 INFO:tasks.workunit.client.1.smithi117.stdout:0/518: chown d0/ca6 1024130466 1 2024-02-16T19:29:06.629 INFO:tasks.workunit.client.1.smithi117.stdout:3/560: creat d1/d30/d9f/dbc/fc5 x:0 0 0 2024-02-16T19:29:06.633 INFO:tasks.workunit.client.1.smithi117.stdout:7/442: unlink d4/f2b 0 2024-02-16T19:29:06.635 INFO:tasks.workunit.client.1.smithi117.stdout:7/443: truncate d4/d1c/d39/d46/d47/f51 848846 0 2024-02-16T19:29:06.635 INFO:tasks.workunit.client.0.smithi022.stdout:2/420: symlink d0/d2/d18/d1f/l87 0 2024-02-16T19:29:06.636 INFO:tasks.workunit.client.1.smithi117.stdout:1/418: fsync d2/d4/f40 0 2024-02-16T19:29:06.639 INFO:tasks.workunit.client.1.smithi117.stdout:5/542: mkdir d2/d3/d21/d2f/d30/d38/db5 0 2024-02-16T19:29:06.640 INFO:tasks.workunit.client.1.smithi117.stdout:5/543: chown d2/d9/d3c/d24/d50/d52/lc 28111 1 2024-02-16T19:29:06.647 INFO:tasks.workunit.client.0.smithi022.stdout:3/447: rename d7/d12/d18/dd/d31/d41 to d7/d12/d18/dac 0 2024-02-16T19:29:06.651 INFO:tasks.workunit.client.1.smithi117.stdout:4/453: rename d0/da/d21/d3d/d2d to d0/da/d21/d3d/d28/d66/d97 0 2024-02-16T19:29:06.654 INFO:tasks.workunit.client.0.smithi022.stdout:2/421: symlink d0/d5d/l88 0 2024-02-16T19:29:06.660 INFO:tasks.workunit.client.1.smithi117.stdout:9/385: dread dd/f4a [0,4194304] 0 2024-02-16T19:29:06.660 INFO:tasks.workunit.client.1.smithi117.stdout:6/456: write d5/d4b/d20/d2d/f96 [848685,31369] 0 2024-02-16T19:29:06.660 INFO:tasks.workunit.client.1.smithi117.stdout:9/386: chown dd/d23/c38 0 1 2024-02-16T19:29:06.661 INFO:tasks.workunit.client.0.smithi022.stdout:2/422: dwrite d0/d32/f62 [0,4194304] 0 2024-02-16T19:29:06.666 INFO:tasks.workunit.client.1.smithi117.stdout:7/444: readlink d4/d1c/d87/d89/l2a 0 2024-02-16T19:29:06.668 INFO:tasks.workunit.client.0.smithi022.stdout:9/518: link db/d14/d16/d21/l7d db/d14/d16/d49/lb2 0 2024-02-16T19:29:06.669 INFO:tasks.workunit.client.0.smithi022.stdout:9/519: chown db/d18/f6d 2 1 2024-02-16T19:29:06.670 INFO:tasks.workunit.client.0.smithi022.stdout:8/471: truncate d3/d12/da6/f4d 507746 0 2024-02-16T19:29:06.672 INFO:tasks.workunit.client.1.smithi117.stdout:1/419: creat d2/d1d/d60/d68/f8a x:0 0 0 2024-02-16T19:29:06.672 INFO:tasks.workunit.client.0.smithi022.stdout:3/448: write d7/d12/d18/d4b/f6f [1546255,104576] 0 2024-02-16T19:29:06.675 INFO:tasks.workunit.client.0.smithi022.stdout:2/423: mknod d0/d2/d2e/d73/c89 0 2024-02-16T19:29:06.676 INFO:tasks.workunit.client.1.smithi117.stdout:5/544: rmdir d2/d3/d21/d2f/d30/d4e 39 2024-02-16T19:29:06.677 INFO:tasks.workunit.client.1.smithi117.stdout:5/545: readlink d2/d9/l15 0 2024-02-16T19:29:06.678 INFO:tasks.workunit.client.0.smithi022.stdout:9/520: mknod db/d14/d16/d21/cb3 0 2024-02-16T19:29:06.679 INFO:tasks.workunit.client.1.smithi117.stdout:0/519: dwrite d0/fd [0,4194304] 0 2024-02-16T19:29:06.680 INFO:tasks.workunit.client.1.smithi117.stdout:2/437: symlink d0/d13/d4d/d80/l89 0 2024-02-16T19:29:06.680 INFO:tasks.workunit.client.0.smithi022.stdout:9/521: fdatasync db/d14/d16/d87/f8f 0 2024-02-16T19:29:06.680 INFO:tasks.workunit.client.0.smithi022.stdout:2/424: read d0/f17 [2712024,73813] 0 2024-02-16T19:29:06.684 INFO:tasks.workunit.client.1.smithi117.stdout:8/470: rename d4/d7/d37/d3a/d54/c59 to d4/d7/d6a/d72/c8f 0 2024-02-16T19:29:06.686 INFO:tasks.workunit.client.1.smithi117.stdout:4/454: mkdir d0/da/d21/d3d/d28/d64/d73/d91/d37/d98 0 2024-02-16T19:29:06.690 INFO:tasks.workunit.client.1.smithi117.stdout:8/471: dwrite d4/d7/d37/d3a/d43/f55 [0,4194304] 0 2024-02-16T19:29:06.690 INFO:tasks.workunit.client.0.smithi022.stdout:9/522: creat db/d18/fb4 x:0 0 0 2024-02-16T19:29:06.693 INFO:tasks.workunit.client.1.smithi117.stdout:9/387: fsync dd/d45/f49 0 2024-02-16T19:29:06.695 INFO:tasks.workunit.client.0.smithi022.stdout:2/425: write d0/d2/d18/f28 [353555,93043] 0 2024-02-16T19:29:06.697 INFO:tasks.workunit.client.1.smithi117.stdout:7/445: stat d4/d13/f18 0 2024-02-16T19:29:06.699 INFO:tasks.workunit.client.0.smithi022.stdout:9/523: dwrite db/d18/f93 [0,4194304] 0 2024-02-16T19:29:06.705 INFO:tasks.workunit.client.1.smithi117.stdout:1/420: symlink d2/d19/d59/d89/d69/d80/l8b 0 2024-02-16T19:29:06.708 INFO:tasks.workunit.client.0.smithi022.stdout:2/426: dread d0/d2/d2e/f51 [0,4194304] 0 2024-02-16T19:29:06.709 INFO:tasks.workunit.client.0.smithi022.stdout:8/472: rename d3/d12/d2b/d2e/c2a to d3/d12/d2b/da9/cb3 0 2024-02-16T19:29:06.714 INFO:tasks.workunit.client.0.smithi022.stdout:9/524: symlink db/d14/d33/lb5 0 2024-02-16T19:29:06.714 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:06 smithi022 ceph-mon[29221]: pgmap v6: 65 pgs: 65 active+clean; 1.5 GiB data, 4.5 GiB used, 532 GiB / 536 GiB avail; 10 MiB/s rd, 25 MiB/s wr, 84 op/s 2024-02-16T19:29:06.715 INFO:tasks.workunit.client.0.smithi022.stdout:9/525: read db/d18/f93 [1873976,46453] 0 2024-02-16T19:29:06.718 INFO:tasks.workunit.client.1.smithi117.stdout:5/546: dread d2/d3/f6 [0,4194304] 0 2024-02-16T19:29:06.723 INFO:tasks.workunit.client.0.smithi022.stdout:8/473: mknod d3/d12/d92/cb4 0 2024-02-16T19:29:06.725 INFO:tasks.workunit.client.1.smithi117.stdout:8/472: creat d4/d7/f90 x:0 0 0 2024-02-16T19:29:06.726 INFO:tasks.workunit.client.1.smithi117.stdout:8/473: dread d4/d1c/f49 [0,4194304] 0 2024-02-16T19:29:06.730 INFO:tasks.workunit.client.0.smithi022.stdout:9/526: rename db/d14/d16/f45 to db/d14/d16/d21/fb6 0 2024-02-16T19:29:06.732 INFO:tasks.workunit.client.1.smithi117.stdout:9/388: readlink dd/d23/l41 0 2024-02-16T19:29:06.733 INFO:tasks.workunit.client.0.smithi022.stdout:2/427: mkdir d0/d2/d2e/d82/d8a 0 2024-02-16T19:29:06.734 INFO:tasks.workunit.client.1.smithi117.stdout:4/455: dread d0/da/d21/d3d/d28/d64/d48/f4c [0,4194304] 0 2024-02-16T19:29:06.737 INFO:tasks.workunit.client.0.smithi022.stdout:8/474: truncate d3/d12/d2b/d54/d74/f84 65502 0 2024-02-16T19:29:06.742 INFO:tasks.workunit.client.1.smithi117.stdout:2/438: truncate d0/d13/d4d/d80/f88 3637443 0 2024-02-16T19:29:06.742 INFO:tasks.workunit.client.1.smithi117.stdout:6/457: truncate d5/d4b/d44/f56 991249 0 2024-02-16T19:29:06.744 INFO:tasks.workunit.client.1.smithi117.stdout:3/561: rename d1/d30/cb8 to d1/d30/d9f/cc6 0 2024-02-16T19:29:06.745 INFO:tasks.workunit.client.0.smithi022.stdout:9/527: symlink db/d14/d16/d5d/d65/d6b/d89/lb7 0 2024-02-16T19:29:06.745 INFO:tasks.workunit.client.1.smithi117.stdout:5/547: fdatasync d2/d9/d3c/d24/d50/d52/fa5 0 2024-02-16T19:29:06.745 INFO:tasks.workunit.client.1.smithi117.stdout:3/562: read - d1/d83/d96/fb6 zero size 2024-02-16T19:29:06.746 INFO:tasks.workunit.client.0.smithi022.stdout:2/428: creat d0/d2/d18/d38/d6b/f8b x:0 0 0 2024-02-16T19:29:06.747 INFO:tasks.workunit.client.1.smithi117.stdout:5/548: chown d2/d3/d21/d49/d59 297 1 2024-02-16T19:29:06.747 INFO:tasks.workunit.client.1.smithi117.stdout:7/446: dwrite d4/f5 [0,4194304] 0 2024-02-16T19:29:06.748 INFO:tasks.workunit.client.0.smithi022.stdout:8/475: symlink d3/d12/d6a/lb5 0 2024-02-16T19:29:06.750 INFO:tasks.workunit.client.1.smithi117.stdout:9/389: creat dd/d47/d5e/f87 x:0 0 0 2024-02-16T19:29:06.751 INFO:tasks.workunit.client.0.smithi022.stdout:8/476: rmdir d3/d12 39 2024-02-16T19:29:06.756 INFO:tasks.workunit.client.1.smithi117.stdout:4/456: dwrite d0/da/d21/d3d/d28/d66/f7f [0,4194304] 0 2024-02-16T19:29:06.758 INFO:tasks.workunit.client.0.smithi022.stdout:8/477: write d3/d12/da6/f7 [2307202,114946] 0 2024-02-16T19:29:06.763 INFO:tasks.workunit.client.0.smithi022.stdout:2/429: fsync d0/d2/d18/d38/d6b/f8b 0 2024-02-16T19:29:06.765 INFO:tasks.workunit.client.1.smithi117.stdout:8/474: write d4/d12/d2a/f5c [447390,58689] 0 2024-02-16T19:29:06.768 INFO:tasks.workunit.client.0.smithi022.stdout:2/430: symlink d0/d32/d3b/l8c 0 2024-02-16T19:29:06.772 INFO:tasks.workunit.client.0.smithi022.stdout:2/431: symlink d0/d2/d18/d19/d40/l8d 0 2024-02-16T19:29:06.775 INFO:tasks.workunit.client.0.smithi022.stdout:8/478: creat d3/d12/d17/d64/d8f/fb6 x:0 0 0 2024-02-16T19:29:06.778 INFO:tasks.workunit.client.0.smithi022.stdout:2/432: unlink d0/d2/f3c 0 2024-02-16T19:29:06.780 INFO:tasks.workunit.client.0.smithi022.stdout:8/479: creat d3/d12/fb7 x:0 0 0 2024-02-16T19:29:06.783 INFO:tasks.workunit.client.0.smithi022.stdout:2/433: mknod d0/d2/d18/c8e 0 2024-02-16T19:29:06.785 INFO:tasks.workunit.client.0.smithi022.stdout:1/583: sync 2024-02-16T19:29:06.785 INFO:tasks.workunit.client.0.smithi022.stdout:1/584: fsync d6/d8/d22/f9a 0 2024-02-16T19:29:06.788 INFO:tasks.workunit.client.0.smithi022.stdout:2/434: creat d0/d2/f8f x:0 0 0 2024-02-16T19:29:06.790 INFO:tasks.workunit.client.0.smithi022.stdout:1/585: symlink d6/d8/d22/d16/d17/d26/lb2 0 2024-02-16T19:29:06.793 INFO:tasks.workunit.client.0.smithi022.stdout:2/435: mkdir d0/d2/d18/d19/d90 0 2024-02-16T19:29:06.796 INFO:tasks.workunit.client.0.smithi022.stdout:1/586: creat d6/d66/fb3 x:0 0 0 2024-02-16T19:29:06.797 INFO:tasks.workunit.client.0.smithi022.stdout:8/480: getdents d3/d12/d2b/d54 0 2024-02-16T19:29:06.801 INFO:tasks.workunit.client.0.smithi022.stdout:8/481: chown d3/d12/d2b/f6c 3344396 1 2024-02-16T19:29:06.804 INFO:tasks.workunit.client.0.smithi022.stdout:2/436: creat d0/d2/d2e/d82/d8a/f91 x:0 0 0 2024-02-16T19:29:06.812 INFO:tasks.workunit.client.1.smithi117.stdout:1/421: mkdir d2/d19/d59/d76/d8c 0 2024-02-16T19:29:06.813 INFO:tasks.workunit.client.0.smithi022.stdout:1/587: dread d6/d8/d22/f28 [0,4194304] 0 2024-02-16T19:29:06.813 INFO:tasks.workunit.client.1.smithi117.stdout:1/422: write d2/d51/f75 [1147022,116826] 0 2024-02-16T19:29:06.814 INFO:tasks.workunit.client.1.smithi117.stdout:2/439: creat d0/d13/d4d/d64/d46/f8a x:0 0 0 2024-02-16T19:29:06.815 INFO:tasks.workunit.client.0.smithi022.stdout:8/482: dread d3/d25/f5c [0,4194304] 0 2024-02-16T19:29:06.827 INFO:tasks.workunit.client.1.smithi117.stdout:0/520: sync 2024-02-16T19:29:06.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:06 smithi117 ceph-mon[35309]: pgmap v6: 65 pgs: 65 active+clean; 1.5 GiB data, 4.5 GiB used, 532 GiB / 536 GiB avail; 10 MiB/s rd, 25 MiB/s wr, 84 op/s 2024-02-16T19:29:06.899 INFO:tasks.workunit.client.1.smithi117.stdout:8/475: sync 2024-02-16T19:29:06.919 INFO:tasks.workunit.client.1.smithi117.stdout:2/440: sync 2024-02-16T19:29:06.924 INFO:tasks.workunit.client.1.smithi117.stdout:6/458: symlink d5/db/d7d/d32/l98 0 2024-02-16T19:29:06.925 INFO:tasks.workunit.client.0.smithi022.stdout:1/588: mknod d6/d8/d22/d16/d17/d5d/cb4 0 2024-02-16T19:29:07.000 INFO:tasks.workunit.client.0.smithi022.stdout:6/474: sync 2024-02-16T19:29:07.000 INFO:tasks.workunit.client.0.smithi022.stdout:0/601: sync 2024-02-16T19:29:07.003 INFO:tasks.workunit.client.0.smithi022.stdout:8/483: mkdir d3/d12/d2b/d54/db8 0 2024-02-16T19:29:07.005 INFO:tasks.workunit.client.1.smithi117.stdout:3/563: unlink d1/d1b/d1e/d29/l8d 0 2024-02-16T19:29:07.006 INFO:tasks.workunit.client.0.smithi022.stdout:2/437: getdents d0/d2/d18/d38 0 2024-02-16T19:29:07.010 INFO:tasks.workunit.client.0.smithi022.stdout:0/602: unlink d10/d11/d27/d3f/f57 0 2024-02-16T19:29:07.012 INFO:tasks.workunit.client.0.smithi022.stdout:6/475: symlink d1/d8/da/d17/d53/d7f/la9 0 2024-02-16T19:29:07.022 INFO:tasks.workunit.client.1.smithi117.stdout:1/423: fsync d2/d19/d59/d89/d49/f4a 0 2024-02-16T19:29:07.023 INFO:tasks.workunit.client.1.smithi117.stdout:9/390: dread dd/d47/d48/f66 [0,4194304] 0 2024-02-16T19:29:07.026 INFO:tasks.workunit.client.1.smithi117.stdout:0/521: fsync d0/f1 0 2024-02-16T19:29:07.027 INFO:tasks.workunit.client.1.smithi117.stdout:0/522: truncate d0/d9/d14/d1d/d60/d76/f9f 402143 0 2024-02-16T19:29:07.028 INFO:tasks.workunit.client.0.smithi022.stdout:6/476: mknod d1/d8/d23/d95/caa 0 2024-02-16T19:29:07.029 INFO:tasks.workunit.client.0.smithi022.stdout:6/477: chown d1/d8/da/d17/d53/l5a 438 1 2024-02-16T19:29:07.031 INFO:tasks.workunit.client.1.smithi117.stdout:2/441: symlink d0/d3b/l8b 0 2024-02-16T19:29:07.033 INFO:tasks.workunit.client.1.smithi117.stdout:2/442: dread - d0/d13/d4d/d64/d46/f83 zero size 2024-02-16T19:29:07.033 INFO:tasks.workunit.client.1.smithi117.stdout:5/549: dwrite d2/db3/fa0 [0,4194304] 0 2024-02-16T19:29:07.034 INFO:tasks.workunit.client.1.smithi117.stdout:6/459: creat d5/db/d7d/d32/d95/d49/d59/f99 x:0 0 0 2024-02-16T19:29:07.034 INFO:tasks.workunit.client.1.smithi117.stdout:5/550: read - d2/d9/d3c/f8d zero size 2024-02-16T19:29:07.038 INFO:tasks.workunit.client.1.smithi117.stdout:6/460: dread d5/d4b/d44/d52/f6c [0,4194304] 0 2024-02-16T19:29:07.039 INFO:tasks.workunit.client.1.smithi117.stdout:4/457: dwrite d0/da/d21/f40 [0,4194304] 0 2024-02-16T19:29:07.040 INFO:tasks.workunit.client.0.smithi022.stdout:2/438: getdents d0/d2/d2e 0 2024-02-16T19:29:07.043 INFO:tasks.workunit.client.0.smithi022.stdout:0/603: truncate d10/d11/d50/d65/f68 322638 0 2024-02-16T19:29:07.043 INFO:tasks.workunit.client.1.smithi117.stdout:1/424: mkdir d2/d19/d59/d89/d8d 0 2024-02-16T19:29:07.049 INFO:tasks.workunit.client.0.smithi022.stdout:6/478: getdents d1/d8 0 2024-02-16T19:29:07.051 INFO:tasks.workunit.client.0.smithi022.stdout:0/604: dread d10/d15/d23/d38/f9e [0,4194304] 0 2024-02-16T19:29:07.055 INFO:tasks.workunit.client.1.smithi117.stdout:0/523: rename d0/d9/d14/d1d/d60/d76/d7b/d8d/c4b to d0/d9/d22/d27/d4e/cbe 0 2024-02-16T19:29:07.055 INFO:tasks.workunit.client.1.smithi117.stdout:8/476: truncate d4/f6 4161986 0 2024-02-16T19:29:07.063 INFO:tasks.workunit.client.1.smithi117.stdout:7/447: truncate d4/d1c/d87/d89/d5f/f7b 2189481 0 2024-02-16T19:29:07.064 INFO:tasks.workunit.client.1.smithi117.stdout:7/448: dread - d4/d1c/d87/d89/d60/d9b/fa8 zero size 2024-02-16T19:29:07.064 INFO:tasks.workunit.client.1.smithi117.stdout:2/443: dread d0/f25 [0,4194304] 0 2024-02-16T19:29:07.064 INFO:tasks.workunit.client.1.smithi117.stdout:7/449: dread - d4/d13/f8d zero size 2024-02-16T19:29:07.065 INFO:tasks.workunit.client.0.smithi022.stdout:2/439: write d0/d2/f3d [1152704,1523] 0 2024-02-16T19:29:07.066 INFO:tasks.workunit.client.0.smithi022.stdout:0/605: mkdir d10/d15/d23/dc6 0 2024-02-16T19:29:07.069 INFO:tasks.workunit.client.1.smithi117.stdout:9/391: dwrite dd/d47/d48/f66 [0,4194304] 0 2024-02-16T19:29:07.076 INFO:tasks.workunit.client.1.smithi117.stdout:9/392: dwrite dd/d47/d5e/f7f [0,4194304] 0 2024-02-16T19:29:07.080 INFO:tasks.workunit.client.1.smithi117.stdout:1/425: mkdir d2/d19/d59/d89/d69/d80/d8e 0 2024-02-16T19:29:07.081 INFO:tasks.workunit.client.1.smithi117.stdout:1/426: chown d2/d1d/d1f/f5e 51886 1 2024-02-16T19:29:07.085 INFO:tasks.workunit.client.0.smithi022.stdout:6/479: getdents d1/d8/da/d29/d61/d57 0 2024-02-16T19:29:07.093 INFO:tasks.workunit.client.0.smithi022.stdout:6/480: creat d1/d8/da/d17/d85/d56/fab x:0 0 0 2024-02-16T19:29:07.095 INFO:tasks.workunit.client.0.smithi022.stdout:2/440: write d0/d2/f1c [1759404,39732] 0 2024-02-16T19:29:07.098 INFO:tasks.workunit.client.1.smithi117.stdout:2/444: unlink d0/d3/f3a 0 2024-02-16T19:29:07.100 INFO:tasks.workunit.client.1.smithi117.stdout:7/450: rename d4/d13/f8d to d4/d1c/d87/d89/d60/fab 0 2024-02-16T19:29:07.101 INFO:tasks.workunit.client.0.smithi022.stdout:0/606: getdents d10/d22/d2c 0 2024-02-16T19:29:07.101 INFO:tasks.workunit.client.1.smithi117.stdout:3/564: link d1/d1b/l4d d1/dd/d3e/d7d/lc7 0 2024-02-16T19:29:07.103 INFO:tasks.workunit.client.1.smithi117.stdout:2/445: dwrite d0/d3/d40/f55 [0,4194304] 0 2024-02-16T19:29:07.103 INFO:tasks.workunit.client.0.smithi022.stdout:2/441: symlink d0/d5d/l92 0 2024-02-16T19:29:07.109 INFO:tasks.workunit.client.1.smithi117.stdout:4/458: creat d0/da/d21/d3d/d28/d64/d73/d91/d37/d98/f99 x:0 0 0 2024-02-16T19:29:07.111 INFO:tasks.workunit.client.1.smithi117.stdout:2/446: dwrite d0/d3/d22/f84 [0,4194304] 0 2024-02-16T19:29:07.111 INFO:tasks.workunit.client.1.smithi117.stdout:2/447: dread - d0/d3/f7c zero size 2024-02-16T19:29:07.112 INFO:tasks.workunit.client.0.smithi022.stdout:6/481: getdents d1/d8/da/d17/d85/d9c 0 2024-02-16T19:29:07.112 INFO:tasks.workunit.client.0.smithi022.stdout:0/607: mknod d10/d15/d23/d38/d5a/d66/cc7 0 2024-02-16T19:29:07.112 INFO:tasks.workunit.client.0.smithi022.stdout:2/442: mknod d0/d2/c93 0 2024-02-16T19:29:07.113 INFO:tasks.workunit.client.0.smithi022.stdout:2/443: truncate d0/f20 2083228 0 2024-02-16T19:29:07.113 INFO:tasks.workunit.client.1.smithi117.stdout:9/393: creat dd/d45/d73/f88 x:0 0 0 2024-02-16T19:29:07.114 INFO:tasks.workunit.client.0.smithi022.stdout:0/608: mknod d10/d11/d27/d3f/d94/cc8 0 2024-02-16T19:29:07.116 INFO:tasks.workunit.client.0.smithi022.stdout:4/457: sync 2024-02-16T19:29:07.117 INFO:tasks.workunit.client.1.smithi117.stdout:1/427: creat d2/d19/d59/d89/d69/f8f x:0 0 0 2024-02-16T19:29:07.118 INFO:tasks.workunit.client.0.smithi022.stdout:4/458: dread - d4/d6/d14/d64/d66/d6d/f89 zero size 2024-02-16T19:29:07.119 INFO:tasks.workunit.client.1.smithi117.stdout:1/428: write d2/d51/f75 [4798413,44856] 0 2024-02-16T19:29:07.120 INFO:tasks.workunit.client.0.smithi022.stdout:0/609: creat d10/d11/fc9 x:0 0 0 2024-02-16T19:29:07.120 INFO:tasks.workunit.client.1.smithi117.stdout:4/459: dwrite d0/da/d21/d3d/f5e [0,4194304] 0 2024-02-16T19:29:07.121 INFO:tasks.workunit.client.1.smithi117.stdout:4/460: chown d0/da/d21/d3d/d28/d64 6156 1 2024-02-16T19:29:07.122 INFO:tasks.workunit.client.1.smithi117.stdout:0/524: mknod d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/dbd/cbf 0 2024-02-16T19:29:07.127 INFO:tasks.workunit.client.1.smithi117.stdout:5/551: creat d2/d3/fb6 x:0 0 0 2024-02-16T19:29:07.129 INFO:tasks.workunit.client.0.smithi022.stdout:0/610: rmdir d10/d11/d79 39 2024-02-16T19:29:07.131 INFO:tasks.workunit.client.1.smithi117.stdout:6/461: truncate d5/db/f55 3768356 0 2024-02-16T19:29:07.131 INFO:tasks.workunit.client.1.smithi117.stdout:6/462: readlink d5/d4b/d44/d52/l61 0 2024-02-16T19:29:07.132 INFO:tasks.workunit.client.1.smithi117.stdout:6/463: readlink d5/db/l76 0 2024-02-16T19:29:07.134 INFO:tasks.workunit.client.1.smithi117.stdout:7/451: symlink d4/d1c/d39/d46/lac 0 2024-02-16T19:29:07.135 INFO:tasks.workunit.client.0.smithi022.stdout:0/611: mkdir d10/d22/d49/dca 0 2024-02-16T19:29:07.137 INFO:tasks.workunit.client.0.smithi022.stdout:6/482: getdents d1/d8/da 0 2024-02-16T19:29:07.141 INFO:tasks.workunit.client.1.smithi117.stdout:9/394: mknod dd/d47/d5e/c89 0 2024-02-16T19:29:07.144 INFO:tasks.workunit.client.1.smithi117.stdout:2/448: rename d0/d13/d4d/d64/c7f to d0/d13/d4d/d64/d46/d68/c8c 0 2024-02-16T19:29:07.149 INFO:tasks.workunit.client.1.smithi117.stdout:8/477: symlink d4/d7/d37/d3a/d54/d70/d76/l91 0 2024-02-16T19:29:07.151 INFO:tasks.workunit.client.0.smithi022.stdout:0/612: rmdir d10/d22/d49/dca 0 2024-02-16T19:29:07.153 INFO:tasks.workunit.client.1.smithi117.stdout:1/429: dread d2/d19/d59/d89/f64 [0,4194304] 0 2024-02-16T19:29:07.156 INFO:tasks.workunit.client.1.smithi117.stdout:5/552: fdatasync d2/d4c/d69/f72 0 2024-02-16T19:29:07.158 INFO:tasks.workunit.client.0.smithi022.stdout:4/459: dwrite d4/d6/dd/df/d10/f19 [0,4194304] 0 2024-02-16T19:29:07.159 INFO:tasks.workunit.client.1.smithi117.stdout:6/464: chown d5/db/f4a 1792329 1 2024-02-16T19:29:07.160 INFO:tasks.workunit.client.1.smithi117.stdout:6/465: read d5/d4b/f46 [223939,65574] 0 2024-02-16T19:29:07.161 INFO:tasks.workunit.client.0.smithi022.stdout:6/483: dread d1/d8/d3c/d4d/f45 [0,4194304] 0 2024-02-16T19:29:07.162 INFO:tasks.workunit.client.0.smithi022.stdout:0/613: dwrite d10/d11/d27/d77/f8e [0,4194304] 0 2024-02-16T19:29:07.162 INFO:tasks.workunit.client.0.smithi022.stdout:2/444: truncate d0/d2/d18/d19/f2f 3078032 0 2024-02-16T19:29:07.164 INFO:tasks.workunit.client.0.smithi022.stdout:4/460: dread d4/d6/dd/df/d10/f35 [0,4194304] 0 2024-02-16T19:29:07.167 INFO:tasks.workunit.client.0.smithi022.stdout:4/461: stat d4/d6/d14/d64/d4e/d54/d5a/d63/c77 0 2024-02-16T19:29:07.167 INFO:tasks.workunit.client.1.smithi117.stdout:9/395: chown dd/d23/d64/c4b 434395 1 2024-02-16T19:29:07.169 INFO:tasks.workunit.client.0.smithi022.stdout:0/614: read d10/d11/d79/d8b/fa5 [723265,33546] 0 2024-02-16T19:29:07.171 INFO:tasks.workunit.client.1.smithi117.stdout:2/449: mknod d0/d29/d56/c8d 0 2024-02-16T19:29:07.172 INFO:tasks.workunit.client.1.smithi117.stdout:2/450: write d0/d3/d22/f7b [257102,81636] 0 2024-02-16T19:29:07.173 INFO:tasks.workunit.client.1.smithi117.stdout:2/451: dread - d0/d13/d4d/d53/f6f zero size 2024-02-16T19:29:07.173 INFO:tasks.workunit.client.0.smithi022.stdout:6/484: mkdir d1/d8/dac 0 2024-02-16T19:29:07.178 INFO:tasks.workunit.client.1.smithi117.stdout:8/478: readlink d4/d7/l20 0 2024-02-16T19:29:07.179 INFO:tasks.workunit.client.1.smithi117.stdout:4/461: write d0/da/d21/d3d/d28/d64/d48/f4c [6654173,103443] 0 2024-02-16T19:29:07.184 INFO:tasks.workunit.client.1.smithi117.stdout:0/525: dwrite d0/d9/d22/d27/f5f [0,4194304] 0 2024-02-16T19:29:07.184 INFO:tasks.workunit.client.0.smithi022.stdout:0/615: dread d10/d11/d27/d3f/f7a [0,4194304] 0 2024-02-16T19:29:07.189 INFO:tasks.workunit.client.1.smithi117.stdout:5/553: symlink d2/d9/d3c/d24/d50/d52/d74/d8c/lb7 0 2024-02-16T19:29:07.189 INFO:tasks.workunit.client.0.smithi022.stdout:2/445: write d0/f5f [1996571,106230] 0 2024-02-16T19:29:07.190 INFO:tasks.workunit.client.1.smithi117.stdout:0/526: dwrite d0/f40 [0,4194304] 0 2024-02-16T19:29:07.196 INFO:tasks.workunit.client.0.smithi022.stdout:6/485: dread d1/d8/da/d17/d85/d56/f90 [0,4194304] 0 2024-02-16T19:29:07.198 INFO:tasks.workunit.client.1.smithi117.stdout:3/565: link d1/dd/d3b/lac d1/dd/d3b/dba/lc8 0 2024-02-16T19:29:07.201 INFO:tasks.workunit.client.0.smithi022.stdout:2/446: creat d0/d32/f94 x:0 0 0 2024-02-16T19:29:07.202 INFO:tasks.workunit.client.1.smithi117.stdout:9/396: creat dd/d45/d73/f8a x:0 0 0 2024-02-16T19:29:07.202 INFO:tasks.workunit.client.0.smithi022.stdout:4/462: link d4/d6/d14/d64/l67 d4/d6/d3a/l8e 0 2024-02-16T19:29:07.204 INFO:tasks.workunit.client.1.smithi117.stdout:2/452: creat d0/d13/d4d/d64/d46/d68/f8e x:0 0 0 2024-02-16T19:29:07.205 INFO:tasks.workunit.client.1.smithi117.stdout:2/453: write d0/f5e [88637,71855] 0 2024-02-16T19:29:07.205 INFO:tasks.workunit.client.1.smithi117.stdout:8/479: truncate d4/f3f 1048724 0 2024-02-16T19:29:07.207 INFO:tasks.workunit.client.0.smithi022.stdout:0/616: mknod d10/d11/d27/d3f/d58/d60/ccb 0 2024-02-16T19:29:07.209 INFO:tasks.workunit.client.0.smithi022.stdout:2/447: symlink d0/d32/d3b/l95 0 2024-02-16T19:29:07.209 INFO:tasks.workunit.client.1.smithi117.stdout:9/397: dwrite dd/d47/f7e [0,4194304] 0 2024-02-16T19:29:07.210 INFO:tasks.workunit.client.0.smithi022.stdout:2/448: dread - d0/d2/d18/d19/d40/d55/d59/f74 zero size 2024-02-16T19:29:07.211 INFO:tasks.workunit.client.1.smithi117.stdout:4/462: symlink d0/da/d21/d3d/d28/d64/d73/d91/d37/l9a 0 2024-02-16T19:29:07.214 INFO:tasks.workunit.client.0.smithi022.stdout:6/486: mkdir d1/d8/da/d17/d53/d7f/da3/dad 0 2024-02-16T19:29:07.216 INFO:tasks.workunit.client.0.smithi022.stdout:0/617: symlink d10/d15/d23/d8f/lcc 0 2024-02-16T19:29:07.216 INFO:tasks.workunit.client.0.smithi022.stdout:6/487: chown d1/f2 1037929169 1 2024-02-16T19:29:07.217 INFO:tasks.workunit.client.0.smithi022.stdout:4/463: mknod d4/d38/d6a/c8f 0 2024-02-16T19:29:07.218 INFO:tasks.workunit.client.1.smithi117.stdout:5/554: creat d2/d3/d21/d49/d59/fb8 x:0 0 0 2024-02-16T19:29:07.220 INFO:tasks.workunit.client.0.smithi022.stdout:4/464: fdatasync d4/d6/dd/d22/f51 0 2024-02-16T19:29:07.220 INFO:tasks.workunit.client.0.smithi022.stdout:2/449: dwrite d0/d2/f57 [0,4194304] 0 2024-02-16T19:29:07.221 INFO:tasks.workunit.client.0.smithi022.stdout:4/465: fsync d4/d6/f87 0 2024-02-16T19:29:07.221 INFO:tasks.workunit.client.0.smithi022.stdout:2/450: dread - d0/f78 zero size 2024-02-16T19:29:07.222 INFO:tasks.workunit.client.0.smithi022.stdout:6/488: truncate d1/d8/da/d17/f99 921348 0 2024-02-16T19:29:07.223 INFO:tasks.workunit.client.0.smithi022.stdout:0/618: symlink d10/d15/d23/d38/lcd 0 2024-02-16T19:29:07.231 INFO:tasks.workunit.client.1.smithi117.stdout:7/452: truncate d4/f10 3513431 0 2024-02-16T19:29:07.231 INFO:tasks.workunit.client.1.smithi117.stdout:7/453: chown d4/f95 251151 1 2024-02-16T19:29:07.231 INFO:tasks.workunit.client.0.smithi022.stdout:4/466: dwrite d4/d6/dd/d22/f59 [0,4194304] 0 2024-02-16T19:29:07.231 INFO:tasks.workunit.client.0.smithi022.stdout:7/506: sync 2024-02-16T19:29:07.233 INFO:tasks.workunit.client.0.smithi022.stdout:4/467: chown d4/d6/d14/d64/d4e/f8b 60449 1 2024-02-16T19:29:07.234 INFO:tasks.workunit.client.0.smithi022.stdout:4/468: chown d4/d6/l4c 42553454 1 2024-02-16T19:29:07.242 INFO:tasks.workunit.client.1.smithi117.stdout:2/454: rename d0/d3/d22/f5a to d0/d13/d4d/d64/d46/d68/f8f 0 2024-02-16T19:29:07.244 INFO:tasks.workunit.client.0.smithi022.stdout:6/489: truncate d1/f32 882066 0 2024-02-16T19:29:07.247 INFO:tasks.workunit.client.1.smithi117.stdout:5/555: fsync d2/d5d/f7d 0 2024-02-16T19:29:07.248 INFO:tasks.workunit.client.1.smithi117.stdout:5/556: stat d2/d9/d3c/d24/f26 0 2024-02-16T19:29:07.248 INFO:tasks.workunit.client.0.smithi022.stdout:7/507: dread d9/da/d2c/f8d [0,4194304] 0 2024-02-16T19:29:07.249 INFO:tasks.workunit.client.0.smithi022.stdout:4/469: rmdir d4/d6/d3a 39 2024-02-16T19:29:07.251 INFO:tasks.workunit.client.1.smithi117.stdout:0/527: symlink d0/d9/d22/d27/d83/lc0 0 2024-02-16T19:29:07.251 INFO:tasks.workunit.client.0.smithi022.stdout:2/451: mknod d0/d2/d18/d19/d90/c96 0 2024-02-16T19:29:07.259 INFO:tasks.workunit.client.0.smithi022.stdout:6/490: fdatasync d1/d8/da/d17/d85/d5b/f6c 0 2024-02-16T19:29:07.259 INFO:tasks.workunit.client.1.smithi117.stdout:3/566: dwrite d1/d98/fa5 [0,4194304] 0 2024-02-16T19:29:07.259 INFO:tasks.workunit.client.1.smithi117.stdout:7/454: creat d4/d85/fad x:0 0 0 2024-02-16T19:29:07.264 INFO:tasks.workunit.client.0.smithi022.stdout:7/508: creat d9/da/d1f/d40/fa5 x:0 0 0 2024-02-16T19:29:07.266 INFO:tasks.workunit.client.0.smithi022.stdout:7/509: dread d9/da/d2c/f8d [0,4194304] 0 2024-02-16T19:29:07.267 INFO:tasks.workunit.client.1.smithi117.stdout:9/398: rename dd/d23/f68 to dd/d47/d5b/d5f/d77/d7c/f8b 0 2024-02-16T19:29:07.268 INFO:tasks.workunit.client.0.smithi022.stdout:4/470: readlink d4/d6/d3a/l75 0 2024-02-16T19:29:07.269 INFO:tasks.workunit.client.0.smithi022.stdout:0/619: truncate d10/d11/d27/d3f/f8a 343605 0 2024-02-16T19:29:07.272 INFO:tasks.workunit.client.1.smithi117.stdout:4/463: creat d0/da/d21/d3d/d28/d64/d90/f9b x:0 0 0 2024-02-16T19:29:07.272 INFO:tasks.workunit.client.1.smithi117.stdout:4/464: readlink d0/da/l53 0 2024-02-16T19:29:07.273 INFO:tasks.workunit.client.1.smithi117.stdout:5/557: mknod d2/d5d/d8e/cb9 0 2024-02-16T19:29:07.273 INFO:tasks.workunit.client.1.smithi117.stdout:4/465: chown d0/da/d21/d3d/d28/d66/d97/d6d/c84 4143 1 2024-02-16T19:29:07.274 INFO:tasks.workunit.client.1.smithi117.stdout:6/466: getdents d5/d4b/d20 0 2024-02-16T19:29:07.275 INFO:tasks.workunit.client.1.smithi117.stdout:5/558: write d2/db3/f48 [832630,21762] 0 2024-02-16T19:29:07.275 INFO:tasks.workunit.client.1.smithi117.stdout:6/467: chown d5/db/c66 12731268 1 2024-02-16T19:29:07.276 INFO:tasks.workunit.client.1.smithi117.stdout:4/466: fdatasync d0/da/d21/d3d/d28/d66/d97/d6d/f95 0 2024-02-16T19:29:07.277 INFO:tasks.workunit.client.0.smithi022.stdout:7/510: write d9/da/d3b/f41 [1751128,120904] 0 2024-02-16T19:29:07.279 INFO:tasks.workunit.client.1.smithi117.stdout:3/567: rename d1/d30/d52/fc0 to d1/d30/d52/fc9 0 2024-02-16T19:29:07.284 INFO:tasks.workunit.client.0.smithi022.stdout:7/511: dwrite d9/da/d3b/f9f [0,4194304] 0 2024-02-16T19:29:07.286 INFO:tasks.workunit.client.0.smithi022.stdout:6/491: mkdir d1/d8/da/d7b/dae 0 2024-02-16T19:29:07.290 INFO:tasks.workunit.client.0.smithi022.stdout:8/484: sync 2024-02-16T19:29:07.290 INFO:tasks.workunit.client.0.smithi022.stdout:3/449: sync 2024-02-16T19:29:07.290 INFO:tasks.workunit.client.0.smithi022.stdout:9/528: sync 2024-02-16T19:29:07.290 INFO:tasks.workunit.client.0.smithi022.stdout:5/477: sync 2024-02-16T19:29:07.290 INFO:tasks.workunit.client.0.smithi022.stdout:1/589: sync 2024-02-16T19:29:07.292 INFO:tasks.workunit.client.0.smithi022.stdout:4/471: mknod d4/d6/d14/d64/d66/d86/c90 0 2024-02-16T19:29:07.292 INFO:tasks.workunit.client.0.smithi022.stdout:5/478: chown dd/f1d 203371304 1 2024-02-16T19:29:07.293 INFO:tasks.workunit.client.0.smithi022.stdout:8/485: readlink d3/d12/da6/l57 0 2024-02-16T19:29:07.295 INFO:tasks.workunit.client.0.smithi022.stdout:4/472: write d4/f23 [2183399,42135] 0 2024-02-16T19:29:07.298 INFO:tasks.workunit.client.1.smithi117.stdout:7/455: write d4/d13/d63/f6b [377092,34948] 0 2024-02-16T19:29:07.298 INFO:tasks.workunit.client.0.smithi022.stdout:0/620: rename d10/d11/d27/d3f/d58/d60/c7e to d10/d15/d23/dc6/cce 0 2024-02-16T19:29:07.299 INFO:tasks.workunit.client.1.smithi117.stdout:5/559: readlink d2/d3/d21/d49/l39 0 2024-02-16T19:29:07.300 INFO:tasks.workunit.client.1.smithi117.stdout:7/456: write d4/d1c/d39/d46/d47/f56 [5225482,103592] 0 2024-02-16T19:29:07.302 INFO:tasks.workunit.client.0.smithi022.stdout:7/512: read - d9/da/d2c/d2f/f3d zero size 2024-02-16T19:29:07.303 INFO:tasks.workunit.client.0.smithi022.stdout:7/513: chown d9/d1b 366680 1 2024-02-16T19:29:07.303 INFO:tasks.workunit.client.0.smithi022.stdout:6/492: dread d1/d8/da/d17/d85/d5b/f63 [0,4194304] 0 2024-02-16T19:29:07.305 INFO:tasks.workunit.client.0.smithi022.stdout:7/514: write d9/da/d2c/d46/f9a [612589,100751] 0 2024-02-16T19:29:07.311 INFO:tasks.workunit.client.0.smithi022.stdout:2/452: getdents d0/d2 0 2024-02-16T19:29:07.313 INFO:tasks.workunit.client.1.smithi117.stdout:8/480: rename d4/d7/d6a/d72/c8f to d4/d7/d37/d3a/d43/d44/c92 0 2024-02-16T19:29:07.313 INFO:tasks.workunit.client.1.smithi117.stdout:8/481: stat d4/l17 0 2024-02-16T19:29:07.316 INFO:tasks.workunit.client.1.smithi117.stdout:9/399: link dd/d45/f4f dd/d47/d5b/f8c 0 2024-02-16T19:29:07.318 INFO:tasks.workunit.client.1.smithi117.stdout:4/467: dwrite d0/da/d21/f65 [0,4194304] 0 2024-02-16T19:29:07.319 INFO:tasks.workunit.client.1.smithi117.stdout:2/455: creat d0/d3/d10/f90 x:0 0 0 2024-02-16T19:29:07.320 INFO:tasks.workunit.client.1.smithi117.stdout:8/482: dwrite d4/d7/d37/f56 [4194304,4194304] 0 2024-02-16T19:29:07.323 INFO:tasks.workunit.client.1.smithi117.stdout:5/560: mkdir d2/d9/dba 0 2024-02-16T19:29:07.324 INFO:tasks.workunit.client.0.smithi022.stdout:5/479: creat dd/d1f/d48/d21/d2f/d40/d49/fa1 x:0 0 0 2024-02-16T19:29:07.330 INFO:tasks.workunit.client.0.smithi022.stdout:9/529: write db/d14/d33/f88 [931350,34526] 0 2024-02-16T19:29:07.331 INFO:tasks.workunit.client.1.smithi117.stdout:7/457: write d4/d1c/d87/d89/d60/fab [69346,5154] 0 2024-02-16T19:29:07.331 INFO:tasks.workunit.client.1.smithi117.stdout:7/458: chown d4/f95 151409 1 2024-02-16T19:29:07.332 INFO:tasks.workunit.client.0.smithi022.stdout:9/530: chown db/d14/d16/d27/fa3 1176735 1 2024-02-16T19:29:07.332 INFO:tasks.workunit.client.1.smithi117.stdout:7/459: chown d4/d1c/d87/l53 1983960162 1 2024-02-16T19:29:07.333 INFO:tasks.workunit.client.0.smithi022.stdout:9/531: chown db/d14/d16/d87/fb0 7 1 2024-02-16T19:29:07.336 INFO:tasks.workunit.client.0.smithi022.stdout:1/590: rename d6/f19 to d6/d66/fb5 0 2024-02-16T19:29:07.337 INFO:tasks.workunit.client.0.smithi022.stdout:1/591: chown d6/d8/d22/d16/d17/d5d/c61 2243 1 2024-02-16T19:29:07.337 INFO:tasks.workunit.client.0.smithi022.stdout:0/621: mknod d10/d22/dad/ccf 0 2024-02-16T19:29:07.339 INFO:tasks.workunit.client.1.smithi117.stdout:4/468: chown d0/da/d21/d3d/d28/f51 280080172 1 2024-02-16T19:29:07.340 INFO:tasks.workunit.client.0.smithi022.stdout:7/515: chown d9/da/d1f/d44/d5b/c77 50915223 1 2024-02-16T19:29:07.340 INFO:tasks.workunit.client.1.smithi117.stdout:2/456: stat d0/d3/d22/c4a 0 2024-02-16T19:29:07.343 INFO:tasks.workunit.client.1.smithi117.stdout:7/460: mkdir d4/d1c/d87/d89/d5f/dae 0 2024-02-16T19:29:07.343 INFO:tasks.workunit.client.1.smithi117.stdout:1/430: sync 2024-02-16T19:29:07.343 INFO:tasks.workunit.client.1.smithi117.stdout:6/468: link d5/db/d7d/d32/d95/f27 d5/d4b/d44/d52/d64/f9a 0 2024-02-16T19:29:07.345 INFO:tasks.workunit.client.1.smithi117.stdout:4/469: dwrite d0/da/d21/d3d/d28/d64/d8a/d92/f85 [0,4194304] 0 2024-02-16T19:29:07.345 INFO:tasks.workunit.client.1.smithi117.stdout:9/400: mkdir dd/d8d 0 2024-02-16T19:29:07.348 INFO:tasks.workunit.client.0.smithi022.stdout:2/453: dwrite d0/d32/f52 [0,4194304] 0 2024-02-16T19:29:07.350 INFO:tasks.workunit.client.1.smithi117.stdout:9/401: dwrite dd/d47/f6d [0,4194304] 0 2024-02-16T19:29:07.354 INFO:tasks.workunit.client.1.smithi117.stdout:9/402: dread dd/d47/f7e [0,4194304] 0 2024-02-16T19:29:07.357 INFO:tasks.workunit.client.0.smithi022.stdout:2/454: read d0/d2/d18/d38/d6b/f76 [156169,49366] 0 2024-02-16T19:29:07.364 INFO:tasks.workunit.client.1.smithi117.stdout:4/470: mkdir d0/da/d21/d3d/d28/d64/d73/d91/d9c 0 2024-02-16T19:29:07.365 INFO:tasks.workunit.client.0.smithi022.stdout:1/592: chown d6/l25 46342605 1 2024-02-16T19:29:07.368 INFO:tasks.workunit.client.1.smithi117.stdout:4/471: dread d0/da/d21/d3d/d28/d64/d8a/d92/f85 [0,4194304] 0 2024-02-16T19:29:07.368 INFO:tasks.workunit.client.1.smithi117.stdout:8/483: rename d4/d7/f90 to d4/d7/d6a/f93 0 2024-02-16T19:29:07.369 INFO:tasks.workunit.client.0.smithi022.stdout:0/622: creat d10/d22/dad/fd0 x:0 0 0 2024-02-16T19:29:07.372 INFO:tasks.workunit.client.0.smithi022.stdout:3/450: dwrite d7/d12/d18/dd/d24/d29/f3f [0,4194304] 0 2024-02-16T19:29:07.373 INFO:tasks.workunit.client.0.smithi022.stdout:7/516: rename d9/da/d1f/l1c to d9/da/d1f/la6 0 2024-02-16T19:29:07.373 INFO:tasks.workunit.client.1.smithi117.stdout:4/472: dwrite d0/da/d21/f40 [0,4194304] 0 2024-02-16T19:29:07.374 INFO:tasks.workunit.client.0.smithi022.stdout:3/451: dread - d7/d12/d18/dd/d24/d58/d6b/f94 zero size 2024-02-16T19:29:07.377 INFO:tasks.workunit.client.1.smithi117.stdout:1/431: creat d2/d19/d59/d89/d69/d80/d8e/f90 x:0 0 0 2024-02-16T19:29:07.379 INFO:tasks.workunit.client.0.smithi022.stdout:8/486: rename d3/d12/d2b/d54/d74/f84 to d3/d12/db0/fb9 0 2024-02-16T19:29:07.381 INFO:tasks.workunit.client.1.smithi117.stdout:1/432: read d2/d1d/d60/f66 [2062621,121343] 0 2024-02-16T19:29:07.381 INFO:tasks.workunit.client.1.smithi117.stdout:4/473: dwrite d0/da/d21/f40 [0,4194304] 0 2024-02-16T19:29:07.383 INFO:tasks.workunit.client.1.smithi117.stdout:4/474: write d0/da/f5b [2060341,115327] 0 2024-02-16T19:29:07.396 INFO:tasks.workunit.client.1.smithi117.stdout:1/433: mkdir d2/d19/d59/d76/d91 0 2024-02-16T19:29:07.401 INFO:tasks.workunit.client.1.smithi117.stdout:1/434: dwrite d2/d1d/d1f/f44 [0,4194304] 0 2024-02-16T19:29:07.402 INFO:tasks.workunit.client.1.smithi117.stdout:7/461: rename d4/d1c/d39/d46/d47/c64 to d4/d13/caf 0 2024-02-16T19:29:07.403 INFO:tasks.workunit.client.1.smithi117.stdout:1/435: write d2/d19/d59/d76/f7a [1272016,94012] 0 2024-02-16T19:29:07.404 INFO:tasks.workunit.client.1.smithi117.stdout:4/475: dread d0/da/d21/f27 [0,4194304] 0 2024-02-16T19:29:07.406 INFO:tasks.workunit.client.1.smithi117.stdout:9/403: mkdir dd/d45/d31/d63/d8e 0 2024-02-16T19:29:07.409 INFO:tasks.workunit.client.1.smithi117.stdout:9/404: dread dd/d45/d62/f32 [0,4194304] 0 2024-02-16T19:29:07.410 INFO:tasks.workunit.client.1.smithi117.stdout:8/484: mkdir d4/d94 0 2024-02-16T19:29:07.412 INFO:tasks.workunit.client.1.smithi117.stdout:6/469: getdents d5/db/d7d/d32/d79 0 2024-02-16T19:29:07.420 INFO:tasks.workunit.client.1.smithi117.stdout:7/462: unlink d4/d1c/d87/l53 0 2024-02-16T19:29:07.422 INFO:tasks.workunit.client.1.smithi117.stdout:1/436: symlink d2/d19/d59/d89/d69/l92 0 2024-02-16T19:29:07.422 INFO:tasks.workunit.client.1.smithi117.stdout:1/437: fsync d2/ff 0 2024-02-16T19:29:07.423 INFO:tasks.workunit.client.1.smithi117.stdout:1/438: dread - d2/d1d/d60/d68/f81 zero size 2024-02-16T19:29:07.423 INFO:tasks.workunit.client.1.smithi117.stdout:4/476: symlink d0/da/d21/d3d/d28/d64/d90/l9d 0 2024-02-16T19:29:07.424 INFO:tasks.workunit.client.1.smithi117.stdout:4/477: readlink d0/da/d21/d3d/d28/d66/l87 0 2024-02-16T19:29:07.425 INFO:tasks.workunit.client.1.smithi117.stdout:9/405: rmdir dd/d45 39 2024-02-16T19:29:07.428 INFO:tasks.workunit.client.1.smithi117.stdout:6/470: mknod d5/db/d7d/d32/c9b 0 2024-02-16T19:29:07.430 INFO:tasks.workunit.client.1.smithi117.stdout:7/463: creat d4/d13/fb0 x:0 0 0 2024-02-16T19:29:07.432 INFO:tasks.workunit.client.0.smithi022.stdout:4/473: creat d4/d6/dd/df/f91 x:0 0 0 2024-02-16T19:29:07.434 INFO:tasks.workunit.client.0.smithi022.stdout:1/593: symlink d6/d8/d22/d3f/d99/lb6 0 2024-02-16T19:29:07.435 INFO:tasks.workunit.client.1.smithi117.stdout:9/406: chown dd/d45/d31/c53 5633834 1 2024-02-16T19:29:07.436 INFO:tasks.workunit.client.0.smithi022.stdout:6/493: creat d1/d8/da/d17/faf x:0 0 0 2024-02-16T19:29:07.437 INFO:tasks.workunit.client.1.smithi117.stdout:4/478: mkdir d0/da/d21/d3d/d28/d64/d73/d91/d9e 0 2024-02-16T19:29:07.437 INFO:tasks.workunit.client.0.smithi022.stdout:6/494: write d1/d8/da/d17/d48/f74 [437540,42000] 0 2024-02-16T19:29:07.439 INFO:tasks.workunit.client.0.smithi022.stdout:0/623: mkdir d10/d11/d79/dd1 0 2024-02-16T19:29:07.441 INFO:tasks.workunit.client.0.smithi022.stdout:3/452: mknod d7/d12/d18/dd/d3c/cad 0 2024-02-16T19:29:07.443 INFO:tasks.workunit.client.0.smithi022.stdout:5/480: link dd/d1f/d48/d21/d2f/d40/d49/c82 dd/d1f/d48/d21/d2f/d40/d49/d4b/ca2 0 2024-02-16T19:29:07.446 INFO:tasks.workunit.client.0.smithi022.stdout:0/624: dwrite d10/d11/d50/fa9 [0,4194304] 0 2024-02-16T19:29:07.448 INFO:tasks.workunit.client.0.smithi022.stdout:0/625: readlink d10/d11/l9b 0 2024-02-16T19:29:07.449 INFO:tasks.workunit.client.0.smithi022.stdout:2/455: rename d0/d2/d2e/d82/d8a/f91 to d0/d32/f97 0 2024-02-16T19:29:07.449 INFO:tasks.workunit.client.1.smithi117.stdout:4/479: truncate d0/da/f44 1317041 0 2024-02-16T19:29:07.452 INFO:tasks.workunit.client.0.smithi022.stdout:3/453: dwrite d7/d12/d18/f1b [0,4194304] 0 2024-02-16T19:29:07.457 INFO:tasks.workunit.client.1.smithi117.stdout:6/471: mkdir d5/d9c 0 2024-02-16T19:29:07.461 INFO:tasks.workunit.client.1.smithi117.stdout:7/464: rename f3 to d4/d13/da3/fb1 0 2024-02-16T19:29:07.464 INFO:tasks.workunit.client.1.smithi117.stdout:1/439: creat d2/f93 x:0 0 0 2024-02-16T19:29:07.465 INFO:tasks.workunit.client.0.smithi022.stdout:1/594: mkdir d6/d8/d22/d47/db7 0 2024-02-16T19:29:07.468 INFO:tasks.workunit.client.1.smithi117.stdout:7/465: dwrite d4/d1c/d87/d89/d60/d9b/fa8 [0,4194304] 0 2024-02-16T19:29:07.468 INFO:tasks.workunit.client.1.smithi117.stdout:7/466: dread - d4/d13/d63/fa2 zero size 2024-02-16T19:29:07.470 INFO:tasks.workunit.client.1.smithi117.stdout:6/472: stat d5/db/d7d/d32/d95/f27 0 2024-02-16T19:29:07.472 INFO:tasks.workunit.client.0.smithi022.stdout:1/595: dwrite d6/d8/d22/d16/d17/d1d/f33 [4194304,4194304] 0 2024-02-16T19:29:07.478 INFO:tasks.workunit.client.1.smithi117.stdout:1/440: dread - d2/d1d/f29 zero size 2024-02-16T19:29:07.481 INFO:tasks.workunit.client.1.smithi117.stdout:5/561: sync 2024-02-16T19:29:07.481 INFO:tasks.workunit.client.1.smithi117.stdout:2/457: sync 2024-02-16T19:29:07.481 INFO:tasks.workunit.client.1.smithi117.stdout:3/568: sync 2024-02-16T19:29:07.482 INFO:tasks.workunit.client.1.smithi117.stdout:0/528: sync 2024-02-16T19:29:07.485 INFO:tasks.workunit.client.0.smithi022.stdout:5/481: truncate dd/d1f/d48/d21/d2f/d52/d5c/f63 17230 0 2024-02-16T19:29:07.489 INFO:tasks.workunit.client.0.smithi022.stdout:4/474: write d4/d6/f7f [823403,7941] 0 2024-02-16T19:29:07.491 INFO:tasks.workunit.client.0.smithi022.stdout:2/456: mknod d0/d2/d18/d19/c98 0 2024-02-16T19:29:07.495 INFO:tasks.workunit.client.1.smithi117.stdout:6/473: rmdir d5/d4b/d20/d69 39 2024-02-16T19:29:07.496 INFO:tasks.workunit.client.1.smithi117.stdout:7/467: dread d4/d13/f35 [0,4194304] 0 2024-02-16T19:29:07.496 INFO:tasks.workunit.client.1.smithi117.stdout:9/407: truncate dd/d47/d5e/f7f 6208188 0 2024-02-16T19:29:07.498 INFO:tasks.workunit.client.0.smithi022.stdout:5/482: dread dd/d1f/d48/d21/d2f/d40/d49/f4c [0,4194304] 0 2024-02-16T19:29:07.499 INFO:tasks.workunit.client.0.smithi022.stdout:0/626: rename ca to d10/d11/d27/d3f/d58/d60/dbc/cd2 0 2024-02-16T19:29:07.503 INFO:tasks.workunit.client.1.smithi117.stdout:1/441: unlink d2/d4/f16 0 2024-02-16T19:29:07.503 INFO:tasks.workunit.client.0.smithi022.stdout:4/475: dread d4/d6/dd/f1e [0,4194304] 0 2024-02-16T19:29:07.504 INFO:tasks.workunit.client.1.smithi117.stdout:1/442: stat d2/d19/d59/d89/d7d/d7f/f85 0 2024-02-16T19:29:07.504 INFO:tasks.workunit.client.0.smithi022.stdout:9/532: getdents db/d14/d16/d5d 0 2024-02-16T19:29:07.510 INFO:tasks.workunit.client.1.smithi117.stdout:2/458: write d0/d13/d4d/d64/d46/d68/f8f [4663578,4217] 0 2024-02-16T19:29:07.514 INFO:tasks.workunit.client.1.smithi117.stdout:9/408: rmdir dd/d23/d64 39 2024-02-16T19:29:07.515 INFO:tasks.workunit.client.1.smithi117.stdout:2/459: dwrite d0/d3/f7c [0,4194304] 0 2024-02-16T19:29:07.520 INFO:tasks.workunit.client.1.smithi117.stdout:1/443: creat d2/d19/d59/d89/d7d/d7f/f94 x:0 0 0 2024-02-16T19:29:07.523 INFO:tasks.workunit.client.1.smithi117.stdout:3/569: creat d1/d31/d89/dbd/fca x:0 0 0 2024-02-16T19:29:07.523 INFO:tasks.workunit.client.1.smithi117.stdout:3/570: chown d1/dd/c59 283436 1 2024-02-16T19:29:07.525 INFO:tasks.workunit.client.1.smithi117.stdout:4/480: getdents d0/da 0 2024-02-16T19:29:07.528 INFO:tasks.workunit.client.1.smithi117.stdout:2/460: rmdir d0/d13/d75 39 2024-02-16T19:29:07.529 INFO:tasks.workunit.client.1.smithi117.stdout:2/461: chown d0/d29/d56 3412 1 2024-02-16T19:29:07.530 INFO:tasks.workunit.client.1.smithi117.stdout:7/468: rename d4/d1c/d87/d89/d60/d9b/fa8 to d4/d1c/d87/d89/d60/d9b/fb2 0 2024-02-16T19:29:07.531 INFO:tasks.workunit.client.1.smithi117.stdout:8/485: rename d4/d7/d37/d3a/d43/d44/c92 to d4/d7/d6a/c95 0 2024-02-16T19:29:07.532 INFO:tasks.workunit.client.1.smithi117.stdout:1/444: creat d2/d19/d59/d89/d69/f95 x:0 0 0 2024-02-16T19:29:07.533 INFO:tasks.workunit.client.1.smithi117.stdout:3/571: rmdir d1 39 2024-02-16T19:29:07.534 INFO:tasks.workunit.client.1.smithi117.stdout:9/409: mknod dd/c8f 0 2024-02-16T19:29:07.535 INFO:tasks.workunit.client.1.smithi117.stdout:2/462: write d0/d13/d75/f59 [270810,100305] 0 2024-02-16T19:29:07.537 INFO:tasks.workunit.client.1.smithi117.stdout:8/486: mkdir d4/d7/d37/d3a/d65/d96 0 2024-02-16T19:29:07.538 INFO:tasks.workunit.client.1.smithi117.stdout:8/487: readlink d4/d7/d37/l5e 0 2024-02-16T19:29:07.539 INFO:tasks.workunit.client.1.smithi117.stdout:3/572: read - d1/dd/d5c/fc1 zero size 2024-02-16T19:29:07.540 INFO:tasks.workunit.client.1.smithi117.stdout:1/445: mknod d2/d19/d59/d89/d69/d80/c96 0 2024-02-16T19:29:07.540 INFO:tasks.workunit.client.1.smithi117.stdout:3/573: read d1/d1b/d1e/d5e/f66 [163753,90378] 0 2024-02-16T19:29:07.541 INFO:tasks.workunit.client.1.smithi117.stdout:9/410: dread - dd/d45/f49 zero size 2024-02-16T19:29:07.542 INFO:tasks.workunit.client.1.smithi117.stdout:3/574: fsync d1/d30/f90 0 2024-02-16T19:29:07.542 INFO:tasks.workunit.client.1.smithi117.stdout:4/481: dread d0/da/f33 [0,4194304] 0 2024-02-16T19:29:07.542 INFO:tasks.workunit.client.1.smithi117.stdout:1/446: write d2/d1d/d1f/f56 [4403384,35198] 0 2024-02-16T19:29:07.543 INFO:tasks.workunit.client.1.smithi117.stdout:7/469: fdatasync d4/d1c/d87/d89/d5f/f7b 0 2024-02-16T19:29:07.545 INFO:tasks.workunit.client.1.smithi117.stdout:2/463: mkdir d0/d13/d4d/d64/d91 0 2024-02-16T19:29:07.547 INFO:tasks.workunit.client.1.smithi117.stdout:4/482: chown d0/da/d21/d3d/d28/d64/d73/d91/d37/l9a 54 1 2024-02-16T19:29:07.549 INFO:tasks.workunit.client.1.smithi117.stdout:6/474: dwrite d5/d4b/d20/f42 [0,4194304] 0 2024-02-16T19:29:07.551 INFO:tasks.workunit.client.1.smithi117.stdout:6/475: write d5/db/d7d/d32/d95/f87 [4512491,83151] 0 2024-02-16T19:29:07.551 INFO:tasks.workunit.client.1.smithi117.stdout:9/411: mkdir dd/d47/d5b/d90 0 2024-02-16T19:29:07.553 INFO:tasks.workunit.client.0.smithi022.stdout:3/454: write d7/d12/d18/dd/d31/f47 [107904,75620] 0 2024-02-16T19:29:07.553 INFO:tasks.workunit.client.0.smithi022.stdout:7/517: creat d9/fa7 x:0 0 0 2024-02-16T19:29:07.554 INFO:tasks.workunit.client.0.smithi022.stdout:1/596: stat d6/d8/c18 0 2024-02-16T19:29:07.555 INFO:tasks.workunit.client.1.smithi117.stdout:3/575: mkdir d1/d30/d9f/dcb 0 2024-02-16T19:29:07.556 INFO:tasks.workunit.client.0.smithi022.stdout:5/483: creat dd/d1f/d48/d21/d2f/d46/d5e/d84/fa3 x:0 0 0 2024-02-16T19:29:07.557 INFO:tasks.workunit.client.0.smithi022.stdout:4/476: creat d4/d6/d14/d64/d4e/d54/d5a/d63/f92 x:0 0 0 2024-02-16T19:29:07.558 INFO:tasks.workunit.client.1.smithi117.stdout:1/447: rmdir d2/d19/d59 39 2024-02-16T19:29:07.560 INFO:tasks.workunit.client.1.smithi117.stdout:8/488: mknod d4/d7/d37/d3a/d65/d96/c97 0 2024-02-16T19:29:07.561 INFO:tasks.workunit.client.1.smithi117.stdout:8/489: fsync d4/d7/d6a/f7b 0 2024-02-16T19:29:07.562 INFO:tasks.workunit.client.0.smithi022.stdout:9/533: creat db/d14/d16/d27/d55/fb8 x:0 0 0 2024-02-16T19:29:07.562 INFO:tasks.workunit.client.0.smithi022.stdout:8/487: dwrite d3/d12/d17/f7e [0,4194304] 0 2024-02-16T19:29:07.565 INFO:tasks.workunit.client.0.smithi022.stdout:5/484: write dd/d1f/d48/d21/d2f/d52/d5c/d61/d8f/f94 [55874,7808] 0 2024-02-16T19:29:07.566 INFO:tasks.workunit.client.0.smithi022.stdout:8/488: fdatasync d3/d12/d6a/fa4 0 2024-02-16T19:29:07.568 INFO:tasks.workunit.client.1.smithi117.stdout:4/483: mknod d0/da/d21/d3d/d28/d64/d8a/c9f 0 2024-02-16T19:29:07.571 INFO:tasks.workunit.client.0.smithi022.stdout:6/495: getdents d1/d8/da/d17/d48/d51 0 2024-02-16T19:29:07.574 INFO:tasks.workunit.client.1.smithi117.stdout:2/464: dread d0/d29/d2f/f44 [0,4194304] 0 2024-02-16T19:29:07.574 INFO:tasks.workunit.client.1.smithi117.stdout:3/576: chown d1/cb3 8566711 1 2024-02-16T19:29:07.578 INFO:tasks.workunit.client.0.smithi022.stdout:4/477: dwrite d4/d6/d14/f2e [0,4194304] 0 2024-02-16T19:29:07.578 INFO:tasks.workunit.client.1.smithi117.stdout:8/490: fdatasync d4/d73/f82 0 2024-02-16T19:29:07.581 INFO:tasks.workunit.client.0.smithi022.stdout:1/597: symlink d6/d8/lb8 0 2024-02-16T19:29:07.583 INFO:tasks.workunit.client.0.smithi022.stdout:4/478: stat d4/d6/dd/df/d10/d24 0 2024-02-16T19:29:07.583 INFO:tasks.workunit.client.0.smithi022.stdout:0/627: dread d10/d22/d49/f4d [0,4194304] 0 2024-02-16T19:29:07.584 INFO:tasks.workunit.client.1.smithi117.stdout:8/491: dwrite d4/f52 [0,4194304] 0 2024-02-16T19:29:07.584 INFO:tasks.workunit.client.0.smithi022.stdout:8/489: dwrite d3/f8d [0,4194304] 0 2024-02-16T19:29:07.587 INFO:tasks.workunit.client.0.smithi022.stdout:6/496: mkdir d1/d8/d23/d95/db0 0 2024-02-16T19:29:07.588 INFO:tasks.workunit.client.0.smithi022.stdout:3/455: dread d7/d21/f7c [0,4194304] 0 2024-02-16T19:29:07.590 INFO:tasks.workunit.client.1.smithi117.stdout:6/476: mknod d5/d4b/c9d 0 2024-02-16T19:29:07.590 INFO:tasks.workunit.client.0.smithi022.stdout:4/479: rename d4/d6/dd/df/d10 to d4/d6/dd/df/d10/d24/d4b/d93 22 2024-02-16T19:29:07.591 INFO:tasks.workunit.client.0.smithi022.stdout:9/534: truncate db/d14/f60 212414 0 2024-02-16T19:29:07.591 INFO:tasks.workunit.client.1.smithi117.stdout:6/477: write d5/db/d7d/d32/d79/f94 [243149,89357] 0 2024-02-16T19:29:07.592 INFO:tasks.workunit.client.0.smithi022.stdout:3/456: write d7/d21/f5d [483117,113741] 0 2024-02-16T19:29:07.592 INFO:tasks.workunit.client.0.smithi022.stdout:6/497: write d1/d8/f3b [426850,62982] 0 2024-02-16T19:29:07.593 INFO:tasks.workunit.client.1.smithi117.stdout:6/478: chown d5/d4b/d20/d2d/f96 29 1 2024-02-16T19:29:07.593 INFO:tasks.workunit.client.0.smithi022.stdout:4/480: write d4/d6/f87 [180015,5982] 0 2024-02-16T19:29:07.593 INFO:tasks.workunit.client.0.smithi022.stdout:9/535: fsync db/d18/fb4 0 2024-02-16T19:29:07.595 INFO:tasks.workunit.client.0.smithi022.stdout:0/628: creat d10/d22/fd3 x:0 0 0 2024-02-16T19:29:07.596 INFO:tasks.workunit.client.0.smithi022.stdout:5/485: dread dd/d1f/d48/d21/d2f/d52/d5c/d61/d6a/f70 [0,4194304] 0 2024-02-16T19:29:07.597 INFO:tasks.workunit.client.1.smithi117.stdout:9/412: rename dd/d45/d73/f78 to dd/d23/d64/f91 0 2024-02-16T19:29:07.597 INFO:tasks.workunit.client.0.smithi022.stdout:1/598: truncate d6/d8/d22/d16/d4f/f8c 523889 0 2024-02-16T19:29:07.597 INFO:tasks.workunit.client.0.smithi022.stdout:9/536: readlink db/d14/d19/da8/l75 0 2024-02-16T19:29:07.598 INFO:tasks.workunit.client.0.smithi022.stdout:8/490: symlink d3/d12/d6a/lba 0 2024-02-16T19:29:07.599 INFO:tasks.workunit.client.0.smithi022.stdout:0/629: fdatasync d10/d22/dad/fc2 0 2024-02-16T19:29:07.599 INFO:tasks.workunit.client.0.smithi022.stdout:6/498: write d1/d8/d3c/f44 [4549942,85603] 0 2024-02-16T19:29:07.600 INFO:tasks.workunit.client.1.smithi117.stdout:1/448: fsync d2/d4/f11 0 2024-02-16T19:29:07.601 INFO:tasks.workunit.client.0.smithi022.stdout:8/491: fdatasync d3/d12/da6/f8c 0 2024-02-16T19:29:07.603 INFO:tasks.workunit.client.1.smithi117.stdout:3/577: symlink d1/d1b/d1e/lcc 0 2024-02-16T19:29:07.605 INFO:tasks.workunit.client.0.smithi022.stdout:7/518: truncate d9/da/f62 709846 0 2024-02-16T19:29:07.606 INFO:tasks.workunit.client.0.smithi022.stdout:1/599: dread d6/d8/d22/f1b [0,4194304] 0 2024-02-16T19:29:07.608 INFO:tasks.workunit.client.1.smithi117.stdout:7/470: link d4/d13/d63/c7c d4/d1c/d39/d46/cb3 0 2024-02-16T19:29:07.609 INFO:tasks.workunit.client.1.smithi117.stdout:7/471: dread - d4/d1c/d39/d46/d6e/f99 zero size 2024-02-16T19:29:07.609 INFO:tasks.workunit.client.1.smithi117.stdout:4/484: symlink d0/da/d21/d3d/d28/d66/d97/la0 0 2024-02-16T19:29:07.610 INFO:tasks.workunit.client.1.smithi117.stdout:7/472: readlink d4/d1c/l33 0 2024-02-16T19:29:07.610 INFO:tasks.workunit.client.0.smithi022.stdout:1/600: fsync d6/d8/d22/f82 0 2024-02-16T19:29:07.611 INFO:tasks.workunit.client.1.smithi117.stdout:2/465: dread d0/d3/d22/d37/f48 [0,4194304] 0 2024-02-16T19:29:07.611 INFO:tasks.workunit.client.0.smithi022.stdout:4/481: symlink d4/d6/d14/d64/d66/d86/l94 0 2024-02-16T19:29:07.611 INFO:tasks.workunit.client.1.smithi117.stdout:2/466: chown d0/d3/d22/c4a 303 1 2024-02-16T19:29:07.614 INFO:tasks.workunit.client.0.smithi022.stdout:6/499: dwrite d1/d8/d3c/f44 [0,4194304] 0 2024-02-16T19:29:07.615 INFO:tasks.workunit.client.0.smithi022.stdout:4/482: dread - d4/d6/d14/d64/d66/d6d/f7d zero size 2024-02-16T19:29:07.616 INFO:tasks.workunit.client.0.smithi022.stdout:0/630: dwrite d10/d22/f53 [0,4194304] 0 2024-02-16T19:29:07.616 INFO:tasks.workunit.client.0.smithi022.stdout:9/537: rmdir db/d14/d16/d27/d8b 39 2024-02-16T19:29:07.618 INFO:tasks.workunit.client.0.smithi022.stdout:9/538: fsync db/d14/d16/d5d/d65/d6b/f9b 0 2024-02-16T19:29:07.619 INFO:tasks.workunit.client.0.smithi022.stdout:9/539: readlink db/d14/d33/l9e 0 2024-02-16T19:29:07.620 INFO:tasks.workunit.client.1.smithi117.stdout:3/578: creat d1/dd/d3b/d9d/fcd x:0 0 0 2024-02-16T19:29:07.623 INFO:tasks.workunit.client.1.smithi117.stdout:8/492: dread d4/d12/d2a/f2b [8388608,4194304] 0 2024-02-16T19:29:07.623 INFO:tasks.workunit.client.1.smithi117.stdout:4/485: unlink d0/da/d21/d3d/d28/d66/l87 0 2024-02-16T19:29:07.624 INFO:tasks.workunit.client.0.smithi022.stdout:4/483: dwrite d4/f23 [4194304,4194304] 0 2024-02-16T19:29:07.626 INFO:tasks.workunit.client.1.smithi117.stdout:4/486: dread d0/da/d21/d3d/d28/d64/f67 [0,4194304] 0 2024-02-16T19:29:07.627 INFO:tasks.workunit.client.0.smithi022.stdout:4/484: read - d4/d38/f79 zero size 2024-02-16T19:29:07.635 INFO:tasks.workunit.client.0.smithi022.stdout:5/486: mknod dd/d1f/ca4 0 2024-02-16T19:29:07.639 INFO:tasks.workunit.client.0.smithi022.stdout:6/500: mkdir d1/d8/d23/db1 0 2024-02-16T19:29:07.641 INFO:tasks.workunit.client.1.smithi117.stdout:9/413: mknod dd/d45/c92 0 2024-02-16T19:29:07.642 INFO:tasks.workunit.client.0.smithi022.stdout:3/457: write d7/d12/d18/d4b/f4d [48512,68737] 0 2024-02-16T19:29:07.643 INFO:tasks.workunit.client.1.smithi117.stdout:3/579: mkdir d1/d67/dce 0 2024-02-16T19:29:07.645 INFO:tasks.workunit.client.1.smithi117.stdout:8/493: rmdir d4/d12/d2a/d64 39 2024-02-16T19:29:07.645 INFO:tasks.workunit.client.1.smithi117.stdout:8/494: dread - d4/d12/d31/f77 zero size 2024-02-16T19:29:07.647 INFO:tasks.workunit.client.0.smithi022.stdout:9/540: rename db/d14/d16/d28/l77 to db/d14/d16/d27/d2e/lb9 0 2024-02-16T19:29:07.652 INFO:tasks.workunit.client.0.smithi022.stdout:1/601: dread d6/d8/d22/d16/f40 [0,4194304] 0 2024-02-16T19:29:07.654 INFO:tasks.workunit.client.0.smithi022.stdout:5/487: fsync dd/d1f/d48/d21/d2f/d52/d5c/d61/d6a/f70 0 2024-02-16T19:29:07.657 INFO:tasks.workunit.client.0.smithi022.stdout:6/501: rmdir d1/d8/da/d29 39 2024-02-16T19:29:07.659 INFO:tasks.workunit.client.1.smithi117.stdout:6/479: dwrite d5/d4b/d44/d52/f6c [0,4194304] 0 2024-02-16T19:29:07.660 INFO:tasks.workunit.client.1.smithi117.stdout:6/480: truncate d5/d4b/d44/f75 1390409 0 2024-02-16T19:29:07.662 INFO:tasks.workunit.client.0.smithi022.stdout:5/488: dwrite f6 [0,4194304] 0 2024-02-16T19:29:07.664 INFO:tasks.workunit.client.0.smithi022.stdout:0/631: truncate d10/d11/d27/d3f/f8a 746048 0 2024-02-16T19:29:07.666 INFO:tasks.workunit.client.0.smithi022.stdout:0/632: chown d10/d11/d50/d65/fbe 43538364 1 2024-02-16T19:29:07.667 INFO:tasks.workunit.client.0.smithi022.stdout:0/633: stat d10/d11/d27/d3f/f7a 0 2024-02-16T19:29:07.667 INFO:tasks.workunit.client.0.smithi022.stdout:4/485: dread d4/d6/dd/f60 [4194304,4194304] 0 2024-02-16T19:29:07.670 INFO:tasks.workunit.client.1.smithi117.stdout:2/467: creat d0/d3/f92 x:0 0 0 2024-02-16T19:29:07.676 INFO:tasks.workunit.client.0.smithi022.stdout:1/602: mknod d6/d8/d22/d47/d51/cb9 0 2024-02-16T19:29:07.677 INFO:tasks.workunit.client.0.smithi022.stdout:1/603: stat d6/d8/d22/d16/d17/d26/lb2 0 2024-02-16T19:29:07.679 INFO:tasks.workunit.client.1.smithi117.stdout:7/473: dwrite d4/d13/da3/fb1 [0,4194304] 0 2024-02-16T19:29:07.679 INFO:tasks.workunit.client.1.smithi117.stdout:6/481: fsync d5/db/f4a 0 2024-02-16T19:29:07.679 INFO:tasks.workunit.client.1.smithi117.stdout:6/482: chown d5/f86 11583 1 2024-02-16T19:29:07.681 INFO:tasks.workunit.client.1.smithi117.stdout:7/474: rename d4/d1c/d87/d89 to d4/d1c/d87/d89/d5f/dae/db4 22 2024-02-16T19:29:07.688 INFO:tasks.workunit.client.1.smithi117.stdout:2/468: mkdir d0/d13/d4d/d64/d46/d68/d93 0 2024-02-16T19:29:07.696 INFO:tasks.workunit.client.0.smithi022.stdout:5/489: rmdir dd/d1f 39 2024-02-16T19:29:07.698 INFO:tasks.workunit.client.0.smithi022.stdout:8/492: getdents d3/d12/d2b/da9 0 2024-02-16T19:29:07.700 INFO:tasks.workunit.client.1.smithi117.stdout:5/562: sync 2024-02-16T19:29:07.700 INFO:tasks.workunit.client.1.smithi117.stdout:0/529: sync 2024-02-16T19:29:07.703 INFO:tasks.workunit.client.0.smithi022.stdout:4/486: mknod d4/d6/dd/df/d10/d24/c95 0 2024-02-16T19:29:07.705 INFO:tasks.workunit.client.0.smithi022.stdout:7/519: getdents d9/da/d4e 0 2024-02-16T19:29:07.707 INFO:tasks.workunit.client.1.smithi117.stdout:7/475: rename d4/f95 to d4/d1c/d39/d46/d47/fb5 0 2024-02-16T19:29:07.709 INFO:tasks.workunit.client.1.smithi117.stdout:4/487: getdents d0/da/d21/d3d/d28/d64/d73/d91/d37 0 2024-02-16T19:29:07.709 INFO:tasks.workunit.client.1.smithi117.stdout:7/476: chown d4/d1c/d39/f5a 3505 1 2024-02-16T19:29:07.709 INFO:tasks.workunit.client.0.smithi022.stdout:1/604: dread - d6/d8/d22/d3a/d80/f86 zero size 2024-02-16T19:29:07.710 INFO:tasks.workunit.client.1.smithi117.stdout:9/414: getdents dd/d47/d5b/d5f/d77/d7c 0 2024-02-16T19:29:07.710 INFO:tasks.workunit.client.1.smithi117.stdout:3/580: getdents d1/d31/d89/dbd 0 2024-02-16T19:29:07.710 INFO:tasks.workunit.client.0.smithi022.stdout:6/502: fsync d1/d8/da/d17/f96 0 2024-02-16T19:29:07.711 INFO:tasks.workunit.client.1.smithi117.stdout:3/581: dread - d1/dd/d5c/fa1 zero size 2024-02-16T19:29:07.714 INFO:tasks.workunit.client.1.smithi117.stdout:9/415: dwrite dd/d47/d48/f55 [4194304,4194304] 0 2024-02-16T19:29:07.719 INFO:tasks.workunit.client.0.smithi022.stdout:4/487: dread d4/d6/dd/f2c [0,4194304] 0 2024-02-16T19:29:07.746 INFO:tasks.workunit.client.1.smithi117.stdout:1/449: sync 2024-02-16T19:29:07.789 INFO:tasks.workunit.client.1.smithi117.stdout:5/563: unlink d2/d9/d3c/d24/f6a 0 2024-02-16T19:29:07.791 INFO:tasks.workunit.client.1.smithi117.stdout:0/530: readlink d0/d9/d22/d27/d31/d6d/l9a 0 2024-02-16T19:29:07.792 INFO:tasks.workunit.client.1.smithi117.stdout:5/564: dread d2/db3/f3f [0,4194304] 0 2024-02-16T19:29:07.795 INFO:tasks.workunit.client.1.smithi117.stdout:6/483: link d5/d4b/d20/d2d/f96 d5/d9c/f9e 0 2024-02-16T19:29:07.797 INFO:tasks.workunit.client.1.smithi117.stdout:8/495: getdents d4/d7/d37/d3a/d43 0 2024-02-16T19:29:07.802 INFO:tasks.workunit.client.1.smithi117.stdout:2/469: rename d0/d13/d4d/d64 to d0/d3/d34/d94 0 2024-02-16T19:29:07.804 INFO:tasks.workunit.client.1.smithi117.stdout:4/488: write d0/da/f33 [2012010,51476] 0 2024-02-16T19:29:07.806 INFO:tasks.workunit.client.1.smithi117.stdout:7/477: stat d4/d13/d63/d71/l7d 0 2024-02-16T19:29:07.808 INFO:tasks.workunit.client.1.smithi117.stdout:7/478: chown d4/d13/d63/d71/f90 2416 1 2024-02-16T19:29:07.809 INFO:tasks.workunit.client.1.smithi117.stdout:4/489: dwrite d0/da/d21/d3d/d28/d66/d97/f94 [0,4194304] 0 2024-02-16T19:29:07.810 INFO:tasks.workunit.client.0.smithi022.stdout:4/488: mknod d4/d38/c96 0 2024-02-16T19:29:07.811 INFO:tasks.workunit.client.0.smithi022.stdout:0/634: truncate d10/d15/d23/d8f/f95 363941 0 2024-02-16T19:29:07.813 INFO:tasks.workunit.client.0.smithi022.stdout:0/635: write d10/d22/d49/fa3 [520601,93986] 0 2024-02-16T19:29:07.814 INFO:tasks.workunit.client.0.smithi022.stdout:5/490: creat dd/d1f/d48/d21/d2f/d40/d4e/d56/fa5 x:0 0 0 2024-02-16T19:29:07.815 INFO:tasks.workunit.client.0.smithi022.stdout:5/491: read dd/f11 [491159,47857] 0 2024-02-16T19:29:07.817 INFO:tasks.workunit.client.0.smithi022.stdout:9/541: getdents db/d14 0 2024-02-16T19:29:07.820 INFO:tasks.workunit.client.0.smithi022.stdout:1/605: mkdir d6/d8/dba 0 2024-02-16T19:29:07.822 INFO:tasks.workunit.client.0.smithi022.stdout:7/520: dwrite d9/da/d1f/d44/f54 [0,4194304] 0 2024-02-16T19:29:07.829 INFO:tasks.workunit.client.0.smithi022.stdout:8/493: creat d3/d12/d17/fbb x:0 0 0 2024-02-16T19:29:07.831 INFO:tasks.workunit.client.1.smithi117.stdout:3/582: dread d1/dd/d3b/f3d [0,4194304] 0 2024-02-16T19:29:07.831 INFO:tasks.workunit.client.0.smithi022.stdout:0/636: read d10/d11/d50/f3e [6334065,93494] 0 2024-02-16T19:29:07.834 INFO:tasks.workunit.client.0.smithi022.stdout:5/492: creat dd/d1f/d48/d21/d2f/d46/d5e/fa6 x:0 0 0 2024-02-16T19:29:07.835 INFO:tasks.workunit.client.0.smithi022.stdout:0/637: dread d10/d22/d49/fa3 [0,4194304] 0 2024-02-16T19:29:07.839 INFO:tasks.workunit.client.0.smithi022.stdout:6/503: link d1/d8/d81/f98 d1/d8/da/d7b/fb2 0 2024-02-16T19:29:07.843 INFO:tasks.workunit.client.0.smithi022.stdout:7/521: creat d9/da/d2c/d88/fa8 x:0 0 0 2024-02-16T19:29:07.856 INFO:tasks.workunit.client.0.smithi022.stdout:0/638: mknod d10/d22/d2c/cd4 0 2024-02-16T19:29:07.857 INFO:tasks.workunit.client.0.smithi022.stdout:6/504: dread d1/d8/da/d17/d85/f2b [0,4194304] 0 2024-02-16T19:29:07.858 INFO:tasks.workunit.client.0.smithi022.stdout:0/639: write d10/d22/d2c/f72 [1622808,56759] 0 2024-02-16T19:29:07.861 INFO:tasks.workunit.client.0.smithi022.stdout:5/493: rename dd/d1f/d76/c86 to dd/d1f/d48/d21/d2f/d46/d5e/d84/d8b/ca7 0 2024-02-16T19:29:07.861 INFO:tasks.workunit.client.0.smithi022.stdout:4/489: dwrite d4/d6/dd/d22/d30/f44 [0,4194304] 0 2024-02-16T19:29:07.871 INFO:tasks.workunit.client.0.smithi022.stdout:5/494: dwrite dd/d1f/d48/d21/d2f/d40/f91 [0,4194304] 0 2024-02-16T19:29:07.872 INFO:tasks.workunit.client.0.smithi022.stdout:7/522: fsync d9/da/d2c/d2f/f3d 0 2024-02-16T19:29:07.873 INFO:tasks.workunit.client.0.smithi022.stdout:5/495: readlink dd/d1f/d48/d21/d2f/d52/d5c/l67 0 2024-02-16T19:29:07.875 INFO:tasks.workunit.client.0.smithi022.stdout:5/496: dread - dd/d1f/d48/d21/d2f/d46/d5e/d84/fa3 zero size 2024-02-16T19:29:07.875 INFO:tasks.workunit.client.0.smithi022.stdout:7/523: write d9/da/d2c/d46/f6f [262131,4391] 0 2024-02-16T19:29:07.880 INFO:tasks.workunit.client.1.smithi117.stdout:1/450: dwrite d2/d1d/f6d [0,4194304] 0 2024-02-16T19:29:07.880 INFO:tasks.workunit.client.1.smithi117.stdout:0/531: creat d0/d9/d2c/fc1 x:0 0 0 2024-02-16T19:29:07.884 INFO:tasks.workunit.client.0.smithi022.stdout:1/606: dwrite d6/d8/d22/f15 [0,4194304] 0 2024-02-16T19:29:07.885 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:07 smithi022 ceph-mon[29221]: pgmap v7: 65 pgs: 65 active+clean; 2.0 GiB data, 5.8 GiB used, 531 GiB / 536 GiB avail; 65 MiB/s rd, 164 MiB/s wr, 343 op/s 2024-02-16T19:29:07.888 INFO:tasks.workunit.client.0.smithi022.stdout:8/494: write d3/d25/f5c [1185925,111489] 0 2024-02-16T19:29:07.889 INFO:tasks.workunit.client.1.smithi117.stdout:8/496: unlink d4/d7/d37/d3a/d43/d44/l4c 0 2024-02-16T19:29:07.895 INFO:tasks.workunit.client.1.smithi117.stdout:2/470: dread - d0/f4f zero size 2024-02-16T19:29:07.895 INFO:tasks.workunit.client.0.smithi022.stdout:6/505: rename d1/d8/d23/d95 to d1/d8/da/d17/d53/d7f/db3 0 2024-02-16T19:29:07.895 INFO:tasks.workunit.client.0.smithi022.stdout:9/542: write db/d14/d19/d1a/f50 [1987509,130928] 0 2024-02-16T19:29:07.897 INFO:tasks.workunit.client.0.smithi022.stdout:1/607: dread d6/d8/d22/f15 [0,4194304] 0 2024-02-16T19:29:07.898 INFO:tasks.workunit.client.0.smithi022.stdout:9/543: chown db/d14/d16/d21/f4c 51 1 2024-02-16T19:29:07.899 INFO:tasks.workunit.client.0.smithi022.stdout:4/490: symlink d4/d6/dd/df/d10/d55/l97 0 2024-02-16T19:29:07.901 INFO:tasks.workunit.client.1.smithi117.stdout:5/565: dread d2/d9/d3c/d24/d50/d52/d74/d8c/f4b [0,4194304] 0 2024-02-16T19:29:07.902 INFO:tasks.workunit.client.0.smithi022.stdout:4/491: write d4/d6/dd/df/d10/f19 [4382949,58691] 0 2024-02-16T19:29:07.904 INFO:tasks.workunit.client.1.smithi117.stdout:7/479: write d4/d13/f35 [195268,50107] 0 2024-02-16T19:29:07.905 INFO:tasks.workunit.client.1.smithi117.stdout:6/484: dread d5/db/d7d/d32/d95/f2c [0,4194304] 0 2024-02-16T19:29:07.906 INFO:tasks.workunit.client.0.smithi022.stdout:9/544: stat db/d14/d16/d27/d2e/f8e 0 2024-02-16T19:29:07.908 INFO:tasks.workunit.client.0.smithi022.stdout:6/506: dwrite d1/d8/d81/fa0 [0,4194304] 0 2024-02-16T19:29:07.913 INFO:tasks.workunit.client.0.smithi022.stdout:1/608: dwrite d6/d8/d22/d16/d17/d63/d6f/d8b/fa3 [0,4194304] 0 2024-02-16T19:29:07.913 INFO:tasks.workunit.client.1.smithi117.stdout:3/583: rename d1/dd/c59 to d1/d30/d52/ccf 0 2024-02-16T19:29:07.919 INFO:tasks.workunit.client.1.smithi117.stdout:2/471: dread d0/d13/f15 [0,4194304] 0 2024-02-16T19:29:07.920 INFO:tasks.workunit.client.1.smithi117.stdout:2/472: chown d0/d13/d75/f59 1868458 1 2024-02-16T19:29:07.923 INFO:tasks.workunit.client.1.smithi117.stdout:4/490: dread d0/da/d21/d3d/d28/f63 [0,4194304] 0 2024-02-16T19:29:07.923 INFO:tasks.workunit.client.1.smithi117.stdout:9/416: truncate dd/d45/d62/f32 3987589 0 2024-02-16T19:29:07.924 INFO:tasks.workunit.client.0.smithi022.stdout:7/524: write d9/da/d2c/f2d [1668121,28954] 0 2024-02-16T19:29:07.929 INFO:tasks.workunit.client.0.smithi022.stdout:8/495: unlink d3/d12/d2b/d2e/c42 0 2024-02-16T19:29:07.939 INFO:tasks.workunit.client.0.smithi022.stdout:7/525: dread d9/da/d4e/f89 [0,4194304] 0 2024-02-16T19:29:07.946 INFO:tasks.workunit.client.1.smithi117.stdout:7/480: dwrite d4/d13/f18 [0,4194304] 0 2024-02-16T19:29:07.948 INFO:tasks.workunit.client.1.smithi117.stdout:2/473: dread - d0/d3/f57 zero size 2024-02-16T19:29:07.952 INFO:tasks.workunit.client.1.smithi117.stdout:1/451: dwrite d2/d19/d59/d89/f42 [0,4194304] 0 2024-02-16T19:29:07.956 INFO:tasks.workunit.client.1.smithi117.stdout:4/491: rename d0/l16 to d0/da/d21/d3d/d28/d66/d97/d6d/la1 0 2024-02-16T19:29:07.956 INFO:tasks.workunit.client.1.smithi117.stdout:4/492: chown d0/da/d21/d3d/d28/d64/c18 8392 1 2024-02-16T19:29:07.958 INFO:tasks.workunit.client.1.smithi117.stdout:8/497: truncate d4/d1c/f7f 252921 0 2024-02-16T19:29:07.960 INFO:tasks.workunit.client.1.smithi117.stdout:5/566: mknod d2/cbb 0 2024-02-16T19:29:07.961 INFO:tasks.workunit.client.0.smithi022.stdout:8/496: truncate d3/d25/f61 213620 0 2024-02-16T19:29:07.963 INFO:tasks.workunit.client.1.smithi117.stdout:6/485: mkdir d5/d4b/d44/d52/d81/d9f 0 2024-02-16T19:29:07.963 INFO:tasks.workunit.client.0.smithi022.stdout:4/492: mknod d4/d6/dd/df/c98 0 2024-02-16T19:29:07.964 INFO:tasks.workunit.client.0.smithi022.stdout:9/545: symlink db/d14/d16/lba 0 2024-02-16T19:29:07.965 INFO:tasks.workunit.client.1.smithi117.stdout:3/584: mknod d1/dd/d3e/cd0 0 2024-02-16T19:29:07.968 INFO:tasks.workunit.client.1.smithi117.stdout:6/486: dwrite d5/ff [4194304,4194304] 0 2024-02-16T19:29:07.972 INFO:tasks.workunit.client.1.smithi117.stdout:0/532: link d0/d9/d22/d27/d31/d6d/l9a d0/lc2 0 2024-02-16T19:29:07.973 INFO:tasks.workunit.client.0.smithi022.stdout:5/497: link dd/d1f/d48/d21/d2f/d40/d49/d4b/c62 dd/d1f/d48/d21/d2f/d46/d5e/d84/ca8 0 2024-02-16T19:29:07.973 INFO:tasks.workunit.client.1.smithi117.stdout:3/585: dwrite d1/d1b/f21 [0,4194304] 0 2024-02-16T19:29:07.974 INFO:tasks.workunit.client.0.smithi022.stdout:0/640: getdents d10/d7d 0 2024-02-16T19:29:07.975 INFO:tasks.workunit.client.0.smithi022.stdout:0/641: stat d10/d11/d27/d3f/d58/dab 0 2024-02-16T19:29:07.977 INFO:tasks.workunit.client.0.smithi022.stdout:0/642: chown d10/d15/f9f 1604 1 2024-02-16T19:29:07.986 INFO:tasks.workunit.client.1.smithi117.stdout:1/452: creat d2/d19/d59/f97 x:0 0 0 2024-02-16T19:29:07.989 INFO:tasks.workunit.client.1.smithi117.stdout:4/493: mkdir d0/da/d21/d3d/d28/d66/da2 0 2024-02-16T19:29:07.990 INFO:tasks.workunit.client.0.smithi022.stdout:7/526: mkdir d9/da9 0 2024-02-16T19:29:07.990 INFO:tasks.workunit.client.0.smithi022.stdout:9/546: dwrite db/d18/f93 [0,4194304] 0 2024-02-16T19:29:07.991 INFO:tasks.workunit.client.1.smithi117.stdout:8/498: stat d4/d12/d2a/d64/c8d 0 2024-02-16T19:29:07.992 INFO:tasks.workunit.client.0.smithi022.stdout:1/609: link d6/d8/d22/d16/d17/d26/l5e d6/d8/d22/d67/lbb 0 2024-02-16T19:29:07.997 INFO:tasks.workunit.client.1.smithi117.stdout:5/567: rename d2/d3/d21/d2f to d2/d9/d3c/d24/d50/d96/dbc 0 2024-02-16T19:29:07.999 INFO:tasks.workunit.client.1.smithi117.stdout:2/474: dread d0/fd [0,4194304] 0 2024-02-16T19:29:08.000 INFO:tasks.workunit.client.0.smithi022.stdout:5/498: mkdir dd/d1f/d48/d21/d2f/d52/d77/da9 0 2024-02-16T19:29:08.000 INFO:tasks.workunit.client.1.smithi117.stdout:2/475: dread - d0/d13/d4d/d53/d58/f78 zero size 2024-02-16T19:29:08.006 INFO:tasks.workunit.client.1.smithi117.stdout:2/476: dwrite d0/d13/d4d/d53/f6f [0,4194304] 0 2024-02-16T19:29:08.007 INFO:tasks.workunit.client.0.smithi022.stdout:8/497: symlink d3/d12/d17/lbc 0 2024-02-16T19:29:08.007 INFO:tasks.workunit.client.0.smithi022.stdout:5/499: dwrite dd/d1f/d4a/f83 [0,4194304] 0 2024-02-16T19:29:08.008 INFO:tasks.workunit.client.1.smithi117.stdout:0/533: dread - d0/d30/f86 zero size 2024-02-16T19:29:08.009 INFO:tasks.workunit.client.0.smithi022.stdout:5/500: chown dd/d1f/d48/d21/d2f/d40/d49/f4c 142 1 2024-02-16T19:29:08.011 INFO:tasks.workunit.client.0.smithi022.stdout:5/501: write dd/d1f/d48/f60 [657486,80221] 0 2024-02-16T19:29:08.011 INFO:tasks.workunit.client.1.smithi117.stdout:3/586: mknod d1/dd/d5c/d85/cd1 0 2024-02-16T19:29:08.012 INFO:tasks.workunit.client.1.smithi117.stdout:3/587: write d1/d1b/f41 [1591001,59792] 0 2024-02-16T19:29:08.017 INFO:tasks.workunit.client.0.smithi022.stdout:7/527: mknod d9/da/d1f/d44/caa 0 2024-02-16T19:29:08.018 INFO:tasks.workunit.client.0.smithi022.stdout:7/528: chown d9/da/d4e/f89 6 1 2024-02-16T19:29:08.019 INFO:tasks.workunit.client.0.smithi022.stdout:9/547: symlink db/d14/d19/d1a/lbb 0 2024-02-16T19:29:08.019 INFO:tasks.workunit.client.1.smithi117.stdout:3/588: dwrite d1/dd/d3e/d7d/fbe [0,4194304] 0 2024-02-16T19:29:08.020 INFO:tasks.workunit.client.1.smithi117.stdout:9/417: link dd/d23/d64/f61 dd/d45/d31/f93 0 2024-02-16T19:29:08.020 INFO:tasks.workunit.client.1.smithi117.stdout:3/589: fdatasync d1/d30/f77 0 2024-02-16T19:29:08.022 INFO:tasks.workunit.client.1.smithi117.stdout:1/453: symlink d2/d19/d59/d89/d69/d80/l98 0 2024-02-16T19:29:08.023 INFO:tasks.workunit.client.1.smithi117.stdout:4/494: mknod d0/da/d21/d3d/d28/ca3 0 2024-02-16T19:29:08.024 INFO:tasks.workunit.client.0.smithi022.stdout:8/498: truncate d3/d25/f69 424636 0 2024-02-16T19:29:08.024 INFO:tasks.workunit.client.1.smithi117.stdout:8/499: chown d4/d7/c2e 237652367 1 2024-02-16T19:29:08.026 INFO:tasks.workunit.client.0.smithi022.stdout:7/529: dwrite d9/fa7 [0,4194304] 0 2024-02-16T19:29:08.028 INFO:tasks.workunit.client.0.smithi022.stdout:5/502: mkdir dd/d1f/d48/d21/d2f/d40/d49/daa 0 2024-02-16T19:29:08.031 INFO:tasks.workunit.client.1.smithi117.stdout:5/568: symlink d2/d4c/d4f/d67/lbd 0 2024-02-16T19:29:08.032 INFO:tasks.workunit.client.1.smithi117.stdout:7/481: write d4/f10 [2014319,56498] 0 2024-02-16T19:29:08.032 INFO:tasks.workunit.client.0.smithi022.stdout:4/493: link d4/d6/dd/f60 d4/d38/d8d/f99 0 2024-02-16T19:29:08.036 INFO:tasks.workunit.client.0.smithi022.stdout:0/643: rename d10/f6d to d10/d11/fd5 0 2024-02-16T19:29:08.040 INFO:tasks.workunit.client.1.smithi117.stdout:4/495: dread d0/da/d21/d3d/d28/d64/f1f [0,4194304] 0 2024-02-16T19:29:08.044 INFO:tasks.workunit.client.0.smithi022.stdout:1/610: rename d6/d8 to d6/d8/dbc 22 2024-02-16T19:29:08.045 INFO:tasks.workunit.client.1.smithi117.stdout:3/590: dread d1/d31/f7e [0,4194304] 0 2024-02-16T19:29:08.046 INFO:tasks.workunit.client.0.smithi022.stdout:1/611: truncate d6/d8/d22/f9a 845269 0 2024-02-16T19:29:08.048 INFO:tasks.workunit.client.1.smithi117.stdout:9/418: chown l8 100 1 2024-02-16T19:29:08.052 INFO:tasks.workunit.client.1.smithi117.stdout:9/419: dread dd/d23/f1a [0,4194304] 0 2024-02-16T19:29:08.054 INFO:tasks.workunit.client.1.smithi117.stdout:2/477: dread d0/d3/d22/f5d [0,4194304] 0 2024-02-16T19:29:08.055 INFO:tasks.workunit.client.1.smithi117.stdout:1/454: mknod d2/d19/d59/d76/c99 0 2024-02-16T19:29:08.055 INFO:tasks.workunit.client.0.smithi022.stdout:9/548: dread db/d14/d19/da8/f81 [0,4194304] 0 2024-02-16T19:29:08.056 INFO:tasks.workunit.client.0.smithi022.stdout:8/499: creat d3/d12/d2b/da9/fbd x:0 0 0 2024-02-16T19:29:08.057 INFO:tasks.workunit.client.0.smithi022.stdout:9/549: read f5 [3369640,74480] 0 2024-02-16T19:29:08.057 INFO:tasks.workunit.client.0.smithi022.stdout:8/500: chown d3/d12/d17/lbc 26 1 2024-02-16T19:29:08.058 INFO:tasks.workunit.client.1.smithi117.stdout:6/487: write d5/d4b/d20/d2d/f3c [1109351,35399] 0 2024-02-16T19:29:08.059 INFO:tasks.workunit.client.1.smithi117.stdout:6/488: fsync d5/db/d7d/d32/f3f 0 2024-02-16T19:29:08.060 INFO:tasks.workunit.client.1.smithi117.stdout:7/482: creat d4/d85/fb6 x:0 0 0 2024-02-16T19:29:08.061 INFO:tasks.workunit.client.1.smithi117.stdout:0/534: write d0/d9/d14/d1d/d60/d76/d7b/d8d/f32 [691135,23958] 0 2024-02-16T19:29:08.062 INFO:tasks.workunit.client.1.smithi117.stdout:7/483: write d4/f5 [1014718,22406] 0 2024-02-16T19:29:08.062 INFO:tasks.workunit.client.1.smithi117.stdout:5/569: creat d2/d9/d3c/d24/d50/d96/dbc/fbe x:0 0 0 2024-02-16T19:29:08.063 INFO:tasks.workunit.client.1.smithi117.stdout:5/570: read - d2/d5d/fa9 zero size 2024-02-16T19:29:08.064 INFO:tasks.workunit.client.0.smithi022.stdout:0/644: write d10/d11/d79/d8b/fa5 [1032173,115756] 0 2024-02-16T19:29:08.065 INFO:tasks.workunit.client.0.smithi022.stdout:7/530: truncate d9/da/d3b/f41 1069624 0 2024-02-16T19:29:08.066 INFO:tasks.workunit.client.0.smithi022.stdout:9/550: symlink db/d14/lbc 0 2024-02-16T19:29:08.067 INFO:tasks.workunit.client.1.smithi117.stdout:5/571: dread d2/d3/d21/f92 [0,4194304] 0 2024-02-16T19:29:08.068 INFO:tasks.workunit.client.0.smithi022.stdout:1/612: dread d6/d8/d22/d16/f1e [4194304,4194304] 0 2024-02-16T19:29:08.069 INFO:tasks.workunit.client.0.smithi022.stdout:8/501: readlink d3/d12/d2b/l4e 0 2024-02-16T19:29:08.071 INFO:tasks.workunit.client.1.smithi117.stdout:8/500: write d4/d12/d31/f40 [4148117,88758] 0 2024-02-16T19:29:08.073 INFO:tasks.workunit.client.0.smithi022.stdout:0/645: unlink d10/d11/d50/d65/fb8 0 2024-02-16T19:29:08.073 INFO:tasks.workunit.client.1.smithi117.stdout:8/501: dread - d4/d7/d37/d3a/d54/f78 zero size 2024-02-16T19:29:08.074 INFO:tasks.workunit.client.0.smithi022.stdout:0/646: readlink d10/d15/d23/d8f/dc1/l6c 0 2024-02-16T19:29:08.074 INFO:tasks.workunit.client.1.smithi117.stdout:8/502: chown d4/d12/l58 12186 1 2024-02-16T19:29:08.077 INFO:tasks.workunit.client.1.smithi117.stdout:9/420: fdatasync dd/f4a 0 2024-02-16T19:29:08.077 INFO:tasks.workunit.client.0.smithi022.stdout:8/502: truncate d3/d12/da6/f27 1137179 0 2024-02-16T19:29:08.078 INFO:tasks.workunit.client.1.smithi117.stdout:2/478: rmdir d0/d13/d75 39 2024-02-16T19:29:08.081 INFO:tasks.workunit.client.1.smithi117.stdout:1/455: mkdir d2/d19/d59/d89/d69/d9a 0 2024-02-16T19:29:08.081 INFO:tasks.workunit.client.1.smithi117.stdout:2/479: dread d0/d29/d56/f72 [0,4194304] 0 2024-02-16T19:29:08.082 INFO:tasks.workunit.client.0.smithi022.stdout:7/531: fdatasync d9/da/d2c/d2f/f35 0 2024-02-16T19:29:08.082 INFO:tasks.workunit.client.1.smithi117.stdout:1/456: chown d2/d1d 230 1 2024-02-16T19:29:08.082 INFO:tasks.workunit.client.1.smithi117.stdout:2/480: chown d0/c3d 20736600 1 2024-02-16T19:29:08.084 INFO:tasks.workunit.client.1.smithi117.stdout:2/481: dread d0/d3/d34/f7a [0,4194304] 0 2024-02-16T19:29:08.086 INFO:tasks.workunit.client.0.smithi022.stdout:9/551: dread db/d18/f73 [4194304,4194304] 0 2024-02-16T19:29:08.091 INFO:tasks.workunit.client.1.smithi117.stdout:6/489: mknod d5/db/d7d/d32/d79/ca0 0 2024-02-16T19:29:08.091 INFO:tasks.workunit.client.0.smithi022.stdout:0/647: dread d10/d22/f45 [0,4194304] 0 2024-02-16T19:29:08.093 INFO:tasks.workunit.client.0.smithi022.stdout:8/503: read d3/d25/f69 [250933,114325] 0 2024-02-16T19:29:08.093 INFO:tasks.workunit.client.1.smithi117.stdout:6/490: write d5/db/d7d/d32/d95/d49/d59/f99 [474854,125957] 0 2024-02-16T19:29:08.093 INFO:tasks.workunit.client.0.smithi022.stdout:0/648: chown d10/d15/d23/d38/d5a/f5d 14 1 2024-02-16T19:29:08.094 INFO:tasks.workunit.client.0.smithi022.stdout:0/649: fdatasync d10/d11/d27/d3f/f5b 0 2024-02-16T19:29:08.097 INFO:tasks.workunit.client.1.smithi117.stdout:0/535: fdatasync d0/d9/d14/d1d/f88 0 2024-02-16T19:29:08.098 INFO:tasks.workunit.client.1.smithi117.stdout:0/536: chown d0/d9/d14/d1d/d1e/f21 3843603 1 2024-02-16T19:29:08.099 INFO:tasks.workunit.client.1.smithi117.stdout:0/537: readlink d0/d9/d14/d1d/d60/d76/d7b/d8d/l57 0 2024-02-16T19:29:08.100 INFO:tasks.workunit.client.1.smithi117.stdout:7/484: symlink d4/d1c/d87/d89/d5f/lb7 0 2024-02-16T19:29:08.101 INFO:tasks.workunit.client.1.smithi117.stdout:0/538: read d0/d9/d14/d1d/d60/d76/f9f [327447,56556] 0 2024-02-16T19:29:08.102 INFO:tasks.workunit.client.0.smithi022.stdout:1/613: truncate d6/d8/d22/f9a 73571 0 2024-02-16T19:29:08.103 INFO:tasks.workunit.client.0.smithi022.stdout:1/614: truncate d6/d8/d22/d16/f40 4862897 0 2024-02-16T19:29:08.105 INFO:tasks.workunit.client.1.smithi117.stdout:6/491: dread d5/f11 [0,4194304] 0 2024-02-16T19:29:08.107 INFO:tasks.workunit.client.1.smithi117.stdout:4/496: unlink d0/da/d21/d3d/d28/d66/d97/l7c 0 2024-02-16T19:29:08.113 INFO:tasks.workunit.client.1.smithi117.stdout:7/485: dread d4/d1c/d87/d89/d5f/f7b [0,4194304] 0 2024-02-16T19:29:08.115 INFO:tasks.workunit.client.1.smithi117.stdout:8/503: unlink d4/d1c/l22 0 2024-02-16T19:29:08.117 INFO:tasks.workunit.client.0.smithi022.stdout:0/650: creat d10/d15/d23/d38/d5a/d66/fd6 x:0 0 0 2024-02-16T19:29:08.118 INFO:tasks.workunit.client.0.smithi022.stdout:0/651: fdatasync d10/d22/d2c/f72 0 2024-02-16T19:29:08.120 INFO:tasks.workunit.client.1.smithi117.stdout:9/421: creat dd/d45/d73/f94 x:0 0 0 2024-02-16T19:29:08.123 INFO:tasks.workunit.client.0.smithi022.stdout:1/615: unlink d6/d8/d22/d47/d54/f79 0 2024-02-16T19:29:08.127 INFO:tasks.workunit.client.0.smithi022.stdout:0/652: mkdir d10/d22/dad/dd7 0 2024-02-16T19:29:08.131 INFO:tasks.workunit.client.0.smithi022.stdout:9/552: creat db/d14/d33/fbd x:0 0 0 2024-02-16T19:29:08.131 INFO:tasks.workunit.client.1.smithi117.stdout:5/572: dwrite d2/d9/d3c/d24/f53 [0,4194304] 0 2024-02-16T19:29:08.133 INFO:tasks.workunit.client.1.smithi117.stdout:3/591: truncate d1/dd/d8e/f94 75692 0 2024-02-16T19:29:08.135 INFO:tasks.workunit.client.1.smithi117.stdout:0/539: mkdir d0/d9/d14/d1d/d60/dc3 0 2024-02-16T19:29:08.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:07 smithi117 ceph-mon[35309]: pgmap v7: 65 pgs: 65 active+clean; 2.0 GiB data, 5.8 GiB used, 531 GiB / 536 GiB avail; 65 MiB/s rd, 164 MiB/s wr, 343 op/s 2024-02-16T19:29:08.138 INFO:tasks.workunit.client.0.smithi022.stdout:8/504: creat d3/d12/d17/fbe x:0 0 0 2024-02-16T19:29:08.138 INFO:tasks.workunit.client.0.smithi022.stdout:7/532: write d9/da/f62 [1276,72237] 0 2024-02-16T19:29:08.138 INFO:tasks.workunit.client.1.smithi117.stdout:0/540: write d0/d9/d14/d1d/d60/d76/d7b/fb1 [514497,41475] 0 2024-02-16T19:29:08.139 INFO:tasks.workunit.client.1.smithi117.stdout:0/541: stat d0/l2b 0 2024-02-16T19:29:08.140 INFO:tasks.workunit.client.1.smithi117.stdout:6/492: unlink d5/d4b/d44/d52/l65 0 2024-02-16T19:29:08.141 INFO:tasks.workunit.client.1.smithi117.stdout:4/497: rmdir d0/da/d4a 39 2024-02-16T19:29:08.142 INFO:tasks.workunit.client.1.smithi117.stdout:3/592: dwrite d1/dd/d3b/d9d/fcd [0,4194304] 0 2024-02-16T19:29:08.142 INFO:tasks.workunit.client.1.smithi117.stdout:7/486: mkdir d4/d1c/d39/d46/d55/db8 0 2024-02-16T19:29:08.144 INFO:tasks.workunit.client.1.smithi117.stdout:7/487: dread - d4/d13/da3/fa7 zero size 2024-02-16T19:29:08.144 INFO:tasks.workunit.client.0.smithi022.stdout:8/505: fdatasync d3/d25/d33/f37 0 2024-02-16T19:29:08.145 INFO:tasks.workunit.client.1.smithi117.stdout:3/593: chown d1/d1b/d1e/lcc 62002 1 2024-02-16T19:29:08.146 INFO:tasks.workunit.client.0.smithi022.stdout:0/653: link d10/d11/d50/fc3 d10/d15/fd8 0 2024-02-16T19:29:08.152 INFO:tasks.workunit.client.0.smithi022.stdout:8/506: stat d3/d12/da6/f27 0 2024-02-16T19:29:08.156 INFO:tasks.workunit.client.1.smithi117.stdout:6/493: creat d5/db/d7d/d32/d37/fa1 x:0 0 0 2024-02-16T19:29:08.158 INFO:tasks.workunit.client.1.smithi117.stdout:7/488: read - d4/d1c/d39/d46/d47/f77 zero size 2024-02-16T19:29:08.159 INFO:tasks.workunit.client.1.smithi117.stdout:1/457: creat d2/f9b x:0 0 0 2024-02-16T19:29:08.160 INFO:tasks.workunit.client.1.smithi117.stdout:2/482: rename d0/fd to d0/d3/d10/f95 0 2024-02-16T19:29:08.161 INFO:tasks.workunit.client.1.smithi117.stdout:1/458: dread - d2/d1d/d60/d68/f8a zero size 2024-02-16T19:29:08.161 INFO:tasks.workunit.client.1.smithi117.stdout:3/594: dread d1/d1b/d1e/d37/f9c [0,4194304] 0 2024-02-16T19:29:08.163 INFO:tasks.workunit.client.1.smithi117.stdout:0/542: link d0/d9/d22/d27/d4e/fac d0/d9/d22/d27/d31/fc4 0 2024-02-16T19:29:08.165 INFO:tasks.workunit.client.1.smithi117.stdout:0/543: stat d0/d9/d14/d23/d33/l7a 0 2024-02-16T19:29:08.165 INFO:tasks.workunit.client.1.smithi117.stdout:3/595: rmdir d1/d1b/d1e/d29 39 2024-02-16T19:29:08.167 INFO:tasks.workunit.client.1.smithi117.stdout:3/596: readlink d1/d1b/d1e/lcc 0 2024-02-16T19:29:08.167 INFO:tasks.workunit.client.1.smithi117.stdout:2/483: rename d0/d3/d22/d37/f48 to d0/d13/d4d/d53/f96 0 2024-02-16T19:29:08.168 INFO:tasks.workunit.client.1.smithi117.stdout:6/494: dwrite d5/db/d7d/d32/d37/f41 [0,4194304] 0 2024-02-16T19:29:08.169 INFO:tasks.workunit.client.1.smithi117.stdout:7/489: link d4/d13/d63/fa2 d4/d1c/d39/d46/d6e/fb9 0 2024-02-16T19:29:08.169 INFO:tasks.workunit.client.1.smithi117.stdout:4/498: dwrite d0/da/d21/d3d/d28/d66/f7f [4194304,4194304] 0 2024-02-16T19:29:08.170 INFO:tasks.workunit.client.0.smithi022.stdout:3/458: sync 2024-02-16T19:29:08.170 INFO:tasks.workunit.client.0.smithi022.stdout:2/457: sync 2024-02-16T19:29:08.170 INFO:tasks.workunit.client.1.smithi117.stdout:3/597: chown d1/d1b/d1e/d37/f86 33259547 1 2024-02-16T19:29:08.171 INFO:tasks.workunit.client.0.smithi022.stdout:0/654: dwrite d10/f6e [0,4194304] 0 2024-02-16T19:29:08.172 INFO:tasks.workunit.client.0.smithi022.stdout:7/533: creat d9/da/fab x:0 0 0 2024-02-16T19:29:08.173 INFO:tasks.workunit.client.0.smithi022.stdout:8/507: readlink d3/d12/d6a/lb5 0 2024-02-16T19:29:08.174 INFO:tasks.workunit.client.1.smithi117.stdout:3/598: read d1/dd/f14 [2744100,39109] 0 2024-02-16T19:29:08.174 INFO:tasks.workunit.client.0.smithi022.stdout:3/459: fdatasync d7/d12/d18/dd/d24/f9f 0 2024-02-16T19:29:08.175 INFO:tasks.workunit.client.0.smithi022.stdout:8/508: chown d3/d12 4086 1 2024-02-16T19:29:08.176 INFO:tasks.workunit.client.0.smithi022.stdout:7/534: fdatasync d9/da/d2c/d88/fa8 0 2024-02-16T19:29:08.177 INFO:tasks.workunit.client.1.smithi117.stdout:2/484: unlink d0/d3b/f79 0 2024-02-16T19:29:08.177 INFO:tasks.workunit.client.0.smithi022.stdout:3/460: chown d7/d12/d18/dd/d3c/c8c 876 1 2024-02-16T19:29:08.177 INFO:tasks.workunit.client.0.smithi022.stdout:0/655: mknod d10/d7d/dac/cd9 0 2024-02-16T19:29:08.178 INFO:tasks.workunit.client.0.smithi022.stdout:7/535: write d9/d5d/f91 [795597,54252] 0 2024-02-16T19:29:08.180 INFO:tasks.workunit.client.0.smithi022.stdout:3/461: chown d7/d12/d59/f60 13 1 2024-02-16T19:29:08.180 INFO:tasks.workunit.client.0.smithi022.stdout:7/536: write d9/da/fab [990034,83198] 0 2024-02-16T19:29:08.183 INFO:tasks.workunit.client.1.smithi117.stdout:7/490: creat d4/d1c/d87/d89/d60/fba x:0 0 0 2024-02-16T19:29:08.184 INFO:tasks.workunit.client.1.smithi117.stdout:4/499: readlink d0/da/d21/d3d/d28/d64/d48/l82 0 2024-02-16T19:29:08.187 INFO:tasks.workunit.client.1.smithi117.stdout:7/491: dread d4/d13/d63/f6b [0,4194304] 0 2024-02-16T19:29:08.187 INFO:tasks.workunit.client.0.smithi022.stdout:1/616: dread d6/d8/d22/d47/f4e [0,4194304] 0 2024-02-16T19:29:08.188 INFO:tasks.workunit.client.1.smithi117.stdout:7/492: dread - d4/d1c/d87/d89/faa zero size 2024-02-16T19:29:08.188 INFO:tasks.workunit.client.1.smithi117.stdout:2/485: dwrite d0/d3/d34/d94/d46/d68/f8f [0,4194304] 0 2024-02-16T19:29:08.191 INFO:tasks.workunit.client.0.smithi022.stdout:3/462: dread d7/d12/d18/dac/faa [0,4194304] 0 2024-02-16T19:29:08.191 INFO:tasks.workunit.client.0.smithi022.stdout:2/458: dwrite d0/d32/f97 [0,4194304] 0 2024-02-16T19:29:08.193 INFO:tasks.workunit.client.0.smithi022.stdout:3/463: chown d7/d12/d18/f91 12883 1 2024-02-16T19:29:08.193 INFO:tasks.workunit.client.1.smithi117.stdout:6/495: creat d5/db/d7d/d32/d95/d72/fa2 x:0 0 0 2024-02-16T19:29:08.201 INFO:tasks.workunit.client.0.smithi022.stdout:0/656: rmdir d10/d11/d27/d3f/d58/dab 39 2024-02-16T19:29:08.205 INFO:tasks.workunit.client.1.smithi117.stdout:3/599: rmdir d1/d30/d80 39 2024-02-16T19:29:08.205 INFO:tasks.workunit.client.1.smithi117.stdout:1/459: getdents d2/d1d/d1f 0 2024-02-16T19:29:08.207 INFO:tasks.workunit.client.0.smithi022.stdout:7/537: rename d9/da/d2c/c33 to d9/d1b/d23/cac 0 2024-02-16T19:29:08.208 INFO:tasks.workunit.client.0.smithi022.stdout:7/538: fsync d9/da/d2c/d46/f9a 0 2024-02-16T19:29:08.211 INFO:tasks.workunit.client.1.smithi117.stdout:1/460: dwrite d2/d19/d59/d89/d49/f4a [0,4194304] 0 2024-02-16T19:29:08.215 INFO:tasks.workunit.client.0.smithi022.stdout:7/539: dwrite d9/d5d/f91 [0,4194304] 0 2024-02-16T19:29:08.217 INFO:tasks.workunit.client.0.smithi022.stdout:1/617: symlink d6/d8/d22/d16/d17/d26/lbd 0 2024-02-16T19:29:08.219 INFO:tasks.workunit.client.0.smithi022.stdout:1/618: stat d6/d8/d6c/d96 0 2024-02-16T19:29:08.221 INFO:tasks.workunit.client.1.smithi117.stdout:7/493: unlink d4/f3a 0 2024-02-16T19:29:08.222 INFO:tasks.workunit.client.0.smithi022.stdout:7/540: dread d9/da/f62 [0,4194304] 0 2024-02-16T19:29:08.225 INFO:tasks.workunit.client.0.smithi022.stdout:3/464: creat d7/d12/d18/dd/d31/fae x:0 0 0 2024-02-16T19:29:08.232 INFO:tasks.workunit.client.0.smithi022.stdout:9/553: dwrite db/d14/d16/d21/d37/f4e [0,4194304] 0 2024-02-16T19:29:08.241 INFO:tasks.workunit.client.1.smithi117.stdout:9/422: write dd/d47/d5e/f7f [5763511,19018] 0 2024-02-16T19:29:08.242 INFO:tasks.workunit.client.1.smithi117.stdout:9/423: chown dd/d47/d48 118328 1 2024-02-16T19:29:08.245 INFO:tasks.workunit.client.1.smithi117.stdout:5/573: write d2/d3/d21/d49/f82 [3225732,69085] 0 2024-02-16T19:29:08.246 INFO:tasks.workunit.client.1.smithi117.stdout:0/544: fsync d0/d30/f66 0 2024-02-16T19:29:08.248 INFO:tasks.workunit.client.0.smithi022.stdout:2/459: mkdir d0/d2/d18/d1f/d6c/d99 0 2024-02-16T19:29:08.249 INFO:tasks.workunit.client.0.smithi022.stdout:2/460: stat d0/d2/d18/d38/d6b/f8b 0 2024-02-16T19:29:08.250 INFO:tasks.workunit.client.0.smithi022.stdout:2/461: write d0/d2/f85 [496272,88891] 0 2024-02-16T19:29:08.251 INFO:tasks.workunit.client.0.smithi022.stdout:7/541: symlink d9/d1b/d23/lad 0 2024-02-16T19:29:08.252 INFO:tasks.workunit.client.1.smithi117.stdout:0/545: dwrite d0/d9/d14/d1d/d60/d76/d7b/d8d/f97 [0,4194304] 0 2024-02-16T19:29:08.253 INFO:tasks.workunit.client.1.smithi117.stdout:2/486: symlink d0/d3/d34/d94/d46/d68/d76/l97 0 2024-02-16T19:29:08.257 INFO:tasks.workunit.client.1.smithi117.stdout:9/424: dread - dd/d2d/f44 zero size 2024-02-16T19:29:08.258 INFO:tasks.workunit.client.1.smithi117.stdout:2/487: dwrite d0/d3/f2a [0,4194304] 0 2024-02-16T19:29:08.260 INFO:tasks.workunit.client.1.smithi117.stdout:4/500: creat d0/da/fa4 x:0 0 0 2024-02-16T19:29:08.262 INFO:tasks.workunit.client.1.smithi117.stdout:4/501: write d0/da/d21/d3d/d28/d66/d97/f94 [3356821,105630] 0 2024-02-16T19:29:08.264 INFO:tasks.workunit.client.1.smithi117.stdout:2/488: dwrite d0/d3b/f5b [0,4194304] 0 2024-02-16T19:29:08.267 INFO:tasks.workunit.client.0.smithi022.stdout:2/462: creat d0/d32/d3b/f9a x:0 0 0 2024-02-16T19:29:08.269 INFO:tasks.workunit.client.0.smithi022.stdout:2/463: read d0/d2/f3d [605192,25443] 0 2024-02-16T19:29:08.270 INFO:tasks.workunit.client.0.smithi022.stdout:7/542: truncate d9/f25 2254965 0 2024-02-16T19:29:08.271 INFO:tasks.workunit.client.0.smithi022.stdout:2/464: write d0/f37 [4209705,118980] 0 2024-02-16T19:29:08.276 INFO:tasks.workunit.client.0.smithi022.stdout:7/543: dread d9/fa7 [0,4194304] 0 2024-02-16T19:29:08.280 INFO:tasks.workunit.client.1.smithi117.stdout:6/496: write d5/fa [3330551,513] 0 2024-02-16T19:29:08.281 INFO:tasks.workunit.client.1.smithi117.stdout:0/546: symlink d0/d9/d14/d1d/d60/d76/lc5 0 2024-02-16T19:29:08.291 INFO:tasks.workunit.client.0.smithi022.stdout:0/657: dwrite d10/d22/f20 [0,4194304] 0 2024-02-16T19:29:08.292 INFO:tasks.workunit.client.0.smithi022.stdout:0/658: write d10/d22/dad/fd0 [947953,121984] 0 2024-02-16T19:29:08.295 INFO:tasks.workunit.client.1.smithi117.stdout:7/494: link d4/d1c/c41 d4/d1c/d98/cbb 0 2024-02-16T19:29:08.297 INFO:tasks.workunit.client.1.smithi117.stdout:7/495: write d4/d13/f18 [1049602,77890] 0 2024-02-16T19:29:08.297 INFO:tasks.workunit.client.0.smithi022.stdout:0/659: unlink d10/d11/d27/d3f/d94/lb0 0 2024-02-16T19:29:08.298 INFO:tasks.workunit.client.1.smithi117.stdout:7/496: readlink d4/d1c/d87/d89/l58 0 2024-02-16T19:29:08.299 INFO:tasks.workunit.client.0.smithi022.stdout:0/660: write d10/d11/fc9 [184760,3541] 0 2024-02-16T19:29:08.299 INFO:tasks.workunit.client.0.smithi022.stdout:1/619: truncate d6/d8/d22/d16/f4b 873806 0 2024-02-16T19:29:08.300 INFO:tasks.workunit.client.1.smithi117.stdout:0/547: fdatasync d0/d9/d14/f92 0 2024-02-16T19:29:08.301 INFO:tasks.workunit.client.1.smithi117.stdout:3/600: getdents d1/dd/d3b/dba 0 2024-02-16T19:29:08.301 INFO:tasks.workunit.client.1.smithi117.stdout:0/548: stat d0/d9/d14/d23/d33/lba 0 2024-02-16T19:29:08.304 INFO:tasks.workunit.client.1.smithi117.stdout:5/574: rename d2/d9/d3c/d42/l66 to d2/d9/d3c/lbf 0 2024-02-16T19:29:08.306 INFO:tasks.workunit.client.0.smithi022.stdout:3/465: truncate f5 570821 0 2024-02-16T19:29:08.307 INFO:tasks.workunit.client.0.smithi022.stdout:0/661: fdatasync d10/d22/f2a 0 2024-02-16T19:29:08.309 INFO:tasks.workunit.client.1.smithi117.stdout:2/489: creat d0/d3/d34/d94/d91/f98 x:0 0 0 2024-02-16T19:29:08.309 INFO:tasks.workunit.client.0.smithi022.stdout:8/509: truncate d3/d12/d2b/d54/f95 1099891 0 2024-02-16T19:29:08.310 INFO:tasks.workunit.client.1.smithi117.stdout:2/490: dread - d0/d13/d4d/d53/d58/f78 zero size 2024-02-16T19:29:08.311 INFO:tasks.workunit.client.0.smithi022.stdout:1/620: creat d6/d8/d22/d16/d17/d1d/fbe x:0 0 0 2024-02-16T19:29:08.311 INFO:tasks.workunit.client.0.smithi022.stdout:1/621: dread - d6/d8/d22/d67/f9f zero size 2024-02-16T19:29:08.313 INFO:tasks.workunit.client.0.smithi022.stdout:3/466: creat d7/d12/d18/dd/d31/d7f/faf x:0 0 0 2024-02-16T19:29:08.314 INFO:tasks.workunit.client.1.smithi117.stdout:6/497: mknod d5/d9c/ca3 0 2024-02-16T19:29:08.314 INFO:tasks.workunit.client.1.smithi117.stdout:2/491: dwrite d0/d3/d34/d94/f6d [0,4194304] 0 2024-02-16T19:29:08.315 INFO:tasks.workunit.client.1.smithi117.stdout:6/498: readlink d5/db/d7d/l6a 0 2024-02-16T19:29:08.319 INFO:tasks.workunit.client.1.smithi117.stdout:2/492: dread d0/d3/f31 [0,4194304] 0 2024-02-16T19:29:08.323 INFO:tasks.workunit.client.1.smithi117.stdout:0/549: truncate d0/d9/d22/d27/d31/fc4 4133242 0 2024-02-16T19:29:08.334 INFO:tasks.workunit.client.1.smithi117.stdout:8/504: sync 2024-02-16T19:29:08.337 INFO:tasks.workunit.client.1.smithi117.stdout:2/493: chown d0/d13/d4d/d53/f96 25747 1 2024-02-16T19:29:08.337 INFO:tasks.workunit.client.0.smithi022.stdout:8/510: write d3/d12/f9e [734443,99265] 0 2024-02-16T19:29:08.340 INFO:tasks.workunit.client.0.smithi022.stdout:0/662: getdents d10/d11/d27/d3f/d58 0 2024-02-16T19:29:08.342 INFO:tasks.workunit.client.1.smithi117.stdout:2/494: dwrite d0/d3/d22/f84 [0,4194304] 0 2024-02-16T19:29:08.343 INFO:tasks.workunit.client.0.smithi022.stdout:3/467: getdents d7/d12/d18/dd/d24/d29 0 2024-02-16T19:29:08.345 INFO:tasks.workunit.client.1.smithi117.stdout:5/575: mknod d2/d36/d6f/d9c/cc0 0 2024-02-16T19:29:08.347 INFO:tasks.workunit.client.1.smithi117.stdout:4/502: getdents d0/da/d21/d3d/d28/d66/d97 0 2024-02-16T19:29:08.347 INFO:tasks.workunit.client.0.smithi022.stdout:0/663: fdatasync d10/d22/d49/f4d 0 2024-02-16T19:29:08.348 INFO:tasks.workunit.client.1.smithi117.stdout:4/503: read - d0/da/d21/d3d/d28/d66/d97/d6d/f95 zero size 2024-02-16T19:29:08.349 INFO:tasks.workunit.client.1.smithi117.stdout:4/504: readlink d0/da/d21/d3d/d28/d64/d48/l82 0 2024-02-16T19:29:08.350 INFO:tasks.workunit.client.0.smithi022.stdout:3/468: rename d7/d12/d18/dd/c2a to d7/d21/cb0 0 2024-02-16T19:29:08.352 INFO:tasks.workunit.client.0.smithi022.stdout:3/469: chown d7/d12/d18/d4b/f6f 5 1 2024-02-16T19:29:08.361 INFO:tasks.workunit.client.0.smithi022.stdout:0/664: rename d10/d11/d50/d65/lb5 to d10/d22/d49/lda 0 2024-02-16T19:29:08.364 INFO:tasks.workunit.client.1.smithi117.stdout:0/550: creat d0/d9/d14/dbc/fc6 x:0 0 0 2024-02-16T19:29:08.365 INFO:tasks.workunit.client.0.smithi022.stdout:8/511: write d3/d12/d17/f83 [578673,112179] 0 2024-02-16T19:29:08.365 INFO:tasks.workunit.client.1.smithi117.stdout:8/505: write d4/d12/f1a [2744400,21759] 0 2024-02-16T19:29:08.366 INFO:tasks.workunit.client.1.smithi117.stdout:8/506: write d4/d7/d37/f56 [220710,70914] 0 2024-02-16T19:29:08.367 INFO:tasks.workunit.client.1.smithi117.stdout:9/425: rename dd/f3e to dd/d23/f95 0 2024-02-16T19:29:08.367 INFO:tasks.workunit.client.1.smithi117.stdout:9/426: dread - dd/d23/d64/f61 zero size 2024-02-16T19:29:08.369 INFO:tasks.workunit.client.1.smithi117.stdout:9/427: write dd/f19 [3549684,112756] 0 2024-02-16T19:29:08.370 INFO:tasks.workunit.client.1.smithi117.stdout:2/495: creat d0/d13/d4d/d53/d58/f99 x:0 0 0 2024-02-16T19:29:08.372 INFO:tasks.workunit.client.1.smithi117.stdout:5/576: mkdir d2/d5d/d8e/dc1 0 2024-02-16T19:29:08.376 INFO:tasks.workunit.client.1.smithi117.stdout:7/497: getdents d4/d13 0 2024-02-16T19:29:08.381 INFO:tasks.workunit.client.1.smithi117.stdout:0/551: stat d0/cb5 0 2024-02-16T19:29:08.382 INFO:tasks.workunit.client.1.smithi117.stdout:0/552: write d0/d9/d14/d1d/f9e [160247,70894] 0 2024-02-16T19:29:08.386 INFO:tasks.workunit.client.1.smithi117.stdout:8/507: fsync d4/d7/d37/d3a/d43/f6b 0 2024-02-16T19:29:08.392 INFO:tasks.workunit.client.1.smithi117.stdout:2/496: dread d0/d13/f15 [0,4194304] 0 2024-02-16T19:29:08.394 INFO:tasks.workunit.client.0.smithi022.stdout:6/507: sync 2024-02-16T19:29:08.395 INFO:tasks.workunit.client.0.smithi022.stdout:6/508: write d1/d8/f3b [1349059,52538] 0 2024-02-16T19:29:08.395 INFO:tasks.workunit.client.1.smithi117.stdout:3/601: getdents d1/d30/d80 0 2024-02-16T19:29:08.396 INFO:tasks.workunit.client.1.smithi117.stdout:3/602: readlink d1/d1b/d1e/l7f 0 2024-02-16T19:29:08.396 INFO:tasks.workunit.client.1.smithi117.stdout:3/603: chown d1/dd/d3b/lac 1003334 1 2024-02-16T19:29:08.397 INFO:tasks.workunit.client.1.smithi117.stdout:0/553: truncate d0/d9/d22/d27/d31/d6d/f82 3064971 0 2024-02-16T19:29:08.398 INFO:tasks.workunit.client.1.smithi117.stdout:0/554: chown d0/d9/d14/c93 56 1 2024-02-16T19:29:08.401 INFO:tasks.workunit.client.1.smithi117.stdout:8/508: creat d4/d1c/f98 x:0 0 0 2024-02-16T19:29:08.402 INFO:tasks.workunit.client.1.smithi117.stdout:9/428: creat dd/d45/d31/d63/d8e/f96 x:0 0 0 2024-02-16T19:29:08.408 INFO:tasks.workunit.client.1.smithi117.stdout:1/461: sync 2024-02-16T19:29:08.408 INFO:tasks.workunit.client.1.smithi117.stdout:3/604: rmdir d1/dd/d5c/d85 39 2024-02-16T19:29:08.409 INFO:tasks.workunit.client.1.smithi117.stdout:1/462: fsync d2/d19/d59/f97 0 2024-02-16T19:29:08.409 INFO:tasks.workunit.client.1.smithi117.stdout:3/605: dread - d1/d30/d52/fc9 zero size 2024-02-16T19:29:08.410 INFO:tasks.workunit.client.1.smithi117.stdout:5/577: dwrite d2/db3/f3f [0,4194304] 0 2024-02-16T19:29:08.420 INFO:tasks.workunit.client.1.smithi117.stdout:8/509: mknod d4/d7/d37/d3a/c99 0 2024-02-16T19:29:08.424 INFO:tasks.workunit.client.1.smithi117.stdout:7/498: truncate d4/d1c/d39/d46/d47/f56 1386836 0 2024-02-16T19:29:08.425 INFO:tasks.workunit.client.1.smithi117.stdout:5/578: mkdir d2/d4c/d69/dc2 0 2024-02-16T19:29:08.426 INFO:tasks.workunit.client.1.smithi117.stdout:7/499: readlink d4/d1c/d39/l3b 0 2024-02-16T19:29:08.426 INFO:tasks.workunit.client.0.smithi022.stdout:6/509: symlink d1/d8/da/d17/d8b/lb4 0 2024-02-16T19:29:08.426 INFO:tasks.workunit.client.1.smithi117.stdout:1/463: dread d2/d4/f15 [0,4194304] 0 2024-02-16T19:29:08.427 INFO:tasks.workunit.client.1.smithi117.stdout:8/510: dwrite d4/f52 [4194304,4194304] 0 2024-02-16T19:29:08.429 INFO:tasks.workunit.client.1.smithi117.stdout:7/500: write d4/d85/fad [433735,70516] 0 2024-02-16T19:29:08.429 INFO:tasks.workunit.client.1.smithi117.stdout:4/505: dwrite d0/da/d4a/f58 [0,4194304] 0 2024-02-16T19:29:08.429 INFO:tasks.workunit.client.1.smithi117.stdout:5/579: write d2/d9/d3c/f8d [313251,27145] 0 2024-02-16T19:29:08.431 INFO:tasks.workunit.client.1.smithi117.stdout:2/497: write d0/d29/f73 [1286004,79824] 0 2024-02-16T19:29:08.432 INFO:tasks.workunit.client.1.smithi117.stdout:9/429: creat dd/d23/f97 x:0 0 0 2024-02-16T19:29:08.432 INFO:tasks.workunit.client.1.smithi117.stdout:5/580: stat d2/d4c/d4f/c78 0 2024-02-16T19:29:08.436 INFO:tasks.workunit.client.0.smithi022.stdout:0/665: link d10/d11/d79/d8b/fb9 d10/d11/d27/fdb 0 2024-02-16T19:29:08.439 INFO:tasks.workunit.client.1.smithi117.stdout:9/430: dwrite dd/d47/f57 [0,4194304] 0 2024-02-16T19:29:08.439 INFO:tasks.workunit.client.1.smithi117.stdout:2/498: dwrite d0/d13/d4d/d53/f6f [0,4194304] 0 2024-02-16T19:29:08.439 INFO:tasks.workunit.client.0.smithi022.stdout:3/470: mknod d7/d12/d18/dd/d98/cb1 0 2024-02-16T19:29:08.439 INFO:tasks.workunit.client.1.smithi117.stdout:9/431: readlink dd/l10 0 2024-02-16T19:29:08.440 INFO:tasks.workunit.client.1.smithi117.stdout:2/499: chown d0/d3/f7c 51 1 2024-02-16T19:29:08.441 INFO:tasks.workunit.client.1.smithi117.stdout:2/500: readlink d0/d13/d4d/d53/l81 0 2024-02-16T19:29:08.441 INFO:tasks.workunit.client.0.smithi022.stdout:8/512: creat d3/d12/d2b/fbf x:0 0 0 2024-02-16T19:29:08.441 INFO:tasks.workunit.client.1.smithi117.stdout:8/511: mknod d4/d73/c9a 0 2024-02-16T19:29:08.442 INFO:tasks.workunit.client.1.smithi117.stdout:4/506: fsync d0/da/d21/f27 0 2024-02-16T19:29:08.444 INFO:tasks.workunit.client.0.smithi022.stdout:0/666: unlink d10/d11/d27/d3f/l91 0 2024-02-16T19:29:08.444 INFO:tasks.workunit.client.1.smithi117.stdout:8/512: truncate d4/d12/d2a/f5c 1530197 0 2024-02-16T19:29:08.444 INFO:tasks.workunit.client.1.smithi117.stdout:4/507: chown d0/da/d21/d3d/d28/f39 0 1 2024-02-16T19:29:08.448 INFO:tasks.workunit.client.1.smithi117.stdout:7/501: dread d4/d13/d63/f6b [0,4194304] 0 2024-02-16T19:29:08.449 INFO:tasks.workunit.client.0.smithi022.stdout:8/513: dread - d3/d12/d17/f6d zero size 2024-02-16T19:29:08.450 INFO:tasks.workunit.client.1.smithi117.stdout:3/606: link d1/d98/fa5 d1/d1b/d20/fd2 0 2024-02-16T19:29:08.451 INFO:tasks.workunit.client.1.smithi117.stdout:3/607: stat d1/d1b/d1e/d5e 0 2024-02-16T19:29:08.453 INFO:tasks.workunit.client.0.smithi022.stdout:0/667: dwrite d10/d11/d27/d3f/d94/faf [0,4194304] 0 2024-02-16T19:29:08.454 INFO:tasks.workunit.client.0.smithi022.stdout:0/668: stat d10/d22/d2c/l9c 0 2024-02-16T19:29:08.457 INFO:tasks.workunit.client.0.smithi022.stdout:3/471: mknod d7/d12/d18/dd/d24/cb2 0 2024-02-16T19:29:08.463 INFO:tasks.workunit.client.0.smithi022.stdout:8/514: mkdir d3/d12/d2b/d54/d74/da2/dc0 0 2024-02-16T19:29:08.463 INFO:tasks.workunit.client.0.smithi022.stdout:6/510: write d1/d8/d23/f30 [398778,34041] 0 2024-02-16T19:29:08.464 INFO:tasks.workunit.client.0.smithi022.stdout:8/515: fsync d3/d25/d33/fab 0 2024-02-16T19:29:08.465 INFO:tasks.workunit.client.0.smithi022.stdout:0/669: symlink d10/d11/d50/d65/ldc 0 2024-02-16T19:29:08.469 INFO:tasks.workunit.client.1.smithi117.stdout:9/432: rename dd/d45/f3c to dd/d47/d48/f98 0 2024-02-16T19:29:08.471 INFO:tasks.workunit.client.1.smithi117.stdout:9/433: dread dd/d47/d48/f98 [0,4194304] 0 2024-02-16T19:29:08.471 INFO:tasks.workunit.client.1.smithi117.stdout:2/501: truncate d0/d3/d22/f5d 1456412 0 2024-02-16T19:29:08.474 INFO:tasks.workunit.client.0.smithi022.stdout:0/670: dwrite d10/d11/d79/d8b/fc0 [0,4194304] 0 2024-02-16T19:29:08.475 INFO:tasks.workunit.client.1.smithi117.stdout:8/513: mknod d4/d7/d37/d3a/d43/c9b 0 2024-02-16T19:29:08.475 INFO:tasks.workunit.client.1.smithi117.stdout:6/499: sync 2024-02-16T19:29:08.477 INFO:tasks.workunit.client.1.smithi117.stdout:5/581: dread d2/d9/d3c/d24/d50/d52/f1e [0,4194304] 0 2024-02-16T19:29:08.477 INFO:tasks.workunit.client.1.smithi117.stdout:5/582: dread - d2/d3/fb6 zero size 2024-02-16T19:29:08.480 INFO:tasks.workunit.client.0.smithi022.stdout:8/516: symlink d3/d12/d2b/da9/lc1 0 2024-02-16T19:29:08.485 INFO:tasks.workunit.client.1.smithi117.stdout:7/502: symlink d4/d1c/d87/d89/d60/lbc 0 2024-02-16T19:29:08.487 INFO:tasks.workunit.client.0.smithi022.stdout:6/511: fsync d1/d8/da/d29/d61/d57/f60 0 2024-02-16T19:29:08.489 INFO:tasks.workunit.client.1.smithi117.stdout:3/608: rmdir d1/d31/d89 39 2024-02-16T19:29:08.492 INFO:tasks.workunit.client.0.smithi022.stdout:0/671: creat d10/d15/d23/dc6/fdd x:0 0 0 2024-02-16T19:29:08.494 INFO:tasks.workunit.client.0.smithi022.stdout:0/672: write d10/d11/d27/fbb [184295,94956] 0 2024-02-16T19:29:08.494 INFO:tasks.workunit.client.0.smithi022.stdout:3/472: dread d7/d12/f25 [0,4194304] 0 2024-02-16T19:29:08.494 INFO:tasks.workunit.client.1.smithi117.stdout:4/508: write d0/da/d21/d3d/d28/d64/f8b [1492910,101456] 0 2024-02-16T19:29:08.497 INFO:tasks.workunit.client.1.smithi117.stdout:9/434: readlink dd/d23/d64/l4c 0 2024-02-16T19:29:08.498 INFO:tasks.workunit.client.1.smithi117.stdout:9/435: truncate dd/d23/f97 504811 0 2024-02-16T19:29:08.498 INFO:tasks.workunit.client.1.smithi117.stdout:9/436: readlink dd/l14 0 2024-02-16T19:29:08.500 INFO:tasks.workunit.client.0.smithi022.stdout:8/517: creat d3/d12/d2b/d54/d97/fc2 x:0 0 0 2024-02-16T19:29:08.503 INFO:tasks.workunit.client.1.smithi117.stdout:2/502: write d0/d3/d22/f28 [11914238,99955] 0 2024-02-16T19:29:08.506 INFO:tasks.workunit.client.1.smithi117.stdout:3/609: rmdir d1/dd/d5c 39 2024-02-16T19:29:08.507 INFO:tasks.workunit.client.1.smithi117.stdout:1/464: getdents d2/d1d/d60 0 2024-02-16T19:29:08.508 INFO:tasks.workunit.client.1.smithi117.stdout:0/555: sync 2024-02-16T19:29:08.508 INFO:tasks.workunit.client.1.smithi117.stdout:4/509: creat d0/da/d21/d3d/d28/d64/d73/d91/fa5 x:0 0 0 2024-02-16T19:29:08.509 INFO:tasks.workunit.client.1.smithi117.stdout:9/437: creat dd/d45/d73/f99 x:0 0 0 2024-02-16T19:29:08.510 INFO:tasks.workunit.client.1.smithi117.stdout:6/500: mkdir d5/d4b/d20/da4 0 2024-02-16T19:29:08.511 INFO:tasks.workunit.client.1.smithi117.stdout:2/503: symlink d0/d13/d4d/d80/l9a 0 2024-02-16T19:29:08.512 INFO:tasks.workunit.client.1.smithi117.stdout:7/503: dread - d4/d1c/d87/d89/d60/d79/f7a zero size 2024-02-16T19:29:08.514 INFO:tasks.workunit.client.1.smithi117.stdout:0/556: creat d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/fc7 x:0 0 0 2024-02-16T19:29:08.514 INFO:tasks.workunit.client.1.smithi117.stdout:9/438: mkdir dd/d47/d5e/d9a 0 2024-02-16T19:29:08.515 INFO:tasks.workunit.client.1.smithi117.stdout:0/557: stat d0/d9/d14/d1d/d1e/f25 0 2024-02-16T19:29:08.515 INFO:tasks.workunit.client.1.smithi117.stdout:8/514: link d4/d7/d37/d3a/d43/c9b d4/d7/d6a/d72/d84/c9c 0 2024-02-16T19:29:08.515 INFO:tasks.workunit.client.0.smithi022.stdout:4/494: sync 2024-02-16T19:29:08.515 INFO:tasks.workunit.client.0.smithi022.stdout:5/503: sync 2024-02-16T19:29:08.515 INFO:tasks.workunit.client.1.smithi117.stdout:9/439: readlink dd/d45/d31/l46 0 2024-02-16T19:29:08.516 INFO:tasks.workunit.client.1.smithi117.stdout:5/583: mknod d2/d9/d3c/d24/d50/d96/dbc/d30/d4e/cc3 0 2024-02-16T19:29:08.516 INFO:tasks.workunit.client.1.smithi117.stdout:5/584: chown d2/d9/d3c/d24/d50/c75 7 1 2024-02-16T19:29:08.517 INFO:tasks.workunit.client.1.smithi117.stdout:7/504: creat d4/d13/d63/d71/fbd x:0 0 0 2024-02-16T19:29:08.522 INFO:tasks.workunit.client.1.smithi117.stdout:2/504: symlink d0/l9b 0 2024-02-16T19:29:08.523 INFO:tasks.workunit.client.1.smithi117.stdout:2/505: fdatasync d0/d3/d34/d94/f6d 0 2024-02-16T19:29:08.524 INFO:tasks.workunit.client.0.smithi022.stdout:5/504: chown dd 38528 1 2024-02-16T19:29:08.524 INFO:tasks.workunit.client.0.smithi022.stdout:0/673: mknod d10/d11/d50/cde 0 2024-02-16T19:29:08.524 INFO:tasks.workunit.client.1.smithi117.stdout:5/585: mkdir d2/d9/d3c/d24/d50/d96/dc4 0 2024-02-16T19:29:08.524 INFO:tasks.workunit.client.0.smithi022.stdout:6/512: getdents d1/d8/da/d7b/dae 0 2024-02-16T19:29:08.526 INFO:tasks.workunit.client.0.smithi022.stdout:0/674: readlink d10/d11/d27/d3f/d58/l5c 0 2024-02-16T19:29:08.526 INFO:tasks.workunit.client.1.smithi117.stdout:7/505: unlink d4/d13/fa1 0 2024-02-16T19:29:08.527 INFO:tasks.workunit.client.1.smithi117.stdout:7/506: chown d4/d1c/d39/d46/c52 4181956 1 2024-02-16T19:29:08.531 INFO:tasks.workunit.client.0.smithi022.stdout:5/505: creat dd/d1f/d76/fab x:0 0 0 2024-02-16T19:29:08.531 INFO:tasks.workunit.client.1.smithi117.stdout:6/501: write d5/db/d7d/f2b [3537550,43934] 0 2024-02-16T19:29:08.532 INFO:tasks.workunit.client.1.smithi117.stdout:9/440: dread dd/d2d/f4d [0,4194304] 0 2024-02-16T19:29:08.533 INFO:tasks.workunit.client.1.smithi117.stdout:6/502: write d5/fa [1296928,44126] 0 2024-02-16T19:29:08.533 INFO:tasks.workunit.client.1.smithi117.stdout:1/465: write d2/d19/d59/d89/f32 [761946,113770] 0 2024-02-16T19:29:08.535 INFO:tasks.workunit.client.1.smithi117.stdout:6/503: write d5/d4b/d20/f21 [1804934,113766] 0 2024-02-16T19:29:08.535 INFO:tasks.workunit.client.1.smithi117.stdout:1/466: chown d2/d1d/d60/l3c 16 1 2024-02-16T19:29:08.536 INFO:tasks.workunit.client.1.smithi117.stdout:1/467: stat d2/d19/d59/d89/d69/l70 0 2024-02-16T19:29:08.542 INFO:tasks.workunit.client.1.smithi117.stdout:2/506: dwrite d0/d13/d75/f59 [0,4194304] 0 2024-02-16T19:29:08.542 INFO:tasks.workunit.client.1.smithi117.stdout:7/507: creat d4/d1c/fbe x:0 0 0 2024-02-16T19:29:08.543 INFO:tasks.workunit.client.1.smithi117.stdout:3/610: link d1/dd/d3b/d9d/cb0 d1/d83/d96/cd3 0 2024-02-16T19:29:08.544 INFO:tasks.workunit.client.1.smithi117.stdout:7/508: chown d4/d85/fad 16519 1 2024-02-16T19:29:08.545 INFO:tasks.workunit.client.1.smithi117.stdout:8/515: rename d4/d1c/f98 to d4/d7/f9d 0 2024-02-16T19:29:08.548 INFO:tasks.workunit.client.1.smithi117.stdout:6/504: mknod d5/db/d7d/d32/d79/ca5 0 2024-02-16T19:29:08.549 INFO:tasks.workunit.client.1.smithi117.stdout:1/468: mkdir d2/d1d/d1f/d9c 0 2024-02-16T19:29:08.550 INFO:tasks.workunit.client.1.smithi117.stdout:7/509: unlink d4/l27 0 2024-02-16T19:29:08.552 INFO:tasks.workunit.client.1.smithi117.stdout:3/611: dwrite d1/d1b/d1e/d37/f9c [0,4194304] 0 2024-02-16T19:29:08.552 INFO:tasks.workunit.client.1.smithi117.stdout:0/558: rename d0/d9/d14/d1d/d60/d76/d7b/d8d/l5e to d0/d30/lc8 0 2024-02-16T19:29:08.553 INFO:tasks.workunit.client.1.smithi117.stdout:2/507: dread d0/f25 [0,4194304] 0 2024-02-16T19:29:08.556 INFO:tasks.workunit.client.1.smithi117.stdout:5/586: rename d2/db3/db0/f77 to d2/d4c/d4f/d67/fc5 0 2024-02-16T19:29:08.557 INFO:tasks.workunit.client.1.smithi117.stdout:5/587: chown d2/d3/d21 287 1 2024-02-16T19:29:08.557 INFO:tasks.workunit.client.1.smithi117.stdout:0/559: creat d0/d9/fc9 x:0 0 0 2024-02-16T19:29:08.558 INFO:tasks.workunit.client.1.smithi117.stdout:2/508: creat d0/d13/d4d/d80/f9c x:0 0 0 2024-02-16T19:29:08.559 INFO:tasks.workunit.client.1.smithi117.stdout:7/510: truncate d4/d1c/d39/d46/d47/f56 1075576 0 2024-02-16T19:29:08.559 INFO:tasks.workunit.client.1.smithi117.stdout:2/509: chown d0/f5e 201 1 2024-02-16T19:29:08.560 INFO:tasks.workunit.client.1.smithi117.stdout:7/511: truncate d4/d13/d63/d71/f9c 120971 0 2024-02-16T19:29:08.563 INFO:tasks.workunit.client.1.smithi117.stdout:8/516: link d4/d7/d6a/d6f/c8b d4/d7/d6a/c9e 0 2024-02-16T19:29:08.566 INFO:tasks.workunit.client.1.smithi117.stdout:8/517: creat d4/d73/f9f x:0 0 0 2024-02-16T19:29:08.569 INFO:tasks.workunit.client.1.smithi117.stdout:7/512: dwrite d4/d1c/d39/d46/d47/f73 [0,4194304] 0 2024-02-16T19:29:08.570 INFO:tasks.workunit.client.1.smithi117.stdout:2/510: getdents d0/d3/d34 0 2024-02-16T19:29:08.577 INFO:tasks.workunit.client.1.smithi117.stdout:7/513: rmdir d4/d13/d63/d71 39 2024-02-16T19:29:08.578 INFO:tasks.workunit.client.1.smithi117.stdout:7/514: fsync d4/d1c/f3c 0 2024-02-16T19:29:08.586 INFO:tasks.workunit.client.1.smithi117.stdout:6/505: dread d5/db/d7d/d32/f3f [0,4194304] 0 2024-02-16T19:29:08.586 INFO:tasks.workunit.client.1.smithi117.stdout:7/515: rmdir d4/d1c/d87 39 2024-02-16T19:29:08.587 INFO:tasks.workunit.client.1.smithi117.stdout:5/588: truncate d2/d3/d21/f33 2930298 0 2024-02-16T19:29:08.588 INFO:tasks.workunit.client.1.smithi117.stdout:8/518: link d4/f3c d4/d73/fa0 0 2024-02-16T19:29:08.590 INFO:tasks.workunit.client.1.smithi117.stdout:1/469: dread d2/d4/f11 [0,4194304] 0 2024-02-16T19:29:08.592 INFO:tasks.workunit.client.1.smithi117.stdout:7/516: rename d4/d1c/d39/d46/d55/c44 to d4/d85/cbf 0 2024-02-16T19:29:08.592 INFO:tasks.workunit.client.1.smithi117.stdout:1/470: write d2/d19/d59/d76/f7a [5002485,126374] 0 2024-02-16T19:29:08.594 INFO:tasks.workunit.client.1.smithi117.stdout:0/560: dread d0/d9/d22/d27/d83/f67 [0,4194304] 0 2024-02-16T19:29:08.594 INFO:tasks.workunit.client.1.smithi117.stdout:8/519: mknod d4/d7/d6a/d72/d84/ca1 0 2024-02-16T19:29:08.596 INFO:tasks.workunit.client.1.smithi117.stdout:6/506: getdents d5/d4b/d20/d69 0 2024-02-16T19:29:08.598 INFO:tasks.workunit.client.1.smithi117.stdout:2/511: dread d0/f19 [0,4194304] 0 2024-02-16T19:29:08.599 INFO:tasks.workunit.client.1.smithi117.stdout:1/471: creat d2/f9d x:0 0 0 2024-02-16T19:29:08.601 INFO:tasks.workunit.client.1.smithi117.stdout:5/589: dwrite d2/d9/d3c/d24/d50/d52/d74/fab [0,4194304] 0 2024-02-16T19:29:08.604 INFO:tasks.workunit.client.1.smithi117.stdout:2/512: fdatasync d0/d29/d56/f72 0 2024-02-16T19:29:08.605 INFO:tasks.workunit.client.1.smithi117.stdout:5/590: read - d2/d3/d21/d2a/fa6 zero size 2024-02-16T19:29:08.606 INFO:tasks.workunit.client.1.smithi117.stdout:1/472: mkdir d2/d1d/d60/d7c/d9e 0 2024-02-16T19:29:08.608 INFO:tasks.workunit.client.1.smithi117.stdout:6/507: dwrite d5/db/d7d/d32/f62 [0,4194304] 0 2024-02-16T19:29:08.613 INFO:tasks.workunit.client.1.smithi117.stdout:5/591: creat d2/d9/d3c/d42/fc6 x:0 0 0 2024-02-16T19:29:08.614 INFO:tasks.workunit.client.1.smithi117.stdout:6/508: truncate d5/db/d7d/d32/f73 2756269 0 2024-02-16T19:29:08.615 INFO:tasks.workunit.client.1.smithi117.stdout:8/520: write d4/d7/d37/d3a/d43/f6b [3645248,83959] 0 2024-02-16T19:29:08.615 INFO:tasks.workunit.client.1.smithi117.stdout:7/517: truncate d4/d1c/d39/d46/f96 557987 0 2024-02-16T19:29:08.615 INFO:tasks.workunit.client.1.smithi117.stdout:0/561: write d0/d9/d22/d27/d4e/fac [1615718,116278] 0 2024-02-16T19:29:08.617 INFO:tasks.workunit.client.1.smithi117.stdout:2/513: dwrite d0/d3/d34/d94/d46/d68/f8f [0,4194304] 0 2024-02-16T19:29:08.619 INFO:tasks.workunit.client.1.smithi117.stdout:5/592: creat d2/d9/d3c/d24/d50/d52/d74/d8f/fc7 x:0 0 0 2024-02-16T19:29:08.622 INFO:tasks.workunit.client.1.smithi117.stdout:2/514: rmdir d0/d3/d34/d94/d46/d68/d76 39 2024-02-16T19:29:08.622 INFO:tasks.workunit.client.1.smithi117.stdout:0/562: chown d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/fc7 2334523 1 2024-02-16T19:29:08.623 INFO:tasks.workunit.client.1.smithi117.stdout:0/563: stat d0/f40 0 2024-02-16T19:29:08.628 INFO:tasks.workunit.client.1.smithi117.stdout:5/593: unlink d2/l1b 0 2024-02-16T19:29:08.628 INFO:tasks.workunit.client.1.smithi117.stdout:8/521: rename d4/d7/l23 to d4/d94/la2 0 2024-02-16T19:29:08.628 INFO:tasks.workunit.client.1.smithi117.stdout:2/515: dread d0/f5e [0,4194304] 0 2024-02-16T19:29:08.629 INFO:tasks.workunit.client.0.smithi022.stdout:6/513: mknod d1/d8/da/d7b/cb5 0 2024-02-16T19:29:08.629 INFO:tasks.workunit.client.0.smithi022.stdout:3/473: rename d7/d12/d18/dd/d31/c37 to d7/d12/d18/dd/d24/d58/d70/cb3 0 2024-02-16T19:29:08.629 INFO:tasks.workunit.client.0.smithi022.stdout:8/518: write d3/d25/f79 [1029854,50693] 0 2024-02-16T19:29:08.629 INFO:tasks.workunit.client.0.smithi022.stdout:0/675: fsync d10/f17 0 2024-02-16T19:29:08.629 INFO:tasks.workunit.client.1.smithi117.stdout:5/594: readlink d2/d3/d21/d49/l39 0 2024-02-16T19:29:08.629 INFO:tasks.workunit.client.1.smithi117.stdout:2/516: readlink d0/d29/d56/d60/l67 0 2024-02-16T19:29:08.631 INFO:tasks.workunit.client.0.smithi022.stdout:5/506: mknod dd/d1f/d48/d21/d2f/d40/d49/d4b/cac 0 2024-02-16T19:29:08.631 INFO:tasks.workunit.client.0.smithi022.stdout:8/519: write d3/d12/d2b/d54/d97/fae [358243,78349] 0 2024-02-16T19:29:08.632 INFO:tasks.workunit.client.0.smithi022.stdout:4/495: mknod d4/d6/dd/d22/d83/c9a 0 2024-02-16T19:29:08.632 INFO:tasks.workunit.client.1.smithi117.stdout:0/564: fsync d0/f53 0 2024-02-16T19:29:08.636 INFO:tasks.workunit.client.0.smithi022.stdout:3/474: creat d7/d12/d18/dd/d24/fb4 x:0 0 0 2024-02-16T19:29:08.637 INFO:tasks.workunit.client.0.smithi022.stdout:0/676: dread d10/d15/d23/d4e/f87 [0,4194304] 0 2024-02-16T19:29:08.639 INFO:tasks.workunit.client.1.smithi117.stdout:8/522: mkdir d4/d7/d37/d3a/d65/d96/da3 0 2024-02-16T19:29:08.640 INFO:tasks.workunit.client.1.smithi117.stdout:8/523: readlink d4/d7/d37/d3a/d43/d44/l5d 0 2024-02-16T19:29:08.640 INFO:tasks.workunit.client.1.smithi117.stdout:8/524: stat d4/d12/d31/f40 0 2024-02-16T19:29:08.642 INFO:tasks.workunit.client.1.smithi117.stdout:2/517: unlink d0/d3/d10/c41 0 2024-02-16T19:29:08.644 INFO:tasks.workunit.client.0.smithi022.stdout:6/514: mkdir d1/d8/da/d7b/dae/db6 0 2024-02-16T19:29:08.644 INFO:tasks.workunit.client.1.smithi117.stdout:6/509: link d5/d4b/d20/d2d/f96 d5/d4b/d44/d52/d81/fa6 0 2024-02-16T19:29:08.646 INFO:tasks.workunit.client.0.smithi022.stdout:5/507: rename dd/d1f/d48/d21/d2f/d40/d49/la0 to dd/d1f/d48/d21/d2f/lad 0 2024-02-16T19:29:08.648 INFO:tasks.workunit.client.1.smithi117.stdout:7/518: getdents d4/d1c/d39/d46/d55 0 2024-02-16T19:29:08.649 INFO:tasks.workunit.client.1.smithi117.stdout:7/519: truncate d4/d13/d63/f88 1001172 0 2024-02-16T19:29:08.650 INFO:tasks.workunit.client.0.smithi022.stdout:3/475: fdatasync d7/d12/d18/dd/d24/d58/d70/f7b 0 2024-02-16T19:29:08.652 INFO:tasks.workunit.client.1.smithi117.stdout:8/525: fdatasync d4/d12/d2a/f2b 0 2024-02-16T19:29:08.653 INFO:tasks.workunit.client.0.smithi022.stdout:0/677: fdatasync d10/f17 0 2024-02-16T19:29:08.656 INFO:tasks.workunit.client.1.smithi117.stdout:2/518: mkdir d0/d3/d22/d37/d9d 0 2024-02-16T19:29:08.657 INFO:tasks.workunit.client.1.smithi117.stdout:2/519: chown d0/d3/c2c 66820023 1 2024-02-16T19:29:08.657 INFO:tasks.workunit.client.1.smithi117.stdout:2/520: dread - d0/f6c zero size 2024-02-16T19:29:08.658 INFO:tasks.workunit.client.0.smithi022.stdout:6/515: rmdir d1/d8/d3c/d4d 39 2024-02-16T19:29:08.661 INFO:tasks.workunit.client.0.smithi022.stdout:4/496: rename d4/d6/c45 to d4/d6/d14/d64/d4e/d54/d5a/c9b 0 2024-02-16T19:29:08.663 INFO:tasks.workunit.client.0.smithi022.stdout:5/508: creat dd/d1f/d48/d21/d2f/d46/fae x:0 0 0 2024-02-16T19:29:08.667 INFO:tasks.workunit.client.0.smithi022.stdout:3/476: creat d7/d12/d18/dd/d24/d29/fb5 x:0 0 0 2024-02-16T19:29:08.669 INFO:tasks.workunit.client.0.smithi022.stdout:8/520: creat d3/d12/d2b/fc3 x:0 0 0 2024-02-16T19:29:08.670 INFO:tasks.workunit.client.1.smithi117.stdout:5/595: write d2/d3/d21/f33 [1291120,13554] 0 2024-02-16T19:29:08.671 INFO:tasks.workunit.client.1.smithi117.stdout:2/521: rename d0/d3/d34/c77 to d0/d13/d4d/d80/c9e 0 2024-02-16T19:29:08.672 INFO:tasks.workunit.client.1.smithi117.stdout:2/522: chown d0/d3/d22/f3c 61 1 2024-02-16T19:29:08.675 INFO:tasks.workunit.client.1.smithi117.stdout:7/520: mkdir d4/d1c/dc0 0 2024-02-16T19:29:08.678 INFO:tasks.workunit.client.1.smithi117.stdout:8/526: creat d4/d7/d37/d3a/d54/d70/d76/d8c/fa4 x:0 0 0 2024-02-16T19:29:08.679 INFO:tasks.workunit.client.1.smithi117.stdout:0/565: truncate d0/d9/d14/d1d/d60/d76/d7b/d8d/f32 531964 0 2024-02-16T19:29:08.679 INFO:tasks.workunit.client.1.smithi117.stdout:8/527: read - d4/d12/d2a/d64/f7d zero size 2024-02-16T19:29:08.680 INFO:tasks.workunit.client.0.smithi022.stdout:3/477: mkdir d7/d12/d18/d4b/db6 0 2024-02-16T19:29:08.681 INFO:tasks.workunit.client.1.smithi117.stdout:8/528: dread d4/d12/d2a/f48 [0,4194304] 0 2024-02-16T19:29:08.685 INFO:tasks.workunit.client.1.smithi117.stdout:2/523: symlink d0/d3/d34/d94/l9f 0 2024-02-16T19:29:08.685 INFO:tasks.workunit.client.1.smithi117.stdout:2/524: dread - d0/f6c zero size 2024-02-16T19:29:08.686 INFO:tasks.workunit.client.1.smithi117.stdout:2/525: write d0/d3/d40/f66 [768883,128731] 0 2024-02-16T19:29:08.687 INFO:tasks.workunit.client.1.smithi117.stdout:6/510: creat d5/fa7 x:0 0 0 2024-02-16T19:29:08.688 INFO:tasks.workunit.client.0.smithi022.stdout:4/497: dwrite d4/d6/d14/f1d [0,4194304] 0 2024-02-16T19:29:08.688 INFO:tasks.workunit.client.1.smithi117.stdout:7/521: write d4/d1c/d39/d46/d47/fb5 [248693,43975] 0 2024-02-16T19:29:08.689 INFO:tasks.workunit.client.1.smithi117.stdout:7/522: readlink d4/d1c/d39/d46/d55/l59 0 2024-02-16T19:29:08.689 INFO:tasks.workunit.client.0.smithi022.stdout:5/509: creat dd/d1f/d48/d21/d64/faf x:0 0 0 2024-02-16T19:29:08.692 INFO:tasks.workunit.client.1.smithi117.stdout:4/510: sync 2024-02-16T19:29:08.695 INFO:tasks.workunit.client.1.smithi117.stdout:0/566: rename d0/d9/d14/d1d/d60/d76/d7b/d8d/fa1 to d0/d78/fca 0 2024-02-16T19:29:08.695 INFO:tasks.workunit.client.1.smithi117.stdout:8/529: symlink d4/d12/d2a/d64/la5 0 2024-02-16T19:29:08.699 INFO:tasks.workunit.client.0.smithi022.stdout:0/678: rename d10/d15/d23/d38/d5a/d66/f9d to d10/d15/d23/fdf 0 2024-02-16T19:29:08.704 INFO:tasks.workunit.client.1.smithi117.stdout:6/511: rmdir d5/d4b/d44/d52/d64 39 2024-02-16T19:29:08.704 INFO:tasks.workunit.client.0.smithi022.stdout:9/554: sync 2024-02-16T19:29:08.705 INFO:tasks.workunit.client.1.smithi117.stdout:6/512: chown d5/db/d7d/d32/d37/f41 22592 1 2024-02-16T19:29:08.706 INFO:tasks.workunit.client.0.smithi022.stdout:5/510: mkdir dd/d1f/d48/d21/d2f/d52/d5c/d61/d6a/db0 0 2024-02-16T19:29:08.707 INFO:tasks.workunit.client.1.smithi117.stdout:7/523: truncate d4/d1c/f2c 861634 0 2024-02-16T19:29:08.708 INFO:tasks.workunit.client.1.smithi117.stdout:7/524: truncate d4/f34 4359835 0 2024-02-16T19:29:08.708 INFO:tasks.workunit.client.0.smithi022.stdout:3/478: rmdir d7/d12/d18/dd/d3c/d5a/d67 39 2024-02-16T19:29:08.712 INFO:tasks.workunit.client.1.smithi117.stdout:7/525: dwrite d4/d13/f35 [0,4194304] 0 2024-02-16T19:29:08.712 INFO:tasks.workunit.client.1.smithi117.stdout:4/511: creat d0/da/d21/d3d/d28/fa6 x:0 0 0 2024-02-16T19:29:08.713 INFO:tasks.workunit.client.1.smithi117.stdout:7/526: truncate d4/f11 4473098 0 2024-02-16T19:29:08.714 INFO:tasks.workunit.client.0.smithi022.stdout:6/516: getdents d1/d8/d23 0 2024-02-16T19:29:08.714 INFO:tasks.workunit.client.1.smithi117.stdout:7/527: chown d4/d1c/d39/d46/d47/f77 250 1 2024-02-16T19:29:08.720 INFO:tasks.workunit.client.1.smithi117.stdout:2/526: write d0/d13/d4d/f50 [51397,44732] 0 2024-02-16T19:29:08.721 INFO:tasks.workunit.client.0.smithi022.stdout:0/679: creat d10/d11/d27/fe0 x:0 0 0 2024-02-16T19:29:08.722 INFO:tasks.workunit.client.0.smithi022.stdout:4/498: write d4/d6/dd/df/d10/d55/f7b [929040,74726] 0 2024-02-16T19:29:08.726 INFO:tasks.workunit.client.1.smithi117.stdout:9/441: sync 2024-02-16T19:29:08.730 INFO:tasks.workunit.client.1.smithi117.stdout:6/513: mkdir d5/db/d7d/d32/da8 0 2024-02-16T19:29:08.730 INFO:tasks.workunit.client.0.smithi022.stdout:8/521: link d3/d12/d2b/d2e/c75 d3/d12/d6a/cc4 0 2024-02-16T19:29:08.734 INFO:tasks.workunit.client.0.smithi022.stdout:6/517: symlink d1/d8/d23/lb7 0 2024-02-16T19:29:08.734 INFO:tasks.workunit.client.1.smithi117.stdout:8/530: dread d4/d7/d37/d3a/d65/d6c/f75 [0,4194304] 0 2024-02-16T19:29:08.738 INFO:tasks.workunit.client.0.smithi022.stdout:9/555: write db/d14/d16/d27/d2e/f38 [5489838,62746] 0 2024-02-16T19:29:08.746 INFO:tasks.workunit.client.0.smithi022.stdout:5/511: write dd/d1f/d4a/f3f [1360239,30367] 0 2024-02-16T19:29:08.749 INFO:tasks.workunit.client.1.smithi117.stdout:5/596: getdents d2/d9/d3c/d24 0 2024-02-16T19:29:08.753 INFO:tasks.workunit.client.1.smithi117.stdout:4/512: dwrite d0/da/d21/d3d/d28/d64/d8a/d92/f85 [0,4194304] 0 2024-02-16T19:29:08.754 INFO:tasks.workunit.client.1.smithi117.stdout:0/567: dwrite d0/d9/d2c/f44 [0,4194304] 0 2024-02-16T19:29:08.756 INFO:tasks.workunit.client.0.smithi022.stdout:6/518: dwrite d1/d8/d3c/f47 [0,4194304] 0 2024-02-16T19:29:08.763 INFO:tasks.workunit.client.0.smithi022.stdout:8/522: write d3/f15 [1007018,39732] 0 2024-02-16T19:29:08.763 INFO:tasks.workunit.client.1.smithi117.stdout:2/527: creat d0/d3/d34/d94/d46/d68/d93/fa0 x:0 0 0 2024-02-16T19:29:08.763 INFO:tasks.workunit.client.1.smithi117.stdout:6/514: dwrite d5/db/d7d/f40 [0,4194304] 0 2024-02-16T19:29:08.764 INFO:tasks.workunit.client.0.smithi022.stdout:4/499: getdents d4/d6/dd 0 2024-02-16T19:29:08.764 INFO:tasks.workunit.client.0.smithi022.stdout:5/512: write dd/d1f/d48/d21/d2f/d52/f65 [182084,3333] 0 2024-02-16T19:29:08.769 INFO:tasks.workunit.client.0.smithi022.stdout:5/513: fsync dd/d1f/d76/f96 0 2024-02-16T19:29:08.772 INFO:tasks.workunit.client.0.smithi022.stdout:9/556: dwrite db/d14/d16/d21/f4c [0,4194304] 0 2024-02-16T19:29:08.775 INFO:tasks.workunit.client.0.smithi022.stdout:0/680: getdents d10/d15/d23/d8f 0 2024-02-16T19:29:08.776 INFO:tasks.workunit.client.0.smithi022.stdout:8/523: truncate d3/d25/d33/f35 82759 0 2024-02-16T19:29:08.777 INFO:tasks.workunit.client.0.smithi022.stdout:0/681: dread - d10/d15/d23/d8f/dc1/f69 zero size 2024-02-16T19:29:08.788 INFO:tasks.workunit.client.0.smithi022.stdout:4/500: mknod d4/d6/d14/d64/d4e/d54/d8a/c9c 0 2024-02-16T19:29:08.790 INFO:tasks.workunit.client.0.smithi022.stdout:4/501: readlink d4/d6/dd/df/d10/l6f 0 2024-02-16T19:29:08.799 INFO:tasks.workunit.client.0.smithi022.stdout:6/519: dwrite d1/d8/da/d29/f3f [4194304,4194304] 0 2024-02-16T19:29:08.800 INFO:tasks.workunit.client.0.smithi022.stdout:8/524: truncate d3/d12/d2b/f3f 1065124 0 2024-02-16T19:29:08.801 INFO:tasks.workunit.client.0.smithi022.stdout:4/502: symlink d4/d6/dd/d22/d30/l9d 0 2024-02-16T19:29:08.804 INFO:tasks.workunit.client.0.smithi022.stdout:5/514: write f5 [572856,14744] 0 2024-02-16T19:29:08.804 INFO:tasks.workunit.client.1.smithi117.stdout:0/568: mknod d0/d78/ccb 0 2024-02-16T19:29:08.805 INFO:tasks.workunit.client.0.smithi022.stdout:0/682: truncate d10/d11/fd5 325502 0 2024-02-16T19:29:08.811 INFO:tasks.workunit.client.0.smithi022.stdout:4/503: dwrite d4/d6/dd/df/d10/f42 [0,4194304] 0 2024-02-16T19:29:08.816 INFO:tasks.workunit.client.0.smithi022.stdout:9/557: dwrite db/d14/d33/fa7 [0,4194304] 0 2024-02-16T19:29:08.819 INFO:tasks.workunit.client.1.smithi117.stdout:9/442: dwrite dd/d2d/f35 [0,4194304] 0 2024-02-16T19:29:08.820 INFO:tasks.workunit.client.1.smithi117.stdout:1/473: sync 2024-02-16T19:29:08.820 INFO:tasks.workunit.client.1.smithi117.stdout:3/612: sync 2024-02-16T19:29:08.820 INFO:tasks.workunit.client.1.smithi117.stdout:4/513: dread d0/da/d21/f3e [0,4194304] 0 2024-02-16T19:29:08.823 INFO:tasks.workunit.client.1.smithi117.stdout:1/474: write d2/d19/d59/d89/d7d/d7f/f94 [623292,81778] 0 2024-02-16T19:29:08.823 INFO:tasks.workunit.client.0.smithi022.stdout:8/525: creat d3/d12/d92/fc5 x:0 0 0 2024-02-16T19:29:08.824 INFO:tasks.workunit.client.1.smithi117.stdout:0/569: dread d0/d9/d14/f1b [0,4194304] 0 2024-02-16T19:29:08.825 INFO:tasks.workunit.client.1.smithi117.stdout:0/570: fsync d0/d9/fc9 0 2024-02-16T19:29:08.825 INFO:tasks.workunit.client.0.smithi022.stdout:4/504: dwrite d4/d6/dd/df/f29 [4194304,4194304] 0 2024-02-16T19:29:08.829 INFO:tasks.workunit.client.1.smithi117.stdout:1/475: dwrite d2/d1d/f4f [0,4194304] 0 2024-02-16T19:29:08.833 INFO:tasks.workunit.client.0.smithi022.stdout:0/683: rename d10/d11/d79/dd1 to d10/d22/d49/de1 0 2024-02-16T19:29:08.838 INFO:tasks.workunit.client.0.smithi022.stdout:7/544: sync 2024-02-16T19:29:08.838 INFO:tasks.workunit.client.1.smithi117.stdout:7/528: creat d4/d1c/d87/d89/d60/d79/fc1 x:0 0 0 2024-02-16T19:29:08.839 INFO:tasks.workunit.client.0.smithi022.stdout:2/465: sync 2024-02-16T19:29:08.843 INFO:tasks.workunit.client.1.smithi117.stdout:2/528: rename d0/d29 to d0/d3/d34/d94/d46/d68/d76/da1 0 2024-02-16T19:29:08.844 INFO:tasks.workunit.client.0.smithi022.stdout:6/520: mknod d1/d8/d3c/cb8 0 2024-02-16T19:29:08.845 INFO:tasks.workunit.client.1.smithi117.stdout:2/529: read d0/d3/d34/d94/d46/d68/f8f [1027099,108111] 0 2024-02-16T19:29:08.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:08.846+0000 7f7c64cf0700 1 -- 172.21.15.22:0/2837534475 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c60071a00 msgr2=0x7f7c60071e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:29:08.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:08.846+0000 7f7c64cf0700 1 --2- 172.21.15.22:0/2837534475 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c60071a00 0x7f7c60071e10 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f7c50009910 tx=0x7f7c50009c20 comp rx=0 tx=0).stop 2024-02-16T19:29:08.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:08.846+0000 7f7c64cf0700 1 -- 172.21.15.22:0/2837534475 shutdown_connections 2024-02-16T19:29:08.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:08.846+0000 7f7c64cf0700 1 --2- 172.21.15.22:0/2837534475 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c600723e0 0x7f7c6010be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:08.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:08.846+0000 7f7c64cf0700 1 --2- 172.21.15.22:0/2837534475 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c60071a00 0x7f7c60071e10 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:08.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:08.846+0000 7f7c64cf0700 1 -- 172.21.15.22:0/2837534475 >> 172.21.15.22:0/2837534475 conn(0x7f7c6006d110 msgr2=0x7f7c6006f560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:29:08.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:08.847+0000 7f7c64cf0700 1 -- 172.21.15.22:0/2837534475 shutdown_connections 2024-02-16T19:29:08.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:08.847+0000 7f7c64cf0700 1 -- 172.21.15.22:0/2837534475 wait complete. 2024-02-16T19:29:08.849 INFO:tasks.workunit.client.0.smithi022.stdout:9/558: symlink db/d14/d16/d49/lbe 0 2024-02-16T19:29:08.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:08.848+0000 7f7c64cf0700 1 Processor -- start 2024-02-16T19:29:08.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:08.848+0000 7f7c64cf0700 1 -- start start 2024-02-16T19:29:08.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:08.848+0000 7f7c64cf0700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c600723e0 0x7f7c60116c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:29:08.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:08.848+0000 7f7c64cf0700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c601171b0 0x7f7c601a1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:29:08.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:08.848+0000 7f7c64cf0700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c60117620 con 0x7f7c600723e0 2024-02-16T19:29:08.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:08.848+0000 7f7c64cf0700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c60117790 con 0x7f7c601171b0 2024-02-16T19:29:08.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:08.848+0000 7f7c5dd9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c601171b0 0x7f7c601a1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:29:08.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:08.848+0000 7f7c5dd9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c601171b0 0x7f7c601a1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.22:50074/0 (socket says 172.21.15.22:50074) 2024-02-16T19:29:08.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:08.848+0000 7f7c5dd9b700 1 -- 172.21.15.22:0/3577356079 learned_addr learned my addr 172.21.15.22:0/3577356079 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:29:08.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:08.848+0000 7f7c5e59c700 1 --2- 172.21.15.22:0/3577356079 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c600723e0 0x7f7c60116c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:29:08.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:08.849+0000 7f7c5dd9b700 1 -- 172.21.15.22:0/3577356079 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c600723e0 msgr2=0x7f7c60116c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:29:08.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:08.849+0000 7f7c5dd9b700 1 --2- 172.21.15.22:0/3577356079 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c600723e0 0x7f7c60116c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:08.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:08.849+0000 7f7c5dd9b700 1 -- 172.21.15.22:0/3577356079 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c500095f0 con 0x7f7c601171b0 2024-02-16T19:29:08.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:08.849+0000 7f7c5dd9b700 1 --2- 172.21.15.22:0/3577356079 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c601171b0 0x7f7c601a1300 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7c5400c930 tx=0x7f7c5400ccf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:29:08.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:08.849+0000 7f7c4f7fe700 1 -- 172.21.15.22:0/3577356079 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c54007ad0 con 0x7f7c601171b0 2024-02-16T19:29:08.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:08.849+0000 7f7c64cf0700 1 -- 172.21.15.22:0/3577356079 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c601a1840 con 0x7f7c601171b0 2024-02-16T19:29:08.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:08.849+0000 7f7c64cf0700 1 -- 172.21.15.22:0/3577356079 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c601a1d60 con 0x7f7c601171b0 2024-02-16T19:29:08.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:08.850+0000 7f7c4f7fe700 1 -- 172.21.15.22:0/3577356079 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7c54007c30 con 0x7f7c601171b0 2024-02-16T19:29:08.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:08.850+0000 7f7c4f7fe700 1 -- 172.21.15.22:0/3577356079 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c54018630 con 0x7f7c601171b0 2024-02-16T19:29:08.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:08.851+0000 7f7c64cf0700 1 -- 172.21.15.22:0/3577356079 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c40005320 con 0x7f7c601171b0 2024-02-16T19:29:08.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:08.851+0000 7f7c4f7fe700 1 -- 172.21.15.22:0/3577356079 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) v1 ==== 46073+0+0 (secure 0 0 0) 0x7f7c540188e0 con 0x7f7c601171b0 2024-02-16T19:29:08.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:08.851+0000 7f7c4f7fe700 1 --2- 172.21.15.22:0/3577356079 >> [v2:172.21.15.117:6828/10500008,v1:172.21.15.117:6829/10500008] conn(0x7f7c48038eb0 0x7f7c4803b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:29:08.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:08.851+0000 7f7c4f7fe700 1 -- 172.21.15.22:0/3577356079 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f7c5404d420 con 0x7f7c601171b0 2024-02-16T19:29:08.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:08.852+0000 7f7c5e59c700 1 --2- 172.21.15.22:0/3577356079 >> [v2:172.21.15.117:6828/10500008,v1:172.21.15.117:6829/10500008] conn(0x7f7c48038eb0 0x7f7c4803b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:29:08.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:08.852+0000 7f7c5e59c700 1 --2- 172.21.15.22:0/3577356079 >> [v2:172.21.15.117:6828/10500008,v1:172.21.15.117:6829/10500008] conn(0x7f7c48038eb0 0x7f7c4803b360 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7c50005090 tx=0x7f7c5000b540 comp rx=0 tx=0).ready entity=mgr.24349 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:29:08.854 INFO:tasks.workunit.client.0.smithi022.stdout:5/515: link dd/d1f/d4a/f83 dd/d1f/d76/fb1 0 2024-02-16T19:29:08.855 INFO:tasks.workunit.client.0.smithi022.stdout:5/516: stat dd/d1f/d48/d21/d2f/d52/d77/d95 0 2024-02-16T19:29:08.855 INFO:tasks.workunit.client.0.smithi022.stdout:5/517: stat f5 0 2024-02-16T19:29:08.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:08.857+0000 7f7c4f7fe700 1 -- 172.21.15.22:0/3577356079 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f7c5401f2f0 con 0x7f7c601171b0 2024-02-16T19:29:08.860 INFO:tasks.workunit.client.1.smithi117.stdout:8/531: sync 2024-02-16T19:29:08.860 INFO:tasks.workunit.client.1.smithi117.stdout:6/515: truncate d5/d4b/f29 2841549 0 2024-02-16T19:29:08.863 INFO:tasks.workunit.client.0.smithi022.stdout:4/505: rename d4/d6/d70 to d4/d6/dd/d22/d83/d9e 0 2024-02-16T19:29:08.863 INFO:tasks.workunit.client.0.smithi022.stdout:5/518: dwrite dd/d1f/d48/d21/d2f/d46/d5e/fa6 [0,4194304] 0 2024-02-16T19:29:08.866 INFO:tasks.workunit.client.1.smithi117.stdout:8/532: dwrite d4/d7/d37/d3a/d54/d70/d76/f85 [0,4194304] 0 2024-02-16T19:29:08.867 INFO:tasks.workunit.client.0.smithi022.stdout:0/684: dread - d10/d15/f9f zero size 2024-02-16T19:29:08.867 INFO:tasks.workunit.client.1.smithi117.stdout:7/529: read d4/d13/d63/d71/f90 [1924802,59991] 0 2024-02-16T19:29:08.867 INFO:tasks.workunit.client.0.smithi022.stdout:4/506: write d4/d6/d14/d64/d66/d6d/f89 [223356,6037] 0 2024-02-16T19:29:08.868 INFO:tasks.workunit.client.0.smithi022.stdout:4/507: chown d4/d6/f87 617402250 1 2024-02-16T19:29:08.869 INFO:tasks.workunit.client.1.smithi117.stdout:4/514: dwrite d0/da/d21/d3d/f68 [0,4194304] 0 2024-02-16T19:29:08.869 INFO:tasks.workunit.client.0.smithi022.stdout:4/508: chown d4/d6/d14 7 1 2024-02-16T19:29:08.870 INFO:tasks.workunit.client.0.smithi022.stdout:4/509: read d4/d6/dd/df/d10/d24/f8c [2441947,16469] 0 2024-02-16T19:29:08.871 INFO:tasks.workunit.client.1.smithi117.stdout:0/571: rename d0/d9/d2c/f91 to d0/d9/d14/d1d/d60/dc3/fcc 0 2024-02-16T19:29:08.873 INFO:tasks.workunit.client.0.smithi022.stdout:7/545: truncate d9/da/f13 9046668 0 2024-02-16T19:29:08.873 INFO:tasks.workunit.client.1.smithi117.stdout:4/515: write d0/da/d21/d3d/d28/d66/d97/d6d/f93 [834910,91917] 0 2024-02-16T19:29:08.877 INFO:tasks.workunit.client.0.smithi022.stdout:2/466: creat d0/d32/d3b/d6f/f9b x:0 0 0 2024-02-16T19:29:08.882 INFO:tasks.workunit.client.1.smithi117.stdout:5/597: getdents d2/d3 0 2024-02-16T19:29:08.884 INFO:tasks.workunit.client.0.smithi022.stdout:0/685: truncate d10/d11/d50/f3e 7110034 0 2024-02-16T19:29:08.885 INFO:tasks.workunit.client.0.smithi022.stdout:0/686: dread - d10/d11/d50/d65/fbe zero size 2024-02-16T19:29:08.890 INFO:tasks.workunit.client.0.smithi022.stdout:0/687: dread d10/f17 [0,4194304] 0 2024-02-16T19:29:08.902 INFO:tasks.workunit.client.1.smithi117.stdout:0/572: dread d0/d30/f66 [0,4194304] 0 2024-02-16T19:29:08.942 INFO:tasks.workunit.client.0.smithi022.stdout:1/622: sync 2024-02-16T19:29:08.942 INFO:tasks.workunit.client.0.smithi022.stdout:3/479: sync 2024-02-16T19:29:08.943 INFO:tasks.workunit.client.0.smithi022.stdout:1/623: read - d6/d8/d22/d59/da7/fa8 zero size 2024-02-16T19:29:08.974 INFO:tasks.workunit.client.0.smithi022.stdout:4/510: fsync d4/d6/dd/df/d10/f35 0 2024-02-16T19:29:08.978 INFO:tasks.workunit.client.0.smithi022.stdout:4/511: dread d4/d6/dd/d22/d30/f44 [0,4194304] 0 2024-02-16T19:29:08.980 INFO:tasks.workunit.client.0.smithi022.stdout:4/512: write d4/d6/f87 [906310,105616] 0 2024-02-16T19:29:08.981 INFO:tasks.workunit.client.0.smithi022.stdout:7/546: mkdir d9/da/d2c/dae 0 2024-02-16T19:29:08.981 INFO:tasks.workunit.client.0.smithi022.stdout:4/513: stat d4/d6/d3a/f41 0 2024-02-16T19:29:08.982 INFO:tasks.workunit.client.1.smithi117.stdout:8/533: dwrite d4/d7/f88 [0,4194304] 0 2024-02-16T19:29:08.982 INFO:tasks.workunit.client.0.smithi022.stdout:7/547: write d9/da/f60 [50813,40859] 0 2024-02-16T19:29:08.982 INFO:tasks.workunit.client.0.smithi022.stdout:4/514: chown d4/d6/dd/df/c2f 617 1 2024-02-16T19:29:08.983 INFO:tasks.workunit.client.1.smithi117.stdout:8/534: chown d4/d73/c9a 3 1 2024-02-16T19:29:08.984 INFO:tasks.workunit.client.1.smithi117.stdout:8/535: dread d4/d1c/f49 [0,4194304] 0 2024-02-16T19:29:08.986 INFO:tasks.workunit.client.1.smithi117.stdout:8/536: truncate d4/d12/d2a/f5c 2431187 0 2024-02-16T19:29:08.988 INFO:tasks.workunit.client.1.smithi117.stdout:3/613: write d1/d1b/d1e/d29/f47 [1694640,120526] 0 2024-02-16T19:29:08.989 INFO:tasks.workunit.client.0.smithi022.stdout:2/467: symlink d0/d2/d18/d1f/d6c/l9c 0 2024-02-16T19:29:08.991 INFO:tasks.workunit.client.0.smithi022.stdout:2/468: write d0/d32/d3b/d6f/f5a [2327667,116799] 0 2024-02-16T19:29:08.992 INFO:tasks.workunit.client.1.smithi117.stdout:1/476: rename d2/d1d/c41 to d2/d19/d59/d89/d7d/c9f 0 2024-02-16T19:29:08.994 INFO:tasks.workunit.client.1.smithi117.stdout:9/443: truncate dd/d47/d48/f55 4886149 0 2024-02-16T19:29:08.995 INFO:tasks.workunit.client.1.smithi117.stdout:1/477: write d2/d19/d59/f97 [958117,106104] 0 2024-02-16T19:29:08.995 INFO:tasks.workunit.client.1.smithi117.stdout:1/478: readlink d2/d19/d59/d89/d49/l6c 0 2024-02-16T19:29:08.996 INFO:tasks.workunit.client.1.smithi117.stdout:9/444: truncate dd/d45/d62/f4e 521107 0 2024-02-16T19:29:08.999 INFO:tasks.workunit.client.0.smithi022.stdout:0/688: fsync d10/d22/d49/f4d 0 2024-02-16T19:29:09.003 INFO:tasks.workunit.client.1.smithi117.stdout:4/516: unlink d0/da/d4a/f78 0 2024-02-16T19:29:09.007 INFO:tasks.workunit.client.1.smithi117.stdout:4/517: dread d0/da/d21/d3d/f68 [0,4194304] 0 2024-02-16T19:29:09.009 INFO:tasks.workunit.client.1.smithi117.stdout:2/530: fsync d0/d3/d34/f7a 0 2024-02-16T19:29:09.010 INFO:tasks.workunit.client.1.smithi117.stdout:2/531: readlink d0/d3/d40/l74 0 2024-02-16T19:29:09.013 INFO:tasks.workunit.client.1.smithi117.stdout:6/516: getdents d5/d4b/d20/da4 0 2024-02-16T19:29:09.013 INFO:tasks.workunit.client.1.smithi117.stdout:6/517: chown d5/f83 17331587 1 2024-02-16T19:29:09.021 INFO:tasks.workunit.client.1.smithi117.stdout:0/573: rmdir d0/d9/d14/d1d/d60/dc3 39 2024-02-16T19:29:09.022 INFO:tasks.workunit.client.1.smithi117.stdout:0/574: write d0/d9/d22/d27/d83/f95 [555458,26883] 0 2024-02-16T19:29:09.026 INFO:tasks.workunit.client.0.smithi022.stdout:4/515: creat d4/d6/dd/d22/d83/f9f x:0 0 0 2024-02-16T19:29:09.028 INFO:tasks.workunit.client.0.smithi022.stdout:4/516: write d4/d6/d14/f1d [2359218,536] 0 2024-02-16T19:29:09.040 INFO:tasks.workunit.client.1.smithi117.stdout:7/530: rename d4/d1c/d39/d46/d47/f73 to d4/d1c/d39/d46/d55/db8/fc2 0 2024-02-16T19:29:09.042 INFO:tasks.workunit.client.0.smithi022.stdout:8/526: getdents d3 0 2024-02-16T19:29:09.044 INFO:tasks.workunit.client.0.smithi022.stdout:6/521: sync 2024-02-16T19:29:09.046 INFO:tasks.workunit.client.1.smithi117.stdout:9/445: fsync dd/d45/f58 0 2024-02-16T19:29:09.046 INFO:tasks.workunit.client.1.smithi117.stdout:9/446: dread - dd/d47/d5e/f6e zero size 2024-02-16T19:29:09.058 INFO:tasks.workunit.client.1.smithi117.stdout:2/532: dread d0/f5e [0,4194304] 0 2024-02-16T19:29:09.060 INFO:tasks.workunit.client.1.smithi117.stdout:2/533: dread d0/d3/d22/f5d [0,4194304] 0 2024-02-16T19:29:09.070 INFO:tasks.workunit.client.0.smithi022.stdout:0/689: symlink d10/d11/d27/d3f/le2 0 2024-02-16T19:29:09.071 INFO:tasks.workunit.client.0.smithi022.stdout:0/690: write d10/d11/d27/fe0 [661356,115732] 0 2024-02-16T19:29:09.077 INFO:tasks.workunit.client.1.smithi117.stdout:4/518: creat d0/da/d21/d3d/d28/d66/d97/fa7 x:0 0 0 2024-02-16T19:29:09.078 INFO:tasks.workunit.client.1.smithi117.stdout:4/519: chown d0/da/d21/f2b 14 1 2024-02-16T19:29:09.084 INFO:tasks.workunit.client.1.smithi117.stdout:4/520: dread d0/da/d21/d3d/d28/f63 [0,4194304] 0 2024-02-16T19:29:09.091 INFO:tasks.workunit.client.1.smithi117.stdout:7/531: mkdir d4/d1c/d87/d89/d60/d79/dc3 0 2024-02-16T19:29:09.093 INFO:tasks.workunit.client.1.smithi117.stdout:9/447: link dd/d45/d73/f8a dd/d47/d5b/d5f/f9b 0 2024-02-16T19:29:09.094 INFO:tasks.workunit.client.1.smithi117.stdout:8/537: getdents d4/d12/d31 0 2024-02-16T19:29:09.099 INFO:tasks.workunit.client.1.smithi117.stdout:7/532: mkdir d4/d1c/d39/d46/dc4 0 2024-02-16T19:29:09.102 INFO:tasks.workunit.client.1.smithi117.stdout:9/448: write dd/d45/d73/f8a [1046415,69148] 0 2024-02-16T19:29:09.109 INFO:tasks.workunit.client.1.smithi117.stdout:7/533: creat d4/d1c/d39/d46/d55/db8/fc5 x:0 0 0 2024-02-16T19:29:09.110 INFO:tasks.workunit.client.1.smithi117.stdout:2/534: getdents d0/d3/d34/d94/d46/d68/d76/da1/d56/d60 0 2024-02-16T19:29:09.117 INFO:tasks.workunit.client.1.smithi117.stdout:2/535: symlink d0/d3/d22/d37/d9d/la2 0 2024-02-16T19:29:09.117 INFO:tasks.workunit.client.1.smithi117.stdout:2/536: chown d0/d3/d34/f7a 32263303 1 2024-02-16T19:29:09.118 INFO:tasks.workunit.client.1.smithi117.stdout:2/537: chown d0/d3/d22/c4a 957793310 1 2024-02-16T19:29:09.118 INFO:tasks.workunit.client.1.smithi117.stdout:2/538: dread - d0/d3/d34/f71 zero size 2024-02-16T19:29:09.119 INFO:tasks.workunit.client.1.smithi117.stdout:2/539: readlink d0/d13/d4d/d53/l81 0 2024-02-16T19:29:09.122 INFO:tasks.workunit.client.1.smithi117.stdout:2/540: fsync d0/d3/fc 0 2024-02-16T19:29:09.126 INFO:tasks.workunit.client.1.smithi117.stdout:2/541: fsync d0/d3b/f51 0 2024-02-16T19:29:09.127 INFO:tasks.workunit.client.0.smithi022.stdout:8/527: creat d3/d12/d17/fc6 x:0 0 0 2024-02-16T19:29:09.131 INFO:tasks.workunit.client.0.smithi022.stdout:8/528: mknod d3/d12/d2b/da9/cc7 0 2024-02-16T19:29:09.132 INFO:tasks.workunit.client.0.smithi022.stdout:8/529: dread - d3/d12/f9c zero size 2024-02-16T19:29:09.133 INFO:tasks.workunit.client.0.smithi022.stdout:8/530: chown d3/d12/da6/f8c 3017906 1 2024-02-16T19:29:09.160 INFO:tasks.workunit.client.0.smithi022.stdout:5/519: link dd/c10 dd/d1f/d48/cb2 0 2024-02-16T19:29:09.161 INFO:tasks.workunit.client.0.smithi022.stdout:5/520: stat dd/d1f/d48/d21/d2f/d46/d5e/d84/fa3 0 2024-02-16T19:29:09.162 INFO:tasks.workunit.client.0.smithi022.stdout:1/624: write d6/d8/d22/d16/d17/d63/f69 [560349,50657] 0 2024-02-16T19:29:09.165 INFO:tasks.workunit.client.0.smithi022.stdout:1/625: creat d6/d8/d22/d16/d17/d5d/fbf x:0 0 0 2024-02-16T19:29:09.171 INFO:tasks.workunit.client.0.smithi022.stdout:1/626: link d6/f6b d6/d8/d22/d59/fc0 0 2024-02-16T19:29:09.174 INFO:tasks.workunit.client.0.smithi022.stdout:1/627: fdatasync d6/f55 0 2024-02-16T19:29:09.182 INFO:tasks.workunit.client.0.smithi022.stdout:1/628: symlink d6/lc1 0 2024-02-16T19:29:09.182 INFO:tasks.workunit.client.1.smithi117.stdout:3/614: dwrite d1/d1b/d1e/d37/f38 [0,4194304] 0 2024-02-16T19:29:09.185 INFO:tasks.workunit.client.0.smithi022.stdout:2/469: dwrite d0/d2/d2e/f2a [0,4194304] 0 2024-02-16T19:29:09.192 INFO:tasks.workunit.client.1.smithi117.stdout:3/615: dwrite d1/dd/d5c/f6b [0,4194304] 0 2024-02-16T19:29:09.192 INFO:tasks.workunit.client.0.smithi022.stdout:6/522: unlink d1/d8/c3e 0 2024-02-16T19:29:09.193 INFO:tasks.workunit.client.1.smithi117.stdout:3/616: write d1/d1b/d1e/d37/f9c [2421572,113429] 0 2024-02-16T19:29:09.201 INFO:tasks.workunit.client.1.smithi117.stdout:3/617: dwrite d1/dd/f34 [0,4194304] 0 2024-02-16T19:29:09.202 INFO:tasks.workunit.client.1.smithi117.stdout:3/618: write d1/d83/d96/fb6 [309824,94701] 0 2024-02-16T19:29:09.208 INFO:tasks.workunit.client.1.smithi117.stdout:3/619: rmdir d1/d1b/d1e/d37 39 2024-02-16T19:29:09.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.206+0000 7f7c64cf0700 1 -- 172.21.15.22:0/3577356079 --> [v2:172.21.15.117:6828/10500008,v1:172.21.15.117:6829/10500008] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7c40000bf0 con 0x7f7c48038eb0 2024-02-16T19:29:09.209 INFO:tasks.workunit.client.0.smithi022.stdout:4/517: dwrite d4/d6/dd/df/d10/f25 [0,4194304] 0 2024-02-16T19:29:09.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.208+0000 7f7c4f7fe700 1 -- 172.21.15.22:0/3577356079 <== mgr.24349 v2:172.21.15.117:6828/10500008 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f7c40000bf0 con 0x7f7c48038eb0 2024-02-16T19:29:09.215 INFO:tasks.workunit.client.1.smithi117.stdout:3/620: getdents d1/d1b/d1e/d5e 0 2024-02-16T19:29:09.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.213+0000 7f7c64cf0700 1 -- 172.21.15.22:0/3577356079 >> [v2:172.21.15.117:6828/10500008,v1:172.21.15.117:6829/10500008] conn(0x7f7c48038eb0 msgr2=0x7f7c4803b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:29:09.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.213+0000 7f7c64cf0700 1 --2- 172.21.15.22:0/3577356079 >> [v2:172.21.15.117:6828/10500008,v1:172.21.15.117:6829/10500008] conn(0x7f7c48038eb0 0x7f7c4803b360 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7c50005090 tx=0x7f7c5000b540 comp rx=0 tx=0).stop 2024-02-16T19:29:09.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.213+0000 7f7c64cf0700 1 -- 172.21.15.22:0/3577356079 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c601171b0 msgr2=0x7f7c601a1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:29:09.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.213+0000 7f7c64cf0700 1 --2- 172.21.15.22:0/3577356079 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c601171b0 0x7f7c601a1300 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7c5400c930 tx=0x7f7c5400ccf0 comp rx=0 tx=0).stop 2024-02-16T19:29:09.216 INFO:tasks.workunit.client.1.smithi117.stdout:3/621: chown d1/d30/d9f/dcb 12 1 2024-02-16T19:29:09.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.213+0000 7f7c64cf0700 1 -- 172.21.15.22:0/3577356079 shutdown_connections 2024-02-16T19:29:09.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.213+0000 7f7c64cf0700 1 --2- 172.21.15.22:0/3577356079 >> [v2:172.21.15.117:6828/10500008,v1:172.21.15.117:6829/10500008] conn(0x7f7c48038eb0 0x7f7c4803b360 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:09.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.213+0000 7f7c64cf0700 1 --2- 172.21.15.22:0/3577356079 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c600723e0 0x7f7c60116c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:09.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.213+0000 7f7c64cf0700 1 --2- 172.21.15.22:0/3577356079 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c601171b0 0x7f7c601a1300 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:09.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.213+0000 7f7c64cf0700 1 -- 172.21.15.22:0/3577356079 >> 172.21.15.22:0/3577356079 conn(0x7f7c6006d110 msgr2=0x7f7c6010b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:29:09.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.213+0000 7f7c64cf0700 1 -- 172.21.15.22:0/3577356079 shutdown_connections 2024-02-16T19:29:09.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.214+0000 7f7c64cf0700 1 -- 172.21.15.22:0/3577356079 wait complete. 2024-02-16T19:29:09.218 INFO:tasks.workunit.client.0.smithi022.stdout:3/480: symlink d7/d12/d18/lb7 0 2024-02-16T19:29:09.222 INFO:tasks.workunit.client.0.smithi022.stdout:8/531: rmdir d3/d12/da6 39 2024-02-16T19:29:09.223 INFO:tasks.workunit.client.1.smithi117.stdout:4/521: write d0/da/d21/d3d/d28/d64/f1f [2711189,51266] 0 2024-02-16T19:29:09.227 INFO:tasks.workunit.client.0.smithi022.stdout:1/629: rmdir d6/d8/d22/d3f 39 2024-02-16T19:29:09.227 INFO:tasks.workunit.client.0.smithi022.stdout:8/532: read d3/d12/da6/f28 [531439,35559] 0 2024-02-16T19:29:09.228 INFO:tasks.workunit.client.1.smithi117.stdout:5/598: rmdir d2/d3 39 2024-02-16T19:29:09.230 INFO:tasks.workunit.client.0.smithi022.stdout:8/533: mkdir d3/d12/d2b/d54/db8/dc8 0 2024-02-16T19:29:09.231 INFO:teuthology.orchestra.run.smithi022.stdout:true 2024-02-16T19:29:09.233 INFO:tasks.workunit.client.0.smithi022.stdout:8/534: rmdir d3/d12/d2b/d2e 39 2024-02-16T19:29:09.236 INFO:tasks.workunit.client.1.smithi117.stdout:1/479: rename d2/f93 to d2/d1d/d77/fa0 0 2024-02-16T19:29:09.238 INFO:tasks.workunit.client.1.smithi117.stdout:7/534: truncate d4/d13/da3/fb1 1083371 0 2024-02-16T19:29:09.242 INFO:tasks.workunit.client.0.smithi022.stdout:0/691: mknod d10/d15/d23/ce3 0 2024-02-16T19:29:09.243 INFO:tasks.workunit.client.1.smithi117.stdout:1/480: dwrite d2/d19/d59/f97 [0,4194304] 0 2024-02-16T19:29:09.244 INFO:tasks.workunit.client.1.smithi117.stdout:7/535: chown d4/d1c/d87/d89/d60/d79/c9d 4 1 2024-02-16T19:29:09.246 INFO:tasks.workunit.client.1.smithi117.stdout:6/518: rename d5/db/d7d/d32/d95/d49/d59/c67 to d5/db/d7d/d32/d95/ca9 0 2024-02-16T19:29:09.247 INFO:tasks.workunit.client.1.smithi117.stdout:7/536: creat d4/d1c/d39/d46/fc6 x:0 0 0 2024-02-16T19:29:09.248 INFO:tasks.workunit.client.1.smithi117.stdout:6/519: truncate d5/d4b/d44/f75 2151075 0 2024-02-16T19:29:09.249 INFO:tasks.workunit.client.1.smithi117.stdout:2/542: rmdir d0/d3/d22 39 2024-02-16T19:29:09.250 INFO:tasks.workunit.client.1.smithi117.stdout:4/522: dread d0/f14 [4194304,4194304] 0 2024-02-16T19:29:09.251 INFO:tasks.workunit.client.1.smithi117.stdout:7/537: mknod d4/d1c/d39/cc7 0 2024-02-16T19:29:09.254 INFO:tasks.workunit.client.1.smithi117.stdout:4/523: dread d0/da/d21/d3d/d28/d66/f7f [4194304,4194304] 0 2024-02-16T19:29:09.254 INFO:tasks.workunit.client.1.smithi117.stdout:4/524: stat d0 0 2024-02-16T19:29:09.256 INFO:tasks.workunit.client.1.smithi117.stdout:9/449: rename dd/d23/d64/f39 to dd/d23/f9c 0 2024-02-16T19:29:09.257 INFO:tasks.workunit.client.1.smithi117.stdout:2/543: creat d0/d3/d40/fa3 x:0 0 0 2024-02-16T19:29:09.261 INFO:tasks.workunit.client.0.smithi022.stdout:0/692: dread d10/d15/d23/f3c [0,4194304] 0 2024-02-16T19:29:09.262 INFO:tasks.workunit.client.1.smithi117.stdout:6/520: truncate d5/d4b/d44/d52/d81/fa6 756637 0 2024-02-16T19:29:09.263 INFO:tasks.workunit.client.1.smithi117.stdout:9/450: chown c6 3800 1 2024-02-16T19:29:09.267 INFO:tasks.workunit.client.1.smithi117.stdout:3/622: rename d1/d1b/d1e/d29/c71 to d1/d30/d80/cd4 0 2024-02-16T19:29:09.268 INFO:tasks.workunit.client.1.smithi117.stdout:3/623: chown d1/dd/d3b/d9d 764331 1 2024-02-16T19:29:09.268 INFO:tasks.workunit.client.1.smithi117.stdout:2/544: rmdir d0/d3 39 2024-02-16T19:29:09.270 INFO:tasks.workunit.client.0.smithi022.stdout:0/693: getdents d10/d22/dad/dd7 0 2024-02-16T19:29:09.272 INFO:tasks.workunit.client.0.smithi022.stdout:7/548: rename d9/da/d2c/d88/fa8 to d9/faf 0 2024-02-16T19:29:09.276 INFO:tasks.workunit.client.1.smithi117.stdout:1/481: rename d2/d1d/d1f/c2d to d2/d19/d59/d89/ca1 0 2024-02-16T19:29:09.281 INFO:tasks.workunit.client.1.smithi117.stdout:3/624: creat d1/d30/d52/d56/fd5 x:0 0 0 2024-02-16T19:29:09.283 INFO:tasks.workunit.client.0.smithi022.stdout:1/630: rename d6/d8/d22/d16/d17/d63/f69 to d6/d8/d22/d16/d4f/fc2 0 2024-02-16T19:29:09.289 INFO:tasks.workunit.client.1.smithi117.stdout:1/482: mknod d2/d19/d59/d89/d7b/ca2 0 2024-02-16T19:29:09.294 INFO:tasks.workunit.client.0.smithi022.stdout:0/694: creat d10/d15/d23/fe4 x:0 0 0 2024-02-16T19:29:09.295 INFO:tasks.workunit.client.0.smithi022.stdout:0/695: write d10/d11/d27/fbb [920238,48811] 0 2024-02-16T19:29:09.296 INFO:tasks.workunit.client.1.smithi117.stdout:6/521: rename d5/d4b/d44/d52/d64 to d5/d9c/daa 0 2024-02-16T19:29:09.304 INFO:tasks.workunit.client.0.smithi022.stdout:9/559: creat db/d14/d16/fbf x:0 0 0 2024-02-16T19:29:09.307 INFO:tasks.workunit.client.1.smithi117.stdout:6/522: dread - d5/f83 zero size 2024-02-16T19:29:09.308 INFO:tasks.workunit.client.1.smithi117.stdout:6/523: read d5/d4b/d20/d2d/f3c [91528,127138] 0 2024-02-16T19:29:09.308 INFO:tasks.workunit.client.1.smithi117.stdout:6/524: chown d5/d9c/daa/c84 64431 1 2024-02-16T19:29:09.309 INFO:tasks.workunit.client.0.smithi022.stdout:6/523: write d1/d8/da/d17/d85/d5b/f88 [136624,129627] 0 2024-02-16T19:29:09.309 INFO:tasks.workunit.client.1.smithi117.stdout:4/525: unlink d0/da/d21/d3d/d28/d64/f55 0 2024-02-16T19:29:09.311 INFO:tasks.workunit.client.1.smithi117.stdout:3/625: creat d1/dd/d3e/fd6 x:0 0 0 2024-02-16T19:29:09.312 INFO:tasks.workunit.client.0.smithi022.stdout:2/470: dwrite d0/d2/f3d [0,4194304] 0 2024-02-16T19:29:09.312 INFO:tasks.workunit.client.0.smithi022.stdout:2/471: stat d0/d32 0 2024-02-16T19:29:09.317 INFO:tasks.workunit.client.0.smithi022.stdout:9/560: rename db/d14/d16/d27/d8b/c95 to db/d14/d16/d5d/d65/cc0 0 2024-02-16T19:29:09.319 INFO:tasks.workunit.client.0.smithi022.stdout:4/518: dwrite d4/d6/dd/f37 [0,4194304] 0 2024-02-16T19:29:09.321 INFO:tasks.workunit.client.1.smithi117.stdout:5/599: write d2/d9/d3c/d24/d50/d96/dbc/d30/d38/f64 [910986,85064] 0 2024-02-16T19:29:09.327 INFO:tasks.workunit.client.0.smithi022.stdout:2/472: mkdir d0/d2/d18/d38/d9d 0 2024-02-16T19:29:09.327 INFO:tasks.workunit.client.1.smithi117.stdout:6/525: dread d5/f13 [0,4194304] 0 2024-02-16T19:29:09.327 INFO:tasks.workunit.client.0.smithi022.stdout:2/473: read - d0/d32/f94 zero size 2024-02-16T19:29:09.329 INFO:tasks.workunit.client.1.smithi117.stdout:4/526: creat d0/da/d4a/fa8 x:0 0 0 2024-02-16T19:29:09.329 INFO:tasks.workunit.client.0.smithi022.stdout:2/474: write d0/d2/d18/d19/d40/d55/f7c [661841,89709] 0 2024-02-16T19:29:09.331 INFO:tasks.workunit.client.0.smithi022.stdout:9/561: creat db/d14/d16/d21/fc1 x:0 0 0 2024-02-16T19:29:09.334 INFO:tasks.workunit.client.0.smithi022.stdout:4/519: creat d4/d6/d3a/fa0 x:0 0 0 2024-02-16T19:29:09.337 INFO:tasks.workunit.client.1.smithi117.stdout:1/483: rename d2/d19/d59/d89/d49/f50 to d2/d4/fa3 0 2024-02-16T19:29:09.338 INFO:tasks.workunit.client.0.smithi022.stdout:8/535: dwrite d3/d12/d2b/d54/f95 [0,4194304] 0 2024-02-16T19:29:09.342 INFO:tasks.workunit.client.0.smithi022.stdout:2/475: creat d0/d2/d18/d1f/d6c/f9e x:0 0 0 2024-02-16T19:29:09.344 INFO:tasks.workunit.client.0.smithi022.stdout:4/520: dwrite d4/d6/f7f [0,4194304] 0 2024-02-16T19:29:09.348 INFO:tasks.workunit.client.1.smithi117.stdout:7/538: dwrite d4/d13/d63/fa2 [0,4194304] 0 2024-02-16T19:29:09.355 INFO:tasks.workunit.client.0.smithi022.stdout:7/549: write d9/faf [917155,39140] 0 2024-02-16T19:29:09.356 INFO:tasks.workunit.client.0.smithi022.stdout:2/476: symlink d0/d2/d2e/d73/l9f 0 2024-02-16T19:29:09.357 INFO:tasks.workunit.client.1.smithi117.stdout:9/451: dwrite dd/d47/d5b/f69 [0,4194304] 0 2024-02-16T19:29:09.357 INFO:tasks.workunit.client.0.smithi022.stdout:7/550: write d9/da/d74/f80 [242478,26221] 0 2024-02-16T19:29:09.364 INFO:tasks.workunit.client.1.smithi117.stdout:5/600: rename d2/db3/l31 to d2/d9/lc8 0 2024-02-16T19:29:09.364 INFO:tasks.workunit.client.1.smithi117.stdout:9/452: dwrite dd/d45/d73/f94 [0,4194304] 0 2024-02-16T19:29:09.365 INFO:tasks.workunit.client.0.smithi022.stdout:8/536: mknod d3/d12/d17/cc9 0 2024-02-16T19:29:09.365 INFO:tasks.workunit.client.1.smithi117.stdout:9/453: stat dd/d45/d73/f82 0 2024-02-16T19:29:09.368 INFO:tasks.workunit.client.1.smithi117.stdout:1/484: rmdir d2/d1d/d60/d68 39 2024-02-16T19:29:09.370 INFO:tasks.workunit.client.0.smithi022.stdout:2/477: rmdir d0/d2/d18/d19 39 2024-02-16T19:29:09.371 INFO:tasks.workunit.client.0.smithi022.stdout:2/478: stat d0/d2/d2e/d73/l9f 0 2024-02-16T19:29:09.375 INFO:tasks.workunit.client.0.smithi022.stdout:7/551: creat d9/d1b/fb0 x:0 0 0 2024-02-16T19:29:09.376 INFO:tasks.workunit.client.1.smithi117.stdout:7/539: write d4/f11 [5016394,57199] 0 2024-02-16T19:29:09.377 INFO:tasks.workunit.client.1.smithi117.stdout:6/526: symlink d5/d4b/d44/lab 0 2024-02-16T19:29:09.380 INFO:tasks.workunit.client.0.smithi022.stdout:0/696: dwrite d10/d15/f9f [0,4194304] 0 2024-02-16T19:29:09.391 INFO:tasks.workunit.client.0.smithi022.stdout:6/524: dwrite d1/d8/da/d17/d53/f7d [0,4194304] 0 2024-02-16T19:29:09.396 INFO:tasks.workunit.client.1.smithi117.stdout:9/454: dread dd/d47/d5b/d5f/f6f [0,4194304] 0 2024-02-16T19:29:09.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.398+0000 7fe206f52700 1 -- 172.21.15.22:0/2090026839 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2001003d0 msgr2=0x7fe200100840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:29:09.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.398+0000 7fe206f52700 1 --2- 172.21.15.22:0/2090026839 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2001003d0 0x7fe200100840 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec009910 tx=0x7fe1ec009c20 comp rx=0 tx=0).stop 2024-02-16T19:29:09.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.398+0000 7fe206f52700 1 -- 172.21.15.22:0/2090026839 shutdown_connections 2024-02-16T19:29:09.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.398+0000 7fe206f52700 1 --2- 172.21.15.22:0/2090026839 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2001003d0 0x7fe200100840 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:09.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.398+0000 7fe206f52700 1 --2- 172.21.15.22:0/2090026839 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2000ff130 0x7fe2000ff540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:09.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.398+0000 7fe206f52700 1 -- 172.21.15.22:0/2090026839 >> 172.21.15.22:0/2090026839 conn(0x7fe2000fa780 msgr2=0x7fe2000fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:29:09.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.398+0000 7fe206f52700 1 -- 172.21.15.22:0/2090026839 shutdown_connections 2024-02-16T19:29:09.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.398+0000 7fe206f52700 1 -- 172.21.15.22:0/2090026839 wait complete. 2024-02-16T19:29:09.400 INFO:tasks.workunit.client.1.smithi117.stdout:8/538: sync 2024-02-16T19:29:09.400 INFO:tasks.workunit.client.1.smithi117.stdout:0/575: sync 2024-02-16T19:29:09.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.399+0000 7fe206f52700 1 Processor -- start 2024-02-16T19:29:09.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.399+0000 7fe206f52700 1 -- start start 2024-02-16T19:29:09.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.400+0000 7fe206f52700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2000ff130 0x7fe20019c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:29:09.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.400+0000 7fe206f52700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2001003d0 0x7fe20019c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:29:09.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.400+0000 7fe206f52700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe20019cc40 con 0x7fe2000ff130 2024-02-16T19:29:09.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.400+0000 7fe206f52700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe20019cd80 con 0x7fe2001003d0 2024-02-16T19:29:09.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.400+0000 7fe1fffff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2001003d0 0x7fe20019c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:29:09.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.400+0000 7fe1fffff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2001003d0 0x7fe20019c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.22:50080/0 (socket says 172.21.15.22:50080) 2024-02-16T19:29:09.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.400+0000 7fe1fffff700 1 -- 172.21.15.22:0/3366505316 learned_addr learned my addr 172.21.15.22:0/3366505316 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:29:09.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.400+0000 7fe204cee700 1 --2- 172.21.15.22:0/3366505316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2000ff130 0x7fe20019c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:29:09.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.401+0000 7fe204cee700 1 -- 172.21.15.22:0/3366505316 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2001003d0 msgr2=0x7fe20019c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:29:09.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.401+0000 7fe204cee700 1 --2- 172.21.15.22:0/3366505316 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2001003d0 0x7fe20019c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:09.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.401+0000 7fe204cee700 1 -- 172.21.15.22:0/3366505316 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1ec0095f0 con 0x7fe2000ff130 2024-02-16T19:29:09.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.401+0000 7fe1fffff700 1 --2- 172.21.15.22:0/3366505316 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2001003d0 0x7fe20019c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T19:29:09.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.401+0000 7fe204cee700 1 --2- 172.21.15.22:0/3366505316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2000ff130 0x7fe20019c0e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe1f400b3d0 tx=0x7fe1f400b6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:29:09.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.401+0000 7fe1fdffb700 1 -- 172.21.15.22:0/3366505316 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1f4011800 con 0x7fe2000ff130 2024-02-16T19:29:09.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.401+0000 7fe206f52700 1 -- 172.21.15.22:0/3366505316 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2001a1830 con 0x7fe2000ff130 2024-02-16T19:29:09.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.402+0000 7fe1fdffb700 1 -- 172.21.15.22:0/3366505316 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe1f400be90 con 0x7fe2000ff130 2024-02-16T19:29:09.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.402+0000 7fe1fdffb700 1 -- 172.21.15.22:0/3366505316 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1f4010e70 con 0x7fe2000ff130 2024-02-16T19:29:09.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.402+0000 7fe206f52700 1 -- 172.21.15.22:0/3366505316 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2001a1d80 con 0x7fe2000ff130 2024-02-16T19:29:09.404 INFO:tasks.workunit.client.1.smithi117.stdout:3/626: getdents d1/dd/d8e 0 2024-02-16T19:29:09.404 INFO:tasks.workunit.client.1.smithi117.stdout:8/539: dread d4/d7/d37/d3a/d65/d6c/f75 [0,4194304] 0 2024-02-16T19:29:09.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.403+0000 7fe1fdffb700 1 -- 172.21.15.22:0/3366505316 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 24) v1 ==== 46073+0+0 (secure 0 0 0) 0x7fe1f401a400 con 0x7fe2000ff130 2024-02-16T19:29:09.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.403+0000 7fe1fdffb700 1 --2- 172.21.15.22:0/3366505316 >> [v2:172.21.15.117:6828/10500008,v1:172.21.15.117:6829/10500008] conn(0x7fe1f0038eb0 0x7fe1f003b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:29:09.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.403+0000 7fe1fdffb700 1 -- 172.21.15.22:0/3366505316 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fe1f404c8f0 con 0x7fe2000ff130 2024-02-16T19:29:09.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.404+0000 7fe1fffff700 1 --2- 172.21.15.22:0/3366505316 >> [v2:172.21.15.117:6828/10500008,v1:172.21.15.117:6829/10500008] conn(0x7fe1f0038eb0 0x7fe1f003b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:29:09.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.404+0000 7fe206f52700 1 -- 172.21.15.22:0/3366505316 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe200107c90 con 0x7fe2000ff130 2024-02-16T19:29:09.405 INFO:tasks.workunit.client.0.smithi022.stdout:9/562: write db/d14/f60 [1139274,12296] 0 2024-02-16T19:29:09.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.404+0000 7fe1fffff700 1 --2- 172.21.15.22:0/3366505316 >> [v2:172.21.15.117:6828/10500008,v1:172.21.15.117:6829/10500008] conn(0x7fe1f0038eb0 0x7fe1f003b360 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec005090 tx=0x7fe1ec00b560 comp rx=0 tx=0).ready entity=mgr.24349 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:29:09.411 INFO:tasks.workunit.client.1.smithi117.stdout:5/601: link d2/d9/d3c/d24/d50/d52/d74/fab d2/d5d/fc9 0 2024-02-16T19:29:09.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.411+0000 7fe1fdffb700 1 -- 172.21.15.22:0/3366505316 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7fe1f40178b0 con 0x7fe2000ff130 2024-02-16T19:29:09.423 INFO:tasks.workunit.client.1.smithi117.stdout:1/485: write d2/d1d/f29 [47854,80391] 0 2024-02-16T19:29:09.425 INFO:tasks.workunit.client.1.smithi117.stdout:4/527: getdents d0/da 0 2024-02-16T19:29:09.429 INFO:tasks.workunit.client.0.smithi022.stdout:2/479: readlink d0/d2/l15 0 2024-02-16T19:29:09.430 INFO:tasks.workunit.client.0.smithi022.stdout:2/480: write d0/d2/d18/d1f/f4d [2345395,45308] 0 2024-02-16T19:29:09.431 INFO:tasks.workunit.client.1.smithi117.stdout:3/627: mknod d1/dd/d8e/cd7 0 2024-02-16T19:29:09.431 INFO:tasks.workunit.client.1.smithi117.stdout:6/527: dwrite d5/db/f1f [4194304,4194304] 0 2024-02-16T19:29:09.431 INFO:tasks.workunit.client.1.smithi117.stdout:6/528: stat d5/f15 0 2024-02-16T19:29:09.431 INFO:tasks.workunit.client.0.smithi022.stdout:2/481: fdatasync d0/d32/d3b/d6f/f5a 0 2024-02-16T19:29:09.432 INFO:tasks.workunit.client.1.smithi117.stdout:6/529: stat d5/fa7 0 2024-02-16T19:29:09.435 INFO:tasks.workunit.client.0.smithi022.stdout:7/552: truncate d9/fa7 197037 0 2024-02-16T19:29:09.440 INFO:tasks.workunit.client.0.smithi022.stdout:0/697: rename d10/d11/l9b to d10/d11/d27/d3f/d94/le5 0 2024-02-16T19:29:09.443 INFO:tasks.workunit.client.1.smithi117.stdout:7/540: mknod d4/d1c/d39/d46/d47/cc8 0 2024-02-16T19:29:09.444 INFO:tasks.workunit.client.0.smithi022.stdout:6/525: readlink d1/d8/da/l3d 0 2024-02-16T19:29:09.446 INFO:tasks.workunit.client.0.smithi022.stdout:6/526: truncate d1/d8/d3c/f47 4383296 0 2024-02-16T19:29:09.446 INFO:tasks.workunit.client.1.smithi117.stdout:9/455: mknod dd/d45/c9d 0 2024-02-16T19:29:09.446 INFO:tasks.workunit.client.0.smithi022.stdout:6/527: chown d1/d8/da/d17/d53/f91 339510745 1 2024-02-16T19:29:09.447 INFO:tasks.workunit.client.1.smithi117.stdout:5/602: dread d2/d3/f6 [0,4194304] 0 2024-02-16T19:29:09.447 INFO:tasks.workunit.client.0.smithi022.stdout:8/537: unlink d3/d12/d2b/f3a 0 2024-02-16T19:29:09.451 INFO:tasks.workunit.client.1.smithi117.stdout:0/576: dwrite d0/f10 [0,4194304] 0 2024-02-16T19:29:09.457 INFO:tasks.workunit.client.1.smithi117.stdout:6/530: rmdir d5 39 2024-02-16T19:29:09.459 INFO:tasks.workunit.client.0.smithi022.stdout:7/553: mknod d9/da/d1f/d93/cb1 0 2024-02-16T19:29:09.462 INFO:tasks.workunit.client.0.smithi022.stdout:9/563: rename db/d14/d16/d5d/d65/d6b/d89/l8a to db/d14/d16/d49/d90/lc2 0 2024-02-16T19:29:09.466 INFO:tasks.workunit.client.1.smithi117.stdout:9/456: mknod dd/d47/d5b/c9e 0 2024-02-16T19:29:09.467 INFO:tasks.workunit.client.0.smithi022.stdout:7/554: dwrite d9/da/d1f/f18 [0,4194304] 0 2024-02-16T19:29:09.469 INFO:tasks.workunit.client.0.smithi022.stdout:6/528: dread - d1/d8/f8f zero size 2024-02-16T19:29:09.470 INFO:tasks.workunit.client.0.smithi022.stdout:6/529: chown d1/d8/d3c/f9d 89 1 2024-02-16T19:29:09.471 INFO:tasks.workunit.client.1.smithi117.stdout:4/528: dwrite d0/f14 [0,4194304] 0 2024-02-16T19:29:09.475 INFO:tasks.workunit.client.0.smithi022.stdout:2/482: creat d0/d2/d18/d19/d90/fa0 x:0 0 0 2024-02-16T19:29:09.476 INFO:tasks.workunit.client.0.smithi022.stdout:2/483: chown d0/d32/f58 82347302 1 2024-02-16T19:29:09.476 INFO:tasks.workunit.client.1.smithi117.stdout:1/486: mkdir d2/d19/d59/d89/d8d/da4 0 2024-02-16T19:29:09.476 INFO:tasks.workunit.client.1.smithi117.stdout:0/577: mkdir d0/d9/d2c/d8b/dcd 0 2024-02-16T19:29:09.477 INFO:tasks.workunit.client.1.smithi117.stdout:0/578: chown d0/f1a 30207 1 2024-02-16T19:29:09.481 INFO:tasks.workunit.client.0.smithi022.stdout:9/564: write db/d14/d16/d21/fb6 [7801295,19866] 0 2024-02-16T19:29:09.484 INFO:tasks.workunit.client.0.smithi022.stdout:2/484: dwrite d0/f5f [0,4194304] 0 2024-02-16T19:29:09.488 INFO:tasks.workunit.client.1.smithi117.stdout:8/540: rename d4/d7/d37 to d4/d7/da6 0 2024-02-16T19:29:09.488 INFO:tasks.workunit.client.0.smithi022.stdout:0/698: write d10/d22/f2a [1844462,72379] 0 2024-02-16T19:29:09.488 INFO:tasks.workunit.client.1.smithi117.stdout:8/541: chown d4/d7/da6/d3a/d43/c4f 28340444 1 2024-02-16T19:29:09.491 INFO:tasks.workunit.client.1.smithi117.stdout:7/541: symlink d4/d1c/d87/d89/d60/d79/lc9 0 2024-02-16T19:29:09.496 INFO:tasks.workunit.client.1.smithi117.stdout:7/542: dread d4/d1c/d39/d46/d47/fb5 [0,4194304] 0 2024-02-16T19:29:09.497 INFO:tasks.workunit.client.1.smithi117.stdout:5/603: dwrite d2/d9/d3c/d24/d50/d96/dbc/d30/f3e [0,4194304] 0 2024-02-16T19:29:09.497 INFO:tasks.workunit.client.1.smithi117.stdout:5/604: chown d2/d36/d6f/d9c 17289 1 2024-02-16T19:29:09.498 INFO:tasks.workunit.client.1.smithi117.stdout:1/487: dread d2/ff [4194304,4194304] 0 2024-02-16T19:29:09.499 INFO:tasks.workunit.client.1.smithi117.stdout:2/545: sync 2024-02-16T19:29:09.501 INFO:tasks.workunit.client.1.smithi117.stdout:0/579: symlink d0/d9/d14/d1d/d60/d76/lce 0 2024-02-16T19:29:09.503 INFO:tasks.workunit.client.0.smithi022.stdout:6/530: mknod d1/d8/da/d7b/cb9 0 2024-02-16T19:29:09.506 INFO:tasks.workunit.client.0.smithi022.stdout:8/538: mkdir d3/d12/d2b/dca 0 2024-02-16T19:29:09.513 INFO:tasks.workunit.client.0.smithi022.stdout:9/565: creat db/d14/d16/d27/d55/fc3 x:0 0 0 2024-02-16T19:29:09.518 INFO:tasks.workunit.client.0.smithi022.stdout:0/699: creat d10/d11/d27/fe6 x:0 0 0 2024-02-16T19:29:09.521 INFO:tasks.workunit.client.0.smithi022.stdout:7/555: chown d9/d1b/f27 0 1 2024-02-16T19:29:09.524 INFO:tasks.workunit.client.0.smithi022.stdout:6/531: creat d1/fba x:0 0 0 2024-02-16T19:29:09.525 INFO:tasks.workunit.client.0.smithi022.stdout:2/485: write d0/d2/d18/d19/f56 [83756,74991] 0 2024-02-16T19:29:09.526 INFO:tasks.workunit.client.0.smithi022.stdout:2/486: readlink d0/d5d/l92 0 2024-02-16T19:29:09.529 INFO:tasks.workunit.client.0.smithi022.stdout:9/566: symlink db/d14/d16/d21/d4a/lc4 0 2024-02-16T19:29:09.529 INFO:tasks.workunit.client.1.smithi117.stdout:2/546: chown d0/d3/d34/c85 79 1 2024-02-16T19:29:09.530 INFO:tasks.workunit.client.0.smithi022.stdout:9/567: write db/d14/d16/d21/d37/f4e [31055,53298] 0 2024-02-16T19:29:09.532 INFO:tasks.workunit.client.0.smithi022.stdout:9/568: write db/d18/fb4 [250302,123077] 0 2024-02-16T19:29:09.533 INFO:tasks.workunit.client.0.smithi022.stdout:0/700: mkdir d10/d22/dad/de7 0 2024-02-16T19:29:09.534 INFO:tasks.workunit.client.0.smithi022.stdout:8/539: write d3/d12/d6a/f6e [774910,125187] 0 2024-02-16T19:29:09.535 INFO:tasks.workunit.client.0.smithi022.stdout:8/540: chown d3/d12/db0/fb9 1 1 2024-02-16T19:29:09.536 INFO:tasks.workunit.client.1.smithi117.stdout:2/547: dwrite d0/d3/d34/d94/d46/d68/f8e [0,4194304] 0 2024-02-16T19:29:09.536 INFO:tasks.workunit.client.0.smithi022.stdout:8/541: chown d3/d12/c31 498434907 1 2024-02-16T19:29:09.539 INFO:tasks.workunit.client.1.smithi117.stdout:8/542: mkdir d4/d12/d2a/da7 0 2024-02-16T19:29:09.540 INFO:tasks.workunit.client.1.smithi117.stdout:5/605: dread d2/db3/f55 [0,4194304] 0 2024-02-16T19:29:09.540 INFO:tasks.workunit.client.0.smithi022.stdout:2/487: mkdir d0/d2/d2e/d82/da1 0 2024-02-16T19:29:09.542 INFO:tasks.workunit.client.0.smithi022.stdout:2/488: write d0/d2/d18/d1f/d6c/f9e [1006752,23342] 0 2024-02-16T19:29:09.543 INFO:tasks.workunit.client.0.smithi022.stdout:2/489: write d0/d2/f8f [139840,89801] 0 2024-02-16T19:29:09.545 INFO:tasks.workunit.client.0.smithi022.stdout:2/490: read - d0/d32/f94 zero size 2024-02-16T19:29:09.546 INFO:tasks.workunit.client.1.smithi117.stdout:9/457: link dd/d2d/f4d dd/d23/d64/f9f 0 2024-02-16T19:29:09.546 INFO:tasks.workunit.client.1.smithi117.stdout:6/531: truncate d5/f12 1187011 0 2024-02-16T19:29:09.546 INFO:tasks.workunit.client.1.smithi117.stdout:6/532: chown d5/db/d7d 3712 1 2024-02-16T19:29:09.546 INFO:tasks.workunit.client.0.smithi022.stdout:2/491: fsync d0/d2/f85 0 2024-02-16T19:29:09.546 INFO:tasks.workunit.client.1.smithi117.stdout:9/458: truncate dd/d47/f7b 47467 0 2024-02-16T19:29:09.547 INFO:tasks.workunit.client.1.smithi117.stdout:4/529: creat d0/da/d21/d3d/d28/d64/d73/fa9 x:0 0 0 2024-02-16T19:29:09.549 INFO:tasks.workunit.client.1.smithi117.stdout:3/628: sync 2024-02-16T19:29:09.550 INFO:tasks.workunit.client.0.smithi022.stdout:9/569: creat db/d14/d16/d21/fc5 x:0 0 0 2024-02-16T19:29:09.551 INFO:tasks.workunit.client.1.smithi117.stdout:1/488: mkdir d2/d1d/d60/d7c/d9e/da5 0 2024-02-16T19:29:09.552 INFO:tasks.workunit.client.0.smithi022.stdout:0/701: stat d10/d15/d23/d38/d5a/cb4 0 2024-02-16T19:29:09.552 INFO:tasks.workunit.client.1.smithi117.stdout:1/489: write d2/d19/d59/d89/d7d/d7f/f94 [32804,47745] 0 2024-02-16T19:29:09.559 INFO:tasks.workunit.client.1.smithi117.stdout:2/548: creat d0/d13/d4d/d53/fa4 x:0 0 0 2024-02-16T19:29:09.560 INFO:tasks.workunit.client.1.smithi117.stdout:8/543: fsync d4/d12/d2a/f71 0 2024-02-16T19:29:09.563 INFO:tasks.workunit.client.0.smithi022.stdout:2/492: creat d0/d2/d18/d1f/d6c/fa2 x:0 0 0 2024-02-16T19:29:09.564 INFO:tasks.workunit.client.1.smithi117.stdout:6/533: creat d5/db/d7d/d32/d95/d49/fac x:0 0 0 2024-02-16T19:29:09.565 INFO:tasks.workunit.client.0.smithi022.stdout:7/556: truncate d9/da/d1f/d44/f54 474854 0 2024-02-16T19:29:09.565 INFO:tasks.workunit.client.1.smithi117.stdout:7/543: dwrite d4/f28 [0,4194304] 0 2024-02-16T19:29:09.565 INFO:tasks.workunit.client.1.smithi117.stdout:6/534: dread d5/d4b/d20/f3a [0,4194304] 0 2024-02-16T19:29:09.566 INFO:tasks.workunit.client.0.smithi022.stdout:9/570: truncate f5 3940314 0 2024-02-16T19:29:09.567 INFO:tasks.workunit.client.0.smithi022.stdout:9/571: chown db/d14/d16/d28/daa 503951931 1 2024-02-16T19:29:09.569 INFO:tasks.workunit.client.1.smithi117.stdout:9/459: unlink dd/d23/f97 0 2024-02-16T19:29:09.571 INFO:tasks.workunit.client.0.smithi022.stdout:0/702: rmdir d10/d7d 39 2024-02-16T19:29:09.573 INFO:tasks.workunit.client.0.smithi022.stdout:7/557: dwrite d9/da/d74/f80 [0,4194304] 0 2024-02-16T19:29:09.573 INFO:tasks.workunit.client.1.smithi117.stdout:4/530: mknod d0/da/d21/caa 0 2024-02-16T19:29:09.578 INFO:tasks.workunit.client.0.smithi022.stdout:2/493: unlink d0/d2/f1c 0 2024-02-16T19:29:09.579 INFO:tasks.workunit.client.0.smithi022.stdout:2/494: dread - d0/d32/d3b/d6f/f9b zero size 2024-02-16T19:29:09.584 INFO:tasks.workunit.client.1.smithi117.stdout:1/490: fsync d2/d19/d59/d89/d7d/d7f/f85 0 2024-02-16T19:29:09.586 INFO:tasks.workunit.client.1.smithi117.stdout:1/491: dread - d2/d19/d59/d89/d69/f8f zero size 2024-02-16T19:29:09.587 INFO:tasks.workunit.client.0.smithi022.stdout:2/495: dwrite d0/d2/d18/f28 [0,4194304] 0 2024-02-16T19:29:09.591 INFO:tasks.workunit.client.0.smithi022.stdout:6/532: getdents d1/d8/da/d17/d85/d56 0 2024-02-16T19:29:09.592 INFO:tasks.workunit.client.1.smithi117.stdout:4/531: dread d0/da/d21/f40 [0,4194304] 0 2024-02-16T19:29:09.594 INFO:tasks.workunit.client.1.smithi117.stdout:5/606: mkdir d2/d9/d3c/d24/d50/d96/dc4/dca 0 2024-02-16T19:29:09.595 INFO:tasks.workunit.client.0.smithi022.stdout:8/542: truncate d3/d12/da6/f4d 364724 0 2024-02-16T19:29:09.596 INFO:tasks.workunit.client.0.smithi022.stdout:8/543: chown d3/d12/f51 92 1 2024-02-16T19:29:09.597 INFO:tasks.workunit.client.1.smithi117.stdout:8/544: rename d4/d7/da6/d3a/d54/d70 to d4/d7/d6a/d6f/da8 0 2024-02-16T19:29:09.599 INFO:tasks.workunit.client.0.smithi022.stdout:5/521: sync 2024-02-16T19:29:09.599 INFO:tasks.workunit.client.0.smithi022.stdout:9/572: mknod db/d14/d16/d27/d92/cc6 0 2024-02-16T19:29:09.600 INFO:tasks.workunit.client.0.smithi022.stdout:5/522: chown dd/d1f/d48/d21/d64 11 1 2024-02-16T19:29:09.600 INFO:tasks.workunit.client.0.smithi022.stdout:9/573: chown db/d18/f93 73593 1 2024-02-16T19:29:09.601 INFO:tasks.workunit.client.0.smithi022.stdout:9/574: write db/d14/d16/d21/fc1 [880483,117584] 0 2024-02-16T19:29:09.614 INFO:tasks.workunit.client.0.smithi022.stdout:2/496: creat d0/d2/d18/d19/d90/fa3 x:0 0 0 2024-02-16T19:29:09.616 INFO:tasks.workunit.client.1.smithi117.stdout:3/629: symlink d1/d30/ld8 0 2024-02-16T19:29:09.618 INFO:tasks.workunit.client.0.smithi022.stdout:6/533: mkdir d1/d8/da/d7b/dae/dbb 0 2024-02-16T19:29:09.618 INFO:tasks.workunit.client.1.smithi117.stdout:0/580: getdents d0/d78 0 2024-02-16T19:29:09.619 INFO:tasks.workunit.client.0.smithi022.stdout:2/497: dread d0/d2/d2e/f2a [0,4194304] 0 2024-02-16T19:29:09.627 INFO:tasks.workunit.client.0.smithi022.stdout:5/523: creat dd/d1f/d48/d21/d5d/fb3 x:0 0 0 2024-02-16T19:29:09.628 INFO:tasks.workunit.client.0.smithi022.stdout:5/524: stat dd/d1f/d48/d21/d2f/d52/d5c/l67 0 2024-02-16T19:29:09.630 INFO:tasks.workunit.client.0.smithi022.stdout:0/703: write d10/d22/d49/f76 [1215693,66894] 0 2024-02-16T19:29:09.633 INFO:tasks.workunit.client.1.smithi117.stdout:9/460: rename dd/d47/f6d to dd/d45/d62/fa0 0 2024-02-16T19:29:09.635 INFO:tasks.workunit.client.1.smithi117.stdout:8/545: mkdir d4/d7/da6/d3a/d65/d6c/da9 0 2024-02-16T19:29:09.638 INFO:tasks.workunit.client.0.smithi022.stdout:8/544: write d3/d12/f48 [1487262,25153] 0 2024-02-16T19:29:09.639 INFO:tasks.workunit.client.0.smithi022.stdout:2/498: symlink d0/d2/d18/d19/d90/la4 0 2024-02-16T19:29:09.641 INFO:tasks.workunit.client.0.smithi022.stdout:2/499: chown d0/d2/d2e/d73/l9f 1693 1 2024-02-16T19:29:09.642 INFO:tasks.workunit.client.1.smithi117.stdout:6/535: creat d5/d4b/d44/d52/d81/d9f/fad x:0 0 0 2024-02-16T19:29:09.642 INFO:tasks.workunit.client.0.smithi022.stdout:5/525: dread dd/f3c [0,4194304] 0 2024-02-16T19:29:09.642 INFO:tasks.workunit.client.0.smithi022.stdout:2/500: fsync d0/d2/d18/d19/d40/d55/f7c 0 2024-02-16T19:29:09.644 INFO:tasks.workunit.client.0.smithi022.stdout:5/526: chown dd/d1f/d48/d21/d2f/d40/f91 161 1 2024-02-16T19:29:09.646 INFO:tasks.workunit.client.1.smithi117.stdout:6/536: dwrite d5/f9 [0,4194304] 0 2024-02-16T19:29:09.650 INFO:tasks.workunit.client.1.smithi117.stdout:0/581: symlink d0/d9/d14/d1d/d60/d76/d7b/d8d/lcf 0 2024-02-16T19:29:09.652 INFO:tasks.workunit.client.0.smithi022.stdout:6/534: symlink d1/d8/da/d17/lbc 0 2024-02-16T19:29:09.653 INFO:tasks.workunit.client.0.smithi022.stdout:6/535: chown d1/d8/d23/f68 5 1 2024-02-16T19:29:09.656 INFO:tasks.workunit.client.0.smithi022.stdout:8/545: chown d3/d12/d2b/f6c 18232 1 2024-02-16T19:29:09.657 INFO:tasks.workunit.client.1.smithi117.stdout:4/532: creat d0/da/d21/d3d/d28/d64/d48/fab x:0 0 0 2024-02-16T19:29:09.658 INFO:tasks.workunit.client.1.smithi117.stdout:7/544: dwrite d4/f1a [0,4194304] 0 2024-02-16T19:29:09.666 INFO:tasks.workunit.client.0.smithi022.stdout:2/501: creat d0/d2/d18/d19/d40/fa5 x:0 0 0 2024-02-16T19:29:09.667 INFO:tasks.workunit.client.1.smithi117.stdout:8/546: creat d4/d7/d6a/d6f/da8/d76/faa x:0 0 0 2024-02-16T19:29:09.669 INFO:tasks.workunit.client.0.smithi022.stdout:5/527: mkdir dd/d1f/d48/d21/d2f/d46/d5e/d84/db4 0 2024-02-16T19:29:09.670 INFO:tasks.workunit.client.0.smithi022.stdout:5/528: write dd/d1f/d48/f60 [4672807,12540] 0 2024-02-16T19:29:09.671 INFO:tasks.workunit.client.0.smithi022.stdout:0/704: creat d10/d22/dad/dd7/fe8 x:0 0 0 2024-02-16T19:29:09.673 INFO:tasks.workunit.client.1.smithi117.stdout:3/630: creat d1/dd/d5c/d85/fd9 x:0 0 0 2024-02-16T19:29:09.676 INFO:tasks.workunit.client.1.smithi117.stdout:0/582: symlink d0/d9/d14/dbc/ld0 0 2024-02-16T19:29:09.679 INFO:tasks.workunit.client.1.smithi117.stdout:1/492: truncate d2/f5b 51085 0 2024-02-16T19:29:09.680 INFO:tasks.workunit.client.1.smithi117.stdout:1/493: stat d2/d19/d59/d76/f7a 0 2024-02-16T19:29:09.681 INFO:tasks.workunit.client.1.smithi117.stdout:4/533: unlink d0/da/l12 0 2024-02-16T19:29:09.684 INFO:tasks.workunit.client.0.smithi022.stdout:5/529: mkdir dd/d1f/d48/d21/d2f/d40/d49/d4b/db5 0 2024-02-16T19:29:09.684 INFO:tasks.workunit.client.0.smithi022.stdout:8/546: write d3/d12/f71 [480461,125683] 0 2024-02-16T19:29:09.687 INFO:tasks.workunit.client.0.smithi022.stdout:6/536: creat d1/d8/da/d7b/dae/dbb/fbd x:0 0 0 2024-02-16T19:29:09.689 INFO:tasks.workunit.client.0.smithi022.stdout:6/537: write d1/d8/da/d17/f99 [595375,24350] 0 2024-02-16T19:29:09.690 INFO:tasks.workunit.client.1.smithi117.stdout:2/549: sync 2024-02-16T19:29:09.691 INFO:tasks.workunit.client.0.smithi022.stdout:2/502: truncate d0/d2/d18/d19/f2f 3938419 0 2024-02-16T19:29:09.692 INFO:tasks.workunit.client.1.smithi117.stdout:0/583: dread d0/d9/d14/d1d/d60/d76/d7b/d8d/f97 [0,4194304] 0 2024-02-16T19:29:09.694 INFO:tasks.workunit.client.1.smithi117.stdout:2/550: dread d0/f25 [0,4194304] 0 2024-02-16T19:29:09.695 INFO:tasks.workunit.client.0.smithi022.stdout:0/705: mkdir d10/d7d/dac/de9 0 2024-02-16T19:29:09.697 INFO:tasks.workunit.client.1.smithi117.stdout:3/631: write d1/d1b/d1e/d37/f79 [1456431,86540] 0 2024-02-16T19:29:09.698 INFO:tasks.workunit.client.1.smithi117.stdout:6/537: mkdir d5/d4b/d44/d93/dae 0 2024-02-16T19:29:09.698 INFO:tasks.workunit.client.1.smithi117.stdout:6/538: stat d5/d4b/d44/d52/f6c 0 2024-02-16T19:29:09.699 INFO:tasks.workunit.client.0.smithi022.stdout:5/530: mkdir dd/d1f/d48/d21/d2f/d46/d5e/db6 0 2024-02-16T19:29:09.702 INFO:tasks.workunit.client.1.smithi117.stdout:1/494: unlink d2/d19/l45 0 2024-02-16T19:29:09.703 INFO:tasks.workunit.client.0.smithi022.stdout:6/538: symlink d1/d8/d3c/d77/lbe 0 2024-02-16T19:29:09.704 INFO:tasks.workunit.client.0.smithi022.stdout:6/539: stat d1/d8/c16 0 2024-02-16T19:29:09.708 INFO:tasks.workunit.client.0.smithi022.stdout:0/706: symlink d10/d11/d27/d3f/d58/lea 0 2024-02-16T19:29:09.709 INFO:tasks.workunit.client.1.smithi117.stdout:5/607: getdents d2/d9/d3c/d42 0 2024-02-16T19:29:09.711 INFO:tasks.workunit.client.1.smithi117.stdout:9/461: rmdir dd/d47/d5b/d90 0 2024-02-16T19:29:09.716 INFO:tasks.workunit.client.1.smithi117.stdout:8/547: unlink d4/d7/d6a/d6f/f7a 0 2024-02-16T19:29:09.716 INFO:tasks.workunit.client.0.smithi022.stdout:0/707: dwrite d10/d11/d27/d3f/f5b [0,4194304] 0 2024-02-16T19:29:09.717 INFO:tasks.workunit.client.0.smithi022.stdout:0/708: write d10/d15/d23/fe4 [273546,117561] 0 2024-02-16T19:29:09.717 INFO:tasks.workunit.client.1.smithi117.stdout:9/462: dwrite dd/d47/d5b/f69 [4194304,4194304] 0 2024-02-16T19:29:09.717 INFO:tasks.workunit.client.0.smithi022.stdout:0/709: dread - d10/d22/fd3 zero size 2024-02-16T19:29:09.719 INFO:tasks.workunit.client.1.smithi117.stdout:9/463: dread dd/d47/f7b [0,4194304] 0 2024-02-16T19:29:09.719 INFO:tasks.workunit.client.0.smithi022.stdout:6/540: mknod d1/d8/da/d7b/dae/dbb/cbf 0 2024-02-16T19:29:09.721 INFO:tasks.workunit.client.1.smithi117.stdout:2/551: mkdir d0/d13/d4d/da5 0 2024-02-16T19:29:09.724 INFO:tasks.workunit.client.0.smithi022.stdout:8/547: creat d3/d12/d2b/d54/fcb x:0 0 0 2024-02-16T19:29:09.727 INFO:tasks.workunit.client.0.smithi022.stdout:5/531: rename dd/d1f/c2a to dd/d1f/d48/d21/d2f/d46/d5e/db6/cb7 0 2024-02-16T19:29:09.729 INFO:tasks.workunit.client.0.smithi022.stdout:6/541: symlink d1/d8/da/d17/d85/d5b/lc0 0 2024-02-16T19:29:09.731 INFO:tasks.workunit.client.1.smithi117.stdout:6/539: creat d5/db/d7d/d32/faf x:0 0 0 2024-02-16T19:29:09.734 INFO:tasks.workunit.client.0.smithi022.stdout:8/548: rmdir d3/d12/d17 39 2024-02-16T19:29:09.738 INFO:tasks.workunit.client.0.smithi022.stdout:0/710: rename d10/d22/d49/de1 to d10/d15/d23/d8f/dc1/deb 0 2024-02-16T19:29:09.739 INFO:tasks.workunit.client.0.smithi022.stdout:6/542: mknod d1/d8/d81/cc1 0 2024-02-16T19:29:09.741 INFO:tasks.workunit.client.0.smithi022.stdout:8/549: rmdir d3/d12/d2b/d54/db8 39 2024-02-16T19:29:09.742 INFO:tasks.workunit.client.1.smithi117.stdout:0/584: symlink d0/d9/d14/d1d/d60/dc3/ld1 0 2024-02-16T19:29:09.742 INFO:tasks.workunit.client.1.smithi117.stdout:0/585: readlink d0/d9/d2c/d8b/l87 0 2024-02-16T19:29:09.743 INFO:tasks.workunit.client.0.smithi022.stdout:5/532: rename dd/d1f/d2e/f88 to dd/d1f/d48/d21/d2f/d52/d77/d95/fb8 0 2024-02-16T19:29:09.744 INFO:tasks.workunit.client.1.smithi117.stdout:8/548: symlink d4/d12/d31/d8e/lab 0 2024-02-16T19:29:09.746 INFO:tasks.workunit.client.0.smithi022.stdout:0/711: fdatasync d10/d22/fa1 0 2024-02-16T19:29:09.747 INFO:tasks.workunit.client.0.smithi022.stdout:0/712: chown d10/d11/d27/cb3 1419 1 2024-02-16T19:29:09.751 INFO:tasks.workunit.client.0.smithi022.stdout:5/533: dread dd/f11 [4194304,4194304] 0 2024-02-16T19:29:09.753 INFO:tasks.workunit.client.0.smithi022.stdout:8/550: write d3/d12/d17/d64/f73 [1435841,26685] 0 2024-02-16T19:29:09.757 INFO:tasks.workunit.client.1.smithi117.stdout:3/632: symlink d1/dd/d3b/da3/lda 0 2024-02-16T19:29:09.759 INFO:tasks.workunit.client.0.smithi022.stdout:2/503: dwrite d0/d2/d18/d19/f2f [4194304,4194304] 0 2024-02-16T19:29:09.760 INFO:tasks.workunit.client.1.smithi117.stdout:1/495: mknod d2/d1d/d1f/d9c/ca6 0 2024-02-16T19:29:09.762 INFO:tasks.workunit.client.1.smithi117.stdout:3/633: dwrite d1/d30/d9f/dbc/fc5 [0,4194304] 0 2024-02-16T19:29:09.764 INFO:tasks.workunit.client.1.smithi117.stdout:6/540: mkdir d5/d9c/daa/d85/db0 0 2024-02-16T19:29:09.768 INFO:tasks.workunit.client.0.smithi022.stdout:6/543: creat d1/d8/d3c/d4d/fc2 x:0 0 0 2024-02-16T19:29:09.769 INFO:tasks.workunit.client.0.smithi022.stdout:5/534: dread dd/d1f/d48/d21/d2f/d52/d5c/d61/d6a/f7f [0,4194304] 0 2024-02-16T19:29:09.772 INFO:tasks.workunit.client.1.smithi117.stdout:7/545: getdents d4/d13/da3 0 2024-02-16T19:29:09.773 INFO:tasks.workunit.client.1.smithi117.stdout:7/546: chown d4/d1c/d39/d46/d6e/f99 696892 1 2024-02-16T19:29:09.777 INFO:tasks.workunit.client.1.smithi117.stdout:0/586: creat d0/d30/fd2 x:0 0 0 2024-02-16T19:29:09.778 INFO:tasks.workunit.client.0.smithi022.stdout:8/551: rmdir d3/d12/d2b/d54/d74/da2 39 2024-02-16T19:29:09.780 INFO:tasks.workunit.client.0.smithi022.stdout:2/504: truncate d0/d2/f54 829990 0 2024-02-16T19:29:09.783 INFO:tasks.workunit.client.0.smithi022.stdout:6/544: creat d1/d8/d3c/d77/fc3 x:0 0 0 2024-02-16T19:29:09.786 INFO:tasks.workunit.client.0.smithi022.stdout:6/545: write d1/d8/d3c/d4d/f4b [2659251,115049] 0 2024-02-16T19:29:09.788 INFO:tasks.workunit.client.0.smithi022.stdout:2/505: dwrite d0/d2/d18/d19/d40/d55/d59/f74 [0,4194304] 0 2024-02-16T19:29:09.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.796+0000 7fe206f52700 1 -- 172.21.15.22:0/3366505316 --> [v2:172.21.15.117:6828/10500008,v1:172.21.15.117:6829/10500008] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe200068c50 con 0x7fe1f0038eb0 2024-02-16T19:29:09.797 INFO:tasks.workunit.client.1.smithi117.stdout:4/534: dwrite d0/da/f69 [0,4194304] 0 2024-02-16T19:29:09.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.798+0000 7fe1fdffb700 1 -- 172.21.15.22:0/3366505316 <== mgr.24349 v2:172.21.15.117:6828/10500008 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7fe200068c50 con 0x7fe1f0038eb0 2024-02-16T19:29:09.800 INFO:tasks.workunit.client.0.smithi022.stdout:8/552: chown d3/d12/d2b/d2e/l9b 4016 1 2024-02-16T19:29:09.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.801+0000 7fe206f52700 1 -- 172.21.15.22:0/3366505316 >> [v2:172.21.15.117:6828/10500008,v1:172.21.15.117:6829/10500008] conn(0x7fe1f0038eb0 msgr2=0x7fe1f003b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:29:09.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.801+0000 7fe206f52700 1 --2- 172.21.15.22:0/3366505316 >> [v2:172.21.15.117:6828/10500008,v1:172.21.15.117:6829/10500008] conn(0x7fe1f0038eb0 0x7fe1f003b360 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec005090 tx=0x7fe1ec00b560 comp rx=0 tx=0).stop 2024-02-16T19:29:09.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.801+0000 7fe206f52700 1 -- 172.21.15.22:0/3366505316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2000ff130 msgr2=0x7fe20019c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:29:09.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.801+0000 7fe206f52700 1 --2- 172.21.15.22:0/3366505316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2000ff130 0x7fe20019c0e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe1f400b3d0 tx=0x7fe1f400b6e0 comp rx=0 tx=0).stop 2024-02-16T19:29:09.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.801+0000 7fe206f52700 1 -- 172.21.15.22:0/3366505316 shutdown_connections 2024-02-16T19:29:09.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.801+0000 7fe206f52700 1 --2- 172.21.15.22:0/3366505316 >> [v2:172.21.15.117:6828/10500008,v1:172.21.15.117:6829/10500008] conn(0x7fe1f0038eb0 0x7fe1f003b360 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:09.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.801+0000 7fe206f52700 1 --2- 172.21.15.22:0/3366505316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2000ff130 0x7fe20019c0e0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:09.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.801+0000 7fe206f52700 1 --2- 172.21.15.22:0/3366505316 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2001003d0 0x7fe20019c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:09.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.801+0000 7fe206f52700 1 -- 172.21.15.22:0/3366505316 >> 172.21.15.22:0/3366505316 conn(0x7fe2000fa780 msgr2=0x7fe200068540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:29:09.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.802+0000 7fe206f52700 1 -- 172.21.15.22:0/3366505316 shutdown_connections 2024-02-16T19:29:09.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.802+0000 7fe206f52700 1 -- 172.21.15.22:0/3366505316 wait complete. 2024-02-16T19:29:09.804 INFO:tasks.workunit.client.0.smithi022.stdout:6/546: rename d1/d8/d81/f98 to d1/d8/da/d17/d85/d56/fc4 0 2024-02-16T19:29:09.808 INFO:tasks.workunit.client.0.smithi022.stdout:2/506: symlink d0/d2/d18/d38/la6 0 2024-02-16T19:29:09.810 INFO:tasks.workunit.client.0.smithi022.stdout:2/507: chown d0/d2/d18/f28 30977811 1 2024-02-16T19:29:09.811 INFO:tasks.workunit.client.0.smithi022.stdout:2/508: fsync d0/d2/d18/d19/d90/fa0 0 2024-02-16T19:29:09.811 INFO:tasks.workunit.client.0.smithi022.stdout:0/713: truncate d10/d22/d49/f76 195714 0 2024-02-16T19:29:09.817 INFO:tasks.workunit.client.0.smithi022.stdout:6/547: symlink d1/d8/da/d17/d53/d7f/da3/lc5 0 2024-02-16T19:29:09.818 INFO:tasks.workunit.client.0.smithi022.stdout:2/509: dwrite d0/d2/d18/d38/d6b/f8b [0,4194304] 0 2024-02-16T19:29:09.820 INFO:tasks.workunit.client.0.smithi022.stdout:2/510: fsync d0/d2/d18/d19/f30 0 2024-02-16T19:29:09.824 INFO:tasks.workunit.client.0.smithi022.stdout:5/535: write dd/d1f/d48/d21/d2f/d52/d5c/f63 [46529,24165] 0 2024-02-16T19:29:09.825 INFO:tasks.workunit.client.0.smithi022.stdout:5/536: chown dd/d1f/d48/d21/f25 827 1 2024-02-16T19:29:09.829 INFO:tasks.workunit.client.0.smithi022.stdout:3/481: sync 2024-02-16T19:29:09.831 INFO:tasks.workunit.client.0.smithi022.stdout:3/482: write d7/d12/d18/dd/d31/d7f/faf [827231,34907] 0 2024-02-16T19:29:09.831 INFO:tasks.workunit.client.0.smithi022.stdout:2/511: dread d0/f20 [0,4194304] 0 2024-02-16T19:29:09.839 INFO:tasks.workunit.client.0.smithi022.stdout:0/714: dread fb [0,4194304] 0 2024-02-16T19:29:09.840 INFO:tasks.workunit.client.0.smithi022.stdout:2/512: dwrite d0/d2/d18/f22 [0,4194304] 0 2024-02-16T19:29:09.842 INFO:tasks.workunit.client.0.smithi022.stdout:8/553: write d3/d25/f43 [699582,65327] 0 2024-02-16T19:29:09.845 INFO:tasks.workunit.client.1.smithi117.stdout:9/464: mknod dd/d47/d5e/d9a/ca1 0 2024-02-16T19:29:09.846 INFO:tasks.workunit.client.1.smithi117.stdout:9/465: write dd/d23/d64/f70 [862759,67136] 0 2024-02-16T19:29:09.853 INFO:tasks.workunit.client.0.smithi022.stdout:3/483: dwrite d7/d12/d18/dac/f9d [0,4194304] 0 2024-02-16T19:29:09.861 INFO:tasks.workunit.client.0.smithi022.stdout:6/548: write d1/d8/f1c [3401334,88409] 0 2024-02-16T19:29:09.865 INFO:tasks.workunit.client.0.smithi022.stdout:5/537: creat dd/d1f/d48/d21/d2f/d46/d5e/d84/db4/fb9 x:0 0 0 2024-02-16T19:29:09.868 INFO:tasks.workunit.client.1.smithi117.stdout:0/587: fsync d0/d9/d14/d1d/d1e/f2d 0 2024-02-16T19:29:09.870 INFO:tasks.workunit.client.1.smithi117.stdout:2/552: write d0/d3/d22/f5d [228370,101800] 0 2024-02-16T19:29:09.873 INFO:tasks.workunit.client.0.smithi022.stdout:8/554: dread d3/d12/da6/f28 [0,4194304] 0 2024-02-16T19:29:09.874 INFO:tasks.workunit.client.0.smithi022.stdout:3/484: chown d7/d12/d18/dd/f7d 3716 1 2024-02-16T19:29:09.876 INFO:tasks.workunit.client.1.smithi117.stdout:1/496: dwrite d2/d4/f17 [4194304,4194304] 0 2024-02-16T19:29:09.877 INFO:tasks.workunit.client.0.smithi022.stdout:6/549: mkdir d1/d8/d23/dc6 0 2024-02-16T19:29:09.878 INFO:tasks.workunit.client.1.smithi117.stdout:1/497: fdatasync d2/d19/d59/d89/f42 0 2024-02-16T19:29:09.879 INFO:tasks.workunit.client.0.smithi022.stdout:5/538: mknod dd/d1f/d48/d21/d5d/cba 0 2024-02-16T19:29:09.881 INFO:tasks.workunit.client.1.smithi117.stdout:9/466: dwrite dd/d47/d5b/d5f/d77/d7c/f8b [0,4194304] 0 2024-02-16T19:29:09.882 INFO:tasks.workunit.client.1.smithi117.stdout:9/467: dread - dd/d23/d64/f61 zero size 2024-02-16T19:29:09.883 INFO:tasks.workunit.client.1.smithi117.stdout:3/634: link d1/d30/f77 d1/fdb 0 2024-02-16T19:29:09.884 INFO:tasks.workunit.client.1.smithi117.stdout:3/635: read d1/d30/f77 [3939899,104447] 0 2024-02-16T19:29:09.884 INFO:tasks.workunit.client.1.smithi117.stdout:6/541: mkdir d5/d4b/d44/db1 0 2024-02-16T19:29:09.886 INFO:tasks.workunit.client.0.smithi022.stdout:3/485: dwrite d7/d21/f5d [0,4194304] 0 2024-02-16T19:29:09.886 INFO:tasks.workunit.client.0.smithi022.stdout:1/631: sync 2024-02-16T19:29:09.889 INFO:tasks.workunit.client.0.smithi022.stdout:6/550: dwrite d1/d8/da/d17/f99 [0,4194304] 0 2024-02-16T19:29:09.893 INFO:tasks.workunit.client.1.smithi117.stdout:0/588: creat d0/d9/d14/d1d/d60/d76/d7b/d8d/fd3 x:0 0 0 2024-02-16T19:29:09.895 INFO:tasks.workunit.client.1.smithi117.stdout:0/589: write d0/d9/d22/d27/d31/f38 [862710,82684] 0 2024-02-16T19:29:09.896 INFO:tasks.workunit.client.1.smithi117.stdout:0/590: read - d0/d9/d14/d1d/d1e/d54/fb4 zero size 2024-02-16T19:29:09.898 INFO:tasks.workunit.client.1.smithi117.stdout:0/591: chown d0/d9/d22/l28 73 1 2024-02-16T19:29:09.898 INFO:tasks.workunit.client.0.smithi022.stdout:2/513: link d0/d5d/l92 d0/d2/d18/d1f/d6c/la7 0 2024-02-16T19:29:09.899 INFO:tasks.workunit.client.0.smithi022.stdout:2/514: write d0/d2/d18/f65 [731427,127549] 0 2024-02-16T19:29:09.900 INFO:tasks.workunit.client.0.smithi022.stdout:2/515: stat d0/d2/c93 0 2024-02-16T19:29:09.900 INFO:tasks.workunit.client.1.smithi117.stdout:2/553: dread d0/d3/d34/d94/d46/d68/f8f [0,4194304] 0 2024-02-16T19:29:09.902 INFO:tasks.workunit.client.0.smithi022.stdout:5/539: rename c7 to dd/d1f/d48/d21/d2f/d46/d5e/d84/cbb 0 2024-02-16T19:29:09.904 INFO:tasks.workunit.client.0.smithi022.stdout:0/715: mkdir d10/d11/d27/d3f/d58/d60/dec 0 2024-02-16T19:29:09.907 INFO:tasks.workunit.client.0.smithi022.stdout:3/486: dread - d7/f3b zero size 2024-02-16T19:29:09.908 INFO:tasks.workunit.client.1.smithi117.stdout:7/547: creat d4/d1c/d39/d9f/fca x:0 0 0 2024-02-16T19:29:09.909 INFO:tasks.workunit.client.0.smithi022.stdout:6/551: mknod d1/d8/d81/cc7 0 2024-02-16T19:29:09.910 INFO:tasks.workunit.client.1.smithi117.stdout:3/636: mkdir d1/d1b/d1e/d37/ddc 0 2024-02-16T19:29:09.913 INFO:tasks.workunit.client.0.smithi022.stdout:5/540: rmdir dd/d1f/d48/d21/d2f/d52/d5c/d61 39 2024-02-16T19:29:09.915 INFO:tasks.workunit.client.1.smithi117.stdout:4/535: mkdir d0/da/d21/d3d/d28/d66/da2/dac 0 2024-02-16T19:29:09.916 INFO:tasks.workunit.client.0.smithi022.stdout:1/632: symlink d6/d8/d22/lc3 0 2024-02-16T19:29:09.919 INFO:tasks.workunit.client.1.smithi117.stdout:2/554: creat d0/d13/d4d/d80/fa6 x:0 0 0 2024-02-16T19:29:09.921 INFO:tasks.workunit.client.1.smithi117.stdout:1/498: symlink d2/la7 0 2024-02-16T19:29:09.922 INFO:tasks.workunit.client.1.smithi117.stdout:1/499: write d2/d1d/d1f/f56 [4818573,38892] 0 2024-02-16T19:29:09.922 INFO:tasks.workunit.client.1.smithi117.stdout:1/500: stat d2/d19/d59/d89/d8d 0 2024-02-16T19:29:09.924 INFO:tasks.workunit.client.1.smithi117.stdout:1/501: write d2/d19/d59/d89/d69/d80/d8e/f90 [901823,128272] 0 2024-02-16T19:29:09.926 INFO:tasks.workunit.client.1.smithi117.stdout:7/548: creat d4/d13/d63/d71/fcb x:0 0 0 2024-02-16T19:29:09.928 INFO:tasks.workunit.client.0.smithi022.stdout:5/541: truncate dd/d1f/d48/d21/d2f/d40/d49/f4c 3310997 0 2024-02-16T19:29:09.931 INFO:tasks.workunit.client.0.smithi022.stdout:1/633: rmdir d6/d66 39 2024-02-16T19:29:09.932 INFO:tasks.workunit.client.1.smithi117.stdout:7/549: dwrite d4/d13/d63/d71/f9c [0,4194304] 0 2024-02-16T19:29:09.934 INFO:tasks.workunit.client.1.smithi117.stdout:3/637: rmdir d1/dd/d32 39 2024-02-16T19:29:09.936 INFO:tasks.workunit.client.0.smithi022.stdout:3/487: write d7/d12/d18/dd/d24/d29/fa3 [351389,63707] 0 2024-02-16T19:29:09.936 INFO:tasks.workunit.client.0.smithi022.stdout:6/552: dread d1/d8/d3c/d4d/f4b [0,4194304] 0 2024-02-16T19:29:09.937 INFO:tasks.workunit.client.1.smithi117.stdout:4/536: creat d0/da/d21/d3d/d28/fad x:0 0 0 2024-02-16T19:29:09.937 INFO:tasks.workunit.client.1.smithi117.stdout:7/550: dwrite d4/f5 [0,4194304] 0 2024-02-16T19:29:09.938 INFO:tasks.workunit.client.1.smithi117.stdout:7/551: dread - d4/d85/fb6 zero size 2024-02-16T19:29:09.939 INFO:tasks.workunit.client.1.smithi117.stdout:4/537: dread - d0/da/d21/d3d/d28/d64/d73/d91/fa5 zero size 2024-02-16T19:29:09.940 INFO:tasks.workunit.client.1.smithi117.stdout:0/592: mkdir d0/d9/d22/dd4 0 2024-02-16T19:29:09.940 INFO:tasks.workunit.client.1.smithi117.stdout:0/593: chown d0/d78/ccb 7 1 2024-02-16T19:29:09.942 INFO:tasks.workunit.client.1.smithi117.stdout:0/594: chown d0/d9/d14/d1d/d60/dc3/ld1 1149448306 1 2024-02-16T19:29:09.945 INFO:tasks.workunit.client.1.smithi117.stdout:9/468: dwrite dd/d45/f4f [0,4194304] 0 2024-02-16T19:29:09.947 INFO:tasks.workunit.client.1.smithi117.stdout:5/608: sync 2024-02-16T19:29:09.952 INFO:tasks.workunit.client.0.smithi022.stdout:3/488: mknod d7/d12/d18/dd/d24/cb8 0 2024-02-16T19:29:09.956 INFO:tasks.workunit.client.0.smithi022.stdout:2/516: dwrite d0/d2/d2e/f51 [0,4194304] 0 2024-02-16T19:29:09.959 INFO:tasks.workunit.client.0.smithi022.stdout:5/542: getdents dd/d1f/d48/d21/d2f/d40/d71 0 2024-02-16T19:29:09.960 INFO:tasks.workunit.client.0.smithi022.stdout:5/543: fsync dd/d1f/f31 0 2024-02-16T19:29:09.963 INFO:tasks.workunit.client.1.smithi117.stdout:8/549: sync 2024-02-16T19:29:09.965 INFO:tasks.workunit.client.1.smithi117.stdout:8/550: dread - d4/d7/d6a/d6f/da8/d76/faa zero size 2024-02-16T19:29:09.965 INFO:tasks.workunit.client.0.smithi022.stdout:6/553: mknod d1/d8/dac/cc8 0 2024-02-16T19:29:09.970 INFO:tasks.workunit.client.0.smithi022.stdout:0/716: dwrite d10/d11/d27/d3f/f8a [0,4194304] 0 2024-02-16T19:29:09.972 INFO:tasks.workunit.client.1.smithi117.stdout:1/502: write d2/d4/f40 [691928,55220] 0 2024-02-16T19:29:09.973 INFO:tasks.workunit.client.0.smithi022.stdout:3/489: truncate d7/d12/d18/dd/d3c/d5a/d67/fa8 76672 0 2024-02-16T19:29:09.973 INFO:tasks.workunit.client.1.smithi117.stdout:4/538: creat d0/da/d21/d3d/d28/d64/d73/d91/d37/fae x:0 0 0 2024-02-16T19:29:09.982 INFO:tasks.workunit.client.0.smithi022.stdout:3/490: dwrite d7/d12/d18/dd/d31/d7f/faf [0,4194304] 0 2024-02-16T19:29:09.983 INFO:tasks.workunit.client.0.smithi022.stdout:1/634: dread d6/d8/d22/d3a/f44 [0,4194304] 0 2024-02-16T19:29:09.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.984+0000 7f1362a15700 1 -- 172.21.15.22:0/3275676555 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f135c0feb50 msgr2=0x7f135c100f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:29:09.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.984+0000 7f1362a15700 1 --2- 172.21.15.22:0/3275676555 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f135c0feb50 0x7f135c100f70 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f1344009910 tx=0x7f1344009c20 comp rx=0 tx=0).stop 2024-02-16T19:29:09.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.985+0000 7f1362a15700 1 -- 172.21.15.22:0/3275676555 shutdown_connections 2024-02-16T19:29:09.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.985+0000 7f1362a15700 1 --2- 172.21.15.22:0/3275676555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f135c1014b0 0x7f135c103930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:09.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.985+0000 7f1362a15700 1 --2- 172.21.15.22:0/3275676555 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f135c0feb50 0x7f135c100f70 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:09.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.985+0000 7f1362a15700 1 -- 172.21.15.22:0/3275676555 >> 172.21.15.22:0/3275676555 conn(0x7f135c0fa780 msgr2=0x7f135c0fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:29:09.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.985+0000 7f1362a15700 1 -- 172.21.15.22:0/3275676555 shutdown_connections 2024-02-16T19:29:09.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.986+0000 7f1362a15700 1 -- 172.21.15.22:0/3275676555 wait complete. 2024-02-16T19:29:09.988 INFO:tasks.workunit.client.0.smithi022.stdout:6/554: rename d1/d8/d23/f68 to d1/d8/da/d17/d85/d5b/fc9 0 2024-02-16T19:29:09.988 INFO:tasks.workunit.client.0.smithi022.stdout:3/491: readlink d7/d12/d18/dd/d24/d58/l83 0 2024-02-16T19:29:09.988 INFO:tasks.workunit.client.0.smithi022.stdout:5/544: dwrite f6 [12582912,4194304] 0 2024-02-16T19:29:09.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.987+0000 7f1362a15700 1 Processor -- start 2024-02-16T19:29:09.991 INFO:tasks.workunit.client.0.smithi022.stdout:3/492: fdatasync d7/d12/d18/d4b/f6f 0 2024-02-16T19:29:09.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.989+0000 7f1362a15700 1 -- start start 2024-02-16T19:29:09.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.990+0000 7f1362a15700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f135c0feb50 0x7f135c10c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:29:09.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.990+0000 7f1362a15700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f135c1014b0 0x7f135c10cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:29:09.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.990+0000 7f1362a15700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f135c10d360 con 0x7f135c0feb50 2024-02-16T19:29:09.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.990+0000 7f1362a15700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f135c1a9e60 con 0x7f135c1014b0 2024-02-16T19:29:09.992 INFO:tasks.workunit.client.1.smithi117.stdout:5/609: mkdir d2/d3/d21/d2a/dcb 0 2024-02-16T19:29:09.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.991+0000 7f135bfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f135c0feb50 0x7f135c10c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:29:09.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.991+0000 7f135bfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f135c0feb50 0x7f135c10c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47620/0 (socket says 172.21.15.22:47620) 2024-02-16T19:29:09.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.991+0000 7f135bfff700 1 -- 172.21.15.22:0/3704560028 learned_addr learned my addr 172.21.15.22:0/3704560028 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:29:09.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.991+0000 7f135b7fe700 1 --2- 172.21.15.22:0/3704560028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f135c1014b0 0x7f135c10cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:29:09.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.991+0000 7f135bfff700 1 -- 172.21.15.22:0/3704560028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f135c1014b0 msgr2=0x7f135c10cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:29:09.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.991+0000 7f135bfff700 1 --2- 172.21.15.22:0/3704560028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f135c1014b0 0x7f135c10cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:09.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.991+0000 7f135bfff700 1 -- 172.21.15.22:0/3704560028 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13440095f0 con 0x7f135c0feb50 2024-02-16T19:29:09.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.992+0000 7f135bfff700 1 --2- 172.21.15.22:0/3704560028 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f135c0feb50 0x7f135c10c800 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f134400bda0 tx=0x7f134400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:29:09.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.992+0000 7f13597fa700 1 -- 172.21.15.22:0/3704560028 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f134401d070 con 0x7f135c0feb50 2024-02-16T19:29:09.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.992+0000 7f1362a15700 1 -- 172.21.15.22:0/3704560028 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f135c1aa000 con 0x7f135c0feb50 2024-02-16T19:29:09.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.992+0000 7f1362a15700 1 -- 172.21.15.22:0/3704560028 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f135c1aa4a0 con 0x7f135c0feb50 2024-02-16T19:29:09.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.993+0000 7f13597fa700 1 -- 172.21.15.22:0/3704560028 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f134400f460 con 0x7f135c0feb50 2024-02-16T19:29:09.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.993+0000 7f13597fa700 1 -- 172.21.15.22:0/3704560028 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1344017610 con 0x7f135c0feb50 2024-02-16T19:29:09.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.994+0000 7f13597fa700 1 -- 172.21.15.22:0/3704560028 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 24) v1 ==== 46073+0+0 (secure 0 0 0) 0x7f1344017770 con 0x7f135c0feb50 2024-02-16T19:29:09.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.994+0000 7f1362a15700 1 -- 172.21.15.22:0/3704560028 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f133c005320 con 0x7f135c0feb50 2024-02-16T19:29:09.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.994+0000 7f13597fa700 1 --2- 172.21.15.22:0/3704560028 >> [v2:172.21.15.117:6828/10500008,v1:172.21.15.117:6829/10500008] conn(0x7f1348038f00 0x7f134803b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:29:09.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.994+0000 7f13597fa700 1 -- 172.21.15.22:0/3704560028 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f134404e210 con 0x7f135c0feb50 2024-02-16T19:29:09.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.995+0000 7f135b7fe700 1 --2- 172.21.15.22:0/3704560028 >> [v2:172.21.15.117:6828/10500008,v1:172.21.15.117:6829/10500008] conn(0x7f1348038f00 0x7f134803b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:29:09.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:09.996+0000 7f135b7fe700 1 --2- 172.21.15.22:0/3704560028 >> [v2:172.21.15.117:6828/10500008,v1:172.21.15.117:6829/10500008] conn(0x7f1348038f00 0x7f134803b3b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f134c005950 tx=0x7f134c0058e0 comp rx=0 tx=0).ready entity=mgr.24349 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:29:10.000 INFO:tasks.workunit.client.1.smithi117.stdout:6/542: getdents d5/d4b/d20 0 2024-02-16T19:29:10.001 INFO:tasks.workunit.client.0.smithi022.stdout:6/555: dwrite d1/d8/f40 [0,4194304] 0 2024-02-16T19:29:10.001 INFO:tasks.workunit.client.1.smithi117.stdout:6/543: dread - d5/db/d7d/d32/faf zero size 2024-02-16T19:29:10.003 INFO:tasks.workunit.client.0.smithi022.stdout:6/556: fdatasync d1/d8/d81/fa0 0 2024-02-16T19:29:10.003 INFO:tasks.workunit.client.1.smithi117.stdout:8/551: mknod d4/d7/da6/d3a/d43/d44/d5f/cac 0 2024-02-16T19:29:10.004 INFO:tasks.workunit.client.0.smithi022.stdout:6/557: write d1/d8/d3c/d77/fc3 [708963,18765] 0 2024-02-16T19:29:10.004 INFO:tasks.workunit.client.1.smithi117.stdout:8/552: write d4/d12/f42 [797281,105283] 0 2024-02-16T19:29:10.007 INFO:tasks.workunit.client.1.smithi117.stdout:3/638: unlink d1/d30/d52/d56/c9b 0 2024-02-16T19:29:10.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:10.007+0000 7f13597fa700 1 -- 172.21.15.22:0/3704560028 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f134401fc30 con 0x7f135c0feb50 2024-02-16T19:29:10.015 INFO:tasks.workunit.client.0.smithi022.stdout:5/545: mknod dd/d1f/d48/d21/d5d/cbc 0 2024-02-16T19:29:10.016 INFO:tasks.workunit.client.0.smithi022.stdout:1/635: dread d6/d8/d22/f15 [0,4194304] 0 2024-02-16T19:29:10.016 INFO:tasks.workunit.client.0.smithi022.stdout:0/717: write d10/d15/d23/d8f/dc1/f69 [84514,49692] 0 2024-02-16T19:29:10.018 INFO:tasks.workunit.client.0.smithi022.stdout:5/546: dread - dd/d1f/d48/d21/d2f/d46/d5e/d84/db4/fb9 zero size 2024-02-16T19:29:10.019 INFO:tasks.workunit.client.0.smithi022.stdout:3/493: creat d7/d12/d59/fb9 x:0 0 0 2024-02-16T19:29:10.020 INFO:tasks.workunit.client.1.smithi117.stdout:4/539: dread - d0/f72 zero size 2024-02-16T19:29:10.025 INFO:tasks.workunit.client.1.smithi117.stdout:0/595: fdatasync d0/d9/d22/d27/f42 0 2024-02-16T19:29:10.026 INFO:tasks.workunit.client.0.smithi022.stdout:2/517: link d0/d2/d18/c64 d0/d2/d2e/d4a/ca8 0 2024-02-16T19:29:10.026 INFO:tasks.workunit.client.1.smithi117.stdout:0/596: dread - d0/d9/d22/d27/d31/fb2 zero size 2024-02-16T19:29:10.031 INFO:tasks.workunit.client.1.smithi117.stdout:9/469: truncate dd/d23/f95 290163 0 2024-02-16T19:29:10.033 INFO:tasks.workunit.client.1.smithi117.stdout:0/597: dwrite d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/fc7 [0,4194304] 0 2024-02-16T19:29:10.033 INFO:tasks.workunit.client.0.smithi022.stdout:6/558: mknod d1/d8/da/d17/d53/d7f/da3/cca 0 2024-02-16T19:29:10.036 INFO:tasks.workunit.client.1.smithi117.stdout:5/610: write d2/d4c/d69/f72 [1004185,63181] 0 2024-02-16T19:29:10.036 INFO:tasks.workunit.client.0.smithi022.stdout:0/718: creat d10/d11/d50/d65/fed x:0 0 0 2024-02-16T19:29:10.039 INFO:tasks.workunit.client.1.smithi117.stdout:6/544: write d5/db/f4a [167673,22010] 0 2024-02-16T19:29:10.040 INFO:tasks.workunit.client.1.smithi117.stdout:6/545: fdatasync d5/db/d7d/d32/f92 0 2024-02-16T19:29:10.040 INFO:tasks.workunit.client.0.smithi022.stdout:1/636: truncate d6/d8/d22/d47/f4e 1554993 0 2024-02-16T19:29:10.041 INFO:tasks.workunit.client.1.smithi117.stdout:8/553: chown d4/d7/d6a/d72/d84/c9c 2 1 2024-02-16T19:29:10.041 INFO:tasks.workunit.client.1.smithi117.stdout:6/546: write d5/d4b/d44/d52/d81/d9f/fad [618095,86271] 0 2024-02-16T19:29:10.043 INFO:tasks.workunit.client.0.smithi022.stdout:5/547: symlink dd/d1f/d48/d21/d2f/d40/d4e/d56/lbd 0 2024-02-16T19:29:10.045 INFO:tasks.workunit.client.1.smithi117.stdout:7/552: truncate d4/d1c/f3c 393015 0 2024-02-16T19:29:10.046 INFO:tasks.workunit.client.0.smithi022.stdout:3/494: mknod d7/d12/d18/dd/d3c/d5a/d67/cba 0 2024-02-16T19:29:10.047 INFO:tasks.workunit.client.1.smithi117.stdout:7/553: write d4/d1c/fbe [984000,20597] 0 2024-02-16T19:29:10.047 INFO:tasks.workunit.client.0.smithi022.stdout:2/518: creat d0/d2/d2e/d4a/fa9 x:0 0 0 2024-02-16T19:29:10.049 INFO:tasks.workunit.client.1.smithi117.stdout:6/547: dwrite d5/db/d7d/d32/d95/d72/fa2 [0,4194304] 0 2024-02-16T19:29:10.050 INFO:tasks.workunit.client.0.smithi022.stdout:6/559: unlink d1/d8/d3c/f47 0 2024-02-16T19:29:10.050 INFO:tasks.workunit.client.1.smithi117.stdout:6/548: chown d5/db/l76 312 1 2024-02-16T19:29:10.050 INFO:tasks.workunit.client.1.smithi117.stdout:4/540: stat d0/da/d21/f65 0 2024-02-16T19:29:10.051 INFO:tasks.workunit.client.0.smithi022.stdout:0/719: chown d10/d11/fd5 11 1 2024-02-16T19:29:10.052 INFO:tasks.workunit.client.0.smithi022.stdout:6/560: write d1/d8/da/d17/d53/f7d [1462885,128162] 0 2024-02-16T19:29:10.052 INFO:tasks.workunit.client.1.smithi117.stdout:2/555: getdents d0/d3/d34 0 2024-02-16T19:29:10.053 INFO:tasks.workunit.client.1.smithi117.stdout:2/556: readlink d0/d3/d22/d37/d9d/la2 0 2024-02-16T19:29:10.054 INFO:tasks.workunit.client.1.smithi117.stdout:2/557: write d0/d3b/f5b [4116260,55354] 0 2024-02-16T19:29:10.055 INFO:tasks.workunit.client.0.smithi022.stdout:5/548: mknod dd/d1f/d48/d21/d2f/d46/d5e/d84/db4/cbe 0 2024-02-16T19:29:10.055 INFO:tasks.workunit.client.0.smithi022.stdout:5/549: chown dd/d1f/d48/d21/l33 179912 1 2024-02-16T19:29:10.060 INFO:tasks.workunit.client.1.smithi117.stdout:9/470: rename dd/d2d/f84 to dd/d47/d5b/d5f/d77/d7c/fa2 0 2024-02-16T19:29:10.064 INFO:tasks.workunit.client.0.smithi022.stdout:6/561: fsync d1/d8/d3c/d4d/f45 0 2024-02-16T19:29:10.067 INFO:tasks.workunit.client.1.smithi117.stdout:8/554: creat d4/d1c/fad x:0 0 0 2024-02-16T19:29:10.069 INFO:tasks.workunit.client.0.smithi022.stdout:1/637: rename d6/d66/f71 to d6/d8/d22/d59/fc4 0 2024-02-16T19:29:10.073 INFO:tasks.workunit.client.0.smithi022.stdout:3/495: mknod d7/d12/d18/dd/cbb 0 2024-02-16T19:29:10.074 INFO:tasks.workunit.client.0.smithi022.stdout:3/496: chown d7/d12/f14 5 1 2024-02-16T19:29:10.077 INFO:tasks.workunit.client.0.smithi022.stdout:5/550: creat dd/d1f/d48/d21/d2f/d40/fbf x:0 0 0 2024-02-16T19:29:10.079 INFO:tasks.workunit.client.0.smithi022.stdout:5/551: write dd/d1f/d48/d21/d2f/d46/d5e/d84/d8b/f8e [249130,31822] 0 2024-02-16T19:29:10.079 INFO:tasks.workunit.client.1.smithi117.stdout:1/503: rmdir d2/d19/d59/d89/d8d/da4 0 2024-02-16T19:29:10.081 INFO:tasks.workunit.client.0.smithi022.stdout:0/720: mkdir d10/d7d/dac/de9/dee 0 2024-02-16T19:29:10.082 INFO:tasks.workunit.client.1.smithi117.stdout:6/549: readlink d5/d4b/d44/d52/l61 0 2024-02-16T19:29:10.083 INFO:tasks.workunit.client.0.smithi022.stdout:0/721: truncate d10/d11/d27/d77/f8e 4883204 0 2024-02-16T19:29:10.086 INFO:tasks.workunit.client.0.smithi022.stdout:3/497: dread d7/d12/d18/f1b [0,4194304] 0 2024-02-16T19:29:10.098 INFO:tasks.workunit.client.1.smithi117.stdout:2/558: rmdir d0 39 2024-02-16T19:29:10.101 INFO:tasks.workunit.client.0.smithi022.stdout:5/552: mkdir dd/d1f/d48/d21/d5d/dc0 0 2024-02-16T19:29:10.106 INFO:tasks.workunit.client.1.smithi117.stdout:9/471: unlink dd/d45/d31/c7d 0 2024-02-16T19:29:10.106 INFO:tasks.workunit.client.0.smithi022.stdout:3/498: read d7/d12/d18/dd/d24/f9f [398391,52292] 0 2024-02-16T19:29:10.110 INFO:tasks.workunit.client.1.smithi117.stdout:5/611: mknod d2/d9/d3c/d24/d50/d52/d74/d8c/dac/ccc 0 2024-02-16T19:29:10.114 INFO:tasks.workunit.client.0.smithi022.stdout:6/562: write d1/d8/da/d17/d48/d51/f59 [640026,107345] 0 2024-02-16T19:29:10.115 INFO:tasks.workunit.client.0.smithi022.stdout:1/638: mkdir d6/d8/d22/d3f/dc5 0 2024-02-16T19:29:10.115 INFO:tasks.workunit.client.0.smithi022.stdout:3/499: dwrite d7/d12/d18/dd/d24/d29/fb5 [0,4194304] 0 2024-02-16T19:29:10.125 INFO:tasks.workunit.client.1.smithi117.stdout:4/541: dread d0/da/d21/d3d/d28/d66/d97/d6d/f7e [0,4194304] 0 2024-02-16T19:29:10.126 INFO:tasks.workunit.client.1.smithi117.stdout:2/559: readlink d0/d3/d34/d94/d46/d68/d76/da1/d56/d60/l67 0 2024-02-16T19:29:10.126 INFO:tasks.workunit.client.1.smithi117.stdout:4/542: read d0/da/d21/f3e [563948,60288] 0 2024-02-16T19:29:10.130 INFO:tasks.workunit.client.1.smithi117.stdout:7/554: rename d4/d1c/d39/f5b to d4/d1c/d39/d46/d55/fcc 0 2024-02-16T19:29:10.132 INFO:tasks.workunit.client.1.smithi117.stdout:9/472: chown dd/d23/f18 119222 1 2024-02-16T19:29:10.133 INFO:tasks.workunit.client.0.smithi022.stdout:6/563: truncate d1/d8/da/d17/d85/d56/f90 1278353 0 2024-02-16T19:29:10.134 INFO:tasks.workunit.client.0.smithi022.stdout:6/564: chown d1/d8/d3c/d4d/c4e 11286 1 2024-02-16T19:29:10.137 INFO:tasks.workunit.client.0.smithi022.stdout:6/565: truncate d1/d8/da/d17/d48/d51/f9b 4533784 0 2024-02-16T19:29:10.138 INFO:tasks.workunit.client.0.smithi022.stdout:0/722: link d10/d15/d23/d8f/dc1/c78 d10/d15/d23/dc6/cef 0 2024-02-16T19:29:10.138 INFO:tasks.workunit.client.0.smithi022.stdout:6/566: write d1/d8/f40 [2095899,26712] 0 2024-02-16T19:29:10.139 INFO:tasks.workunit.client.0.smithi022.stdout:0/723: chown fb 8237295 1 2024-02-16T19:29:10.140 INFO:tasks.workunit.client.0.smithi022.stdout:0/724: chown d10/d11/d27/d77/f8e 143849854 1 2024-02-16T19:29:10.144 INFO:tasks.workunit.client.1.smithi117.stdout:6/550: link d5/d4b/d44/d52/f6c d5/d9c/daa/fb2 0 2024-02-16T19:29:10.151 INFO:tasks.workunit.client.1.smithi117.stdout:2/560: rename d0/d3/d22/d37/d9d/la2 to d0/d3/d22/d37/d9d/la7 0 2024-02-16T19:29:10.155 INFO:tasks.workunit.client.0.smithi022.stdout:6/567: dread d1/d8/f11 [0,4194304] 0 2024-02-16T19:29:10.155 INFO:tasks.workunit.client.0.smithi022.stdout:1/639: write d6/d8/d22/d47/d54/f65 [4153988,112105] 0 2024-02-16T19:29:10.157 INFO:tasks.workunit.client.1.smithi117.stdout:3/639: sync 2024-02-16T19:29:10.157 INFO:tasks.workunit.client.1.smithi117.stdout:3/640: chown d1/d30/c39 842139049 1 2024-02-16T19:29:10.158 INFO:tasks.workunit.client.1.smithi117.stdout:7/555: creat d4/d85/fcd x:0 0 0 2024-02-16T19:29:10.159 INFO:tasks.workunit.client.1.smithi117.stdout:3/641: dread - d1/dd/d3e/fd6 zero size 2024-02-16T19:29:10.160 INFO:tasks.workunit.client.0.smithi022.stdout:7/558: sync 2024-02-16T19:29:10.160 INFO:tasks.workunit.client.0.smithi022.stdout:4/521: sync 2024-02-16T19:29:10.160 INFO:tasks.workunit.client.0.smithi022.stdout:9/575: sync 2024-02-16T19:29:10.160 INFO:tasks.workunit.client.1.smithi117.stdout:2/561: dread d0/d3/d34/d94/d46/d68/d76/da1/d2f/f44 [0,4194304] 0 2024-02-16T19:29:10.161 INFO:tasks.workunit.client.1.smithi117.stdout:2/562: chown d0/d3/d34/d94/d46/d68 175949 1 2024-02-16T19:29:10.161 INFO:tasks.workunit.client.0.smithi022.stdout:0/725: mkdir d10/d11/d27/d3f/df0 0 2024-02-16T19:29:10.161 INFO:tasks.workunit.client.1.smithi117.stdout:2/563: readlink d0/d13/d4d/d80/l9a 0 2024-02-16T19:29:10.163 INFO:tasks.workunit.client.0.smithi022.stdout:6/568: dread d1/d8/d3c/d4d/f4b [0,4194304] 0 2024-02-16T19:29:10.165 INFO:tasks.workunit.client.1.smithi117.stdout:8/555: truncate d4/d12/d2a/f2b 11156404 0 2024-02-16T19:29:10.165 INFO:tasks.workunit.client.0.smithi022.stdout:3/500: write f5 [322999,44256] 0 2024-02-16T19:29:10.166 INFO:tasks.workunit.client.0.smithi022.stdout:7/559: write d9/da/fab [1988716,124600] 0 2024-02-16T19:29:10.167 INFO:tasks.workunit.client.1.smithi117.stdout:1/504: dwrite d2/d1d/f3a [0,4194304] 0 2024-02-16T19:29:10.168 INFO:tasks.workunit.client.0.smithi022.stdout:5/553: dwrite dd/d1f/d48/d21/d2f/d52/d77/d95/fb8 [0,4194304] 0 2024-02-16T19:29:10.170 INFO:tasks.workunit.client.1.smithi117.stdout:2/564: dread d0/d3/d10/f16 [0,4194304] 0 2024-02-16T19:29:10.174 INFO:tasks.workunit.client.1.smithi117.stdout:6/551: mkdir d5/d4b/d20/db3 0 2024-02-16T19:29:10.176 INFO:tasks.workunit.client.0.smithi022.stdout:1/640: mknod d6/d8/cc6 0 2024-02-16T19:29:10.178 INFO:tasks.workunit.client.1.smithi117.stdout:9/473: dread dd/d23/f3b [0,4194304] 0 2024-02-16T19:29:10.179 INFO:tasks.workunit.client.1.smithi117.stdout:9/474: stat dd/d47/d48/c6c 0 2024-02-16T19:29:10.180 INFO:tasks.workunit.client.1.smithi117.stdout:9/475: write dd/d47/f79 [738569,77182] 0 2024-02-16T19:29:10.180 INFO:tasks.workunit.client.1.smithi117.stdout:5/612: write d2/d4c/d4f/d67/fc5 [3797257,96511] 0 2024-02-16T19:29:10.181 INFO:tasks.workunit.client.0.smithi022.stdout:3/501: fdatasync d7/d21/f7c 0 2024-02-16T19:29:10.182 INFO:tasks.workunit.client.0.smithi022.stdout:3/502: fdatasync f5 0 2024-02-16T19:29:10.185 INFO:tasks.workunit.client.0.smithi022.stdout:6/569: fdatasync d1/d8/da/d17/d53/f58 0 2024-02-16T19:29:10.186 INFO:tasks.workunit.client.1.smithi117.stdout:3/642: mknod d1/d1b/d1e/d37/cdd 0 2024-02-16T19:29:10.187 INFO:tasks.workunit.client.0.smithi022.stdout:7/560: dread - d9/d1b/d23/f96 zero size 2024-02-16T19:29:10.188 INFO:tasks.workunit.client.0.smithi022.stdout:5/554: truncate dd/f1d 3136363 0 2024-02-16T19:29:10.189 INFO:tasks.workunit.client.1.smithi117.stdout:3/643: dread d1/fdb [0,4194304] 0 2024-02-16T19:29:10.190 INFO:tasks.workunit.client.1.smithi117.stdout:3/644: fsync d1/d30/f90 0 2024-02-16T19:29:10.191 INFO:tasks.workunit.client.1.smithi117.stdout:0/598: sync 2024-02-16T19:29:10.194 INFO:tasks.workunit.client.0.smithi022.stdout:9/576: mknod db/d14/d19/d1a/cc7 0 2024-02-16T19:29:10.196 INFO:tasks.workunit.client.0.smithi022.stdout:9/577: write db/d14/d16/d49/f9f [707113,83823] 0 2024-02-16T19:29:10.196 INFO:tasks.workunit.client.1.smithi117.stdout:1/505: symlink d2/d19/d59/d89/la8 0 2024-02-16T19:29:10.204 INFO:tasks.workunit.client.0.smithi022.stdout:0/726: link d10/d22/f20 d10/d11/d27/d3f/df0/ff1 0 2024-02-16T19:29:10.206 INFO:tasks.workunit.client.1.smithi117.stdout:6/552: unlink d5/f83 0 2024-02-16T19:29:10.207 INFO:tasks.workunit.client.1.smithi117.stdout:9/476: symlink dd/d45/d31/la3 0 2024-02-16T19:29:10.208 INFO:tasks.workunit.client.0.smithi022.stdout:6/570: creat d1/d8/da/d17/d53/d7f/da3/fcb x:0 0 0 2024-02-16T19:29:10.208 INFO:tasks.workunit.client.1.smithi117.stdout:9/477: dread dd/d45/d62/f4e [0,4194304] 0 2024-02-16T19:29:10.212 INFO:tasks.workunit.client.1.smithi117.stdout:4/543: rename d0/da/d21/d3d/d28/l32 to d0/da/d21/d3d/laf 0 2024-02-16T19:29:10.213 INFO:tasks.workunit.client.1.smithi117.stdout:4/544: fdatasync d0/da/d21/d3d/d28/d66/d97/fa7 0 2024-02-16T19:29:10.214 INFO:tasks.workunit.client.0.smithi022.stdout:6/571: dwrite d1/d8/da/d17/d48/d51/f59 [0,4194304] 0 2024-02-16T19:29:10.214 INFO:tasks.workunit.client.0.smithi022.stdout:5/555: fsync dd/d1f/d48/d21/d2f/d52/d5c/d61/d8f/f94 0 2024-02-16T19:29:10.216 INFO:tasks.workunit.client.0.smithi022.stdout:1/641: mkdir d6/d8/dba/dc7 0 2024-02-16T19:29:10.217 INFO:tasks.workunit.client.0.smithi022.stdout:6/572: chown d1/d8/d3c/d4d/f7e 42434902 1 2024-02-16T19:29:10.217 INFO:tasks.workunit.client.0.smithi022.stdout:7/561: dread d9/da/f62 [0,4194304] 0 2024-02-16T19:29:10.223 INFO:tasks.workunit.client.1.smithi117.stdout:8/556: creat d4/d7/da6/d3a/d65/d6c/da9/fae x:0 0 0 2024-02-16T19:29:10.224 INFO:tasks.workunit.client.0.smithi022.stdout:2/519: sync 2024-02-16T19:29:10.224 INFO:tasks.workunit.client.0.smithi022.stdout:8/555: sync 2024-02-16T19:29:10.227 INFO:tasks.workunit.client.1.smithi117.stdout:8/557: dread d4/d73/f82 [0,4194304] 0 2024-02-16T19:29:10.229 INFO:tasks.workunit.client.1.smithi117.stdout:8/558: dread d4/d7/d6a/d6f/da8/f87 [0,4194304] 0 2024-02-16T19:29:10.231 INFO:tasks.workunit.client.1.smithi117.stdout:2/565: symlink d0/la8 0 2024-02-16T19:29:10.232 INFO:tasks.workunit.client.0.smithi022.stdout:2/520: dwrite d0/d2/d18/d1f/f4d [4194304,4194304] 0 2024-02-16T19:29:10.234 INFO:tasks.workunit.client.0.smithi022.stdout:4/522: link d4/d6/d14/d64/d4e/d54/d5a/d63/l65 d4/d6/dd/d22/d30/la1 0 2024-02-16T19:29:10.236 INFO:tasks.workunit.client.0.smithi022.stdout:4/523: fsync d4/d6/d14/d64/d66/d6d/f7d 0 2024-02-16T19:29:10.237 INFO:tasks.workunit.client.1.smithi117.stdout:3/645: dwrite d1/dd/d3b/f3d [0,4194304] 0 2024-02-16T19:29:10.237 INFO:tasks.workunit.client.1.smithi117.stdout:5/613: symlink d2/d9/dba/lcd 0 2024-02-16T19:29:10.238 INFO:tasks.workunit.client.1.smithi117.stdout:3/646: chown d1/d1b/d1e/d29/c2e 329738859 1 2024-02-16T19:29:10.238 INFO:tasks.workunit.client.0.smithi022.stdout:4/524: write d4/d6/d14/d64/d66/f82 [358950,21333] 0 2024-02-16T19:29:10.240 INFO:tasks.workunit.client.0.smithi022.stdout:9/578: mkdir db/d14/dc8 0 2024-02-16T19:29:10.241 INFO:tasks.workunit.client.1.smithi117.stdout:7/556: creat d4/d13/d63/fce x:0 0 0 2024-02-16T19:29:10.242 INFO:tasks.workunit.client.0.smithi022.stdout:5/556: dread dd/d1f/d48/f74 [0,4194304] 0 2024-02-16T19:29:10.244 INFO:tasks.workunit.client.0.smithi022.stdout:6/573: symlink d1/d8/da/d17/d85/d5b/lcc 0 2024-02-16T19:29:10.246 INFO:tasks.workunit.client.1.smithi117.stdout:9/478: dwrite dd/d47/d48/f98 [0,4194304] 0 2024-02-16T19:29:10.247 INFO:tasks.workunit.client.1.smithi117.stdout:9/479: read - dd/d45/d73/f82 zero size 2024-02-16T19:29:10.248 INFO:tasks.workunit.client.1.smithi117.stdout:7/557: dwrite d4/f5 [0,4194304] 0 2024-02-16T19:29:10.255 INFO:tasks.workunit.client.1.smithi117.stdout:8/559: unlink d4/d12/f26 0 2024-02-16T19:29:10.256 INFO:tasks.workunit.client.0.smithi022.stdout:2/521: truncate d0/f20 1298372 0 2024-02-16T19:29:10.260 INFO:tasks.workunit.client.1.smithi117.stdout:4/545: dread d0/da/f33 [4194304,4194304] 0 2024-02-16T19:29:10.263 INFO:tasks.workunit.client.0.smithi022.stdout:2/522: dread d0/d2/d18/d19/f2f [4194304,4194304] 0 2024-02-16T19:29:10.263 INFO:tasks.workunit.client.0.smithi022.stdout:4/525: symlink d4/d6/d14/d64/d66/la2 0 2024-02-16T19:29:10.263 INFO:tasks.workunit.client.1.smithi117.stdout:3/647: mknod d1/d30/d9f/dbc/cde 0 2024-02-16T19:29:10.264 INFO:tasks.workunit.client.0.smithi022.stdout:2/523: readlink d0/d2/d18/d38/l4b 0 2024-02-16T19:29:10.266 INFO:tasks.workunit.client.0.smithi022.stdout:3/503: getdents d7 0 2024-02-16T19:29:10.273 INFO:tasks.workunit.client.0.smithi022.stdout:9/579: dwrite db/d14/d16/d27/d55/fa6 [0,4194304] 0 2024-02-16T19:29:10.276 INFO:tasks.workunit.client.0.smithi022.stdout:8/556: write d3/d25/d33/f37 [836561,32996] 0 2024-02-16T19:29:10.277 INFO:tasks.workunit.client.0.smithi022.stdout:8/557: dread - d3/d12/f9c zero size 2024-02-16T19:29:10.279 INFO:tasks.workunit.client.1.smithi117.stdout:7/558: creat d4/d1c/d87/d89/d60/d9b/fcf x:0 0 0 2024-02-16T19:29:10.283 INFO:tasks.workunit.client.1.smithi117.stdout:1/506: truncate d2/d19/d59/d89/f42 1417067 0 2024-02-16T19:29:10.284 INFO:tasks.workunit.client.0.smithi022.stdout:9/580: dwrite db/d14/d16/d87/f8f [0,4194304] 0 2024-02-16T19:29:10.288 INFO:tasks.workunit.client.0.smithi022.stdout:9/581: chown db/d14/d16/d5d/f7e 37096 1 2024-02-16T19:29:10.289 INFO:tasks.workunit.client.0.smithi022.stdout:7/562: truncate d9/faf 733592 0 2024-02-16T19:29:10.290 INFO:tasks.workunit.client.0.smithi022.stdout:8/558: dwrite d3/d12/fb7 [0,4194304] 0 2024-02-16T19:29:10.295 INFO:tasks.workunit.client.0.smithi022.stdout:4/526: stat d4/d6/dd/df/c80 0 2024-02-16T19:29:10.296 INFO:tasks.workunit.client.0.smithi022.stdout:1/642: dwrite d6/d8/d22/d59/fc4 [0,4194304] 0 2024-02-16T19:29:10.297 INFO:tasks.workunit.client.1.smithi117.stdout:6/553: mknod d5/d4b/d20/db3/cb4 0 2024-02-16T19:29:10.298 INFO:tasks.workunit.client.0.smithi022.stdout:4/527: write d4/d6/d14/d64/d4e/f8b [3360063,96653] 0 2024-02-16T19:29:10.301 INFO:tasks.workunit.client.0.smithi022.stdout:2/524: symlink d0/d2/d2e/laa 0 2024-02-16T19:29:10.303 INFO:tasks.workunit.client.1.smithi117.stdout:5/614: mknod d2/d3/d21/d2a/dcb/cce 0 2024-02-16T19:29:10.306 INFO:tasks.workunit.client.1.smithi117.stdout:8/560: write d4/d7/da6/d3a/d43/d44/f62 [1104145,61171] 0 2024-02-16T19:29:10.307 INFO:tasks.workunit.client.1.smithi117.stdout:4/546: readlink d0/da/d21/d3d/d28/d64/d73/d91/l45 0 2024-02-16T19:29:10.309 INFO:tasks.workunit.client.1.smithi117.stdout:3/648: mknod d1/dd/d5c/d85/cdf 0 2024-02-16T19:29:10.309 INFO:tasks.workunit.client.0.smithi022.stdout:0/727: getdents d10/d11/d27/d3f/d58/d60/dbc 0 2024-02-16T19:29:10.312 INFO:tasks.workunit.client.1.smithi117.stdout:0/599: link d0/d9/d22/d27/d4e/l7d d0/d9/d22/ld5 0 2024-02-16T19:29:10.313 INFO:tasks.workunit.client.1.smithi117.stdout:7/559: creat d4/d1c/d39/d9f/fd0 x:0 0 0 2024-02-16T19:29:10.317 INFO:tasks.workunit.client.0.smithi022.stdout:5/557: write dd/d1f/d48/d21/d2f/d46/d5e/d84/d8b/f9a [848350,61720] 0 2024-02-16T19:29:10.322 INFO:tasks.workunit.client.1.smithi117.stdout:5/615: unlink d2/d9/d3c/d24/d50/d96/dbc/d30/f3e 0 2024-02-16T19:29:10.328 INFO:tasks.workunit.client.0.smithi022.stdout:2/525: creat d0/d2/d18/d19/fab x:0 0 0 2024-02-16T19:29:10.329 INFO:tasks.workunit.client.1.smithi117.stdout:6/554: dwrite d5/d4b/f1e [0,4194304] 0 2024-02-16T19:29:10.329 INFO:tasks.workunit.client.1.smithi117.stdout:3/649: mknod d1/d1b/d20/ce0 0 2024-02-16T19:29:10.332 INFO:tasks.workunit.client.0.smithi022.stdout:6/574: rename d1/l1b to d1/d8/da/d17/d85/lcd 0 2024-02-16T19:29:10.334 INFO:tasks.workunit.client.0.smithi022.stdout:0/728: mknod d10/d11/d50/cf2 0 2024-02-16T19:29:10.335 INFO:tasks.workunit.client.1.smithi117.stdout:2/566: getdents d0/d13/d4d/d53 0 2024-02-16T19:29:10.335 INFO:tasks.workunit.client.0.smithi022.stdout:0/729: chown d10/d11/d79/d8b/l90 453608018 1 2024-02-16T19:29:10.336 INFO:tasks.workunit.client.0.smithi022.stdout:9/582: mkdir db/d14/dc8/dc9 0 2024-02-16T19:29:10.337 INFO:tasks.workunit.client.0.smithi022.stdout:9/583: write db/d14/d16/d5d/d65/d6b/faf [34910,2601] 0 2024-02-16T19:29:10.338 INFO:tasks.workunit.client.0.smithi022.stdout:9/584: stat db/d14/d16/d27/d55/cae 0 2024-02-16T19:29:10.340 INFO:tasks.workunit.client.0.smithi022.stdout:9/585: chown db/d14/d16/d27/fa3 0 1 2024-02-16T19:29:10.341 INFO:tasks.workunit.client.0.smithi022.stdout:9/586: chown db/d14/d16/fbf 2205742 1 2024-02-16T19:29:10.341 INFO:tasks.workunit.client.0.smithi022.stdout:9/587: chown db/d14/d33/fa7 111074 1 2024-02-16T19:29:10.344 INFO:tasks.workunit.client.1.smithi117.stdout:0/600: dwrite d0/d9/d22/d27/d4e/f61 [0,4194304] 0 2024-02-16T19:29:10.344 INFO:tasks.workunit.client.0.smithi022.stdout:1/643: fsync d6/d8/d22/d59/fc0 0 2024-02-16T19:29:10.347 INFO:tasks.workunit.client.1.smithi117.stdout:4/547: creat d0/da/d21/d3d/d28/d64/d90/fb0 x:0 0 0 2024-02-16T19:29:10.348 INFO:tasks.workunit.client.0.smithi022.stdout:4/528: symlink d4/d6/d14/d64/la3 0 2024-02-16T19:29:10.351 INFO:tasks.workunit.client.1.smithi117.stdout:8/561: rename d4/d7/da6/d3a/f61 to d4/d7/da6/d3a/d43/d44/faf 0 2024-02-16T19:29:10.352 INFO:tasks.workunit.client.1.smithi117.stdout:4/548: dread d0/da/d21/d3d/d28/d66/d97/d6d/f93 [0,4194304] 0 2024-02-16T19:29:10.355 INFO:tasks.workunit.client.0.smithi022.stdout:2/526: unlink d0/d2/d18/d19/d40/d55/d59/f74 0 2024-02-16T19:29:10.357 INFO:tasks.workunit.client.0.smithi022.stdout:7/563: dwrite d9/da/d3b/f41 [0,4194304] 0 2024-02-16T19:29:10.358 INFO:tasks.workunit.client.0.smithi022.stdout:3/504: rename d7/d12/d18/dac/l6d to d7/d12/d18/d4b/lbc 0 2024-02-16T19:29:10.359 INFO:tasks.workunit.client.1.smithi117.stdout:9/480: sync 2024-02-16T19:29:10.362 INFO:tasks.workunit.client.0.smithi022.stdout:7/564: write d9/d1b/fb0 [5352,125822] 0 2024-02-16T19:29:10.363 INFO:tasks.workunit.client.0.smithi022.stdout:3/505: fdatasync d7/d12/d59/f60 0 2024-02-16T19:29:10.364 INFO:tasks.workunit.client.1.smithi117.stdout:7/560: mknod d4/cd1 0 2024-02-16T19:29:10.365 INFO:tasks.workunit.client.1.smithi117.stdout:5/616: dwrite d2/d9/f18 [4194304,4194304] 0 2024-02-16T19:29:10.367 INFO:tasks.workunit.client.1.smithi117.stdout:9/481: dread dd/d45/d73/f94 [0,4194304] 0 2024-02-16T19:29:10.368 INFO:tasks.workunit.client.1.smithi117.stdout:2/567: truncate d0/d3b/f51 3469579 0 2024-02-16T19:29:10.369 INFO:tasks.workunit.client.0.smithi022.stdout:6/575: truncate d1/d8/d3c/d77/f80 699261 0 2024-02-16T19:29:10.371 INFO:tasks.workunit.client.1.smithi117.stdout:2/568: write d0/d13/d4d/d53/fa4 [553667,123836] 0 2024-02-16T19:29:10.371 INFO:tasks.workunit.client.1.smithi117.stdout:0/601: dread - d0/d9/d22/d27/d4e/fa5 zero size 2024-02-16T19:29:10.372 INFO:tasks.workunit.client.0.smithi022.stdout:8/559: dwrite d3/d12/d2b/f3f [0,4194304] 0 2024-02-16T19:29:10.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:10.372+0000 7f1362a15700 1 -- 172.21.15.22:0/3704560028 --> [v2:172.21.15.117:6828/10500008,v1:172.21.15.117:6829/10500008] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f133c000bf0 con 0x7f1348038f00 2024-02-16T19:29:10.373 INFO:tasks.workunit.client.0.smithi022.stdout:8/560: dread - d3/d12/d92/fc5 zero size 2024-02-16T19:29:10.375 INFO:tasks.workunit.client.0.smithi022.stdout:1/644: creat d6/d8/d22/d16/d17/d5d/fc8 x:0 0 0 2024-02-16T19:29:10.375 INFO:tasks.workunit.client.0.smithi022.stdout:9/588: chown db/f7b 0 1 2024-02-16T19:29:10.377 INFO:tasks.workunit.client.0.smithi022.stdout:2/527: creat d0/d32/d3b/d6f/fac x:0 0 0 2024-02-16T19:29:10.378 INFO:tasks.workunit.client.0.smithi022.stdout:5/558: rename dd/d1f/d4a/c9b to dd/d1f/d48/d21/d64/cc1 0 2024-02-16T19:29:10.378 INFO:tasks.workunit.client.0.smithi022.stdout:4/529: dread d4/d6/dd/d22/f51 [0,4194304] 0 2024-02-16T19:29:10.380 INFO:tasks.workunit.client.1.smithi117.stdout:3/650: mkdir d1/d31/d89/de1 0 2024-02-16T19:29:10.381 INFO:tasks.workunit.client.0.smithi022.stdout:6/576: dread d1/d8/d3c/d77/fc3 [0,4194304] 0 2024-02-16T19:29:10.382 INFO:tasks.workunit.client.1.smithi117.stdout:3/651: readlink d1/d1b/d1e/l1a 0 2024-02-16T19:29:10.383 INFO:tasks.workunit.client.0.smithi022.stdout:2/528: fsync d0/d2/d18/d1f/f21 0 2024-02-16T19:29:10.384 INFO:tasks.workunit.client.1.smithi117.stdout:3/652: write d1/d30/d9f/dbc/fc5 [4733251,87908] 0 2024-02-16T19:29:10.390 INFO:tasks.workunit.client.0.smithi022.stdout:0/730: dwrite d10/d15/d23/fdf [4194304,4194304] 0 2024-02-16T19:29:10.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:10.391+0000 7f13597fa700 1 -- 172.21.15.22:0/3704560028 <== mgr.24349 v2:172.21.15.117:6828/10500008 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f133c000bf0 con 0x7f1348038f00 2024-02-16T19:29:10.393 INFO:teuthology.orchestra.run.smithi022.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T19:29:10.393 INFO:teuthology.orchestra.run.smithi022.stdout:alertmanager.smithi022 smithi022 *:9093,9094 running (6m) 2m ago 8m 27.3M - 0.25.0 c8568f914cd2 4dac585d8372 2024-02-16T19:29:10.393 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-exporter.smithi022 smithi022 running (8m) 2m ago 8m 7428k - 18.2.0 dc2bc1663786 3f8bef4349ba 2024-02-16T19:29:10.393 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-exporter.smithi117 smithi117 running (7m) 25s ago 7m 7314k - 18.2.0 dc2bc1663786 59b65a7e309f 2024-02-16T19:29:10.393 INFO:teuthology.orchestra.run.smithi022.stdout:crash.smithi022 smithi022 running (8m) 2m ago 8m 6669k - 18.2.0 dc2bc1663786 efef476aa486 2024-02-16T19:29:10.393 INFO:teuthology.orchestra.run.smithi022.stdout:crash.smithi117 smithi117 running (7m) 25s ago 7m 6660k - 18.2.0 dc2bc1663786 5466453eb149 2024-02-16T19:29:10.393 INFO:teuthology.orchestra.run.smithi022.stdout:grafana.smithi022 smithi022 *:3000 running (6m) 2m ago 8m 84.6M - 9.4.7 954c08fa6188 a48872ecbbf2 2024-02-16T19:29:10.393 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi022.btppwm smithi022 running (2m) 2m ago 2m 14.9M - 18.2.0 dc2bc1663786 22971fb3a907 2024-02-16T19:29:10.393 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi022.ocukvs smithi022 running (2m) 2m ago 2m 11.5M - 18.2.0 dc2bc1663786 81ee7bd5fd39 2024-02-16T19:29:10.393 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi117.ibmppn smithi117 running (2m) 25s ago 2m 14.2M - 18.2.0 dc2bc1663786 8c606cbeeb86 2024-02-16T19:29:10.393 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi117.orzors smithi117 running (2m) 25s ago 2m 13.2M - 18.2.0 dc2bc1663786 7d2bc4df3feb 2024-02-16T19:29:10.393 INFO:teuthology.orchestra.run.smithi022.stdout:mgr.smithi022.gmdcch smithi022 *:9283,8765,8443 running (9m) 2m ago 9m 487M - 18.2.0 dc2bc1663786 e8435b394d04 2024-02-16T19:29:10.393 INFO:teuthology.orchestra.run.smithi022.stdout:mgr.smithi117.cohbtv smithi117 *:8443,9283,8765 running (34s) 25s ago 7m 51.8M - 19.0.0-1450-gf78a58c0 f081f847070a 688882bc227b 2024-02-16T19:29:10.394 INFO:teuthology.orchestra.run.smithi022.stdout:mon.smithi022 smithi022 running (9m) 2m ago 9m 45.2M 2048M 18.2.0 dc2bc1663786 b4ffc3032e12 2024-02-16T19:29:10.394 INFO:teuthology.orchestra.run.smithi022.stdout:mon.smithi117 smithi117 running (7m) 25s ago 7m 36.1M 2048M 18.2.0 dc2bc1663786 aff3e21a73a0 2024-02-16T19:29:10.394 INFO:teuthology.orchestra.run.smithi022.stdout:node-exporter.smithi022 smithi022 *:9100 running (8m) 2m ago 8m 16.0M - 1.5.0 0da6a335fe13 d727f51df007 2024-02-16T19:29:10.394 INFO:teuthology.orchestra.run.smithi022.stdout:node-exporter.smithi117 smithi117 *:9100 running (7m) 25s ago 7m 16.7M - 1.5.0 0da6a335fe13 fc8b2df5a29d 2024-02-16T19:29:10.394 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0 smithi022 running (6m) 2m ago 6m 47.4M 3948M 18.2.0 dc2bc1663786 0027f86f16ce 2024-02-16T19:29:10.394 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1 smithi022 running (6m) 2m ago 6m 48.5M 3948M 18.2.0 dc2bc1663786 87300382b8a7 2024-02-16T19:29:10.394 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2 smithi022 running (5m) 2m ago 5m 46.7M 3948M 18.2.0 dc2bc1663786 be8a631b4f93 2024-02-16T19:29:10.394 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3 smithi117 running (5m) 25s ago 5m 94.8M 4970M 18.2.0 dc2bc1663786 9c0ce9a981cf 2024-02-16T19:29:10.394 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4 smithi117 running (4m) 25s ago 4m 88.5M 4970M 18.2.0 dc2bc1663786 1779c3496f4d 2024-02-16T19:29:10.394 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5 smithi117 running (4m) 25s ago 4m 81.4M 4970M 18.2.0 dc2bc1663786 fefaabca0638 2024-02-16T19:29:10.394 INFO:teuthology.orchestra.run.smithi022.stdout:prometheus.smithi022 smithi022 *:9095 running (6m) 2m ago 7m 60.2M - 2.43.0 a07b618ecd1d a1a5b92b5ef7 2024-02-16T19:29:10.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:10.395+0000 7f1362a15700 1 -- 172.21.15.22:0/3704560028 >> [v2:172.21.15.117:6828/10500008,v1:172.21.15.117:6829/10500008] conn(0x7f1348038f00 msgr2=0x7f134803b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:29:10.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:10.395+0000 7f1362a15700 1 --2- 172.21.15.22:0/3704560028 >> [v2:172.21.15.117:6828/10500008,v1:172.21.15.117:6829/10500008] conn(0x7f1348038f00 0x7f134803b3b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f134c005950 tx=0x7f134c0058e0 comp rx=0 tx=0).stop 2024-02-16T19:29:10.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:10.395+0000 7f1362a15700 1 -- 172.21.15.22:0/3704560028 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f135c0feb50 msgr2=0x7f135c10c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:29:10.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:10.395+0000 7f1362a15700 1 --2- 172.21.15.22:0/3704560028 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f135c0feb50 0x7f135c10c800 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f134400bda0 tx=0x7f134400be80 comp rx=0 tx=0).stop 2024-02-16T19:29:10.397 INFO:tasks.workunit.client.0.smithi022.stdout:2/529: dread d0/d2/d18/d38/d6b/f8b [0,4194304] 0 2024-02-16T19:29:10.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:10.395+0000 7f1362a15700 1 -- 172.21.15.22:0/3704560028 shutdown_connections 2024-02-16T19:29:10.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:10.395+0000 7f1362a15700 1 --2- 172.21.15.22:0/3704560028 >> [v2:172.21.15.117:6828/10500008,v1:172.21.15.117:6829/10500008] conn(0x7f1348038f00 0x7f134803b3b0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:10.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:10.395+0000 7f1362a15700 1 --2- 172.21.15.22:0/3704560028 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f135c0feb50 0x7f135c10c800 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:10.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:10.395+0000 7f1362a15700 1 --2- 172.21.15.22:0/3704560028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f135c1014b0 0x7f135c10cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:10.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:10.395+0000 7f1362a15700 1 -- 172.21.15.22:0/3704560028 >> 172.21.15.22:0/3704560028 conn(0x7f135c0fa780 msgr2=0x7f135c0fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:29:10.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:10.395+0000 7f1362a15700 1 -- 172.21.15.22:0/3704560028 shutdown_connections 2024-02-16T19:29:10.398 INFO:tasks.workunit.client.0.smithi022.stdout:9/589: dwrite db/d14/d16/d87/fa9 [4194304,4194304] 0 2024-02-16T19:29:10.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:10.396+0000 7f1362a15700 1 -- 172.21.15.22:0/3704560028 wait complete. 2024-02-16T19:29:10.405 INFO:tasks.workunit.client.0.smithi022.stdout:2/530: read d0/d2/f57 [2026408,10944] 0 2024-02-16T19:29:10.406 INFO:tasks.workunit.client.1.smithi117.stdout:2/569: mkdir d0/d3/d34/d94/d46/d68/d93/da9 0 2024-02-16T19:29:10.408 INFO:tasks.workunit.client.1.smithi117.stdout:8/562: mknod d4/d12/d2a/da7/cb0 0 2024-02-16T19:29:10.413 INFO:tasks.workunit.client.1.smithi117.stdout:0/602: read d0/d9/d22/d27/d31/d6d/fa2 [406780,75274] 0 2024-02-16T19:29:10.414 INFO:tasks.workunit.client.1.smithi117.stdout:7/561: dwrite d4/d1c/d39/d46/f62 [0,4194304] 0 2024-02-16T19:29:10.416 INFO:tasks.workunit.client.1.smithi117.stdout:4/549: creat d0/da/d21/d3d/d28/d64/d8a/fb1 x:0 0 0 2024-02-16T19:29:10.416 INFO:tasks.workunit.client.1.smithi117.stdout:7/562: readlink d4/d13/d63/d71/l7d 0 2024-02-16T19:29:10.416 INFO:tasks.workunit.client.0.smithi022.stdout:1/645: dread d6/d8/d22/d3a/d3e/f76 [0,4194304] 0 2024-02-16T19:29:10.422 INFO:tasks.workunit.client.1.smithi117.stdout:0/603: dwrite d0/d30/fd2 [0,4194304] 0 2024-02-16T19:29:10.423 INFO:tasks.workunit.client.1.smithi117.stdout:1/507: read d2/d19/d59/d89/f42 [573545,46614] 0 2024-02-16T19:29:10.423 INFO:tasks.workunit.client.1.smithi117.stdout:0/604: chown d0/f5 1 1 2024-02-16T19:29:10.425 INFO:tasks.workunit.client.1.smithi117.stdout:7/563: dwrite d4/d1c/d39/d46/d6e/fb9 [0,4194304] 0 2024-02-16T19:29:10.427 INFO:tasks.workunit.client.1.smithi117.stdout:5/617: symlink d2/d5d/d8e/dc1/lcf 0 2024-02-16T19:29:10.434 INFO:tasks.workunit.client.1.smithi117.stdout:2/570: mknod d0/d3b/caa 0 2024-02-16T19:29:10.435 INFO:tasks.workunit.client.1.smithi117.stdout:5/618: dwrite d2/d9/d3c/f8d [0,4194304] 0 2024-02-16T19:29:10.442 INFO:tasks.workunit.client.1.smithi117.stdout:5/619: dwrite d2/d5d/fa9 [0,4194304] 0 2024-02-16T19:29:10.460 INFO:tasks.workunit.client.1.smithi117.stdout:0/605: symlink d0/d9/d14/d1d/d60/d76/d7b/ld6 0 2024-02-16T19:29:10.463 INFO:tasks.workunit.client.1.smithi117.stdout:7/564: symlink d4/d1c/d87/d89/d60/ld2 0 2024-02-16T19:29:10.466 INFO:tasks.workunit.client.0.smithi022.stdout:7/565: creat d9/da/d2c/d88/fb2 x:0 0 0 2024-02-16T19:29:10.466 INFO:tasks.workunit.client.1.smithi117.stdout:9/482: link dd/d23/d64/f9f dd/d47/d5b/fa4 0 2024-02-16T19:29:10.467 INFO:tasks.workunit.client.0.smithi022.stdout:7/566: chown d9/da/d1f/l38 330725 1 2024-02-16T19:29:10.470 INFO:tasks.workunit.client.1.smithi117.stdout:9/483: read dd/d2d/f35 [2146644,126429] 0 2024-02-16T19:29:10.480 INFO:tasks.workunit.client.1.smithi117.stdout:1/508: dwrite d2/d19/d59/d89/f2a [0,4194304] 0 2024-02-16T19:29:10.480 INFO:tasks.workunit.client.1.smithi117.stdout:5/620: creat d2/d4c/d4f/fd0 x:0 0 0 2024-02-16T19:29:10.485 INFO:tasks.workunit.client.1.smithi117.stdout:3/653: creat d1/d1b/fe2 x:0 0 0 2024-02-16T19:29:10.486 INFO:tasks.workunit.client.1.smithi117.stdout:6/555: link d5/d4b/d20/f3a d5/d4b/d20/fb5 0 2024-02-16T19:29:10.492 INFO:tasks.workunit.client.1.smithi117.stdout:8/563: dwrite d4/d1c/f7f [0,4194304] 0 2024-02-16T19:29:10.492 INFO:tasks.workunit.client.0.smithi022.stdout:4/530: unlink d4/d6/dd/df/d10/d55/f7b 0 2024-02-16T19:29:10.495 INFO:tasks.workunit.client.0.smithi022.stdout:5/559: dwrite dd/f17 [4194304,4194304] 0 2024-02-16T19:29:10.497 INFO:tasks.workunit.client.1.smithi117.stdout:9/484: mknod dd/d45/d31/d63/ca5 0 2024-02-16T19:29:10.497 INFO:tasks.workunit.client.0.smithi022.stdout:0/731: fdatasync d10/d15/f39 0 2024-02-16T19:29:10.502 INFO:tasks.workunit.client.0.smithi022.stdout:6/577: rename d1/d8/da/d17/d85/d5b/f8a to d1/d8/da/d17/d85/d9c/fce 0 2024-02-16T19:29:10.510 INFO:tasks.workunit.client.0.smithi022.stdout:9/590: mknod db/d14/d16/d87/cca 0 2024-02-16T19:29:10.511 INFO:tasks.workunit.client.0.smithi022.stdout:9/591: dread - db/d14/d16/d21/d37/fb1 zero size 2024-02-16T19:29:10.517 INFO:tasks.workunit.client.1.smithi117.stdout:5/621: creat d2/d5d/d8e/dc1/fd1 x:0 0 0 2024-02-16T19:29:10.520 INFO:tasks.workunit.client.0.smithi022.stdout:1/646: mkdir d6/d8/d6c/dc9 0 2024-02-16T19:29:10.524 INFO:tasks.workunit.client.0.smithi022.stdout:9/592: dread db/d14/d16/d5d/f7e [0,4194304] 0 2024-02-16T19:29:10.527 INFO:tasks.workunit.client.1.smithi117.stdout:1/509: dwrite d2/d1d/d77/fa0 [0,4194304] 0 2024-02-16T19:29:10.527 INFO:tasks.workunit.client.0.smithi022.stdout:3/506: write d7/d12/d18/dd/f28 [776044,27354] 0 2024-02-16T19:29:10.528 INFO:tasks.workunit.client.0.smithi022.stdout:3/507: stat d7/d12/d18/d4b/db6 0 2024-02-16T19:29:10.529 INFO:tasks.workunit.client.0.smithi022.stdout:3/508: chown d7/d12/d18/dd/d24/fb4 186672 1 2024-02-16T19:29:10.532 INFO:tasks.workunit.client.1.smithi117.stdout:2/571: rename d0/d3/d40/l74 to d0/d3/d34/d94/d46/d68/lab 0 2024-02-16T19:29:10.534 INFO:tasks.workunit.client.1.smithi117.stdout:9/485: creat dd/d2d/fa6 x:0 0 0 2024-02-16T19:29:10.534 INFO:tasks.workunit.client.0.smithi022.stdout:3/509: dread d7/d12/d18/dac/f9d [0,4194304] 0 2024-02-16T19:29:10.535 INFO:tasks.workunit.client.1.smithi117.stdout:4/550: getdents d0 0 2024-02-16T19:29:10.537 INFO:tasks.workunit.client.1.smithi117.stdout:5/622: fdatasync d2/d3/d21/f92 0 2024-02-16T19:29:10.547 INFO:tasks.workunit.client.0.smithi022.stdout:5/560: truncate dd/d1f/d48/d21/d2f/d40/d49/d4b/f66 807906 0 2024-02-16T19:29:10.548 INFO:tasks.workunit.client.0.smithi022.stdout:5/561: write f6 [3941535,61025] 0 2024-02-16T19:29:10.553 INFO:tasks.workunit.client.0.smithi022.stdout:4/531: rename d4/d6/f16 to d4/d6/dd/d61/fa4 0 2024-02-16T19:29:10.555 INFO:tasks.workunit.client.1.smithi117.stdout:0/606: rename d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/fc7 to d0/d9/d22/dd4/fd7 0 2024-02-16T19:29:10.556 INFO:tasks.workunit.client.1.smithi117.stdout:8/564: truncate d4/f79 3635361 0 2024-02-16T19:29:10.557 INFO:tasks.workunit.client.0.smithi022.stdout:6/578: symlink d1/d8/da/d17/d53/d7f/da3/lcf 0 2024-02-16T19:29:10.562 INFO:tasks.workunit.client.1.smithi117.stdout:9/486: read - dd/d47/d5b/d5f/f65 zero size 2024-02-16T19:29:10.569 INFO:tasks.workunit.client.1.smithi117.stdout:4/551: dread d0/da/d21/f40 [0,4194304] 0 2024-02-16T19:29:10.569 INFO:tasks.workunit.client.0.smithi022.stdout:7/567: unlink d9/da/c30 0 2024-02-16T19:29:10.570 INFO:tasks.workunit.client.1.smithi117.stdout:2/572: write d0/d3/d40/f55 [4184220,107074] 0 2024-02-16T19:29:10.572 INFO:tasks.workunit.client.0.smithi022.stdout:9/593: creat db/d14/d16/d5d/d65/d6b/fcb x:0 0 0 2024-02-16T19:29:10.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:10.572+0000 7fdef6048700 1 -- 172.21.15.22:0/1784144444 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdef00ff130 msgr2=0x7fdef00ff540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:29:10.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:10.572+0000 7fdef6048700 1 --2- 172.21.15.22:0/1784144444 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdef00ff130 0x7fdef00ff540 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fdee0009910 tx=0x7fdee0009c20 comp rx=0 tx=0).stop 2024-02-16T19:29:10.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:10.572+0000 7fdef6048700 1 -- 172.21.15.22:0/1784144444 shutdown_connections 2024-02-16T19:29:10.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:10.572+0000 7fdef6048700 1 --2- 172.21.15.22:0/1784144444 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdef01003d0 0x7fdef0100840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:10.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:10.572+0000 7fdef6048700 1 --2- 172.21.15.22:0/1784144444 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdef00ff130 0x7fdef00ff540 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:10.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:10.572+0000 7fdef6048700 1 -- 172.21.15.22:0/1784144444 >> 172.21.15.22:0/1784144444 conn(0x7fdef00fa780 msgr2=0x7fdef00fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:29:10.574 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:10 smithi022 ceph-mon[29221]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:29:10.574 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:10 smithi022 ceph-mon[29221]: pgmap v8: 65 pgs: 65 active+clean; 2.0 GiB data, 5.8 GiB used, 531 GiB / 536 GiB avail; 51 MiB/s rd, 128 MiB/s wr, 267 op/s 2024-02-16T19:29:10.574 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:10 smithi022 ceph-mon[29221]: Standby manager daemon smithi022.gmdcch started 2024-02-16T19:29:10.574 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:10 smithi022 ceph-mon[29221]: from='mgr.? 172.21.15.22:0/896396138' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi022.gmdcch/crt"}]: dispatch 2024-02-16T19:29:10.574 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:10 smithi022 ceph-mon[29221]: from='mgr.? 172.21.15.22:0/896396138' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-16T19:29:10.574 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:10 smithi022 ceph-mon[29221]: from='mgr.? 172.21.15.22:0/896396138' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi022.gmdcch/key"}]: dispatch 2024-02-16T19:29:10.574 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:10 smithi022 ceph-mon[29221]: from='mgr.? 172.21.15.22:0/896396138' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-16T19:29:10.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:10.572+0000 7fdef6048700 1 -- 172.21.15.22:0/1784144444 shutdown_connections 2024-02-16T19:29:10.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:10.572+0000 7fdef6048700 1 -- 172.21.15.22:0/1784144444 wait complete. 2024-02-16T19:29:10.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:10.574+0000 7fdef6048700 1 Processor -- start 2024-02-16T19:29:10.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:10.574+0000 7fdef6048700 1 -- start start 2024-02-16T19:29:10.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:10.574+0000 7fdef6048700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdef00ff130 0x7fdef0193840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:29:10.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:10.574+0000 7fdef6048700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdef01003d0 0x7fdef0193d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:29:10.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:10.575+0000 7fdef6048700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef01943a0 con 0x7fdef01003d0 2024-02-16T19:29:10.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:10.575+0000 7fdeef7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdef00ff130 0x7fdef0193840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:29:10.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:10.575+0000 7fdeef7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdef00ff130 0x7fdef0193840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.22:50112/0 (socket says 172.21.15.22:50112) 2024-02-16T19:29:10.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:10.575+0000 7fdeef7fe700 1 -- 172.21.15.22:0/347594203 learned_addr learned my addr 172.21.15.22:0/347594203 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:29:10.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:10.575+0000 7fdef6048700 1 -- 172.21.15.22:0/347594203 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef01944e0 con 0x7fdef00ff130 2024-02-16T19:29:10.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:10.575+0000 7fdee7fff700 1 --2- 172.21.15.22:0/347594203 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdef01003d0 0x7fdef0193d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:29:10.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:10.575+0000 7fdee7fff700 1 -- 172.21.15.22:0/347594203 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdef00ff130 msgr2=0x7fdef0193840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:29:10.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:10.575+0000 7fdee7fff700 1 --2- 172.21.15.22:0/347594203 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdef00ff130 0x7fdef0193840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:10.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:10.575+0000 7fdee7fff700 1 -- 172.21.15.22:0/347594203 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fded8009520 con 0x7fdef01003d0 2024-02-16T19:29:10.577 INFO:tasks.workunit.client.0.smithi022.stdout:2/531: truncate d0/f2d 2131452 0 2024-02-16T19:29:10.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:10.575+0000 7fdeef7fe700 1 --2- 172.21.15.22:0/347594203 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdef00ff130 0x7fdef0193840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T19:29:10.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:10.576+0000 7fdee7fff700 1 --2- 172.21.15.22:0/347594203 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdef01003d0 0x7fdef0193d80 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fded8010d00 tx=0x7fded800b5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:29:10.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:10.576+0000 7fdeed7fa700 1 -- 172.21.15.22:0/347594203 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fded8015070 con 0x7fdef01003d0 2024-02-16T19:29:10.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:10.576+0000 7fdeed7fa700 1 -- 172.21.15.22:0/347594203 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fded8010e50 con 0x7fdef01003d0 2024-02-16T19:29:10.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:10.576+0000 7fdeed7fa700 1 -- 172.21.15.22:0/347594203 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fded8019410 con 0x7fdef01003d0 2024-02-16T19:29:10.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:10.576+0000 7fdef6048700 1 -- 172.21.15.22:0/347594203 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdee00095f0 con 0x7fdef01003d0 2024-02-16T19:29:10.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:10.576+0000 7fdef6048700 1 -- 172.21.15.22:0/347594203 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdef0199320 con 0x7fdef01003d0 2024-02-16T19:29:10.579 INFO:tasks.workunit.client.1.smithi117.stdout:6/556: link d5/db/d7d/d32/c82 d5/d9c/cb6 0 2024-02-16T19:29:10.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:10.578+0000 7fdef6048700 1 -- 172.21.15.22:0/347594203 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdef0106ae0 con 0x7fdef01003d0 2024-02-16T19:29:10.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:10.579+0000 7fdeed7fa700 1 -- 172.21.15.22:0/347594203 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 25) v1 ==== 90734+0+0 (secure 0 0 0) 0x7fded8004000 con 0x7fdef01003d0 2024-02-16T19:29:10.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:10.580+0000 7fdeed7fa700 1 --2- 172.21.15.22:0/347594203 >> [v2:172.21.15.117:6828/10500008,v1:172.21.15.117:6829/10500008] conn(0x7fded006cf00 0x7fded006f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:29:10.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:10.580+0000 7fdeed7fa700 1 -- 172.21.15.22:0/347594203 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fded80901a0 con 0x7fdef01003d0 2024-02-16T19:29:10.582 INFO:tasks.workunit.client.1.smithi117.stdout:1/510: symlink d2/d1d/d60/la9 0 2024-02-16T19:29:10.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:10.580+0000 7fdeef7fe700 1 --2- 172.21.15.22:0/347594203 >> [v2:172.21.15.117:6828/10500008,v1:172.21.15.117:6829/10500008] conn(0x7fded006cf00 0x7fded006f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:29:10.583 INFO:tasks.workunit.client.0.smithi022.stdout:8/561: link d3/d12/d2b/l4e d3/d12/d92/lcc 0 2024-02-16T19:29:10.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:10.581+0000 7fdeef7fe700 1 --2- 172.21.15.22:0/347594203 >> [v2:172.21.15.117:6828/10500008,v1:172.21.15.117:6829/10500008] conn(0x7fded006cf00 0x7fded006f3b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fdee0006010 tx=0x7fdee0005a90 comp rx=0 tx=0).ready entity=mgr.24349 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:29:10.585 INFO:tasks.workunit.client.0.smithi022.stdout:2/532: dwrite d0/d2/d18/d19/d90/fa3 [0,4194304] 0 2024-02-16T19:29:10.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:10.587+0000 7fdeed7fa700 1 -- 172.21.15.22:0/347594203 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7fded805d960 con 0x7fdef01003d0 2024-02-16T19:29:10.593 INFO:tasks.workunit.client.0.smithi022.stdout:0/732: rename d10/d11/d27/d3f/d58/d60/fa7 to d10/d22/dad/de7/ff3 0 2024-02-16T19:29:10.594 INFO:tasks.workunit.client.0.smithi022.stdout:0/733: stat d10/d11/d27/lc4 0 2024-02-16T19:29:10.598 INFO:tasks.workunit.client.0.smithi022.stdout:7/568: mkdir d9/da/d1f/d67/db3 0 2024-02-16T19:29:10.600 INFO:tasks.workunit.client.0.smithi022.stdout:9/594: creat db/d14/d16/d21/fcc x:0 0 0 2024-02-16T19:29:10.601 INFO:tasks.workunit.client.0.smithi022.stdout:9/595: write db/d14/d19/f17 [4821542,94867] 0 2024-02-16T19:29:10.605 INFO:tasks.workunit.client.0.smithi022.stdout:2/533: chown d0/d2/d18/c33 2149 1 2024-02-16T19:29:10.606 INFO:tasks.workunit.client.0.smithi022.stdout:2/534: write d0/d2/d18/f28 [1075146,21486] 0 2024-02-16T19:29:10.608 INFO:tasks.workunit.client.0.smithi022.stdout:1/647: rename d6/d8/d22/d67/c6e to d6/d8/d22/d3a/d80/cca 0 2024-02-16T19:29:10.611 INFO:tasks.workunit.client.0.smithi022.stdout:0/734: truncate d10/d22/fa1 4766265 0 2024-02-16T19:29:10.613 INFO:tasks.workunit.client.0.smithi022.stdout:8/562: mkdir d3/d12/d2b/d54/dcd 0 2024-02-16T19:29:10.615 INFO:tasks.workunit.client.0.smithi022.stdout:8/563: read d3/d12/d2b/f3f [2884879,49065] 0 2024-02-16T19:29:10.616 INFO:tasks.workunit.client.0.smithi022.stdout:8/564: chown d3/d12/d6a/fa8 583497 1 2024-02-16T19:29:10.619 INFO:tasks.workunit.client.0.smithi022.stdout:4/532: link d4/d6/dd/df/d10/d55/c5e d4/d6/dd/df/d10/d24/d6b/ca5 0 2024-02-16T19:29:10.620 INFO:tasks.workunit.client.0.smithi022.stdout:2/535: dread d0/d2/d18/d1f/f4d [0,4194304] 0 2024-02-16T19:29:10.622 INFO:tasks.workunit.client.0.smithi022.stdout:7/569: creat d9/da/d1f/d67/db3/fb4 x:0 0 0 2024-02-16T19:29:10.622 INFO:tasks.workunit.client.0.smithi022.stdout:0/735: mknod d10/d15/d23/d4e/cf4 0 2024-02-16T19:29:10.623 INFO:tasks.workunit.client.0.smithi022.stdout:0/736: write d10/d22/f53 [1805618,24708] 0 2024-02-16T19:29:10.625 INFO:tasks.workunit.client.0.smithi022.stdout:8/565: creat d3/d12/d6a/fce x:0 0 0 2024-02-16T19:29:10.627 INFO:tasks.workunit.client.0.smithi022.stdout:1/648: rename d6/d8/d22/d3a/d7a/caf to d6/d8/d22/d3a/d7a/ccb 0 2024-02-16T19:29:10.628 INFO:tasks.workunit.client.0.smithi022.stdout:2/536: symlink d0/d2/d18/d38/lad 0 2024-02-16T19:29:10.630 INFO:tasks.workunit.client.0.smithi022.stdout:4/533: creat d4/d6/dd/df/d10/d24/fa6 x:0 0 0 2024-02-16T19:29:10.637 INFO:tasks.workunit.client.0.smithi022.stdout:4/534: dwrite d4/d6/f7f [0,4194304] 0 2024-02-16T19:29:10.639 INFO:tasks.workunit.client.0.smithi022.stdout:9/596: getdents db/d14/dc8 0 2024-02-16T19:29:10.645 INFO:tasks.workunit.client.0.smithi022.stdout:5/562: rename dd/d1f/d48/d21/d2f/lad to dd/d1f/d48/d21/lc2 0 2024-02-16T19:29:10.645 INFO:tasks.workunit.client.0.smithi022.stdout:1/649: dwrite d6/d8/d22/d16/f40 [0,4194304] 0 2024-02-16T19:29:10.649 INFO:tasks.workunit.client.0.smithi022.stdout:0/737: link d10/l2b d10/d15/d23/d38/d5a/d66/lf5 0 2024-02-16T19:29:10.651 INFO:tasks.workunit.client.0.smithi022.stdout:6/579: rename d1/d8/da/d17/c8e to d1/d8/da/d17/d48/cd0 0 2024-02-16T19:29:10.654 INFO:tasks.workunit.client.0.smithi022.stdout:1/650: symlink d6/d8/d22/d3a/d3e/lcc 0 2024-02-16T19:29:10.655 INFO:tasks.workunit.client.0.smithi022.stdout:5/563: mknod dd/d1f/d48/d21/d5d/cc3 0 2024-02-16T19:29:10.656 INFO:tasks.workunit.client.1.smithi117.stdout:1/511: symlink d2/d19/d59/d89/d7d/laa 0 2024-02-16T19:29:10.657 INFO:tasks.workunit.client.1.smithi117.stdout:9/487: write dd/d45/d73/f94 [4992195,6433] 0 2024-02-16T19:29:10.659 INFO:tasks.workunit.client.1.smithi117.stdout:0/607: dread d0/d30/f8c [0,4194304] 0 2024-02-16T19:29:10.659 INFO:tasks.workunit.client.1.smithi117.stdout:1/512: chown d2/d1d/d60/f3e 1124951 1 2024-02-16T19:29:10.659 INFO:tasks.workunit.client.1.smithi117.stdout:5/623: dwrite d2/d3/f6 [0,4194304] 0 2024-02-16T19:29:10.659 INFO:tasks.workunit.client.0.smithi022.stdout:4/535: dwrite d4/f23 [0,4194304] 0 2024-02-16T19:29:10.662 INFO:tasks.workunit.client.0.smithi022.stdout:6/580: dread d1/d8/ff [0,4194304] 0 2024-02-16T19:29:10.663 INFO:tasks.workunit.client.0.smithi022.stdout:4/536: read d4/f23 [5925501,27220] 0 2024-02-16T19:29:10.666 INFO:tasks.workunit.client.1.smithi117.stdout:4/552: symlink d0/da/d21/d3d/d28/d66/d97/lb2 0 2024-02-16T19:29:10.667 INFO:tasks.workunit.client.0.smithi022.stdout:7/570: rename d9/f6e to d9/da/d1f/d40/fb5 0 2024-02-16T19:29:10.668 INFO:tasks.workunit.client.1.smithi117.stdout:6/557: symlink d5/d4b/d44/d52/lb7 0 2024-02-16T19:29:10.669 INFO:tasks.workunit.client.0.smithi022.stdout:7/571: truncate d9/da/d1f/d40/fa5 696389 0 2024-02-16T19:29:10.671 INFO:tasks.workunit.client.0.smithi022.stdout:3/510: truncate d7/d12/d18/dac/f9d 3378916 0 2024-02-16T19:29:10.672 INFO:tasks.workunit.client.1.smithi117.stdout:1/513: dwrite d2/d1d/f29 [0,4194304] 0 2024-02-16T19:29:10.672 INFO:tasks.workunit.client.0.smithi022.stdout:5/564: read dd/d1f/d48/d21/d2f/d40/f3b [3025364,24141] 0 2024-02-16T19:29:10.678 INFO:tasks.workunit.client.0.smithi022.stdout:0/738: creat d10/d11/d27/d3f/d58/dab/ff6 x:0 0 0 2024-02-16T19:29:10.679 INFO:tasks.workunit.client.1.smithi117.stdout:2/573: dread d0/d3/f31 [0,4194304] 0 2024-02-16T19:29:10.681 INFO:tasks.workunit.client.1.smithi117.stdout:9/488: creat dd/d47/d5b/d5f/d77/fa7 x:0 0 0 2024-02-16T19:29:10.682 INFO:tasks.workunit.client.1.smithi117.stdout:9/489: write dd/d47/d5b/d5f/d77/d7c/f8b [4249295,2275] 0 2024-02-16T19:29:10.683 INFO:tasks.workunit.client.0.smithi022.stdout:6/581: creat d1/d8/d3c/d77/fd1 x:0 0 0 2024-02-16T19:29:10.685 INFO:tasks.workunit.client.0.smithi022.stdout:6/582: read d1/d8/f1c [1159399,659] 0 2024-02-16T19:29:10.687 INFO:tasks.workunit.client.1.smithi117.stdout:7/565: rename d4/d1c/d39/d46/d6e to d4/dd3 0 2024-02-16T19:29:10.689 INFO:tasks.workunit.client.0.smithi022.stdout:4/537: unlink d4/d6/l4c 0 2024-02-16T19:29:10.693 INFO:tasks.workunit.client.0.smithi022.stdout:1/651: rename d6/d8/d22/d59/fc0 to d6/d8/d22/d47/fcd 0 2024-02-16T19:29:10.695 INFO:tasks.workunit.client.0.smithi022.stdout:7/572: symlink d9/da/d1f/d67/db3/lb6 0 2024-02-16T19:29:10.696 INFO:tasks.workunit.client.0.smithi022.stdout:7/573: chown d9/d5d/f97 514 1 2024-02-16T19:29:10.697 INFO:tasks.workunit.client.0.smithi022.stdout:7/574: readlink d9/da/d4e/l94 0 2024-02-16T19:29:10.697 INFO:tasks.workunit.client.1.smithi117.stdout:6/558: creat d5/d9c/daa/fb8 x:0 0 0 2024-02-16T19:29:10.700 INFO:tasks.workunit.client.0.smithi022.stdout:3/511: creat d7/d21/fbd x:0 0 0 2024-02-16T19:29:10.701 INFO:tasks.workunit.client.1.smithi117.stdout:2/574: creat d0/d13/d75/fac x:0 0 0 2024-02-16T19:29:10.705 INFO:tasks.workunit.client.0.smithi022.stdout:2/537: dwrite d0/d2/d2e/f72 [0,4194304] 0 2024-02-16T19:29:10.707 INFO:tasks.workunit.client.0.smithi022.stdout:2/538: write d0/d32/f52 [4103077,28907] 0 2024-02-16T19:29:10.707 INFO:tasks.workunit.client.1.smithi117.stdout:9/490: fdatasync dd/d45/d62/f4e 0 2024-02-16T19:29:10.712 INFO:tasks.workunit.client.0.smithi022.stdout:8/566: write d3/d12/d17/f3d [179145,110806] 0 2024-02-16T19:29:10.713 INFO:tasks.workunit.client.1.smithi117.stdout:3/654: rename d1/d31/f36 to d1/d83/fe3 0 2024-02-16T19:29:10.720 INFO:tasks.workunit.client.0.smithi022.stdout:9/597: dwrite db/d14/d16/d5d/d65/f96 [4194304,4194304] 0 2024-02-16T19:29:10.723 INFO:tasks.workunit.client.1.smithi117.stdout:6/559: dread d5/db/d7d/d32/f48 [0,4194304] 0 2024-02-16T19:29:10.728 INFO:tasks.workunit.client.1.smithi117.stdout:1/514: truncate d2/f5b 219284 0 2024-02-16T19:29:10.730 INFO:tasks.workunit.client.0.smithi022.stdout:5/565: symlink dd/d1f/d48/d21/d2f/d40/d71/lc4 0 2024-02-16T19:29:10.731 INFO:tasks.workunit.client.0.smithi022.stdout:9/598: dwrite db/d14/d19/d1a/f24 [0,4194304] 0 2024-02-16T19:29:10.732 INFO:tasks.workunit.client.0.smithi022.stdout:5/566: truncate dd/d1f/d48/f74 5040597 0 2024-02-16T19:29:10.733 INFO:tasks.workunit.client.1.smithi117.stdout:2/575: rmdir d0/d13/d4d/d80 39 2024-02-16T19:29:10.747 INFO:tasks.workunit.client.0.smithi022.stdout:8/567: creat d3/d25/d33/fcf x:0 0 0 2024-02-16T19:29:10.748 INFO:tasks.workunit.client.0.smithi022.stdout:8/568: readlink d3/d12/d2b/l4e 0 2024-02-16T19:29:10.749 INFO:tasks.workunit.client.0.smithi022.stdout:8/569: write d3/d12/f8b [2166820,114265] 0 2024-02-16T19:29:10.749 INFO:tasks.workunit.client.1.smithi117.stdout:4/553: creat d0/da/d21/d3d/d28/d64/fb3 x:0 0 0 2024-02-16T19:29:10.750 INFO:tasks.workunit.client.0.smithi022.stdout:8/570: chown d3/d12/f9c 450735 1 2024-02-16T19:29:10.754 INFO:tasks.workunit.client.1.smithi117.stdout:6/560: chown d5/d4b/d44/f56 78 1 2024-02-16T19:29:10.758 INFO:tasks.workunit.client.1.smithi117.stdout:2/576: truncate d0/d3/d10/f16 1275073 0 2024-02-16T19:29:10.759 INFO:tasks.workunit.client.1.smithi117.stdout:2/577: truncate d0/d3/d40/f66 1148361 0 2024-02-16T19:29:10.761 INFO:tasks.workunit.client.0.smithi022.stdout:3/512: dwrite d7/d12/d18/dac/f4c [0,4194304] 0 2024-02-16T19:29:10.765 INFO:tasks.workunit.client.1.smithi117.stdout:3/655: symlink d1/d30/d9f/dcb/le4 0 2024-02-16T19:29:10.765 INFO:tasks.workunit.client.0.smithi022.stdout:9/599: dread db/d14/d16/d28/daa/fad [0,4194304] 0 2024-02-16T19:29:10.766 INFO:tasks.workunit.client.1.smithi117.stdout:3/656: fdatasync d1/dd/d3b/f3d 0 2024-02-16T19:29:10.767 INFO:tasks.workunit.client.0.smithi022.stdout:3/513: truncate d7/d12/d18/dd/d24/d58/f95 996242 0 2024-02-16T19:29:10.767 INFO:tasks.workunit.client.1.smithi117.stdout:3/657: write d1/d1b/d20/fa2 [2030400,109496] 0 2024-02-16T19:29:10.767 INFO:tasks.workunit.client.0.smithi022.stdout:2/539: rename d0/d32/d3b/c49 to d0/d2/d18/d19/d90/cae 0 2024-02-16T19:29:10.768 INFO:tasks.workunit.client.1.smithi117.stdout:7/566: truncate d4/d1c/d39/d46/d47/f82 3347150 0 2024-02-16T19:29:10.769 INFO:tasks.workunit.client.0.smithi022.stdout:5/567: dwrite dd/d1f/d48/d21/f24 [0,4194304] 0 2024-02-16T19:29:10.775 INFO:tasks.workunit.client.1.smithi117.stdout:7/567: dwrite d4/d1c/fbe [0,4194304] 0 2024-02-16T19:29:10.777 INFO:tasks.workunit.client.0.smithi022.stdout:6/583: link d1/d8/da/d17/d85/d5b/lcc d1/ld2 0 2024-02-16T19:29:10.791 INFO:tasks.workunit.client.0.smithi022.stdout:9/600: symlink db/d14/d16/d5d/d65/d6b/d89/lcd 0 2024-02-16T19:29:10.794 INFO:tasks.workunit.client.0.smithi022.stdout:3/514: dwrite d7/d12/f25 [4194304,4194304] 0 2024-02-16T19:29:10.799 INFO:tasks.workunit.client.0.smithi022.stdout:5/568: read dd/d1f/d48/d21/d2f/d40/d4e/f68 [63700,70686] 0 2024-02-16T19:29:10.800 INFO:tasks.workunit.client.0.smithi022.stdout:0/739: getdents d10/d22 0 2024-02-16T19:29:10.802 INFO:tasks.workunit.client.0.smithi022.stdout:6/584: creat d1/d8/da/d17/d53/d7f/fd3 x:0 0 0 2024-02-16T19:29:10.805 INFO:tasks.workunit.client.0.smithi022.stdout:0/740: read d10/d22/f2a [1405088,11325] 0 2024-02-16T19:29:10.806 INFO:tasks.workunit.client.0.smithi022.stdout:9/601: creat db/d14/d16/d49/fce x:0 0 0 2024-02-16T19:29:10.811 INFO:tasks.workunit.client.0.smithi022.stdout:6/585: dwrite d1/d8/da/d17/d85/d56/fa5 [0,4194304] 0 2024-02-16T19:29:10.815 INFO:tasks.workunit.client.0.smithi022.stdout:6/586: write d1/d8/f3b [3293431,50233] 0 2024-02-16T19:29:10.816 INFO:tasks.workunit.client.0.smithi022.stdout:9/602: dwrite db/d14/d16/d21/fc1 [0,4194304] 0 2024-02-16T19:29:10.818 INFO:tasks.workunit.client.0.smithi022.stdout:5/569: creat dd/d1f/d48/d21/d64/fc5 x:0 0 0 2024-02-16T19:29:10.822 INFO:tasks.workunit.client.0.smithi022.stdout:5/570: write dd/d1f/d48/d21/d2f/d52/d5c/f63 [208434,87777] 0 2024-02-16T19:29:10.827 INFO:tasks.workunit.client.1.smithi117.stdout:8/565: sync 2024-02-16T19:29:10.827 INFO:tasks.workunit.client.1.smithi117.stdout:5/624: sync 2024-02-16T19:29:10.828 INFO:tasks.workunit.client.0.smithi022.stdout:2/540: truncate d0/d32/f52 2984274 0 2024-02-16T19:29:10.833 INFO:tasks.workunit.client.0.smithi022.stdout:3/515: dread d7/d12/d18/dd/d31/d7f/fa5 [0,4194304] 0 2024-02-16T19:29:10.834 INFO:tasks.workunit.client.0.smithi022.stdout:6/587: mkdir d1/d8/d81/dd4 0 2024-02-16T19:29:10.835 INFO:tasks.workunit.client.0.smithi022.stdout:6/588: fsync d1/f3 0 2024-02-16T19:29:10.836 INFO:tasks.workunit.client.1.smithi117.stdout:1/515: mkdir d2/dab 0 2024-02-16T19:29:10.836 INFO:tasks.workunit.client.0.smithi022.stdout:6/589: readlink d1/d8/da/d17/d53/l5a 0 2024-02-16T19:29:10.836 INFO:tasks.workunit.client.0.smithi022.stdout:6/590: stat d1/d8/d23/lb7 0 2024-02-16T19:29:10.840 INFO:tasks.workunit.client.1.smithi117.stdout:9/491: creat dd/d23/fa8 x:0 0 0 2024-02-16T19:29:10.841 INFO:tasks.workunit.client.0.smithi022.stdout:5/571: creat dd/d1f/d4a/fc6 x:0 0 0 2024-02-16T19:29:10.843 INFO:tasks.workunit.client.1.smithi117.stdout:0/608: rename d0/d9/fc9 to d0/d9/d14/d23/d33/fd8 0 2024-02-16T19:29:10.844 INFO:tasks.workunit.client.0.smithi022.stdout:7/575: getdents d9/d1b/d23 0 2024-02-16T19:29:10.850 INFO:tasks.workunit.client.1.smithi117.stdout:4/554: unlink d0/da/f44 0 2024-02-16T19:29:10.853 INFO:tasks.workunit.client.1.smithi117.stdout:6/561: dread d5/db/d7d/d32/d95/f50 [0,4194304] 0 2024-02-16T19:29:10.856 INFO:tasks.workunit.client.0.smithi022.stdout:2/541: dread d0/d2/f57 [0,4194304] 0 2024-02-16T19:29:10.856 INFO:tasks.workunit.client.0.smithi022.stdout:6/591: rename d1/d8/da/d29/d61/d57/c87 to d1/d8/da/d17/d53/d7f/da3/cd5 0 2024-02-16T19:29:10.857 INFO:tasks.workunit.client.1.smithi117.stdout:2/578: write d0/d3/d22/f3c [253172,92098] 0 2024-02-16T19:29:10.859 INFO:tasks.workunit.client.0.smithi022.stdout:9/603: dwrite db/d14/d19/f5f [0,4194304] 0 2024-02-16T19:29:10.864 INFO:tasks.workunit.client.1.smithi117.stdout:5/625: rmdir d2/d9/d3c/d24/d50/d52/d74/d8f 39 2024-02-16T19:29:10.867 INFO:tasks.workunit.client.0.smithi022.stdout:9/604: dwrite db/d14/d19/fa2 [0,4194304] 0 2024-02-16T19:29:10.868 INFO:tasks.workunit.client.0.smithi022.stdout:9/605: chown db/d14/d16/d28/daa 848 1 2024-02-16T19:29:10.869 INFO:tasks.workunit.client.0.smithi022.stdout:0/741: creat d10/d11/d27/d3f/d58/d60/ff7 x:0 0 0 2024-02-16T19:29:10.871 INFO:tasks.workunit.client.0.smithi022.stdout:3/516: getdents d7/d12/d18/dd/d24/d93 0 2024-02-16T19:29:10.882 INFO:tasks.workunit.client.1.smithi117.stdout:3/658: getdents d1/d31/d89/dbd 0 2024-02-16T19:29:10.882 INFO:tasks.workunit.client.1.smithi117.stdout:8/566: dread d4/d12/d2a/f2c [0,4194304] 0 2024-02-16T19:29:10.886 INFO:tasks.workunit.client.0.smithi022.stdout:9/606: creat db/d14/d19/fcf x:0 0 0 2024-02-16T19:29:10.886 INFO:tasks.workunit.client.0.smithi022.stdout:7/576: write d9/da/f13 [8529229,79031] 0 2024-02-16T19:29:10.886 INFO:tasks.workunit.client.0.smithi022.stdout:9/607: stat db/d14/lbc 0 2024-02-16T19:29:10.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:10 smithi117 ceph-mon[35309]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:29:10.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:10 smithi117 ceph-mon[35309]: pgmap v8: 65 pgs: 65 active+clean; 2.0 GiB data, 5.8 GiB used, 531 GiB / 536 GiB avail; 51 MiB/s rd, 128 MiB/s wr, 267 op/s 2024-02-16T19:29:10.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:10 smithi117 ceph-mon[35309]: Standby manager daemon smithi022.gmdcch started 2024-02-16T19:29:10.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:10 smithi117 ceph-mon[35309]: from='mgr.? 172.21.15.22:0/896396138' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi022.gmdcch/crt"}]: dispatch 2024-02-16T19:29:10.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:10 smithi117 ceph-mon[35309]: from='mgr.? 172.21.15.22:0/896396138' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-16T19:29:10.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:10 smithi117 ceph-mon[35309]: from='mgr.? 172.21.15.22:0/896396138' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi022.gmdcch/key"}]: dispatch 2024-02-16T19:29:10.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:10 smithi117 ceph-mon[35309]: from='mgr.? 172.21.15.22:0/896396138' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-16T19:29:10.888 INFO:tasks.workunit.client.1.smithi117.stdout:3/659: dwrite d1/d83/d96/fb6 [0,4194304] 0 2024-02-16T19:29:10.889 INFO:tasks.workunit.client.1.smithi117.stdout:2/579: creat d0/d3/d22/d37/d9d/fad x:0 0 0 2024-02-16T19:29:10.889 INFO:tasks.workunit.client.1.smithi117.stdout:2/580: stat d0/d3/d34/d94/d46/f83 0 2024-02-16T19:29:10.890 INFO:tasks.workunit.client.0.smithi022.stdout:0/742: read d10/d15/d23/d38/f41 [2742365,79653] 0 2024-02-16T19:29:10.891 INFO:tasks.workunit.client.1.smithi117.stdout:5/626: rmdir d2/d4c/d4f 39 2024-02-16T19:29:10.893 INFO:tasks.workunit.client.0.smithi022.stdout:5/572: rename dd/d1f/d48/c42 to dd/d1f/d48/d21/d2f/d40/d49/d4b/db5/cc7 0 2024-02-16T19:29:10.894 INFO:tasks.workunit.client.0.smithi022.stdout:7/577: dwrite d9/d1b/fb0 [0,4194304] 0 2024-02-16T19:29:10.902 INFO:tasks.workunit.client.0.smithi022.stdout:6/592: write d1/d8/da/d17/d85/d56/f90 [650822,120000] 0 2024-02-16T19:29:10.902 INFO:tasks.workunit.client.0.smithi022.stdout:0/743: creat d10/d22/d49/ff8 x:0 0 0 2024-02-16T19:29:10.904 INFO:tasks.workunit.client.1.smithi117.stdout:8/567: symlink d4/d7/da6/d3a/d43/lb1 0 2024-02-16T19:29:10.905 INFO:tasks.workunit.client.0.smithi022.stdout:3/517: truncate d7/d12/d18/dd/ff 936677 0 2024-02-16T19:29:10.908 INFO:tasks.workunit.client.0.smithi022.stdout:9/608: rename db/d14/d19/d1a/cc7 to db/d14/dc8/cd0 0 2024-02-16T19:29:10.911 INFO:tasks.workunit.client.0.smithi022.stdout:2/542: link d0/d32/l5b d0/d2/d2e/laf 0 2024-02-16T19:29:10.912 INFO:tasks.workunit.client.0.smithi022.stdout:2/543: dread - d0/d2/d18/d19/fab zero size 2024-02-16T19:29:10.913 INFO:tasks.workunit.client.1.smithi117.stdout:9/492: creat dd/fa9 x:0 0 0 2024-02-16T19:29:10.915 INFO:tasks.workunit.client.1.smithi117.stdout:3/660: dwrite d1/d30/d52/d56/f3f [4194304,4194304] 0 2024-02-16T19:29:10.915 INFO:tasks.workunit.client.1.smithi117.stdout:7/568: rename d4/d1c/f2e to d4/d1c/d87/d89/fd4 0 2024-02-16T19:29:10.916 INFO:tasks.workunit.client.1.smithi117.stdout:3/661: chown d1/d1b/d1e/d37/cdd 174 1 2024-02-16T19:29:10.916 INFO:tasks.workunit.client.0.smithi022.stdout:5/573: creat dd/d1f/d48/d21/d2f/d52/d77/da9/fc8 x:0 0 0 2024-02-16T19:29:10.917 INFO:tasks.workunit.client.1.smithi117.stdout:3/662: stat d1/d1b/d1e/l7f 0 2024-02-16T19:29:10.917 INFO:tasks.workunit.client.0.smithi022.stdout:0/744: dread d10/d15/d23/d8f/f95 [0,4194304] 0 2024-02-16T19:29:10.918 INFO:tasks.workunit.client.0.smithi022.stdout:6/593: mknod d1/d8/da/d7b/dae/db6/cd6 0 2024-02-16T19:29:10.919 INFO:tasks.workunit.client.1.smithi117.stdout:3/663: write d1/dd/d3b/d9d/fcd [3001266,6985] 0 2024-02-16T19:29:10.919 INFO:tasks.workunit.client.0.smithi022.stdout:5/574: truncate dd/d1f/f58 666050 0 2024-02-16T19:29:10.920 INFO:tasks.workunit.client.1.smithi117.stdout:2/581: symlink d0/d3/d34/d94/d46/d68/lae 0 2024-02-16T19:29:10.920 INFO:tasks.workunit.client.1.smithi117.stdout:3/664: chown d1/dd/d3e/cd0 4 1 2024-02-16T19:29:10.921 INFO:tasks.workunit.client.0.smithi022.stdout:2/544: write d0/d2/d18/d38/d6b/f8b [3478516,42586] 0 2024-02-16T19:29:10.922 INFO:tasks.workunit.client.1.smithi117.stdout:2/582: chown d0/f12 6 1 2024-02-16T19:29:10.923 INFO:tasks.workunit.client.1.smithi117.stdout:4/555: dread d0/da/d21/f2b [0,4194304] 0 2024-02-16T19:29:10.923 INFO:tasks.workunit.client.1.smithi117.stdout:2/583: write d0/f6c [359521,39341] 0 2024-02-16T19:29:10.924 INFO:tasks.workunit.client.1.smithi117.stdout:8/568: dwrite d4/d7/d6a/f93 [0,4194304] 0 2024-02-16T19:29:10.926 INFO:tasks.workunit.client.1.smithi117.stdout:1/516: getdents d2/d1d/d60/d7c 0 2024-02-16T19:29:10.927 INFO:tasks.workunit.client.1.smithi117.stdout:2/584: truncate d0/d3/d10/f90 981901 0 2024-02-16T19:29:10.930 INFO:tasks.workunit.client.1.smithi117.stdout:0/609: symlink d0/d9/d22/d27/ld9 0 2024-02-16T19:29:10.931 INFO:tasks.workunit.client.1.smithi117.stdout:6/562: write d5/d4b/d20/d2d/f3c [1351205,114806] 0 2024-02-16T19:29:10.931 INFO:tasks.workunit.client.1.smithi117.stdout:6/563: chown d5/db/d7d/d32/c82 1373 1 2024-02-16T19:29:10.936 INFO:tasks.workunit.client.0.smithi022.stdout:6/594: unlink d1/d8/da/d17/d85/f38 0 2024-02-16T19:29:10.938 INFO:tasks.workunit.client.1.smithi117.stdout:9/493: write dd/d47/d5b/d5f/f6f [2368738,35737] 0 2024-02-16T19:29:10.939 INFO:tasks.workunit.client.0.smithi022.stdout:2/545: rmdir d0/d2/d18/d1f/d6c 39 2024-02-16T19:29:10.939 INFO:tasks.workunit.client.0.smithi022.stdout:7/578: dwrite d9/da/d2c/d2f/f3d [0,4194304] 0 2024-02-16T19:29:10.945 INFO:tasks.workunit.client.0.smithi022.stdout:9/609: creat db/d14/d16/d27/fd1 x:0 0 0 2024-02-16T19:29:10.946 INFO:tasks.workunit.client.1.smithi117.stdout:7/569: rename d4/d85/fad to d4/d1c/d39/d46/d55/db8/fd5 0 2024-02-16T19:29:10.947 INFO:tasks.workunit.client.0.smithi022.stdout:0/745: write d10/d15/f61 [401532,30031] 0 2024-02-16T19:29:10.948 INFO:tasks.workunit.client.1.smithi117.stdout:7/570: write d4/d13/d63/d71/fcb [18560,70076] 0 2024-02-16T19:29:10.948 INFO:tasks.workunit.client.1.smithi117.stdout:7/571: chown d4 15671912 1 2024-02-16T19:29:10.949 INFO:tasks.workunit.client.0.smithi022.stdout:0/746: dread - d10/d11/d27/d3f/d58/dab/ff6 zero size 2024-02-16T19:29:10.950 INFO:tasks.workunit.client.0.smithi022.stdout:5/575: mkdir dd/d1f/d48/d21/dc9 0 2024-02-16T19:29:10.950 INFO:tasks.workunit.client.1.smithi117.stdout:7/572: write d4/d85/fcd [459261,114071] 0 2024-02-16T19:29:10.953 INFO:tasks.workunit.client.1.smithi117.stdout:5/627: creat d2/d9/d3c/d24/d50/d52/d74/fd2 x:0 0 0 2024-02-16T19:29:10.957 INFO:tasks.workunit.client.0.smithi022.stdout:6/595: read - d1/d8/d81/f8c zero size 2024-02-16T19:29:10.957 INFO:tasks.workunit.client.1.smithi117.stdout:7/573: dwrite d4/d1c/d98/fa6 [0,4194304] 0 2024-02-16T19:29:10.958 INFO:tasks.workunit.client.0.smithi022.stdout:6/596: dread - d1/d8/d23/f84 zero size 2024-02-16T19:29:10.960 INFO:tasks.workunit.client.0.smithi022.stdout:5/576: dread dd/d1f/d48/d21/d2f/d52/d5c/d61/d6a/f7f [0,4194304] 0 2024-02-16T19:29:10.961 INFO:tasks.workunit.client.1.smithi117.stdout:4/556: mkdir d0/da/d21/db4 0 2024-02-16T19:29:10.961 INFO:tasks.workunit.client.0.smithi022.stdout:2/546: creat d0/d5d/fb0 x:0 0 0 2024-02-16T19:29:10.962 INFO:tasks.workunit.client.0.smithi022.stdout:0/747: dread d10/d11/d79/d8b/fa5 [0,4194304] 0 2024-02-16T19:29:10.962 INFO:tasks.workunit.client.1.smithi117.stdout:8/569: symlink d4/d12/d2a/d64/lb2 0 2024-02-16T19:29:10.963 INFO:tasks.workunit.client.1.smithi117.stdout:8/570: read - d4/d12/d31/f77 zero size 2024-02-16T19:29:10.964 INFO:tasks.workunit.client.0.smithi022.stdout:0/748: write d10/d22/f2f [131292,121197] 0 2024-02-16T19:29:10.967 INFO:tasks.workunit.client.1.smithi117.stdout:4/557: dwrite d0/da/d21/d3d/d28/d66/d97/fa7 [0,4194304] 0 2024-02-16T19:29:10.973 INFO:tasks.workunit.client.0.smithi022.stdout:9/610: dread db/d18/f93 [0,4194304] 0 2024-02-16T19:29:10.975 INFO:tasks.workunit.client.0.smithi022.stdout:6/597: rename d1/d8/d23/c52 to d1/d8/d81/dd4/cd7 0 2024-02-16T19:29:10.979 INFO:tasks.workunit.client.1.smithi117.stdout:3/665: dwrite d1/d1b/d1e/d29/f8b [4194304,4194304] 0 2024-02-16T19:29:10.980 INFO:tasks.workunit.client.1.smithi117.stdout:6/564: rename d5/d4b/d20/d2d/f3c to d5/d4b/d44/d7c/fb9 0 2024-02-16T19:29:10.980 INFO:tasks.workunit.client.1.smithi117.stdout:3/666: fsync d1/dd/d3e/d7d/fbe 0 2024-02-16T19:29:10.981 INFO:tasks.workunit.client.0.smithi022.stdout:2/547: mkdir d0/d2/d18/d19/d40/db1 0 2024-02-16T19:29:10.982 INFO:tasks.workunit.client.1.smithi117.stdout:6/565: fdatasync d5/db/d7d/d32/d95/d49/d59/f99 0 2024-02-16T19:29:10.983 INFO:tasks.workunit.client.0.smithi022.stdout:0/749: creat d10/d22/d2c/ff9 x:0 0 0 2024-02-16T19:29:10.984 INFO:tasks.workunit.client.0.smithi022.stdout:0/750: chown d10/d11/d27/d3f/d58 1419 1 2024-02-16T19:29:10.985 INFO:tasks.workunit.client.0.smithi022.stdout:5/577: dread dd/d1f/d48/d21/d2f/d52/d5c/d61/f92 [0,4194304] 0 2024-02-16T19:29:10.986 INFO:tasks.workunit.client.1.smithi117.stdout:7/574: creat d4/d1c/d39/d9f/fd6 x:0 0 0 2024-02-16T19:29:10.988 INFO:tasks.workunit.client.1.smithi117.stdout:8/571: rmdir d4/d7/d6a/d6f/da8 39 2024-02-16T19:29:10.988 INFO:tasks.workunit.client.0.smithi022.stdout:9/611: mkdir db/d14/d16/d5d/d65/d6b/d89/dd2 0 2024-02-16T19:29:10.989 INFO:tasks.workunit.client.0.smithi022.stdout:6/598: truncate d1/d8/d81/f8c 600907 0 2024-02-16T19:29:10.990 INFO:tasks.workunit.client.0.smithi022.stdout:6/599: chown d1/d8/d81/cc7 0 1 2024-02-16T19:29:10.992 INFO:tasks.workunit.client.0.smithi022.stdout:2/548: symlink d0/d5d/lb2 0 2024-02-16T19:29:10.994 INFO:tasks.workunit.client.1.smithi117.stdout:0/610: creat d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/dbd/fda x:0 0 0 2024-02-16T19:29:10.994 INFO:tasks.workunit.client.0.smithi022.stdout:2/549: fdatasync d0/d32/d3b/f9a 0 2024-02-16T19:29:10.997 INFO:tasks.workunit.client.0.smithi022.stdout:9/612: dwrite db/d14/d16/d87/fb0 [0,4194304] 0 2024-02-16T19:29:11.001 INFO:tasks.workunit.client.1.smithi117.stdout:9/494: mknod dd/d45/caa 0 2024-02-16T19:29:11.002 INFO:tasks.workunit.client.0.smithi022.stdout:6/600: creat d1/d8/da/d17/d53/fd8 x:0 0 0 2024-02-16T19:29:11.006 INFO:tasks.workunit.client.0.smithi022.stdout:2/550: rmdir d0/d2/d18/d19/d40/d55 39 2024-02-16T19:29:11.011 INFO:tasks.workunit.client.1.smithi117.stdout:5/628: mknod d2/d9/d3c/d24/d50/cd3 0 2024-02-16T19:29:11.012 INFO:tasks.workunit.client.1.smithi117.stdout:3/667: read - d1/d1b/d1e/d29/f82 zero size 2024-02-16T19:29:11.013 INFO:tasks.workunit.client.1.smithi117.stdout:3/668: dread - d1/dd/d5c/d85/fd9 zero size 2024-02-16T19:29:11.016 INFO:tasks.workunit.client.0.smithi022.stdout:2/551: rmdir d0/d2/d2e/d4a 39 2024-02-16T19:29:11.017 INFO:tasks.workunit.client.0.smithi022.stdout:9/613: dwrite db/d14/d16/d5d/f7e [0,4194304] 0 2024-02-16T19:29:11.018 INFO:tasks.workunit.client.1.smithi117.stdout:6/566: truncate d5/db/f6f 1250846 0 2024-02-16T19:29:11.019 INFO:tasks.workunit.client.0.smithi022.stdout:9/614: fdatasync db/d14/d16/d5d/d65/d6b/faf 0 2024-02-16T19:29:11.020 INFO:tasks.workunit.client.0.smithi022.stdout:0/751: write d10/d11/d27/fa4 [550574,17698] 0 2024-02-16T19:29:11.021 INFO:tasks.workunit.client.0.smithi022.stdout:0/752: chown d10/d15/d23/d8f/dc1 2 1 2024-02-16T19:29:11.021 INFO:tasks.workunit.client.1.smithi117.stdout:3/669: dread d1/d1b/f41 [0,4194304] 0 2024-02-16T19:29:11.024 INFO:tasks.workunit.client.0.smithi022.stdout:6/601: link d1/d8/da/d17/f99 d1/d8/d81/fd9 0 2024-02-16T19:29:11.026 INFO:tasks.workunit.client.1.smithi117.stdout:8/572: symlink d4/d94/lb3 0 2024-02-16T19:29:11.027 INFO:tasks.workunit.client.1.smithi117.stdout:0/611: rmdir d0/d9/d14/d1d/d60 39 2024-02-16T19:29:11.028 INFO:tasks.workunit.client.0.smithi022.stdout:2/552: chown d0/d32/l5b 12259 1 2024-02-16T19:29:11.030 INFO:tasks.workunit.client.1.smithi117.stdout:9/495: mknod dd/d47/d5b/d5f/cab 0 2024-02-16T19:29:11.032 INFO:tasks.workunit.client.0.smithi022.stdout:6/602: symlink d1/d8/da/d17/d53/lda 0 2024-02-16T19:29:11.033 INFO:tasks.workunit.client.0.smithi022.stdout:5/578: dwrite dd/d1f/d48/d21/d2f/d52/d5c/d61/d6a/f70 [4194304,4194304] 0 2024-02-16T19:29:11.033 INFO:tasks.workunit.client.0.smithi022.stdout:2/553: creat d0/d2/fb3 x:0 0 0 2024-02-16T19:29:11.035 INFO:tasks.workunit.client.0.smithi022.stdout:6/603: dread - d1/d8/d3c/d4d/fc2 zero size 2024-02-16T19:29:11.037 INFO:tasks.workunit.client.0.smithi022.stdout:0/753: creat d10/d7d/dac/de9/dee/ffa x:0 0 0 2024-02-16T19:29:11.038 INFO:tasks.workunit.client.0.smithi022.stdout:6/604: read - d1/d8/d23/f84 zero size 2024-02-16T19:29:11.039 INFO:tasks.workunit.client.1.smithi117.stdout:3/670: creat d1/d1b/d20/fe5 x:0 0 0 2024-02-16T19:29:11.040 INFO:tasks.workunit.client.0.smithi022.stdout:6/605: write d1/d8/da/d29/f3f [9377497,103236] 0 2024-02-16T19:29:11.041 INFO:tasks.workunit.client.1.smithi117.stdout:8/573: rmdir d4/d1c 39 2024-02-16T19:29:11.042 INFO:tasks.workunit.client.1.smithi117.stdout:8/574: chown d4/l16 247494 1 2024-02-16T19:29:11.044 INFO:tasks.workunit.client.0.smithi022.stdout:2/554: rename d0/d2/d18/d19 to d0/d2/d18/d19/d40/d55/d59/db4 22 2024-02-16T19:29:11.046 INFO:tasks.workunit.client.0.smithi022.stdout:0/754: rmdir d10/d15/d23/d8f 39 2024-02-16T19:29:11.049 INFO:tasks.workunit.client.0.smithi022.stdout:5/579: creat dd/d1f/d48/d21/dc9/fca x:0 0 0 2024-02-16T19:29:11.050 INFO:tasks.workunit.client.1.smithi117.stdout:5/629: mkdir d2/d9/d3c/d24/d90/dd4 0 2024-02-16T19:29:11.052 INFO:tasks.workunit.client.1.smithi117.stdout:9/496: truncate dd/d23/d64/f61 1024141 0 2024-02-16T19:29:11.055 INFO:tasks.workunit.client.0.smithi022.stdout:2/555: truncate d0/d2/d18/d19/d40/f6d 1159790 0 2024-02-16T19:29:11.056 INFO:tasks.workunit.client.0.smithi022.stdout:0/755: mknod d10/d11/d27/d3f/d58/dab/cfb 0 2024-02-16T19:29:11.057 INFO:tasks.workunit.client.1.smithi117.stdout:6/567: dwrite d5/db/d7d/d32/f5b [0,4194304] 0 2024-02-16T19:29:11.057 INFO:tasks.workunit.client.1.smithi117.stdout:6/568: chown d5/db/d7d/d32/d95/d49 22 1 2024-02-16T19:29:11.058 INFO:tasks.workunit.client.1.smithi117.stdout:7/575: link d4/d1c/d39/d9f/fd6 d4/d1c/d87/d89/d60/d79/fd7 0 2024-02-16T19:29:11.059 INFO:tasks.workunit.client.1.smithi117.stdout:7/576: write d4/d13/d63/fce [45595,114591] 0 2024-02-16T19:29:11.060 INFO:tasks.workunit.client.1.smithi117.stdout:3/671: unlink d1/d1b/d1e/d37/f6d 0 2024-02-16T19:29:11.060 INFO:tasks.workunit.client.1.smithi117.stdout:3/672: stat d1/d98 0 2024-02-16T19:29:11.060 INFO:tasks.workunit.client.0.smithi022.stdout:6/606: mkdir d1/d8/d23/dc6/ddb 0 2024-02-16T19:29:11.061 INFO:tasks.workunit.client.0.smithi022.stdout:2/556: dwrite d0/d2/d18/d1f/f5e [0,4194304] 0 2024-02-16T19:29:11.062 INFO:tasks.workunit.client.1.smithi117.stdout:8/575: mknod d4/d7/da6/d3a/d65/cb4 0 2024-02-16T19:29:11.065 INFO:tasks.workunit.client.0.smithi022.stdout:0/756: mkdir d10/d7d/dac/de9/dfc 0 2024-02-16T19:29:11.065 INFO:tasks.workunit.client.1.smithi117.stdout:0/612: fsync d0/d9/d14/d23/d33/f5c 0 2024-02-16T19:29:11.068 INFO:tasks.workunit.client.1.smithi117.stdout:4/558: getdents d0/da/d21/d3d/d28/d64/d48 0 2024-02-16T19:29:11.068 INFO:tasks.workunit.client.1.smithi117.stdout:4/559: read d0/da/d21/f3e [3532782,74770] 0 2024-02-16T19:29:11.069 INFO:tasks.workunit.client.1.smithi117.stdout:4/560: chown d0/da/d21/d3d/d28/d64/d8a/d92/f89 321580 1 2024-02-16T19:29:11.072 INFO:tasks.workunit.client.1.smithi117.stdout:9/497: creat dd/d47/d5b/d5f/d77/fac x:0 0 0 2024-02-16T19:29:11.075 INFO:tasks.workunit.client.1.smithi117.stdout:6/569: creat d5/db/d7d/d32/d79/fba x:0 0 0 2024-02-16T19:29:11.076 INFO:tasks.workunit.client.1.smithi117.stdout:6/570: read d5/d4b/d20/f3a [25212,20025] 0 2024-02-16T19:29:11.076 INFO:tasks.workunit.client.1.smithi117.stdout:3/673: dread d1/d1b/d1e/d37/f65 [0,4194304] 0 2024-02-16T19:29:11.082 INFO:tasks.workunit.client.1.smithi117.stdout:4/561: symlink d0/da/d21/d3d/d28/d64/d8a/lb5 0 2024-02-16T19:29:11.084 INFO:tasks.workunit.client.1.smithi117.stdout:5/630: rename d2/d5d/d8e/dc1/fd1 to d2/d4c/d4f/d67/fd5 0 2024-02-16T19:29:11.085 INFO:tasks.workunit.client.1.smithi117.stdout:5/631: chown d2/d3/d21/d49/d59/c80 27 1 2024-02-16T19:29:11.086 INFO:tasks.workunit.client.1.smithi117.stdout:9/498: mknod dd/d47/d48/cad 0 2024-02-16T19:29:11.093 INFO:tasks.workunit.client.1.smithi117.stdout:8/576: symlink d4/d7/d6a/d6f/da8/lb5 0 2024-02-16T19:29:11.093 INFO:tasks.workunit.client.1.smithi117.stdout:8/577: chown d4/d12/l18 0 1 2024-02-16T19:29:11.094 INFO:tasks.workunit.client.1.smithi117.stdout:3/674: truncate d1/d31/f8c 3402583 0 2024-02-16T19:29:11.094 INFO:tasks.workunit.client.1.smithi117.stdout:1/517: sync 2024-02-16T19:29:11.095 INFO:tasks.workunit.client.1.smithi117.stdout:2/585: sync 2024-02-16T19:29:11.095 INFO:tasks.workunit.client.1.smithi117.stdout:1/518: chown d2 35 1 2024-02-16T19:29:11.096 INFO:tasks.workunit.client.1.smithi117.stdout:2/586: write d0/d3/d34/d94/f6d [3964333,45760] 0 2024-02-16T19:29:11.098 INFO:tasks.workunit.client.1.smithi117.stdout:5/632: dread - d2/d9/d3c/d24/d50/d96/dbc/d30/d5a/fa1 zero size 2024-02-16T19:29:11.099 INFO:tasks.workunit.client.1.smithi117.stdout:9/499: symlink dd/d47/d5e/lae 0 2024-02-16T19:29:11.100 INFO:tasks.workunit.client.1.smithi117.stdout:8/578: rmdir d4/d7/d6a/d6f/da8 39 2024-02-16T19:29:11.103 INFO:tasks.workunit.client.0.smithi022.stdout:5/580: truncate dd/d1f/d76/fb1 539622 0 2024-02-16T19:29:11.104 INFO:tasks.workunit.client.0.smithi022.stdout:6/607: mkdir d1/ddc 0 2024-02-16T19:29:11.106 INFO:tasks.workunit.client.0.smithi022.stdout:6/608: write d1/fba [497997,62839] 0 2024-02-16T19:29:11.107 INFO:tasks.workunit.client.0.smithi022.stdout:6/609: chown d1/d8/da/d17/d53 77461924 1 2024-02-16T19:29:11.110 INFO:tasks.workunit.client.0.smithi022.stdout:0/757: symlink d10/d7d/dac/de9/dee/lfd 0 2024-02-16T19:29:11.111 INFO:tasks.workunit.client.1.smithi117.stdout:7/577: truncate d4/d1c/d39/d46/d55/db8/fc2 54992 0 2024-02-16T19:29:11.112 INFO:tasks.workunit.client.0.smithi022.stdout:2/557: dread d0/f2d [0,4194304] 0 2024-02-16T19:29:11.113 INFO:tasks.workunit.client.1.smithi117.stdout:6/571: dwrite d5/d4b/d44/f56 [0,4194304] 0 2024-02-16T19:29:11.114 INFO:tasks.workunit.client.1.smithi117.stdout:7/578: read - d4/d1c/d87/d89/d60/d79/f7a zero size 2024-02-16T19:29:11.115 INFO:tasks.workunit.client.1.smithi117.stdout:9/500: fsync dd/f4a 0 2024-02-16T19:29:11.115 INFO:tasks.workunit.client.1.smithi117.stdout:7/579: chown d4/d13/da3/fb1 162443 1 2024-02-16T19:29:11.116 INFO:tasks.workunit.client.0.smithi022.stdout:5/581: readlink dd/d1f/d48/d21/d2f/d46/d5e/d84/l81 0 2024-02-16T19:29:11.116 INFO:tasks.workunit.client.0.smithi022.stdout:6/610: dwrite d1/d8/da/d17/d48/d51/f59 [0,4194304] 0 2024-02-16T19:29:11.117 INFO:tasks.workunit.client.1.smithi117.stdout:7/580: dread - d4/d1c/d39/d9f/fd6 zero size 2024-02-16T19:29:11.118 INFO:tasks.workunit.client.1.smithi117.stdout:8/579: mknod d4/d7/da6/d3a/d65/d96/cb6 0 2024-02-16T19:29:11.119 INFO:tasks.workunit.client.1.smithi117.stdout:7/581: chown d4/d1c/d39/d46/f57 745368 1 2024-02-16T19:29:11.122 INFO:tasks.workunit.client.1.smithi117.stdout:2/587: unlink d0/d3/d34/d94/d46/d68/lab 0 2024-02-16T19:29:11.124 INFO:tasks.workunit.client.1.smithi117.stdout:4/562: link d0/da/d21/d3d/d28/f4e d0/da/d21/d3d/d28/d64/d73/d91/d9e/fb6 0 2024-02-16T19:29:11.128 INFO:tasks.workunit.client.1.smithi117.stdout:8/580: dwrite d4/d12/d2a/d64/f7d [0,4194304] 0 2024-02-16T19:29:11.128 INFO:tasks.workunit.client.0.smithi022.stdout:5/582: dwrite dd/f11 [4194304,4194304] 0 2024-02-16T19:29:11.130 INFO:tasks.workunit.client.0.smithi022.stdout:0/758: link d10/d22/d49/ff8 d10/d22/ffe 0 2024-02-16T19:29:11.132 INFO:tasks.workunit.client.0.smithi022.stdout:0/759: stat d10/d15/d23/d38/d5a/d66 0 2024-02-16T19:29:11.134 INFO:tasks.workunit.client.1.smithi117.stdout:4/563: dwrite d0/da/d21/d3d/d28/fa6 [0,4194304] 0 2024-02-16T19:29:11.135 INFO:tasks.workunit.client.1.smithi117.stdout:6/572: symlink d5/db/d7d/d32/d79/lbb 0 2024-02-16T19:29:11.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:11.138+0000 7fdef6048700 1 -- 172.21.15.22:0/347594203 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fdef01999b0 con 0x7fdef01003d0 2024-02-16T19:29:11.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:11.138+0000 7fdeed7fa700 1 -- 172.21.15.22:0/347594203 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+766 (secure 0 0 0) 0x7fded805d4f0 con 0x7fdef01003d0 2024-02-16T19:29:11.139 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-02-16T19:29:11.139 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": { 2024-02-16T19:29:11.139 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-02-16T19:29:11.139 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:29:11.140 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": { 2024-02-16T19:29:11.140 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-16T19:29:11.140 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 1 2024-02-16T19:29:11.140 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:29:11.140 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": { 2024-02-16T19:29:11.140 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-02-16T19:29:11.140 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:29:11.140 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": { 2024-02-16T19:29:11.140 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-02-16T19:29:11.140 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:29:11.140 INFO:teuthology.orchestra.run.smithi022.stdout: "overall": { 2024-02-16T19:29:11.140 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 13, 2024-02-16T19:29:11.140 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 1 2024-02-16T19:29:11.140 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-02-16T19:29:11.140 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-02-16T19:29:11.141 INFO:tasks.workunit.client.0.smithi022.stdout:5/583: rename dd/d1f/d48/d21/d64/faf to dd/d1f/d48/d21/d2f/d46/d5e/d84/d8b/fcb 0 2024-02-16T19:29:11.142 INFO:tasks.workunit.client.0.smithi022.stdout:0/760: creat d10/d22/dad/dd7/fff x:0 0 0 2024-02-16T19:29:11.143 INFO:tasks.workunit.client.1.smithi117.stdout:5/633: dwrite d2/db3/f55 [0,4194304] 0 2024-02-16T19:29:11.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:11.141+0000 7fdef6048700 1 -- 172.21.15.22:0/347594203 >> [v2:172.21.15.117:6828/10500008,v1:172.21.15.117:6829/10500008] conn(0x7fded006cf00 msgr2=0x7fded006f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:29:11.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:11.141+0000 7fdef6048700 1 --2- 172.21.15.22:0/347594203 >> [v2:172.21.15.117:6828/10500008,v1:172.21.15.117:6829/10500008] conn(0x7fded006cf00 0x7fded006f3b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fdee0006010 tx=0x7fdee0005a90 comp rx=0 tx=0).stop 2024-02-16T19:29:11.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:11.141+0000 7fdef6048700 1 -- 172.21.15.22:0/347594203 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdef01003d0 msgr2=0x7fdef0193d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:29:11.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:11.141+0000 7fdef6048700 1 --2- 172.21.15.22:0/347594203 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdef01003d0 0x7fdef0193d80 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fded8010d00 tx=0x7fded800b5a0 comp rx=0 tx=0).stop 2024-02-16T19:29:11.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:11.142+0000 7fdef6048700 1 -- 172.21.15.22:0/347594203 shutdown_connections 2024-02-16T19:29:11.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:11.142+0000 7fdef6048700 1 --2- 172.21.15.22:0/347594203 >> [v2:172.21.15.117:6828/10500008,v1:172.21.15.117:6829/10500008] conn(0x7fded006cf00 0x7fded006f3b0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:11.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:11.142+0000 7fdef6048700 1 --2- 172.21.15.22:0/347594203 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdef00ff130 0x7fdef0193840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:11.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:11.142+0000 7fdef6048700 1 --2- 172.21.15.22:0/347594203 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdef01003d0 0x7fdef0193d80 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:11.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:11.142+0000 7fdef6048700 1 -- 172.21.15.22:0/347594203 >> 172.21.15.22:0/347594203 conn(0x7fdef00fa780 msgr2=0x7fdef0103600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:29:11.144 INFO:tasks.workunit.client.0.smithi022.stdout:0/761: dread - d10/d11/d50/fc3 zero size 2024-02-16T19:29:11.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:11.142+0000 7fdef6048700 1 -- 172.21.15.22:0/347594203 shutdown_connections 2024-02-16T19:29:11.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:11.142+0000 7fdef6048700 1 -- 172.21.15.22:0/347594203 wait complete. 2024-02-16T19:29:11.145 INFO:tasks.workunit.client.0.smithi022.stdout:0/762: write d10/d11/d79/d8b/fc0 [2963907,108636] 0 2024-02-16T19:29:11.146 INFO:tasks.workunit.client.0.smithi022.stdout:0/763: write d10/d11/d27/fbb [581342,100035] 0 2024-02-16T19:29:11.146 INFO:tasks.workunit.client.1.smithi117.stdout:0/613: getdents d0/d9/d14/d1d/d60/d76/d7b/d8d/d55 0 2024-02-16T19:29:11.149 INFO:tasks.workunit.client.1.smithi117.stdout:3/675: rename d1/dd/d5c/d85/l91 to d1/d1b/d1e/le6 0 2024-02-16T19:29:11.151 INFO:tasks.workunit.client.0.smithi022.stdout:5/584: unlink dd/d1f/d4a/d7a/l9c 0 2024-02-16T19:29:11.154 INFO:tasks.workunit.client.1.smithi117.stdout:2/588: creat d0/d13/d4d/d53/d58/faf x:0 0 0 2024-02-16T19:29:11.155 INFO:tasks.workunit.client.1.smithi117.stdout:8/581: stat d4/d7/c24 0 2024-02-16T19:29:11.156 INFO:tasks.workunit.client.1.smithi117.stdout:7/582: dwrite d4/d13/d63/d71/f90 [0,4194304] 0 2024-02-16T19:29:11.157 INFO:tasks.workunit.client.0.smithi022.stdout:2/558: getdents d0/d2/d2e/d82/d8a 0 2024-02-16T19:29:11.157 INFO:tasks.workunit.client.1.smithi117.stdout:2/589: write d0/d3/d22/f84 [2557160,48561] 0 2024-02-16T19:29:11.159 INFO:tasks.workunit.client.1.smithi117.stdout:2/590: chown d0/d3/d34/d94/d46/d68/d76/da1/d56/d60 465148 1 2024-02-16T19:29:11.159 INFO:tasks.workunit.client.1.smithi117.stdout:4/564: unlink d0/da/d21/d3d/d28/d64/fb3 0 2024-02-16T19:29:11.161 INFO:tasks.workunit.client.1.smithi117.stdout:4/565: chown d0/da/d21/d3d/d28/d64 1319681 1 2024-02-16T19:29:11.163 INFO:tasks.workunit.client.0.smithi022.stdout:6/611: getdents d1/d8/da/d17/d53 0 2024-02-16T19:29:11.163 INFO:tasks.workunit.client.0.smithi022.stdout:6/612: read - d1/d8/da/d17/d85/d56/fc4 zero size 2024-02-16T19:29:11.164 INFO:tasks.workunit.client.1.smithi117.stdout:7/583: dread d4/d1c/d39/d46/f62 [0,4194304] 0 2024-02-16T19:29:11.165 INFO:tasks.workunit.client.1.smithi117.stdout:9/501: mkdir dd/d23/daf 0 2024-02-16T19:29:11.167 INFO:tasks.workunit.client.0.smithi022.stdout:2/559: mknod d0/d2/d18/d38/cb5 0 2024-02-16T19:29:11.168 INFO:tasks.workunit.client.0.smithi022.stdout:2/560: readlink d0/d2/d18/l42 0 2024-02-16T19:29:11.168 INFO:tasks.workunit.client.1.smithi117.stdout:4/566: chown d0/da/d21/d3d/d28/d66/d97/lb2 397 1 2024-02-16T19:29:11.169 INFO:tasks.workunit.client.1.smithi117.stdout:5/634: mknod d2/d9/dba/cd6 0 2024-02-16T19:29:11.169 INFO:tasks.workunit.client.1.smithi117.stdout:2/591: dwrite d0/d13/d4d/d53/d58/faf [0,4194304] 0 2024-02-16T19:29:11.171 INFO:tasks.workunit.client.1.smithi117.stdout:4/567: write d0/f5c [1587134,22791] 0 2024-02-16T19:29:11.171 INFO:tasks.workunit.client.1.smithi117.stdout:2/592: chown d0/d3/d34/d94/d91/f98 10172 1 2024-02-16T19:29:11.174 INFO:tasks.workunit.client.0.smithi022.stdout:5/585: read dd/d1f/d48/d21/d2f/d46/d5e/fa6 [1825186,15208] 0 2024-02-16T19:29:11.178 INFO:tasks.workunit.client.0.smithi022.stdout:6/613: mkdir d1/d8/d3c/d4d/ddd 0 2024-02-16T19:29:11.179 INFO:tasks.workunit.client.1.smithi117.stdout:3/676: rmdir d1/dd/d8e 39 2024-02-16T19:29:11.180 INFO:tasks.workunit.client.0.smithi022.stdout:6/614: fdatasync d1/d8/da/d17/d85/d5b/fa4 0 2024-02-16T19:29:11.186 INFO:tasks.workunit.client.1.smithi117.stdout:3/677: dwrite d1/d31/d89/dbd/fca [0,4194304] 0 2024-02-16T19:29:11.187 INFO:tasks.workunit.client.0.smithi022.stdout:5/586: creat dd/d1f/d48/d21/d2f/fcc x:0 0 0 2024-02-16T19:29:11.190 INFO:tasks.workunit.client.0.smithi022.stdout:2/561: dwrite d0/d2/f57 [4194304,4194304] 0 2024-02-16T19:29:11.193 INFO:tasks.workunit.client.1.smithi117.stdout:6/573: dwrite d5/d4b/f46 [0,4194304] 0 2024-02-16T19:29:11.195 INFO:tasks.workunit.client.0.smithi022.stdout:5/587: read dd/d1f/d48/d21/d2f/d40/f3b [365185,110050] 0 2024-02-16T19:29:11.201 INFO:tasks.workunit.client.0.smithi022.stdout:2/562: dwrite d0/d2/d18/f65 [0,4194304] 0 2024-02-16T19:29:11.202 INFO:tasks.workunit.client.0.smithi022.stdout:6/615: rename d1/d8/da/d17/d85/d56/f90 to d1/d8/da/d7b/dae/dbb/fde 0 2024-02-16T19:29:11.204 INFO:tasks.workunit.client.0.smithi022.stdout:6/616: write d1/d8/f40 [1062754,51002] 0 2024-02-16T19:29:11.209 INFO:tasks.workunit.client.1.smithi117.stdout:9/502: mknod dd/d47/d5b/d5f/d77/d7c/cb0 0 2024-02-16T19:29:11.211 INFO:tasks.workunit.client.1.smithi117.stdout:0/614: dwrite d0/f1 [0,4194304] 0 2024-02-16T19:29:11.211 INFO:tasks.workunit.client.0.smithi022.stdout:5/588: mkdir dd/d1f/d48/d21/d2f/d40/d4e/d56/dcd 0 2024-02-16T19:29:11.212 INFO:tasks.workunit.client.0.smithi022.stdout:6/617: dwrite d1/d8/f3b [0,4194304] 0 2024-02-16T19:29:11.219 INFO:tasks.workunit.client.1.smithi117.stdout:2/593: creat d0/d3/d34/d94/d46/d68/d76/da1/fb0 x:0 0 0 2024-02-16T19:29:11.222 INFO:tasks.workunit.client.0.smithi022.stdout:5/589: unlink dd/d1f/d48/d21/l33 0 2024-02-16T19:29:11.224 INFO:tasks.workunit.client.1.smithi117.stdout:8/582: truncate d4/f3f 879683 0 2024-02-16T19:29:11.224 INFO:tasks.workunit.client.1.smithi117.stdout:4/568: dwrite d0/da/f5b [0,4194304] 0 2024-02-16T19:29:11.225 INFO:tasks.workunit.client.0.smithi022.stdout:6/618: truncate d1/d8/da/d17/d85/f5f 397014 0 2024-02-16T19:29:11.226 INFO:tasks.workunit.client.1.smithi117.stdout:8/583: fdatasync d4/d73/f82 0 2024-02-16T19:29:11.229 INFO:tasks.workunit.client.1.smithi117.stdout:3/678: creat d1/d30/d9f/dcb/fe7 x:0 0 0 2024-02-16T19:29:11.234 INFO:tasks.workunit.client.0.smithi022.stdout:6/619: unlink d1/d8/da/d17/d53/fd8 0 2024-02-16T19:29:11.240 INFO:tasks.workunit.client.0.smithi022.stdout:6/620: truncate d1/f72 896848 0 2024-02-16T19:29:11.242 INFO:tasks.workunit.client.1.smithi117.stdout:0/615: mknod d0/d9/d22/d27/d31/d94/cdb 0 2024-02-16T19:29:11.244 INFO:tasks.workunit.client.1.smithi117.stdout:1/519: rename d2/d4/f40 to d2/d19/fac 0 2024-02-16T19:29:11.245 INFO:tasks.workunit.client.1.smithi117.stdout:0/616: dread - d0/d9/d14/d1d/d60/d76/d7b/d8d/fd3 zero size 2024-02-16T19:29:11.248 INFO:tasks.workunit.client.1.smithi117.stdout:9/503: link dd/f19 dd/d45/d62/fb1 0 2024-02-16T19:29:11.251 INFO:tasks.workunit.client.1.smithi117.stdout:1/520: mkdir d2/d19/d59/d89/d69/dad 0 2024-02-16T19:29:11.251 INFO:tasks.workunit.client.1.smithi117.stdout:5/635: getdents d2/d9/d3c/d24/d50/d52/d74/d8c 0 2024-02-16T19:29:11.252 INFO:tasks.workunit.client.1.smithi117.stdout:1/521: chown d2/d19/d59/d76 1 1 2024-02-16T19:29:11.252 INFO:tasks.workunit.client.1.smithi117.stdout:7/584: dread d4/d1c/f37 [0,4194304] 0 2024-02-16T19:29:11.254 INFO:tasks.workunit.client.1.smithi117.stdout:4/569: getdents d0/da/d21/d3d/d28/d64/d8a/d92 0 2024-02-16T19:29:11.255 INFO:tasks.workunit.client.1.smithi117.stdout:5/636: creat d2/d4c/fd7 x:0 0 0 2024-02-16T19:29:11.255 INFO:tasks.workunit.client.1.smithi117.stdout:1/522: mknod d2/d19/d59/d89/d69/cae 0 2024-02-16T19:29:11.258 INFO:tasks.workunit.client.1.smithi117.stdout:4/570: creat d0/da/d21/d3d/d28/fb7 x:0 0 0 2024-02-16T19:29:11.259 INFO:tasks.workunit.client.1.smithi117.stdout:1/523: fdatasync d2/d4/f47 0 2024-02-16T19:29:11.265 INFO:tasks.workunit.client.1.smithi117.stdout:9/504: dwrite dd/d47/d5b/d5f/d77/d7c/fa2 [0,4194304] 0 2024-02-16T19:29:11.266 INFO:tasks.workunit.client.1.smithi117.stdout:5/637: dread d2/d9/f18 [4194304,4194304] 0 2024-02-16T19:29:11.270 INFO:tasks.workunit.client.0.smithi022.stdout:5/590: truncate dd/d1f/d48/d21/d2f/d46/d5e/d84/fa3 696626 0 2024-02-16T19:29:11.274 INFO:tasks.workunit.client.1.smithi117.stdout:9/505: fdatasync dd/d45/d62/fb1 0 2024-02-16T19:29:11.276 INFO:tasks.workunit.client.1.smithi117.stdout:5/638: symlink d2/d9/dba/ld8 0 2024-02-16T19:29:11.276 INFO:tasks.workunit.client.0.smithi022.stdout:5/591: dread dd/d1f/d48/d21/d2f/d52/d5c/d61/f92 [4194304,4194304] 0 2024-02-16T19:29:11.277 INFO:tasks.workunit.client.0.smithi022.stdout:5/592: write dd/d1f/d48/d21/d2f/d46/d5e/f89 [4510717,100386] 0 2024-02-16T19:29:11.278 INFO:tasks.workunit.client.1.smithi117.stdout:1/524: mknod d2/dab/caf 0 2024-02-16T19:29:11.279 INFO:tasks.workunit.client.0.smithi022.stdout:5/593: write dd/d1f/d48/d21/d2f/d40/f91 [3146061,121253] 0 2024-02-16T19:29:11.280 INFO:tasks.workunit.client.1.smithi117.stdout:4/571: truncate d0/da/d21/d3d/d28/d64/d73/d91/d9e/fb6 3861978 0 2024-02-16T19:29:11.281 INFO:tasks.workunit.client.0.smithi022.stdout:2/563: write d0/f20 [1601299,45037] 0 2024-02-16T19:29:11.282 INFO:tasks.workunit.client.1.smithi117.stdout:9/506: mkdir dd/d47/d5e/d9a/db2 0 2024-02-16T19:29:11.286 INFO:tasks.workunit.client.1.smithi117.stdout:3/679: dread d1/dd/d32/fc3 [4194304,4194304] 0 2024-02-16T19:29:11.287 INFO:tasks.workunit.client.1.smithi117.stdout:5/639: symlink d2/d9/d3c/d24/d50/d96/dbc/d30/d38/d51/ld9 0 2024-02-16T19:29:11.289 INFO:tasks.workunit.client.1.smithi117.stdout:0/617: dread d0/d9/d22/d27/d31/f38 [0,4194304] 0 2024-02-16T19:29:11.291 INFO:tasks.workunit.client.1.smithi117.stdout:4/572: stat d0/da/c20 0 2024-02-16T19:29:11.297 INFO:tasks.workunit.client.1.smithi117.stdout:1/525: mkdir d2/d1d/d60/d7c/d9e/da5/db0 0 2024-02-16T19:29:11.298 INFO:tasks.workunit.client.1.smithi117.stdout:1/526: chown d2/d19/d59/d89/f42 19138912 1 2024-02-16T19:29:11.299 INFO:tasks.workunit.client.1.smithi117.stdout:2/594: write d0/d3/d10/f1d [4690702,43057] 0 2024-02-16T19:29:11.300 INFO:tasks.workunit.client.1.smithi117.stdout:8/584: write d4/d7/da6/f63 [310213,23016] 0 2024-02-16T19:29:11.300 INFO:tasks.workunit.client.1.smithi117.stdout:8/585: stat d4/d12/d2a/f5c 0 2024-02-16T19:29:11.302 INFO:tasks.workunit.client.0.smithi022.stdout:5/594: getdents dd/d1f/d48/d21 0 2024-02-16T19:29:11.304 INFO:tasks.workunit.client.0.smithi022.stdout:2/564: getdents d0/d32 0 2024-02-16T19:29:11.306 INFO:tasks.workunit.client.0.smithi022.stdout:5/595: write dd/d1f/d48/f2d [1713163,33482] 0 2024-02-16T19:29:11.308 INFO:tasks.workunit.client.0.smithi022.stdout:5/596: chown dd 0 1 2024-02-16T19:29:11.308 INFO:tasks.workunit.client.1.smithi117.stdout:0/618: truncate d0/d9/d2c/da9/faa 869228 0 2024-02-16T19:29:11.308 INFO:tasks.workunit.client.1.smithi117.stdout:6/574: dwrite d5/d9c/f9e [0,4194304] 0 2024-02-16T19:29:11.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:11.308+0000 7f36a247a700 1 -- 172.21.15.22:0/2760268447 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f369c100750 msgr2=0x7f369c100b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:29:11.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:11.308+0000 7f36a247a700 1 --2- 172.21.15.22:0/2760268447 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f369c100750 0x7f369c100b60 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f368c009910 tx=0x7f368c009c20 comp rx=0 tx=0).stop 2024-02-16T19:29:11.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:11.308+0000 7f36a247a700 1 -- 172.21.15.22:0/2760268447 shutdown_connections 2024-02-16T19:29:11.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:11.308+0000 7f36a247a700 1 --2- 172.21.15.22:0/2760268447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f369c1010a0 0x7f369c0ff4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:11.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:11.308+0000 7f36a247a700 1 --2- 172.21.15.22:0/2760268447 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f369c100750 0x7f369c100b60 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:11.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:11.308+0000 7f36a247a700 1 -- 172.21.15.22:0/2760268447 >> 172.21.15.22:0/2760268447 conn(0x7f369c0f9f10 msgr2=0x7f369c0fc340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:29:11.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:11.309+0000 7f36a247a700 1 -- 172.21.15.22:0/2760268447 shutdown_connections 2024-02-16T19:29:11.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:11.309+0000 7f36a247a700 1 -- 172.21.15.22:0/2760268447 wait complete. 2024-02-16T19:29:11.310 INFO:tasks.workunit.client.1.smithi117.stdout:4/573: dwrite d0/da/d4a/f58 [0,4194304] 0 2024-02-16T19:29:11.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:11.309+0000 7f36a247a700 1 Processor -- start 2024-02-16T19:29:11.311 INFO:tasks.workunit.client.1.smithi117.stdout:8/586: dread d4/d7/d6a/f93 [0,4194304] 0 2024-02-16T19:29:11.311 INFO:tasks.workunit.client.1.smithi117.stdout:2/595: mknod d0/d13/d4d/d53/d58/cb1 0 2024-02-16T19:29:11.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:11.310+0000 7f36a247a700 1 -- start start 2024-02-16T19:29:11.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:11.310+0000 7f36a247a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f369c100750 0x7f369c190d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:29:11.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:11.310+0000 7f36a247a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f369c1010a0 0x7f369c191270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:29:11.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:11.310+0000 7f36a247a700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f369c18d820 con 0x7f369c1010a0 2024-02-16T19:29:11.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:11.310+0000 7f36a247a700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f369c18d990 con 0x7f369c100750 2024-02-16T19:29:11.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:11.310+0000 7f369bfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f369c100750 0x7f369c190d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:29:11.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:11.310+0000 7f369bfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f369c100750 0x7f369c190d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.22:50130/0 (socket says 172.21.15.22:50130) 2024-02-16T19:29:11.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:11.310+0000 7f369bfff700 1 -- 172.21.15.22:0/4141552039 learned_addr learned my addr 172.21.15.22:0/4141552039 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:29:11.313 INFO:tasks.workunit.client.0.smithi022.stdout:2/565: dwrite d0/d32/d3b/d6f/fac [0,4194304] 0 2024-02-16T19:29:11.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:11.311+0000 7f369b7fe700 1 --2- 172.21.15.22:0/4141552039 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f369c1010a0 0x7f369c191270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:29:11.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:11.311+0000 7f369bfff700 1 -- 172.21.15.22:0/4141552039 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f369c1010a0 msgr2=0x7f369c191270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:29:11.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:11.311+0000 7f369bfff700 1 --2- 172.21.15.22:0/4141552039 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f369c1010a0 0x7f369c191270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:11.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:11.311+0000 7f369bfff700 1 -- 172.21.15.22:0/4141552039 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f368c0095f0 con 0x7f369c100750 2024-02-16T19:29:11.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:11.312+0000 7f369bfff700 1 --2- 172.21.15.22:0/4141552039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f369c100750 0x7f369c190d30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f368c00bd70 tx=0x7f368c00be50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:29:11.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:11.313+0000 7f36997fa700 1 -- 172.21.15.22:0/4141552039 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f368c01d070 con 0x7f369c100750 2024-02-16T19:29:11.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:11.313+0000 7f36a247a700 1 -- 172.21.15.22:0/4141552039 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f369c18dc10 con 0x7f369c100750 2024-02-16T19:29:11.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:11.313+0000 7f36a247a700 1 -- 172.21.15.22:0/4141552039 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f369c18e100 con 0x7f369c100750 2024-02-16T19:29:11.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:11.313+0000 7f36997fa700 1 -- 172.21.15.22:0/4141552039 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f368c004be0 con 0x7f369c100750 2024-02-16T19:29:11.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:11.313+0000 7f36997fa700 1 -- 172.21.15.22:0/4141552039 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f368c021640 con 0x7f369c100750 2024-02-16T19:29:11.315 INFO:tasks.workunit.client.0.smithi022.stdout:2/566: dread d0/d2/d18/d38/d6b/f76 [0,4194304] 0 2024-02-16T19:29:11.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:11.315+0000 7f36a247a700 1 -- 172.21.15.22:0/4141552039 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3688005320 con 0x7f369c100750 2024-02-16T19:29:11.316 INFO:tasks.workunit.client.0.smithi022.stdout:2/567: stat d0/d2/d18 0 2024-02-16T19:29:11.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:11.317+0000 7f36997fa700 1 -- 172.21.15.22:0/4141552039 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 25) v1 ==== 90734+0+0 (secure 0 0 0) 0x7f368c004d50 con 0x7f369c100750 2024-02-16T19:29:11.318 INFO:tasks.workunit.client.1.smithi117.stdout:6/575: readlink d5/d4b/d20/l97 0 2024-02-16T19:29:11.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:11.317+0000 7f36997fa700 1 --2- 172.21.15.22:0/4141552039 >> [v2:172.21.15.117:6828/10500008,v1:172.21.15.117:6829/10500008] conn(0x7f368406cf00 0x7f368406f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:29:11.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:11.317+0000 7f36997fa700 1 -- 172.21.15.22:0/4141552039 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f368c08da20 con 0x7f369c100750 2024-02-16T19:29:11.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:11.318+0000 7f369b7fe700 1 --2- 172.21.15.22:0/4141552039 >> [v2:172.21.15.117:6828/10500008,v1:172.21.15.117:6829/10500008] conn(0x7f368406cf00 0x7f368406f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:29:11.319 INFO:tasks.workunit.client.1.smithi117.stdout:0/619: dread - d0/d9/d22/d27/d83/faf zero size 2024-02-16T19:29:11.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:11.319+0000 7f369b7fe700 1 --2- 172.21.15.22:0/4141552039 >> [v2:172.21.15.117:6828/10500008,v1:172.21.15.117:6829/10500008] conn(0x7f368406cf00 0x7f368406f3b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3690006fd0 tx=0x7f3690008040 comp rx=0 tx=0).ready entity=mgr.24349 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:29:11.322 INFO:tasks.workunit.client.1.smithi117.stdout:5/640: getdents d2/d4c/d4f/d67 0 2024-02-16T19:29:11.323 INFO:tasks.workunit.client.1.smithi117.stdout:6/576: truncate d5/f15 4509880 0 2024-02-16T19:29:11.324 INFO:tasks.workunit.client.1.smithi117.stdout:1/527: dread d2/ff [0,4194304] 0 2024-02-16T19:29:11.325 INFO:tasks.workunit.client.1.smithi117.stdout:7/585: dwrite d4/d1c/d87/d89/d60/d79/fd7 [0,4194304] 0 2024-02-16T19:29:11.329 INFO:tasks.workunit.client.0.smithi022.stdout:5/597: link dd/d1f/d48/d21/d2f/d52/d5c/d61/d6a/f7f dd/d1f/d48/d21/d2f/d52/d5c/fce 0 2024-02-16T19:29:11.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:11.328+0000 7f36997fa700 1 -- 172.21.15.22:0/4141552039 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f368c05b1c0 con 0x7f369c100750 2024-02-16T19:29:11.330 INFO:tasks.workunit.client.0.smithi022.stdout:5/598: stat dd/d1f/d48/d21/d2f/d46/d5e/f89 0 2024-02-16T19:29:11.333 INFO:tasks.workunit.client.1.smithi117.stdout:2/596: rename d0/c3d to d0/d3/d34/d94/d46/d68/d76/da1/d56/cb2 0 2024-02-16T19:29:11.334 INFO:tasks.workunit.client.1.smithi117.stdout:2/597: write d0/d3/d10/f90 [1838898,117525] 0 2024-02-16T19:29:11.338 INFO:tasks.workunit.client.1.smithi117.stdout:4/574: truncate d0/da/d21/d3d/d28/d64/d8a/d92/f89 845726 0 2024-02-16T19:29:11.339 INFO:tasks.workunit.client.1.smithi117.stdout:2/598: dwrite d0/d3/f92 [0,4194304] 0 2024-02-16T19:29:11.339 INFO:tasks.workunit.client.0.smithi022.stdout:5/599: dwrite dd/d1f/d48/d21/d5d/fb3 [0,4194304] 0 2024-02-16T19:29:11.340 INFO:tasks.workunit.client.1.smithi117.stdout:3/680: write d1/d1b/f41 [2599643,102161] 0 2024-02-16T19:29:11.350 INFO:tasks.workunit.client.1.smithi117.stdout:0/620: chown d0/d9/d14/d1d/d60/d76/d7b/d8d/f32 1043373458 1 2024-02-16T19:29:11.351 INFO:tasks.workunit.client.0.smithi022.stdout:5/600: rename dd/d1f/d48/f47 to dd/d1f/d48/d21/d2f/d46/d5e/d84/db4/fcf 0 2024-02-16T19:29:11.353 INFO:tasks.workunit.client.1.smithi117.stdout:4/575: dwrite d0/da/d21/d3d/d28/d64/d73/d91/d37/fae [0,4194304] 0 2024-02-16T19:29:11.356 INFO:tasks.workunit.client.0.smithi022.stdout:5/601: readlink dd/d1f/d48/d21/d2f/d46/d5e/d84/l81 0 2024-02-16T19:29:11.357 INFO:tasks.workunit.client.1.smithi117.stdout:8/587: symlink d4/d7/da6/d3a/d65/d96/da3/lb7 0 2024-02-16T19:29:11.360 INFO:tasks.workunit.client.1.smithi117.stdout:7/586: mkdir d4/d1c/dd8 0 2024-02-16T19:29:11.361 INFO:tasks.workunit.client.0.smithi022.stdout:2/568: dread d0/d32/f62 [0,4194304] 0 2024-02-16T19:29:11.363 INFO:tasks.workunit.client.0.smithi022.stdout:2/569: write d0/d5d/f7a [7935,55358] 0 2024-02-16T19:29:11.365 INFO:tasks.workunit.client.0.smithi022.stdout:2/570: dread d0/f2d [0,4194304] 0 2024-02-16T19:29:11.368 INFO:tasks.workunit.client.1.smithi117.stdout:5/641: mkdir d2/d9/d3c/d24/d50/d96/dbc/d30/d38/db5/dda 0 2024-02-16T19:29:11.371 INFO:tasks.workunit.client.1.smithi117.stdout:1/528: creat d2/d19/d59/d76/d8c/fb1 x:0 0 0 2024-02-16T19:29:11.373 INFO:tasks.workunit.client.0.smithi022.stdout:2/571: mkdir d0/d2/d18/d38/db6 0 2024-02-16T19:29:11.377 INFO:tasks.workunit.client.1.smithi117.stdout:8/588: truncate d4/d12/d2a/f71 68711 0 2024-02-16T19:29:11.378 INFO:tasks.workunit.client.1.smithi117.stdout:6/577: dwrite d5/db/d7d/d32/d37/f58 [0,4194304] 0 2024-02-16T19:29:11.380 INFO:tasks.workunit.client.0.smithi022.stdout:5/602: write dd/d1f/d48/d21/d2f/d46/d5e/f7b [103387,129113] 0 2024-02-16T19:29:11.383 INFO:tasks.workunit.client.1.smithi117.stdout:5/642: dread d2/d3/d21/f33 [0,4194304] 0 2024-02-16T19:29:11.385 INFO:tasks.workunit.client.1.smithi117.stdout:3/681: dwrite d1/d1b/d1e/d37/f65 [0,4194304] 0 2024-02-16T19:29:11.389 INFO:tasks.workunit.client.1.smithi117.stdout:4/576: write d0/da/d21/d3d/d28/f51 [43136,20052] 0 2024-02-16T19:29:11.391 INFO:tasks.workunit.client.0.smithi022.stdout:5/603: dread dd/f3c [0,4194304] 0 2024-02-16T19:29:11.392 INFO:tasks.workunit.client.0.smithi022.stdout:5/604: chown dd/d1f/d48/d21/d2f/d52/f65 100327104 1 2024-02-16T19:29:11.393 INFO:tasks.workunit.client.1.smithi117.stdout:0/621: dwrite d0/f5 [4194304,4194304] 0 2024-02-16T19:29:11.394 INFO:tasks.workunit.client.0.smithi022.stdout:5/605: read dd/d1f/d48/d21/f24 [1635663,29238] 0 2024-02-16T19:29:11.397 INFO:tasks.workunit.client.1.smithi117.stdout:7/587: write d4/f4d [690358,19129] 0 2024-02-16T19:29:11.400 INFO:tasks.workunit.client.1.smithi117.stdout:6/578: creat d5/db/d7d/d32/d95/d49/fbc x:0 0 0 2024-02-16T19:29:11.402 INFO:tasks.workunit.client.1.smithi117.stdout:0/622: dread d0/d9/d14/d23/d33/f5c [0,4194304] 0 2024-02-16T19:29:11.409 INFO:tasks.workunit.client.1.smithi117.stdout:8/589: mknod d4/d7/d6a/d6f/da8/cb8 0 2024-02-16T19:29:11.411 INFO:tasks.workunit.client.1.smithi117.stdout:7/588: creat d4/d1c/d39/d46/d47/fd9 x:0 0 0 2024-02-16T19:29:11.413 INFO:tasks.workunit.client.1.smithi117.stdout:6/579: truncate d5/d4b/d20/f42 4532118 0 2024-02-16T19:29:11.416 INFO:tasks.workunit.client.1.smithi117.stdout:0/623: creat d0/d9/d22/dd4/fdc x:0 0 0 2024-02-16T19:29:11.421 INFO:tasks.workunit.client.1.smithi117.stdout:1/529: creat d2/d1d/d60/fb2 x:0 0 0 2024-02-16T19:29:11.423 INFO:tasks.workunit.client.1.smithi117.stdout:4/577: rename d0/da/d21/d3d/d28/d64/l15 to d0/da/d21/d3d/d28/d66/lb8 0 2024-02-16T19:29:11.425 INFO:tasks.workunit.client.1.smithi117.stdout:2/599: getdents d0/d13/d4d/d80 0 2024-02-16T19:29:11.426 INFO:tasks.workunit.client.1.smithi117.stdout:5/643: write d2/d9/d3c/d24/d50/d96/dbc/d30/d38/f85 [483991,53973] 0 2024-02-16T19:29:11.428 INFO:tasks.workunit.client.1.smithi117.stdout:8/590: rmdir d4/d7/d6a 39 2024-02-16T19:29:11.432 INFO:tasks.workunit.client.1.smithi117.stdout:6/580: mknod d5/d4b/d44/d52/d81/d9f/cbd 0 2024-02-16T19:29:11.435 INFO:tasks.workunit.client.1.smithi117.stdout:0/624: mknod d0/d9/d22/d27/d31/cdd 0 2024-02-16T19:29:11.437 INFO:tasks.workunit.client.1.smithi117.stdout:3/682: creat d1/d31/fe8 x:0 0 0 2024-02-16T19:29:11.442 INFO:tasks.workunit.client.1.smithi117.stdout:2/600: creat d0/d3/d40/fb3 x:0 0 0 2024-02-16T19:29:11.443 INFO:tasks.workunit.client.1.smithi117.stdout:9/507: sync 2024-02-16T19:29:11.446 INFO:tasks.workunit.client.1.smithi117.stdout:8/591: write d4/d7/d6a/d6f/da8/d76/faa [131875,114379] 0 2024-02-16T19:29:11.458 INFO:tasks.workunit.client.1.smithi117.stdout:6/581: dread d5/d4b/d20/f21 [0,4194304] 0 2024-02-16T19:29:11.550 INFO:tasks.workunit.client.1.smithi117.stdout:3/683: chown d1/dd/f14 19536416 1 2024-02-16T19:29:11.550 INFO:tasks.workunit.client.1.smithi117.stdout:1/530: creat d2/d19/d59/d89/d69/d9a/fb3 x:0 0 0 2024-02-16T19:29:11.551 INFO:tasks.workunit.client.0.smithi022.stdout:5/606: creat dd/d1f/d48/d21/d2f/fd0 x:0 0 0 2024-02-16T19:29:11.551 INFO:tasks.workunit.client.1.smithi117.stdout:4/578: creat d0/da/d21/d3d/d28/d64/d73/d91/d9e/fb9 x:0 0 0 2024-02-16T19:29:11.552 INFO:tasks.workunit.client.1.smithi117.stdout:4/579: dread - d0/da/d21/d3d/d28/d64/d73/fa9 zero size 2024-02-16T19:29:11.553 INFO:tasks.workunit.client.1.smithi117.stdout:4/580: write d0/da/d21/d3d/d28/fad [35561,48584] 0 2024-02-16T19:29:11.602 INFO:tasks.workunit.client.0.smithi022.stdout:2/572: unlink d0/l16 0 2024-02-16T19:29:11.604 INFO:tasks.workunit.client.1.smithi117.stdout:2/601: unlink d0/d3/d40/fb3 0 2024-02-16T19:29:11.605 INFO:tasks.workunit.client.1.smithi117.stdout:2/602: chown d0/la8 92899 1 2024-02-16T19:29:11.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:11.605+0000 7f36a247a700 1 -- 172.21.15.22:0/4141552039 --> [v2:172.21.15.117:6828/10500008,v1:172.21.15.117:6829/10500008] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3688000bf0 con 0x7f368406cf00 2024-02-16T19:29:11.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:11.606+0000 7f36997fa700 1 -- 172.21.15.22:0/4141552039 <== mgr.24349 v2:172.21.15.117:6828/10500008 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f3688000bf0 con 0x7f368406cf00 2024-02-16T19:29:11.607 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-02-16T19:29:11.607 INFO:teuthology.orchestra.run.smithi022.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T19:29:11.607 INFO:teuthology.orchestra.run.smithi022.stdout: "in_progress": true, 2024-02-16T19:29:11.607 INFO:teuthology.orchestra.run.smithi022.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-02-16T19:29:11.607 INFO:teuthology.orchestra.run.smithi022.stdout: "services_complete": [], 2024-02-16T19:29:11.607 INFO:teuthology.orchestra.run.smithi022.stdout: "progress": "1/2 daemons upgraded", 2024-02-16T19:29:11.608 INFO:teuthology.orchestra.run.smithi022.stdout: "message": "", 2024-02-16T19:29:11.608 INFO:teuthology.orchestra.run.smithi022.stdout: "is_paused": false 2024-02-16T19:29:11.608 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-02-16T19:29:11.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:11.608+0000 7f36a247a700 1 -- 172.21.15.22:0/4141552039 >> [v2:172.21.15.117:6828/10500008,v1:172.21.15.117:6829/10500008] conn(0x7f368406cf00 msgr2=0x7f368406f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:29:11.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:11.608+0000 7f36a247a700 1 --2- 172.21.15.22:0/4141552039 >> [v2:172.21.15.117:6828/10500008,v1:172.21.15.117:6829/10500008] conn(0x7f368406cf00 0x7f368406f3b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3690006fd0 tx=0x7f3690008040 comp rx=0 tx=0).stop 2024-02-16T19:29:11.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:11.608+0000 7f36a247a700 1 -- 172.21.15.22:0/4141552039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f369c100750 msgr2=0x7f369c190d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:29:11.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:11.608+0000 7f36a247a700 1 --2- 172.21.15.22:0/4141552039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f369c100750 0x7f369c190d30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f368c00bd70 tx=0x7f368c00be50 comp rx=0 tx=0).stop 2024-02-16T19:29:11.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:11.609+0000 7f36a247a700 1 -- 172.21.15.22:0/4141552039 shutdown_connections 2024-02-16T19:29:11.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:11.609+0000 7f36a247a700 1 --2- 172.21.15.22:0/4141552039 >> [v2:172.21.15.117:6828/10500008,v1:172.21.15.117:6829/10500008] conn(0x7f368406cf00 0x7f368406f3b0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:11.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:11.609+0000 7f36a247a700 1 --2- 172.21.15.22:0/4141552039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f369c100750 0x7f369c190d30 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:11.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:11.609+0000 7f36a247a700 1 --2- 172.21.15.22:0/4141552039 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f369c1010a0 0x7f369c191270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:11.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:11.609+0000 7f36a247a700 1 -- 172.21.15.22:0/4141552039 >> 172.21.15.22:0/4141552039 conn(0x7f369c0f9f10 msgr2=0x7f369c0fe340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:29:11.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:11.609+0000 7f36a247a700 1 -- 172.21.15.22:0/4141552039 shutdown_connections 2024-02-16T19:29:11.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:11.609+0000 7f36a247a700 1 -- 172.21.15.22:0/4141552039 wait complete. 2024-02-16T19:29:11.611 INFO:tasks.workunit.client.1.smithi117.stdout:8/592: mknod d4/d7/da6/d3a/d43/d44/cb9 0 2024-02-16T19:29:11.613 INFO:tasks.workunit.client.0.smithi022.stdout:5/607: dwrite dd/d1f/d48/f74 [0,4194304] 0 2024-02-16T19:29:11.614 INFO:tasks.workunit.client.1.smithi117.stdout:6/582: chown d5/d4b/d71 110 1 2024-02-16T19:29:11.615 INFO:tasks.workunit.client.1.smithi117.stdout:3/684: mkdir d1/dd/d3b/dba/de9 0 2024-02-16T19:29:11.616 INFO:tasks.workunit.client.1.smithi117.stdout:1/531: mknod d2/d51/cb4 0 2024-02-16T19:29:11.617 INFO:tasks.workunit.client.0.smithi022.stdout:2/573: dwrite d0/d2/d18/d1f/d6c/fa2 [0,4194304] 0 2024-02-16T19:29:11.623 INFO:tasks.workunit.client.1.smithi117.stdout:9/508: unlink f5 0 2024-02-16T19:29:11.624 INFO:tasks.workunit.client.1.smithi117.stdout:1/532: dread d2/d19/fac [0,4194304] 0 2024-02-16T19:29:11.624 INFO:tasks.workunit.client.0.smithi022.stdout:2/574: mknod d0/d2/d18/d19/d40/d55/d59/cb7 0 2024-02-16T19:29:11.624 INFO:tasks.workunit.client.1.smithi117.stdout:7/589: link d4/d1c/c8e d4/d1c/d87/d89/d5f/dae/cda 0 2024-02-16T19:29:11.625 INFO:tasks.workunit.client.0.smithi022.stdout:2/575: chown d0/d2/c93 7366903 1 2024-02-16T19:29:11.628 INFO:tasks.workunit.client.1.smithi117.stdout:8/593: write d4/d1c/f7f [3732442,14486] 0 2024-02-16T19:29:11.628 INFO:tasks.workunit.client.1.smithi117.stdout:8/594: stat d4/d12/d2a/d64/lb2 0 2024-02-16T19:29:11.628 INFO:tasks.workunit.client.0.smithi022.stdout:2/576: rmdir d0/d32 39 2024-02-16T19:29:11.634 INFO:tasks.workunit.client.0.smithi022.stdout:2/577: creat d0/d5d/fb8 x:0 0 0 2024-02-16T19:29:11.635 INFO:tasks.workunit.client.1.smithi117.stdout:6/583: rmdir d5/d4b/d20 39 2024-02-16T19:29:11.635 INFO:tasks.workunit.client.1.smithi117.stdout:6/584: write d5/db/d7d/f40 [4080613,104920] 0 2024-02-16T19:29:11.640 INFO:tasks.workunit.client.0.smithi022.stdout:5/608: link dd/d1f/d48/d21/d2f/d40/d49/d4b/f66 dd/d1f/d48/d21/d2f/d46/d5e/d84/fd1 0 2024-02-16T19:29:11.641 INFO:tasks.workunit.client.1.smithi117.stdout:5/644: getdents d2/d3/d21 0 2024-02-16T19:29:11.642 INFO:tasks.workunit.client.0.smithi022.stdout:2/578: creat d0/d2/d18/d38/d9d/fb9 x:0 0 0 2024-02-16T19:29:11.645 INFO:tasks.workunit.client.0.smithi022.stdout:5/609: creat dd/d1f/d2e/fd2 x:0 0 0 2024-02-16T19:29:11.645 INFO:tasks.workunit.client.1.smithi117.stdout:8/595: dread d4/d7/da6/f56 [4194304,4194304] 0 2024-02-16T19:29:11.646 INFO:tasks.workunit.client.1.smithi117.stdout:9/509: rmdir dd/d2d 39 2024-02-16T19:29:11.647 INFO:tasks.workunit.client.1.smithi117.stdout:8/596: truncate d4/d73/f82 1441184 0 2024-02-16T19:29:11.653 INFO:tasks.workunit.client.0.smithi022.stdout:2/579: rename d0/d2/fb3 to d0/d2/d2e/fba 0 2024-02-16T19:29:11.654 INFO:tasks.workunit.client.1.smithi117.stdout:0/625: getdents d0/d9/d14 0 2024-02-16T19:29:11.658 INFO:tasks.workunit.client.0.smithi022.stdout:5/610: rename dd/d1f/d48/d21/d2f/d40/f91 to dd/d1f/d48/d21/d2f/d46/d5e/fd3 0 2024-02-16T19:29:11.659 INFO:tasks.workunit.client.0.smithi022.stdout:2/580: dwrite d0/d2/d18/d38/d9d/fb9 [0,4194304] 0 2024-02-16T19:29:11.660 INFO:tasks.workunit.client.0.smithi022.stdout:2/581: fsync d0/d2/d18/f65 0 2024-02-16T19:29:11.670 INFO:tasks.workunit.client.1.smithi117.stdout:7/590: symlink d4/d1c/dc0/ldb 0 2024-02-16T19:29:11.670 INFO:tasks.workunit.client.1.smithi117.stdout:7/591: readlink d4/d1c/d87/d89/d60/d79/lc9 0 2024-02-16T19:29:11.672 INFO:tasks.workunit.client.1.smithi117.stdout:0/626: mknod d0/d9/d14/d1d/d60/d65/cde 0 2024-02-16T19:29:11.674 INFO:tasks.workunit.client.1.smithi117.stdout:4/581: link d0/da/f33 d0/da/d21/d3d/d28/d64/d73/d91/fba 0 2024-02-16T19:29:11.675 INFO:tasks.workunit.client.1.smithi117.stdout:8/597: fdatasync d4/d12/d2a/f2b 0 2024-02-16T19:29:11.692 INFO:tasks.workunit.client.0.smithi022.stdout:5/611: dwrite dd/d1f/d48/d21/d2f/d40/d49/d4b/f66 [0,4194304] 0 2024-02-16T19:29:11.698 INFO:tasks.workunit.client.0.smithi022.stdout:5/612: dread dd/d1f/d48/d21/d2f/d52/d5c/d61/d6a/f70 [4194304,4194304] 0 2024-02-16T19:29:11.701 INFO:tasks.workunit.client.0.smithi022.stdout:5/613: unlink dd/d1f/d48/d21/d64/fc5 0 2024-02-16T19:29:11.706 INFO:tasks.workunit.client.0.smithi022.stdout:5/614: dwrite dd/f11 [0,4194304] 0 2024-02-16T19:29:11.706 INFO:tasks.workunit.client.0.smithi022.stdout:5/615: chown dd/d1f/d2e/l5b 198039343 1 2024-02-16T19:29:11.716 INFO:tasks.workunit.client.0.smithi022.stdout:5/616: dwrite f6 [12582912,4194304] 0 2024-02-16T19:29:11.718 INFO:tasks.workunit.client.0.smithi022.stdout:5/617: write dd/d1f/d48/d21/d2f/d46/fae [664179,78598] 0 2024-02-16T19:29:11.719 INFO:tasks.workunit.client.0.smithi022.stdout:5/618: dread - dd/d1f/d48/d21/d2f/d40/d4e/d56/fa5 zero size 2024-02-16T19:29:11.726 INFO:tasks.workunit.client.0.smithi022.stdout:5/619: truncate dd/d1f/d48/f60 1373056 0 2024-02-16T19:29:11.727 INFO:tasks.workunit.client.0.smithi022.stdout:5/620: chown dd/d1f/d48/d21/d2f/d46/c50 668 1 2024-02-16T19:29:11.729 INFO:tasks.workunit.client.0.smithi022.stdout:5/621: mkdir dd/d1f/d2e/dd4 0 2024-02-16T19:29:11.730 INFO:tasks.workunit.client.0.smithi022.stdout:5/622: rename dd/d1f/d48/d21/d2f/d52 to dd/d1f/d48/d21/d2f/d52/d77/da9/dd5 22 2024-02-16T19:29:11.731 INFO:tasks.workunit.client.0.smithi022.stdout:5/623: write dd/f17 [7853076,60544] 0 2024-02-16T19:29:11.733 INFO:tasks.workunit.client.0.smithi022.stdout:5/624: creat dd/d1f/d76/fd6 x:0 0 0 2024-02-16T19:29:11.735 INFO:tasks.workunit.client.1.smithi117.stdout:7/592: creat d4/d13/da3/fdc x:0 0 0 2024-02-16T19:29:11.736 INFO:tasks.workunit.client.1.smithi117.stdout:4/582: mkdir d0/da/d21/d3d/d28/d64/d73/d91/d9e/dbb 0 2024-02-16T19:29:11.737 INFO:tasks.workunit.client.1.smithi117.stdout:2/603: truncate d0/d3/d34/d94/d46/d68/d76/da1/f73 1214341 0 2024-02-16T19:29:11.738 INFO:tasks.workunit.client.1.smithi117.stdout:4/583: chown d0/da/d4a/f58 1121546200 1 2024-02-16T19:29:11.738 INFO:tasks.workunit.client.0.smithi022.stdout:5/625: dread dd/d1f/d48/d21/d2f/d52/d5c/d61/f92 [0,4194304] 0 2024-02-16T19:29:11.740 INFO:tasks.workunit.client.1.smithi117.stdout:9/510: link dd/d47/d5b/d5f/cab dd/d47/d5b/cb3 0 2024-02-16T19:29:11.744 INFO:tasks.workunit.client.1.smithi117.stdout:7/593: rmdir d4/d13/da3 39 2024-02-16T19:29:11.744 INFO:tasks.workunit.client.1.smithi117.stdout:3/685: dwrite d1/d30/f77 [0,4194304] 0 2024-02-16T19:29:11.745 INFO:tasks.workunit.client.1.smithi117.stdout:1/533: dwrite d2/d1d/f39 [0,4194304] 0 2024-02-16T19:29:11.745 INFO:tasks.workunit.client.1.smithi117.stdout:3/686: stat d1/d67 0 2024-02-16T19:29:11.746 INFO:tasks.workunit.client.1.smithi117.stdout:4/584: write d0/f47 [872579,10669] 0 2024-02-16T19:29:11.746 INFO:tasks.workunit.client.1.smithi117.stdout:2/604: creat d0/d13/d4d/d53/fb4 x:0 0 0 2024-02-16T19:29:11.747 INFO:tasks.workunit.client.1.smithi117.stdout:6/585: link d5/d4b/d20/c70 d5/d9c/daa/d85/db0/cbe 0 2024-02-16T19:29:11.748 INFO:tasks.workunit.client.1.smithi117.stdout:3/687: fdatasync d1/d30/d52/d56/fd5 0 2024-02-16T19:29:11.749 INFO:tasks.workunit.client.1.smithi117.stdout:4/585: chown d0/da/d21/d3d/d28/d66/d97 898060 1 2024-02-16T19:29:11.750 INFO:tasks.workunit.client.1.smithi117.stdout:3/688: chown d1/d1b/d1e/d29 28 1 2024-02-16T19:29:11.752 INFO:tasks.workunit.client.1.smithi117.stdout:8/598: dread d4/f53 [4194304,4194304] 0 2024-02-16T19:29:11.752 INFO:tasks.workunit.client.1.smithi117.stdout:9/511: mknod dd/d45/d73/cb4 0 2024-02-16T19:29:11.753 INFO:tasks.workunit.client.0.smithi022.stdout:5/626: getdents dd/d1f/d48/d21/d2f/d46/d5e/d84/d8b 0 2024-02-16T19:29:11.759 INFO:tasks.workunit.client.1.smithi117.stdout:1/534: symlink d2/d19/d59/lb5 0 2024-02-16T19:29:11.760 INFO:tasks.workunit.client.0.smithi022.stdout:5/627: unlink dd/d1f/d48/d21/d5d/c75 0 2024-02-16T19:29:11.761 INFO:tasks.workunit.client.1.smithi117.stdout:4/586: mknod d0/da/d21/d3d/d28/d66/cbc 0 2024-02-16T19:29:11.762 INFO:tasks.workunit.client.1.smithi117.stdout:7/594: dread d4/d1c/d39/d46/d55/db8/fd5 [0,4194304] 0 2024-02-16T19:29:11.763 INFO:tasks.workunit.client.0.smithi022.stdout:5/628: mknod dd/d1f/d48/d21/d2f/d52/d77/da9/cd7 0 2024-02-16T19:29:11.764 INFO:tasks.workunit.client.1.smithi117.stdout:3/689: dread d1/dd/d8e/f94 [0,4194304] 0 2024-02-16T19:29:11.765 INFO:tasks.workunit.client.1.smithi117.stdout:0/627: dwrite d0/d9/d14/d23/f56 [0,4194304] 0 2024-02-16T19:29:11.767 INFO:tasks.workunit.client.0.smithi022.stdout:5/629: dwrite dd/d1f/d48/d21/d2f/d40/d49/fa1 [0,4194304] 0 2024-02-16T19:29:11.770 INFO:tasks.workunit.client.1.smithi117.stdout:2/605: dwrite d0/d3/d34/d94/d46/d68/d93/fa0 [0,4194304] 0 2024-02-16T19:29:11.774 INFO:tasks.workunit.client.1.smithi117.stdout:6/586: write d5/f86 [1467033,26503] 0 2024-02-16T19:29:11.774 INFO:tasks.workunit.client.1.smithi117.stdout:6/587: chown d5/d4b/d44/c6d 756898 1 2024-02-16T19:29:11.777 INFO:tasks.workunit.client.0.smithi022.stdout:5/630: read dd/d1f/d48/d21/f24 [7041592,114910] 0 2024-02-16T19:29:11.780 INFO:tasks.workunit.client.0.smithi022.stdout:5/631: rename dd/d1f/d2e/dd4 to dd/d1f/d2e/dd4/dd8 22 2024-02-16T19:29:11.785 INFO:tasks.workunit.client.0.smithi022.stdout:5/632: dwrite dd/f3c [0,4194304] 0 2024-02-16T19:29:11.792 INFO:tasks.workunit.client.1.smithi117.stdout:4/587: mknod d0/da/d4a/cbd 0 2024-02-16T19:29:11.793 INFO:tasks.workunit.client.0.smithi022.stdout:4/538: sync 2024-02-16T19:29:11.794 INFO:tasks.workunit.client.0.smithi022.stdout:5/633: dwrite dd/d1f/d48/d21/d2f/d40/d49/fa1 [0,4194304] 0 2024-02-16T19:29:11.797 INFO:tasks.workunit.client.1.smithi117.stdout:6/588: mknod d5/db/d7d/cbf 0 2024-02-16T19:29:11.798 INFO:tasks.workunit.client.1.smithi117.stdout:8/599: truncate d4/d7/d6a/f7b 34282 0 2024-02-16T19:29:11.800 INFO:tasks.workunit.client.0.smithi022.stdout:4/539: dread d4/d6/d14/d64/d4e/f8b [0,4194304] 0 2024-02-16T19:29:11.802 INFO:tasks.workunit.client.1.smithi117.stdout:9/512: rename dd/d45/d62 to dd/db5 0 2024-02-16T19:29:11.803 INFO:tasks.workunit.client.1.smithi117.stdout:5/645: sync 2024-02-16T19:29:11.803 INFO:tasks.workunit.client.1.smithi117.stdout:9/513: fsync dd/d47/d5e/f6e 0 2024-02-16T19:29:11.804 INFO:tasks.workunit.client.0.smithi022.stdout:5/634: mknod dd/d1f/d48/d21/d2f/d52/d5c/d61/cd9 0 2024-02-16T19:29:11.805 INFO:tasks.workunit.client.1.smithi117.stdout:4/588: truncate d0/da/d21/d3d/d28/d64/d48/f5d 2503661 0 2024-02-16T19:29:11.806 INFO:tasks.workunit.client.1.smithi117.stdout:4/589: truncate d0/f47 1532088 0 2024-02-16T19:29:11.807 INFO:tasks.workunit.client.1.smithi117.stdout:7/595: mkdir d4/d13/d63/ddd 0 2024-02-16T19:29:11.809 INFO:tasks.workunit.client.0.smithi022.stdout:4/540: symlink d4/d6/dd/d22/d83/la7 0 2024-02-16T19:29:11.810 INFO:tasks.workunit.client.1.smithi117.stdout:3/690: mknod d1/d31/cea 0 2024-02-16T19:29:11.811 INFO:tasks.workunit.client.0.smithi022.stdout:5/635: dwrite dd/d1f/d48/d21/d2f/d46/d5e/d84/d8b/f9a [0,4194304] 0 2024-02-16T19:29:11.814 INFO:tasks.workunit.client.0.smithi022.stdout:5/636: truncate dd/d1f/d48/d21/d2f/d52/f65 699232 0 2024-02-16T19:29:11.819 INFO:tasks.workunit.client.0.smithi022.stdout:5/637: chown dd/d1f/d48/f60 8142613 1 2024-02-16T19:29:11.820 INFO:tasks.workunit.client.0.smithi022.stdout:5/638: stat dd/d1f/d48/d21/d2f/d52/d77/da9/fc8 0 2024-02-16T19:29:11.820 INFO:tasks.workunit.client.1.smithi117.stdout:0/628: rename d0/d9/d22/d27/d31/d94/cdb to d0/d9/d14/dbc/cdf 0 2024-02-16T19:29:11.822 INFO:tasks.workunit.client.1.smithi117.stdout:4/590: dread d0/da/d21/d3d/d28/f63 [0,4194304] 0 2024-02-16T19:29:11.823 INFO:tasks.workunit.client.0.smithi022.stdout:8/571: sync 2024-02-16T19:29:11.823 INFO:tasks.workunit.client.0.smithi022.stdout:1/652: sync 2024-02-16T19:29:11.823 INFO:tasks.workunit.client.0.smithi022.stdout:7/579: sync 2024-02-16T19:29:11.823 INFO:tasks.workunit.client.0.smithi022.stdout:6/621: sync 2024-02-16T19:29:11.823 INFO:tasks.workunit.client.0.smithi022.stdout:9/615: sync 2024-02-16T19:29:11.823 INFO:tasks.workunit.client.0.smithi022.stdout:3/518: sync 2024-02-16T19:29:11.823 INFO:tasks.workunit.client.0.smithi022.stdout:0/764: sync 2024-02-16T19:29:11.823 INFO:tasks.workunit.client.0.smithi022.stdout:2/582: sync 2024-02-16T19:29:11.824 INFO:tasks.workunit.client.1.smithi117.stdout:0/629: write d0/d9/d14/d1d/d1e/d54/fb4 [351322,125855] 0 2024-02-16T19:29:11.824 INFO:tasks.workunit.client.1.smithi117.stdout:4/591: chown d0/da/d4a/fa8 87381 1 2024-02-16T19:29:11.824 INFO:tasks.workunit.client.1.smithi117.stdout:5/646: creat d2/d9/d3c/d24/d50/d96/dbc/d30/d5a/d62/fdb x:0 0 0 2024-02-16T19:29:11.825 INFO:tasks.workunit.client.1.smithi117.stdout:4/592: stat d0/da/d21/d3d/c7d 0 2024-02-16T19:29:11.826 INFO:tasks.workunit.client.0.smithi022.stdout:2/583: chown d0/d2/d18/f22 50 1 2024-02-16T19:29:11.827 INFO:tasks.workunit.client.0.smithi022.stdout:8/572: stat d3/d12/d92/cb4 0 2024-02-16T19:29:11.829 INFO:tasks.workunit.client.0.smithi022.stdout:0/765: rmdir d10/d7d/dac 39 2024-02-16T19:29:11.830 INFO:tasks.workunit.client.1.smithi117.stdout:1/535: link d2/d4/cd d2/d1d/d60/d7c/d9e/da5/db0/cb6 0 2024-02-16T19:29:11.830 INFO:tasks.workunit.client.1.smithi117.stdout:0/630: dwrite d0/d9/d22/dd4/fdc [0,4194304] 0 2024-02-16T19:29:11.830 INFO:tasks.workunit.client.1.smithi117.stdout:1/536: dread - d2/f9b zero size 2024-02-16T19:29:11.831 INFO:tasks.workunit.client.0.smithi022.stdout:5/639: dread dd/d1f/d48/d21/d2f/d40/d49/f4c [4194304,4194304] 0 2024-02-16T19:29:11.831 INFO:tasks.workunit.client.0.smithi022.stdout:2/584: fdatasync d0/d2/d18/d1f/f21 0 2024-02-16T19:29:11.831 INFO:tasks.workunit.client.0.smithi022.stdout:1/653: dread d6/d8/d22/d16/d17/d5d/f95 [0,4194304] 0 2024-02-16T19:29:11.832 INFO:tasks.workunit.client.0.smithi022.stdout:8/573: write d3/d12/d6a/fa4 [798823,48081] 0 2024-02-16T19:29:11.833 INFO:tasks.workunit.client.0.smithi022.stdout:0/766: chown d10/d15/d23/ce3 0 1 2024-02-16T19:29:11.834 INFO:tasks.workunit.client.0.smithi022.stdout:1/654: readlink d6/l25 0 2024-02-16T19:29:11.835 INFO:tasks.workunit.client.1.smithi117.stdout:8/600: truncate d4/f3f 1190982 0 2024-02-16T19:29:11.839 INFO:tasks.workunit.client.0.smithi022.stdout:5/640: write dd/d1f/d48/d21/d2f/d46/d5e/d84/d8b/f8e [889160,36112] 0 2024-02-16T19:29:11.839 INFO:tasks.workunit.client.1.smithi117.stdout:6/589: symlink d5/d4b/d44/d93/dae/lc0 0 2024-02-16T19:29:11.840 INFO:tasks.workunit.client.1.smithi117.stdout:5/647: dread d2/db3/f48 [0,4194304] 0 2024-02-16T19:29:11.841 INFO:tasks.workunit.client.0.smithi022.stdout:6/622: symlink d1/d8/da/d17/d53/d7f/da3/ldf 0 2024-02-16T19:29:11.842 INFO:tasks.workunit.client.1.smithi117.stdout:5/648: write d2/d9/d3c/d24/d50/d52/d74/fd2 [215589,63719] 0 2024-02-16T19:29:11.842 INFO:tasks.workunit.client.1.smithi117.stdout:9/514: mknod dd/d47/cb6 0 2024-02-16T19:29:11.848 INFO:tasks.workunit.client.1.smithi117.stdout:2/606: dwrite d0/f12 [0,4194304] 0 2024-02-16T19:29:11.855 INFO:tasks.workunit.client.1.smithi117.stdout:2/607: dwrite d0/d3/d34/d94/d46/d68/d93/fa0 [0,4194304] 0 2024-02-16T19:29:11.857 INFO:tasks.workunit.client.1.smithi117.stdout:4/593: creat d0/da/d21/d3d/d28/d64/d73/d91/fbe x:0 0 0 2024-02-16T19:29:11.859 INFO:tasks.workunit.client.1.smithi117.stdout:0/631: mkdir d0/d9/d2c/d8b/de0 0 2024-02-16T19:29:11.860 INFO:tasks.workunit.client.1.smithi117.stdout:2/608: dwrite d0/d13/d75/fac [0,4194304] 0 2024-02-16T19:29:11.864 INFO:tasks.workunit.client.1.smithi117.stdout:6/590: read - d5/d4b/d44/d52/f7e zero size 2024-02-16T19:29:11.868 INFO:tasks.workunit.client.1.smithi117.stdout:7/596: write d4/d1c/d39/d46/f75 [1153386,7584] 0 2024-02-16T19:29:11.871 INFO:tasks.workunit.client.1.smithi117.stdout:3/691: truncate d1/dd/d3b/f3d 13652 0 2024-02-16T19:29:11.873 INFO:tasks.workunit.client.1.smithi117.stdout:3/692: truncate d1/d30/f90 4639492 0 2024-02-16T19:29:11.874 INFO:tasks.workunit.client.1.smithi117.stdout:4/594: readlink d0/da/l53 0 2024-02-16T19:29:11.878 INFO:tasks.workunit.client.1.smithi117.stdout:1/537: creat d2/d19/d59/d89/d7b/d86/fb7 x:0 0 0 2024-02-16T19:29:11.880 INFO:tasks.workunit.client.1.smithi117.stdout:0/632: symlink d0/d9/d14/d1d/d60/d76/le1 0 2024-02-16T19:29:11.883 INFO:tasks.workunit.client.1.smithi117.stdout:6/591: read d5/f12 [623010,109457] 0 2024-02-16T19:29:11.886 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:11 smithi117 ceph-mon[35309]: from='client.14628 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:29:11.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:11 smithi117 ceph-mon[35309]: from='client.14632 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:29:11.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:11 smithi117 ceph-mon[35309]: mgrmap e25: smithi117.cohbtv(active, since 11s), standbys: smithi022.gmdcch 2024-02-16T19:29:11.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:11 smithi117 ceph-mon[35309]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "mgr metadata", "who": "smithi022.gmdcch", "id": "smithi022.gmdcch"}]: dispatch 2024-02-16T19:29:11.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:11 smithi117 ceph-mon[35309]: from='client.? 172.21.15.22:0/347594203' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:29:11.889 INFO:tasks.workunit.client.1.smithi117.stdout:1/538: rmdir d2/d19/d59/d89 39 2024-02-16T19:29:11.890 INFO:tasks.workunit.client.0.smithi022.stdout:3/519: dread f5 [0,4194304] 0 2024-02-16T19:29:11.891 INFO:tasks.workunit.client.1.smithi117.stdout:0/633: write d0/d9/d14/d23/d33/fd8 [1024828,84136] 0 2024-02-16T19:29:11.891 INFO:tasks.workunit.client.1.smithi117.stdout:9/515: write dd/d23/f25 [4507053,32615] 0 2024-02-16T19:29:11.892 INFO:tasks.workunit.client.0.smithi022.stdout:9/616: dwrite db/d14/d16/d27/d55/fa6 [0,4194304] 0 2024-02-16T19:29:11.892 INFO:tasks.workunit.client.1.smithi117.stdout:5/649: truncate d2/d5d/f7d 3969882 0 2024-02-16T19:29:11.893 INFO:tasks.workunit.client.1.smithi117.stdout:5/650: chown d2/d9/d3c/d24/d50/d52/d74 512675 1 2024-02-16T19:29:11.893 INFO:tasks.workunit.client.0.smithi022.stdout:4/541: dread d4/d6/dd/df/d10/f19 [0,4194304] 0 2024-02-16T19:29:11.894 INFO:tasks.workunit.client.0.smithi022.stdout:1/655: truncate d6/d8/d22/d16/d17/f52 1363151 0 2024-02-16T19:29:11.894 INFO:tasks.workunit.client.0.smithi022.stdout:8/574: truncate d3/d12/d17/d64/f80 413578 0 2024-02-16T19:29:11.896 INFO:tasks.workunit.client.1.smithi117.stdout:7/597: getdents d4/d1c/d39/d46/dc4 0 2024-02-16T19:29:11.897 INFO:tasks.workunit.client.0.smithi022.stdout:8/575: stat d3/d12/c49 0 2024-02-16T19:29:11.900 INFO:tasks.workunit.client.0.smithi022.stdout:4/542: write d4/d6/dd/d22/d83/f9f [408685,88943] 0 2024-02-16T19:29:11.905 INFO:tasks.workunit.client.0.smithi022.stdout:2/585: dwrite d0/d2/d2e/f2a [0,4194304] 0 2024-02-16T19:29:11.905 INFO:tasks.workunit.client.0.smithi022.stdout:8/576: fsync d3/d12/d6a/fa4 0 2024-02-16T19:29:11.906 INFO:tasks.workunit.client.1.smithi117.stdout:1/539: unlink d2/d4/l73 0 2024-02-16T19:29:11.909 INFO:tasks.workunit.client.0.smithi022.stdout:5/641: dwrite dd/d1f/d48/d21/d2f/d52/d5c/d61/f92 [0,4194304] 0 2024-02-16T19:29:11.910 INFO:tasks.workunit.client.0.smithi022.stdout:8/577: stat d3/d12/d2b/d54/d97/c9f 0 2024-02-16T19:29:11.910 INFO:tasks.workunit.client.1.smithi117.stdout:8/601: getdents d4/d7 0 2024-02-16T19:29:11.910 INFO:tasks.workunit.client.1.smithi117.stdout:3/693: dwrite d1/d1b/d1e/f22 [0,4194304] 0 2024-02-16T19:29:11.912 INFO:tasks.workunit.client.0.smithi022.stdout:7/580: truncate d9/da/d74/f80 3635203 0 2024-02-16T19:29:11.913 INFO:tasks.workunit.client.0.smithi022.stdout:5/642: read dd/d1f/d48/d21/d2f/d40/d4e/f68 [16120,101678] 0 2024-02-16T19:29:11.914 INFO:tasks.workunit.client.0.smithi022.stdout:9/617: rmdir db/d18 39 2024-02-16T19:29:11.916 INFO:tasks.workunit.client.0.smithi022.stdout:3/520: read d7/d12/d18/dd/d24/d58/d6b/f78 [2853303,34531] 0 2024-02-16T19:29:11.919 INFO:tasks.workunit.client.1.smithi117.stdout:9/516: fsync dd/d47/f7b 0 2024-02-16T19:29:11.920 INFO:tasks.workunit.client.0.smithi022.stdout:1/656: creat d6/d8/dba/fce x:0 0 0 2024-02-16T19:29:11.921 INFO:tasks.workunit.client.0.smithi022.stdout:1/657: dread - d6/d8/d22/d16/d17/d5d/fc8 zero size 2024-02-16T19:29:11.922 INFO:tasks.workunit.client.0.smithi022.stdout:1/658: chown d6/d8/d22/d3a/f44 1597391 1 2024-02-16T19:29:11.930 INFO:tasks.workunit.client.0.smithi022.stdout:5/643: fsync dd/d1f/d48/d21/d2f/d40/d4e/f87 0 2024-02-16T19:29:11.937 INFO:tasks.workunit.client.0.smithi022.stdout:1/659: truncate d6/d8/d22/d67/d7e/fa9 252670 0 2024-02-16T19:29:11.937 INFO:tasks.workunit.client.0.smithi022.stdout:4/543: write d4/d6/dd/f60 [4429283,40943] 0 2024-02-16T19:29:11.939 INFO:tasks.workunit.client.0.smithi022.stdout:2/586: creat d0/d32/fbb x:0 0 0 2024-02-16T19:29:11.943 INFO:tasks.workunit.client.1.smithi117.stdout:7/598: creat d4/dd3/fde x:0 0 0 2024-02-16T19:29:11.944 INFO:tasks.workunit.client.0.smithi022.stdout:5/644: creat dd/d1f/d48/d21/d2f/d52/d5c/d61/fda x:0 0 0 2024-02-16T19:29:11.944 INFO:tasks.workunit.client.1.smithi117.stdout:8/602: symlink d4/d12/d31/d8e/lba 0 2024-02-16T19:29:11.944 INFO:tasks.workunit.client.1.smithi117.stdout:3/694: rename d1/c58 to d1/dd/d3b/d9d/ceb 0 2024-02-16T19:29:11.945 INFO:tasks.workunit.client.1.smithi117.stdout:6/592: truncate d5/db/d7d/d32/f5b 790156 0 2024-02-16T19:29:11.945 INFO:tasks.workunit.client.0.smithi022.stdout:6/623: getdents d1/d8/da/d17/d53 0 2024-02-16T19:29:11.948 INFO:tasks.workunit.client.1.smithi117.stdout:6/593: chown d5/db/l45 214673508 1 2024-02-16T19:29:11.948 INFO:tasks.workunit.client.1.smithi117.stdout:4/595: truncate d0/da/d21/d3d/d28/d64/f1f 3691500 0 2024-02-16T19:29:11.948 INFO:tasks.workunit.client.0.smithi022.stdout:1/660: fdatasync d6/d8/d22/d3f/f73 0 2024-02-16T19:29:11.949 INFO:tasks.workunit.client.1.smithi117.stdout:7/599: read - d4/d1c/d39/d46/d47/f77 zero size 2024-02-16T19:29:11.949 INFO:tasks.workunit.client.1.smithi117.stdout:9/517: dwrite dd/db5/fb1 [0,4194304] 0 2024-02-16T19:29:11.949 INFO:tasks.workunit.client.0.smithi022.stdout:1/661: dread - d6/d8/d22/f82 zero size 2024-02-16T19:29:11.950 INFO:tasks.workunit.client.0.smithi022.stdout:8/578: creat d3/d12/d2b/d54/d74/da2/dc0/fd0 x:0 0 0 2024-02-16T19:29:11.951 INFO:tasks.workunit.client.1.smithi117.stdout:7/600: chown d4/d1c/d39/d46/d47 120143451 1 2024-02-16T19:29:11.952 INFO:tasks.workunit.client.0.smithi022.stdout:4/544: rename d4/d6/d14/d64/d4e/d54/d8a to d4/d6/dd/d22/da8 0 2024-02-16T19:29:11.954 INFO:tasks.workunit.client.1.smithi117.stdout:0/634: dwrite d0/d9/d14/d1d/d60/d76/f9f [0,4194304] 0 2024-02-16T19:29:11.954 INFO:tasks.workunit.client.0.smithi022.stdout:4/545: write d4/d6/d14/f1d [940533,57814] 0 2024-02-16T19:29:11.954 INFO:tasks.workunit.client.0.smithi022.stdout:9/618: dread db/d18/f73 [0,4194304] 0 2024-02-16T19:29:11.955 INFO:tasks.workunit.client.1.smithi117.stdout:0/635: fdatasync d0/d9/d22/d27/d83/f95 0 2024-02-16T19:29:11.958 INFO:tasks.workunit.client.0.smithi022.stdout:2/587: symlink d0/d2/d18/d38/d6b/lbc 0 2024-02-16T19:29:11.960 INFO:tasks.workunit.client.0.smithi022.stdout:3/521: dread d7/d21/f36 [4194304,4194304] 0 2024-02-16T19:29:11.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:11 smithi022 ceph-mon[29221]: from='client.14628 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:29:11.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:11 smithi022 ceph-mon[29221]: from='client.14632 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:29:11.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:11 smithi022 ceph-mon[29221]: mgrmap e25: smithi117.cohbtv(active, since 11s), standbys: smithi022.gmdcch 2024-02-16T19:29:11.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:11 smithi022 ceph-mon[29221]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "mgr metadata", "who": "smithi022.gmdcch", "id": "smithi022.gmdcch"}]: dispatch 2024-02-16T19:29:11.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:11 smithi022 ceph-mon[29221]: from='client.? 172.21.15.22:0/347594203' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:29:11.967 INFO:tasks.workunit.client.1.smithi117.stdout:6/594: creat d5/d4b/d71/fc1 x:0 0 0 2024-02-16T19:29:11.968 INFO:tasks.workunit.client.1.smithi117.stdout:4/596: chown d0/da/d21/d3d/d28/d64/d73/d91/d9e/fb6 429 1 2024-02-16T19:29:11.973 INFO:tasks.workunit.client.1.smithi117.stdout:3/695: fdatasync d1/dd/d3b/f3d 0 2024-02-16T19:29:11.975 INFO:tasks.workunit.client.0.smithi022.stdout:7/581: getdents d9/da/d1f 0 2024-02-16T19:29:11.978 INFO:tasks.workunit.client.1.smithi117.stdout:0/636: creat d0/d9/d22/d27/d31/fe2 x:0 0 0 2024-02-16T19:29:11.979 INFO:tasks.workunit.client.1.smithi117.stdout:5/651: getdents d2/d9/d3c/d24/d90 0 2024-02-16T19:29:11.979 INFO:tasks.workunit.client.1.smithi117.stdout:5/652: readlink d2/d9/dba/lcd 0 2024-02-16T19:29:11.984 INFO:tasks.workunit.client.1.smithi117.stdout:7/601: mknod d4/d1c/cdf 0 2024-02-16T19:29:11.984 INFO:tasks.workunit.client.1.smithi117.stdout:7/602: chown d4/d1c/dc0 214 1 2024-02-16T19:29:11.986 INFO:tasks.workunit.client.1.smithi117.stdout:3/696: mknod d1/dd/d3b/cec 0 2024-02-16T19:29:11.988 INFO:tasks.workunit.client.0.smithi022.stdout:8/579: mkdir d3/d12/d2b/d46/dd1 0 2024-02-16T19:29:11.988 INFO:tasks.workunit.client.0.smithi022.stdout:6/624: write d1/d8/da/d17/d85/f5f [910408,23318] 0 2024-02-16T19:29:11.992 INFO:tasks.workunit.client.0.smithi022.stdout:4/546: fdatasync d4/d6/dd/df/d10/d24/f88 0 2024-02-16T19:29:11.994 INFO:tasks.workunit.client.1.smithi117.stdout:8/603: link d4/d1c/l28 d4/d7/d6a/d6f/da8/lbb 0 2024-02-16T19:29:11.994 INFO:tasks.workunit.client.0.smithi022.stdout:4/547: write d4/d6/d14/d64/d66/d6d/f7e [4791930,11854] 0 2024-02-16T19:29:11.996 INFO:tasks.workunit.client.0.smithi022.stdout:9/619: link db/d14/d19/fa2 db/d14/d16/d28/daa/fd3 0 2024-02-16T19:29:11.997 INFO:tasks.workunit.client.1.smithi117.stdout:6/595: creat d5/d4b/d20/d69/fc2 x:0 0 0 2024-02-16T19:29:11.997 INFO:tasks.workunit.client.0.smithi022.stdout:9/620: write db/d14/d16/d5d/d65/d6b/fcb [21319,126863] 0 2024-02-16T19:29:11.999 INFO:tasks.workunit.client.0.smithi022.stdout:2/588: link d0/d2/d18/d19/d90/fa3 d0/d2/d18/d19/d90/fbd 0 2024-02-16T19:29:12.000 INFO:tasks.workunit.client.0.smithi022.stdout:2/589: truncate d0/d2/d2e/fba 1011860 0 2024-02-16T19:29:12.001 INFO:tasks.workunit.client.1.smithi117.stdout:4/597: link d0/da/d21/d3d/d28/d64/d73/d91/f31 d0/da/d21/d3d/d28/d64/d73/fbf 0 2024-02-16T19:29:12.001 INFO:tasks.workunit.client.0.smithi022.stdout:5/645: link dd/d1f/d2e/l5b dd/d1f/d48/d21/d2f/d40/ldb 0 2024-02-16T19:29:12.002 INFO:tasks.workunit.client.1.smithi117.stdout:5/653: write d2/d4c/d4f/f81 [4438372,9078] 0 2024-02-16T19:29:12.003 INFO:tasks.workunit.client.1.smithi117.stdout:0/637: dwrite d0/d9/d14/d1d/d60/d76/d7b/d8d/f48 [0,4194304] 0 2024-02-16T19:29:12.005 INFO:tasks.workunit.client.0.smithi022.stdout:7/582: dwrite d9/da/f3a [0,4194304] 0 2024-02-16T19:29:12.007 INFO:tasks.workunit.client.1.smithi117.stdout:7/603: creat d4/d1c/d39/d46/d55/fe0 x:0 0 0 2024-02-16T19:29:12.008 INFO:tasks.workunit.client.1.smithi117.stdout:7/604: fsync d4/d13/d63/d71/fcb 0 2024-02-16T19:29:12.009 INFO:tasks.workunit.client.0.smithi022.stdout:1/662: creat d6/d8/d22/fcf x:0 0 0 2024-02-16T19:29:12.013 INFO:tasks.workunit.client.0.smithi022.stdout:8/580: rename d3/d12/d2b/l7b to d3/d25/d33/ld2 0 2024-02-16T19:29:12.013 INFO:tasks.workunit.client.0.smithi022.stdout:5/646: dwrite dd/d1f/d48/d21/d2f/d46/d5e/d84/fa3 [0,4194304] 0 2024-02-16T19:29:12.015 INFO:tasks.workunit.client.1.smithi117.stdout:7/605: dwrite d4/d1c/d87/d89/d60/d79/fc1 [0,4194304] 0 2024-02-16T19:29:12.015 INFO:tasks.workunit.client.0.smithi022.stdout:5/647: chown dd/d1f/d4a/f3f 196486 1 2024-02-16T19:29:12.018 INFO:tasks.workunit.client.1.smithi117.stdout:6/596: mkdir d5/d4b/d44/d93/dae/dc3 0 2024-02-16T19:29:12.019 INFO:tasks.workunit.client.0.smithi022.stdout:6/625: dread d1/d8/da/d17/d85/f36 [0,4194304] 0 2024-02-16T19:29:12.022 INFO:tasks.workunit.client.1.smithi117.stdout:2/609: sync 2024-02-16T19:29:12.023 INFO:tasks.workunit.client.0.smithi022.stdout:3/522: truncate d7/d12/d18/d4b/f4d 119025 0 2024-02-16T19:29:12.024 INFO:tasks.workunit.client.1.smithi117.stdout:5/654: symlink d2/db3/ldc 0 2024-02-16T19:29:12.025 INFO:tasks.workunit.client.1.smithi117.stdout:5/655: dread - d2/d4c/d4f/d67/fd5 zero size 2024-02-16T19:29:12.025 INFO:tasks.workunit.client.0.smithi022.stdout:4/548: creat d4/d38/d6a/fa9 x:0 0 0 2024-02-16T19:29:12.026 INFO:tasks.workunit.client.1.smithi117.stdout:3/697: creat d1/dd/d3b/dba/de9/fed x:0 0 0 2024-02-16T19:29:12.027 INFO:tasks.workunit.client.0.smithi022.stdout:9/621: truncate db/d14/d33/f4f 1369497 0 2024-02-16T19:29:12.031 INFO:tasks.workunit.client.1.smithi117.stdout:7/606: mkdir d4/d1c/d39/d9f/de1 0 2024-02-16T19:29:12.033 INFO:tasks.workunit.client.1.smithi117.stdout:5/656: fdatasync d2/d4c/d4f/d67/f7f 0 2024-02-16T19:29:12.034 INFO:tasks.workunit.client.1.smithi117.stdout:5/657: stat d2/d3/fb6 0 2024-02-16T19:29:12.037 INFO:tasks.workunit.client.0.smithi022.stdout:8/581: dread d3/d12/d2b/d54/f70 [0,4194304] 0 2024-02-16T19:29:12.038 INFO:tasks.workunit.client.0.smithi022.stdout:8/582: chown d3/d12/d2b/d54/dcd 1 1 2024-02-16T19:29:12.039 INFO:tasks.workunit.client.0.smithi022.stdout:6/626: mknod d1/d8/da/d17/d53/ce0 0 2024-02-16T19:29:12.039 INFO:tasks.workunit.client.0.smithi022.stdout:0/767: sync 2024-02-16T19:29:12.040 INFO:tasks.workunit.client.0.smithi022.stdout:3/523: mknod d7/d12/d18/dd/d24/d58/d6b/cbe 0 2024-02-16T19:29:12.040 INFO:tasks.workunit.client.0.smithi022.stdout:5/648: dwrite dd/d1f/d48/d21/d2f/d52/d5c/d61/d6a/f70 [4194304,4194304] 0 2024-02-16T19:29:12.041 INFO:tasks.workunit.client.0.smithi022.stdout:8/583: truncate d3/d25/f79 1363604 0 2024-02-16T19:29:12.042 INFO:tasks.workunit.client.0.smithi022.stdout:8/584: chown d3/d12/d2b/dca 42 1 2024-02-16T19:29:12.042 INFO:tasks.workunit.client.0.smithi022.stdout:4/549: unlink d4/d6/d14/d64/d4e/d54/d5a/c72 0 2024-02-16T19:29:12.044 INFO:tasks.workunit.client.0.smithi022.stdout:4/550: chown d4/d6/dd/d61 1 1 2024-02-16T19:29:12.049 INFO:tasks.workunit.client.0.smithi022.stdout:0/768: write d10/f17 [1319366,43428] 0 2024-02-16T19:29:12.050 INFO:tasks.workunit.client.0.smithi022.stdout:0/769: chown fb 13585 1 2024-02-16T19:29:12.050 INFO:tasks.workunit.client.0.smithi022.stdout:1/663: mknod d6/d8/d22/d16/d17/cd0 0 2024-02-16T19:29:12.054 INFO:tasks.workunit.client.0.smithi022.stdout:6/627: fsync d1/d8/da/d29/f35 0 2024-02-16T19:29:12.056 INFO:tasks.workunit.client.0.smithi022.stdout:3/524: mkdir d7/d12/d18/dd/d31/dbf 0 2024-02-16T19:29:12.057 INFO:tasks.workunit.client.0.smithi022.stdout:8/585: write d3/d12/da6/f7 [2379893,2488] 0 2024-02-16T19:29:12.058 INFO:tasks.workunit.client.0.smithi022.stdout:5/649: creat dd/d1f/d48/d21/d2f/d40/d49/d4b/fdc x:0 0 0 2024-02-16T19:29:12.060 INFO:tasks.workunit.client.0.smithi022.stdout:4/551: creat d4/d6/d14/d64/d66/d6d/faa x:0 0 0 2024-02-16T19:29:12.060 INFO:tasks.workunit.client.0.smithi022.stdout:4/552: readlink d4/l7a 0 2024-02-16T19:29:12.062 INFO:tasks.workunit.client.0.smithi022.stdout:2/590: creat d0/fbe x:0 0 0 2024-02-16T19:29:12.062 INFO:tasks.workunit.client.0.smithi022.stdout:0/770: dread d10/d22/d2c/f72 [0,4194304] 0 2024-02-16T19:29:12.063 INFO:tasks.workunit.client.0.smithi022.stdout:0/771: dread - d10/d7d/f93 zero size 2024-02-16T19:29:12.064 INFO:tasks.workunit.client.0.smithi022.stdout:7/583: link d9/da/d2c/d46/f6f d9/d1b/fb7 0 2024-02-16T19:29:12.067 INFO:tasks.workunit.client.0.smithi022.stdout:6/628: fsync d1/d8/da/d17/d85/f2b 0 2024-02-16T19:29:12.068 INFO:tasks.workunit.client.1.smithi117.stdout:8/604: creat d4/d12/d2a/fbc x:0 0 0 2024-02-16T19:29:12.068 INFO:tasks.workunit.client.1.smithi117.stdout:2/610: symlink d0/d3/d34/d94/d46/d68/d93/da9/lb5 0 2024-02-16T19:29:12.069 INFO:tasks.workunit.client.1.smithi117.stdout:5/658: read d2/d36/fb2 [6276710,116069] 0 2024-02-16T19:29:12.070 INFO:tasks.workunit.client.1.smithi117.stdout:5/659: chown d2/f63 0 1 2024-02-16T19:29:12.070 INFO:tasks.workunit.client.1.smithi117.stdout:4/598: write d0/da/d21/f22 [8751928,74219] 0 2024-02-16T19:29:12.070 INFO:tasks.workunit.client.0.smithi022.stdout:3/525: symlink d7/d12/d18/dd/d24/lc0 0 2024-02-16T19:29:12.072 INFO:tasks.workunit.client.0.smithi022.stdout:8/586: mkdir d3/dd3 0 2024-02-16T19:29:12.072 INFO:tasks.workunit.client.0.smithi022.stdout:6/629: dread d1/d8/da/d17/d48/d51/f59 [0,4194304] 0 2024-02-16T19:29:12.073 INFO:tasks.workunit.client.0.smithi022.stdout:6/630: dread - d1/d8/f6f zero size 2024-02-16T19:29:12.075 INFO:tasks.workunit.client.1.smithi117.stdout:3/698: dwrite d1/d1b/d1e/d29/f82 [0,4194304] 0 2024-02-16T19:29:12.079 INFO:tasks.workunit.client.1.smithi117.stdout:2/611: mknod d0/d3/d34/d94/d46/d68/d76/da1/d2f/cb6 0 2024-02-16T19:29:12.079 INFO:tasks.workunit.client.0.smithi022.stdout:9/622: dwrite db/d18/f2a [0,4194304] 0 2024-02-16T19:29:12.081 INFO:tasks.workunit.client.0.smithi022.stdout:4/553: rename d4/d6/d14/d64/d66/d6d/f73 to d4/d6/dd/d22/d83/d9e/fab 0 2024-02-16T19:29:12.085 INFO:tasks.workunit.client.0.smithi022.stdout:7/584: fsync d9/d1b/f27 0 2024-02-16T19:29:12.087 INFO:tasks.workunit.client.1.smithi117.stdout:5/660: truncate d2/d3/d21/f33 1515410 0 2024-02-16T19:29:12.087 INFO:tasks.workunit.client.1.smithi117.stdout:2/612: dwrite d0/f6c [0,4194304] 0 2024-02-16T19:29:12.087 INFO:tasks.workunit.client.0.smithi022.stdout:1/664: truncate d6/f6b 2057028 0 2024-02-16T19:29:12.089 INFO:tasks.workunit.client.1.smithi117.stdout:5/661: stat d2/d9/dba/cd6 0 2024-02-16T19:29:12.089 INFO:tasks.workunit.client.1.smithi117.stdout:2/613: stat d0/d3/d34/d94/d46/d68/f8f 0 2024-02-16T19:29:12.089 INFO:tasks.workunit.client.1.smithi117.stdout:7/607: link d4/d13/d63/c6a d4/d1c/dd8/ce2 0 2024-02-16T19:29:12.090 INFO:tasks.workunit.client.1.smithi117.stdout:1/540: sync 2024-02-16T19:29:12.092 INFO:tasks.workunit.client.1.smithi117.stdout:8/605: mknod d4/d7/da6/d3a/cbd 0 2024-02-16T19:29:12.093 INFO:tasks.workunit.client.1.smithi117.stdout:2/614: chown d0/d3/d34/d94/d46/d68/d76/da1/d56 719498 1 2024-02-16T19:29:12.093 INFO:tasks.workunit.client.1.smithi117.stdout:8/606: chown d4/d7/da6/d3a/d54/f78 55922 1 2024-02-16T19:29:12.096 INFO:tasks.workunit.client.1.smithi117.stdout:3/699: unlink d1/dd/f43 0 2024-02-16T19:29:12.097 INFO:tasks.workunit.client.1.smithi117.stdout:2/615: dread d0/d3/d22/f7b [0,4194304] 0 2024-02-16T19:29:12.097 INFO:tasks.workunit.client.1.smithi117.stdout:2/616: fdatasync d0/d3/d40/fa3 0 2024-02-16T19:29:12.098 INFO:tasks.workunit.client.0.smithi022.stdout:3/526: stat d7/d12/d18/dd/d31/l8a 0 2024-02-16T19:29:12.101 INFO:tasks.workunit.client.1.smithi117.stdout:4/599: fdatasync d0/da/d21/d3d/d28/d64/d8a/d92/f85 0 2024-02-16T19:29:12.103 INFO:tasks.workunit.client.1.smithi117.stdout:4/600: dread - d0/da/d21/d3d/d28/d64/d73/fa9 zero size 2024-02-16T19:29:12.103 INFO:tasks.workunit.client.0.smithi022.stdout:6/631: unlink d1/d8/d3c/d4d/f7e 0 2024-02-16T19:29:12.105 INFO:tasks.workunit.client.1.smithi117.stdout:5/662: creat d2/d9/d3c/d24/d50/d96/dbc/d30/d38/fdd x:0 0 0 2024-02-16T19:29:12.106 INFO:tasks.workunit.client.0.smithi022.stdout:2/591: dwrite d0/f17 [0,4194304] 0 2024-02-16T19:29:12.108 INFO:tasks.workunit.client.1.smithi117.stdout:5/663: write d2/d9/d3c/d24/d50/d96/dbc/d30/d38/f64 [1464542,25672] 0 2024-02-16T19:29:12.108 INFO:tasks.workunit.client.1.smithi117.stdout:1/541: symlink d2/d1d/d60/d7c/lb8 0 2024-02-16T19:29:12.112 INFO:tasks.workunit.client.1.smithi117.stdout:7/608: creat d4/d1c/d39/d46/d55/db8/fe3 x:0 0 0 2024-02-16T19:29:12.115 INFO:tasks.workunit.client.1.smithi117.stdout:1/542: dwrite d2/d19/d59/f97 [0,4194304] 0 2024-02-16T19:29:12.121 INFO:tasks.workunit.client.1.smithi117.stdout:8/607: mknod d4/d7/da6/d3a/d43/cbe 0 2024-02-16T19:29:12.127 INFO:tasks.workunit.client.1.smithi117.stdout:9/518: sync 2024-02-16T19:29:12.127 INFO:tasks.workunit.client.1.smithi117.stdout:0/638: sync 2024-02-16T19:29:12.128 INFO:tasks.workunit.client.0.smithi022.stdout:4/554: dread d4/d6/dd/d22/f68 [0,4194304] 0 2024-02-16T19:29:12.129 INFO:tasks.workunit.client.1.smithi117.stdout:9/519: truncate dd/d45/d73/f99 167109 0 2024-02-16T19:29:12.129 INFO:tasks.workunit.client.1.smithi117.stdout:0/639: write d0/d9/d14/d1d/d60/d76/d7b/fb1 [659080,116383] 0 2024-02-16T19:29:12.130 INFO:tasks.workunit.client.0.smithi022.stdout:7/585: creat d9/da/d1f/d93/fb8 x:0 0 0 2024-02-16T19:29:12.132 INFO:tasks.workunit.client.0.smithi022.stdout:1/665: creat d6/d8/d22/d59/da7/fd1 x:0 0 0 2024-02-16T19:29:12.133 INFO:tasks.workunit.client.1.smithi117.stdout:4/601: mkdir d0/da/d21/d3d/d28/d64/d90/dc0 0 2024-02-16T19:29:12.138 INFO:tasks.workunit.client.0.smithi022.stdout:8/587: write d3/d12/d92/fc5 [1033170,28736] 0 2024-02-16T19:29:12.139 INFO:tasks.workunit.client.1.smithi117.stdout:4/602: dwrite d0/da/d21/f22 [4194304,4194304] 0 2024-02-16T19:29:12.141 INFO:tasks.workunit.client.0.smithi022.stdout:5/650: link dd/d1f/d48/d21/d2f/d40/d4e/f87 dd/d1f/d48/d21/d2f/d52/d5c/d61/d6a/db0/fdd 0 2024-02-16T19:29:12.142 INFO:tasks.workunit.client.1.smithi117.stdout:4/603: read - d0/da/d21/d3d/d28/d64/d73/d91/fa5 zero size 2024-02-16T19:29:12.144 INFO:tasks.workunit.client.0.smithi022.stdout:2/592: mknod d0/d2/d2e/d82/cbf 0 2024-02-16T19:29:12.146 INFO:tasks.workunit.client.1.smithi117.stdout:7/609: dread d4/f10 [0,4194304] 0 2024-02-16T19:29:12.148 INFO:tasks.workunit.client.0.smithi022.stdout:6/632: dread d1/d8/da/d17/f2c [0,4194304] 0 2024-02-16T19:29:12.148 INFO:tasks.workunit.client.0.smithi022.stdout:9/623: dwrite db/d14/d19/da8/f81 [0,4194304] 0 2024-02-16T19:29:12.148 INFO:tasks.workunit.client.0.smithi022.stdout:0/772: write d10/d11/d79/d8b/fb9 [3335161,44150] 0 2024-02-16T19:29:12.152 INFO:tasks.workunit.client.0.smithi022.stdout:4/555: creat d4/d6/dd/fac x:0 0 0 2024-02-16T19:29:12.152 INFO:tasks.workunit.client.1.smithi117.stdout:8/608: dread d4/d7/d6a/f93 [0,4194304] 0 2024-02-16T19:29:12.156 INFO:tasks.workunit.client.1.smithi117.stdout:3/700: dwrite d1/dd/d3b/f3d [0,4194304] 0 2024-02-16T19:29:12.161 INFO:tasks.workunit.client.1.smithi117.stdout:5/664: mkdir d2/d4c/dde 0 2024-02-16T19:29:12.165 INFO:tasks.workunit.client.1.smithi117.stdout:1/543: symlink d2/d4/lb9 0 2024-02-16T19:29:12.166 INFO:tasks.workunit.client.1.smithi117.stdout:1/544: fdatasync d2/d4/f9 0 2024-02-16T19:29:12.167 INFO:tasks.workunit.client.1.smithi117.stdout:9/520: dread dd/db5/f32 [0,4194304] 0 2024-02-16T19:29:12.168 INFO:tasks.workunit.client.1.smithi117.stdout:7/610: mkdir d4/d1c/d39/d46/d47/de4 0 2024-02-16T19:29:12.174 INFO:tasks.workunit.client.1.smithi117.stdout:8/609: mkdir d4/d12/d31/d8e/dbf 0 2024-02-16T19:29:12.176 INFO:tasks.workunit.client.1.smithi117.stdout:3/701: creat d1/d67/fee x:0 0 0 2024-02-16T19:29:12.176 INFO:tasks.workunit.client.1.smithi117.stdout:2/617: dwrite d0/d3/d10/f16 [0,4194304] 0 2024-02-16T19:29:12.181 INFO:tasks.workunit.client.1.smithi117.stdout:9/521: chown dd/d45/d31/f93 21 1 2024-02-16T19:29:12.182 INFO:tasks.workunit.client.1.smithi117.stdout:8/610: mknod d4/d94/cc0 0 2024-02-16T19:29:12.183 INFO:tasks.workunit.client.1.smithi117.stdout:3/702: symlink d1/d30/d52/lef 0 2024-02-16T19:29:12.184 INFO:tasks.workunit.client.1.smithi117.stdout:8/611: write d4/d1c/fad [769004,18489] 0 2024-02-16T19:29:12.185 INFO:tasks.workunit.client.1.smithi117.stdout:2/618: creat d0/d3/d22/d37/fb7 x:0 0 0 2024-02-16T19:29:12.187 INFO:tasks.workunit.client.1.smithi117.stdout:0/640: dwrite d0/d9/d22/d27/d4e/f62 [0,4194304] 0 2024-02-16T19:29:12.187 INFO:tasks.workunit.client.1.smithi117.stdout:1/545: mkdir d2/d1d/d60/d7c/d9e/da5/db0/dba 0 2024-02-16T19:29:12.187 INFO:tasks.workunit.client.1.smithi117.stdout:3/703: chown d1/d30/d52/fc9 727 1 2024-02-16T19:29:12.189 INFO:tasks.workunit.client.1.smithi117.stdout:0/641: chown d0/d9/d22/d27/d83/fb6 56965502 1 2024-02-16T19:29:12.194 INFO:tasks.workunit.client.1.smithi117.stdout:8/612: dread d4/d12/d2a/f48 [0,4194304] 0 2024-02-16T19:29:12.196 INFO:tasks.workunit.client.1.smithi117.stdout:7/611: dread d4/d1c/d87/d89/fd4 [0,4194304] 0 2024-02-16T19:29:12.197 INFO:tasks.workunit.client.1.smithi117.stdout:1/546: write d2/d19/d59/d89/d69/f8f [287943,27803] 0 2024-02-16T19:29:12.197 INFO:tasks.workunit.client.1.smithi117.stdout:3/704: symlink d1/d30/lf0 0 2024-02-16T19:29:12.200 INFO:tasks.workunit.client.1.smithi117.stdout:5/665: dwrite d2/d3/f47 [0,4194304] 0 2024-02-16T19:29:12.202 INFO:tasks.workunit.client.1.smithi117.stdout:7/612: chown d4/d1c/d39/d46/f96 487 1 2024-02-16T19:29:12.206 INFO:tasks.workunit.client.1.smithi117.stdout:0/642: rename d0/d9/d22/d27/d31/d6d/f82 to d0/d9/d14/d1d/d60/d65/fe3 0 2024-02-16T19:29:12.207 INFO:tasks.workunit.client.1.smithi117.stdout:8/613: truncate d4/d12/d31/f40 1858323 0 2024-02-16T19:29:12.208 INFO:tasks.workunit.client.1.smithi117.stdout:3/705: rename d1/d1b/d1e/d29/f82 to d1/dd/d3b/ff1 0 2024-02-16T19:29:12.208 INFO:tasks.workunit.client.1.smithi117.stdout:0/643: chown d0/d9/d14/d1d/d60/dc3/fcc 23178 1 2024-02-16T19:29:12.208 INFO:tasks.workunit.client.1.smithi117.stdout:9/522: write dd/d2d/f4d [3826324,85340] 0 2024-02-16T19:29:12.210 INFO:tasks.workunit.client.1.smithi117.stdout:3/706: write d1/dd/d5c/d85/fd9 [565442,15928] 0 2024-02-16T19:29:12.212 INFO:tasks.workunit.client.1.smithi117.stdout:7/613: fsync d4/d13/da3/fb1 0 2024-02-16T19:29:12.212 INFO:tasks.workunit.client.1.smithi117.stdout:3/707: readlink d1/l27 0 2024-02-16T19:29:12.213 INFO:tasks.workunit.client.1.smithi117.stdout:8/614: write d4/d7/f9d [242512,48323] 0 2024-02-16T19:29:12.215 INFO:tasks.workunit.client.1.smithi117.stdout:5/666: rmdir d2/d9/d3c/d24/d50/d96/dbc/d30/d38/db5/dda 0 2024-02-16T19:29:12.215 INFO:tasks.workunit.client.1.smithi117.stdout:9/523: write dd/db5/f52 [989876,33289] 0 2024-02-16T19:29:12.215 INFO:tasks.workunit.client.1.smithi117.stdout:0/644: creat d0/d9/d22/db7/fe4 x:0 0 0 2024-02-16T19:29:12.216 INFO:tasks.workunit.client.1.smithi117.stdout:8/615: symlink d4/d1c/lc1 0 2024-02-16T19:29:12.216 INFO:tasks.workunit.client.1.smithi117.stdout:3/708: fdatasync d1/dd/d3b/dba/de9/fed 0 2024-02-16T19:29:12.219 INFO:tasks.workunit.client.1.smithi117.stdout:9/524: mknod dd/d45/d31/d63/cb7 0 2024-02-16T19:29:12.221 INFO:tasks.workunit.client.1.smithi117.stdout:8/616: rmdir d4/d7/d6a/d6f/da8/d76 39 2024-02-16T19:29:12.223 INFO:tasks.workunit.client.1.smithi117.stdout:5/667: link d2/f63 d2/d5d/fdf 0 2024-02-16T19:29:12.225 INFO:tasks.workunit.client.1.smithi117.stdout:3/709: rename d1/dd/c5d to d1/d30/d9f/dbc/cf2 0 2024-02-16T19:29:12.230 INFO:tasks.workunit.client.1.smithi117.stdout:9/525: dwrite dd/d23/f25 [0,4194304] 0 2024-02-16T19:29:12.230 INFO:tasks.workunit.client.1.smithi117.stdout:7/614: truncate d4/d13/da3/fb1 92943 0 2024-02-16T19:29:12.232 INFO:tasks.workunit.client.1.smithi117.stdout:3/710: chown d1/d1b/d1e/d29/f33 0 1 2024-02-16T19:29:12.235 INFO:tasks.workunit.client.1.smithi117.stdout:1/547: dread d2/d1d/f3a [0,4194304] 0 2024-02-16T19:29:12.235 INFO:tasks.workunit.client.1.smithi117.stdout:6/597: sync 2024-02-16T19:29:12.236 INFO:tasks.workunit.client.1.smithi117.stdout:0/645: dread d0/d9/d22/d27/d4e/fac [0,4194304] 0 2024-02-16T19:29:12.236 INFO:tasks.workunit.client.1.smithi117.stdout:9/526: write dd/db5/f32 [37868,97537] 0 2024-02-16T19:29:12.238 INFO:tasks.workunit.client.1.smithi117.stdout:3/711: rename d1/dd/d3b/da3/lda to d1/d83/lf3 0 2024-02-16T19:29:12.241 INFO:tasks.workunit.client.1.smithi117.stdout:8/617: dwrite d4/d12/d2a/fbc [0,4194304] 0 2024-02-16T19:29:12.243 INFO:tasks.workunit.client.1.smithi117.stdout:6/598: symlink d5/d9c/daa/lc4 0 2024-02-16T19:29:12.243 INFO:tasks.workunit.client.1.smithi117.stdout:5/668: link d2/db3/db0/f43 d2/d9/d3c/d24/d50/d96/dbc/d30/d38/d51/fe0 0 2024-02-16T19:29:12.246 INFO:tasks.workunit.client.1.smithi117.stdout:3/712: symlink d1/d1b/d1e/d29/lf4 0 2024-02-16T19:29:12.247 INFO:tasks.workunit.client.1.smithi117.stdout:8/618: read d4/d12/d2a/f48 [61083,17888] 0 2024-02-16T19:29:12.247 INFO:tasks.workunit.client.1.smithi117.stdout:0/646: dread d0/d9/f1c [0,4194304] 0 2024-02-16T19:29:12.250 INFO:tasks.workunit.client.1.smithi117.stdout:6/599: chown d5/db/d7d/d32/l7b 6325433 1 2024-02-16T19:29:12.251 INFO:tasks.workunit.client.1.smithi117.stdout:9/527: getdents dd/d47/d5b/d5f/d77/d7c 0 2024-02-16T19:29:12.251 INFO:tasks.workunit.client.1.smithi117.stdout:7/615: dwrite d4/d13/d63/fce [0,4194304] 0 2024-02-16T19:29:12.253 INFO:tasks.workunit.client.1.smithi117.stdout:6/600: chown d5/f11 0 1 2024-02-16T19:29:12.254 INFO:tasks.workunit.client.1.smithi117.stdout:8/619: chown d4/d7/da6/d3a/d54/c67 136131759 1 2024-02-16T19:29:12.255 INFO:tasks.workunit.client.1.smithi117.stdout:9/528: stat dd/d47/d5e/lae 0 2024-02-16T19:29:12.256 INFO:tasks.workunit.client.1.smithi117.stdout:0/647: truncate d0/d9/d22/d27/d83/f67 1126277 0 2024-02-16T19:29:12.256 INFO:tasks.workunit.client.1.smithi117.stdout:3/713: dread d1/dd/d5c/d85/fd9 [0,4194304] 0 2024-02-16T19:29:12.257 INFO:tasks.workunit.client.1.smithi117.stdout:9/529: chown dd/d23/daf 1 1 2024-02-16T19:29:12.257 INFO:tasks.workunit.client.1.smithi117.stdout:7/616: rmdir d4/d1c/d87/d89/d5f 39 2024-02-16T19:29:12.259 INFO:tasks.workunit.client.1.smithi117.stdout:8/620: chown d4/d12/d2a/f2b 83 1 2024-02-16T19:29:12.261 INFO:tasks.workunit.client.1.smithi117.stdout:9/530: dread - dd/d45/f49 zero size 2024-02-16T19:29:12.261 INFO:tasks.workunit.client.1.smithi117.stdout:7/617: write d4/dd3/fb9 [4865619,34652] 0 2024-02-16T19:29:12.262 INFO:tasks.workunit.client.1.smithi117.stdout:3/714: symlink d1/d1b/d1e/d37/ddc/lf5 0 2024-02-16T19:29:12.264 INFO:tasks.workunit.client.1.smithi117.stdout:0/648: creat d0/d9/d2c/da9/fe5 x:0 0 0 2024-02-16T19:29:12.267 INFO:tasks.workunit.client.1.smithi117.stdout:9/531: unlink dd/l14 0 2024-02-16T19:29:12.267 INFO:tasks.workunit.client.1.smithi117.stdout:5/669: dread d2/d9/d3c/f1a [0,4194304] 0 2024-02-16T19:29:12.272 INFO:tasks.workunit.client.1.smithi117.stdout:0/649: creat d0/d9/d14/d1d/d60/d76/d7b/fe6 x:0 0 0 2024-02-16T19:29:12.272 INFO:tasks.workunit.client.1.smithi117.stdout:9/532: read - dd/d47/d5b/d5f/f65 zero size 2024-02-16T19:29:12.274 INFO:tasks.workunit.client.1.smithi117.stdout:6/601: dwrite d5/d4b/d44/f75 [0,4194304] 0 2024-02-16T19:29:12.275 INFO:tasks.workunit.client.1.smithi117.stdout:3/715: link d1/dd/d5c/l7c d1/d1b/d1e/d37/ddc/lf6 0 2024-02-16T19:29:12.276 INFO:tasks.workunit.client.1.smithi117.stdout:8/621: dwrite d4/d7/da6/d3a/d43/f6b [0,4194304] 0 2024-02-16T19:29:12.277 INFO:tasks.workunit.client.0.smithi022.stdout:3/527: write d7/d12/d18/dd/d24/d58/d70/f81 [4467823,78348] 0 2024-02-16T19:29:12.277 INFO:tasks.workunit.client.0.smithi022.stdout:2/593: rmdir d0/d2/d18/d38/d6b 39 2024-02-16T19:29:12.278 INFO:tasks.workunit.client.0.smithi022.stdout:6/633: rmdir d1/d8/da/d17/d53 39 2024-02-16T19:29:12.278 INFO:tasks.workunit.client.1.smithi117.stdout:0/650: fdatasync d0/d30/f66 0 2024-02-16T19:29:12.279 INFO:tasks.workunit.client.0.smithi022.stdout:0/773: rename d10/d11/fc9 to d10/d15/f100 0 2024-02-16T19:29:12.280 INFO:tasks.workunit.client.1.smithi117.stdout:3/716: chown d1/dd/d3e/c72 924 1 2024-02-16T19:29:12.280 INFO:tasks.workunit.client.1.smithi117.stdout:3/717: chown d1/dd 0 1 2024-02-16T19:29:12.282 INFO:tasks.workunit.client.0.smithi022.stdout:2/594: chown d0/d2/f3d 17412687 1 2024-02-16T19:29:12.283 INFO:tasks.workunit.client.0.smithi022.stdout:6/634: read - d1/d8/da/d29/d61/d57/f60 zero size 2024-02-16T19:29:12.283 INFO:tasks.workunit.client.1.smithi117.stdout:3/718: chown d1/dd/d3b/cc4 0 1 2024-02-16T19:29:12.286 INFO:tasks.workunit.client.1.smithi117.stdout:3/719: write d1/d30/f90 [4599158,58854] 0 2024-02-16T19:29:12.287 INFO:tasks.workunit.client.0.smithi022.stdout:7/586: dread d9/da/d2c/d46/d73/f8e [0,4194304] 0 2024-02-16T19:29:12.288 INFO:tasks.workunit.client.0.smithi022.stdout:9/624: rename db/d14/d16/d21/fc5 to db/d14/d16/d28/fd4 0 2024-02-16T19:29:12.288 INFO:tasks.workunit.client.1.smithi117.stdout:5/670: dwrite d2/d5d/faf [0,4194304] 0 2024-02-16T19:29:12.289 INFO:tasks.workunit.client.1.smithi117.stdout:0/651: mkdir d0/d9/d22/d27/d31/de7 0 2024-02-16T19:29:12.289 INFO:tasks.workunit.client.0.smithi022.stdout:1/666: dwrite d6/d8/d22/d16/d17/d5d/f95 [0,4194304] 0 2024-02-16T19:29:12.290 INFO:tasks.workunit.client.1.smithi117.stdout:0/652: readlink d0/d9/d14/d23/la7 0 2024-02-16T19:29:12.292 INFO:tasks.workunit.client.0.smithi022.stdout:0/774: unlink d10/d15/d23/f47 0 2024-02-16T19:29:12.295 INFO:tasks.workunit.client.0.smithi022.stdout:3/528: symlink d7/d12/d18/dd/d24/d53/lc1 0 2024-02-16T19:29:12.299 INFO:tasks.workunit.client.1.smithi117.stdout:6/602: write d5/db/d7d/d32/d95/f50 [2680653,68811] 0 2024-02-16T19:29:12.301 INFO:tasks.workunit.client.0.smithi022.stdout:1/667: dwrite d6/d8/d22/d16/d17/d1d/fbe [0,4194304] 0 2024-02-16T19:29:12.302 INFO:tasks.workunit.client.0.smithi022.stdout:2/595: symlink d0/d2/d18/d19/d90/lc0 0 2024-02-16T19:29:12.304 INFO:tasks.workunit.client.1.smithi117.stdout:3/720: symlink d1/dd/d3b/dba/de9/lf7 0 2024-02-16T19:29:12.309 INFO:tasks.workunit.client.0.smithi022.stdout:7/587: creat d9/da/d1f/d40/fb9 x:0 0 0 2024-02-16T19:29:12.311 INFO:tasks.workunit.client.1.smithi117.stdout:9/533: link dd/d47/d5e/c89 dd/d47/d5e/d9a/cb8 0 2024-02-16T19:29:12.315 INFO:tasks.workunit.client.1.smithi117.stdout:3/721: chown d1/d83/lf3 0 1 2024-02-16T19:29:12.316 INFO:tasks.workunit.client.0.smithi022.stdout:9/625: dwrite db/d14/d16/d28/daa/fad [0,4194304] 0 2024-02-16T19:29:12.317 INFO:tasks.workunit.client.0.smithi022.stdout:9/626: fsync db/d14/d16/d21/fb6 0 2024-02-16T19:29:12.318 INFO:tasks.workunit.client.0.smithi022.stdout:0/775: read d10/d11/d27/d3f/d58/f6a [3391547,64520] 0 2024-02-16T19:29:12.319 INFO:tasks.workunit.client.0.smithi022.stdout:9/627: readlink db/d14/d16/d27/d2e/l6f 0 2024-02-16T19:29:12.321 INFO:tasks.workunit.client.0.smithi022.stdout:3/529: creat d7/d21/fc2 x:0 0 0 2024-02-16T19:29:12.322 INFO:tasks.workunit.client.0.smithi022.stdout:3/530: stat d7/d21/f5d 0 2024-02-16T19:29:12.323 INFO:tasks.workunit.client.0.smithi022.stdout:8/588: getdents d3/d12/d17/d64 0 2024-02-16T19:29:12.324 INFO:tasks.workunit.client.0.smithi022.stdout:0/776: dread d10/f17 [0,4194304] 0 2024-02-16T19:29:12.325 INFO:tasks.workunit.client.0.smithi022.stdout:0/777: chown d10/d11/f73 29838002 1 2024-02-16T19:29:12.326 INFO:tasks.workunit.client.0.smithi022.stdout:0/778: chown d10/d11/f73 10216 1 2024-02-16T19:29:12.329 INFO:tasks.workunit.client.0.smithi022.stdout:2/596: creat d0/d2/d18/d19/fc1 x:0 0 0 2024-02-16T19:29:12.333 INFO:tasks.workunit.client.0.smithi022.stdout:7/588: mkdir d9/da/d3b/dba 0 2024-02-16T19:29:12.334 INFO:tasks.workunit.client.1.smithi117.stdout:5/671: write d2/d9/d3c/d24/d50/d96/dbc/d30/d38/d51/fe0 [1511880,130464] 0 2024-02-16T19:29:12.334 INFO:tasks.workunit.client.1.smithi117.stdout:0/653: dwrite d0/d9/d14/d1d/d60/d76/d7b/d8d/f36 [0,4194304] 0 2024-02-16T19:29:12.336 INFO:tasks.workunit.client.1.smithi117.stdout:8/622: getdents d4/d12/d2a/d64 0 2024-02-16T19:29:12.338 INFO:tasks.workunit.client.1.smithi117.stdout:3/722: mkdir d1/dd/df8 0 2024-02-16T19:29:12.340 INFO:tasks.workunit.client.1.smithi117.stdout:0/654: dread d0/d9/f1c [0,4194304] 0 2024-02-16T19:29:12.340 INFO:tasks.workunit.client.1.smithi117.stdout:6/603: dwrite d5/d4b/d44/d52/f6c [4194304,4194304] 0 2024-02-16T19:29:12.340 INFO:tasks.workunit.client.0.smithi022.stdout:1/668: dwrite d6/d8/f2d [0,4194304] 0 2024-02-16T19:29:12.342 INFO:tasks.workunit.client.0.smithi022.stdout:1/669: write d6/d8/d22/d59/da7/fd1 [649057,62051] 0 2024-02-16T19:29:12.343 INFO:tasks.workunit.client.1.smithi117.stdout:0/655: truncate d0/d9/d14/d23/d33/fd8 1168852 0 2024-02-16T19:29:12.345 INFO:tasks.workunit.client.0.smithi022.stdout:0/779: write d10/d11/d27/d3f/f7a [2411823,78956] 0 2024-02-16T19:29:12.347 INFO:tasks.workunit.client.0.smithi022.stdout:6/635: link d1/d8/da/d17/c27 d1/d8/da/d7b/dae/dbb/ce1 0 2024-02-16T19:29:12.348 INFO:tasks.workunit.client.1.smithi117.stdout:3/723: unlink d1/dd/c35 0 2024-02-16T19:29:12.349 INFO:tasks.workunit.client.0.smithi022.stdout:7/589: creat d9/da/d1f/d44/fbb x:0 0 0 2024-02-16T19:29:12.350 INFO:tasks.workunit.client.1.smithi117.stdout:9/534: dread dd/d47/d48/f55 [0,4194304] 0 2024-02-16T19:29:12.351 INFO:tasks.workunit.client.1.smithi117.stdout:3/724: write d1/dd/d3b/dba/de9/fed [339842,1733] 0 2024-02-16T19:29:12.352 INFO:tasks.workunit.client.1.smithi117.stdout:0/656: chown d0/d9/d2c/da9/faa 1668870551 1 2024-02-16T19:29:12.353 INFO:tasks.workunit.client.0.smithi022.stdout:2/597: mknod d0/d2/d2e/d82/da1/cc2 0 2024-02-16T19:29:12.361 INFO:tasks.workunit.client.1.smithi117.stdout:6/604: rename d5/db/d7d/d32/f43 to d5/d4b/d44/d93/fc5 0 2024-02-16T19:29:12.363 INFO:tasks.workunit.client.0.smithi022.stdout:0/780: mknod d10/d11/d27/d3f/d58/d60/dbc/c101 0 2024-02-16T19:29:12.364 INFO:tasks.workunit.client.1.smithi117.stdout:4/604: sync 2024-02-16T19:29:12.367 INFO:tasks.workunit.client.1.smithi117.stdout:4/605: truncate d0/da/d21/d3d/d28/d64/d48/fab 787027 0 2024-02-16T19:29:12.369 INFO:tasks.workunit.client.1.smithi117.stdout:9/535: mknod dd/d8d/cb9 0 2024-02-16T19:29:12.370 INFO:tasks.workunit.client.1.smithi117.stdout:9/536: dread - dd/d23/d64/f91 zero size 2024-02-16T19:29:12.371 INFO:tasks.workunit.client.1.smithi117.stdout:0/657: dread d0/d30/fd2 [0,4194304] 0 2024-02-16T19:29:12.371 INFO:tasks.workunit.client.0.smithi022.stdout:9/628: write db/d14/d16/d21/fc1 [4400396,20520] 0 2024-02-16T19:29:12.372 INFO:tasks.workunit.client.0.smithi022.stdout:4/556: getdents d4/d6/dd/d22/da8 0 2024-02-16T19:29:12.375 INFO:tasks.workunit.client.1.smithi117.stdout:8/623: rename d4/d12/d2a/d64/la5 to d4/d7/d6a/d6f/da8/d76/d8c/lc2 0 2024-02-16T19:29:12.376 INFO:tasks.workunit.client.1.smithi117.stdout:8/624: chown d4/d7/d6a/d72/d84 265760820 1 2024-02-16T19:29:12.377 INFO:tasks.workunit.client.0.smithi022.stdout:4/557: chown d4/d38/d8d/f99 14 1 2024-02-16T19:29:12.377 INFO:tasks.workunit.client.0.smithi022.stdout:3/531: creat d7/d12/d18/dd/d24/d29/fc3 x:0 0 0 2024-02-16T19:29:12.378 INFO:tasks.workunit.client.1.smithi117.stdout:5/672: write d2/d9/d3c/d24/d50/d52/d74/d8c/f4b [4454689,83624] 0 2024-02-16T19:29:12.383 INFO:tasks.workunit.client.1.smithi117.stdout:4/606: dread - d0/da/d21/d3d/d28/d64/d73/d91/f8f zero size 2024-02-16T19:29:12.385 INFO:tasks.workunit.client.0.smithi022.stdout:8/589: creat d3/fd4 x:0 0 0 2024-02-16T19:29:12.387 INFO:tasks.workunit.client.1.smithi117.stdout:0/658: write d0/fd [2810598,88582] 0 2024-02-16T19:29:12.388 INFO:tasks.workunit.client.1.smithi117.stdout:3/725: dwrite d1/dd/f48 [0,4194304] 0 2024-02-16T19:29:12.388 INFO:tasks.workunit.client.0.smithi022.stdout:0/781: mkdir d10/d11/d50/d102 0 2024-02-16T19:29:12.389 INFO:tasks.workunit.client.0.smithi022.stdout:0/782: fsync d10/d11/d27/d3f/f5b 0 2024-02-16T19:29:12.390 INFO:tasks.workunit.client.0.smithi022.stdout:6/636: mknod d1/d8/d23/db1/ce2 0 2024-02-16T19:29:12.393 INFO:tasks.workunit.client.1.smithi117.stdout:8/625: rename d4/d12/f1a to d4/d7/da6/d50/fc3 0 2024-02-16T19:29:12.394 INFO:tasks.workunit.client.0.smithi022.stdout:7/590: mkdir d9/da9/dbc 0 2024-02-16T19:29:12.400 INFO:tasks.workunit.client.1.smithi117.stdout:2/619: sync 2024-02-16T19:29:12.400 INFO:tasks.workunit.client.1.smithi117.stdout:1/548: sync 2024-02-16T19:29:12.401 INFO:tasks.workunit.client.0.smithi022.stdout:3/532: dread d7/d12/d18/dd/d31/d7f/fa0 [0,4194304] 0 2024-02-16T19:29:12.401 INFO:tasks.workunit.client.1.smithi117.stdout:3/726: stat d1/c3 0 2024-02-16T19:29:12.402 INFO:tasks.workunit.client.1.smithi117.stdout:2/620: chown d0/d3/d10/f17 49 1 2024-02-16T19:29:12.402 INFO:tasks.workunit.client.1.smithi117.stdout:1/549: read d2/d1d/d1f/f56 [4913936,31689] 0 2024-02-16T19:29:12.404 INFO:tasks.workunit.client.1.smithi117.stdout:4/607: dread d0/da/d21/d3d/d28/fa6 [0,4194304] 0 2024-02-16T19:29:12.406 INFO:tasks.workunit.client.1.smithi117.stdout:5/673: rename d2/d4c/d4f/d67/f94 to d2/d9/d3c/d24/d50/d96/dbc/d30/d38/fe1 0 2024-02-16T19:29:12.408 INFO:tasks.workunit.client.1.smithi117.stdout:0/659: dwrite d0/d9/d14/d1d/d60/dc3/fcc [0,4194304] 0 2024-02-16T19:29:12.415 INFO:tasks.workunit.client.1.smithi117.stdout:9/537: truncate dd/d47/d5b/fa4 1277738 0 2024-02-16T19:29:12.415 INFO:tasks.workunit.client.1.smithi117.stdout:9/538: readlink l8 0 2024-02-16T19:29:12.416 INFO:tasks.workunit.client.1.smithi117.stdout:3/727: symlink d1/d1b/d1e/d37/ddc/lf9 0 2024-02-16T19:29:12.424 INFO:tasks.workunit.client.0.smithi022.stdout:4/558: creat d4/d6/dd/d22/d30/fad x:0 0 0 2024-02-16T19:29:12.425 INFO:tasks.workunit.client.1.smithi117.stdout:5/674: chown d2/d5d/ca4 1392989322 1 2024-02-16T19:29:12.427 INFO:tasks.workunit.client.1.smithi117.stdout:6/605: getdents d5/db/d7d/d32/d95/d49/d59 0 2024-02-16T19:29:12.427 INFO:tasks.workunit.client.0.smithi022.stdout:1/670: link d6/d8/d22/d67/f77 d6/d8/dba/dc7/fd2 0 2024-02-16T19:29:12.428 INFO:tasks.workunit.client.0.smithi022.stdout:1/671: chown d6/f5b 521 1 2024-02-16T19:29:12.430 INFO:tasks.workunit.client.1.smithi117.stdout:8/626: dwrite d4/d7/d6a/f93 [0,4194304] 0 2024-02-16T19:29:12.430 INFO:tasks.workunit.client.1.smithi117.stdout:0/660: truncate d0/d9/d14/f1b 2523977 0 2024-02-16T19:29:12.430 INFO:tasks.workunit.client.0.smithi022.stdout:2/598: link d0/d32/d3b/d6f/fac d0/d2/d2e/d82/fc3 0 2024-02-16T19:29:12.431 INFO:tasks.workunit.client.1.smithi117.stdout:1/550: write d2/d19/f2f [4085029,45354] 0 2024-02-16T19:29:12.432 INFO:tasks.workunit.client.1.smithi117.stdout:8/627: write d4/d7/da6/f63 [903940,3238] 0 2024-02-16T19:29:12.432 INFO:tasks.workunit.client.1.smithi117.stdout:7/618: sync 2024-02-16T19:29:12.433 INFO:tasks.workunit.client.0.smithi022.stdout:8/590: mkdir d3/d12/d2b/da9/dd5 0 2024-02-16T19:29:12.434 INFO:tasks.workunit.client.0.smithi022.stdout:8/591: stat d3/d12/d2b/d54/fcb 0 2024-02-16T19:29:12.434 INFO:tasks.workunit.client.1.smithi117.stdout:8/628: stat d4/d73/f82 0 2024-02-16T19:29:12.436 INFO:tasks.workunit.client.1.smithi117.stdout:5/675: dwrite d2/d9/d3c/d24/d50/d96/dbc/d30/d38/f64 [0,4194304] 0 2024-02-16T19:29:12.436 INFO:tasks.workunit.client.1.smithi117.stdout:9/539: mkdir dd/d45/d31/d63/dba 0 2024-02-16T19:29:12.438 INFO:tasks.workunit.client.0.smithi022.stdout:0/783: truncate d10/d11/f73 1000862 0 2024-02-16T19:29:12.444 INFO:tasks.workunit.client.1.smithi117.stdout:1/551: dread d2/d19/d59/d89/d69/f8f [0,4194304] 0 2024-02-16T19:29:12.444 INFO:tasks.workunit.client.1.smithi117.stdout:4/608: symlink d0/da/lc1 0 2024-02-16T19:29:12.445 INFO:tasks.workunit.client.1.smithi117.stdout:4/609: stat d0/da/d21/d3d/f5e 0 2024-02-16T19:29:12.446 INFO:tasks.workunit.client.0.smithi022.stdout:9/629: dwrite db/d14/d16/d27/f68 [0,4194304] 0 2024-02-16T19:29:12.446 INFO:tasks.workunit.client.0.smithi022.stdout:1/672: dread d6/d8/d22/d59/fc4 [0,4194304] 0 2024-02-16T19:29:12.450 INFO:tasks.workunit.client.0.smithi022.stdout:9/630: dread - db/d14/d16/d21/d37/fb1 zero size 2024-02-16T19:29:12.453 INFO:tasks.workunit.client.0.smithi022.stdout:3/533: read d7/d12/d18/dd/ff [765562,70949] 0 2024-02-16T19:29:12.455 INFO:tasks.workunit.client.0.smithi022.stdout:3/534: stat d7/d21/f7c 0 2024-02-16T19:29:12.459 INFO:tasks.workunit.client.0.smithi022.stdout:4/559: dread d4/d6/dd/df/d10/f19 [0,4194304] 0 2024-02-16T19:29:12.459 INFO:tasks.workunit.client.1.smithi117.stdout:0/661: truncate d0/d9/d14/f92 861721 0 2024-02-16T19:29:12.460 INFO:tasks.workunit.client.1.smithi117.stdout:0/662: dread d0/f53 [0,4194304] 0 2024-02-16T19:29:12.463 INFO:tasks.workunit.client.0.smithi022.stdout:0/784: readlink d10/d15/d23/d8f/dc1/l63 0 2024-02-16T19:29:12.466 INFO:tasks.workunit.client.0.smithi022.stdout:6/637: mknod d1/d8/da/d17/d53/d7f/db3/ce3 0 2024-02-16T19:29:12.466 INFO:tasks.workunit.client.1.smithi117.stdout:0/663: dwrite d0/d9/d14/d1d/d60/d76/f9f [0,4194304] 0 2024-02-16T19:29:12.469 INFO:tasks.workunit.client.1.smithi117.stdout:0/664: read d0/d9/d14/d1d/d60/d76/d7b/fb1 [198139,47309] 0 2024-02-16T19:29:12.470 INFO:tasks.workunit.client.1.smithi117.stdout:0/665: write d0/f5 [5652333,115691] 0 2024-02-16T19:29:12.474 INFO:tasks.workunit.client.1.smithi117.stdout:7/619: write d4/d1c/f26 [925514,100662] 0 2024-02-16T19:29:12.475 INFO:tasks.workunit.client.0.smithi022.stdout:9/631: mkdir db/d14/d16/d5d/d65/d6b/dd5 0 2024-02-16T19:29:12.475 INFO:tasks.workunit.client.1.smithi117.stdout:7/620: dread - d4/d1c/d87/d89/d60/fba zero size 2024-02-16T19:29:12.476 INFO:tasks.workunit.client.0.smithi022.stdout:7/591: dread d9/da/d3b/f9f [0,4194304] 0 2024-02-16T19:29:12.477 INFO:tasks.workunit.client.1.smithi117.stdout:8/629: write d4/d1c/f49 [673957,120676] 0 2024-02-16T19:29:12.477 INFO:tasks.workunit.client.1.smithi117.stdout:1/552: write d2/d4/f15 [2702859,66587] 0 2024-02-16T19:29:12.479 INFO:tasks.workunit.client.0.smithi022.stdout:3/535: mkdir d7/d12/d59/dc4 0 2024-02-16T19:29:12.479 INFO:tasks.workunit.client.0.smithi022.stdout:3/536: chown d7/d12/c2b 24393 1 2024-02-16T19:29:12.480 INFO:tasks.workunit.client.1.smithi117.stdout:4/610: mkdir d0/da/d21/dc2 0 2024-02-16T19:29:12.480 INFO:tasks.workunit.client.1.smithi117.stdout:9/540: write dd/d45/d31/f75 [932058,53934] 0 2024-02-16T19:29:12.481 INFO:tasks.workunit.client.0.smithi022.stdout:1/673: write d6/d8/d22/d16/d17/d26/f43 [2909216,3523] 0 2024-02-16T19:29:12.483 INFO:tasks.workunit.client.1.smithi117.stdout:2/621: rename d0/d3/d22/d37/c47 to d0/d13/cb8 0 2024-02-16T19:29:12.484 INFO:tasks.workunit.client.0.smithi022.stdout:4/560: symlink d4/d6/d14/d64/d66/d6d/lae 0 2024-02-16T19:29:12.484 INFO:tasks.workunit.client.0.smithi022.stdout:8/592: truncate d3/f2d 419310 0 2024-02-16T19:29:12.486 INFO:tasks.workunit.client.0.smithi022.stdout:0/785: symlink d10/d15/d23/d8f/dc1/l103 0 2024-02-16T19:29:12.498 INFO:tasks.workunit.client.0.smithi022.stdout:5/651: sync 2024-02-16T19:29:12.506 INFO:tasks.workunit.client.0.smithi022.stdout:7/592: rename d9/da/d1f/d44/fbb to d9/da/d1f/d40/fbd 0 2024-02-16T19:29:12.509 INFO:tasks.workunit.client.0.smithi022.stdout:6/638: dread d1/d8/da/d17/d85/d5b/f63 [0,4194304] 0 2024-02-16T19:29:12.510 INFO:tasks.workunit.client.1.smithi117.stdout:5/676: mkdir d2/d3/d21/d49/d59/daa/de2 0 2024-02-16T19:29:12.517 INFO:tasks.workunit.client.1.smithi117.stdout:8/630: mknod d4/d7/da6/d3a/d43/d44/cc4 0 2024-02-16T19:29:12.522 INFO:tasks.workunit.client.1.smithi117.stdout:5/677: dread d2/db3/db0/f43 [0,4194304] 0 2024-02-16T19:29:12.523 INFO:tasks.workunit.client.1.smithi117.stdout:9/541: mkdir dd/d45/d31/d63/d8e/dbb 0 2024-02-16T19:29:12.525 INFO:tasks.workunit.client.1.smithi117.stdout:6/606: rename d5/d9c/daa/fb8 to d5/d4b/d20/da4/fc6 0 2024-02-16T19:29:12.528 INFO:tasks.workunit.client.1.smithi117.stdout:4/611: write d0/da/d21/d3d/d28/f4e [511253,95792] 0 2024-02-16T19:29:12.530 INFO:tasks.workunit.client.1.smithi117.stdout:7/621: dwrite d4/d1c/d39/d46/f96 [0,4194304] 0 2024-02-16T19:29:12.530 INFO:tasks.workunit.client.1.smithi117.stdout:3/728: getdents d1/d1b/d1e/d37/ddc 0 2024-02-16T19:29:12.532 INFO:tasks.workunit.client.1.smithi117.stdout:8/631: creat d4/d7/d6a/d6f/da8/d76/fc5 x:0 0 0 2024-02-16T19:29:12.532 INFO:tasks.workunit.client.1.smithi117.stdout:4/612: chown d0/da/d21/l71 166229369 1 2024-02-16T19:29:12.533 INFO:tasks.workunit.client.1.smithi117.stdout:3/729: fsync d1/d67/fee 0 2024-02-16T19:29:12.533 INFO:tasks.workunit.client.1.smithi117.stdout:9/542: dwrite dd/d45/d73/f8a [0,4194304] 0 2024-02-16T19:29:12.536 INFO:tasks.workunit.client.1.smithi117.stdout:8/632: chown d4/d7/da6/d3a/cbd 132 1 2024-02-16T19:29:12.538 INFO:tasks.workunit.client.1.smithi117.stdout:7/622: truncate d4/d1c/d39/d9f/fd0 1019389 0 2024-02-16T19:29:12.539 INFO:tasks.workunit.client.1.smithi117.stdout:7/623: truncate d4/f4d 1391565 0 2024-02-16T19:29:12.539 INFO:tasks.workunit.client.1.smithi117.stdout:9/543: creat dd/d47/d48/fbc x:0 0 0 2024-02-16T19:29:12.541 INFO:tasks.workunit.client.1.smithi117.stdout:1/553: rename d2/d19/d59/d89/d7d to d2/d4/dbb 0 2024-02-16T19:29:12.543 INFO:tasks.workunit.client.1.smithi117.stdout:9/544: truncate dd/d23/f1a 291940 0 2024-02-16T19:29:12.544 INFO:tasks.workunit.client.1.smithi117.stdout:9/545: dread - dd/fa9 zero size 2024-02-16T19:29:12.545 INFO:tasks.workunit.client.1.smithi117.stdout:3/730: rmdir d1/d67/dce 0 2024-02-16T19:29:12.546 INFO:tasks.workunit.client.1.smithi117.stdout:2/622: rename d0/d3/d34/d94/d46/d68/d76/da1/fb0 to d0/d13/d4d/d80/fb9 0 2024-02-16T19:29:12.548 INFO:tasks.workunit.client.1.smithi117.stdout:3/731: truncate d1/d1b/d1e/d37/f65 4392568 0 2024-02-16T19:29:12.548 INFO:tasks.workunit.client.1.smithi117.stdout:7/624: getdents d4/d1c/d39/d46/da4 0 2024-02-16T19:29:12.549 INFO:tasks.workunit.client.1.smithi117.stdout:7/625: readlink d4/d13/l15 0 2024-02-16T19:29:12.550 INFO:tasks.workunit.client.1.smithi117.stdout:9/546: creat dd/d47/d5b/fbd x:0 0 0 2024-02-16T19:29:12.553 INFO:tasks.workunit.client.1.smithi117.stdout:6/607: rename d5/db/d7d/d32/d95 to d5/d4b/d71/dc7 0 2024-02-16T19:29:12.556 INFO:tasks.workunit.client.1.smithi117.stdout:0/666: sync 2024-02-16T19:29:12.557 INFO:tasks.workunit.client.1.smithi117.stdout:4/613: rename d0/da/d21/d3d/d28/d66/d97/lb2 to d0/da/d21/dc2/lc3 0 2024-02-16T19:29:12.558 INFO:tasks.workunit.client.1.smithi117.stdout:0/667: chown d0/d30/f66 9912775 1 2024-02-16T19:29:12.558 INFO:tasks.workunit.client.1.smithi117.stdout:5/678: truncate d2/db3/f3f 2372562 0 2024-02-16T19:29:12.559 INFO:tasks.workunit.client.1.smithi117.stdout:3/732: symlink d1/dd/df8/lfa 0 2024-02-16T19:29:12.565 INFO:tasks.workunit.client.1.smithi117.stdout:8/633: rename d4/d7/da6/d3a/d65/d6c/da9 to d4/d12/d2a/d64/dc6 0 2024-02-16T19:29:12.566 INFO:tasks.workunit.client.1.smithi117.stdout:8/634: chown d4/d7/da6/d3a/d65/d96/c97 5373 1 2024-02-16T19:29:12.567 INFO:tasks.workunit.client.0.smithi022.stdout:0/786: symlink d10/d11/d27/d77/l104 0 2024-02-16T19:29:12.567 INFO:tasks.workunit.client.0.smithi022.stdout:5/652: mknod dd/d1f/d48/d21/d2f/d40/d49/cde 0 2024-02-16T19:29:12.568 INFO:tasks.workunit.client.1.smithi117.stdout:1/554: write d2/d19/d59/d89/d69/f8f [727474,52589] 0 2024-02-16T19:29:12.572 INFO:tasks.workunit.client.0.smithi022.stdout:9/632: mknod db/d14/cd6 0 2024-02-16T19:29:12.572 INFO:tasks.workunit.client.0.smithi022.stdout:3/537: mkdir d7/d12/d18/dd/d31/dbf/dc5 0 2024-02-16T19:29:12.572 INFO:tasks.workunit.client.0.smithi022.stdout:2/599: getdents d0/d2/d18/d19/d40/d55/d59 0 2024-02-16T19:29:12.572 INFO:tasks.workunit.client.0.smithi022.stdout:6/639: dwrite d1/d8/da/d17/d53/d7f/fd3 [0,4194304] 0 2024-02-16T19:29:12.572 INFO:tasks.workunit.client.1.smithi117.stdout:1/555: truncate d2/f9b 50514 0 2024-02-16T19:29:12.572 INFO:tasks.workunit.client.1.smithi117.stdout:1/556: write d2/d1d/f29 [590456,107008] 0 2024-02-16T19:29:12.572 INFO:tasks.workunit.client.1.smithi117.stdout:5/679: creat d2/d9/fe3 x:0 0 0 2024-02-16T19:29:12.572 INFO:tasks.workunit.client.1.smithi117.stdout:2/623: write d0/d13/f15 [3812404,32602] 0 2024-02-16T19:29:12.573 INFO:tasks.workunit.client.1.smithi117.stdout:2/624: write d0/d3/d10/f90 [686869,41458] 0 2024-02-16T19:29:12.576 INFO:tasks.workunit.client.1.smithi117.stdout:0/668: creat d0/d9/d22/db7/fe8 x:0 0 0 2024-02-16T19:29:12.579 INFO:tasks.workunit.client.0.smithi022.stdout:4/561: dread d4/d6/d3a/f41 [0,4194304] 0 2024-02-16T19:29:12.581 INFO:tasks.workunit.client.0.smithi022.stdout:4/562: chown d4/d6/d14/d64/d66 250174650 1 2024-02-16T19:29:12.581 INFO:tasks.workunit.client.0.smithi022.stdout:9/633: dwrite db/d18/f2a [0,4194304] 0 2024-02-16T19:29:12.581 INFO:tasks.workunit.client.0.smithi022.stdout:7/593: mknod d9/da/d1f/d44/d5b/d95/cbe 0 2024-02-16T19:29:12.583 INFO:tasks.workunit.client.1.smithi117.stdout:9/547: creat dd/d47/fbe x:0 0 0 2024-02-16T19:29:12.584 INFO:tasks.workunit.client.0.smithi022.stdout:7/594: read d9/da/d2c/d46/d73/f8e [593918,111362] 0 2024-02-16T19:29:12.588 INFO:tasks.workunit.client.1.smithi117.stdout:7/626: rename d4/d1c/d39/d46/d55/l59 to d4/d13/d63/ddd/le5 0 2024-02-16T19:29:12.589 INFO:tasks.workunit.client.1.smithi117.stdout:6/608: truncate d5/fa 3125951 0 2024-02-16T19:29:12.591 INFO:tasks.workunit.client.1.smithi117.stdout:4/614: symlink d0/da/d21/db4/lc4 0 2024-02-16T19:29:12.592 INFO:tasks.workunit.client.0.smithi022.stdout:3/538: symlink d7/lc6 0 2024-02-16T19:29:12.596 INFO:tasks.workunit.client.0.smithi022.stdout:2/600: rename d0/d2/d18/d19/d40/db1 to d0/d5d/dc4 0 2024-02-16T19:29:12.597 INFO:tasks.workunit.client.0.smithi022.stdout:8/593: write d3/d25/f69 [111569,14875] 0 2024-02-16T19:29:12.597 INFO:tasks.workunit.client.1.smithi117.stdout:1/557: creat d2/d51/fbc x:0 0 0 2024-02-16T19:29:12.599 INFO:tasks.workunit.client.0.smithi022.stdout:4/563: dread d4/d38/d8d/f99 [0,4194304] 0 2024-02-16T19:29:12.600 INFO:tasks.workunit.client.0.smithi022.stdout:6/640: mkdir d1/d8/da/d17/d48/de4 0 2024-02-16T19:29:12.601 INFO:tasks.workunit.client.1.smithi117.stdout:5/680: symlink d2/d9/dba/le4 0 2024-02-16T19:29:12.604 INFO:tasks.workunit.client.1.smithi117.stdout:2/625: creat d0/d3/d40/fba x:0 0 0 2024-02-16T19:29:12.605 INFO:tasks.workunit.client.1.smithi117.stdout:2/626: chown d0/d3/fc 1387350239 1 2024-02-16T19:29:12.606 INFO:tasks.workunit.client.1.smithi117.stdout:2/627: dread - d0/d3/d34/d94/d46/f8a zero size 2024-02-16T19:29:12.606 INFO:tasks.workunit.client.1.smithi117.stdout:0/669: unlink d0/d9/d14/d1d/d60/d65/fa3 0 2024-02-16T19:29:12.607 INFO:tasks.workunit.client.1.smithi117.stdout:0/670: fdatasync d0/f53 0 2024-02-16T19:29:12.607 INFO:tasks.workunit.client.1.smithi117.stdout:2/628: read d0/d3/d40/f66 [227173,92050] 0 2024-02-16T19:29:12.608 INFO:tasks.workunit.client.1.smithi117.stdout:0/671: dread - d0/d9/d22/d27/d83/faf zero size 2024-02-16T19:29:12.608 INFO:tasks.workunit.client.0.smithi022.stdout:9/634: symlink db/d14/d16/d49/d90/ld7 0 2024-02-16T19:29:12.609 INFO:tasks.workunit.client.1.smithi117.stdout:0/672: dread d0/d9/d2c/da9/faa [0,4194304] 0 2024-02-16T19:29:12.610 INFO:tasks.workunit.client.1.smithi117.stdout:2/629: dread d0/d3/d34/d94/d46/d68/d76/da1/d56/f72 [0,4194304] 0 2024-02-16T19:29:12.611 INFO:tasks.workunit.client.0.smithi022.stdout:5/653: mkdir dd/d1f/d2e/dd4/ddf 0 2024-02-16T19:29:12.612 INFO:tasks.workunit.client.0.smithi022.stdout:7/595: dread - d9/d5d/f97 zero size 2024-02-16T19:29:12.613 INFO:tasks.workunit.client.1.smithi117.stdout:0/673: dwrite d0/f40 [0,4194304] 0 2024-02-16T19:29:12.617 INFO:tasks.workunit.client.0.smithi022.stdout:8/594: unlink d3/d12/db0/f78 0 2024-02-16T19:29:12.617 INFO:tasks.workunit.client.1.smithi117.stdout:7/627: mknod d4/d1c/d39/d46/d55/ce6 0 2024-02-16T19:29:12.618 INFO:tasks.workunit.client.0.smithi022.stdout:4/564: creat d4/d6/dd/df/d10/d24/d4b/faf x:0 0 0 2024-02-16T19:29:12.625 INFO:tasks.workunit.client.1.smithi117.stdout:4/615: rmdir d0/da/d21/d3d/d28 39 2024-02-16T19:29:12.631 INFO:tasks.workunit.client.0.smithi022.stdout:9/635: rename ca to db/d14/d16/d5d/d65/d6b/d89/dd2/cd8 0 2024-02-16T19:29:12.639 INFO:tasks.workunit.client.0.smithi022.stdout:2/601: mknod d0/d2/d18/d38/db6/cc5 0 2024-02-16T19:29:12.639 INFO:tasks.workunit.client.1.smithi117.stdout:9/548: write dd/d23/f3b [5194086,69254] 0 2024-02-16T19:29:12.640 INFO:tasks.workunit.client.1.smithi117.stdout:3/733: link d1/f8 d1/d1b/d1e/d29/daf/ffb 0 2024-02-16T19:29:12.640 INFO:tasks.workunit.client.1.smithi117.stdout:9/549: chown dd/d47/d5e/d9a 37319 1 2024-02-16T19:29:12.641 INFO:tasks.workunit.client.1.smithi117.stdout:9/550: fdatasync dd/d47/d5b/f8c 0 2024-02-16T19:29:12.642 INFO:tasks.workunit.client.0.smithi022.stdout:4/565: symlink d4/d6/dd/d22/da8/lb0 0 2024-02-16T19:29:12.645 INFO:tasks.workunit.client.0.smithi022.stdout:7/596: dread d9/da/d2c/d46/f6a [0,4194304] 0 2024-02-16T19:29:12.649 INFO:tasks.workunit.client.0.smithi022.stdout:5/654: creat dd/d1f/d48/d21/d2f/d40/d4e/d56/d9d/fe0 x:0 0 0 2024-02-16T19:29:12.654 INFO:tasks.workunit.client.1.smithi117.stdout:8/635: rename d4/d7/da6/d3a/d43/d44/faf to d4/d7/da6/d3a/d65/d96/fc7 0 2024-02-16T19:29:12.656 INFO:tasks.workunit.client.0.smithi022.stdout:2/602: rmdir d0/d2/d18/d19/d40 39 2024-02-16T19:29:12.657 INFO:tasks.workunit.client.0.smithi022.stdout:2/603: chown d0/d2/d18/d1f/d6c/f9e 5752583 1 2024-02-16T19:29:12.657 INFO:tasks.workunit.client.1.smithi117.stdout:7/628: symlink d4/d1c/d39/d46/d55/le7 0 2024-02-16T19:29:12.660 INFO:tasks.workunit.client.1.smithi117.stdout:1/558: symlink d2/d1d/d60/d68/lbd 0 2024-02-16T19:29:12.660 INFO:tasks.workunit.client.1.smithi117.stdout:2/630: dwrite d0/d3/d34/d94/d46/d68/f8f [0,4194304] 0 2024-02-16T19:29:12.662 INFO:tasks.workunit.client.1.smithi117.stdout:1/559: dread - d2/d19/d59/d89/d7b/d86/fb7 zero size 2024-02-16T19:29:12.663 INFO:tasks.workunit.client.1.smithi117.stdout:5/681: getdents d2/d4c/d69/dc2 0 2024-02-16T19:29:12.663 INFO:tasks.workunit.client.1.smithi117.stdout:0/674: dread d0/d9/d22/d27/f5f [0,4194304] 0 2024-02-16T19:29:12.663 INFO:tasks.workunit.client.0.smithi022.stdout:8/595: rename d3/d12/da6/l57 to d3/d25/ld6 0 2024-02-16T19:29:12.663 INFO:tasks.workunit.client.0.smithi022.stdout:8/596: stat d3 0 2024-02-16T19:29:12.664 INFO:tasks.workunit.client.1.smithi117.stdout:5/682: dread - d2/d3/d21/d49/d59/fb8 zero size 2024-02-16T19:29:12.669 INFO:tasks.workunit.client.1.smithi117.stdout:3/734: creat d1/d1b/d1e/d37/ffc x:0 0 0 2024-02-16T19:29:12.671 INFO:tasks.workunit.client.0.smithi022.stdout:4/566: fdatasync d4/d6/dd/d61/fa4 0 2024-02-16T19:29:12.672 INFO:tasks.workunit.client.0.smithi022.stdout:6/641: getdents d1/d8/da/d17/d85 0 2024-02-16T19:29:12.674 INFO:tasks.workunit.client.0.smithi022.stdout:7/597: dwrite d9/da/d1f/f81 [0,4194304] 0 2024-02-16T19:29:12.674 INFO:tasks.workunit.client.1.smithi117.stdout:6/609: rename d5/db/d7d/d32/d79/ca5 to d5/d4b/d20/d69/cc8 0 2024-02-16T19:29:12.676 INFO:tasks.workunit.client.0.smithi022.stdout:8/597: unlink d3/d12/d17/f6d 0 2024-02-16T19:29:12.677 INFO:tasks.workunit.client.0.smithi022.stdout:8/598: chown d3/d12/f9c 0 1 2024-02-16T19:29:12.683 INFO:tasks.workunit.client.0.smithi022.stdout:2/604: creat d0/d2/d18/d1f/d6c/d99/fc6 x:0 0 0 2024-02-16T19:29:12.684 INFO:tasks.workunit.client.1.smithi117.stdout:0/675: mkdir d0/d9/d14/d1d/d60/dc3/de9 0 2024-02-16T19:29:12.691 INFO:tasks.workunit.client.0.smithi022.stdout:6/642: chown d1/d8/da/d7b/dae/dbb/ce1 357 1 2024-02-16T19:29:12.692 INFO:tasks.workunit.client.1.smithi117.stdout:3/735: creat d1/dd/d3e/d7d/ffd x:0 0 0 2024-02-16T19:29:12.693 INFO:tasks.workunit.client.0.smithi022.stdout:6/643: write d1/d8/da/d7b/dae/dbb/fbd [685749,117418] 0 2024-02-16T19:29:12.694 INFO:tasks.workunit.client.0.smithi022.stdout:6/644: chown d1/d8/d23/f30 2549 1 2024-02-16T19:29:12.695 INFO:tasks.workunit.client.0.smithi022.stdout:7/598: fsync d9/da/d1f/d44/f49 0 2024-02-16T19:29:12.695 INFO:tasks.workunit.client.1.smithi117.stdout:1/560: dwrite d2/f57 [0,4194304] 0 2024-02-16T19:29:12.696 INFO:tasks.workunit.client.1.smithi117.stdout:1/561: readlink d2/d1d/d60/d68/lbd 0 2024-02-16T19:29:12.697 INFO:tasks.workunit.client.0.smithi022.stdout:3/539: getdents d7/d12/d59 0 2024-02-16T19:29:12.697 INFO:tasks.workunit.client.1.smithi117.stdout:5/683: write d2/d4c/f71 [565364,23498] 0 2024-02-16T19:29:12.699 INFO:tasks.workunit.client.1.smithi117.stdout:9/551: rename dd/d47/f7e to dd/d2d/fbf 0 2024-02-16T19:29:12.700 INFO:tasks.workunit.client.0.smithi022.stdout:5/655: link dd/d1f/d48/d21/d2f/d52/d5c/d61/cd9 dd/d1f/d48/d21/d2f/d40/d4e/d56/d9d/ce1 0 2024-02-16T19:29:12.701 INFO:tasks.workunit.client.1.smithi117.stdout:4/616: link d0/da/d21/d3d/f29 d0/fc5 0 2024-02-16T19:29:12.702 INFO:tasks.workunit.client.0.smithi022.stdout:4/567: dwrite d4/d6/d14/d64/f34 [0,4194304] 0 2024-02-16T19:29:12.703 INFO:tasks.workunit.client.0.smithi022.stdout:2/605: creat d0/d2/d18/d1f/d6c/d99/fc7 x:0 0 0 2024-02-16T19:29:12.703 INFO:tasks.workunit.client.0.smithi022.stdout:5/656: fdatasync dd/d1f/d48/d21/d2f/d46/d5e/d84/fd1 0 2024-02-16T19:29:12.704 INFO:tasks.workunit.client.0.smithi022.stdout:5/657: chown dd/l28 13176453 1 2024-02-16T19:29:12.704 INFO:tasks.workunit.client.1.smithi117.stdout:7/629: creat d4/d1c/d39/d46/da4/fe8 x:0 0 0 2024-02-16T19:29:12.706 INFO:tasks.workunit.client.0.smithi022.stdout:6/645: creat d1/d8/d3c/d77/fe5 x:0 0 0 2024-02-16T19:29:12.711 INFO:tasks.workunit.client.1.smithi117.stdout:3/736: creat d1/dd/d8e/ffe x:0 0 0 2024-02-16T19:29:12.712 INFO:tasks.workunit.client.1.smithi117.stdout:3/737: chown d1/d31/f7e 13 1 2024-02-16T19:29:12.714 INFO:tasks.workunit.client.0.smithi022.stdout:5/658: creat dd/d1f/d48/d21/d2f/fe2 x:0 0 0 2024-02-16T19:29:12.714 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:12 smithi022 ceph-mon[29221]: pgmap v9: 65 pgs: 65 active+clean; 2.4 GiB data, 7.0 GiB used, 529 GiB / 536 GiB avail; 83 MiB/s rd, 196 MiB/s wr, 424 op/s 2024-02-16T19:29:12.715 INFO:tasks.workunit.client.1.smithi117.stdout:5/684: chown d2/d9/d3c/d24/d50/d96/dbc/d30/lb4 62892 1 2024-02-16T19:29:12.719 INFO:tasks.workunit.client.1.smithi117.stdout:6/610: rename d5/db/l3e to d5/d4b/d44/d93/lc9 0 2024-02-16T19:29:12.720 INFO:tasks.workunit.client.1.smithi117.stdout:0/676: write d0/d9/d14/d1d/f88 [544455,108220] 0 2024-02-16T19:29:12.721 INFO:tasks.workunit.client.0.smithi022.stdout:5/659: dwrite dd/d1f/d2e/fd2 [0,4194304] 0 2024-02-16T19:29:12.722 INFO:tasks.workunit.client.1.smithi117.stdout:0/677: chown d0/d78/c98 0 1 2024-02-16T19:29:12.723 INFO:tasks.workunit.client.1.smithi117.stdout:5/685: dwrite d2/d9/d3c/d24/d50/d52/d74/d8c/f4b [0,4194304] 0 2024-02-16T19:29:12.724 INFO:tasks.workunit.client.1.smithi117.stdout:6/611: read d5/d4b/d71/dc7/d72/fa2 [3136267,11011] 0 2024-02-16T19:29:12.726 INFO:tasks.workunit.client.0.smithi022.stdout:3/540: read d7/d12/d18/dd/d3c/f49 [95486,77042] 0 2024-02-16T19:29:12.728 INFO:tasks.workunit.client.1.smithi117.stdout:7/630: mknod d4/d1c/d39/d46/d47/ce9 0 2024-02-16T19:29:12.730 INFO:tasks.workunit.client.1.smithi117.stdout:7/631: write d4/d13/d63/fce [2939240,42490] 0 2024-02-16T19:29:12.731 INFO:tasks.workunit.client.0.smithi022.stdout:2/606: mknod d0/d2/d18/d38/cc8 0 2024-02-16T19:29:12.735 INFO:tasks.workunit.client.1.smithi117.stdout:8/636: getdents d4/d1c 0 2024-02-16T19:29:12.736 INFO:tasks.workunit.client.0.smithi022.stdout:3/541: dread d7/d12/d59/f85 [0,4194304] 0 2024-02-16T19:29:12.737 INFO:tasks.workunit.client.1.smithi117.stdout:2/631: sync 2024-02-16T19:29:12.737 INFO:tasks.workunit.client.1.smithi117.stdout:8/637: chown d4/d12/c2f 203 1 2024-02-16T19:29:12.739 INFO:tasks.workunit.client.1.smithi117.stdout:4/617: dwrite d0/da/d21/f65 [4194304,4194304] 0 2024-02-16T19:29:12.739 INFO:tasks.workunit.client.1.smithi117.stdout:1/562: rename d2/d4/lb9 to d2/d19/d59/d76/d8c/lbe 0 2024-02-16T19:29:12.743 INFO:tasks.workunit.client.0.smithi022.stdout:5/660: mknod dd/d1f/d48/ce3 0 2024-02-16T19:29:12.746 INFO:tasks.workunit.client.0.smithi022.stdout:6/646: truncate d1/d8/f3b 2440987 0 2024-02-16T19:29:12.746 INFO:tasks.workunit.client.0.smithi022.stdout:6/647: read - d1/d8/f6f zero size 2024-02-16T19:29:12.747 INFO:tasks.workunit.client.0.smithi022.stdout:6/648: readlink d1/l5 0 2024-02-16T19:29:12.747 INFO:tasks.workunit.client.1.smithi117.stdout:5/686: dread d2/d9/d3c/d24/f53 [0,4194304] 0 2024-02-16T19:29:12.747 INFO:tasks.workunit.client.1.smithi117.stdout:6/612: mknod d5/d4b/d20/da4/cca 0 2024-02-16T19:29:12.749 INFO:tasks.workunit.client.1.smithi117.stdout:4/618: dwrite d0/da/d4a/fa8 [0,4194304] 0 2024-02-16T19:29:12.750 INFO:tasks.workunit.client.1.smithi117.stdout:6/613: write d5/d4b/d20/d69/fc2 [321973,129115] 0 2024-02-16T19:29:12.754 INFO:tasks.workunit.client.0.smithi022.stdout:5/661: read dd/d1f/d4a/f83 [82628,81509] 0 2024-02-16T19:29:12.755 INFO:tasks.workunit.client.0.smithi022.stdout:2/607: mkdir d0/d2/d18/d19/d40/dc9 0 2024-02-16T19:29:12.757 INFO:tasks.workunit.client.0.smithi022.stdout:5/662: write dd/d1f/d76/fab [416216,75731] 0 2024-02-16T19:29:12.758 INFO:tasks.workunit.client.0.smithi022.stdout:3/542: creat d7/d12/d18/dd/d3c/d5a/fc7 x:0 0 0 2024-02-16T19:29:12.761 INFO:tasks.workunit.client.0.smithi022.stdout:5/663: read dd/d1f/d48/d21/d2f/d46/d5e/f7b [10343,110400] 0 2024-02-16T19:29:12.763 INFO:tasks.workunit.client.0.smithi022.stdout:5/664: readlink dd/d1f/d48/d21/d2f/d40/d71/lc4 0 2024-02-16T19:29:12.763 INFO:tasks.workunit.client.0.smithi022.stdout:6/649: dread d1/d8/da/d17/f99 [0,4194304] 0 2024-02-16T19:29:12.765 INFO:tasks.workunit.client.1.smithi117.stdout:8/638: mknod d4/d12/d2a/d64/dc6/cc8 0 2024-02-16T19:29:12.765 INFO:tasks.workunit.client.1.smithi117.stdout:9/552: rename dd/d2d to dd/d47/d48/dc0 0 2024-02-16T19:29:12.767 INFO:tasks.workunit.client.1.smithi117.stdout:9/553: dread - dd/d47/d48/dc0/fa6 zero size 2024-02-16T19:29:12.768 INFO:tasks.workunit.client.0.smithi022.stdout:3/543: symlink d7/d12/d18/dac/lc8 0 2024-02-16T19:29:12.770 INFO:tasks.workunit.client.0.smithi022.stdout:3/544: dread d7/d12/d18/dd/d31/d7f/fa5 [0,4194304] 0 2024-02-16T19:29:12.771 INFO:tasks.workunit.client.1.smithi117.stdout:1/563: mknod d2/d19/d59/d89/d69/d80/cbf 0 2024-02-16T19:29:12.771 INFO:tasks.workunit.client.1.smithi117.stdout:3/738: truncate d1/d1b/d1e/d29/f33 134998 0 2024-02-16T19:29:12.773 INFO:tasks.workunit.client.0.smithi022.stdout:5/665: dread dd/d1f/d4a/f3f [0,4194304] 0 2024-02-16T19:29:12.774 INFO:tasks.workunit.client.0.smithi022.stdout:2/608: write d0/d2/f14 [757331,90894] 0 2024-02-16T19:29:12.775 INFO:tasks.workunit.client.1.smithi117.stdout:0/678: dread d0/d9/d22/d27/d31/f38 [0,4194304] 0 2024-02-16T19:29:12.776 INFO:tasks.workunit.client.0.smithi022.stdout:2/609: write d0/d2/d18/d19/d40/fa5 [668766,112929] 0 2024-02-16T19:29:12.777 INFO:tasks.workunit.client.1.smithi117.stdout:5/687: symlink d2/db3/db0/d9b/le5 0 2024-02-16T19:29:12.777 INFO:tasks.workunit.client.1.smithi117.stdout:0/679: readlink d0/d9/d14/d1d/d60/d76/lc5 0 2024-02-16T19:29:12.781 INFO:tasks.workunit.client.1.smithi117.stdout:7/632: mknod d4/d1c/d87/d89/cea 0 2024-02-16T19:29:12.783 INFO:tasks.workunit.client.1.smithi117.stdout:2/632: symlink d0/d3/d34/d94/d82/lbb 0 2024-02-16T19:29:12.784 INFO:tasks.workunit.client.1.smithi117.stdout:3/739: dread d1/f8 [0,4194304] 0 2024-02-16T19:29:12.796 INFO:tasks.workunit.client.0.smithi022.stdout:3/545: dwrite d7/d12/d18/dd/d24/d58/f8b [0,4194304] 0 2024-02-16T19:29:12.801 INFO:tasks.workunit.client.0.smithi022.stdout:5/666: dwrite dd/d1f/d48/d21/d2f/d40/d4e/d56/f59 [0,4194304] 0 2024-02-16T19:29:12.804 INFO:tasks.workunit.client.0.smithi022.stdout:2/610: dwrite d0/d2/d18/d19/d90/fa3 [0,4194304] 0 2024-02-16T19:29:12.808 INFO:tasks.workunit.client.0.smithi022.stdout:5/667: unlink dd/d1f/d4a/d7a/l93 0 2024-02-16T19:29:12.812 INFO:tasks.workunit.client.0.smithi022.stdout:2/611: unlink d0/d2/d18/d19/f56 0 2024-02-16T19:29:12.814 INFO:tasks.workunit.client.0.smithi022.stdout:2/612: read d0/d2/d18/d19/d90/fbd [2587606,916] 0 2024-02-16T19:29:12.816 INFO:tasks.workunit.client.0.smithi022.stdout:5/668: dwrite dd/d1f/d48/d21/f25 [4194304,4194304] 0 2024-02-16T19:29:12.819 INFO:tasks.workunit.client.1.smithi117.stdout:5/688: mknod d2/db3/ce6 0 2024-02-16T19:29:12.819 INFO:tasks.workunit.client.1.smithi117.stdout:5/689: chown d2/d9/c7b 8627 1 2024-02-16T19:29:12.823 INFO:tasks.workunit.client.1.smithi117.stdout:6/614: creat d5/d9c/daa/d85/db0/fcb x:0 0 0 2024-02-16T19:29:12.824 INFO:tasks.workunit.client.0.smithi022.stdout:5/669: creat dd/d1f/d48/d21/d2f/d40/d71/fe4 x:0 0 0 2024-02-16T19:29:12.826 INFO:tasks.workunit.client.0.smithi022.stdout:2/613: dwrite d0/d2/d18/d19/d40/d55/f7c [0,4194304] 0 2024-02-16T19:29:12.831 INFO:tasks.workunit.client.1.smithi117.stdout:2/633: mkdir d0/d13/d4d/d53/d58/dbc 0 2024-02-16T19:29:12.831 INFO:tasks.workunit.client.1.smithi117.stdout:6/615: dwrite d5/d9c/daa/fb2 [8388608,4194304] 0 2024-02-16T19:29:12.831 INFO:tasks.workunit.client.1.smithi117.stdout:7/633: read d4/d1c/d87/d89/d60/fab [56447,87658] 0 2024-02-16T19:29:12.833 INFO:tasks.workunit.client.1.smithi117.stdout:7/634: write d4/d1c/d87/d89/d60/d79/fc1 [2849447,54536] 0 2024-02-16T19:29:12.835 INFO:tasks.workunit.client.1.smithi117.stdout:7/635: write d4/dd3/fb9 [687855,104375] 0 2024-02-16T19:29:12.836 INFO:tasks.workunit.client.1.smithi117.stdout:7/636: stat d4/d1c/d87/d89/d60/fba 0 2024-02-16T19:29:12.837 INFO:tasks.workunit.client.1.smithi117.stdout:7/637: chown d4/d1c/d39/d9f/fca 81 1 2024-02-16T19:29:12.839 INFO:tasks.workunit.client.0.smithi022.stdout:3/546: dread d7/d12/d18/dd/d24/d53/f5e [4194304,4194304] 0 2024-02-16T19:29:12.844 INFO:tasks.workunit.client.1.smithi117.stdout:6/616: dread d5/f9 [0,4194304] 0 2024-02-16T19:29:12.863 INFO:tasks.workunit.client.1.smithi117.stdout:4/619: sync 2024-02-16T19:29:12.867 INFO:tasks.workunit.client.1.smithi117.stdout:3/740: symlink d1/d31/d89/dbd/lff 0 2024-02-16T19:29:12.868 INFO:tasks.workunit.client.0.smithi022.stdout:2/614: fdatasync d0/f78 0 2024-02-16T19:29:12.869 INFO:tasks.workunit.client.0.smithi022.stdout:2/615: readlink d0/d2/d18/l42 0 2024-02-16T19:29:12.873 INFO:tasks.workunit.client.0.smithi022.stdout:3/547: mknod d7/d12/d18/dd/d31/d7f/cc9 0 2024-02-16T19:29:12.874 INFO:tasks.workunit.client.1.smithi117.stdout:3/741: dwrite d1/d1b/fe2 [0,4194304] 0 2024-02-16T19:29:12.875 INFO:tasks.workunit.client.1.smithi117.stdout:8/639: write d4/d7/da6/d3a/d43/f55 [2926701,87117] 0 2024-02-16T19:29:12.877 INFO:tasks.workunit.client.1.smithi117.stdout:1/564: getdents d2/d19/d59/d89/d69/dad 0 2024-02-16T19:29:12.884 INFO:tasks.workunit.client.0.smithi022.stdout:2/616: dread d0/d2/d2e/f72 [0,4194304] 0 2024-02-16T19:29:12.886 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:12 smithi117 ceph-mon[35309]: pgmap v9: 65 pgs: 65 active+clean; 2.4 GiB data, 7.0 GiB used, 529 GiB / 536 GiB avail; 83 MiB/s rd, 196 MiB/s wr, 424 op/s 2024-02-16T19:29:12.890 INFO:tasks.workunit.client.0.smithi022.stdout:3/548: dread d7/d12/d18/dd/d31/d7f/faf [0,4194304] 0 2024-02-16T19:29:12.890 INFO:tasks.workunit.client.0.smithi022.stdout:5/670: write dd/d1f/d48/d21/d2f/d52/d5c/fce [585512,39247] 0 2024-02-16T19:29:12.891 INFO:tasks.workunit.client.0.smithi022.stdout:2/617: dwrite d0/f17 [0,4194304] 0 2024-02-16T19:29:12.892 INFO:tasks.workunit.client.1.smithi117.stdout:7/638: mknod d4/d1c/d87/d89/d60/d9b/ceb 0 2024-02-16T19:29:12.894 INFO:tasks.workunit.client.1.smithi117.stdout:5/690: sync 2024-02-16T19:29:12.894 INFO:tasks.workunit.client.1.smithi117.stdout:9/554: sync 2024-02-16T19:29:12.895 INFO:tasks.workunit.client.0.smithi022.stdout:3/549: readlink d7/l11 0 2024-02-16T19:29:12.900 INFO:tasks.workunit.client.0.smithi022.stdout:5/671: dwrite f6 [4194304,4194304] 0 2024-02-16T19:29:12.903 INFO:tasks.workunit.client.0.smithi022.stdout:3/550: rmdir d7/d12/d59 39 2024-02-16T19:29:12.905 INFO:tasks.workunit.client.1.smithi117.stdout:3/742: write d1/d31/f7e [4160561,1980] 0 2024-02-16T19:29:12.905 INFO:tasks.workunit.client.1.smithi117.stdout:0/680: dwrite d0/d9/d14/d1d/d60/d65/fe3 [0,4194304] 0 2024-02-16T19:29:12.905 INFO:tasks.workunit.client.0.smithi022.stdout:5/672: unlink dd/d1f/d48/d21/d2f/fd0 0 2024-02-16T19:29:12.907 INFO:tasks.workunit.client.0.smithi022.stdout:3/551: mknod d7/d12/d18/dd/d31/cca 0 2024-02-16T19:29:12.908 INFO:tasks.workunit.client.1.smithi117.stdout:0/681: readlink d0/d9/d22/d27/d31/d6d/l9a 0 2024-02-16T19:29:12.910 INFO:tasks.workunit.client.1.smithi117.stdout:1/565: readlink d2/d19/d59/d76/d8c/lbe 0 2024-02-16T19:29:12.910 INFO:tasks.workunit.client.1.smithi117.stdout:6/617: write d5/d4b/d71/dc7/f87 [4089904,78504] 0 2024-02-16T19:29:12.913 INFO:tasks.workunit.client.1.smithi117.stdout:2/634: symlink d0/d3/d10/lbd 0 2024-02-16T19:29:12.915 INFO:tasks.workunit.client.0.smithi022.stdout:3/552: dwrite d7/d12/d18/dd/d3c/f65 [0,4194304] 0 2024-02-16T19:29:12.917 INFO:tasks.workunit.client.1.smithi117.stdout:5/691: mkdir d2/d4c/d4f/de7 0 2024-02-16T19:29:12.921 INFO:tasks.workunit.client.0.smithi022.stdout:3/553: read d7/d12/d18/dac/f4c [59894,29544] 0 2024-02-16T19:29:12.924 INFO:tasks.workunit.client.0.smithi022.stdout:3/554: dread - d7/d21/fbd zero size 2024-02-16T19:29:12.927 INFO:tasks.workunit.client.1.smithi117.stdout:3/743: mknod d1/dd/d5c/c100 0 2024-02-16T19:29:12.927 INFO:tasks.workunit.client.0.smithi022.stdout:5/673: dread dd/d1f/d48/f1c [0,4194304] 0 2024-02-16T19:29:12.928 INFO:tasks.workunit.client.1.smithi117.stdout:3/744: chown d1/d1b/c6f 13 1 2024-02-16T19:29:12.933 INFO:tasks.workunit.client.1.smithi117.stdout:4/620: write d0/da/d21/d3d/d28/f63 [2609731,29291] 0 2024-02-16T19:29:12.934 INFO:tasks.workunit.client.1.smithi117.stdout:4/621: dread - d0/da/d21/d3d/d28/d64/d73/d91/fbe zero size 2024-02-16T19:29:12.936 INFO:tasks.workunit.client.1.smithi117.stdout:7/639: creat d4/d1c/d39/fec x:0 0 0 2024-02-16T19:29:12.937 INFO:tasks.workunit.client.1.smithi117.stdout:6/618: write d5/d4b/d71/dc7/d49/d59/f5e [603585,125818] 0 2024-02-16T19:29:12.939 INFO:tasks.workunit.client.1.smithi117.stdout:7/640: truncate d4/f4d 2282497 0 2024-02-16T19:29:12.939 INFO:tasks.workunit.client.1.smithi117.stdout:8/640: dwrite d4/d7/da6/d3a/d65/d96/fc7 [0,4194304] 0 2024-02-16T19:29:12.941 INFO:tasks.workunit.client.1.smithi117.stdout:0/682: rename d0/d9/d14/d1d/d1e/d54/fb4 to d0/d9/d14/d23/d33/fea 0 2024-02-16T19:29:12.942 INFO:tasks.workunit.client.1.smithi117.stdout:1/566: truncate d2/d1d/f29 580083 0 2024-02-16T19:29:12.942 INFO:tasks.workunit.client.1.smithi117.stdout:9/555: getdents dd/d45/d31/d63 0 2024-02-16T19:29:12.943 INFO:tasks.workunit.client.0.smithi022.stdout:3/555: dwrite d7/d12/d18/dd/d24/f32 [0,4194304] 0 2024-02-16T19:29:12.945 INFO:tasks.workunit.client.0.smithi022.stdout:3/556: chown d7/d12/d18/dd/d31/d7f/l99 223 1 2024-02-16T19:29:12.945 INFO:tasks.workunit.client.1.smithi117.stdout:2/635: truncate d0/f6c 1587478 0 2024-02-16T19:29:12.945 INFO:tasks.workunit.client.1.smithi117.stdout:9/556: truncate dd/d45/d73/f82 229632 0 2024-02-16T19:29:12.946 INFO:tasks.workunit.client.1.smithi117.stdout:9/557: read dd/d23/f25 [2085034,18825] 0 2024-02-16T19:29:12.946 INFO:tasks.workunit.client.1.smithi117.stdout:4/622: mkdir d0/da/d21/db4/dc6 0 2024-02-16T19:29:12.948 INFO:tasks.workunit.client.1.smithi117.stdout:5/692: write d2/db3/f3f [2361533,44012] 0 2024-02-16T19:29:12.951 INFO:tasks.workunit.client.1.smithi117.stdout:7/641: creat d4/d1c/d39/d46/da4/fed x:0 0 0 2024-02-16T19:29:12.953 INFO:tasks.workunit.client.0.smithi022.stdout:3/557: rename f5 to d7/d12/d18/dd/fcb 0 2024-02-16T19:29:12.953 INFO:tasks.workunit.client.1.smithi117.stdout:7/642: readlink d4/d1c/d39/d46/d55/l6d 0 2024-02-16T19:29:12.955 INFO:tasks.workunit.client.1.smithi117.stdout:1/567: mknod d2/d19/d59/d89/d7b/cc0 0 2024-02-16T19:29:12.955 INFO:tasks.workunit.client.0.smithi022.stdout:5/674: dread dd/d1f/f31 [0,4194304] 0 2024-02-16T19:29:12.956 INFO:tasks.workunit.client.1.smithi117.stdout:1/568: chown d2/d4/c3f 184305 1 2024-02-16T19:29:12.956 INFO:tasks.workunit.client.1.smithi117.stdout:2/636: mknod d0/d3/d34/d94/d46/d68/d76/da1/d2f/cbe 0 2024-02-16T19:29:12.957 INFO:tasks.workunit.client.0.smithi022.stdout:5/675: dread - dd/d1f/d48/d21/d2f/d52/d5c/d61/fda zero size 2024-02-16T19:29:12.958 INFO:tasks.workunit.client.1.smithi117.stdout:0/683: dread d0/d9/d22/d27/f5f [0,4194304] 0 2024-02-16T19:29:12.960 INFO:tasks.workunit.client.0.smithi022.stdout:5/676: creat dd/d1f/d48/d21/d2f/d52/d77/da9/fe5 x:0 0 0 2024-02-16T19:29:12.962 INFO:tasks.workunit.client.0.smithi022.stdout:5/677: dread - dd/d1f/d48/d21/d2f/d40/fbf zero size 2024-02-16T19:29:12.965 INFO:tasks.workunit.client.1.smithi117.stdout:8/641: dread d4/d7/da6/f63 [0,4194304] 0 2024-02-16T19:29:12.965 INFO:tasks.workunit.client.0.smithi022.stdout:5/678: truncate dd/d1f/d48/f60 2182692 0 2024-02-16T19:29:12.966 INFO:tasks.workunit.client.0.smithi022.stdout:5/679: write dd/d1f/d48/d21/d2f/fcc [896183,46003] 0 2024-02-16T19:29:12.968 INFO:tasks.workunit.client.1.smithi117.stdout:4/623: creat d0/da/d21/d3d/d28/d66/da2/dac/fc7 x:0 0 0 2024-02-16T19:29:12.971 INFO:tasks.workunit.client.1.smithi117.stdout:9/558: dread dd/d23/d64/f70 [0,4194304] 0 2024-02-16T19:29:12.973 INFO:tasks.workunit.client.1.smithi117.stdout:5/693: mknod d2/d9/d3c/d24/d50/d96/dbc/d30/d4e/ce8 0 2024-02-16T19:29:12.973 INFO:tasks.workunit.client.1.smithi117.stdout:5/694: chown d2/d9/l89 10425 1 2024-02-16T19:29:12.976 INFO:tasks.workunit.client.1.smithi117.stdout:2/637: mknod d0/d13/d4d/cbf 0 2024-02-16T19:29:12.979 INFO:tasks.workunit.client.1.smithi117.stdout:0/684: symlink d0/d9/leb 0 2024-02-16T19:29:12.980 INFO:tasks.workunit.client.1.smithi117.stdout:7/643: write d4/d1c/d39/d46/f62 [1160447,19773] 0 2024-02-16T19:29:12.982 INFO:tasks.workunit.client.1.smithi117.stdout:3/745: link d1/d30/d52/ccf d1/d30/c101 0 2024-02-16T19:29:12.982 INFO:tasks.workunit.client.1.smithi117.stdout:1/569: write d2/f4d [259660,121836] 0 2024-02-16T19:29:12.983 INFO:tasks.workunit.client.1.smithi117.stdout:3/746: chown d1/d83/d96/cad 11 1 2024-02-16T19:29:12.984 INFO:tasks.workunit.client.1.smithi117.stdout:9/559: symlink dd/d47/d5e/d9a/lc1 0 2024-02-16T19:29:12.986 INFO:tasks.workunit.client.1.smithi117.stdout:8/642: mkdir d4/d7/da6/d3a/d65/d7c/dc9 0 2024-02-16T19:29:12.987 INFO:tasks.workunit.client.1.smithi117.stdout:2/638: symlink d0/d3/d34/d94/d46/d68/d76/da1/d2f/lc0 0 2024-02-16T19:29:12.989 INFO:tasks.workunit.client.0.smithi022.stdout:5/680: write dd/d1f/d48/f1c [1666944,79067] 0 2024-02-16T19:29:12.989 INFO:tasks.workunit.client.0.smithi022.stdout:5/681: readlink dd/l28 0 2024-02-16T19:29:12.991 INFO:tasks.workunit.client.1.smithi117.stdout:1/570: fsync d2/d1d/d1f/f5e 0 2024-02-16T19:29:12.992 INFO:tasks.workunit.client.1.smithi117.stdout:3/747: mkdir d1/dd/df8/d102 0 2024-02-16T19:29:12.994 INFO:tasks.workunit.client.0.smithi022.stdout:5/682: rmdir dd/d1f/d76 39 2024-02-16T19:29:13.000 INFO:tasks.workunit.client.1.smithi117.stdout:2/639: rename d0/f25 to d0/d3/d40/fc1 0 2024-02-16T19:29:13.000 INFO:tasks.workunit.client.1.smithi117.stdout:2/640: chown d0/d3/f31 15 1 2024-02-16T19:29:13.002 INFO:tasks.workunit.client.1.smithi117.stdout:3/748: mkdir d1/d30/d9f/dcb/d103 0 2024-02-16T19:29:13.004 INFO:tasks.workunit.client.1.smithi117.stdout:7/644: write d4/f10 [126987,58921] 0 2024-02-16T19:29:13.005 INFO:tasks.workunit.client.1.smithi117.stdout:5/695: dwrite d2/d9/d3c/d24/d50/d52/f1e [0,4194304] 0 2024-02-16T19:29:13.007 INFO:tasks.workunit.client.1.smithi117.stdout:1/571: rename d2/d19/fac to d2/d19/d59/d89/d69/d80/d8e/fc1 0 2024-02-16T19:29:13.008 INFO:tasks.workunit.client.1.smithi117.stdout:9/560: creat dd/d47/fc2 x:0 0 0 2024-02-16T19:29:13.009 INFO:tasks.workunit.client.1.smithi117.stdout:2/641: truncate d0/d3/d34/d94/d46/d68/d76/da1/f49 1358392 0 2024-02-16T19:29:13.013 INFO:tasks.workunit.client.1.smithi117.stdout:3/749: rename d1/d1b/d1e/l1a to d1/dd/d3b/da3/l104 0 2024-02-16T19:29:13.015 INFO:tasks.workunit.client.1.smithi117.stdout:4/624: dwrite d0/da/d21/d3d/d28/d66/d97/f41 [0,4194304] 0 2024-02-16T19:29:13.017 INFO:tasks.workunit.client.1.smithi117.stdout:6/619: sync 2024-02-16T19:29:13.018 INFO:tasks.workunit.client.1.smithi117.stdout:2/642: rename d0/d3b/l8b to d0/d3b/lc2 0 2024-02-16T19:29:13.018 INFO:tasks.workunit.client.1.smithi117.stdout:7/645: truncate d4/f1e 1956776 0 2024-02-16T19:29:13.020 INFO:tasks.workunit.client.1.smithi117.stdout:1/572: dwrite d2/f57 [0,4194304] 0 2024-02-16T19:29:13.024 INFO:tasks.workunit.client.1.smithi117.stdout:3/750: symlink d1/d1b/d1e/d29/l105 0 2024-02-16T19:29:13.027 INFO:tasks.workunit.client.1.smithi117.stdout:8/643: dwrite d4/d12/d2a/f2b [4194304,4194304] 0 2024-02-16T19:29:13.027 INFO:tasks.workunit.client.1.smithi117.stdout:7/646: chown d4/d1c/d87/d89/d60/d79/dc3 1599039 1 2024-02-16T19:29:13.027 INFO:tasks.workunit.client.1.smithi117.stdout:2/643: write d0/d3/d10/f1d [1673494,74634] 0 2024-02-16T19:29:13.030 INFO:tasks.workunit.client.1.smithi117.stdout:3/751: mknod d1/d1b/d1e/d37/c106 0 2024-02-16T19:29:13.031 INFO:tasks.workunit.client.1.smithi117.stdout:8/644: write d4/d7/da6/d3a/d65/d96/fc7 [1460891,100655] 0 2024-02-16T19:29:13.032 INFO:tasks.workunit.client.1.smithi117.stdout:9/561: getdents dd/d45/d31/d63/d8e 0 2024-02-16T19:29:13.033 INFO:tasks.workunit.client.1.smithi117.stdout:9/562: stat dd/d23/f1f 0 2024-02-16T19:29:13.033 INFO:tasks.workunit.client.1.smithi117.stdout:7/647: creat d4/d1c/d87/fee x:0 0 0 2024-02-16T19:29:13.039 INFO:tasks.workunit.client.1.smithi117.stdout:2/644: rename d0/d3/d34/d94/d46/d68/d76/da1/d2f/cbe to d0/d3/d22/cc3 0 2024-02-16T19:29:13.040 INFO:tasks.workunit.client.1.smithi117.stdout:8/645: creat d4/d7/fca x:0 0 0 2024-02-16T19:29:13.040 INFO:tasks.workunit.client.1.smithi117.stdout:7/648: mkdir d4/d1c/d98/def 0 2024-02-16T19:29:13.042 INFO:tasks.workunit.client.1.smithi117.stdout:9/563: symlink dd/d23/daf/lc3 0 2024-02-16T19:29:13.043 INFO:tasks.workunit.client.1.smithi117.stdout:2/645: mkdir d0/d13/dc4 0 2024-02-16T19:29:13.044 INFO:tasks.workunit.client.1.smithi117.stdout:2/646: rmdir d0/d13 39 2024-02-16T19:29:13.046 INFO:tasks.workunit.client.1.smithi117.stdout:9/564: fdatasync dd/d45/f58 0 2024-02-16T19:29:13.051 INFO:tasks.workunit.client.1.smithi117.stdout:4/625: dread d0/da/f33 [0,4194304] 0 2024-02-16T19:29:13.052 INFO:tasks.workunit.client.1.smithi117.stdout:6/620: dread d5/fa [0,4194304] 0 2024-02-16T19:29:13.053 INFO:tasks.workunit.client.1.smithi117.stdout:6/621: chown d5/d4b/d71/dc7/d49 30325302 1 2024-02-16T19:29:13.053 INFO:tasks.workunit.client.1.smithi117.stdout:3/752: rmdir d1/d1b/d1e/d37 39 2024-02-16T19:29:13.054 INFO:tasks.workunit.client.1.smithi117.stdout:4/626: creat d0/da/d21/fc8 x:0 0 0 2024-02-16T19:29:13.057 INFO:tasks.workunit.client.1.smithi117.stdout:3/753: write d1/dd/d5c/d85/fd9 [1114878,21169] 0 2024-02-16T19:29:13.057 INFO:tasks.workunit.client.1.smithi117.stdout:4/627: dread - d0/da/d21/d3d/d28/d66/da2/dac/fc7 zero size 2024-02-16T19:29:13.058 INFO:tasks.workunit.client.0.smithi022.stdout:5/683: unlink dd/d1f/d48/cb2 0 2024-02-16T19:29:13.060 INFO:tasks.workunit.client.1.smithi117.stdout:3/754: write d1/d1b/fe2 [1636505,21541] 0 2024-02-16T19:29:13.061 INFO:tasks.workunit.client.1.smithi117.stdout:0/685: sync 2024-02-16T19:29:13.063 INFO:tasks.workunit.client.0.smithi022.stdout:5/684: creat dd/d1f/d48/d21/d2f/d52/fe6 x:0 0 0 2024-02-16T19:29:13.064 INFO:tasks.workunit.client.1.smithi117.stdout:8/646: truncate d4/d73/fa0 3911270 0 2024-02-16T19:29:13.064 INFO:tasks.workunit.client.1.smithi117.stdout:1/573: dread d2/ff [4194304,4194304] 0 2024-02-16T19:29:13.066 INFO:tasks.workunit.client.1.smithi117.stdout:7/649: write d4/d1c/d39/d46/d55/db8/fc2 [970391,84536] 0 2024-02-16T19:29:13.069 INFO:tasks.workunit.client.0.smithi022.stdout:5/685: rename dd/d1f/d4a to dd/d1f/d48/d21/d5d/de7 0 2024-02-16T19:29:13.069 INFO:tasks.workunit.client.1.smithi117.stdout:6/622: link d5/d4b/d20/d69/l6e d5/d4b/d44/d93/lcc 0 2024-02-16T19:29:13.070 INFO:tasks.workunit.client.1.smithi117.stdout:9/565: truncate dd/d23/f3b 3347554 0 2024-02-16T19:29:13.072 INFO:tasks.workunit.client.1.smithi117.stdout:0/686: mkdir d0/d9/d14/dec 0 2024-02-16T19:29:13.074 INFO:tasks.workunit.client.1.smithi117.stdout:4/628: dread d0/da/f5b [0,4194304] 0 2024-02-16T19:29:13.077 INFO:tasks.workunit.client.1.smithi117.stdout:2/647: link d0/d3/d22/cc3 d0/d3/d34/d94/d46/d68/d76/da1/cc5 0 2024-02-16T19:29:13.080 INFO:tasks.workunit.client.1.smithi117.stdout:1/574: unlink d2/d19/d59/d89/d69/d80/d8e/fc1 0 2024-02-16T19:29:13.082 INFO:tasks.workunit.client.1.smithi117.stdout:7/650: mkdir d4/d1c/d87/df0 0 2024-02-16T19:29:13.083 INFO:tasks.workunit.client.1.smithi117.stdout:7/651: write d4/d13/d63/d71/fbd [269687,46457] 0 2024-02-16T19:29:13.083 INFO:tasks.workunit.client.1.smithi117.stdout:6/623: mknod d5/d4b/d20/d69/ccd 0 2024-02-16T19:29:13.085 INFO:tasks.workunit.client.1.smithi117.stdout:3/755: creat d1/d1b/d1e/d37/ddc/f107 x:0 0 0 2024-02-16T19:29:13.086 INFO:tasks.workunit.client.1.smithi117.stdout:6/624: chown d5/d4b/d20/db3/cb4 612750 1 2024-02-16T19:29:13.087 INFO:tasks.workunit.client.1.smithi117.stdout:9/566: dread dd/d47/d5b/d5f/d77/d7c/f8b [0,4194304] 0 2024-02-16T19:29:13.087 INFO:tasks.workunit.client.1.smithi117.stdout:6/625: chown d5/d4b/d44/d7c 0 1 2024-02-16T19:29:13.087 INFO:tasks.workunit.client.0.smithi022.stdout:5/686: write dd/d1f/d48/d21/d2f/d40/d4e/f6d [588093,126253] 0 2024-02-16T19:29:13.090 INFO:tasks.workunit.client.1.smithi117.stdout:2/648: creat d0/d3/d34/fc6 x:0 0 0 2024-02-16T19:29:13.093 INFO:tasks.workunit.client.1.smithi117.stdout:5/696: sync 2024-02-16T19:29:13.096 INFO:tasks.workunit.client.1.smithi117.stdout:8/647: dwrite d4/d7/da6/d3a/d54/f78 [0,4194304] 0 2024-02-16T19:29:13.097 INFO:tasks.workunit.client.1.smithi117.stdout:7/652: mknod d4/d13/da3/cf1 0 2024-02-16T19:29:13.101 INFO:tasks.workunit.client.0.smithi022.stdout:5/687: getdents dd/d1f/d48 0 2024-02-16T19:29:13.102 INFO:tasks.workunit.client.1.smithi117.stdout:7/653: write d4/d1c/d39/d46/f62 [1703450,101412] 0 2024-02-16T19:29:13.102 INFO:tasks.workunit.client.1.smithi117.stdout:5/697: dread d2/d9/d3c/d24/d50/d52/d74/d8c/f4b [0,4194304] 0 2024-02-16T19:29:13.103 INFO:tasks.workunit.client.1.smithi117.stdout:9/567: unlink dd/d45/d31/f75 0 2024-02-16T19:29:13.103 INFO:tasks.workunit.client.0.smithi022.stdout:5/688: write dd/d1f/d48/d21/d2f/d40/d4e/d56/f59 [1745537,52674] 0 2024-02-16T19:29:13.110 INFO:tasks.workunit.client.1.smithi117.stdout:7/654: truncate d4/d1c/d87/fee 429070 0 2024-02-16T19:29:13.111 INFO:tasks.workunit.client.1.smithi117.stdout:2/649: rmdir d0/d3/d34/d94/d91 39 2024-02-16T19:29:13.112 INFO:tasks.workunit.client.1.smithi117.stdout:3/756: write d1/d83/d96/fa0 [2818778,25753] 0 2024-02-16T19:29:13.112 INFO:tasks.workunit.client.1.smithi117.stdout:4/629: dwrite d0/da/d21/f2b [0,4194304] 0 2024-02-16T19:29:13.113 INFO:tasks.workunit.client.1.smithi117.stdout:7/655: dread - d4/d1c/d39/d46/fc6 zero size 2024-02-16T19:29:13.114 INFO:tasks.workunit.client.1.smithi117.stdout:8/648: creat d4/d7/da6/d3a/d65/d96/fcb x:0 0 0 2024-02-16T19:29:13.116 INFO:tasks.workunit.client.1.smithi117.stdout:1/575: dwrite d2/d1d/d60/f66 [0,4194304] 0 2024-02-16T19:29:13.119 INFO:tasks.workunit.client.1.smithi117.stdout:9/568: dwrite dd/d47/d5e/f6e [0,4194304] 0 2024-02-16T19:29:13.122 INFO:tasks.workunit.client.1.smithi117.stdout:6/626: rename d5/d4b/d44/d52/d81/d9f to d5/d4b/d71/dc7/dce 0 2024-02-16T19:29:13.124 INFO:tasks.workunit.client.1.smithi117.stdout:9/569: mkdir dd/db5/dc4 0 2024-02-16T19:29:13.125 INFO:tasks.workunit.client.0.smithi022.stdout:5/689: truncate dd/d1f/d48/d21/d2f/d52/d77/d95/fb8 2942987 0 2024-02-16T19:29:13.126 INFO:tasks.workunit.client.1.smithi117.stdout:4/630: creat d0/da/d21/d3d/d28/d64/d90/dc0/fc9 x:0 0 0 2024-02-16T19:29:13.128 INFO:tasks.workunit.client.1.smithi117.stdout:3/757: rmdir d1/d1b/d1e/d29/daf 39 2024-02-16T19:29:13.133 INFO:tasks.workunit.client.1.smithi117.stdout:2/650: mknod d0/d3/d34/d94/d82/cc7 0 2024-02-16T19:29:13.135 INFO:tasks.workunit.client.1.smithi117.stdout:6/627: creat d5/d4b/d44/fcf x:0 0 0 2024-02-16T19:29:13.136 INFO:tasks.workunit.client.1.smithi117.stdout:3/758: mknod d1/dd/df8/c108 0 2024-02-16T19:29:13.137 INFO:tasks.workunit.client.1.smithi117.stdout:5/698: rename d2/d9/d3c/d24/d50/d96/dbc/d30/d4e/f34 to d2/d9/d3c/d24/d50/d96/dbc/d30/d5a/d62/fe9 0 2024-02-16T19:29:13.139 INFO:tasks.workunit.client.1.smithi117.stdout:0/687: dread d0/d9/d14/d1d/d1e/d54/f72 [0,4194304] 0 2024-02-16T19:29:13.141 INFO:tasks.workunit.client.1.smithi117.stdout:2/651: creat d0/d13/d4d/d80/fc8 x:0 0 0 2024-02-16T19:29:13.142 INFO:tasks.workunit.client.0.smithi022.stdout:0/787: sync 2024-02-16T19:29:13.142 INFO:tasks.workunit.client.0.smithi022.stdout:1/674: sync 2024-02-16T19:29:13.142 INFO:tasks.workunit.client.1.smithi117.stdout:9/570: dwrite dd/d47/d5e/f6e [0,4194304] 0 2024-02-16T19:29:13.143 INFO:tasks.workunit.client.1.smithi117.stdout:6/628: dread - d5/d4b/d20/d69/f91 zero size 2024-02-16T19:29:13.145 INFO:tasks.workunit.client.1.smithi117.stdout:4/631: link d0/da/d21/d3d/d28/l5a d0/da/d21/d3d/d28/d64/d73/d91/d37/lca 0 2024-02-16T19:29:13.146 INFO:tasks.workunit.client.1.smithi117.stdout:8/649: rename d4/d7/d6a/f7b to d4/d7/da6/d3a/d65/d96/da3/fcc 0 2024-02-16T19:29:13.148 INFO:tasks.workunit.client.0.smithi022.stdout:0/788: symlink d10/d11/d50/d102/l105 0 2024-02-16T19:29:13.148 INFO:tasks.workunit.client.1.smithi117.stdout:7/656: dwrite d4/d1c/d39/d46/d55/db8/fc2 [0,4194304] 0 2024-02-16T19:29:13.149 INFO:tasks.workunit.client.0.smithi022.stdout:5/690: truncate dd/d1f/f58 97402 0 2024-02-16T19:29:13.150 INFO:tasks.workunit.client.1.smithi117.stdout:2/652: write d0/d3/d22/f28 [950244,38697] 0 2024-02-16T19:29:13.151 INFO:tasks.workunit.client.0.smithi022.stdout:5/691: write dd/d1f/d48/d21/d2f/d52/d5c/d61/f92 [2113417,33240] 0 2024-02-16T19:29:13.154 INFO:tasks.workunit.client.1.smithi117.stdout:5/699: creat d2/fea x:0 0 0 2024-02-16T19:29:13.156 INFO:tasks.workunit.client.1.smithi117.stdout:1/576: dread d2/d4/f47 [0,4194304] 0 2024-02-16T19:29:13.157 INFO:tasks.workunit.client.1.smithi117.stdout:9/571: creat dd/d47/d5b/d5f/fc5 x:0 0 0 2024-02-16T19:29:13.158 INFO:tasks.workunit.client.1.smithi117.stdout:1/577: stat d2/d1d/c27 0 2024-02-16T19:29:13.159 INFO:tasks.workunit.client.0.smithi022.stdout:0/789: rename d10/d22/dad/de7/ff3 to d10/d22/dad/de7/f106 0 2024-02-16T19:29:13.160 INFO:tasks.workunit.client.1.smithi117.stdout:3/759: dwrite d1/d67/fee [0,4194304] 0 2024-02-16T19:29:13.161 INFO:tasks.workunit.client.0.smithi022.stdout:5/692: creat dd/d1f/d48/d21/d2f/d40/fe8 x:0 0 0 2024-02-16T19:29:13.161 INFO:tasks.workunit.client.1.smithi117.stdout:0/688: creat d0/d9/d14/dbc/fed x:0 0 0 2024-02-16T19:29:13.161 INFO:tasks.workunit.client.0.smithi022.stdout:5/693: readlink dd/d1f/d2e/l57 0 2024-02-16T19:29:13.162 INFO:tasks.workunit.client.0.smithi022.stdout:5/694: read - dd/d1f/d48/d21/d2f/d40/d49/d4b/fdc zero size 2024-02-16T19:29:13.163 INFO:tasks.workunit.client.1.smithi117.stdout:8/650: dwrite d4/d7/fca [0,4194304] 0 2024-02-16T19:29:13.163 INFO:tasks.workunit.client.1.smithi117.stdout:6/629: rmdir d5/d4b/d20/d2d 39 2024-02-16T19:29:13.167 INFO:tasks.workunit.client.1.smithi117.stdout:7/657: symlink d4/d1c/d39/d46/da4/lf2 0 2024-02-16T19:29:13.168 INFO:tasks.workunit.client.1.smithi117.stdout:4/632: rename d0/da/d21/db4/dc6 to d0/da/d21/d3d/d28/d66/da2/dac/dcb 0 2024-02-16T19:29:13.168 INFO:tasks.workunit.client.1.smithi117.stdout:7/658: dread - d4/d1c/d87/d89/d60/d79/f7a zero size 2024-02-16T19:29:13.169 INFO:tasks.workunit.client.0.smithi022.stdout:0/790: dwrite d10/d15/d23/d38/d5a/f5d [0,4194304] 0 2024-02-16T19:29:13.169 INFO:tasks.workunit.client.0.smithi022.stdout:5/695: creat dd/d1f/d48/d21/d5d/dc0/fe9 x:0 0 0 2024-02-16T19:29:13.170 INFO:tasks.workunit.client.1.smithi117.stdout:2/653: truncate d0/d13/f36 837624 0 2024-02-16T19:29:13.171 INFO:tasks.workunit.client.0.smithi022.stdout:5/696: fsync dd/d1f/d48/d21/d2f/d46/d5e/d84/fa3 0 2024-02-16T19:29:13.174 INFO:tasks.workunit.client.1.smithi117.stdout:1/578: symlink d2/d4/dbb/d7f/lc2 0 2024-02-16T19:29:13.174 INFO:tasks.workunit.client.1.smithi117.stdout:9/572: creat dd/d47/d5b/fc6 x:0 0 0 2024-02-16T19:29:13.177 INFO:tasks.workunit.client.1.smithi117.stdout:3/760: read - d1/dd/d5c/fa1 zero size 2024-02-16T19:29:13.177 INFO:tasks.workunit.client.1.smithi117.stdout:6/630: symlink d5/db/d7d/ld0 0 2024-02-16T19:29:13.178 INFO:tasks.workunit.client.0.smithi022.stdout:0/791: dwrite d10/d11/d50/d65/fbe [0,4194304] 0 2024-02-16T19:29:13.179 INFO:tasks.workunit.client.1.smithi117.stdout:8/651: mknod d4/d7/da6/d3a/d65/d7c/ccd 0 2024-02-16T19:29:13.179 INFO:tasks.workunit.client.1.smithi117.stdout:3/761: chown d1/d1b/c6f 18 1 2024-02-16T19:29:13.180 INFO:tasks.workunit.client.1.smithi117.stdout:8/652: chown d4/d12/d2a/f48 8 1 2024-02-16T19:29:13.181 INFO:tasks.workunit.client.0.smithi022.stdout:5/697: rmdir dd/d1f/d48/d21/d2f/d52/d5c/d61/d6a 39 2024-02-16T19:29:13.181 INFO:tasks.workunit.client.1.smithi117.stdout:5/700: mknod d2/d9/d3c/ceb 0 2024-02-16T19:29:13.181 INFO:tasks.workunit.client.0.smithi022.stdout:0/792: chown d10/d11/d27/l64 352 1 2024-02-16T19:29:13.183 INFO:tasks.workunit.client.0.smithi022.stdout:5/698: stat dd/d1f/d48/d21/d2f/d46/d5e/d84/d8b/f9a 0 2024-02-16T19:29:13.185 INFO:tasks.workunit.client.1.smithi117.stdout:2/654: mkdir d0/d3/d34/d94/d46/d68/d76/da1/d56/d60/dc9 0 2024-02-16T19:29:13.188 INFO:tasks.workunit.client.1.smithi117.stdout:9/573: fdatasync dd/d23/f25 0 2024-02-16T19:29:13.188 INFO:tasks.workunit.client.0.smithi022.stdout:5/699: read dd/d1f/d48/d21/d2f/d46/fae [676605,124175] 0 2024-02-16T19:29:13.189 INFO:tasks.workunit.client.1.smithi117.stdout:9/574: chown dd/d47/d5b/fc6 15431562 1 2024-02-16T19:29:13.189 INFO:tasks.workunit.client.0.smithi022.stdout:0/793: write d10/d11/d27/d3f/d58/f6a [6902530,71572] 0 2024-02-16T19:29:13.193 INFO:tasks.workunit.client.0.smithi022.stdout:0/794: read d10/d11/d27/d3f/f5b [1100436,108641] 0 2024-02-16T19:29:13.194 INFO:tasks.workunit.client.1.smithi117.stdout:6/631: creat d5/db/d7d/fd1 x:0 0 0 2024-02-16T19:29:13.195 INFO:tasks.workunit.client.0.smithi022.stdout:5/700: dwrite dd/d1f/d48/d21/d2f/d40/d49/d4b/f66 [0,4194304] 0 2024-02-16T19:29:13.202 INFO:tasks.workunit.client.1.smithi117.stdout:4/633: rmdir d0/da/d21/d3d/d28/d66/d97/d6d 39 2024-02-16T19:29:13.209 INFO:tasks.workunit.client.0.smithi022.stdout:0/795: getdents d10/d11/d27/d3f/d58/d60 0 2024-02-16T19:29:13.210 INFO:tasks.workunit.client.0.smithi022.stdout:0/796: truncate d10/d22/ffe 205899 0 2024-02-16T19:29:13.211 INFO:tasks.workunit.client.0.smithi022.stdout:0/797: read d10/d11/d50/d65/fbe [984752,9143] 0 2024-02-16T19:29:13.211 INFO:tasks.workunit.client.1.smithi117.stdout:9/575: mknod dd/db5/cc7 0 2024-02-16T19:29:13.213 INFO:tasks.workunit.client.1.smithi117.stdout:0/689: creat d0/d9/d14/fee x:0 0 0 2024-02-16T19:29:13.215 INFO:tasks.workunit.client.1.smithi117.stdout:6/632: mknod d5/db/d7d/cd2 0 2024-02-16T19:29:13.217 INFO:tasks.workunit.client.0.smithi022.stdout:0/798: truncate d10/d22/d49/fa3 920185 0 2024-02-16T19:29:13.218 INFO:tasks.workunit.client.1.smithi117.stdout:3/762: creat d1/d31/d89/de1/f109 x:0 0 0 2024-02-16T19:29:13.221 INFO:tasks.workunit.client.1.smithi117.stdout:2/655: dread d0/d3/d40/fc1 [0,4194304] 0 2024-02-16T19:29:13.222 INFO:tasks.workunit.client.1.smithi117.stdout:5/701: mknod d2/d9/d3c/d24/d50/d96/dc4/dca/cec 0 2024-02-16T19:29:13.222 INFO:tasks.workunit.client.1.smithi117.stdout:2/656: fdatasync d0/d13/d75/f59 0 2024-02-16T19:29:13.223 INFO:tasks.workunit.client.0.smithi022.stdout:0/799: symlink d10/d11/d27/d3f/d58/l107 0 2024-02-16T19:29:13.224 INFO:tasks.workunit.client.0.smithi022.stdout:0/800: write d10/d15/d23/d38/d5a/f5d [2147994,43573] 0 2024-02-16T19:29:13.229 INFO:tasks.workunit.client.0.smithi022.stdout:0/801: creat d10/d22/dad/f108 x:0 0 0 2024-02-16T19:29:13.235 INFO:tasks.workunit.client.0.smithi022.stdout:0/802: symlink d10/d7d/dac/de9/l109 0 2024-02-16T19:29:13.238 INFO:tasks.workunit.client.0.smithi022.stdout:0/803: unlink d10/d15/l97 0 2024-02-16T19:29:13.239 INFO:tasks.workunit.client.0.smithi022.stdout:0/804: mkdir d10/d15/d23/dc6/d10a 0 2024-02-16T19:29:13.241 INFO:tasks.workunit.client.0.smithi022.stdout:0/805: rmdir d10/d11 39 2024-02-16T19:29:13.244 INFO:tasks.workunit.client.0.smithi022.stdout:0/806: truncate d10/d11/d27/d3f/d94/faa 690462 0 2024-02-16T19:29:13.245 INFO:tasks.workunit.client.0.smithi022.stdout:0/807: write d10/d22/dad/fd0 [917616,45258] 0 2024-02-16T19:29:13.245 INFO:tasks.workunit.client.1.smithi117.stdout:1/579: link d2/d4/f11 d2/d1d/d60/d7c/d9e/fc3 0 2024-02-16T19:29:13.245 INFO:tasks.workunit.client.1.smithi117.stdout:5/702: stat d2/d9/d3c/d24/d50/d96/dbc 0 2024-02-16T19:29:13.245 INFO:tasks.workunit.client.1.smithi117.stdout:3/763: symlink d1/d1b/d1e/d29/daf/l10a 0 2024-02-16T19:29:13.246 INFO:tasks.workunit.client.1.smithi117.stdout:7/659: truncate d4/d1c/d39/d9f/fd0 149003 0 2024-02-16T19:29:13.246 INFO:tasks.workunit.client.0.smithi022.stdout:0/808: chown d10/d11/d79/d8b/la6 0 1 2024-02-16T19:29:13.247 INFO:tasks.workunit.client.1.smithi117.stdout:2/657: creat d0/d3b/fca x:0 0 0 2024-02-16T19:29:13.249 INFO:tasks.workunit.client.1.smithi117.stdout:8/653: getdents d4/d7/d6a/d72/d84 0 2024-02-16T19:29:13.250 INFO:tasks.workunit.client.1.smithi117.stdout:4/634: link d0/da/l53 d0/da/d4a/lcc 0 2024-02-16T19:29:13.250 INFO:tasks.workunit.client.1.smithi117.stdout:3/764: mknod d1/d30/d9f/dbc/c10b 0 2024-02-16T19:29:13.252 INFO:tasks.workunit.client.0.smithi022.stdout:0/809: dwrite d10/d15/f61 [0,4194304] 0 2024-02-16T19:29:13.253 INFO:tasks.workunit.client.0.smithi022.stdout:0/810: write d10/d11/d27/fdb [4080564,13115] 0 2024-02-16T19:29:13.255 INFO:tasks.workunit.client.0.smithi022.stdout:0/811: read d10/d11/d27/fa4 [389928,20931] 0 2024-02-16T19:29:13.255 INFO:tasks.workunit.client.0.smithi022.stdout:0/812: stat d10/f17 0 2024-02-16T19:29:13.256 INFO:tasks.workunit.client.0.smithi022.stdout:0/813: fdatasync d10/d11/d27/d3f/f8a 0 2024-02-16T19:29:13.260 INFO:tasks.workunit.client.1.smithi117.stdout:0/690: truncate d0/d30/fd2 98741 0 2024-02-16T19:29:13.262 INFO:tasks.workunit.client.0.smithi022.stdout:0/814: creat d10/d15/d23/dc6/f10b x:0 0 0 2024-02-16T19:29:13.266 INFO:tasks.workunit.client.1.smithi117.stdout:8/654: rename d4/d12/d2a/f5c to d4/d7/da6/d3a/d65/d96/fce 0 2024-02-16T19:29:13.266 INFO:tasks.workunit.client.1.smithi117.stdout:6/633: dwrite d5/db/f55 [0,4194304] 0 2024-02-16T19:29:13.267 INFO:tasks.workunit.client.0.smithi022.stdout:0/815: fsync d10/d11/d27/d3f/df0/ff1 0 2024-02-16T19:29:13.268 INFO:tasks.workunit.client.1.smithi117.stdout:8/655: chown d4/d7/d6a/d6f/da8/d76/fc5 737 1 2024-02-16T19:29:13.268 INFO:tasks.workunit.client.1.smithi117.stdout:4/635: truncate d0/da/d21/d3d/d28/f39 5064096 0 2024-02-16T19:29:13.269 INFO:tasks.workunit.client.1.smithi117.stdout:8/656: fdatasync d4/d7/da6/d3a/d43/f6b 0 2024-02-16T19:29:13.275 INFO:tasks.workunit.client.1.smithi117.stdout:0/691: creat d0/d9/d14/d1d/d60/dc3/fef x:0 0 0 2024-02-16T19:29:13.276 INFO:tasks.workunit.client.0.smithi022.stdout:0/816: getdents d10/d15/d23/dc6/d10a 0 2024-02-16T19:29:13.277 INFO:tasks.workunit.client.1.smithi117.stdout:2/658: dwrite d0/d3/d22/f7b [0,4194304] 0 2024-02-16T19:29:13.278 INFO:tasks.workunit.client.1.smithi117.stdout:1/580: link d2/d1d/d60/d68/f81 d2/d1d/d1f/fc4 0 2024-02-16T19:29:13.280 INFO:tasks.workunit.client.0.smithi022.stdout:0/817: read d10/d11/d50/d65/fbe [2585065,113033] 0 2024-02-16T19:29:13.281 INFO:tasks.workunit.client.0.smithi022.stdout:4/568: sync 2024-02-16T19:29:13.281 INFO:tasks.workunit.client.0.smithi022.stdout:8/599: sync 2024-02-16T19:29:13.281 INFO:tasks.workunit.client.0.smithi022.stdout:2/618: sync 2024-02-16T19:29:13.281 INFO:tasks.workunit.client.0.smithi022.stdout:7/599: sync 2024-02-16T19:29:13.281 INFO:tasks.workunit.client.0.smithi022.stdout:6/650: sync 2024-02-16T19:29:13.281 INFO:tasks.workunit.client.0.smithi022.stdout:9/636: sync 2024-02-16T19:29:13.282 INFO:tasks.workunit.client.0.smithi022.stdout:3/558: sync 2024-02-16T19:29:13.283 INFO:tasks.workunit.client.1.smithi117.stdout:7/660: mkdir d4/d1c/d87/d89/d5f/dae/df3 0 2024-02-16T19:29:13.285 INFO:tasks.workunit.client.1.smithi117.stdout:0/692: dwrite d0/f1 [4194304,4194304] 0 2024-02-16T19:29:13.285 INFO:tasks.workunit.client.1.smithi117.stdout:7/661: chown d4/f28 21294153 1 2024-02-16T19:29:13.287 INFO:tasks.workunit.client.1.smithi117.stdout:0/693: fsync d0/d9/d22/db7/fe8 0 2024-02-16T19:29:13.288 INFO:tasks.workunit.client.1.smithi117.stdout:8/657: chown d4/d7/l29 351 1 2024-02-16T19:29:13.290 INFO:tasks.workunit.client.0.smithi022.stdout:8/600: fdatasync d3/d25/f7d 0 2024-02-16T19:29:13.292 INFO:tasks.workunit.client.0.smithi022.stdout:3/559: dread d7/d12/d18/f42 [0,4194304] 0 2024-02-16T19:29:13.292 INFO:tasks.workunit.client.1.smithi117.stdout:6/634: dread d5/d4b/d71/dc7/f87 [0,4194304] 0 2024-02-16T19:29:13.293 INFO:tasks.workunit.client.1.smithi117.stdout:6/635: write d5/d4b/d71/dc7/f50 [1302974,65145] 0 2024-02-16T19:29:13.294 INFO:tasks.workunit.client.0.smithi022.stdout:7/600: dread d9/da/d1f/f81 [0,4194304] 0 2024-02-16T19:29:13.294 INFO:tasks.workunit.client.1.smithi117.stdout:5/703: link d2/d9/d3c/d24/d50/d52/l19 d2/d9/d3c/d24/d90/led 0 2024-02-16T19:29:13.295 INFO:tasks.workunit.client.0.smithi022.stdout:7/601: chown d9/da/d3b/dba 428845897 1 2024-02-16T19:29:13.295 INFO:tasks.workunit.client.1.smithi117.stdout:6/636: chown d5/d4b/d71/dc7/d49/d59/f5e 10321572 1 2024-02-16T19:29:13.298 INFO:tasks.workunit.client.0.smithi022.stdout:6/651: rmdir d1/d8/da/d17/d48/d51 39 2024-02-16T19:29:13.300 INFO:tasks.workunit.client.1.smithi117.stdout:3/765: creat d1/d30/f10c x:0 0 0 2024-02-16T19:29:13.301 INFO:tasks.workunit.client.1.smithi117.stdout:3/766: chown d1/dd/d5c/d85/cdf 29 1 2024-02-16T19:29:13.302 INFO:tasks.workunit.client.0.smithi022.stdout:4/569: unlink d4/d38/d6a/c8f 0 2024-02-16T19:29:13.302 INFO:tasks.workunit.client.0.smithi022.stdout:6/652: dread d1/d8/d23/f30 [0,4194304] 0 2024-02-16T19:29:13.303 INFO:tasks.workunit.client.0.smithi022.stdout:0/818: write d10/d11/d50/d65/fbe [2503571,22718] 0 2024-02-16T19:29:13.303 INFO:tasks.workunit.client.1.smithi117.stdout:2/659: creat d0/d13/d4d/d53/d58/fcb x:0 0 0 2024-02-16T19:29:13.309 INFO:tasks.workunit.client.1.smithi117.stdout:0/694: dread d0/f5 [4194304,4194304] 0 2024-02-16T19:29:13.311 INFO:tasks.workunit.client.0.smithi022.stdout:2/619: dwrite d0/d2/d18/d19/d90/fa3 [4194304,4194304] 0 2024-02-16T19:29:13.311 INFO:tasks.workunit.client.0.smithi022.stdout:9/637: mkdir db/d14/d16/d5d/d65/d6b/dd5/dd9 0 2024-02-16T19:29:13.312 INFO:tasks.workunit.client.0.smithi022.stdout:2/620: chown d0/d2/d18/d1f/d6c/d99 13073995 1 2024-02-16T19:29:13.314 INFO:tasks.workunit.client.1.smithi117.stdout:8/658: write d4/d73/fa0 [1146593,22716] 0 2024-02-16T19:29:13.315 INFO:tasks.workunit.client.1.smithi117.stdout:7/662: dwrite d4/d1c/d87/d89/fd4 [4194304,4194304] 0 2024-02-16T19:29:13.316 INFO:tasks.workunit.client.0.smithi022.stdout:7/602: creat d9/da/d1f/d44/d5b/d95/fbf x:0 0 0 2024-02-16T19:29:13.316 INFO:tasks.workunit.client.0.smithi022.stdout:3/560: write d7/d12/d18/f1b [4889005,14514] 0 2024-02-16T19:29:13.317 INFO:tasks.workunit.client.0.smithi022.stdout:7/603: chown d9/da/d2c/d2f/l2a 3095778 1 2024-02-16T19:29:13.317 INFO:tasks.workunit.client.0.smithi022.stdout:3/561: chown d7/d12/f14 11282667 1 2024-02-16T19:29:13.319 INFO:tasks.workunit.client.0.smithi022.stdout:4/570: mknod d4/d38/d6a/cb1 0 2024-02-16T19:29:13.320 INFO:tasks.workunit.client.1.smithi117.stdout:9/576: sync 2024-02-16T19:29:13.322 INFO:tasks.workunit.client.1.smithi117.stdout:5/704: rename d2/d9/d3c/d24/d50/d96/dbc/l5c to d2/d3/d21/d2a/dcb/lee 0 2024-02-16T19:29:13.323 INFO:tasks.workunit.client.1.smithi117.stdout:5/705: stat d2/d9/d3c/d24/d50/l87 0 2024-02-16T19:29:13.324 INFO:tasks.workunit.client.0.smithi022.stdout:6/653: creat d1/d8/da/d17/d85/fe6 x:0 0 0 2024-02-16T19:29:13.325 INFO:tasks.workunit.client.0.smithi022.stdout:6/654: write d1/d8/d3c/f44 [332839,2598] 0 2024-02-16T19:29:13.327 INFO:tasks.workunit.client.0.smithi022.stdout:8/601: creat d3/dd3/fd7 x:0 0 0 2024-02-16T19:29:13.327 INFO:tasks.workunit.client.0.smithi022.stdout:6/655: stat d1/d8/d23/f30 0 2024-02-16T19:29:13.327 INFO:tasks.workunit.client.0.smithi022.stdout:8/602: readlink d3/d25/d33/l44 0 2024-02-16T19:29:13.328 INFO:tasks.workunit.client.1.smithi117.stdout:2/660: creat d0/d3/d34/d94/d46/d68/d76/da1/d56/fcc x:0 0 0 2024-02-16T19:29:13.329 INFO:tasks.workunit.client.1.smithi117.stdout:2/661: write d0/d3/d40/fba [935416,125640] 0 2024-02-16T19:29:13.330 INFO:tasks.workunit.client.1.smithi117.stdout:4/636: link d0/l49 d0/da/d21/d3d/d28/d64/d73/lcd 0 2024-02-16T19:29:13.333 INFO:tasks.workunit.client.0.smithi022.stdout:9/638: chown db/d14/f2d 59405 1 2024-02-16T19:29:13.334 INFO:tasks.workunit.client.1.smithi117.stdout:6/637: write d5/d4b/d71/dc7/f2c [1614208,19267] 0 2024-02-16T19:29:13.337 INFO:tasks.workunit.client.1.smithi117.stdout:8/659: mknod d4/d7/da6/d3a/d65/d7c/ccf 0 2024-02-16T19:29:13.337 INFO:tasks.workunit.client.0.smithi022.stdout:7/604: stat d9/d1b/c68 0 2024-02-16T19:29:13.338 INFO:tasks.workunit.client.0.smithi022.stdout:3/562: creat d7/d12/d18/dd/d3c/fcc x:0 0 0 2024-02-16T19:29:13.341 INFO:tasks.workunit.client.0.smithi022.stdout:4/571: unlink d4/d6/f7f 0 2024-02-16T19:29:13.343 INFO:tasks.workunit.client.0.smithi022.stdout:0/819: mknod d10/d15/d23/c10c 0 2024-02-16T19:29:13.346 INFO:tasks.workunit.client.1.smithi117.stdout:1/581: creat d2/d19/fc5 x:0 0 0 2024-02-16T19:29:13.346 INFO:tasks.workunit.client.0.smithi022.stdout:6/656: chown d1/d8/da/d17/d53/d7f/da3/cd5 14 1 2024-02-16T19:29:13.348 INFO:tasks.workunit.client.1.smithi117.stdout:2/662: creat d0/d3/d34/d94/d46/d68/d93/da9/fcd x:0 0 0 2024-02-16T19:29:13.349 INFO:tasks.workunit.client.1.smithi117.stdout:7/663: write d4/d13/fb0 [339051,45891] 0 2024-02-16T19:29:13.350 INFO:tasks.workunit.client.1.smithi117.stdout:7/664: write d4/d1c/d39/d46/f75 [1791570,15675] 0 2024-02-16T19:29:13.351 INFO:tasks.workunit.client.1.smithi117.stdout:4/637: rename d0/da/d21/d3d/d28/d64/d73/d91/c7b to d0/da/d21/d3d/d28/d66/da2/cce 0 2024-02-16T19:29:13.354 INFO:tasks.workunit.client.0.smithi022.stdout:2/621: rename d0/d2/d18/d19/f30 to d0/d2/d18/d38/d6b/fca 0 2024-02-16T19:29:13.355 INFO:tasks.workunit.client.1.smithi117.stdout:9/577: write dd/d47/d48/f55 [140849,12112] 0 2024-02-16T19:29:13.356 INFO:tasks.workunit.client.1.smithi117.stdout:6/638: creat d5/d4b/d71/dc7/d49/d59/fd3 x:0 0 0 2024-02-16T19:29:13.357 INFO:tasks.workunit.client.0.smithi022.stdout:7/605: creat d9/da/d1f/d44/fc0 x:0 0 0 2024-02-16T19:29:13.357 INFO:tasks.workunit.client.1.smithi117.stdout:5/706: write d2/d5d/f9d [2891333,48923] 0 2024-02-16T19:29:13.358 INFO:tasks.workunit.client.0.smithi022.stdout:3/563: creat d7/d12/d18/dd/d3c/fcd x:0 0 0 2024-02-16T19:29:13.360 INFO:tasks.workunit.client.0.smithi022.stdout:0/820: creat d10/d11/d27/d77/f10d x:0 0 0 2024-02-16T19:29:13.362 INFO:tasks.workunit.client.0.smithi022.stdout:9/639: mknod db/d14/cda 0 2024-02-16T19:29:13.367 INFO:tasks.workunit.client.0.smithi022.stdout:9/640: dread db/d14/f60 [0,4194304] 0 2024-02-16T19:29:13.368 INFO:tasks.workunit.client.1.smithi117.stdout:4/638: fsync d0/da/d21/d3d/d28/d64/d73/d91/f31 0 2024-02-16T19:29:13.370 INFO:tasks.workunit.client.0.smithi022.stdout:4/572: symlink d4/lb2 0 2024-02-16T19:29:13.371 INFO:tasks.workunit.client.1.smithi117.stdout:0/695: mknod d0/d9/d14/d23/d33/cf0 0 2024-02-16T19:29:13.376 INFO:tasks.workunit.client.0.smithi022.stdout:8/603: creat d3/d12/d2b/d54/d74/fd8 x:0 0 0 2024-02-16T19:29:13.377 INFO:tasks.workunit.client.1.smithi117.stdout:5/707: mkdir d2/d3/d21/d2a/dcb/def 0 2024-02-16T19:29:13.379 INFO:tasks.workunit.client.1.smithi117.stdout:2/663: write d0/d3/f31 [968945,81097] 0 2024-02-16T19:29:13.379 INFO:tasks.workunit.client.1.smithi117.stdout:8/660: mknod d4/d7/da6/d3a/d65/d7c/dc9/cd0 0 2024-02-16T19:29:13.381 INFO:tasks.workunit.client.0.smithi022.stdout:7/606: write d9/da/f62 [1154616,81452] 0 2024-02-16T19:29:13.381 INFO:tasks.workunit.client.1.smithi117.stdout:3/767: getdents d1/d98 0 2024-02-16T19:29:13.382 INFO:tasks.workunit.client.1.smithi117.stdout:3/768: dread - d1/d1b/d20/fe5 zero size 2024-02-16T19:29:13.384 INFO:tasks.workunit.client.1.smithi117.stdout:1/582: mkdir d2/d19/d59/d89/d69/dad/dc6 0 2024-02-16T19:29:13.386 INFO:tasks.workunit.client.0.smithi022.stdout:6/657: truncate d1/d8/d3c/f9d 3326776 0 2024-02-16T19:29:13.390 INFO:tasks.workunit.client.1.smithi117.stdout:4/639: chown d0/da/d21/d3d/d28/d64/d73/d91/c46 1835 1 2024-02-16T19:29:13.393 INFO:tasks.workunit.client.1.smithi117.stdout:9/578: link dd/d45/f4f dd/d45/d31/d63/d8e/fc8 0 2024-02-16T19:29:13.396 INFO:tasks.workunit.client.1.smithi117.stdout:6/639: symlink d5/d4b/d44/d52/ld4 0 2024-02-16T19:29:13.399 INFO:tasks.workunit.client.1.smithi117.stdout:0/696: write d0/d9/d14/d1d/d60/d76/d7b/d8d/f32 [570794,33145] 0 2024-02-16T19:29:13.400 INFO:tasks.workunit.client.1.smithi117.stdout:8/661: mknod d4/d73/cd1 0 2024-02-16T19:29:13.401 INFO:tasks.workunit.client.1.smithi117.stdout:6/640: dwrite d5/db/d7d/d32/f92 [0,4194304] 0 2024-02-16T19:29:13.402 INFO:tasks.workunit.client.1.smithi117.stdout:2/664: creat d0/d13/d4d/d53/fce x:0 0 0 2024-02-16T19:29:13.403 INFO:tasks.workunit.client.1.smithi117.stdout:8/662: write d4/d7/d6a/d6f/da8/d76/faa [913294,92949] 0 2024-02-16T19:29:13.404 INFO:tasks.workunit.client.1.smithi117.stdout:3/769: creat d1/d30/d80/f10d x:0 0 0 2024-02-16T19:29:13.405 INFO:tasks.workunit.client.1.smithi117.stdout:8/663: dread d4/d12/d2a/f48 [0,4194304] 0 2024-02-16T19:29:13.406 INFO:tasks.workunit.client.1.smithi117.stdout:1/583: mkdir d2/d19/d59/d76/d8c/dc7 0 2024-02-16T19:29:13.409 INFO:tasks.workunit.client.1.smithi117.stdout:7/665: rename d4/d13/d63/d71/f9c to d4/d1c/d39/ff4 0 2024-02-16T19:29:13.417 INFO:tasks.workunit.client.1.smithi117.stdout:1/584: dread d2/d4/f15 [0,4194304] 0 2024-02-16T19:29:13.417 INFO:tasks.workunit.client.1.smithi117.stdout:0/697: symlink d0/d9/d2c/da9/lf1 0 2024-02-16T19:29:13.417 INFO:tasks.workunit.client.1.smithi117.stdout:1/585: stat d2/d1d/d1f/f5e 0 2024-02-16T19:29:13.422 INFO:tasks.workunit.client.1.smithi117.stdout:3/770: creat d1/d1b/d1e/d37/ddc/f10e x:0 0 0 2024-02-16T19:29:13.422 INFO:tasks.workunit.client.1.smithi117.stdout:3/771: chown d1/d67 2739083 1 2024-02-16T19:29:13.426 INFO:tasks.workunit.client.1.smithi117.stdout:8/664: symlink d4/d7/da6/d3a/d54/ld2 0 2024-02-16T19:29:13.429 INFO:tasks.workunit.client.1.smithi117.stdout:4/640: rename d0/da/d21/db4/lc4 to d0/da/d21/dc2/lcf 0 2024-02-16T19:29:13.431 INFO:tasks.workunit.client.1.smithi117.stdout:7/666: creat d4/d1c/d87/d89/d60/d9b/ff5 x:0 0 0 2024-02-16T19:29:13.432 INFO:tasks.workunit.client.1.smithi117.stdout:9/579: creat dd/d45/d31/d63/d8e/dbb/fc9 x:0 0 0 2024-02-16T19:29:13.438 INFO:tasks.workunit.client.1.smithi117.stdout:6/641: fsync d5/d4b/d20/f78 0 2024-02-16T19:29:13.439 INFO:tasks.workunit.client.1.smithi117.stdout:6/642: dread - d5/d4b/d71/dc7/d49/d59/fd3 zero size 2024-02-16T19:29:13.440 INFO:tasks.workunit.client.1.smithi117.stdout:3/772: mkdir d1/d30/d52/d10f 0 2024-02-16T19:29:13.442 INFO:tasks.workunit.client.1.smithi117.stdout:8/665: mknod d4/d7/da6/cd3 0 2024-02-16T19:29:13.445 INFO:tasks.workunit.client.1.smithi117.stdout:3/773: dwrite d1/dd/f48 [0,4194304] 0 2024-02-16T19:29:13.445 INFO:tasks.workunit.client.1.smithi117.stdout:4/641: chown d0/da/d21/d3d/d28/d66/d97/c70 15 1 2024-02-16T19:29:13.447 INFO:tasks.workunit.client.1.smithi117.stdout:4/642: dread - d0/da/d21/d3d/d28/d64/d73/d91/fa5 zero size 2024-02-16T19:29:13.449 INFO:tasks.workunit.client.1.smithi117.stdout:9/580: creat dd/d47/d5b/d5f/d77/d7c/fca x:0 0 0 2024-02-16T19:29:13.449 INFO:tasks.workunit.client.1.smithi117.stdout:7/667: read d4/d1c/d39/d46/d47/f82 [2216211,59732] 0 2024-02-16T19:29:13.450 INFO:tasks.workunit.client.1.smithi117.stdout:5/708: getdents d2/d3/d21/d2a/dcb 0 2024-02-16T19:29:13.454 INFO:tasks.workunit.client.1.smithi117.stdout:3/774: dwrite d1/d31/d89/de1/f109 [0,4194304] 0 2024-02-16T19:29:13.454 INFO:tasks.workunit.client.1.smithi117.stdout:9/581: read dd/d47/d5b/d5f/d77/d7c/fa2 [2356745,42460] 0 2024-02-16T19:29:13.457 INFO:tasks.workunit.client.0.smithi022.stdout:7/607: creat d9/da/d2c/fc1 x:0 0 0 2024-02-16T19:29:13.459 INFO:tasks.workunit.client.0.smithi022.stdout:9/641: mknod db/d14/d16/d21/d37/cdb 0 2024-02-16T19:29:13.459 INFO:tasks.workunit.client.1.smithi117.stdout:0/698: dwrite d0/d9/d14/d23/d33/fb3 [0,4194304] 0 2024-02-16T19:29:13.460 INFO:tasks.workunit.client.0.smithi022.stdout:3/564: link d7/d12/l2d d7/d12/d18/d4b/db6/lce 0 2024-02-16T19:29:13.461 INFO:tasks.workunit.client.0.smithi022.stdout:3/565: fdatasync d7/d12/d18/dd/d3c/fcd 0 2024-02-16T19:29:13.464 INFO:tasks.workunit.client.0.smithi022.stdout:0/821: link d10/d7d/dac/de9/dee/lfd d10/d7d/dac/de9/l10e 0 2024-02-16T19:29:13.465 INFO:tasks.workunit.client.0.smithi022.stdout:0/822: readlink d10/d15/d23/d8f/dc1/l103 0 2024-02-16T19:29:13.466 INFO:tasks.workunit.client.0.smithi022.stdout:0/823: dread - d10/d15/d23/dc6/f10b zero size 2024-02-16T19:29:13.467 INFO:tasks.workunit.client.0.smithi022.stdout:0/824: chown d10/d22/dad/f108 0 1 2024-02-16T19:29:13.469 INFO:tasks.workunit.client.1.smithi117.stdout:0/699: dread d0/d9/d14/d23/d33/fd8 [0,4194304] 0 2024-02-16T19:29:13.470 INFO:tasks.workunit.client.1.smithi117.stdout:5/709: rmdir d2/d3/d21/d49/d59/daa 39 2024-02-16T19:29:13.471 INFO:tasks.workunit.client.0.smithi022.stdout:6/658: mknod d1/ddc/ce7 0 2024-02-16T19:29:13.475 INFO:tasks.workunit.client.0.smithi022.stdout:6/659: dread d1/fba [0,4194304] 0 2024-02-16T19:29:13.476 INFO:tasks.workunit.client.0.smithi022.stdout:3/566: creat d7/d12/d18/dd/d31/dbf/fcf x:0 0 0 2024-02-16T19:29:13.478 INFO:tasks.workunit.client.0.smithi022.stdout:3/567: write d7/d12/d18/dd/f28 [1767101,90490] 0 2024-02-16T19:29:13.478 INFO:tasks.workunit.client.0.smithi022.stdout:8/604: link d3/d12/d17/d64/f7c d3/d12/db0/fd9 0 2024-02-16T19:29:13.484 INFO:tasks.workunit.client.1.smithi117.stdout:9/582: creat dd/d47/d5b/d5f/d77/fcb x:0 0 0 2024-02-16T19:29:13.485 INFO:tasks.workunit.client.0.smithi022.stdout:3/568: dwrite d7/d12/d18/dd/d3c/d5a/fc7 [0,4194304] 0 2024-02-16T19:29:13.487 INFO:tasks.workunit.client.0.smithi022.stdout:3/569: dread - d7/d12/d18/dd/d31/dbf/fcf zero size 2024-02-16T19:29:13.487 INFO:tasks.workunit.client.1.smithi117.stdout:9/583: dread dd/d47/d5b/d5f/f9b [0,4194304] 0 2024-02-16T19:29:13.488 INFO:tasks.workunit.client.1.smithi117.stdout:9/584: chown dd/d47/d5b/cb3 169 1 2024-02-16T19:29:13.489 INFO:tasks.workunit.client.1.smithi117.stdout:9/585: dread - dd/d47/fbe zero size 2024-02-16T19:29:13.489 INFO:tasks.workunit.client.0.smithi022.stdout:6/660: mkdir d1/d8/da/d17/d53/d7f/db3/de8 0 2024-02-16T19:29:13.491 INFO:tasks.workunit.client.0.smithi022.stdout:6/661: stat d1/d8/da/d17/d53/c7a 0 2024-02-16T19:29:13.494 INFO:tasks.workunit.client.1.smithi117.stdout:2/665: rename d0/l9b to d0/d13/d4d/lcf 0 2024-02-16T19:29:13.494 INFO:tasks.workunit.client.1.smithi117.stdout:3/775: write d1/dd/f17 [2871733,107114] 0 2024-02-16T19:29:13.494 INFO:tasks.workunit.client.0.smithi022.stdout:7/608: dread d9/da/d1f/d40/f5e [0,4194304] 0 2024-02-16T19:29:13.495 INFO:tasks.workunit.client.0.smithi022.stdout:7/609: readlink d9/da/d1f/d40/l79 0 2024-02-16T19:29:13.496 INFO:tasks.workunit.client.1.smithi117.stdout:6/643: truncate d5/db/d7d/d32/d37/f41 4161492 0 2024-02-16T19:29:13.499 INFO:tasks.workunit.client.1.smithi117.stdout:0/700: creat d0/d9/d22/d27/d31/ff2 x:0 0 0 2024-02-16T19:29:13.499 INFO:tasks.workunit.client.0.smithi022.stdout:9/642: link db/d14/d19/d1a/c66 db/d14/d16/d49/d90/cdc 0 2024-02-16T19:29:13.500 INFO:tasks.workunit.client.1.smithi117.stdout:5/710: read - d2/d3/d21/d2a/f84 zero size 2024-02-16T19:29:13.501 INFO:tasks.workunit.client.1.smithi117.stdout:1/586: getdents d2/d19/d59/d76/d8c 0 2024-02-16T19:29:13.503 INFO:tasks.workunit.client.0.smithi022.stdout:6/662: rename d1/d8/d23/f30 to d1/d8/da/d17/d53/d7f/fe9 0 2024-02-16T19:29:13.503 INFO:tasks.workunit.client.1.smithi117.stdout:6/644: dwrite d5/d4b/d44/f56 [0,4194304] 0 2024-02-16T19:29:13.504 INFO:tasks.workunit.client.0.smithi022.stdout:8/605: creat d3/d12/d2b/d54/db8/fda x:0 0 0 2024-02-16T19:29:13.507 INFO:tasks.workunit.client.1.smithi117.stdout:8/666: creat d4/d12/fd4 x:0 0 0 2024-02-16T19:29:13.513 INFO:tasks.workunit.client.0.smithi022.stdout:8/606: mkdir d3/d25/ddb 0 2024-02-16T19:29:13.513 INFO:tasks.workunit.client.0.smithi022.stdout:8/607: readlink d3/d12/l89 0 2024-02-16T19:29:13.514 INFO:tasks.workunit.client.0.smithi022.stdout:1/675: sync 2024-02-16T19:29:13.514 INFO:tasks.workunit.client.0.smithi022.stdout:0/825: truncate d10/d22/f53 2999222 0 2024-02-16T19:29:13.515 INFO:tasks.workunit.client.0.smithi022.stdout:7/610: symlink d9/da/d4e/d83/lc2 0 2024-02-16T19:29:13.517 INFO:tasks.workunit.client.0.smithi022.stdout:9/643: mkdir db/d14/d16/d5d/d65/d6b/dd5/dd9/ddd 0 2024-02-16T19:29:13.522 INFO:tasks.workunit.client.0.smithi022.stdout:8/608: creat d3/d12/d2b/da9/fdc x:0 0 0 2024-02-16T19:29:13.523 INFO:tasks.workunit.client.1.smithi117.stdout:0/701: mknod d0/d9/d22/db7/cf3 0 2024-02-16T19:29:13.523 INFO:tasks.workunit.client.0.smithi022.stdout:3/570: dwrite d7/d12/d18/dd/ff [0,4194304] 0 2024-02-16T19:29:13.525 INFO:tasks.workunit.client.0.smithi022.stdout:3/571: fdatasync d7/d12/d18/d4b/f9e 0 2024-02-16T19:29:13.525 INFO:tasks.workunit.client.1.smithi117.stdout:5/711: mknod d2/d9/d3c/d24/d50/d96/dbc/d30/d5a/cf0 0 2024-02-16T19:29:13.527 INFO:tasks.workunit.client.0.smithi022.stdout:7/611: mkdir d9/d5d/dc3 0 2024-02-16T19:29:13.529 INFO:tasks.workunit.client.1.smithi117.stdout:1/587: mkdir d2/d4/dbb/d7f/dc8 0 2024-02-16T19:29:13.532 INFO:tasks.workunit.client.1.smithi117.stdout:6/645: chown d5/d4b/d20/d2d/f96 48752841 1 2024-02-16T19:29:13.536 INFO:tasks.workunit.client.1.smithi117.stdout:8/667: mknod d4/d1c/cd5 0 2024-02-16T19:29:13.537 INFO:tasks.workunit.client.0.smithi022.stdout:8/609: rmdir d3/d12/d2b/d2e 39 2024-02-16T19:29:13.538 INFO:tasks.workunit.client.0.smithi022.stdout:1/676: dread d6/f1c [0,4194304] 0 2024-02-16T19:29:13.539 INFO:tasks.workunit.client.1.smithi117.stdout:9/586: creat dd/d45/d31/d63/dba/fcc x:0 0 0 2024-02-16T19:29:13.540 INFO:tasks.workunit.client.1.smithi117.stdout:8/668: dread d4/f3c [0,4194304] 0 2024-02-16T19:29:13.540 INFO:tasks.workunit.client.1.smithi117.stdout:8/669: chown d4/d7/da6/d3a/d43 86122 1 2024-02-16T19:29:13.541 INFO:tasks.workunit.client.1.smithi117.stdout:2/666: symlink d0/d3/ld0 0 2024-02-16T19:29:13.543 INFO:tasks.workunit.client.0.smithi022.stdout:3/572: creat d7/d12/d18/dd/d24/fd0 x:0 0 0 2024-02-16T19:29:13.544 INFO:tasks.workunit.client.0.smithi022.stdout:3/573: chown d7/d12/d18/dd/d3c/cad 1124420 1 2024-02-16T19:29:13.544 INFO:tasks.workunit.client.0.smithi022.stdout:7/612: creat d9/da/d4e/fc4 x:0 0 0 2024-02-16T19:29:13.545 INFO:tasks.workunit.client.1.smithi117.stdout:5/712: dread d2/d9/d3c/d24/f26 [0,4194304] 0 2024-02-16T19:29:13.546 INFO:tasks.workunit.client.0.smithi022.stdout:3/574: write d7/d12/d18/dd/d3c/fcc [486803,51667] 0 2024-02-16T19:29:13.547 INFO:tasks.workunit.client.1.smithi117.stdout:4/643: getdents d0/da 0 2024-02-16T19:29:13.548 INFO:tasks.workunit.client.1.smithi117.stdout:7/668: getdents d4/d1c/d39 0 2024-02-16T19:29:13.549 INFO:tasks.workunit.client.0.smithi022.stdout:6/663: getdents d1/d8/da/d17/d53/d7f 0 2024-02-16T19:29:13.550 INFO:tasks.workunit.client.0.smithi022.stdout:9/644: rename db/d14/d16/d27/d2e/lb9 to db/d14/d16/d5d/d65/d6b/d89/dd2/lde 0 2024-02-16T19:29:13.551 INFO:tasks.workunit.client.0.smithi022.stdout:0/826: dread d10/d22/f20 [0,4194304] 0 2024-02-16T19:29:13.554 INFO:tasks.workunit.client.0.smithi022.stdout:7/613: dwrite d9/da/d1f/d40/fb9 [0,4194304] 0 2024-02-16T19:29:13.554 INFO:tasks.workunit.client.1.smithi117.stdout:6/646: write d5/d4b/d20/da4/fc6 [616285,36006] 0 2024-02-16T19:29:13.558 INFO:tasks.workunit.client.1.smithi117.stdout:9/587: mknod dd/d47/d48/ccd 0 2024-02-16T19:29:13.559 INFO:tasks.workunit.client.1.smithi117.stdout:9/588: stat dd/d23/daf/lc3 0 2024-02-16T19:29:13.562 INFO:tasks.workunit.client.1.smithi117.stdout:8/670: mknod d4/d7/da6/d3a/d43/d44/d5f/cd6 0 2024-02-16T19:29:13.562 INFO:tasks.workunit.client.0.smithi022.stdout:8/610: symlink d3/d12/d2b/dca/ldd 0 2024-02-16T19:29:13.565 INFO:tasks.workunit.client.0.smithi022.stdout:9/645: fsync db/d14/d16/d27/d55/f99 0 2024-02-16T19:29:13.566 INFO:tasks.workunit.client.1.smithi117.stdout:1/588: dread d2/d19/d59/d89/d7b/f7e [0,4194304] 0 2024-02-16T19:29:13.568 INFO:tasks.workunit.client.0.smithi022.stdout:7/614: creat d9/da/d1f/fc5 x:0 0 0 2024-02-16T19:29:13.569 INFO:tasks.workunit.client.1.smithi117.stdout:4/644: symlink d0/da/d21/d3d/d28/d64/d8a/ld0 0 2024-02-16T19:29:13.570 INFO:tasks.workunit.client.1.smithi117.stdout:4/645: fsync d0/da/d21/d3d/d28/d64/d73/d91/fbe 0 2024-02-16T19:29:13.572 INFO:tasks.workunit.client.1.smithi117.stdout:7/669: mknod d4/d1c/dc0/cf6 0 2024-02-16T19:29:13.574 INFO:tasks.workunit.client.0.smithi022.stdout:8/611: truncate d3/d25/f5c 1754041 0 2024-02-16T19:29:13.576 INFO:tasks.workunit.client.1.smithi117.stdout:0/702: dwrite d0/d9/d22/fb8 [0,4194304] 0 2024-02-16T19:29:13.576 INFO:tasks.workunit.client.0.smithi022.stdout:1/677: dwrite d6/d8/d22/d3f/f73 [0,4194304] 0 2024-02-16T19:29:13.577 INFO:tasks.workunit.client.1.smithi117.stdout:6/647: truncate d5/d4b/d71/dc7/d72/f90 823606 0 2024-02-16T19:29:13.578 INFO:tasks.workunit.client.1.smithi117.stdout:4/646: dwrite d0/da/d21/d3d/d28/d66/d97/f94 [0,4194304] 0 2024-02-16T19:29:13.580 INFO:tasks.workunit.client.1.smithi117.stdout:6/648: chown d5/db/d7d/d32/d79/lbb 3 1 2024-02-16T19:29:13.580 INFO:tasks.workunit.client.1.smithi117.stdout:9/589: dread dd/d47/f7b [0,4194304] 0 2024-02-16T19:29:13.582 INFO:tasks.workunit.client.1.smithi117.stdout:8/671: creat d4/d7/d6a/d6f/da8/d76/d8c/fd7 x:0 0 0 2024-02-16T19:29:13.583 INFO:tasks.workunit.client.1.smithi117.stdout:2/667: truncate d0/d13/d4d/d80/f88 331266 0 2024-02-16T19:29:13.584 INFO:tasks.workunit.client.0.smithi022.stdout:6/664: fsync d1/d8/da/d17/d85/d5b/f6c 0 2024-02-16T19:29:13.585 INFO:tasks.workunit.client.1.smithi117.stdout:3/776: getdents d1/d31/d89 0 2024-02-16T19:29:13.585 INFO:tasks.workunit.client.0.smithi022.stdout:8/612: readlink d3/l4b 0 2024-02-16T19:29:13.586 INFO:tasks.workunit.client.1.smithi117.stdout:1/589: dread d2/d1d/f4f [0,4194304] 0 2024-02-16T19:29:13.586 INFO:tasks.workunit.client.0.smithi022.stdout:8/613: write d3/d25/f79 [1879434,57741] 0 2024-02-16T19:29:13.590 INFO:tasks.workunit.client.0.smithi022.stdout:1/678: creat d6/d8/d22/d47/fd3 x:0 0 0 2024-02-16T19:29:13.591 INFO:tasks.workunit.client.0.smithi022.stdout:3/575: link d7/l64 d7/d12/d18/d4b/ld1 0 2024-02-16T19:29:13.596 INFO:tasks.workunit.client.0.smithi022.stdout:9/646: link db/f7b db/d14/dc8/dc9/fdf 0 2024-02-16T19:29:13.597 INFO:tasks.workunit.client.0.smithi022.stdout:9/647: readlink db/d14/d16/d5d/l97 0 2024-02-16T19:29:13.599 INFO:tasks.workunit.client.1.smithi117.stdout:8/672: mknod d4/d12/d2a/d64/cd8 0 2024-02-16T19:29:13.602 INFO:tasks.workunit.client.0.smithi022.stdout:7/615: write d9/da/d2c/d2f/f9e [936090,110163] 0 2024-02-16T19:29:13.606 INFO:tasks.workunit.client.0.smithi022.stdout:6/665: symlink d1/d8/da/d17/d53/d7f/db3/de8/lea 0 2024-02-16T19:29:13.608 INFO:tasks.workunit.client.1.smithi117.stdout:1/590: symlink d2/d4/dbb/d7f/lc9 0 2024-02-16T19:29:13.610 INFO:tasks.workunit.client.0.smithi022.stdout:9/648: chown db/d14/f5e 7929924 1 2024-02-16T19:29:13.611 INFO:tasks.workunit.client.1.smithi117.stdout:5/713: rename d2/d36/d6f/d9c/cc0 to d2/d9/d3c/cf1 0 2024-02-16T19:29:13.615 INFO:tasks.workunit.client.1.smithi117.stdout:0/703: mknod d0/d9/d14/d23/d33/cf4 0 2024-02-16T19:29:13.616 INFO:tasks.workunit.client.0.smithi022.stdout:7/616: rename d9/da/d2c/dae to d9/da/d1f/dc6 0 2024-02-16T19:29:13.617 INFO:tasks.workunit.client.1.smithi117.stdout:6/649: symlink d5/d4b/d44/ld5 0 2024-02-16T19:29:13.618 INFO:tasks.workunit.client.1.smithi117.stdout:6/650: stat d5/d4b/d44/d93/dae/lc0 0 2024-02-16T19:29:13.619 INFO:tasks.workunit.client.0.smithi022.stdout:3/576: unlink d7/f61 0 2024-02-16T19:29:13.621 INFO:tasks.workunit.client.0.smithi022.stdout:6/666: mkdir d1/d8/da/d29/deb 0 2024-02-16T19:29:13.621 INFO:tasks.workunit.client.1.smithi117.stdout:4/647: write d0/da/d21/d3d/d28/d64/d8a/d92/f85 [1816213,49660] 0 2024-02-16T19:29:13.621 INFO:tasks.workunit.client.1.smithi117.stdout:8/673: mknod d4/d94/cd9 0 2024-02-16T19:29:13.624 INFO:tasks.workunit.client.1.smithi117.stdout:4/648: read d0/da/d21/d3d/d28/d64/d8a/d92/f85 [1068937,88994] 0 2024-02-16T19:29:13.624 INFO:tasks.workunit.client.0.smithi022.stdout:8/614: creat d3/d12/d2b/fde x:0 0 0 2024-02-16T19:29:13.628 INFO:tasks.workunit.client.1.smithi117.stdout:7/670: write d4/d1c/d39/d9f/fd0 [580321,80679] 0 2024-02-16T19:29:13.628 INFO:tasks.workunit.client.0.smithi022.stdout:1/679: creat d6/d8/d22/d16/d17/fd4 x:0 0 0 2024-02-16T19:29:13.629 INFO:tasks.workunit.client.0.smithi022.stdout:1/680: chown d6/d8/d22/d16/d17/cd0 9 1 2024-02-16T19:29:13.630 INFO:tasks.workunit.client.1.smithi117.stdout:1/591: creat d2/d19/d59/d76/d8c/fca x:0 0 0 2024-02-16T19:29:13.633 INFO:tasks.workunit.client.0.smithi022.stdout:9/649: dread db/f12 [0,4194304] 0 2024-02-16T19:29:13.634 INFO:tasks.workunit.client.1.smithi117.stdout:3/777: rename d1/d30/f77 to d1/d83/d96/f110 0 2024-02-16T19:29:13.634 INFO:tasks.workunit.client.0.smithi022.stdout:1/681: dread d6/d8/d22/d59/da7/fd1 [0,4194304] 0 2024-02-16T19:29:13.634 INFO:tasks.workunit.client.0.smithi022.stdout:3/577: creat d7/d21/fd2 x:0 0 0 2024-02-16T19:29:13.639 INFO:tasks.workunit.client.0.smithi022.stdout:9/650: creat db/d14/d16/d5d/d65/d6b/fe0 x:0 0 0 2024-02-16T19:29:13.640 INFO:tasks.workunit.client.1.smithi117.stdout:9/590: creat dd/d23/fce x:0 0 0 2024-02-16T19:29:13.641 INFO:tasks.workunit.client.1.smithi117.stdout:9/591: write dd/d23/fa8 [281209,37191] 0 2024-02-16T19:29:13.645 INFO:tasks.workunit.client.0.smithi022.stdout:1/682: rename d6/d8/d22/d3f/f73 to d6/d8/d22/d47/d51/fd5 0 2024-02-16T19:29:13.646 INFO:tasks.workunit.client.1.smithi117.stdout:8/674: mkdir d4/d7/d6a/d72/d84/dda 0 2024-02-16T19:29:13.650 INFO:tasks.workunit.client.0.smithi022.stdout:9/651: mknod db/d14/d19/ce1 0 2024-02-16T19:29:13.650 INFO:tasks.workunit.client.0.smithi022.stdout:9/652: stat db/d14/d16/d28/f9a 0 2024-02-16T19:29:13.652 INFO:tasks.workunit.client.1.smithi117.stdout:8/675: dwrite d4/d7/d6a/d6f/da8/d76/fc5 [0,4194304] 0 2024-02-16T19:29:13.654 INFO:tasks.workunit.client.1.smithi117.stdout:7/671: symlink d4/d1c/d87/d89/d60/lf7 0 2024-02-16T19:29:13.657 INFO:tasks.workunit.client.0.smithi022.stdout:1/683: unlink d6/d8/l35 0 2024-02-16T19:29:13.658 INFO:tasks.workunit.client.0.smithi022.stdout:1/684: readlink d6/d8/d22/d16/d17/d26/l58 0 2024-02-16T19:29:13.659 INFO:tasks.workunit.client.0.smithi022.stdout:8/615: getdents d3/dd3 0 2024-02-16T19:29:13.661 INFO:tasks.workunit.client.1.smithi117.stdout:3/778: mknod d1/dd/df8/c111 0 2024-02-16T19:29:13.661 INFO:tasks.workunit.client.0.smithi022.stdout:3/578: write d7/d12/d18/dd/d24/d58/d70/f8e [955506,105799] 0 2024-02-16T19:29:13.662 INFO:tasks.workunit.client.1.smithi117.stdout:0/704: dwrite d0/d9/d14/d23/d33/fd8 [0,4194304] 0 2024-02-16T19:29:13.664 INFO:tasks.workunit.client.0.smithi022.stdout:9/653: dread db/d18/f74 [0,4194304] 0 2024-02-16T19:29:13.665 INFO:tasks.workunit.client.0.smithi022.stdout:1/685: creat d6/d8/d22/d67/fd6 x:0 0 0 2024-02-16T19:29:13.669 INFO:tasks.workunit.client.1.smithi117.stdout:6/651: truncate d5/db/d7d/d32/d37/f41 3473072 0 2024-02-16T19:29:13.673 INFO:tasks.workunit.client.0.smithi022.stdout:9/654: chown db/d14/d16/d49/d90/lc2 105 1 2024-02-16T19:29:13.676 INFO:tasks.workunit.client.0.smithi022.stdout:1/686: read d6/f5b [132484,11918] 0 2024-02-16T19:29:13.678 INFO:tasks.workunit.client.0.smithi022.stdout:3/579: mknod d7/d12/d59/cd3 0 2024-02-16T19:29:13.679 INFO:tasks.workunit.client.0.smithi022.stdout:3/580: dread d7/d12/d18/dd/d3c/d5a/d67/fa8 [0,4194304] 0 2024-02-16T19:29:13.680 INFO:tasks.workunit.client.0.smithi022.stdout:9/655: creat db/d14/d16/d28/fe2 x:0 0 0 2024-02-16T19:29:13.680 INFO:tasks.workunit.client.1.smithi117.stdout:2/668: link d0/d3b/lc2 d0/d3/d34/ld1 0 2024-02-16T19:29:13.681 INFO:tasks.workunit.client.0.smithi022.stdout:9/656: fdatasync db/d14/d16/d27/d2e/f38 0 2024-02-16T19:29:13.681 INFO:tasks.workunit.client.1.smithi117.stdout:2/669: read d0/d3/d22/f7b [826959,95961] 0 2024-02-16T19:29:13.681 INFO:tasks.workunit.client.1.smithi117.stdout:8/676: creat d4/d73/fdb x:0 0 0 2024-02-16T19:29:13.683 INFO:tasks.workunit.client.1.smithi117.stdout:8/677: chown d4/d7/d6a/d6f/da8/d76/d8c 3 1 2024-02-16T19:29:13.684 INFO:tasks.workunit.client.1.smithi117.stdout:7/672: mkdir d4/d1c/d39/df8 0 2024-02-16T19:29:13.684 INFO:tasks.workunit.client.1.smithi117.stdout:7/673: chown d4 32 1 2024-02-16T19:29:13.686 INFO:tasks.workunit.client.1.smithi117.stdout:1/592: mkdir d2/d19/d59/d76/d8c/dc7/dcb 0 2024-02-16T19:29:13.687 INFO:tasks.workunit.client.1.smithi117.stdout:8/678: dread d4/d7/d6a/d6f/da8/d76/fc5 [0,4194304] 0 2024-02-16T19:29:13.691 INFO:tasks.workunit.client.1.smithi117.stdout:1/593: dread d2/d19/d59/d89/d7b/f7e [0,4194304] 0 2024-02-16T19:29:13.691 INFO:tasks.workunit.client.1.smithi117.stdout:5/714: creat d2/d9/d3c/d24/d50/d96/dbc/d30/ff2 x:0 0 0 2024-02-16T19:29:13.692 INFO:tasks.workunit.client.1.smithi117.stdout:1/594: write d2/d4/f17 [5852411,107788] 0 2024-02-16T19:29:13.694 INFO:tasks.workunit.client.0.smithi022.stdout:9/657: symlink db/d14/d16/d49/d90/le3 0 2024-02-16T19:29:13.698 INFO:tasks.workunit.client.0.smithi022.stdout:1/687: dwrite d6/d8/d22/d47/d54/f8a [0,4194304] 0 2024-02-16T19:29:13.698 INFO:tasks.workunit.client.1.smithi117.stdout:9/592: mkdir dd/db5/dc4/dcf 0 2024-02-16T19:29:13.702 INFO:tasks.workunit.client.0.smithi022.stdout:3/581: symlink d7/d12/d18/dd/d98/ld4 0 2024-02-16T19:29:13.707 INFO:tasks.workunit.client.1.smithi117.stdout:2/670: creat d0/d3/d34/d94/d46/d68/d76/fd2 x:0 0 0 2024-02-16T19:29:13.708 INFO:tasks.workunit.client.1.smithi117.stdout:2/671: truncate d0/d3b/f5b 4758002 0 2024-02-16T19:29:13.717 INFO:tasks.workunit.client.0.smithi022.stdout:3/582: mknod d7/d12/d18/dd/d24/d29/cd5 0 2024-02-16T19:29:13.717 INFO:tasks.workunit.client.0.smithi022.stdout:3/583: read - d7/d12/d59/f71 zero size 2024-02-16T19:29:13.719 INFO:tasks.workunit.client.1.smithi117.stdout:5/715: mkdir d2/d9/d3c/d24/d50/d96/dc4/dca/df3 0 2024-02-16T19:29:13.720 INFO:tasks.workunit.client.0.smithi022.stdout:9/658: link db/d14/d16/d27/d2e/l8c db/d14/dc8/le4 0 2024-02-16T19:29:13.721 INFO:tasks.workunit.client.1.smithi117.stdout:8/679: dwrite d4/f52 [0,4194304] 0 2024-02-16T19:29:13.722 INFO:tasks.workunit.client.1.smithi117.stdout:3/779: truncate d1/d31/f8c 2313240 0 2024-02-16T19:29:13.723 INFO:tasks.workunit.client.0.smithi022.stdout:1/688: unlink d6/d8/d22/d16/d17/d26/f43 0 2024-02-16T19:29:13.725 INFO:tasks.workunit.client.0.smithi022.stdout:3/584: symlink d7/d12/d18/dd/d31/dbf/ld6 0 2024-02-16T19:29:13.727 INFO:tasks.workunit.client.1.smithi117.stdout:9/593: rmdir dd/d23/d64 39 2024-02-16T19:29:13.728 INFO:tasks.workunit.client.0.smithi022.stdout:9/659: mkdir db/d14/d19/d1a/de5 0 2024-02-16T19:29:13.731 INFO:tasks.workunit.client.1.smithi117.stdout:4/649: getdents d0/da/d21/d3d/d28/d64/d73/d91/d9e 0 2024-02-16T19:29:13.732 INFO:tasks.workunit.client.1.smithi117.stdout:1/595: dwrite d2/d1d/d60/d68/f81 [0,4194304] 0 2024-02-16T19:29:13.732 INFO:tasks.workunit.client.0.smithi022.stdout:1/689: symlink d6/d8/d22/d16/d17/d63/ld7 0 2024-02-16T19:29:13.732 INFO:tasks.workunit.client.1.smithi117.stdout:4/650: dread - d0/da/d21/fc8 zero size 2024-02-16T19:29:13.733 INFO:tasks.workunit.client.0.smithi022.stdout:3/585: fdatasync d7/d12/d18/dac/faa 0 2024-02-16T19:29:13.734 INFO:tasks.workunit.client.0.smithi022.stdout:9/660: symlink db/d14/d33/le6 0 2024-02-16T19:29:13.734 INFO:tasks.workunit.client.1.smithi117.stdout:9/594: dread dd/d23/f30 [0,4194304] 0 2024-02-16T19:29:13.736 INFO:tasks.workunit.client.1.smithi117.stdout:7/674: readlink d4/d1c/d39/d46/lac 0 2024-02-16T19:29:13.738 INFO:tasks.workunit.client.0.smithi022.stdout:1/690: truncate d6/d8/d22/d3a/d80/f86 795812 0 2024-02-16T19:29:13.738 INFO:tasks.workunit.client.1.smithi117.stdout:7/675: chown d4/d1c/d39/d46/d47/l74 16665 1 2024-02-16T19:29:13.740 INFO:tasks.workunit.client.1.smithi117.stdout:8/680: symlink d4/d94/ldc 0 2024-02-16T19:29:13.740 INFO:tasks.workunit.client.1.smithi117.stdout:6/652: dwrite d5/db/d7d/d32/f73 [0,4194304] 0 2024-02-16T19:29:13.740 INFO:tasks.workunit.client.1.smithi117.stdout:3/780: mknod d1/c112 0 2024-02-16T19:29:13.742 INFO:tasks.workunit.client.1.smithi117.stdout:6/653: readlink d5/d4b/d44/d52/l61 0 2024-02-16T19:29:13.744 INFO:tasks.workunit.client.1.smithi117.stdout:7/676: truncate d4/d1c/d39/d46/d55/db8/fe3 47354 0 2024-02-16T19:29:13.745 INFO:tasks.workunit.client.1.smithi117.stdout:7/677: chown d4/d13/f35 13 1 2024-02-16T19:29:13.748 INFO:tasks.workunit.client.1.smithi117.stdout:8/681: dwrite d4/d7/fca [0,4194304] 0 2024-02-16T19:29:13.765 INFO:tasks.workunit.client.0.smithi022.stdout:5/701: sync 2024-02-16T19:29:13.767 INFO:tasks.workunit.client.0.smithi022.stdout:5/702: chown dd/d1f/d48/f60 51617 1 2024-02-16T19:29:13.770 INFO:tasks.workunit.client.0.smithi022.stdout:5/703: fsync dd/d1f/d48/d21/d2f/d46/d5e/d84/fd1 0 2024-02-16T19:29:13.773 INFO:tasks.workunit.client.0.smithi022.stdout:5/704: write dd/d1f/d48/d21/d2f/d40/d49/d4b/f66 [3374591,57189] 0 2024-02-16T19:29:13.774 INFO:tasks.workunit.client.0.smithi022.stdout:5/705: chown dd/d1f/d48/d21/d2f/d52/d77/d95 221320 1 2024-02-16T19:29:13.779 INFO:tasks.workunit.client.0.smithi022.stdout:5/706: dread dd/d1f/d48/d21/d2f/d52/f65 [0,4194304] 0 2024-02-16T19:29:13.781 INFO:tasks.workunit.client.0.smithi022.stdout:1/691: symlink d6/d8/dba/dc7/ld8 0 2024-02-16T19:29:13.800 INFO:tasks.workunit.client.1.smithi117.stdout:1/596: unlink d2/d19/f2f 0 2024-02-16T19:29:13.800 INFO:tasks.workunit.client.1.smithi117.stdout:7/678: dread d4/d1c/fbe [0,4194304] 0 2024-02-16T19:29:13.801 INFO:tasks.workunit.client.1.smithi117.stdout:4/651: creat d0/da/d21/d3d/d28/d64/d73/d91/d37/fd1 x:0 0 0 2024-02-16T19:29:13.801 INFO:tasks.workunit.client.1.smithi117.stdout:3/781: unlink d1/d1b/d1e/l7f 0 2024-02-16T19:29:13.801 INFO:tasks.workunit.client.1.smithi117.stdout:5/716: write d2/d3/f12 [3532186,19289] 0 2024-02-16T19:29:13.803 INFO:tasks.workunit.client.1.smithi117.stdout:2/672: truncate d0/d3/d34/f7a 1286560 0 2024-02-16T19:29:13.815 INFO:tasks.workunit.client.1.smithi117.stdout:6/654: mkdir d5/d4b/dd6 0 2024-02-16T19:29:13.815 INFO:tasks.workunit.client.1.smithi117.stdout:5/717: dread d2/d5d/faf [0,4194304] 0 2024-02-16T19:29:13.815 INFO:tasks.workunit.client.1.smithi117.stdout:4/652: dread - d0/da/d21/d3d/d28/f86 zero size 2024-02-16T19:29:13.817 INFO:tasks.workunit.client.1.smithi117.stdout:2/673: unlink d0/d3/d40/fc1 0 2024-02-16T19:29:13.818 INFO:tasks.workunit.client.1.smithi117.stdout:2/674: chown d0/d3/cb 1 1 2024-02-16T19:29:13.818 INFO:tasks.workunit.client.1.smithi117.stdout:2/675: chown d0/d3/f31 3939845 1 2024-02-16T19:29:13.818 INFO:tasks.workunit.client.1.smithi117.stdout:9/595: rmdir dd/d47/d5e/d9a/db2 0 2024-02-16T19:29:13.819 INFO:tasks.workunit.client.1.smithi117.stdout:1/597: mkdir d2/d19/d59/d89/d69/dad/dc6/dcc 0 2024-02-16T19:29:13.819 INFO:tasks.workunit.client.1.smithi117.stdout:2/676: chown d0/d13/d4d/d53/d58/dbc 53 1 2024-02-16T19:29:13.819 INFO:tasks.workunit.client.1.smithi117.stdout:1/598: readlink d2/d1d/l2e 0 2024-02-16T19:29:13.820 INFO:tasks.workunit.client.1.smithi117.stdout:2/677: chown d0/d3/d10 3018389 1 2024-02-16T19:29:13.821 INFO:tasks.workunit.client.1.smithi117.stdout:4/653: truncate d0/da/d21/f40 435551 0 2024-02-16T19:29:13.822 INFO:tasks.workunit.client.1.smithi117.stdout:9/596: mkdir dd/d47/d48/dd0 0 2024-02-16T19:29:13.823 INFO:tasks.workunit.client.1.smithi117.stdout:4/654: chown d0/da/d21/d3d/d28/d64 14702534 1 2024-02-16T19:29:13.823 INFO:tasks.workunit.client.1.smithi117.stdout:6/655: fsync d5/f7 0 2024-02-16T19:29:13.825 INFO:tasks.workunit.client.1.smithi117.stdout:1/599: rename d2/d1d/d1f/f5e to d2/d19/d59/d89/d49/fcd 0 2024-02-16T19:29:13.826 INFO:tasks.workunit.client.1.smithi117.stdout:2/678: creat d0/d3/d34/d94/d46/d68/d76/da1/d56/fd3 x:0 0 0 2024-02-16T19:29:13.826 INFO:tasks.workunit.client.1.smithi117.stdout:3/782: getdents d1/dd/d5c/d85 0 2024-02-16T19:29:13.828 INFO:tasks.workunit.client.1.smithi117.stdout:9/597: fsync dd/d47/d48/f50 0 2024-02-16T19:29:13.829 INFO:tasks.workunit.client.1.smithi117.stdout:4/655: readlink d0/da/d21/d3d/l7a 0 2024-02-16T19:29:13.829 INFO:tasks.workunit.client.1.smithi117.stdout:6/656: mkdir d5/d4b/d44/d52/d81/dd7 0 2024-02-16T19:29:13.830 INFO:tasks.workunit.client.1.smithi117.stdout:5/718: dread d2/d4c/d4f/d67/f7f [0,4194304] 0 2024-02-16T19:29:13.830 INFO:tasks.workunit.client.1.smithi117.stdout:1/600: fsync d2/d1d/d1f/f4e 0 2024-02-16T19:29:13.832 INFO:tasks.workunit.client.1.smithi117.stdout:3/783: dread d1/dd/d3e/d7d/fbf [0,4194304] 0 2024-02-16T19:29:13.832 INFO:tasks.workunit.client.1.smithi117.stdout:2/679: unlink d0/d3/d22/f3c 0 2024-02-16T19:29:13.833 INFO:tasks.workunit.client.1.smithi117.stdout:5/719: read d2/d4c/f71 [49334,31853] 0 2024-02-16T19:29:13.836 INFO:tasks.workunit.client.1.smithi117.stdout:5/720: fsync d2/db3/f55 0 2024-02-16T19:29:13.837 INFO:tasks.workunit.client.0.smithi022.stdout:1/692: creat d6/d8/d22/d16/d17/d63/d6f/d8b/fd9 x:0 0 0 2024-02-16T19:29:13.839 INFO:tasks.workunit.client.1.smithi117.stdout:6/657: dwrite d5/db/d7d/d32/f92 [4194304,4194304] 0 2024-02-16T19:29:13.845 INFO:tasks.workunit.client.0.smithi022.stdout:3/586: dwrite d7/d21/fa6 [0,4194304] 0 2024-02-16T19:29:13.847 INFO:tasks.workunit.client.1.smithi117.stdout:4/656: dread d0/f25 [4194304,4194304] 0 2024-02-16T19:29:13.848 INFO:tasks.workunit.client.0.smithi022.stdout:9/661: dwrite db/d14/f5e [0,4194304] 0 2024-02-16T19:29:13.849 INFO:tasks.workunit.client.0.smithi022.stdout:1/693: rename d6/d8/d6c/l8f to d6/d8/d6c/lda 0 2024-02-16T19:29:13.849 INFO:tasks.workunit.client.1.smithi117.stdout:0/705: sync 2024-02-16T19:29:13.850 INFO:tasks.workunit.client.0.smithi022.stdout:5/707: dread dd/f17 [4194304,4194304] 0 2024-02-16T19:29:13.855 INFO:tasks.workunit.client.1.smithi117.stdout:1/601: rename d2/d4/f47 to d2/d19/d59/d89/d69/dad/dc6/dcc/fce 0 2024-02-16T19:29:13.855 INFO:tasks.workunit.client.1.smithi117.stdout:4/657: dwrite d0/da/d21/d3d/d28/d64/d8a/fb1 [0,4194304] 0 2024-02-16T19:29:13.857 INFO:tasks.workunit.client.0.smithi022.stdout:3/587: rmdir d7/d12/d18/dd/d24/d58/d70 39 2024-02-16T19:29:13.857 INFO:tasks.workunit.client.1.smithi117.stdout:1/602: write d2/f57 [2277796,29937] 0 2024-02-16T19:29:13.858 INFO:tasks.workunit.client.0.smithi022.stdout:3/588: read d7/d12/d18/dd/d3c/f65 [1631686,123119] 0 2024-02-16T19:29:13.860 INFO:tasks.workunit.client.1.smithi117.stdout:5/721: symlink d2/d9/d3c/d24/d50/d96/dbc/d30/d5a/d62/lf4 0 2024-02-16T19:29:13.866 INFO:tasks.workunit.client.0.smithi022.stdout:3/589: mkdir d7/d12/d18/dac/dd7 0 2024-02-16T19:29:13.866 INFO:tasks.workunit.client.1.smithi117.stdout:9/598: dread dd/d23/d64/f5a [0,4194304] 0 2024-02-16T19:29:13.868 INFO:tasks.workunit.client.1.smithi117.stdout:9/599: dread dd/d45/d73/f82 [0,4194304] 0 2024-02-16T19:29:13.872 INFO:tasks.workunit.client.0.smithi022.stdout:3/590: read - d7/d12/d18/f4e zero size 2024-02-16T19:29:13.876 INFO:tasks.workunit.client.0.smithi022.stdout:9/662: getdents db/d18 0 2024-02-16T19:29:13.878 INFO:tasks.workunit.client.0.smithi022.stdout:9/663: chown db/d14/d19/da8/f81 183622182 1 2024-02-16T19:29:13.881 INFO:tasks.workunit.client.0.smithi022.stdout:1/694: dread d6/d8/d22/f3d [0,4194304] 0 2024-02-16T19:29:13.881 INFO:tasks.workunit.client.0.smithi022.stdout:5/708: getdents dd/d1f/d48/d21/d2f/d40/d49 0 2024-02-16T19:29:13.882 INFO:tasks.workunit.client.0.smithi022.stdout:3/591: read d7/d12/d18/dd/d3c/f92 [633389,73123] 0 2024-02-16T19:29:13.883 INFO:tasks.workunit.client.0.smithi022.stdout:5/709: chown dd/d1f/d48/d21/d2f/d46/d5e/d84/db4/fb9 100234441 1 2024-02-16T19:29:13.884 INFO:tasks.workunit.client.0.smithi022.stdout:9/664: creat db/d14/d16/d49/fe7 x:0 0 0 2024-02-16T19:29:13.886 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:13 smithi117 ceph-mon[35309]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:29:13.886 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:13 smithi117 ceph-mon[35309]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' 2024-02-16T19:29:13.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:13 smithi117 ceph-mon[35309]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' 2024-02-16T19:29:13.908 INFO:tasks.workunit.client.0.smithi022.stdout:2/622: sync 2024-02-16T19:29:13.908 INFO:tasks.workunit.client.0.smithi022.stdout:4/573: sync 2024-02-16T19:29:13.910 INFO:tasks.workunit.client.0.smithi022.stdout:2/623: read d0/d2/f14 [327997,67773] 0 2024-02-16T19:29:13.911 INFO:tasks.workunit.client.0.smithi022.stdout:2/624: dread - d0/d2/d18/d19/d90/fa0 zero size 2024-02-16T19:29:13.947 INFO:tasks.workunit.client.1.smithi117.stdout:7/679: sync 2024-02-16T19:29:13.947 INFO:tasks.workunit.client.1.smithi117.stdout:8/682: sync 2024-02-16T19:29:13.964 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:13 smithi022 ceph-mon[29221]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:29:13.964 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:13 smithi022 ceph-mon[29221]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' 2024-02-16T19:29:13.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:13 smithi022 ceph-mon[29221]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' 2024-02-16T19:29:13.982 INFO:tasks.workunit.client.0.smithi022.stdout:7/617: sync 2024-02-16T19:29:13.982 INFO:tasks.workunit.client.0.smithi022.stdout:0/827: sync 2024-02-16T19:29:13.989 INFO:tasks.workunit.client.0.smithi022.stdout:5/710: dread - dd/d1f/d76/fd6 zero size 2024-02-16T19:29:13.997 INFO:tasks.workunit.client.1.smithi117.stdout:5/722: symlink d2/d3/d21/d49/d59/daa/lf5 0 2024-02-16T19:29:13.998 INFO:tasks.workunit.client.0.smithi022.stdout:2/625: truncate d0/f78 363826 0 2024-02-16T19:29:14.002 INFO:tasks.workunit.client.1.smithi117.stdout:3/784: getdents d1/dd/d3e 0 2024-02-16T19:29:14.002 INFO:tasks.workunit.client.0.smithi022.stdout:7/618: mkdir d9/d5d/dc7 0 2024-02-16T19:29:14.003 INFO:tasks.workunit.client.0.smithi022.stdout:7/619: readlink d9/da/d1f/d40/l79 0 2024-02-16T19:29:14.007 INFO:tasks.workunit.client.1.smithi117.stdout:2/680: getdents d0/d3/d34/d94/d82 0 2024-02-16T19:29:14.011 INFO:tasks.workunit.client.0.smithi022.stdout:9/665: creat db/d14/dc8/dc9/fe8 x:0 0 0 2024-02-16T19:29:14.017 INFO:tasks.workunit.client.1.smithi117.stdout:4/658: write d0/da/d21/f27 [3438633,27960] 0 2024-02-16T19:29:14.017 INFO:tasks.workunit.client.1.smithi117.stdout:6/658: getdents d5/d4b/d71/dc7/dce 0 2024-02-16T19:29:14.019 INFO:tasks.workunit.client.1.smithi117.stdout:0/706: dwrite d0/d9/d14/d1d/d1e/d54/f72 [0,4194304] 0 2024-02-16T19:29:14.021 INFO:tasks.workunit.client.1.smithi117.stdout:6/659: fdatasync d5/db/d7d/d32/faf 0 2024-02-16T19:29:14.025 INFO:tasks.workunit.client.0.smithi022.stdout:8/616: sync 2024-02-16T19:29:14.025 INFO:tasks.workunit.client.0.smithi022.stdout:6/667: sync 2024-02-16T19:29:14.025 INFO:tasks.workunit.client.1.smithi117.stdout:3/785: dread - d1/dd/d5c/fc1 zero size 2024-02-16T19:29:14.025 INFO:tasks.workunit.client.1.smithi117.stdout:1/603: dwrite d2/d1d/f6d [0,4194304] 0 2024-02-16T19:29:14.026 INFO:tasks.workunit.client.1.smithi117.stdout:3/786: write d1/d30/f10c [1029743,26080] 0 2024-02-16T19:29:14.030 INFO:tasks.workunit.client.0.smithi022.stdout:6/668: dread d1/d8/da/d17/f99 [0,4194304] 0 2024-02-16T19:29:14.030 INFO:tasks.workunit.client.0.smithi022.stdout:7/620: unlink l0 0 2024-02-16T19:29:14.042 INFO:tasks.workunit.client.1.smithi117.stdout:5/723: creat d2/d3/d21/d49/d59/daa/de2/ff6 x:0 0 0 2024-02-16T19:29:14.048 INFO:tasks.workunit.client.1.smithi117.stdout:0/707: fdatasync d0/d9/d2c/fc1 0 2024-02-16T19:29:14.049 INFO:tasks.workunit.client.0.smithi022.stdout:4/574: dwrite d4/d6/d14/d64/d4e/f8b [0,4194304] 0 2024-02-16T19:29:14.050 INFO:tasks.workunit.client.0.smithi022.stdout:1/695: getdents d6/d8/d22/d67 0 2024-02-16T19:29:14.055 INFO:tasks.workunit.client.0.smithi022.stdout:8/617: readlink d3/d25/d33/ld2 0 2024-02-16T19:29:14.062 INFO:tasks.workunit.client.0.smithi022.stdout:6/669: symlink d1/d8/d81/lec 0 2024-02-16T19:29:14.062 INFO:tasks.workunit.client.0.smithi022.stdout:6/670: chown d1/d8/dac/cc8 30269 1 2024-02-16T19:29:14.064 INFO:tasks.workunit.client.0.smithi022.stdout:7/621: creat d9/d5d/fc8 x:0 0 0 2024-02-16T19:29:14.065 INFO:tasks.workunit.client.0.smithi022.stdout:7/622: rename d9/da/d1f/d67/db3 to d9/da/d1f/d67/db3/dc9 22 2024-02-16T19:29:14.072 INFO:tasks.workunit.client.1.smithi117.stdout:1/604: mkdir d2/d19/d59/d76/d91/dcf 0 2024-02-16T19:29:14.079 INFO:tasks.workunit.client.0.smithi022.stdout:7/623: mkdir d9/da/d2c/d46/d73/dca 0 2024-02-16T19:29:14.082 INFO:tasks.workunit.client.1.smithi117.stdout:4/659: creat d0/da/d21/d3d/d28/d66/d97/fd2 x:0 0 0 2024-02-16T19:29:14.084 INFO:tasks.workunit.client.1.smithi117.stdout:1/605: creat d2/d19/d59/d89/d69/dad/dc6/fd0 x:0 0 0 2024-02-16T19:29:14.086 INFO:tasks.workunit.client.1.smithi117.stdout:5/724: truncate d2/db3/f55 2633632 0 2024-02-16T19:29:14.088 INFO:tasks.workunit.client.1.smithi117.stdout:4/660: write d0/da/d21/d3d/d28/fa6 [2342711,65690] 0 2024-02-16T19:29:14.089 INFO:tasks.workunit.client.1.smithi117.stdout:8/683: creat d4/d7/d6a/d6f/da8/fdd x:0 0 0 2024-02-16T19:29:14.090 INFO:tasks.workunit.client.1.smithi117.stdout:1/606: creat d2/d19/d59/d89/d69/d9a/fd1 x:0 0 0 2024-02-16T19:29:14.091 INFO:tasks.workunit.client.1.smithi117.stdout:4/661: readlink d0/da/d21/dc2/lc3 0 2024-02-16T19:29:14.092 INFO:tasks.workunit.client.1.smithi117.stdout:8/684: mknod d4/d7/da6/d3a/d65/d96/cde 0 2024-02-16T19:29:14.093 INFO:tasks.workunit.client.1.smithi117.stdout:1/607: mkdir d2/d19/d59/d76/d8c/dc7/dd2 0 2024-02-16T19:29:14.094 INFO:tasks.workunit.client.1.smithi117.stdout:1/608: readlink d2/d19/d59/l5a 0 2024-02-16T19:29:14.109 INFO:tasks.workunit.client.1.smithi117.stdout:1/609: creat d2/d1d/d60/fd3 x:0 0 0 2024-02-16T19:29:14.110 INFO:tasks.workunit.client.1.smithi117.stdout:1/610: chown d2/d19/d59/d76/d91 479800987 1 2024-02-16T19:29:14.111 INFO:tasks.workunit.client.1.smithi117.stdout:1/611: write d2/d1d/d77/fa0 [5152577,58449] 0 2024-02-16T19:29:14.113 INFO:tasks.workunit.client.1.smithi117.stdout:1/612: write d2/d19/d59/d89/d69/d80/d8e/f90 [719719,96349] 0 2024-02-16T19:29:14.127 INFO:tasks.workunit.client.1.smithi117.stdout:7/680: mknod d4/d13/cf9 0 2024-02-16T19:29:14.129 INFO:tasks.workunit.client.1.smithi117.stdout:7/681: symlink d4/d1c/d87/lfa 0 2024-02-16T19:29:14.132 INFO:tasks.workunit.client.1.smithi117.stdout:7/682: mkdir d4/d13/dfb 0 2024-02-16T19:29:14.134 INFO:tasks.workunit.client.1.smithi117.stdout:7/683: mknod d4/d13/d63/ddd/cfc 0 2024-02-16T19:29:14.135 INFO:tasks.workunit.client.1.smithi117.stdout:7/684: chown d4/d1c/d39/f91 267063112 1 2024-02-16T19:29:14.137 INFO:tasks.workunit.client.1.smithi117.stdout:7/685: read d4/d1c/d39/d46/f62 [2825289,90588] 0 2024-02-16T19:29:14.139 INFO:tasks.workunit.client.1.smithi117.stdout:7/686: mkdir d4/d13/dfd 0 2024-02-16T19:29:14.148 INFO:tasks.workunit.client.1.smithi117.stdout:6/660: sync 2024-02-16T19:29:14.149 INFO:tasks.workunit.client.1.smithi117.stdout:9/600: dwrite dd/d45/f4f [0,4194304] 0 2024-02-16T19:29:14.151 INFO:tasks.workunit.client.1.smithi117.stdout:6/661: symlink d5/d4b/d44/d93/dae/ld8 0 2024-02-16T19:29:14.154 INFO:tasks.workunit.client.1.smithi117.stdout:6/662: read d5/db/f6f [1052337,47986] 0 2024-02-16T19:29:14.167 INFO:tasks.workunit.client.1.smithi117.stdout:6/663: dread d5/d4b/f46 [0,4194304] 0 2024-02-16T19:29:14.170 INFO:tasks.workunit.client.1.smithi117.stdout:8/685: rmdir d4/d7/d6a 39 2024-02-16T19:29:14.172 INFO:tasks.workunit.client.1.smithi117.stdout:7/687: unlink d4/d1c/d39/f91 0 2024-02-16T19:29:14.173 INFO:tasks.workunit.client.1.smithi117.stdout:8/686: creat d4/d12/d2a/d64/fdf x:0 0 0 2024-02-16T19:29:14.173 INFO:tasks.workunit.client.1.smithi117.stdout:7/688: readlink d4/d13/d63/l78 0 2024-02-16T19:29:14.173 INFO:tasks.workunit.client.1.smithi117.stdout:2/681: dread d0/f5e [0,4194304] 0 2024-02-16T19:29:14.178 INFO:tasks.workunit.client.1.smithi117.stdout:3/787: write d1/d1b/d1e/d29/daf/ffb [5111230,21354] 0 2024-02-16T19:29:14.179 INFO:tasks.workunit.client.0.smithi022.stdout:9/666: creat db/d14/d16/fe9 x:0 0 0 2024-02-16T19:29:14.179 INFO:tasks.workunit.client.0.smithi022.stdout:2/626: getdents d0/d2/d18/d38 0 2024-02-16T19:29:14.179 INFO:tasks.workunit.client.0.smithi022.stdout:7/624: stat d9/d1b/d23 0 2024-02-16T19:29:14.180 INFO:tasks.workunit.client.0.smithi022.stdout:3/592: write d7/d12/d18/dd/d3c/f49 [10481,5451] 0 2024-02-16T19:29:14.181 INFO:tasks.workunit.client.0.smithi022.stdout:5/711: write dd/d1f/d48/d21/d2f/d52/f65 [572946,28170] 0 2024-02-16T19:29:14.182 INFO:tasks.workunit.client.0.smithi022.stdout:0/828: truncate d10/d11/d27/d77/f8e 2894697 0 2024-02-16T19:29:14.183 INFO:tasks.workunit.client.0.smithi022.stdout:7/625: write d9/da/d1f/d44/d5b/d95/fbf [397727,503] 0 2024-02-16T19:29:14.186 INFO:tasks.workunit.client.0.smithi022.stdout:7/626: chown d9/d1b/d23 8169 1 2024-02-16T19:29:14.188 INFO:tasks.workunit.client.0.smithi022.stdout:4/575: rename d4/d6/d14/d64/d4e/l5d to d4/d6/d14/d64/d4e/d54/d5a/lb3 0 2024-02-16T19:29:14.189 INFO:tasks.workunit.client.0.smithi022.stdout:3/593: dread d7/d12/d18/d4b/f9e [0,4194304] 0 2024-02-16T19:29:14.189 INFO:tasks.workunit.client.1.smithi117.stdout:3/788: fdatasync d1/f28 0 2024-02-16T19:29:14.193 INFO:tasks.workunit.client.0.smithi022.stdout:9/667: dread db/d14/d33/f4f [0,4194304] 0 2024-02-16T19:29:14.194 INFO:tasks.workunit.client.1.smithi117.stdout:0/708: dwrite d0/d9/d22/d27/d31/fc4 [0,4194304] 0 2024-02-16T19:29:14.196 INFO:tasks.workunit.client.0.smithi022.stdout:5/712: dread dd/f11 [0,4194304] 0 2024-02-16T19:29:14.196 INFO:tasks.workunit.client.1.smithi117.stdout:7/689: rmdir d4/d1c/d39/d46/d47/de4 0 2024-02-16T19:29:14.198 INFO:tasks.workunit.client.0.smithi022.stdout:2/627: dread d0/d32/d3b/d6f/f5a [0,4194304] 0 2024-02-16T19:29:14.198 INFO:tasks.workunit.client.0.smithi022.stdout:6/671: write d1/d8/d81/fd9 [4038039,120521] 0 2024-02-16T19:29:14.203 INFO:tasks.workunit.client.1.smithi117.stdout:3/789: symlink d1/d30/d9f/l113 0 2024-02-16T19:29:14.203 INFO:tasks.workunit.client.1.smithi117.stdout:0/709: dwrite d0/d9/d22/db7/fe4 [0,4194304] 0 2024-02-16T19:29:14.204 INFO:tasks.workunit.client.0.smithi022.stdout:4/576: creat d4/d6/d14/d64/d4e/d54/fb4 x:0 0 0 2024-02-16T19:29:14.205 INFO:tasks.workunit.client.0.smithi022.stdout:4/577: dread - d4/d6/dd/fac zero size 2024-02-16T19:29:14.206 INFO:tasks.workunit.client.0.smithi022.stdout:4/578: chown d4/d6/dd/d22/f59 2515 1 2024-02-16T19:29:14.206 INFO:tasks.workunit.client.1.smithi117.stdout:7/690: rename d4/d13/dfb to d4/d1c/d87/df0/dfe 0 2024-02-16T19:29:14.211 INFO:tasks.workunit.client.1.smithi117.stdout:2/682: dread d0/d13/d4d/d53/f6f [0,4194304] 0 2024-02-16T19:29:14.211 INFO:tasks.workunit.client.0.smithi022.stdout:8/618: getdents d3/d12/d2b/d2e 0 2024-02-16T19:29:14.216 INFO:tasks.workunit.client.0.smithi022.stdout:9/668: rename db/d14/d16/d21/fc1 to db/d14/d16/d27/d8b/fea 0 2024-02-16T19:29:14.219 INFO:tasks.workunit.client.0.smithi022.stdout:5/713: mknod dd/d1f/d48/d21/cea 0 2024-02-16T19:29:14.220 INFO:tasks.workunit.client.0.smithi022.stdout:5/714: chown dd/d1f/d2e/l57 31 1 2024-02-16T19:29:14.224 INFO:tasks.workunit.client.1.smithi117.stdout:5/725: dwrite d2/d5d/faf [0,4194304] 0 2024-02-16T19:29:14.225 INFO:tasks.workunit.client.1.smithi117.stdout:5/726: stat d2/d3/d21/d2a/dcb/cce 0 2024-02-16T19:29:14.226 INFO:tasks.workunit.client.0.smithi022.stdout:6/672: creat d1/d8/da/d17/d53/fed x:0 0 0 2024-02-16T19:29:14.227 INFO:tasks.workunit.client.0.smithi022.stdout:6/673: chown d1/d8/da/d17/d85/d5b/fa4 291331303 1 2024-02-16T19:29:14.227 INFO:tasks.workunit.client.0.smithi022.stdout:6/674: write d1/f3 [663784,4322] 0 2024-02-16T19:29:14.229 INFO:tasks.workunit.client.0.smithi022.stdout:6/675: chown d1/d8/da/d7b/dae/db6/cd6 10991 1 2024-02-16T19:29:14.229 INFO:tasks.workunit.client.0.smithi022.stdout:8/619: creat d3/d12/d2b/dca/fdf x:0 0 0 2024-02-16T19:29:14.233 INFO:tasks.workunit.client.1.smithi117.stdout:7/691: link d4/d85/fcd d4/d1c/d87/fff 0 2024-02-16T19:29:14.236 INFO:tasks.workunit.client.0.smithi022.stdout:4/579: rename d4/d6/dd/df/d10/d24/d4b/faf to d4/d6/dd/d61/fb5 0 2024-02-16T19:29:14.239 INFO:tasks.workunit.client.0.smithi022.stdout:5/715: creat dd/d1f/d48/d21/dc9/feb x:0 0 0 2024-02-16T19:29:14.240 INFO:tasks.workunit.client.0.smithi022.stdout:5/716: dread - dd/d1f/d48/d21/d2f/fe2 zero size 2024-02-16T19:29:14.241 INFO:tasks.workunit.client.0.smithi022.stdout:0/829: truncate d10/d22/d2c/f72 2372264 0 2024-02-16T19:29:14.241 INFO:tasks.workunit.client.0.smithi022.stdout:0/830: dread - d10/d7d/f93 zero size 2024-02-16T19:29:14.243 INFO:tasks.workunit.client.1.smithi117.stdout:7/692: symlink d4/d1c/d87/d89/d60/l100 0 2024-02-16T19:29:14.245 INFO:tasks.workunit.client.1.smithi117.stdout:5/727: mknod d2/d9/d3c/d24/d90/dd4/cf7 0 2024-02-16T19:29:14.246 INFO:tasks.workunit.client.0.smithi022.stdout:6/676: symlink d1/d8/da/d17/d53/d7f/db3/lee 0 2024-02-16T19:29:14.247 INFO:tasks.workunit.client.1.smithi117.stdout:1/613: write d2/d19/d59/d89/d7b/f7e [179845,49169] 0 2024-02-16T19:29:14.250 INFO:tasks.workunit.client.1.smithi117.stdout:7/693: unlink d4/d13/d63/c6a 0 2024-02-16T19:29:14.251 INFO:tasks.workunit.client.0.smithi022.stdout:7/627: mkdir d9/da/d2c/dcb 0 2024-02-16T19:29:14.254 INFO:tasks.workunit.client.0.smithi022.stdout:5/717: creat dd/d1f/d48/d21/d2f/d52/d77/d95/fec x:0 0 0 2024-02-16T19:29:14.256 INFO:tasks.workunit.client.0.smithi022.stdout:2/628: creat d0/d2/d18/d19/d40/fcb x:0 0 0 2024-02-16T19:29:14.258 INFO:tasks.workunit.client.1.smithi117.stdout:7/694: fsync d4/d1c/d87/d89/d60/d9b/fb2 0 2024-02-16T19:29:14.259 INFO:tasks.workunit.client.0.smithi022.stdout:8/620: dread d3/d12/da6/f5e [0,4194304] 0 2024-02-16T19:29:14.266 INFO:tasks.workunit.client.1.smithi117.stdout:7/695: readlink d4/d1c/d39/l45 0 2024-02-16T19:29:14.266 INFO:tasks.workunit.client.1.smithi117.stdout:7/696: stat d4/d13/d63/d71/fcb 0 2024-02-16T19:29:14.267 INFO:tasks.workunit.client.0.smithi022.stdout:7/628: mkdir d9/da/d2c/d46/d73/dcc 0 2024-02-16T19:29:14.269 INFO:tasks.workunit.client.1.smithi117.stdout:5/728: getdents d2/d9/d3c/d24/d50/d96/dbc 0 2024-02-16T19:29:14.270 INFO:tasks.workunit.client.0.smithi022.stdout:2/629: fsync d0/d2/d2e/f72 0 2024-02-16T19:29:14.271 INFO:tasks.workunit.client.0.smithi022.stdout:2/630: write d0/d2/d2e/f51 [250659,103802] 0 2024-02-16T19:29:14.274 INFO:tasks.workunit.client.0.smithi022.stdout:8/621: mknod d3/d12/db0/ce0 0 2024-02-16T19:29:14.275 INFO:tasks.workunit.client.1.smithi117.stdout:5/729: creat d2/d3/ff8 x:0 0 0 2024-02-16T19:29:14.276 INFO:tasks.workunit.client.0.smithi022.stdout:6/677: fsync d1/d8/da/d17/d48/d51/f59 0 2024-02-16T19:29:14.277 INFO:tasks.workunit.client.1.smithi117.stdout:1/614: getdents d2/d19/d59 0 2024-02-16T19:29:14.278 INFO:tasks.workunit.client.1.smithi117.stdout:1/615: truncate d2/d4/f9 4821033 0 2024-02-16T19:29:14.279 INFO:tasks.workunit.client.1.smithi117.stdout:1/616: write d2/d1d/d60/f66 [2133925,49031] 0 2024-02-16T19:29:14.280 INFO:tasks.workunit.client.0.smithi022.stdout:9/669: getdents db/d14/d16/d5d/d65/d6b/dd5/dd9 0 2024-02-16T19:29:14.281 INFO:tasks.workunit.client.1.smithi117.stdout:7/697: dread d4/d1c/d39/d46/f96 [0,4194304] 0 2024-02-16T19:29:14.281 INFO:tasks.workunit.client.1.smithi117.stdout:6/664: sync 2024-02-16T19:29:14.281 INFO:tasks.workunit.client.1.smithi117.stdout:5/730: mkdir d2/d9/df9 0 2024-02-16T19:29:14.282 INFO:tasks.workunit.client.1.smithi117.stdout:9/601: write dd/d47/d48/f86 [3032724,107751] 0 2024-02-16T19:29:14.284 INFO:tasks.workunit.client.1.smithi117.stdout:7/698: read d4/d1c/d39/d46/f75 [1132171,78258] 0 2024-02-16T19:29:14.284 INFO:tasks.workunit.client.0.smithi022.stdout:7/629: unlink d9/d5d/f91 0 2024-02-16T19:29:14.287 INFO:tasks.workunit.client.0.smithi022.stdout:9/670: dwrite db/d14/d16/d27/d55/fc3 [0,4194304] 0 2024-02-16T19:29:14.288 INFO:tasks.workunit.client.1.smithi117.stdout:4/662: creat d0/da/d21/d3d/d28/d64/fd3 x:0 0 0 2024-02-16T19:29:14.289 INFO:tasks.workunit.client.1.smithi117.stdout:1/617: rmdir d2/d19/d59/d89 39 2024-02-16T19:29:14.290 INFO:tasks.workunit.client.0.smithi022.stdout:8/622: rmdir d3/d12/d2b/d54/d74 39 2024-02-16T19:29:14.290 INFO:tasks.workunit.client.1.smithi117.stdout:1/618: write d2/d1d/f6d [1002381,80610] 0 2024-02-16T19:29:14.295 INFO:tasks.workunit.client.1.smithi117.stdout:9/602: fdatasync dd/d23/f1f 0 2024-02-16T19:29:14.298 INFO:tasks.workunit.client.0.smithi022.stdout:7/630: fdatasync d9/da/d1f/d93/fa2 0 2024-02-16T19:29:14.299 INFO:tasks.workunit.client.1.smithi117.stdout:5/731: dread d2/d9/d3c/d24/d50/d52/f1e [0,4194304] 0 2024-02-16T19:29:14.299 INFO:tasks.workunit.client.1.smithi117.stdout:5/732: readlink d2/d9/l89 0 2024-02-16T19:29:14.300 INFO:tasks.workunit.client.1.smithi117.stdout:5/733: write d2/d4c/d4f/fd0 [316562,68163] 0 2024-02-16T19:29:14.302 INFO:tasks.workunit.client.0.smithi022.stdout:9/671: unlink db/d14/dc8/le4 0 2024-02-16T19:29:14.303 INFO:tasks.workunit.client.1.smithi117.stdout:7/699: mknod d4/d85/c101 0 2024-02-16T19:29:14.303 INFO:tasks.workunit.client.0.smithi022.stdout:9/672: write db/d14/d16/d5d/d65/f96 [2509513,85851] 0 2024-02-16T19:29:14.308 INFO:tasks.workunit.client.1.smithi117.stdout:4/663: mknod d0/da/d21/d3d/d28/d64/d73/d91/d9e/cd4 0 2024-02-16T19:29:14.309 INFO:tasks.workunit.client.1.smithi117.stdout:8/687: dwrite d4/d7/f3d [0,4194304] 0 2024-02-16T19:29:14.311 INFO:tasks.workunit.client.0.smithi022.stdout:7/631: truncate d9/da/d2c/d2f/f5c 497871 0 2024-02-16T19:29:14.312 INFO:tasks.workunit.client.0.smithi022.stdout:7/632: write d9/da/d1f/d93/fb8 [80075,5028] 0 2024-02-16T19:29:14.315 INFO:tasks.workunit.client.1.smithi117.stdout:2/683: rename d0/d3/d34/d94/d46/d68/d76/da1/c32 to d0/d3/d34/cd4 0 2024-02-16T19:29:14.316 INFO:tasks.workunit.client.1.smithi117.stdout:5/734: dread d2/d9/d3c/f8d [0,4194304] 0 2024-02-16T19:29:14.317 INFO:tasks.workunit.client.1.smithi117.stdout:9/603: mknod dd/d8d/cd1 0 2024-02-16T19:29:14.322 INFO:tasks.workunit.client.0.smithi022.stdout:8/623: getdents d3/d12/d2b/d54/d97 0 2024-02-16T19:29:14.324 INFO:tasks.workunit.client.0.smithi022.stdout:3/594: dwrite d7/d12/d18/dd/d3c/fa9 [0,4194304] 0 2024-02-16T19:29:14.324 INFO:tasks.workunit.client.1.smithi117.stdout:7/700: symlink d4/dd3/l102 0 2024-02-16T19:29:14.324 INFO:tasks.workunit.client.1.smithi117.stdout:7/701: chown d4/f11 3 1 2024-02-16T19:29:14.333 INFO:tasks.workunit.client.1.smithi117.stdout:0/710: dwrite d0/d9/d14/d1d/d60/d76/d7b/f9c [0,4194304] 0 2024-02-16T19:29:14.333 INFO:tasks.workunit.client.1.smithi117.stdout:1/619: write d2/d19/d59/d89/d69/d9a/fd1 [849728,108807] 0 2024-02-16T19:29:14.335 INFO:tasks.workunit.client.1.smithi117.stdout:6/665: rename d5/db/d7d/ld0 to d5/db/d7d/d32/d79/ld9 0 2024-02-16T19:29:14.339 INFO:tasks.workunit.client.0.smithi022.stdout:4/580: rename d4/d6/dd/d61 to d4/d6/db6 0 2024-02-16T19:29:14.339 INFO:tasks.workunit.client.1.smithi117.stdout:3/790: dwrite d1/d1b/d1e/d29/f33 [0,4194304] 0 2024-02-16T19:29:14.347 INFO:tasks.workunit.client.1.smithi117.stdout:4/664: dread d0/da/d21/d3d/d28/d64/d73/fbf [0,4194304] 0 2024-02-16T19:29:14.347 INFO:tasks.workunit.client.0.smithi022.stdout:5/718: rename dd/d1f/d48/d21/d2f/d40/d49/daa to dd/d1f/d48/d21/d2f/d52/d77/d95/ded 0 2024-02-16T19:29:14.348 INFO:tasks.workunit.client.0.smithi022.stdout:5/719: fdatasync dd/d1f/d48/d21/d5d/de7/fc6 0 2024-02-16T19:29:14.351 INFO:tasks.workunit.client.0.smithi022.stdout:4/581: dread - d4/d6/d14/d64/d4e/d54/d5a/d63/f92 zero size 2024-02-16T19:29:14.351 INFO:tasks.workunit.client.0.smithi022.stdout:5/720: write dd/d1f/d48/d21/d2f/d40/d49/d4b/fdc [737793,89135] 0 2024-02-16T19:29:14.356 INFO:tasks.workunit.client.0.smithi022.stdout:0/831: write d10/d11/d79/d8b/fa5 [1176893,65921] 0 2024-02-16T19:29:14.358 INFO:tasks.workunit.client.1.smithi117.stdout:8/688: dread d4/f6 [0,4194304] 0 2024-02-16T19:29:14.360 INFO:tasks.workunit.client.1.smithi117.stdout:1/620: dread d2/ff [0,4194304] 0 2024-02-16T19:29:14.361 INFO:tasks.workunit.client.1.smithi117.stdout:6/666: creat d5/d4b/d20/da4/fda x:0 0 0 2024-02-16T19:29:14.363 INFO:tasks.workunit.client.1.smithi117.stdout:5/735: dread d2/d9/d3c/d24/d50/d96/dbc/d30/d38/d51/f7e [0,4194304] 0 2024-02-16T19:29:14.363 INFO:tasks.workunit.client.0.smithi022.stdout:2/631: rename d0/d2/d18/d19/d40/d55/d59/cb7 to d0/d2/d18/d1f/d6c/d99/ccc 0 2024-02-16T19:29:14.367 INFO:tasks.workunit.client.0.smithi022.stdout:5/721: dread - dd/d1f/d48/d21/d2f/d52/d5c/d61/d6a/db0/fdd zero size 2024-02-16T19:29:14.367 INFO:tasks.workunit.client.1.smithi117.stdout:7/702: creat d4/d1c/d87/d89/d5f/dae/df3/f103 x:0 0 0 2024-02-16T19:29:14.368 INFO:tasks.workunit.client.0.smithi022.stdout:2/632: dwrite d0/d2/d18/f22 [0,4194304] 0 2024-02-16T19:29:14.370 INFO:tasks.workunit.client.1.smithi117.stdout:4/665: fdatasync d0/da/d21/d3d/d28/f39 0 2024-02-16T19:29:14.371 INFO:tasks.workunit.client.1.smithi117.stdout:5/736: dread d2/d3/f65 [0,4194304] 0 2024-02-16T19:29:14.371 INFO:tasks.workunit.client.0.smithi022.stdout:3/595: link d7/d12/d18/d4b/db6/lce d7/d12/d18/dd/d24/d53/ld8 0 2024-02-16T19:29:14.378 INFO:tasks.workunit.client.0.smithi022.stdout:3/596: dread d7/d12/d18/dd/d3c/d5a/fc7 [0,4194304] 0 2024-02-16T19:29:14.384 INFO:tasks.workunit.client.1.smithi117.stdout:7/703: dread d4/f5 [0,4194304] 0 2024-02-16T19:29:14.385 INFO:tasks.workunit.client.1.smithi117.stdout:1/621: symlink d2/d19/d59/d89/d7b/ld4 0 2024-02-16T19:29:14.385 INFO:tasks.workunit.client.0.smithi022.stdout:5/722: symlink dd/d1f/d48/d21/d2f/d40/d49/lee 0 2024-02-16T19:29:14.386 INFO:tasks.workunit.client.1.smithi117.stdout:1/622: write d2/d4/dbb/d7f/f85 [813038,34451] 0 2024-02-16T19:29:14.386 INFO:tasks.workunit.client.0.smithi022.stdout:2/633: creat d0/d32/d3b/fcd x:0 0 0 2024-02-16T19:29:14.392 INFO:tasks.workunit.client.0.smithi022.stdout:3/597: mkdir d7/d21/dd9 0 2024-02-16T19:29:14.396 INFO:tasks.workunit.client.1.smithi117.stdout:4/666: creat d0/da/d4a/fd5 x:0 0 0 2024-02-16T19:29:14.398 INFO:tasks.workunit.client.1.smithi117.stdout:4/667: dread d0/da/d21/d3d/d28/d64/d73/fbf [0,4194304] 0 2024-02-16T19:29:14.399 INFO:tasks.workunit.client.1.smithi117.stdout:5/737: unlink d2/d9/d3c/d24/d50/d96/dbc/d30/f68 0 2024-02-16T19:29:14.401 INFO:tasks.workunit.client.0.smithi022.stdout:8/624: write d3/d12/d17/fbe [921397,63206] 0 2024-02-16T19:29:14.405 INFO:tasks.workunit.client.1.smithi117.stdout:2/684: rename d0/d3/d40/c6b to d0/d3/cd5 0 2024-02-16T19:29:14.405 INFO:tasks.workunit.client.0.smithi022.stdout:8/625: dread d3/d12/d6a/fa4 [0,4194304] 0 2024-02-16T19:29:14.405 INFO:tasks.workunit.client.1.smithi117.stdout:9/604: write dd/d45/f49 [237935,28315] 0 2024-02-16T19:29:14.406 INFO:tasks.workunit.client.1.smithi117.stdout:2/685: write d0/d13/f15 [1597439,17485] 0 2024-02-16T19:29:14.407 INFO:tasks.workunit.client.1.smithi117.stdout:9/605: chown dd/d45/d31/d63/d8e/f96 4983283 1 2024-02-16T19:29:14.409 INFO:tasks.workunit.client.1.smithi117.stdout:8/689: unlink d4/d7/da6/d3a/d54/ld2 0 2024-02-16T19:29:14.416 INFO:tasks.workunit.client.1.smithi117.stdout:5/738: unlink d2/d9/dba/le4 0 2024-02-16T19:29:14.418 INFO:tasks.workunit.client.1.smithi117.stdout:6/667: rename d5/d9c/ca3 to d5/db/d7d/d32/da8/cdb 0 2024-02-16T19:29:14.423 INFO:tasks.workunit.client.1.smithi117.stdout:2/686: mknod d0/d3/d34/d94/d82/cd6 0 2024-02-16T19:29:14.423 INFO:tasks.workunit.client.1.smithi117.stdout:0/711: dwrite d0/d9/f12 [0,4194304] 0 2024-02-16T19:29:14.424 INFO:tasks.workunit.client.1.smithi117.stdout:1/623: dread d2/d19/d59/d89/d49/f4a [0,4194304] 0 2024-02-16T19:29:14.425 INFO:tasks.workunit.client.0.smithi022.stdout:0/832: truncate d10/d11/d27/d3f/d58/f6a 4550209 0 2024-02-16T19:29:14.426 INFO:tasks.workunit.client.1.smithi117.stdout:0/712: stat d0/d9/d22/d27/d31/d94/ca0 0 2024-02-16T19:29:14.429 INFO:tasks.workunit.client.1.smithi117.stdout:3/791: getdents d1/dd 0 2024-02-16T19:29:14.433 INFO:tasks.workunit.client.1.smithi117.stdout:7/704: rename d4/d13/f4b to d4/d1c/d87/d89/d5f/dae/f104 0 2024-02-16T19:29:14.434 INFO:tasks.workunit.client.1.smithi117.stdout:8/690: dwrite d4/d7/f9d [0,4194304] 0 2024-02-16T19:29:14.437 INFO:tasks.workunit.client.1.smithi117.stdout:6/668: dread d5/d4b/d71/dc7/d49/d59/f5e [0,4194304] 0 2024-02-16T19:29:14.439 INFO:tasks.workunit.client.1.smithi117.stdout:6/669: write d5/d4b/d71/dc7/dce/fad [728446,73625] 0 2024-02-16T19:29:14.446 INFO:tasks.workunit.client.1.smithi117.stdout:4/668: rename d0/da/d21/d3d/d28/d64/d73/fbf to d0/da/d21/d3d/d28/d64/d73/d91/d9c/fd6 0 2024-02-16T19:29:14.448 INFO:tasks.workunit.client.1.smithi117.stdout:2/687: dread d0/d13/d4d/d53/d58/faf [0,4194304] 0 2024-02-16T19:29:14.449 INFO:tasks.workunit.client.1.smithi117.stdout:7/705: creat d4/d1c/d87/d89/d5f/dae/f105 x:0 0 0 2024-02-16T19:29:14.457 INFO:tasks.workunit.client.1.smithi117.stdout:6/670: unlink d5/d9c/daa/lc4 0 2024-02-16T19:29:14.457 INFO:tasks.workunit.client.1.smithi117.stdout:6/671: chown d5/db/d7d/d32/d79/fba 126 1 2024-02-16T19:29:14.458 INFO:tasks.workunit.client.1.smithi117.stdout:6/672: chown d5/db/d7d/d32 2079439 1 2024-02-16T19:29:14.459 INFO:tasks.workunit.client.1.smithi117.stdout:6/673: dread d5/d4b/d20/fb5 [0,4194304] 0 2024-02-16T19:29:14.460 INFO:tasks.workunit.client.1.smithi117.stdout:6/674: write d5/d4b/d71/dc7/d49/d59/fd3 [1040756,60814] 0 2024-02-16T19:29:14.461 INFO:tasks.workunit.client.0.smithi022.stdout:5/723: creat dd/d1f/d48/d21/d5d/fef x:0 0 0 2024-02-16T19:29:14.462 INFO:tasks.workunit.client.0.smithi022.stdout:2/634: mkdir d0/d2/dce 0 2024-02-16T19:29:14.463 INFO:tasks.workunit.client.0.smithi022.stdout:2/635: read d0/d2/d18/d19/d90/fbd [2714414,7388] 0 2024-02-16T19:29:14.463 INFO:tasks.workunit.client.1.smithi117.stdout:9/606: rename dd/d47/f79 to dd/d45/d31/d63/d8e/dbb/fd2 0 2024-02-16T19:29:14.464 INFO:tasks.workunit.client.1.smithi117.stdout:4/669: mkdir d0/da/d21/d3d/d28/d64/d90/dc0/dd7 0 2024-02-16T19:29:14.464 INFO:tasks.workunit.client.1.smithi117.stdout:9/607: write dd/d47/fbe [441224,111032] 0 2024-02-16T19:29:14.466 INFO:tasks.workunit.client.1.smithi117.stdout:8/691: mknod d4/d12/d31/d8e/dbf/ce0 0 2024-02-16T19:29:14.466 INFO:tasks.workunit.client.1.smithi117.stdout:8/692: chown d4/d12/d2a/d64 2734 1 2024-02-16T19:29:14.467 INFO:tasks.workunit.client.1.smithi117.stdout:1/624: creat d2/d19/d59/fd5 x:0 0 0 2024-02-16T19:29:14.468 INFO:tasks.workunit.client.1.smithi117.stdout:8/693: dread - d4/d12/d2a/d64/fdf zero size 2024-02-16T19:29:14.468 INFO:tasks.workunit.client.1.smithi117.stdout:0/713: link d0/d9/d14/dbc/ld0 d0/d9/d14/dec/lf5 0 2024-02-16T19:29:14.468 INFO:tasks.workunit.client.1.smithi117.stdout:1/625: chown d2/d19/d59/d89/d69/l70 1 1 2024-02-16T19:29:14.469 INFO:tasks.workunit.client.1.smithi117.stdout:6/675: creat d5/d9c/fdc x:0 0 0 2024-02-16T19:29:14.472 INFO:tasks.workunit.client.1.smithi117.stdout:5/739: rename d2/d9/d3c/d24/d50/d96/dbc/d30/d5a/cf0 to d2/d9/d3c/d24/d50/d96/dbc/cfa 0 2024-02-16T19:29:14.474 INFO:tasks.workunit.client.0.smithi022.stdout:2/636: fsync d0/d2/d2e/d73/f7d 0 2024-02-16T19:29:14.475 INFO:tasks.workunit.client.1.smithi117.stdout:9/608: mkdir dd/db5/dc4/dd3 0 2024-02-16T19:29:14.477 INFO:tasks.workunit.client.1.smithi117.stdout:9/609: truncate dd/d45/d31/d63/dba/fcc 532664 0 2024-02-16T19:29:14.477 INFO:tasks.workunit.client.1.smithi117.stdout:4/670: dread - d0/da/d21/d3d/d28/d66/d97/d6d/f83 zero size 2024-02-16T19:29:14.478 INFO:tasks.workunit.client.1.smithi117.stdout:4/671: chown d0/f25 2356048 1 2024-02-16T19:29:14.479 INFO:tasks.workunit.client.0.smithi022.stdout:6/678: rename d1/d8/f3b to d1/d8/da/d17/d48/fef 0 2024-02-16T19:29:14.481 INFO:tasks.workunit.client.0.smithi022.stdout:2/637: dwrite d0/d32/f94 [0,4194304] 0 2024-02-16T19:29:14.487 INFO:tasks.workunit.client.1.smithi117.stdout:6/676: fdatasync d5/db/d7d/d32/f51 0 2024-02-16T19:29:14.488 INFO:tasks.workunit.client.1.smithi117.stdout:6/677: stat d5/db/d7d/d32/l7b 0 2024-02-16T19:29:14.489 INFO:tasks.workunit.client.1.smithi117.stdout:6/678: write d5/d9c/daa/d85/f8b [3542497,91582] 0 2024-02-16T19:29:14.494 INFO:tasks.workunit.client.1.smithi117.stdout:9/610: creat dd/d45/d31/d63/d8e/fd4 x:0 0 0 2024-02-16T19:29:14.494 INFO:tasks.workunit.client.0.smithi022.stdout:3/598: creat d7/d12/d18/dd/d24/d58/fda x:0 0 0 2024-02-16T19:29:14.495 INFO:tasks.workunit.client.0.smithi022.stdout:3/599: fsync d7/d12/d18/dd/d24/f32 0 2024-02-16T19:29:14.496 INFO:tasks.workunit.client.1.smithi117.stdout:4/672: symlink d0/da/d21/d3d/d28/d66/ld8 0 2024-02-16T19:29:14.498 INFO:tasks.workunit.client.1.smithi117.stdout:0/714: mkdir d0/d9/d22/df6 0 2024-02-16T19:29:14.499 INFO:tasks.workunit.client.0.smithi022.stdout:7/633: rename d9/da/d1f/d40/f92 to d9/da/d1f/fcd 0 2024-02-16T19:29:14.506 INFO:tasks.workunit.client.0.smithi022.stdout:8/626: rmdir d3/d12/d2b/d46/dd1 0 2024-02-16T19:29:14.508 INFO:tasks.workunit.client.0.smithi022.stdout:4/582: fdatasync d4/d6/dd/df/d10/f35 0 2024-02-16T19:29:14.508 INFO:tasks.workunit.client.0.smithi022.stdout:1/696: sync 2024-02-16T19:29:14.510 INFO:tasks.workunit.client.1.smithi117.stdout:5/740: rename d2/d9/d3c/d24/d50/d96/dbc/d30/d5a to d2/d9/d3c/d24/d90/dd4/dfb 0 2024-02-16T19:29:14.511 INFO:tasks.workunit.client.1.smithi117.stdout:6/679: mknod d5/d4b/d71/dc7/d72/cdd 0 2024-02-16T19:29:14.511 INFO:tasks.workunit.client.0.smithi022.stdout:5/724: link dd/d1f/l69 dd/d1f/d48/d21/d2f/d52/d5c/d61/d6a/lf0 0 2024-02-16T19:29:14.517 INFO:tasks.workunit.client.1.smithi117.stdout:1/626: symlink d2/d19/d59/d76/d8c/dc7/dd2/ld6 0 2024-02-16T19:29:14.517 INFO:tasks.workunit.client.1.smithi117.stdout:2/688: dwrite d0/d3/d34/d94/d91/f98 [0,4194304] 0 2024-02-16T19:29:14.518 INFO:tasks.workunit.client.1.smithi117.stdout:1/627: readlink d2/d19/d59/d89/l6e 0 2024-02-16T19:29:14.519 INFO:tasks.workunit.client.1.smithi117.stdout:5/741: symlink d2/d3/lfc 0 2024-02-16T19:29:14.521 INFO:tasks.workunit.client.1.smithi117.stdout:6/680: creat d5/d4b/d71/dc7/d72/fde x:0 0 0 2024-02-16T19:29:14.522 INFO:tasks.workunit.client.1.smithi117.stdout:6/681: chown d5/d4b/d44/d52/f7e 1094563 1 2024-02-16T19:29:14.525 INFO:tasks.workunit.client.0.smithi022.stdout:8/627: creat d3/d12/d2b/dca/fe1 x:0 0 0 2024-02-16T19:29:14.528 INFO:tasks.workunit.client.1.smithi117.stdout:8/694: truncate d4/d7/f9d 247660 0 2024-02-16T19:29:14.528 INFO:tasks.workunit.client.0.smithi022.stdout:1/697: dread d6/d8/d22/d16/d17/d5d/f7b [0,4194304] 0 2024-02-16T19:29:14.529 INFO:tasks.workunit.client.1.smithi117.stdout:2/689: read - d0/d13/d4d/d80/f9c zero size 2024-02-16T19:29:14.534 INFO:tasks.workunit.client.1.smithi117.stdout:8/695: dwrite d4/d12/f42 [0,4194304] 0 2024-02-16T19:29:14.535 INFO:tasks.workunit.client.0.smithi022.stdout:1/698: dwrite d6/d8/d22/f82 [0,4194304] 0 2024-02-16T19:29:14.536 INFO:tasks.workunit.client.0.smithi022.stdout:1/699: dread - d6/d8/d22/d67/f9f zero size 2024-02-16T19:29:14.538 INFO:tasks.workunit.client.0.smithi022.stdout:3/600: write d7/d12/d18/dd/d31/d7f/fa5 [1049774,91525] 0 2024-02-16T19:29:14.539 INFO:tasks.workunit.client.1.smithi117.stdout:6/682: read d5/db/d7d/d32/f48 [403698,109981] 0 2024-02-16T19:29:14.540 INFO:tasks.workunit.client.1.smithi117.stdout:5/742: dread d2/d4c/f71 [0,4194304] 0 2024-02-16T19:29:14.541 INFO:tasks.workunit.client.1.smithi117.stdout:3/792: sync 2024-02-16T19:29:14.541 INFO:tasks.workunit.client.1.smithi117.stdout:7/706: sync 2024-02-16T19:29:14.542 INFO:tasks.workunit.client.0.smithi022.stdout:9/673: sync 2024-02-16T19:29:14.545 INFO:tasks.workunit.client.0.smithi022.stdout:3/601: dread d7/d12/d18/dd/d3c/f65 [0,4194304] 0 2024-02-16T19:29:14.547 INFO:tasks.workunit.client.1.smithi117.stdout:9/611: truncate dd/d45/d31/d63/d8e/fc8 1861896 0 2024-02-16T19:29:14.549 INFO:tasks.workunit.client.0.smithi022.stdout:2/638: truncate d0/f5f 1252518 0 2024-02-16T19:29:14.556 INFO:tasks.workunit.client.1.smithi117.stdout:0/715: getdents d0/d9/d14/dec 0 2024-02-16T19:29:14.560 INFO:tasks.workunit.client.1.smithi117.stdout:6/683: creat d5/db/d7d/d32/d79/fdf x:0 0 0 2024-02-16T19:29:14.561 INFO:tasks.workunit.client.1.smithi117.stdout:6/684: truncate d5/db/d7d/d32/faf 214787 0 2024-02-16T19:29:14.562 INFO:tasks.workunit.client.1.smithi117.stdout:5/743: fdatasync d2/d9/d3c/f1a 0 2024-02-16T19:29:14.563 INFO:tasks.workunit.client.0.smithi022.stdout:1/700: dread - d6/d8/d22/d3a/d3e/fa1 zero size 2024-02-16T19:29:14.564 INFO:tasks.workunit.client.1.smithi117.stdout:5/744: dread - d2/d3/d21/d49/d59/daa/de2/ff6 zero size 2024-02-16T19:29:14.565 INFO:tasks.workunit.client.1.smithi117.stdout:4/673: sync 2024-02-16T19:29:14.566 INFO:tasks.workunit.client.0.smithi022.stdout:8/628: write d3/d12/d17/d64/d8f/f9a [234143,64908] 0 2024-02-16T19:29:14.570 INFO:tasks.workunit.client.0.smithi022.stdout:7/634: creat d9/da/fce x:0 0 0 2024-02-16T19:29:14.571 INFO:tasks.workunit.client.0.smithi022.stdout:4/583: dwrite d4/d6/dd/d22/f59 [8388608,4194304] 0 2024-02-16T19:29:14.571 INFO:tasks.workunit.client.1.smithi117.stdout:2/690: mknod d0/d13/cd7 0 2024-02-16T19:29:14.573 INFO:tasks.workunit.client.0.smithi022.stdout:4/584: read d4/d6/d14/d64/d66/d6d/f7e [3195829,15154] 0 2024-02-16T19:29:14.574 INFO:tasks.workunit.client.0.smithi022.stdout:6/679: link d1/d8/da/c10 d1/d8/d23/dc6/ddb/cf0 0 2024-02-16T19:29:14.575 INFO:tasks.workunit.client.1.smithi117.stdout:8/696: dread - d4/d7/d6a/d6f/da8/d76/d8c/fa4 zero size 2024-02-16T19:29:14.576 INFO:tasks.workunit.client.1.smithi117.stdout:0/716: truncate d0/d30/f8c 716732 0 2024-02-16T19:29:14.578 INFO:tasks.workunit.client.0.smithi022.stdout:5/725: link dd/d1f/d48/d21/d2f/d40/d49/d4b/ca2 dd/d1f/d48/d21/d2f/d40/d49/d4b/db5/cf1 0 2024-02-16T19:29:14.581 INFO:tasks.workunit.client.1.smithi117.stdout:7/707: dwrite d4/d13/f35 [4194304,4194304] 0 2024-02-16T19:29:14.583 INFO:tasks.workunit.client.1.smithi117.stdout:7/708: chown d4/d85/fcd 263074729 1 2024-02-16T19:29:14.583 INFO:tasks.workunit.client.0.smithi022.stdout:0/833: rename d10/d22/d49/f76 to d10/f10f 0 2024-02-16T19:29:14.584 INFO:tasks.workunit.client.1.smithi117.stdout:5/745: mkdir d2/d4c/d4f/dfd 0 2024-02-16T19:29:14.585 INFO:tasks.workunit.client.0.smithi022.stdout:3/602: creat d7/d12/d18/dac/dd7/fdb x:0 0 0 2024-02-16T19:29:14.587 INFO:tasks.workunit.client.1.smithi117.stdout:4/674: mkdir d0/da/d21/d3d/d28/d66/dd9 0 2024-02-16T19:29:14.588 INFO:tasks.workunit.client.0.smithi022.stdout:8/629: dread - d3/d25/f7d zero size 2024-02-16T19:29:14.590 INFO:tasks.workunit.client.1.smithi117.stdout:3/793: mkdir d1/d30/d52/d10f/d114 0 2024-02-16T19:29:14.591 INFO:tasks.workunit.client.1.smithi117.stdout:9/612: getdents dd/d47/d48/dd0 0 2024-02-16T19:29:14.593 INFO:tasks.workunit.client.1.smithi117.stdout:2/691: chown d0/c18 0 1 2024-02-16T19:29:14.594 INFO:tasks.workunit.client.1.smithi117.stdout:2/692: read d0/d13/d4d/d53/f6f [2498780,87456] 0 2024-02-16T19:29:14.595 INFO:tasks.workunit.client.1.smithi117.stdout:1/628: getdents d2/d1d/d60/d7c 0 2024-02-16T19:29:14.599 INFO:tasks.workunit.client.1.smithi117.stdout:8/697: rename d4/c5 to d4/d12/d31/d8e/dbf/ce1 0 2024-02-16T19:29:14.599 INFO:tasks.workunit.client.0.smithi022.stdout:5/726: readlink dd/d1f/d48/d21/d2f/d46/l99 0 2024-02-16T19:29:14.600 INFO:tasks.workunit.client.0.smithi022.stdout:1/701: write d6/d8/d22/d16/f2e [42600,111444] 0 2024-02-16T19:29:14.604 INFO:tasks.workunit.client.0.smithi022.stdout:0/834: creat d10/d15/d23/d8f/f110 x:0 0 0 2024-02-16T19:29:14.605 INFO:tasks.workunit.client.1.smithi117.stdout:6/685: mkdir d5/d4b/d44/d52/d81/dd7/de0 0 2024-02-16T19:29:14.606 INFO:tasks.workunit.client.0.smithi022.stdout:3/603: write d7/d12/d18/dd/fcb [1256063,53220] 0 2024-02-16T19:29:14.610 INFO:tasks.workunit.client.1.smithi117.stdout:4/675: creat d0/da/d21/d3d/d28/d66/d97/d6d/fda x:0 0 0 2024-02-16T19:29:14.611 INFO:tasks.workunit.client.1.smithi117.stdout:3/794: creat d1/dd/d8e/f115 x:0 0 0 2024-02-16T19:29:14.613 INFO:tasks.workunit.client.0.smithi022.stdout:7/635: write d9/d1b/f57 [1054933,51327] 0 2024-02-16T19:29:14.616 INFO:tasks.workunit.client.0.smithi022.stdout:6/680: fsync d1/f72 0 2024-02-16T19:29:14.619 INFO:tasks.workunit.client.1.smithi117.stdout:1/629: mknod d2/d1d/d77/cd7 0 2024-02-16T19:29:14.620 INFO:tasks.workunit.client.0.smithi022.stdout:5/727: creat dd/d1f/d48/d21/d2f/d46/d5e/ff2 x:0 0 0 2024-02-16T19:29:14.622 INFO:tasks.workunit.client.0.smithi022.stdout:0/835: dread d10/d15/d23/fdf [4194304,4194304] 0 2024-02-16T19:29:14.622 INFO:tasks.workunit.client.1.smithi117.stdout:0/717: rename d0/d9/d14/l8f to d0/d9/d14/d1d/d60/d65/lf7 0 2024-02-16T19:29:14.623 INFO:tasks.workunit.client.1.smithi117.stdout:9/613: dwrite dd/d47/d48/f50 [0,4194304] 0 2024-02-16T19:29:14.623 INFO:tasks.workunit.client.0.smithi022.stdout:0/836: chown d10/d15/d23/d8f/dc1/l63 4227868 1 2024-02-16T19:29:14.625 INFO:tasks.workunit.client.1.smithi117.stdout:5/746: creat d2/d3/d21/d2a/dcb/def/ffe x:0 0 0 2024-02-16T19:29:14.625 INFO:tasks.workunit.client.0.smithi022.stdout:3/604: mknod d7/d12/d18/dd/d24/d93/cdc 0 2024-02-16T19:29:14.628 INFO:tasks.workunit.client.1.smithi117.stdout:3/795: mkdir d1/dd/d3b/dba/d116 0 2024-02-16T19:29:14.630 INFO:tasks.workunit.client.1.smithi117.stdout:4/676: dread d0/da/d21/d3d/d28/d64/d73/d91/d9c/fd6 [0,4194304] 0 2024-02-16T19:29:14.630 INFO:tasks.workunit.client.1.smithi117.stdout:6/686: dwrite d5/d4b/f7a [0,4194304] 0 2024-02-16T19:29:14.631 INFO:tasks.workunit.client.1.smithi117.stdout:4/677: chown d0/da/f33 492813 1 2024-02-16T19:29:14.632 INFO:tasks.workunit.client.1.smithi117.stdout:2/693: creat d0/d3/d34/d94/d46/d68/d76/da1/d56/d60/dc9/fd8 x:0 0 0 2024-02-16T19:29:14.634 INFO:tasks.workunit.client.0.smithi022.stdout:7/636: creat d9/da/d1f/d67/fcf x:0 0 0 2024-02-16T19:29:14.639 INFO:tasks.workunit.client.1.smithi117.stdout:9/614: stat dd/d45/f4f 0 2024-02-16T19:29:14.643 INFO:tasks.workunit.client.0.smithi022.stdout:1/702: creat d6/d8/d22/d47/db7/fdb x:0 0 0 2024-02-16T19:29:14.644 INFO:tasks.workunit.client.0.smithi022.stdout:2/639: dread d0/f5f [0,4194304] 0 2024-02-16T19:29:14.645 INFO:tasks.workunit.client.1.smithi117.stdout:5/747: creat d2/d9/d3c/d24/d50/d96/dc4/fff x:0 0 0 2024-02-16T19:29:14.646 INFO:tasks.workunit.client.1.smithi117.stdout:3/796: dread d1/dd/f48 [0,4194304] 0 2024-02-16T19:29:14.648 INFO:tasks.workunit.client.0.smithi022.stdout:0/837: mknod d10/d11/d50/d102/c111 0 2024-02-16T19:29:14.650 INFO:tasks.workunit.client.0.smithi022.stdout:3/605: mknod d7/d12/d18/dd/d24/cdd 0 2024-02-16T19:29:14.652 INFO:tasks.workunit.client.0.smithi022.stdout:8/630: rmdir d3/d12/d2b/d54/db8/dc8 0 2024-02-16T19:29:14.655 INFO:tasks.workunit.client.0.smithi022.stdout:7/637: rename d9/da/d2c/f2d to d9/da/d2c/d46/d73/fd0 0 2024-02-16T19:29:14.656 INFO:tasks.workunit.client.0.smithi022.stdout:4/585: getdents d4/d6/dd/d22/d30 0 2024-02-16T19:29:14.658 INFO:tasks.workunit.client.0.smithi022.stdout:1/703: rmdir d6/d8/d22/d16/d17/d63/d6f/d8b 39 2024-02-16T19:29:14.660 INFO:tasks.workunit.client.0.smithi022.stdout:3/606: truncate d7/d21/f38 1408859 0 2024-02-16T19:29:14.660 INFO:tasks.workunit.client.1.smithi117.stdout:8/698: write d4/f79 [1716317,103102] 0 2024-02-16T19:29:14.660 INFO:tasks.workunit.client.1.smithi117.stdout:1/630: write d2/d19/d59/d89/f42 [1336110,1408] 0 2024-02-16T19:29:14.661 INFO:tasks.workunit.client.1.smithi117.stdout:0/718: write d0/d9/d14/d23/d33/f5c [383030,27700] 0 2024-02-16T19:29:14.663 INFO:tasks.workunit.client.0.smithi022.stdout:6/681: rename d1/f18 to d1/d8/da/d17/d85/d9c/ff1 0 2024-02-16T19:29:14.666 INFO:tasks.workunit.client.0.smithi022.stdout:2/640: mknod d0/d2/d2e/ccf 0 2024-02-16T19:29:14.667 INFO:tasks.workunit.client.1.smithi117.stdout:4/678: getdents d0/da/d21/d3d/d28/d64/d90/dc0/dd7 0 2024-02-16T19:29:14.667 INFO:tasks.workunit.client.0.smithi022.stdout:2/641: chown d0/d2/d18/l42 191716072 1 2024-02-16T19:29:14.670 INFO:tasks.workunit.client.1.smithi117.stdout:2/694: mkdir d0/d13/dc4/dd9 0 2024-02-16T19:29:14.672 INFO:tasks.workunit.client.0.smithi022.stdout:8/631: dread d3/d12/d2b/d46/f53 [0,4194304] 0 2024-02-16T19:29:14.672 INFO:tasks.workunit.client.1.smithi117.stdout:5/748: symlink d2/d9/d3c/d24/d50/d96/dbc/d30/d4e/l100 0 2024-02-16T19:29:14.673 INFO:tasks.workunit.client.1.smithi117.stdout:5/749: readlink d2/d3/d21/d49/l39 0 2024-02-16T19:29:14.675 INFO:tasks.workunit.client.1.smithi117.stdout:8/699: write d4/f53 [7804590,41778] 0 2024-02-16T19:29:14.676 INFO:tasks.workunit.client.0.smithi022.stdout:1/704: creat d6/d8/d22/d67/d7e/fdc x:0 0 0 2024-02-16T19:29:14.678 INFO:tasks.workunit.client.1.smithi117.stdout:8/700: dread d4/d1c/fad [0,4194304] 0 2024-02-16T19:29:14.678 INFO:tasks.workunit.client.0.smithi022.stdout:3/607: write d7/d12/d18/dd/f7d [3237372,110924] 0 2024-02-16T19:29:14.679 INFO:tasks.workunit.client.1.smithi117.stdout:8/701: write d4/d73/fdb [352574,40516] 0 2024-02-16T19:29:14.679 INFO:tasks.workunit.client.1.smithi117.stdout:1/631: fsync d2/d4/dbb/d7f/f84 0 2024-02-16T19:29:14.680 INFO:tasks.workunit.client.0.smithi022.stdout:4/586: write d4/d38/f3d [3714436,85583] 0 2024-02-16T19:29:14.681 INFO:tasks.workunit.client.0.smithi022.stdout:7/638: dread d9/da/d2c/d6d/f8b [0,4194304] 0 2024-02-16T19:29:14.681 INFO:tasks.workunit.client.1.smithi117.stdout:8/702: fdatasync d4/d7/da6/d3a/d43/f55 0 2024-02-16T19:29:14.681 INFO:tasks.workunit.client.0.smithi022.stdout:6/682: symlink d1/d8/d81/lf2 0 2024-02-16T19:29:14.683 INFO:tasks.workunit.client.0.smithi022.stdout:6/683: chown d1/d8/d23/c76 1 1 2024-02-16T19:29:14.683 INFO:tasks.workunit.client.1.smithi117.stdout:0/719: creat d0/d9/d14/d1d/d60/d76/ff8 x:0 0 0 2024-02-16T19:29:14.685 INFO:tasks.workunit.client.0.smithi022.stdout:7/639: truncate d9/da/d1f/d67/db3/fb4 215935 0 2024-02-16T19:29:14.685 INFO:tasks.workunit.client.0.smithi022.stdout:2/642: write d0/d2/d2e/fba [23238,85352] 0 2024-02-16T19:29:14.685 INFO:tasks.workunit.client.0.smithi022.stdout:0/838: truncate d10/d11/d27/fbb 208205 0 2024-02-16T19:29:14.685 INFO:tasks.workunit.client.0.smithi022.stdout:6/684: read d1/d8/da/d7b/dae/dbb/fde [776448,105938] 0 2024-02-16T19:29:14.686 INFO:tasks.workunit.client.0.smithi022.stdout:7/640: dread - d9/da/fce zero size 2024-02-16T19:29:14.686 INFO:tasks.workunit.client.1.smithi117.stdout:4/679: symlink d0/da/d21/d3d/d28/d64/d73/d91/d37/d98/ldb 0 2024-02-16T19:29:14.687 INFO:tasks.workunit.client.1.smithi117.stdout:8/703: dwrite d4/d7/da6/d3a/d43/d44/f62 [0,4194304] 0 2024-02-16T19:29:14.690 INFO:tasks.workunit.client.0.smithi022.stdout:1/705: mknod d6/d8/d22/d47/db7/cdd 0 2024-02-16T19:29:14.692 INFO:tasks.workunit.client.1.smithi117.stdout:3/797: link d1/d1b/d1e/d37/ddc/f10e d1/dd/d3b/dba/f117 0 2024-02-16T19:29:14.692 INFO:tasks.workunit.client.1.smithi117.stdout:4/680: dwrite d0/da/d21/d3d/d28/d64/d8a/fb1 [0,4194304] 0 2024-02-16T19:29:14.693 INFO:tasks.workunit.client.1.smithi117.stdout:3/798: chown d1/d1b/d1e/lcc 7631241 1 2024-02-16T19:29:14.694 INFO:tasks.workunit.client.1.smithi117.stdout:3/799: chown d1/cb 1017924139 1 2024-02-16T19:29:14.695 INFO:tasks.workunit.client.0.smithi022.stdout:3/608: read d7/d12/f19 [854684,26840] 0 2024-02-16T19:29:14.695 INFO:tasks.workunit.client.0.smithi022.stdout:4/587: mkdir d4/d6/dd/d22/d83/d9e/db7 0 2024-02-16T19:29:14.696 INFO:tasks.workunit.client.1.smithi117.stdout:1/632: symlink d2/d19/d59/d89/ld8 0 2024-02-16T19:29:14.697 INFO:tasks.workunit.client.0.smithi022.stdout:2/643: mkdir d0/d5d/dd0 0 2024-02-16T19:29:14.702 INFO:tasks.workunit.client.0.smithi022.stdout:7/641: mknod d9/da/d74/cd1 0 2024-02-16T19:29:14.707 INFO:tasks.workunit.client.0.smithi022.stdout:2/644: mknod d0/d32/d3b/cd1 0 2024-02-16T19:29:14.710 INFO:tasks.workunit.client.1.smithi117.stdout:3/800: creat d1/dd/d32/f118 x:0 0 0 2024-02-16T19:29:14.712 INFO:tasks.workunit.client.1.smithi117.stdout:1/633: creat d2/d19/d59/d89/d7b/fd9 x:0 0 0 2024-02-16T19:29:14.713 INFO:tasks.workunit.client.1.smithi117.stdout:0/720: mkdir d0/d9/d22/d27/df9 0 2024-02-16T19:29:14.713 INFO:tasks.workunit.client.0.smithi022.stdout:2/645: dwrite d0/d2/d18/d19/d90/fa0 [0,4194304] 0 2024-02-16T19:29:14.714 INFO:tasks.workunit.client.0.smithi022.stdout:6/685: dread d1/d8/d3c/d77/f80 [0,4194304] 0 2024-02-16T19:29:14.715 INFO:tasks.workunit.client.1.smithi117.stdout:3/801: dwrite d1/d83/d96/fb6 [0,4194304] 0 2024-02-16T19:29:14.715 INFO:tasks.workunit.client.0.smithi022.stdout:8/632: rename d3/d12/d2b/f65 to d3/fe2 0 2024-02-16T19:29:14.716 INFO:tasks.workunit.client.1.smithi117.stdout:7/709: sync 2024-02-16T19:29:14.716 INFO:tasks.workunit.client.0.smithi022.stdout:6/686: readlink d1/d8/da/d17/l69 0 2024-02-16T19:29:14.717 INFO:tasks.workunit.client.1.smithi117.stdout:2/695: link d0/d13/f15 d0/d3b/fda 0 2024-02-16T19:29:14.727 INFO:tasks.workunit.client.0.smithi022.stdout:3/609: rename d7/d12/d18/dd/d31/l6a to d7/d12/d18/dd/d31/d7f/lde 0 2024-02-16T19:29:14.730 INFO:tasks.workunit.client.0.smithi022.stdout:8/633: mknod d3/d12/d92/ce3 0 2024-02-16T19:29:14.732 INFO:tasks.workunit.client.0.smithi022.stdout:6/687: symlink d1/d8/da/d17/d53/lf3 0 2024-02-16T19:29:14.733 INFO:tasks.workunit.client.0.smithi022.stdout:7/642: link d9/da/d4e/f89 d9/da/d4e/fd2 0 2024-02-16T19:29:14.735 INFO:tasks.workunit.client.1.smithi117.stdout:7/710: rename d4/d1c/l83 to d4/d1c/d39/df8/l106 0 2024-02-16T19:29:14.736 INFO:tasks.workunit.client.1.smithi117.stdout:2/696: fdatasync d0/d3/d22/f7b 0 2024-02-16T19:29:14.738 INFO:tasks.workunit.client.1.smithi117.stdout:5/750: link d2/d9/d3c/d24/d50/d96/dbc/cfa d2/d9/d3c/d24/d50/d96/dbc/d30/d95/c101 0 2024-02-16T19:29:14.739 INFO:tasks.workunit.client.1.smithi117.stdout:3/802: read d1/d1b/d1e/d5e/f62 [5243139,28182] 0 2024-02-16T19:29:14.741 INFO:tasks.workunit.client.0.smithi022.stdout:0/839: dwrite d10/d22/ffe [0,4194304] 0 2024-02-16T19:29:14.741 INFO:tasks.workunit.client.1.smithi117.stdout:0/721: mknod d0/d9/d2c/d8b/dcd/cfa 0 2024-02-16T19:29:14.742 INFO:tasks.workunit.client.1.smithi117.stdout:8/704: dwrite d4/d7/da6/d3a/d43/d44/f4b [0,4194304] 0 2024-02-16T19:29:14.743 INFO:tasks.workunit.client.1.smithi117.stdout:8/705: write d4/d7/da6/d3a/d43/d44/f4b [1019181,75350] 0 2024-02-16T19:29:14.745 INFO:tasks.workunit.client.0.smithi022.stdout:7/643: creat d9/da/d2c/d88/fd3 x:0 0 0 2024-02-16T19:29:14.747 INFO:tasks.workunit.client.1.smithi117.stdout:7/711: creat d4/d1c/d87/f107 x:0 0 0 2024-02-16T19:29:14.747 INFO:tasks.workunit.client.0.smithi022.stdout:4/588: dwrite d4/d6/d14/d64/d66/d6d/f89 [0,4194304] 0 2024-02-16T19:29:14.749 INFO:tasks.workunit.client.0.smithi022.stdout:4/589: dread - d4/d38/d6a/fa9 zero size 2024-02-16T19:29:14.749 INFO:tasks.workunit.client.0.smithi022.stdout:2/646: creat d0/d2/d2e/fd2 x:0 0 0 2024-02-16T19:29:14.751 INFO:tasks.workunit.client.0.smithi022.stdout:4/590: write d4/d6/d14/d64/d4e/d54/fb4 [159646,63785] 0 2024-02-16T19:29:14.752 INFO:tasks.workunit.client.0.smithi022.stdout:4/591: chown d4/d6/d14/d64/d4e/d54/fb4 963926590 1 2024-02-16T19:29:14.753 INFO:tasks.workunit.client.0.smithi022.stdout:4/592: write d4/d6/d14/d64/f34 [5084124,53580] 0 2024-02-16T19:29:14.758 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:14 smithi117 ceph-mon[35309]: pgmap v10: 65 pgs: 65 active+clean; 2.4 GiB data, 7.0 GiB used, 529 GiB / 536 GiB avail; 76 MiB/s rd, 179 MiB/s wr, 388 op/s 2024-02-16T19:29:14.758 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:14 smithi117 ceph-mon[35309]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' 2024-02-16T19:29:14.758 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:14 smithi117 ceph-mon[35309]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' 2024-02-16T19:29:14.758 INFO:tasks.workunit.client.1.smithi117.stdout:0/722: creat d0/d9/d14/d1d/d60/d76/ffb x:0 0 0 2024-02-16T19:29:14.763 INFO:tasks.workunit.client.1.smithi117.stdout:8/706: mknod d4/d7/da6/d3a/d65/d7c/ce2 0 2024-02-16T19:29:14.764 INFO:tasks.workunit.client.0.smithi022.stdout:0/840: unlink d10/d11/d27/fdb 0 2024-02-16T19:29:14.766 INFO:tasks.workunit.client.1.smithi117.stdout:2/697: creat d0/d13/d4d/d53/d58/dbc/fdb x:0 0 0 2024-02-16T19:29:14.767 INFO:tasks.workunit.client.0.smithi022.stdout:7/644: rmdir d9/da/d2c/d6d 39 2024-02-16T19:29:14.767 INFO:tasks.workunit.client.1.smithi117.stdout:2/698: stat d0/d3/d34 0 2024-02-16T19:29:14.767 INFO:tasks.workunit.client.1.smithi117.stdout:1/634: dwrite d2/d19/d59/d89/f32 [0,4194304] 0 2024-02-16T19:29:14.768 INFO:tasks.workunit.client.0.smithi022.stdout:2/647: symlink d0/d2/d18/d1f/d6c/ld3 0 2024-02-16T19:29:14.769 INFO:tasks.workunit.client.1.smithi117.stdout:0/723: fsync d0/d30/f86 0 2024-02-16T19:29:14.769 INFO:tasks.workunit.client.0.smithi022.stdout:2/648: truncate d0/d2/f85 1123777 0 2024-02-16T19:29:14.774 INFO:tasks.workunit.client.0.smithi022.stdout:8/634: link d3/d12/d2b/d54/f70 d3/d12/d2b/da9/fe4 0 2024-02-16T19:29:14.776 INFO:tasks.workunit.client.1.smithi117.stdout:3/803: creat d1/d30/f119 x:0 0 0 2024-02-16T19:29:14.778 INFO:tasks.workunit.client.1.smithi117.stdout:2/699: mkdir d0/d3/d34/d94/d46/d68/d93/ddc 0 2024-02-16T19:29:14.779 INFO:tasks.workunit.client.0.smithi022.stdout:0/841: creat d10/d15/d23/d38/f112 x:0 0 0 2024-02-16T19:29:14.782 INFO:tasks.workunit.client.1.smithi117.stdout:8/707: mkdir d4/d12/d2a/d6d/de3 0 2024-02-16T19:29:14.783 INFO:tasks.workunit.client.1.smithi117.stdout:8/708: read d4/d7/fca [1251764,15896] 0 2024-02-16T19:29:14.791 INFO:tasks.workunit.client.1.smithi117.stdout:0/724: creat d0/d9/d22/ffc x:0 0 0 2024-02-16T19:29:14.792 INFO:tasks.workunit.client.1.smithi117.stdout:8/709: creat d4/d7/d6a/fe4 x:0 0 0 2024-02-16T19:29:14.792 INFO:tasks.workunit.client.1.smithi117.stdout:0/725: write d0/d9/d14/d23/d33/f5c [565339,15637] 0 2024-02-16T19:29:14.793 INFO:tasks.workunit.client.1.smithi117.stdout:8/710: chown d4/d12/d2a/d64/dc6 414824840 1 2024-02-16T19:29:14.794 INFO:tasks.workunit.client.1.smithi117.stdout:0/726: chown d0/d9/d14/d1d/d60/d76/d7b/d8d/d55/l7e 2628506 1 2024-02-16T19:29:14.796 INFO:tasks.workunit.client.0.smithi022.stdout:7/645: mkdir d9/da/d2c/d88/dd4 0 2024-02-16T19:29:14.797 INFO:tasks.workunit.client.0.smithi022.stdout:2/649: symlink d0/d2/d18/d38/ld4 0 2024-02-16T19:29:14.797 INFO:tasks.workunit.client.1.smithi117.stdout:0/727: fdatasync d0/d9/d22/d27/d83/f95 0 2024-02-16T19:29:14.798 INFO:tasks.workunit.client.0.smithi022.stdout:2/650: fdatasync d0/d5d/f7a 0 2024-02-16T19:29:14.798 INFO:tasks.workunit.client.0.smithi022.stdout:3/610: link d7/d12/d18/l10 d7/d12/d18/dd/ldf 0 2024-02-16T19:29:14.798 INFO:tasks.workunit.client.1.smithi117.stdout:0/728: fsync d0/d9/d14/d1d/d1e/d54/f72 0 2024-02-16T19:29:14.800 INFO:tasks.workunit.client.0.smithi022.stdout:6/688: getdents d1/d8/d3c/d4d 0 2024-02-16T19:29:14.800 INFO:tasks.workunit.client.0.smithi022.stdout:0/842: fdatasync d10/d11/d27/d3f/d58/d60/dbc/fbd 0 2024-02-16T19:29:14.801 INFO:tasks.workunit.client.0.smithi022.stdout:3/611: readlink d7/d12/d18/dd/d24/lc0 0 2024-02-16T19:29:14.804 INFO:tasks.workunit.client.1.smithi117.stdout:8/711: link d4/d7/d6a/fe4 d4/d7/d6a/d72/fe5 0 2024-02-16T19:29:14.804 INFO:tasks.workunit.client.1.smithi117.stdout:8/712: chown d4/d1c 274251326 1 2024-02-16T19:29:14.806 INFO:tasks.workunit.client.0.smithi022.stdout:4/593: dread d4/d6/dd/df/d10/f50 [0,4194304] 0 2024-02-16T19:29:14.807 INFO:tasks.workunit.client.1.smithi117.stdout:0/729: mkdir d0/d9/d14/d1d/d60/dc3/dfd 0 2024-02-16T19:29:14.807 INFO:tasks.workunit.client.0.smithi022.stdout:6/689: dwrite d1/d8/f40 [0,4194304] 0 2024-02-16T19:29:14.808 INFO:tasks.workunit.client.0.smithi022.stdout:2/651: creat d0/d5d/fd5 x:0 0 0 2024-02-16T19:29:14.808 INFO:tasks.workunit.client.1.smithi117.stdout:7/712: dwrite d4/d1c/d87/d89/d5f/dae/f104 [0,4194304] 0 2024-02-16T19:29:14.809 INFO:tasks.workunit.client.1.smithi117.stdout:7/713: fdatasync d4/f10 0 2024-02-16T19:29:14.823 INFO:tasks.workunit.client.1.smithi117.stdout:2/700: dwrite d0/d3/d34/d94/d46/d68/d76/da1/f73 [0,4194304] 0 2024-02-16T19:29:14.823 INFO:tasks.workunit.client.0.smithi022.stdout:4/594: creat d4/d6/d14/d64/d4e/d54/d5a/fb8 x:0 0 0 2024-02-16T19:29:14.828 INFO:tasks.workunit.client.0.smithi022.stdout:7/646: link d9/da/d2c/d88/fd3 d9/d1b/d23/fd5 0 2024-02-16T19:29:14.834 INFO:tasks.workunit.client.0.smithi022.stdout:8/635: getdents d3/d12/db0 0 2024-02-16T19:29:14.836 INFO:tasks.workunit.client.0.smithi022.stdout:4/595: truncate d4/d6/d14/f2e 4471158 0 2024-02-16T19:29:14.839 INFO:tasks.workunit.client.0.smithi022.stdout:7/647: unlink d9/l3e 0 2024-02-16T19:29:14.842 INFO:tasks.workunit.client.0.smithi022.stdout:0/843: rename d10/d15/d23/d4e/cf4 to d10/d15/d23/c113 0 2024-02-16T19:29:14.843 INFO:tasks.workunit.client.0.smithi022.stdout:8/636: mkdir d3/d12/d2b/d54/db8/de5 0 2024-02-16T19:29:14.844 INFO:tasks.workunit.client.0.smithi022.stdout:8/637: fsync d3/d12/d6a/f6e 0 2024-02-16T19:29:14.847 INFO:tasks.workunit.client.1.smithi117.stdout:8/713: mkdir d4/d7/d6a/d72/de6 0 2024-02-16T19:29:14.848 INFO:tasks.workunit.client.1.smithi117.stdout:8/714: chown d4/d7/da6/d3a/d43/c4f 3336070 1 2024-02-16T19:29:14.850 INFO:tasks.workunit.client.0.smithi022.stdout:6/690: rename d1/d8/da/d17/d53/d7f/da3/fcb to d1/ff4 0 2024-02-16T19:29:14.852 INFO:tasks.workunit.client.0.smithi022.stdout:0/844: symlink d10/d15/d23/d4e/l114 0 2024-02-16T19:29:14.855 INFO:tasks.workunit.client.1.smithi117.stdout:7/714: symlink d4/d13/da3/l108 0 2024-02-16T19:29:14.857 INFO:tasks.workunit.client.0.smithi022.stdout:4/596: mknod d4/d6/dd/cb9 0 2024-02-16T19:29:14.860 INFO:tasks.workunit.client.1.smithi117.stdout:2/701: symlink d0/d3/d34/d94/d46/ldd 0 2024-02-16T19:29:14.861 INFO:tasks.workunit.client.1.smithi117.stdout:2/702: stat d0/d3/d34/d94/d46/d68/d76/fd2 0 2024-02-16T19:29:14.861 INFO:tasks.workunit.client.1.smithi117.stdout:8/715: mknod d4/d7/da6/d3a/d65/d6c/ce7 0 2024-02-16T19:29:14.862 INFO:tasks.workunit.client.1.smithi117.stdout:0/730: symlink d0/d9/d14/d1d/d60/dc3/de9/lfe 0 2024-02-16T19:29:14.864 INFO:tasks.workunit.client.0.smithi022.stdout:3/612: getdents d7/d12/d18/dd/d24/d58/d70 0 2024-02-16T19:29:14.864 INFO:tasks.workunit.client.1.smithi117.stdout:0/731: fsync d0/d9/d14/d1d/d60/d76/d7b/d8d/f48 0 2024-02-16T19:29:14.867 INFO:tasks.workunit.client.1.smithi117.stdout:2/703: dwrite d0/d13/d4d/d80/fa6 [0,4194304] 0 2024-02-16T19:29:14.874 INFO:tasks.workunit.client.1.smithi117.stdout:0/732: symlink d0/d9/d22/d27/d31/d6d/lff 0 2024-02-16T19:29:14.878 INFO:tasks.workunit.client.0.smithi022.stdout:2/652: getdents d0/d2/d18/d38 0 2024-02-16T19:29:14.880 INFO:tasks.workunit.client.1.smithi117.stdout:2/704: unlink d0/d3/d34/d94/d46/d68/d76/da1/d2f/cb6 0 2024-02-16T19:29:14.882 INFO:tasks.workunit.client.0.smithi022.stdout:3/613: readlink d7/d12/d18/l10 0 2024-02-16T19:29:14.882 INFO:tasks.workunit.client.1.smithi117.stdout:0/733: dwrite d0/d9/d22/d27/d31/fb2 [0,4194304] 0 2024-02-16T19:29:14.883 INFO:tasks.workunit.client.1.smithi117.stdout:8/716: dread d4/d73/fa0 [0,4194304] 0 2024-02-16T19:29:14.884 INFO:tasks.workunit.client.1.smithi117.stdout:8/717: dread - d4/d12/d31/f77 zero size 2024-02-16T19:29:14.884 INFO:tasks.workunit.client.0.smithi022.stdout:6/691: rename d1/d8/da/d17/d85/lcd to d1/d8/da/d17/d85/d5b/lf5 0 2024-02-16T19:29:14.888 INFO:tasks.workunit.client.1.smithi117.stdout:2/705: dread d0/d3/d34/d94/d91/f98 [0,4194304] 0 2024-02-16T19:29:14.889 INFO:tasks.workunit.client.1.smithi117.stdout:0/734: rmdir d0/d9/d14/d1d/d1e 39 2024-02-16T19:29:14.889 INFO:tasks.workunit.client.1.smithi117.stdout:2/706: chown d0/d3/d22/d37/d9d/fad 228 1 2024-02-16T19:29:14.891 INFO:tasks.workunit.client.1.smithi117.stdout:2/707: write d0/d13/d4d/d53/fb4 [824908,111056] 0 2024-02-16T19:29:14.893 INFO:tasks.workunit.client.1.smithi117.stdout:7/715: dwrite d4/dd3/f99 [0,4194304] 0 2024-02-16T19:29:14.893 INFO:tasks.workunit.client.1.smithi117.stdout:8/718: creat d4/d7/da6/d3a/d65/d96/fe8 x:0 0 0 2024-02-16T19:29:14.893 INFO:tasks.workunit.client.0.smithi022.stdout:7/648: link d9/da/d4e/fd2 d9/da/d2c/d46/d73/fd6 0 2024-02-16T19:29:14.894 INFO:tasks.workunit.client.0.smithi022.stdout:8/638: truncate d3/d12/d17/f3d 1230214 0 2024-02-16T19:29:14.896 INFO:tasks.workunit.client.0.smithi022.stdout:7/649: chown d9/da/d2c/d46/c6c 35 1 2024-02-16T19:29:14.896 INFO:tasks.workunit.client.0.smithi022.stdout:0/845: dwrite d10/d15/d23/d38/f41 [0,4194304] 0 2024-02-16T19:29:14.898 INFO:tasks.workunit.client.0.smithi022.stdout:7/650: readlink d9/da/d4e/d83/lc2 0 2024-02-16T19:29:14.898 INFO:tasks.workunit.client.0.smithi022.stdout:7/651: readlink d9/da/d1f/d44/d5b/l63 0 2024-02-16T19:29:14.902 INFO:tasks.workunit.client.1.smithi117.stdout:7/716: dwrite d4/d1c/d39/d46/d47/fd9 [0,4194304] 0 2024-02-16T19:29:14.903 INFO:tasks.workunit.client.0.smithi022.stdout:4/597: write d4/d6/dd/df/d10/d24/f8c [4249255,93074] 0 2024-02-16T19:29:14.904 INFO:tasks.workunit.client.1.smithi117.stdout:0/735: symlink d0/d9/d22/d27/d31/d6d/l100 0 2024-02-16T19:29:14.905 INFO:tasks.workunit.client.1.smithi117.stdout:2/708: creat d0/d3/d34/d94/d46/d68/d76/da1/fde x:0 0 0 2024-02-16T19:29:14.907 INFO:tasks.workunit.client.1.smithi117.stdout:8/719: unlink d4/d7/da6/d3a/d65/d96/da3/lb7 0 2024-02-16T19:29:14.908 INFO:tasks.workunit.client.1.smithi117.stdout:8/720: chown d4/d73/c9a 165 1 2024-02-16T19:29:14.909 INFO:tasks.workunit.client.0.smithi022.stdout:2/653: dwrite d0/d32/f58 [0,4194304] 0 2024-02-16T19:29:14.911 INFO:tasks.workunit.client.0.smithi022.stdout:3/614: symlink d7/le0 0 2024-02-16T19:29:14.915 INFO:tasks.workunit.client.0.smithi022.stdout:7/652: rename d9/da/d3b/f41 to d9/da/d3b/fd7 0 2024-02-16T19:29:14.916 INFO:tasks.workunit.client.1.smithi117.stdout:8/721: rmdir d4/d12/d2a/d64 39 2024-02-16T19:29:14.918 INFO:tasks.workunit.client.1.smithi117.stdout:2/709: symlink d0/ldf 0 2024-02-16T19:29:14.918 INFO:tasks.workunit.client.0.smithi022.stdout:4/598: truncate d4/d6/f87 949098 0 2024-02-16T19:29:14.921 INFO:tasks.workunit.client.0.smithi022.stdout:8/639: creat d3/d25/ddb/fe6 x:0 0 0 2024-02-16T19:29:14.924 INFO:tasks.workunit.client.1.smithi117.stdout:2/710: dread - d0/d3/d34/f71 zero size 2024-02-16T19:29:14.928 INFO:tasks.workunit.client.1.smithi117.stdout:0/736: rename d0/d9/d22/d27/d31/de7 to d0/d9/d22/d27/d83/d101 0 2024-02-16T19:29:14.929 INFO:tasks.workunit.client.0.smithi022.stdout:8/640: dwrite d3/d12/d2b/dca/fdf [0,4194304] 0 2024-02-16T19:29:14.931 INFO:tasks.workunit.client.1.smithi117.stdout:8/722: mkdir d4/d7/d6a/d6f/de9 0 2024-02-16T19:29:14.931 INFO:tasks.workunit.client.0.smithi022.stdout:0/846: fsync d10/d22/f53 0 2024-02-16T19:29:14.933 INFO:tasks.workunit.client.1.smithi117.stdout:2/711: symlink d0/d3/d34/d94/d46/d68/d76/da1/d2f/le0 0 2024-02-16T19:29:14.934 INFO:tasks.workunit.client.1.smithi117.stdout:2/712: chown d0/d13/d4d/f50 1 1 2024-02-16T19:29:14.944 INFO:tasks.workunit.client.0.smithi022.stdout:6/692: creat d1/d8/da/d17/d85/d9c/ff6 x:0 0 0 2024-02-16T19:29:14.945 INFO:tasks.workunit.client.1.smithi117.stdout:2/713: symlink d0/d3/d34/d94/d46/d68/d76/le1 0 2024-02-16T19:29:14.946 INFO:tasks.workunit.client.1.smithi117.stdout:2/714: chown d0/d3/fc 7 1 2024-02-16T19:29:14.946 INFO:tasks.workunit.client.1.smithi117.stdout:2/715: chown d0/d3/d34/d94/d46/d68/d76/da1 1 1 2024-02-16T19:29:14.947 INFO:tasks.workunit.client.0.smithi022.stdout:4/599: rename d4/d6/d14/d64/d4e/d54/d5a/d63 to d4/d6/d14/d64/d4e/d54/dba 0 2024-02-16T19:29:14.948 INFO:tasks.workunit.client.1.smithi117.stdout:2/716: chown d0/d3b/fca 110558 1 2024-02-16T19:29:14.948 INFO:tasks.workunit.client.1.smithi117.stdout:2/717: readlink d0/ldf 0 2024-02-16T19:29:14.949 INFO:tasks.workunit.client.1.smithi117.stdout:8/723: symlink d4/d7/d6a/d72/d84/dda/lea 0 2024-02-16T19:29:14.950 INFO:tasks.workunit.client.1.smithi117.stdout:8/724: truncate d4/d7/da6/d3a/d65/d96/fe8 382268 0 2024-02-16T19:29:14.950 INFO:tasks.workunit.client.1.smithi117.stdout:0/737: rename d0/d9/d22 to d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102 0 2024-02-16T19:29:14.952 INFO:tasks.workunit.client.1.smithi117.stdout:2/718: mknod d0/d3/d22/ce2 0 2024-02-16T19:29:14.952 INFO:tasks.workunit.client.1.smithi117.stdout:2/719: stat d0/d13/d4d/d53/d58/dbc 0 2024-02-16T19:29:14.955 INFO:tasks.workunit.client.0.smithi022.stdout:2/654: dwrite d0/d2/d18/d19/d90/fbd [4194304,4194304] 0 2024-02-16T19:29:14.957 INFO:tasks.workunit.client.1.smithi117.stdout:0/738: creat d0/d9/d14/d1d/d60/d65/f103 x:0 0 0 2024-02-16T19:29:14.960 INFO:tasks.workunit.client.0.smithi022.stdout:0/847: creat d10/d15/d23/d8f/dc1/f115 x:0 0 0 2024-02-16T19:29:14.962 INFO:tasks.workunit.client.0.smithi022.stdout:3/615: dwrite d7/d12/d18/dd/d24/d58/d6b/f78 [0,4194304] 0 2024-02-16T19:29:14.962 INFO:tasks.workunit.client.1.smithi117.stdout:8/725: unlink d4/d12/d2a/d64/c8d 0 2024-02-16T19:29:14.964 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:14 smithi022 ceph-mon[29221]: pgmap v10: 65 pgs: 65 active+clean; 2.4 GiB data, 7.0 GiB used, 529 GiB / 536 GiB avail; 76 MiB/s rd, 179 MiB/s wr, 388 op/s 2024-02-16T19:29:14.964 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:14 smithi022 ceph-mon[29221]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' 2024-02-16T19:29:14.964 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:14 smithi022 ceph-mon[29221]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' 2024-02-16T19:29:14.966 INFO:tasks.workunit.client.0.smithi022.stdout:0/848: dread d10/d15/f61 [0,4194304] 0 2024-02-16T19:29:14.967 INFO:tasks.workunit.client.0.smithi022.stdout:0/849: chown d10/d15/d23/ce3 0 1 2024-02-16T19:29:14.968 INFO:tasks.workunit.client.0.smithi022.stdout:0/850: stat d10/d11/d50/d65 0 2024-02-16T19:29:14.972 INFO:tasks.workunit.client.0.smithi022.stdout:8/641: write d3/d12/d2b/d54/d97/fae [1399846,13565] 0 2024-02-16T19:29:14.972 INFO:tasks.workunit.client.1.smithi117.stdout:8/726: write d4/d7/d6a/d6f/da8/d76/fc5 [513089,116980] 0 2024-02-16T19:29:14.972 INFO:tasks.workunit.client.1.smithi117.stdout:2/720: dread d0/d3/d34/d94/d46/d68/f8f [0,4194304] 0 2024-02-16T19:29:14.975 INFO:tasks.workunit.client.1.smithi117.stdout:0/739: link d0/d9/d14/d1d/d60/d76/d7b/d8d/f48 d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/d27/d83/d101/f104 0 2024-02-16T19:29:14.976 INFO:tasks.workunit.client.1.smithi117.stdout:8/727: fsync d4/f3c 0 2024-02-16T19:29:14.977 INFO:tasks.workunit.client.0.smithi022.stdout:2/655: fsync d0/d32/d3b/d6f/fac 0 2024-02-16T19:29:14.978 INFO:tasks.workunit.client.1.smithi117.stdout:8/728: dread d4/d7/d6a/d6f/da8/f87 [0,4194304] 0 2024-02-16T19:29:14.986 INFO:tasks.workunit.client.0.smithi022.stdout:7/653: dwrite d9/da/d1f/d40/fb5 [0,4194304] 0 2024-02-16T19:29:14.991 INFO:tasks.workunit.client.0.smithi022.stdout:4/600: dread d4/d6/dd/df/f29 [0,4194304] 0 2024-02-16T19:29:14.991 INFO:tasks.workunit.client.1.smithi117.stdout:8/729: symlink d4/d1c/leb 0 2024-02-16T19:29:14.992 INFO:tasks.workunit.client.0.smithi022.stdout:3/616: dread d7/d12/d18/dd/d24/d58/f8b [0,4194304] 0 2024-02-16T19:29:14.993 INFO:tasks.workunit.client.0.smithi022.stdout:9/674: sync 2024-02-16T19:29:14.996 INFO:tasks.workunit.client.1.smithi117.stdout:0/740: write d0/fd [3179156,56207] 0 2024-02-16T19:29:14.996 INFO:tasks.workunit.client.1.smithi117.stdout:8/730: fdatasync d4/d7/da6/d3a/d65/d96/fce 0 2024-02-16T19:29:14.996 INFO:tasks.workunit.client.0.smithi022.stdout:2/656: write d0/d32/d3b/d6f/fac [1878533,58402] 0 2024-02-16T19:29:15.002 INFO:tasks.workunit.client.1.smithi117.stdout:0/741: truncate d0/d30/f86 324592 0 2024-02-16T19:29:15.003 INFO:tasks.workunit.client.0.smithi022.stdout:0/851: mknod d10/d11/d27/d3f/df0/c116 0 2024-02-16T19:29:15.004 INFO:tasks.workunit.client.1.smithi117.stdout:2/721: truncate d0/d3/d10/f16 235984 0 2024-02-16T19:29:15.005 INFO:tasks.workunit.client.1.smithi117.stdout:2/722: dread - d0/d3/d22/d37/fb7 zero size 2024-02-16T19:29:15.005 INFO:tasks.workunit.client.0.smithi022.stdout:2/657: dwrite d0/d2/d18/d19/d40/fa5 [0,4194304] 0 2024-02-16T19:29:15.006 INFO:tasks.workunit.client.0.smithi022.stdout:0/852: chown d10/d22/d49/c4c 1 1 2024-02-16T19:29:15.006 INFO:tasks.workunit.client.0.smithi022.stdout:7/654: dwrite d9/da/d3b/fd7 [0,4194304] 0 2024-02-16T19:29:15.008 INFO:tasks.workunit.client.0.smithi022.stdout:7/655: stat d9/da/d2c/d46/f6a 0 2024-02-16T19:29:15.009 INFO:tasks.workunit.client.1.smithi117.stdout:8/731: rename d4/d7/d6a/d6f/da8 to d4/d12/d2a/d64/dec 0 2024-02-16T19:29:15.013 INFO:tasks.workunit.client.1.smithi117.stdout:0/742: chown d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/d27/f4d 141209 1 2024-02-16T19:29:15.013 INFO:tasks.workunit.client.0.smithi022.stdout:6/693: link d1/d8/da/d17/l73 d1/d8/d3c/d4d/lf7 0 2024-02-16T19:29:15.013 INFO:tasks.workunit.client.1.smithi117.stdout:0/743: chown d0/d9/d14/d1d 119 1 2024-02-16T19:29:15.015 INFO:tasks.workunit.client.0.smithi022.stdout:0/853: dwrite d10/d11/d27/d3f/f8a [0,4194304] 0 2024-02-16T19:29:15.018 INFO:tasks.workunit.client.0.smithi022.stdout:9/675: read - db/f7b zero size 2024-02-16T19:29:15.019 INFO:tasks.workunit.client.0.smithi022.stdout:9/676: chown db/d14/d19 9 1 2024-02-16T19:29:15.020 INFO:tasks.workunit.client.0.smithi022.stdout:9/677: write db/d14/dc8/dc9/fe8 [1016780,48348] 0 2024-02-16T19:29:15.021 INFO:tasks.workunit.client.1.smithi117.stdout:8/732: creat d4/d12/d31/d8e/fed x:0 0 0 2024-02-16T19:29:15.022 INFO:tasks.workunit.client.0.smithi022.stdout:8/642: fsync d3/d12/da6/f2f 0 2024-02-16T19:29:15.023 INFO:tasks.workunit.client.1.smithi117.stdout:0/744: fdatasync d0/d9/d14/d23/d33/fea 0 2024-02-16T19:29:15.026 INFO:tasks.workunit.client.1.smithi117.stdout:8/733: creat d4/d7/da6/fee x:0 0 0 2024-02-16T19:29:15.027 INFO:tasks.workunit.client.1.smithi117.stdout:8/734: chown d4/d7/da6/d3a/d65/d96/c97 915100 1 2024-02-16T19:29:15.028 INFO:tasks.workunit.client.1.smithi117.stdout:8/735: dread - d4/d12/d2a/d64/dec/d76/d8c/fd7 zero size 2024-02-16T19:29:15.028 INFO:tasks.workunit.client.0.smithi022.stdout:2/658: symlink d0/d2/d18/d38/ld6 0 2024-02-16T19:29:15.029 INFO:tasks.workunit.client.1.smithi117.stdout:8/736: readlink d4/d94/lb3 0 2024-02-16T19:29:15.029 INFO:tasks.workunit.client.0.smithi022.stdout:2/659: write d0/d2/d18/d1f/f5e [531169,121982] 0 2024-02-16T19:29:15.033 INFO:tasks.workunit.client.0.smithi022.stdout:0/854: mkdir d10/d11/d50/d117 0 2024-02-16T19:29:15.034 INFO:tasks.workunit.client.1.smithi117.stdout:8/737: rename d4/d12/d2a/d64/dec/d76/faa to d4/d12/d2a/d6d/de3/fef 0 2024-02-16T19:29:15.036 INFO:tasks.workunit.client.0.smithi022.stdout:4/601: rename d4/d6/dd/df/d10/d24/d6b/ca5 to d4/d6/dd/df/d10/d24/d4b/cbb 0 2024-02-16T19:29:15.036 INFO:tasks.workunit.client.1.smithi117.stdout:2/723: dread d0/d3/f7c [0,4194304] 0 2024-02-16T19:29:15.036 INFO:tasks.workunit.client.1.smithi117.stdout:8/738: write d4/d7/da6/d3a/d65/d96/fce [679114,2009] 0 2024-02-16T19:29:15.037 INFO:tasks.workunit.client.0.smithi022.stdout:9/678: symlink db/d18/leb 0 2024-02-16T19:29:15.037 INFO:tasks.workunit.client.1.smithi117.stdout:2/724: truncate d0/d3b/fca 582339 0 2024-02-16T19:29:15.038 INFO:tasks.workunit.client.1.smithi117.stdout:2/725: chown d0/d3/d22/c24 2043011955 1 2024-02-16T19:29:15.039 INFO:tasks.workunit.client.0.smithi022.stdout:2/660: dread - d0/d2/d18/d1f/f7b zero size 2024-02-16T19:29:15.039 INFO:tasks.workunit.client.0.smithi022.stdout:8/643: dread d3/d25/f61 [0,4194304] 0 2024-02-16T19:29:15.040 INFO:tasks.workunit.client.1.smithi117.stdout:2/726: chown d0/d3/d34/d94/d46/d68/d76/da1/d56/d60/dc9 493 1 2024-02-16T19:29:15.040 INFO:tasks.workunit.client.0.smithi022.stdout:7/656: truncate d9/da/d4e/f89 6749741 0 2024-02-16T19:29:15.040 INFO:tasks.workunit.client.1.smithi117.stdout:8/739: truncate d4/d7/da6/d3a/d65/d96/fc7 3647351 0 2024-02-16T19:29:15.041 INFO:tasks.workunit.client.0.smithi022.stdout:9/679: dread db/d18/fb4 [0,4194304] 0 2024-02-16T19:29:15.041 INFO:tasks.workunit.client.0.smithi022.stdout:0/855: mknod d10/d11/d50/d102/c118 0 2024-02-16T19:29:15.043 INFO:tasks.workunit.client.0.smithi022.stdout:4/602: creat d4/d6/dd/df/d10/d24/fbc x:0 0 0 2024-02-16T19:29:15.045 INFO:tasks.workunit.client.1.smithi117.stdout:2/727: mknod d0/d3/ce3 0 2024-02-16T19:29:15.046 INFO:tasks.workunit.client.1.smithi117.stdout:6/687: sync 2024-02-16T19:29:15.046 INFO:tasks.workunit.client.1.smithi117.stdout:9/615: sync 2024-02-16T19:29:15.046 INFO:tasks.workunit.client.1.smithi117.stdout:8/740: dwrite d4/d7/da6/d3a/d54/f78 [0,4194304] 0 2024-02-16T19:29:15.047 INFO:tasks.workunit.client.0.smithi022.stdout:9/680: dwrite db/d14/f5e [0,4194304] 0 2024-02-16T19:29:15.048 INFO:tasks.workunit.client.1.smithi117.stdout:2/728: readlink d0/d3/d34/d94/d46/d68/d93/da9/lb5 0 2024-02-16T19:29:15.048 INFO:tasks.workunit.client.0.smithi022.stdout:9/681: write db/d14/d19/da8/f81 [3561346,120223] 0 2024-02-16T19:29:15.050 INFO:tasks.workunit.client.0.smithi022.stdout:9/682: chown db/d14/d16/d28/fe2 3 1 2024-02-16T19:29:15.050 INFO:tasks.workunit.client.1.smithi117.stdout:0/745: dread d0/fe [0,4194304] 0 2024-02-16T19:29:15.050 INFO:tasks.workunit.client.1.smithi117.stdout:2/729: stat d0/d13/d4d/d80/l9a 0 2024-02-16T19:29:15.050 INFO:tasks.workunit.client.0.smithi022.stdout:9/683: chown db/d14/d19/d1a/lbb 6988828 1 2024-02-16T19:29:15.051 INFO:tasks.workunit.client.1.smithi117.stdout:0/746: read - d0/d9/d2c/fc1 zero size 2024-02-16T19:29:15.051 INFO:tasks.workunit.client.1.smithi117.stdout:8/741: dread d4/d12/d2a/f48 [0,4194304] 0 2024-02-16T19:29:15.051 INFO:tasks.workunit.client.0.smithi022.stdout:9/684: chown db/d14/d16/d21/fcc 6353681 1 2024-02-16T19:29:15.054 INFO:tasks.workunit.client.0.smithi022.stdout:4/603: truncate d4/f47 1233093 0 2024-02-16T19:29:15.056 INFO:tasks.workunit.client.1.smithi117.stdout:9/616: rmdir dd/d47/d5b/d5f/d77/d7c 39 2024-02-16T19:29:15.060 INFO:tasks.workunit.client.1.smithi117.stdout:2/730: mkdir d0/d13/dc4/de4 0 2024-02-16T19:29:15.062 INFO:tasks.workunit.client.1.smithi117.stdout:0/747: dwrite d0/d9/d14/d1d/d60/d65/f103 [0,4194304] 0 2024-02-16T19:29:15.063 INFO:tasks.workunit.client.1.smithi117.stdout:2/731: chown d0/d3/d22/f7b 13952 1 2024-02-16T19:29:15.064 INFO:tasks.workunit.client.1.smithi117.stdout:8/742: dwrite d4/d12/d31/d8e/fed [0,4194304] 0 2024-02-16T19:29:15.066 INFO:tasks.workunit.client.1.smithi117.stdout:9/617: fdatasync dd/d23/d64/f61 0 2024-02-16T19:29:15.067 INFO:tasks.workunit.client.1.smithi117.stdout:2/732: dread - d0/d13/d4d/d80/f9c zero size 2024-02-16T19:29:15.072 INFO:tasks.workunit.client.0.smithi022.stdout:8/644: write d3/d12/d2b/d2e/f87 [276867,85972] 0 2024-02-16T19:29:15.073 INFO:tasks.workunit.client.1.smithi117.stdout:0/748: symlink d0/d9/d14/d1d/d60/d76/l105 0 2024-02-16T19:29:15.073 INFO:tasks.workunit.client.1.smithi117.stdout:2/733: dwrite d0/d3b/f5b [0,4194304] 0 2024-02-16T19:29:15.074 INFO:tasks.workunit.client.1.smithi117.stdout:6/688: write d5/db/f1f [8199361,52512] 0 2024-02-16T19:29:15.075 INFO:tasks.workunit.client.1.smithi117.stdout:6/689: fsync d5/d4b/d44/d52/d81/fa6 0 2024-02-16T19:29:15.081 INFO:tasks.workunit.client.0.smithi022.stdout:9/685: mkdir db/d14/d16/dec 0 2024-02-16T19:29:15.081 INFO:tasks.workunit.client.1.smithi117.stdout:8/743: dwrite d4/d7/da6/d50/fc3 [0,4194304] 0 2024-02-16T19:29:15.082 INFO:tasks.workunit.client.0.smithi022.stdout:9/686: stat db/d14/d16/c9d 0 2024-02-16T19:29:15.083 INFO:tasks.workunit.client.1.smithi117.stdout:8/744: dread - d4/d12/fd4 zero size 2024-02-16T19:29:15.086 INFO:tasks.workunit.client.0.smithi022.stdout:0/856: getdents d10/d11 0 2024-02-16T19:29:15.086 INFO:tasks.workunit.client.0.smithi022.stdout:7/657: write d9/da/d1f/d44/f49 [5098301,28210] 0 2024-02-16T19:29:15.090 INFO:tasks.workunit.client.1.smithi117.stdout:4/681: sync 2024-02-16T19:29:15.091 INFO:tasks.workunit.client.1.smithi117.stdout:2/734: creat d0/d13/dc4/de4/fe5 x:0 0 0 2024-02-16T19:29:15.092 INFO:tasks.workunit.client.0.smithi022.stdout:0/857: dread d10/d11/d27/d3f/df0/ff1 [0,4194304] 0 2024-02-16T19:29:15.093 INFO:tasks.workunit.client.1.smithi117.stdout:8/745: mkdir d4/d7/da6/d3a/d65/d96/df0 0 2024-02-16T19:29:15.093 INFO:tasks.workunit.client.1.smithi117.stdout:8/746: chown d4/l89 64278 1 2024-02-16T19:29:15.095 INFO:tasks.workunit.client.0.smithi022.stdout:9/687: unlink db/d14/d16/d5d/d65/f96 0 2024-02-16T19:29:15.100 INFO:tasks.workunit.client.0.smithi022.stdout:4/604: rename d4/d6/d14/l39 to d4/lbd 0 2024-02-16T19:29:15.101 INFO:tasks.workunit.client.0.smithi022.stdout:7/658: symlink d9/da/d1f/d67/ld8 0 2024-02-16T19:29:15.103 INFO:tasks.workunit.client.1.smithi117.stdout:2/735: mkdir d0/d3/d22/d37/d9d/de6 0 2024-02-16T19:29:15.103 INFO:tasks.workunit.client.1.smithi117.stdout:2/736: readlink d0/ldf 0 2024-02-16T19:29:15.105 INFO:tasks.workunit.client.0.smithi022.stdout:0/858: creat d10/d15/d23/d8f/dc1/deb/f119 x:0 0 0 2024-02-16T19:29:15.106 INFO:tasks.workunit.client.1.smithi117.stdout:8/747: mknod d4/d7/da6/d3a/d65/d96/cf1 0 2024-02-16T19:29:15.107 INFO:tasks.workunit.client.1.smithi117.stdout:9/618: link dd/d45/d31/c37 dd/d45/cd5 0 2024-02-16T19:29:15.107 INFO:tasks.workunit.client.1.smithi117.stdout:6/690: dread d5/d4b/d71/dc7/d72/fa2 [0,4194304] 0 2024-02-16T19:29:15.108 INFO:tasks.workunit.client.0.smithi022.stdout:8/645: write d3/d12/d17/f3d [698608,67394] 0 2024-02-16T19:29:15.111 INFO:tasks.workunit.client.1.smithi117.stdout:4/682: creat d0/da/d21/d3d/d28/d64/d90/dc0/dd7/fdc x:0 0 0 2024-02-16T19:29:15.117 INFO:tasks.workunit.client.0.smithi022.stdout:4/605: dread d4/d6/dd/df/d10/f4f [0,4194304] 0 2024-02-16T19:29:15.118 INFO:tasks.workunit.client.0.smithi022.stdout:0/859: creat d10/d15/d23/d8f/f11a x:0 0 0 2024-02-16T19:29:15.118 INFO:tasks.workunit.client.0.smithi022.stdout:4/606: chown d4/d6/dd/df/d10/d24/f88 1 1 2024-02-16T19:29:15.119 INFO:tasks.workunit.client.1.smithi117.stdout:6/691: read - d5/d4b/d71/dc7/d49/fac zero size 2024-02-16T19:29:15.119 INFO:tasks.workunit.client.1.smithi117.stdout:6/692: fsync d5/db/f55 0 2024-02-16T19:29:15.122 INFO:tasks.workunit.client.0.smithi022.stdout:4/607: dread d4/d6/dd/df/d10/f4f [0,4194304] 0 2024-02-16T19:29:15.122 INFO:tasks.workunit.client.1.smithi117.stdout:4/683: creat d0/da/d21/d3d/d28/d64/d8a/d92/fdd x:0 0 0 2024-02-16T19:29:15.125 INFO:tasks.workunit.client.1.smithi117.stdout:8/748: creat d4/d7/d6a/d72/d83/ff2 x:0 0 0 2024-02-16T19:29:15.126 INFO:tasks.workunit.client.0.smithi022.stdout:7/659: creat d9/da/d3b/dba/fd9 x:0 0 0 2024-02-16T19:29:15.127 INFO:tasks.workunit.client.0.smithi022.stdout:4/608: dwrite d4/d6/d14/d64/d66/f82 [0,4194304] 0 2024-02-16T19:29:15.131 INFO:tasks.workunit.client.0.smithi022.stdout:0/860: mknod d10/d11/d50/c11b 0 2024-02-16T19:29:15.132 INFO:tasks.workunit.client.1.smithi117.stdout:4/684: creat d0/da/d21/d3d/d28/d64/d90/dc0/dd7/fde x:0 0 0 2024-02-16T19:29:15.133 INFO:tasks.workunit.client.0.smithi022.stdout:0/861: stat d10/d11/d50/d102/c118 0 2024-02-16T19:29:15.133 INFO:tasks.workunit.client.0.smithi022.stdout:7/660: dwrite f5 [0,4194304] 0 2024-02-16T19:29:15.133 INFO:tasks.workunit.client.0.smithi022.stdout:9/688: write db/d14/d16/fbf [429439,36224] 0 2024-02-16T19:29:15.135 INFO:tasks.workunit.client.0.smithi022.stdout:0/862: chown d10/d11/d27/d3f/d58/d60/ff7 378665451 1 2024-02-16T19:29:15.136 INFO:tasks.workunit.client.0.smithi022.stdout:7/661: write d9/d1b/d23/f96 [279903,66786] 0 2024-02-16T19:29:15.136 INFO:tasks.workunit.client.1.smithi117.stdout:8/749: unlink d4/d12/d2a/f2b 0 2024-02-16T19:29:15.138 INFO:tasks.workunit.client.0.smithi022.stdout:0/863: write d10/d15/d23/d8f/dc1/f69 [1124260,20713] 0 2024-02-16T19:29:15.140 INFO:tasks.workunit.client.1.smithi117.stdout:4/685: read - d0/da/d21/d3d/d28/d64/d73/d91/f62 zero size 2024-02-16T19:29:15.143 INFO:tasks.workunit.client.1.smithi117.stdout:4/686: chown d0/da/d21/d3d/d28/d64/d8a/d92/f85 162422 1 2024-02-16T19:29:15.145 INFO:tasks.workunit.client.0.smithi022.stdout:8/646: dwrite d3/d25/d33/fab [0,4194304] 0 2024-02-16T19:29:15.145 INFO:tasks.workunit.client.1.smithi117.stdout:2/737: dwrite d0/d3/d34/d94/d46/d68/d76/da1/f2d [0,4194304] 0 2024-02-16T19:29:15.146 INFO:tasks.workunit.client.1.smithi117.stdout:2/738: truncate d0/d13/d4d/d53/fce 515135 0 2024-02-16T19:29:15.148 INFO:tasks.workunit.client.0.smithi022.stdout:9/689: creat db/d14/d16/d21/d4a/fed x:0 0 0 2024-02-16T19:29:15.149 INFO:tasks.workunit.client.1.smithi117.stdout:6/693: write d5/db/f6f [1948515,64101] 0 2024-02-16T19:29:15.150 INFO:tasks.workunit.client.1.smithi117.stdout:8/750: unlink d4/d7/da6/d3a/d54/c74 0 2024-02-16T19:29:15.155 INFO:tasks.workunit.client.1.smithi117.stdout:2/739: dwrite d0/d13/d4d/d80/f88 [0,4194304] 0 2024-02-16T19:29:15.157 INFO:tasks.workunit.client.0.smithi022.stdout:9/690: fsync db/d14/d19/da8/fab 0 2024-02-16T19:29:15.157 INFO:tasks.workunit.client.1.smithi117.stdout:2/740: readlink d0/d3b/lc2 0 2024-02-16T19:29:15.157 INFO:tasks.workunit.client.0.smithi022.stdout:9/691: chown db/d14/dc8/dc9 56039 1 2024-02-16T19:29:15.158 INFO:tasks.workunit.client.0.smithi022.stdout:7/662: unlink d9/d1b/d23/c47 0 2024-02-16T19:29:15.158 INFO:tasks.workunit.client.0.smithi022.stdout:9/692: readlink db/d14/d16/d21/l3a 0 2024-02-16T19:29:15.159 INFO:tasks.workunit.client.0.smithi022.stdout:8/647: symlink d3/d12/d17/le7 0 2024-02-16T19:29:15.163 INFO:tasks.workunit.client.0.smithi022.stdout:7/663: dread d9/d1b/d23/f96 [0,4194304] 0 2024-02-16T19:29:15.164 INFO:tasks.workunit.client.1.smithi117.stdout:2/741: dwrite d0/d3b/fca [0,4194304] 0 2024-02-16T19:29:15.166 INFO:tasks.workunit.client.1.smithi117.stdout:4/687: fsync d0/da/d21/d3d/d28/d64/d90/dc0/dd7/fde 0 2024-02-16T19:29:15.167 INFO:tasks.workunit.client.0.smithi022.stdout:9/693: dwrite db/d14/d19/f5f [0,4194304] 0 2024-02-16T19:29:15.169 INFO:tasks.workunit.client.1.smithi117.stdout:2/742: mknod d0/d3/d22/d37/d9d/ce7 0 2024-02-16T19:29:15.173 INFO:tasks.workunit.client.1.smithi117.stdout:4/688: dwrite d0/da/d4a/fa8 [0,4194304] 0 2024-02-16T19:29:15.175 INFO:tasks.workunit.client.0.smithi022.stdout:9/694: rename db/d14/d16/d27/d92 to db/d14/d16/dec/dee 0 2024-02-16T19:29:15.178 INFO:tasks.workunit.client.1.smithi117.stdout:2/743: creat d0/d3/d22/d37/d9d/fe8 x:0 0 0 2024-02-16T19:29:15.181 INFO:tasks.workunit.client.1.smithi117.stdout:4/689: rename d0/da/d21/d3d/d28/d64/d73/d91/d9e/fb9 to d0/da/d21/d3d/d28/d64/d73/fdf 0 2024-02-16T19:29:15.182 INFO:tasks.workunit.client.0.smithi022.stdout:9/695: mknod db/d14/d16/dec/cef 0 2024-02-16T19:29:15.183 INFO:tasks.workunit.client.0.smithi022.stdout:9/696: chown db/d14/d33/f4f 74841204 1 2024-02-16T19:29:15.184 INFO:tasks.workunit.client.0.smithi022.stdout:9/697: read db/d14/f5e [3250746,107496] 0 2024-02-16T19:29:15.184 INFO:tasks.workunit.client.1.smithi117.stdout:6/694: write d5/ff [627418,8508] 0 2024-02-16T19:29:15.185 INFO:tasks.workunit.client.1.smithi117.stdout:6/695: write d5/ff [305065,58062] 0 2024-02-16T19:29:15.186 INFO:tasks.workunit.client.1.smithi117.stdout:4/690: symlink d0/da/d21/d3d/d28/d66/d97/le0 0 2024-02-16T19:29:15.218 INFO:tasks.workunit.client.1.smithi117.stdout:5/751: sync 2024-02-16T19:29:15.219 INFO:tasks.workunit.client.1.smithi117.stdout:3/804: sync 2024-02-16T19:29:15.219 INFO:tasks.workunit.client.1.smithi117.stdout:1/635: sync 2024-02-16T19:29:15.220 INFO:tasks.workunit.client.1.smithi117.stdout:5/752: dread - d2/d9/d3c/d24/d50/d96/dc4/fff zero size 2024-02-16T19:29:15.221 INFO:tasks.workunit.client.0.smithi022.stdout:5/728: sync 2024-02-16T19:29:15.223 INFO:tasks.workunit.client.1.smithi117.stdout:6/696: rename d5/db/d7d/d32/d79 to d5/db/d7d/d32/da8/de1 0 2024-02-16T19:29:15.225 INFO:tasks.workunit.client.1.smithi117.stdout:4/691: creat d0/da/d21/d3d/d28/d66/da2/fe1 x:0 0 0 2024-02-16T19:29:15.227 INFO:tasks.workunit.client.1.smithi117.stdout:2/744: getdents d0/d13/d4d/d53/d58 0 2024-02-16T19:29:15.230 INFO:tasks.workunit.client.0.smithi022.stdout:8/648: write d3/d25/f61 [289868,78619] 0 2024-02-16T19:29:15.237 INFO:tasks.workunit.client.1.smithi117.stdout:4/692: mknod d0/da/d21/d3d/d28/d66/d97/ce2 0 2024-02-16T19:29:15.238 INFO:tasks.workunit.client.1.smithi117.stdout:5/753: link d2/d3/c8 d2/d4c/dde/c102 0 2024-02-16T19:29:15.240 INFO:tasks.workunit.client.0.smithi022.stdout:7/664: dwrite d9/da/d74/f80 [4194304,4194304] 0 2024-02-16T19:29:15.241 INFO:tasks.workunit.client.1.smithi117.stdout:6/697: dread d5/db/d7d/d32/f3f [0,4194304] 0 2024-02-16T19:29:15.241 INFO:tasks.workunit.client.1.smithi117.stdout:2/745: link d0/d13/dc4/de4/fe5 d0/d3/d34/d94/d46/d68/d76/da1/fe9 0 2024-02-16T19:29:15.242 INFO:tasks.workunit.client.0.smithi022.stdout:7/665: write d9/da/d1f/d40/fa5 [1568745,52486] 0 2024-02-16T19:29:15.242 INFO:tasks.workunit.client.0.smithi022.stdout:7/666: chown d9/da/d2c/d2f/c1e 1 1 2024-02-16T19:29:15.243 INFO:tasks.workunit.client.0.smithi022.stdout:7/667: fdatasync d9/da/d2c/fc1 0 2024-02-16T19:29:15.246 INFO:tasks.workunit.client.1.smithi117.stdout:3/805: rename d1/dd/d3e/d7d/ffd to d1/d31/f11a 0 2024-02-16T19:29:15.248 INFO:tasks.workunit.client.1.smithi117.stdout:4/693: mknod d0/da/d21/d3d/d28/d66/ce3 0 2024-02-16T19:29:15.249 INFO:tasks.workunit.client.1.smithi117.stdout:2/746: dread d0/d3/fc [0,4194304] 0 2024-02-16T19:29:15.251 INFO:tasks.workunit.client.1.smithi117.stdout:6/698: dread d5/ff [0,4194304] 0 2024-02-16T19:29:15.252 INFO:tasks.workunit.client.0.smithi022.stdout:9/698: dwrite db/d18/f74 [0,4194304] 0 2024-02-16T19:29:15.252 INFO:tasks.workunit.client.1.smithi117.stdout:6/699: dread - d5/d4b/d71/dc7/d49/fbc zero size 2024-02-16T19:29:15.253 INFO:tasks.workunit.client.0.smithi022.stdout:7/668: mknod d9/da9/dbc/cda 0 2024-02-16T19:29:15.253 INFO:tasks.workunit.client.1.smithi117.stdout:5/754: dread d2/d9/d3c/f8d [0,4194304] 0 2024-02-16T19:29:15.254 INFO:tasks.workunit.client.0.smithi022.stdout:7/669: chown d9/da/d2c/d46/d73/f8e 1557 1 2024-02-16T19:29:15.261 INFO:tasks.workunit.client.0.smithi022.stdout:8/649: dwrite d3/d12/d17/f7e [0,4194304] 0 2024-02-16T19:29:15.262 INFO:tasks.workunit.client.0.smithi022.stdout:7/670: rename d9/da/d2c/d88 to d9/da/d1f/d67/ddb 0 2024-02-16T19:29:15.263 INFO:tasks.workunit.client.1.smithi117.stdout:3/806: dread d1/f28 [0,4194304] 0 2024-02-16T19:29:15.265 INFO:tasks.workunit.client.1.smithi117.stdout:1/636: rename d2/d19/d59/d89/c5c to d2/d19/d59/d76/d91/cda 0 2024-02-16T19:29:15.266 INFO:tasks.workunit.client.1.smithi117.stdout:1/637: fsync d2/d19/d59/d89/f42 0 2024-02-16T19:29:15.266 INFO:tasks.workunit.client.1.smithi117.stdout:4/694: readlink d0/da/d21/d3d/d28/d64/l6e 0 2024-02-16T19:29:15.272 INFO:tasks.workunit.client.0.smithi022.stdout:7/671: mkdir d9/d5d/dc7/ddc 0 2024-02-16T19:29:15.273 INFO:tasks.workunit.client.0.smithi022.stdout:7/672: chown d9/da/d1f/d44/fc0 10 1 2024-02-16T19:29:15.273 INFO:tasks.workunit.client.1.smithi117.stdout:6/700: readlink d5/d4b/d20/d2d/l36 0 2024-02-16T19:29:15.277 INFO:tasks.workunit.client.1.smithi117.stdout:1/638: creat d2/d19/d59/d89/d69/dad/fdb x:0 0 0 2024-02-16T19:29:15.279 INFO:tasks.workunit.client.0.smithi022.stdout:7/673: dwrite d9/da/d4e/fc4 [0,4194304] 0 2024-02-16T19:29:15.283 INFO:tasks.workunit.client.0.smithi022.stdout:7/674: truncate d9/da/d1f/d40/f61 1023539 0 2024-02-16T19:29:15.284 INFO:tasks.workunit.client.0.smithi022.stdout:8/650: truncate d3/d12/d17/f7e 2445141 0 2024-02-16T19:29:15.286 INFO:tasks.workunit.client.1.smithi117.stdout:5/755: mkdir d2/d9/d3c/d24/d50/d103 0 2024-02-16T19:29:15.286 INFO:tasks.workunit.client.1.smithi117.stdout:3/807: write d1/f19 [2903103,26205] 0 2024-02-16T19:29:15.289 INFO:tasks.workunit.client.0.smithi022.stdout:8/651: mknod d3/d12/d2b/d2e/ce8 0 2024-02-16T19:29:15.290 INFO:tasks.workunit.client.0.smithi022.stdout:7/675: creat d9/da/d2c/dcb/fdd x:0 0 0 2024-02-16T19:29:15.291 INFO:tasks.workunit.client.0.smithi022.stdout:7/676: stat d9/da/d1f/d44 0 2024-02-16T19:29:15.299 INFO:tasks.workunit.client.1.smithi117.stdout:6/701: creat d5/d4b/d44/db1/fe2 x:0 0 0 2024-02-16T19:29:15.299 INFO:tasks.workunit.client.1.smithi117.stdout:3/808: unlink d1/d30/f90 0 2024-02-16T19:29:15.301 INFO:tasks.workunit.client.1.smithi117.stdout:6/702: rmdir d5/d4b/d20/d2d 39 2024-02-16T19:29:15.302 INFO:tasks.workunit.client.1.smithi117.stdout:2/747: dwrite d0/d3b/f51 [0,4194304] 0 2024-02-16T19:29:15.303 INFO:tasks.workunit.client.1.smithi117.stdout:5/756: write d2/d9/d3c/d24/d90/dd4/dfb/fa1 [273901,25425] 0 2024-02-16T19:29:15.307 INFO:tasks.workunit.client.1.smithi117.stdout:2/748: creat d0/d3/d34/d94/d46/d68/d76/da1/d56/d60/dc9/fea x:0 0 0 2024-02-16T19:29:15.308 INFO:tasks.workunit.client.1.smithi117.stdout:6/703: getdents d5/d4b/d44/d52/d81/dd7/de0 0 2024-02-16T19:29:15.312 INFO:tasks.workunit.client.0.smithi022.stdout:8/652: dwrite d3/d12/d2b/d54/f76 [0,4194304] 0 2024-02-16T19:29:15.313 INFO:tasks.workunit.client.1.smithi117.stdout:2/749: creat d0/d3/d34/d94/d46/d68/d93/ddc/feb x:0 0 0 2024-02-16T19:29:15.313 INFO:tasks.workunit.client.1.smithi117.stdout:5/757: getdents d2/d9/d3c/d24/d50/d96/dbc/d30/d38/d51 0 2024-02-16T19:29:15.314 INFO:tasks.workunit.client.0.smithi022.stdout:7/677: dread d9/da/d1f/f22 [0,4194304] 0 2024-02-16T19:29:15.315 INFO:tasks.workunit.client.1.smithi117.stdout:1/639: dwrite d2/d4/f11 [0,4194304] 0 2024-02-16T19:29:15.316 INFO:tasks.workunit.client.0.smithi022.stdout:7/678: write d9/d1b/f57 [816225,27829] 0 2024-02-16T19:29:15.317 INFO:tasks.workunit.client.1.smithi117.stdout:6/704: rename d5/d4b/d71/dc7/f87 to d5/d4b/d44/d93/dae/fe3 0 2024-02-16T19:29:15.318 INFO:tasks.workunit.client.1.smithi117.stdout:3/809: dwrite d1/d1b/d1e/d5e/f74 [0,4194304] 0 2024-02-16T19:29:15.318 INFO:tasks.workunit.client.0.smithi022.stdout:8/653: mkdir d3/d25/ddb/de9 0 2024-02-16T19:29:15.319 INFO:tasks.workunit.client.0.smithi022.stdout:7/679: stat d9/da/d1f/d40/ca1 0 2024-02-16T19:29:15.320 INFO:tasks.workunit.client.1.smithi117.stdout:1/640: read d2/d19/d59/d89/d69/d80/d8e/f90 [996695,94881] 0 2024-02-16T19:29:15.324 INFO:tasks.workunit.client.0.smithi022.stdout:7/680: mkdir d9/da/d1f/d40/dde 0 2024-02-16T19:29:15.331 INFO:tasks.workunit.client.1.smithi117.stdout:3/810: creat d1/dd/d5c/d85/f11b x:0 0 0 2024-02-16T19:29:15.332 INFO:tasks.workunit.client.1.smithi117.stdout:3/811: stat d1/d1b/d1e/d29/daf/ffb 0 2024-02-16T19:29:15.333 INFO:tasks.workunit.client.0.smithi022.stdout:7/681: rename d9/da/d74/cd1 to d9/d5d/dc7/cdf 0 2024-02-16T19:29:15.333 INFO:tasks.workunit.client.1.smithi117.stdout:7/717: sync 2024-02-16T19:29:15.333 INFO:tasks.workunit.client.1.smithi117.stdout:0/749: sync 2024-02-16T19:29:15.334 INFO:tasks.workunit.client.1.smithi117.stdout:3/812: dread - d1/d30/d52/d56/fd5 zero size 2024-02-16T19:29:15.335 INFO:tasks.workunit.client.1.smithi117.stdout:2/750: rename d0/d3/d34/cd4 to d0/d3/d40/cec 0 2024-02-16T19:29:15.337 INFO:tasks.workunit.client.1.smithi117.stdout:1/641: symlink d2/d51/ldc 0 2024-02-16T19:29:15.337 INFO:tasks.workunit.client.1.smithi117.stdout:5/758: dread d2/d5d/f5e [0,4194304] 0 2024-02-16T19:29:15.339 INFO:tasks.workunit.client.0.smithi022.stdout:7/682: unlink d9/da/d2c/d2f/d21/f48 0 2024-02-16T19:29:15.339 INFO:tasks.workunit.client.1.smithi117.stdout:5/759: dread - d2/d4c/fae zero size 2024-02-16T19:29:15.342 INFO:tasks.workunit.client.1.smithi117.stdout:7/718: dwrite d4/d1c/f3c [0,4194304] 0 2024-02-16T19:29:15.343 INFO:tasks.workunit.client.1.smithi117.stdout:7/719: stat d4/d1c/d98 0 2024-02-16T19:29:15.344 INFO:tasks.workunit.client.1.smithi117.stdout:3/813: rmdir d1/dd/d8e 39 2024-02-16T19:29:15.346 INFO:tasks.workunit.client.0.smithi022.stdout:7/683: creat d9/da/d2c/d6d/fe0 x:0 0 0 2024-02-16T19:29:15.347 INFO:tasks.workunit.client.1.smithi117.stdout:2/751: chown d0/d13/cb8 1529812 1 2024-02-16T19:29:15.347 INFO:tasks.workunit.client.1.smithi117.stdout:6/705: dwrite d5/d4b/d71/dc7/d49/f88 [0,4194304] 0 2024-02-16T19:29:15.349 INFO:tasks.workunit.client.0.smithi022.stdout:8/654: dwrite d3/f55 [0,4194304] 0 2024-02-16T19:29:15.354 INFO:tasks.workunit.client.1.smithi117.stdout:7/720: unlink d4/d13/da3/cf1 0 2024-02-16T19:29:15.359 INFO:tasks.workunit.client.0.smithi022.stdout:8/655: mknod d3/d12/d2b/dca/cea 0 2024-02-16T19:29:15.360 INFO:tasks.workunit.client.1.smithi117.stdout:8/751: sync 2024-02-16T19:29:15.360 INFO:tasks.workunit.client.1.smithi117.stdout:9/619: sync 2024-02-16T19:29:15.364 INFO:tasks.workunit.client.0.smithi022.stdout:7/684: link d9/da/d4e/fc4 d9/da/d2c/d46/d73/fe1 0 2024-02-16T19:29:15.366 INFO:tasks.workunit.client.0.smithi022.stdout:8/656: unlink d3/d12/d17/f7e 0 2024-02-16T19:29:15.367 INFO:tasks.workunit.client.0.smithi022.stdout:8/657: chown d3/d12/d2b/dca/fdf 49 1 2024-02-16T19:29:15.367 INFO:tasks.workunit.client.1.smithi117.stdout:8/752: dwrite d4/d12/d2a/d64/dec/d76/d8c/fd7 [0,4194304] 0 2024-02-16T19:29:15.367 INFO:tasks.workunit.client.1.smithi117.stdout:2/752: fsync d0/d3/d34/d94/d46/d68/d76/da1/d2f/f44 0 2024-02-16T19:29:15.368 INFO:tasks.workunit.client.1.smithi117.stdout:1/642: write d2/d19/f54 [583875,72512] 0 2024-02-16T19:29:15.368 INFO:tasks.workunit.client.0.smithi022.stdout:7/685: creat d9/da/d4e/d83/fe2 x:0 0 0 2024-02-16T19:29:15.370 INFO:tasks.workunit.client.0.smithi022.stdout:7/686: dread - d9/da/d1f/d67/ddb/fd3 zero size 2024-02-16T19:29:15.371 INFO:tasks.workunit.client.1.smithi117.stdout:5/760: write d2/db3/f48 [2516158,25375] 0 2024-02-16T19:29:15.372 INFO:tasks.workunit.client.0.smithi022.stdout:7/687: write d9/da/fab [1091333,44414] 0 2024-02-16T19:29:15.373 INFO:tasks.workunit.client.0.smithi022.stdout:7/688: chown d9/da/d1f/c37 55 1 2024-02-16T19:29:15.373 INFO:tasks.workunit.client.1.smithi117.stdout:0/750: dwrite d0/d9/d14/d1d/f9e [0,4194304] 0 2024-02-16T19:29:15.375 INFO:tasks.workunit.client.1.smithi117.stdout:5/761: dread - d2/d9/d3c/d24/d50/d96/dbc/fbe zero size 2024-02-16T19:29:15.382 INFO:tasks.workunit.client.0.smithi022.stdout:7/689: stat d9/da/d2c/d2f/c4a 0 2024-02-16T19:29:15.384 INFO:tasks.workunit.client.0.smithi022.stdout:7/690: write d9/da/d1f/d44/fc0 [523540,104563] 0 2024-02-16T19:29:15.384 INFO:tasks.workunit.client.0.smithi022.stdout:8/658: write d3/d12/db0/f81 [1091281,89958] 0 2024-02-16T19:29:15.386 INFO:tasks.workunit.client.1.smithi117.stdout:7/721: write d4/d1c/d39/d46/d47/fb5 [207816,121976] 0 2024-02-16T19:29:15.388 INFO:tasks.workunit.client.1.smithi117.stdout:2/753: creat d0/d3/d34/d94/d82/fed x:0 0 0 2024-02-16T19:29:15.390 INFO:tasks.workunit.client.1.smithi117.stdout:1/643: symlink d2/d1d/ldd 0 2024-02-16T19:29:15.395 INFO:tasks.workunit.client.1.smithi117.stdout:0/751: creat d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/d27/d31/d94/f106 x:0 0 0 2024-02-16T19:29:15.396 INFO:tasks.workunit.client.1.smithi117.stdout:7/722: dwrite d4/d1c/d87/d89/d60/fba [0,4194304] 0 2024-02-16T19:29:15.396 INFO:tasks.workunit.client.1.smithi117.stdout:6/706: creat d5/d4b/fe4 x:0 0 0 2024-02-16T19:29:15.397 INFO:tasks.workunit.client.1.smithi117.stdout:9/620: dwrite dd/d47/d5b/f8c [0,4194304] 0 2024-02-16T19:29:15.398 INFO:tasks.workunit.client.1.smithi117.stdout:8/753: truncate d4/d7/da6/d3a/d65/d96/da3/fcc 812421 0 2024-02-16T19:29:15.402 INFO:tasks.workunit.client.1.smithi117.stdout:5/762: mknod d2/d9/d3c/d24/d50/d103/c104 0 2024-02-16T19:29:15.402 INFO:tasks.workunit.client.1.smithi117.stdout:1/644: dwrite d2/d19/f54 [0,4194304] 0 2024-02-16T19:29:15.405 INFO:tasks.workunit.client.1.smithi117.stdout:6/707: mkdir d5/db/de5 0 2024-02-16T19:29:15.406 INFO:tasks.workunit.client.1.smithi117.stdout:6/708: fsync d5/d4b/f7a 0 2024-02-16T19:29:15.407 INFO:tasks.workunit.client.1.smithi117.stdout:0/752: symlink d0/d9/d14/d1d/d60/d76/d7b/d8d/d55/l107 0 2024-02-16T19:29:15.411 INFO:tasks.workunit.client.1.smithi117.stdout:7/723: chown d4/d1c/d87/d89/d5f/dae/cda 830768 1 2024-02-16T19:29:15.412 INFO:tasks.workunit.client.1.smithi117.stdout:9/621: dread dd/d23/f1a [0,4194304] 0 2024-02-16T19:29:15.415 INFO:tasks.workunit.client.1.smithi117.stdout:3/814: rename d1/dd/d32/f75 to d1/dd/f11c 0 2024-02-16T19:29:15.415 INFO:tasks.workunit.client.1.smithi117.stdout:4/695: sync 2024-02-16T19:29:15.416 INFO:tasks.workunit.client.1.smithi117.stdout:3/815: chown d1/dd/d32 9960 1 2024-02-16T19:29:15.417 INFO:tasks.workunit.client.1.smithi117.stdout:8/754: creat d4/d12/d2a/d64/dc6/ff3 x:0 0 0 2024-02-16T19:29:15.417 INFO:tasks.workunit.client.1.smithi117.stdout:4/696: write d0/da/d21/d3d/d28/d64/fd3 [1033035,121060] 0 2024-02-16T19:29:15.424 INFO:tasks.workunit.client.1.smithi117.stdout:8/755: dwrite d4/d12/d2a/d64/f7d [4194304,4194304] 0 2024-02-16T19:29:15.439 INFO:tasks.workunit.client.0.smithi022.stdout:1/706: sync 2024-02-16T19:29:15.441 INFO:tasks.workunit.client.1.smithi117.stdout:1/645: symlink d2/d1d/d1f/lde 0 2024-02-16T19:29:15.442 INFO:tasks.workunit.client.1.smithi117.stdout:1/646: write d2/d19/d59/d89/f2a [4352946,14805] 0 2024-02-16T19:29:15.442 INFO:tasks.workunit.client.1.smithi117.stdout:5/763: mknod d2/d9/d3c/d24/d50/d96/dbc/c105 0 2024-02-16T19:29:15.454 INFO:tasks.workunit.client.1.smithi117.stdout:9/622: unlink dd/d47/fbe 0 2024-02-16T19:29:15.455 INFO:tasks.workunit.client.1.smithi117.stdout:5/764: dread d2/d5d/fa9 [0,4194304] 0 2024-02-16T19:29:15.456 INFO:tasks.workunit.client.1.smithi117.stdout:7/724: fdatasync d4/d1c/d39/d46/d55/db8/fc5 0 2024-02-16T19:29:15.457 INFO:tasks.workunit.client.1.smithi117.stdout:5/765: truncate d2/fea 554606 0 2024-02-16T19:29:15.458 INFO:tasks.workunit.client.1.smithi117.stdout:0/753: dread d0/d9/d14/d1d/d60/d76/d7b/d8d/f2e [0,4194304] 0 2024-02-16T19:29:15.459 INFO:tasks.workunit.client.1.smithi117.stdout:0/754: write d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/db7/fe4 [1332543,49161] 0 2024-02-16T19:29:15.460 INFO:tasks.workunit.client.1.smithi117.stdout:2/754: rename d0/d3/d34/d94/d46/f8a to d0/d3/d34/d94/d46/d68/d76/da1/fee 0 2024-02-16T19:29:15.460 INFO:tasks.workunit.client.1.smithi117.stdout:0/755: chown d0/d9/d14/d1d/d60/d76/d7b/d8d/f48 0 1 2024-02-16T19:29:15.461 INFO:tasks.workunit.client.1.smithi117.stdout:2/755: truncate d0/d13/d4d/d80/fc8 893437 0 2024-02-16T19:29:15.463 INFO:tasks.workunit.client.1.smithi117.stdout:2/756: write d0/d3/d34/d94/d46/d68/d93/fa0 [2890545,22526] 0 2024-02-16T19:29:15.464 INFO:tasks.workunit.client.1.smithi117.stdout:2/757: read - d0/d3/d34/d94/d46/d68/d76/da1/fee zero size 2024-02-16T19:29:15.466 INFO:tasks.workunit.client.1.smithi117.stdout:2/758: write d0/d3/d34/d94/d46/d68/d93/da9/fcd [1006580,99488] 0 2024-02-16T19:29:15.468 INFO:tasks.workunit.client.1.smithi117.stdout:8/756: creat d4/d12/d2a/d64/dec/d76/ff4 x:0 0 0 2024-02-16T19:29:15.472 INFO:tasks.workunit.client.1.smithi117.stdout:8/757: dread d4/d7/da6/d3a/d54/f78 [0,4194304] 0 2024-02-16T19:29:15.478 INFO:tasks.workunit.client.1.smithi117.stdout:2/759: dread d0/d13/d75/fac [0,4194304] 0 2024-02-16T19:29:15.483 INFO:tasks.workunit.client.0.smithi022.stdout:8/659: creat d3/feb x:0 0 0 2024-02-16T19:29:15.483 INFO:tasks.workunit.client.0.smithi022.stdout:1/707: dwrite d6/d8/d22/d16/d17/d63/d6f/d8b/fd9 [0,4194304] 0 2024-02-16T19:29:15.488 INFO:tasks.workunit.client.0.smithi022.stdout:8/660: creat d3/d12/d17/d64/d8f/fec x:0 0 0 2024-02-16T19:29:15.489 INFO:tasks.workunit.client.0.smithi022.stdout:8/661: chown d3/d12/d2b/dca/fdf 4378662 1 2024-02-16T19:29:15.494 INFO:tasks.workunit.client.1.smithi117.stdout:0/756: rename d0/d9/d2c/f44 to d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/d27/d31/d94/f108 0 2024-02-16T19:29:15.497 INFO:tasks.workunit.client.1.smithi117.stdout:6/709: truncate d5/d4b/f46 1624873 0 2024-02-16T19:29:15.499 INFO:tasks.workunit.client.0.smithi022.stdout:8/662: rename d3/d12/d2b/d54/c88 to d3/d12/db0/ced 0 2024-02-16T19:29:15.502 INFO:tasks.workunit.client.0.smithi022.stdout:1/708: getdents d6/d8/d22/d16/d17 0 2024-02-16T19:29:15.505 INFO:tasks.workunit.client.1.smithi117.stdout:1/647: mknod d2/d19/d59/d76/d91/dcf/cdf 0 2024-02-16T19:29:15.506 INFO:tasks.workunit.client.0.smithi022.stdout:1/709: mknod d6/d8/d22/d47/db7/cde 0 2024-02-16T19:29:15.507 INFO:tasks.workunit.client.1.smithi117.stdout:8/758: truncate d4/d12/d2a/d64/dec/d76/d8c/fa4 1033217 0 2024-02-16T19:29:15.508 INFO:tasks.workunit.client.1.smithi117.stdout:8/759: readlink d4/d1c/leb 0 2024-02-16T19:29:15.508 INFO:tasks.workunit.client.1.smithi117.stdout:6/710: dread d5/d4b/d44/d52/d81/fa6 [0,4194304] 0 2024-02-16T19:29:15.510 INFO:tasks.workunit.client.1.smithi117.stdout:8/760: write d4/d12/d31/f40 [1253412,8675] 0 2024-02-16T19:29:15.511 INFO:tasks.workunit.client.0.smithi022.stdout:1/710: rmdir d6/d8/d22/d3f 39 2024-02-16T19:29:15.511 INFO:tasks.workunit.client.1.smithi117.stdout:2/760: write d0/f12 [957963,6759] 0 2024-02-16T19:29:15.511 INFO:tasks.workunit.client.1.smithi117.stdout:4/697: sync 2024-02-16T19:29:15.513 INFO:tasks.workunit.client.1.smithi117.stdout:9/623: creat dd/db5/dc4/dcf/fd6 x:0 0 0 2024-02-16T19:29:15.514 INFO:tasks.workunit.client.1.smithi117.stdout:9/624: fsync dd/d47/d48/f98 0 2024-02-16T19:29:15.514 INFO:tasks.workunit.client.1.smithi117.stdout:9/625: dread - dd/d47/d5b/fbd zero size 2024-02-16T19:29:15.516 INFO:tasks.workunit.client.1.smithi117.stdout:5/766: mknod d2/d4c/d4f/de7/c106 0 2024-02-16T19:29:15.516 INFO:tasks.workunit.client.1.smithi117.stdout:5/767: readlink d2/d5d/d8e/dc1/lcf 0 2024-02-16T19:29:15.517 INFO:tasks.workunit.client.1.smithi117.stdout:0/757: readlink d0/d9/d14/d1d/d60/d65/lf7 0 2024-02-16T19:29:15.517 INFO:tasks.workunit.client.0.smithi022.stdout:8/663: write d3/f15 [1786944,33415] 0 2024-02-16T19:29:15.517 INFO:tasks.workunit.client.1.smithi117.stdout:9/626: dread dd/d45/f4f [0,4194304] 0 2024-02-16T19:29:15.519 INFO:tasks.workunit.client.1.smithi117.stdout:9/627: write dd/d45/f49 [702076,115013] 0 2024-02-16T19:29:15.523 INFO:tasks.workunit.client.1.smithi117.stdout:1/648: creat d2/d19/d59/d89/d69/fe0 x:0 0 0 2024-02-16T19:29:15.525 INFO:tasks.workunit.client.0.smithi022.stdout:1/711: dread d6/d8/d22/d16/d17/d1d/f33 [4194304,4194304] 0 2024-02-16T19:29:15.525 INFO:tasks.workunit.client.1.smithi117.stdout:6/711: rename d5/f12 to d5/d4b/d44/d7c/fe6 0 2024-02-16T19:29:15.526 INFO:tasks.workunit.client.1.smithi117.stdout:1/649: dread d2/d1d/d60/f66 [4194304,4194304] 0 2024-02-16T19:29:15.526 INFO:tasks.workunit.client.1.smithi117.stdout:6/712: truncate d5/d4b/fe4 743246 0 2024-02-16T19:29:15.527 INFO:tasks.workunit.client.1.smithi117.stdout:6/713: chown d5/db/d7d/cbf 44 1 2024-02-16T19:29:15.527 INFO:tasks.workunit.client.0.smithi022.stdout:8/664: write d3/d25/f43 [982195,52733] 0 2024-02-16T19:29:15.529 INFO:tasks.workunit.client.1.smithi117.stdout:4/698: mkdir d0/da/d21/db4/de4 0 2024-02-16T19:29:15.532 INFO:tasks.workunit.client.0.smithi022.stdout:1/712: creat d6/d66/fdf x:0 0 0 2024-02-16T19:29:15.532 INFO:tasks.workunit.client.0.smithi022.stdout:1/713: readlink d6/d8/d22/d16/d17/d63/ld7 0 2024-02-16T19:29:15.533 INFO:tasks.workunit.client.0.smithi022.stdout:1/714: truncate d6/d66/fb5 4197196 0 2024-02-16T19:29:15.538 INFO:tasks.workunit.client.0.smithi022.stdout:8/665: write d3/d12/da6/f5e [3362885,7535] 0 2024-02-16T19:29:15.541 INFO:tasks.workunit.client.1.smithi117.stdout:5/768: read - d2/d5d/fdf zero size 2024-02-16T19:29:15.542 INFO:tasks.workunit.client.1.smithi117.stdout:5/769: chown d2/d9/d3c/d24/d50/d52/d74/c9a 6639571 1 2024-02-16T19:29:15.542 INFO:tasks.workunit.client.0.smithi022.stdout:8/666: symlink d3/d12/d2b/d54/dcd/lee 0 2024-02-16T19:29:15.543 INFO:tasks.workunit.client.0.smithi022.stdout:1/715: truncate d6/d8/d22/d16/d17/d63/d6f/d8b/fd9 1366517 0 2024-02-16T19:29:15.543 INFO:tasks.workunit.client.0.smithi022.stdout:8/667: write d3/d12/d2b/d2e/f87 [749854,108180] 0 2024-02-16T19:29:15.544 INFO:tasks.workunit.client.1.smithi117.stdout:3/816: link d1/d30/c6a d1/d1b/c11d 0 2024-02-16T19:29:15.544 INFO:tasks.workunit.client.1.smithi117.stdout:3/817: dread - d1/d30/d9f/dcb/fe7 zero size 2024-02-16T19:29:15.545 INFO:tasks.workunit.client.1.smithi117.stdout:3/818: write d1/d30/f119 [328491,24353] 0 2024-02-16T19:29:15.546 INFO:tasks.workunit.client.1.smithi117.stdout:9/628: mkdir dd/d23/d64/dd7 0 2024-02-16T19:29:15.546 INFO:tasks.workunit.client.0.smithi022.stdout:8/668: mknod d3/d25/ddb/cef 0 2024-02-16T19:29:15.547 INFO:tasks.workunit.client.0.smithi022.stdout:1/716: creat d6/d8/d22/d16/d17/d5d/fe0 x:0 0 0 2024-02-16T19:29:15.549 INFO:tasks.workunit.client.0.smithi022.stdout:1/717: dread - d6/d8/d22/d47/d51/fa2 zero size 2024-02-16T19:29:15.549 INFO:tasks.workunit.client.1.smithi117.stdout:1/650: creat d2/d1d/d60/d7c/d9e/da5/fe1 x:0 0 0 2024-02-16T19:29:15.550 INFO:tasks.workunit.client.0.smithi022.stdout:1/718: write d6/d8/d22/f82 [2263090,45658] 0 2024-02-16T19:29:15.552 INFO:tasks.workunit.client.0.smithi022.stdout:8/669: rename d3/d12/d17/d64/c96 to d3/d12/d2b/d54/db8/cf0 0 2024-02-16T19:29:15.552 INFO:tasks.workunit.client.0.smithi022.stdout:8/670: chown d3/d25/d33 13246 1 2024-02-16T19:29:15.558 INFO:tasks.workunit.client.1.smithi117.stdout:4/699: rename d0/da/d21/d3d/d28/l38 to d0/da/d21/d3d/d28/d64/d73/le5 0 2024-02-16T19:29:15.559 INFO:tasks.workunit.client.1.smithi117.stdout:7/725: getdents d4/d13 0 2024-02-16T19:29:15.560 INFO:tasks.workunit.client.1.smithi117.stdout:7/726: chown d4/d1c/d87/d89 745603417 1 2024-02-16T19:29:15.562 INFO:tasks.workunit.client.0.smithi022.stdout:8/671: read d3/d12/d2b/d54/d74/f82 [93077,70230] 0 2024-02-16T19:29:15.563 INFO:tasks.workunit.client.1.smithi117.stdout:5/770: symlink d2/d9/d3c/d24/l107 0 2024-02-16T19:29:15.564 INFO:tasks.workunit.client.1.smithi117.stdout:3/819: unlink d1/d1b/d1e/d5e/f74 0 2024-02-16T19:29:15.565 INFO:tasks.workunit.client.1.smithi117.stdout:8/761: dwrite d4/d12/d2a/d64/dec/d76/d8c/fa4 [0,4194304] 0 2024-02-16T19:29:15.569 INFO:tasks.workunit.client.1.smithi117.stdout:0/758: truncate d0/fd 2998520 0 2024-02-16T19:29:15.570 INFO:tasks.workunit.client.1.smithi117.stdout:1/651: mknod d2/d1d/d1f/ce2 0 2024-02-16T19:29:15.571 INFO:tasks.workunit.client.1.smithi117.stdout:6/714: mkdir d5/d4b/d20/de7 0 2024-02-16T19:29:15.571 INFO:tasks.workunit.client.1.smithi117.stdout:0/759: readlink d0/d9/d14/d1d/d60/d65/lf7 0 2024-02-16T19:29:15.573 INFO:tasks.workunit.client.1.smithi117.stdout:3/820: dwrite d1/d30/d9f/dcb/fe7 [0,4194304] 0 2024-02-16T19:29:15.573 INFO:tasks.workunit.client.1.smithi117.stdout:4/700: creat d0/da/d21/d3d/d28/d66/da2/dac/fe6 x:0 0 0 2024-02-16T19:29:15.573 INFO:tasks.workunit.client.1.smithi117.stdout:7/727: symlink d4/d1c/d39/d46/d55/db8/l109 0 2024-02-16T19:29:15.575 INFO:tasks.workunit.client.1.smithi117.stdout:3/821: fsync d1/d1b/d1e/d37/ffc 0 2024-02-16T19:29:15.577 INFO:tasks.workunit.client.1.smithi117.stdout:7/728: readlink d4/d13/da3/l108 0 2024-02-16T19:29:15.578 INFO:tasks.workunit.client.1.smithi117.stdout:5/771: chown d2/l11 5674 1 2024-02-16T19:29:15.579 INFO:tasks.workunit.client.1.smithi117.stdout:8/762: read d4/d7/da6/d3a/d65/d96/da3/fcc [218765,71263] 0 2024-02-16T19:29:15.580 INFO:tasks.workunit.client.1.smithi117.stdout:1/652: symlink d2/d19/d59/d89/d7b/le3 0 2024-02-16T19:29:15.580 INFO:tasks.workunit.client.1.smithi117.stdout:6/715: dread d5/d9c/f9e [0,4194304] 0 2024-02-16T19:29:15.581 INFO:tasks.workunit.client.1.smithi117.stdout:2/761: link d0/d3/d34/d94/d46/d68/d76/da1/c3e d0/d3/d10/cef 0 2024-02-16T19:29:15.583 INFO:tasks.workunit.client.1.smithi117.stdout:3/822: fdatasync d1/dd/d5c/d85/f11b 0 2024-02-16T19:29:15.584 INFO:tasks.workunit.client.1.smithi117.stdout:2/762: stat d0/d13/dc4/de4/fe5 0 2024-02-16T19:29:15.585 INFO:tasks.workunit.client.1.smithi117.stdout:4/701: creat d0/da/d21/d3d/d28/d64/d73/d91/d37/d98/fe7 x:0 0 0 2024-02-16T19:29:15.585 INFO:tasks.workunit.client.1.smithi117.stdout:9/629: dread dd/d47/d5b/d5f/d77/d7c/fa2 [0,4194304] 0 2024-02-16T19:29:15.586 INFO:tasks.workunit.client.1.smithi117.stdout:6/716: truncate d5/d4b/d20/d69/fc2 516934 0 2024-02-16T19:29:15.586 INFO:tasks.workunit.client.1.smithi117.stdout:7/729: write d4/d1c/d39/d46/f96 [151883,79606] 0 2024-02-16T19:29:15.588 INFO:tasks.workunit.client.1.smithi117.stdout:9/630: write dd/d47/d5b/fc6 [853275,104540] 0 2024-02-16T19:29:15.589 INFO:tasks.workunit.client.1.smithi117.stdout:6/717: write d5/db/f4a [2313164,84272] 0 2024-02-16T19:29:15.589 INFO:tasks.workunit.client.1.smithi117.stdout:8/763: creat d4/d7/da6/d50/ff5 x:0 0 0 2024-02-16T19:29:15.590 INFO:tasks.workunit.client.0.smithi022.stdout:8/672: dread d3/d12/d17/d64/f73 [0,4194304] 0 2024-02-16T19:29:15.592 INFO:tasks.workunit.client.0.smithi022.stdout:8/673: chown d3/d12/d2b/d54/d97/c9f 0 1 2024-02-16T19:29:15.593 INFO:tasks.workunit.client.1.smithi117.stdout:1/653: unlink d2/d19/d59/d89/d7b/f7e 0 2024-02-16T19:29:15.595 INFO:tasks.workunit.client.1.smithi117.stdout:1/654: write d2/d1d/d60/fb2 [532091,125024] 0 2024-02-16T19:29:15.595 INFO:tasks.workunit.client.1.smithi117.stdout:3/823: fdatasync d1/d30/d52/fa4 0 2024-02-16T19:29:15.599 INFO:tasks.workunit.client.1.smithi117.stdout:7/730: mknod d4/d1c/d98/c10a 0 2024-02-16T19:29:15.600 INFO:tasks.workunit.client.1.smithi117.stdout:4/702: mkdir d0/da/d21/d3d/d28/d64/d8a/d92/de8 0 2024-02-16T19:29:15.601 INFO:tasks.workunit.client.0.smithi022.stdout:8/674: link d3/d12/d17/f9d d3/d12/d2b/d54/db8/ff1 0 2024-02-16T19:29:15.601 INFO:tasks.workunit.client.1.smithi117.stdout:4/703: chown d0/da/d21/d3d/d28/f39 892 1 2024-02-16T19:29:15.604 INFO:tasks.workunit.client.0.smithi022.stdout:8/675: creat d3/d25/ff2 x:0 0 0 2024-02-16T19:29:15.605 INFO:tasks.workunit.client.1.smithi117.stdout:5/772: write d2/d9/d3c/d24/d50/d96/dbc/fbe [640687,103346] 0 2024-02-16T19:29:15.605 INFO:tasks.workunit.client.0.smithi022.stdout:8/676: stat d3/d12/d17/cc9 0 2024-02-16T19:29:15.607 INFO:tasks.workunit.client.1.smithi117.stdout:9/631: fsync dd/d45/f4f 0 2024-02-16T19:29:15.612 INFO:tasks.workunit.client.1.smithi117.stdout:1/655: symlink d2/d4/dbb/le4 0 2024-02-16T19:29:15.612 INFO:tasks.workunit.client.1.smithi117.stdout:0/760: truncate d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/db7/fe4 2438312 0 2024-02-16T19:29:15.613 INFO:tasks.workunit.client.1.smithi117.stdout:8/764: dwrite d4/d12/d2a/d6d/de3/fef [0,4194304] 0 2024-02-16T19:29:15.613 INFO:tasks.workunit.client.0.smithi022.stdout:3/617: sync 2024-02-16T19:29:15.615 INFO:tasks.workunit.client.1.smithi117.stdout:2/763: mkdir d0/d3/d34/d94/df0 0 2024-02-16T19:29:15.615 INFO:tasks.workunit.client.0.smithi022.stdout:8/677: truncate d3/d12/da6/f7 2815962 0 2024-02-16T19:29:15.615 INFO:tasks.workunit.client.0.smithi022.stdout:8/678: dread - d3/d12/d2b/fde zero size 2024-02-16T19:29:15.616 INFO:tasks.workunit.client.1.smithi117.stdout:4/704: creat d0/da/d21/d3d/d28/d66/da2/dac/dcb/fe9 x:0 0 0 2024-02-16T19:29:15.617 INFO:tasks.workunit.client.1.smithi117.stdout:5/773: creat d2/d3/d21/d2a/dcb/f108 x:0 0 0 2024-02-16T19:29:15.618 INFO:tasks.workunit.client.0.smithi022.stdout:3/618: creat d7/d12/d18/dd/d24/d58/d6b/fe1 x:0 0 0 2024-02-16T19:29:15.618 INFO:tasks.workunit.client.1.smithi117.stdout:4/705: readlink d0/da/d21/l71 0 2024-02-16T19:29:15.623 INFO:tasks.workunit.client.1.smithi117.stdout:2/764: dwrite d0/d13/d75/f59 [0,4194304] 0 2024-02-16T19:29:15.623 INFO:tasks.workunit.client.0.smithi022.stdout:3/619: rename d7/d12/d18/dd/d3c/fa2 to d7/d12/d18/dd/d3c/fe2 0 2024-02-16T19:29:15.625 INFO:tasks.workunit.client.0.smithi022.stdout:3/620: read d7/d12/d59/f85 [323989,29395] 0 2024-02-16T19:29:15.629 INFO:tasks.workunit.client.0.smithi022.stdout:3/621: creat d7/d21/dd9/fe3 x:0 0 0 2024-02-16T19:29:15.632 INFO:tasks.workunit.client.1.smithi117.stdout:8/765: mknod d4/d7/da6/d3a/cf6 0 2024-02-16T19:29:15.633 INFO:tasks.workunit.client.1.smithi117.stdout:8/766: chown d4/d12/d2a/f71 239791 1 2024-02-16T19:29:15.636 INFO:tasks.workunit.client.1.smithi117.stdout:9/632: rename dd/d47/d5e/d9a/lc1 to dd/d47/d5b/d5f/d77/ld8 0 2024-02-16T19:29:15.636 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:15 smithi117 ceph-mon[35309]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:29:15.637 INFO:tasks.workunit.client.1.smithi117.stdout:5/774: symlink d2/d4c/dde/l109 0 2024-02-16T19:29:15.638 INFO:tasks.workunit.client.1.smithi117.stdout:6/718: truncate d5/d4b/d71/dc7/d72/fa2 137979 0 2024-02-16T19:29:15.640 INFO:tasks.workunit.client.1.smithi117.stdout:3/824: getdents d1/d1b/d20 0 2024-02-16T19:29:15.642 INFO:tasks.workunit.client.1.smithi117.stdout:7/731: dread d4/f28 [0,4194304] 0 2024-02-16T19:29:15.643 INFO:tasks.workunit.client.1.smithi117.stdout:5/775: dwrite d2/db3/fa0 [0,4194304] 0 2024-02-16T19:29:15.644 INFO:tasks.workunit.client.0.smithi022.stdout:3/622: link d7/d12/d18/dac/faa d7/d12/d18/dd/d24/d53/fe4 0 2024-02-16T19:29:15.646 INFO:tasks.workunit.client.0.smithi022.stdout:8/679: dwrite d3/d12/d2b/da9/fe4 [0,4194304] 0 2024-02-16T19:29:15.649 INFO:tasks.workunit.client.0.smithi022.stdout:3/623: mkdir d7/d12/d18/dd/d24/de5 0 2024-02-16T19:29:15.651 INFO:tasks.workunit.client.0.smithi022.stdout:3/624: chown d7/d12/d18/dd/d24/d58/d70/f7b 5790 1 2024-02-16T19:29:15.654 INFO:tasks.workunit.client.1.smithi117.stdout:6/719: rmdir d5/d9c/daa/d85/db0 39 2024-02-16T19:29:15.654 INFO:tasks.workunit.client.1.smithi117.stdout:3/825: symlink d1/d30/d52/d10f/l11e 0 2024-02-16T19:29:15.657 INFO:tasks.workunit.client.1.smithi117.stdout:2/765: link d0/d3/d34/d94/d46/d68/d76/da1/fde d0/d3/d34/d94/d46/d68/d76/da1/d2f/ff1 0 2024-02-16T19:29:15.658 INFO:tasks.workunit.client.0.smithi022.stdout:3/625: unlink d7/d12/d18/dd/d24/d58/l74 0 2024-02-16T19:29:15.659 INFO:tasks.workunit.client.1.smithi117.stdout:2/766: dread - d0/d3/d34/d94/d46/d68/d76/da1/d56/fd3 zero size 2024-02-16T19:29:15.659 INFO:tasks.workunit.client.1.smithi117.stdout:2/767: stat d0/d3/d34/d94/d46/d68/d76/l97 0 2024-02-16T19:29:15.660 INFO:tasks.workunit.client.1.smithi117.stdout:7/732: creat d4/d13/d63/ddd/f10b x:0 0 0 2024-02-16T19:29:15.662 INFO:tasks.workunit.client.1.smithi117.stdout:5/776: write d2/d9/f18 [6451693,56012] 0 2024-02-16T19:29:15.662 INFO:tasks.workunit.client.0.smithi022.stdout:3/626: mkdir d7/d12/d18/dd/d24/d58/d6b/de6 0 2024-02-16T19:29:15.663 INFO:tasks.workunit.client.1.smithi117.stdout:0/761: getdents d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/d27/d31/d6d 0 2024-02-16T19:29:15.666 INFO:tasks.workunit.client.1.smithi117.stdout:8/767: mkdir d4/d12/d2a/df7 0 2024-02-16T19:29:15.668 INFO:tasks.workunit.client.0.smithi022.stdout:3/627: rename d7/d12/d18/dd/d24/f63 to d7/d12/d18/dd/d31/dbf/fe7 0 2024-02-16T19:29:15.669 INFO:tasks.workunit.client.1.smithi117.stdout:9/633: symlink dd/db5/dc4/dd3/ld9 0 2024-02-16T19:29:15.671 INFO:tasks.workunit.client.1.smithi117.stdout:3/826: mkdir d1/d30/d52/d10f/d11f 0 2024-02-16T19:29:15.673 INFO:tasks.workunit.client.1.smithi117.stdout:9/634: dread dd/d45/d31/d63/d8e/dbb/fd2 [0,4194304] 0 2024-02-16T19:29:15.673 INFO:tasks.workunit.client.1.smithi117.stdout:2/768: creat d0/d13/dc4/de4/ff2 x:0 0 0 2024-02-16T19:29:15.674 INFO:tasks.workunit.client.1.smithi117.stdout:2/769: chown d0/d3/d40 2702 1 2024-02-16T19:29:15.675 INFO:tasks.workunit.client.0.smithi022.stdout:3/628: link d7/d12/d18/dd/d31/dbf/fcf d7/d12/d18/dd/d31/dbf/dc5/fe8 0 2024-02-16T19:29:15.676 INFO:tasks.workunit.client.1.smithi117.stdout:8/768: dread d4/d7/da6/f63 [0,4194304] 0 2024-02-16T19:29:15.677 INFO:tasks.workunit.client.1.smithi117.stdout:8/769: chown d4/d7/da6/d3a/d65/c66 0 1 2024-02-16T19:29:15.677 INFO:tasks.workunit.client.1.smithi117.stdout:0/762: mkdir d0/d9/d14/d1d/d60/dc3/d109 0 2024-02-16T19:29:15.678 INFO:tasks.workunit.client.0.smithi022.stdout:8/680: dwrite d3/fe2 [8388608,4194304] 0 2024-02-16T19:29:15.680 INFO:tasks.workunit.client.0.smithi022.stdout:8/681: write d3/f15 [865668,14487] 0 2024-02-16T19:29:15.680 INFO:tasks.workunit.client.1.smithi117.stdout:1/656: rename d2/d19/d59/d89/d7b/le3 to d2/d1d/d60/le5 0 2024-02-16T19:29:15.681 INFO:tasks.workunit.client.1.smithi117.stdout:6/720: getdents d5/d4b/d44/d93/dae/dc3 0 2024-02-16T19:29:15.685 INFO:tasks.workunit.client.1.smithi117.stdout:3/827: mkdir d1/d31/d89/dbd/d120 0 2024-02-16T19:29:15.690 INFO:tasks.workunit.client.1.smithi117.stdout:0/763: write d0/d9/d14/d1d/d60/d76/d7b/d8d/f97 [2460886,128155] 0 2024-02-16T19:29:15.690 INFO:tasks.workunit.client.1.smithi117.stdout:0/764: chown d0/cb5 162 1 2024-02-16T19:29:15.693 INFO:tasks.workunit.client.1.smithi117.stdout:5/777: rename d2/d9/d3c/d24/d90/dd4/dfb/l60 to d2/d9/dba/l10a 0 2024-02-16T19:29:15.696 INFO:tasks.workunit.client.0.smithi022.stdout:8/682: creat d3/d12/da6/ff3 x:0 0 0 2024-02-16T19:29:15.697 INFO:tasks.workunit.client.1.smithi117.stdout:6/721: creat d5/d4b/d71/fe8 x:0 0 0 2024-02-16T19:29:15.699 INFO:tasks.workunit.client.0.smithi022.stdout:8/683: dread d3/d12/d2b/dca/fdf [0,4194304] 0 2024-02-16T19:29:15.701 INFO:tasks.workunit.client.0.smithi022.stdout:8/684: readlink d3/d12/d2b/da9/lc1 0 2024-02-16T19:29:15.702 INFO:tasks.workunit.client.1.smithi117.stdout:7/733: creat d4/d1c/d87/d89/f10c x:0 0 0 2024-02-16T19:29:15.704 INFO:tasks.workunit.client.0.smithi022.stdout:3/629: dwrite d7/d12/d18/dd/d24/d58/d70/f7b [0,4194304] 0 2024-02-16T19:29:15.711 INFO:tasks.workunit.client.1.smithi117.stdout:6/722: mknod d5/db/d7d/d32/da8/ce9 0 2024-02-16T19:29:15.714 INFO:tasks.workunit.client.1.smithi117.stdout:3/828: dwrite d1/d30/d52/fc9 [0,4194304] 0 2024-02-16T19:29:15.716 INFO:tasks.workunit.client.1.smithi117.stdout:9/635: rmdir dd/d47/d48/dd0 0 2024-02-16T19:29:15.716 INFO:tasks.workunit.client.1.smithi117.stdout:6/723: dread d5/d4b/d20/da4/fc6 [0,4194304] 0 2024-02-16T19:29:15.717 INFO:tasks.workunit.client.1.smithi117.stdout:1/657: write d2/d19/d59/d89/d69/dad/dc6/dcc/fce [4328245,26845] 0 2024-02-16T19:29:15.720 INFO:tasks.workunit.client.1.smithi117.stdout:7/734: mknod d4/d1c/dc0/c10d 0 2024-02-16T19:29:15.720 INFO:tasks.workunit.client.1.smithi117.stdout:6/724: truncate d5/d4b/d44/fcf 459853 0 2024-02-16T19:29:15.720 INFO:tasks.workunit.client.1.smithi117.stdout:0/765: mkdir d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/d27/d10a 0 2024-02-16T19:29:15.722 INFO:tasks.workunit.client.1.smithi117.stdout:4/706: dread d0/da/d21/f2b [0,4194304] 0 2024-02-16T19:29:15.729 INFO:tasks.workunit.client.0.smithi022.stdout:8/685: dwrite d3/f8d [0,4194304] 0 2024-02-16T19:29:15.730 INFO:tasks.workunit.client.1.smithi117.stdout:2/770: rename d0/d3/d34/d94/d46/d68/d76/da1/d2f/le0 to d0/d3/lf3 0 2024-02-16T19:29:15.731 INFO:tasks.workunit.client.1.smithi117.stdout:9/636: write dd/d47/d5b/d5f/d77/d7c/fa2 [803257,13549] 0 2024-02-16T19:29:15.733 INFO:tasks.workunit.client.1.smithi117.stdout:6/725: creat d5/d4b/dd6/fea x:0 0 0 2024-02-16T19:29:15.734 INFO:tasks.workunit.client.0.smithi022.stdout:8/686: dread d3/d12/da6/f28 [0,4194304] 0 2024-02-16T19:29:15.734 INFO:tasks.workunit.client.0.smithi022.stdout:3/630: dwrite d7/d21/f38 [0,4194304] 0 2024-02-16T19:29:15.735 INFO:tasks.workunit.client.1.smithi117.stdout:8/770: rename d4/d7/da6/d3a/d43/d44/cb9 to d4/d12/d2a/df7/cf8 0 2024-02-16T19:29:15.737 INFO:tasks.workunit.client.1.smithi117.stdout:9/637: write dd/db5/dc4/dcf/fd6 [319319,103072] 0 2024-02-16T19:29:15.737 INFO:tasks.workunit.client.1.smithi117.stdout:8/771: stat d4/d7/da6/d3a/d65/d96/cf1 0 2024-02-16T19:29:15.738 INFO:tasks.workunit.client.0.smithi022.stdout:3/631: truncate d7/d12/d18/dd/d24/d58/fda 427174 0 2024-02-16T19:29:15.738 INFO:tasks.workunit.client.1.smithi117.stdout:4/707: unlink d0/da/d21/d3d/d28/d66/d97/d6d/f93 0 2024-02-16T19:29:15.740 INFO:tasks.workunit.client.1.smithi117.stdout:7/735: creat d4/d1c/d87/d89/f10e x:0 0 0 2024-02-16T19:29:15.741 INFO:tasks.workunit.client.1.smithi117.stdout:0/766: dwrite d0/f53 [0,4194304] 0 2024-02-16T19:29:15.743 INFO:tasks.workunit.client.0.smithi022.stdout:8/687: rename d3/d12/d2b/d2e/c47 to d3/d12/d2b/dca/cf4 0 2024-02-16T19:29:15.745 INFO:tasks.workunit.client.0.smithi022.stdout:3/632: readlink d7/d12/d18/dd/la4 0 2024-02-16T19:29:15.748 INFO:tasks.workunit.client.0.smithi022.stdout:8/688: symlink d3/d12/d2b/da9/lf5 0 2024-02-16T19:29:15.749 INFO:tasks.workunit.client.1.smithi117.stdout:1/658: truncate d2/d19/d59/d89/d69/f8f 241661 0 2024-02-16T19:29:15.751 INFO:tasks.workunit.client.1.smithi117.stdout:5/778: rename d2/d4c/dde/l109 to d2/d9/d3c/d24/d90/dd4/dfb/d62/l10b 0 2024-02-16T19:29:15.752 INFO:tasks.workunit.client.1.smithi117.stdout:5/779: chown d2/d3/d21/d49 6913411 1 2024-02-16T19:29:15.752 INFO:tasks.workunit.client.0.smithi022.stdout:3/633: rename d7/d12/d59 to d7/d12/d18/dd/d24/d93/de9 0 2024-02-16T19:29:15.753 INFO:tasks.workunit.client.1.smithi117.stdout:2/771: symlink d0/d13/d4d/d53/d58/dbc/lf4 0 2024-02-16T19:29:15.754 INFO:tasks.workunit.client.0.smithi022.stdout:8/689: symlink d3/d12/d92/lf6 0 2024-02-16T19:29:15.754 INFO:tasks.workunit.client.1.smithi117.stdout:1/659: dwrite d2/d1d/d77/fa0 [0,4194304] 0 2024-02-16T19:29:15.755 INFO:tasks.workunit.client.0.smithi022.stdout:8/690: truncate d3/fd4 465833 0 2024-02-16T19:29:15.756 INFO:tasks.workunit.client.1.smithi117.stdout:9/638: mkdir dd/db5/dc4/dcf/dda 0 2024-02-16T19:29:15.756 INFO:tasks.workunit.client.1.smithi117.stdout:4/708: readlink d0/da/d21/d3d/d28/d66/lb8 0 2024-02-16T19:29:15.757 INFO:tasks.workunit.client.0.smithi022.stdout:8/691: fdatasync d3/d12/d2b/f6c 0 2024-02-16T19:29:15.763 INFO:tasks.workunit.client.1.smithi117.stdout:4/709: dwrite d0/da/d21/d3d/d28/d66/da2/dac/dcb/fe9 [0,4194304] 0 2024-02-16T19:29:15.823 INFO:tasks.workunit.client.0.smithi022.stdout:6/694: sync 2024-02-16T19:29:15.833 INFO:tasks.workunit.client.0.smithi022.stdout:6/695: creat d1/d8/da/d17/d53/d7f/db3/ff8 x:0 0 0 2024-02-16T19:29:15.834 INFO:tasks.workunit.client.1.smithi117.stdout:2/772: write d0/d3/d34/d94/d91/f98 [1670649,28515] 0 2024-02-16T19:29:15.837 INFO:tasks.workunit.client.1.smithi117.stdout:1/660: mkdir d2/d19/d59/d89/d69/d80/de6 0 2024-02-16T19:29:15.845 INFO:tasks.workunit.client.0.smithi022.stdout:3/634: truncate d7/d12/d18/dd/f7d 3440495 0 2024-02-16T19:29:15.847 INFO:tasks.workunit.client.1.smithi117.stdout:4/710: creat d0/da/d21/d3d/d28/d64/d8a/fea x:0 0 0 2024-02-16T19:29:15.848 INFO:tasks.workunit.client.1.smithi117.stdout:4/711: read - d0/da/d21/d3d/d28/f86 zero size 2024-02-16T19:29:15.851 INFO:tasks.workunit.client.0.smithi022.stdout:8/692: write d3/d12/da6/f27 [1306867,98313] 0 2024-02-16T19:29:15.854 INFO:tasks.workunit.client.0.smithi022.stdout:6/696: write d1/d8/da/d17/d85/f2b [1618605,17143] 0 2024-02-16T19:29:15.856 INFO:tasks.workunit.client.0.smithi022.stdout:8/693: write d3/d12/d2b/dca/fdf [2526544,39901] 0 2024-02-16T19:29:15.857 INFO:tasks.workunit.client.0.smithi022.stdout:3/635: dread d7/d21/fa6 [0,4194304] 0 2024-02-16T19:29:15.857 INFO:tasks.workunit.client.0.smithi022.stdout:6/697: read d1/d8/d81/fd9 [3478420,30648] 0 2024-02-16T19:29:15.861 INFO:tasks.workunit.client.0.smithi022.stdout:8/694: creat d3/d12/d2b/d54/d97/ff7 x:0 0 0 2024-02-16T19:29:15.863 INFO:tasks.workunit.client.0.smithi022.stdout:6/698: creat d1/d8/dac/ff9 x:0 0 0 2024-02-16T19:29:15.864 INFO:tasks.workunit.client.0.smithi022.stdout:3/636: mknod d7/d21/dd9/cea 0 2024-02-16T19:29:15.865 INFO:tasks.workunit.client.0.smithi022.stdout:3/637: dread - d7/d21/dd9/fe3 zero size 2024-02-16T19:29:15.869 INFO:tasks.workunit.client.0.smithi022.stdout:8/695: mknod d3/d12/d17/cf8 0 2024-02-16T19:29:15.874 INFO:tasks.workunit.client.0.smithi022.stdout:3/638: link d7/d12/d18/dd/d24/d29/c30 d7/d12/d18/dd/d31/d7f/ceb 0 2024-02-16T19:29:15.877 INFO:tasks.workunit.client.0.smithi022.stdout:3/639: write d7/d12/d18/dd/d24/d58/f95 [1712642,32746] 0 2024-02-16T19:29:15.879 INFO:tasks.workunit.client.1.smithi117.stdout:0/767: dwrite d0/d9/d14/d1d/d60/d65/fab [0,4194304] 0 2024-02-16T19:29:15.880 INFO:tasks.workunit.client.1.smithi117.stdout:2/773: symlink d0/d3/d22/lf5 0 2024-02-16T19:29:15.880 INFO:tasks.workunit.client.0.smithi022.stdout:8/696: dread d3/d12/da6/f2f [0,4194304] 0 2024-02-16T19:29:15.880 INFO:tasks.workunit.client.0.smithi022.stdout:3/640: mknod d7/d12/d18/dd/d24/d53/cec 0 2024-02-16T19:29:15.881 INFO:tasks.workunit.client.0.smithi022.stdout:8/697: chown d3/d12/d2b/d54/f76 42275 1 2024-02-16T19:29:15.881 INFO:tasks.workunit.client.1.smithi117.stdout:7/736: creat d4/d1c/d39/f10f x:0 0 0 2024-02-16T19:29:15.885 INFO:tasks.workunit.client.1.smithi117.stdout:8/772: dwrite d4/d7/f88 [0,4194304] 0 2024-02-16T19:29:15.885 INFO:tasks.workunit.client.1.smithi117.stdout:4/712: dwrite d0/da/d21/d3d/d28/f4e [0,4194304] 0 2024-02-16T19:29:15.886 INFO:tasks.workunit.client.1.smithi117.stdout:5/780: getdents d2/d9/d3c/d24 0 2024-02-16T19:29:15.886 INFO:tasks.workunit.client.0.smithi022.stdout:8/698: dread d3/d12/f9e [0,4194304] 0 2024-02-16T19:29:15.887 INFO:tasks.workunit.client.0.smithi022.stdout:8/699: chown d3/d25/f7d 13 1 2024-02-16T19:29:15.887 INFO:tasks.workunit.client.1.smithi117.stdout:1/661: link d2/d51/cb4 d2/d4/dbb/d7f/dc8/ce7 0 2024-02-16T19:29:15.888 INFO:tasks.workunit.client.0.smithi022.stdout:8/700: chown d3/d12/d2b/d46/f91 240665703 1 2024-02-16T19:29:15.889 INFO:tasks.workunit.client.0.smithi022.stdout:8/701: fsync d3/d12/f8b 0 2024-02-16T19:29:15.890 INFO:tasks.workunit.client.0.smithi022.stdout:3/641: rename d7/d12/d18/dac/f48 to d7/d12/d18/dd/d31/fed 0 2024-02-16T19:29:15.890 INFO:tasks.workunit.client.1.smithi117.stdout:9/639: dwrite dd/d47/d5b/d5f/f65 [0,4194304] 0 2024-02-16T19:29:15.890 INFO:tasks.workunit.client.1.smithi117.stdout:8/773: dread - d4/d7/da6/d3a/d65/d96/fcb zero size 2024-02-16T19:29:15.891 INFO:tasks.workunit.client.1.smithi117.stdout:1/662: chown d2/d1d/d1f/c3d 11 1 2024-02-16T19:29:15.891 INFO:tasks.workunit.client.0.smithi022.stdout:3/642: chown d7/d12/d18/dd/d24/d58/d70/f7b 5584455 1 2024-02-16T19:29:15.894 INFO:tasks.workunit.client.1.smithi117.stdout:0/768: unlink d0/d9/d14/d23/c9b 0 2024-02-16T19:29:15.897 INFO:tasks.workunit.client.1.smithi117.stdout:0/769: chown d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b 4049652 1 2024-02-16T19:29:15.898 INFO:tasks.workunit.client.1.smithi117.stdout:2/774: creat d0/d3/d22/d37/d9d/ff6 x:0 0 0 2024-02-16T19:29:15.901 INFO:tasks.workunit.client.1.smithi117.stdout:9/640: dwrite dd/db5/dc4/dcf/fd6 [0,4194304] 0 2024-02-16T19:29:15.902 INFO:tasks.workunit.client.0.smithi022.stdout:3/643: read - d7/f3b zero size 2024-02-16T19:29:15.903 INFO:tasks.workunit.client.0.smithi022.stdout:3/644: chown d7/d12/d18/dd/d3c/d5a/fc7 78317 1 2024-02-16T19:29:15.904 INFO:tasks.workunit.client.0.smithi022.stdout:3/645: chown d7/d21/f5d 4031 1 2024-02-16T19:29:15.905 INFO:tasks.workunit.client.1.smithi117.stdout:8/774: fsync d4/d7/d6a/fe4 0 2024-02-16T19:29:15.906 INFO:tasks.workunit.client.0.smithi022.stdout:6/699: write d1/d8/f6f [664698,2215] 0 2024-02-16T19:29:15.907 INFO:tasks.workunit.client.1.smithi117.stdout:9/641: dread dd/d45/f4f [0,4194304] 0 2024-02-16T19:29:15.907 INFO:tasks.workunit.client.1.smithi117.stdout:8/775: write d4/d73/fdb [674795,81842] 0 2024-02-16T19:29:15.912 INFO:tasks.workunit.client.1.smithi117.stdout:4/713: truncate d0/da/d21/d3d/d28/d64/d90/f9b 364998 0 2024-02-16T19:29:15.913 INFO:tasks.workunit.client.1.smithi117.stdout:4/714: chown d0/da/d21/d3d/d28/d66/da2/dac/fe6 10115 1 2024-02-16T19:29:15.914 INFO:tasks.workunit.client.1.smithi117.stdout:0/770: truncate d0/d9/d2c/da9/faa 1055201 0 2024-02-16T19:29:15.915 INFO:tasks.workunit.client.0.smithi022.stdout:8/702: getdents d3/d12/d17/d64 0 2024-02-16T19:29:15.916 INFO:tasks.workunit.client.0.smithi022.stdout:8/703: truncate d3/d25/f61 1157571 0 2024-02-16T19:29:15.917 INFO:tasks.workunit.client.1.smithi117.stdout:7/737: symlink d4/d1c/d98/def/l110 0 2024-02-16T19:29:15.917 INFO:tasks.workunit.client.1.smithi117.stdout:5/781: dread d2/d4c/d4f/f81 [0,4194304] 0 2024-02-16T19:29:15.918 INFO:tasks.workunit.client.1.smithi117.stdout:7/738: write d4/d1c/d87/d89/d60/d79/fd7 [2621442,23475] 0 2024-02-16T19:29:15.924 INFO:tasks.workunit.client.0.smithi022.stdout:8/704: creat d3/ff9 x:0 0 0 2024-02-16T19:29:15.926 INFO:tasks.workunit.client.0.smithi022.stdout:8/705: write d3/d12/d17/d64/d8f/fec [107878,32715] 0 2024-02-16T19:29:15.926 INFO:tasks.workunit.client.1.smithi117.stdout:9/642: truncate dd/d45/d31/d63/d8e/f96 843065 0 2024-02-16T19:29:15.926 INFO:tasks.workunit.client.0.smithi022.stdout:8/706: chown d3/d12/d92/cb4 155 1 2024-02-16T19:29:15.927 INFO:tasks.workunit.client.1.smithi117.stdout:9/643: write dd/d45/d31/d63/d8e/dbb/fc9 [546021,66292] 0 2024-02-16T19:29:15.928 INFO:tasks.workunit.client.1.smithi117.stdout:4/715: unlink d0/da/d21/d3d/d28/d66/ld8 0 2024-02-16T19:29:15.934 INFO:tasks.workunit.client.1.smithi117.stdout:5/782: unlink d2/d5d/ca4 0 2024-02-16T19:29:15.935 INFO:tasks.workunit.client.0.smithi022.stdout:3/646: dwrite f2 [0,4194304] 0 2024-02-16T19:29:15.937 INFO:tasks.workunit.client.0.smithi022.stdout:3/647: chown d7/d12/d18/dd/d3c/fa9 12113745 1 2024-02-16T19:29:15.937 INFO:tasks.workunit.client.0.smithi022.stdout:8/707: creat d3/d12/da6/ffa x:0 0 0 2024-02-16T19:29:15.937 INFO:tasks.workunit.client.0.smithi022.stdout:3/648: readlink d7/d12/d18/l10 0 2024-02-16T19:29:15.938 INFO:tasks.workunit.client.1.smithi117.stdout:9/644: dread dd/db5/f32 [0,4194304] 0 2024-02-16T19:29:15.938 INFO:tasks.workunit.client.0.smithi022.stdout:6/700: write d1/d8/da/d17/d85/d56/fc4 [224918,91279] 0 2024-02-16T19:29:15.939 INFO:tasks.workunit.client.1.smithi117.stdout:9/645: read dd/db5/dc4/dcf/fd6 [3841244,88333] 0 2024-02-16T19:29:15.939 INFO:tasks.workunit.client.1.smithi117.stdout:1/663: link d2/dab/caf d2/d4/dbb/d7f/ce8 0 2024-02-16T19:29:15.940 INFO:tasks.workunit.client.1.smithi117.stdout:4/716: rmdir d0/da/d21/dc2 39 2024-02-16T19:29:15.941 INFO:tasks.workunit.client.0.smithi022.stdout:6/701: read d1/d8/da/d17/d53/f5c [1033139,69344] 0 2024-02-16T19:29:15.942 INFO:tasks.workunit.client.1.smithi117.stdout:4/717: write d0/da/d21/d3d/d28/d66/d97/f41 [4087785,45794] 0 2024-02-16T19:29:15.947 INFO:tasks.workunit.client.0.smithi022.stdout:3/649: truncate d7/f3b 331887 0 2024-02-16T19:29:15.950 INFO:tasks.workunit.client.1.smithi117.stdout:5/783: fsync d2/d4c/d4f/f81 0 2024-02-16T19:29:15.952 INFO:tasks.workunit.client.1.smithi117.stdout:7/739: fsync d4/d13/da3/fb1 0 2024-02-16T19:29:15.952 INFO:tasks.workunit.client.1.smithi117.stdout:8/776: write d4/f3f [1579,64811] 0 2024-02-16T19:29:15.953 INFO:tasks.workunit.client.0.smithi022.stdout:3/650: rmdir d7/d12/d18/dd/d24/d53 39 2024-02-16T19:29:15.954 INFO:tasks.workunit.client.0.smithi022.stdout:3/651: dread - d7/d12/d18/dd/d24/d58/d70/fa7 zero size 2024-02-16T19:29:15.955 INFO:tasks.workunit.client.0.smithi022.stdout:8/708: write d3/d12/da6/f1f [3608436,123227] 0 2024-02-16T19:29:15.956 INFO:tasks.workunit.client.1.smithi117.stdout:0/771: dread d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/d27/d4e/f61 [0,4194304] 0 2024-02-16T19:29:15.961 INFO:tasks.workunit.client.0.smithi022.stdout:6/702: dwrite d1/f72 [0,4194304] 0 2024-02-16T19:29:15.961 INFO:tasks.workunit.client.1.smithi117.stdout:9/646: creat dd/d45/d31/d63/fdb x:0 0 0 2024-02-16T19:29:15.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:15 smithi022 ceph-mon[29221]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:29:15.965 INFO:tasks.workunit.client.1.smithi117.stdout:4/718: symlink d0/da/d21/d3d/d28/d66/leb 0 2024-02-16T19:29:15.965 INFO:tasks.workunit.client.0.smithi022.stdout:8/709: dwrite d3/d12/d2b/da9/fe4 [0,4194304] 0 2024-02-16T19:29:15.968 INFO:tasks.workunit.client.1.smithi117.stdout:2/775: getdents d0/d3/d22 0 2024-02-16T19:29:15.969 INFO:tasks.workunit.client.0.smithi022.stdout:6/703: truncate d1/d8/ff 318311 0 2024-02-16T19:29:15.969 INFO:tasks.workunit.client.1.smithi117.stdout:2/776: chown d0/d3/d34/d94/d46/d68/d93 1 1 2024-02-16T19:29:15.970 INFO:tasks.workunit.client.0.smithi022.stdout:3/652: write d7/d12/d18/d4b/f4d [841676,67272] 0 2024-02-16T19:29:15.972 INFO:tasks.workunit.client.1.smithi117.stdout:4/719: dwrite d0/da/d21/d3d/d28/d66/d97/f41 [0,4194304] 0 2024-02-16T19:29:15.972 INFO:tasks.workunit.client.0.smithi022.stdout:3/653: truncate d7/d12/d18/d4b/f4d 973389 0 2024-02-16T19:29:15.974 INFO:tasks.workunit.client.0.smithi022.stdout:8/710: dread - d3/d12/d2b/fc3 zero size 2024-02-16T19:29:15.976 INFO:tasks.workunit.client.0.smithi022.stdout:8/711: dread d3/d12/da6/f2f [0,4194304] 0 2024-02-16T19:29:15.977 INFO:tasks.workunit.client.1.smithi117.stdout:2/777: read d0/d3/d34/d94/d46/d68/d76/da1/f2e [1073466,85856] 0 2024-02-16T19:29:15.977 INFO:tasks.workunit.client.1.smithi117.stdout:7/740: symlink d4/d13/da3/l111 0 2024-02-16T19:29:15.980 INFO:tasks.workunit.client.1.smithi117.stdout:8/777: symlink d4/d12/d31/lf9 0 2024-02-16T19:29:15.981 INFO:tasks.workunit.client.0.smithi022.stdout:8/712: mknod d3/d25/cfb 0 2024-02-16T19:29:15.983 INFO:tasks.workunit.client.0.smithi022.stdout:8/713: write d3/f8d [515098,110068] 0 2024-02-16T19:29:15.984 INFO:tasks.workunit.client.1.smithi117.stdout:9/647: rmdir dd/d23/daf 39 2024-02-16T19:29:15.985 INFO:tasks.workunit.client.1.smithi117.stdout:5/784: dwrite d2/d4c/d4f/d67/fd5 [0,4194304] 0 2024-02-16T19:29:15.989 INFO:tasks.workunit.client.1.smithi117.stdout:4/720: unlink d0/da/d21/d3d/d28/d66/d97/f94 0 2024-02-16T19:29:15.990 INFO:tasks.workunit.client.0.smithi022.stdout:6/704: write d1/d8/d3c/d77/f80 [45508,80289] 0 2024-02-16T19:29:15.994 INFO:tasks.workunit.client.1.smithi117.stdout:2/778: mkdir d0/d3/d34/d94/d82/df7 0 2024-02-16T19:29:15.997 INFO:tasks.workunit.client.1.smithi117.stdout:5/785: dwrite d2/d3/d21/d49/d59/fb8 [0,4194304] 0 2024-02-16T19:29:15.997 INFO:tasks.workunit.client.1.smithi117.stdout:7/741: mkdir d4/d1c/d112 0 2024-02-16T19:29:15.999 INFO:tasks.workunit.client.1.smithi117.stdout:8/778: mkdir d4/d7/dfa 0 2024-02-16T19:29:16.001 INFO:tasks.workunit.client.1.smithi117.stdout:7/742: write d4/d13/fb0 [438003,34489] 0 2024-02-16T19:29:16.002 INFO:tasks.workunit.client.1.smithi117.stdout:4/721: rename d0/da/d21/d3d/d28/d64/d73/d91/d37/fd1 to d0/da/d21/d3d/d28/d66/da2/dac/fec 0 2024-02-16T19:29:16.003 INFO:tasks.workunit.client.1.smithi117.stdout:0/772: dwrite d0/d78/fca [0,4194304] 0 2024-02-16T19:29:16.004 INFO:tasks.workunit.client.1.smithi117.stdout:5/786: mknod d2/d9/d3c/d24/c10c 0 2024-02-16T19:29:16.005 INFO:tasks.workunit.client.1.smithi117.stdout:9/648: link dd/d23/l36 dd/d45/d31/d63/d8e/dbb/ldc 0 2024-02-16T19:29:16.005 INFO:tasks.workunit.client.1.smithi117.stdout:4/722: dread - d0/da/d21/d3d/d28/f86 zero size 2024-02-16T19:29:16.006 INFO:tasks.workunit.client.0.smithi022.stdout:3/654: dread d7/d12/d18/dd/d24/d58/d6b/f89 [0,4194304] 0 2024-02-16T19:29:16.008 INFO:tasks.workunit.client.0.smithi022.stdout:3/655: chown d7/d12/d18/dd/d24/d58/l72 12 1 2024-02-16T19:29:16.009 INFO:tasks.workunit.client.0.smithi022.stdout:6/705: dread d1/f32 [0,4194304] 0 2024-02-16T19:29:16.010 INFO:tasks.workunit.client.1.smithi117.stdout:0/773: fsync d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/dbd/fda 0 2024-02-16T19:29:16.010 INFO:tasks.workunit.client.1.smithi117.stdout:4/723: stat d0/da/d21/d3d/d28/d66/d97/ce2 0 2024-02-16T19:29:16.010 INFO:tasks.workunit.client.1.smithi117.stdout:7/743: mkdir d4/d1c/d39/d46/d55/d113 0 2024-02-16T19:29:16.012 INFO:tasks.workunit.client.1.smithi117.stdout:1/664: write d2/f5b [518371,53866] 0 2024-02-16T19:29:16.013 INFO:tasks.workunit.client.0.smithi022.stdout:3/656: rmdir d7/d12/d18/dd/d31/d7f 39 2024-02-16T19:29:16.015 INFO:tasks.workunit.client.0.smithi022.stdout:8/714: write d3/d12/d2b/d46/f53 [3776444,96323] 0 2024-02-16T19:29:16.017 INFO:tasks.workunit.client.0.smithi022.stdout:6/706: mkdir d1/d8/da/d17/d53/d7f/da3/dfa 0 2024-02-16T19:29:16.019 INFO:tasks.workunit.client.1.smithi117.stdout:5/787: dwrite d2/d3/d21/d2a/dcb/def/ffe [0,4194304] 0 2024-02-16T19:29:16.019 INFO:tasks.workunit.client.0.smithi022.stdout:3/657: rmdir d7/d12 39 2024-02-16T19:29:16.020 INFO:tasks.workunit.client.1.smithi117.stdout:3/829: sync 2024-02-16T19:29:16.020 INFO:tasks.workunit.client.1.smithi117.stdout:6/726: sync 2024-02-16T19:29:16.023 INFO:tasks.workunit.client.0.smithi022.stdout:8/715: dwrite d3/d12/d17/fbe [0,4194304] 0 2024-02-16T19:29:16.023 INFO:tasks.workunit.client.1.smithi117.stdout:1/665: symlink d2/d19/d59/d89/d69/le9 0 2024-02-16T19:29:16.025 INFO:tasks.workunit.client.1.smithi117.stdout:1/666: chown d2/f57 7076050 1 2024-02-16T19:29:16.026 INFO:tasks.workunit.client.1.smithi117.stdout:7/744: dwrite d4/dd3/fde [0,4194304] 0 2024-02-16T19:29:16.029 INFO:tasks.workunit.client.0.smithi022.stdout:8/716: dread d3/d12/d17/d64/f73 [0,4194304] 0 2024-02-16T19:29:16.029 INFO:tasks.workunit.client.0.smithi022.stdout:6/707: rename d1/d8/da/d17/d53/d7f/db3/de8 to d1/d8/da/d17/d48/de4/dfb 0 2024-02-16T19:29:16.032 INFO:tasks.workunit.client.1.smithi117.stdout:9/649: rename dd/d45/d31/l3d to dd/d23/daf/ldd 0 2024-02-16T19:29:16.035 INFO:tasks.workunit.client.1.smithi117.stdout:0/774: dread d0/fe [0,4194304] 0 2024-02-16T19:29:16.038 INFO:tasks.workunit.client.0.smithi022.stdout:8/717: rmdir d3/d12/d17 39 2024-02-16T19:29:16.039 INFO:tasks.workunit.client.1.smithi117.stdout:2/779: truncate d0/d3/d22/f84 3131426 0 2024-02-16T19:29:16.041 INFO:tasks.workunit.client.0.smithi022.stdout:8/718: chown d3/d12/d2b/d54/l60 191109767 1 2024-02-16T19:29:16.042 INFO:tasks.workunit.client.1.smithi117.stdout:5/788: rmdir d2/d9/d3c/d24/d50/d52/d74 39 2024-02-16T19:29:16.043 INFO:tasks.workunit.client.0.smithi022.stdout:6/708: mknod d1/d8/d3c/cfc 0 2024-02-16T19:29:16.044 INFO:tasks.workunit.client.1.smithi117.stdout:3/830: dread d1/d30/d52/fa4 [0,4194304] 0 2024-02-16T19:29:16.044 INFO:tasks.workunit.client.1.smithi117.stdout:4/724: symlink d0/da/d21/d3d/d28/d66/d97/led 0 2024-02-16T19:29:16.044 INFO:tasks.workunit.client.0.smithi022.stdout:6/709: chown d1/d8/da/d29/f35 0 1 2024-02-16T19:29:16.046 INFO:tasks.workunit.client.1.smithi117.stdout:7/745: unlink d4/d1c/d39/d46/d47/fd9 0 2024-02-16T19:29:16.050 INFO:tasks.workunit.client.1.smithi117.stdout:8/779: getdents d4/d7 0 2024-02-16T19:29:16.050 INFO:tasks.workunit.client.0.smithi022.stdout:8/719: fdatasync d3/d12/d17/d64/d8f/fb6 0 2024-02-16T19:29:16.051 INFO:tasks.workunit.client.0.smithi022.stdout:6/710: dread d1/d8/ff [0,4194304] 0 2024-02-16T19:29:16.051 INFO:tasks.workunit.client.0.smithi022.stdout:8/720: stat d3/d12/d6a/c77 0 2024-02-16T19:29:16.053 INFO:tasks.workunit.client.0.smithi022.stdout:6/711: write d1/d8/da/d17/d53/d7f/fd3 [4743937,35996] 0 2024-02-16T19:29:16.055 INFO:tasks.workunit.client.1.smithi117.stdout:6/727: symlink d5/leb 0 2024-02-16T19:29:16.057 INFO:tasks.workunit.client.1.smithi117.stdout:2/780: fdatasync d0/d13/d75/fac 0 2024-02-16T19:29:16.057 INFO:tasks.workunit.client.0.smithi022.stdout:3/658: dread f2 [4194304,4194304] 0 2024-02-16T19:29:16.058 INFO:tasks.workunit.client.0.smithi022.stdout:8/721: creat d3/d12/d2b/d54/d97/ffc x:0 0 0 2024-02-16T19:29:16.058 INFO:tasks.workunit.client.0.smithi022.stdout:8/722: readlink d3/d12/d2b/d2e/l85 0 2024-02-16T19:29:16.059 INFO:tasks.workunit.client.0.smithi022.stdout:8/723: chown d3/ff9 0 1 2024-02-16T19:29:16.059 INFO:tasks.workunit.client.1.smithi117.stdout:5/789: truncate d2/d9/d3c/f1a 465553 0 2024-02-16T19:29:16.060 INFO:tasks.workunit.client.0.smithi022.stdout:8/724: readlink d3/d25/d33/l44 0 2024-02-16T19:29:16.060 INFO:tasks.workunit.client.1.smithi117.stdout:7/746: dread d4/d1c/fbe [0,4194304] 0 2024-02-16T19:29:16.061 INFO:tasks.workunit.client.0.smithi022.stdout:8/725: chown d3/d12/d2b/d54/d74/f82 10122 1 2024-02-16T19:29:16.061 INFO:tasks.workunit.client.1.smithi117.stdout:7/747: stat d4/d13/d63/l69 0 2024-02-16T19:29:16.062 INFO:tasks.workunit.client.1.smithi117.stdout:7/748: truncate d4/d1c/d39/d46/d55/db8/fc2 4446052 0 2024-02-16T19:29:16.063 INFO:tasks.workunit.client.0.smithi022.stdout:6/712: chown d1/d8/da/d17/d85/d9c/fce 56 1 2024-02-16T19:29:16.066 INFO:tasks.workunit.client.0.smithi022.stdout:3/659: readlink d7/d12/d18/dd/d24/d58/l83 0 2024-02-16T19:29:16.066 INFO:tasks.workunit.client.0.smithi022.stdout:8/726: dwrite d3/d12/da6/f1f [0,4194304] 0 2024-02-16T19:29:16.069 INFO:tasks.workunit.client.1.smithi117.stdout:9/650: truncate dd/d45/d31/d63/d8e/f96 54135 0 2024-02-16T19:29:16.074 INFO:tasks.workunit.client.0.smithi022.stdout:6/713: rename d1/d8/da/d17 to d1/d8/da/d29/deb/dfd 0 2024-02-16T19:29:16.076 INFO:tasks.workunit.client.1.smithi117.stdout:8/780: rename d4/d7/da6/d3a/d43/d44/f62 to d4/d7/d6a/d72/d84/dda/ffb 0 2024-02-16T19:29:16.079 INFO:tasks.workunit.client.1.smithi117.stdout:0/775: creat d0/d9/d14/d1d/d60/dc3/dfd/f10b x:0 0 0 2024-02-16T19:29:16.079 INFO:tasks.workunit.client.0.smithi022.stdout:8/727: creat d3/d12/d2b/da9/ffd x:0 0 0 2024-02-16T19:29:16.080 INFO:tasks.workunit.client.1.smithi117.stdout:6/728: mkdir d5/d4b/d71/dc7/d72/dec 0 2024-02-16T19:29:16.083 INFO:tasks.workunit.client.1.smithi117.stdout:2/781: mknod d0/d3b/cf8 0 2024-02-16T19:29:16.086 INFO:tasks.workunit.client.1.smithi117.stdout:3/831: mkdir d1/dd/d3e/d7d/d121 0 2024-02-16T19:29:16.090 INFO:tasks.workunit.client.0.smithi022.stdout:3/660: dwrite d7/d21/f5d [0,4194304] 0 2024-02-16T19:29:16.090 INFO:tasks.workunit.client.1.smithi117.stdout:4/725: truncate d0/da/d21/d3d/d28/d64/d90/f9b 1023543 0 2024-02-16T19:29:16.092 INFO:tasks.workunit.client.1.smithi117.stdout:9/651: creat dd/d47/d5b/d5f/fde x:0 0 0 2024-02-16T19:29:16.093 INFO:tasks.workunit.client.0.smithi022.stdout:6/714: dread d1/d8/f11 [0,4194304] 0 2024-02-16T19:29:16.094 INFO:tasks.workunit.client.0.smithi022.stdout:3/661: dread d7/d21/f38 [0,4194304] 0 2024-02-16T19:29:16.097 INFO:tasks.workunit.client.1.smithi117.stdout:8/781: creat d4/d7/da6/ffc x:0 0 0 2024-02-16T19:29:16.103 INFO:tasks.workunit.client.1.smithi117.stdout:0/776: creat d0/d9/d14/d1d/d60/d65/f10c x:0 0 0 2024-02-16T19:29:16.103 INFO:tasks.workunit.client.0.smithi022.stdout:8/728: dwrite d3/d12/d17/f9d [0,4194304] 0 2024-02-16T19:29:16.106 INFO:tasks.workunit.client.1.smithi117.stdout:6/729: mkdir d5/d4b/d71/dc7/dce/ded 0 2024-02-16T19:29:16.108 INFO:tasks.workunit.client.1.smithi117.stdout:2/782: readlink d0/l1a 0 2024-02-16T19:29:16.109 INFO:tasks.workunit.client.0.smithi022.stdout:6/715: creat d1/d8/d23/dc6/ddb/ffe x:0 0 0 2024-02-16T19:29:16.111 INFO:tasks.workunit.client.1.smithi117.stdout:5/790: truncate d2/d9/d3c/d24/d90/dd4/dfb/d62/fe9 1285727 0 2024-02-16T19:29:16.111 INFO:tasks.workunit.client.0.smithi022.stdout:2/661: sync 2024-02-16T19:29:16.111 INFO:tasks.workunit.client.0.smithi022.stdout:4/609: sync 2024-02-16T19:29:16.113 INFO:tasks.workunit.client.0.smithi022.stdout:4/610: chown d4/d6/d3a 21569337 1 2024-02-16T19:29:16.113 INFO:tasks.workunit.client.0.smithi022.stdout:3/662: dread d7/d21/f7c [0,4194304] 0 2024-02-16T19:29:16.114 INFO:tasks.workunit.client.1.smithi117.stdout:3/832: unlink d1/f8 0 2024-02-16T19:29:16.115 INFO:tasks.workunit.client.0.smithi022.stdout:3/663: write d7/d12/d18/dd/d24/fd0 [290179,36137] 0 2024-02-16T19:29:16.118 INFO:tasks.workunit.client.1.smithi117.stdout:7/749: mknod d4/d13/dfd/c114 0 2024-02-16T19:29:16.119 INFO:tasks.workunit.client.1.smithi117.stdout:7/750: write d4/d13/d63/d71/f90 [4716796,120987] 0 2024-02-16T19:29:16.123 INFO:tasks.workunit.client.0.smithi022.stdout:3/664: dwrite d7/d12/d18/dd/fcb [0,4194304] 0 2024-02-16T19:29:16.125 INFO:tasks.workunit.client.1.smithi117.stdout:4/726: rename d0/da/d21/d3d/d28/d64/d73/d91/d9c/fd6 to d0/da/d21/d3d/d28/d64/d73/d91/d37/d98/fee 0 2024-02-16T19:29:16.128 INFO:tasks.workunit.client.1.smithi117.stdout:9/652: read dd/db5/f32 [1147864,66952] 0 2024-02-16T19:29:16.128 INFO:tasks.workunit.client.1.smithi117.stdout:4/727: dread - d0/da/d21/d3d/d28/d64/d73/d91/d37/d98/fe7 zero size 2024-02-16T19:29:16.129 INFO:tasks.workunit.client.0.smithi022.stdout:3/665: dread d7/d12/d18/dd/f7d [0,4194304] 0 2024-02-16T19:29:16.129 INFO:tasks.workunit.client.1.smithi117.stdout:9/653: dread - dd/d47/d5b/d5f/fc5 zero size 2024-02-16T19:29:16.130 INFO:tasks.workunit.client.1.smithi117.stdout:8/782: mkdir d4/d7/da6/dfd 0 2024-02-16T19:29:16.134 INFO:tasks.workunit.client.0.smithi022.stdout:8/729: write d3/d12/d17/fc6 [540459,54624] 0 2024-02-16T19:29:16.136 INFO:tasks.workunit.client.1.smithi117.stdout:6/730: rmdir d5/db 39 2024-02-16T19:29:16.137 INFO:tasks.workunit.client.0.smithi022.stdout:3/666: symlink d7/d12/d18/dd/d24/d58/lee 0 2024-02-16T19:29:16.137 INFO:tasks.workunit.client.0.smithi022.stdout:6/716: write d1/d8/da/d29/deb/dfd/d48/d51/f59 [1246182,67280] 0 2024-02-16T19:29:16.139 INFO:tasks.workunit.client.0.smithi022.stdout:6/717: dread - d1/d8/dac/ff9 zero size 2024-02-16T19:29:16.139 INFO:tasks.workunit.client.1.smithi117.stdout:2/783: mkdir d0/d3/d22/d37/d9d/df9 0 2024-02-16T19:29:16.140 INFO:tasks.workunit.client.1.smithi117.stdout:2/784: read - d0/d3/d34/d94/d46/d68/d76/da1/d56/fcc zero size 2024-02-16T19:29:16.140 INFO:tasks.workunit.client.1.smithi117.stdout:5/791: rmdir d2/d9/d3c/d24/d90/dd4/dfb/d62 39 2024-02-16T19:29:16.141 INFO:tasks.workunit.client.0.smithi022.stdout:3/667: mkdir d7/d12/d18/dd/d24/d93/de9/def 0 2024-02-16T19:29:16.144 INFO:tasks.workunit.client.1.smithi117.stdout:2/785: dread d0/d13/d75/fac [0,4194304] 0 2024-02-16T19:29:16.144 INFO:tasks.workunit.client.0.smithi022.stdout:6/718: chown d1/d8/da/d29/deb/dfd/d85/d5b/fc9 31412 1 2024-02-16T19:29:16.145 INFO:tasks.workunit.client.0.smithi022.stdout:8/730: creat d3/ffe x:0 0 0 2024-02-16T19:29:16.147 INFO:tasks.workunit.client.0.smithi022.stdout:3/668: chown d7/d12/d18/dd/d31/fed 2 1 2024-02-16T19:29:16.148 INFO:tasks.workunit.client.0.smithi022.stdout:6/719: creat d1/d8/d3c/d4d/fff x:0 0 0 2024-02-16T19:29:16.149 INFO:tasks.workunit.client.1.smithi117.stdout:1/667: sync 2024-02-16T19:29:16.149 INFO:tasks.workunit.client.1.smithi117.stdout:1/668: readlink d2/d4/dbb/d7f/lc9 0 2024-02-16T19:29:16.153 INFO:tasks.workunit.client.1.smithi117.stdout:8/783: rename d4/d7/da6/d3a/d65/d96/c97 to d4/d12/d31/d8e/cfe 0 2024-02-16T19:29:16.153 INFO:tasks.workunit.client.0.smithi022.stdout:4/611: write d4/d6/f87 [885190,26383] 0 2024-02-16T19:29:16.156 INFO:tasks.workunit.client.0.smithi022.stdout:6/720: creat d1/d8/da/d7b/dae/db6/f100 x:0 0 0 2024-02-16T19:29:16.158 INFO:tasks.workunit.client.1.smithi117.stdout:0/777: truncate d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/d27/f42 204922 0 2024-02-16T19:29:16.159 INFO:tasks.workunit.client.1.smithi117.stdout:3/833: write d1/d30/d52/fa4 [290010,18667] 0 2024-02-16T19:29:16.160 INFO:tasks.workunit.client.1.smithi117.stdout:6/731: mkdir d5/d4b/d71/dc7/d72/dee 0 2024-02-16T19:29:16.160 INFO:tasks.workunit.client.1.smithi117.stdout:0/778: chown d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/db7/cf3 9103 1 2024-02-16T19:29:16.160 INFO:tasks.workunit.client.0.smithi022.stdout:3/669: creat d7/d12/d18/dd/d31/dbf/dc5/ff0 x:0 0 0 2024-02-16T19:29:16.161 INFO:tasks.workunit.client.1.smithi117.stdout:0/779: write d0/f53 [4794675,41551] 0 2024-02-16T19:29:16.162 INFO:tasks.workunit.client.1.smithi117.stdout:0/780: chown d0/d9/d14/d1d/d60/d65/fe3 38717 1 2024-02-16T19:29:16.164 INFO:tasks.workunit.client.1.smithi117.stdout:9/654: write dd/d45/f58 [958840,129666] 0 2024-02-16T19:29:16.165 INFO:tasks.workunit.client.1.smithi117.stdout:5/792: creat d2/d36/f10d x:0 0 0 2024-02-16T19:29:16.166 INFO:tasks.workunit.client.1.smithi117.stdout:5/793: stat d2/d9/d3c/c3a 0 2024-02-16T19:29:16.167 INFO:tasks.workunit.client.1.smithi117.stdout:5/794: chown d2/d9/d3c/d24/d50/d96/dbc/d30/d38/fdd 2 1 2024-02-16T19:29:16.168 INFO:tasks.workunit.client.0.smithi022.stdout:6/721: fsync d1/d8/da/d29/deb/dfd/d53/d7f/fe9 0 2024-02-16T19:29:16.169 INFO:tasks.workunit.client.1.smithi117.stdout:5/795: chown d2/d9/d3c/d24/d50/d96/dbc/d30/d38/d51/ca3 7657267 1 2024-02-16T19:29:16.171 INFO:tasks.workunit.client.0.smithi022.stdout:8/731: link d3/d12/d2b/d2e/f59 d3/d12/d2b/da9/dd5/fff 0 2024-02-16T19:29:16.171 INFO:tasks.workunit.client.0.smithi022.stdout:8/732: readlink d3/d12/d2b/l4e 0 2024-02-16T19:29:16.172 INFO:tasks.workunit.client.0.smithi022.stdout:8/733: chown d3/d12/d6a/c77 1388294465 1 2024-02-16T19:29:16.176 INFO:tasks.workunit.client.1.smithi117.stdout:0/781: dread d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/d27/f5f [0,4194304] 0 2024-02-16T19:29:16.178 INFO:tasks.workunit.client.1.smithi117.stdout:2/786: write d0/d3/d34/d94/d46/f83 [628625,114713] 0 2024-02-16T19:29:16.179 INFO:tasks.workunit.client.1.smithi117.stdout:7/751: truncate d4/d1c/d39/d46/d55/db8/fc2 3362061 0 2024-02-16T19:29:16.180 INFO:tasks.workunit.client.0.smithi022.stdout:6/722: write d1/d8/da/d29/deb/dfd/d53/d7f/fe9 [1187463,86971] 0 2024-02-16T19:29:16.181 INFO:tasks.workunit.client.1.smithi117.stdout:1/669: mkdir d2/d19/d59/d89/d69/d80/d8e/dea 0 2024-02-16T19:29:16.183 INFO:tasks.workunit.client.0.smithi022.stdout:8/734: read d3/d12/d2b/f3f [173033,114787] 0 2024-02-16T19:29:16.185 INFO:tasks.workunit.client.1.smithi117.stdout:3/834: unlink d1/dd/d5c/fa1 0 2024-02-16T19:29:16.185 INFO:tasks.workunit.client.1.smithi117.stdout:7/752: dwrite d4/d13/d63/ddd/f10b [0,4194304] 0 2024-02-16T19:29:16.186 INFO:tasks.workunit.client.0.smithi022.stdout:8/735: truncate d3/d12/da6/ff3 338166 0 2024-02-16T19:29:16.187 INFO:tasks.workunit.client.0.smithi022.stdout:4/612: creat d4/d6/fbe x:0 0 0 2024-02-16T19:29:16.188 INFO:tasks.workunit.client.0.smithi022.stdout:6/723: dwrite d1/d8/d23/dc6/ddb/ffe [0,4194304] 0 2024-02-16T19:29:16.189 INFO:tasks.workunit.client.0.smithi022.stdout:8/736: chown d3/d12/da6/f27 569465 1 2024-02-16T19:29:16.190 INFO:tasks.workunit.client.1.smithi117.stdout:8/784: write d4/d12/d2a/d64/dc6/fae [940427,48470] 0 2024-02-16T19:29:16.193 INFO:tasks.workunit.client.0.smithi022.stdout:3/670: rename d7/d12/d18/dd/d31/c50 to d7/d12/d18/cf1 0 2024-02-16T19:29:16.195 INFO:tasks.workunit.client.1.smithi117.stdout:9/655: dread dd/d45/f42 [0,4194304] 0 2024-02-16T19:29:16.196 INFO:tasks.workunit.client.1.smithi117.stdout:5/796: mknod d2/d5d/d8e/c10e 0 2024-02-16T19:29:16.197 INFO:tasks.workunit.client.1.smithi117.stdout:9/656: dread dd/d45/f42 [0,4194304] 0 2024-02-16T19:29:16.199 INFO:tasks.workunit.client.1.smithi117.stdout:4/728: creat d0/da/d21/fef x:0 0 0 2024-02-16T19:29:16.202 INFO:tasks.workunit.client.0.smithi022.stdout:4/613: symlink d4/d6/dd/d22/d83/lbf 0 2024-02-16T19:29:16.203 INFO:tasks.workunit.client.0.smithi022.stdout:4/614: dread - d4/d38/d6a/fa9 zero size 2024-02-16T19:29:16.207 INFO:tasks.workunit.client.1.smithi117.stdout:1/670: rename d2/d19/d59/d89/d69/l70 to d2/d19/d59/d89/leb 0 2024-02-16T19:29:16.209 INFO:tasks.workunit.client.1.smithi117.stdout:3/835: rmdir d1/dd/d5c 39 2024-02-16T19:29:16.211 INFO:tasks.workunit.client.1.smithi117.stdout:7/753: fdatasync d4/d13/d63/f94 0 2024-02-16T19:29:16.212 INFO:tasks.workunit.client.0.smithi022.stdout:8/737: write d3/d12/d17/f50 [3385683,28083] 0 2024-02-16T19:29:16.213 INFO:tasks.workunit.client.1.smithi117.stdout:0/782: dread d0/d30/f64 [0,4194304] 0 2024-02-16T19:29:16.214 INFO:tasks.workunit.client.1.smithi117.stdout:6/732: creat d5/d4b/d71/dc7/d72/dec/fef x:0 0 0 2024-02-16T19:29:16.215 INFO:tasks.workunit.client.0.smithi022.stdout:6/724: getdents d1/d8/d81 0 2024-02-16T19:29:16.216 INFO:tasks.workunit.client.1.smithi117.stdout:8/785: truncate d4/d7/d6a/d72/fe5 186169 0 2024-02-16T19:29:16.217 INFO:tasks.workunit.client.0.smithi022.stdout:6/725: dread d1/f32 [0,4194304] 0 2024-02-16T19:29:16.218 INFO:tasks.workunit.client.0.smithi022.stdout:8/738: dwrite d3/d12/d2b/d46/f53 [0,4194304] 0 2024-02-16T19:29:16.219 INFO:tasks.workunit.client.0.smithi022.stdout:4/615: link d4/d6/dd/df/d10/d24/fbc d4/d38/fc0 0 2024-02-16T19:29:16.220 INFO:tasks.workunit.client.0.smithi022.stdout:3/671: getdents d7/d12/d18/dd/d31/dbf 0 2024-02-16T19:29:16.221 INFO:tasks.workunit.client.0.smithi022.stdout:4/616: fdatasync d4/d6/dd/df/d10/d24/fa6 0 2024-02-16T19:29:16.222 INFO:tasks.workunit.client.0.smithi022.stdout:8/739: dread d3/d12/da6/f28 [0,4194304] 0 2024-02-16T19:29:16.223 INFO:tasks.workunit.client.0.smithi022.stdout:8/740: stat d3/d12/d2b/d2e/l9b 0 2024-02-16T19:29:16.225 INFO:tasks.workunit.client.1.smithi117.stdout:1/671: creat d2/d19/d59/d89/fec x:0 0 0 2024-02-16T19:29:16.226 INFO:tasks.workunit.client.0.smithi022.stdout:6/726: rename d1/d8/da/d29/deb/dfd/d53/d7f to d1/d8/dac/d101 0 2024-02-16T19:29:16.227 INFO:tasks.workunit.client.1.smithi117.stdout:3/836: stat d1/dd/f48 0 2024-02-16T19:29:16.228 INFO:tasks.workunit.client.0.smithi022.stdout:6/727: chown d1/d8/da/d29/deb/dfd/faf 592219756 1 2024-02-16T19:29:16.229 INFO:tasks.workunit.client.1.smithi117.stdout:7/754: symlink d4/d13/d63/d71/l115 0 2024-02-16T19:29:16.230 INFO:tasks.workunit.client.0.smithi022.stdout:3/672: symlink d7/d21/lf2 0 2024-02-16T19:29:16.231 INFO:tasks.workunit.client.1.smithi117.stdout:6/733: fdatasync d5/db/f1f 0 2024-02-16T19:29:16.232 INFO:tasks.workunit.client.1.smithi117.stdout:8/786: mkdir d4/d73/dff 0 2024-02-16T19:29:16.233 INFO:tasks.workunit.client.1.smithi117.stdout:8/787: chown d4/d7/da6/dfd 0 1 2024-02-16T19:29:16.234 INFO:tasks.workunit.client.1.smithi117.stdout:9/657: symlink dd/db5/dc4/dcf/dda/ldf 0 2024-02-16T19:29:16.236 INFO:tasks.workunit.client.0.smithi022.stdout:3/673: chown d7/d12/d18/dd/d31/d7f/faf 373137955 1 2024-02-16T19:29:16.238 INFO:tasks.workunit.client.1.smithi117.stdout:4/729: dread d0/da/d21/d3d/d28/d64/f1f [0,4194304] 0 2024-02-16T19:29:16.240 INFO:tasks.workunit.client.1.smithi117.stdout:5/797: rename d2/d9/d3c/d24/d50/d96/dbc/d30/d95/c101 to d2/d4c/d69/c10f 0 2024-02-16T19:29:16.240 INFO:tasks.workunit.client.1.smithi117.stdout:9/658: dwrite dd/d47/d5b/d5f/f6f [0,4194304] 0 2024-02-16T19:29:16.243 INFO:tasks.workunit.client.0.smithi022.stdout:8/741: creat d3/d12/da6/f100 x:0 0 0 2024-02-16T19:29:16.244 INFO:tasks.workunit.client.1.smithi117.stdout:9/659: dread dd/db5/dc4/dcf/fd6 [0,4194304] 0 2024-02-16T19:29:16.246 INFO:tasks.workunit.client.1.smithi117.stdout:7/755: creat d4/d1c/d87/df0/dfe/f116 x:0 0 0 2024-02-16T19:29:16.248 INFO:tasks.workunit.client.0.smithi022.stdout:8/742: readlink d3/d25/ld6 0 2024-02-16T19:29:16.249 INFO:tasks.workunit.client.0.smithi022.stdout:6/728: write d1/d8/f66 [2854123,53954] 0 2024-02-16T19:29:16.249 INFO:tasks.workunit.client.0.smithi022.stdout:4/617: dwrite d4/d6/dd/f85 [0,4194304] 0 2024-02-16T19:29:16.252 INFO:tasks.workunit.client.1.smithi117.stdout:8/788: creat d4/d12/d31/d8e/dbf/f100 x:0 0 0 2024-02-16T19:29:16.252 INFO:tasks.workunit.client.1.smithi117.stdout:2/787: getdents d0/d3/d10 0 2024-02-16T19:29:16.257 INFO:tasks.workunit.client.0.smithi022.stdout:7/691: sync 2024-02-16T19:29:16.257 INFO:tasks.workunit.client.0.smithi022.stdout:0/864: sync 2024-02-16T19:29:16.257 INFO:tasks.workunit.client.0.smithi022.stdout:5/729: sync 2024-02-16T19:29:16.257 INFO:tasks.workunit.client.0.smithi022.stdout:1/719: sync 2024-02-16T19:29:16.257 INFO:tasks.workunit.client.0.smithi022.stdout:9/699: sync 2024-02-16T19:29:16.259 INFO:tasks.workunit.client.1.smithi117.stdout:2/788: dwrite d0/d3/d34/fc6 [0,4194304] 0 2024-02-16T19:29:16.260 INFO:tasks.workunit.client.0.smithi022.stdout:8/743: rename d3/d12/d17/le7 to d3/d12/d2b/d2e/l101 0 2024-02-16T19:29:16.260 INFO:tasks.workunit.client.0.smithi022.stdout:1/720: stat d6/d8/d22/f28 0 2024-02-16T19:29:16.261 INFO:tasks.workunit.client.1.smithi117.stdout:2/789: truncate d0/d13/d4d/d53/d58/dbc/fdb 516993 0 2024-02-16T19:29:16.261 INFO:tasks.workunit.client.0.smithi022.stdout:8/744: readlink d3/d12/d6a/lba 0 2024-02-16T19:29:16.263 INFO:tasks.workunit.client.1.smithi117.stdout:0/783: rename d0/d9/d14/d1d/d60/d76/d7b/f9c to d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/d27/d31/d94/f10d 0 2024-02-16T19:29:16.263 INFO:tasks.workunit.client.1.smithi117.stdout:8/789: rename d4/d12/d31 to d4/d12/d31/d8e/dbf/d101 22 2024-02-16T19:29:16.266 INFO:tasks.workunit.client.0.smithi022.stdout:4/618: mkdir d4/d6/dd/df/dc1 0 2024-02-16T19:29:16.266 INFO:tasks.workunit.client.1.smithi117.stdout:1/672: mknod d2/d19/d59/d89/d69/d80/d8e/dea/ced 0 2024-02-16T19:29:16.267 INFO:tasks.workunit.client.0.smithi022.stdout:7/692: mkdir d9/da/d2c/d46/de3 0 2024-02-16T19:29:16.268 INFO:tasks.workunit.client.0.smithi022.stdout:7/693: chown d9/da/d1f/d44 20427 1 2024-02-16T19:29:16.270 INFO:tasks.workunit.client.1.smithi117.stdout:3/837: truncate d1/dd/d5c/f6b 1407753 0 2024-02-16T19:29:16.271 INFO:tasks.workunit.client.0.smithi022.stdout:0/865: creat d10/d15/d23/d38/d5a/d66/f11c x:0 0 0 2024-02-16T19:29:16.271 INFO:tasks.workunit.client.1.smithi117.stdout:8/790: dwrite d4/d7/da6/d3a/d65/d96/fe8 [0,4194304] 0 2024-02-16T19:29:16.274 INFO:tasks.workunit.client.1.smithi117.stdout:5/798: dread d2/d3/f35 [0,4194304] 0 2024-02-16T19:29:16.274 INFO:tasks.workunit.client.1.smithi117.stdout:2/790: dread d0/d3/d34/d94/d46/d68/d76/da1/f2e [0,4194304] 0 2024-02-16T19:29:16.276 INFO:tasks.workunit.client.0.smithi022.stdout:9/700: creat db/d14/d16/d5d/d65/d6b/d89/dd2/ff0 x:0 0 0 2024-02-16T19:29:16.280 INFO:tasks.workunit.client.0.smithi022.stdout:9/701: truncate db/d14/d19/d1a/f24 4603983 0 2024-02-16T19:29:16.282 INFO:tasks.workunit.client.0.smithi022.stdout:8/745: symlink d3/d12/d2b/d54/db8/l102 0 2024-02-16T19:29:16.282 INFO:tasks.workunit.client.0.smithi022.stdout:6/729: write d1/d8/da/f14 [4877411,51604] 0 2024-02-16T19:29:16.294 INFO:tasks.workunit.client.1.smithi117.stdout:7/756: dwrite d4/f34 [0,4194304] 0 2024-02-16T19:29:16.294 INFO:tasks.workunit.client.0.smithi022.stdout:7/694: dread d9/da/d1f/d40/f61 [0,4194304] 0 2024-02-16T19:29:16.295 INFO:tasks.workunit.client.1.smithi117.stdout:1/673: fsync d2/f9d 0 2024-02-16T19:29:16.296 INFO:tasks.workunit.client.1.smithi117.stdout:1/674: write d2/d1d/f6d [3723900,125436] 0 2024-02-16T19:29:16.297 INFO:tasks.workunit.client.0.smithi022.stdout:1/721: fdatasync d6/d8/d22/f9a 0 2024-02-16T19:29:16.300 INFO:tasks.workunit.client.1.smithi117.stdout:1/675: dread d2/d4/dbb/d7f/f85 [0,4194304] 0 2024-02-16T19:29:16.304 INFO:tasks.workunit.client.1.smithi117.stdout:3/838: chown d1/dd/d8e/f94 1579322400 1 2024-02-16T19:29:16.304 INFO:tasks.workunit.client.1.smithi117.stdout:3/839: dread - d1/d30/d52/d56/fd5 zero size 2024-02-16T19:29:16.307 INFO:tasks.workunit.client.0.smithi022.stdout:5/730: dwrite dd/d1f/d48/d21/d2f/d52/d77/d95/fb8 [0,4194304] 0 2024-02-16T19:29:16.309 INFO:tasks.workunit.client.1.smithi117.stdout:2/791: mknod d0/d3/d34/d94/d46/d68/d76/cfa 0 2024-02-16T19:29:16.311 INFO:tasks.workunit.client.1.smithi117.stdout:6/734: rename d5/d4b/d20/f78 to d5/d4b/d20/ff0 0 2024-02-16T19:29:16.312 INFO:tasks.workunit.client.0.smithi022.stdout:1/722: unlink d6/d8/d22/d16/d17/d1d/c2b 0 2024-02-16T19:29:16.314 INFO:tasks.workunit.client.1.smithi117.stdout:0/784: symlink d0/d9/d14/l10e 0 2024-02-16T19:29:16.317 INFO:tasks.workunit.client.1.smithi117.stdout:7/757: mkdir d4/d13/d63/ddd/d117 0 2024-02-16T19:29:16.318 INFO:tasks.workunit.client.0.smithi022.stdout:7/695: creat d9/da/d2c/d46/de3/fe4 x:0 0 0 2024-02-16T19:29:16.320 INFO:tasks.workunit.client.0.smithi022.stdout:5/731: fsync dd/d1f/d48/d21/d2f/d52/d5c/d61/f92 0 2024-02-16T19:29:16.322 INFO:tasks.workunit.client.1.smithi117.stdout:7/758: dread d4/d85/fcd [0,4194304] 0 2024-02-16T19:29:16.323 INFO:tasks.workunit.client.0.smithi022.stdout:1/723: rmdir d6/d8/d22/d47/db7 39 2024-02-16T19:29:16.325 INFO:tasks.workunit.client.0.smithi022.stdout:6/730: dread d1/d8/f1c [0,4194304] 0 2024-02-16T19:29:16.327 INFO:tasks.workunit.client.0.smithi022.stdout:0/866: getdents d10/d15/d23/d38/d5a/d66 0 2024-02-16T19:29:16.327 INFO:tasks.workunit.client.1.smithi117.stdout:7/759: dwrite d4/d13/d63/fce [0,4194304] 0 2024-02-16T19:29:16.328 INFO:tasks.workunit.client.1.smithi117.stdout:7/760: dread - d4/d1c/d39/d46/da4/fed zero size 2024-02-16T19:29:16.328 INFO:tasks.workunit.client.0.smithi022.stdout:7/696: rmdir d9/da/d2c/dcb 39 2024-02-16T19:29:16.329 INFO:tasks.workunit.client.1.smithi117.stdout:8/791: getdents d4/d7/dfa 0 2024-02-16T19:29:16.334 INFO:tasks.workunit.client.0.smithi022.stdout:5/732: rename dd/d1f/d48/f1b to dd/d1f/d48/d21/d2f/d46/ff3 0 2024-02-16T19:29:16.338 INFO:tasks.workunit.client.0.smithi022.stdout:9/702: getdents db/d14/d16/d21/d37 0 2024-02-16T19:29:16.339 INFO:tasks.workunit.client.1.smithi117.stdout:5/799: creat d2/d9/df9/f110 x:0 0 0 2024-02-16T19:29:16.341 INFO:tasks.workunit.client.0.smithi022.stdout:6/731: dread d1/d8/da/d29/deb/dfd/d85/d56/fa5 [0,4194304] 0 2024-02-16T19:29:16.342 INFO:tasks.workunit.client.0.smithi022.stdout:1/724: rename d6/d8/d22/d16/d17/d5d/f7b to d6/d8/d22/d16/d17/d5d/fe1 0 2024-02-16T19:29:16.343 INFO:tasks.workunit.client.1.smithi117.stdout:6/735: mknod d5/d4b/d71/dc7/dce/cf1 0 2024-02-16T19:29:16.344 INFO:tasks.workunit.client.1.smithi117.stdout:6/736: stat d5/db/d7d/d32/da8/de1 0 2024-02-16T19:29:16.345 INFO:tasks.workunit.client.1.smithi117.stdout:4/730: getdents d0/da/d21/d3d/d28/d66/da2/dac 0 2024-02-16T19:29:16.346 INFO:tasks.workunit.client.0.smithi022.stdout:0/867: chown d10/d11/d27/c30 11057194 1 2024-02-16T19:29:16.346 INFO:tasks.workunit.client.1.smithi117.stdout:4/731: dread - d0/da/d21/d3d/d28/d66/da2/dac/fec zero size 2024-02-16T19:29:16.349 INFO:tasks.workunit.client.0.smithi022.stdout:7/697: creat d9/da/d2c/d46/fe5 x:0 0 0 2024-02-16T19:29:16.349 INFO:tasks.workunit.client.0.smithi022.stdout:1/725: dread d6/d8/d22/d67/f77 [0,4194304] 0 2024-02-16T19:29:16.350 INFO:tasks.workunit.client.0.smithi022.stdout:5/733: unlink dd/d1f/f2b 0 2024-02-16T19:29:16.351 INFO:tasks.workunit.client.1.smithi117.stdout:8/792: dread d4/d12/d2a/d64/dec/d76/fc5 [0,4194304] 0 2024-02-16T19:29:16.351 INFO:tasks.workunit.client.0.smithi022.stdout:9/703: dread - db/d14/d33/fbd zero size 2024-02-16T19:29:16.355 INFO:tasks.workunit.client.1.smithi117.stdout:3/840: write d1/d1b/d1e/d29/f47 [2441420,52793] 0 2024-02-16T19:29:16.356 INFO:tasks.workunit.client.1.smithi117.stdout:3/841: chown d1/d30/d9f/dbc/fc5 81937 1 2024-02-16T19:29:16.357 INFO:tasks.workunit.client.1.smithi117.stdout:3/842: truncate d1/dd/d8e/f115 750075 0 2024-02-16T19:29:16.359 INFO:tasks.workunit.client.1.smithi117.stdout:9/660: getdents dd/db5/dc4/dcf 0 2024-02-16T19:29:16.359 INFO:tasks.workunit.client.1.smithi117.stdout:9/661: readlink dd/l1d 0 2024-02-16T19:29:16.361 INFO:tasks.workunit.client.1.smithi117.stdout:0/785: write d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/d27/d83/f50 [708598,75364] 0 2024-02-16T19:29:16.362 INFO:tasks.workunit.client.0.smithi022.stdout:0/868: creat d10/d11/d79/f11d x:0 0 0 2024-02-16T19:29:16.364 INFO:tasks.workunit.client.1.smithi117.stdout:0/786: dread d0/d30/f64 [0,4194304] 0 2024-02-16T19:29:16.365 INFO:tasks.workunit.client.1.smithi117.stdout:0/787: write d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/db7/fe8 [856736,31459] 0 2024-02-16T19:29:16.367 INFO:tasks.workunit.client.0.smithi022.stdout:6/732: write d1/d8/ff [48929,126896] 0 2024-02-16T19:29:16.370 INFO:tasks.workunit.client.0.smithi022.stdout:9/704: truncate db/d14/d16/d27/d55/fb8 405107 0 2024-02-16T19:29:16.372 INFO:tasks.workunit.client.0.smithi022.stdout:0/869: symlink d10/d11/d50/d102/l11e 0 2024-02-16T19:29:16.375 INFO:tasks.workunit.client.0.smithi022.stdout:1/726: fsync d6/f1c 0 2024-02-16T19:29:16.377 INFO:tasks.workunit.client.1.smithi117.stdout:3/843: truncate d1/dd/f48 5232088 0 2024-02-16T19:29:16.380 INFO:tasks.workunit.client.1.smithi117.stdout:9/662: mknod dd/d45/d73/ce0 0 2024-02-16T19:29:16.381 INFO:tasks.workunit.client.1.smithi117.stdout:9/663: read dd/d45/d73/f8a [331995,63258] 0 2024-02-16T19:29:16.383 INFO:tasks.workunit.client.0.smithi022.stdout:7/698: dwrite d9/da/d1f/d40/f61 [0,4194304] 0 2024-02-16T19:29:16.383 INFO:tasks.workunit.client.1.smithi117.stdout:1/676: truncate d2/ff 3353424 0 2024-02-16T19:29:16.386 INFO:tasks.workunit.client.1.smithi117.stdout:4/732: dread d0/da/d21/f3f [0,4194304] 0 2024-02-16T19:29:16.387 INFO:tasks.workunit.client.1.smithi117.stdout:5/800: mkdir d2/d4c/d4f/dfd/d111 0 2024-02-16T19:29:16.387 INFO:tasks.workunit.client.1.smithi117.stdout:9/664: dwrite dd/d47/d48/fbc [0,4194304] 0 2024-02-16T19:29:16.389 INFO:tasks.workunit.client.1.smithi117.stdout:9/665: chown dd/d45/d31/c53 30005744 1 2024-02-16T19:29:16.393 INFO:tasks.workunit.client.1.smithi117.stdout:6/737: link d5/db/f1f d5/d4b/d44/d52/d81/dd7/de0/ff2 0 2024-02-16T19:29:16.396 INFO:tasks.workunit.client.1.smithi117.stdout:8/793: creat d4/d73/dff/f102 x:0 0 0 2024-02-16T19:29:16.398 INFO:tasks.workunit.client.1.smithi117.stdout:3/844: creat d1/d1b/d1e/d5e/f122 x:0 0 0 2024-02-16T19:29:16.398 INFO:tasks.workunit.client.0.smithi022.stdout:0/870: write d10/d11/d27/d77/f81 [456045,101416] 0 2024-02-16T19:29:16.400 INFO:tasks.workunit.client.0.smithi022.stdout:0/871: readlink d10/d11/d27/d3f/d58/dab/lc5 0 2024-02-16T19:29:16.401 INFO:tasks.workunit.client.1.smithi117.stdout:7/761: creat d4/f118 x:0 0 0 2024-02-16T19:29:16.401 INFO:tasks.workunit.client.1.smithi117.stdout:7/762: chown d4/d1c/d39/cc7 3482 1 2024-02-16T19:29:16.404 INFO:tasks.workunit.client.0.smithi022.stdout:9/705: symlink db/d14/d16/d5d/d65/d6b/dd5/dd9/ddd/lf1 0 2024-02-16T19:29:16.410 INFO:tasks.workunit.client.0.smithi022.stdout:1/727: dwrite d6/d8/d22/d16/d17/d63/d6f/d8b/fa3 [0,4194304] 0 2024-02-16T19:29:16.413 INFO:tasks.workunit.client.1.smithi117.stdout:8/794: symlink d4/d7/d6a/d72/l103 0 2024-02-16T19:29:16.415 INFO:tasks.workunit.client.1.smithi117.stdout:3/845: symlink d1/l123 0 2024-02-16T19:29:16.417 INFO:tasks.workunit.client.0.smithi022.stdout:6/733: dwrite d1/d8/da/d29/deb/dfd/d85/f50 [0,4194304] 0 2024-02-16T19:29:16.417 INFO:tasks.workunit.client.1.smithi117.stdout:7/763: creat d4/d1c/d39/d46/d47/f119 x:0 0 0 2024-02-16T19:29:16.419 INFO:tasks.workunit.client.1.smithi117.stdout:9/666: write dd/d47/d5e/f87 [205941,1983] 0 2024-02-16T19:29:16.420 INFO:tasks.workunit.client.1.smithi117.stdout:4/733: mkdir d0/da/d21/d3d/d28/d64/d73/d91/d9e/dbb/df0 0 2024-02-16T19:29:16.421 INFO:tasks.workunit.client.1.smithi117.stdout:4/734: dread - d0/da/d21/d3d/d28/d64/d73/d91/f62 zero size 2024-02-16T19:29:16.422 INFO:tasks.workunit.client.0.smithi022.stdout:6/734: chown d1/d8/da/d7b/cb9 253101993 1 2024-02-16T19:29:16.422 INFO:tasks.workunit.client.0.smithi022.stdout:7/699: dwrite d9/da/d2c/f8d [0,4194304] 0 2024-02-16T19:29:16.424 INFO:tasks.workunit.client.0.smithi022.stdout:2/662: sync 2024-02-16T19:29:16.427 INFO:tasks.workunit.client.1.smithi117.stdout:6/738: mkdir d5/d4b/d71/dc7/dce/ded/df3 0 2024-02-16T19:29:16.431 INFO:tasks.workunit.client.1.smithi117.stdout:7/764: read - d4/d13/d63/f94 zero size 2024-02-16T19:29:16.431 INFO:tasks.workunit.client.0.smithi022.stdout:6/735: dread d1/d8/d3c/d77/f80 [0,4194304] 0 2024-02-16T19:29:16.433 INFO:tasks.workunit.client.1.smithi117.stdout:7/765: truncate d4/d1c/d39/d46/fc6 1024031 0 2024-02-16T19:29:16.434 INFO:tasks.workunit.client.1.smithi117.stdout:5/801: rename d2/db3/ce6 to d2/d9/d3c/d24/d50/d96/dbc/d30/c112 0 2024-02-16T19:29:16.434 INFO:tasks.workunit.client.1.smithi117.stdout:5/802: chown d2/d3/d21/d49 29035115 1 2024-02-16T19:29:16.435 INFO:tasks.workunit.client.1.smithi117.stdout:3/846: fdatasync d1/dd/d3b/ff1 0 2024-02-16T19:29:16.436 INFO:tasks.workunit.client.1.smithi117.stdout:5/803: write d2/d5d/f9d [2335759,92355] 0 2024-02-16T19:29:16.448 INFO:tasks.workunit.client.1.smithi117.stdout:7/766: mkdir d4/d13/d63/ddd/d117/d11a 0 2024-02-16T19:29:16.451 INFO:tasks.workunit.client.1.smithi117.stdout:4/735: rename d0/da/d21/d3d/d28/d64/l75 to d0/da/d21/d3d/d28/d64/d48/lf1 0 2024-02-16T19:29:16.451 INFO:tasks.workunit.client.1.smithi117.stdout:9/667: write dd/d45/d31/f93 [572817,88910] 0 2024-02-16T19:29:16.451 INFO:tasks.workunit.client.1.smithi117.stdout:8/795: dwrite d4/d12/d2a/f2c [0,4194304] 0 2024-02-16T19:29:16.456 INFO:tasks.workunit.client.1.smithi117.stdout:5/804: fdatasync d2/d9/d3c/d24/d50/d52/d74/d8f/fc7 0 2024-02-16T19:29:16.457 INFO:tasks.workunit.client.1.smithi117.stdout:6/739: dread d5/d4b/f1e [0,4194304] 0 2024-02-16T19:29:16.459 INFO:tasks.workunit.client.1.smithi117.stdout:9/668: dread dd/d47/d5b/d5f/f6f [0,4194304] 0 2024-02-16T19:29:16.460 INFO:tasks.workunit.client.1.smithi117.stdout:3/847: link d1/d30/d52/d56/lb7 d1/d30/d52/d10f/d114/l124 0 2024-02-16T19:29:16.466 INFO:tasks.workunit.client.1.smithi117.stdout:8/796: mkdir d4/d7/da6/d3a/d65/d96/d104 0 2024-02-16T19:29:16.470 INFO:tasks.workunit.client.1.smithi117.stdout:5/805: truncate d2/d3/f76 1210984 0 2024-02-16T19:29:16.471 INFO:tasks.workunit.client.1.smithi117.stdout:5/806: chown d2/db3/f5f 257776 1 2024-02-16T19:29:16.478 INFO:tasks.workunit.client.1.smithi117.stdout:8/797: symlink d4/d7/d6a/d72/d84/dda/l105 0 2024-02-16T19:29:16.478 INFO:tasks.workunit.client.1.smithi117.stdout:4/736: write d0/da/d21/d3d/d28/f39 [4066,9565] 0 2024-02-16T19:29:16.479 INFO:tasks.workunit.client.1.smithi117.stdout:4/737: chown d0/f25 210739 1 2024-02-16T19:29:16.482 INFO:tasks.workunit.client.1.smithi117.stdout:8/798: dread d4/d73/fdb [0,4194304] 0 2024-02-16T19:29:16.483 INFO:tasks.workunit.client.1.smithi117.stdout:5/807: write d2/d9/d3c/d24/d50/d52/d74/d8f/fc7 [994648,46174] 0 2024-02-16T19:29:16.486 INFO:tasks.workunit.client.0.smithi022.stdout:5/734: mknod dd/d1f/d48/d21/d2f/d52/d5c/d61/cf4 0 2024-02-16T19:29:16.487 INFO:tasks.workunit.client.1.smithi117.stdout:4/738: dwrite d0/da/d21/d3d/d28/d64/d8a/d92/fdd [0,4194304] 0 2024-02-16T19:29:16.490 INFO:tasks.workunit.client.0.smithi022.stdout:3/674: sync 2024-02-16T19:29:16.491 INFO:tasks.workunit.client.0.smithi022.stdout:3/675: write d7/d12/d18/dd/fcb [2844403,12421] 0 2024-02-16T19:29:16.492 INFO:tasks.workunit.client.0.smithi022.stdout:9/706: mknod db/d14/d16/d49/d90/cf2 0 2024-02-16T19:29:16.493 INFO:tasks.workunit.client.0.smithi022.stdout:9/707: stat db/d14/d16/d27/d2e/c76 0 2024-02-16T19:29:16.497 INFO:tasks.workunit.client.1.smithi117.stdout:9/669: dwrite dd/db5/f32 [0,4194304] 0 2024-02-16T19:29:16.499 INFO:tasks.workunit.client.1.smithi117.stdout:9/670: readlink dd/d45/d31/l34 0 2024-02-16T19:29:16.499 INFO:tasks.workunit.client.1.smithi117.stdout:3/848: dwrite d1/dd/d32/fc3 [0,4194304] 0 2024-02-16T19:29:16.500 INFO:tasks.workunit.client.1.smithi117.stdout:8/799: truncate d4/d12/d2a/f48 256171 0 2024-02-16T19:29:16.501 INFO:tasks.workunit.client.1.smithi117.stdout:6/740: write d5/db/d7d/d32/d37/f41 [4110217,119353] 0 2024-02-16T19:29:16.503 INFO:tasks.workunit.client.1.smithi117.stdout:7/767: link d4/d1c/d87/d89/d5f/f7b d4/d1c/dd8/f11b 0 2024-02-16T19:29:16.504 INFO:tasks.workunit.client.0.smithi022.stdout:0/872: dread d10/d11/d50/f3e [4194304,4194304] 0 2024-02-16T19:29:16.505 INFO:tasks.workunit.client.0.smithi022.stdout:2/663: symlink d0/d32/ld7 0 2024-02-16T19:29:16.506 INFO:tasks.workunit.client.1.smithi117.stdout:4/739: truncate d0/da/d21/d3d/f29 2031784 0 2024-02-16T19:29:16.506 INFO:tasks.workunit.client.1.smithi117.stdout:7/768: rename d4 to d4/d1c/d39/d46/dc4/d11c 22 2024-02-16T19:29:16.506 INFO:tasks.workunit.client.1.smithi117.stdout:5/808: dread d2/db3/f5f [0,4194304] 0 2024-02-16T19:29:16.506 INFO:tasks.workunit.client.1.smithi117.stdout:8/800: write d4/d12/d2a/f2c [4791151,90392] 0 2024-02-16T19:29:16.510 INFO:tasks.workunit.client.1.smithi117.stdout:5/809: dread - d2/d9/d3c/f98 zero size 2024-02-16T19:29:16.515 INFO:tasks.workunit.client.1.smithi117.stdout:9/671: creat dd/d23/daf/fe1 x:0 0 0 2024-02-16T19:29:16.516 INFO:tasks.workunit.client.1.smithi117.stdout:2/792: sync 2024-02-16T19:29:16.517 INFO:tasks.workunit.client.1.smithi117.stdout:2/793: fsync d0/d3/d34/d94/d46/d68/d76/da1/d56/fd3 0 2024-02-16T19:29:16.518 INFO:tasks.workunit.client.1.smithi117.stdout:3/849: mkdir d1/dd/d3e/d125 0 2024-02-16T19:29:16.522 INFO:tasks.workunit.client.1.smithi117.stdout:6/741: creat d5/d4b/d71/dc7/d72/dec/ff4 x:0 0 0 2024-02-16T19:29:16.523 INFO:tasks.workunit.client.0.smithi022.stdout:2/664: truncate d0/d2/f14 1809652 0 2024-02-16T19:29:16.525 INFO:tasks.workunit.client.1.smithi117.stdout:3/850: dwrite d1/d30/d52/fc9 [0,4194304] 0 2024-02-16T19:29:16.525 INFO:tasks.workunit.client.0.smithi022.stdout:0/873: read d10/d15/d23/d8f/dc1/f4f [2103216,93719] 0 2024-02-16T19:29:16.526 INFO:tasks.workunit.client.0.smithi022.stdout:7/700: rename d9/da/d4e/fd2 to d9/da/d2c/dcb/fe6 0 2024-02-16T19:29:16.527 INFO:tasks.workunit.client.1.smithi117.stdout:3/851: chown d1/dd/d3e/d7d/fbe 4 1 2024-02-16T19:29:16.528 INFO:tasks.workunit.client.1.smithi117.stdout:5/810: truncate d2/d3/f65 995886 0 2024-02-16T19:29:16.530 INFO:tasks.workunit.client.0.smithi022.stdout:5/735: mkdir dd/d1f/d48/d21/d2f/d46/d5e/d84/df5 0 2024-02-16T19:29:16.531 INFO:tasks.workunit.client.1.smithi117.stdout:6/742: dwrite d5/d4b/d71/dc7/d72/fde [0,4194304] 0 2024-02-16T19:29:16.532 INFO:tasks.workunit.client.1.smithi117.stdout:2/794: truncate d0/d13/d4d/d53/fa4 705840 0 2024-02-16T19:29:16.534 INFO:tasks.workunit.client.0.smithi022.stdout:3/676: write d7/d12/d18/dd/d24/d29/f2c [4903979,100012] 0 2024-02-16T19:29:16.537 INFO:tasks.workunit.client.0.smithi022.stdout:6/736: dwrite d1/d8/d81/f8c [0,4194304] 0 2024-02-16T19:29:16.539 INFO:tasks.workunit.client.1.smithi117.stdout:7/769: mkdir d4/d1c/d87/d89/d60/d79/dc3/d11d 0 2024-02-16T19:29:16.540 INFO:tasks.workunit.client.1.smithi117.stdout:7/770: readlink d4/d1c/d87/d89/l2a 0 2024-02-16T19:29:16.545 INFO:tasks.workunit.client.1.smithi117.stdout:6/743: dread d5/db/d7d/d32/f5b [0,4194304] 0 2024-02-16T19:29:16.545 INFO:tasks.workunit.client.0.smithi022.stdout:2/665: readlink d0/d2/d18/d38/ld6 0 2024-02-16T19:29:16.546 INFO:tasks.workunit.client.1.smithi117.stdout:6/744: dread - d5/d4b/d71/fc1 zero size 2024-02-16T19:29:16.546 INFO:tasks.workunit.client.1.smithi117.stdout:4/740: fsync d0/da/d21/d3d/d28/d64/d73/d91/f31 0 2024-02-16T19:29:16.548 INFO:tasks.workunit.client.0.smithi022.stdout:0/874: mknod d10/d22/dad/c11f 0 2024-02-16T19:29:16.549 INFO:tasks.workunit.client.1.smithi117.stdout:9/672: dread dd/d23/d64/f9f [0,4194304] 0 2024-02-16T19:29:16.550 INFO:tasks.workunit.client.0.smithi022.stdout:7/701: read d9/da/d2c/d2f/f5c [349783,20797] 0 2024-02-16T19:29:16.553 INFO:tasks.workunit.client.1.smithi117.stdout:5/811: creat d2/d9/d3c/d24/f113 x:0 0 0 2024-02-16T19:29:16.554 INFO:tasks.workunit.client.0.smithi022.stdout:4/619: sync 2024-02-16T19:29:16.554 INFO:tasks.workunit.client.0.smithi022.stdout:8/746: sync 2024-02-16T19:29:16.555 INFO:tasks.workunit.client.1.smithi117.stdout:2/795: creat d0/d13/dc4/ffb x:0 0 0 2024-02-16T19:29:16.556 INFO:tasks.workunit.client.0.smithi022.stdout:3/677: symlink d7/d21/lf3 0 2024-02-16T19:29:16.557 INFO:tasks.workunit.client.1.smithi117.stdout:7/771: rename d4/d1c/d87/d89/d60/d79/c9d to d4/d1c/d39/d46/d55/db8/c11e 0 2024-02-16T19:29:16.559 INFO:tasks.workunit.client.1.smithi117.stdout:7/772: read d4/d1c/d39/d46/d55/fcc [20765,125716] 0 2024-02-16T19:29:16.562 INFO:tasks.workunit.client.0.smithi022.stdout:3/678: dwrite d7/d12/d18/dd/d24/d58/d70/f81 [4194304,4194304] 0 2024-02-16T19:29:16.563 INFO:tasks.workunit.client.0.smithi022.stdout:1/728: link d6/d8/d22/d47/db7/cde d6/d8/d22/d47/db7/ce2 0 2024-02-16T19:29:16.565 INFO:tasks.workunit.client.1.smithi117.stdout:6/745: symlink d5/d4b/d44/d93/dae/lf5 0 2024-02-16T19:29:16.565 INFO:tasks.workunit.client.1.smithi117.stdout:3/852: write d1/d1b/f21 [2297803,53731] 0 2024-02-16T19:29:16.571 INFO:tasks.workunit.client.0.smithi022.stdout:2/666: creat d0/d2/d2e/d73/fd8 x:0 0 0 2024-02-16T19:29:16.571 INFO:tasks.workunit.client.0.smithi022.stdout:1/729: dwrite d6/d8/d22/d16/d17/d5d/fc8 [0,4194304] 0 2024-02-16T19:29:16.571 INFO:tasks.workunit.client.1.smithi117.stdout:8/801: creat d4/d7/d6a/d6f/f106 x:0 0 0 2024-02-16T19:29:16.578 INFO:tasks.workunit.client.0.smithi022.stdout:5/736: symlink dd/d1f/d48/d21/d2f/d52/d5c/lf6 0 2024-02-16T19:29:16.579 INFO:tasks.workunit.client.0.smithi022.stdout:4/620: unlink d4/d38/c96 0 2024-02-16T19:29:16.580 INFO:tasks.workunit.client.0.smithi022.stdout:6/737: dwrite d1/d8/d81/fa0 [0,4194304] 0 2024-02-16T19:29:16.582 INFO:tasks.workunit.client.0.smithi022.stdout:8/747: creat d3/d12/d2b/da9/f103 x:0 0 0 2024-02-16T19:29:16.584 INFO:tasks.workunit.client.0.smithi022.stdout:8/748: chown d3/d12/f71 105758490 1 2024-02-16T19:29:16.584 INFO:tasks.workunit.client.0.smithi022.stdout:8/749: write d3/d12/da6/f5e [946961,75121] 0 2024-02-16T19:29:16.585 INFO:tasks.workunit.client.1.smithi117.stdout:9/673: rename dd/d47/d5e/f6e to dd/d23/daf/fe2 0 2024-02-16T19:29:16.587 INFO:tasks.workunit.client.0.smithi022.stdout:9/708: getdents db 0 2024-02-16T19:29:16.589 INFO:tasks.workunit.client.1.smithi117.stdout:6/746: symlink d5/d4b/d71/dc7/dce/ded/lf6 0 2024-02-16T19:29:16.592 INFO:tasks.workunit.client.1.smithi117.stdout:3/853: rmdir d1/dd/d3b 39 2024-02-16T19:29:16.596 INFO:tasks.workunit.client.1.smithi117.stdout:5/812: dwrite d2/d4c/f71 [0,4194304] 0 2024-02-16T19:29:16.598 INFO:tasks.workunit.client.1.smithi117.stdout:8/802: creat d4/d7/da6/d3a/d65/d7c/f107 x:0 0 0 2024-02-16T19:29:16.600 INFO:tasks.workunit.client.0.smithi022.stdout:7/702: creat d9/da/d1f/d40/dde/fe7 x:0 0 0 2024-02-16T19:29:16.602 INFO:tasks.workunit.client.0.smithi022.stdout:5/737: dread - dd/d1f/d48/d21/d2f/d52/d5c/f7d zero size 2024-02-16T19:29:16.605 INFO:tasks.workunit.client.1.smithi117.stdout:1/677: sync 2024-02-16T19:29:16.605 INFO:tasks.workunit.client.1.smithi117.stdout:0/788: sync 2024-02-16T19:29:16.606 INFO:tasks.workunit.client.1.smithi117.stdout:0/789: readlink d0/d9/d14/l10e 0 2024-02-16T19:29:16.606 INFO:tasks.workunit.client.1.smithi117.stdout:1/678: write d2/d4/f9 [423821,111637] 0 2024-02-16T19:29:16.609 INFO:tasks.workunit.client.1.smithi117.stdout:0/790: chown d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102 1 1 2024-02-16T19:29:16.609 INFO:tasks.workunit.client.0.smithi022.stdout:7/703: dwrite d9/da/fab [0,4194304] 0 2024-02-16T19:29:16.609 INFO:tasks.workunit.client.0.smithi022.stdout:6/738: read d1/d8/da/d29/deb/dfd/d53/f7d [4157968,63117] 0 2024-02-16T19:29:16.610 INFO:tasks.workunit.client.1.smithi117.stdout:6/747: stat d5/d4b/f46 0 2024-02-16T19:29:16.610 INFO:tasks.workunit.client.1.smithi117.stdout:1/679: write d2/d19/d59/d89/d69/d9a/fd1 [155695,58062] 0 2024-02-16T19:29:16.613 INFO:tasks.workunit.client.0.smithi022.stdout:9/709: mknod db/d14/d16/d28/cf3 0 2024-02-16T19:29:16.613 INFO:tasks.workunit.client.0.smithi022.stdout:4/621: write d4/d6/dd/d22/f51 [1718209,7563] 0 2024-02-16T19:29:16.614 INFO:tasks.workunit.client.0.smithi022.stdout:4/622: chown d4/d6 136479896 1 2024-02-16T19:29:16.614 INFO:tasks.workunit.client.1.smithi117.stdout:7/773: truncate d4/d1c/f3c 7359 0 2024-02-16T19:29:16.616 INFO:tasks.workunit.client.0.smithi022.stdout:3/679: rename d7/d12/d18/dd/d31/d7f/lde to d7/d12/d18/dd/d24/d93/de9/dc4/lf4 0 2024-02-16T19:29:16.618 INFO:tasks.workunit.client.1.smithi117.stdout:4/741: dwrite d0/da/d21/f40 [0,4194304] 0 2024-02-16T19:29:16.618 INFO:tasks.workunit.client.1.smithi117.stdout:5/813: symlink d2/d9/df9/l114 0 2024-02-16T19:29:16.620 INFO:tasks.workunit.client.1.smithi117.stdout:2/796: creat d0/d3/d34/d94/ffc x:0 0 0 2024-02-16T19:29:16.622 INFO:tasks.workunit.client.0.smithi022.stdout:2/667: creat d0/d2/d18/d38/db6/fd9 x:0 0 0 2024-02-16T19:29:16.623 INFO:tasks.workunit.client.0.smithi022.stdout:2/668: dread - d0/d32/fbb zero size 2024-02-16T19:29:16.624 INFO:tasks.workunit.client.0.smithi022.stdout:2/669: readlink d0/d32/ld7 0 2024-02-16T19:29:16.625 INFO:tasks.workunit.client.1.smithi117.stdout:8/803: rename d4/l17 to d4/d7/da6/d3a/d65/d7c/l108 0 2024-02-16T19:29:16.626 INFO:tasks.workunit.client.0.smithi022.stdout:0/875: creat d10/d15/d23/f120 x:0 0 0 2024-02-16T19:29:16.630 INFO:tasks.workunit.client.0.smithi022.stdout:5/738: mknod dd/d1f/d48/d21/d2f/d52/d77/cf7 0 2024-02-16T19:29:16.633 INFO:tasks.workunit.client.1.smithi117.stdout:1/680: dread d2/d1d/d1f/fc4 [0,4194304] 0 2024-02-16T19:29:16.636 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:16 smithi117 ceph-mon[35309]: pgmap v11: 65 pgs: 65 active+clean; 2.5 GiB data, 7.3 GiB used, 529 GiB / 536 GiB avail; 84 MiB/s rd, 198 MiB/s wr, 412 op/s 2024-02-16T19:29:16.637 INFO:tasks.workunit.client.0.smithi022.stdout:8/750: mkdir d3/d12/d2b/d54/db8/de5/d104 0 2024-02-16T19:29:16.638 INFO:tasks.workunit.client.0.smithi022.stdout:1/730: dwrite d6/f55 [0,4194304] 0 2024-02-16T19:29:16.638 INFO:tasks.workunit.client.1.smithi117.stdout:6/748: mknod d5/d4b/d20/d69/cf7 0 2024-02-16T19:29:16.642 INFO:tasks.workunit.client.1.smithi117.stdout:7/774: rename d4/d1c/d87/d89/d60/d9b to d4/d1c/d87/d89/d60/d11f 0 2024-02-16T19:29:16.644 INFO:tasks.workunit.client.0.smithi022.stdout:6/739: dread d1/d8/d3c/f44 [0,4194304] 0 2024-02-16T19:29:16.646 INFO:tasks.workunit.client.1.smithi117.stdout:5/814: unlink d2/d5d/fdf 0 2024-02-16T19:29:16.647 INFO:tasks.workunit.client.1.smithi117.stdout:6/749: dread d5/f86 [0,4194304] 0 2024-02-16T19:29:16.647 INFO:tasks.workunit.client.1.smithi117.stdout:5/815: write d2/d36/f10d [25331,110445] 0 2024-02-16T19:29:16.652 INFO:tasks.workunit.client.1.smithi117.stdout:3/854: rename d1/dd/d3b/cec to d1/d31/d89/dbd/d120/c126 0 2024-02-16T19:29:16.652 INFO:tasks.workunit.client.1.smithi117.stdout:3/855: chown d1/l4 326 1 2024-02-16T19:29:16.653 INFO:tasks.workunit.client.0.smithi022.stdout:2/670: rename d0/d2/d18/d19/d40/c45 to d0/d2/dce/cda 0 2024-02-16T19:29:16.658 INFO:tasks.workunit.client.1.smithi117.stdout:7/775: dread d4/d13/d63/fa2 [0,4194304] 0 2024-02-16T19:29:16.660 INFO:tasks.workunit.client.0.smithi022.stdout:4/623: write d4/d6/dd/df/d10/f50 [2323912,71425] 0 2024-02-16T19:29:16.660 INFO:tasks.workunit.client.0.smithi022.stdout:7/704: symlink d9/da/d2c/d65/le8 0 2024-02-16T19:29:16.661 INFO:tasks.workunit.client.1.smithi117.stdout:2/797: write d0/f5e [530888,22194] 0 2024-02-16T19:29:16.662 INFO:tasks.workunit.client.1.smithi117.stdout:0/791: mknod d0/d9/d14/c10f 0 2024-02-16T19:29:16.663 INFO:tasks.workunit.client.1.smithi117.stdout:6/750: dread d5/d4b/d71/dc7/d49/d59/f5e [0,4194304] 0 2024-02-16T19:29:16.666 INFO:tasks.workunit.client.0.smithi022.stdout:1/731: symlink d6/d8/dba/dc7/le3 0 2024-02-16T19:29:16.671 INFO:tasks.workunit.client.0.smithi022.stdout:6/740: symlink d1/ddc/l102 0 2024-02-16T19:29:16.675 INFO:tasks.workunit.client.1.smithi117.stdout:4/742: write d0/da/d21/d3d/d28/d64/f23 [2273143,92685] 0 2024-02-16T19:29:16.676 INFO:tasks.workunit.client.0.smithi022.stdout:2/671: mknod d0/d2/d18/d19/d40/d55/cdb 0 2024-02-16T19:29:16.677 INFO:tasks.workunit.client.0.smithi022.stdout:8/751: dwrite d3/d25/d33/f37 [0,4194304] 0 2024-02-16T19:29:16.680 INFO:tasks.workunit.client.1.smithi117.stdout:9/674: getdents dd/db5/dc4/dd3 0 2024-02-16T19:29:16.685 INFO:tasks.workunit.client.0.smithi022.stdout:2/672: dwrite d0/d2/d18/d1f/d6c/d99/fc6 [0,4194304] 0 2024-02-16T19:29:16.686 INFO:tasks.workunit.client.1.smithi117.stdout:8/804: write d4/d7/d6a/fe4 [41027,18628] 0 2024-02-16T19:29:16.687 INFO:tasks.workunit.client.0.smithi022.stdout:0/876: write d10/d11/d27/d3f/d58/f6a [2937022,19743] 0 2024-02-16T19:29:16.692 INFO:tasks.workunit.client.0.smithi022.stdout:1/732: creat d6/d8/d22/d16/d17/d63/fe4 x:0 0 0 2024-02-16T19:29:16.693 INFO:tasks.workunit.client.1.smithi117.stdout:0/792: unlink d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/d27/d31/fc4 0 2024-02-16T19:29:16.694 INFO:tasks.workunit.client.0.smithi022.stdout:9/710: creat db/d14/ff4 x:0 0 0 2024-02-16T19:29:16.694 INFO:tasks.workunit.client.1.smithi117.stdout:7/776: dwrite d4/d1c/d87/d89/d60/fab [0,4194304] 0 2024-02-16T19:29:16.695 INFO:tasks.workunit.client.1.smithi117.stdout:7/777: chown d4/d1c/d39/d46/fc6 0 1 2024-02-16T19:29:16.696 INFO:tasks.workunit.client.0.smithi022.stdout:6/741: mkdir d1/d8/da/d29/deb/dfd/d53/d103 0 2024-02-16T19:29:16.696 INFO:tasks.workunit.client.0.smithi022.stdout:3/680: link d7/d21/dd9/cea d7/d12/d18/dd/d24/d93/cf5 0 2024-02-16T19:29:16.697 INFO:tasks.workunit.client.1.smithi117.stdout:5/816: fsync d2/d3/d21/f33 0 2024-02-16T19:29:16.698 INFO:tasks.workunit.client.0.smithi022.stdout:6/742: write d1/d8/d23/dc6/ddb/ffe [3687732,59335] 0 2024-02-16T19:29:16.700 INFO:tasks.workunit.client.0.smithi022.stdout:8/752: creat d3/d12/d2b/d54/d74/da2/f105 x:0 0 0 2024-02-16T19:29:16.702 INFO:tasks.workunit.client.1.smithi117.stdout:4/743: creat d0/da/d21/d3d/d28/d66/da2/dac/ff2 x:0 0 0 2024-02-16T19:29:16.708 INFO:tasks.workunit.client.1.smithi117.stdout:9/675: creat dd/db5/dc4/dd3/fe3 x:0 0 0 2024-02-16T19:29:16.708 INFO:tasks.workunit.client.0.smithi022.stdout:8/753: dwrite d3/d12/d2b/d46/f53 [0,4194304] 0 2024-02-16T19:29:16.710 INFO:tasks.workunit.client.1.smithi117.stdout:4/744: dwrite d0/da/d21/d3d/d28/d64/d90/dc0/dd7/fde [0,4194304] 0 2024-02-16T19:29:16.710 INFO:tasks.workunit.client.1.smithi117.stdout:8/805: dread d4/d7/da6/d3a/d43/f55 [0,4194304] 0 2024-02-16T19:29:16.711 INFO:tasks.workunit.client.1.smithi117.stdout:8/806: dread - d4/d7/d6a/d6f/f106 zero size 2024-02-16T19:29:16.715 INFO:tasks.workunit.client.1.smithi117.stdout:6/751: symlink d5/d4b/d44/lf8 0 2024-02-16T19:29:16.717 INFO:tasks.workunit.client.0.smithi022.stdout:1/733: chown d6/d8/d22/d47/fcd 11268 1 2024-02-16T19:29:16.718 INFO:tasks.workunit.client.0.smithi022.stdout:9/711: truncate db/d14/d16/d49/f9f 1693372 0 2024-02-16T19:29:16.720 INFO:tasks.workunit.client.0.smithi022.stdout:9/712: chown db/d14/d16/d27/c6c 459208 1 2024-02-16T19:29:16.722 INFO:tasks.workunit.client.0.smithi022.stdout:9/713: fsync db/d14/d16/d5d/d65/d6b/fcb 0 2024-02-16T19:29:16.725 INFO:tasks.workunit.client.0.smithi022.stdout:9/714: readlink db/d14/d16/d49/d90/ld7 0 2024-02-16T19:29:16.726 INFO:tasks.workunit.client.0.smithi022.stdout:3/681: unlink d7/d12/d18/dd/d24/cdd 0 2024-02-16T19:29:16.726 INFO:tasks.workunit.client.0.smithi022.stdout:1/734: dwrite d6/d8/d22/d16/d17/d5d/fe0 [0,4194304] 0 2024-02-16T19:29:16.730 INFO:tasks.workunit.client.0.smithi022.stdout:6/743: read - d1/d8/da/d29/deb/dfd/d85/d5b/fa4 zero size 2024-02-16T19:29:16.733 INFO:tasks.workunit.client.0.smithi022.stdout:7/705: dwrite d9/f25 [0,4194304] 0 2024-02-16T19:29:16.733 INFO:tasks.workunit.client.0.smithi022.stdout:5/739: link dd/d1f/d48/f60 dd/d1f/d48/d21/d2f/ff8 0 2024-02-16T19:29:16.736 INFO:tasks.workunit.client.0.smithi022.stdout:6/744: stat d1/d8/da/d29/deb/dfd/d85/d56/f6a 0 2024-02-16T19:29:16.739 INFO:tasks.workunit.client.1.smithi117.stdout:5/817: write d2/d4c/d69/f72 [372273,49725] 0 2024-02-16T19:29:16.741 INFO:tasks.workunit.client.1.smithi117.stdout:7/778: dwrite d4/d1c/d87/fff [0,4194304] 0 2024-02-16T19:29:16.742 INFO:tasks.workunit.client.1.smithi117.stdout:7/779: readlink d4/l97 0 2024-02-16T19:29:16.742 INFO:tasks.workunit.client.1.smithi117.stdout:5/818: stat d2/d9/d3c/d24/d50/d52/d74/fd2 0 2024-02-16T19:29:16.743 INFO:tasks.workunit.client.0.smithi022.stdout:3/682: dread d7/d12/d18/dd/d3c/fcc [0,4194304] 0 2024-02-16T19:29:16.743 INFO:tasks.workunit.client.1.smithi117.stdout:4/745: symlink d0/da/d21/d3d/d28/d64/d73/d91/lf3 0 2024-02-16T19:29:16.744 INFO:tasks.workunit.client.0.smithi022.stdout:3/683: stat d7/d12/d18/dd/d24/f3e 0 2024-02-16T19:29:16.745 INFO:tasks.workunit.client.1.smithi117.stdout:5/819: read d2/d9/f18 [8363974,30711] 0 2024-02-16T19:29:16.746 INFO:tasks.workunit.client.1.smithi117.stdout:5/820: chown d2/d9/d3c/d24/d50/d52/fa5 1843 1 2024-02-16T19:29:16.747 INFO:tasks.workunit.client.1.smithi117.stdout:0/793: creat d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/d27/df9/f110 x:0 0 0 2024-02-16T19:29:16.751 INFO:tasks.workunit.client.0.smithi022.stdout:9/715: rmdir db/d14/d16/d21/d37 39 2024-02-16T19:29:16.754 INFO:tasks.workunit.client.1.smithi117.stdout:9/676: dwrite dd/d23/f18 [0,4194304] 0 2024-02-16T19:29:16.756 INFO:tasks.workunit.client.1.smithi117.stdout:1/681: rename d2/d1d/d60/d7c/d9e/da5/db0/cb6 to d2/d19/cee 0 2024-02-16T19:29:16.758 INFO:tasks.workunit.client.1.smithi117.stdout:3/856: creat d1/d1b/d1e/d29/daf/f127 x:0 0 0 2024-02-16T19:29:16.760 INFO:tasks.workunit.client.1.smithi117.stdout:3/857: dread d1/dd/d8e/f115 [0,4194304] 0 2024-02-16T19:29:16.761 INFO:tasks.workunit.client.1.smithi117.stdout:3/858: chown d1/d1b/l4d 1518040439 1 2024-02-16T19:29:16.763 INFO:tasks.workunit.client.0.smithi022.stdout:8/754: fsync d3/d12/da6/f8c 0 2024-02-16T19:29:16.764 INFO:tasks.workunit.client.1.smithi117.stdout:7/780: write d4/d1c/fbe [2928898,48440] 0 2024-02-16T19:29:16.767 INFO:tasks.workunit.client.0.smithi022.stdout:6/745: symlink d1/d8/da/d29/l104 0 2024-02-16T19:29:16.770 INFO:tasks.workunit.client.1.smithi117.stdout:2/798: getdents d0/d3/d34/d94/d46/d68/d76 0 2024-02-16T19:29:16.770 INFO:tasks.workunit.client.0.smithi022.stdout:4/624: link d4/d6/d14/d64/c48 d4/d6/dd/df/d10/d55/cc2 0 2024-02-16T19:29:16.771 INFO:tasks.workunit.client.1.smithi117.stdout:4/746: creat d0/da/d21/d3d/d28/d64/d73/d91/d9e/ff4 x:0 0 0 2024-02-16T19:29:16.775 INFO:tasks.workunit.client.0.smithi022.stdout:9/716: mkdir db/d14/d16/dec/dee/df5 0 2024-02-16T19:29:16.775 INFO:tasks.workunit.client.1.smithi117.stdout:0/794: rmdir d0/d9/d14/d1d/d60 39 2024-02-16T19:29:16.776 INFO:tasks.workunit.client.0.smithi022.stdout:9/717: fdatasync db/d14/d16/d28/fe2 0 2024-02-16T19:29:16.780 INFO:tasks.workunit.client.0.smithi022.stdout:7/706: symlink d9/le9 0 2024-02-16T19:29:16.781 INFO:tasks.workunit.client.0.smithi022.stdout:7/707: truncate d9/da/d2c/dcb/fdd 360628 0 2024-02-16T19:29:16.782 INFO:tasks.workunit.client.1.smithi117.stdout:1/682: unlink d2/c87 0 2024-02-16T19:29:16.782 INFO:tasks.workunit.client.0.smithi022.stdout:1/735: write d6/d8/f2d [4779480,91773] 0 2024-02-16T19:29:16.784 INFO:tasks.workunit.client.0.smithi022.stdout:1/736: dread - d6/d8/d22/d47/db7/fdb zero size 2024-02-16T19:29:16.785 INFO:tasks.workunit.client.0.smithi022.stdout:1/737: chown d6/d8/d22/d47/d51/d8d/cae 64242 1 2024-02-16T19:29:16.785 INFO:tasks.workunit.client.0.smithi022.stdout:8/755: mkdir d3/d12/d2b/da9/d106 0 2024-02-16T19:29:16.787 INFO:tasks.workunit.client.0.smithi022.stdout:2/673: getdents d0/d5d 0 2024-02-16T19:29:16.788 INFO:tasks.workunit.client.1.smithi117.stdout:9/677: dwrite dd/d47/d5b/d5f/d77/fa7 [0,4194304] 0 2024-02-16T19:29:16.788 INFO:tasks.workunit.client.1.smithi117.stdout:2/799: creat d0/d13/dc4/de4/ffd x:0 0 0 2024-02-16T19:29:16.789 INFO:tasks.workunit.client.1.smithi117.stdout:9/678: stat dd/d47/d5b/fc6 0 2024-02-16T19:29:16.790 INFO:tasks.workunit.client.1.smithi117.stdout:9/679: truncate dd/d47/d5b/d5f/f6f 5100289 0 2024-02-16T19:29:16.792 INFO:tasks.workunit.client.0.smithi022.stdout:4/625: rmdir d4/d6 39 2024-02-16T19:29:16.795 INFO:tasks.workunit.client.0.smithi022.stdout:2/674: dwrite d0/d2/d2e/fba [0,4194304] 0 2024-02-16T19:29:16.799 INFO:tasks.workunit.client.0.smithi022.stdout:0/877: getdents d10/d7d/dac/de9/dee 0 2024-02-16T19:29:16.801 INFO:tasks.workunit.client.0.smithi022.stdout:9/718: mkdir db/d14/d16/d5d/d65/d6b/df6 0 2024-02-16T19:29:16.801 INFO:tasks.workunit.client.0.smithi022.stdout:7/708: rmdir d9/da/d2c/dcb 39 2024-02-16T19:29:16.803 INFO:tasks.workunit.client.1.smithi117.stdout:1/683: mknod d2/d19/d59/d76/d8c/dc7/cef 0 2024-02-16T19:29:16.804 INFO:tasks.workunit.client.0.smithi022.stdout:2/675: dwrite d0/d5d/fb8 [0,4194304] 0 2024-02-16T19:29:16.805 INFO:tasks.workunit.client.1.smithi117.stdout:1/684: chown d2/d19/d59/d89/d69/d80/c96 1 1 2024-02-16T19:29:16.806 INFO:tasks.workunit.client.0.smithi022.stdout:2/676: stat d0/d2/d18/d19/c98 0 2024-02-16T19:29:16.806 INFO:tasks.workunit.client.0.smithi022.stdout:1/738: dread - d6/d8/d22/d16/d17/d5d/fbf zero size 2024-02-16T19:29:16.813 INFO:tasks.workunit.client.0.smithi022.stdout:3/684: creat d7/d12/d18/dd/d3c/d5a/ff6 x:0 0 0 2024-02-16T19:29:16.814 INFO:tasks.workunit.client.1.smithi117.stdout:7/781: rename d4/d1c/d98/cbb to d4/d1c/d39/d46/d55/db8/c120 0 2024-02-16T19:29:16.814 INFO:tasks.workunit.client.1.smithi117.stdout:2/800: rmdir d0/d13/d4d/d80 39 2024-02-16T19:29:16.815 INFO:tasks.workunit.client.1.smithi117.stdout:8/807: getdents d4/d7/da6/d3a/d65/d96/da3 0 2024-02-16T19:29:16.815 INFO:tasks.workunit.client.1.smithi117.stdout:4/747: link d0/da/d21/d3d/d28/d64/d48/fab d0/da/d21/d3d/d28/d66/ff5 0 2024-02-16T19:29:16.817 INFO:tasks.workunit.client.1.smithi117.stdout:5/821: creat d2/d9/d3c/d24/d50/d52/f115 x:0 0 0 2024-02-16T19:29:16.818 INFO:tasks.workunit.client.1.smithi117.stdout:2/801: chown d0/d3/d34/d94/d82/cd6 58 1 2024-02-16T19:29:16.818 INFO:tasks.workunit.client.1.smithi117.stdout:0/795: dwrite d0/d9/d14/d23/f24 [0,4194304] 0 2024-02-16T19:29:16.819 INFO:tasks.workunit.client.0.smithi022.stdout:7/709: rmdir d9/da/d4e/d83 39 2024-02-16T19:29:16.819 INFO:tasks.workunit.client.1.smithi117.stdout:8/808: write d4/d7/d6a/f93 [1284836,57620] 0 2024-02-16T19:29:16.819 INFO:tasks.workunit.client.1.smithi117.stdout:4/748: chown d0/da/d21/d3d/d28/d64/d73/d91/d37/d98 292034 1 2024-02-16T19:29:16.820 INFO:tasks.workunit.client.0.smithi022.stdout:9/719: symlink db/d14/d16/d28/lf7 0 2024-02-16T19:29:16.821 INFO:tasks.workunit.client.1.smithi117.stdout:5/822: write d2/d3/d21/d49/d59/daa/de2/ff6 [557867,129166] 0 2024-02-16T19:29:16.821 INFO:tasks.workunit.client.0.smithi022.stdout:9/720: dread - db/d14/d16/d28/fd4 zero size 2024-02-16T19:29:16.821 INFO:tasks.workunit.client.1.smithi117.stdout:1/685: unlink d2/d1d/d77/cd7 0 2024-02-16T19:29:16.822 INFO:tasks.workunit.client.0.smithi022.stdout:5/740: link dd/d1f/d48/d21/d2f/d46/fae dd/ff9 0 2024-02-16T19:29:16.823 INFO:tasks.workunit.client.1.smithi117.stdout:1/686: write d2/d1d/d60/d7c/d9e/fc3 [1513720,125882] 0 2024-02-16T19:29:16.824 INFO:tasks.workunit.client.0.smithi022.stdout:2/677: rmdir d0/d2/d18 39 2024-02-16T19:29:16.827 INFO:tasks.workunit.client.0.smithi022.stdout:1/739: read d6/d8/d22/d47/d51/f85 [1886701,435] 0 2024-02-16T19:29:16.828 INFO:tasks.workunit.client.0.smithi022.stdout:6/746: getdents d1/d8/da/d29/deb/dfd/d85 0 2024-02-16T19:29:16.829 INFO:tasks.workunit.client.1.smithi117.stdout:1/687: dwrite d2/d19/d59/d89/d69/dad/fdb [0,4194304] 0 2024-02-16T19:29:16.831 INFO:tasks.workunit.client.0.smithi022.stdout:4/626: creat d4/d6/d14/d64/d66/fc3 x:0 0 0 2024-02-16T19:29:16.832 INFO:tasks.workunit.client.0.smithi022.stdout:5/741: dwrite dd/d1f/d48/d21/d2f/d52/f65 [0,4194304] 0 2024-02-16T19:29:16.833 INFO:tasks.workunit.client.0.smithi022.stdout:3/685: truncate d7/d12/d18/dd/d31/fae 287063 0 2024-02-16T19:29:16.834 INFO:tasks.workunit.client.0.smithi022.stdout:6/747: read d1/d8/d3c/d4d/f45 [378515,5675] 0 2024-02-16T19:29:16.834 INFO:tasks.workunit.client.0.smithi022.stdout:3/686: chown d7/d21/f23 0 1 2024-02-16T19:29:16.837 INFO:tasks.workunit.client.0.smithi022.stdout:6/748: chown d1/d8/d23/l79 96132 1 2024-02-16T19:29:16.838 INFO:tasks.workunit.client.0.smithi022.stdout:4/627: truncate d4/d6/d14/d64/d66/d6d/f89 4991450 0 2024-02-16T19:29:16.844 INFO:tasks.workunit.client.1.smithi117.stdout:1/688: dread d2/d19/d59/d89/f42 [0,4194304] 0 2024-02-16T19:29:16.844 INFO:tasks.workunit.client.0.smithi022.stdout:5/742: dwrite dd/d1f/d48/d21/d2f/d52/d77/d95/fec [0,4194304] 0 2024-02-16T19:29:16.846 INFO:tasks.workunit.client.0.smithi022.stdout:0/878: write d10/d15/f9f [4758525,110185] 0 2024-02-16T19:29:16.853 INFO:tasks.workunit.client.0.smithi022.stdout:1/740: mknod d6/d8/d6c/ce5 0 2024-02-16T19:29:16.860 INFO:tasks.workunit.client.0.smithi022.stdout:9/721: symlink db/d14/d16/d21/lf8 0 2024-02-16T19:29:16.864 INFO:tasks.workunit.client.0.smithi022.stdout:2/678: truncate d0/f78 372453 0 2024-02-16T19:29:16.864 INFO:tasks.workunit.client.0.smithi022.stdout:5/743: dread dd/d1f/d48/f74 [4194304,4194304] 0 2024-02-16T19:29:16.866 INFO:tasks.workunit.client.0.smithi022.stdout:8/756: getdents d3/d25 0 2024-02-16T19:29:16.867 INFO:tasks.workunit.client.0.smithi022.stdout:0/879: symlink d10/d15/d23/d38/l121 0 2024-02-16T19:29:16.869 INFO:tasks.workunit.client.0.smithi022.stdout:4/628: rename d4/d6/d14/d64/d66 to d4/d6/dd/d22/da8/dc4 0 2024-02-16T19:29:16.871 INFO:tasks.workunit.client.0.smithi022.stdout:1/741: fsync d6/d8/d22/d59/fc4 0 2024-02-16T19:29:16.875 INFO:tasks.workunit.client.0.smithi022.stdout:3/687: dread d7/d12/d18/dd/d24/d53/f5e [0,4194304] 0 2024-02-16T19:29:16.875 INFO:tasks.workunit.client.0.smithi022.stdout:3/688: stat d7/d12/d18/dd/d24/d58/f8b 0 2024-02-16T19:29:16.879 INFO:tasks.workunit.client.0.smithi022.stdout:9/722: dread db/d14/d16/d5d/d65/d6b/faf [0,4194304] 0 2024-02-16T19:29:16.883 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:16 smithi022 ceph-mon[29221]: pgmap v11: 65 pgs: 65 active+clean; 2.5 GiB data, 7.3 GiB used, 529 GiB / 536 GiB avail; 84 MiB/s rd, 198 MiB/s wr, 412 op/s 2024-02-16T19:29:16.884 INFO:tasks.workunit.client.0.smithi022.stdout:5/744: truncate dd/d1f/d48/f51 4512639 0 2024-02-16T19:29:16.885 INFO:tasks.workunit.client.0.smithi022.stdout:8/757: read - d3/d12/f9c zero size 2024-02-16T19:29:16.888 INFO:tasks.workunit.client.0.smithi022.stdout:2/679: dwrite d0/d2/d18/d19/d90/fa3 [4194304,4194304] 0 2024-02-16T19:29:16.891 INFO:tasks.workunit.client.0.smithi022.stdout:4/629: rename d4/d6/dd/df/d10/d24/d4b to d4/d6/dd/d22/d83/dc5 0 2024-02-16T19:29:16.894 INFO:tasks.workunit.client.0.smithi022.stdout:8/758: dwrite d3/d12/d2b/d2e/f87 [0,4194304] 0 2024-02-16T19:29:16.895 INFO:tasks.workunit.client.0.smithi022.stdout:8/759: chown d3/d12/d2b/dca 2055624441 1 2024-02-16T19:29:16.897 INFO:tasks.workunit.client.0.smithi022.stdout:8/760: truncate d3/dd3/fd7 715760 0 2024-02-16T19:29:16.898 INFO:tasks.workunit.client.0.smithi022.stdout:7/710: getdents d9/da/d1f/d40/dde 0 2024-02-16T19:29:16.902 INFO:tasks.workunit.client.1.smithi117.stdout:7/782: creat d4/d13/d63/ddd/f121 x:0 0 0 2024-02-16T19:29:16.903 INFO:tasks.workunit.client.0.smithi022.stdout:6/749: truncate d1/d8/da/f14 165549 0 2024-02-16T19:29:16.905 INFO:tasks.workunit.client.0.smithi022.stdout:9/723: symlink db/d14/dc8/dc9/lf9 0 2024-02-16T19:29:16.906 INFO:tasks.workunit.client.1.smithi117.stdout:3/859: getdents d1/dd/d32 0 2024-02-16T19:29:16.906 INFO:tasks.workunit.client.0.smithi022.stdout:7/711: dwrite d9/da/f62 [0,4194304] 0 2024-02-16T19:29:16.907 INFO:tasks.workunit.client.0.smithi022.stdout:3/689: read d7/d12/d18/dd/d24/d29/f3f [103106,74348] 0 2024-02-16T19:29:16.908 INFO:tasks.workunit.client.0.smithi022.stdout:7/712: chown d9/c4f 658 1 2024-02-16T19:29:16.908 INFO:tasks.workunit.client.1.smithi117.stdout:8/809: mkdir d4/d94/d109 0 2024-02-16T19:29:16.910 INFO:tasks.workunit.client.1.smithi117.stdout:4/749: mknod d0/da/d21/d3d/d28/d64/d73/d91/d37/d98/cf6 0 2024-02-16T19:29:16.911 INFO:tasks.workunit.client.1.smithi117.stdout:5/823: symlink d2/d9/d3c/d24/l116 0 2024-02-16T19:29:16.912 INFO:tasks.workunit.client.0.smithi022.stdout:5/745: mknod dd/d1f/d48/d21/d5d/dc0/cfa 0 2024-02-16T19:29:16.913 INFO:tasks.workunit.client.1.smithi117.stdout:4/750: truncate d0/da/d21/d3d/d28/d64/d73/fdf 47056 0 2024-02-16T19:29:16.913 INFO:tasks.workunit.client.0.smithi022.stdout:2/680: creat d0/d5d/dc4/fdc x:0 0 0 2024-02-16T19:29:16.917 INFO:tasks.workunit.client.0.smithi022.stdout:4/630: mknod d4/d6/dd/d22/d83/dc5/cc6 0 2024-02-16T19:29:16.918 INFO:tasks.workunit.client.1.smithi117.stdout:1/689: mkdir d2/d1d/d60/d7c/d9e/df0 0 2024-02-16T19:29:16.919 INFO:tasks.workunit.client.1.smithi117.stdout:1/690: rename d2/d1d/d77 to d2/d1d/d77/df1 22 2024-02-16T19:29:16.919 INFO:tasks.workunit.client.0.smithi022.stdout:1/742: dread - d6/d8/d22/d3f/f84 zero size 2024-02-16T19:29:16.920 INFO:tasks.workunit.client.1.smithi117.stdout:9/680: link dd/d45/d31/l5d dd/d23/d64/le4 0 2024-02-16T19:29:16.920 INFO:tasks.workunit.client.1.smithi117.stdout:9/681: stat dd/d23/d64/c4b 0 2024-02-16T19:29:16.922 INFO:tasks.workunit.client.1.smithi117.stdout:7/783: creat d4/d13/f122 x:0 0 0 2024-02-16T19:29:16.922 INFO:tasks.workunit.client.1.smithi117.stdout:7/784: fsync d4/f118 0 2024-02-16T19:29:16.924 INFO:tasks.workunit.client.1.smithi117.stdout:3/860: mkdir d1/d67/d128 0 2024-02-16T19:29:16.925 INFO:tasks.workunit.client.0.smithi022.stdout:6/750: rmdir d1/d8/d23/dc6/ddb 39 2024-02-16T19:29:16.928 INFO:tasks.workunit.client.0.smithi022.stdout:9/724: mknod db/d14/d16/d5d/d65/d6b/dd5/cfa 0 2024-02-16T19:29:16.930 INFO:tasks.workunit.client.1.smithi117.stdout:6/752: sync 2024-02-16T19:29:16.930 INFO:tasks.workunit.client.1.smithi117.stdout:8/810: creat d4/d7/da6/d3a/d43/f10a x:0 0 0 2024-02-16T19:29:16.932 INFO:tasks.workunit.client.1.smithi117.stdout:4/751: dread d0/da/d21/d3d/d28/d64/d8a/fb1 [0,4194304] 0 2024-02-16T19:29:16.933 INFO:tasks.workunit.client.1.smithi117.stdout:4/752: readlink d0/da/d21/d3d/d28/d64/d48/l82 0 2024-02-16T19:29:16.933 INFO:tasks.workunit.client.0.smithi022.stdout:7/713: creat d9/da/d4e/d7a/fea x:0 0 0 2024-02-16T19:29:16.934 INFO:tasks.workunit.client.0.smithi022.stdout:3/690: dread d7/d12/d18/dd/d24/d53/fe4 [0,4194304] 0 2024-02-16T19:29:16.934 INFO:tasks.workunit.client.1.smithi117.stdout:1/691: chown d2/d4/dbb/c9f 19662939 1 2024-02-16T19:29:16.935 INFO:tasks.workunit.client.0.smithi022.stdout:0/880: creat d10/d7d/f122 x:0 0 0 2024-02-16T19:29:16.937 INFO:tasks.workunit.client.1.smithi117.stdout:9/682: unlink dd/l1b 0 2024-02-16T19:29:16.937 INFO:tasks.workunit.client.1.smithi117.stdout:9/683: readlink dd/db5/l54 0 2024-02-16T19:29:16.938 INFO:tasks.workunit.client.0.smithi022.stdout:5/746: mkdir dd/d1f/d48/d21/d2f/d52/d5c/d61/d6a/dfb 0 2024-02-16T19:29:16.941 INFO:tasks.workunit.client.1.smithi117.stdout:3/861: mknod d1/dd/d5c/d85/c129 0 2024-02-16T19:29:16.943 INFO:tasks.workunit.client.0.smithi022.stdout:4/631: rename d4/d38 to d4/d6/d14/dc7 0 2024-02-16T19:29:16.945 INFO:tasks.workunit.client.1.smithi117.stdout:6/753: rmdir d5/d4b/d20/da4 39 2024-02-16T19:29:16.947 INFO:tasks.workunit.client.1.smithi117.stdout:8/811: fsync d4/d73/fdb 0 2024-02-16T19:29:16.948 INFO:tasks.workunit.client.1.smithi117.stdout:8/812: write d4/d7/da6/d3a/d65/d7c/f107 [376995,30062] 0 2024-02-16T19:29:16.951 INFO:tasks.workunit.client.1.smithi117.stdout:4/753: mknod d0/da/d4a/cf7 0 2024-02-16T19:29:16.954 INFO:tasks.workunit.client.0.smithi022.stdout:0/881: creat d10/d11/d50/f123 x:0 0 0 2024-02-16T19:29:16.956 INFO:tasks.workunit.client.1.smithi117.stdout:9/684: read dd/d47/d5e/f7f [2808910,111925] 0 2024-02-16T19:29:16.959 INFO:tasks.workunit.client.0.smithi022.stdout:2/681: fsync d0/d2/d18/d19/d40/f6d 0 2024-02-16T19:29:16.959 INFO:tasks.workunit.client.1.smithi117.stdout:4/754: dwrite d0/da/d21/f40 [4194304,4194304] 0 2024-02-16T19:29:16.959 INFO:tasks.workunit.client.1.smithi117.stdout:0/796: dwrite d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/d27/d83/fb6 [0,4194304] 0 2024-02-16T19:29:16.960 INFO:tasks.workunit.client.0.smithi022.stdout:2/682: stat d0/d32/d3b/d6f/l81 0 2024-02-16T19:29:16.960 INFO:tasks.workunit.client.1.smithi117.stdout:4/755: chown d0/da/fa4 499051851 1 2024-02-16T19:29:16.961 INFO:tasks.workunit.client.1.smithi117.stdout:3/862: fdatasync d1/d1b/f41 0 2024-02-16T19:29:16.962 INFO:tasks.workunit.client.0.smithi022.stdout:4/632: rmdir d4/d6/d14/d64/d4e/d54/d5a 39 2024-02-16T19:29:16.964 INFO:tasks.workunit.client.0.smithi022.stdout:5/747: dwrite dd/d1f/d48/d21/d2f/d52/d5c/f85 [0,4194304] 0 2024-02-16T19:29:16.965 INFO:tasks.workunit.client.1.smithi117.stdout:6/754: mkdir d5/d4b/d44/d52/d81/dd7/df9 0 2024-02-16T19:29:16.965 INFO:tasks.workunit.client.1.smithi117.stdout:0/797: stat d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/db7 0 2024-02-16T19:29:16.966 INFO:tasks.workunit.client.1.smithi117.stdout:9/685: dwrite dd/d47/d48/f86 [0,4194304] 0 2024-02-16T19:29:16.970 INFO:tasks.workunit.client.0.smithi022.stdout:8/761: creat d3/f107 x:0 0 0 2024-02-16T19:29:16.979 INFO:tasks.workunit.client.0.smithi022.stdout:0/882: symlink d10/d11/d27/l124 0 2024-02-16T19:29:16.980 INFO:tasks.workunit.client.1.smithi117.stdout:1/692: mkdir d2/d1d/d60/df2 0 2024-02-16T19:29:16.981 INFO:tasks.workunit.client.0.smithi022.stdout:3/691: dread d7/d12/f44 [0,4194304] 0 2024-02-16T19:29:16.982 INFO:tasks.workunit.client.0.smithi022.stdout:3/692: chown d7/d12/d18/dd/d24/d58/d70/f7b 4157721 1 2024-02-16T19:29:16.984 INFO:tasks.workunit.client.1.smithi117.stdout:7/785: creat d4/d1c/d39/d9f/de1/f123 x:0 0 0 2024-02-16T19:29:16.991 INFO:tasks.workunit.client.1.smithi117.stdout:3/863: unlink d1/dd/d5c/fc1 0 2024-02-16T19:29:16.991 INFO:tasks.workunit.client.0.smithi022.stdout:2/683: rename d0/d5d/fb8 to d0/d2/d18/d1f/d6c/d99/fdd 0 2024-02-16T19:29:16.992 INFO:tasks.workunit.client.0.smithi022.stdout:5/748: creat dd/d1f/d48/d21/d5d/dc0/ffc x:0 0 0 2024-02-16T19:29:16.992 INFO:tasks.workunit.client.1.smithi117.stdout:4/756: unlink d0/da/d21/d3d/d28/d64/d73/d91/fbe 0 2024-02-16T19:29:16.995 INFO:tasks.workunit.client.1.smithi117.stdout:9/686: symlink dd/db5/le5 0 2024-02-16T19:29:16.996 INFO:tasks.workunit.client.0.smithi022.stdout:1/743: write d6/f6b [1012607,8987] 0 2024-02-16T19:29:17.003 INFO:tasks.workunit.client.1.smithi117.stdout:1/693: write d2/d1d/d1f/fc4 [3203316,82305] 0 2024-02-16T19:29:17.003 INFO:tasks.workunit.client.0.smithi022.stdout:7/714: dwrite d9/fa7 [0,4194304] 0 2024-02-16T19:29:17.005 INFO:tasks.workunit.client.0.smithi022.stdout:8/762: creat d3/d12/d2b/da9/dd5/f108 x:0 0 0 2024-02-16T19:29:17.008 INFO:tasks.workunit.client.0.smithi022.stdout:2/684: truncate d0/d5d/fb0 52258 0 2024-02-16T19:29:17.008 INFO:tasks.workunit.client.1.smithi117.stdout:7/786: rmdir d4/d1c/d39/d9f/de1 39 2024-02-16T19:29:17.008 INFO:tasks.workunit.client.1.smithi117.stdout:2/802: sync 2024-02-16T19:29:17.011 INFO:tasks.workunit.client.1.smithi117.stdout:2/803: write d0/d3/d34/d94/d46/d68/d76/da1/d56/d60/dc9/fea [124680,17122] 0 2024-02-16T19:29:17.012 INFO:tasks.workunit.client.1.smithi117.stdout:2/804: dread - d0/d3/d34/d94/d82/fed zero size 2024-02-16T19:29:17.015 INFO:tasks.workunit.client.1.smithi117.stdout:3/864: write d1/d1b/f41 [6266,33822] 0 2024-02-16T19:29:17.016 INFO:tasks.workunit.client.0.smithi022.stdout:1/744: creat d6/d8/d22/d16/d17/d63/fe6 x:0 0 0 2024-02-16T19:29:17.017 INFO:tasks.workunit.client.1.smithi117.stdout:6/755: mkdir d5/dfa 0 2024-02-16T19:29:17.018 INFO:tasks.workunit.client.1.smithi117.stdout:6/756: dread - d5/d4b/d44/db1/fe2 zero size 2024-02-16T19:29:17.019 INFO:tasks.workunit.client.1.smithi117.stdout:4/757: mknod d0/da/d21/d3d/d28/d64/d73/d91/d37/cf8 0 2024-02-16T19:29:17.023 INFO:tasks.workunit.client.0.smithi022.stdout:4/633: rename d4/d6/dd/d22/d83/d9e/fab to d4/d6/d3a/fc8 0 2024-02-16T19:29:17.025 INFO:tasks.workunit.client.0.smithi022.stdout:7/715: mkdir d9/da/d1f/d40/deb 0 2024-02-16T19:29:17.026 INFO:tasks.workunit.client.1.smithi117.stdout:8/813: link d4/d7/da6/d3a/d43/d44/l5d d4/d7/da6/l10b 0 2024-02-16T19:29:17.028 INFO:tasks.workunit.client.0.smithi022.stdout:8/763: creat d3/d12/d17/d64/d8f/f109 x:0 0 0 2024-02-16T19:29:17.029 INFO:tasks.workunit.client.0.smithi022.stdout:0/883: dwrite d10/d11/d27/d77/f8e [0,4194304] 0 2024-02-16T19:29:17.029 INFO:tasks.workunit.client.0.smithi022.stdout:8/764: fsync d3/d12/da6/f100 0 2024-02-16T19:29:17.031 INFO:tasks.workunit.client.0.smithi022.stdout:2/685: unlink d0/d2/c93 0 2024-02-16T19:29:17.032 INFO:tasks.workunit.client.0.smithi022.stdout:8/765: dread d3/dd3/fd7 [0,4194304] 0 2024-02-16T19:29:17.033 INFO:tasks.workunit.client.0.smithi022.stdout:8/766: chown d3/d12/d6a 2750 1 2024-02-16T19:29:17.036 INFO:tasks.workunit.client.1.smithi117.stdout:2/805: dread - d0/d13/d4d/d80/f9c zero size 2024-02-16T19:29:17.037 INFO:tasks.workunit.client.1.smithi117.stdout:7/787: dread d4/d1c/d87/f66 [0,4194304] 0 2024-02-16T19:29:17.038 INFO:tasks.workunit.client.0.smithi022.stdout:1/745: dread - d6/d8/d22/d59/da7/fa8 zero size 2024-02-16T19:29:17.039 INFO:tasks.workunit.client.1.smithi117.stdout:2/806: dread - d0/d3/d34/d94/d46/d68/d76/da1/d56/d60/dc9/fd8 zero size 2024-02-16T19:29:17.040 INFO:tasks.workunit.client.1.smithi117.stdout:6/757: unlink d5/db/f55 0 2024-02-16T19:29:17.041 INFO:tasks.workunit.client.0.smithi022.stdout:2/686: dwrite d0/d32/fbb [0,4194304] 0 2024-02-16T19:29:17.043 INFO:tasks.workunit.client.1.smithi117.stdout:4/758: unlink d0/da/d21/d3d/d28/fb7 0 2024-02-16T19:29:17.043 INFO:tasks.workunit.client.0.smithi022.stdout:3/693: link d7/d12/d18/dd/d3c/cad d7/d12/d18/dd/d98/cf7 0 2024-02-16T19:29:17.046 INFO:tasks.workunit.client.0.smithi022.stdout:0/884: dread d10/d11/d27/d3f/df0/ff1 [0,4194304] 0 2024-02-16T19:29:17.048 INFO:tasks.workunit.client.0.smithi022.stdout:7/716: creat d9/da/d2c/d2f/d21/fec x:0 0 0 2024-02-16T19:29:17.049 INFO:tasks.workunit.client.1.smithi117.stdout:8/814: dread d4/d7/da6/d50/fc3 [0,4194304] 0 2024-02-16T19:29:17.050 INFO:tasks.workunit.client.1.smithi117.stdout:1/694: mkdir d2/d1d/d60/d7c/d9e/da5/db0/dba/df3 0 2024-02-16T19:29:17.051 INFO:tasks.workunit.client.0.smithi022.stdout:5/749: creat dd/d1f/d48/d21/d2f/d52/d5c/ffd x:0 0 0 2024-02-16T19:29:17.053 INFO:tasks.workunit.client.0.smithi022.stdout:8/767: creat d3/d12/d2b/dca/f10a x:0 0 0 2024-02-16T19:29:17.056 INFO:tasks.workunit.client.1.smithi117.stdout:8/815: dread d4/f3f [0,4194304] 0 2024-02-16T19:29:17.059 INFO:tasks.workunit.client.1.smithi117.stdout:6/758: creat d5/db/d7d/d32/da8/de1/ffb x:0 0 0 2024-02-16T19:29:17.060 INFO:tasks.workunit.client.0.smithi022.stdout:1/746: truncate d6/d8/d22/f3d 4860788 0 2024-02-16T19:29:17.060 INFO:tasks.workunit.client.1.smithi117.stdout:9/687: truncate dd/d47/d5b/d5f/f6f 3899375 0 2024-02-16T19:29:17.061 INFO:tasks.workunit.client.1.smithi117.stdout:4/759: creat d0/da/d21/d3d/d28/d64/d90/dc0/dd7/ff9 x:0 0 0 2024-02-16T19:29:17.061 INFO:tasks.workunit.client.1.smithi117.stdout:9/688: dread dd/d45/d73/f99 [0,4194304] 0 2024-02-16T19:29:17.062 INFO:tasks.workunit.client.0.smithi022.stdout:4/634: write d4/d6/db6/f78 [5248144,109693] 0 2024-02-16T19:29:17.064 INFO:tasks.workunit.client.1.smithi117.stdout:0/798: rename d0/d9/d14/d23/d33/c84 to d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/d27/c111 0 2024-02-16T19:29:17.065 INFO:tasks.workunit.client.0.smithi022.stdout:0/885: mknod d10/d11/d27/c125 0 2024-02-16T19:29:17.066 INFO:tasks.workunit.client.0.smithi022.stdout:0/886: dread - d10/d11/d79/f11d zero size 2024-02-16T19:29:17.067 INFO:tasks.workunit.client.1.smithi117.stdout:9/689: dwrite dd/d45/d73/f99 [0,4194304] 0 2024-02-16T19:29:17.072 INFO:tasks.workunit.client.0.smithi022.stdout:5/750: unlink dd/d1f/d48/d21/d2f/d40/fbf 0 2024-02-16T19:29:17.075 INFO:tasks.workunit.client.1.smithi117.stdout:7/788: write d4/d13/d63/fce [1877190,119856] 0 2024-02-16T19:29:17.075 INFO:tasks.workunit.client.1.smithi117.stdout:1/695: dwrite d2/d4/fa3 [0,4194304] 0 2024-02-16T19:29:17.075 INFO:tasks.workunit.client.0.smithi022.stdout:5/751: truncate dd/d1f/d48/d21/d2f/d52/d5c/f85 5102011 0 2024-02-16T19:29:17.076 INFO:tasks.workunit.client.0.smithi022.stdout:2/687: mkdir d0/d2/dde 0 2024-02-16T19:29:17.077 INFO:tasks.workunit.client.0.smithi022.stdout:5/752: readlink dd/d1f/d48/d21/d2f/d46/d5e/d84/l81 0 2024-02-16T19:29:17.077 INFO:tasks.workunit.client.0.smithi022.stdout:1/747: mknod d6/d8/d22/d16/d17/ce7 0 2024-02-16T19:29:17.078 INFO:tasks.workunit.client.1.smithi117.stdout:7/789: truncate d4/d1c/d87/d89/d5f/dae/f104 5033760 0 2024-02-16T19:29:17.079 INFO:tasks.workunit.client.0.smithi022.stdout:2/688: truncate d0/d2/d18/d19/d90/fbd 9271818 0 2024-02-16T19:29:17.080 INFO:tasks.workunit.client.0.smithi022.stdout:5/753: dread - dd/d1f/d48/d21/d2f/d46/d5e/ff2 zero size 2024-02-16T19:29:17.080 INFO:tasks.workunit.client.0.smithi022.stdout:5/754: fdatasync f6 0 2024-02-16T19:29:17.082 INFO:tasks.workunit.client.0.smithi022.stdout:4/635: symlink d4/d6/d14/lc9 0 2024-02-16T19:29:17.082 INFO:tasks.workunit.client.1.smithi117.stdout:4/760: unlink d0/da/d21/d3d/d28/d66/d97/d6d/c84 0 2024-02-16T19:29:17.082 INFO:tasks.workunit.client.1.smithi117.stdout:9/690: dread dd/d47/d48/f55 [0,4194304] 0 2024-02-16T19:29:17.082 INFO:tasks.workunit.client.1.smithi117.stdout:4/761: chown d0/da/d21/d3d/d28/d64/d8a/d92 385 1 2024-02-16T19:29:17.084 INFO:tasks.workunit.client.0.smithi022.stdout:0/887: fdatasync d10/d15/f39 0 2024-02-16T19:29:17.085 INFO:tasks.workunit.client.0.smithi022.stdout:7/717: mkdir d9/da/d2c/ded 0 2024-02-16T19:29:17.087 INFO:tasks.workunit.client.0.smithi022.stdout:7/718: dread - d9/da/d2c/d6d/fe0 zero size 2024-02-16T19:29:17.088 INFO:tasks.workunit.client.1.smithi117.stdout:9/691: dwrite dd/d23/f18 [0,4194304] 0 2024-02-16T19:29:17.088 INFO:tasks.workunit.client.0.smithi022.stdout:8/768: unlink d3/d12/d2b/da9/cb3 0 2024-02-16T19:29:17.089 INFO:tasks.workunit.client.0.smithi022.stdout:5/755: dwrite dd/d1f/d48/d21/d2f/d40/d4e/d56/f59 [0,4194304] 0 2024-02-16T19:29:17.089 INFO:tasks.workunit.client.0.smithi022.stdout:8/769: chown d3/d25/ddb/de9 1655090000 1 2024-02-16T19:29:17.090 INFO:tasks.workunit.client.1.smithi117.stdout:3/865: link d1/d1b/c1d d1/dd/df8/c12a 0 2024-02-16T19:29:17.090 INFO:tasks.workunit.client.0.smithi022.stdout:3/694: creat d7/d12/d18/dd/d24/d58/ff8 x:0 0 0 2024-02-16T19:29:17.095 INFO:tasks.workunit.client.0.smithi022.stdout:1/748: creat d6/d8/d22/d67/d7e/fe8 x:0 0 0 2024-02-16T19:29:17.097 INFO:tasks.workunit.client.1.smithi117.stdout:3/866: dwrite d1/dd/d3b/f3d [4194304,4194304] 0 2024-02-16T19:29:17.105 INFO:tasks.workunit.client.0.smithi022.stdout:4/636: symlink d4/d6/dd/df/d10/d55/lca 0 2024-02-16T19:29:17.114 INFO:tasks.workunit.client.0.smithi022.stdout:4/637: dwrite d4/d6/dd/d22/da8/dc4/d6d/faa [0,4194304] 0 2024-02-16T19:29:17.119 INFO:tasks.workunit.client.0.smithi022.stdout:8/770: mkdir d3/d12/d2b/da9/dd5/d10b 0 2024-02-16T19:29:17.121 INFO:tasks.workunit.client.0.smithi022.stdout:3/695: creat d7/d12/d18/dd/d98/ff9 x:0 0 0 2024-02-16T19:29:17.123 INFO:tasks.workunit.client.1.smithi117.stdout:9/692: dread dd/db5/f4e [0,4194304] 0 2024-02-16T19:29:17.123 INFO:tasks.workunit.client.0.smithi022.stdout:3/696: dread - d7/d12/d18/dd/d3c/d5a/ff6 zero size 2024-02-16T19:29:17.124 INFO:tasks.workunit.client.0.smithi022.stdout:5/756: rename dd/d1f/d48/d21/d2f/d52/d5c/d61/d8f/f94 to dd/d1f/d48/d21/d2f/d52/d5c/d61/d6a/dfb/ffe 0 2024-02-16T19:29:17.126 INFO:tasks.workunit.client.0.smithi022.stdout:3/697: read d7/d12/d18/dd/d24/d58/d70/f8e [2818593,117634] 0 2024-02-16T19:29:17.127 INFO:tasks.workunit.client.0.smithi022.stdout:2/689: mkdir d0/d2/d18/d19/ddf 0 2024-02-16T19:29:17.139 INFO:tasks.workunit.client.1.smithi117.stdout:5/824: sync 2024-02-16T19:29:17.141 INFO:tasks.workunit.client.1.smithi117.stdout:3/867: rename d1/d83/d96/fb6 to d1/dd/d3b/f12b 0 2024-02-16T19:29:17.146 INFO:tasks.workunit.client.1.smithi117.stdout:3/868: dwrite d1/d1b/d1e/f22 [0,4194304] 0 2024-02-16T19:29:17.147 INFO:tasks.workunit.client.0.smithi022.stdout:8/771: dread - d3/d12/d2b/d46/f91 zero size 2024-02-16T19:29:17.149 INFO:tasks.workunit.client.1.smithi117.stdout:8/816: getdents d4/d12/d2a/d64/dec/d76 0 2024-02-16T19:29:17.150 INFO:tasks.workunit.client.0.smithi022.stdout:5/757: unlink dd/l28 0 2024-02-16T19:29:17.207 INFO:tasks.workunit.client.0.smithi022.stdout:2/690: rename d0/d32/d3b/d6f/f5a to d0/d2/d18/d1f/d6c/fe0 0 2024-02-16T19:29:17.208 INFO:tasks.workunit.client.1.smithi117.stdout:6/759: getdents d5/d4b/d44/d52/d81 0 2024-02-16T19:29:17.213 INFO:tasks.workunit.client.0.smithi022.stdout:7/719: dwrite d9/da/f60 [0,4194304] 0 2024-02-16T19:29:17.213 INFO:tasks.workunit.client.1.smithi117.stdout:7/790: rename d4/d13/d63/ddd/f10b to d4/d13/da3/f124 0 2024-02-16T19:29:17.214 INFO:tasks.workunit.client.0.smithi022.stdout:7/720: fsync d9/da/d1f/d40/f61 0 2024-02-16T19:29:17.215 INFO:tasks.workunit.client.0.smithi022.stdout:4/638: symlink d4/d6/d14/d64/d4e/d54/d5a/lcb 0 2024-02-16T19:29:17.218 INFO:tasks.workunit.client.0.smithi022.stdout:8/772: creat d3/d12/d2b/d2e/f10c x:0 0 0 2024-02-16T19:29:17.219 INFO:tasks.workunit.client.1.smithi117.stdout:0/799: getdents d0/d9/d14/d23 0 2024-02-16T19:29:17.220 INFO:tasks.workunit.client.1.smithi117.stdout:0/800: chown d0/d9/d14/d1d/d60/d76/d7b/ld6 396183518 1 2024-02-16T19:29:17.223 INFO:tasks.workunit.client.1.smithi117.stdout:2/807: sync 2024-02-16T19:29:17.224 INFO:tasks.workunit.client.0.smithi022.stdout:5/758: read dd/d1f/d48/d21/d2f/d52/d5c/f63 [60565,66455] 0 2024-02-16T19:29:17.225 INFO:tasks.workunit.client.0.smithi022.stdout:2/691: mkdir d0/d5d/dc4/de1 0 2024-02-16T19:29:17.229 INFO:tasks.workunit.client.0.smithi022.stdout:1/749: mkdir d6/d8/d22/d16/d17/d5d/de9 0 2024-02-16T19:29:17.230 INFO:tasks.workunit.client.1.smithi117.stdout:1/696: write d2/d19/d59/d89/d69/f8f [508553,114630] 0 2024-02-16T19:29:17.231 INFO:tasks.workunit.client.1.smithi117.stdout:1/697: chown d2/d19/d59/d89/d69/dad 9895791 1 2024-02-16T19:29:17.233 INFO:tasks.workunit.client.1.smithi117.stdout:1/698: stat d2/d19/d59/d89/d69/d80/de6 0 2024-02-16T19:29:17.233 INFO:tasks.workunit.client.0.smithi022.stdout:5/759: dwrite dd/d1f/d48/d21/d5d/de7/f3f [0,4194304] 0 2024-02-16T19:29:17.234 INFO:tasks.workunit.client.1.smithi117.stdout:6/760: truncate d5/db/d7d/f40 551915 0 2024-02-16T19:29:17.235 INFO:tasks.workunit.client.1.smithi117.stdout:0/801: dread d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/d27/d4e/f61 [0,4194304] 0 2024-02-16T19:29:17.235 INFO:tasks.workunit.client.0.smithi022.stdout:7/721: dread d9/da/d2c/d46/d73/fe1 [0,4194304] 0 2024-02-16T19:29:17.236 INFO:tasks.workunit.client.0.smithi022.stdout:5/760: write dd/d1f/d2e/fd2 [854221,32902] 0 2024-02-16T19:29:17.236 INFO:tasks.workunit.client.0.smithi022.stdout:7/722: write d9/da/d1f/fc5 [698360,22277] 0 2024-02-16T19:29:17.239 INFO:tasks.workunit.client.0.smithi022.stdout:5/761: stat dd/d1f/d48/d21/d5d/de7/f83 0 2024-02-16T19:29:17.239 INFO:tasks.workunit.client.0.smithi022.stdout:7/723: fsync d9/da/f62 0 2024-02-16T19:29:17.239 INFO:tasks.workunit.client.1.smithi117.stdout:4/762: truncate d0/da/d21/d3d/d28/d64/fd3 827318 0 2024-02-16T19:29:17.243 INFO:tasks.workunit.client.0.smithi022.stdout:0/888: dwrite d10/f10f [0,4194304] 0 2024-02-16T19:29:17.248 INFO:tasks.workunit.client.1.smithi117.stdout:2/808: symlink d0/d3/d22/d37/d9d/lfe 0 2024-02-16T19:29:17.253 INFO:tasks.workunit.client.1.smithi117.stdout:8/817: dread d4/d7/da6/d3a/d54/f78 [0,4194304] 0 2024-02-16T19:29:17.254 INFO:tasks.workunit.client.1.smithi117.stdout:1/699: mkdir d2/d19/d59/d89/df4 0 2024-02-16T19:29:17.254 INFO:tasks.workunit.client.0.smithi022.stdout:2/692: creat d0/d2/d18/d1f/d6c/fe2 x:0 0 0 2024-02-16T19:29:17.256 INFO:tasks.workunit.client.1.smithi117.stdout:9/693: write dd/d23/f3b [3241240,72031] 0 2024-02-16T19:29:17.260 INFO:tasks.workunit.client.1.smithi117.stdout:3/869: dwrite d1/dd/d3b/d9d/fcd [0,4194304] 0 2024-02-16T19:29:17.261 INFO:tasks.workunit.client.1.smithi117.stdout:4/763: mknod d0/da/d4a/cfa 0 2024-02-16T19:29:17.262 INFO:tasks.workunit.client.1.smithi117.stdout:3/870: dread - d1/d1b/d1e/d37/ddc/f10e zero size 2024-02-16T19:29:17.265 INFO:tasks.workunit.client.1.smithi117.stdout:7/791: write d4/d13/d63/fa2 [2803578,28732] 0 2024-02-16T19:29:17.268 INFO:tasks.workunit.client.1.smithi117.stdout:1/700: rmdir d2/d19/d59/d89/d7b 39 2024-02-16T19:29:17.270 INFO:tasks.workunit.client.0.smithi022.stdout:5/762: mknod dd/d1f/d48/d21/d5d/dc0/cff 0 2024-02-16T19:29:17.271 INFO:tasks.workunit.client.1.smithi117.stdout:6/761: dread d5/db/d7d/d32/d37/f58 [0,4194304] 0 2024-02-16T19:29:17.271 INFO:tasks.workunit.client.1.smithi117.stdout:8/818: mknod d4/d7/da6/d3a/d43/c10c 0 2024-02-16T19:29:17.272 INFO:tasks.workunit.client.1.smithi117.stdout:1/701: dread d2/d4/fa3 [0,4194304] 0 2024-02-16T19:29:17.273 INFO:tasks.workunit.client.1.smithi117.stdout:1/702: dread - d2/d19/d59/d89/d69/fe0 zero size 2024-02-16T19:29:17.275 INFO:tasks.workunit.client.1.smithi117.stdout:0/802: creat d0/d9/d14/d1d/d60/dc3/d109/f112 x:0 0 0 2024-02-16T19:29:17.276 INFO:tasks.workunit.client.0.smithi022.stdout:0/889: creat d10/d22/dad/de7/f126 x:0 0 0 2024-02-16T19:29:17.277 INFO:tasks.workunit.client.1.smithi117.stdout:5/825: sync 2024-02-16T19:29:17.277 INFO:tasks.workunit.client.1.smithi117.stdout:9/694: mkdir dd/d47/d5b/d5f/de6 0 2024-02-16T19:29:17.278 INFO:tasks.workunit.client.1.smithi117.stdout:0/803: chown d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/d27/d4e/fa5 8336464 1 2024-02-16T19:29:17.279 INFO:tasks.workunit.client.1.smithi117.stdout:1/703: dwrite d2/d4/fa3 [0,4194304] 0 2024-02-16T19:29:17.280 INFO:tasks.workunit.client.0.smithi022.stdout:2/693: read d0/d32/f97 [3844452,3914] 0 2024-02-16T19:29:17.281 INFO:tasks.workunit.client.0.smithi022.stdout:7/724: write d9/da/d2c/d6d/f8b [557448,1701] 0 2024-02-16T19:29:17.282 INFO:tasks.workunit.client.1.smithi117.stdout:5/826: write d2/d9/d3c/d24/d50/d96/dbc/d30/d38/fdd [184599,93491] 0 2024-02-16T19:29:17.286 INFO:tasks.workunit.client.1.smithi117.stdout:4/764: readlink d0/da/d21/d3d/d28/d66/d97/la0 0 2024-02-16T19:29:17.286 INFO:tasks.workunit.client.0.smithi022.stdout:4/639: write d4/d6/dd/df/d10/f25 [3738338,36819] 0 2024-02-16T19:29:17.288 INFO:tasks.workunit.client.0.smithi022.stdout:8/773: rename d3/d12/d2b/da9/d106 to d3/d12/d17/d10d 0 2024-02-16T19:29:17.289 INFO:tasks.workunit.client.0.smithi022.stdout:2/694: dwrite d0/d2/d18/d19/d90/fa3 [8388608,4194304] 0 2024-02-16T19:29:17.290 INFO:tasks.workunit.client.1.smithi117.stdout:2/809: mkdir d0/d3/dff 0 2024-02-16T19:29:17.295 INFO:tasks.workunit.client.0.smithi022.stdout:7/725: creat d9/da9/dbc/fee x:0 0 0 2024-02-16T19:29:17.297 INFO:tasks.workunit.client.0.smithi022.stdout:8/774: dwrite d3/f55 [0,4194304] 0 2024-02-16T19:29:17.297 INFO:tasks.workunit.client.0.smithi022.stdout:7/726: chown d9/da/d1f/d40/fa5 64137 1 2024-02-16T19:29:17.297 INFO:tasks.workunit.client.0.smithi022.stdout:2/695: dread d0/d2/d18/d1f/d6c/d99/fdd [0,4194304] 0 2024-02-16T19:29:17.298 INFO:tasks.workunit.client.0.smithi022.stdout:8/775: read - d3/d12/d2b/d54/db8/fda zero size 2024-02-16T19:29:17.300 INFO:tasks.workunit.client.1.smithi117.stdout:6/762: unlink d5/d4b/c9d 0 2024-02-16T19:29:17.302 INFO:tasks.workunit.client.0.smithi022.stdout:5/763: rename dd/d1f/d76/f96 to dd/d1f/d2e/dd4/f100 0 2024-02-16T19:29:17.304 INFO:tasks.workunit.client.1.smithi117.stdout:9/695: creat dd/d47/d48/dc0/fe7 x:0 0 0 2024-02-16T19:29:17.306 INFO:tasks.workunit.client.0.smithi022.stdout:8/776: dwrite d3/d12/da6/f5e [0,4194304] 0 2024-02-16T19:29:17.307 INFO:tasks.workunit.client.0.smithi022.stdout:8/777: stat d3/d12/l89 0 2024-02-16T19:29:17.309 INFO:tasks.workunit.client.0.smithi022.stdout:8/778: write d3/d12/d2b/fa3 [3499863,11490] 0 2024-02-16T19:29:17.316 INFO:tasks.workunit.client.0.smithi022.stdout:4/640: rename d4/d6/dd/df/d10/d55/cc2 to d4/d6/db6/ccc 0 2024-02-16T19:29:17.318 INFO:tasks.workunit.client.0.smithi022.stdout:4/641: write d4/d6/d14/dc7/d6a/fa9 [453782,123423] 0 2024-02-16T19:29:17.320 INFO:tasks.workunit.client.0.smithi022.stdout:0/890: getdents d10/d11/d27 0 2024-02-16T19:29:17.327 INFO:tasks.workunit.client.1.smithi117.stdout:1/704: mkdir d2/d19/d59/d89/d69/d80/d8e/df5 0 2024-02-16T19:29:17.327 INFO:tasks.workunit.client.1.smithi117.stdout:0/804: rmdir d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/d27/d31/d6d 39 2024-02-16T19:29:17.328 INFO:tasks.workunit.client.1.smithi117.stdout:7/792: creat d4/d1c/d112/f125 x:0 0 0 2024-02-16T19:29:17.329 INFO:tasks.workunit.client.0.smithi022.stdout:8/779: creat d3/d12/d92/f10e x:0 0 0 2024-02-16T19:29:17.329 INFO:tasks.workunit.client.0.smithi022.stdout:4/642: readlink d4/lbd 0 2024-02-16T19:29:17.329 INFO:tasks.workunit.client.1.smithi117.stdout:6/763: dwrite d5/d4b/d71/dc7/d49/f88 [4194304,4194304] 0 2024-02-16T19:29:17.330 INFO:tasks.workunit.client.0.smithi022.stdout:4/643: fsync d4/d6/fbe 0 2024-02-16T19:29:17.331 INFO:tasks.workunit.client.1.smithi117.stdout:1/705: truncate d2/d4/f17 9073023 0 2024-02-16T19:29:17.331 INFO:tasks.workunit.client.0.smithi022.stdout:5/764: write dd/d1f/d48/d21/d5d/de7/f83 [1057033,76768] 0 2024-02-16T19:29:17.332 INFO:tasks.workunit.client.1.smithi117.stdout:1/706: chown d2/d4 198518471 1 2024-02-16T19:29:17.332 INFO:tasks.workunit.client.0.smithi022.stdout:0/891: creat d10/d11/d50/d102/f127 x:0 0 0 2024-02-16T19:29:17.333 INFO:tasks.workunit.client.0.smithi022.stdout:5/765: chown dd/d1f/d48/d21/d2f/d40/d71/fe4 17 1 2024-02-16T19:29:17.334 INFO:tasks.workunit.client.1.smithi117.stdout:9/696: readlink dd/d47/d5b/d5f/d77/ld8 0 2024-02-16T19:29:17.335 INFO:tasks.workunit.client.0.smithi022.stdout:7/727: link d9/da/d2c/d2f/f35 d9/da9/dbc/fef 0 2024-02-16T19:29:17.336 INFO:tasks.workunit.client.0.smithi022.stdout:7/728: stat d9/da/d74/f80 0 2024-02-16T19:29:17.338 INFO:tasks.workunit.client.0.smithi022.stdout:7/729: fsync d9/da/d1f/fc5 0 2024-02-16T19:29:17.338 INFO:tasks.workunit.client.0.smithi022.stdout:8/780: rmdir d3/d12/d2b/da9 39 2024-02-16T19:29:17.338 INFO:tasks.workunit.client.1.smithi117.stdout:4/765: dwrite d0/da/d21/d3d/d28/d64/d73/d91/f62 [0,4194304] 0 2024-02-16T19:29:17.339 INFO:tasks.workunit.client.1.smithi117.stdout:8/819: dwrite d4/f3c [0,4194304] 0 2024-02-16T19:29:17.339 INFO:tasks.workunit.client.1.smithi117.stdout:0/805: dread d0/d30/fd2 [0,4194304] 0 2024-02-16T19:29:17.340 INFO:tasks.workunit.client.0.smithi022.stdout:4/644: stat d4/d6/dd/f37 0 2024-02-16T19:29:17.340 INFO:tasks.workunit.client.1.smithi117.stdout:3/871: link d1/d1b/d1e/d37/f9c d1/dd/d3b/dba/f12c 0 2024-02-16T19:29:17.342 INFO:tasks.workunit.client.1.smithi117.stdout:7/793: symlink d4/d1c/d39/d46/d55/l126 0 2024-02-16T19:29:17.344 INFO:tasks.workunit.client.0.smithi022.stdout:2/696: getdents d0/d2/d18/d19 0 2024-02-16T19:29:17.346 INFO:tasks.workunit.client.1.smithi117.stdout:3/872: fdatasync d1/dd/d5c/d85/f11b 0 2024-02-16T19:29:17.346 INFO:tasks.workunit.client.1.smithi117.stdout:1/707: mknod d2/d1d/d60/d7c/d9e/da5/cf6 0 2024-02-16T19:29:17.349 INFO:tasks.workunit.client.0.smithi022.stdout:7/730: creat d9/da/d1f/d67/db3/ff0 x:0 0 0 2024-02-16T19:29:17.350 INFO:tasks.workunit.client.0.smithi022.stdout:5/766: dread dd/d1f/d48/d21/d2f/d46/d5e/d84/d8b/f9a [0,4194304] 0 2024-02-16T19:29:17.351 INFO:tasks.workunit.client.0.smithi022.stdout:5/767: stat dd/d1f/d48/d21/d2f/d40/c55 0 2024-02-16T19:29:17.352 INFO:tasks.workunit.client.1.smithi117.stdout:3/873: mkdir d1/dd/d3b/d9d/d12d 0 2024-02-16T19:29:17.353 INFO:tasks.workunit.client.1.smithi117.stdout:4/766: dwrite d0/da/d21/f65 [4194304,4194304] 0 2024-02-16T19:29:17.355 INFO:tasks.workunit.client.1.smithi117.stdout:8/820: fdatasync d4/d12/d2a/d64/dec/d76/f85 0 2024-02-16T19:29:17.358 INFO:tasks.workunit.client.1.smithi117.stdout:6/764: dread d5/f9 [0,4194304] 0 2024-02-16T19:29:17.358 INFO:tasks.workunit.client.1.smithi117.stdout:9/697: symlink dd/d47/d5b/d5f/de6/le8 0 2024-02-16T19:29:17.358 INFO:tasks.workunit.client.1.smithi117.stdout:1/708: dwrite d2/f57 [0,4194304] 0 2024-02-16T19:29:17.360 INFO:tasks.workunit.client.0.smithi022.stdout:5/768: dwrite dd/d1f/d48/d21/d5d/de7/fc6 [0,4194304] 0 2024-02-16T19:29:17.361 INFO:tasks.workunit.client.0.smithi022.stdout:4/645: rename d4/d6/dd/d22/da8/dc4/d86/l94 to d4/d6/dd/d22/da8/dc4/lcd 0 2024-02-16T19:29:17.361 INFO:tasks.workunit.client.0.smithi022.stdout:0/892: write d10/d11/d50/fc3 [598110,112848] 0 2024-02-16T19:29:17.361 INFO:tasks.workunit.client.0.smithi022.stdout:5/769: stat dd/d1f/c41 0 2024-02-16T19:29:17.363 INFO:tasks.workunit.client.0.smithi022.stdout:4/646: dread - d4/d6/dd/df/d10/d24/fbc zero size 2024-02-16T19:29:17.363 INFO:tasks.workunit.client.1.smithi117.stdout:0/806: symlink d0/d9/d14/d1d/d60/dc3/d109/l113 0 2024-02-16T19:29:17.364 INFO:tasks.workunit.client.0.smithi022.stdout:5/770: readlink dd/d1f/d48/d21/d2f/d52/d5c/lf6 0 2024-02-16T19:29:17.365 INFO:tasks.workunit.client.0.smithi022.stdout:2/697: mkdir d0/d2/d2e/d73/de3 0 2024-02-16T19:29:17.366 INFO:tasks.workunit.client.0.smithi022.stdout:5/771: chown dd/d1f/d48/d21/d2f/d40/d49/c54 2384 1 2024-02-16T19:29:17.367 INFO:tasks.workunit.client.0.smithi022.stdout:2/698: stat d0/d2/d18/d1f/d6c 0 2024-02-16T19:29:17.368 INFO:tasks.workunit.client.0.smithi022.stdout:5/772: readlink dd/d1f/d48/d21/d2f/d40/d71/lc4 0 2024-02-16T19:29:17.368 INFO:tasks.workunit.client.1.smithi117.stdout:9/698: dread dd/d45/d31/f93 [0,4194304] 0 2024-02-16T19:29:17.369 INFO:tasks.workunit.client.0.smithi022.stdout:5/773: chown dd/c44 29872 1 2024-02-16T19:29:17.369 INFO:tasks.workunit.client.1.smithi117.stdout:7/794: rename d4/d1c/d87/f50 to d4/d1c/d39/d46/dc4/f127 0 2024-02-16T19:29:17.371 INFO:tasks.workunit.client.0.smithi022.stdout:7/731: symlink d9/da/d3b/lf1 0 2024-02-16T19:29:17.371 INFO:tasks.workunit.client.0.smithi022.stdout:7/732: readlink d9/da/d2c/d46/l56 0 2024-02-16T19:29:17.373 INFO:tasks.workunit.client.1.smithi117.stdout:3/874: truncate d1/d1b/d1e/d37/f79 5029308 0 2024-02-16T19:29:17.374 INFO:tasks.workunit.client.1.smithi117.stdout:3/875: chown d1/dd/d3b/da3 2746 1 2024-02-16T19:29:17.375 INFO:tasks.workunit.client.1.smithi117.stdout:8/821: mkdir d4/d7/da6/d50/d10d 0 2024-02-16T19:29:17.375 INFO:tasks.workunit.client.1.smithi117.stdout:8/822: fsync d4/d12/d2a/d64/dec/d76/ff4 0 2024-02-16T19:29:17.377 INFO:tasks.workunit.client.0.smithi022.stdout:0/893: creat d10/d15/f128 x:0 0 0 2024-02-16T19:29:17.378 INFO:tasks.workunit.client.1.smithi117.stdout:6/765: unlink d5/d4b/d71/dc7/d49/l5c 0 2024-02-16T19:29:17.379 INFO:tasks.workunit.client.1.smithi117.stdout:6/766: write d5/d9c/f9e [2205145,48666] 0 2024-02-16T19:29:17.380 INFO:tasks.workunit.client.0.smithi022.stdout:4/647: read - d4/d6/d3a/fc8 zero size 2024-02-16T19:29:17.385 INFO:tasks.workunit.client.1.smithi117.stdout:9/699: symlink dd/db5/dc4/le9 0 2024-02-16T19:29:17.386 INFO:tasks.workunit.client.0.smithi022.stdout:7/733: write d9/da/d2c/dcb/fdd [863208,9536] 0 2024-02-16T19:29:17.388 INFO:tasks.workunit.client.1.smithi117.stdout:4/767: rename d0/da/d21/d3d/d28/d64/d90/dc0/dd7/ff9 to d0/da/d21/db4/de4/ffb 0 2024-02-16T19:29:17.388 INFO:tasks.workunit.client.0.smithi022.stdout:8/781: link d3/d12/d6a/c77 d3/d12/d92/c10f 0 2024-02-16T19:29:17.391 INFO:tasks.workunit.client.1.smithi117.stdout:4/768: dread d0/da/d21/d3d/d28/d64/d8a/fb1 [0,4194304] 0 2024-02-16T19:29:17.393 INFO:tasks.workunit.client.1.smithi117.stdout:3/876: symlink d1/d1b/d1e/da6/l12e 0 2024-02-16T19:29:17.394 INFO:tasks.workunit.client.1.smithi117.stdout:3/877: fsync d1/d1b/d1e/d37/ddc/f107 0 2024-02-16T19:29:17.394 INFO:tasks.workunit.client.0.smithi022.stdout:4/648: mkdir d4/d6/d14/d64/d4e/d54/d5a/dce 0 2024-02-16T19:29:17.398 INFO:tasks.workunit.client.1.smithi117.stdout:8/823: mknod d4/d12/d31/d8e/c10e 0 2024-02-16T19:29:17.398 INFO:tasks.workunit.client.0.smithi022.stdout:4/649: fsync d4/f40 0 2024-02-16T19:29:17.399 INFO:tasks.workunit.client.1.smithi117.stdout:8/824: read d4/d7/f88 [3010170,20961] 0 2024-02-16T19:29:17.400 INFO:tasks.workunit.client.1.smithi117.stdout:1/709: creat d2/d19/d59/d76/d8c/dc7/dcb/ff7 x:0 0 0 2024-02-16T19:29:17.400 INFO:tasks.workunit.client.1.smithi117.stdout:8/825: fsync d4/d12/d31/f77 0 2024-02-16T19:29:17.403 INFO:tasks.workunit.client.0.smithi022.stdout:7/734: creat d9/da/d74/ff2 x:0 0 0 2024-02-16T19:29:17.405 INFO:tasks.workunit.client.1.smithi117.stdout:1/710: dwrite d2/d1d/d60/d68/f81 [0,4194304] 0 2024-02-16T19:29:17.407 INFO:tasks.workunit.client.0.smithi022.stdout:7/735: dread d9/da/d4e/fc4 [0,4194304] 0 2024-02-16T19:29:17.408 INFO:tasks.workunit.client.1.smithi117.stdout:6/767: rename d5/d4b/d44/d52/d81/dd7/df9 to d5/d4b/d44/d52/d81/dfc 0 2024-02-16T19:29:17.409 INFO:tasks.workunit.client.1.smithi117.stdout:0/807: write d0/d30/f64 [3351795,38368] 0 2024-02-16T19:29:17.410 INFO:tasks.workunit.client.1.smithi117.stdout:7/795: write d4/d1c/d39/ff4 [4967531,125160] 0 2024-02-16T19:29:17.411 INFO:tasks.workunit.client.1.smithi117.stdout:8/826: dread d4/f6 [0,4194304] 0 2024-02-16T19:29:17.411 INFO:tasks.workunit.client.1.smithi117.stdout:8/827: readlink d4/d12/l58 0 2024-02-16T19:29:17.412 INFO:tasks.workunit.client.1.smithi117.stdout:4/769: mkdir d0/da/d21/d3d/d28/d66/da2/dac/dcb/dfc 0 2024-02-16T19:29:17.413 INFO:tasks.workunit.client.1.smithi117.stdout:4/770: chown d0/da/d21/d3d/d28/d64/d73/d91/d37 21499715 1 2024-02-16T19:29:17.413 INFO:tasks.workunit.client.1.smithi117.stdout:4/771: dread - d0/f72 zero size 2024-02-16T19:29:17.415 INFO:tasks.workunit.client.1.smithi117.stdout:3/878: mkdir d1/d30/d9f/dbc/d12f 0 2024-02-16T19:29:17.415 INFO:tasks.workunit.client.0.smithi022.stdout:2/699: dwrite d0/d2/d2e/d73/f7d [0,4194304] 0 2024-02-16T19:29:17.420 INFO:tasks.workunit.client.1.smithi117.stdout:3/879: dwrite d1/dd/d32/f118 [0,4194304] 0 2024-02-16T19:29:17.421 INFO:tasks.workunit.client.1.smithi117.stdout:3/880: dread - d1/d31/f11a zero size 2024-02-16T19:29:17.427 INFO:tasks.workunit.client.1.smithi117.stdout:1/711: symlink d2/d19/d59/d89/d69/d80/lf8 0 2024-02-16T19:29:17.427 INFO:tasks.workunit.client.1.smithi117.stdout:5/827: sync 2024-02-16T19:29:17.429 INFO:tasks.workunit.client.1.smithi117.stdout:9/700: rename dd/d47/d5b/f69 to dd/d23/d64/fea 0 2024-02-16T19:29:17.430 INFO:tasks.workunit.client.1.smithi117.stdout:9/701: readlink dd/d23/daf/lc3 0 2024-02-16T19:29:17.433 INFO:tasks.workunit.client.1.smithi117.stdout:1/712: dwrite d2/d1d/d60/d7c/d9e/fc3 [0,4194304] 0 2024-02-16T19:29:17.434 INFO:tasks.workunit.client.1.smithi117.stdout:7/796: write d4/d1c/d39/fec [10625,3801] 0 2024-02-16T19:29:17.435 INFO:tasks.workunit.client.1.smithi117.stdout:0/808: unlink d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/d27/d4e/l7d 0 2024-02-16T19:29:17.443 INFO:tasks.workunit.client.1.smithi117.stdout:7/797: dwrite d4/f34 [4194304,4194304] 0 2024-02-16T19:29:17.443 INFO:tasks.workunit.client.1.smithi117.stdout:3/881: chown d1/dd/f48 21448848 1 2024-02-16T19:29:17.444 INFO:tasks.workunit.client.1.smithi117.stdout:1/713: dwrite d2/f4d [0,4194304] 0 2024-02-16T19:29:17.454 INFO:tasks.workunit.client.1.smithi117.stdout:6/768: mkdir d5/d4b/d44/dfd 0 2024-02-16T19:29:17.457 INFO:tasks.workunit.client.1.smithi117.stdout:9/702: creat dd/db5/dc4/feb x:0 0 0 2024-02-16T19:29:17.463 INFO:tasks.workunit.client.1.smithi117.stdout:8/828: link d4/d7/da6/d50/ff5 d4/d7/da6/d3a/d65/d7c/f10f 0 2024-02-16T19:29:17.464 INFO:tasks.workunit.client.1.smithi117.stdout:8/829: fsync d4/d7/d6a/d72/d83/ff2 0 2024-02-16T19:29:17.465 INFO:tasks.workunit.client.1.smithi117.stdout:8/830: chown d4/d7/da6/l5e 132777659 1 2024-02-16T19:29:17.466 INFO:tasks.workunit.client.1.smithi117.stdout:0/809: unlink d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/fb8 0 2024-02-16T19:29:17.466 INFO:tasks.workunit.client.1.smithi117.stdout:8/831: dread - d4/d7/da6/ffc zero size 2024-02-16T19:29:17.467 INFO:tasks.workunit.client.1.smithi117.stdout:8/832: fsync d4/d12/d2a/d64/dc6/fae 0 2024-02-16T19:29:17.467 INFO:tasks.workunit.client.1.smithi117.stdout:0/810: chown d0/d9/d14/d1d/d60/d76/d7b/d8d 29716 1 2024-02-16T19:29:17.470 INFO:tasks.workunit.client.0.smithi022.stdout:0/894: dwrite d10/d15/d23/d38/d5a/d66/fd6 [0,4194304] 0 2024-02-16T19:29:17.470 INFO:tasks.workunit.client.1.smithi117.stdout:3/882: dread d1/d83/d96/fa0 [0,4194304] 0 2024-02-16T19:29:17.471 INFO:tasks.workunit.client.1.smithi117.stdout:1/714: mknod d2/d1d/cf9 0 2024-02-16T19:29:17.474 INFO:tasks.workunit.client.1.smithi117.stdout:1/715: dread d2/f57 [0,4194304] 0 2024-02-16T19:29:17.475 INFO:tasks.workunit.client.0.smithi022.stdout:8/782: dwrite d3/d12/d17/fbe [4194304,4194304] 0 2024-02-16T19:29:17.478 INFO:tasks.workunit.client.1.smithi117.stdout:4/772: rename d0/da/d21/d3d/d28/d64/d73/d91/d37/lca to d0/da/d21/d3d/d28/d66/da2/dac/lfd 0 2024-02-16T19:29:17.480 INFO:tasks.workunit.client.1.smithi117.stdout:4/773: chown d0/da/d21/f40 163 1 2024-02-16T19:29:17.480 INFO:tasks.workunit.client.1.smithi117.stdout:6/769: mknod d5/d4b/d44/d93/dae/cfe 0 2024-02-16T19:29:17.481 INFO:tasks.workunit.client.0.smithi022.stdout:5/774: creat dd/d1f/d48/d21/d2f/d52/d5c/d61/f101 x:0 0 0 2024-02-16T19:29:17.482 INFO:tasks.workunit.client.1.smithi117.stdout:1/716: dwrite d2/d19/d59/d89/d69/fe0 [0,4194304] 0 2024-02-16T19:29:17.482 INFO:tasks.workunit.client.1.smithi117.stdout:9/703: fsync dd/d45/d31/f93 0 2024-02-16T19:29:17.483 INFO:tasks.workunit.client.0.smithi022.stdout:7/736: rmdir d9/da/d1f 39 2024-02-16T19:29:17.486 INFO:tasks.workunit.client.1.smithi117.stdout:9/704: write dd/d47/d5b/d5f/fde [890897,112502] 0 2024-02-16T19:29:17.486 INFO:tasks.workunit.client.0.smithi022.stdout:5/775: dread dd/d1f/d48/d21/d2f/d52/d77/d95/fb8 [0,4194304] 0 2024-02-16T19:29:17.491 INFO:tasks.workunit.client.1.smithi117.stdout:5/828: dwrite d2/d9/d3c/d24/d50/d52/d74/d8c/f4b [4194304,4194304] 0 2024-02-16T19:29:17.491 INFO:tasks.workunit.client.0.smithi022.stdout:2/700: creat d0/d2/d18/d38/db6/fe4 x:0 0 0 2024-02-16T19:29:17.492 INFO:tasks.workunit.client.1.smithi117.stdout:1/717: dwrite d2/f4d [0,4194304] 0 2024-02-16T19:29:17.492 INFO:tasks.workunit.client.0.smithi022.stdout:2/701: stat d0/d2/d18/d38/db6/cc5 0 2024-02-16T19:29:17.493 INFO:tasks.workunit.client.0.smithi022.stdout:2/702: chown d0/d2/d18/f22 1534489265 1 2024-02-16T19:29:17.493 INFO:tasks.workunit.client.1.smithi117.stdout:5/829: fdatasync d2/d3/d21/d49/d59/daa/de2/ff6 0 2024-02-16T19:29:17.494 INFO:tasks.workunit.client.1.smithi117.stdout:1/718: write d2/d19/d59/d89/f2a [3351849,67645] 0 2024-02-16T19:29:17.494 INFO:tasks.workunit.client.0.smithi022.stdout:0/895: creat d10/d11/d27/d3f/d58/d60/f129 x:0 0 0 2024-02-16T19:29:17.496 INFO:tasks.workunit.client.1.smithi117.stdout:0/811: fsync d0/d9/d14/d23/f56 0 2024-02-16T19:29:17.496 INFO:tasks.workunit.client.1.smithi117.stdout:5/830: dread - d2/d9/d3c/d24/d50/d96/dbc/fb1 zero size 2024-02-16T19:29:17.496 INFO:tasks.workunit.client.1.smithi117.stdout:1/719: dread d2/d19/d59/d89/f42 [0,4194304] 0 2024-02-16T19:29:17.504 INFO:tasks.workunit.client.0.smithi022.stdout:0/896: dwrite d10/d11/d50/fc3 [0,4194304] 0 2024-02-16T19:29:17.514 INFO:tasks.workunit.client.0.smithi022.stdout:0/897: dwrite d10/d15/d23/d8f/dc1/f69 [0,4194304] 0 2024-02-16T19:29:17.528 INFO:tasks.workunit.client.1.smithi117.stdout:9/705: rename dd/d23/d64/f70 to dd/d47/d5e/d9a/fec 0 2024-02-16T19:29:17.528 INFO:tasks.workunit.client.0.smithi022.stdout:4/650: link d4/d6/dd/f37 d4/d6/dd/d22/d30/fcf 0 2024-02-16T19:29:17.529 INFO:tasks.workunit.client.0.smithi022.stdout:4/651: chown d4/d6/d14/d64/d4e/d54/dba/c74 1490138874 1 2024-02-16T19:29:17.533 INFO:tasks.workunit.client.0.smithi022.stdout:8/783: write d3/d12/d17/fbb [668137,113821] 0 2024-02-16T19:29:17.535 INFO:tasks.workunit.client.0.smithi022.stdout:0/898: rename d10/d15/d23/f120 to d10/d15/d23/d4e/f12a 0 2024-02-16T19:29:17.536 INFO:tasks.workunit.client.0.smithi022.stdout:4/652: dread d4/d6/dd/df/d10/f4f [0,4194304] 0 2024-02-16T19:29:17.537 INFO:tasks.workunit.client.0.smithi022.stdout:8/784: truncate d3/d12/d2b/d54/d74/da2/f105 394785 0 2024-02-16T19:29:17.537 INFO:tasks.workunit.client.0.smithi022.stdout:7/737: mknod d9/da/d4e/cf3 0 2024-02-16T19:29:17.538 INFO:tasks.workunit.client.0.smithi022.stdout:0/899: truncate d10/d22/dad/f108 526710 0 2024-02-16T19:29:17.542 INFO:tasks.workunit.client.0.smithi022.stdout:5/776: symlink dd/d1f/d48/d21/d2f/d46/d5e/d84/l102 0 2024-02-16T19:29:17.544 INFO:tasks.workunit.client.0.smithi022.stdout:7/738: dwrite d9/da/d2c/d2f/f9e [0,4194304] 0 2024-02-16T19:29:17.545 INFO:tasks.workunit.client.1.smithi117.stdout:7/798: symlink d4/d1c/dd8/l128 0 2024-02-16T19:29:17.549 INFO:tasks.workunit.client.1.smithi117.stdout:6/770: dwrite d5/db/f1f [4194304,4194304] 0 2024-02-16T19:29:17.549 INFO:tasks.workunit.client.0.smithi022.stdout:2/703: dwrite d0/d2/f3d [0,4194304] 0 2024-02-16T19:29:17.549 INFO:tasks.workunit.client.1.smithi117.stdout:3/883: mkdir d1/dd/d3e/d7d/d121/d130 0 2024-02-16T19:29:17.550 INFO:tasks.workunit.client.0.smithi022.stdout:2/704: stat d0/d32/d3b/d6f 0 2024-02-16T19:29:17.550 INFO:tasks.workunit.client.1.smithi117.stdout:1/720: dwrite d2/d1d/f3a [4194304,4194304] 0 2024-02-16T19:29:17.552 INFO:tasks.workunit.client.0.smithi022.stdout:4/653: chown d4/d6/dd/d22/d83/dc5/c6c 414 1 2024-02-16T19:29:17.555 INFO:tasks.workunit.client.1.smithi117.stdout:3/884: write d1/d1b/d1e/d37/f65 [1075832,110663] 0 2024-02-16T19:29:17.556 INFO:tasks.workunit.client.0.smithi022.stdout:0/900: mkdir d10/d11/d12b 0 2024-02-16T19:29:17.558 INFO:tasks.workunit.client.1.smithi117.stdout:9/706: symlink dd/d45/d31/d63/d8e/led 0 2024-02-16T19:29:17.558 INFO:tasks.workunit.client.1.smithi117.stdout:8/833: creat d4/d12/d2a/f110 x:0 0 0 2024-02-16T19:29:17.559 INFO:tasks.workunit.client.1.smithi117.stdout:7/799: dwrite d4/d13/d63/fa2 [0,4194304] 0 2024-02-16T19:29:17.559 INFO:tasks.workunit.client.0.smithi022.stdout:7/739: chown d9/da/d1f/d67/ddb/dd4 1 1 2024-02-16T19:29:17.561 INFO:tasks.workunit.client.0.smithi022.stdout:7/740: chown d9/da/d1f/d93/cb1 24891 1 2024-02-16T19:29:17.561 INFO:tasks.workunit.client.0.smithi022.stdout:2/705: mkdir d0/d32/d3b/de5 0 2024-02-16T19:29:17.562 INFO:tasks.workunit.client.1.smithi117.stdout:8/834: chown d4/d7/da6/d3a/d65/cb4 32968795 1 2024-02-16T19:29:17.562 INFO:tasks.workunit.client.0.smithi022.stdout:7/741: fsync d9/da/d1f/d40/fb5 0 2024-02-16T19:29:17.563 INFO:tasks.workunit.client.1.smithi117.stdout:8/835: read - d4/d12/fd4 zero size 2024-02-16T19:29:17.564 INFO:tasks.workunit.client.1.smithi117.stdout:7/800: dread d4/d1c/d39/fec [0,4194304] 0 2024-02-16T19:29:17.565 INFO:tasks.workunit.client.0.smithi022.stdout:8/785: dread - d3/d12/d2b/da9/fbd zero size 2024-02-16T19:29:17.566 INFO:tasks.workunit.client.0.smithi022.stdout:8/786: readlink d3/d12/d17/d64/d8f/laa 0 2024-02-16T19:29:17.566 INFO:tasks.workunit.client.1.smithi117.stdout:6/771: mknod d5/d4b/d20/db3/cff 0 2024-02-16T19:29:17.567 INFO:tasks.workunit.client.0.smithi022.stdout:0/901: fsync d10/d11/d27/d3f/df0/ff1 0 2024-02-16T19:29:17.569 INFO:tasks.workunit.client.0.smithi022.stdout:0/902: chown d10/d15/d23/d8f/lb7 6040 1 2024-02-16T19:29:17.572 INFO:tasks.workunit.client.1.smithi117.stdout:8/836: dwrite d4/f79 [0,4194304] 0 2024-02-16T19:29:17.574 INFO:tasks.workunit.client.0.smithi022.stdout:2/706: unlink d0/d32/d3b/l95 0 2024-02-16T19:29:17.576 INFO:tasks.workunit.client.1.smithi117.stdout:9/707: dread - dd/d23/f9c zero size 2024-02-16T19:29:17.577 INFO:tasks.workunit.client.0.smithi022.stdout:7/742: unlink d9/da/d2c/d65/c9c 0 2024-02-16T19:29:17.578 INFO:tasks.workunit.client.1.smithi117.stdout:9/708: truncate dd/d47/d48/dc0/fa6 624391 0 2024-02-16T19:29:17.580 INFO:tasks.workunit.client.0.smithi022.stdout:4/654: link d4/d6/dd/f85 d4/d6/dd/d22/d30/fd0 0 2024-02-16T19:29:17.582 INFO:tasks.workunit.client.1.smithi117.stdout:0/812: creat d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/dd4/f114 x:0 0 0 2024-02-16T19:29:17.583 INFO:tasks.workunit.client.1.smithi117.stdout:2/810: sync 2024-02-16T19:29:17.583 INFO:tasks.workunit.client.0.smithi022.stdout:8/787: rename d3/d12/d2b/d54/db8/de5/d104 to d3/d12/d17/d10d/d110 0 2024-02-16T19:29:17.584 INFO:tasks.workunit.client.0.smithi022.stdout:5/777: link dd/d1f/d48/d21/d2f/d52/d5c/l67 dd/d1f/d2e/dd4/ddf/l103 0 2024-02-16T19:29:17.585 INFO:tasks.workunit.client.0.smithi022.stdout:8/788: write d3/d12/d2b/d46/f53 [4834181,118171] 0 2024-02-16T19:29:17.586 INFO:tasks.workunit.client.0.smithi022.stdout:5/778: chown dd/d1f/d48/d21/d2f/d52/d77/da9 1 1 2024-02-16T19:29:17.586 INFO:tasks.workunit.client.0.smithi022.stdout:8/789: write d3/feb [914359,83606] 0 2024-02-16T19:29:17.588 INFO:tasks.workunit.client.0.smithi022.stdout:5/779: chown dd/d1f/d48/d21/d2f/d40/d4e/f6d 24009 1 2024-02-16T19:29:17.589 INFO:tasks.workunit.client.1.smithi117.stdout:6/772: mkdir d5/d4b/d71/dc7/dce/ded/d100 0 2024-02-16T19:29:17.589 INFO:tasks.workunit.client.1.smithi117.stdout:0/813: dwrite d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/dd4/fdc [0,4194304] 0 2024-02-16T19:29:17.590 INFO:tasks.workunit.client.0.smithi022.stdout:5/780: chown dd/d1f/d48/d21/d5d/de7/l5f 31 1 2024-02-16T19:29:17.590 INFO:tasks.workunit.client.1.smithi117.stdout:8/837: dread d4/f52 [4194304,4194304] 0 2024-02-16T19:29:17.591 INFO:tasks.workunit.client.0.smithi022.stdout:7/743: mkdir d9/da/d1f/d44/d5b/df4 0 2024-02-16T19:29:17.591 INFO:tasks.workunit.client.0.smithi022.stdout:7/744: read - d9/d5d/fc8 zero size 2024-02-16T19:29:17.596 INFO:tasks.workunit.client.1.smithi117.stdout:9/709: write dd/db5/fa0 [1621117,107085] 0 2024-02-16T19:29:17.596 INFO:tasks.workunit.client.0.smithi022.stdout:8/790: mkdir d3/d12/d2b/d54/d97/d111 0 2024-02-16T19:29:17.597 INFO:tasks.workunit.client.1.smithi117.stdout:5/831: getdents d2/d4c/d69 0 2024-02-16T19:29:17.597 INFO:tasks.workunit.client.1.smithi117.stdout:5/832: write d2/d3/f47 [76409,122454] 0 2024-02-16T19:29:17.601 INFO:tasks.workunit.client.0.smithi022.stdout:4/655: mkdir d4/d6/d14/dc7/d8d/dd1 0 2024-02-16T19:29:17.602 INFO:tasks.workunit.client.1.smithi117.stdout:4/774: sync 2024-02-16T19:29:17.603 INFO:tasks.workunit.client.1.smithi117.stdout:4/775: fsync d0/da/d21/d3d/d28/d64/d90/dc0/dd7/fdc 0 2024-02-16T19:29:17.603 INFO:tasks.workunit.client.0.smithi022.stdout:7/745: rename d9/d1b/fb7 to d9/da/d3b/dba/ff5 0 2024-02-16T19:29:17.604 INFO:tasks.workunit.client.0.smithi022.stdout:7/746: fsync d9/da/d1f/d40/fb5 0 2024-02-16T19:29:17.604 INFO:tasks.workunit.client.1.smithi117.stdout:8/838: symlink d4/d12/d31/d8e/l111 0 2024-02-16T19:29:17.608 INFO:tasks.workunit.client.0.smithi022.stdout:8/791: dread d3/f2d [0,4194304] 0 2024-02-16T19:29:17.611 INFO:tasks.workunit.client.0.smithi022.stdout:7/747: mknod d9/da/d3b/dba/cf6 0 2024-02-16T19:29:17.613 INFO:tasks.workunit.client.1.smithi117.stdout:9/710: rmdir dd/d47/d48 39 2024-02-16T19:29:17.614 INFO:tasks.workunit.client.0.smithi022.stdout:4/656: dwrite d4/d6/dd/f60 [4194304,4194304] 0 2024-02-16T19:29:17.617 INFO:tasks.workunit.client.1.smithi117.stdout:4/776: dread d0/da/d21/d3d/d28/d64/d48/f4c [0,4194304] 0 2024-02-16T19:29:17.619 INFO:tasks.workunit.client.1.smithi117.stdout:4/777: write d0/da/d21/d3d/d28/d64/d8a/fea [24737,76323] 0 2024-02-16T19:29:17.620 INFO:tasks.workunit.client.1.smithi117.stdout:8/839: dread d4/d12/d2a/fbc [0,4194304] 0 2024-02-16T19:29:17.623 INFO:tasks.workunit.client.0.smithi022.stdout:8/792: mknod d3/d12/d2b/d54/db8/de5/c112 0 2024-02-16T19:29:17.625 INFO:tasks.workunit.client.1.smithi117.stdout:6/773: mknod d5/d4b/d71/dc7/d72/dee/c101 0 2024-02-16T19:29:17.626 INFO:tasks.workunit.client.0.smithi022.stdout:7/748: mknod d9/da/d1f/d44/cf7 0 2024-02-16T19:29:17.627 INFO:tasks.workunit.client.0.smithi022.stdout:0/903: write d10/d11/d27/d3f/df0/ff1 [902141,16735] 0 2024-02-16T19:29:17.629 INFO:tasks.workunit.client.1.smithi117.stdout:0/814: fdatasync d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/d27/f4d 0 2024-02-16T19:29:17.631 INFO:tasks.workunit.client.0.smithi022.stdout:8/793: symlink d3/d12/d2b/d54/db8/de5/l113 0 2024-02-16T19:29:17.632 INFO:tasks.workunit.client.1.smithi117.stdout:1/721: write d2/d19/d59/d89/d69/f95 [669994,4714] 0 2024-02-16T19:29:17.632 INFO:tasks.workunit.client.0.smithi022.stdout:8/794: readlink d3/d12/l89 0 2024-02-16T19:29:17.633 INFO:tasks.workunit.client.0.smithi022.stdout:8/795: fsync d3/d12/d17/d64/d8f/fec 0 2024-02-16T19:29:17.634 INFO:tasks.workunit.client.1.smithi117.stdout:5/833: mkdir d2/d9/d3c/d24/d50/d52/d117 0 2024-02-16T19:29:17.637 INFO:tasks.workunit.client.0.smithi022.stdout:0/904: creat d10/d15/d23/d38/d5a/d66/f12c x:0 0 0 2024-02-16T19:29:17.638 INFO:tasks.workunit.client.0.smithi022.stdout:0/905: stat d10/d11/c98 0 2024-02-16T19:29:17.639 INFO:tasks.workunit.client.1.smithi117.stdout:8/840: unlink d4/d1c/leb 0 2024-02-16T19:29:17.641 INFO:tasks.workunit.client.1.smithi117.stdout:6/774: creat d5/d4b/d71/dc7/d72/dec/f102 x:0 0 0 2024-02-16T19:29:17.644 INFO:tasks.workunit.client.1.smithi117.stdout:7/801: write d4/d1c/d39/d46/d47/f82 [3350463,32044] 0 2024-02-16T19:29:17.646 INFO:tasks.workunit.client.0.smithi022.stdout:8/796: mkdir d3/d12/d17/d64/d8f/d114 0 2024-02-16T19:29:17.649 INFO:tasks.workunit.client.0.smithi022.stdout:0/906: symlink d10/d22/d49/l12d 0 2024-02-16T19:29:17.650 INFO:tasks.workunit.client.1.smithi117.stdout:0/815: fsync d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/ffc 0 2024-02-16T19:29:17.651 INFO:tasks.workunit.client.1.smithi117.stdout:0/816: chown d0/d9/d2c/d8b/l87 44013 1 2024-02-16T19:29:17.653 INFO:tasks.workunit.client.0.smithi022.stdout:7/749: link d9/da/d1f/d44/f49 d9/da/d2c/ff8 0 2024-02-16T19:29:17.653 INFO:tasks.workunit.client.1.smithi117.stdout:3/885: write d1/fdb [517666,109246] 0 2024-02-16T19:29:17.654 INFO:tasks.workunit.client.1.smithi117.stdout:0/817: write d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/d27/d83/f50 [687602,48882] 0 2024-02-16T19:29:17.654 INFO:tasks.workunit.client.1.smithi117.stdout:8/841: dread d4/d7/fca [0,4194304] 0 2024-02-16T19:29:17.656 INFO:tasks.workunit.client.0.smithi022.stdout:2/707: dwrite d0/d32/d3b/f6e [0,4194304] 0 2024-02-16T19:29:17.657 INFO:tasks.workunit.client.1.smithi117.stdout:8/842: truncate d4/d12/fd4 908711 0 2024-02-16T19:29:17.658 INFO:tasks.workunit.client.0.smithi022.stdout:0/907: mknod d10/d15/d23/d38/d5a/c12e 0 2024-02-16T19:29:17.658 INFO:tasks.workunit.client.1.smithi117.stdout:6/775: dread d5/d9c/daa/fb2 [4194304,4194304] 0 2024-02-16T19:29:17.658 INFO:tasks.workunit.client.1.smithi117.stdout:0/818: write d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/d27/d83/f95 [104711,97045] 0 2024-02-16T19:29:17.662 INFO:tasks.workunit.client.1.smithi117.stdout:5/834: rename d2/d3/l45 to d2/d4c/d69/l118 0 2024-02-16T19:29:17.663 INFO:tasks.workunit.client.0.smithi022.stdout:7/750: symlink d9/da/d1f/d93/lf9 0 2024-02-16T19:29:17.665 INFO:tasks.workunit.client.0.smithi022.stdout:5/781: write dd/d1f/d48/d21/d5d/fb3 [1522923,25315] 0 2024-02-16T19:29:17.666 INFO:tasks.workunit.client.1.smithi117.stdout:4/778: unlink d0/da/d21/d3d/d28/d64/l19 0 2024-02-16T19:29:17.667 INFO:tasks.workunit.client.1.smithi117.stdout:1/722: dread d2/d4/dbb/d7f/f94 [0,4194304] 0 2024-02-16T19:29:17.668 INFO:tasks.workunit.client.0.smithi022.stdout:8/797: dread d3/d12/d2b/f3f [0,4194304] 0 2024-02-16T19:29:17.670 INFO:tasks.workunit.client.1.smithi117.stdout:1/723: stat d2/d1d/d60/fb2 0 2024-02-16T19:29:17.670 INFO:tasks.workunit.client.1.smithi117.stdout:2/811: dwrite d0/d3/f2a [0,4194304] 0 2024-02-16T19:29:17.671 INFO:tasks.workunit.client.1.smithi117.stdout:4/779: stat d0/da/d21/d3d/d28/d64/d8a/fea 0 2024-02-16T19:29:17.672 INFO:tasks.workunit.client.1.smithi117.stdout:7/802: dread d4/d1c/d39/d46/f48 [0,4194304] 0 2024-02-16T19:29:17.673 INFO:tasks.workunit.client.1.smithi117.stdout:3/886: creat d1/d31/d89/dbd/f131 x:0 0 0 2024-02-16T19:29:17.674 INFO:tasks.workunit.client.0.smithi022.stdout:2/708: mkdir d0/d2/d2e/d73/de3/de6 0 2024-02-16T19:29:17.674 INFO:tasks.workunit.client.1.smithi117.stdout:9/711: link dd/d47/d5e/f7f dd/d45/d73/fee 0 2024-02-16T19:29:17.676 INFO:tasks.workunit.client.0.smithi022.stdout:5/782: mknod dd/d1f/d2e/dd4/c104 0 2024-02-16T19:29:17.678 INFO:tasks.workunit.client.0.smithi022.stdout:5/783: fsync dd/d1f/d48/d21/d2f/d40/d71/fe4 0 2024-02-16T19:29:17.678 INFO:tasks.workunit.client.1.smithi117.stdout:7/803: dwrite d4/d1c/d87/fff [0,4194304] 0 2024-02-16T19:29:17.681 INFO:tasks.workunit.client.0.smithi022.stdout:5/784: stat dd/d1f/d48/d21/d2f/d40/d49/d4b/ca2 0 2024-02-16T19:29:17.681 INFO:tasks.workunit.client.0.smithi022.stdout:8/798: dread d3/d12/f9e [0,4194304] 0 2024-02-16T19:29:17.681 INFO:tasks.workunit.client.1.smithi117.stdout:6/776: symlink d5/d4b/d44/d52/d81/dd7/l103 0 2024-02-16T19:29:17.682 INFO:tasks.workunit.client.0.smithi022.stdout:5/785: chown dd/d1f/d48/d21/d2f/d52/d5c/d61/f92 282566167 1 2024-02-16T19:29:17.682 INFO:tasks.workunit.client.0.smithi022.stdout:5/786: fdatasync f6 0 2024-02-16T19:29:17.684 INFO:tasks.workunit.client.0.smithi022.stdout:4/657: write d4/d6/d3a/fa0 [1030888,122613] 0 2024-02-16T19:29:17.686 INFO:tasks.workunit.client.0.smithi022.stdout:5/787: rename dd/d1f/d48/d21/d2f/d52/d77 to dd/d1f/d48/d21/d2f/d52/d77/d95/ded/d105 22 2024-02-16T19:29:17.686 INFO:tasks.workunit.client.1.smithi117.stdout:0/819: dread d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/d27/d83/d101/f104 [0,4194304] 0 2024-02-16T19:29:17.687 INFO:tasks.workunit.client.0.smithi022.stdout:2/709: rmdir d0/d2/d18/d19/d40/d55/d59 39 2024-02-16T19:29:17.687 INFO:tasks.workunit.client.0.smithi022.stdout:4/658: write d4/d6/dd/d22/da8/dc4/fc3 [1022800,52570] 0 2024-02-16T19:29:17.690 INFO:tasks.workunit.client.1.smithi117.stdout:5/835: creat d2/db3/db0/d9b/f119 x:0 0 0 2024-02-16T19:29:17.691 INFO:tasks.workunit.client.0.smithi022.stdout:0/908: link d10/d11/d27/fbb d10/d15/d23/dc6/f12f 0 2024-02-16T19:29:17.692 INFO:tasks.workunit.client.0.smithi022.stdout:0/909: chown d10/d7d/dac/de9/dee 113390755 1 2024-02-16T19:29:17.694 INFO:tasks.workunit.client.0.smithi022.stdout:8/799: fdatasync d3/d12/da6/f2f 0 2024-02-16T19:29:17.704 INFO:tasks.workunit.client.0.smithi022.stdout:4/659: dread d4/d6/d14/dc7/d8d/f99 [0,4194304] 0 2024-02-16T19:29:17.704 INFO:tasks.workunit.client.0.smithi022.stdout:0/910: unlink d10/d15/d23/d8f/dc1/deb/f119 0 2024-02-16T19:29:17.705 INFO:tasks.workunit.client.0.smithi022.stdout:0/911: stat d10/d22/d2c/l8d 0 2024-02-16T19:29:17.706 INFO:tasks.workunit.client.0.smithi022.stdout:4/660: write d4/d6/dd/d22/da8/dc4/d6d/faa [50388,5384] 0 2024-02-16T19:29:17.707 INFO:tasks.workunit.client.1.smithi117.stdout:2/812: symlink d0/d3/d34/d94/d46/d68/d76/da1/d56/d60/l100 0 2024-02-16T19:29:17.708 INFO:tasks.workunit.client.0.smithi022.stdout:5/788: dread dd/d1f/d48/d21/d2f/d40/d49/fa1 [0,4194304] 0 2024-02-16T19:29:17.712 INFO:tasks.workunit.client.0.smithi022.stdout:8/800: mkdir d3/d12/d2b/d54/dcd/d115 0 2024-02-16T19:29:17.713 INFO:tasks.workunit.client.0.smithi022.stdout:0/912: dwrite d10/d15/d23/d38/d5a/d66/fd6 [0,4194304] 0 2024-02-16T19:29:17.713 INFO:tasks.workunit.client.1.smithi117.stdout:1/724: dread d2/d4/f15 [0,4194304] 0 2024-02-16T19:29:17.714 INFO:tasks.workunit.client.0.smithi022.stdout:8/801: fsync d3/d12/d2b/d54/d97/fae 0 2024-02-16T19:29:17.714 INFO:tasks.workunit.client.1.smithi117.stdout:9/712: dwrite dd/d45/d73/f99 [4194304,4194304] 0 2024-02-16T19:29:17.718 INFO:tasks.workunit.client.0.smithi022.stdout:2/710: rename d0/d2/d18/d19/d40/c6a to d0/d5d/dd0/ce7 0 2024-02-16T19:29:17.720 INFO:tasks.workunit.client.1.smithi117.stdout:6/777: creat d5/d4b/d44/d7c/f104 x:0 0 0 2024-02-16T19:29:17.725 INFO:tasks.workunit.client.0.smithi022.stdout:7/751: truncate d9/da/d3b/fd7 1283465 0 2024-02-16T19:29:17.726 INFO:tasks.workunit.client.0.smithi022.stdout:7/752: fdatasync d9/da/d1f/d40/fa5 0 2024-02-16T19:29:17.728 INFO:tasks.workunit.client.1.smithi117.stdout:5/836: mknod d2/d5d/d8e/dc1/c11a 0 2024-02-16T19:29:17.729 INFO:tasks.workunit.client.0.smithi022.stdout:4/661: mkdir d4/d6/d14/dd2 0 2024-02-16T19:29:17.729 INFO:tasks.workunit.client.1.smithi117.stdout:5/837: write d2/d3/f47 [54760,108692] 0 2024-02-16T19:29:17.730 INFO:tasks.workunit.client.0.smithi022.stdout:4/662: chown d4/d6/dd/d22/d83/la7 382884 1 2024-02-16T19:29:17.730 INFO:tasks.workunit.client.1.smithi117.stdout:4/780: mknod d0/da/d21/cfe 0 2024-02-16T19:29:17.731 INFO:tasks.workunit.client.1.smithi117.stdout:2/813: creat d0/d3/d22/f101 x:0 0 0 2024-02-16T19:29:17.732 INFO:tasks.workunit.client.1.smithi117.stdout:2/814: fsync d0/d3/d22/f28 0 2024-02-16T19:29:17.735 INFO:tasks.workunit.client.1.smithi117.stdout:9/713: stat dd/d23/l1c 0 2024-02-16T19:29:17.736 INFO:tasks.workunit.client.1.smithi117.stdout:8/843: truncate d4/d12/d31/f40 2829901 0 2024-02-16T19:29:17.736 INFO:tasks.workunit.client.1.smithi117.stdout:8/844: dread - d4/d7/da6/fee zero size 2024-02-16T19:29:17.737 INFO:tasks.workunit.client.1.smithi117.stdout:1/725: symlink d2/d19/d59/d76/d8c/dc7/dd2/lfa 0 2024-02-16T19:29:17.738 INFO:tasks.workunit.client.1.smithi117.stdout:8/845: dread d4/d12/fd4 [0,4194304] 0 2024-02-16T19:29:17.738 INFO:tasks.workunit.client.1.smithi117.stdout:1/726: chown d2/d1d/d60/d68 29 1 2024-02-16T19:29:17.738 INFO:tasks.workunit.client.1.smithi117.stdout:8/846: fsync d4/d12/d2a/d64/dc6/fae 0 2024-02-16T19:29:17.739 INFO:tasks.workunit.client.0.smithi022.stdout:2/711: symlink d0/d2/dce/le8 0 2024-02-16T19:29:17.743 INFO:tasks.workunit.client.1.smithi117.stdout:0/820: read d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/db7/fe4 [1184898,25454] 0 2024-02-16T19:29:17.743 INFO:tasks.workunit.client.0.smithi022.stdout:5/789: getdents dd/d1f/d48/d21/d2f/d52/d77/d95/ded 0 2024-02-16T19:29:17.744 INFO:tasks.workunit.client.1.smithi117.stdout:5/838: creat d2/d9/d3c/d24/d50/d96/dc4/f11b x:0 0 0 2024-02-16T19:29:17.745 INFO:tasks.workunit.client.0.smithi022.stdout:4/663: symlink d4/d6/d14/d64/d4e/d54/d5a/ld3 0 2024-02-16T19:29:17.745 INFO:tasks.workunit.client.1.smithi117.stdout:5/839: chown d2/d3/d21/d49/d59/c29 1201 1 2024-02-16T19:29:17.746 INFO:tasks.workunit.client.0.smithi022.stdout:4/664: fdatasync d4/d6/dd/d22/da8/dc4/fc3 0 2024-02-16T19:29:17.746 INFO:tasks.workunit.client.1.smithi117.stdout:4/781: creat d0/da/d21/d3d/d28/d64/d8a/d92/fff x:0 0 0 2024-02-16T19:29:17.750 INFO:tasks.workunit.client.1.smithi117.stdout:2/815: symlink d0/d3/d34/l102 0 2024-02-16T19:29:17.750 INFO:tasks.workunit.client.1.smithi117.stdout:5/840: dread d2/d9/d3c/d24/d50/d52/d74/d8f/fc7 [0,4194304] 0 2024-02-16T19:29:17.752 INFO:tasks.workunit.client.1.smithi117.stdout:4/782: dwrite d0/da/d21/d3d/d28/fa6 [0,4194304] 0 2024-02-16T19:29:17.753 INFO:tasks.workunit.client.1.smithi117.stdout:7/804: truncate d4/d1c/d39/d46/f48 1594064 0 2024-02-16T19:29:17.755 INFO:tasks.workunit.client.0.smithi022.stdout:7/753: mknod d9/da/d4e/cfa 0 2024-02-16T19:29:17.760 INFO:tasks.workunit.client.1.smithi117.stdout:8/847: mkdir d4/d7/d112 0 2024-02-16T19:29:17.760 INFO:tasks.workunit.client.1.smithi117.stdout:8/848: chown d4/d7/d6a/d72/d83 1 1 2024-02-16T19:29:17.761 INFO:tasks.workunit.client.1.smithi117.stdout:6/778: symlink d5/d4b/d20/de7/l105 0 2024-02-16T19:29:17.762 INFO:tasks.workunit.client.0.smithi022.stdout:4/665: rmdir d4/d6/dd/d22 39 2024-02-16T19:29:17.762 INFO:tasks.workunit.client.1.smithi117.stdout:6/779: truncate d5/d4b/dd6/fea 999939 0 2024-02-16T19:29:17.767 INFO:tasks.workunit.client.1.smithi117.stdout:4/783: dread d0/da/d21/f2b [0,4194304] 0 2024-02-16T19:29:17.768 INFO:tasks.workunit.client.1.smithi117.stdout:4/784: truncate d0/da/d21/f22 9051413 0 2024-02-16T19:29:17.769 INFO:tasks.workunit.client.1.smithi117.stdout:9/714: mknod dd/d45/cef 0 2024-02-16T19:29:17.770 INFO:tasks.workunit.client.0.smithi022.stdout:2/712: dwrite d0/f20 [0,4194304] 0 2024-02-16T19:29:17.772 INFO:tasks.workunit.client.0.smithi022.stdout:7/754: mkdir d9/da/d1f/d40/dfb 0 2024-02-16T19:29:17.773 INFO:tasks.workunit.client.0.smithi022.stdout:5/790: unlink dd/d1f/f58 0 2024-02-16T19:29:17.776 INFO:tasks.workunit.client.0.smithi022.stdout:4/666: write d4/d6/dd/d22/da8/dc4/fc3 [1894541,100711] 0 2024-02-16T19:29:17.777 INFO:tasks.workunit.client.0.smithi022.stdout:0/913: getdents d10/d7d/dac 0 2024-02-16T19:29:17.778 INFO:tasks.workunit.client.1.smithi117.stdout:1/727: dwrite d2/d1d/f25 [4194304,4194304] 0 2024-02-16T19:29:17.779 INFO:tasks.workunit.client.0.smithi022.stdout:8/802: getdents d3/d12 0 2024-02-16T19:29:17.780 INFO:tasks.workunit.client.1.smithi117.stdout:6/780: dread d5/d4b/f46 [0,4194304] 0 2024-02-16T19:29:17.781 INFO:tasks.workunit.client.1.smithi117.stdout:8/849: rename d4/d7/d112 to d4/d12/d2a/d64/d113 0 2024-02-16T19:29:17.782 INFO:tasks.workunit.client.1.smithi117.stdout:8/850: read - d4/d12/d2a/d64/dec/d76/ff4 zero size 2024-02-16T19:29:17.784 INFO:tasks.workunit.client.1.smithi117.stdout:4/785: read d0/f47 [896061,77333] 0 2024-02-16T19:29:17.786 INFO:tasks.workunit.client.1.smithi117.stdout:9/715: dread dd/d45/d73/f82 [0,4194304] 0 2024-02-16T19:29:17.787 INFO:tasks.workunit.client.0.smithi022.stdout:5/791: symlink dd/d1f/d48/d21/d2f/d52/l106 0 2024-02-16T19:29:17.791 INFO:tasks.workunit.client.1.smithi117.stdout:9/716: dread dd/f4a [0,4194304] 0 2024-02-16T19:29:17.796 INFO:tasks.workunit.client.0.smithi022.stdout:2/713: creat d0/d2/d18/d38/d9d/fe9 x:0 0 0 2024-02-16T19:29:17.797 INFO:tasks.workunit.client.1.smithi117.stdout:0/821: write d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/d27/d4e/fac [3853526,57262] 0 2024-02-16T19:29:17.799 INFO:tasks.workunit.client.1.smithi117.stdout:6/781: rmdir d5/d4b/d44/d93/dae 39 2024-02-16T19:29:17.800 INFO:tasks.workunit.client.0.smithi022.stdout:5/792: mknod dd/d1f/d2e/dd4/c107 0 2024-02-16T19:29:17.803 INFO:tasks.workunit.client.0.smithi022.stdout:4/667: rename d4/c15 to d4/d6/d14/dc7/d6a/cd4 0 2024-02-16T19:29:17.806 INFO:tasks.workunit.client.0.smithi022.stdout:7/755: write d9/faf [712142,52920] 0 2024-02-16T19:29:17.807 INFO:tasks.workunit.client.0.smithi022.stdout:0/914: dwrite d10/d22/d49/f4d [4194304,4194304] 0 2024-02-16T19:29:17.807 INFO:tasks.workunit.client.1.smithi117.stdout:4/786: symlink d0/da/d21/d3d/d28/d64/d73/d91/d9e/dbb/l100 0 2024-02-16T19:29:17.808 INFO:tasks.workunit.client.0.smithi022.stdout:4/668: read d4/d6/dd/df/d10/d24/f8c [3133766,31804] 0 2024-02-16T19:29:17.809 INFO:tasks.workunit.client.0.smithi022.stdout:2/714: mknod d0/d2/d2e/d82/da1/cea 0 2024-02-16T19:29:17.811 INFO:tasks.workunit.client.0.smithi022.stdout:8/803: dwrite d3/d12/f48 [0,4194304] 0 2024-02-16T19:29:17.813 INFO:tasks.workunit.client.1.smithi117.stdout:5/841: link d2/db3/f5f d2/d9/d3c/d24/d50/d52/d74/f11c 0 2024-02-16T19:29:17.813 INFO:tasks.workunit.client.1.smithi117.stdout:8/851: write d4/d7/da6/d3a/d65/d6c/f75 [1943245,6741] 0 2024-02-16T19:29:17.813 INFO:tasks.workunit.client.1.smithi117.stdout:7/805: link d4/d1c/d39/d46/d55/ce6 d4/dd3/c129 0 2024-02-16T19:29:17.820 INFO:tasks.workunit.client.1.smithi117.stdout:0/822: stat d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/df6 0 2024-02-16T19:29:17.821 INFO:tasks.workunit.client.1.smithi117.stdout:6/782: rename d5/dfa to d5/d4b/d44/d52/d81/dfc/d106 0 2024-02-16T19:29:17.822 INFO:tasks.workunit.client.1.smithi117.stdout:6/783: write d5/db/d7d/d32/da8/de1/ffb [457011,94074] 0 2024-02-16T19:29:17.825 INFO:tasks.workunit.client.0.smithi022.stdout:4/669: rmdir d4/d6/dd/d22/d83 39 2024-02-16T19:29:17.827 INFO:tasks.workunit.client.0.smithi022.stdout:8/804: symlink d3/d12/d17/d10d/d110/l116 0 2024-02-16T19:29:17.829 INFO:tasks.workunit.client.0.smithi022.stdout:0/915: truncate d10/d22/fa1 45590 0 2024-02-16T19:29:17.831 INFO:tasks.workunit.client.0.smithi022.stdout:7/756: rename d9/da/d4e/d83 to d9/da/d1f/d44/d5b/dfc 0 2024-02-16T19:29:17.832 INFO:tasks.workunit.client.0.smithi022.stdout:2/715: creat d0/d2/d18/d19/ddf/feb x:0 0 0 2024-02-16T19:29:17.833 INFO:tasks.workunit.client.0.smithi022.stdout:4/670: fsync d4/d6/dd/d22/da8/dc4/d6d/f7e 0 2024-02-16T19:29:17.837 INFO:tasks.workunit.client.0.smithi022.stdout:8/805: rename d3/d12/d17/d64/d8f/d114 to d3/d12/d117 0 2024-02-16T19:29:17.837 INFO:tasks.workunit.client.0.smithi022.stdout:0/916: read d10/f6e [2489057,84368] 0 2024-02-16T19:29:17.837 INFO:tasks.workunit.client.0.smithi022.stdout:0/917: fsync d10/d22/dad/fd0 0 2024-02-16T19:29:17.838 INFO:tasks.workunit.client.1.smithi117.stdout:4/787: write d0/da/d21/d3d/d28/f39 [4494556,27826] 0 2024-02-16T19:29:17.839 INFO:tasks.workunit.client.0.smithi022.stdout:7/757: rmdir d9/da/d74 39 2024-02-16T19:29:17.848 INFO:tasks.workunit.client.0.smithi022.stdout:4/671: read d4/d6/d14/d64/d4e/d54/d5a/f5f [1132561,53462] 0 2024-02-16T19:29:17.858 INFO:tasks.workunit.client.0.smithi022.stdout:2/716: symlink d0/d2/d18/d19/d40/dc9/lec 0 2024-02-16T19:29:17.863 INFO:tasks.workunit.client.0.smithi022.stdout:0/918: write d10/d7d/dac/de9/dee/ffa [705536,12789] 0 2024-02-16T19:29:17.866 INFO:tasks.workunit.client.0.smithi022.stdout:2/717: rename d0/d2/d2e/l46 to d0/d2/d2e/d82/d8a/led 0 2024-02-16T19:29:17.866 INFO:tasks.workunit.client.0.smithi022.stdout:2/718: stat d0/d2/d18/d38/ld4 0 2024-02-16T19:29:17.867 INFO:tasks.workunit.client.0.smithi022.stdout:4/672: truncate d4/d6/dd/d22/d30/fd0 3853877 0 2024-02-16T19:29:17.869 INFO:tasks.workunit.client.1.smithi117.stdout:3/887: sync 2024-02-16T19:29:17.869 INFO:tasks.workunit.client.1.smithi117.stdout:2/816: sync 2024-02-16T19:29:17.869 INFO:tasks.workunit.client.0.smithi022.stdout:7/758: creat d9/da/d2c/ffd x:0 0 0 2024-02-16T19:29:17.873 INFO:tasks.workunit.client.1.smithi117.stdout:3/888: dwrite d1/d1b/d20/fe5 [0,4194304] 0 2024-02-16T19:29:17.874 INFO:tasks.workunit.client.0.smithi022.stdout:2/719: mkdir d0/d5d/dc4/de1/dee 0 2024-02-16T19:29:17.875 INFO:tasks.workunit.client.0.smithi022.stdout:2/720: truncate d0/d2/d18/d1f/d6c/fe2 585831 0 2024-02-16T19:29:17.876 INFO:tasks.workunit.client.0.smithi022.stdout:7/759: mkdir d9/da/d1f/d67/ddb/dfe 0 2024-02-16T19:29:17.878 INFO:tasks.workunit.client.1.smithi117.stdout:9/717: unlink dd/d47/d5e/c89 0 2024-02-16T19:29:17.878 INFO:tasks.workunit.client.0.smithi022.stdout:4/673: truncate d4/d6/d14/f2e 4074873 0 2024-02-16T19:29:17.879 INFO:tasks.workunit.client.0.smithi022.stdout:8/806: creat d3/d12/d2b/d54/f118 x:0 0 0 2024-02-16T19:29:17.880 INFO:tasks.workunit.client.1.smithi117.stdout:8/852: mkdir d4/d7/da6/d3a/d43/d44/d5f/d114 0 2024-02-16T19:29:17.880 INFO:tasks.workunit.client.0.smithi022.stdout:8/807: dread - d3/d12/d2b/d54/f118 zero size 2024-02-16T19:29:17.881 INFO:tasks.workunit.client.0.smithi022.stdout:8/808: fsync d3/d12/d2b/d54/d74/fd8 0 2024-02-16T19:29:17.882 INFO:tasks.workunit.client.0.smithi022.stdout:2/721: fsync d0/d2/d18/d1f/f21 0 2024-02-16T19:29:17.886 INFO:tasks.workunit.client.0.smithi022.stdout:0/919: getdents d10/d11 0 2024-02-16T19:29:17.889 INFO:tasks.workunit.client.1.smithi117.stdout:4/788: rename d0/da/d21/d3d/d28/d64/d73/d91/d37/c88 to d0/da/d21/d3d/d28/d66/c101 0 2024-02-16T19:29:17.895 INFO:tasks.workunit.client.0.smithi022.stdout:4/674: creat d4/d6/dd/df/dc1/fd5 x:0 0 0 2024-02-16T19:29:17.896 INFO:tasks.workunit.client.1.smithi117.stdout:9/718: fsync dd/d47/d5b/d5f/d77/d7c/f8b 0 2024-02-16T19:29:17.898 INFO:tasks.workunit.client.1.smithi117.stdout:8/853: mkdir d4/d7/da6/d3a/d65/d115 0 2024-02-16T19:29:17.898 INFO:tasks.workunit.client.1.smithi117.stdout:8/854: chown d4/d12/fd4 724725662 1 2024-02-16T19:29:17.899 INFO:tasks.workunit.client.0.smithi022.stdout:8/809: symlink d3/d12/d2b/d54/dcd/d115/l119 0 2024-02-16T19:29:17.901 INFO:tasks.workunit.client.0.smithi022.stdout:2/722: dread d0/d2/d18/d19/d90/fa3 [0,4194304] 0 2024-02-16T19:29:17.902 INFO:tasks.workunit.client.0.smithi022.stdout:2/723: dread - d0/d2/d18/d1f/f7b zero size 2024-02-16T19:29:17.902 INFO:tasks.workunit.client.1.smithi117.stdout:0/823: mkdir d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/d27/d10a/d115 0 2024-02-16T19:29:17.903 INFO:tasks.workunit.client.0.smithi022.stdout:2/724: write d0/d32/d3b/f6e [4731720,63961] 0 2024-02-16T19:29:17.904 INFO:tasks.workunit.client.0.smithi022.stdout:2/725: readlink d0/d2/d18/d1f/d6c/ld3 0 2024-02-16T19:29:17.905 INFO:tasks.workunit.client.0.smithi022.stdout:2/726: readlink d0/d2/d18/d1f/l4e 0 2024-02-16T19:29:17.905 INFO:tasks.workunit.client.0.smithi022.stdout:4/675: chown d4/d6/dd/d22/d83/lbf 301 1 2024-02-16T19:29:17.910 INFO:tasks.workunit.client.1.smithi117.stdout:2/817: symlink d0/d13/dc4/dd9/l103 0 2024-02-16T19:29:17.911 INFO:tasks.workunit.client.0.smithi022.stdout:8/810: chown d3/d12/d17/lbc 1050 1 2024-02-16T19:29:17.912 INFO:tasks.workunit.client.1.smithi117.stdout:1/728: sync 2024-02-16T19:29:17.912 INFO:tasks.workunit.client.1.smithi117.stdout:4/789: mkdir d0/da/d21/d3d/d28/d64/d73/d91/d37/d102 0 2024-02-16T19:29:17.912 INFO:tasks.workunit.client.1.smithi117.stdout:1/729: chown d2/f57 263 1 2024-02-16T19:29:17.914 INFO:tasks.workunit.client.1.smithi117.stdout:3/889: unlink d1/d1b/d1e/le6 0 2024-02-16T19:29:17.915 INFO:tasks.workunit.client.0.smithi022.stdout:7/760: link d9/d1b/c36 d9/da9/cff 0 2024-02-16T19:29:17.920 INFO:tasks.workunit.client.0.smithi022.stdout:7/761: dwrite d9/d5d/fc8 [0,4194304] 0 2024-02-16T19:29:17.992 INFO:tasks.workunit.client.1.smithi117.stdout:9/719: rename dd/d23 to dd/d45/d31/d63/dba/df0 0 2024-02-16T19:29:17.996 INFO:tasks.workunit.client.1.smithi117.stdout:8/855: creat d4/d12/d2a/d64/dec/d76/d8c/f116 x:0 0 0 2024-02-16T19:29:17.996 INFO:tasks.workunit.client.1.smithi117.stdout:8/856: fsync d4/d12/d31/d8e/fed 0 2024-02-16T19:29:17.998 INFO:tasks.workunit.client.1.smithi117.stdout:8/857: dread d4/d12/d2a/d64/dec/f87 [0,4194304] 0 2024-02-16T19:29:18.001 INFO:tasks.workunit.client.0.smithi022.stdout:8/811: rmdir d3/d25 39 2024-02-16T19:29:18.001 INFO:tasks.workunit.client.0.smithi022.stdout:8/812: fsync d3/d12/d2b/da9/fdc 0 2024-02-16T19:29:18.004 INFO:tasks.workunit.client.1.smithi117.stdout:5/842: link d2/d4c/d69/l118 d2/d9/d3c/d24/d50/d96/dc4/l11d 0 2024-02-16T19:29:18.005 INFO:tasks.workunit.client.1.smithi117.stdout:7/806: write d4/d1c/d39/d46/d55/db8/fc2 [3858745,58889] 0 2024-02-16T19:29:18.009 INFO:tasks.workunit.client.0.smithi022.stdout:4/676: rename d4/d6/dd/d22/d83/dc5/cbb to d4/d6/dd/d22/da8/dc4/d6d/cd6 0 2024-02-16T19:29:18.010 INFO:tasks.workunit.client.1.smithi117.stdout:1/730: creat d2/d19/d59/d89/d49/ffb x:0 0 0 2024-02-16T19:29:18.013 INFO:tasks.workunit.client.0.smithi022.stdout:8/813: dread d3/d12/db0/fb9 [0,4194304] 0 2024-02-16T19:29:18.013 INFO:tasks.workunit.client.1.smithi117.stdout:3/890: fdatasync d1/d1b/d1e/d29/daf/f127 0 2024-02-16T19:29:18.014 INFO:tasks.workunit.client.1.smithi117.stdout:1/731: dread d2/d19/d59/d89/d69/fe0 [0,4194304] 0 2024-02-16T19:29:18.015 INFO:tasks.workunit.client.0.smithi022.stdout:0/920: dwrite d10/d22/fa1 [0,4194304] 0 2024-02-16T19:29:18.016 INFO:tasks.workunit.client.1.smithi117.stdout:9/720: creat dd/d45/d73/ff1 x:0 0 0 2024-02-16T19:29:18.023 INFO:tasks.workunit.client.1.smithi117.stdout:9/721: dwrite dd/d45/d73/ff1 [0,4194304] 0 2024-02-16T19:29:18.024 INFO:tasks.workunit.client.0.smithi022.stdout:2/727: write d0/f5f [1737288,17475] 0 2024-02-16T19:29:18.032 INFO:tasks.workunit.client.1.smithi117.stdout:0/824: fsync d0/d9/d14/f1b 0 2024-02-16T19:29:18.035 INFO:tasks.workunit.client.1.smithi117.stdout:6/784: getdents d5/db/d7d/d32/da8/de1 0 2024-02-16T19:29:18.035 INFO:tasks.workunit.client.0.smithi022.stdout:0/921: read d10/d11/d50/d65/f68 [211781,103814] 0 2024-02-16T19:29:18.036 INFO:tasks.workunit.client.0.smithi022.stdout:7/762: write d9/da/d2c/d2f/f35 [2263971,42686] 0 2024-02-16T19:29:18.037 INFO:tasks.workunit.client.0.smithi022.stdout:8/814: dread d3/f8d [0,4194304] 0 2024-02-16T19:29:18.037 INFO:tasks.workunit.client.0.smithi022.stdout:0/922: chown d10/d15/c1a 171449 1 2024-02-16T19:29:18.041 INFO:tasks.workunit.client.1.smithi117.stdout:2/818: symlink d0/d3/d22/d37/d9d/df9/l104 0 2024-02-16T19:29:18.041 INFO:tasks.workunit.client.0.smithi022.stdout:8/815: dread d3/d12/d17/d64/d8f/fec [0,4194304] 0 2024-02-16T19:29:18.042 INFO:tasks.workunit.client.0.smithi022.stdout:2/728: mkdir d0/d5d/dd0/def 0 2024-02-16T19:29:18.043 INFO:tasks.workunit.client.1.smithi117.stdout:2/819: truncate d0/d3/d34/d94/d46/d68/d76/da1/d56/fd3 272705 0 2024-02-16T19:29:18.044 INFO:tasks.workunit.client.1.smithi117.stdout:2/820: write d0/d3/f31 [1148169,58513] 0 2024-02-16T19:29:18.044 INFO:tasks.workunit.client.1.smithi117.stdout:4/790: symlink d0/da/d21/d3d/d28/d66/d97/l103 0 2024-02-16T19:29:18.046 INFO:tasks.workunit.client.0.smithi022.stdout:4/677: dwrite d4/d6/d14/d64/d4e/d54/d5a/f5f [0,4194304] 0 2024-02-16T19:29:18.049 INFO:tasks.workunit.client.1.smithi117.stdout:1/732: rename d2/d51 to d2/d19/d59/d89/df4/dfc 0 2024-02-16T19:29:18.050 INFO:tasks.workunit.client.1.smithi117.stdout:2/821: dwrite d0/d13/d4d/d53/fb4 [0,4194304] 0 2024-02-16T19:29:18.053 INFO:tasks.workunit.client.0.smithi022.stdout:0/923: symlink d10/d11/d79/l130 0 2024-02-16T19:29:18.055 INFO:tasks.workunit.client.1.smithi117.stdout:8/858: getdents d4/d7/da6/dfd 0 2024-02-16T19:29:18.058 INFO:tasks.workunit.client.1.smithi117.stdout:6/785: symlink d5/db/d7d/l107 0 2024-02-16T19:29:18.059 INFO:tasks.workunit.client.0.smithi022.stdout:7/763: mknod d9/da/d2c/d46/d73/dca/c100 0 2024-02-16T19:29:18.060 INFO:tasks.workunit.client.1.smithi117.stdout:3/891: dwrite d1/dd/f48 [0,4194304] 0 2024-02-16T19:29:18.060 INFO:tasks.workunit.client.1.smithi117.stdout:6/786: write d5/db/f4a [1740869,30502] 0 2024-02-16T19:29:18.062 INFO:tasks.workunit.client.0.smithi022.stdout:0/924: dwrite d10/d11/d27/d77/f10d [0,4194304] 0 2024-02-16T19:29:18.064 INFO:tasks.workunit.client.0.smithi022.stdout:2/729: rename d0/d2/d18/d38/d6b to d0/d2/d2e/d82/df0 0 2024-02-16T19:29:18.065 INFO:tasks.workunit.client.1.smithi117.stdout:7/807: mknod d4/d1c/d39/d46/d55/d113/c12a 0 2024-02-16T19:29:18.072 INFO:tasks.workunit.client.0.smithi022.stdout:0/925: dwrite d10/d15/d23/d38/f41 [0,4194304] 0 2024-02-16T19:29:18.073 INFO:tasks.workunit.client.0.smithi022.stdout:0/926: dread - d10/d7d/f93 zero size 2024-02-16T19:29:18.074 INFO:tasks.workunit.client.0.smithi022.stdout:0/927: fsync d10/d11/d79/f11d 0 2024-02-16T19:29:18.075 INFO:tasks.workunit.client.0.smithi022.stdout:8/816: creat d3/d12/d17/f11a x:0 0 0 2024-02-16T19:29:18.076 INFO:tasks.workunit.client.1.smithi117.stdout:4/791: creat d0/da/d21/d3d/d28/d64/d48/f104 x:0 0 0 2024-02-16T19:29:18.076 INFO:tasks.workunit.client.0.smithi022.stdout:0/928: chown d10/d11/d27/d3f/f75 27241626 1 2024-02-16T19:29:18.076 INFO:tasks.workunit.client.1.smithi117.stdout:4/792: chown d0/da/d4a/cfa 83022 1 2024-02-16T19:29:18.082 INFO:tasks.workunit.client.1.smithi117.stdout:9/722: mkdir dd/d45/df2 0 2024-02-16T19:29:18.083 INFO:tasks.workunit.client.0.smithi022.stdout:0/929: dread d10/d7d/dac/de9/dee/ffa [0,4194304] 0 2024-02-16T19:29:18.084 INFO:tasks.workunit.client.1.smithi117.stdout:8/859: rmdir d4/d7/da6/d3a/d65/d7c 39 2024-02-16T19:29:18.084 INFO:tasks.workunit.client.0.smithi022.stdout:6/751: sync 2024-02-16T19:29:18.085 INFO:tasks.workunit.client.0.smithi022.stdout:4/678: link d4/d6/dd/l3b d4/d6/d14/dd2/ld7 0 2024-02-16T19:29:18.091 INFO:tasks.workunit.client.1.smithi117.stdout:5/843: rmdir d2/d9/d3c/d24/d50/d52/d117 0 2024-02-16T19:29:18.093 INFO:tasks.workunit.client.1.smithi117.stdout:6/787: fdatasync d5/f86 0 2024-02-16T19:29:18.094 INFO:tasks.workunit.client.0.smithi022.stdout:0/930: fsync d10/d22/d2c/ff9 0 2024-02-16T19:29:18.096 INFO:tasks.workunit.client.1.smithi117.stdout:3/892: creat d1/d30/d9f/dbc/f132 x:0 0 0 2024-02-16T19:29:18.096 INFO:tasks.workunit.client.1.smithi117.stdout:3/893: chown d1/l123 73 1 2024-02-16T19:29:18.097 INFO:tasks.workunit.client.1.smithi117.stdout:3/894: chown d1/dd/d5c 1209 1 2024-02-16T19:29:18.097 INFO:tasks.workunit.client.1.smithi117.stdout:3/895: write d1/d31/f8c [1405079,6114] 0 2024-02-16T19:29:18.098 INFO:tasks.workunit.client.0.smithi022.stdout:7/764: truncate d9/da/d2c/d2f/f9e 2656504 0 2024-02-16T19:29:18.099 INFO:tasks.workunit.client.0.smithi022.stdout:6/752: dread d1/d8/da/d29/d61/f9e [0,4194304] 0 2024-02-16T19:29:18.100 INFO:tasks.workunit.client.0.smithi022.stdout:7/765: write d9/da/d2c/d46/fe5 [621531,62404] 0 2024-02-16T19:29:18.101 INFO:tasks.workunit.client.0.smithi022.stdout:7/766: dread - d9/da/d2c/fc1 zero size 2024-02-16T19:29:18.102 INFO:tasks.workunit.client.1.smithi117.stdout:4/793: unlink d0/da/d21/d3d/d28/d64/d73/d91/d37/fae 0 2024-02-16T19:29:18.103 INFO:tasks.workunit.client.0.smithi022.stdout:2/730: getdents d0/d2/d18/d19/ddf 0 2024-02-16T19:29:18.104 INFO:tasks.workunit.client.1.smithi117.stdout:4/794: write d0/da/d21/d3d/d28/d64/d73/d91/d37/d98/fe7 [454066,136] 0 2024-02-16T19:29:18.107 INFO:tasks.workunit.client.1.smithi117.stdout:0/825: dwrite d0/d9/d14/d1d/d1e/f21 [0,4194304] 0 2024-02-16T19:29:18.111 INFO:tasks.workunit.client.0.smithi022.stdout:0/931: creat d10/d15/d23/d38/d5a/f131 x:0 0 0 2024-02-16T19:29:18.117 INFO:tasks.workunit.client.0.smithi022.stdout:8/817: getdents d3/d12/d17/d10d 0 2024-02-16T19:29:18.118 INFO:tasks.workunit.client.0.smithi022.stdout:8/818: dread - d3/d12/d2b/fde zero size 2024-02-16T19:29:18.120 INFO:tasks.workunit.client.0.smithi022.stdout:6/753: symlink d1/d8/d81/l105 0 2024-02-16T19:29:18.123 INFO:tasks.workunit.client.0.smithi022.stdout:7/767: truncate d9/da/d2c/d2f/f76 234662 0 2024-02-16T19:29:18.126 INFO:tasks.workunit.client.1.smithi117.stdout:3/896: creat d1/dd/d3e/d7d/d121/f133 x:0 0 0 2024-02-16T19:29:18.127 INFO:tasks.workunit.client.0.smithi022.stdout:7/768: dread d9/da/f60 [0,4194304] 0 2024-02-16T19:29:18.127 INFO:tasks.workunit.client.0.smithi022.stdout:5/793: sync 2024-02-16T19:29:18.127 INFO:tasks.workunit.client.0.smithi022.stdout:1/750: sync 2024-02-16T19:29:18.127 INFO:tasks.workunit.client.0.smithi022.stdout:3/698: sync 2024-02-16T19:29:18.127 INFO:tasks.workunit.client.0.smithi022.stdout:9/725: sync 2024-02-16T19:29:18.129 INFO:tasks.workunit.client.1.smithi117.stdout:7/808: truncate d4/d1c/d87/f66 739594 0 2024-02-16T19:29:18.129 INFO:tasks.workunit.client.0.smithi022.stdout:3/699: fdatasync d7/d12/d18/f1b 0 2024-02-16T19:29:18.130 INFO:tasks.workunit.client.0.smithi022.stdout:4/679: truncate d4/d6/d14/d64/d4e/d54/d5a/f5f 1119237 0 2024-02-16T19:29:18.132 INFO:tasks.workunit.client.1.smithi117.stdout:8/860: dwrite d4/d12/d2a/d64/dec/d76/f85 [0,4194304] 0 2024-02-16T19:29:18.133 INFO:tasks.workunit.client.1.smithi117.stdout:3/897: dwrite d1/d1b/d1e/d29/f47 [0,4194304] 0 2024-02-16T19:29:18.135 INFO:tasks.workunit.client.1.smithi117.stdout:2/822: truncate d0/d13/d75/fac 2279017 0 2024-02-16T19:29:18.139 INFO:tasks.workunit.client.1.smithi117.stdout:2/823: dread d0/d3/d10/f16 [0,4194304] 0 2024-02-16T19:29:18.141 INFO:tasks.workunit.client.1.smithi117.stdout:4/795: rmdir d0/da/d21 39 2024-02-16T19:29:18.143 INFO:tasks.workunit.client.1.smithi117.stdout:3/898: dwrite d1/d1b/d1e/d29/daf/f127 [0,4194304] 0 2024-02-16T19:29:18.144 INFO:tasks.workunit.client.0.smithi022.stdout:2/731: rename d0/d2/d2e/d4a to d0/d2/d18/d1f/d6c/df1 0 2024-02-16T19:29:18.150 INFO:tasks.workunit.client.1.smithi117.stdout:0/826: dread d0/d9/d14/d1d/f88 [0,4194304] 0 2024-02-16T19:29:18.151 INFO:tasks.workunit.client.1.smithi117.stdout:5/844: stat d2/d9/d3c/d24/d90/dd4/dfb/d62/l10b 0 2024-02-16T19:29:18.156 INFO:tasks.workunit.client.1.smithi117.stdout:5/845: dwrite d2/d4c/d4f/fd0 [0,4194304] 0 2024-02-16T19:29:18.158 INFO:tasks.workunit.client.0.smithi022.stdout:5/794: symlink dd/d1f/d48/d21/d2f/d52/d5c/d61/d6a/dfb/l108 0 2024-02-16T19:29:18.158 INFO:tasks.workunit.client.1.smithi117.stdout:5/846: chown d2/d9/d3c/d24/d90/dd4/dfb/d62/lf4 151 1 2024-02-16T19:29:18.159 INFO:tasks.workunit.client.1.smithi117.stdout:5/847: readlink d2/d5d/d8e/dc1/lcf 0 2024-02-16T19:29:18.161 INFO:tasks.workunit.client.0.smithi022.stdout:6/754: write d1/d8/da/d29/d61/f9e [456645,123029] 0 2024-02-16T19:29:18.165 INFO:tasks.workunit.client.1.smithi117.stdout:1/733: getdents d2/d19/d59/d89/d69/d80/d8e 0 2024-02-16T19:29:18.166 INFO:tasks.workunit.client.0.smithi022.stdout:6/755: truncate d1/d8/d3c/d77/fe5 222352 0 2024-02-16T19:29:18.167 INFO:tasks.workunit.client.0.smithi022.stdout:5/795: dwrite dd/f11 [0,4194304] 0 2024-02-16T19:29:18.172 INFO:tasks.workunit.client.1.smithi117.stdout:1/734: dwrite d2/d19/d59/d89/f42 [0,4194304] 0 2024-02-16T19:29:18.178 INFO:tasks.workunit.client.0.smithi022.stdout:5/796: dwrite dd/d1f/d48/d21/d2f/d52/d77/d95/fec [4194304,4194304] 0 2024-02-16T19:29:18.178 INFO:tasks.workunit.client.0.smithi022.stdout:1/751: write d6/d8/d22/d59/fc4 [3647477,60828] 0 2024-02-16T19:29:18.180 INFO:tasks.workunit.client.1.smithi117.stdout:2/824: dread d0/d3/d22/f5d [0,4194304] 0 2024-02-16T19:29:18.181 INFO:tasks.workunit.client.0.smithi022.stdout:9/726: symlink db/d14/d16/d5d/d65/d6b/df6/lfb 0 2024-02-16T19:29:18.184 INFO:tasks.workunit.client.0.smithi022.stdout:7/769: link d9/da/d1f/d40/fa5 d9/da/d1f/d93/f101 0 2024-02-16T19:29:18.184 INFO:tasks.workunit.client.1.smithi117.stdout:0/827: dread - d0/d9/d14/d1d/d60/d65/fad zero size 2024-02-16T19:29:18.186 INFO:tasks.workunit.client.1.smithi117.stdout:7/809: write d4/d1c/d39/d46/d47/f77 [500998,70800] 0 2024-02-16T19:29:18.188 INFO:tasks.workunit.client.1.smithi117.stdout:2/825: dwrite d0/d3/d34/d94/d46/d68/d76/da1/d56/d60/dc9/fd8 [0,4194304] 0 2024-02-16T19:29:18.190 INFO:tasks.workunit.client.0.smithi022.stdout:3/700: unlink d7/d12/d18/dd/d24/d29/l3d 0 2024-02-16T19:29:18.194 INFO:tasks.workunit.client.0.smithi022.stdout:6/756: dread d1/d8/f66 [0,4194304] 0 2024-02-16T19:29:18.196 INFO:tasks.workunit.client.1.smithi117.stdout:8/861: rename d4/d12/d31/d8e/dbf/ce1 to d4/d7/d6a/d72/c117 0 2024-02-16T19:29:18.198 INFO:tasks.workunit.client.0.smithi022.stdout:8/819: rename d3/d12/da6/f7 to d3/d12/d2b/da9/dd5/f11b 0 2024-02-16T19:29:18.199 INFO:tasks.workunit.client.0.smithi022.stdout:5/797: dread dd/d1f/d48/d21/d2f/d40/d49/d4b/f66 [0,4194304] 0 2024-02-16T19:29:18.200 INFO:tasks.workunit.client.0.smithi022.stdout:0/932: link d10/d15/d23/dc6/f12f d10/d11/d27/d3f/df0/f132 0 2024-02-16T19:29:18.200 INFO:tasks.workunit.client.1.smithi117.stdout:1/735: stat d2/d19/d59/d89/d7b/ca2 0 2024-02-16T19:29:18.201 INFO:tasks.workunit.client.0.smithi022.stdout:0/933: truncate d10/d15/f128 861900 0 2024-02-16T19:29:18.203 INFO:tasks.workunit.client.0.smithi022.stdout:1/752: symlink d6/d8/d22/d3a/d7a/lea 0 2024-02-16T19:29:18.207 INFO:tasks.workunit.client.1.smithi117.stdout:0/828: mkdir d0/d9/d14/d1d/d60/dc3/dfd/d116 0 2024-02-16T19:29:18.208 INFO:tasks.workunit.client.0.smithi022.stdout:7/770: symlink d9/da/d1f/l102 0 2024-02-16T19:29:18.211 INFO:tasks.workunit.client.0.smithi022.stdout:4/680: creat d4/d6/dd/df/d10/fd8 x:0 0 0 2024-02-16T19:29:18.211 INFO:tasks.workunit.client.1.smithi117.stdout:0/829: dwrite d0/d9/d14/d1d/d60/d65/fe3 [0,4194304] 0 2024-02-16T19:29:18.212 INFO:tasks.workunit.client.0.smithi022.stdout:5/798: dread dd/d1f/d48/d21/d2f/d52/f65 [0,4194304] 0 2024-02-16T19:29:18.212 INFO:tasks.workunit.client.0.smithi022.stdout:4/681: readlink d4/d6/dd/d22/d30/l9d 0 2024-02-16T19:29:18.214 INFO:tasks.workunit.client.0.smithi022.stdout:3/701: mknod d7/d12/d18/dd/d24/d93/cfa 0 2024-02-16T19:29:18.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:17 smithi022 ceph-mon[29221]: pgmap v12: 65 pgs: 65 active+clean; 2.9 GiB data, 8.4 GiB used, 528 GiB / 536 GiB avail; 115 MiB/s rd, 260 MiB/s wr, 560 op/s 2024-02-16T19:29:18.215 INFO:tasks.workunit.client.0.smithi022.stdout:3/702: readlink d7/d12/l26 0 2024-02-16T19:29:18.216 INFO:tasks.workunit.client.0.smithi022.stdout:3/703: dread - d7/d12/d18/dd/d24/d93/de9/f71 zero size 2024-02-16T19:29:18.216 INFO:tasks.workunit.client.1.smithi117.stdout:6/788: getdents d5/d4b/d71/dc7/d49/d59 0 2024-02-16T19:29:18.219 INFO:tasks.workunit.client.1.smithi117.stdout:3/899: rename d1/dd/d3b/d9d to d1/d30/d9f/dcb/d103/d134 0 2024-02-16T19:29:18.219 INFO:tasks.workunit.client.1.smithi117.stdout:3/900: dread - d1/d31/d89/dbd/f131 zero size 2024-02-16T19:29:18.225 INFO:tasks.workunit.client.1.smithi117.stdout:3/901: dwrite d1/fdb [0,4194304] 0 2024-02-16T19:29:18.226 INFO:tasks.workunit.client.1.smithi117.stdout:4/796: creat d0/da/d4a/f105 x:0 0 0 2024-02-16T19:29:18.228 INFO:tasks.workunit.client.1.smithi117.stdout:9/723: link dd/d47/d5e/d9a/ca1 dd/d45/d31/d63/dba/df0/d64/cf3 0 2024-02-16T19:29:18.230 INFO:tasks.workunit.client.1.smithi117.stdout:9/724: chown dd/d45/d31/d63/d8e/led 0 1 2024-02-16T19:29:18.230 INFO:tasks.workunit.client.0.smithi022.stdout:0/934: mkdir d10/d15/d23/d8f/dc1/d133 0 2024-02-16T19:29:18.232 INFO:tasks.workunit.client.1.smithi117.stdout:3/902: dwrite d1/dd/d3b/ff1 [4194304,4194304] 0 2024-02-16T19:29:18.235 INFO:tasks.workunit.client.1.smithi117.stdout:7/810: write d4/d1c/d39/d46/f57 [1362591,36952] 0 2024-02-16T19:29:18.240 INFO:tasks.workunit.client.1.smithi117.stdout:5/848: rename d2/d9/d3c/d24/d90/dd4/dfb to d2/d4c/d4f/d67/d11e 0 2024-02-16T19:29:18.241 INFO:tasks.workunit.client.0.smithi022.stdout:5/799: mknod dd/d1f/d48/d21/d2f/d46/d5e/db6/c109 0 2024-02-16T19:29:18.244 INFO:tasks.workunit.client.0.smithi022.stdout:3/704: read d7/d12/d18/dd/d31/fae [159476,106769] 0 2024-02-16T19:29:18.245 INFO:tasks.workunit.client.1.smithi117.stdout:8/862: creat d4/d7/da6/d3a/d65/d7c/dc9/f118 x:0 0 0 2024-02-16T19:29:18.245 INFO:tasks.workunit.client.1.smithi117.stdout:8/863: stat d4/d7/da6/d50/l51 0 2024-02-16T19:29:18.246 INFO:tasks.workunit.client.0.smithi022.stdout:6/757: creat d1/d8/dac/d101/da3/dad/f106 x:0 0 0 2024-02-16T19:29:18.248 INFO:tasks.workunit.client.0.smithi022.stdout:2/732: rename d0/d2/dce/cda to d0/d2/d2e/d73/de3/cf2 0 2024-02-16T19:29:18.251 INFO:tasks.workunit.client.0.smithi022.stdout:4/682: dwrite d4/d6/d14/f28 [4194304,4194304] 0 2024-02-16T19:29:18.266 INFO:tasks.workunit.client.1.smithi117.stdout:0/830: rename d0/d9/d14/d1d/d60/d76/d7b/d8d/f97 to d0/d9/d14/d23/f117 0 2024-02-16T19:29:18.267 INFO:tasks.workunit.client.1.smithi117.stdout:6/789: truncate d5/db/d7d/d32/d37/f41 2261842 0 2024-02-16T19:29:18.269 INFO:tasks.workunit.client.1.smithi117.stdout:5/849: read - d2/d4c/fd7 zero size 2024-02-16T19:29:18.270 INFO:tasks.workunit.client.1.smithi117.stdout:5/850: write d2/d9/d3c/d24/d50/d96/dbc/d30/ff2 [235002,58587] 0 2024-02-16T19:29:18.270 INFO:tasks.workunit.client.0.smithi022.stdout:7/771: symlink d9/d5d/dc3/l103 0 2024-02-16T19:29:18.271 INFO:tasks.workunit.client.0.smithi022.stdout:8/820: dwrite d3/d12/d92/fc5 [0,4194304] 0 2024-02-16T19:29:18.271 INFO:tasks.workunit.client.0.smithi022.stdout:7/772: chown d9/da/d3b/lf1 798 1 2024-02-16T19:29:18.273 INFO:tasks.workunit.client.0.smithi022.stdout:5/800: dread - dd/d1f/d48/f43 zero size 2024-02-16T19:29:18.275 INFO:tasks.workunit.client.0.smithi022.stdout:5/801: fdatasync dd/d1f/d48/d21/d2f/d52/d5c/d61/f92 0 2024-02-16T19:29:18.276 INFO:tasks.workunit.client.1.smithi117.stdout:4/797: dread d0/da/d21/d3d/d28/fad [0,4194304] 0 2024-02-16T19:29:18.276 INFO:tasks.workunit.client.1.smithi117.stdout:7/811: creat d4/d13/dfd/f12b x:0 0 0 2024-02-16T19:29:18.277 INFO:tasks.workunit.client.1.smithi117.stdout:4/798: write d0/da/d21/fef [560065,85617] 0 2024-02-16T19:29:18.280 INFO:tasks.workunit.client.1.smithi117.stdout:9/725: rename dd/d45/d31/d63/dba/df0/d64 to dd/d45/d31/d63/dba/df0/daf/df4 0 2024-02-16T19:29:18.284 INFO:tasks.workunit.client.0.smithi022.stdout:2/733: dwrite d0/d2/d2e/d82/df0/fca [0,4194304] 0 2024-02-16T19:29:18.285 INFO:tasks.workunit.client.0.smithi022.stdout:2/734: dread - d0/d2/d2e/fd2 zero size 2024-02-16T19:29:18.287 INFO:tasks.workunit.client.0.smithi022.stdout:6/758: write d1/d8/da/d29/deb/dfd/d53/f91 [765596,62713] 0 2024-02-16T19:29:18.287 INFO:tasks.workunit.client.1.smithi117.stdout:6/790: creat d5/db/f108 x:0 0 0 2024-02-16T19:29:18.290 INFO:tasks.workunit.client.0.smithi022.stdout:9/727: link db/d14/l32 db/d14/d33/lfc 0 2024-02-16T19:29:18.295 INFO:tasks.workunit.client.1.smithi117.stdout:1/736: getdents d2/d19/d59/d89 0 2024-02-16T19:29:18.296 INFO:tasks.workunit.client.0.smithi022.stdout:7/773: symlink d9/da/d1f/d44/d5b/dfc/l104 0 2024-02-16T19:29:18.296 INFO:tasks.workunit.client.1.smithi117.stdout:1/737: write d2/d19/d59/d89/d7b/fd9 [165087,64542] 0 2024-02-16T19:29:18.297 INFO:tasks.workunit.client.0.smithi022.stdout:8/821: truncate d3/d12/d2b/f6c 869766 0 2024-02-16T19:29:18.300 INFO:tasks.workunit.client.1.smithi117.stdout:4/799: symlink d0/da/d21/d3d/d28/d64/d90/dc0/l106 0 2024-02-16T19:29:18.302 INFO:tasks.workunit.client.0.smithi022.stdout:0/935: truncate d10/d15/d23/fdf 3822239 0 2024-02-16T19:29:18.302 INFO:tasks.workunit.client.1.smithi117.stdout:9/726: read dd/d47/d5e/f7f [4928427,51805] 0 2024-02-16T19:29:18.304 INFO:tasks.workunit.client.0.smithi022.stdout:9/728: dread db/d14/d16/d27/f68 [0,4194304] 0 2024-02-16T19:29:18.308 INFO:tasks.workunit.client.1.smithi117.stdout:7/812: write d4/d1c/d39/d46/d55/db8/fd5 [223614,88572] 0 2024-02-16T19:29:18.309 INFO:tasks.workunit.client.0.smithi022.stdout:6/759: fsync d1/d8/da/d29/deb/dfd/d85/d56/fa5 0 2024-02-16T19:29:18.310 INFO:tasks.workunit.client.1.smithi117.stdout:8/864: link d4/d1c/fad d4/d7/f119 0 2024-02-16T19:29:18.315 INFO:tasks.workunit.client.0.smithi022.stdout:7/774: fdatasync d9/da/d1f/d40/f5e 0 2024-02-16T19:29:18.315 INFO:tasks.workunit.client.1.smithi117.stdout:0/831: write d0/d9/d14/f92 [104293,6036] 0 2024-02-16T19:29:18.316 INFO:tasks.workunit.client.1.smithi117.stdout:8/865: dwrite d4/d12/d31/d8e/fed [0,4194304] 0 2024-02-16T19:29:18.318 INFO:tasks.workunit.client.1.smithi117.stdout:4/800: creat d0/da/d21/d3d/d28/d64/d73/d91/d37/f107 x:0 0 0 2024-02-16T19:29:18.321 INFO:tasks.workunit.client.0.smithi022.stdout:5/802: mkdir dd/d1f/d48/d21/d2f/d52/d5c/d61/d10a 0 2024-02-16T19:29:18.321 INFO:tasks.workunit.client.0.smithi022.stdout:7/775: dwrite d9/da/d1f/fc5 [0,4194304] 0 2024-02-16T19:29:18.324 INFO:tasks.workunit.client.0.smithi022.stdout:0/936: dread - d10/d11/d27/d3f/d58/dab/ff6 zero size 2024-02-16T19:29:18.327 INFO:tasks.workunit.client.0.smithi022.stdout:5/803: write dd/d1f/d48/d21/d2f/d52/d77/d95/fec [6943384,56642] 0 2024-02-16T19:29:18.328 INFO:tasks.workunit.client.1.smithi117.stdout:5/851: dwrite d2/d3/f65 [0,4194304] 0 2024-02-16T19:29:18.328 INFO:tasks.workunit.client.0.smithi022.stdout:1/753: rename d6/d66/lab to d6/leb 0 2024-02-16T19:29:18.330 INFO:tasks.workunit.client.0.smithi022.stdout:5/804: dread - dd/d1f/d48/d21/d2f/d52/d5c/d61/d6a/db0/fdd zero size 2024-02-16T19:29:18.331 INFO:tasks.workunit.client.0.smithi022.stdout:5/805: dread - dd/d1f/d48/d21/d2f/d40/d4e/f87 zero size 2024-02-16T19:29:18.334 INFO:tasks.workunit.client.0.smithi022.stdout:5/806: read dd/d1f/d48/d21/d2f/d40/d49/fa1 [242894,91671] 0 2024-02-16T19:29:18.336 INFO:tasks.workunit.client.1.smithi117.stdout:8/866: creat d4/d12/d2a/df7/f11a x:0 0 0 2024-02-16T19:29:18.337 INFO:tasks.workunit.client.1.smithi117.stdout:0/832: read d0/d9/d14/f1b [1854374,17490] 0 2024-02-16T19:29:18.338 INFO:tasks.workunit.client.1.smithi117.stdout:9/727: dwrite dd/d45/d31/d63/dba/df0/daf/df4/f91 [0,4194304] 0 2024-02-16T19:29:18.338 INFO:tasks.workunit.client.0.smithi022.stdout:6/760: mknod d1/d8/da/d29/deb/dfd/d53/c107 0 2024-02-16T19:29:18.342 INFO:tasks.workunit.client.0.smithi022.stdout:7/776: dread d9/da/d2c/d46/f6a [0,4194304] 0 2024-02-16T19:29:18.342 INFO:tasks.workunit.client.0.smithi022.stdout:8/822: mknod d3/d12/c11c 0 2024-02-16T19:29:18.343 INFO:tasks.workunit.client.0.smithi022.stdout:7/777: dread - d9/da/d2c/d2f/d21/fec zero size 2024-02-16T19:29:18.344 INFO:tasks.workunit.client.1.smithi117.stdout:6/791: rename d5/db/l76 to d5/d4b/d20/l109 0 2024-02-16T19:29:18.345 INFO:tasks.workunit.client.0.smithi022.stdout:7/778: dread - d9/da/d2c/d2f/d21/fec zero size 2024-02-16T19:29:18.345 INFO:tasks.workunit.client.1.smithi117.stdout:6/792: chown d5/d4b/d44/d52/lb7 335748605 1 2024-02-16T19:29:18.349 INFO:tasks.workunit.client.1.smithi117.stdout:0/833: truncate d0/d9/d14/d1d/d60/d76/d7b/d8d/f36 1777161 0 2024-02-16T19:29:18.350 INFO:tasks.workunit.client.0.smithi022.stdout:2/735: write d0/d2/f14 [2611261,22203] 0 2024-02-16T19:29:18.353 INFO:tasks.workunit.client.0.smithi022.stdout:7/779: dwrite d9/da/fce [0,4194304] 0 2024-02-16T19:29:18.353 INFO:tasks.workunit.client.1.smithi117.stdout:5/852: rename d2/d5d/d8e/dc1 to d2/d4c/dde/d11f 0 2024-02-16T19:29:18.356 INFO:tasks.workunit.client.0.smithi022.stdout:4/683: rename d4/d6/dd/d22/d83/dc5/c6c to d4/d6/dd/d22/da8/dc4/d86/cd9 0 2024-02-16T19:29:18.358 INFO:tasks.workunit.client.1.smithi117.stdout:6/793: creat d5/db/d7d/d32/da8/de1/f10a x:0 0 0 2024-02-16T19:29:18.361 INFO:tasks.workunit.client.1.smithi117.stdout:5/853: dwrite d2/d9/d3c/d24/d50/d52/d74/d8c/f4b [4194304,4194304] 0 2024-02-16T19:29:18.363 INFO:tasks.workunit.client.1.smithi117.stdout:7/813: dread d4/d1c/d98/fa6 [0,4194304] 0 2024-02-16T19:29:18.363 INFO:tasks.workunit.client.1.smithi117.stdout:4/801: dread d0/da/d4a/f58 [0,4194304] 0 2024-02-16T19:29:18.364 INFO:tasks.workunit.client.1.smithi117.stdout:0/834: creat d0/d9/d14/d1d/d60/dc3/dfd/f118 x:0 0 0 2024-02-16T19:29:18.365 INFO:tasks.workunit.client.0.smithi022.stdout:6/761: symlink d1/d8/d23/db1/l108 0 2024-02-16T19:29:18.369 INFO:tasks.workunit.client.1.smithi117.stdout:0/835: read d0/d9/d14/d1d/d60/d65/fab [2599435,45321] 0 2024-02-16T19:29:18.371 INFO:tasks.workunit.client.1.smithi117.stdout:9/728: creat dd/ff5 x:0 0 0 2024-02-16T19:29:18.383 INFO:tasks.workunit.client.0.smithi022.stdout:2/736: mkdir d0/d2/d2e/d82/df0/df3 0 2024-02-16T19:29:18.385 INFO:tasks.workunit.client.0.smithi022.stdout:7/780: unlink d9/da/d1f/c37 0 2024-02-16T19:29:18.386 INFO:tasks.workunit.client.1.smithi117.stdout:9/729: dread dd/d45/d31/d63/dba/df0/f3b [0,4194304] 0 2024-02-16T19:29:18.386 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:17 smithi117 ceph-mon[35309]: pgmap v12: 65 pgs: 65 active+clean; 2.9 GiB data, 8.4 GiB used, 528 GiB / 536 GiB avail; 115 MiB/s rd, 260 MiB/s wr, 560 op/s 2024-02-16T19:29:18.387 INFO:tasks.workunit.client.0.smithi022.stdout:7/781: dread - d9/da/d2c/d46/de3/fe4 zero size 2024-02-16T19:29:18.387 INFO:tasks.workunit.client.0.smithi022.stdout:7/782: stat d9/d1b/fb0 0 2024-02-16T19:29:18.389 INFO:tasks.workunit.client.0.smithi022.stdout:4/684: stat d4/d6/d14/c2d 0 2024-02-16T19:29:18.390 INFO:tasks.workunit.client.0.smithi022.stdout:6/762: mkdir d1/d8/dac/d101/da3/d109 0 2024-02-16T19:29:18.391 INFO:tasks.workunit.client.1.smithi117.stdout:7/814: mkdir d4/d1c/d39/d12c 0 2024-02-16T19:29:18.393 INFO:tasks.workunit.client.0.smithi022.stdout:2/737: symlink d0/d2/d2e/d82/da1/lf4 0 2024-02-16T19:29:18.395 INFO:tasks.workunit.client.0.smithi022.stdout:2/738: dread - d0/d2/d18/d19/ddf/feb zero size 2024-02-16T19:29:18.395 INFO:tasks.workunit.client.0.smithi022.stdout:7/783: chown d9/da9/cff 67567172 1 2024-02-16T19:29:18.396 INFO:tasks.workunit.client.0.smithi022.stdout:9/729: rename db/d14/d16/d5d/d65/d6b/d89/dd2/cd8 to db/d14/d16/d21/d37/cfd 0 2024-02-16T19:29:18.397 INFO:tasks.workunit.client.1.smithi117.stdout:9/730: symlink dd/d45/d31/d63/lf6 0 2024-02-16T19:29:18.401 INFO:tasks.workunit.client.1.smithi117.stdout:6/794: link d5/db/l45 d5/db/d7d/d32/da8/l10b 0 2024-02-16T19:29:18.402 INFO:tasks.workunit.client.0.smithi022.stdout:6/763: creat d1/d8/dac/d101/f10a x:0 0 0 2024-02-16T19:29:18.402 INFO:tasks.workunit.client.0.smithi022.stdout:8/823: unlink d3/d12/d2b/d54/db8/cf0 0 2024-02-16T19:29:18.403 INFO:tasks.workunit.client.0.smithi022.stdout:0/937: getdents d10/d11/d27/d77 0 2024-02-16T19:29:18.404 INFO:tasks.workunit.client.0.smithi022.stdout:5/807: rename dd/d1f/d48/d21/lc2 to dd/d1f/d48/d21/d5d/de7/l10b 0 2024-02-16T19:29:18.407 INFO:tasks.workunit.client.0.smithi022.stdout:7/784: dwrite d9/da9/dbc/fee [0,4194304] 0 2024-02-16T19:29:18.407 INFO:tasks.workunit.client.1.smithi117.stdout:5/854: rename d2/d3/d21/d49/d59/daa/de2 to d2/d3/d21/d49/d59/d120 0 2024-02-16T19:29:18.408 INFO:tasks.workunit.client.1.smithi117.stdout:5/855: readlink d2/d9/d3c/d24/d50/d96/dbc/d30/lb4 0 2024-02-16T19:29:18.409 INFO:tasks.workunit.client.0.smithi022.stdout:4/685: mkdir d4/d6/d14/d64/d4e/d54/d5a/dce/dda 0 2024-02-16T19:29:18.409 INFO:tasks.workunit.client.1.smithi117.stdout:9/731: creat dd/d45/d73/ff7 x:0 0 0 2024-02-16T19:29:18.414 INFO:tasks.workunit.client.1.smithi117.stdout:5/856: rename d2/d9/d3c/d24/d50/d96/dc4/fff to d2/d9/d3c/d24/d50/d52/d74/d8c/f121 0 2024-02-16T19:29:18.416 INFO:tasks.workunit.client.1.smithi117.stdout:6/795: dread d5/f86 [0,4194304] 0 2024-02-16T19:29:18.417 INFO:tasks.workunit.client.1.smithi117.stdout:6/796: chown d5/d4b/d44/d93 64862 1 2024-02-16T19:29:18.420 INFO:tasks.workunit.client.1.smithi117.stdout:6/797: truncate d5/db/d7d/d32/faf 729839 0 2024-02-16T19:29:18.420 INFO:tasks.workunit.client.1.smithi117.stdout:5/857: dwrite d2/d4c/d4f/fd0 [0,4194304] 0 2024-02-16T19:29:18.422 INFO:tasks.workunit.client.1.smithi117.stdout:5/858: write d2/d4c/d69/f72 [1980834,123298] 0 2024-02-16T19:29:18.422 INFO:tasks.workunit.client.1.smithi117.stdout:5/859: chown d2/d4c/d4f 152439 1 2024-02-16T19:29:18.425 INFO:tasks.workunit.client.1.smithi117.stdout:9/732: dread dd/db5/dc4/dcf/fd6 [0,4194304] 0 2024-02-16T19:29:18.435 INFO:tasks.workunit.client.1.smithi117.stdout:9/733: dwrite dd/d47/d48/f98 [0,4194304] 0 2024-02-16T19:29:18.436 INFO:tasks.workunit.client.0.smithi022.stdout:8/824: rmdir d3/d12/d2b/d2e 39 2024-02-16T19:29:18.438 INFO:tasks.workunit.client.0.smithi022.stdout:9/730: truncate db/d14/d16/d21/fb6 6344504 0 2024-02-16T19:29:18.439 INFO:tasks.workunit.client.1.smithi117.stdout:9/734: fsync dd/d45/d31/d63/dba/df0/f1a 0 2024-02-16T19:29:18.441 INFO:tasks.workunit.client.0.smithi022.stdout:2/739: rename d0/d5d/dc4/de1/dee to d0/d2/d18/d19/d40/dc9/df5 0 2024-02-16T19:29:18.446 INFO:tasks.workunit.client.0.smithi022.stdout:5/808: mknod dd/d1f/d48/d21/d2f/c10c 0 2024-02-16T19:29:18.448 INFO:tasks.workunit.client.1.smithi117.stdout:9/735: creat dd/d47/d5b/d5f/d77/d7c/ff8 x:0 0 0 2024-02-16T19:29:18.451 INFO:tasks.workunit.client.0.smithi022.stdout:4/686: creat d4/d6/dd/d22/d83/dc5/fdb x:0 0 0 2024-02-16T19:29:18.456 INFO:tasks.workunit.client.0.smithi022.stdout:2/740: creat d0/d2/d18/d1f/d6c/d99/ff6 x:0 0 0 2024-02-16T19:29:18.456 INFO:tasks.workunit.client.0.smithi022.stdout:5/809: write dd/d1f/d48/d21/d2f/d52/fe6 [52795,14860] 0 2024-02-16T19:29:18.461 INFO:tasks.workunit.client.0.smithi022.stdout:5/810: dread - dd/d1f/d48/d21/d2f/d52/d77/da9/fc8 zero size 2024-02-16T19:29:18.462 INFO:tasks.workunit.client.0.smithi022.stdout:9/731: rmdir db/d14/d16/dec/dee/df5 0 2024-02-16T19:29:18.464 INFO:tasks.workunit.client.0.smithi022.stdout:9/732: symlink db/d14/d16/d21/d4a/lfe 0 2024-02-16T19:29:18.470 INFO:tasks.workunit.client.0.smithi022.stdout:9/733: dwrite db/d14/d16/d27/f78 [0,4194304] 0 2024-02-16T19:29:18.471 INFO:tasks.workunit.client.0.smithi022.stdout:9/734: dread - db/d14/d33/fbd zero size 2024-02-16T19:29:18.474 INFO:tasks.workunit.client.0.smithi022.stdout:9/735: creat db/d14/d16/d49/fff x:0 0 0 2024-02-16T19:29:18.497 INFO:tasks.workunit.client.0.smithi022.stdout:1/754: dread d6/d8/d22/f15 [0,4194304] 0 2024-02-16T19:29:18.501 INFO:tasks.workunit.client.0.smithi022.stdout:1/755: creat d6/d8/d6c/dc9/fec x:0 0 0 2024-02-16T19:29:18.504 INFO:tasks.workunit.client.0.smithi022.stdout:1/756: rmdir d6/d8/d22/d47/d51 39 2024-02-16T19:29:18.505 INFO:tasks.workunit.client.1.smithi117.stdout:4/802: dwrite d0/da/d21/d3d/d28/d64/d73/d91/f31 [0,4194304] 0 2024-02-16T19:29:18.505 INFO:tasks.workunit.client.0.smithi022.stdout:1/757: readlink d6/d8/d22/l2c 0 2024-02-16T19:29:18.507 INFO:tasks.workunit.client.1.smithi117.stdout:0/836: dwrite d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/d27/d83/f67 [0,4194304] 0 2024-02-16T19:29:18.512 INFO:tasks.workunit.client.1.smithi117.stdout:4/803: symlink d0/da/d21/d3d/d28/d66/d97/l108 0 2024-02-16T19:29:18.514 INFO:tasks.workunit.client.0.smithi022.stdout:1/758: rename d6/d8/cb to d6/d8/d22/d47/d51/ced 0 2024-02-16T19:29:18.519 INFO:tasks.workunit.client.1.smithi117.stdout:4/804: creat d0/da/d21/d3d/d28/d66/d97/d6d/f109 x:0 0 0 2024-02-16T19:29:18.520 INFO:tasks.workunit.client.0.smithi022.stdout:2/741: dread d0/f37 [0,4194304] 0 2024-02-16T19:29:18.520 INFO:tasks.workunit.client.1.smithi117.stdout:7/815: dwrite d4/d1c/f26 [0,4194304] 0 2024-02-16T19:29:18.522 INFO:tasks.workunit.client.1.smithi117.stdout:7/816: write d4/d1c/d87/d89/d5f/dae/f105 [839949,18741] 0 2024-02-16T19:29:18.524 INFO:tasks.workunit.client.0.smithi022.stdout:1/759: mkdir d6/d8/d22/d16/d17/d63/dee 0 2024-02-16T19:29:18.527 INFO:tasks.workunit.client.0.smithi022.stdout:2/742: mknod d0/d5d/cf7 0 2024-02-16T19:29:18.532 INFO:tasks.workunit.client.1.smithi117.stdout:2/826: sync 2024-02-16T19:29:18.533 INFO:tasks.workunit.client.0.smithi022.stdout:2/743: dread d0/d2/d2e/d82/df0/fca [0,4194304] 0 2024-02-16T19:29:18.534 INFO:tasks.workunit.client.1.smithi117.stdout:0/837: truncate d0/d9/d14/d1d/d1e/d54/f72 918308 0 2024-02-16T19:29:18.536 INFO:tasks.workunit.client.1.smithi117.stdout:7/817: symlink d4/d13/dfd/l12d 0 2024-02-16T19:29:18.536 INFO:tasks.workunit.client.0.smithi022.stdout:5/811: truncate dd/d1f/d48/d21/d2f/d40/d4e/d56/d9d/fe0 353555 0 2024-02-16T19:29:18.537 INFO:tasks.workunit.client.0.smithi022.stdout:1/760: stat d6/d8/d22/d3f/d99/f9b 0 2024-02-16T19:29:18.538 INFO:tasks.workunit.client.0.smithi022.stdout:1/761: chown d6/d8/d22/d3a/d3e 166319473 1 2024-02-16T19:29:18.539 INFO:tasks.workunit.client.1.smithi117.stdout:4/805: fsync d0/da/d21/d3d/f29 0 2024-02-16T19:29:18.541 INFO:tasks.workunit.client.0.smithi022.stdout:2/744: rename d0/d2/d18/d38/d9d/fe9 to d0/d2/dce/ff8 0 2024-02-16T19:29:18.541 INFO:tasks.workunit.client.1.smithi117.stdout:4/806: read d0/da/d21/d3d/f29 [1560234,64764] 0 2024-02-16T19:29:18.542 INFO:tasks.workunit.client.1.smithi117.stdout:7/818: dwrite d4/d1c/d39/d46/d47/f82 [0,4194304] 0 2024-02-16T19:29:18.543 INFO:tasks.workunit.client.1.smithi117.stdout:2/827: creat d0/d3/d34/d94/d46/f105 x:0 0 0 2024-02-16T19:29:18.549 INFO:tasks.workunit.client.1.smithi117.stdout:0/838: dwrite d0/d9/d14/d1d/d60/d76/d7b/d8d/f48 [0,4194304] 0 2024-02-16T19:29:18.550 INFO:tasks.workunit.client.1.smithi117.stdout:0/839: chown d0/d9/d14/d1d/d60/d76/d7b/d8d/la8 279 1 2024-02-16T19:29:18.557 INFO:tasks.workunit.client.1.smithi117.stdout:2/828: symlink d0/d13/d75/l106 0 2024-02-16T19:29:18.557 INFO:tasks.workunit.client.1.smithi117.stdout:6/798: write d5/db/d7d/d32/f5b [4803,125126] 0 2024-02-16T19:29:18.561 INFO:tasks.workunit.client.1.smithi117.stdout:4/807: rename d0/da/d21/d3d/d28/d64/d73/d91/d9e/fb6 to d0/da/d21/d3d/d28/d66/da2/dac/f10a 0 2024-02-16T19:29:18.565 INFO:tasks.workunit.client.0.smithi022.stdout:5/812: rmdir dd/d1f/d48/d21/d2f/d40/d4e/d56/dcd 0 2024-02-16T19:29:18.567 INFO:tasks.workunit.client.0.smithi022.stdout:2/745: dread d0/d2/f39 [0,4194304] 0 2024-02-16T19:29:18.569 INFO:tasks.workunit.client.0.smithi022.stdout:2/746: write d0/d2/d2e/d82/fc3 [4527148,48174] 0 2024-02-16T19:29:18.570 INFO:tasks.workunit.client.1.smithi117.stdout:2/829: creat d0/d3/d34/d94/d91/f107 x:0 0 0 2024-02-16T19:29:18.571 INFO:tasks.workunit.client.0.smithi022.stdout:2/747: dread - d0/d2/d18/d38/db6/fd9 zero size 2024-02-16T19:29:18.573 INFO:tasks.workunit.client.1.smithi117.stdout:5/860: dwrite d2/f63 [0,4194304] 0 2024-02-16T19:29:18.574 INFO:tasks.workunit.client.0.smithi022.stdout:5/813: dwrite dd/d1f/d48/d21/d2f/d52/d5c/d61/f101 [0,4194304] 0 2024-02-16T19:29:18.574 INFO:tasks.workunit.client.1.smithi117.stdout:5/861: truncate d2/fea 1208446 0 2024-02-16T19:29:18.575 INFO:tasks.workunit.client.1.smithi117.stdout:4/808: creat d0/da/d21/d3d/d28/d66/da2/dac/f10b x:0 0 0 2024-02-16T19:29:18.579 INFO:tasks.workunit.client.1.smithi117.stdout:5/862: dread d2/d4c/d4f/fd0 [0,4194304] 0 2024-02-16T19:29:18.582 INFO:tasks.workunit.client.1.smithi117.stdout:3/903: sync 2024-02-16T19:29:18.582 INFO:tasks.workunit.client.1.smithi117.stdout:8/867: sync 2024-02-16T19:29:18.583 INFO:tasks.workunit.client.1.smithi117.stdout:1/738: sync 2024-02-16T19:29:18.584 INFO:tasks.workunit.client.0.smithi022.stdout:5/814: creat dd/d1f/d48/d21/d2f/d52/f10d x:0 0 0 2024-02-16T19:29:18.586 INFO:tasks.workunit.client.1.smithi117.stdout:2/830: rmdir d0/d13/d4d/d53/d58 39 2024-02-16T19:29:18.589 INFO:tasks.workunit.client.1.smithi117.stdout:4/809: rename d0/da/d21/d3d/d28/d64/d73/d91/f8f to d0/da/d21/d3d/d28/d64/d48/f10c 0 2024-02-16T19:29:18.597 INFO:tasks.workunit.client.0.smithi022.stdout:6/764: dwrite d1/d8/da/d29/deb/dfd/d48/f7c [0,4194304] 0 2024-02-16T19:29:18.598 INFO:tasks.workunit.client.0.smithi022.stdout:7/785: write d9/da/d1f/d67/ddb/fd3 [361109,50205] 0 2024-02-16T19:29:18.599 INFO:tasks.workunit.client.1.smithi117.stdout:2/831: creat d0/d13/d4d/d80/f108 x:0 0 0 2024-02-16T19:29:18.601 INFO:tasks.workunit.client.1.smithi117.stdout:2/832: write d0/d3/d34/d94/d46/d68/d76/da1/d56/d60/dc9/fea [300823,103507] 0 2024-02-16T19:29:18.604 INFO:tasks.workunit.client.1.smithi117.stdout:3/904: rename d1/d1b/d20 to d1/dd/d5c/d85/d135 0 2024-02-16T19:29:18.604 INFO:tasks.workunit.client.0.smithi022.stdout:5/815: link f5 dd/d1f/d48/d21/d2f/d46/d5e/d84/db4/f10e 0 2024-02-16T19:29:18.607 INFO:tasks.workunit.client.1.smithi117.stdout:9/736: dwrite dd/d47/d5b/fa4 [0,4194304] 0 2024-02-16T19:29:18.607 INFO:tasks.workunit.client.1.smithi117.stdout:4/810: creat d0/da/d21/d3d/d28/d66/da2/f10d x:0 0 0 2024-02-16T19:29:18.608 INFO:tasks.workunit.client.0.smithi022.stdout:2/748: rmdir d0/d2/d18/d19/d40/dc9/df5 0 2024-02-16T19:29:18.610 INFO:tasks.workunit.client.0.smithi022.stdout:0/938: dwrite d10/d15/d23/fdf [0,4194304] 0 2024-02-16T19:29:18.612 INFO:tasks.workunit.client.0.smithi022.stdout:4/687: write d4/d6/dd/f85 [3510034,110382] 0 2024-02-16T19:29:18.619 INFO:tasks.workunit.client.0.smithi022.stdout:8/825: dwrite d3/d25/f5c [0,4194304] 0 2024-02-16T19:29:18.620 INFO:tasks.workunit.client.0.smithi022.stdout:5/816: dwrite dd/d1f/d48/d21/d2f/d40/d4e/d56/f59 [0,4194304] 0 2024-02-16T19:29:18.620 INFO:tasks.workunit.client.0.smithi022.stdout:2/749: write d0/d2/d18/d19/f2f [2451033,59095] 0 2024-02-16T19:29:18.625 INFO:tasks.workunit.client.1.smithi117.stdout:3/905: rmdir d1/d30/d52/d10f 39 2024-02-16T19:29:18.626 INFO:tasks.workunit.client.0.smithi022.stdout:9/736: dwrite db/f79 [0,4194304] 0 2024-02-16T19:29:18.635 INFO:tasks.workunit.client.1.smithi117.stdout:4/811: dread - d0/da/d21/d3d/d28/d64/d73/fa9 zero size 2024-02-16T19:29:18.636 INFO:tasks.workunit.client.0.smithi022.stdout:4/688: dread d4/d6/db6/f78 [4194304,4194304] 0 2024-02-16T19:29:18.640 INFO:tasks.workunit.client.0.smithi022.stdout:5/817: mkdir dd/d1f/d48/d21/d5d/de7/d7a/d10f 0 2024-02-16T19:29:18.641 INFO:tasks.workunit.client.0.smithi022.stdout:5/818: chown dd/d1f/d48/d21/d2f/d52/d5c/d61/fda 30386 1 2024-02-16T19:29:18.642 INFO:tasks.workunit.client.0.smithi022.stdout:0/939: mkdir d10/d11/d12b/d134 0 2024-02-16T19:29:18.643 INFO:tasks.workunit.client.0.smithi022.stdout:0/940: stat d10/d11/d50/d65/c83 0 2024-02-16T19:29:18.645 INFO:tasks.workunit.client.0.smithi022.stdout:0/941: dread d10/d11/f73 [0,4194304] 0 2024-02-16T19:29:18.645 INFO:tasks.workunit.client.0.smithi022.stdout:2/750: creat d0/d2/d18/d19/d40/dc9/ff9 x:0 0 0 2024-02-16T19:29:18.647 INFO:tasks.workunit.client.1.smithi117.stdout:3/906: mkdir d1/d30/d9f/dcb/d103/d134/d136 0 2024-02-16T19:29:18.647 INFO:tasks.workunit.client.1.smithi117.stdout:3/907: chown d1/d30/d9f/dcb/le4 10555980 1 2024-02-16T19:29:18.648 INFO:tasks.workunit.client.1.smithi117.stdout:3/908: stat d1/l10 0 2024-02-16T19:29:18.650 INFO:tasks.workunit.client.0.smithi022.stdout:8/826: rename d3/d12/d2b/d54/dcd/d115 to d3/d12/db0/d11d 0 2024-02-16T19:29:18.650 INFO:tasks.workunit.client.1.smithi117.stdout:1/739: link d2/d19/d59/d76/d91/cda d2/d19/d59/d76/d8c/dc7/dd2/cfd 0 2024-02-16T19:29:18.652 INFO:tasks.workunit.client.0.smithi022.stdout:4/689: chown d4/d6/d14/dd2/ld7 152 1 2024-02-16T19:29:18.656 INFO:tasks.workunit.client.1.smithi117.stdout:7/819: dwrite d4/d13/da3/fa7 [0,4194304] 0 2024-02-16T19:29:18.656 INFO:tasks.workunit.client.1.smithi117.stdout:2/833: rmdir d0/d3/d34/d94/df0 0 2024-02-16T19:29:18.658 INFO:tasks.workunit.client.1.smithi117.stdout:9/737: rename dd/d45/d31/d63/dba/df0/f18 to dd/ff9 0 2024-02-16T19:29:18.659 INFO:tasks.workunit.client.0.smithi022.stdout:0/942: write d10/d15/d23/d4e/f12a [561940,31377] 0 2024-02-16T19:29:18.661 INFO:tasks.workunit.client.1.smithi117.stdout:0/840: dwrite d0/d9/d14/d1d/d60/dc3/fcc [0,4194304] 0 2024-02-16T19:29:18.661 INFO:tasks.workunit.client.1.smithi117.stdout:1/740: dwrite d2/d1d/d60/d68/f81 [0,4194304] 0 2024-02-16T19:29:18.667 INFO:tasks.workunit.client.1.smithi117.stdout:6/799: dwrite d5/f13 [0,4194304] 0 2024-02-16T19:29:18.669 INFO:tasks.workunit.client.0.smithi022.stdout:5/819: dread dd/d1f/d48/f2d [4194304,4194304] 0 2024-02-16T19:29:18.672 INFO:tasks.workunit.client.1.smithi117.stdout:8/868: write d4/d73/f82 [772388,29635] 0 2024-02-16T19:29:18.672 INFO:tasks.workunit.client.0.smithi022.stdout:2/751: creat d0/d5d/dd0/def/ffa x:0 0 0 2024-02-16T19:29:18.673 INFO:tasks.workunit.client.1.smithi117.stdout:8/869: stat d4/d12/d2a/d64/f7d 0 2024-02-16T19:29:18.675 INFO:tasks.workunit.client.0.smithi022.stdout:3/705: sync 2024-02-16T19:29:18.675 INFO:tasks.workunit.client.0.smithi022.stdout:0/943: symlink d10/d15/l135 0 2024-02-16T19:29:18.679 INFO:tasks.workunit.client.0.smithi022.stdout:9/737: link db/d18/f93 db/d14/d16/d27/d8b/f100 0 2024-02-16T19:29:18.681 INFO:tasks.workunit.client.1.smithi117.stdout:2/834: mkdir d0/d3/d22/d37/d109 0 2024-02-16T19:29:18.683 INFO:tasks.workunit.client.0.smithi022.stdout:0/944: dwrite d10/d11/d27/d3f/df0/ff1 [0,4194304] 0 2024-02-16T19:29:18.686 INFO:tasks.workunit.client.0.smithi022.stdout:8/827: unlink d3/d12/d17/f50 0 2024-02-16T19:29:18.687 INFO:tasks.workunit.client.1.smithi117.stdout:3/909: symlink d1/d1b/d1e/d29/daf/l137 0 2024-02-16T19:29:18.687 INFO:tasks.workunit.client.0.smithi022.stdout:6/765: write d1/d8/d3c/d4d/f45 [2584425,19375] 0 2024-02-16T19:29:18.690 INFO:tasks.workunit.client.0.smithi022.stdout:5/820: mknod dd/d1f/d48/d21/d2f/d52/d5c/d61/d6a/dfb/c110 0 2024-02-16T19:29:18.694 INFO:tasks.workunit.client.1.smithi117.stdout:6/800: creat d5/d4b/d71/dc7/d49/d59/f10c x:0 0 0 2024-02-16T19:29:18.694 INFO:tasks.workunit.client.1.smithi117.stdout:6/801: read d5/d9c/daa/fb2 [9622753,117479] 0 2024-02-16T19:29:18.695 INFO:tasks.workunit.client.1.smithi117.stdout:6/802: write d5/d4b/d71/dc7/d49/f88 [4497589,9643] 0 2024-02-16T19:29:18.696 INFO:tasks.workunit.client.1.smithi117.stdout:6/803: dread - d5/d9c/fdc zero size 2024-02-16T19:29:18.700 INFO:tasks.workunit.client.1.smithi117.stdout:6/804: readlink d5/d4b/d20/de7/l105 0 2024-02-16T19:29:18.700 INFO:tasks.workunit.client.1.smithi117.stdout:2/835: creat d0/d3/d34/d94/d46/d68/d93/ddc/f10a x:0 0 0 2024-02-16T19:29:18.700 INFO:tasks.workunit.client.1.smithi117.stdout:6/805: write d5/db/d7d/fd1 [813239,23824] 0 2024-02-16T19:29:18.700 INFO:tasks.workunit.client.1.smithi117.stdout:2/836: stat d0/d3/d34/d94/d46/d68/d76/da1/d56/fd3 0 2024-02-16T19:29:18.700 INFO:tasks.workunit.client.1.smithi117.stdout:6/806: write d5/d9c/f9e [1926814,77624] 0 2024-02-16T19:29:18.701 INFO:tasks.workunit.client.0.smithi022.stdout:0/945: dread - d10/d11/d50/d65/fed zero size 2024-02-16T19:29:18.701 INFO:tasks.workunit.client.0.smithi022.stdout:8/828: dread - d3/d12/da6/ffa zero size 2024-02-16T19:29:18.702 INFO:tasks.workunit.client.1.smithi117.stdout:4/812: getdents d0 0 2024-02-16T19:29:18.702 INFO:tasks.workunit.client.0.smithi022.stdout:8/829: chown d3/d12/d2b/d54/d74/da2/fac 222 1 2024-02-16T19:29:18.703 INFO:tasks.workunit.client.0.smithi022.stdout:6/766: fdatasync d1/f2 0 2024-02-16T19:29:18.704 INFO:tasks.workunit.client.0.smithi022.stdout:6/767: chown d1/d8/d23/dc6 59 1 2024-02-16T19:29:18.707 INFO:tasks.workunit.client.0.smithi022.stdout:5/821: creat dd/d1f/d48/d21/d2f/d46/f111 x:0 0 0 2024-02-16T19:29:18.707 INFO:tasks.workunit.client.0.smithi022.stdout:6/768: dread d1/d8/da/d7b/fb2 [0,4194304] 0 2024-02-16T19:29:18.707 INFO:tasks.workunit.client.0.smithi022.stdout:5/822: fsync dd/d1f/d76/fd6 0 2024-02-16T19:29:18.712 INFO:tasks.workunit.client.0.smithi022.stdout:3/706: mknod d7/d12/d18/dd/cfb 0 2024-02-16T19:29:18.713 INFO:tasks.workunit.client.0.smithi022.stdout:3/707: chown d7/d12/d18/dd/d24/c45 38949893 1 2024-02-16T19:29:18.713 INFO:tasks.workunit.client.0.smithi022.stdout:5/823: dread dd/d1f/d48/d21/d5d/de7/f83 [0,4194304] 0 2024-02-16T19:29:18.714 INFO:tasks.workunit.client.1.smithi117.stdout:3/910: creat d1/d30/d9f/dbc/d12f/f138 x:0 0 0 2024-02-16T19:29:18.717 INFO:tasks.workunit.client.1.smithi117.stdout:6/807: creat d5/d4b/d20/de7/f10d x:0 0 0 2024-02-16T19:29:18.718 INFO:tasks.workunit.client.0.smithi022.stdout:0/946: creat d10/d11/d50/d102/f136 x:0 0 0 2024-02-16T19:29:18.719 INFO:tasks.workunit.client.1.smithi117.stdout:3/911: dwrite d1/d31/f7e [0,4194304] 0 2024-02-16T19:29:18.719 INFO:tasks.workunit.client.0.smithi022.stdout:4/690: getdents d4/d6/dd/d22/da8/dc4/d86 0 2024-02-16T19:29:18.723 INFO:tasks.workunit.client.0.smithi022.stdout:8/830: unlink d3/d12/d2b/d54/d74/da2/fac 0 2024-02-16T19:29:18.724 INFO:tasks.workunit.client.1.smithi117.stdout:6/808: dread d5/d4b/d71/dc7/dce/fad [0,4194304] 0 2024-02-16T19:29:18.725 INFO:tasks.workunit.client.1.smithi117.stdout:7/820: write d4/f5 [2023160,70461] 0 2024-02-16T19:29:18.727 INFO:tasks.workunit.client.1.smithi117.stdout:9/738: getdents dd/d45/d31 0 2024-02-16T19:29:18.727 INFO:tasks.workunit.client.1.smithi117.stdout:9/739: chown dd/d45/d31 37 1 2024-02-16T19:29:18.735 INFO:tasks.workunit.client.0.smithi022.stdout:9/738: write db/d18/f2a [409100,5388] 0 2024-02-16T19:29:18.736 INFO:tasks.workunit.client.0.smithi022.stdout:3/708: write d7/d21/fa6 [1382838,88977] 0 2024-02-16T19:29:18.737 INFO:tasks.workunit.client.1.smithi117.stdout:8/870: getdents d4/d12/d2a/d6d/de3 0 2024-02-16T19:29:18.737 INFO:tasks.workunit.client.0.smithi022.stdout:3/709: stat d7/d12 0 2024-02-16T19:29:18.740 INFO:tasks.workunit.client.1.smithi117.stdout:0/841: sync 2024-02-16T19:29:18.740 INFO:tasks.workunit.client.1.smithi117.stdout:5/863: sync 2024-02-16T19:29:18.741 INFO:tasks.workunit.client.1.smithi117.stdout:1/741: sync 2024-02-16T19:29:18.741 INFO:tasks.workunit.client.0.smithi022.stdout:0/947: mkdir d10/d22/d137 0 2024-02-16T19:29:18.747 INFO:tasks.workunit.client.1.smithi117.stdout:3/912: symlink d1/d30/d52/l139 0 2024-02-16T19:29:18.748 INFO:tasks.workunit.client.1.smithi117.stdout:8/871: fdatasync d4/d7/f9d 0 2024-02-16T19:29:18.748 INFO:tasks.workunit.client.0.smithi022.stdout:6/769: write d1/d8/da/d7b/dae/dbb/fbd [54334,10298] 0 2024-02-16T19:29:18.750 INFO:tasks.workunit.client.0.smithi022.stdout:2/752: getdents d0/d32/d3b 0 2024-02-16T19:29:18.750 INFO:tasks.workunit.client.1.smithi117.stdout:2/837: truncate d0/d3/f31 143995 0 2024-02-16T19:29:18.751 INFO:tasks.workunit.client.1.smithi117.stdout:2/838: readlink d0/d3/d34/d94/d46/d68/d76/le1 0 2024-02-16T19:29:18.752 INFO:tasks.workunit.client.1.smithi117.stdout:5/864: symlink d2/d9/l122 0 2024-02-16T19:29:18.752 INFO:tasks.workunit.client.0.smithi022.stdout:5/824: write dd/d1f/d48/d21/d2f/d52/d5c/fce [3313698,67949] 0 2024-02-16T19:29:18.753 INFO:tasks.workunit.client.1.smithi117.stdout:5/865: stat d2/d4c/d4f/d67/d11e/d62/lf4 0 2024-02-16T19:29:18.753 INFO:tasks.workunit.client.0.smithi022.stdout:5/825: fdatasync dd/d1f/d76/fd6 0 2024-02-16T19:29:18.754 INFO:tasks.workunit.client.1.smithi117.stdout:4/813: write d0/da/d21/d3d/d28/d64/f67 [545095,97324] 0 2024-02-16T19:29:18.754 INFO:tasks.workunit.client.1.smithi117.stdout:5/866: chown d2/d9/d3c/d24/d50/d52/l19 39390 1 2024-02-16T19:29:18.755 INFO:tasks.workunit.client.0.smithi022.stdout:5/826: truncate dd/d1f/d48/d21/d2f/d46/f111 861151 0 2024-02-16T19:29:18.757 INFO:tasks.workunit.client.1.smithi117.stdout:7/821: symlink d4/d1c/d39/d9f/l12e 0 2024-02-16T19:29:18.759 INFO:tasks.workunit.client.0.smithi022.stdout:0/948: unlink d10/d15/f128 0 2024-02-16T19:29:18.760 INFO:tasks.workunit.client.0.smithi022.stdout:0/949: readlink d10/d15/d23/l8c 0 2024-02-16T19:29:18.761 INFO:tasks.workunit.client.0.smithi022.stdout:1/762: sync 2024-02-16T19:29:18.762 INFO:tasks.workunit.client.0.smithi022.stdout:4/691: mkdir d4/d6/dd/df/d10/ddc 0 2024-02-16T19:29:18.762 INFO:tasks.workunit.client.0.smithi022.stdout:4/692: readlink d4/d6/dd/d22/d30/la1 0 2024-02-16T19:29:18.763 INFO:tasks.workunit.client.1.smithi117.stdout:3/913: chown d1/d30/d52/d10f/d114 9493 1 2024-02-16T19:29:18.763 INFO:tasks.workunit.client.1.smithi117.stdout:3/914: readlink d1/d83/la7 0 2024-02-16T19:29:18.764 INFO:tasks.workunit.client.0.smithi022.stdout:8/831: creat d3/d12/d2b/d2e/f11e x:0 0 0 2024-02-16T19:29:18.765 INFO:tasks.workunit.client.0.smithi022.stdout:6/770: rmdir d1/d8/d23/dc6 39 2024-02-16T19:29:18.767 INFO:tasks.workunit.client.0.smithi022.stdout:3/710: dwrite d7/d12/d18/dd/d31/d7f/faf [0,4194304] 0 2024-02-16T19:29:18.768 INFO:tasks.workunit.client.0.smithi022.stdout:2/753: dread d0/f78 [0,4194304] 0 2024-02-16T19:29:18.771 INFO:tasks.workunit.client.0.smithi022.stdout:2/754: truncate d0/d2/d18/d19/ddf/feb 844987 0 2024-02-16T19:29:18.771 INFO:tasks.workunit.client.1.smithi117.stdout:5/867: fsync d2/d9/fe3 0 2024-02-16T19:29:18.772 INFO:tasks.workunit.client.0.smithi022.stdout:5/827: creat dd/d1f/d48/d21/d5d/f112 x:0 0 0 2024-02-16T19:29:18.774 INFO:tasks.workunit.client.1.smithi117.stdout:7/822: creat d4/d1c/d87/d89/d5f/dae/df3/f12f x:0 0 0 2024-02-16T19:29:18.775 INFO:tasks.workunit.client.1.smithi117.stdout:7/823: chown d4/d1c/d39/d46/da4/fed 3069202 1 2024-02-16T19:29:18.778 INFO:tasks.workunit.client.1.smithi117.stdout:0/842: link d0/d9/d14/d23/d33/f5c d0/d9/d2c/d8b/de0/f119 0 2024-02-16T19:29:18.780 INFO:tasks.workunit.client.0.smithi022.stdout:0/950: unlink d10/d22/d49/c89 0 2024-02-16T19:29:18.784 INFO:tasks.workunit.client.1.smithi117.stdout:9/740: creat dd/d47/d5b/d5f/d77/ffa x:0 0 0 2024-02-16T19:29:18.784 INFO:tasks.workunit.client.1.smithi117.stdout:1/742: write d2/d19/d59/d89/d49/fcd [197143,110230] 0 2024-02-16T19:29:18.787 INFO:tasks.workunit.client.0.smithi022.stdout:9/739: dwrite db/d14/d16/d27/d8b/f100 [4194304,4194304] 0 2024-02-16T19:29:18.796 INFO:tasks.workunit.client.0.smithi022.stdout:1/763: write d6/d8/d22/d67/f9f [325571,104497] 0 2024-02-16T19:29:18.799 INFO:tasks.workunit.client.0.smithi022.stdout:5/828: fdatasync dd/d1f/d48/d21/d2f/d46/d5e/f7b 0 2024-02-16T19:29:18.801 INFO:tasks.workunit.client.1.smithi117.stdout:7/824: mknod d4/d1c/d98/def/c130 0 2024-02-16T19:29:18.801 INFO:tasks.workunit.client.1.smithi117.stdout:4/814: dwrite d0/da/f33 [0,4194304] 0 2024-02-16T19:29:18.802 INFO:tasks.workunit.client.0.smithi022.stdout:5/829: read dd/d1f/d48/d21/d2f/d52/d5c/fce [3793264,108817] 0 2024-02-16T19:29:18.802 INFO:tasks.workunit.client.0.smithi022.stdout:0/951: rename d10/d7d/dac/de9/dee/ffa to d10/d15/d23/d38/d5a/f138 0 2024-02-16T19:29:18.803 INFO:tasks.workunit.client.0.smithi022.stdout:8/832: dwrite d3/d12/d2b/f3f [0,4194304] 0 2024-02-16T19:29:18.804 INFO:tasks.workunit.client.0.smithi022.stdout:5/830: chown dd/d1f/d48/d21/d2f/d40/d49/d4b/ca2 15231972 1 2024-02-16T19:29:18.804 INFO:tasks.workunit.client.1.smithi117.stdout:3/915: dwrite d1/d1b/d1e/d37/ddc/f10e [0,4194304] 0 2024-02-16T19:29:18.805 INFO:tasks.workunit.client.0.smithi022.stdout:4/693: symlink d4/d6/dd/d22/ldd 0 2024-02-16T19:29:18.808 INFO:tasks.workunit.client.0.smithi022.stdout:6/771: symlink d1/d8/dac/d101/da3/dfa/l10b 0 2024-02-16T19:29:18.808 INFO:tasks.workunit.client.1.smithi117.stdout:3/916: chown d1/dd/d3b 1 1 2024-02-16T19:29:18.809 INFO:tasks.workunit.client.1.smithi117.stdout:4/815: dread - d0/da/d21/d3d/d28/d66/da2/dac/fec zero size 2024-02-16T19:29:18.812 INFO:tasks.workunit.client.1.smithi117.stdout:8/872: dwrite d4/d12/d31/f40 [0,4194304] 0 2024-02-16T19:29:18.816 INFO:tasks.workunit.client.1.smithi117.stdout:7/825: dwrite d4/d1c/d87/fff [0,4194304] 0 2024-02-16T19:29:18.824 INFO:tasks.workunit.client.0.smithi022.stdout:0/952: creat d10/d15/d23/d4e/f139 x:0 0 0 2024-02-16T19:29:18.827 INFO:tasks.workunit.client.1.smithi117.stdout:4/816: symlink d0/da/d21/db4/l10e 0 2024-02-16T19:29:18.828 INFO:tasks.workunit.client.0.smithi022.stdout:8/833: creat d3/d12/d2b/da9/dd5/f11f x:0 0 0 2024-02-16T19:29:18.828 INFO:tasks.workunit.client.1.smithi117.stdout:8/873: dread - d4/d73/f9f zero size 2024-02-16T19:29:18.828 INFO:tasks.workunit.client.1.smithi117.stdout:3/917: dwrite d1/dd/f48 [0,4194304] 0 2024-02-16T19:29:18.829 INFO:tasks.workunit.client.0.smithi022.stdout:5/831: dread - dd/d1f/d48/d21/d5d/dc0/fe9 zero size 2024-02-16T19:29:18.831 INFO:tasks.workunit.client.1.smithi117.stdout:8/874: chown d4/d12/d2a/d64/dec/d76/d8c/fd7 2425401 1 2024-02-16T19:29:18.832 INFO:tasks.workunit.client.1.smithi117.stdout:4/817: read d0/da/d21/d3d/d28/d64/d73/d91/d37/d98/fe7 [154223,118440] 0 2024-02-16T19:29:18.832 INFO:tasks.workunit.client.0.smithi022.stdout:4/694: unlink d4/d6/d14/d64/d4e/d54/dba/f92 0 2024-02-16T19:29:18.835 INFO:tasks.workunit.client.0.smithi022.stdout:8/834: dread d3/f8d [0,4194304] 0 2024-02-16T19:29:18.839 INFO:tasks.workunit.client.1.smithi117.stdout:9/741: truncate dd/d47/d48/f55 4543270 0 2024-02-16T19:29:18.839 INFO:tasks.workunit.client.1.smithi117.stdout:1/743: write d2/d19/d59/f97 [2496028,103959] 0 2024-02-16T19:29:18.839 INFO:tasks.workunit.client.0.smithi022.stdout:1/764: dwrite d6/d8/d22/d47/d54/f65 [4194304,4194304] 0 2024-02-16T19:29:18.840 INFO:tasks.workunit.client.0.smithi022.stdout:3/711: dread d7/d12/d18/dd/d24/d58/d70/f81 [4194304,4194304] 0 2024-02-16T19:29:18.843 INFO:tasks.workunit.client.0.smithi022.stdout:5/832: dread dd/d1f/d48/d21/f23 [0,4194304] 0 2024-02-16T19:29:18.846 INFO:tasks.workunit.client.0.smithi022.stdout:5/833: write dd/d1f/d48/d21/d5d/fb3 [3723185,82840] 0 2024-02-16T19:29:18.847 INFO:tasks.workunit.client.0.smithi022.stdout:2/755: rename d0/c12 to d0/d2/d18/d19/d40/d55/d59/cfb 0 2024-02-16T19:29:18.848 INFO:tasks.workunit.client.1.smithi117.stdout:0/843: dread d0/f10 [4194304,4194304] 0 2024-02-16T19:29:18.848 INFO:tasks.workunit.client.0.smithi022.stdout:0/953: mknod d10/d22/c13a 0 2024-02-16T19:29:18.850 INFO:tasks.workunit.client.0.smithi022.stdout:4/695: chown d4/l7a 611 1 2024-02-16T19:29:18.852 INFO:tasks.workunit.client.0.smithi022.stdout:9/740: creat db/d14/f101 x:0 0 0 2024-02-16T19:29:18.854 INFO:tasks.workunit.client.0.smithi022.stdout:1/765: rmdir d6/d8/d22/d16/d17/d63/d6f/d8b 39 2024-02-16T19:29:18.858 INFO:tasks.workunit.client.0.smithi022.stdout:5/834: mkdir dd/d1f/d48/d21/d2f/d46/d113 0 2024-02-16T19:29:18.859 INFO:tasks.workunit.client.0.smithi022.stdout:5/835: write dd/f3c [3924964,52310] 0 2024-02-16T19:29:18.859 INFO:tasks.workunit.client.0.smithi022.stdout:2/756: mknod d0/d2/d18/d19/d40/d55/d59/cfc 0 2024-02-16T19:29:18.862 INFO:tasks.workunit.client.0.smithi022.stdout:0/954: mkdir d10/d15/d23/d8f/d13b 0 2024-02-16T19:29:18.869 INFO:tasks.workunit.client.0.smithi022.stdout:5/836: dwrite dd/d1f/d48/d21/d5d/dc0/ffc [0,4194304] 0 2024-02-16T19:29:18.869 INFO:tasks.workunit.client.0.smithi022.stdout:9/741: rename db/d14/d19/l39 to db/d14/d16/d5d/d65/d6b/dd5/l102 0 2024-02-16T19:29:18.871 INFO:tasks.workunit.client.0.smithi022.stdout:0/955: dwrite d10/d22/d49/ff8 [0,4194304] 0 2024-02-16T19:29:18.872 INFO:tasks.workunit.client.0.smithi022.stdout:5/837: chown dd/d1f/d48/d21/d2f/d40/c6c 1274581 1 2024-02-16T19:29:18.879 INFO:tasks.workunit.client.0.smithi022.stdout:2/757: mkdir d0/d2/d2e/d82/d8a/dfd 0 2024-02-16T19:29:18.883 INFO:tasks.workunit.client.0.smithi022.stdout:3/712: dread d7/d12/d18/dd/d31/d7f/fa0 [0,4194304] 0 2024-02-16T19:29:18.883 INFO:tasks.workunit.client.0.smithi022.stdout:8/835: link d3/d12/db0/f81 d3/f120 0 2024-02-16T19:29:18.889 INFO:tasks.workunit.client.0.smithi022.stdout:8/836: dwrite d3/d25/f61 [0,4194304] 0 2024-02-16T19:29:18.891 INFO:tasks.workunit.client.0.smithi022.stdout:8/837: write d3/d25/f43 [528843,13920] 0 2024-02-16T19:29:18.892 INFO:tasks.workunit.client.0.smithi022.stdout:8/838: read d3/d12/d2b/f3f [417231,95378] 0 2024-02-16T19:29:18.939 INFO:tasks.workunit.client.0.smithi022.stdout:4/696: fsync d4/d6/d14/d64/d4e/d54/d5a/f5f 0 2024-02-16T19:29:18.941 INFO:tasks.workunit.client.0.smithi022.stdout:4/697: stat d4/d6/d14/dc7/d6a/cb1 0 2024-02-16T19:29:18.956 INFO:tasks.workunit.client.0.smithi022.stdout:7/786: sync 2024-02-16T19:29:18.957 INFO:tasks.workunit.client.0.smithi022.stdout:7/787: chown d9/da/d4e/d7a/fea 14 1 2024-02-16T19:29:18.958 INFO:tasks.workunit.client.0.smithi022.stdout:7/788: chown d9/da/d1f/la6 20 1 2024-02-16T19:29:18.967 INFO:tasks.workunit.client.1.smithi117.stdout:2/839: sync 2024-02-16T19:29:18.967 INFO:tasks.workunit.client.1.smithi117.stdout:6/809: sync 2024-02-16T19:29:18.969 INFO:tasks.workunit.client.1.smithi117.stdout:2/840: dread - d0/d3/d34/d94/d46/d68/d76/da1/fe9 zero size 2024-02-16T19:29:18.969 INFO:tasks.workunit.client.1.smithi117.stdout:2/841: chown d0/ldf 479229977 1 2024-02-16T19:29:18.994 INFO:tasks.workunit.client.0.smithi022.stdout:9/742: mkdir db/d18/d103 0 2024-02-16T19:29:18.995 INFO:tasks.workunit.client.0.smithi022.stdout:9/743: read - db/d14/ff4 zero size 2024-02-16T19:29:18.996 INFO:tasks.workunit.client.0.smithi022.stdout:6/772: sync 2024-02-16T19:29:18.998 INFO:tasks.workunit.client.0.smithi022.stdout:1/766: mkdir d6/d8/d22/d47/def 0 2024-02-16T19:29:19.004 INFO:tasks.workunit.client.0.smithi022.stdout:6/773: dwrite d1/d8/da/d29/deb/dfd/d53/fed [0,4194304] 0 2024-02-16T19:29:19.007 INFO:tasks.workunit.client.0.smithi022.stdout:5/838: read - dd/d1f/d48/d21/d2f/d46/d5e/d84/f5a zero size 2024-02-16T19:29:19.011 INFO:tasks.workunit.client.0.smithi022.stdout:5/839: chown dd/d1f/d48/d21/d2f/d52/d5c/d61/d6a/dfb/l108 268295476 1 2024-02-16T19:29:19.011 INFO:tasks.workunit.client.0.smithi022.stdout:0/956: dwrite d10/d11/d27/d3f/d58/d60/f129 [0,4194304] 0 2024-02-16T19:29:19.024 INFO:tasks.workunit.client.1.smithi117.stdout:5/868: sync 2024-02-16T19:29:19.026 INFO:tasks.workunit.client.0.smithi022.stdout:8/839: dread - d3/d25/d33/fcf zero size 2024-02-16T19:29:19.027 INFO:tasks.workunit.client.0.smithi022.stdout:8/840: chown d3/d25/d33/fcf 24 1 2024-02-16T19:29:19.027 INFO:tasks.workunit.client.0.smithi022.stdout:8/841: chown d3/d12/d17/d10d/d110 140 1 2024-02-16T19:29:19.032 INFO:tasks.workunit.client.0.smithi022.stdout:4/698: symlink d4/d6/db6/lde 0 2024-02-16T19:29:19.033 INFO:tasks.workunit.client.0.smithi022.stdout:4/699: chown d4/d6/dd/d22/da8/dc4/d6d/f89 0 1 2024-02-16T19:29:19.034 INFO:tasks.workunit.client.0.smithi022.stdout:4/700: read d4/d6/dd/d22/f68 [1101949,61235] 0 2024-02-16T19:29:19.035 INFO:tasks.workunit.client.0.smithi022.stdout:7/789: mknod d9/da/d1f/d93/c105 0 2024-02-16T19:29:19.042 INFO:tasks.workunit.client.0.smithi022.stdout:8/842: dread d3/d12/d17/f9d [0,4194304] 0 2024-02-16T19:29:19.050 INFO:tasks.workunit.client.0.smithi022.stdout:6/774: dread d1/d8/da/d29/deb/dfd/d48/d51/f9b [0,4194304] 0 2024-02-16T19:29:19.062 INFO:tasks.workunit.client.0.smithi022.stdout:0/957: rename d10/d11/d27/d3f/d58/dab/cfb to d10/d15/d23/d8f/d13b/c13c 0 2024-02-16T19:29:19.072 INFO:tasks.workunit.client.1.smithi117.stdout:3/918: symlink d1/dd/d5c/d85/d135/l13a 0 2024-02-16T19:29:19.078 INFO:tasks.workunit.client.1.smithi117.stdout:8/875: rmdir d4/d7/da6/d3a/d43/d44/d5f 39 2024-02-16T19:29:19.081 INFO:tasks.workunit.client.1.smithi117.stdout:4/818: mknod d0/c10f 0 2024-02-16T19:29:19.085 INFO:tasks.workunit.client.1.smithi117.stdout:4/819: dread d0/da/f33 [8388608,4194304] 0 2024-02-16T19:29:19.085 INFO:tasks.workunit.client.1.smithi117.stdout:1/744: creat d2/d19/d59/d89/d69/ffe x:0 0 0 2024-02-16T19:29:19.086 INFO:tasks.workunit.client.1.smithi117.stdout:4/820: fsync d0/da/d4a/fd5 0 2024-02-16T19:29:19.086 INFO:tasks.workunit.client.1.smithi117.stdout:1/745: fsync d2/d1d/d60/d7c/d9e/da5/fe1 0 2024-02-16T19:29:19.088 INFO:tasks.workunit.client.0.smithi022.stdout:8/843: creat d3/d25/ddb/f121 x:0 0 0 2024-02-16T19:29:19.091 INFO:tasks.workunit.client.0.smithi022.stdout:1/767: symlink d6/d8/d22/d16/d17/d1d/lf0 0 2024-02-16T19:29:19.092 INFO:tasks.workunit.client.0.smithi022.stdout:1/768: chown d6/d8/d22/d16/d17/d63/d6f 83 1 2024-02-16T19:29:19.093 INFO:tasks.workunit.client.0.smithi022.stdout:1/769: stat d6/d8/d22/f9a 0 2024-02-16T19:29:19.093 INFO:tasks.workunit.client.1.smithi117.stdout:6/810: creat d5/db/d7d/d32/f10e x:0 0 0 2024-02-16T19:29:19.095 INFO:tasks.workunit.client.0.smithi022.stdout:5/840: creat dd/d1f/d2e/dd4/ddf/f114 x:0 0 0 2024-02-16T19:29:19.103 INFO:tasks.workunit.client.1.smithi117.stdout:2/842: symlink d0/d3/d40/l10b 0 2024-02-16T19:29:19.105 INFO:tasks.workunit.client.0.smithi022.stdout:0/958: dread - d10/d11/d27/d3f/d58/d60/ff7 zero size 2024-02-16T19:29:19.113 INFO:tasks.workunit.client.0.smithi022.stdout:4/701: creat d4/d6/dd/df/d10/ddc/fdf x:0 0 0 2024-02-16T19:29:19.115 INFO:tasks.workunit.client.0.smithi022.stdout:7/790: mkdir d9/d5d/dc7/ddc/d106 0 2024-02-16T19:29:19.116 INFO:tasks.workunit.client.0.smithi022.stdout:7/791: chown d9/da/d1f/d67/ddb/c8a 1 1 2024-02-16T19:29:19.117 INFO:tasks.workunit.client.0.smithi022.stdout:7/792: truncate d9/da/d1f/d67/db3/ff0 836915 0 2024-02-16T19:29:19.118 INFO:tasks.workunit.client.0.smithi022.stdout:8/844: fdatasync d3/d25/d33/fa0 0 2024-02-16T19:29:19.122 INFO:tasks.workunit.client.0.smithi022.stdout:1/770: stat d6/c14 0 2024-02-16T19:29:19.122 INFO:tasks.workunit.client.0.smithi022.stdout:1/771: write d6/d8/d22/d47/db7/fdb [180113,52307] 0 2024-02-16T19:29:19.123 INFO:tasks.workunit.client.0.smithi022.stdout:1/772: write d6/f55 [3327989,71853] 0 2024-02-16T19:29:19.124 INFO:tasks.workunit.client.0.smithi022.stdout:0/959: creat d10/d11/d27/d77/f13d x:0 0 0 2024-02-16T19:29:19.241 INFO:tasks.workunit.client.1.smithi117.stdout:9/742: creat dd/d45/d31/d63/dba/df0/daf/df4/dd7/ffb x:0 0 0 2024-02-16T19:29:19.242 INFO:tasks.workunit.client.0.smithi022.stdout:8/845: symlink d3/d25/d33/l122 0 2024-02-16T19:29:19.244 INFO:tasks.workunit.client.0.smithi022.stdout:8/846: write d3/d12/d2b/da9/f103 [921091,111844] 0 2024-02-16T19:29:19.245 INFO:tasks.workunit.client.0.smithi022.stdout:8/847: chown d3/d12/da6/f2f 3281587 1 2024-02-16T19:29:19.248 INFO:tasks.workunit.client.1.smithi117.stdout:1/746: mknod d2/d19/d59/d89/d69/dad/dc6/dcc/cff 0 2024-02-16T19:29:19.253 INFO:tasks.workunit.client.1.smithi117.stdout:6/811: dread - d5/d4b/d20/da4/fda zero size 2024-02-16T19:29:19.254 INFO:tasks.workunit.client.0.smithi022.stdout:0/960: mknod d10/d15/d23/d8f/dc1/c13e 0 2024-02-16T19:29:19.255 INFO:tasks.workunit.client.1.smithi117.stdout:2/843: chown d0/d3/d34/d94/d46/d68/d76/da1/f49 62 1 2024-02-16T19:29:19.255 INFO:tasks.workunit.client.1.smithi117.stdout:2/844: chown d0/d3/d40/fba 9148021 1 2024-02-16T19:29:19.255 INFO:tasks.workunit.client.1.smithi117.stdout:2/845: dread - d0/d13/dc4/ffb zero size 2024-02-16T19:29:19.256 INFO:tasks.workunit.client.0.smithi022.stdout:7/793: mknod d9/da/d1f/d44/d5b/df4/c107 0 2024-02-16T19:29:19.256 INFO:tasks.workunit.client.1.smithi117.stdout:2/846: chown d0/d3/d34/d94/d46/d68/d76/da1/d56/d60/l67 16 1 2024-02-16T19:29:19.258 INFO:tasks.workunit.client.0.smithi022.stdout:8/848: symlink d3/d12/d2b/dca/l123 0 2024-02-16T19:29:19.259 INFO:tasks.workunit.client.1.smithi117.stdout:3/919: creat d1/d30/d52/d10f/d11f/f13b x:0 0 0 2024-02-16T19:29:19.263 INFO:tasks.workunit.client.0.smithi022.stdout:7/794: symlink d9/da/d1f/d40/l108 0 2024-02-16T19:29:19.267 INFO:tasks.workunit.client.1.smithi117.stdout:4/821: mkdir d0/da/d21/d3d/d28/d64/d8a/d92/de8/d110 0 2024-02-16T19:29:19.268 INFO:tasks.workunit.client.1.smithi117.stdout:4/822: dread - d0/da/d21/d3d/d28/d66/d97/d6d/f109 zero size 2024-02-16T19:29:19.270 INFO:tasks.workunit.client.1.smithi117.stdout:0/844: link d0/d9/f1c d0/d9/d14/d1d/d60/d76/d7b/d8d/f11a 0 2024-02-16T19:29:19.275 INFO:tasks.workunit.client.1.smithi117.stdout:0/845: dwrite d0/f53 [0,4194304] 0 2024-02-16T19:29:19.275 INFO:tasks.workunit.client.0.smithi022.stdout:7/795: dread d9/da/d1f/f22 [4194304,4194304] 0 2024-02-16T19:29:19.276 INFO:tasks.workunit.client.1.smithi117.stdout:2/847: unlink d0/d3/d34/d94/d46/d68/d93/ddc/f10a 0 2024-02-16T19:29:19.278 INFO:tasks.workunit.client.1.smithi117.stdout:8/876: creat d4/d12/d2a/d64/dec/f11b x:0 0 0 2024-02-16T19:29:19.281 INFO:tasks.workunit.client.1.smithi117.stdout:9/743: mknod dd/d45/cfc 0 2024-02-16T19:29:19.282 INFO:tasks.workunit.client.1.smithi117.stdout:4/823: readlink d0/da/d4a/lcc 0 2024-02-16T19:29:19.284 INFO:tasks.workunit.client.1.smithi117.stdout:4/824: stat d0/da/d21/d3d/d28/d64/d73/d91/d37/f107 0 2024-02-16T19:29:19.288 INFO:tasks.workunit.client.1.smithi117.stdout:0/846: read - d0/d9/d14/fee zero size 2024-02-16T19:29:19.288 INFO:tasks.workunit.client.1.smithi117.stdout:0/847: readlink d0/d9/d2c/d8b/l8a 0 2024-02-16T19:29:19.289 INFO:tasks.workunit.client.1.smithi117.stdout:2/848: symlink d0/d3/d34/d94/d91/l10c 0 2024-02-16T19:29:19.291 INFO:tasks.workunit.client.1.smithi117.stdout:2/849: write d0/d3/f2a [2724505,111089] 0 2024-02-16T19:29:19.294 INFO:tasks.workunit.client.1.smithi117.stdout:4/825: truncate d0/da/d21/f3f 1383276 0 2024-02-16T19:29:19.298 INFO:tasks.workunit.client.1.smithi117.stdout:0/848: write d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/d27/d31/d94/f10d [434812,76962] 0 2024-02-16T19:29:19.305 INFO:tasks.workunit.client.1.smithi117.stdout:0/849: mkdir d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/d27/d4e/d11b 0 2024-02-16T19:29:19.306 INFO:tasks.workunit.client.1.smithi117.stdout:0/850: chown d0/d30/cae 138882806 1 2024-02-16T19:29:19.306 INFO:tasks.workunit.client.1.smithi117.stdout:2/850: truncate d0/d13/d4d/d53/fa4 1005996 0 2024-02-16T19:29:19.308 INFO:tasks.workunit.client.1.smithi117.stdout:4/826: symlink d0/da/d21/dc2/l111 0 2024-02-16T19:29:19.309 INFO:tasks.workunit.client.1.smithi117.stdout:8/877: getdents d4/d12/d2a/d6d 0 2024-02-16T19:29:19.310 INFO:tasks.workunit.client.1.smithi117.stdout:9/744: getdents dd/db5 0 2024-02-16T19:29:19.311 INFO:tasks.workunit.client.1.smithi117.stdout:4/827: write d0/da/d21/d3d/d28/d64/f67 [454456,55664] 0 2024-02-16T19:29:19.313 INFO:tasks.workunit.client.1.smithi117.stdout:0/851: dwrite d0/d30/f66 [0,4194304] 0 2024-02-16T19:29:19.322 INFO:tasks.workunit.client.1.smithi117.stdout:0/852: mknod d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/d27/d31/d6d/c11c 0 2024-02-16T19:29:19.327 INFO:tasks.workunit.client.1.smithi117.stdout:0/853: read d0/d9/d14/d1d/d60/d76/d7b/d8d/f36 [301989,105022] 0 2024-02-16T19:29:19.332 INFO:tasks.workunit.client.1.smithi117.stdout:4/828: dread d0/da/f5b [0,4194304] 0 2024-02-16T19:29:19.334 INFO:tasks.workunit.client.1.smithi117.stdout:4/829: write d0/da/d21/d3d/d28/d64/d90/dc0/dd7/fdc [40959,56984] 0 2024-02-16T19:29:19.335 INFO:tasks.workunit.client.1.smithi117.stdout:8/878: dread d4/d12/d2a/d64/dec/d76/d8c/fa4 [0,4194304] 0 2024-02-16T19:29:19.335 INFO:tasks.workunit.client.1.smithi117.stdout:0/854: truncate d0/d9/d14/d1d/d60/d76/d7b/d8d/f11a 1003434 0 2024-02-16T19:29:19.337 INFO:tasks.workunit.client.1.smithi117.stdout:8/879: write d4/d12/d2a/d64/dc6/ff3 [403968,119932] 0 2024-02-16T19:29:19.338 INFO:tasks.workunit.client.1.smithi117.stdout:0/855: stat d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/d27/d10a 0 2024-02-16T19:29:19.354 INFO:tasks.workunit.client.0.smithi022.stdout:9/744: write db/d14/d16/f69 [731482,83133] 0 2024-02-16T19:29:19.356 INFO:tasks.workunit.client.0.smithi022.stdout:9/745: mkdir db/d14/d16/d87/d104 0 2024-02-16T19:29:19.360 INFO:tasks.workunit.client.0.smithi022.stdout:4/702: sync 2024-02-16T19:29:19.361 INFO:tasks.workunit.client.0.smithi022.stdout:4/703: chown d4/d6/d14/d64/c48 14 1 2024-02-16T19:29:19.374 INFO:tasks.workunit.client.0.smithi022.stdout:3/713: truncate d7/d12/d18/dd/d31/d7f/faf 466697 0 2024-02-16T19:29:19.378 INFO:tasks.workunit.client.1.smithi117.stdout:7/826: write d4/d1c/f3c [146809,118490] 0 2024-02-16T19:29:19.378 INFO:tasks.workunit.client.0.smithi022.stdout:3/714: getdents d7/d12/d18/dd/d24/d29 0 2024-02-16T19:29:19.385 INFO:tasks.workunit.client.0.smithi022.stdout:3/715: link d7/d12/d18/dd/c34 d7/d12/d18/dd/d24/d58/d70/cfc 0 2024-02-16T19:29:19.388 INFO:tasks.workunit.client.0.smithi022.stdout:3/716: getdents d7 0 2024-02-16T19:29:19.392 INFO:tasks.workunit.client.0.smithi022.stdout:3/717: dread d7/d12/d18/f42 [0,4194304] 0 2024-02-16T19:29:19.394 INFO:tasks.workunit.client.0.smithi022.stdout:3/718: stat d7/d12/d18/dd/d24/d58/d6b/f89 0 2024-02-16T19:29:19.395 INFO:tasks.workunit.client.0.smithi022.stdout:3/719: chown d7/d12/d18/dd/d31/dbf/dc5 4435970 1 2024-02-16T19:29:19.401 INFO:tasks.workunit.client.0.smithi022.stdout:3/720: dwrite d7/d12/d18/dd/d3c/f65 [0,4194304] 0 2024-02-16T19:29:19.409 INFO:tasks.workunit.client.1.smithi117.stdout:5/869: dwrite d2/d9/f7c [0,4194304] 0 2024-02-16T19:29:19.411 INFO:tasks.workunit.client.1.smithi117.stdout:5/870: write d2/fea [1996171,122141] 0 2024-02-16T19:29:19.417 INFO:tasks.workunit.client.1.smithi117.stdout:5/871: creat d2/d36/d6f/f123 x:0 0 0 2024-02-16T19:29:19.419 INFO:tasks.workunit.client.1.smithi117.stdout:5/872: readlink d2/d9/d3c/d24/d50/d96/dbc/d30/d38/d51/ld9 0 2024-02-16T19:29:19.435 INFO:tasks.workunit.client.0.smithi022.stdout:3/721: write d7/d12/d18/dd/d24/d58/d70/f8e [881764,9444] 0 2024-02-16T19:29:19.438 INFO:tasks.workunit.client.1.smithi117.stdout:3/920: link d1/d30/d52/d56/lb7 d1/d31/d89/l13c 0 2024-02-16T19:29:19.439 INFO:tasks.workunit.client.0.smithi022.stdout:0/961: write d10/d15/d23/d38/d5a/f5d [1571917,35026] 0 2024-02-16T19:29:19.440 INFO:tasks.workunit.client.1.smithi117.stdout:3/921: dread - d1/d31/fe8 zero size 2024-02-16T19:29:19.441 INFO:tasks.workunit.client.0.smithi022.stdout:0/962: write d10/d11/d50/f123 [422753,101163] 0 2024-02-16T19:29:19.443 INFO:tasks.workunit.client.1.smithi117.stdout:3/922: mkdir d1/d30/d9f/dbc/d12f/d13d 0 2024-02-16T19:29:19.444 INFO:tasks.workunit.client.0.smithi022.stdout:7/796: mkdir d9/da/d1f/d44/d5b/d109 0 2024-02-16T19:29:19.446 INFO:tasks.workunit.client.1.smithi117.stdout:3/923: rmdir d1/d1b/d1e 39 2024-02-16T19:29:19.448 INFO:tasks.workunit.client.0.smithi022.stdout:8/849: dwrite d3/d12/db0/fd9 [0,4194304] 0 2024-02-16T19:29:19.448 INFO:tasks.workunit.client.0.smithi022.stdout:7/797: mkdir d9/da/d1f/d44/d5b/d95/d10a 0 2024-02-16T19:29:19.449 INFO:tasks.workunit.client.0.smithi022.stdout:8/850: stat d3/d12/db0/ce0 0 2024-02-16T19:29:19.457 INFO:tasks.workunit.client.0.smithi022.stdout:2/758: mkdir d0/d2/dfe 0 2024-02-16T19:29:19.457 INFO:tasks.workunit.client.0.smithi022.stdout:7/798: dwrite d9/da/d2c/d46/de3/fe4 [0,4194304] 0 2024-02-16T19:29:19.461 INFO:tasks.workunit.client.0.smithi022.stdout:8/851: rmdir d3/d12/d2b/d54/d97/d111 0 2024-02-16T19:29:19.462 INFO:tasks.workunit.client.0.smithi022.stdout:7/799: rmdir d9/d1b/d23 39 2024-02-16T19:29:19.465 INFO:tasks.workunit.client.1.smithi117.stdout:6/812: write d5/d9c/daa/d85/db0/fcb [734388,72499] 0 2024-02-16T19:29:19.465 INFO:tasks.workunit.client.0.smithi022.stdout:2/759: mkdir d0/d2/d2e/d82/df0/dff 0 2024-02-16T19:29:19.467 INFO:tasks.workunit.client.0.smithi022.stdout:8/852: creat d3/dd3/f124 x:0 0 0 2024-02-16T19:29:19.468 INFO:tasks.workunit.client.0.smithi022.stdout:2/760: chown d0/d2/dce 18702611 1 2024-02-16T19:29:19.468 INFO:tasks.workunit.client.0.smithi022.stdout:7/800: mkdir d9/da/d2c/d46/d10b 0 2024-02-16T19:29:19.474 INFO:tasks.workunit.client.0.smithi022.stdout:6/775: rename d1/ff4 to d1/d8/da/d29/deb/dfd/d48/de4/dfb/f10c 0 2024-02-16T19:29:19.475 INFO:tasks.workunit.client.0.smithi022.stdout:2/761: mkdir d0/d2/d18/d19/d90/d100 0 2024-02-16T19:29:19.477 INFO:tasks.workunit.client.0.smithi022.stdout:8/853: symlink d3/d12/d6a/l125 0 2024-02-16T19:29:19.484 INFO:tasks.workunit.client.0.smithi022.stdout:7/801: creat d9/da/d2c/f10c x:0 0 0 2024-02-16T19:29:19.489 INFO:tasks.workunit.client.0.smithi022.stdout:8/854: creat d3/d12/d2b/f126 x:0 0 0 2024-02-16T19:29:19.490 INFO:tasks.workunit.client.0.smithi022.stdout:8/855: chown d3/d12/d2b/d2e/c39 769 1 2024-02-16T19:29:19.491 INFO:tasks.workunit.client.0.smithi022.stdout:5/841: rename dd/d1f/d48/c20 to dd/d1f/d48/d21/d2f/d46/d5e/d84/c115 0 2024-02-16T19:29:19.493 INFO:tasks.workunit.client.0.smithi022.stdout:8/856: read d3/d12/d2b/da9/fe4 [2651933,73228] 0 2024-02-16T19:29:19.495 INFO:tasks.workunit.client.0.smithi022.stdout:3/722: rename d7/d12/d18/dd/d3c/c8c to d7/d12/d18/dac/dd7/cfd 0 2024-02-16T19:29:19.501 INFO:tasks.workunit.client.0.smithi022.stdout:0/963: rename d10/d11/d50/d65/fed to d10/d15/d23/d38/f13f 0 2024-02-16T19:29:19.501 INFO:tasks.workunit.client.1.smithi117.stdout:9/745: dwrite dd/d47/d5b/d5f/d77/fac [0,4194304] 0 2024-02-16T19:29:19.502 INFO:tasks.workunit.client.0.smithi022.stdout:3/723: truncate d7/d12/d18/dd/d24/d58/d6b/f94 312585 0 2024-02-16T19:29:19.509 INFO:tasks.workunit.client.1.smithi117.stdout:1/747: rename d2/d19/d59/d76/d8c/dc7/dcb to d2/d19/d59/d89/d69/dad/d100 0 2024-02-16T19:29:19.515 INFO:tasks.workunit.client.0.smithi022.stdout:2/762: rmdir d0/d5d 39 2024-02-16T19:29:19.515 INFO:tasks.workunit.client.1.smithi117.stdout:2/851: rename d0/d3/d10/f16 to d0/d3/d34/d94/d46/f10d 0 2024-02-16T19:29:19.516 INFO:tasks.workunit.client.0.smithi022.stdout:2/763: readlink d0/d2/d18/d38/ld6 0 2024-02-16T19:29:19.517 INFO:tasks.workunit.client.0.smithi022.stdout:3/724: getdents d7/d12/d18/dd/d24/d93 0 2024-02-16T19:29:19.519 INFO:tasks.workunit.client.1.smithi117.stdout:4/830: dwrite d0/f47 [0,4194304] 0 2024-02-16T19:29:19.522 INFO:tasks.workunit.client.1.smithi117.stdout:0/856: dwrite d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/db7/fe4 [0,4194304] 0 2024-02-16T19:29:19.522 INFO:tasks.workunit.client.1.smithi117.stdout:4/831: chown d0/da/d21/d3d/d28/d66/d97/la0 0 1 2024-02-16T19:29:19.524 INFO:tasks.workunit.client.1.smithi117.stdout:4/832: chown d0/da/d21/d3d/d28/d66/f7f 311139 1 2024-02-16T19:29:19.529 INFO:tasks.workunit.client.0.smithi022.stdout:3/725: link d7/d12/d18/dd/fcb d7/d12/d18/dd/d3c/d5a/d67/ffe 0 2024-02-16T19:29:19.533 INFO:tasks.workunit.client.1.smithi117.stdout:1/748: link d2/d19/d59/d76/d8c/dc7/dd2/lfa d2/d4/l101 0 2024-02-16T19:29:19.536 INFO:tasks.workunit.client.0.smithi022.stdout:3/726: unlink d7/d12/d18/d4b/c5c 0 2024-02-16T19:29:19.538 INFO:tasks.workunit.client.1.smithi117.stdout:4/833: creat d0/da/d21/d3d/d28/d64/d73/d91/d9e/f112 x:0 0 0 2024-02-16T19:29:19.544 INFO:tasks.workunit.client.1.smithi117.stdout:1/749: mknod d2/d19/d59/d89/d69/d80/d8e/c102 0 2024-02-16T19:29:19.544 INFO:tasks.workunit.client.1.smithi117.stdout:2/852: getdents d0/d3/d34/d94/d46/d68/d76/da1/d56/d60/dc9 0 2024-02-16T19:29:19.545 INFO:tasks.workunit.client.1.smithi117.stdout:2/853: read d0/d3/d34/d94/d46/d68/d76/da1/d56/fd3 [98952,124151] 0 2024-02-16T19:29:19.546 INFO:tasks.workunit.client.1.smithi117.stdout:2/854: fsync d0/d3/d34/d94/d46/f83 0 2024-02-16T19:29:19.546 INFO:tasks.workunit.client.1.smithi117.stdout:4/834: mkdir d0/da/d21/d3d/d28/d64/d8a/d92/de8/d113 0 2024-02-16T19:29:19.547 INFO:tasks.workunit.client.0.smithi022.stdout:9/746: dwrite db/d14/d33/f88 [4194304,4194304] 0 2024-02-16T19:29:19.548 INFO:tasks.workunit.client.1.smithi117.stdout:4/835: chown d0/da/d21/d3d/d28/d64/d73/d91/d9c 7100 1 2024-02-16T19:29:19.548 INFO:tasks.workunit.client.0.smithi022.stdout:9/747: chown db/f79 6608 1 2024-02-16T19:29:19.550 INFO:tasks.workunit.client.0.smithi022.stdout:9/748: dread - db/d14/d16/d27/d55/f99 zero size 2024-02-16T19:29:19.552 INFO:tasks.workunit.client.0.smithi022.stdout:4/704: dwrite d4/d6/dd/d22/d30/f44 [0,4194304] 0 2024-02-16T19:29:19.559 INFO:tasks.workunit.client.0.smithi022.stdout:9/749: rename db/d14/d33 to db/d14/dc8/dc9/d105 0 2024-02-16T19:29:19.561 INFO:tasks.workunit.client.1.smithi117.stdout:2/855: creat d0/d3/d34/d94/d46/d68/d76/da1/d2f/f10e x:0 0 0 2024-02-16T19:29:19.562 INFO:tasks.workunit.client.0.smithi022.stdout:4/705: truncate d4/d6/dd/df/d10/f19 4394401 0 2024-02-16T19:29:19.568 INFO:tasks.workunit.client.1.smithi117.stdout:6/813: rename d5/d4b/d44/d93/dae/dc3 to d5/d4b/d71/d10f 0 2024-02-16T19:29:19.572 INFO:tasks.workunit.client.0.smithi022.stdout:1/773: rmdir d6/d8/d22/d67 39 2024-02-16T19:29:19.574 INFO:tasks.workunit.client.1.smithi117.stdout:1/750: getdents d2/d1d 0 2024-02-16T19:29:19.575 INFO:tasks.workunit.client.1.smithi117.stdout:2/856: dread d0/f4c [0,4194304] 0 2024-02-16T19:29:19.579 INFO:tasks.workunit.client.0.smithi022.stdout:1/774: symlink d6/d8/d22/d67/d7e/lf1 0 2024-02-16T19:29:19.586 INFO:tasks.workunit.client.0.smithi022.stdout:1/775: dread d6/f6b [0,4194304] 0 2024-02-16T19:29:19.587 INFO:tasks.workunit.client.0.smithi022.stdout:1/776: chown d6/d8/d22/d3a/d3e/lcc 2708396 1 2024-02-16T19:29:19.589 INFO:tasks.workunit.client.0.smithi022.stdout:1/777: symlink d6/d8/d22/d3a/d7a/lf2 0 2024-02-16T19:29:19.591 INFO:tasks.workunit.client.0.smithi022.stdout:1/778: mkdir d6/d8/d22/d16/d17/d5d/df3 0 2024-02-16T19:29:19.593 INFO:tasks.workunit.client.0.smithi022.stdout:1/779: mknod d6/d8/d22/d3f/dc5/cf4 0 2024-02-16T19:29:19.595 INFO:tasks.workunit.client.0.smithi022.stdout:1/780: mkdir d6/d8/d6c/df5 0 2024-02-16T19:29:19.598 INFO:tasks.workunit.client.0.smithi022.stdout:1/781: fdatasync d6/d8/d22/d3f/d99/fb0 0 2024-02-16T19:29:19.599 INFO:tasks.workunit.client.0.smithi022.stdout:1/782: write d6/d8/d22/d16/f2e [737318,31983] 0 2024-02-16T19:29:19.604 INFO:tasks.workunit.client.0.smithi022.stdout:1/783: rename d6/d8/d22/d16/f2e to d6/d8/d22/d3a/d3e/ff6 0 2024-02-16T19:29:19.607 INFO:tasks.workunit.client.0.smithi022.stdout:1/784: fsync d6/d8/d22/d16/d17/d1d/f21 0 2024-02-16T19:29:19.613 INFO:tasks.workunit.client.0.smithi022.stdout:1/785: getdents d6/d8/d6c/d96 0 2024-02-16T19:29:19.617 INFO:tasks.workunit.client.0.smithi022.stdout:1/786: creat d6/d8/d22/d67/ff7 x:0 0 0 2024-02-16T19:29:19.619 INFO:tasks.workunit.client.0.smithi022.stdout:5/842: write dd/d1f/d48/d21/f24 [2090254,81151] 0 2024-02-16T19:29:19.620 INFO:tasks.workunit.client.0.smithi022.stdout:6/776: dwrite d1/f32 [0,4194304] 0 2024-02-16T19:29:19.622 INFO:tasks.workunit.client.0.smithi022.stdout:1/787: chown d6/d8/d22/d47/d51/cb9 0 1 2024-02-16T19:29:19.623 INFO:tasks.workunit.client.0.smithi022.stdout:7/802: dwrite d9/da/f60 [4194304,4194304] 0 2024-02-16T19:29:19.624 INFO:tasks.workunit.client.0.smithi022.stdout:0/964: write d10/d22/d2c/ff9 [423655,5862] 0 2024-02-16T19:29:19.624 INFO:tasks.workunit.client.1.smithi117.stdout:9/746: rename dd/d47/d5b/d5f/d77/fcb to dd/d47/d5e/ffd 0 2024-02-16T19:29:19.626 INFO:tasks.workunit.client.1.smithi117.stdout:3/924: truncate d1/d30/d9f/dcb/d103/d134/fcd 1938205 0 2024-02-16T19:29:19.629 INFO:tasks.workunit.client.1.smithi117.stdout:1/751: rename d2/d19/fc5 to d2/d1d/d77/f103 0 2024-02-16T19:29:19.630 INFO:tasks.workunit.client.0.smithi022.stdout:1/788: mknod d6/d8/d6c/cf8 0 2024-02-16T19:29:19.631 INFO:tasks.workunit.client.0.smithi022.stdout:8/857: dwrite d3/d25/d33/f35 [0,4194304] 0 2024-02-16T19:29:19.634 INFO:tasks.workunit.client.0.smithi022.stdout:6/777: fdatasync d1/d8/da/d29/deb/dfd/d85/f36 0 2024-02-16T19:29:19.636 INFO:tasks.workunit.client.0.smithi022.stdout:6/778: readlink d1/d8/da/d29/deb/dfd/d53/l9f 0 2024-02-16T19:29:19.636 INFO:tasks.workunit.client.1.smithi117.stdout:5/873: dwrite d2/d9/d3c/f98 [0,4194304] 0 2024-02-16T19:29:19.637 INFO:tasks.workunit.client.0.smithi022.stdout:7/803: creat d9/da/d2c/d46/f10d x:0 0 0 2024-02-16T19:29:19.637 INFO:tasks.workunit.client.0.smithi022.stdout:6/779: readlink d1/d8/da/d29/d61/l6d 0 2024-02-16T19:29:19.638 INFO:tasks.workunit.client.0.smithi022.stdout:7/804: chown d9/da/d1f/f81 134936 1 2024-02-16T19:29:19.638 INFO:tasks.workunit.client.1.smithi117.stdout:2/857: dwrite d0/d3/d34/d94/d46/d68/d76/da1/fee [0,4194304] 0 2024-02-16T19:29:19.649 INFO:tasks.workunit.client.1.smithi117.stdout:0/857: dwrite d0/d9/d14/d1d/d60/d65/fab [4194304,4194304] 0 2024-02-16T19:29:19.652 INFO:tasks.workunit.client.0.smithi022.stdout:3/727: dwrite d7/d12/d18/f88 [0,4194304] 0 2024-02-16T19:29:19.654 INFO:tasks.workunit.client.1.smithi117.stdout:1/752: dread - d2/d4/dbb/d7f/f84 zero size 2024-02-16T19:29:19.655 INFO:tasks.workunit.client.0.smithi022.stdout:8/858: fsync d3/d12/d2b/fa5 0 2024-02-16T19:29:19.656 INFO:tasks.workunit.client.0.smithi022.stdout:1/789: truncate d6/d8/d22/d16/d4f/f8c 3771342 0 2024-02-16T19:29:19.657 INFO:tasks.workunit.client.1.smithi117.stdout:1/753: chown d2/d1d/d60/d68/f81 1231063 1 2024-02-16T19:29:19.659 INFO:tasks.workunit.client.1.smithi117.stdout:0/858: dwrite d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/dd4/fdc [4194304,4194304] 0 2024-02-16T19:29:19.662 INFO:tasks.workunit.client.1.smithi117.stdout:2/858: mkdir d0/d13/dc4/d10f 0 2024-02-16T19:29:19.664 INFO:tasks.workunit.client.0.smithi022.stdout:7/805: mknod d9/d5d/dc7/c10e 0 2024-02-16T19:29:19.666 INFO:tasks.workunit.client.1.smithi117.stdout:3/925: creat d1/dd/d3b/dba/d116/f13e x:0 0 0 2024-02-16T19:29:19.670 INFO:tasks.workunit.client.1.smithi117.stdout:2/859: dwrite d0/d3b/fca [4194304,4194304] 0 2024-02-16T19:29:19.672 INFO:tasks.workunit.client.1.smithi117.stdout:2/860: chown d0/d3/d22/d37/d9d/fe8 607959517 1 2024-02-16T19:29:19.674 INFO:tasks.workunit.client.0.smithi022.stdout:9/750: dwrite db/d18/f6d [0,4194304] 0 2024-02-16T19:29:19.678 INFO:tasks.workunit.client.1.smithi117.stdout:6/814: dwrite d5/fa7 [0,4194304] 0 2024-02-16T19:29:19.680 INFO:tasks.workunit.client.1.smithi117.stdout:6/815: chown d5/db/d7d/d32/da8/de1/ca0 160741 1 2024-02-16T19:29:19.680 INFO:tasks.workunit.client.0.smithi022.stdout:5/843: link dd/d1f/d48/d21/d5d/de7/fc6 dd/d1f/d48/d21/d64/f116 0 2024-02-16T19:29:19.681 INFO:tasks.workunit.client.0.smithi022.stdout:4/706: dwrite d4/d6/dd/df/f91 [0,4194304] 0 2024-02-16T19:29:19.682 INFO:tasks.workunit.client.1.smithi117.stdout:1/754: mknod d2/d1d/d1f/c104 0 2024-02-16T19:29:19.684 INFO:tasks.workunit.client.0.smithi022.stdout:4/707: dread - d4/d6/dd/d22/d83/dc5/fdb zero size 2024-02-16T19:29:19.686 INFO:tasks.workunit.client.0.smithi022.stdout:3/728: creat d7/d12/d18/d4b/fff x:0 0 0 2024-02-16T19:29:19.686 INFO:tasks.workunit.client.0.smithi022.stdout:4/708: readlink d4/d6/dd/d22/da8/dc4/l76 0 2024-02-16T19:29:19.688 INFO:tasks.workunit.client.0.smithi022.stdout:4/709: write d4/d6/dd/df/d10/f50 [1140553,68480] 0 2024-02-16T19:29:19.689 INFO:tasks.workunit.client.1.smithi117.stdout:6/816: dwrite d5/db/f6f [0,4194304] 0 2024-02-16T19:29:19.689 INFO:tasks.workunit.client.0.smithi022.stdout:3/729: stat d7/d12/d18/dd/d24/d58/d70/cb3 0 2024-02-16T19:29:19.690 INFO:tasks.workunit.client.1.smithi117.stdout:6/817: stat d5/d4b/d71/dc7/d72/dec/f102 0 2024-02-16T19:29:19.694 INFO:tasks.workunit.client.1.smithi117.stdout:0/859: rmdir d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b 39 2024-02-16T19:29:19.700 INFO:tasks.workunit.client.1.smithi117.stdout:3/926: creat d1/dd/d3e/d7d/d121/f13f x:0 0 0 2024-02-16T19:29:19.702 INFO:tasks.workunit.client.0.smithi022.stdout:4/710: dwrite d4/d6/dd/df/dc1/fd5 [0,4194304] 0 2024-02-16T19:29:19.706 INFO:tasks.workunit.client.1.smithi117.stdout:8/880: sync 2024-02-16T19:29:19.706 INFO:tasks.workunit.client.1.smithi117.stdout:7/827: sync 2024-02-16T19:29:19.708 INFO:tasks.workunit.client.1.smithi117.stdout:8/881: write d4/d12/d2a/d64/f7d [3985782,103395] 0 2024-02-16T19:29:19.708 INFO:tasks.workunit.client.0.smithi022.stdout:1/790: dwrite d6/d8/d22/d47/d51/fd5 [0,4194304] 0 2024-02-16T19:29:19.713 INFO:tasks.workunit.client.1.smithi117.stdout:1/755: creat d2/d19/d59/d89/d49/f105 x:0 0 0 2024-02-16T19:29:19.719 INFO:tasks.workunit.client.0.smithi022.stdout:2/764: sync 2024-02-16T19:29:19.720 INFO:tasks.workunit.client.0.smithi022.stdout:5/844: fdatasync dd/d1f/f31 0 2024-02-16T19:29:19.724 INFO:tasks.workunit.client.0.smithi022.stdout:3/730: unlink d7/d12/d18/dd/d31/dbf/ld6 0 2024-02-16T19:29:19.725 INFO:tasks.workunit.client.1.smithi117.stdout:3/927: creat d1/d83/d96/f140 x:0 0 0 2024-02-16T19:29:19.727 INFO:tasks.workunit.client.0.smithi022.stdout:3/731: stat d7/d12/d18/dd/d24/d58/d70 0 2024-02-16T19:29:19.727 INFO:tasks.workunit.client.0.smithi022.stdout:8/859: dread d3/d12/d2b/da9/fe4 [0,4194304] 0 2024-02-16T19:29:19.728 INFO:tasks.workunit.client.0.smithi022.stdout:2/765: dwrite d0/d2/d2e/fd2 [0,4194304] 0 2024-02-16T19:29:19.732 INFO:tasks.workunit.client.0.smithi022.stdout:8/860: write d3/d12/d2b/d54/d74/da2/f105 [293365,23818] 0 2024-02-16T19:29:19.733 INFO:tasks.workunit.client.1.smithi117.stdout:2/861: dwrite d0/d13/d4d/d53/d58/faf [0,4194304] 0 2024-02-16T19:29:19.735 INFO:tasks.workunit.client.1.smithi117.stdout:7/828: fdatasync d4/d1c/d39/d46/d55/fcc 0 2024-02-16T19:29:19.736 INFO:tasks.workunit.client.1.smithi117.stdout:2/862: write d0/d3/d34/d94/d46/d68/d93/ddc/feb [680073,82353] 0 2024-02-16T19:29:19.738 INFO:tasks.workunit.client.0.smithi022.stdout:0/965: getdents d10/d15/d23/d8f 0 2024-02-16T19:29:19.740 INFO:tasks.workunit.client.0.smithi022.stdout:8/861: dread d3/d12/d6a/fa4 [0,4194304] 0 2024-02-16T19:29:19.740 INFO:tasks.workunit.client.1.smithi117.stdout:5/874: dwrite d2/d4c/d4f/fd0 [0,4194304] 0 2024-02-16T19:29:19.741 INFO:tasks.workunit.client.0.smithi022.stdout:0/966: read - d10/d15/d23/d38/d5a/f131 zero size 2024-02-16T19:29:19.742 INFO:tasks.workunit.client.1.smithi117.stdout:8/882: rename d4/f6 to d4/d7/da6/d50/f11c 0 2024-02-16T19:29:19.748 INFO:tasks.workunit.client.0.smithi022.stdout:4/711: dread - d4/d6/dd/d22/d30/fad zero size 2024-02-16T19:29:19.749 INFO:tasks.workunit.client.1.smithi117.stdout:9/747: truncate dd/d45/d73/ff1 330667 0 2024-02-16T19:29:19.753 INFO:tasks.workunit.client.0.smithi022.stdout:9/751: mknod db/d18/d103/c106 0 2024-02-16T19:29:19.757 INFO:tasks.workunit.client.1.smithi117.stdout:2/863: mkdir d0/d13/d4d/d80/d110 0 2024-02-16T19:29:19.760 INFO:tasks.workunit.client.1.smithi117.stdout:3/928: dwrite d1/dd/d3e/d7d/fbf [0,4194304] 0 2024-02-16T19:29:19.767 INFO:tasks.workunit.client.0.smithi022.stdout:2/766: symlink d0/d2/d2e/d73/de3/l101 0 2024-02-16T19:29:19.768 INFO:tasks.workunit.client.0.smithi022.stdout:2/767: fsync d0/d2/d18/d19/f2f 0 2024-02-16T19:29:19.773 INFO:tasks.workunit.client.0.smithi022.stdout:6/780: rename d1/d8/da/d29/d61/l6d to d1/d8/d23/dc6/l10d 0 2024-02-16T19:29:19.774 INFO:tasks.workunit.client.1.smithi117.stdout:9/748: read dd/d47/d5b/d5f/f6f [2965984,17737] 0 2024-02-16T19:29:19.776 INFO:tasks.workunit.client.0.smithi022.stdout:0/967: mknod d10/d11/d79/c140 0 2024-02-16T19:29:19.777 INFO:tasks.workunit.client.0.smithi022.stdout:7/806: write d9/da/d1f/fcd [925987,26962] 0 2024-02-16T19:29:19.778 INFO:tasks.workunit.client.0.smithi022.stdout:0/968: chown d10/d15/d23/f3c 583 1 2024-02-16T19:29:19.779 INFO:tasks.workunit.client.0.smithi022.stdout:8/862: mknod d3/d12/d17/d64/d8f/c127 0 2024-02-16T19:29:19.783 INFO:tasks.workunit.client.1.smithi117.stdout:3/929: write d1/d1b/d1e/d37/ffc [758471,116578] 0 2024-02-16T19:29:19.783 INFO:tasks.workunit.client.0.smithi022.stdout:1/791: write d6/d8/d22/f1b [1427768,31305] 0 2024-02-16T19:29:19.788 INFO:tasks.workunit.client.1.smithi117.stdout:4/836: sync 2024-02-16T19:29:19.788 INFO:tasks.workunit.client.0.smithi022.stdout:0/969: dwrite d10/d11/d27/d77/f13d [0,4194304] 0 2024-02-16T19:29:19.788 INFO:tasks.workunit.client.0.smithi022.stdout:8/863: dwrite d3/d12/da6/f1f [4194304,4194304] 0 2024-02-16T19:29:19.790 INFO:tasks.workunit.client.1.smithi117.stdout:1/756: rename d2/d1d/c22 to d2/d1d/d60/d7c/d9e/da5/db0/dba/c106 0 2024-02-16T19:29:19.794 INFO:tasks.workunit.client.1.smithi117.stdout:6/818: link d5/d4b/d71/dc7/dce/cf1 d5/d4b/d44/d52/c110 0 2024-02-16T19:29:19.796 INFO:tasks.workunit.client.1.smithi117.stdout:4/837: dwrite d0/da/d21/d3d/d28/d66/da2/f10d [0,4194304] 0 2024-02-16T19:29:19.797 INFO:tasks.workunit.client.1.smithi117.stdout:0/860: getdents d0/d9/d14/d1d/d1e/d54 0 2024-02-16T19:29:19.799 INFO:tasks.workunit.client.0.smithi022.stdout:5/845: dwrite dd/d1f/d48/d21/d2f/d52/d77/d95/fb8 [0,4194304] 0 2024-02-16T19:29:19.800 INFO:tasks.workunit.client.1.smithi117.stdout:0/861: write d0/d9/d14/d1d/d60/dc3/fcc [4533649,111450] 0 2024-02-16T19:29:19.802 INFO:tasks.workunit.client.0.smithi022.stdout:3/732: dwrite d7/d21/f38 [0,4194304] 0 2024-02-16T19:29:19.804 INFO:tasks.workunit.client.0.smithi022.stdout:6/781: mkdir d1/d8/da/d29/d10e 0 2024-02-16T19:29:19.805 INFO:tasks.workunit.client.0.smithi022.stdout:3/733: write d7/d21/f5d [328119,9910] 0 2024-02-16T19:29:19.805 INFO:tasks.workunit.client.0.smithi022.stdout:5/846: write dd/d1f/d48/d21/d2f/d52/d5c/d61/f92 [82951,47464] 0 2024-02-16T19:29:19.812 INFO:tasks.workunit.client.1.smithi117.stdout:7/829: write d4/d1c/d39/d46/d47/f56 [830446,31007] 0 2024-02-16T19:29:19.812 INFO:tasks.workunit.client.0.smithi022.stdout:7/807: mkdir d9/da9/dbc/d10f 0 2024-02-16T19:29:19.814 INFO:tasks.workunit.client.1.smithi117.stdout:3/930: creat d1/d30/d9f/dbc/d12f/f141 x:0 0 0 2024-02-16T19:29:19.814 INFO:tasks.workunit.client.0.smithi022.stdout:7/808: write d9/da/d1f/d67/db3/ff0 [1238207,50200] 0 2024-02-16T19:29:19.815 INFO:tasks.workunit.client.1.smithi117.stdout:3/931: write d1/d1b/d1e/d5e/f122 [138820,61047] 0 2024-02-16T19:29:19.817 INFO:tasks.workunit.client.1.smithi117.stdout:5/875: write d2/db3/f55 [3267327,5356] 0 2024-02-16T19:29:19.821 INFO:tasks.workunit.client.0.smithi022.stdout:4/712: mkdir d4/d6/d14/d64/de0 0 2024-02-16T19:29:19.824 INFO:tasks.workunit.client.1.smithi117.stdout:8/883: dwrite d4/d12/d2a/f48 [0,4194304] 0 2024-02-16T19:29:19.824 INFO:tasks.workunit.client.0.smithi022.stdout:9/752: creat db/d14/d19/d1a/de5/f107 x:0 0 0 2024-02-16T19:29:19.825 INFO:tasks.workunit.client.1.smithi117.stdout:8/884: read - d4/d7/d6a/d72/d83/ff2 zero size 2024-02-16T19:29:19.825 INFO:tasks.workunit.client.0.smithi022.stdout:1/792: mkdir d6/d8/d22/d16/d17/d1d/df9 0 2024-02-16T19:29:19.826 INFO:tasks.workunit.client.1.smithi117.stdout:8/885: chown d4/d7/da6/d50 3687 1 2024-02-16T19:29:19.827 INFO:tasks.workunit.client.0.smithi022.stdout:2/768: dwrite d0/d2/d2e/f2a [0,4194304] 0 2024-02-16T19:29:19.829 INFO:tasks.workunit.client.0.smithi022.stdout:8/864: unlink d3/d25/d33/fab 0 2024-02-16T19:29:19.831 INFO:tasks.workunit.client.1.smithi117.stdout:6/819: mknod d5/db/d7d/d32/da8/c111 0 2024-02-16T19:29:19.832 INFO:tasks.workunit.client.1.smithi117.stdout:4/838: creat d0/da/d21/d3d/d28/d64/d90/f114 x:0 0 0 2024-02-16T19:29:19.835 INFO:tasks.workunit.client.1.smithi117.stdout:9/749: mknod dd/d45/cfe 0 2024-02-16T19:29:19.836 INFO:tasks.workunit.client.1.smithi117.stdout:6/820: rename d5/d4b/d71/dc7 to d5/d4b/d71/dc7/dce/ded/df3/d112 22 2024-02-16T19:29:19.837 INFO:tasks.workunit.client.1.smithi117.stdout:4/839: read d0/da/d21/d3d/d28/d64/d90/dc0/dd7/fde [1463597,30159] 0 2024-02-16T19:29:19.840 INFO:tasks.workunit.client.1.smithi117.stdout:8/886: dwrite d4/d7/da6/d3a/d65/d7c/f107 [0,4194304] 0 2024-02-16T19:29:19.840 INFO:tasks.workunit.client.0.smithi022.stdout:6/782: rename d1/d8/da/d29/deb/dfd/d8b/c93 to d1/d8/da/d29/d10e/c10f 0 2024-02-16T19:29:19.842 INFO:tasks.workunit.client.0.smithi022.stdout:3/734: mknod d7/d12/d18/dd/d31/dbf/dc5/c100 0 2024-02-16T19:29:19.843 INFO:tasks.workunit.client.1.smithi117.stdout:0/862: mknod d0/d9/d14/dec/c11d 0 2024-02-16T19:29:19.850 INFO:tasks.workunit.client.0.smithi022.stdout:4/713: dread - d4/d6/dd/d22/da8/dc4/d6d/f7d zero size 2024-02-16T19:29:19.852 INFO:tasks.workunit.client.1.smithi117.stdout:1/757: dwrite d2/d19/d59/d76/d8c/fca [0,4194304] 0 2024-02-16T19:29:19.853 INFO:tasks.workunit.client.0.smithi022.stdout:9/753: creat db/d14/d16/d87/f108 x:0 0 0 2024-02-16T19:29:19.853 INFO:tasks.workunit.client.0.smithi022.stdout:0/970: write d10/d22/dad/fc2 [379134,73404] 0 2024-02-16T19:29:19.855 INFO:tasks.workunit.client.0.smithi022.stdout:2/769: dread d0/d32/f58 [0,4194304] 0 2024-02-16T19:29:19.861 INFO:tasks.workunit.client.0.smithi022.stdout:1/793: creat d6/d8/d22/d47/d51/ffa x:0 0 0 2024-02-16T19:29:19.864 INFO:tasks.workunit.client.1.smithi117.stdout:7/830: dwrite d4/f1a [0,4194304] 0 2024-02-16T19:29:19.867 INFO:tasks.workunit.client.0.smithi022.stdout:1/794: dread d6/d8/d22/d67/f77 [0,4194304] 0 2024-02-16T19:29:19.976 INFO:tasks.workunit.client.0.smithi022.stdout:8/865: unlink d3/d25/ld6 0 2024-02-16T19:29:19.980 INFO:tasks.workunit.client.0.smithi022.stdout:6/783: creat d1/d8/da/d29/f110 x:0 0 0 2024-02-16T19:29:19.981 INFO:tasks.workunit.client.0.smithi022.stdout:6/784: fdatasync d1/d8/da/d29/deb/dfd/d48/d51/f59 0 2024-02-16T19:29:19.981 INFO:tasks.workunit.client.0.smithi022.stdout:3/735: rmdir d7/d21 39 2024-02-16T19:29:19.983 INFO:tasks.workunit.client.0.smithi022.stdout:7/809: mknod d9/da/d1f/d44/d5b/d95/d10a/c110 0 2024-02-16T19:29:19.985 INFO:tasks.workunit.client.0.smithi022.stdout:0/971: creat d10/d15/d23/d8f/dc1/f141 x:0 0 0 2024-02-16T19:29:19.987 INFO:tasks.workunit.client.0.smithi022.stdout:9/754: fdatasync db/d14/d16/d21/d4a/fa4 0 2024-02-16T19:29:19.989 INFO:tasks.workunit.client.0.smithi022.stdout:9/755: dread - db/d14/d16/d5d/d65/d6b/d89/dd2/ff0 zero size 2024-02-16T19:29:19.995 INFO:tasks.workunit.client.0.smithi022.stdout:1/795: mknod d6/d8/d22/d3f/dc5/cfb 0 2024-02-16T19:29:19.997 INFO:tasks.workunit.client.0.smithi022.stdout:8/866: rmdir d3/d12/db0/d11d 39 2024-02-16T19:29:19.997 INFO:tasks.workunit.client.0.smithi022.stdout:2/770: write d0/d2/d18/d1f/f21 [2409206,85735] 0 2024-02-16T19:29:20.000 INFO:tasks.workunit.client.0.smithi022.stdout:5/847: creat dd/d1f/d48/f117 x:0 0 0 2024-02-16T19:29:20.003 INFO:tasks.workunit.client.0.smithi022.stdout:8/867: dwrite d3/d12/d2b/fde [0,4194304] 0 2024-02-16T19:29:20.004 INFO:tasks.workunit.client.0.smithi022.stdout:6/785: fsync d1/d8/da/d29/deb/dfd/d85/d9c/fce 0 2024-02-16T19:29:20.014 INFO:tasks.workunit.client.0.smithi022.stdout:2/771: dread - d0/d5d/fd5 zero size 2024-02-16T19:29:20.019 INFO:tasks.workunit.client.0.smithi022.stdout:5/848: creat dd/d1f/d48/d21/d5d/f118 x:0 0 0 2024-02-16T19:29:20.026 INFO:tasks.workunit.client.0.smithi022.stdout:6/786: fdatasync d1/d8/d3c/d4d/f4b 0 2024-02-16T19:29:20.028 INFO:tasks.workunit.client.0.smithi022.stdout:9/756: dwrite db/f7b [0,4194304] 0 2024-02-16T19:29:20.031 INFO:tasks.workunit.client.0.smithi022.stdout:0/972: mkdir d10/d7d/dac/de9/dfc/d142 0 2024-02-16T19:29:20.034 INFO:tasks.workunit.client.1.smithi117.stdout:9/750: unlink dd/d47/d5b/d5f/d77/d7c/cb0 0 2024-02-16T19:29:20.040 INFO:tasks.workunit.client.1.smithi117.stdout:2/864: getdents d0/d3/d22 0 2024-02-16T19:29:20.047 INFO:tasks.workunit.client.0.smithi022.stdout:7/810: rename d9/da/d3b/ca4 to d9/da/d4e/c111 0 2024-02-16T19:29:20.051 INFO:tasks.workunit.client.1.smithi117.stdout:1/758: mkdir d2/d1d/d60/d7c/d9e/d107 0 2024-02-16T19:29:20.052 INFO:tasks.workunit.client.0.smithi022.stdout:9/757: fdatasync db/d14/d16/d27/d55/f99 0 2024-02-16T19:29:20.053 INFO:tasks.workunit.client.1.smithi117.stdout:5/876: write d2/d9/d3c/f1a [757812,122650] 0 2024-02-16T19:29:20.054 INFO:tasks.workunit.client.1.smithi117.stdout:7/831: fdatasync d4/d1c/d39/d46/d55/db8/fc5 0 2024-02-16T19:29:20.055 INFO:tasks.workunit.client.1.smithi117.stdout:8/887: dread d4/d12/f42 [0,4194304] 0 2024-02-16T19:29:20.056 INFO:tasks.workunit.client.0.smithi022.stdout:0/973: mknod d10/d15/d23/d38/d5a/d66/c143 0 2024-02-16T19:29:20.057 INFO:tasks.workunit.client.1.smithi117.stdout:8/888: write d4/d7/d6a/f93 [1264412,47508] 0 2024-02-16T19:29:20.059 INFO:tasks.workunit.client.1.smithi117.stdout:9/751: symlink dd/d45/d31/d63/dba/df0/daf/df4/dd7/lff 0 2024-02-16T19:29:20.060 INFO:tasks.workunit.client.0.smithi022.stdout:3/736: truncate d7/d12/d18/dd/d3c/fe2 259806 0 2024-02-16T19:29:20.061 INFO:tasks.workunit.client.1.smithi117.stdout:4/840: mknod d0/da/c115 0 2024-02-16T19:29:20.062 INFO:tasks.workunit.client.0.smithi022.stdout:8/868: creat d3/d12/d2b/d54/d74/f128 x:0 0 0 2024-02-16T19:29:20.062 INFO:tasks.workunit.client.1.smithi117.stdout:7/832: dwrite d4/d13/dfd/f12b [0,4194304] 0 2024-02-16T19:29:20.065 INFO:tasks.workunit.client.1.smithi117.stdout:0/863: symlink d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/d27/d31/d94/l11e 0 2024-02-16T19:29:20.065 INFO:tasks.workunit.client.1.smithi117.stdout:3/932: link d1/d83/d96/cd3 d1/d1b/d1e/d37/ddc/c142 0 2024-02-16T19:29:20.066 INFO:tasks.workunit.client.1.smithi117.stdout:3/933: truncate d1/d83/d96/f110 4492132 0 2024-02-16T19:29:20.066 INFO:tasks.workunit.client.1.smithi117.stdout:3/934: readlink d1/l10 0 2024-02-16T19:29:20.067 INFO:tasks.workunit.client.1.smithi117.stdout:3/935: chown d1/d31/f7e 250 1 2024-02-16T19:29:20.068 INFO:tasks.workunit.client.1.smithi117.stdout:3/936: readlink d1/d30/d9f/dcb/le4 0 2024-02-16T19:29:20.069 INFO:tasks.workunit.client.0.smithi022.stdout:2/772: link d0/d2/d18/d1f/d6c/df1/fa9 d0/d2/d18/d19/d90/d100/f102 0 2024-02-16T19:29:20.074 INFO:tasks.workunit.client.0.smithi022.stdout:8/869: symlink d3/d12/d2b/d54/d74/da2/dc0/l129 0 2024-02-16T19:29:20.076 INFO:tasks.workunit.client.1.smithi117.stdout:8/889: symlink d4/d12/d2a/d64/dc6/l11d 0 2024-02-16T19:29:20.079 INFO:tasks.workunit.client.0.smithi022.stdout:8/870: dread d3/d12/d2b/d54/f70 [0,4194304] 0 2024-02-16T19:29:20.080 INFO:tasks.workunit.client.0.smithi022.stdout:8/871: chown d3/d12/d2b/d2e/f11e 19 1 2024-02-16T19:29:20.081 INFO:tasks.workunit.client.1.smithi117.stdout:9/752: write dd/d47/d48/dc0/fbf [2739095,94889] 0 2024-02-16T19:29:20.081 INFO:tasks.workunit.client.0.smithi022.stdout:8/872: stat d3/d12/d2b/d54/db8/de5/c112 0 2024-02-16T19:29:20.082 INFO:tasks.workunit.client.0.smithi022.stdout:9/758: write db/d14/f60 [1299483,101332] 0 2024-02-16T19:29:20.083 INFO:tasks.workunit.client.1.smithi117.stdout:1/759: write d2/d19/d59/d89/f64 [1066374,40943] 0 2024-02-16T19:29:20.084 INFO:tasks.workunit.client.1.smithi117.stdout:7/833: fdatasync d4/d1c/d39/d46/d55/fe0 0 2024-02-16T19:29:20.090 INFO:tasks.workunit.client.0.smithi022.stdout:8/873: dwrite d3/d12/db0/fd9 [0,4194304] 0 2024-02-16T19:29:20.091 INFO:tasks.workunit.client.0.smithi022.stdout:3/737: dwrite d7/d12/d18/dd/d3c/f49 [0,4194304] 0 2024-02-16T19:29:20.094 INFO:tasks.workunit.client.1.smithi117.stdout:4/841: write d0/da/d21/d3d/d28/d64/d90/dc0/fc9 [957200,123533] 0 2024-02-16T19:29:20.097 INFO:tasks.workunit.client.0.smithi022.stdout:1/796: rename d6/d8/d22/d47/d51/fa2 to d6/d8/d22/d3f/d99/ffc 0 2024-02-16T19:29:20.099 INFO:tasks.workunit.client.0.smithi022.stdout:6/787: link d1/d8/da/d29/d10e/c10f d1/d8/da/d29/c111 0 2024-02-16T19:29:20.100 INFO:tasks.workunit.client.0.smithi022.stdout:6/788: dread - d1/d8/da/d29/deb/dfd/d85/d9c/ff6 zero size 2024-02-16T19:29:20.100 INFO:tasks.workunit.client.1.smithi117.stdout:3/937: creat d1/d1b/d1e/d37/f143 x:0 0 0 2024-02-16T19:29:20.102 INFO:tasks.workunit.client.1.smithi117.stdout:5/877: symlink d2/d4c/d69/dc2/l124 0 2024-02-16T19:29:20.103 INFO:tasks.workunit.client.0.smithi022.stdout:9/759: mknod db/d14/d16/d28/c109 0 2024-02-16T19:29:20.107 INFO:tasks.workunit.client.1.smithi117.stdout:6/821: link d5/d4b/d44/d52/c110 d5/d4b/d71/dc7/d49/d59/c113 0 2024-02-16T19:29:20.108 INFO:tasks.workunit.client.1.smithi117.stdout:6/822: dread d5/d4b/d20/f3a [0,4194304] 0 2024-02-16T19:29:20.109 INFO:tasks.workunit.client.0.smithi022.stdout:2/773: symlink d0/d2/d2e/l103 0 2024-02-16T19:29:20.110 INFO:tasks.workunit.client.1.smithi117.stdout:0/864: dread d0/d9/d14/d23/d33/fb3 [0,4194304] 0 2024-02-16T19:29:20.112 INFO:tasks.workunit.client.1.smithi117.stdout:7/834: symlink d4/d1c/d39/d46/dc4/l131 0 2024-02-16T19:29:20.113 INFO:tasks.workunit.client.0.smithi022.stdout:1/797: rmdir d6 39 2024-02-16T19:29:20.114 INFO:tasks.workunit.client.1.smithi117.stdout:2/865: rename d0/d13/d4d/da5 to d0/d3/d34/d94/d46/d68/d111 0 2024-02-16T19:29:20.115 INFO:tasks.workunit.client.1.smithi117.stdout:2/866: chown d0/d3/c39 7932734 1 2024-02-16T19:29:20.116 INFO:tasks.workunit.client.1.smithi117.stdout:4/842: mkdir d0/d116 0 2024-02-16T19:29:20.118 INFO:tasks.workunit.client.1.smithi117.stdout:4/843: fsync d0/da/d21/d3d/d28/d64/d73/d91/d37/f107 0 2024-02-16T19:29:20.119 INFO:tasks.workunit.client.1.smithi117.stdout:3/938: rmdir d1/dd/d3e/d7d/d121 39 2024-02-16T19:29:20.119 INFO:tasks.workunit.client.0.smithi022.stdout:0/974: link d10/d15/d23/d38/d5a/d66/lf5 d10/d11/d79/d8b/l144 0 2024-02-16T19:29:20.123 INFO:tasks.workunit.client.1.smithi117.stdout:8/890: mkdir d4/d94/d109/d11e 0 2024-02-16T19:29:20.127 INFO:tasks.workunit.client.1.smithi117.stdout:6/823: readlink d5/db/l45 0 2024-02-16T19:29:20.127 INFO:tasks.workunit.client.1.smithi117.stdout:9/753: write dd/d47/d48/f50 [4835016,51375] 0 2024-02-16T19:29:20.127 INFO:tasks.workunit.client.1.smithi117.stdout:9/754: stat dd/d47/d5b/d5f/f9b 0 2024-02-16T19:29:20.127 INFO:tasks.workunit.client.1.smithi117.stdout:9/755: dread - dd/d47/d5b/d5f/fc5 zero size 2024-02-16T19:29:20.129 INFO:tasks.workunit.client.1.smithi117.stdout:9/756: write dd/d45/d31/d63/dba/df0/daf/fe1 [529485,12409] 0 2024-02-16T19:29:20.129 INFO:tasks.workunit.client.0.smithi022.stdout:9/760: mkdir db/d14/d10a 0 2024-02-16T19:29:20.129 INFO:tasks.workunit.client.0.smithi022.stdout:9/761: chown db 2 1 2024-02-16T19:29:20.131 INFO:tasks.workunit.client.0.smithi022.stdout:0/975: unlink d10/d15/d23/d8f/dc1/l3b 0 2024-02-16T19:29:20.133 INFO:tasks.workunit.client.1.smithi117.stdout:7/835: rmdir d4/d13/da3 39 2024-02-16T19:29:20.134 INFO:tasks.workunit.client.0.smithi022.stdout:0/976: read d10/d11/d27/d77/f81 [488404,108157] 0 2024-02-16T19:29:20.135 INFO:tasks.workunit.client.0.smithi022.stdout:8/874: dwrite d3/d12/d17/d64/d8f/fb6 [0,4194304] 0 2024-02-16T19:29:20.139 INFO:tasks.workunit.client.0.smithi022.stdout:9/762: unlink db/d14/d19/d1a/lbb 0 2024-02-16T19:29:20.139 INFO:tasks.workunit.client.0.smithi022.stdout:6/789: write d1/d8/da/d29/deb/dfd/d85/d5b/f63 [501457,65655] 0 2024-02-16T19:29:20.140 INFO:tasks.workunit.client.0.smithi022.stdout:9/763: write db/d18/f93 [6763011,17163] 0 2024-02-16T19:29:20.141 INFO:tasks.workunit.client.1.smithi117.stdout:4/844: creat d0/da/d21/db4/de4/f117 x:0 0 0 2024-02-16T19:29:20.142 INFO:tasks.workunit.client.0.smithi022.stdout:6/790: truncate d1/d8/dac/d101/f10a 627207 0 2024-02-16T19:29:20.145 INFO:tasks.workunit.client.0.smithi022.stdout:2/774: rename d0/d32/d3b/d6f to d0/d2/d18/d104 0 2024-02-16T19:29:20.147 INFO:tasks.workunit.client.0.smithi022.stdout:3/738: getdents d7/d12/d18/dd/d24/d93/de9 0 2024-02-16T19:29:20.149 INFO:tasks.workunit.client.1.smithi117.stdout:6/824: write d5/d9c/daa/fb2 [13381659,96544] 0 2024-02-16T19:29:20.149 INFO:tasks.workunit.client.1.smithi117.stdout:5/878: dwrite d2/d3/f12 [0,4194304] 0 2024-02-16T19:29:20.150 INFO:tasks.workunit.client.1.smithi117.stdout:4/845: dwrite d0/da/d21/d3d/d28/fa6 [0,4194304] 0 2024-02-16T19:29:20.152 INFO:tasks.workunit.client.1.smithi117.stdout:0/865: link d0/d9/d14/d1d/d60/d65/f10c d0/d9/d2c/d8b/dcd/f11f 0 2024-02-16T19:29:20.154 INFO:tasks.workunit.client.1.smithi117.stdout:7/836: read d4/d1c/d87/d89/d5f/f81 [2623775,46930] 0 2024-02-16T19:29:20.154 INFO:tasks.workunit.client.1.smithi117.stdout:2/867: unlink d0/f6c 0 2024-02-16T19:29:20.155 INFO:tasks.workunit.client.0.smithi022.stdout:9/764: mknod db/d14/d16/dec/dee/c10b 0 2024-02-16T19:29:20.156 INFO:tasks.workunit.client.0.smithi022.stdout:6/791: dread - d1/d8/d3c/d77/fd1 zero size 2024-02-16T19:29:20.157 INFO:tasks.workunit.client.1.smithi117.stdout:3/939: getdents d1/d30/d9f/dcb/d103/d134/d136 0 2024-02-16T19:29:20.158 INFO:tasks.workunit.client.1.smithi117.stdout:5/879: dwrite d2/d3/f47 [0,4194304] 0 2024-02-16T19:29:20.159 INFO:tasks.workunit.client.0.smithi022.stdout:2/775: creat d0/d5d/dc4/f105 x:0 0 0 2024-02-16T19:29:20.159 INFO:tasks.workunit.client.1.smithi117.stdout:3/940: dread - d1/dd/d3b/dba/d116/f13e zero size 2024-02-16T19:29:20.160 INFO:tasks.workunit.client.0.smithi022.stdout:3/739: symlink d7/d12/d18/dd/d31/d7f/l101 0 2024-02-16T19:29:20.162 INFO:tasks.workunit.client.0.smithi022.stdout:8/875: mknod d3/d12/d117/c12a 0 2024-02-16T19:29:20.163 INFO:tasks.workunit.client.0.smithi022.stdout:8/876: write d3/d12/d17/fc6 [614083,123471] 0 2024-02-16T19:29:20.164 INFO:tasks.workunit.client.0.smithi022.stdout:8/877: write d3/fe2 [10350945,47646] 0 2024-02-16T19:29:20.207 INFO:tasks.workunit.client.1.smithi117.stdout:4/846: symlink d0/da/d21/d3d/d28/d64/d73/d91/l118 0 2024-02-16T19:29:20.208 INFO:tasks.workunit.client.1.smithi117.stdout:0/866: rename d0/d9/d14/dec/c11d to d0/d9/d2c/d8b/c120 0 2024-02-16T19:29:20.211 INFO:tasks.workunit.client.1.smithi117.stdout:2/868: unlink d0/d13/l7d 0 2024-02-16T19:29:20.211 INFO:tasks.workunit.client.1.smithi117.stdout:8/891: getdents d4/d12/d2a/d64/dec/d76 0 2024-02-16T19:29:20.212 INFO:tasks.workunit.client.0.smithi022.stdout:3/740: rmdir d7/d12/d18/dd/d24/d58 39 2024-02-16T19:29:20.213 INFO:tasks.workunit.client.1.smithi117.stdout:5/880: creat d2/d3/d21/f125 x:0 0 0 2024-02-16T19:29:20.215 INFO:tasks.workunit.client.1.smithi117.stdout:2/869: dread d0/d3/d34/d94/d46/d68/d76/da1/fee [0,4194304] 0 2024-02-16T19:29:20.218 INFO:tasks.workunit.client.1.smithi117.stdout:7/837: readlink d4/d13/d63/d71/l115 0 2024-02-16T19:29:20.219 INFO:tasks.workunit.client.1.smithi117.stdout:7/838: fsync d4/d85/fcd 0 2024-02-16T19:29:20.220 INFO:tasks.workunit.client.1.smithi117.stdout:0/867: creat d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/d27/d4e/f121 x:0 0 0 2024-02-16T19:29:20.224 INFO:tasks.workunit.client.0.smithi022.stdout:2/776: creat d0/d32/d3b/de5/f106 x:0 0 0 2024-02-16T19:29:20.228 INFO:tasks.workunit.client.0.smithi022.stdout:8/878: symlink d3/d12/d2b/l12b 0 2024-02-16T19:29:20.229 INFO:tasks.workunit.client.1.smithi117.stdout:8/892: dwrite d4/d12/fd4 [0,4194304] 0 2024-02-16T19:29:20.230 INFO:tasks.workunit.client.0.smithi022.stdout:0/977: getdents d10/d11/d79/d8b 0 2024-02-16T19:29:20.231 INFO:tasks.workunit.client.1.smithi117.stdout:8/893: write d4/d7/da6/d3a/d65/d96/fcb [917534,18331] 0 2024-02-16T19:29:20.231 INFO:tasks.workunit.client.1.smithi117.stdout:2/870: rmdir d0/d13/dc4/de4 39 2024-02-16T19:29:20.231 INFO:tasks.workunit.client.1.smithi117.stdout:8/894: chown d4/c2d 7 1 2024-02-16T19:29:20.234 INFO:tasks.workunit.client.0.smithi022.stdout:6/792: link d1/d8/d81/l105 d1/d8/dac/d101/da3/dad/l112 0 2024-02-16T19:29:20.236 INFO:tasks.workunit.client.1.smithi117.stdout:3/941: truncate d1/d1b/d1e/d5e/f66 1004842 0 2024-02-16T19:29:20.237 INFO:tasks.workunit.client.0.smithi022.stdout:3/741: symlink d7/d21/dd9/l102 0 2024-02-16T19:29:20.240 INFO:tasks.workunit.client.1.smithi117.stdout:6/825: dwrite d5/db/d7d/f40 [0,4194304] 0 2024-02-16T19:29:20.243 INFO:tasks.workunit.client.1.smithi117.stdout:0/868: truncate d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/d27/d4e/f62 221189 0 2024-02-16T19:29:20.244 INFO:tasks.workunit.client.0.smithi022.stdout:0/978: mkdir d10/d15/d23/d8f/dc1/deb/d145 0 2024-02-16T19:29:20.244 INFO:tasks.workunit.client.1.smithi117.stdout:5/881: write d2/d9/d3c/d24/d50/d52/f1e [3813118,127755] 0 2024-02-16T19:29:20.244 INFO:tasks.workunit.client.0.smithi022.stdout:0/979: chown d10/d22/fd3 48404706 1 2024-02-16T19:29:20.246 INFO:tasks.workunit.client.0.smithi022.stdout:9/765: creat db/d14/d16/d21/d37/f10c x:0 0 0 2024-02-16T19:29:20.247 INFO:tasks.workunit.client.0.smithi022.stdout:6/793: fsync d1/d8/da/d29/deb/dfd/d85/d56/fc4 0 2024-02-16T19:29:20.248 INFO:tasks.workunit.client.0.smithi022.stdout:0/980: read d10/d11/d50/fc3 [1701567,123674] 0 2024-02-16T19:29:20.250 INFO:tasks.workunit.client.0.smithi022.stdout:3/742: creat d7/d12/d18/dd/d31/dbf/f103 x:0 0 0 2024-02-16T19:29:20.252 INFO:tasks.workunit.client.0.smithi022.stdout:8/879: mkdir d3/d12/d17/d12c 0 2024-02-16T19:29:20.253 INFO:tasks.workunit.client.1.smithi117.stdout:4/847: creat d0/da/d21/f119 x:0 0 0 2024-02-16T19:29:20.254 INFO:tasks.workunit.client.0.smithi022.stdout:8/880: write d3/d12/d6a/fa4 [549353,84161] 0 2024-02-16T19:29:20.256 INFO:tasks.workunit.client.1.smithi117.stdout:4/848: read d0/da/d21/f65 [4964605,25261] 0 2024-02-16T19:29:20.256 INFO:tasks.workunit.client.1.smithi117.stdout:3/942: rename d1/d83/d96/cd3 to d1/d30/d9f/dcb/d103/c144 0 2024-02-16T19:29:20.262 INFO:tasks.workunit.client.1.smithi117.stdout:6/826: rmdir d5/db/d7d 39 2024-02-16T19:29:20.263 INFO:tasks.workunit.client.1.smithi117.stdout:6/827: dread - d5/d4b/d71/dc7/d49/fbc zero size 2024-02-16T19:29:20.264 INFO:tasks.workunit.client.0.smithi022.stdout:0/981: chown d10/d15/d23/d38/f9e 58 1 2024-02-16T19:29:20.264 INFO:tasks.workunit.client.1.smithi117.stdout:7/839: chown d4/d1c/dd8/ce2 2 1 2024-02-16T19:29:20.268 INFO:tasks.workunit.client.1.smithi117.stdout:2/871: mknod d0/d13/dc4/c112 0 2024-02-16T19:29:20.268 INFO:tasks.workunit.client.0.smithi022.stdout:2/777: link d0/d32/l3a d0/d2/d2e/d82/df0/df3/l107 0 2024-02-16T19:29:20.268 INFO:tasks.workunit.client.0.smithi022.stdout:4/714: sync 2024-02-16T19:29:20.270 INFO:tasks.workunit.client.1.smithi117.stdout:8/895: creat d4/d7/da6/d3a/d65/d115/f11f x:0 0 0 2024-02-16T19:29:20.270 INFO:tasks.workunit.client.0.smithi022.stdout:2/778: dread d0/d2/d18/d19/ddf/feb [0,4194304] 0 2024-02-16T19:29:20.272 INFO:tasks.workunit.client.0.smithi022.stdout:9/766: write db/d14/d16/d27/d2e/f38 [2912514,38721] 0 2024-02-16T19:29:20.273 INFO:tasks.workunit.client.0.smithi022.stdout:8/881: rmdir d3/d25/d33 39 2024-02-16T19:29:20.274 INFO:tasks.workunit.client.0.smithi022.stdout:9/767: readlink db/d14/d16/d5d/d65/d6b/la1 0 2024-02-16T19:29:20.276 INFO:tasks.workunit.client.0.smithi022.stdout:4/715: dwrite d4/d6/dd/df/d10/f50 [0,4194304] 0 2024-02-16T19:29:20.276 INFO:tasks.workunit.client.1.smithi117.stdout:3/943: mkdir d1/d67/d145 0 2024-02-16T19:29:20.277 INFO:tasks.workunit.client.1.smithi117.stdout:3/944: read - d1/d30/d9f/dbc/d12f/f138 zero size 2024-02-16T19:29:20.279 INFO:tasks.workunit.client.1.smithi117.stdout:4/849: unlink d0/da/d21/d3d/d28/d66/da2/f10d 0 2024-02-16T19:29:20.280 INFO:tasks.workunit.client.1.smithi117.stdout:4/850: chown d0/da/d21/d3d/d28/d64/d73/d91/d9c 0 1 2024-02-16T19:29:20.282 INFO:tasks.workunit.client.0.smithi022.stdout:0/982: rename d10/d15/d23/d38/f41 to d10/d15/d23/d4e/f146 0 2024-02-16T19:29:20.284 INFO:tasks.workunit.client.0.smithi022.stdout:3/743: symlink d7/d12/d18/dd/d24/d58/d70/l104 0 2024-02-16T19:29:20.285 INFO:tasks.workunit.client.1.smithi117.stdout:6/828: read d5/d4b/d71/dc7/d72/fa2 [51760,90181] 0 2024-02-16T19:29:20.285 INFO:tasks.workunit.client.0.smithi022.stdout:6/794: truncate d1/d8/dac/d101/fe9 1061325 0 2024-02-16T19:29:20.286 INFO:tasks.workunit.client.1.smithi117.stdout:6/829: fdatasync d5/d4b/fe4 0 2024-02-16T19:29:20.286 INFO:tasks.workunit.client.1.smithi117.stdout:6/830: stat d5/d4b/d44/fcf 0 2024-02-16T19:29:20.287 INFO:tasks.workunit.client.1.smithi117.stdout:2/872: chown d0/d3/cd5 7190 1 2024-02-16T19:29:20.290 INFO:tasks.workunit.client.0.smithi022.stdout:9/768: mknod db/d14/dc8/c10d 0 2024-02-16T19:29:20.291 INFO:tasks.workunit.client.0.smithi022.stdout:9/769: write db/d14/dc8/dc9/fdf [5058793,127131] 0 2024-02-16T19:29:20.294 INFO:tasks.workunit.client.1.smithi117.stdout:3/945: mknod d1/d31/d89/c146 0 2024-02-16T19:29:20.295 INFO:tasks.workunit.client.1.smithi117.stdout:4/851: mkdir d0/da/d21/d3d/d28/d66/d11a 0 2024-02-16T19:29:20.301 INFO:tasks.workunit.client.0.smithi022.stdout:6/795: symlink d1/d8/da/d29/d61/d57/l113 0 2024-02-16T19:29:20.303 INFO:tasks.workunit.client.1.smithi117.stdout:6/831: symlink d5/d4b/d44/d52/d81/dd7/l114 0 2024-02-16T19:29:20.305 INFO:tasks.workunit.client.1.smithi117.stdout:6/832: chown d5/d4b/d44/d52/d81/dfc/d106 864434486 1 2024-02-16T19:29:20.305 INFO:tasks.workunit.client.1.smithi117.stdout:8/896: mknod d4/d7/da6/d3a/d43/d44/c120 0 2024-02-16T19:29:20.305 INFO:tasks.workunit.client.1.smithi117.stdout:6/833: fdatasync d5/d9c/daa/fb2 0 2024-02-16T19:29:20.307 INFO:tasks.workunit.client.1.smithi117.stdout:6/834: write d5/db/f1f [7033880,130935] 0 2024-02-16T19:29:20.308 INFO:tasks.workunit.client.1.smithi117.stdout:3/946: chown d1/dd/l61 345223970 1 2024-02-16T19:29:20.309 INFO:tasks.workunit.client.0.smithi022.stdout:0/983: dread d10/d11/d50/d65/fbe [0,4194304] 0 2024-02-16T19:29:20.309 INFO:tasks.workunit.client.0.smithi022.stdout:2/779: dwrite d0/d2/d18/f65 [0,4194304] 0 2024-02-16T19:29:20.310 INFO:tasks.workunit.client.0.smithi022.stdout:0/984: fdatasync d10/d11/d50/f123 0 2024-02-16T19:29:20.311 INFO:tasks.workunit.client.0.smithi022.stdout:4/716: dread d4/d6/dd/df/d10/d24/f8c [0,4194304] 0 2024-02-16T19:29:20.312 INFO:tasks.workunit.client.1.smithi117.stdout:4/852: mknod d0/da/d21/d3d/d28/d64/d73/d91/d37/c11b 0 2024-02-16T19:29:20.312 INFO:tasks.workunit.client.0.smithi022.stdout:0/985: write d10/d15/d23/d38/d5a/f131 [686469,130143] 0 2024-02-16T19:29:20.313 INFO:tasks.workunit.client.0.smithi022.stdout:9/770: rmdir db/d14/dc8/dc9/d105 39 2024-02-16T19:29:20.315 INFO:tasks.workunit.client.1.smithi117.stdout:5/882: getdents d2/d3 0 2024-02-16T19:29:20.317 INFO:tasks.workunit.client.1.smithi117.stdout:5/883: fdatasync d2/d9/d3c/d24/d50/d96/dbc/fbe 0 2024-02-16T19:29:20.318 INFO:tasks.workunit.client.1.smithi117.stdout:2/873: symlink d0/d3/d22/d37/d9d/de6/l113 0 2024-02-16T19:29:20.320 INFO:tasks.workunit.client.0.smithi022.stdout:6/796: unlink d1/d8/dac/d101/da3/dad/f106 0 2024-02-16T19:29:20.323 INFO:tasks.workunit.client.1.smithi117.stdout:8/897: mkdir d4/d12/d2a/d64/dec/d76/d121 0 2024-02-16T19:29:20.323 INFO:tasks.workunit.client.1.smithi117.stdout:8/898: fdatasync d4/d7/da6/ffc 0 2024-02-16T19:29:20.324 INFO:tasks.workunit.client.0.smithi022.stdout:2/780: read d0/d2/d18/f34 [2766494,44290] 0 2024-02-16T19:29:20.331 INFO:tasks.workunit.client.0.smithi022.stdout:4/717: rename d4/d6/dd/d22/d30/f44 to d4/d6/d14/d64/d4e/d54/d5a/dce/fe1 0 2024-02-16T19:29:20.335 INFO:tasks.workunit.client.0.smithi022.stdout:4/718: dwrite d4/d6/dd/d22/d30/fd0 [0,4194304] 0 2024-02-16T19:29:20.336 INFO:tasks.workunit.client.0.smithi022.stdout:4/719: fdatasync d4/d6/dd/df/dc1/fd5 0 2024-02-16T19:29:20.337 INFO:tasks.workunit.client.1.smithi117.stdout:5/884: creat d2/d3/d21/d49/d59/d120/f126 x:0 0 0 2024-02-16T19:29:20.339 INFO:tasks.workunit.client.0.smithi022.stdout:3/744: creat d7/f105 x:0 0 0 2024-02-16T19:29:20.344 INFO:tasks.workunit.client.1.smithi117.stdout:5/885: dwrite d2/d5d/f9d [0,4194304] 0 2024-02-16T19:29:20.344 INFO:tasks.workunit.client.0.smithi022.stdout:6/797: fdatasync d1/d8/da/d29/deb/dfd/faf 0 2024-02-16T19:29:20.345 INFO:tasks.workunit.client.0.smithi022.stdout:8/882: link d3/d12/d2b/d46/f91 d3/d12/d2b/d54/d74/f12d 0 2024-02-16T19:29:20.347 INFO:tasks.workunit.client.0.smithi022.stdout:8/883: write d3/d12/d6a/f6e [2007380,39298] 0 2024-02-16T19:29:20.348 INFO:tasks.workunit.client.1.smithi117.stdout:4/853: write d0/da/d21/d3d/d28/d64/d73/d91/d37/d98/fe7 [1472305,4974] 0 2024-02-16T19:29:20.352 INFO:tasks.workunit.client.1.smithi117.stdout:6/835: dwrite d5/d4b/d20/f42 [0,4194304] 0 2024-02-16T19:29:20.353 INFO:tasks.workunit.client.1.smithi117.stdout:2/874: write d0/d3/d22/f84 [278847,99715] 0 2024-02-16T19:29:20.353 INFO:tasks.workunit.client.1.smithi117.stdout:8/899: write d4/d7/f119 [374106,117428] 0 2024-02-16T19:29:20.356 INFO:tasks.workunit.client.1.smithi117.stdout:3/947: dwrite d1/dd/d5c/d85/d135/fd2 [0,4194304] 0 2024-02-16T19:29:20.362 INFO:tasks.workunit.client.1.smithi117.stdout:6/836: read d5/db/d7d/d32/f3f [1307324,103918] 0 2024-02-16T19:29:20.363 INFO:tasks.workunit.client.1.smithi117.stdout:2/875: creat d0/d3/d22/d37/d9d/f114 x:0 0 0 2024-02-16T19:29:20.364 INFO:tasks.workunit.client.1.smithi117.stdout:5/886: symlink d2/d3/d21/d49/d59/l127 0 2024-02-16T19:29:20.367 INFO:tasks.workunit.client.0.smithi022.stdout:9/771: write db/d14/dc8/dc9/d105/fbd [360394,80426] 0 2024-02-16T19:29:20.368 INFO:tasks.workunit.client.0.smithi022.stdout:6/798: read d1/d8/d3c/d77/f80 [207703,72023] 0 2024-02-16T19:29:20.369 INFO:tasks.workunit.client.0.smithi022.stdout:4/720: write d4/d6/dd/d22/d30/fad [733276,17319] 0 2024-02-16T19:29:20.370 INFO:tasks.workunit.client.1.smithi117.stdout:3/948: chown d1/d30/d52/d56/lb7 23899 1 2024-02-16T19:29:20.370 INFO:tasks.workunit.client.1.smithi117.stdout:3/949: readlink d1/d83/lf3 0 2024-02-16T19:29:20.373 INFO:tasks.workunit.client.1.smithi117.stdout:5/887: unlink d2/d3/d21/d49/d59/c88 0 2024-02-16T19:29:20.376 INFO:tasks.workunit.client.1.smithi117.stdout:2/876: symlink d0/d3/d34/d94/l115 0 2024-02-16T19:29:20.377 INFO:tasks.workunit.client.1.smithi117.stdout:4/854: getdents d0/da/d21/d3d/d28/d66 0 2024-02-16T19:29:20.378 INFO:tasks.workunit.client.1.smithi117.stdout:6/837: dwrite d5/d4b/d44/d93/dae/fe3 [0,4194304] 0 2024-02-16T19:29:20.380 INFO:tasks.workunit.client.1.smithi117.stdout:5/888: mkdir d2/d9/d3c/d24/d50/d96/dc4/dca/df3/d128 0 2024-02-16T19:29:20.381 INFO:tasks.workunit.client.0.smithi022.stdout:2/781: link d0/d2/d18/d19/d40/c4c d0/d2/d2e/d82/df0/dff/c108 0 2024-02-16T19:29:20.385 INFO:tasks.workunit.client.0.smithi022.stdout:0/986: getdents d10/d11/d27/d77 0 2024-02-16T19:29:20.385 INFO:tasks.workunit.client.0.smithi022.stdout:5/849: sync 2024-02-16T19:29:20.387 INFO:tasks.workunit.client.0.smithi022.stdout:8/884: dwrite d3/d25/f79 [0,4194304] 0 2024-02-16T19:29:20.387 INFO:tasks.workunit.client.1.smithi117.stdout:5/889: mkdir d2/d3/d21/d2a/dcb/d129 0 2024-02-16T19:29:20.388 INFO:tasks.workunit.client.1.smithi117.stdout:2/877: dwrite d0/d13/d4d/d53/fce [0,4194304] 0 2024-02-16T19:29:20.388 INFO:tasks.workunit.client.1.smithi117.stdout:6/838: dwrite d5/d4b/d20/de7/f10d [0,4194304] 0 2024-02-16T19:29:20.389 INFO:tasks.workunit.client.1.smithi117.stdout:2/878: readlink d0/d3/d22/lf5 0 2024-02-16T19:29:20.390 INFO:tasks.workunit.client.1.smithi117.stdout:6/839: readlink d5/d4b/d44/d93/dae/lc0 0 2024-02-16T19:29:20.390 INFO:tasks.workunit.client.0.smithi022.stdout:9/772: truncate f5 3943512 0 2024-02-16T19:29:20.393 INFO:tasks.workunit.client.0.smithi022.stdout:6/799: dread d1/d8/dac/d101/fe9 [0,4194304] 0 2024-02-16T19:29:20.394 INFO:tasks.workunit.client.0.smithi022.stdout:3/745: dwrite d7/d12/d18/dd/d24/d58/d6b/f94 [0,4194304] 0 2024-02-16T19:29:20.395 INFO:tasks.workunit.client.0.smithi022.stdout:9/773: chown db/d14/d16/d27/d2e 1 1 2024-02-16T19:29:20.396 INFO:tasks.workunit.client.1.smithi117.stdout:6/840: fdatasync d5/d4b/d44/d52/d81/dd7/de0/ff2 0 2024-02-16T19:29:20.396 INFO:tasks.workunit.client.1.smithi117.stdout:4/855: mkdir d0/da/d21/d11c 0 2024-02-16T19:29:20.397 INFO:tasks.workunit.client.0.smithi022.stdout:2/782: dread d0/d2/d2e/fd2 [0,4194304] 0 2024-02-16T19:29:20.400 INFO:tasks.workunit.client.0.smithi022.stdout:0/987: dread d10/d11/d27/d77/f13d [0,4194304] 0 2024-02-16T19:29:20.401 INFO:tasks.workunit.client.0.smithi022.stdout:8/885: mknod d3/d12/d17/d10d/c12e 0 2024-02-16T19:29:20.404 INFO:tasks.workunit.client.1.smithi117.stdout:4/856: creat d0/da/d21/db4/de4/f11d x:0 0 0 2024-02-16T19:29:20.406 INFO:tasks.workunit.client.1.smithi117.stdout:4/857: write d0/da/d4a/fd5 [301773,57500] 0 2024-02-16T19:29:20.407 INFO:tasks.workunit.client.1.smithi117.stdout:2/879: truncate d0/f19 4699657 0 2024-02-16T19:29:20.408 INFO:tasks.workunit.client.1.smithi117.stdout:4/858: fsync d0/da/d21/d3d/d28/d66/da2/fe1 0 2024-02-16T19:29:20.408 INFO:tasks.workunit.client.1.smithi117.stdout:2/880: write d0/d3/d22/d37/d9d/ff6 [391453,110656] 0 2024-02-16T19:29:20.411 INFO:tasks.workunit.client.0.smithi022.stdout:6/800: creat d1/d8/da/d29/deb/f114 x:0 0 0 2024-02-16T19:29:20.411 INFO:tasks.workunit.client.0.smithi022.stdout:2/783: unlink d0/d2/d18/d38/d9d/fb9 0 2024-02-16T19:29:20.412 INFO:tasks.workunit.client.0.smithi022.stdout:0/988: creat d10/d11/d27/f147 x:0 0 0 2024-02-16T19:29:20.413 INFO:tasks.workunit.client.1.smithi117.stdout:4/859: chown d0/da/d21/d3d/d28/d64/d73/le5 504 1 2024-02-16T19:29:20.414 INFO:tasks.workunit.client.1.smithi117.stdout:4/860: dread - d0/f72 zero size 2024-02-16T19:29:20.415 INFO:tasks.workunit.client.1.smithi117.stdout:2/881: unlink d0/d13/d4d/d53/d58/fcb 0 2024-02-16T19:29:20.417 INFO:tasks.workunit.client.0.smithi022.stdout:9/774: dwrite db/d14/dc8/dc9/d105/fbd [0,4194304] 0 2024-02-16T19:29:20.418 INFO:tasks.workunit.client.1.smithi117.stdout:4/861: symlink d0/da/d21/d3d/d28/d64/d8a/d92/de8/l11e 0 2024-02-16T19:29:20.421 INFO:tasks.workunit.client.1.smithi117.stdout:5/890: dread d2/d9/d3c/d24/d50/d96/dbc/d30/d38/f64 [0,4194304] 0 2024-02-16T19:29:20.421 INFO:tasks.workunit.client.1.smithi117.stdout:4/862: chown d0/da/d21/d3d/d28/d64/d48/f4c 112199 1 2024-02-16T19:29:20.422 INFO:tasks.workunit.client.0.smithi022.stdout:6/801: dwrite d1/d8/dac/d101/db3/ff8 [0,4194304] 0 2024-02-16T19:29:20.432 INFO:tasks.workunit.client.1.smithi117.stdout:6/841: dread d5/db/d7d/d32/f5b [0,4194304] 0 2024-02-16T19:29:20.443 INFO:tasks.workunit.client.0.smithi022.stdout:8/886: rename d3/d12/d2b/dca/cf4 to d3/d12/d2b/dca/c12f 0 2024-02-16T19:29:20.444 INFO:tasks.workunit.client.0.smithi022.stdout:3/746: creat d7/d12/d18/dd/f106 x:0 0 0 2024-02-16T19:29:20.445 INFO:tasks.workunit.client.0.smithi022.stdout:0/989: mknod d10/d11/d27/d77/c148 0 2024-02-16T19:29:20.446 INFO:tasks.workunit.client.0.smithi022.stdout:2/784: mknod d0/d2/d18/d19/d90/c109 0 2024-02-16T19:29:20.447 INFO:tasks.workunit.client.0.smithi022.stdout:9/775: truncate db/d14/d16/d27/fa3 404788 0 2024-02-16T19:29:20.449 INFO:tasks.workunit.client.0.smithi022.stdout:8/887: chown d3/c62 2978 1 2024-02-16T19:29:20.451 INFO:tasks.workunit.client.0.smithi022.stdout:3/747: rename d7/d12/d18/dd/d3c/fcd to d7/d21/dd9/f107 0 2024-02-16T19:29:20.452 INFO:tasks.workunit.client.1.smithi117.stdout:5/891: dread d2/d3/f6 [0,4194304] 0 2024-02-16T19:29:20.452 INFO:tasks.workunit.client.0.smithi022.stdout:0/990: creat d10/d15/d23/d8f/dc1/f149 x:0 0 0 2024-02-16T19:29:20.453 INFO:tasks.workunit.client.0.smithi022.stdout:9/776: creat db/d14/d16/d5d/d65/d6b/d89/f10e x:0 0 0 2024-02-16T19:29:20.454 INFO:tasks.workunit.client.0.smithi022.stdout:8/888: mknod d3/d12/d17/d64/d8f/c130 0 2024-02-16T19:29:20.457 INFO:tasks.workunit.client.0.smithi022.stdout:9/777: fsync db/d14/d16/d28/daa/fad 0 2024-02-16T19:29:20.461 INFO:tasks.workunit.client.0.smithi022.stdout:8/889: dread d3/d25/f43 [0,4194304] 0 2024-02-16T19:29:20.462 INFO:tasks.workunit.client.0.smithi022.stdout:3/748: dwrite d7/d21/f36 [0,4194304] 0 2024-02-16T19:29:20.464 INFO:tasks.workunit.client.0.smithi022.stdout:9/778: creat db/d14/d19/da8/f10f x:0 0 0 2024-02-16T19:29:20.464 INFO:tasks.workunit.client.0.smithi022.stdout:4/721: truncate d4/d6/dd/d22/da8/dc4/d6d/faa 378502 0 2024-02-16T19:29:20.464 INFO:tasks.workunit.client.0.smithi022.stdout:4/722: chown d4/d6/d14 3 1 2024-02-16T19:29:20.465 INFO:tasks.workunit.client.1.smithi117.stdout:1/760: sync 2024-02-16T19:29:20.466 INFO:tasks.workunit.client.0.smithi022.stdout:3/749: creat d7/d12/d18/dd/d24/d58/d70/f108 x:0 0 0 2024-02-16T19:29:20.468 INFO:tasks.workunit.client.0.smithi022.stdout:9/779: rmdir db/d14/d16/d5d/d65 39 2024-02-16T19:29:20.469 INFO:tasks.workunit.client.1.smithi117.stdout:1/761: readlink d2/d1d/d60/le5 0 2024-02-16T19:29:20.471 INFO:tasks.workunit.client.0.smithi022.stdout:4/723: write d4/d6/dd/d22/f59 [1832635,49163] 0 2024-02-16T19:29:20.471 INFO:tasks.workunit.client.0.smithi022.stdout:3/750: truncate d7/d21/f96 102092 0 2024-02-16T19:29:20.473 INFO:tasks.workunit.client.1.smithi117.stdout:1/762: symlink d2/dab/l108 0 2024-02-16T19:29:20.473 INFO:tasks.workunit.client.0.smithi022.stdout:9/780: mknod db/d14/d16/d28/daa/c110 0 2024-02-16T19:29:20.474 INFO:tasks.workunit.client.1.smithi117.stdout:1/763: write d2/d19/d59/d89/d7b/fd9 [999498,45875] 0 2024-02-16T19:29:20.475 INFO:tasks.workunit.client.0.smithi022.stdout:4/724: mknod d4/d6/dd/d22/d83/ce2 0 2024-02-16T19:29:20.477 INFO:tasks.workunit.client.0.smithi022.stdout:3/751: dread - d7/d21/fbd zero size 2024-02-16T19:29:20.480 INFO:tasks.workunit.client.0.smithi022.stdout:4/725: dread d4/d6/dd/df/f91 [0,4194304] 0 2024-02-16T19:29:20.487 INFO:tasks.workunit.client.0.smithi022.stdout:4/726: dwrite d4/d6/dd/df/f91 [4194304,4194304] 0 2024-02-16T19:29:20.489 INFO:tasks.workunit.client.0.smithi022.stdout:3/752: fdatasync d7/d12/d18/dd/d24/d58/f8b 0 2024-02-16T19:29:20.491 INFO:tasks.workunit.client.0.smithi022.stdout:9/781: rmdir db/d14/d16/d87/d104 0 2024-02-16T19:29:20.492 INFO:tasks.workunit.client.1.smithi117.stdout:2/882: dwrite d0/d3/f31 [0,4194304] 0 2024-02-16T19:29:20.496 INFO:tasks.workunit.client.0.smithi022.stdout:3/753: read d7/d12/d18/d4b/f6f [752371,47303] 0 2024-02-16T19:29:20.497 INFO:tasks.workunit.client.1.smithi117.stdout:4/863: write d0/da/d21/d3d/d28/d64/d8a/d92/f89 [1136076,102093] 0 2024-02-16T19:29:20.498 INFO:tasks.workunit.client.0.smithi022.stdout:3/754: write d7/d12/d18/dd/d24/d93/de9/f60 [4510456,112439] 0 2024-02-16T19:29:20.498 INFO:tasks.workunit.client.1.smithi117.stdout:2/883: mknod d0/d3/c116 0 2024-02-16T19:29:20.500 INFO:tasks.workunit.client.1.smithi117.stdout:2/884: write d0/d3/d22/d37/d9d/fe8 [386377,94214] 0 2024-02-16T19:29:20.501 INFO:tasks.workunit.client.1.smithi117.stdout:4/864: mknod d0/da/d21/d3d/d28/d64/d73/c11f 0 2024-02-16T19:29:20.502 INFO:tasks.workunit.client.1.smithi117.stdout:2/885: read - d0/d13/d4d/d80/f9c zero size 2024-02-16T19:29:20.503 INFO:tasks.workunit.client.0.smithi022.stdout:3/755: rename d7/d12/l26 to d7/d12/d18/dd/d24/d53/l109 0 2024-02-16T19:29:20.507 INFO:tasks.workunit.client.1.smithi117.stdout:2/886: symlink d0/d13/dc4/d10f/l117 0 2024-02-16T19:29:20.510 INFO:tasks.workunit.client.0.smithi022.stdout:3/756: dwrite d7/d12/d18/dd/f106 [0,4194304] 0 2024-02-16T19:29:20.514 INFO:tasks.workunit.client.0.smithi022.stdout:3/757: fdatasync d7/d12/d18/dd/d3c/f92 0 2024-02-16T19:29:20.516 INFO:tasks.workunit.client.0.smithi022.stdout:3/758: write d7/f105 [393386,38692] 0 2024-02-16T19:29:20.520 INFO:tasks.workunit.client.0.smithi022.stdout:3/759: creat d7/d12/d18/dd/d24/f10a x:0 0 0 2024-02-16T19:29:20.525 INFO:tasks.workunit.client.0.smithi022.stdout:6/802: write d1/d8/da/d29/f35 [2270536,13410] 0 2024-02-16T19:29:20.527 INFO:tasks.workunit.client.0.smithi022.stdout:2/785: write d0/f2d [1387681,3789] 0 2024-02-16T19:29:20.527 INFO:tasks.workunit.client.0.smithi022.stdout:6/803: write d1/d8/da/d7b/dae/db6/f100 [778434,78817] 0 2024-02-16T19:29:20.528 INFO:tasks.workunit.client.0.smithi022.stdout:0/991: write d10/d11/d79/d8b/fa5 [2111527,130305] 0 2024-02-16T19:29:20.530 INFO:tasks.workunit.client.0.smithi022.stdout:2/786: mknod d0/d2/d18/d19/d90/c10a 0 2024-02-16T19:29:20.536 INFO:tasks.workunit.client.0.smithi022.stdout:0/992: truncate d10/d11/d27/d3f/d58/dab/ff6 1021582 0 2024-02-16T19:29:20.537 INFO:tasks.workunit.client.0.smithi022.stdout:0/993: stat d10/d11/d50/d102/l11e 0 2024-02-16T19:29:20.538 INFO:tasks.workunit.client.0.smithi022.stdout:0/994: stat d10/d11/d50/d102/c118 0 2024-02-16T19:29:20.539 INFO:tasks.workunit.client.1.smithi117.stdout:1/764: write d2/d1d/d77/f103 [746098,47859] 0 2024-02-16T19:29:20.540 INFO:tasks.workunit.client.0.smithi022.stdout:2/787: getdents d0 0 2024-02-16T19:29:20.541 INFO:tasks.workunit.client.0.smithi022.stdout:2/788: chown d0/d2/d18/d19/d40/d55/l61 389911399 1 2024-02-16T19:29:20.542 INFO:tasks.workunit.client.0.smithi022.stdout:4/727: write d4/d6/dd/df/d10/d24/f88 [549481,50165] 0 2024-02-16T19:29:20.544 INFO:tasks.workunit.client.0.smithi022.stdout:2/789: creat d0/f10b x:0 0 0 2024-02-16T19:29:20.546 INFO:tasks.workunit.client.0.smithi022.stdout:4/728: mknod d4/d6/dd/d22/d30/ce3 0 2024-02-16T19:29:20.547 INFO:tasks.workunit.client.0.smithi022.stdout:0/995: rmdir d10/d22/d137 0 2024-02-16T19:29:20.550 INFO:tasks.workunit.client.0.smithi022.stdout:4/729: truncate d4/d6/d14/dc7/f79 916202 0 2024-02-16T19:29:20.551 INFO:tasks.workunit.client.0.smithi022.stdout:4/730: stat d4/d6/d14/dc7/d8d/f99 0 2024-02-16T19:29:20.552 INFO:tasks.workunit.client.0.smithi022.stdout:0/996: creat d10/d11/d12b/d134/f14a x:0 0 0 2024-02-16T19:29:20.554 INFO:tasks.workunit.client.1.smithi117.stdout:1/765: dread d2/d1d/d60/d7c/d9e/fc3 [0,4194304] 0 2024-02-16T19:29:20.554 INFO:tasks.workunit.client.1.smithi117.stdout:4/865: dwrite d0/da/d21/d3d/d28/d66/d97/fd2 [0,4194304] 0 2024-02-16T19:29:20.556 INFO:tasks.workunit.client.0.smithi022.stdout:0/997: creat d10/d15/d23/d38/f14b x:0 0 0 2024-02-16T19:29:20.557 INFO:tasks.workunit.client.0.smithi022.stdout:3/760: write d7/d21/fbd [350228,100977] 0 2024-02-16T19:29:20.559 INFO:tasks.workunit.client.1.smithi117.stdout:2/887: dwrite d0/d3/d10/f95 [4194304,4194304] 0 2024-02-16T19:29:20.559 INFO:tasks.workunit.client.0.smithi022.stdout:6/804: write d1/d8/da/d29/deb/dfd/d53/f58 [603070,115895] 0 2024-02-16T19:29:20.561 INFO:tasks.workunit.client.1.smithi117.stdout:1/766: dread d2/d1d/d60/d7c/d9e/fc3 [0,4194304] 0 2024-02-16T19:29:20.569 INFO:tasks.workunit.client.1.smithi117.stdout:4/866: getdents d0/da/d21/d3d/d28/d64/d73/d91/d37/d102 0 2024-02-16T19:29:20.570 INFO:tasks.workunit.client.1.smithi117.stdout:4/867: stat d0/da/d21/d3d/d28/d64/d73/d91/d37/f107 0 2024-02-16T19:29:20.571 INFO:tasks.workunit.client.1.smithi117.stdout:1/767: read - d2/d19/d59/d89/df4/dfc/fbc zero size 2024-02-16T19:29:20.572 INFO:tasks.workunit.client.1.smithi117.stdout:1/768: chown d2/d19/d59/d76/d8c/dc7 76670 1 2024-02-16T19:29:20.573 INFO:tasks.workunit.client.0.smithi022.stdout:0/998: write d10/d15/d23/d8f/f95 [691068,14314] 0 2024-02-16T19:29:20.574 INFO:tasks.workunit.client.1.smithi117.stdout:1/769: creat d2/d19/d59/d76/f109 x:0 0 0 2024-02-16T19:29:20.575 INFO:tasks.workunit.client.0.smithi022.stdout:3/761: write d7/d12/d18/dd/f7d [3608362,83887] 0 2024-02-16T19:29:20.577 INFO:tasks.workunit.client.0.smithi022.stdout:4/731: dwrite d4/d6/dd/df/d10/d24/fbc [0,4194304] 0 2024-02-16T19:29:20.578 INFO:tasks.workunit.client.1.smithi117.stdout:1/770: symlink d2/d19/d59/d76/d8c/l10a 0 2024-02-16T19:29:20.579 INFO:tasks.workunit.client.0.smithi022.stdout:4/732: readlink d4/lbd 0 2024-02-16T19:29:20.580 INFO:tasks.workunit.client.1.smithi117.stdout:1/771: rmdir d2/dab 39 2024-02-16T19:29:20.581 INFO:tasks.workunit.client.0.smithi022.stdout:3/762: truncate d7/d12/d18/dd/d31/fed 566287 0 2024-02-16T19:29:20.582 INFO:tasks.workunit.client.0.smithi022.stdout:4/733: write d4/d6/dd/d22/da8/dc4/fc3 [167476,57359] 0 2024-02-16T19:29:20.584 INFO:tasks.workunit.client.0.smithi022.stdout:1/798: sync 2024-02-16T19:29:20.584 INFO:tasks.workunit.client.0.smithi022.stdout:7/811: sync 2024-02-16T19:29:20.586 INFO:tasks.workunit.client.1.smithi117.stdout:1/772: rename d2/d19/d59/d89/d69/dad/dc6 to d2/d4/dbb/d10b 0 2024-02-16T19:29:20.587 INFO:tasks.workunit.client.0.smithi022.stdout:4/734: stat d4/d6/dd/df/d10/c3c 0 2024-02-16T19:29:20.587 INFO:tasks.workunit.client.1.smithi117.stdout:4/868: dread d0/da/d4a/fa8 [0,4194304] 0 2024-02-16T19:29:20.589 INFO:tasks.workunit.client.1.smithi117.stdout:2/888: dread d0/d13/d4d/d80/f88 [0,4194304] 0 2024-02-16T19:29:20.590 INFO:tasks.workunit.client.1.smithi117.stdout:1/773: mknod d2/d4/dbb/d10b/c10c 0 2024-02-16T19:29:20.594 INFO:tasks.workunit.client.1.smithi117.stdout:2/889: rename d0/d13/dc4/dd9/l103 to d0/d13/dc4/d10f/l118 0 2024-02-16T19:29:20.594 INFO:tasks.workunit.client.0.smithi022.stdout:4/735: mknod d4/d6/dd/d22/d83/d9e/db7/ce4 0 2024-02-16T19:29:20.604 INFO:tasks.workunit.client.0.smithi022.stdout:2/790: dread d0/d32/f97 [0,4194304] 0 2024-02-16T19:29:20.642 INFO:tasks.workunit.client.1.smithi117.stdout:0/869: sync 2024-02-16T19:29:20.642 INFO:tasks.workunit.client.1.smithi117.stdout:8/900: sync 2024-02-16T19:29:20.642 INFO:tasks.workunit.client.1.smithi117.stdout:7/840: sync 2024-02-16T19:29:20.643 INFO:tasks.workunit.client.1.smithi117.stdout:3/950: sync 2024-02-16T19:29:20.643 INFO:tasks.workunit.client.1.smithi117.stdout:9/757: sync 2024-02-16T19:29:20.644 INFO:tasks.workunit.client.1.smithi117.stdout:9/758: chown dd/d47/d48/dc0/fbf 31046195 1 2024-02-16T19:29:20.644 INFO:tasks.workunit.client.1.smithi117.stdout:7/841: write d4/d13/d63/fce [292887,39224] 0 2024-02-16T19:29:20.644 INFO:tasks.workunit.client.1.smithi117.stdout:9/759: fdatasync dd/ff5 0 2024-02-16T19:29:20.649 INFO:tasks.workunit.client.1.smithi117.stdout:7/842: dwrite d4/d1c/d39/d9f/fd6 [0,4194304] 0 2024-02-16T19:29:20.659 INFO:tasks.workunit.client.1.smithi117.stdout:9/760: dread dd/d45/d31/d63/dba/df0/f1f [0,4194304] 0 2024-02-16T19:29:20.692 INFO:tasks.workunit.client.1.smithi117.stdout:1/774: mknod d2/d19/d59/d89/d69/d80/d8e/dea/c10d 0 2024-02-16T19:29:20.695 INFO:tasks.workunit.client.1.smithi117.stdout:2/890: rename d0/d3/d34/d94/l115 to d0/d13/dc4/dd9/l119 0 2024-02-16T19:29:20.698 INFO:tasks.workunit.client.0.smithi022.stdout:0/999: truncate d10/d11/d27/d3f/d58/d60/f129 3302271 0 2024-02-16T19:29:20.700 INFO:tasks.workunit.client.0.smithi022.stdout:4/736: link d4/d6/d3a/fa0 d4/d6/db6/fe5 0 2024-02-16T19:29:20.702 INFO:tasks.workunit.client.0.smithi022.stdout:3/763: write d7/d12/d18/dd/d31/fae [160303,59768] 0 2024-02-16T19:29:20.703 INFO:tasks.workunit.client.0.smithi022.stdout:2/791: mkdir d0/d2/d18/d19/ddf/d10c 0 2024-02-16T19:29:20.705 INFO:tasks.workunit.client.0.smithi022.stdout:4/737: mknod d4/d6/dd/df/d10/ddc/ce6 0 2024-02-16T19:29:20.707 INFO:tasks.workunit.client.1.smithi117.stdout:9/761: rmdir dd/d45/d31 39 2024-02-16T19:29:20.708 INFO:tasks.workunit.client.0.smithi022.stdout:1/799: dwrite d6/d8/d22/d3a/d3e/f76 [0,4194304] 0 2024-02-16T19:29:20.709 INFO:tasks.workunit.client.1.smithi117.stdout:4/869: write d0/da/d21/d3d/d28/d64/d90/f9b [1696139,25755] 0 2024-02-16T19:29:20.712 INFO:tasks.workunit.client.1.smithi117.stdout:4/870: dread - d0/da/d21/d3d/d28/d66/da2/dac/f10b zero size 2024-02-16T19:29:20.713 INFO:tasks.workunit.client.0.smithi022.stdout:2/792: mkdir d0/d2/d2e/d82/da1/d10d 0 2024-02-16T19:29:20.714 INFO:tasks.workunit.client.1.smithi117.stdout:4/871: write d0/da/d21/d3d/d28/d64/d90/dc0/fc9 [1445623,107188] 0 2024-02-16T19:29:20.715 INFO:tasks.workunit.client.1.smithi117.stdout:1/775: mknod d2/d19/d59/d89/d69/d80/d8e/c10e 0 2024-02-16T19:29:20.717 INFO:tasks.workunit.client.1.smithi117.stdout:4/872: truncate d0/da/d21/d3d/d28/d64/d73/d91/d9e/f112 686493 0 2024-02-16T19:29:20.720 INFO:tasks.workunit.client.0.smithi022.stdout:1/800: unlink d6/d8/d22/c36 0 2024-02-16T19:29:20.722 INFO:tasks.workunit.client.1.smithi117.stdout:0/870: creat d0/d9/d2c/d8b/de0/f122 x:0 0 0 2024-02-16T19:29:20.722 INFO:tasks.workunit.client.1.smithi117.stdout:0/871: chown d0/d9/d14/d1d/d60/d76/d7b 738700 1 2024-02-16T19:29:20.728 INFO:tasks.workunit.client.1.smithi117.stdout:1/776: symlink d2/d19/d59/d89/d7b/d86/l10f 0 2024-02-16T19:29:20.729 INFO:tasks.workunit.client.1.smithi117.stdout:2/891: mknod d0/d3/dff/c11a 0 2024-02-16T19:29:20.731 INFO:tasks.workunit.client.0.smithi022.stdout:3/764: dwrite d7/d12/d18/dac/dd7/fdb [0,4194304] 0 2024-02-16T19:29:20.735 INFO:tasks.workunit.client.0.smithi022.stdout:1/801: dread - d6/d8/dba/fce zero size 2024-02-16T19:29:20.739 INFO:tasks.workunit.client.1.smithi117.stdout:0/872: mknod d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/c123 0 2024-02-16T19:29:20.739 INFO:tasks.workunit.client.0.smithi022.stdout:4/738: creat d4/fe7 x:0 0 0 2024-02-16T19:29:20.740 INFO:tasks.workunit.client.1.smithi117.stdout:5/892: sync 2024-02-16T19:29:20.740 INFO:tasks.workunit.client.1.smithi117.stdout:6/842: sync 2024-02-16T19:29:20.742 INFO:tasks.workunit.client.1.smithi117.stdout:6/843: chown d5/db/d7d/d32/l7b 2652666 1 2024-02-16T19:29:20.742 INFO:tasks.workunit.client.1.smithi117.stdout:9/762: dwrite dd/d47/f57 [0,4194304] 0 2024-02-16T19:29:20.744 INFO:tasks.workunit.client.0.smithi022.stdout:3/765: symlink d7/d12/d18/dd/d24/d93/l10b 0 2024-02-16T19:29:20.745 INFO:tasks.workunit.client.1.smithi117.stdout:4/873: write d0/da/d4a/f58 [1488470,40640] 0 2024-02-16T19:29:20.747 INFO:tasks.workunit.client.1.smithi117.stdout:0/873: creat d0/d9/d14/d1d/d60/d65/f124 x:0 0 0 2024-02-16T19:29:20.751 INFO:tasks.workunit.client.1.smithi117.stdout:5/893: mkdir d2/d4c/d4f/d67/d12a 0 2024-02-16T19:29:20.753 INFO:tasks.workunit.client.1.smithi117.stdout:1/777: write d2/d1d/d60/f66 [1987404,90531] 0 2024-02-16T19:29:20.756 INFO:tasks.workunit.client.0.smithi022.stdout:3/766: mknod d7/d12/d18/dd/d24/d93/de9/c10c 0 2024-02-16T19:29:20.757 INFO:tasks.workunit.client.0.smithi022.stdout:1/802: dread d6/d8/d22/d47/d54/f65 [0,4194304] 0 2024-02-16T19:29:20.757 INFO:tasks.workunit.client.0.smithi022.stdout:3/767: chown d7/d12/d18/dd 87 1 2024-02-16T19:29:20.758 INFO:tasks.workunit.client.1.smithi117.stdout:6/844: mkdir d5/d4b/d20/d69/d115 0 2024-02-16T19:29:20.758 INFO:tasks.workunit.client.1.smithi117.stdout:1/778: dwrite d2/d19/d59/d89/d69/f8f [0,4194304] 0 2024-02-16T19:29:20.759 INFO:tasks.workunit.client.0.smithi022.stdout:5/850: sync 2024-02-16T19:29:20.761 INFO:tasks.workunit.client.1.smithi117.stdout:6/845: truncate d5/d4b/d71/dc7/d49/d59/f10c 166180 0 2024-02-16T19:29:20.761 INFO:tasks.workunit.client.1.smithi117.stdout:2/892: creat d0/d3/d22/d37/d109/f11b x:0 0 0 2024-02-16T19:29:20.762 INFO:tasks.workunit.client.0.smithi022.stdout:4/739: write d4/d6/d3a/f41 [2459620,79817] 0 2024-02-16T19:29:20.762 INFO:tasks.workunit.client.1.smithi117.stdout:9/763: write dd/d47/fc2 [471047,100936] 0 2024-02-16T19:29:20.764 INFO:tasks.workunit.client.1.smithi117.stdout:2/893: truncate d0/d3b/f51 5109931 0 2024-02-16T19:29:20.765 INFO:tasks.workunit.client.1.smithi117.stdout:2/894: stat d0/d3/d40/f55 0 2024-02-16T19:29:20.766 INFO:tasks.workunit.client.1.smithi117.stdout:3/951: sync 2024-02-16T19:29:20.767 INFO:tasks.workunit.client.0.smithi022.stdout:1/803: rename d6/d8/d22/d16/d17/d63/fe4 to d6/d8/d22/d16/d4f/ffd 0 2024-02-16T19:29:20.768 INFO:tasks.workunit.client.1.smithi117.stdout:5/894: rmdir d2/d9/d3c/d24 39 2024-02-16T19:29:20.769 INFO:tasks.workunit.client.1.smithi117.stdout:5/895: chown d2/d4c/d4f/d67/f7f 3 1 2024-02-16T19:29:20.770 INFO:tasks.workunit.client.0.smithi022.stdout:2/793: getdents d0/d2/d18/d19 0 2024-02-16T19:29:20.772 INFO:tasks.workunit.client.0.smithi022.stdout:5/851: mkdir dd/d1f/d48/d21/d2f/d40/d49/d4b/d119 0 2024-02-16T19:29:20.777 INFO:tasks.workunit.client.0.smithi022.stdout:4/740: dread d4/d6/d14/f28 [0,4194304] 0 2024-02-16T19:29:20.780 INFO:tasks.workunit.client.1.smithi117.stdout:0/874: mkdir d0/d9/d14/d1d/d125 0 2024-02-16T19:29:20.781 INFO:tasks.workunit.client.1.smithi117.stdout:4/874: dread d0/da/d21/f27 [0,4194304] 0 2024-02-16T19:29:20.781 INFO:tasks.workunit.client.1.smithi117.stdout:9/764: mknod dd/db5/dc4/dcf/c100 0 2024-02-16T19:29:20.782 INFO:tasks.workunit.client.0.smithi022.stdout:2/794: rmdir d0/d2/d2e/d82 39 2024-02-16T19:29:20.784 INFO:tasks.workunit.client.1.smithi117.stdout:2/895: rename d0/d3/d34/d94/d46/d68/d76/da1/d56/d60/dc9/fd8 to d0/d3/d22/f11c 0 2024-02-16T19:29:20.787 INFO:tasks.workunit.client.1.smithi117.stdout:3/952: mknod d1/dd/d5c/d85/c147 0 2024-02-16T19:29:20.791 INFO:tasks.workunit.client.0.smithi022.stdout:3/768: write d7/d21/f96 [545192,94547] 0 2024-02-16T19:29:20.793 INFO:tasks.workunit.client.0.smithi022.stdout:3/769: chown d7/d12/d18/dd/d3c/f92 0 1 2024-02-16T19:29:20.796 INFO:tasks.workunit.client.1.smithi117.stdout:6/846: dwrite d5/d4b/d71/dc7/f27 [4194304,4194304] 0 2024-02-16T19:29:20.800 INFO:tasks.workunit.client.0.smithi022.stdout:5/852: write dd/d1f/f31 [762019,46242] 0 2024-02-16T19:29:20.801 INFO:tasks.workunit.client.0.smithi022.stdout:3/770: dwrite d7/d12/d18/d4b/f4d [0,4194304] 0 2024-02-16T19:29:20.804 INFO:tasks.workunit.client.1.smithi117.stdout:5/896: write d2/d4c/d4f/d67/fc5 [3969939,50477] 0 2024-02-16T19:29:20.806 INFO:tasks.workunit.client.0.smithi022.stdout:4/741: mkdir d4/d6/dd/df/d10/de8 0 2024-02-16T19:29:20.808 INFO:tasks.workunit.client.0.smithi022.stdout:1/804: write d6/d8/d22/d67/d7e/fa9 [833196,48910] 0 2024-02-16T19:29:20.809 INFO:tasks.workunit.client.1.smithi117.stdout:0/875: write d0/d9/d14/d1d/d60/d65/fad [944415,32086] 0 2024-02-16T19:29:20.811 INFO:tasks.workunit.client.1.smithi117.stdout:2/896: rename d0/d3/d34/d94/d82/cd6 to d0/d3/d34/c11d 0 2024-02-16T19:29:20.813 INFO:tasks.workunit.client.1.smithi117.stdout:4/875: write d0/da/f33 [5605125,26124] 0 2024-02-16T19:29:20.813 INFO:tasks.workunit.client.1.smithi117.stdout:2/897: chown d0/d13/dc4/ffb 183093166 1 2024-02-16T19:29:20.813 INFO:tasks.workunit.client.0.smithi022.stdout:2/795: mknod d0/d2/d2e/d82/da1/c10e 0 2024-02-16T19:29:20.814 INFO:tasks.workunit.client.1.smithi117.stdout:0/876: write d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/ffc [523740,13064] 0 2024-02-16T19:29:20.816 INFO:tasks.workunit.client.1.smithi117.stdout:1/779: link d2/d19/d59/d89/d69/l6b d2/d1d/d1f/d9c/l110 0 2024-02-16T19:29:20.817 INFO:tasks.workunit.client.0.smithi022.stdout:3/771: creat d7/d12/d18/dd/d3c/d5a/d67/f10d x:0 0 0 2024-02-16T19:29:20.817 INFO:tasks.workunit.client.1.smithi117.stdout:2/898: chown d0/d3/d40/fa3 59412 1 2024-02-16T19:29:20.817 INFO:tasks.workunit.client.1.smithi117.stdout:6/847: dread d5/f9 [0,4194304] 0 2024-02-16T19:29:20.818 INFO:tasks.workunit.client.1.smithi117.stdout:1/780: fdatasync d2/d19/d59/d89/d69/dad/fdb 0 2024-02-16T19:29:20.819 INFO:tasks.workunit.client.1.smithi117.stdout:6/848: chown d5/f15 2 1 2024-02-16T19:29:20.819 INFO:tasks.workunit.client.0.smithi022.stdout:3/772: rename d7/d12/d18/dd/d24/d93/de9/def to d7/d12/d18/dd/d24/d93/de9/def/d10e 22 2024-02-16T19:29:20.819 INFO:tasks.workunit.client.1.smithi117.stdout:6/849: stat d5/d4b/d20/d69/f91 0 2024-02-16T19:29:20.822 INFO:tasks.workunit.client.1.smithi117.stdout:4/876: creat d0/da/d21/d3d/d28/d64/d90/f120 x:0 0 0 2024-02-16T19:29:20.823 INFO:tasks.workunit.client.1.smithi117.stdout:8/901: sync 2024-02-16T19:29:20.823 INFO:tasks.workunit.client.1.smithi117.stdout:7/843: sync 2024-02-16T19:29:20.825 INFO:tasks.workunit.client.1.smithi117.stdout:2/899: creat d0/d13/d4d/d80/f11e x:0 0 0 2024-02-16T19:29:20.825 INFO:tasks.workunit.client.0.smithi022.stdout:5/853: rmdir dd/d1f/d48/d21/d5d/de7/d7a/d10f 0 2024-02-16T19:29:20.826 INFO:tasks.workunit.client.1.smithi117.stdout:4/877: dread d0/da/d21/fef [0,4194304] 0 2024-02-16T19:29:20.827 INFO:tasks.workunit.client.1.smithi117.stdout:1/781: stat d2/d19/cee 0 2024-02-16T19:29:20.828 INFO:tasks.workunit.client.1.smithi117.stdout:6/850: creat d5/d9c/f116 x:0 0 0 2024-02-16T19:29:20.829 INFO:tasks.workunit.client.1.smithi117.stdout:6/851: chown d5/d4b/f46 30960956 1 2024-02-16T19:29:20.829 INFO:tasks.workunit.client.0.smithi022.stdout:5/854: mkdir dd/d1f/d48/d21/d2f/d40/d11a 0 2024-02-16T19:29:20.830 INFO:tasks.workunit.client.1.smithi117.stdout:6/852: stat d5/db/d7d/d32/da8/de1/lbb 0 2024-02-16T19:29:20.833 INFO:tasks.workunit.client.0.smithi022.stdout:5/855: dread dd/d1f/d48/d21/d2f/d52/d77/d95/fb8 [0,4194304] 0 2024-02-16T19:29:20.835 INFO:tasks.workunit.client.1.smithi117.stdout:5/897: creat d2/d9/d3c/d24/d50/d96/dbc/d30/d38/d51/f12b x:0 0 0 2024-02-16T19:29:20.835 INFO:tasks.workunit.client.0.smithi022.stdout:3/773: link d7/d21/dd9/f107 d7/d12/d18/dac/dd7/f10f 0 2024-02-16T19:29:20.836 INFO:tasks.workunit.client.0.smithi022.stdout:4/742: write d4/d6/dd/d22/da8/dc4/f82 [686928,34055] 0 2024-02-16T19:29:20.837 INFO:tasks.workunit.client.1.smithi117.stdout:9/765: write dd/d47/d48/f55 [4147807,57489] 0 2024-02-16T19:29:20.838 INFO:tasks.workunit.client.0.smithi022.stdout:4/743: write d4/d6/d14/dc7/d6a/fa9 [739565,42751] 0 2024-02-16T19:29:20.840 INFO:tasks.workunit.client.1.smithi117.stdout:8/902: creat d4/d1c/f122 x:0 0 0 2024-02-16T19:29:20.841 INFO:tasks.workunit.client.0.smithi022.stdout:5/856: fsync dd/d1f/d48/d21/d2f/d40/d4e/d56/fa5 0 2024-02-16T19:29:20.841 INFO:tasks.workunit.client.0.smithi022.stdout:1/805: dwrite d6/d8/d22/d3a/f49 [0,4194304] 0 2024-02-16T19:29:20.843 INFO:tasks.workunit.client.1.smithi117.stdout:7/844: mknod d4/d1c/d87/d89/d60/d79/dc3/c132 0 2024-02-16T19:29:20.845 INFO:tasks.workunit.client.1.smithi117.stdout:9/766: dwrite dd/d47/d5b/d5f/d77/d7c/ff8 [0,4194304] 0 2024-02-16T19:29:20.848 INFO:tasks.workunit.client.0.smithi022.stdout:4/744: symlink d4/d6/dd/df/d10/d24/d6b/le9 0 2024-02-16T19:29:20.852 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:20 smithi117 ceph-mon[35309]: pgmap v13: 65 pgs: 65 active+clean; 2.9 GiB data, 8.4 GiB used, 528 GiB / 536 GiB avail; 82 MiB/s rd, 174 MiB/s wr, 395 op/s 2024-02-16T19:29:20.857 INFO:tasks.workunit.client.0.smithi022.stdout:5/857: truncate dd/d1f/d48/d21/dc9/fca 327504 0 2024-02-16T19:29:20.860 INFO:tasks.workunit.client.0.smithi022.stdout:3/774: write d7/d12/d18/dd/d3c/fa9 [1800817,100999] 0 2024-02-16T19:29:20.862 INFO:tasks.workunit.client.0.smithi022.stdout:3/775: chown d7/d12/d18/dd/fcb 221 1 2024-02-16T19:29:20.863 INFO:tasks.workunit.client.0.smithi022.stdout:4/745: rename d4/d6/d14/d64/d4e/d54/d5a/fb8 to d4/d6/d14/dc7/d6a/fea 0 2024-02-16T19:29:20.870 INFO:tasks.workunit.client.1.smithi117.stdout:4/878: write d0/da/d21/d3d/d28/d64/d48/f5d [1801983,14169] 0 2024-02-16T19:29:20.870 INFO:tasks.workunit.client.1.smithi117.stdout:4/879: chown d0 26839 1 2024-02-16T19:29:20.871 INFO:tasks.workunit.client.0.smithi022.stdout:5/858: creat dd/d1f/d48/d21/d2f/d52/d5c/d61/d8f/f11b x:0 0 0 2024-02-16T19:29:20.872 INFO:tasks.workunit.client.1.smithi117.stdout:1/782: dread d2/f53 [0,4194304] 0 2024-02-16T19:29:20.879 INFO:tasks.workunit.client.0.smithi022.stdout:1/806: truncate d6/d8/d22/d3a/f49 3570270 0 2024-02-16T19:29:20.882 INFO:tasks.workunit.client.0.smithi022.stdout:4/746: creat d4/d6/d14/d64/d4e/d54/d5a/dce/dda/feb x:0 0 0 2024-02-16T19:29:20.884 INFO:tasks.workunit.client.0.smithi022.stdout:3/776: write d7/d12/d18/dd/d24/d53/f5e [622063,14491] 0 2024-02-16T19:29:20.889 INFO:tasks.workunit.client.0.smithi022.stdout:3/777: fdatasync d7/d12/d18/dd/d24/d29/fb5 0 2024-02-16T19:29:20.891 INFO:tasks.workunit.client.0.smithi022.stdout:5/859: dwrite dd/d1f/d48/d21/d64/f116 [0,4194304] 0 2024-02-16T19:29:20.898 INFO:tasks.workunit.client.0.smithi022.stdout:5/860: symlink dd/d1f/d48/d21/d2f/d52/d5c/d61/d8f/l11c 0 2024-02-16T19:29:20.900 INFO:tasks.workunit.client.0.smithi022.stdout:4/747: getdents d4/d6/dd/d22/da8/dc4/d6d 0 2024-02-16T19:29:20.901 INFO:tasks.workunit.client.0.smithi022.stdout:4/748: fdatasync d4/d6/fbe 0 2024-02-16T19:29:20.901 INFO:tasks.workunit.client.0.smithi022.stdout:4/749: stat d4/d6/db6/c71 0 2024-02-16T19:29:20.905 INFO:tasks.workunit.client.0.smithi022.stdout:1/807: truncate d6/d8/d22/d16/d17/d5d/fe0 1993958 0 2024-02-16T19:29:20.907 INFO:tasks.workunit.client.0.smithi022.stdout:4/750: creat d4/d6/d14/dd2/fec x:0 0 0 2024-02-16T19:29:20.909 INFO:tasks.workunit.client.0.smithi022.stdout:1/808: chown d6/la 28107 1 2024-02-16T19:29:20.913 INFO:tasks.workunit.client.0.smithi022.stdout:4/751: dwrite d4/d6/dd/df/d10/f25 [0,4194304] 0 2024-02-16T19:29:20.915 INFO:tasks.workunit.client.0.smithi022.stdout:1/809: truncate d6/d8/d22/d67/f72 71200 0 2024-02-16T19:29:20.917 INFO:tasks.workunit.client.0.smithi022.stdout:4/752: symlink d4/d6/dd/df/d10/d24/d6b/led 0 2024-02-16T19:29:20.918 INFO:tasks.workunit.client.0.smithi022.stdout:5/861: dwrite dd/d1f/d48/f60 [0,4194304] 0 2024-02-16T19:29:20.921 INFO:tasks.workunit.client.0.smithi022.stdout:4/753: fdatasync d4/d6/dd/d22/f68 0 2024-02-16T19:29:20.923 INFO:tasks.workunit.client.0.smithi022.stdout:4/754: write d4/d6/dd/d22/da8/dc4/fc3 [607751,69502] 0 2024-02-16T19:29:20.925 INFO:tasks.workunit.client.0.smithi022.stdout:1/810: truncate d6/f5b 1536286 0 2024-02-16T19:29:20.933 INFO:tasks.workunit.client.0.smithi022.stdout:1/811: dwrite d6/d8/d22/d16/d17/d5d/fe1 [0,4194304] 0 2024-02-16T19:29:20.934 INFO:tasks.workunit.client.0.smithi022.stdout:1/812: read d6/d8/d22/d3a/f49 [2525354,61134] 0 2024-02-16T19:29:20.935 INFO:tasks.workunit.client.0.smithi022.stdout:1/813: chown d6/d8/d22/d59/fc4 1 1 2024-02-16T19:29:20.936 INFO:tasks.workunit.client.0.smithi022.stdout:1/814: stat d6/d8/d22/d3f 0 2024-02-16T19:29:20.938 INFO:tasks.workunit.client.1.smithi117.stdout:3/953: getdents d1/d30/d9f 0 2024-02-16T19:29:20.939 INFO:tasks.workunit.client.1.smithi117.stdout:3/954: write d1/dd/d3e/d7d/fbf [1421166,119503] 0 2024-02-16T19:29:20.940 INFO:tasks.workunit.client.1.smithi117.stdout:5/898: fsync d2/d3/f4d 0 2024-02-16T19:29:20.941 INFO:tasks.workunit.client.1.smithi117.stdout:5/899: stat d2/db3/f48 0 2024-02-16T19:29:20.941 INFO:tasks.workunit.client.0.smithi022.stdout:1/815: rmdir d6/d8/d22/d3f/d99 39 2024-02-16T19:29:20.941 INFO:tasks.workunit.client.1.smithi117.stdout:3/955: write d1/d1b/d1e/d37/ddc/f107 [438107,22176] 0 2024-02-16T19:29:20.942 INFO:tasks.workunit.client.0.smithi022.stdout:1/816: write d6/d8/d22/d47/db7/fdb [197483,129084] 0 2024-02-16T19:29:20.944 INFO:tasks.workunit.client.1.smithi117.stdout:8/903: rmdir d4 39 2024-02-16T19:29:20.950 INFO:tasks.workunit.client.1.smithi117.stdout:0/877: link d0/d9/d14/d1d/d60/d65/f10c d0/d9/d14/d1d/d60/dc3/dfd/f126 0 2024-02-16T19:29:20.953 INFO:tasks.workunit.client.1.smithi117.stdout:6/853: unlink d5/d4b/d44/d93/fc5 0 2024-02-16T19:29:20.954 INFO:tasks.workunit.client.1.smithi117.stdout:6/854: read - d5/d4b/d20/da4/fda zero size 2024-02-16T19:29:20.954 INFO:tasks.workunit.client.1.smithi117.stdout:4/880: creat d0/da/d21/d3d/d28/d64/d8a/d92/de8/f121 x:0 0 0 2024-02-16T19:29:20.959 INFO:tasks.workunit.client.1.smithi117.stdout:3/956: creat d1/d31/d89/de1/f148 x:0 0 0 2024-02-16T19:29:20.961 INFO:tasks.workunit.client.1.smithi117.stdout:8/904: fdatasync d4/d7/da6/d3a/d65/d96/fc7 0 2024-02-16T19:29:20.962 INFO:tasks.workunit.client.1.smithi117.stdout:9/767: truncate dd/d45/d31/f93 1046612 0 2024-02-16T19:29:20.963 INFO:tasks.workunit.client.1.smithi117.stdout:0/878: chown d0/d9/f1c 1849786161 1 2024-02-16T19:29:20.963 INFO:tasks.workunit.client.1.smithi117.stdout:9/768: write dd/d47/d5e/f87 [135634,405] 0 2024-02-16T19:29:20.964 INFO:tasks.workunit.client.0.smithi022.stdout:5/862: dwrite dd/d1f/d48/d21/d2f/d46/d5e/f7b [0,4194304] 0 2024-02-16T19:29:20.964 INFO:tasks.workunit.client.1.smithi117.stdout:2/900: link d0/d3/d34/d94/d82/lbb d0/d3/d34/d94/d46/d68/d76/da1/d56/d60/l11f 0 2024-02-16T19:29:20.964 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:20 smithi022 ceph-mon[29221]: pgmap v13: 65 pgs: 65 active+clean; 2.9 GiB data, 8.4 GiB used, 528 GiB / 536 GiB avail; 82 MiB/s rd, 174 MiB/s wr, 395 op/s 2024-02-16T19:29:20.966 INFO:tasks.workunit.client.0.smithi022.stdout:5/863: write dd/d1f/d48/d21/d2f/d52/fe6 [448776,4570] 0 2024-02-16T19:29:20.969 INFO:tasks.workunit.client.0.smithi022.stdout:4/755: write d4/d6/dd/d22/d83/f9f [59194,117424] 0 2024-02-16T19:29:20.975 INFO:tasks.workunit.client.0.smithi022.stdout:5/864: rename dd/d1f/d48/d21/d2f/d52/d5c/d61 to dd/d1f/d48/d21/d64/d11d 0 2024-02-16T19:29:20.980 INFO:tasks.workunit.client.1.smithi117.stdout:8/905: rename d4/d7/da6/d3a/d65/d115 to d4/d12/d2a/d64/d113/d123 0 2024-02-16T19:29:20.982 INFO:tasks.workunit.client.0.smithi022.stdout:1/817: write d6/d8/d22/d67/d7e/fdc [415792,33983] 0 2024-02-16T19:29:20.986 INFO:tasks.workunit.client.1.smithi117.stdout:0/879: dwrite d0/d9/d14/d23/f117 [0,4194304] 0 2024-02-16T19:29:20.987 INFO:tasks.workunit.client.1.smithi117.stdout:0/880: chown d0/d9/d14/dbc 974372398 1 2024-02-16T19:29:20.989 INFO:tasks.workunit.client.0.smithi022.stdout:5/865: link dd/d1f/c41 dd/d1f/d2e/dd4/c11e 0 2024-02-16T19:29:20.991 INFO:tasks.workunit.client.0.smithi022.stdout:1/818: truncate d6/d8/d22/d3f/d99/fb0 1038268 0 2024-02-16T19:29:20.992 INFO:tasks.workunit.client.1.smithi117.stdout:1/783: dwrite d2/d4/dbb/d7f/f84 [0,4194304] 0 2024-02-16T19:29:20.996 INFO:tasks.workunit.client.1.smithi117.stdout:5/900: dwrite d2/d3/d21/d49/f82 [0,4194304] 0 2024-02-16T19:29:20.997 INFO:tasks.workunit.client.1.smithi117.stdout:1/784: read d2/d1d/d60/f66 [1568228,49145] 0 2024-02-16T19:29:20.998 INFO:tasks.workunit.client.0.smithi022.stdout:5/866: dwrite dd/d1f/d48/d21/d64/f116 [0,4194304] 0 2024-02-16T19:29:20.999 INFO:tasks.workunit.client.0.smithi022.stdout:1/819: dread d6/d8/d22/d3a/d3e/f76 [0,4194304] 0 2024-02-16T19:29:21.007 INFO:tasks.workunit.client.1.smithi117.stdout:6/855: dwrite d5/d4b/d20/d69/f91 [0,4194304] 0 2024-02-16T19:29:21.009 INFO:tasks.workunit.client.1.smithi117.stdout:8/906: dwrite d4/d7/d6a/d72/d84/dda/ffb [0,4194304] 0 2024-02-16T19:29:21.009 INFO:tasks.workunit.client.1.smithi117.stdout:0/881: unlink d0/d9/d14/d1d/d1e/d54/f72 0 2024-02-16T19:29:21.012 INFO:tasks.workunit.client.1.smithi117.stdout:8/907: dread - d4/d12/d2a/d64/dec/d76/ff4 zero size 2024-02-16T19:29:21.013 INFO:tasks.workunit.client.1.smithi117.stdout:8/908: chown d4/l33 33 1 2024-02-16T19:29:21.015 INFO:tasks.workunit.client.0.smithi022.stdout:4/756: dwrite d4/d6/dd/d22/f68 [0,4194304] 0 2024-02-16T19:29:21.018 INFO:tasks.workunit.client.1.smithi117.stdout:3/957: link d1/d30/lf0 d1/d1b/d1e/d37/ddc/l149 0 2024-02-16T19:29:21.019 INFO:tasks.workunit.client.1.smithi117.stdout:4/881: dwrite d0/da/fa4 [0,4194304] 0 2024-02-16T19:29:21.022 INFO:tasks.workunit.client.1.smithi117.stdout:9/769: write dd/d45/d31/d63/dba/df0/daf/df4/fea [8802157,103703] 0 2024-02-16T19:29:21.024 INFO:tasks.workunit.client.1.smithi117.stdout:7/845: getdents d4/d1c/d39/d9f/de1 0 2024-02-16T19:29:21.025 INFO:tasks.workunit.client.1.smithi117.stdout:7/846: write d4/d1c/d39/d46/d55/fcc [2480623,100819] 0 2024-02-16T19:29:21.028 INFO:tasks.workunit.client.1.smithi117.stdout:6/856: creat d5/d4b/dd6/f117 x:0 0 0 2024-02-16T19:29:21.032 INFO:tasks.workunit.client.1.smithi117.stdout:8/909: rename d4/d73/f82 to d4/d12/d2a/d64/dec/d76/d8c/f124 0 2024-02-16T19:29:21.033 INFO:tasks.workunit.client.0.smithi022.stdout:4/757: dwrite d4/d6/db6/fb5 [0,4194304] 0 2024-02-16T19:29:21.034 INFO:tasks.workunit.client.1.smithi117.stdout:8/910: write d4/d7/da6/d3a/d65/d96/fc7 [2782764,33923] 0 2024-02-16T19:29:21.035 INFO:tasks.workunit.client.0.smithi022.stdout:8/890: sync 2024-02-16T19:29:21.035 INFO:tasks.workunit.client.1.smithi117.stdout:4/882: mknod d0/da/d21/d3d/d28/d66/d97/d6d/c122 0 2024-02-16T19:29:21.035 INFO:tasks.workunit.client.1.smithi117.stdout:8/911: stat d4/d7/d6a/d6f/f106 0 2024-02-16T19:29:21.037 INFO:tasks.workunit.client.1.smithi117.stdout:7/847: mknod d4/d1c/d39/d46/d47/c133 0 2024-02-16T19:29:21.037 INFO:tasks.workunit.client.0.smithi022.stdout:4/758: rmdir d4/d6/d14/dc7/d8d 39 2024-02-16T19:29:21.038 INFO:tasks.workunit.client.1.smithi117.stdout:5/901: dread d2/d5d/fc9 [0,4194304] 0 2024-02-16T19:29:21.039 INFO:tasks.workunit.client.1.smithi117.stdout:5/902: stat d2/d36/d6f/d9c 0 2024-02-16T19:29:21.043 INFO:tasks.workunit.client.1.smithi117.stdout:3/958: rename d1/d83/d96/f140 to d1/d30/d9f/dcb/d103/d134/f14a 0 2024-02-16T19:29:21.046 INFO:tasks.workunit.client.1.smithi117.stdout:4/883: symlink d0/da/d21/d3d/d28/d64/d73/d91/d9c/l123 0 2024-02-16T19:29:21.048 INFO:tasks.workunit.client.1.smithi117.stdout:7/848: creat d4/d1c/d39/d46/da4/f134 x:0 0 0 2024-02-16T19:29:21.049 INFO:tasks.workunit.client.1.smithi117.stdout:5/903: dread - d2/d3/d21/d2a/dcb/f108 zero size 2024-02-16T19:29:21.049 INFO:tasks.workunit.client.1.smithi117.stdout:4/884: write d0/da/d21/d3d/d28/d64/d73/d91/d9e/f112 [1410713,62794] 0 2024-02-16T19:29:21.050 INFO:tasks.workunit.client.1.smithi117.stdout:7/849: stat d4/d1c/d87/d89/d60/d11f/ceb 0 2024-02-16T19:29:21.050 INFO:tasks.workunit.client.1.smithi117.stdout:0/882: creat d0/d9/d14/d1d/d60/f127 x:0 0 0 2024-02-16T19:29:21.051 INFO:tasks.workunit.client.0.smithi022.stdout:4/759: dread d4/d6/d14/d64/d4e/d54/d5a/dce/fe1 [0,4194304] 0 2024-02-16T19:29:21.051 INFO:tasks.workunit.client.1.smithi117.stdout:9/770: link dd/d45/d31/d63/dba/df0/fce dd/d47/d5b/d5f/de6/f101 0 2024-02-16T19:29:21.052 INFO:tasks.workunit.client.1.smithi117.stdout:4/885: chown d0/da/d21/d3d/d28/d66/d97/l103 10 1 2024-02-16T19:29:21.055 INFO:tasks.workunit.client.1.smithi117.stdout:7/850: truncate d4/d1c/d39/d46/f75 2287382 0 2024-02-16T19:29:21.058 INFO:tasks.workunit.client.0.smithi022.stdout:4/760: getdents d4/d6/dd/d22/d30 0 2024-02-16T19:29:21.058 INFO:tasks.workunit.client.1.smithi117.stdout:4/886: dwrite d0/da/d21/d3d/d28/d64/f23 [0,4194304] 0 2024-02-16T19:29:21.060 INFO:tasks.workunit.client.1.smithi117.stdout:0/883: link d0/d9/d2c/da9/lf1 d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/d27/d31/d94/l128 0 2024-02-16T19:29:21.061 INFO:tasks.workunit.client.1.smithi117.stdout:9/771: getdents dd/d45/d73 0 2024-02-16T19:29:21.061 INFO:tasks.workunit.client.0.smithi022.stdout:4/761: creat d4/d6/dd/df/d10/de8/fee x:0 0 0 2024-02-16T19:29:21.065 INFO:tasks.workunit.client.0.smithi022.stdout:6/805: sync 2024-02-16T19:29:21.065 INFO:tasks.workunit.client.0.smithi022.stdout:7/812: sync 2024-02-16T19:29:21.065 INFO:tasks.workunit.client.0.smithi022.stdout:9/782: sync 2024-02-16T19:29:21.066 INFO:tasks.workunit.client.1.smithi117.stdout:9/772: symlink dd/d45/d31/l102 0 2024-02-16T19:29:21.066 INFO:tasks.workunit.client.0.smithi022.stdout:6/806: readlink d1/d8/dac/d101/da3/ldf 0 2024-02-16T19:29:21.068 INFO:tasks.workunit.client.1.smithi117.stdout:7/851: link d4/d1c/d98/c10a d4/d1c/d39/d46/da4/c135 0 2024-02-16T19:29:21.068 INFO:tasks.workunit.client.1.smithi117.stdout:7/852: fsync d4/dd3/fb9 0 2024-02-16T19:29:21.069 INFO:tasks.workunit.client.0.smithi022.stdout:6/807: dread d1/d8/dac/d101/fe9 [0,4194304] 0 2024-02-16T19:29:21.070 INFO:tasks.workunit.client.0.smithi022.stdout:6/808: readlink d1/d8/da/d29/deb/dfd/lbc 0 2024-02-16T19:29:21.070 INFO:tasks.workunit.client.0.smithi022.stdout:7/813: mknod d9/d5d/c112 0 2024-02-16T19:29:21.071 INFO:tasks.workunit.client.1.smithi117.stdout:9/773: mknod dd/d8d/c103 0 2024-02-16T19:29:21.071 INFO:tasks.workunit.client.1.smithi117.stdout:9/774: chown dd/f19 1235 1 2024-02-16T19:29:21.073 INFO:tasks.workunit.client.0.smithi022.stdout:4/762: symlink d4/d6/d14/d64/d4e/lef 0 2024-02-16T19:29:21.074 INFO:tasks.workunit.client.1.smithi117.stdout:7/853: chown d4/d1c/c41 890519780 1 2024-02-16T19:29:21.077 INFO:tasks.workunit.client.1.smithi117.stdout:7/854: mkdir d4/d1c/d39/d46/d136 0 2024-02-16T19:29:21.077 INFO:tasks.workunit.client.1.smithi117.stdout:7/855: chown d4/d1c/d39/fec 20857022 1 2024-02-16T19:29:21.079 INFO:tasks.workunit.client.0.smithi022.stdout:7/814: rename d9/d1b to d9/da/d1f/d44/d5b/d95/d10a/d113 0 2024-02-16T19:29:21.080 INFO:tasks.workunit.client.0.smithi022.stdout:7/815: chown d9/da/d2c/d2f/c1e 5 1 2024-02-16T19:29:21.080 INFO:tasks.workunit.client.1.smithi117.stdout:7/856: symlink d4/d1c/d39/df8/l137 0 2024-02-16T19:29:21.080 INFO:tasks.workunit.client.1.smithi117.stdout:7/857: stat d4/f34 0 2024-02-16T19:29:21.080 INFO:tasks.workunit.client.0.smithi022.stdout:4/763: mknod d4/d6/dd/df/d10/ddc/cf0 0 2024-02-16T19:29:21.094 INFO:tasks.workunit.client.0.smithi022.stdout:1/820: write d6/d8/d22/d47/d54/f8a [1735593,79450] 0 2024-02-16T19:29:21.096 INFO:tasks.workunit.client.0.smithi022.stdout:1/821: mknod d6/d8/d22/d3f/dc5/cfe 0 2024-02-16T19:29:21.098 INFO:tasks.workunit.client.0.smithi022.stdout:1/822: unlink d6/d8/d22/d47/d51/ffa 0 2024-02-16T19:29:21.104 INFO:tasks.workunit.client.0.smithi022.stdout:1/823: dwrite d6/d8/d22/d47/fcd [0,4194304] 0 2024-02-16T19:29:21.108 INFO:tasks.workunit.client.0.smithi022.stdout:1/824: creat d6/d8/d22/d3a/d7a/fff x:0 0 0 2024-02-16T19:29:21.109 INFO:tasks.workunit.client.0.smithi022.stdout:5/867: write dd/ff9 [1524850,35444] 0 2024-02-16T19:29:21.113 INFO:tasks.workunit.client.0.smithi022.stdout:1/825: mkdir d6/d8/d22/d3f/dc5/d100 0 2024-02-16T19:29:21.114 INFO:tasks.workunit.client.0.smithi022.stdout:5/868: truncate dd/d1f/d48/d21/d64/d11d/d6a/f70 2171491 0 2024-02-16T19:29:21.115 INFO:tasks.workunit.client.1.smithi117.stdout:1/785: write d2/d1d/d60/f3e [823162,97038] 0 2024-02-16T19:29:21.115 INFO:tasks.workunit.client.0.smithi022.stdout:1/826: readlink d6/d8/d22/d16/d17/l78 0 2024-02-16T19:29:21.117 INFO:tasks.workunit.client.0.smithi022.stdout:5/869: mknod dd/d1f/d48/d21/d2f/d46/c11f 0 2024-02-16T19:29:21.119 INFO:tasks.workunit.client.1.smithi117.stdout:2/901: sync 2024-02-16T19:29:21.120 INFO:tasks.workunit.client.0.smithi022.stdout:1/827: readlink d6/d8/d22/d16/d17/d26/lb2 0 2024-02-16T19:29:21.121 INFO:tasks.workunit.client.1.smithi117.stdout:1/786: dwrite d2/d4/dbb/d7f/f84 [4194304,4194304] 0 2024-02-16T19:29:21.122 INFO:tasks.workunit.client.0.smithi022.stdout:1/828: dread - d6/d8/d22/d16/d17/fd4 zero size 2024-02-16T19:29:21.124 INFO:tasks.workunit.client.1.smithi117.stdout:2/902: creat d0/d13/d4d/d53/d58/f120 x:0 0 0 2024-02-16T19:29:21.131 INFO:tasks.workunit.client.1.smithi117.stdout:6/857: dwrite d5/d4b/d20/d69/fc2 [0,4194304] 0 2024-02-16T19:29:21.134 INFO:tasks.workunit.client.1.smithi117.stdout:6/858: write d5/d4b/d71/dc7/d49/f88 [3986381,101910] 0 2024-02-16T19:29:21.134 INFO:tasks.workunit.client.1.smithi117.stdout:8/912: dwrite d4/d7/f3d [0,4194304] 0 2024-02-16T19:29:21.140 INFO:tasks.workunit.client.1.smithi117.stdout:6/859: creat d5/d4b/d71/dc7/d49/f118 x:0 0 0 2024-02-16T19:29:21.144 INFO:tasks.workunit.client.1.smithi117.stdout:4/887: write d0/f72 [138684,83132] 0 2024-02-16T19:29:21.147 INFO:tasks.workunit.client.1.smithi117.stdout:0/884: write d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/d27/d31/d94/f106 [640304,85050] 0 2024-02-16T19:29:21.148 INFO:tasks.workunit.client.1.smithi117.stdout:8/913: dwrite d4/d12/d31/f77 [0,4194304] 0 2024-02-16T19:29:21.148 INFO:tasks.workunit.client.1.smithi117.stdout:3/959: dwrite d1/d31/d89/de1/f109 [0,4194304] 0 2024-02-16T19:29:21.152 INFO:tasks.workunit.client.1.smithi117.stdout:5/904: dwrite d2/db3/db0/f43 [0,4194304] 0 2024-02-16T19:29:21.153 INFO:tasks.workunit.client.1.smithi117.stdout:0/885: write d0/d30/f66 [3435855,43760] 0 2024-02-16T19:29:21.157 INFO:tasks.workunit.client.0.smithi022.stdout:9/783: dwrite db/d14/f5e [4194304,4194304] 0 2024-02-16T19:29:21.158 INFO:tasks.workunit.client.1.smithi117.stdout:4/888: rmdir d0/da/d21/d3d 39 2024-02-16T19:29:21.159 INFO:tasks.workunit.client.1.smithi117.stdout:9/775: dwrite dd/d45/d31/d63/dba/df0/fa8 [0,4194304] 0 2024-02-16T19:29:21.159 INFO:tasks.workunit.client.1.smithi117.stdout:4/889: chown d0/l42 17973693 1 2024-02-16T19:29:21.160 INFO:tasks.workunit.client.0.smithi022.stdout:6/809: write d1/d8/d3c/f9d [1138556,50253] 0 2024-02-16T19:29:21.161 INFO:tasks.workunit.client.0.smithi022.stdout:9/784: write db/d14/d16/d28/fe2 [292186,129740] 0 2024-02-16T19:29:21.161 INFO:tasks.workunit.client.0.smithi022.stdout:6/810: readlink d1/l15 0 2024-02-16T19:29:21.161 INFO:tasks.workunit.client.1.smithi117.stdout:9/776: write dd/d47/d5b/fa4 [628078,92121] 0 2024-02-16T19:29:21.168 INFO:tasks.workunit.client.0.smithi022.stdout:4/764: write d4/d6/d3a/fc8 [19308,38716] 0 2024-02-16T19:29:21.168 INFO:tasks.workunit.client.1.smithi117.stdout:5/905: creat d2/d9/d3c/d24/d50/d103/f12c x:0 0 0 2024-02-16T19:29:21.170 INFO:tasks.workunit.client.1.smithi117.stdout:5/906: chown d2/d5d/f5e 22722179 1 2024-02-16T19:29:21.170 INFO:tasks.workunit.client.0.smithi022.stdout:9/785: creat db/d14/d16/d28/daa/f111 x:0 0 0 2024-02-16T19:29:21.170 INFO:tasks.workunit.client.0.smithi022.stdout:7/816: dwrite d9/da/d1f/d40/f87 [0,4194304] 0 2024-02-16T19:29:21.172 INFO:tasks.workunit.client.1.smithi117.stdout:7/858: dwrite d4/d1c/d87/fee [0,4194304] 0 2024-02-16T19:29:21.175 INFO:tasks.workunit.client.1.smithi117.stdout:3/960: mknod d1/dd/d5c/c14b 0 2024-02-16T19:29:21.176 INFO:tasks.workunit.client.1.smithi117.stdout:3/961: readlink d1/l4 0 2024-02-16T19:29:21.178 INFO:tasks.workunit.client.0.smithi022.stdout:3/778: sync 2024-02-16T19:29:21.178 INFO:tasks.workunit.client.0.smithi022.stdout:2/796: sync 2024-02-16T19:29:21.179 INFO:tasks.workunit.client.1.smithi117.stdout:0/886: creat d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/d27/df9/f129 x:0 0 0 2024-02-16T19:29:21.180 INFO:tasks.workunit.client.0.smithi022.stdout:7/817: dwrite d9/da/d1f/d40/f61 [0,4194304] 0 2024-02-16T19:29:21.182 INFO:tasks.workunit.client.0.smithi022.stdout:3/779: chown d7/d12/d18/dd/d24/d58/d70 1 1 2024-02-16T19:29:21.183 INFO:tasks.workunit.client.0.smithi022.stdout:3/780: readlink d7/d21/l40 0 2024-02-16T19:29:21.184 INFO:tasks.workunit.client.0.smithi022.stdout:4/765: creat d4/d6/d14/d64/d4e/d54/d5a/dce/dda/ff1 x:0 0 0 2024-02-16T19:29:21.190 INFO:tasks.workunit.client.0.smithi022.stdout:6/811: fsync d1/d8/da/d29/deb/dfd/f2d 0 2024-02-16T19:29:21.192 INFO:tasks.workunit.client.0.smithi022.stdout:9/786: mknod db/d14/d16/d27/c112 0 2024-02-16T19:29:21.193 INFO:tasks.workunit.client.0.smithi022.stdout:9/787: readlink db/d14/dc8/dc9/d105/le6 0 2024-02-16T19:29:21.194 INFO:tasks.workunit.client.1.smithi117.stdout:9/777: mknod dd/db5/dc4/c104 0 2024-02-16T19:29:21.195 INFO:tasks.workunit.client.0.smithi022.stdout:5/870: write dd/d1f/d48/d21/d2f/d40/d4e/f87 [780136,8461] 0 2024-02-16T19:29:21.198 INFO:tasks.workunit.client.1.smithi117.stdout:5/907: truncate d2/db3/f3f 2382774 0 2024-02-16T19:29:21.199 INFO:tasks.workunit.client.1.smithi117.stdout:7/859: rmdir d4/d1c/d39/df8 39 2024-02-16T19:29:21.200 INFO:tasks.workunit.client.0.smithi022.stdout:4/766: mkdir d4/d6/dd/d22/da8/dc4/d6d/df2 0 2024-02-16T19:29:21.201 INFO:tasks.workunit.client.1.smithi117.stdout:1/787: write d2/d1d/f29 [559829,54586] 0 2024-02-16T19:29:21.201 INFO:tasks.workunit.client.1.smithi117.stdout:3/962: symlink d1/d30/d9f/dcb/l14c 0 2024-02-16T19:29:21.203 INFO:tasks.workunit.client.1.smithi117.stdout:3/963: write d1/dd/d32/fc3 [2114121,14335] 0 2024-02-16T19:29:21.203 INFO:tasks.workunit.client.0.smithi022.stdout:1/829: write d6/d8/d22/d16/d17/d63/f68 [790522,7966] 0 2024-02-16T19:29:21.204 INFO:tasks.workunit.client.1.smithi117.stdout:0/887: mknod d0/d9/d14/d1d/d60/dc3/d109/c12a 0 2024-02-16T19:29:21.204 INFO:tasks.workunit.client.1.smithi117.stdout:3/964: fdatasync d1/dd/d5c/d85/d135/fe5 0 2024-02-16T19:29:21.205 INFO:tasks.workunit.client.0.smithi022.stdout:6/812: symlink d1/d8/da/d29/l115 0 2024-02-16T19:29:21.205 INFO:tasks.workunit.client.1.smithi117.stdout:3/965: write d1/dd/d5c/d85/d135/fe5 [936946,121285] 0 2024-02-16T19:29:21.206 INFO:tasks.workunit.client.1.smithi117.stdout:4/890: mknod d0/da/d21/d3d/d28/d64/d90/dc0/c124 0 2024-02-16T19:29:21.206 INFO:tasks.workunit.client.1.smithi117.stdout:3/966: write d1/dd/d3b/dba/f12c [2951605,21232] 0 2024-02-16T19:29:21.207 INFO:tasks.workunit.client.1.smithi117.stdout:2/903: truncate d0/d3/d22/d37/d9d/fe8 104587 0 2024-02-16T19:29:21.208 INFO:tasks.workunit.client.1.smithi117.stdout:4/891: readlink d0/da/d21/d3d/d28/d64/d73/d91/d9e/dbb/l100 0 2024-02-16T19:29:21.208 INFO:tasks.workunit.client.0.smithi022.stdout:9/788: creat db/d14/d16/d28/f113 x:0 0 0 2024-02-16T19:29:21.211 INFO:tasks.workunit.client.1.smithi117.stdout:5/908: fsync d2/d3/d21/d2a/dcb/f108 0 2024-02-16T19:29:21.213 INFO:tasks.workunit.client.0.smithi022.stdout:5/871: creat dd/d1f/d48/d21/d64/d11d/d6a/dfb/f120 x:0 0 0 2024-02-16T19:29:21.214 INFO:tasks.workunit.client.1.smithi117.stdout:7/860: symlink d4/d1c/d87/l138 0 2024-02-16T19:29:21.217 INFO:tasks.workunit.client.0.smithi022.stdout:1/830: rmdir d6/d8/d22/d16/d17/d1d 39 2024-02-16T19:29:21.220 INFO:tasks.workunit.client.0.smithi022.stdout:8/891: sync 2024-02-16T19:29:21.221 INFO:tasks.workunit.client.0.smithi022.stdout:6/813: symlink d1/d8/da/d29/deb/dfd/d85/d5b/l116 0 2024-02-16T19:29:21.221 INFO:tasks.workunit.client.1.smithi117.stdout:3/967: dread d1/d1b/d1e/d5e/f62 [0,4194304] 0 2024-02-16T19:29:21.222 INFO:tasks.workunit.client.0.smithi022.stdout:9/789: creat db/d14/d16/d28/f114 x:0 0 0 2024-02-16T19:29:21.224 INFO:tasks.workunit.client.1.smithi117.stdout:2/904: unlink d0/d3/d40/fa3 0 2024-02-16T19:29:21.225 INFO:tasks.workunit.client.0.smithi022.stdout:5/872: creat dd/d1f/d2e/dd4/f121 x:0 0 0 2024-02-16T19:29:21.226 INFO:tasks.workunit.client.0.smithi022.stdout:7/818: creat d9/da/f114 x:0 0 0 2024-02-16T19:29:21.227 INFO:tasks.workunit.client.0.smithi022.stdout:5/873: truncate dd/d1f/d76/fd6 687428 0 2024-02-16T19:29:21.228 INFO:tasks.workunit.client.0.smithi022.stdout:3/781: dread d7/d12/d18/dd/d24/d93/de9/f60 [0,4194304] 0 2024-02-16T19:29:21.229 INFO:tasks.workunit.client.0.smithi022.stdout:7/819: readlink d9/da/d1f/d44/d5b/dfc/l104 0 2024-02-16T19:29:21.231 INFO:tasks.workunit.client.0.smithi022.stdout:6/814: fsync d1/d8/da/d29/deb/dfd/d85/d56/f6a 0 2024-02-16T19:29:21.234 INFO:tasks.workunit.client.1.smithi117.stdout:3/968: mkdir d1/d83/d96/d14d 0 2024-02-16T19:29:21.234 INFO:tasks.workunit.client.0.smithi022.stdout:6/815: stat d1/d8/dac/d101/da3/dad 0 2024-02-16T19:29:21.235 INFO:tasks.workunit.client.0.smithi022.stdout:4/767: creat d4/d6/d14/d64/d4e/d54/ff3 x:0 0 0 2024-02-16T19:29:21.237 INFO:tasks.workunit.client.0.smithi022.stdout:8/892: dwrite d3/d12/db0/fb9 [0,4194304] 0 2024-02-16T19:29:21.239 INFO:tasks.workunit.client.0.smithi022.stdout:3/782: fsync d7/d21/fc2 0 2024-02-16T19:29:21.241 INFO:tasks.workunit.client.0.smithi022.stdout:7/820: mkdir d9/da/d2c/d46/de3/d115 0 2024-02-16T19:29:21.245 INFO:tasks.workunit.client.0.smithi022.stdout:1/831: link d6/d8/d22/d47/l9d d6/d8/d22/d3a/d3e/l101 0 2024-02-16T19:29:21.247 INFO:tasks.workunit.client.0.smithi022.stdout:8/893: mknod d3/dd3/c131 0 2024-02-16T19:29:21.249 INFO:tasks.workunit.client.0.smithi022.stdout:6/816: dread d1/d8/f66 [0,4194304] 0 2024-02-16T19:29:21.250 INFO:tasks.workunit.client.0.smithi022.stdout:4/768: truncate d4/d6/dd/d22/da8/dc4/d6d/f7d 981073 0 2024-02-16T19:29:21.251 INFO:tasks.workunit.client.0.smithi022.stdout:3/783: creat d7/d12/d18/dd/d98/f110 x:0 0 0 2024-02-16T19:29:21.251 INFO:tasks.workunit.client.0.smithi022.stdout:6/817: chown d1/d8/da/d29/deb/dfd/d53 145 1 2024-02-16T19:29:21.252 INFO:tasks.workunit.client.0.smithi022.stdout:7/821: mkdir d9/da/d4e/d7a/d116 0 2024-02-16T19:29:21.253 INFO:tasks.workunit.client.0.smithi022.stdout:3/784: chown d7/d12/d18/dd/d24/fd0 47 1 2024-02-16T19:29:21.255 INFO:tasks.workunit.client.0.smithi022.stdout:8/894: readlink d3/d25/d33/l122 0 2024-02-16T19:29:21.257 INFO:tasks.workunit.client.0.smithi022.stdout:7/822: symlink d9/da/d1f/d44/d5b/l117 0 2024-02-16T19:29:21.257 INFO:tasks.workunit.client.0.smithi022.stdout:6/818: dread d1/d8/dac/d101/db3/ff8 [0,4194304] 0 2024-02-16T19:29:21.262 INFO:tasks.workunit.client.0.smithi022.stdout:8/895: mkdir d3/d12/da6/d132 0 2024-02-16T19:29:21.264 INFO:tasks.workunit.client.0.smithi022.stdout:3/785: creat d7/d12/d18/dd/d3c/d5a/f111 x:0 0 0 2024-02-16T19:29:21.265 INFO:tasks.workunit.client.1.smithi117.stdout:3/969: dread d1/d31/f7e [0,4194304] 0 2024-02-16T19:29:21.266 INFO:tasks.workunit.client.1.smithi117.stdout:8/914: write d4/d73/f9f [605892,101406] 0 2024-02-16T19:29:21.267 INFO:tasks.workunit.client.0.smithi022.stdout:4/769: link d4/d6/dd/df/d10/d55/c5e d4/d6/dd/d22/d30/cf4 0 2024-02-16T19:29:21.267 INFO:tasks.workunit.client.1.smithi117.stdout:8/915: readlink d4/d7/d6a/d72/d84/dda/lea 0 2024-02-16T19:29:21.268 INFO:tasks.workunit.client.1.smithi117.stdout:6/860: dwrite d5/d4b/d44/f75 [0,4194304] 0 2024-02-16T19:29:21.269 INFO:tasks.workunit.client.1.smithi117.stdout:3/970: mkdir d1/d1b/d1e/d37/ddc/d14e 0 2024-02-16T19:29:21.269 INFO:tasks.workunit.client.0.smithi022.stdout:3/786: dread d7/d12/d18/dd/d24/d93/de9/f85 [0,4194304] 0 2024-02-16T19:29:21.269 INFO:tasks.workunit.client.1.smithi117.stdout:6/861: stat d5/db/d7d/d32/faf 0 2024-02-16T19:29:21.270 INFO:tasks.workunit.client.0.smithi022.stdout:3/787: write d7/d21/fbd [1243343,64091] 0 2024-02-16T19:29:21.273 INFO:tasks.workunit.client.0.smithi022.stdout:4/770: rename d4/d6/dd/df/d10/d55/lca to d4/d6/dd/d22/da8/dc4/lf5 0 2024-02-16T19:29:21.275 INFO:tasks.workunit.client.0.smithi022.stdout:8/896: dread d3/d12/d2b/da9/dd5/f11b [0,4194304] 0 2024-02-16T19:29:21.275 INFO:tasks.workunit.client.1.smithi117.stdout:3/971: creat d1/d31/d89/f14f x:0 0 0 2024-02-16T19:29:21.276 INFO:tasks.workunit.client.1.smithi117.stdout:8/916: dwrite d4/d12/d2a/d64/dec/d76/ff4 [0,4194304] 0 2024-02-16T19:29:21.278 INFO:tasks.workunit.client.1.smithi117.stdout:6/862: dwrite d5/d9c/daa/fb2 [0,4194304] 0 2024-02-16T19:29:21.278 INFO:tasks.workunit.client.1.smithi117.stdout:8/917: dread - d4/d12/d31/d8e/dbf/f100 zero size 2024-02-16T19:29:21.280 INFO:tasks.workunit.client.1.smithi117.stdout:6/863: write d5/d4b/d71/dc7/d49/d59/f10c [652709,114486] 0 2024-02-16T19:29:21.281 INFO:tasks.workunit.client.0.smithi022.stdout:3/788: rename d7/d21/dd9/l102 to d7/d12/d18/dd/d98/l112 0 2024-02-16T19:29:21.284 INFO:tasks.workunit.client.0.smithi022.stdout:4/771: fdatasync d4/d6/db6/fe5 0 2024-02-16T19:29:21.285 INFO:tasks.workunit.client.1.smithi117.stdout:3/972: dwrite d1/d31/f8c [0,4194304] 0 2024-02-16T19:29:21.288 INFO:tasks.workunit.client.1.smithi117.stdout:8/918: dread d4/d7/da6/d3a/d65/d96/fcb [0,4194304] 0 2024-02-16T19:29:21.289 INFO:tasks.workunit.client.1.smithi117.stdout:8/919: readlink d4/d7/d6a/d72/d84/dda/lea 0 2024-02-16T19:29:21.290 INFO:tasks.workunit.client.1.smithi117.stdout:8/920: write d4/d12/d2a/f110 [1014182,62942] 0 2024-02-16T19:29:21.291 INFO:tasks.workunit.client.1.smithi117.stdout:6/864: mknod d5/db/d7d/c119 0 2024-02-16T19:29:21.291 INFO:tasks.workunit.client.0.smithi022.stdout:2/797: write d0/d2/d18/d19/fab [536479,47176] 0 2024-02-16T19:29:21.296 INFO:tasks.workunit.client.1.smithi117.stdout:3/973: chown d1/d1b/d1e/d5e/f66 190 1 2024-02-16T19:29:21.296 INFO:tasks.workunit.client.0.smithi022.stdout:6/819: dread d1/d8/da/d29/deb/dfd/d48/d51/f59 [0,4194304] 0 2024-02-16T19:29:21.304 INFO:tasks.workunit.client.0.smithi022.stdout:3/789: rename d7/d12/d18/dd/d24/d58/f95 to d7/d12/d18/dd/d31/d7f/f113 0 2024-02-16T19:29:21.306 INFO:tasks.workunit.client.1.smithi117.stdout:9/778: write dd/d47/d5b/fbd [748961,95812] 0 2024-02-16T19:29:21.308 INFO:tasks.workunit.client.1.smithi117.stdout:6/865: rename d5/ld to d5/d4b/d44/db1/l11a 0 2024-02-16T19:29:21.310 INFO:tasks.workunit.client.0.smithi022.stdout:8/897: creat d3/d12/d17/f133 x:0 0 0 2024-02-16T19:29:21.311 INFO:tasks.workunit.client.1.smithi117.stdout:1/788: write d2/d1d/d60/fd3 [1000010,46893] 0 2024-02-16T19:29:21.313 INFO:tasks.workunit.client.1.smithi117.stdout:3/974: creat d1/d30/d9f/dcb/d103/d134/d12d/f150 x:0 0 0 2024-02-16T19:29:21.313 INFO:tasks.workunit.client.1.smithi117.stdout:0/888: write d0/d9/d14/d23/d33/fea [148143,74655] 0 2024-02-16T19:29:21.316 INFO:tasks.workunit.client.1.smithi117.stdout:6/866: fdatasync d5/d4b/d44/f56 0 2024-02-16T19:29:21.319 INFO:tasks.workunit.client.1.smithi117.stdout:3/975: dwrite d1/d30/d9f/dbc/d12f/f141 [0,4194304] 0 2024-02-16T19:29:21.321 INFO:tasks.workunit.client.0.smithi022.stdout:8/898: mknod d3/d12/c134 0 2024-02-16T19:29:21.321 INFO:tasks.workunit.client.0.smithi022.stdout:9/790: write db/d14/d16/d87/f8f [1119950,122389] 0 2024-02-16T19:29:21.323 INFO:tasks.workunit.client.0.smithi022.stdout:8/899: read d3/f2d [45704,1374] 0 2024-02-16T19:29:21.326 INFO:tasks.workunit.client.0.smithi022.stdout:6/820: getdents d1/d8/da/d29/deb/dfd/d48/de4 0 2024-02-16T19:29:21.326 INFO:tasks.workunit.client.1.smithi117.stdout:5/909: dwrite d2/d4c/d4f/d67/d11e/d62/fdb [0,4194304] 0 2024-02-16T19:29:21.328 INFO:tasks.workunit.client.1.smithi117.stdout:7/861: truncate d4/d13/d63/d71/f90 3954695 0 2024-02-16T19:29:21.333 INFO:tasks.workunit.client.1.smithi117.stdout:9/779: creat dd/d45/d31/d63/dba/df0/f105 x:0 0 0 2024-02-16T19:29:21.333 INFO:tasks.workunit.client.1.smithi117.stdout:5/910: dwrite d2/d9/d3c/d24/d50/d96/dc4/f11b [0,4194304] 0 2024-02-16T19:29:21.335 INFO:tasks.workunit.client.1.smithi117.stdout:0/889: mkdir d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/d27/d83/d7c/d12b 0 2024-02-16T19:29:21.337 INFO:tasks.workunit.client.1.smithi117.stdout:2/905: dwrite d0/d13/dc4/de4/fe5 [0,4194304] 0 2024-02-16T19:29:21.339 INFO:tasks.workunit.client.1.smithi117.stdout:3/976: symlink d1/d1b/d1e/d5e/l151 0 2024-02-16T19:29:21.342 INFO:tasks.workunit.client.1.smithi117.stdout:7/862: creat d4/d1c/d39/d46/dc4/f139 x:0 0 0 2024-02-16T19:29:21.345 INFO:tasks.workunit.client.1.smithi117.stdout:0/890: dwrite d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/d27/d31/d94/f10d [0,4194304] 0 2024-02-16T19:29:21.345 INFO:tasks.workunit.client.1.smithi117.stdout:9/780: dread dd/d45/d31/d63/dba/fcc [0,4194304] 0 2024-02-16T19:29:21.346 INFO:tasks.workunit.client.1.smithi117.stdout:5/911: rename d2/d9/d3c/d24/d50/d96/dbc/d30/d4e/cc3 to d2/d9/d3c/d24/d50/d103/c12d 0 2024-02-16T19:29:21.347 INFO:tasks.workunit.client.1.smithi117.stdout:9/781: chown dd/d45/d31/d63/dba/df0/daf/df4 10 1 2024-02-16T19:29:21.349 INFO:tasks.workunit.client.1.smithi117.stdout:3/977: fdatasync d1/dd/d3e/fd6 0 2024-02-16T19:29:21.349 INFO:tasks.workunit.client.1.smithi117.stdout:0/891: write d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/db7/fe8 [138135,118711] 0 2024-02-16T19:29:21.351 INFO:tasks.workunit.client.1.smithi117.stdout:7/863: unlink d4/d1c/d39/d46/d55/db8/l109 0 2024-02-16T19:29:21.351 INFO:tasks.workunit.client.1.smithi117.stdout:2/906: fsync d0/d3/d10/f1d 0 2024-02-16T19:29:21.353 INFO:tasks.workunit.client.1.smithi117.stdout:9/782: mkdir dd/d45/d31/d63/dba/df0/daf/d106 0 2024-02-16T19:29:21.355 INFO:tasks.workunit.client.1.smithi117.stdout:3/978: readlink d1/d1b/d1e/d37/ddc/lf6 0 2024-02-16T19:29:21.355 INFO:tasks.workunit.client.1.smithi117.stdout:5/912: dwrite d2/d3/fa8 [0,4194304] 0 2024-02-16T19:29:21.357 INFO:tasks.workunit.client.1.smithi117.stdout:0/892: symlink d0/d9/d14/d1d/d60/d65/l12c 0 2024-02-16T19:29:21.357 INFO:tasks.workunit.client.1.smithi117.stdout:7/864: creat d4/d1c/d87/df0/dfe/f13a x:0 0 0 2024-02-16T19:29:21.358 INFO:tasks.workunit.client.1.smithi117.stdout:2/907: mknod d0/d13/d4d/d80/c121 0 2024-02-16T19:29:21.359 INFO:tasks.workunit.client.1.smithi117.stdout:9/783: symlink dd/d45/d73/l107 0 2024-02-16T19:29:21.363 INFO:tasks.workunit.client.1.smithi117.stdout:7/865: symlink d4/d1c/d39/d46/da4/l13b 0 2024-02-16T19:29:21.364 INFO:tasks.workunit.client.1.smithi117.stdout:2/908: symlink d0/d3/d34/d94/d46/d68/d76/da1/d56/l122 0 2024-02-16T19:29:21.367 INFO:tasks.workunit.client.1.smithi117.stdout:9/784: link dd/d45/d31/d63/dba/df0/daf/df4/f91 dd/d45/d31/d63/f108 0 2024-02-16T19:29:21.367 INFO:tasks.workunit.client.1.smithi117.stdout:7/866: truncate d4/d1c/f2c 1278635 0 2024-02-16T19:29:21.368 INFO:tasks.workunit.client.1.smithi117.stdout:2/909: mknod d0/d13/d4d/d80/d110/c123 0 2024-02-16T19:29:21.368 INFO:tasks.workunit.client.1.smithi117.stdout:2/910: chown d0/d3/d22 0 1 2024-02-16T19:29:21.370 INFO:tasks.workunit.client.1.smithi117.stdout:9/785: creat dd/d47/d5b/d5f/d77/f109 x:0 0 0 2024-02-16T19:29:21.370 INFO:tasks.workunit.client.1.smithi117.stdout:7/867: chown d4/d13/d63/d71/f90 474305317 1 2024-02-16T19:29:21.371 INFO:tasks.workunit.client.1.smithi117.stdout:9/786: symlink dd/db5/l10a 0 2024-02-16T19:29:21.376 INFO:tasks.workunit.client.1.smithi117.stdout:9/787: dwrite dd/db5/dc4/dd3/fe3 [0,4194304] 0 2024-02-16T19:29:21.376 INFO:tasks.workunit.client.0.smithi022.stdout:9/791: mknod db/d14/d16/d49/c115 0 2024-02-16T19:29:21.377 INFO:tasks.workunit.client.1.smithi117.stdout:4/892: sync 2024-02-16T19:29:21.378 INFO:tasks.workunit.client.1.smithi117.stdout:4/893: chown d0/f5c 228078458 1 2024-02-16T19:29:21.378 INFO:tasks.workunit.client.1.smithi117.stdout:4/894: chown d0/da/d21/d3d/f29 3256615 1 2024-02-16T19:29:21.379 INFO:tasks.workunit.client.0.smithi022.stdout:5/874: write dd/d1f/d48/d21/d2f/d46/ff3 [7208870,111461] 0 2024-02-16T19:29:21.387 INFO:tasks.workunit.client.0.smithi022.stdout:5/875: stat dd/d1f/d48/d21/d2f/d46/d5e/fd3 0 2024-02-16T19:29:21.397 INFO:tasks.workunit.client.0.smithi022.stdout:1/832: dwrite d6/d8/d22/d59/da7/fd1 [0,4194304] 0 2024-02-16T19:29:21.402 INFO:tasks.workunit.client.0.smithi022.stdout:7/823: dwrite d9/da/f13 [4194304,4194304] 0 2024-02-16T19:29:21.402 INFO:tasks.workunit.client.0.smithi022.stdout:5/876: mkdir dd/d1f/d48/d21/d2f/d40/d122 0 2024-02-16T19:29:21.404 INFO:tasks.workunit.client.0.smithi022.stdout:2/798: dwrite d0/d2/d2e/fba [4194304,4194304] 0 2024-02-16T19:29:21.406 INFO:tasks.workunit.client.0.smithi022.stdout:5/877: dread - dd/d1f/d2e/dd4/f121 zero size 2024-02-16T19:29:21.407 INFO:tasks.workunit.client.0.smithi022.stdout:6/821: dread d1/d8/da/d29/deb/dfd/d53/fed [0,4194304] 0 2024-02-16T19:29:21.412 INFO:tasks.workunit.client.0.smithi022.stdout:6/822: chown d1/d8/f1c 10328 1 2024-02-16T19:29:21.421 INFO:tasks.workunit.client.0.smithi022.stdout:3/790: write d7/d12/d18/dd/d3c/fe2 [1000051,113284] 0 2024-02-16T19:29:21.423 INFO:tasks.workunit.client.0.smithi022.stdout:1/833: dread d6/f5b [0,4194304] 0 2024-02-16T19:29:21.429 INFO:tasks.workunit.client.0.smithi022.stdout:8/900: write d3/d12/d2b/d54/d97/fc2 [201621,111978] 0 2024-02-16T19:29:21.435 INFO:tasks.workunit.client.0.smithi022.stdout:9/792: dread db/d14/dc8/dc9/d105/f88 [4194304,4194304] 0 2024-02-16T19:29:21.438 INFO:tasks.workunit.client.1.smithi117.stdout:9/788: symlink dd/db5/dc4/l10b 0 2024-02-16T19:29:21.440 INFO:tasks.workunit.client.1.smithi117.stdout:8/921: truncate d4/d7/d6a/d72/fe5 89238 0 2024-02-16T19:29:21.441 INFO:tasks.workunit.client.1.smithi117.stdout:1/789: truncate d2/d1d/d60/d68/f81 2665642 0 2024-02-16T19:29:21.441 INFO:tasks.workunit.client.1.smithi117.stdout:4/895: chown d0/da/d21/d3d/d28/d64/d73/d91/d37/d98/cf6 228 1 2024-02-16T19:29:21.441 INFO:tasks.workunit.client.1.smithi117.stdout:9/789: chown dd/d47/d5b/d5f/d77/d7c/f8b 12668 1 2024-02-16T19:29:21.442 INFO:tasks.workunit.client.1.smithi117.stdout:5/913: rename d2/d9/d3c/d24/d50 to d2/d9/d12e 0 2024-02-16T19:29:21.444 INFO:tasks.workunit.client.1.smithi117.stdout:8/922: rmdir d4/d7/da6/d3a/d65/d7c 39 2024-02-16T19:29:21.445 INFO:tasks.workunit.client.1.smithi117.stdout:3/979: write d1/d30/d9f/dcb/fe7 [5209341,13080] 0 2024-02-16T19:29:21.448 INFO:tasks.workunit.client.1.smithi117.stdout:1/790: creat d2/d19/d59/d76/d91/f111 x:0 0 0 2024-02-16T19:29:21.451 INFO:tasks.workunit.client.1.smithi117.stdout:5/914: fsync d2/d9/d12e/d52/d74/f11c 0 2024-02-16T19:29:21.454 INFO:tasks.workunit.client.1.smithi117.stdout:4/896: dwrite d0/da/d21/d3d/d28/d64/d73/d91/d37/d98/fe7 [0,4194304] 0 2024-02-16T19:29:21.454 INFO:tasks.workunit.client.1.smithi117.stdout:6/867: dread d5/db/f6f [0,4194304] 0 2024-02-16T19:29:21.455 INFO:tasks.workunit.client.1.smithi117.stdout:5/915: chown d2/d5d/d8e/c10e 418351277 1 2024-02-16T19:29:21.455 INFO:tasks.workunit.client.1.smithi117.stdout:0/893: dwrite d0/d9/d14/d23/d33/f5c [0,4194304] 0 2024-02-16T19:29:21.457 INFO:tasks.workunit.client.1.smithi117.stdout:2/911: dread d0/d3/d34/fc6 [0,4194304] 0 2024-02-16T19:29:21.458 INFO:tasks.workunit.client.1.smithi117.stdout:6/868: dread d5/d4b/fe4 [0,4194304] 0 2024-02-16T19:29:21.460 INFO:tasks.workunit.client.0.smithi022.stdout:6/823: rmdir d1/d8/da/d29/d10e 39 2024-02-16T19:29:21.461 INFO:tasks.workunit.client.1.smithi117.stdout:6/869: dread d5/db/d7d/d32/f48 [0,4194304] 0 2024-02-16T19:29:21.465 INFO:tasks.workunit.client.1.smithi117.stdout:9/790: creat dd/d47/f10c x:0 0 0 2024-02-16T19:29:21.466 INFO:tasks.workunit.client.0.smithi022.stdout:2/799: rename d0/d32/f58 to d0/d2/d2e/d73/de3/f10f 0 2024-02-16T19:29:21.474 INFO:tasks.workunit.client.0.smithi022.stdout:3/791: dwrite d7/d12/d18/dd/d24/d58/d70/f7b [0,4194304] 0 2024-02-16T19:29:21.475 INFO:tasks.workunit.client.1.smithi117.stdout:4/897: rename d0/da/d21/d3d/d28/d64/d73/d91/f31 to d0/da/d4a/f125 0 2024-02-16T19:29:21.476 INFO:tasks.workunit.client.0.smithi022.stdout:8/901: fdatasync d3/d12/d6a/fb2 0 2024-02-16T19:29:21.478 INFO:tasks.workunit.client.1.smithi117.stdout:0/894: truncate d0/d9/d14/d1d/d60/d65/f10c 212391 0 2024-02-16T19:29:21.484 INFO:tasks.workunit.client.0.smithi022.stdout:6/824: symlink d1/d8/l117 0 2024-02-16T19:29:21.484 INFO:tasks.workunit.client.0.smithi022.stdout:8/902: dwrite d3/d12/db0/fd9 [0,4194304] 0 2024-02-16T19:29:21.485 INFO:tasks.workunit.client.1.smithi117.stdout:9/791: mkdir dd/d45/d31/d63/dba/df0/daf/df4/d10d 0 2024-02-16T19:29:21.485 INFO:tasks.workunit.client.0.smithi022.stdout:8/903: read - d3/d12/d2b/d54/d74/fd8 zero size 2024-02-16T19:29:21.486 INFO:tasks.workunit.client.1.smithi117.stdout:9/792: chown dd/d47/d5e/lae 2117720 1 2024-02-16T19:29:21.487 INFO:tasks.workunit.client.1.smithi117.stdout:9/793: chown dd/d47/d48/fbc 520686646 1 2024-02-16T19:29:21.488 INFO:tasks.workunit.client.1.smithi117.stdout:8/923: getdents d4/d12/d2a/d6d 0 2024-02-16T19:29:21.490 INFO:tasks.workunit.client.0.smithi022.stdout:1/834: mknod d6/d8/d22/d16/d17/c102 0 2024-02-16T19:29:21.493 INFO:tasks.workunit.client.0.smithi022.stdout:3/792: creat d7/d12/d18/dd/d3c/f114 x:0 0 0 2024-02-16T19:29:21.496 INFO:tasks.workunit.client.1.smithi117.stdout:4/898: truncate d0/da/d21/d3d/d28/d64/d73/d91/fa5 592452 0 2024-02-16T19:29:21.499 INFO:tasks.workunit.client.1.smithi117.stdout:9/794: symlink dd/d47/d5e/l10e 0 2024-02-16T19:29:21.500 INFO:tasks.workunit.client.0.smithi022.stdout:7/824: getdents d9/da/d2c 0 2024-02-16T19:29:21.501 INFO:tasks.workunit.client.1.smithi117.stdout:8/924: symlink d4/d12/d2a/d64/dec/d76/d8c/l125 0 2024-02-16T19:29:21.501 INFO:tasks.workunit.client.0.smithi022.stdout:7/825: readlink d9/da/d2c/l7c 0 2024-02-16T19:29:21.502 INFO:tasks.workunit.client.1.smithi117.stdout:4/899: dwrite d0/da/d21/d3d/d28/d64/d8a/d92/f85 [0,4194304] 0 2024-02-16T19:29:21.502 INFO:tasks.workunit.client.0.smithi022.stdout:6/825: mknod d1/d8/dac/c118 0 2024-02-16T19:29:21.504 INFO:tasks.workunit.client.0.smithi022.stdout:8/904: dread - d3/d12/d2b/d54/fcb zero size 2024-02-16T19:29:21.507 INFO:tasks.workunit.client.1.smithi117.stdout:5/916: dread d2/d9/f18 [4194304,4194304] 0 2024-02-16T19:29:21.507 INFO:tasks.workunit.client.1.smithi117.stdout:6/870: creat d5/d4b/d44/d93/f11b x:0 0 0 2024-02-16T19:29:21.508 INFO:tasks.workunit.client.1.smithi117.stdout:8/925: dwrite d4/d7/d6a/d72/d83/ff2 [0,4194304] 0 2024-02-16T19:29:21.509 INFO:tasks.workunit.client.1.smithi117.stdout:9/795: write dd/ff9 [3713658,94544] 0 2024-02-16T19:29:21.510 INFO:tasks.workunit.client.0.smithi022.stdout:3/793: fsync d7/d12/d18/dd/d3c/f65 0 2024-02-16T19:29:21.512 INFO:tasks.workunit.client.1.smithi117.stdout:4/900: symlink d0/da/d21/d3d/d28/d64/d90/dc0/l126 0 2024-02-16T19:29:21.513 INFO:tasks.workunit.client.1.smithi117.stdout:1/791: write d2/d1d/f39 [2870068,9519] 0 2024-02-16T19:29:21.520 INFO:tasks.workunit.client.0.smithi022.stdout:6/826: mknod d1/d8/dac/d101/da3/c119 0 2024-02-16T19:29:21.521 INFO:tasks.workunit.client.0.smithi022.stdout:5/878: write dd/d1f/d48/d21/d2f/d40/d49/d4b/f7e [216665,88565] 0 2024-02-16T19:29:21.522 INFO:tasks.workunit.client.1.smithi117.stdout:8/926: fdatasync d4/d7/da6/d3a/d43/d44/f4b 0 2024-02-16T19:29:21.524 INFO:tasks.workunit.client.0.smithi022.stdout:1/835: symlink d6/d8/d22/d16/d17/d63/d6f/l103 0 2024-02-16T19:29:21.528 INFO:tasks.workunit.client.1.smithi117.stdout:1/792: fsync d2/d1d/d60/f66 0 2024-02-16T19:29:21.533 INFO:tasks.workunit.client.0.smithi022.stdout:7/826: rename d9/da/d1f/d44/d5b/d95/d10a/d113/d23 to d9/d5d/dc7/d118 0 2024-02-16T19:29:21.533 INFO:tasks.workunit.client.0.smithi022.stdout:6/827: rename d1/d8 to d1/d8/dac/d101/db3/db0/d11a 22 2024-02-16T19:29:21.535 INFO:tasks.workunit.client.0.smithi022.stdout:6/828: dread - d1/d8/da/d29/deb/dfd/d85/d5b/f6c zero size 2024-02-16T19:29:21.536 INFO:tasks.workunit.client.0.smithi022.stdout:5/879: symlink dd/d1f/d2e/dd4/ddf/l123 0 2024-02-16T19:29:21.540 INFO:tasks.workunit.client.0.smithi022.stdout:7/827: dwrite d9/da9/dbc/fee [0,4194304] 0 2024-02-16T19:29:21.540 INFO:tasks.workunit.client.1.smithi117.stdout:9/796: creat dd/d45/d31/d63/dba/df0/daf/d106/f10f x:0 0 0 2024-02-16T19:29:21.541 INFO:tasks.workunit.client.1.smithi117.stdout:9/797: write dd/d47/d48/dc0/f4d [4988382,33944] 0 2024-02-16T19:29:21.541 INFO:tasks.workunit.client.0.smithi022.stdout:9/793: getdents db/d14/d16/d5d/d65/d6b/d89 0 2024-02-16T19:29:21.541 INFO:tasks.workunit.client.0.smithi022.stdout:7/828: chown d9/da/d1f/d44/d5b/d95/d10a/d113/f27 3 1 2024-02-16T19:29:21.543 INFO:tasks.workunit.client.0.smithi022.stdout:9/794: write db/d14/d16/d28/daa/f111 [803404,105041] 0 2024-02-16T19:29:21.547 INFO:tasks.workunit.client.0.smithi022.stdout:6/829: mknod d1/d8/da/d29/deb/dfd/d85/c11b 0 2024-02-16T19:29:21.555 INFO:tasks.workunit.client.1.smithi117.stdout:6/871: rename d5/db/d7d/d32/da8/l10b to d5/d4b/d44/d52/l11c 0 2024-02-16T19:29:21.555 INFO:tasks.workunit.client.1.smithi117.stdout:8/927: symlink d4/d7/da6/d3a/d65/d96/df0/l126 0 2024-02-16T19:29:21.558 INFO:tasks.workunit.client.1.smithi117.stdout:9/798: creat dd/d47/d5e/f110 x:0 0 0 2024-02-16T19:29:21.558 INFO:tasks.workunit.client.1.smithi117.stdout:5/917: dwrite d2/d36/fb2 [4194304,4194304] 0 2024-02-16T19:29:21.560 INFO:tasks.workunit.client.1.smithi117.stdout:4/901: link d0/da/d21/d3d/d28/d66/d97/l108 d0/da/d21/d3d/d28/d64/d73/d91/l127 0 2024-02-16T19:29:21.562 INFO:tasks.workunit.client.1.smithi117.stdout:6/872: unlink d5/db/d7d/d32/c9b 0 2024-02-16T19:29:21.563 INFO:tasks.workunit.client.1.smithi117.stdout:8/928: rename d4/d12/d31/d8e/l111 to d4/d94/l127 0 2024-02-16T19:29:21.564 INFO:tasks.workunit.client.1.smithi117.stdout:5/918: creat d2/d9/d3c/d24/d90/f12f x:0 0 0 2024-02-16T19:29:21.565 INFO:tasks.workunit.client.1.smithi117.stdout:4/902: mkdir d0/da/d21/d3d/d28/d64/d8a/d92/de8/d110/d128 0 2024-02-16T19:29:21.565 INFO:tasks.workunit.client.1.smithi117.stdout:9/799: dread dd/f4a [0,4194304] 0 2024-02-16T19:29:21.566 INFO:tasks.workunit.client.1.smithi117.stdout:5/919: stat d2/d9/d12e/d96/dbc/d30/c83 0 2024-02-16T19:29:21.566 INFO:tasks.workunit.client.1.smithi117.stdout:6/873: chown d5/c17 45145 1 2024-02-16T19:29:21.568 INFO:tasks.workunit.client.1.smithi117.stdout:5/920: dread - d2/db3/db0/d9b/f119 zero size 2024-02-16T19:29:21.569 INFO:tasks.workunit.client.1.smithi117.stdout:5/921: fdatasync d2/d3/f47 0 2024-02-16T19:29:21.570 INFO:tasks.workunit.client.1.smithi117.stdout:5/922: chown d2/d9/d12e/d52/d74/d8f 203 1 2024-02-16T19:29:21.571 INFO:tasks.workunit.client.1.smithi117.stdout:5/923: chown d2/d3/d21/d49/d59/d120 67290577 1 2024-02-16T19:29:21.572 INFO:tasks.workunit.client.1.smithi117.stdout:9/800: dread dd/d47/d5b/d5f/fde [0,4194304] 0 2024-02-16T19:29:21.572 INFO:tasks.workunit.client.1.smithi117.stdout:5/924: chown d2/d4c/dde/c102 28475 1 2024-02-16T19:29:21.574 INFO:tasks.workunit.client.1.smithi117.stdout:5/925: readlink d2/d3/d21/d49/d59/daa/lf5 0 2024-02-16T19:29:21.575 INFO:tasks.workunit.client.1.smithi117.stdout:5/926: fdatasync d2/d3/f6 0 2024-02-16T19:29:21.578 INFO:tasks.workunit.client.1.smithi117.stdout:8/929: dread d4/d12/d2a/d64/f7d [4194304,4194304] 0 2024-02-16T19:29:21.585 INFO:tasks.workunit.client.1.smithi117.stdout:8/930: dwrite d4/d12/d2a/d64/d113/d123/f11f [0,4194304] 0 2024-02-16T19:29:21.585 INFO:tasks.workunit.client.1.smithi117.stdout:9/801: dread dd/d45/d31/d63/dba/df0/f3b [0,4194304] 0 2024-02-16T19:29:21.586 INFO:tasks.workunit.client.0.smithi022.stdout:6/830: dread d1/d8/da/f14 [0,4194304] 0 2024-02-16T19:29:21.586 INFO:tasks.workunit.client.1.smithi117.stdout:8/931: stat d4/d12/d31/d8e/lab 0 2024-02-16T19:29:21.586 INFO:tasks.workunit.client.0.smithi022.stdout:8/905: write d3/d12/d2b/d46/f99 [213032,110427] 0 2024-02-16T19:29:21.588 INFO:tasks.workunit.client.0.smithi022.stdout:1/836: creat d6/f104 x:0 0 0 2024-02-16T19:29:21.591 INFO:tasks.workunit.client.1.smithi117.stdout:8/932: rmdir d4/d7/d6a/d6f 39 2024-02-16T19:29:21.593 INFO:tasks.workunit.client.0.smithi022.stdout:3/794: dwrite d7/d12/d18/dd/d31/d7f/fa0 [0,4194304] 0 2024-02-16T19:29:21.593 INFO:tasks.workunit.client.1.smithi117.stdout:8/933: rmdir d4/d7/da6/d3a 39 2024-02-16T19:29:21.594 INFO:tasks.workunit.client.1.smithi117.stdout:8/934: truncate d4/d12/d2a/f110 1599564 0 2024-02-16T19:29:21.594 INFO:tasks.workunit.client.0.smithi022.stdout:8/906: truncate d3/d12/d2b/da9/dd5/fff 1076836 0 2024-02-16T19:29:21.595 INFO:tasks.workunit.client.0.smithi022.stdout:3/795: chown d7/d12/d18/dd/d3c/fcc 1 1 2024-02-16T19:29:21.595 INFO:tasks.workunit.client.0.smithi022.stdout:9/795: truncate db/d14/d16/d21/d37/f63 1955731 0 2024-02-16T19:29:21.596 INFO:tasks.workunit.client.1.smithi117.stdout:6/874: dread d5/d4b/d20/d2d/f96 [0,4194304] 0 2024-02-16T19:29:21.600 INFO:tasks.workunit.client.0.smithi022.stdout:9/796: fdatasync db/d14/d16/d5d/f7e 0 2024-02-16T19:29:21.603 INFO:tasks.workunit.client.0.smithi022.stdout:8/907: rename d3/d12/d2b/d54/d74/da2 to d3/d12/d92/d135 0 2024-02-16T19:29:21.604 INFO:tasks.workunit.client.1.smithi117.stdout:7/868: sync 2024-02-16T19:29:21.606 INFO:tasks.workunit.client.0.smithi022.stdout:3/796: truncate d7/d12/d18/dac/f9d 392681 0 2024-02-16T19:29:21.607 INFO:tasks.workunit.client.1.smithi117.stdout:6/875: unlink d5/d4b/d20/c70 0 2024-02-16T19:29:21.608 INFO:tasks.workunit.client.1.smithi117.stdout:6/876: write d5/d9c/f116 [949247,49852] 0 2024-02-16T19:29:21.611 INFO:tasks.workunit.client.0.smithi022.stdout:9/797: dread - db/d14/d16/d21/d4a/fa4 zero size 2024-02-16T19:29:21.613 INFO:tasks.workunit.client.0.smithi022.stdout:3/797: chown d7/d12/d18/dd/d31/d7f/ceb 32354 1 2024-02-16T19:29:21.615 INFO:tasks.workunit.client.1.smithi117.stdout:7/869: mknod d4/d1c/d87/d89/d60/d79/dc3/d11d/c13c 0 2024-02-16T19:29:21.617 INFO:tasks.workunit.client.0.smithi022.stdout:9/798: truncate db/d14/d19/d1a/f24 111088 0 2024-02-16T19:29:21.619 INFO:tasks.workunit.client.0.smithi022.stdout:3/798: creat d7/d12/d18/dd/d3c/f115 x:0 0 0 2024-02-16T19:29:21.622 INFO:tasks.workunit.client.0.smithi022.stdout:9/799: fdatasync db/d14/d19/f5f 0 2024-02-16T19:29:21.622 INFO:tasks.workunit.client.0.smithi022.stdout:7/829: dwrite d9/da/f3a [0,4194304] 0 2024-02-16T19:29:21.624 INFO:tasks.workunit.client.0.smithi022.stdout:7/830: stat d9/da/d2c/d46/d73/fd6 0 2024-02-16T19:29:21.625 INFO:tasks.workunit.client.1.smithi117.stdout:7/870: symlink d4/d1c/l13d 0 2024-02-16T19:29:21.626 INFO:tasks.workunit.client.1.smithi117.stdout:1/793: truncate d2/d1d/f3a 7195779 0 2024-02-16T19:29:21.627 INFO:tasks.workunit.client.1.smithi117.stdout:1/794: write d2/d4/dbb/d7f/f84 [3655402,57988] 0 2024-02-16T19:29:21.628 INFO:tasks.workunit.client.1.smithi117.stdout:6/877: dread d5/db/d7d/d32/f92 [4194304,4194304] 0 2024-02-16T19:29:21.629 INFO:tasks.workunit.client.0.smithi022.stdout:9/800: creat db/d14/d16/d5d/d65/d6b/dd5/f116 x:0 0 0 2024-02-16T19:29:21.630 INFO:tasks.workunit.client.0.smithi022.stdout:3/799: read d7/f1e [2405195,79308] 0 2024-02-16T19:29:21.636 INFO:tasks.workunit.client.0.smithi022.stdout:3/800: unlink d7/d21/dd9/cea 0 2024-02-16T19:29:21.637 INFO:tasks.workunit.client.1.smithi117.stdout:6/878: mknod d5/d4b/d44/d7c/c11d 0 2024-02-16T19:29:21.638 INFO:tasks.workunit.client.1.smithi117.stdout:4/903: truncate d0/da/d21/d3d/d28/f51 90930 0 2024-02-16T19:29:21.639 INFO:tasks.workunit.client.1.smithi117.stdout:7/871: mkdir d4/d1c/d39/d13e 0 2024-02-16T19:29:21.640 INFO:tasks.workunit.client.1.smithi117.stdout:1/795: symlink d2/d19/d59/d89/d69/d80/de6/l112 0 2024-02-16T19:29:21.641 INFO:tasks.workunit.client.0.smithi022.stdout:3/801: stat d7/ca 0 2024-02-16T19:29:21.641 INFO:tasks.workunit.client.1.smithi117.stdout:6/879: dread d5/d9c/f9e [0,4194304] 0 2024-02-16T19:29:21.642 INFO:tasks.workunit.client.1.smithi117.stdout:4/904: creat d0/da/d21/db4/de4/f129 x:0 0 0 2024-02-16T19:29:21.643 INFO:tasks.workunit.client.1.smithi117.stdout:4/905: chown d0/da/d21/l71 1002747 1 2024-02-16T19:29:21.644 INFO:tasks.workunit.client.1.smithi117.stdout:7/872: mknod d4/d1c/d39/d46/c13f 0 2024-02-16T19:29:21.646 INFO:tasks.workunit.client.1.smithi117.stdout:9/802: dwrite dd/d45/d31/d63/d8e/dbb/fd2 [0,4194304] 0 2024-02-16T19:29:21.646 INFO:tasks.workunit.client.1.smithi117.stdout:1/796: creat d2/d19/d59/d89/d69/d80/de6/f113 x:0 0 0 2024-02-16T19:29:21.648 INFO:tasks.workunit.client.0.smithi022.stdout:1/837: dwrite d6/d8/d22/d47/fd3 [0,4194304] 0 2024-02-16T19:29:21.649 INFO:tasks.workunit.client.0.smithi022.stdout:7/831: rename d9/da/d1f/d40/c8f to d9/c119 0 2024-02-16T19:29:21.652 INFO:tasks.workunit.client.1.smithi117.stdout:8/935: write d4/d7/da6/d3a/d65/d7c/f10f [799537,11377] 0 2024-02-16T19:29:21.655 INFO:tasks.workunit.client.0.smithi022.stdout:9/801: dwrite db/d14/d16/d5d/f7e [4194304,4194304] 0 2024-02-16T19:29:21.657 INFO:tasks.workunit.client.0.smithi022.stdout:9/802: readlink db/d14/d16/d49/d90/ld7 0 2024-02-16T19:29:21.658 INFO:tasks.workunit.client.0.smithi022.stdout:9/803: chown db/d14/d16/d21/d4a/fed 4269 1 2024-02-16T19:29:21.668 INFO:tasks.workunit.client.0.smithi022.stdout:3/802: rename d7/d12/d18/dd/d24/d58/d6b to d7/d12/d18/dd/d24/d53/d116 0 2024-02-16T19:29:21.669 INFO:tasks.workunit.client.1.smithi117.stdout:4/906: mkdir d0/da/d21/d3d/d28/d66/d11a/d12a 0 2024-02-16T19:29:21.670 INFO:tasks.workunit.client.1.smithi117.stdout:9/803: dread dd/d47/d48/fbc [0,4194304] 0 2024-02-16T19:29:21.671 INFO:tasks.workunit.client.0.smithi022.stdout:1/838: symlink d6/d8/d22/d47/d54/l105 0 2024-02-16T19:29:21.671 INFO:tasks.workunit.client.1.smithi117.stdout:9/804: write dd/d47/f57 [5169437,57167] 0 2024-02-16T19:29:21.672 INFO:tasks.workunit.client.0.smithi022.stdout:1/839: write d6/d8/d22/d67/d7e/fe8 [920305,100945] 0 2024-02-16T19:29:21.673 INFO:tasks.workunit.client.1.smithi117.stdout:1/797: dread d2/d1d/f4f [0,4194304] 0 2024-02-16T19:29:21.673 INFO:tasks.workunit.client.0.smithi022.stdout:1/840: dread - d6/d8/dba/fce zero size 2024-02-16T19:29:21.679 INFO:tasks.workunit.client.1.smithi117.stdout:6/880: rename d5/d4b/d71/dc7/dce/cf1 to d5/d4b/d44/c11e 0 2024-02-16T19:29:21.680 INFO:tasks.workunit.client.0.smithi022.stdout:7/832: rename d9/da/d74/f99 to d9/da/d2c/d6d/f11a 0 2024-02-16T19:29:21.685 INFO:tasks.workunit.client.0.smithi022.stdout:1/841: rmdir d6/d8/d22/d16 39 2024-02-16T19:29:21.688 INFO:tasks.workunit.client.0.smithi022.stdout:9/804: creat db/d14/d10a/f117 x:0 0 0 2024-02-16T19:29:21.688 INFO:tasks.workunit.client.0.smithi022.stdout:4/772: sync 2024-02-16T19:29:21.689 INFO:tasks.workunit.client.0.smithi022.stdout:9/805: chown db/d14/d19/l34 21121 1 2024-02-16T19:29:21.690 INFO:tasks.workunit.client.0.smithi022.stdout:9/806: chown db/d14/d16/d5d/lac 16 1 2024-02-16T19:29:21.699 INFO:tasks.workunit.client.0.smithi022.stdout:1/842: unlink d6/d8/dba/dc7/fd2 0 2024-02-16T19:29:21.701 INFO:tasks.workunit.client.1.smithi117.stdout:9/805: dread dd/d45/d31/d63/dba/df0/f30 [0,4194304] 0 2024-02-16T19:29:21.706 INFO:tasks.workunit.client.0.smithi022.stdout:9/807: truncate db/d14/d19/d1a/f50 2987712 0 2024-02-16T19:29:21.707 INFO:tasks.workunit.client.0.smithi022.stdout:9/808: read db/f7b [2390425,72410] 0 2024-02-16T19:29:21.708 INFO:tasks.workunit.client.0.smithi022.stdout:3/803: write d7/d12/f44 [3579088,62867] 0 2024-02-16T19:29:21.711 INFO:tasks.workunit.client.1.smithi117.stdout:4/907: write d0/da/d21/d3d/d28/d66/d97/d6d/f95 [81393,86119] 0 2024-02-16T19:29:21.713 INFO:tasks.workunit.client.1.smithi117.stdout:7/873: rename d4/d1c/d87/d89/f10c to d4/d13/f140 0 2024-02-16T19:29:21.716 INFO:tasks.workunit.client.0.smithi022.stdout:4/773: truncate d4/d6/dd/df/d10/f42 1738886 0 2024-02-16T19:29:21.718 INFO:tasks.workunit.client.1.smithi117.stdout:1/798: dwrite d2/d19/d59/fd5 [0,4194304] 0 2024-02-16T19:29:21.719 INFO:tasks.workunit.client.0.smithi022.stdout:7/833: dwrite d9/da/d2c/d46/d73/fe1 [0,4194304] 0 2024-02-16T19:29:21.720 INFO:tasks.workunit.client.1.smithi117.stdout:3/980: sync 2024-02-16T19:29:21.723 INFO:tasks.workunit.client.0.smithi022.stdout:9/809: unlink db/d14/l32 0 2024-02-16T19:29:21.724 INFO:tasks.workunit.client.0.smithi022.stdout:9/810: read - db/d14/d16/d21/d37/f10c zero size 2024-02-16T19:29:21.726 INFO:tasks.workunit.client.1.smithi117.stdout:4/908: dread d0/da/d21/d3d/d28/d64/d73/d91/fa5 [0,4194304] 0 2024-02-16T19:29:21.729 INFO:tasks.workunit.client.1.smithi117.stdout:6/881: getdents d5 0 2024-02-16T19:29:21.731 INFO:tasks.workunit.client.1.smithi117.stdout:9/806: dwrite dd/d45/d31/d63/dba/df0/f1a [0,4194304] 0 2024-02-16T19:29:21.733 INFO:tasks.workunit.client.0.smithi022.stdout:7/834: mknod d9/da/d3b/c11b 0 2024-02-16T19:29:21.734 INFO:tasks.workunit.client.0.smithi022.stdout:1/843: dwrite d6/d8/d22/d3f/d99/f9b [0,4194304] 0 2024-02-16T19:29:21.736 INFO:tasks.workunit.client.0.smithi022.stdout:7/835: write d9/da/d2c/f10c [797128,79648] 0 2024-02-16T19:29:21.737 INFO:tasks.workunit.client.1.smithi117.stdout:3/981: rmdir d1/d31 39 2024-02-16T19:29:21.738 INFO:tasks.workunit.client.0.smithi022.stdout:7/836: truncate d9/da/d4e/d7a/fea 93866 0 2024-02-16T19:29:21.739 INFO:tasks.workunit.client.0.smithi022.stdout:7/837: dread - d9/da/f114 zero size 2024-02-16T19:29:21.741 INFO:tasks.workunit.client.0.smithi022.stdout:9/811: creat db/d14/d16/dec/dee/f118 x:0 0 0 2024-02-16T19:29:21.744 INFO:tasks.workunit.client.0.smithi022.stdout:1/844: fsync d6/d66/fb5 0 2024-02-16T19:29:21.746 INFO:tasks.workunit.client.0.smithi022.stdout:7/838: fdatasync d9/da/d1f/d93/fa2 0 2024-02-16T19:29:21.747 INFO:tasks.workunit.client.1.smithi117.stdout:1/799: mknod d2/d1d/d60/df2/c114 0 2024-02-16T19:29:21.747 INFO:tasks.workunit.client.0.smithi022.stdout:7/839: readlink d9/d5d/dc7/d118/l4d 0 2024-02-16T19:29:21.748 INFO:tasks.workunit.client.0.smithi022.stdout:9/812: mkdir db/d14/d16/dec/d119 0 2024-02-16T19:29:21.756 INFO:tasks.workunit.client.0.smithi022.stdout:3/804: dwrite d7/d21/dd9/fe3 [0,4194304] 0 2024-02-16T19:29:21.756 INFO:tasks.workunit.client.1.smithi117.stdout:1/800: rmdir d2/d19/d59/d89 39 2024-02-16T19:29:21.760 INFO:tasks.workunit.client.1.smithi117.stdout:3/982: creat d1/d31/d89/dbd/f152 x:0 0 0 2024-02-16T19:29:21.763 INFO:tasks.workunit.client.1.smithi117.stdout:2/912: sync 2024-02-16T19:29:21.763 INFO:tasks.workunit.client.1.smithi117.stdout:0/895: sync 2024-02-16T19:29:21.763 INFO:tasks.workunit.client.1.smithi117.stdout:5/927: sync 2024-02-16T19:29:21.763 INFO:tasks.workunit.client.1.smithi117.stdout:8/936: sync 2024-02-16T19:29:21.764 INFO:tasks.workunit.client.0.smithi022.stdout:1/845: rename d6/leb to d6/d8/d22/d67/d7e/l106 0 2024-02-16T19:29:21.764 INFO:tasks.workunit.client.1.smithi117.stdout:9/807: write dd/d45/d73/ff1 [341911,60391] 0 2024-02-16T19:29:21.766 INFO:tasks.workunit.client.1.smithi117.stdout:4/909: dwrite d0/da/d21/d3d/f5e [4194304,4194304] 0 2024-02-16T19:29:21.769 INFO:tasks.workunit.client.0.smithi022.stdout:7/840: write d9/d5d/dc7/d118/f96 [107317,17211] 0 2024-02-16T19:29:21.771 INFO:tasks.workunit.client.1.smithi117.stdout:8/937: truncate d4/f3c 4324307 0 2024-02-16T19:29:21.771 INFO:tasks.workunit.client.1.smithi117.stdout:1/801: fdatasync d2/d19/d59/d89/d69/d9a/fb3 0 2024-02-16T19:29:21.772 INFO:tasks.workunit.client.0.smithi022.stdout:3/805: unlink d7/le0 0 2024-02-16T19:29:21.773 INFO:tasks.workunit.client.1.smithi117.stdout:9/808: fdatasync dd/d45/d31/d63/dba/df0/daf/df4/f9f 0 2024-02-16T19:29:21.773 INFO:tasks.workunit.client.1.smithi117.stdout:6/882: dwrite d5/d4b/f7a [0,4194304] 0 2024-02-16T19:29:21.775 INFO:tasks.workunit.client.1.smithi117.stdout:2/913: unlink d0/d3/d34/d94/d91/l10c 0 2024-02-16T19:29:21.777 INFO:tasks.workunit.client.1.smithi117.stdout:8/938: fsync d4/d7/da6/d3a/d65/d7c/dc9/f118 0 2024-02-16T19:29:21.777 INFO:tasks.workunit.client.1.smithi117.stdout:3/983: dwrite d1/d30/d9f/dbc/d12f/f141 [0,4194304] 0 2024-02-16T19:29:21.779 INFO:tasks.workunit.client.1.smithi117.stdout:0/896: unlink d0/d9/d14/d23/d33/fd8 0 2024-02-16T19:29:21.780 INFO:tasks.workunit.client.1.smithi117.stdout:2/914: write d0/d3/d34/d94/d91/f107 [1002074,16120] 0 2024-02-16T19:29:21.780 INFO:tasks.workunit.client.1.smithi117.stdout:5/928: dread d2/d9/d12e/d96/dbc/d30/d38/d51/fe0 [0,4194304] 0 2024-02-16T19:29:21.785 INFO:tasks.workunit.client.0.smithi022.stdout:7/841: unlink d9/da/d4e/l94 0 2024-02-16T19:29:21.786 INFO:tasks.workunit.client.0.smithi022.stdout:7/842: fsync d9/da/d2c/fc1 0 2024-02-16T19:29:21.786 INFO:tasks.workunit.client.1.smithi117.stdout:4/910: write d0/da/d21/db4/de4/ffb [200011,9493] 0 2024-02-16T19:29:21.786 INFO:tasks.workunit.client.0.smithi022.stdout:7/843: chown d9/da9/dbc 0 1 2024-02-16T19:29:21.789 INFO:tasks.workunit.client.0.smithi022.stdout:9/813: link db/d14/d19/da8/c64 db/d14/d16/d5d/d65/d6b/df6/c11a 0 2024-02-16T19:29:21.790 INFO:tasks.workunit.client.0.smithi022.stdout:1/846: mkdir d6/d8/d22/d16/d17/d1d/d107 0 2024-02-16T19:29:21.791 INFO:tasks.workunit.client.0.smithi022.stdout:1/847: write d6/d8/f2d [5169556,90671] 0 2024-02-16T19:29:21.793 INFO:tasks.workunit.client.1.smithi117.stdout:9/809: fsync dd/d45/d31/d63/dba/df0/daf/df4/f5a 0 2024-02-16T19:29:21.794 INFO:tasks.workunit.client.0.smithi022.stdout:9/814: fsync db/d14/d19/fcf 0 2024-02-16T19:29:21.797 INFO:tasks.workunit.client.0.smithi022.stdout:1/848: creat d6/d8/dba/dc7/f108 x:0 0 0 2024-02-16T19:29:21.803 INFO:tasks.workunit.client.0.smithi022.stdout:3/806: dread d7/d12/d18/dd/d24/f3e [4194304,4194304] 0 2024-02-16T19:29:21.803 INFO:tasks.workunit.client.1.smithi117.stdout:8/939: rename d4/d7/c24 to d4/d12/d2a/d64/d113/d123/c128 0 2024-02-16T19:29:21.804 INFO:tasks.workunit.client.0.smithi022.stdout:3/807: fdatasync d7/d12/d18/dd/d31/fae 0 2024-02-16T19:29:21.804 INFO:tasks.workunit.client.1.smithi117.stdout:8/940: stat d4/d12/d2a/f71 0 2024-02-16T19:29:21.806 INFO:tasks.workunit.client.0.smithi022.stdout:9/815: truncate db/d14/d16/fe9 627486 0 2024-02-16T19:29:21.808 INFO:tasks.workunit.client.1.smithi117.stdout:4/911: mknod d0/da/d21/d3d/d28/d64/d8a/d92/c12b 0 2024-02-16T19:29:21.809 INFO:tasks.workunit.client.1.smithi117.stdout:4/912: chown d0/da/d21/d3d/d28/d64/d73/d91/d37/c5f 94 1 2024-02-16T19:29:21.809 INFO:tasks.workunit.client.1.smithi117.stdout:6/883: write d5/db/d7d/d32/d37/fa1 [603328,69094] 0 2024-02-16T19:29:21.811 INFO:tasks.workunit.client.0.smithi022.stdout:7/844: link d9/d5d/dc7/d118/l4d d9/da/d3b/l11c 0 2024-02-16T19:29:21.816 INFO:tasks.workunit.client.1.smithi117.stdout:9/810: mkdir dd/d45/d31/d63/dba/df0/daf/d111 0 2024-02-16T19:29:21.817 INFO:tasks.workunit.client.1.smithi117.stdout:3/984: mknod d1/dd/d3e/d7d/c153 0 2024-02-16T19:29:21.821 INFO:tasks.workunit.client.0.smithi022.stdout:7/845: creat d9/da/d1f/d40/dde/f11d x:0 0 0 2024-02-16T19:29:21.822 INFO:tasks.workunit.client.0.smithi022.stdout:3/808: write d7/d12/d18/f91 [248021,100565] 0 2024-02-16T19:29:21.825 INFO:tasks.workunit.client.1.smithi117.stdout:0/897: rename d0/d30/cae to d0/d9/d14/dbc/c12d 0 2024-02-16T19:29:21.826 INFO:tasks.workunit.client.0.smithi022.stdout:9/816: fdatasync db/d14/d16/d21/fb6 0 2024-02-16T19:29:21.827 INFO:tasks.workunit.client.0.smithi022.stdout:1/849: truncate d6/d8/f2d 5247829 0 2024-02-16T19:29:21.828 INFO:tasks.workunit.client.0.smithi022.stdout:9/817: dread - db/d14/d16/d5d/d65/d6b/d89/f10e zero size 2024-02-16T19:29:21.829 INFO:tasks.workunit.client.0.smithi022.stdout:9/818: chown db/d14/d16/d27/d8b 15613556 1 2024-02-16T19:29:21.831 INFO:tasks.workunit.client.1.smithi117.stdout:1/802: truncate d2/d19/d59/d89/f64 934803 0 2024-02-16T19:29:21.832 INFO:tasks.workunit.client.1.smithi117.stdout:4/913: mknod d0/da/d21/d3d/d28/d64/d73/d91/d37/c12c 0 2024-02-16T19:29:21.836 INFO:tasks.workunit.client.0.smithi022.stdout:7/846: dread d9/da9/dbc/fef [0,4194304] 0 2024-02-16T19:29:21.840 INFO:tasks.workunit.client.1.smithi117.stdout:8/941: write d4/f3f [2158139,42205] 0 2024-02-16T19:29:21.845 INFO:tasks.workunit.client.0.smithi022.stdout:3/809: link d7/d12/d18/d4b/fff d7/d12/d18/dd/d31/dbf/dc5/f117 0 2024-02-16T19:29:21.845 INFO:tasks.workunit.client.1.smithi117.stdout:3/985: write d1/f28 [4395231,29810] 0 2024-02-16T19:29:21.846 INFO:tasks.workunit.client.1.smithi117.stdout:3/986: readlink d1/d83/d96/lb9 0 2024-02-16T19:29:21.847 INFO:tasks.workunit.client.1.smithi117.stdout:1/803: creat d2/d19/d59/d89/d69/f115 x:0 0 0 2024-02-16T19:29:21.849 INFO:tasks.workunit.client.0.smithi022.stdout:7/847: unlink d9/da/d1f/d40/f5e 0 2024-02-16T19:29:21.853 INFO:tasks.workunit.client.0.smithi022.stdout:9/819: dwrite db/d14/d19/fa2 [4194304,4194304] 0 2024-02-16T19:29:21.854 INFO:tasks.workunit.client.0.smithi022.stdout:3/810: unlink d7/d12/d18/dac/c5f 0 2024-02-16T19:29:21.857 INFO:tasks.workunit.client.0.smithi022.stdout:7/848: rename d9/da/d1f/d93/f101 to d9/da/d2c/d46/de3/f11e 0 2024-02-16T19:29:21.858 INFO:tasks.workunit.client.0.smithi022.stdout:2/800: sync 2024-02-16T19:29:21.859 INFO:tasks.workunit.client.0.smithi022.stdout:1/850: dread d6/d8/d22/d47/d54/f62 [0,4194304] 0 2024-02-16T19:29:21.861 INFO:tasks.workunit.client.0.smithi022.stdout:1/851: stat d6/d8/dba/fce 0 2024-02-16T19:29:21.862 INFO:tasks.workunit.client.0.smithi022.stdout:2/801: write d0/d2/d18/f65 [256856,42041] 0 2024-02-16T19:29:21.862 INFO:tasks.workunit.client.0.smithi022.stdout:9/820: rmdir db/d14/d16/d5d/d65/d6b/df6 39 2024-02-16T19:29:21.862 INFO:tasks.workunit.client.1.smithi117.stdout:3/987: dread d1/f28 [0,4194304] 0 2024-02-16T19:29:21.863 INFO:tasks.workunit.client.0.smithi022.stdout:3/811: dwrite d7/d12/d18/dd/d31/dbf/fe7 [0,4194304] 0 2024-02-16T19:29:21.867 INFO:tasks.workunit.client.1.smithi117.stdout:6/884: write d5/d4b/d44/d7c/fe6 [15447,32400] 0 2024-02-16T19:29:21.869 INFO:tasks.workunit.client.1.smithi117.stdout:7/874: sync 2024-02-16T19:29:21.873 INFO:tasks.workunit.client.0.smithi022.stdout:1/852: dwrite d6/d8/d22/d67/ff7 [0,4194304] 0 2024-02-16T19:29:21.874 INFO:tasks.workunit.client.0.smithi022.stdout:2/802: dwrite d0/d2/d2e/d73/fd8 [0,4194304] 0 2024-02-16T19:29:21.885 INFO:tasks.workunit.client.0.smithi022.stdout:7/849: mkdir d9/da/d2c/d46/d73/dcc/d11f 0 2024-02-16T19:29:21.889 INFO:tasks.workunit.client.0.smithi022.stdout:2/803: dread d0/d2/f54 [0,4194304] 0 2024-02-16T19:29:21.893 INFO:tasks.workunit.client.0.smithi022.stdout:1/853: creat d6/d8/d22/d47/db7/f109 x:0 0 0 2024-02-16T19:29:21.898 INFO:tasks.workunit.client.0.smithi022.stdout:7/850: mkdir d9/da/d2c/d46/d73/dcc/d120 0 2024-02-16T19:29:21.898 INFO:tasks.workunit.client.0.smithi022.stdout:9/821: write db/d14/d19/fcf [724173,9844] 0 2024-02-16T19:29:21.900 INFO:tasks.workunit.client.0.smithi022.stdout:2/804: rmdir d0/d2/d18/d19/ddf 39 2024-02-16T19:29:21.901 INFO:tasks.workunit.client.0.smithi022.stdout:3/812: write d7/d12/d18/f4e [816172,33008] 0 2024-02-16T19:29:21.903 INFO:tasks.workunit.client.0.smithi022.stdout:2/805: write d0/d2/d18/d1f/d6c/d99/fc6 [4084361,21972] 0 2024-02-16T19:29:21.905 INFO:tasks.workunit.client.0.smithi022.stdout:9/822: rmdir db/d14/d10a 39 2024-02-16T19:29:21.907 INFO:tasks.workunit.client.0.smithi022.stdout:9/823: dread - db/d14/d16/d5d/d65/d6b/dd5/f116 zero size 2024-02-16T19:29:21.908 INFO:tasks.workunit.client.0.smithi022.stdout:2/806: dread d0/d2/d18/d19/fab [0,4194304] 0 2024-02-16T19:29:21.908 INFO:tasks.workunit.client.0.smithi022.stdout:3/813: dread d7/d12/d18/dac/dd7/fdb [0,4194304] 0 2024-02-16T19:29:21.913 INFO:tasks.workunit.client.1.smithi117.stdout:2/915: sync 2024-02-16T19:29:21.914 INFO:tasks.workunit.client.1.smithi117.stdout:2/916: chown d0/d13/dc4/d10f/l117 1188007788 1 2024-02-16T19:29:21.917 INFO:tasks.workunit.client.0.smithi022.stdout:9/824: dread db/d14/d16/fe9 [0,4194304] 0 2024-02-16T19:29:21.919 INFO:tasks.workunit.client.0.smithi022.stdout:7/851: creat d9/da/d2c/d46/d10b/f121 x:0 0 0 2024-02-16T19:29:21.925 INFO:tasks.workunit.client.0.smithi022.stdout:7/852: dwrite d9/da/f62 [0,4194304] 0 2024-02-16T19:29:21.927 INFO:tasks.workunit.client.1.smithi117.stdout:2/917: dread d0/d3/d10/f95 [4194304,4194304] 0 2024-02-16T19:29:21.929 INFO:tasks.workunit.client.0.smithi022.stdout:9/825: truncate db/d14/d16/fe9 387895 0 2024-02-16T19:29:21.932 INFO:tasks.workunit.client.0.smithi022.stdout:3/814: chown d7/d12/d18/dd/d24/d93/de9/def 78033 1 2024-02-16T19:29:21.934 INFO:tasks.workunit.client.0.smithi022.stdout:7/853: chown d9/d5d/dc7/d118/l4d 151906 1 2024-02-16T19:29:21.935 INFO:tasks.workunit.client.0.smithi022.stdout:2/807: dwrite d0/d2/d2e/f72 [0,4194304] 0 2024-02-16T19:29:21.944 INFO:tasks.workunit.client.0.smithi022.stdout:7/854: mkdir d9/da/d1f/d67/ddb/dd4/d122 0 2024-02-16T19:29:21.946 INFO:tasks.workunit.client.0.smithi022.stdout:3/815: creat d7/f118 x:0 0 0 2024-02-16T19:29:21.947 INFO:tasks.workunit.client.0.smithi022.stdout:3/816: dread - d7/f118 zero size 2024-02-16T19:29:21.948 INFO:tasks.workunit.client.0.smithi022.stdout:2/808: link d0/d2/d18/d19/fab d0/d5d/f110 0 2024-02-16T19:29:21.950 INFO:tasks.workunit.client.0.smithi022.stdout:9/826: dread db/d18/f2a [0,4194304] 0 2024-02-16T19:29:21.951 INFO:tasks.workunit.client.0.smithi022.stdout:7/855: rename d9/da/d2c/ff8 to d9/d5d/dc7/d118/f123 0 2024-02-16T19:29:21.952 INFO:tasks.workunit.client.0.smithi022.stdout:2/809: dread d0/d2/d18/d1f/f5e [0,4194304] 0 2024-02-16T19:29:21.954 INFO:tasks.workunit.client.0.smithi022.stdout:9/827: write db/d14/d16/d28/fd4 [275402,90667] 0 2024-02-16T19:29:21.956 INFO:tasks.workunit.client.0.smithi022.stdout:7/856: rmdir d9/d5d/dc7 39 2024-02-16T19:29:21.959 INFO:tasks.workunit.client.0.smithi022.stdout:9/828: rename db/d18/c98 to db/d14/d19/d1a/c11b 0 2024-02-16T19:29:21.960 INFO:tasks.workunit.client.0.smithi022.stdout:9/829: readlink db/d14/d19/l31 0 2024-02-16T19:29:21.963 INFO:tasks.workunit.client.0.smithi022.stdout:9/830: chown db/d14/d19/da8/c72 1635636986 1 2024-02-16T19:29:21.963 INFO:tasks.workunit.client.0.smithi022.stdout:3/817: getdents d7/d12/d18/dd/d24/d93/de9 0 2024-02-16T19:29:21.966 INFO:tasks.workunit.client.0.smithi022.stdout:7/857: link d9/da/d1f/d67/ddb/fd3 d9/f124 0 2024-02-16T19:29:21.967 INFO:tasks.workunit.client.0.smithi022.stdout:7/858: readlink d9/l98 0 2024-02-16T19:29:21.970 INFO:tasks.workunit.client.1.smithi117.stdout:5/929: getdents d2/d3/d21/d2a 0 2024-02-16T19:29:21.971 INFO:tasks.workunit.client.0.smithi022.stdout:3/818: dwrite d7/d12/d18/dd/d3c/d5a/ff6 [0,4194304] 0 2024-02-16T19:29:21.971 INFO:tasks.workunit.client.0.smithi022.stdout:9/831: dwrite db/d18/f2a [0,4194304] 0 2024-02-16T19:29:21.975 INFO:tasks.workunit.client.0.smithi022.stdout:3/819: write d7/d21/f96 [1540325,20142] 0 2024-02-16T19:29:21.976 INFO:tasks.workunit.client.0.smithi022.stdout:7/859: mkdir d9/da/d1f/d44/d5b/dfc/d125 0 2024-02-16T19:29:21.984 INFO:tasks.workunit.client.0.smithi022.stdout:7/860: dwrite d9/da/f13 [8388608,4194304] 0 2024-02-16T19:29:21.985 INFO:tasks.workunit.client.1.smithi117.stdout:6/885: symlink d5/d4b/d71/dc7/d49/l11f 0 2024-02-16T19:29:21.986 INFO:tasks.workunit.client.1.smithi117.stdout:6/886: write d5/d9c/daa/f9a [5347126,5746] 0 2024-02-16T19:29:21.987 INFO:tasks.workunit.client.1.smithi117.stdout:7/875: mkdir d4/d1c/d39/d46/d55/db8/d141 0 2024-02-16T19:29:21.991 INFO:tasks.workunit.client.1.smithi117.stdout:2/918: chown d0/d3/d34/d94/d46/d68/d76/da1/c3e 22 1 2024-02-16T19:29:21.992 INFO:tasks.workunit.client.1.smithi117.stdout:8/942: dread - d4/d7/d6a/d6f/f106 zero size 2024-02-16T19:29:21.994 INFO:tasks.workunit.client.1.smithi117.stdout:0/898: creat d0/d9/d14/d1d/d60/d76/d7b/d8d/f12e x:0 0 0 2024-02-16T19:29:21.995 INFO:tasks.workunit.client.0.smithi022.stdout:9/832: fsync db/d18/f2a 0 2024-02-16T19:29:21.997 INFO:tasks.workunit.client.1.smithi117.stdout:4/914: rmdir d0/da/d21/d3d/d28/d64/d73/d91/d9e/dbb/df0 0 2024-02-16T19:29:22.001 INFO:tasks.workunit.client.1.smithi117.stdout:6/887: rename d5/d9c/fdc to d5/d4b/d20/d69/f120 0 2024-02-16T19:29:22.001 INFO:tasks.workunit.client.1.smithi117.stdout:7/876: mkdir d4/d1c/d39/d46/da4/d142 0 2024-02-16T19:29:22.003 INFO:tasks.workunit.client.1.smithi117.stdout:4/915: dwrite d0/da/d21/d3d/d28/d64/d48/f4c [0,4194304] 0 2024-02-16T19:29:22.004 INFO:tasks.workunit.client.1.smithi117.stdout:7/877: truncate d4/d1c/d39/d46/d55/fcc 5015784 0 2024-02-16T19:29:22.004 INFO:tasks.workunit.client.1.smithi117.stdout:9/811: getdents dd/d47 0 2024-02-16T19:29:22.005 INFO:tasks.workunit.client.1.smithi117.stdout:4/916: write d0/da/d21/d3d/d28/f39 [5762814,91664] 0 2024-02-16T19:29:22.017 INFO:tasks.workunit.client.0.smithi022.stdout:7/861: rmdir d9/da/d2c/d46/de3/d115 0 2024-02-16T19:29:22.017 INFO:tasks.workunit.client.0.smithi022.stdout:7/862: readlink d9/da/d1f/la6 0 2024-02-16T19:29:22.017 INFO:tasks.workunit.client.0.smithi022.stdout:5/880: sync 2024-02-16T19:29:22.018 INFO:tasks.workunit.client.0.smithi022.stdout:5/881: readlink dd/d1f/d48/d21/d2f/d40/l90 0 2024-02-16T19:29:22.019 INFO:tasks.workunit.client.1.smithi117.stdout:6/888: creat d5/d4b/d71/f121 x:0 0 0 2024-02-16T19:29:22.023 INFO:tasks.workunit.client.1.smithi117.stdout:4/917: chown d0/da/d21/d3d/d28/d64/c18 898763657 1 2024-02-16T19:29:22.024 INFO:tasks.workunit.client.1.smithi117.stdout:4/918: chown d0/da/d21/d3d/d28/d64/d73/d91/lf3 30 1 2024-02-16T19:29:22.024 INFO:tasks.workunit.client.1.smithi117.stdout:7/878: creat d4/d1c/d39/d9f/de1/f143 x:0 0 0 2024-02-16T19:29:22.025 INFO:tasks.workunit.client.1.smithi117.stdout:9/812: mkdir dd/d45/d31/d63/d8e/dbb/d112 0 2024-02-16T19:29:22.027 INFO:tasks.workunit.client.1.smithi117.stdout:1/804: write d2/d4/dbb/d10b/fd0 [144472,119135] 0 2024-02-16T19:29:22.027 INFO:tasks.workunit.client.1.smithi117.stdout:1/805: fdatasync d2/d4/dbb/d7f/f84 0 2024-02-16T19:29:22.029 INFO:tasks.workunit.client.1.smithi117.stdout:3/988: getdents d1/d1b/d1e/d29/daf 0 2024-02-16T19:29:22.029 INFO:tasks.workunit.client.1.smithi117.stdout:3/989: dread - d1/d30/d9f/dbc/d12f/f138 zero size 2024-02-16T19:29:22.032 INFO:tasks.workunit.client.1.smithi117.stdout:2/919: rename d0/d3/d10/lbd to d0/d3/d34/d94/d46/d68/d93/l124 0 2024-02-16T19:29:22.033 INFO:tasks.workunit.client.0.smithi022.stdout:5/882: link dd/d1f/d48/d21/f25 dd/f124 0 2024-02-16T19:29:22.034 INFO:tasks.workunit.client.0.smithi022.stdout:5/883: readlink dd/d1f/d48/d21/d2f/d40/l90 0 2024-02-16T19:29:22.034 INFO:tasks.workunit.client.1.smithi117.stdout:5/930: getdents d2/d4c/d4f/dfd 0 2024-02-16T19:29:22.035 INFO:tasks.workunit.client.0.smithi022.stdout:5/884: stat dd/d1f/d48/d21/d2f/d46/d5e/f7b 0 2024-02-16T19:29:22.035 INFO:tasks.workunit.client.1.smithi117.stdout:5/931: read d2/d3/f4d [437465,124949] 0 2024-02-16T19:29:22.035 INFO:tasks.workunit.client.0.smithi022.stdout:5/885: chown dd/d1f/d48/f117 174 1 2024-02-16T19:29:22.037 INFO:tasks.workunit.client.0.smithi022.stdout:5/886: chown dd/d1f/d48/d21/d2f/d40/d49/d4b/cac 486901 1 2024-02-16T19:29:22.037 INFO:tasks.workunit.client.0.smithi022.stdout:5/887: stat dd/d1f/f9f 0 2024-02-16T19:29:22.040 INFO:tasks.workunit.client.1.smithi117.stdout:9/813: dread dd/d45/d31/d63/dba/df0/daf/df4/f61 [0,4194304] 0 2024-02-16T19:29:22.040 INFO:tasks.workunit.client.0.smithi022.stdout:5/888: symlink dd/d1f/d48/d21/d2f/d46/d5e/db6/l125 0 2024-02-16T19:29:22.043 INFO:tasks.workunit.client.1.smithi117.stdout:2/920: dread d0/d13/f15 [0,4194304] 0 2024-02-16T19:29:22.043 INFO:tasks.workunit.client.1.smithi117.stdout:6/889: rename d5/d4b/d44/d52/d81/dd7/l114 to d5/d4b/d71/dc7/d72/l122 0 2024-02-16T19:29:22.044 INFO:tasks.workunit.client.1.smithi117.stdout:8/943: getdents d4/d7/d6a 0 2024-02-16T19:29:22.047 INFO:tasks.workunit.client.1.smithi117.stdout:4/919: link d0/da/d21/d3d/d28/d64/d8a/d92/fff d0/da/d21/d3d/d28/d64/d73/d91/d9e/f12d 0 2024-02-16T19:29:22.048 INFO:tasks.workunit.client.0.smithi022.stdout:7/863: write d9/da/d3b/dba/ff5 [868949,71925] 0 2024-02-16T19:29:22.048 INFO:tasks.workunit.client.1.smithi117.stdout:4/920: chown d0/da/d21/d3d/d28/d64/d8a/d92/fdd 2 1 2024-02-16T19:29:22.050 INFO:tasks.workunit.client.1.smithi117.stdout:5/932: fdatasync d2/d9/d12e/d52/d74/fd2 0 2024-02-16T19:29:22.052 INFO:tasks.workunit.client.1.smithi117.stdout:1/806: unlink d2/dab/l108 0 2024-02-16T19:29:22.053 INFO:tasks.workunit.client.0.smithi022.stdout:7/864: mkdir d9/d5d/dc3/d126 0 2024-02-16T19:29:22.053 INFO:tasks.workunit.client.1.smithi117.stdout:0/899: write d0/d9/d2c/d8b/dcd/f11f [430561,79964] 0 2024-02-16T19:29:22.104 INFO:tasks.workunit.client.1.smithi117.stdout:3/990: creat d1/d30/d9f/dbc/d12f/d13d/f154 x:0 0 0 2024-02-16T19:29:22.107 INFO:tasks.workunit.client.1.smithi117.stdout:6/890: write d5/d9c/f9e [1752205,63906] 0 2024-02-16T19:29:22.111 INFO:tasks.workunit.client.1.smithi117.stdout:4/921: unlink d0/da/d21/caa 0 2024-02-16T19:29:22.116 INFO:tasks.workunit.client.1.smithi117.stdout:7/879: sync 2024-02-16T19:29:22.118 INFO:tasks.workunit.client.0.smithi022.stdout:5/889: dwrite dd/d1f/d48/d21/d2f/d46/d5e/d84/db4/fcf [4194304,4194304] 0 2024-02-16T19:29:22.118 INFO:tasks.workunit.client.1.smithi117.stdout:1/807: rename d2/d19/d59/d89/ld8 to d2/d19/d59/d89/d69/l116 0 2024-02-16T19:29:22.122 INFO:tasks.workunit.client.0.smithi022.stdout:7/865: write d9/da/d74/f80 [3905040,7816] 0 2024-02-16T19:29:22.124 INFO:tasks.workunit.client.0.smithi022.stdout:5/890: dread - dd/d1f/d48/d21/dc9/feb zero size 2024-02-16T19:29:22.127 INFO:tasks.workunit.client.1.smithi117.stdout:9/814: dwrite dd/d47/d5b/d5f/f6f [0,4194304] 0 2024-02-16T19:29:22.134 INFO:tasks.workunit.client.0.smithi022.stdout:7/866: mknod d9/da/d1f/d67/ddb/dd4/d122/c127 0 2024-02-16T19:29:22.134 INFO:tasks.workunit.client.1.smithi117.stdout:4/922: write d0/da/d4a/fa8 [691140,88047] 0 2024-02-16T19:29:22.135 INFO:tasks.workunit.client.0.smithi022.stdout:7/867: chown d9/da/d1f/d44/d5b/dfc/d125 12659 1 2024-02-16T19:29:22.140 INFO:tasks.workunit.client.1.smithi117.stdout:3/991: dwrite d1/d31/f11a [0,4194304] 0 2024-02-16T19:29:22.142 INFO:tasks.workunit.client.1.smithi117.stdout:6/891: write d5/db/d7d/d32/f92 [1107473,120582] 0 2024-02-16T19:29:22.143 INFO:tasks.workunit.client.0.smithi022.stdout:7/868: creat d9/da/f128 x:0 0 0 2024-02-16T19:29:22.144 INFO:tasks.workunit.client.1.smithi117.stdout:8/944: getdents d4 0 2024-02-16T19:29:22.144 INFO:tasks.workunit.client.1.smithi117.stdout:2/921: dwrite d0/d3/d34/d94/d46/d68/f8e [0,4194304] 0 2024-02-16T19:29:22.145 INFO:tasks.workunit.client.1.smithi117.stdout:4/923: chown d0/da/d21/d3d/d28/d64/d8a 44029080 1 2024-02-16T19:29:22.150 INFO:tasks.workunit.client.1.smithi117.stdout:5/933: dwrite d2/d5d/fc9 [0,4194304] 0 2024-02-16T19:29:22.150 INFO:tasks.workunit.client.1.smithi117.stdout:9/815: dwrite dd/d47/f10c [0,4194304] 0 2024-02-16T19:29:22.151 INFO:tasks.workunit.client.0.smithi022.stdout:5/891: dwrite dd/d1f/d48/d21/d2f/d40/d4e/f98 [0,4194304] 0 2024-02-16T19:29:22.152 INFO:tasks.workunit.client.1.smithi117.stdout:7/880: creat d4/f144 x:0 0 0 2024-02-16T19:29:22.153 INFO:tasks.workunit.client.0.smithi022.stdout:5/892: chown dd/d1f/d48/d21/d2f/d40/d49/lee 11055 1 2024-02-16T19:29:22.154 INFO:tasks.workunit.client.1.smithi117.stdout:0/900: dwrite d0/d9/d14/d1d/d60/d76/f9f [0,4194304] 0 2024-02-16T19:29:22.155 INFO:tasks.workunit.client.0.smithi022.stdout:7/869: rename d9/da/d4e/f89 to d9/da/d2c/ded/f129 0 2024-02-16T19:29:22.156 INFO:tasks.workunit.client.1.smithi117.stdout:2/922: truncate d0/d3/d22/d37/d109/f11b 965351 0 2024-02-16T19:29:22.171 INFO:tasks.workunit.client.0.smithi022.stdout:4/774: sync 2024-02-16T19:29:22.171 INFO:tasks.workunit.client.0.smithi022.stdout:8/908: sync 2024-02-16T19:29:22.171 INFO:tasks.workunit.client.0.smithi022.stdout:6/831: sync 2024-02-16T19:29:22.172 INFO:tasks.workunit.client.0.smithi022.stdout:8/909: dread - d3/d12/d17/f11a zero size 2024-02-16T19:29:22.178 INFO:tasks.workunit.client.1.smithi117.stdout:8/945: rename d4/d73/c9a to d4/d7/d6a/d72/de6/c129 0 2024-02-16T19:29:22.182 INFO:tasks.workunit.client.1.smithi117.stdout:3/992: symlink d1/dd/d3e/d7d/l155 0 2024-02-16T19:29:22.186 INFO:tasks.workunit.client.0.smithi022.stdout:6/832: mknod d1/d8/dac/d101/da3/c11c 0 2024-02-16T19:29:22.187 INFO:tasks.workunit.client.1.smithi117.stdout:7/881: mkdir d4/d1c/d39/d46/da4/d145 0 2024-02-16T19:29:22.187 INFO:tasks.workunit.client.1.smithi117.stdout:9/816: dread dd/d47/d5b/fc6 [0,4194304] 0 2024-02-16T19:29:22.189 INFO:tasks.workunit.client.1.smithi117.stdout:6/892: creat d5/d4b/d71/dc7/dce/ded/d100/f123 x:0 0 0 2024-02-16T19:29:22.193 INFO:tasks.workunit.client.1.smithi117.stdout:2/923: symlink d0/d3/d34/d94/d46/d68/d76/da1/d56/d60/dc9/l125 0 2024-02-16T19:29:22.195 INFO:tasks.workunit.client.1.smithi117.stdout:5/934: truncate d2/db3/f9f 4219002 0 2024-02-16T19:29:22.196 INFO:tasks.workunit.client.0.smithi022.stdout:5/893: write dd/d1f/d48/d21/d2f/d46/d5e/d84/fa3 [4577759,33709] 0 2024-02-16T19:29:22.196 INFO:tasks.workunit.client.1.smithi117.stdout:5/935: dread - d2/d3/d21/d2a/f93 zero size 2024-02-16T19:29:22.198 INFO:tasks.workunit.client.1.smithi117.stdout:4/924: rename d0/da/d21/d3d/d28/d64/d73/fdf to d0/da/d21/d3d/d28/d64/d73/d91/d37/d102/f12e 0 2024-02-16T19:29:22.199 INFO:tasks.workunit.client.1.smithi117.stdout:4/925: dread - d0/da/d21/d3d/d28/d64/d48/f104 zero size 2024-02-16T19:29:22.200 INFO:tasks.workunit.client.1.smithi117.stdout:8/946: symlink d4/d7/da6/d50/l12a 0 2024-02-16T19:29:22.202 INFO:tasks.workunit.client.0.smithi022.stdout:6/833: dread - d1/d8/da/d29/deb/dfd/d85/fe6 zero size 2024-02-16T19:29:22.205 INFO:tasks.workunit.client.0.smithi022.stdout:8/910: rename d3/d25/f69 to d3/d12/d2b/d54/d74/f136 0 2024-02-16T19:29:22.206 INFO:tasks.workunit.client.1.smithi117.stdout:1/808: sync 2024-02-16T19:29:22.210 INFO:tasks.workunit.client.1.smithi117.stdout:9/817: creat dd/d47/d5e/f113 x:0 0 0 2024-02-16T19:29:22.211 INFO:tasks.workunit.client.1.smithi117.stdout:9/818: dread - dd/d45/d73/ff7 zero size 2024-02-16T19:29:22.212 INFO:tasks.workunit.client.0.smithi022.stdout:7/870: getdents d9/da/d1f/d44 0 2024-02-16T19:29:22.213 INFO:tasks.workunit.client.0.smithi022.stdout:4/775: truncate d4/d6/dd/d22/da8/dc4/f82 1026852 0 2024-02-16T19:29:22.214 INFO:tasks.workunit.client.1.smithi117.stdout:6/893: mknod d5/d4b/d71/dc7/dce/ded/d100/c124 0 2024-02-16T19:29:22.216 INFO:tasks.workunit.client.0.smithi022.stdout:6/834: creat d1/d8/d81/f11d x:0 0 0 2024-02-16T19:29:22.219 INFO:tasks.workunit.client.1.smithi117.stdout:2/924: readlink d0/d13/dc4/dd9/l119 0 2024-02-16T19:29:22.220 INFO:tasks.workunit.client.1.smithi117.stdout:3/993: dwrite d1/d30/d9f/dbc/fc5 [0,4194304] 0 2024-02-16T19:29:22.220 INFO:tasks.workunit.client.0.smithi022.stdout:6/835: dread d1/d8/da/d29/deb/dfd/d53/f58 [0,4194304] 0 2024-02-16T19:29:22.223 INFO:tasks.workunit.client.0.smithi022.stdout:7/871: creat d9/da/d2c/d46/f12a x:0 0 0 2024-02-16T19:29:22.225 INFO:tasks.workunit.client.1.smithi117.stdout:8/947: symlink d4/d73/dff/l12b 0 2024-02-16T19:29:22.225 INFO:tasks.workunit.client.0.smithi022.stdout:5/894: dwrite dd/d1f/d48/d21/d2f/d46/d5e/d84/f5a [0,4194304] 0 2024-02-16T19:29:22.226 INFO:tasks.workunit.client.1.smithi117.stdout:2/925: dread d0/f4c [0,4194304] 0 2024-02-16T19:29:22.228 INFO:tasks.workunit.client.1.smithi117.stdout:8/948: read d4/d12/d2a/f110 [851066,105850] 0 2024-02-16T19:29:22.230 INFO:tasks.workunit.client.1.smithi117.stdout:0/901: dwrite d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/d27/f42 [0,4194304] 0 2024-02-16T19:29:22.230 INFO:tasks.workunit.client.1.smithi117.stdout:7/882: creat d4/d13/da3/f146 x:0 0 0 2024-02-16T19:29:22.236 INFO:tasks.workunit.client.1.smithi117.stdout:9/819: symlink dd/d45/d31/d63/d8e/l114 0 2024-02-16T19:29:22.240 INFO:tasks.workunit.client.1.smithi117.stdout:6/894: rename d5/d4b/d71/dc7/d49/d59 to d5/d4b/d44/d7c/d125 0 2024-02-16T19:29:22.241 INFO:tasks.workunit.client.0.smithi022.stdout:4/776: dwrite d4/f23 [8388608,4194304] 0 2024-02-16T19:29:22.243 INFO:tasks.workunit.client.0.smithi022.stdout:7/872: chown d9/da/d2c/f8d 3471 1 2024-02-16T19:29:22.243 INFO:tasks.workunit.client.1.smithi117.stdout:3/994: creat d1/dd/d3e/f156 x:0 0 0 2024-02-16T19:29:22.244 INFO:tasks.workunit.client.0.smithi022.stdout:8/911: link d3/d12/d2b/d46/f91 d3/d12/d2b/dca/f137 0 2024-02-16T19:29:22.247 INFO:tasks.workunit.client.0.smithi022.stdout:4/777: creat d4/d6/dd/d22/da8/dc4/d86/ff6 x:0 0 0 2024-02-16T19:29:22.249 INFO:tasks.workunit.client.1.smithi117.stdout:8/949: mknod d4/d12/d2a/d64/d113/c12c 0 2024-02-16T19:29:22.253 INFO:tasks.workunit.client.1.smithi117.stdout:9/820: dread dd/db5/fa0 [0,4194304] 0 2024-02-16T19:29:22.254 INFO:tasks.workunit.client.0.smithi022.stdout:4/778: dwrite d4/d6/dd/d22/f59 [4194304,4194304] 0 2024-02-16T19:29:22.254 INFO:tasks.workunit.client.1.smithi117.stdout:0/902: creat d0/d9/d2c/d8b/de0/f12f x:0 0 0 2024-02-16T19:29:22.255 INFO:tasks.workunit.client.0.smithi022.stdout:4/779: chown d4/d6/d14/dc7/d6a/cd4 995183 1 2024-02-16T19:29:22.256 INFO:tasks.workunit.client.1.smithi117.stdout:1/809: fdatasync d2/d19/d59/d89/f2a 0 2024-02-16T19:29:22.261 INFO:tasks.workunit.client.0.smithi022.stdout:8/912: mkdir d3/d25/d138 0 2024-02-16T19:29:22.263 INFO:tasks.workunit.client.0.smithi022.stdout:6/836: dwrite d1/d8/da/d29/deb/dfd/f2c [0,4194304] 0 2024-02-16T19:29:22.268 INFO:tasks.workunit.client.1.smithi117.stdout:3/995: dread d1/dd/d3b/dba/de9/fed [0,4194304] 0 2024-02-16T19:29:22.272 INFO:tasks.workunit.client.1.smithi117.stdout:8/950: unlink d4/d1c/lc1 0 2024-02-16T19:29:22.273 INFO:tasks.workunit.client.0.smithi022.stdout:7/873: dwrite d9/da/d1f/d67/db3/fb4 [0,4194304] 0 2024-02-16T19:29:22.273 INFO:tasks.workunit.client.0.smithi022.stdout:4/780: write d4/d6/dd/d22/da8/dc4/d6d/f7e [2085480,101711] 0 2024-02-16T19:29:22.273 INFO:tasks.workunit.client.1.smithi117.stdout:8/951: write d4/d12/d31/f40 [4749642,128182] 0 2024-02-16T19:29:22.274 INFO:tasks.workunit.client.1.smithi117.stdout:7/883: mkdir d4/d1c/d147 0 2024-02-16T19:29:22.275 INFO:tasks.workunit.client.1.smithi117.stdout:8/952: write d4/d12/d2a/d64/d113/d123/f11f [5000400,53907] 0 2024-02-16T19:29:22.276 INFO:tasks.workunit.client.1.smithi117.stdout:9/821: symlink dd/d8d/l115 0 2024-02-16T19:29:22.278 INFO:tasks.workunit.client.1.smithi117.stdout:0/903: creat d0/d9/d14/d1d/d60/d76/d7b/d8d/d55/f130 x:0 0 0 2024-02-16T19:29:22.280 INFO:tasks.workunit.client.0.smithi022.stdout:8/913: link d3/d12/d92/f10e d3/d12/d2b/d54/d97/f139 0 2024-02-16T19:29:22.283 INFO:tasks.workunit.client.0.smithi022.stdout:6/837: write d1/d8/ff [1292979,130480] 0 2024-02-16T19:29:22.283 INFO:tasks.workunit.client.0.smithi022.stdout:4/781: fdatasync d4/d6/d14/dc7/f3d 0 2024-02-16T19:29:22.284 INFO:tasks.workunit.client.0.smithi022.stdout:6/838: chown d1/d8/d81/f11d 15 1 2024-02-16T19:29:22.285 INFO:tasks.workunit.client.0.smithi022.stdout:6/839: stat d1/ld2 0 2024-02-16T19:29:22.288 INFO:tasks.workunit.client.1.smithi117.stdout:6/895: unlink d5/db/d7d/d32/da8/cdb 0 2024-02-16T19:29:22.290 INFO:tasks.workunit.client.0.smithi022.stdout:7/874: rename d9/da/d2c/d65/le8 to d9/da/d1f/d67/db3/l12b 0 2024-02-16T19:29:22.291 INFO:tasks.workunit.client.0.smithi022.stdout:7/875: chown d9/da/d1f/d44/d5b/l117 1848 1 2024-02-16T19:29:22.291 INFO:tasks.workunit.client.1.smithi117.stdout:5/936: sync 2024-02-16T19:29:22.293 INFO:tasks.workunit.client.1.smithi117.stdout:2/926: creat d0/d3/f126 x:0 0 0 2024-02-16T19:29:22.297 INFO:tasks.workunit.client.0.smithi022.stdout:8/914: symlink d3/d12/d2b/d54/l13a 0 2024-02-16T19:29:22.301 INFO:tasks.workunit.client.1.smithi117.stdout:7/884: mkdir d4/d1c/d98/def/d148 0 2024-02-16T19:29:22.302 INFO:tasks.workunit.client.0.smithi022.stdout:7/876: fsync d9/da/d2c/d46/f6a 0 2024-02-16T19:29:22.305 INFO:tasks.workunit.client.0.smithi022.stdout:6/840: mkdir d1/d8/dac/d101/db3/db0/d11e 0 2024-02-16T19:29:22.307 INFO:tasks.workunit.client.0.smithi022.stdout:4/782: dread d4/d6/d14/dc7/d8d/f99 [8388608,4194304] 0 2024-02-16T19:29:22.309 INFO:tasks.workunit.client.0.smithi022.stdout:8/915: unlink d3/d12/d17/fbe 0 2024-02-16T19:29:22.310 INFO:tasks.workunit.client.1.smithi117.stdout:0/904: mknod d0/d9/d14/d1d/d60/d65/c131 0 2024-02-16T19:29:22.314 INFO:tasks.workunit.client.0.smithi022.stdout:8/916: dread d3/feb [0,4194304] 0 2024-02-16T19:29:22.314 INFO:tasks.workunit.client.0.smithi022.stdout:4/783: dwrite d4/d6/dd/df/dc1/fd5 [0,4194304] 0 2024-02-16T19:29:22.316 INFO:tasks.workunit.client.0.smithi022.stdout:7/877: truncate d9/da/d1f/d44/d5b/dfc/fe2 905363 0 2024-02-16T19:29:22.316 INFO:tasks.workunit.client.1.smithi117.stdout:9/822: write dd/d45/f42 [1368098,53362] 0 2024-02-16T19:29:22.320 INFO:tasks.workunit.client.0.smithi022.stdout:6/841: mknod d1/d8/c11f 0 2024-02-16T19:29:22.321 INFO:tasks.workunit.client.0.smithi022.stdout:6/842: write d1/d8/d3c/d77/fe5 [1138637,109822] 0 2024-02-16T19:29:22.321 INFO:tasks.workunit.client.1.smithi117.stdout:6/896: write d5/d4b/d71/dc7/d72/dec/fef [463031,118811] 0 2024-02-16T19:29:22.323 INFO:tasks.workunit.client.1.smithi117.stdout:2/927: read - d0/d3/d34/d94/d46/d68/d76/fd2 zero size 2024-02-16T19:29:22.324 INFO:tasks.workunit.client.1.smithi117.stdout:7/885: creat d4/d1c/d98/f149 x:0 0 0 2024-02-16T19:29:22.325 INFO:tasks.workunit.client.1.smithi117.stdout:7/886: fsync d4/d1c/f26 0 2024-02-16T19:29:22.326 INFO:tasks.workunit.client.0.smithi022.stdout:7/878: rmdir d9/da/d4e 39 2024-02-16T19:29:22.327 INFO:tasks.workunit.client.1.smithi117.stdout:5/937: dwrite d2/d9/d12e/d52/d74/d8c/f121 [0,4194304] 0 2024-02-16T19:29:22.329 INFO:tasks.workunit.client.1.smithi117.stdout:0/905: rename d0/d9/d2c/d8b/dcd to d0/d9/d14/d1d/d60/dc3/d109/d132 0 2024-02-16T19:29:22.333 INFO:tasks.workunit.client.0.smithi022.stdout:6/843: fdatasync d1/d8/d3c/d4d/fc2 0 2024-02-16T19:29:22.334 INFO:tasks.workunit.client.1.smithi117.stdout:0/906: dread d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/d27/d31/d94/f106 [0,4194304] 0 2024-02-16T19:29:22.335 INFO:tasks.workunit.client.1.smithi117.stdout:3/996: creat d1/d1b/f157 x:0 0 0 2024-02-16T19:29:22.337 INFO:tasks.workunit.client.1.smithi117.stdout:9/823: symlink dd/db5/l116 0 2024-02-16T19:29:22.338 INFO:tasks.workunit.client.0.smithi022.stdout:1/854: sync 2024-02-16T19:29:22.338 INFO:tasks.workunit.client.0.smithi022.stdout:2/810: sync 2024-02-16T19:29:22.340 INFO:tasks.workunit.client.1.smithi117.stdout:6/897: creat d5/d4b/d44/d52/d81/dd7/f126 x:0 0 0 2024-02-16T19:29:22.340 INFO:tasks.workunit.client.1.smithi117.stdout:4/926: sync 2024-02-16T19:29:22.342 INFO:tasks.workunit.client.1.smithi117.stdout:3/997: dwrite d1/d1b/d1e/d37/f9c [0,4194304] 0 2024-02-16T19:29:22.342 INFO:tasks.workunit.client.0.smithi022.stdout:7/879: rename d9/da/d1f/d44/d5b/l63 to d9/da/d1f/d40/deb/l12c 0 2024-02-16T19:29:22.342 INFO:tasks.workunit.client.0.smithi022.stdout:4/784: write d4/d6/d14/d64/d4e/d54/d5a/f5f [1569717,89075] 0 2024-02-16T19:29:22.346 INFO:tasks.workunit.client.1.smithi117.stdout:8/953: creat d4/d12/f12d x:0 0 0 2024-02-16T19:29:22.347 INFO:tasks.workunit.client.1.smithi117.stdout:8/954: stat d4/d7/d6a/d72/d83 0 2024-02-16T19:29:22.351 INFO:tasks.workunit.client.1.smithi117.stdout:9/824: dread dd/d47/d48/dc0/f4d [0,4194304] 0 2024-02-16T19:29:22.353 INFO:tasks.workunit.client.0.smithi022.stdout:8/917: link d3/d12/d17/f83 d3/d25/f13b 0 2024-02-16T19:29:22.354 INFO:tasks.workunit.client.1.smithi117.stdout:5/938: rmdir d2/d4c 39 2024-02-16T19:29:22.355 INFO:tasks.workunit.client.0.smithi022.stdout:1/855: truncate d6/d8/f7d 1109469 0 2024-02-16T19:29:22.356 INFO:tasks.workunit.client.0.smithi022.stdout:6/844: write d1/d8/f6f [1203777,128404] 0 2024-02-16T19:29:22.356 INFO:tasks.workunit.client.0.smithi022.stdout:1/856: chown d6/d8/d22/d3f/d99/f9b 29 1 2024-02-16T19:29:22.356 INFO:tasks.workunit.client.1.smithi117.stdout:1/810: link d2/d1d/f6d d2/d19/d59/d89/d8d/f117 0 2024-02-16T19:29:22.359 INFO:tasks.workunit.client.0.smithi022.stdout:7/880: creat d9/da/d1f/d44/d5b/d95/d10a/f12d x:0 0 0 2024-02-16T19:29:22.359 INFO:tasks.workunit.client.1.smithi117.stdout:9/825: dwrite dd/d47/d5e/f110 [0,4194304] 0 2024-02-16T19:29:22.360 INFO:tasks.workunit.client.0.smithi022.stdout:7/881: write d9/da/d3b/dba/ff5 [546884,86060] 0 2024-02-16T19:29:22.361 INFO:tasks.workunit.client.1.smithi117.stdout:5/939: dread d2/d9/d12e/d52/d74/fab [0,4194304] 0 2024-02-16T19:29:22.361 INFO:tasks.workunit.client.0.smithi022.stdout:7/882: dread - d9/da/d1f/d93/fa2 zero size 2024-02-16T19:29:22.362 INFO:tasks.workunit.client.1.smithi117.stdout:4/927: creat d0/da/d21/d3d/d28/d64/d8a/d92/f12f x:0 0 0 2024-02-16T19:29:22.363 INFO:tasks.workunit.client.1.smithi117.stdout:3/998: mkdir d1/dd/d3b/dba/de9/d158 0 2024-02-16T19:29:22.366 INFO:tasks.workunit.client.1.smithi117.stdout:6/898: rename d5/d4b/d71/dc7/d49/fac to d5/d4b/d44/d52/d81/dd7/de0/f127 0 2024-02-16T19:29:22.367 INFO:tasks.workunit.client.0.smithi022.stdout:4/785: symlink d4/d6/lf7 0 2024-02-16T19:29:22.371 INFO:tasks.workunit.client.1.smithi117.stdout:7/887: creat d4/d1c/d39/d13e/f14a x:0 0 0 2024-02-16T19:29:22.376 INFO:tasks.workunit.client.0.smithi022.stdout:7/883: mknod d9/da/d1f/d67/ddb/c12e 0 2024-02-16T19:29:22.378 INFO:tasks.workunit.client.0.smithi022.stdout:8/918: truncate d3/d12/d2b/dca/f137 786995 0 2024-02-16T19:29:22.379 INFO:tasks.workunit.client.1.smithi117.stdout:0/907: dwrite d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/d27/d4e/fa5 [0,4194304] 0 2024-02-16T19:29:22.380 INFO:tasks.workunit.client.1.smithi117.stdout:1/811: chown d2/d19/d59/d89/leb 1780051 1 2024-02-16T19:29:22.380 INFO:tasks.workunit.client.0.smithi022.stdout:1/857: creat d6/d8/d22/d3f/dc5/d100/f10a x:0 0 0 2024-02-16T19:29:22.382 INFO:tasks.workunit.client.1.smithi117.stdout:9/826: mkdir dd/db5/d117 0 2024-02-16T19:29:22.384 INFO:tasks.workunit.client.1.smithi117.stdout:5/940: readlink d2/d9/lc8 0 2024-02-16T19:29:22.388 INFO:tasks.workunit.client.1.smithi117.stdout:4/928: mknod d0/c130 0 2024-02-16T19:29:22.389 INFO:tasks.workunit.client.0.smithi022.stdout:8/919: fdatasync d3/d12/d2b/d54/f95 0 2024-02-16T19:29:22.389 INFO:tasks.workunit.client.1.smithi117.stdout:9/827: dread dd/d47/fc2 [0,4194304] 0 2024-02-16T19:29:22.391 INFO:tasks.workunit.client.0.smithi022.stdout:1/858: fdatasync d6/d8/d22/f82 0 2024-02-16T19:29:22.396 INFO:tasks.workunit.client.1.smithi117.stdout:8/955: rename d4/d1c to d4/d12/d2a/d64/dec/d76/d121/d12e 0 2024-02-16T19:29:22.397 INFO:tasks.workunit.client.0.smithi022.stdout:6/845: write d1/d8/da/d29/deb/dfd/d85/d56/fa5 [1002275,127601] 0 2024-02-16T19:29:22.398 INFO:tasks.workunit.client.1.smithi117.stdout:6/899: unlink d5/d4b/d44/c6d 0 2024-02-16T19:29:22.400 INFO:tasks.workunit.client.1.smithi117.stdout:2/928: creat d0/d13/f127 x:0 0 0 2024-02-16T19:29:22.402 INFO:tasks.workunit.client.1.smithi117.stdout:3/999: dread d1/d1b/d1e/d29/daf/ffb [0,4194304] 0 2024-02-16T19:29:22.404 INFO:tasks.workunit.client.0.smithi022.stdout:8/920: unlink d3/d12/d92/cb4 0 2024-02-16T19:29:22.408 INFO:tasks.workunit.client.0.smithi022.stdout:1/859: mknod d6/d8/d22/d67/d7e/c10b 0 2024-02-16T19:29:22.409 INFO:tasks.workunit.client.1.smithi117.stdout:4/929: write d0/da/d21/d3d/d28/d64/d90/dc0/dd7/fde [3975973,38423] 0 2024-02-16T19:29:22.409 INFO:tasks.workunit.client.0.smithi022.stdout:4/786: link d4/d6/dd/d22/d83/lbf d4/d6/dd/df/d10/d55/lf8 0 2024-02-16T19:29:22.411 INFO:tasks.workunit.client.0.smithi022.stdout:6/846: fdatasync d1/d8/da/d29/deb/dfd/d85/d9c/ff6 0 2024-02-16T19:29:22.411 INFO:tasks.workunit.client.1.smithi117.stdout:4/930: dread d0/da/d21/d3d/d28/d64/d73/d91/d37/d102/f12e [0,4194304] 0 2024-02-16T19:29:22.412 INFO:tasks.workunit.client.1.smithi117.stdout:4/931: chown d0/da/d21/d3d/d28/d64/d90/fb0 1382 1 2024-02-16T19:29:22.413 INFO:tasks.workunit.client.1.smithi117.stdout:9/828: dread dd/d45/d31/d63/d8e/dbb/fc9 [0,4194304] 0 2024-02-16T19:29:22.413 INFO:tasks.workunit.client.0.smithi022.stdout:6/847: chown d1/d8/dac/d101/da3/ldf 1 1 2024-02-16T19:29:22.414 INFO:tasks.workunit.client.1.smithi117.stdout:9/829: stat dd/db5/cc7 0 2024-02-16T19:29:22.414 INFO:tasks.workunit.client.1.smithi117.stdout:6/900: chown d5/d4b/d71/dc7/d72/f90 45842 1 2024-02-16T19:29:22.415 INFO:tasks.workunit.client.1.smithi117.stdout:4/932: chown d0/da/d21/d3d/d28/d64/d73/d91/l45 1 1 2024-02-16T19:29:22.415 INFO:tasks.workunit.client.0.smithi022.stdout:1/860: dwrite d6/d8/d22/d67/d7e/fa9 [0,4194304] 0 2024-02-16T19:29:22.416 INFO:tasks.workunit.client.0.smithi022.stdout:1/861: chown d6/d8/dba/fce 225680127 1 2024-02-16T19:29:22.417 INFO:tasks.workunit.client.0.smithi022.stdout:8/921: mknod d3/d12/d117/c13c 0 2024-02-16T19:29:22.418 INFO:tasks.workunit.client.1.smithi117.stdout:5/941: dread d2/d9/d3c/d24/f53 [0,4194304] 0 2024-02-16T19:29:22.420 INFO:tasks.workunit.client.1.smithi117.stdout:2/929: creat d0/d3/d34/d94/d46/f128 x:0 0 0 2024-02-16T19:29:22.421 INFO:tasks.workunit.client.1.smithi117.stdout:1/812: rmdir d2/d19/d59/d89/d7b/d86 39 2024-02-16T19:29:22.424 INFO:tasks.workunit.client.0.smithi022.stdout:7/884: truncate d9/da/d2c/dcb/fdd 40971 0 2024-02-16T19:29:22.424 INFO:tasks.workunit.client.0.smithi022.stdout:4/787: fsync d4/d6/dd/f60 0 2024-02-16T19:29:22.425 INFO:tasks.workunit.client.0.smithi022.stdout:4/788: readlink d4/d6/dd/l3b 0 2024-02-16T19:29:22.426 INFO:tasks.workunit.client.1.smithi117.stdout:6/901: symlink d5/d9c/l128 0 2024-02-16T19:29:22.429 INFO:tasks.workunit.client.1.smithi117.stdout:5/942: fsync d2/d3/d21/f92 0 2024-02-16T19:29:22.430 INFO:tasks.workunit.client.1.smithi117.stdout:5/943: readlink d2/d9/d12e/d96/dbc/d30/l58 0 2024-02-16T19:29:22.430 INFO:tasks.workunit.client.1.smithi117.stdout:5/944: chown d2/d9/d12e/d103/f12c 254689 1 2024-02-16T19:29:22.431 INFO:tasks.workunit.client.1.smithi117.stdout:2/930: fsync d0/f5e 0 2024-02-16T19:29:22.443 INFO:tasks.workunit.client.1.smithi117.stdout:1/813: fdatasync d2/d19/d59/d89/fec 0 2024-02-16T19:29:22.444 INFO:tasks.workunit.client.0.smithi022.stdout:8/922: dread d3/d12/da6/f34 [0,4194304] 0 2024-02-16T19:29:22.445 INFO:tasks.workunit.client.1.smithi117.stdout:5/945: dread d2/d9/d12e/d96/dbc/d30/d38/d51/f7e [0,4194304] 0 2024-02-16T19:29:22.445 INFO:tasks.workunit.client.1.smithi117.stdout:5/946: stat d2/d3/d21/f33 0 2024-02-16T19:29:22.446 INFO:tasks.workunit.client.1.smithi117.stdout:9/830: link dd/db5/dc4/l10b dd/d47/d5b/d5f/de6/l118 0 2024-02-16T19:29:22.449 INFO:tasks.workunit.client.1.smithi117.stdout:0/908: sync 2024-02-16T19:29:22.449 INFO:tasks.workunit.client.1.smithi117.stdout:7/888: sync 2024-02-16T19:29:22.452 INFO:tasks.workunit.client.0.smithi022.stdout:9/833: sync 2024-02-16T19:29:22.452 INFO:tasks.workunit.client.0.smithi022.stdout:5/895: sync 2024-02-16T19:29:22.452 INFO:tasks.workunit.client.0.smithi022.stdout:3/820: sync 2024-02-16T19:29:22.455 INFO:tasks.workunit.client.1.smithi117.stdout:8/956: rename d4/d7/da6/dfd to d4/d12f 0 2024-02-16T19:29:22.457 INFO:tasks.workunit.client.1.smithi117.stdout:1/814: dwrite d2/d19/d59/d89/d69/dad/d100/ff7 [0,4194304] 0 2024-02-16T19:29:22.457 INFO:tasks.workunit.client.0.smithi022.stdout:3/821: truncate d7/d12/d18/dd/d31/dbf/dc5/fe8 234708 0 2024-02-16T19:29:22.460 INFO:tasks.workunit.client.0.smithi022.stdout:5/896: dwrite dd/d1f/d48/d21/d2f/ff8 [0,4194304] 0 2024-02-16T19:29:22.464 INFO:tasks.workunit.client.1.smithi117.stdout:4/933: getdents d0/da/d21/d3d/d28/d64/d90 0 2024-02-16T19:29:22.467 INFO:tasks.workunit.client.1.smithi117.stdout:0/909: read d0/d9/d14/d23/f24 [3619000,46831] 0 2024-02-16T19:29:22.472 INFO:tasks.workunit.client.1.smithi117.stdout:6/902: rename d5/d4b/d20/d69/cf7 to d5/d4b/d71/c129 0 2024-02-16T19:29:22.475 INFO:tasks.workunit.client.1.smithi117.stdout:1/815: stat d2/d19/d59/d76/d8c/dc7/dd2/cfd 0 2024-02-16T19:29:22.476 INFO:tasks.workunit.client.1.smithi117.stdout:1/816: stat d2/d1d/d60/l1a 0 2024-02-16T19:29:22.479 INFO:tasks.workunit.client.1.smithi117.stdout:4/934: stat d0/da/c20 0 2024-02-16T19:29:22.480 INFO:tasks.workunit.client.1.smithi117.stdout:4/935: dread - d0/da/d21/d3d/d28/d64/d90/f120 zero size 2024-02-16T19:29:22.481 INFO:tasks.workunit.client.1.smithi117.stdout:0/910: creat d0/f133 x:0 0 0 2024-02-16T19:29:22.482 INFO:tasks.workunit.client.1.smithi117.stdout:0/911: readlink d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/d27/d31/d6d/l100 0 2024-02-16T19:29:22.483 INFO:tasks.workunit.client.1.smithi117.stdout:8/957: creat d4/d12/d31/f130 x:0 0 0 2024-02-16T19:29:22.487 INFO:tasks.workunit.client.1.smithi117.stdout:6/903: mkdir d5/d4b/d71/dc7/d72/d12a 0 2024-02-16T19:29:22.489 INFO:tasks.workunit.client.1.smithi117.stdout:1/817: mkdir d2/d1d/d118 0 2024-02-16T19:29:22.496 INFO:tasks.workunit.client.1.smithi117.stdout:2/931: rename d0/d3/d10/l21 to d0/d13/d4d/d53/d58/l129 0 2024-02-16T19:29:22.497 INFO:tasks.workunit.client.1.smithi117.stdout:2/932: truncate d0/d3/d34/d94/d46/f128 523456 0 2024-02-16T19:29:22.498 INFO:tasks.workunit.client.1.smithi117.stdout:5/947: getdents d2/d9/d12e/d96/dbc/d30/d95 0 2024-02-16T19:29:22.498 INFO:tasks.workunit.client.0.smithi022.stdout:2/811: sync 2024-02-16T19:29:22.500 INFO:tasks.workunit.client.1.smithi117.stdout:6/904: mkdir d5/d4b/d71/dc7/d49/d12b 0 2024-02-16T19:29:22.502 INFO:tasks.workunit.client.1.smithi117.stdout:1/818: creat d2/d4/dbb/f119 x:0 0 0 2024-02-16T19:29:22.503 INFO:tasks.workunit.client.1.smithi117.stdout:9/831: link dd/d45/d31/d63/dba/df0/daf/df4/c5c dd/d45/d31/d63/dba/df0/daf/df4/d10d/c119 0 2024-02-16T19:29:22.504 INFO:tasks.workunit.client.1.smithi117.stdout:9/832: dread - dd/d47/d5b/d5f/fc5 zero size 2024-02-16T19:29:22.508 INFO:tasks.workunit.client.0.smithi022.stdout:6/848: creat d1/d8/da/d29/deb/dfd/d53/d103/f120 x:0 0 0 2024-02-16T19:29:22.510 INFO:tasks.workunit.client.1.smithi117.stdout:2/933: mkdir d0/d13/dc4/de4/d12a 0 2024-02-16T19:29:22.510 INFO:tasks.workunit.client.1.smithi117.stdout:2/934: stat d0/d3/d22/d37/d9d 0 2024-02-16T19:29:22.510 INFO:tasks.workunit.client.1.smithi117.stdout:2/935: chown d0/d3 125 1 2024-02-16T19:29:22.511 INFO:tasks.workunit.client.0.smithi022.stdout:7/885: creat d9/da/d1f/d67/f12f x:0 0 0 2024-02-16T19:29:22.511 INFO:tasks.workunit.client.1.smithi117.stdout:2/936: readlink d0/d3/d22/d37/d9d/df9/l104 0 2024-02-16T19:29:22.514 INFO:tasks.workunit.client.0.smithi022.stdout:1/862: truncate d6/d8/d22/d16/d17/d1d/f33 5317785 0 2024-02-16T19:29:22.516 INFO:tasks.workunit.client.1.smithi117.stdout:6/905: creat d5/d4b/d71/dc7/dce/ded/f12c x:0 0 0 2024-02-16T19:29:22.517 INFO:tasks.workunit.client.0.smithi022.stdout:4/789: read d4/d6/d14/d64/d4e/d54/fb4 [84525,72] 0 2024-02-16T19:29:22.517 INFO:tasks.workunit.client.1.smithi117.stdout:6/906: dread - d5/d4b/d44/db1/fe2 zero size 2024-02-16T19:29:22.519 INFO:tasks.workunit.client.1.smithi117.stdout:1/819: unlink d2/d19/d59/d89/d69/cae 0 2024-02-16T19:29:22.519 INFO:tasks.workunit.client.1.smithi117.stdout:4/936: dwrite d0/da/d21/f3e [0,4194304] 0 2024-02-16T19:29:22.525 INFO:tasks.workunit.client.0.smithi022.stdout:8/923: rename d3/d12/d2b/l4e to d3/d12/d92/d135/dc0/l13d 0 2024-02-16T19:29:22.528 INFO:tasks.workunit.client.1.smithi117.stdout:0/912: creat d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/d27/f134 x:0 0 0 2024-02-16T19:29:22.533 INFO:tasks.workunit.client.0.smithi022.stdout:4/790: dread d4/d6/d14/dc7/d8d/f99 [0,4194304] 0 2024-02-16T19:29:22.534 INFO:tasks.workunit.client.0.smithi022.stdout:3/822: mknod d7/d12/d18/dd/d3c/d5a/d67/c119 0 2024-02-16T19:29:22.534 INFO:tasks.workunit.client.0.smithi022.stdout:4/791: chown d4/d6/d3a/fa0 220578669 1 2024-02-16T19:29:22.536 INFO:tasks.workunit.client.1.smithi117.stdout:6/907: mkdir d5/d4b/d71/dc7/d49/d12d 0 2024-02-16T19:29:22.536 INFO:tasks.workunit.client.0.smithi022.stdout:4/792: fdatasync d4/d6/d14/dc7/d6a/fa9 0 2024-02-16T19:29:22.537 INFO:tasks.workunit.client.0.smithi022.stdout:4/793: readlink d4/lbd 0 2024-02-16T19:29:22.537 INFO:tasks.workunit.client.0.smithi022.stdout:6/849: write d1/d8/d3c/f44 [4255662,27363] 0 2024-02-16T19:29:22.540 INFO:tasks.workunit.client.1.smithi117.stdout:5/948: write d2/db3/db0/f43 [3818283,66926] 0 2024-02-16T19:29:22.544 INFO:tasks.workunit.client.1.smithi117.stdout:5/949: dread d2/d9/d3c/f1a [0,4194304] 0 2024-02-16T19:29:22.547 INFO:tasks.workunit.client.1.smithi117.stdout:0/913: truncate d0/d9/d14/d1d/d60/dc3/fef 598930 0 2024-02-16T19:29:22.547 INFO:tasks.workunit.client.1.smithi117.stdout:9/833: write dd/d45/d31/d63/d8e/fc8 [2599354,24949] 0 2024-02-16T19:29:22.550 INFO:tasks.workunit.client.0.smithi022.stdout:2/812: rename d0/d32/d3b/f6e to d0/d2/d18/d19/d90/f111 0 2024-02-16T19:29:22.553 INFO:tasks.workunit.client.1.smithi117.stdout:7/889: rename d4/dd3/c129 to d4/d1c/c14b 0 2024-02-16T19:29:22.557 INFO:tasks.workunit.client.0.smithi022.stdout:5/897: creat dd/d1f/d48/d21/d64/d11d/d6a/db0/f126 x:0 0 0 2024-02-16T19:29:22.558 INFO:tasks.workunit.client.0.smithi022.stdout:5/898: readlink dd/d1f/d48/d21/d64/d11d/d8f/l11c 0 2024-02-16T19:29:22.558 INFO:tasks.workunit.client.1.smithi117.stdout:2/937: dwrite d0/d3/d34/d94/d46/d68/d76/fd2 [0,4194304] 0 2024-02-16T19:29:22.560 INFO:tasks.workunit.client.1.smithi117.stdout:4/937: mkdir d0/d116/d131 0 2024-02-16T19:29:22.561 INFO:tasks.workunit.client.1.smithi117.stdout:4/938: dread - d0/da/d21/d3d/d28/d64/d8a/d92/fff zero size 2024-02-16T19:29:22.563 INFO:tasks.workunit.client.0.smithi022.stdout:4/794: unlink d4/d6/d3a/l7c 0 2024-02-16T19:29:22.565 INFO:tasks.workunit.client.1.smithi117.stdout:5/950: fsync d2/d9/f18 0 2024-02-16T19:29:22.566 INFO:tasks.workunit.client.1.smithi117.stdout:5/951: fsync d2/d9/d12e/d52/f1e 0 2024-02-16T19:29:22.567 INFO:tasks.workunit.client.1.smithi117.stdout:4/939: dwrite d0/da/d21/d3d/d28/d64/f23 [4194304,4194304] 0 2024-02-16T19:29:22.568 INFO:tasks.workunit.client.0.smithi022.stdout:1/863: dwrite d6/d8/d22/d16/d17/fd4 [0,4194304] 0 2024-02-16T19:29:22.572 INFO:tasks.workunit.client.0.smithi022.stdout:6/850: dread - d1/d8/da/d29/deb/dfd/d85/d56/fab zero size 2024-02-16T19:29:22.572 INFO:tasks.workunit.client.1.smithi117.stdout:8/958: rename d4/d12/fd4 to d4/d12/d31/d8e/dbf/f131 0 2024-02-16T19:29:22.574 INFO:tasks.workunit.client.1.smithi117.stdout:0/914: dread d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/d27/f52 [0,4194304] 0 2024-02-16T19:29:22.575 INFO:tasks.workunit.client.1.smithi117.stdout:7/890: unlink d4/d1c/d39/d46/lac 0 2024-02-16T19:29:22.575 INFO:tasks.workunit.client.0.smithi022.stdout:3/823: dread d7/d12/d18/dd/d31/fed [0,4194304] 0 2024-02-16T19:29:22.581 INFO:tasks.workunit.client.1.smithi117.stdout:8/959: dread d4/d12/d2a/d64/dc6/ff3 [0,4194304] 0 2024-02-16T19:29:22.583 INFO:tasks.workunit.client.1.smithi117.stdout:1/820: rmdir d2/d1d/d60/d7c/d9e/d107 0 2024-02-16T19:29:22.587 INFO:tasks.workunit.client.1.smithi117.stdout:2/938: rmdir d0/d3/d40 39 2024-02-16T19:29:22.588 INFO:tasks.workunit.client.1.smithi117.stdout:2/939: chown d0/d3/d34/d94/d46/d68/d93/da9/fcd 0 1 2024-02-16T19:29:22.589 INFO:tasks.workunit.client.0.smithi022.stdout:5/899: chown dd/f1d 1054 1 2024-02-16T19:29:22.591 INFO:tasks.workunit.client.1.smithi117.stdout:2/940: dread d0/d13/f15 [0,4194304] 0 2024-02-16T19:29:22.592 INFO:tasks.workunit.client.0.smithi022.stdout:7/886: write d9/da/d3b/fd7 [1619713,107782] 0 2024-02-16T19:29:22.593 INFO:tasks.workunit.client.1.smithi117.stdout:2/941: write d0/d3/d34/d94/d46/d68/d76/da1/d56/d60/dc9/fea [106283,50221] 0 2024-02-16T19:29:22.600 INFO:tasks.workunit.client.0.smithi022.stdout:4/795: mknod d4/d6/db6/cf9 0 2024-02-16T19:29:22.601 INFO:tasks.workunit.client.0.smithi022.stdout:4/796: stat d4/d6/d3a/l75 0 2024-02-16T19:29:22.604 INFO:tasks.workunit.client.0.smithi022.stdout:6/851: mkdir d1/d8/da/d29/d61/d57/d121 0 2024-02-16T19:29:22.606 INFO:tasks.workunit.client.0.smithi022.stdout:1/864: write d6/d66/fdf [875019,130577] 0 2024-02-16T19:29:22.607 INFO:tasks.workunit.client.1.smithi117.stdout:9/834: creat dd/d45/d31/d63/dba/df0/daf/df4/d10d/f11a x:0 0 0 2024-02-16T19:29:22.610 INFO:tasks.workunit.client.1.smithi117.stdout:6/908: rename d5/db/d7d/c119 to d5/d4b/d71/dc7/d49/d12d/c12e 0 2024-02-16T19:29:22.611 INFO:tasks.workunit.client.1.smithi117.stdout:6/909: dread - d5/d4b/d20/d69/f120 zero size 2024-02-16T19:29:22.611 INFO:tasks.workunit.client.0.smithi022.stdout:8/924: link d3/d12/d2b/d2e/l85 d3/d25/d33/l13e 0 2024-02-16T19:29:22.611 INFO:tasks.workunit.client.0.smithi022.stdout:8/925: chown d3/d12/db0/f81 1510 1 2024-02-16T19:29:22.614 INFO:tasks.workunit.client.0.smithi022.stdout:9/834: getdents db/d14/dc8 0 2024-02-16T19:29:22.617 INFO:tasks.workunit.client.1.smithi117.stdout:1/821: write d2/d19/d59/d89/d49/ffb [320418,74616] 0 2024-02-16T19:29:22.618 INFO:tasks.workunit.client.0.smithi022.stdout:6/852: mkdir d1/d8/dac/d101/db3/db0/d122 0 2024-02-16T19:29:22.620 INFO:tasks.workunit.client.0.smithi022.stdout:2/813: mknod d0/d2/d18/d19/ddf/d10c/c112 0 2024-02-16T19:29:22.621 INFO:tasks.workunit.client.1.smithi117.stdout:2/942: mkdir d0/d3/d34/d94/d46/d68/d76/da1/d56/d60/d12b 0 2024-02-16T19:29:22.621 INFO:tasks.workunit.client.1.smithi117.stdout:2/943: stat d0/d3/d34/d94/d46/d68/d76/da1/fe9 0 2024-02-16T19:29:22.621 INFO:tasks.workunit.client.0.smithi022.stdout:1/865: creat d6/d8/d22/d3a/d80/f10c x:0 0 0 2024-02-16T19:29:22.623 INFO:tasks.workunit.client.0.smithi022.stdout:3/824: mkdir d7/d12/d18/dd/d24/d53/d116/de6/d11a 0 2024-02-16T19:29:22.624 INFO:tasks.workunit.client.0.smithi022.stdout:3/825: fdatasync d7/d12/d18/dd/d3c/d5a/ff6 0 2024-02-16T19:29:22.625 INFO:tasks.workunit.client.0.smithi022.stdout:3/826: readlink d7/d12/d18/dd/d31/d7f/l101 0 2024-02-16T19:29:22.625 INFO:tasks.workunit.client.0.smithi022.stdout:8/926: rename d3/d12/d2b/da9/cc7 to d3/d12/d2b/d54/db8/c13f 0 2024-02-16T19:29:22.626 INFO:tasks.workunit.client.0.smithi022.stdout:1/866: dread d6/d66/fdf [0,4194304] 0 2024-02-16T19:29:22.628 INFO:tasks.workunit.client.0.smithi022.stdout:9/835: creat db/d14/d16/d5d/d65/f11c x:0 0 0 2024-02-16T19:29:22.631 INFO:tasks.workunit.client.0.smithi022.stdout:7/887: dread d9/da/d2c/d6d/f11a [0,4194304] 0 2024-02-16T19:29:22.632 INFO:tasks.workunit.client.1.smithi117.stdout:0/915: rename d0/d30 to d0/d78/d135 0 2024-02-16T19:29:22.633 INFO:tasks.workunit.client.1.smithi117.stdout:0/916: write d0/d9/d2c/d8b/de0/f12f [981850,39672] 0 2024-02-16T19:29:22.634 INFO:tasks.workunit.client.0.smithi022.stdout:6/853: mknod d1/d8/da/d7b/c123 0 2024-02-16T19:29:22.638 INFO:tasks.workunit.client.1.smithi117.stdout:7/891: mknod d4/d1c/d39/d46/d55/db8/d141/c14c 0 2024-02-16T19:29:22.639 INFO:tasks.workunit.client.1.smithi117.stdout:7/892: read d4/d1c/d87/d89/fd4 [839907,64322] 0 2024-02-16T19:29:22.643 INFO:tasks.workunit.client.0.smithi022.stdout:3/827: mkdir d7/d21/dd9/d11b 0 2024-02-16T19:29:22.647 INFO:tasks.workunit.client.0.smithi022.stdout:8/927: truncate d3/f55 921614 0 2024-02-16T19:29:22.648 INFO:tasks.workunit.client.1.smithi117.stdout:7/893: dwrite d4/d1c/d87/d89/d60/d79/fd7 [0,4194304] 0 2024-02-16T19:29:22.648 INFO:tasks.workunit.client.0.smithi022.stdout:1/867: creat d6/d8/d22/d67/d7e/f10d x:0 0 0 2024-02-16T19:29:22.650 INFO:tasks.workunit.client.1.smithi117.stdout:9/835: dwrite dd/d45/d31/d63/d8e/fd4 [0,4194304] 0 2024-02-16T19:29:22.652 INFO:tasks.workunit.client.0.smithi022.stdout:2/814: dread d0/d2/d2e/d82/df0/fca [0,4194304] 0 2024-02-16T19:29:22.652 INFO:tasks.workunit.client.1.smithi117.stdout:8/960: rename d4/d12/d2a/d64/dec/d76/fc5 to d4/d7/d6a/d72/d84/f132 0 2024-02-16T19:29:22.654 INFO:tasks.workunit.client.1.smithi117.stdout:8/961: write d4/d12/d31/f77 [4488663,49359] 0 2024-02-16T19:29:22.655 INFO:tasks.workunit.client.0.smithi022.stdout:9/836: truncate db/d14/d16/d49/fe7 28013 0 2024-02-16T19:29:22.658 INFO:tasks.workunit.client.1.smithi117.stdout:1/822: dread d2/d19/d59/d89/f42 [0,4194304] 0 2024-02-16T19:29:22.659 INFO:tasks.workunit.client.1.smithi117.stdout:2/944: dread d0/d13/d75/f59 [0,4194304] 0 2024-02-16T19:29:22.661 INFO:tasks.workunit.client.0.smithi022.stdout:4/797: creat d4/d6/d14/d64/d4e/ffa x:0 0 0 2024-02-16T19:29:22.662 INFO:tasks.workunit.client.0.smithi022.stdout:4/798: readlink d4/d6/dd/l3b 0 2024-02-16T19:29:22.663 INFO:tasks.workunit.client.1.smithi117.stdout:5/952: link d2/d9/d3c/d24/l116 d2/d4c/dde/d11f/l130 0 2024-02-16T19:29:22.668 INFO:tasks.workunit.client.0.smithi022.stdout:3/828: symlink d7/d12/d18/dd/d3c/l11c 0 2024-02-16T19:29:22.670 INFO:tasks.workunit.client.1.smithi117.stdout:7/894: rename d4/d1c/dd8/l128 to d4/d13/d63/d71/l14d 0 2024-02-16T19:29:22.672 INFO:tasks.workunit.client.1.smithi117.stdout:6/910: truncate d5/d4b/f7a 2668024 0 2024-02-16T19:29:22.674 INFO:tasks.workunit.client.1.smithi117.stdout:7/895: dread d4/d1c/d39/d46/d47/f77 [0,4194304] 0 2024-02-16T19:29:22.674 INFO:tasks.workunit.client.0.smithi022.stdout:1/868: creat d6/d8/d22/d3f/dc5/d100/f10e x:0 0 0 2024-02-16T19:29:22.677 INFO:tasks.workunit.client.0.smithi022.stdout:2/815: symlink d0/d2/d2e/d82/da1/l113 0 2024-02-16T19:29:22.677 INFO:tasks.workunit.client.1.smithi117.stdout:8/962: dread d4/d7/d6a/d72/fe5 [0,4194304] 0 2024-02-16T19:29:22.678 INFO:tasks.workunit.client.1.smithi117.stdout:4/940: sync 2024-02-16T19:29:22.678 INFO:tasks.workunit.client.0.smithi022.stdout:9/837: rmdir db/d14/dc8/dc9/d105 39 2024-02-16T19:29:22.678 INFO:tasks.workunit.client.0.smithi022.stdout:2/816: write d0/d2/d18/d1f/d6c/d99/ff6 [80643,122837] 0 2024-02-16T19:29:22.680 INFO:tasks.workunit.client.1.smithi117.stdout:0/917: dwrite d0/d9/d14/d1d/d60/d76/d7b/fe6 [0,4194304] 0 2024-02-16T19:29:22.681 INFO:tasks.workunit.client.0.smithi022.stdout:9/838: dread db/d14/d19/da8/fab [0,4194304] 0 2024-02-16T19:29:22.681 INFO:tasks.workunit.client.1.smithi117.stdout:0/918: stat d0/cb0 0 2024-02-16T19:29:22.682 INFO:tasks.workunit.client.0.smithi022.stdout:8/928: dread d3/d12/d2b/dca/fdf [0,4194304] 0 2024-02-16T19:29:22.682 INFO:tasks.workunit.client.1.smithi117.stdout:1/823: creat d2/d19/d59/d76/d91/f11a x:0 0 0 2024-02-16T19:29:22.683 INFO:tasks.workunit.client.0.smithi022.stdout:4/799: dread - d4/d6/dd/df/d10/d24/fa6 zero size 2024-02-16T19:29:22.688 INFO:tasks.workunit.client.1.smithi117.stdout:4/941: dwrite d0/da/d21/d3d/d28/d64/f67 [0,4194304] 0 2024-02-16T19:29:22.688 INFO:tasks.workunit.client.1.smithi117.stdout:5/953: dread - d2/d9/df9/f110 zero size 2024-02-16T19:29:22.690 INFO:tasks.workunit.client.1.smithi117.stdout:5/954: readlink d2/d9/df9/l114 0 2024-02-16T19:29:22.690 INFO:tasks.workunit.client.0.smithi022.stdout:4/800: dwrite d4/d6/fbe [0,4194304] 0 2024-02-16T19:29:22.691 INFO:tasks.workunit.client.1.smithi117.stdout:9/836: unlink dd/d45/d31/d63/dba/df0/daf/df4/d10d/c119 0 2024-02-16T19:29:22.693 INFO:tasks.workunit.client.1.smithi117.stdout:9/837: readlink dd/d45/d31/d63/lf6 0 2024-02-16T19:29:22.694 INFO:tasks.workunit.client.0.smithi022.stdout:1/869: mkdir d6/d8/d22/d16/d4f/d10f 0 2024-02-16T19:29:22.696 INFO:tasks.workunit.client.0.smithi022.stdout:7/888: link d9/da/d2c/d46/la3 d9/da/d1f/d40/dfb/l130 0 2024-02-16T19:29:22.698 INFO:tasks.workunit.client.1.smithi117.stdout:7/896: fdatasync d4/d1c/d39/d46/d47/fb5 0 2024-02-16T19:29:22.698 INFO:tasks.workunit.client.1.smithi117.stdout:5/955: dwrite d2/d9/d12e/d96/dbc/d30/d38/d51/fe0 [0,4194304] 0 2024-02-16T19:29:22.698 INFO:tasks.workunit.client.0.smithi022.stdout:2/817: truncate d0/d2/d18/d1f/f5e 2646861 0 2024-02-16T19:29:22.700 INFO:tasks.workunit.client.1.smithi117.stdout:8/963: rename d4/d73 to d4/d73/dff/d133 22 2024-02-16T19:29:22.701 INFO:tasks.workunit.client.0.smithi022.stdout:9/839: mkdir db/d14/d16/d49/d90/d11d 0 2024-02-16T19:29:22.707 INFO:tasks.workunit.client.0.smithi022.stdout:8/929: fdatasync d3/d12/da6/f2f 0 2024-02-16T19:29:22.709 INFO:tasks.workunit.client.0.smithi022.stdout:6/854: write d1/d8/da/d29/deb/dfd/d85/f2b [1988885,27872] 0 2024-02-16T19:29:22.711 INFO:tasks.workunit.client.1.smithi117.stdout:2/945: unlink d0/d3/d40/f66 0 2024-02-16T19:29:22.713 INFO:tasks.workunit.client.1.smithi117.stdout:0/919: dread d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/d27/d4e/fac [0,4194304] 0 2024-02-16T19:29:22.714 INFO:tasks.workunit.client.0.smithi022.stdout:4/801: creat d4/d6/d14/ffb x:0 0 0 2024-02-16T19:29:22.714 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:22 smithi022 ceph-mon[29221]: pgmap v14: 65 pgs: 65 active+clean; 3.5 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 117 MiB/s rd, 266 MiB/s wr, 593 op/s 2024-02-16T19:29:22.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:22 smithi022 ceph-mon[29221]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' 2024-02-16T19:29:22.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:22 smithi022 ceph-mon[29221]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' 2024-02-16T19:29:22.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:22 smithi022 ceph-mon[29221]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:29:22.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:22 smithi022 ceph-mon[29221]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:29:22.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:22 smithi022 ceph-mon[29221]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:29:22.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:22 smithi022 ceph-mon[29221]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:29:22.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:22 smithi022 ceph-mon[29221]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:29:22.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:22 smithi022 ceph-mon[29221]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:29:22.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:22 smithi022 ceph-mon[29221]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' 2024-02-16T19:29:22.715 INFO:tasks.workunit.client.1.smithi117.stdout:2/946: dread d0/d3/d34/d94/d46/d68/d76/da1/d56/f72 [0,4194304] 0 2024-02-16T19:29:22.716 INFO:tasks.workunit.client.0.smithi022.stdout:4/802: chown d4/d6/dd/d22/d30/ce3 105588 1 2024-02-16T19:29:22.719 INFO:tasks.workunit.client.0.smithi022.stdout:7/889: mkdir d9/da/d74/d131 0 2024-02-16T19:29:22.720 INFO:tasks.workunit.client.0.smithi022.stdout:3/829: dwrite d7/d12/d18/dd/d24/d93/de9/f85 [0,4194304] 0 2024-02-16T19:29:22.722 INFO:tasks.workunit.client.1.smithi117.stdout:9/838: creat dd/d45/d31/d63/dba/df0/daf/df4/d10d/f11b x:0 0 0 2024-02-16T19:29:22.728 INFO:tasks.workunit.client.1.smithi117.stdout:7/897: mknod d4/d13/d63/ddd/c14e 0 2024-02-16T19:29:22.730 INFO:tasks.workunit.client.1.smithi117.stdout:5/956: readlink d2/d9/d3c/d24/l116 0 2024-02-16T19:29:22.731 INFO:tasks.workunit.client.1.smithi117.stdout:5/957: dread - d2/d9/d12e/d96/dbc/d30/d38/d51/f12b zero size 2024-02-16T19:29:22.732 INFO:tasks.workunit.client.1.smithi117.stdout:8/964: symlink d4/d7/d6a/d72/l134 0 2024-02-16T19:29:22.734 INFO:tasks.workunit.client.0.smithi022.stdout:1/870: mkdir d6/d8/d22/d16/d17/d63/dee/d110 0 2024-02-16T19:29:22.742 INFO:tasks.workunit.client.0.smithi022.stdout:2/818: symlink d0/d2/d18/d19/d90/d100/l114 0 2024-02-16T19:29:22.742 INFO:tasks.workunit.client.1.smithi117.stdout:2/947: dread - d0/d3/d22/d37/fb7 zero size 2024-02-16T19:29:22.744 INFO:tasks.workunit.client.0.smithi022.stdout:3/830: mkdir d7/d12/d18/dd/d3c/d11d 0 2024-02-16T19:29:22.747 INFO:tasks.workunit.client.0.smithi022.stdout:9/840: creat db/d14/d16/d5d/d65/d6b/df6/f11e x:0 0 0 2024-02-16T19:29:22.748 INFO:tasks.workunit.client.0.smithi022.stdout:7/890: write d9/da/d2c/d46/f9a [1027649,77299] 0 2024-02-16T19:29:22.750 INFO:tasks.workunit.client.1.smithi117.stdout:9/839: creat dd/db5/f11c x:0 0 0 2024-02-16T19:29:22.750 INFO:tasks.workunit.client.0.smithi022.stdout:6/855: getdents d1/d8/da/d29/d61/d57/d121 0 2024-02-16T19:29:22.754 INFO:tasks.workunit.client.0.smithi022.stdout:9/841: dwrite db/d14/d16/d5d/d65/d6b/dd5/f116 [0,4194304] 0 2024-02-16T19:29:22.757 INFO:tasks.workunit.client.1.smithi117.stdout:8/965: unlink d4/d12/d2a/d64/dc6/ff3 0 2024-02-16T19:29:22.758 INFO:tasks.workunit.client.1.smithi117.stdout:4/942: link d0/da/d21/d3d/d28/d66/d97/d6d/fda d0/da/d21/d3d/d28/d66/da2/dac/dcb/f132 0 2024-02-16T19:29:22.762 INFO:tasks.workunit.client.1.smithi117.stdout:4/943: dread d0/da/d21/d3d/d28/d64/d48/fab [0,4194304] 0 2024-02-16T19:29:22.764 INFO:tasks.workunit.client.1.smithi117.stdout:8/966: dwrite d4/d12/d31/f77 [0,4194304] 0 2024-02-16T19:29:22.765 INFO:tasks.workunit.client.1.smithi117.stdout:6/911: link d5/d4b/d44/d7c/f104 d5/d9c/daa/d85/db0/f12f 0 2024-02-16T19:29:22.766 INFO:tasks.workunit.client.0.smithi022.stdout:1/871: write d6/d8/d22/d59/da7/fa8 [931574,112012] 0 2024-02-16T19:29:22.766 INFO:tasks.workunit.client.1.smithi117.stdout:6/912: readlink d5/d4b/d20/l97 0 2024-02-16T19:29:22.767 INFO:tasks.workunit.client.0.smithi022.stdout:8/930: creat d3/d12/f140 x:0 0 0 2024-02-16T19:29:22.767 INFO:tasks.workunit.client.1.smithi117.stdout:6/913: read d5/f86 [1079068,16131] 0 2024-02-16T19:29:22.767 INFO:tasks.workunit.client.1.smithi117.stdout:5/958: write d2/d9/d3c/d24/f53 [5614996,45840] 0 2024-02-16T19:29:22.768 INFO:tasks.workunit.client.1.smithi117.stdout:9/840: truncate dd/d47/d48/dc0/f35 351300 0 2024-02-16T19:29:22.769 INFO:tasks.workunit.client.0.smithi022.stdout:7/891: rmdir d9/da/d1f/d67/ddb/dd4 39 2024-02-16T19:29:22.771 INFO:tasks.workunit.client.0.smithi022.stdout:3/831: write d7/d12/d18/dd/d3c/d5a/d67/fa8 [269350,46322] 0 2024-02-16T19:29:22.771 INFO:tasks.workunit.client.1.smithi117.stdout:6/914: write d5/d4b/d71/dc7/d72/dec/f102 [972473,60209] 0 2024-02-16T19:29:22.771 INFO:tasks.workunit.client.0.smithi022.stdout:7/892: truncate d9/da/d1f/d67/f12f 152274 0 2024-02-16T19:29:22.772 INFO:tasks.workunit.client.1.smithi117.stdout:6/915: readlink d5/d4b/d44/d52/ld4 0 2024-02-16T19:29:22.774 INFO:tasks.workunit.client.0.smithi022.stdout:6/856: write d1/d8/da/d29/deb/dfd/d85/d5b/f88 [1048301,44578] 0 2024-02-16T19:29:22.775 INFO:tasks.workunit.client.0.smithi022.stdout:6/857: chown d1/d8/da/d29/d61/d57 651434 1 2024-02-16T19:29:22.777 INFO:tasks.workunit.client.1.smithi117.stdout:0/920: rename d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/d27/d31/d6d/l100 to d0/d9/d14/d1d/d60/d76/d7b/d8d/l136 0 2024-02-16T19:29:22.777 INFO:tasks.workunit.client.0.smithi022.stdout:4/803: creat d4/d6/ffc x:0 0 0 2024-02-16T19:29:22.779 INFO:tasks.workunit.client.1.smithi117.stdout:8/967: creat d4/d12/d2a/d6d/de3/f135 x:0 0 0 2024-02-16T19:29:22.779 INFO:tasks.workunit.client.0.smithi022.stdout:9/842: dread - db/d14/d16/d5d/d65/d6b/fe0 zero size 2024-02-16T19:29:22.781 INFO:tasks.workunit.client.1.smithi117.stdout:4/944: truncate d0/da/d21/d3d/d28/d66/d97/d6d/fda 694243 0 2024-02-16T19:29:22.783 INFO:tasks.workunit.client.1.smithi117.stdout:9/841: mkdir dd/db5/dc4/dcf/d11d 0 2024-02-16T19:29:22.784 INFO:tasks.workunit.client.0.smithi022.stdout:2/819: dread d0/d2/d18/f22 [0,4194304] 0 2024-02-16T19:29:22.785 INFO:tasks.workunit.client.0.smithi022.stdout:8/931: rename d3/d12/da6/f1f to d3/d12/d2b/d46/f141 0 2024-02-16T19:29:22.785 INFO:tasks.workunit.client.1.smithi117.stdout:8/968: dwrite d4/d7/da6/d3a/d65/d96/fc7 [0,4194304] 0 2024-02-16T19:29:22.785 INFO:tasks.workunit.client.0.smithi022.stdout:7/893: rmdir d9/da/d74 39 2024-02-16T19:29:22.786 INFO:tasks.workunit.client.1.smithi117.stdout:8/969: write d4/d12/d2a/d64/d113/d123/f11f [162579,11164] 0 2024-02-16T19:29:22.787 INFO:tasks.workunit.client.1.smithi117.stdout:2/948: rename d0/d13/d75/f59 to d0/d13/dc4/de4/d12a/f12c 0 2024-02-16T19:29:22.789 INFO:tasks.workunit.client.1.smithi117.stdout:0/921: mknod d0/d9/d14/d1d/d1e/c137 0 2024-02-16T19:29:22.793 INFO:tasks.workunit.client.1.smithi117.stdout:9/842: mkdir dd/d45/d31/d63/dba/d11e 0 2024-02-16T19:29:22.794 INFO:tasks.workunit.client.0.smithi022.stdout:4/804: rename d4/d6/dd/d22/da8/dc4/d6d to d4/d6/dd/d22/da8/dc4/d86/dfd 0 2024-02-16T19:29:22.796 INFO:tasks.workunit.client.1.smithi117.stdout:7/898: getdents d4/d1c 0 2024-02-16T19:29:22.797 INFO:tasks.workunit.client.0.smithi022.stdout:7/894: mkdir d9/da/d2c/d46/de3/d132 0 2024-02-16T19:29:22.799 INFO:tasks.workunit.client.0.smithi022.stdout:9/843: mkdir db/d14/d16/d5d/d65/d6b/d11f 0 2024-02-16T19:29:22.800 INFO:tasks.workunit.client.1.smithi117.stdout:8/970: creat d4/d12/d31/d8e/f136 x:0 0 0 2024-02-16T19:29:22.801 INFO:tasks.workunit.client.1.smithi117.stdout:1/824: sync 2024-02-16T19:29:22.802 INFO:tasks.workunit.client.1.smithi117.stdout:2/949: read d0/d13/d4d/f50 [99114,85562] 0 2024-02-16T19:29:22.803 INFO:tasks.workunit.client.0.smithi022.stdout:8/932: fdatasync d3/d12/d2b/dca/f137 0 2024-02-16T19:29:22.804 INFO:tasks.workunit.client.1.smithi117.stdout:9/843: write dd/d47/d48/fbc [3758557,5580] 0 2024-02-16T19:29:22.805 INFO:tasks.workunit.client.1.smithi117.stdout:7/899: chown d4/d85/cbf 5921524 1 2024-02-16T19:29:22.805 INFO:tasks.workunit.client.1.smithi117.stdout:9/844: chown dd/d45/d73/f88 423744 1 2024-02-16T19:29:22.806 INFO:tasks.workunit.client.0.smithi022.stdout:9/844: fsync db/d14/d16/d49/fce 0 2024-02-16T19:29:22.806 INFO:tasks.workunit.client.1.smithi117.stdout:7/900: chown d4/d1c/d87/d89/fd4 6241810 1 2024-02-16T19:29:22.807 INFO:tasks.workunit.client.1.smithi117.stdout:6/916: creat d5/d4b/d44/f130 x:0 0 0 2024-02-16T19:29:22.807 INFO:tasks.workunit.client.1.smithi117.stdout:9/845: dread dd/d47/d48/dc0/f4d [4194304,4194304] 0 2024-02-16T19:29:22.808 INFO:tasks.workunit.client.1.smithi117.stdout:8/971: readlink d4/d7/l29 0 2024-02-16T19:29:22.808 INFO:tasks.workunit.client.1.smithi117.stdout:7/901: write d4/d13/d63/ddd/f121 [546973,74553] 0 2024-02-16T19:29:22.808 INFO:tasks.workunit.client.1.smithi117.stdout:6/917: stat d5/db/c8e 0 2024-02-16T19:29:22.809 INFO:tasks.workunit.client.0.smithi022.stdout:4/805: creat d4/d6/dd/d22/da8/dc4/d86/dfd/df2/ffe x:0 0 0 2024-02-16T19:29:22.809 INFO:tasks.workunit.client.0.smithi022.stdout:8/933: dwrite d3/d12/d2b/fde [0,4194304] 0 2024-02-16T19:29:22.813 INFO:tasks.workunit.client.0.smithi022.stdout:6/858: getdents d1/d8/da/d7b 0 2024-02-16T19:29:22.814 INFO:tasks.workunit.client.1.smithi117.stdout:2/950: symlink d0/d13/d4d/d53/d58/l12d 0 2024-02-16T19:29:22.816 INFO:tasks.workunit.client.0.smithi022.stdout:8/934: dread d3/d12/da6/f28 [0,4194304] 0 2024-02-16T19:29:22.820 INFO:tasks.workunit.client.1.smithi117.stdout:8/972: creat d4/d7/d6a/d72/d84/f137 x:0 0 0 2024-02-16T19:29:22.826 INFO:tasks.workunit.client.1.smithi117.stdout:0/922: rename d0/d78/d135/f66 to d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/d27/f138 0 2024-02-16T19:29:22.827 INFO:tasks.workunit.client.1.smithi117.stdout:5/959: dread d2/db3/fa0 [0,4194304] 0 2024-02-16T19:29:22.828 INFO:tasks.workunit.client.1.smithi117.stdout:1/825: symlink d2/d19/d59/d89/d69/d80/d8e/df5/l11b 0 2024-02-16T19:29:22.829 INFO:tasks.workunit.client.0.smithi022.stdout:1/872: write d6/f5b [1306669,7307] 0 2024-02-16T19:29:22.831 INFO:tasks.workunit.client.1.smithi117.stdout:8/973: fdatasync d4/d7/f9d 0 2024-02-16T19:29:22.833 INFO:tasks.workunit.client.1.smithi117.stdout:7/902: symlink d4/d1c/d39/d46/da4/d142/l14f 0 2024-02-16T19:29:22.836 INFO:tasks.workunit.client.1.smithi117.stdout:2/951: rename d0/d3/d22/d37/d9d/ce7 to d0/d3/d34/d94/d82/c12e 0 2024-02-16T19:29:22.844 INFO:tasks.workunit.client.1.smithi117.stdout:5/960: mknod d2/d9/d12e/d96/dc4/dca/df3/c131 0 2024-02-16T19:29:22.845 INFO:tasks.workunit.client.0.smithi022.stdout:1/873: fdatasync d6/d8/dba/fce 0 2024-02-16T19:29:22.846 INFO:tasks.workunit.client.1.smithi117.stdout:1/826: mkdir d2/d19/d59/d89/d69/d11c 0 2024-02-16T19:29:22.847 INFO:tasks.workunit.client.1.smithi117.stdout:4/945: dwrite d0/da/d21/f27 [4194304,4194304] 0 2024-02-16T19:29:22.848 INFO:tasks.workunit.client.0.smithi022.stdout:2/820: write d0/f17 [4380894,123115] 0 2024-02-16T19:29:22.848 INFO:tasks.workunit.client.0.smithi022.stdout:8/935: symlink d3/d12/d2b/l142 0 2024-02-16T19:29:22.849 INFO:tasks.workunit.client.1.smithi117.stdout:4/946: chown d0/da/d21/d3d/d28/d64/d90/f114 235738 1 2024-02-16T19:29:22.850 INFO:tasks.workunit.client.1.smithi117.stdout:8/974: mkdir d4/d12/d31/d8e/dbf/d138 0 2024-02-16T19:29:22.851 INFO:tasks.workunit.client.0.smithi022.stdout:7/895: link d9/da/d2c/f8d d9/da/f133 0 2024-02-16T19:29:22.853 INFO:tasks.workunit.client.1.smithi117.stdout:7/903: rename d4/d13/da3/fb1 to d4/d13/d63/d71/f150 0 2024-02-16T19:29:22.855 INFO:tasks.workunit.client.1.smithi117.stdout:2/952: creat d0/d3/d34/d94/d82/f12f x:0 0 0 2024-02-16T19:29:22.856 INFO:tasks.workunit.client.1.smithi117.stdout:4/947: dwrite d0/da/f33 [8388608,4194304] 0 2024-02-16T19:29:22.860 INFO:tasks.workunit.client.1.smithi117.stdout:6/918: dwrite d5/d4b/f46 [0,4194304] 0 2024-02-16T19:29:22.864 INFO:tasks.workunit.client.0.smithi022.stdout:6/859: write d1/d8/da/d7b/fb2 [631321,62473] 0 2024-02-16T19:29:22.877 INFO:tasks.workunit.client.0.smithi022.stdout:9/845: dwrite db/d14/d19/d1a/f50 [0,4194304] 0 2024-02-16T19:29:22.878 INFO:tasks.workunit.client.0.smithi022.stdout:4/806: dwrite d4/d6/dd/f2c [4194304,4194304] 0 2024-02-16T19:29:22.885 INFO:tasks.workunit.client.0.smithi022.stdout:2/821: write d0/d2/d18/d104/f9b [642596,60969] 0 2024-02-16T19:29:22.886 INFO:tasks.workunit.client.0.smithi022.stdout:2/822: read - d0/d2/dce/ff8 zero size 2024-02-16T19:29:22.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:22 smithi117 ceph-mon[35309]: pgmap v14: 65 pgs: 65 active+clean; 3.5 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 117 MiB/s rd, 266 MiB/s wr, 593 op/s 2024-02-16T19:29:22.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:22 smithi117 ceph-mon[35309]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' 2024-02-16T19:29:22.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:22 smithi117 ceph-mon[35309]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' 2024-02-16T19:29:22.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:22 smithi117 ceph-mon[35309]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:29:22.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:22 smithi117 ceph-mon[35309]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:29:22.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:22 smithi117 ceph-mon[35309]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:29:22.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:22 smithi117 ceph-mon[35309]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:29:22.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:22 smithi117 ceph-mon[35309]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:29:22.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:22 smithi117 ceph-mon[35309]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:29:22.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:22 smithi117 ceph-mon[35309]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' 2024-02-16T19:29:22.891 INFO:tasks.workunit.client.0.smithi022.stdout:1/874: rename d6/d8/d22/f3d to d6/d8/f111 0 2024-02-16T19:29:22.893 INFO:tasks.workunit.client.0.smithi022.stdout:8/936: dwrite d3/fd4 [0,4194304] 0 2024-02-16T19:29:22.895 INFO:tasks.workunit.client.0.smithi022.stdout:9/846: creat db/d14/dc8/dc9/d105/f120 x:0 0 0 2024-02-16T19:29:22.897 INFO:tasks.workunit.client.0.smithi022.stdout:6/860: write d1/d8/d23/f84 [321061,65277] 0 2024-02-16T19:29:22.901 INFO:tasks.workunit.client.0.smithi022.stdout:7/896: truncate d9/da/d2c/d46/d73/fe1 1261393 0 2024-02-16T19:29:22.906 INFO:tasks.workunit.client.0.smithi022.stdout:1/875: mknod d6/d8/d22/d47/d51/d8d/c112 0 2024-02-16T19:29:22.911 INFO:tasks.workunit.client.0.smithi022.stdout:6/861: fsync d1/d8/da/d29/d61/d57/f60 0 2024-02-16T19:29:22.914 INFO:tasks.workunit.client.0.smithi022.stdout:2/823: fsync d0/d2/d2e/f72 0 2024-02-16T19:29:22.915 INFO:tasks.workunit.client.0.smithi022.stdout:4/807: dwrite d4/d6/dd/d22/d30/fcf [0,4194304] 0 2024-02-16T19:29:22.919 INFO:tasks.workunit.client.0.smithi022.stdout:7/897: creat d9/da/d2c/d46/d73/dca/f134 x:0 0 0 2024-02-16T19:29:22.921 INFO:tasks.workunit.client.0.smithi022.stdout:1/876: dread - d6/d66/f9c zero size 2024-02-16T19:29:22.922 INFO:tasks.workunit.client.0.smithi022.stdout:1/877: write d6/d8/d22/f1b [825034,102242] 0 2024-02-16T19:29:22.924 INFO:tasks.workunit.client.0.smithi022.stdout:8/937: write d3/d12/f9e [2808963,108086] 0 2024-02-16T19:29:22.929 INFO:tasks.workunit.client.0.smithi022.stdout:6/862: mkdir d1/d8/da/d29/deb/dfd/d8b/d124 0 2024-02-16T19:29:22.931 INFO:tasks.workunit.client.0.smithi022.stdout:2/824: fdatasync d0/d2/d18/d19/ddf/feb 0 2024-02-16T19:29:22.938 INFO:tasks.workunit.client.0.smithi022.stdout:8/938: mknod d3/d12/d2b/d46/c143 0 2024-02-16T19:29:22.939 INFO:tasks.workunit.client.0.smithi022.stdout:9/847: link db/d14/d16/d49/c115 db/d14/d16/d28/daa/c121 0 2024-02-16T19:29:22.944 INFO:tasks.workunit.client.0.smithi022.stdout:6/863: creat d1/d8/da/d29/d61/d57/f125 x:0 0 0 2024-02-16T19:29:22.945 INFO:tasks.workunit.client.1.smithi117.stdout:9/846: sync 2024-02-16T19:29:22.947 INFO:tasks.workunit.client.0.smithi022.stdout:2/825: chown d0/c5 3955 1 2024-02-16T19:29:22.949 INFO:tasks.workunit.client.0.smithi022.stdout:7/898: write d9/da/d1f/d40/dde/fe7 [742980,74657] 0 2024-02-16T19:29:22.952 INFO:tasks.workunit.client.0.smithi022.stdout:8/939: mknod d3/d12/d2b/da9/c144 0 2024-02-16T19:29:22.953 INFO:tasks.workunit.client.1.smithi117.stdout:5/961: mknod d2/d9/d12e/d103/c132 0 2024-02-16T19:29:22.956 INFO:tasks.workunit.client.0.smithi022.stdout:1/878: dwrite d6/d66/fb5 [0,4194304] 0 2024-02-16T19:29:22.956 INFO:tasks.workunit.client.0.smithi022.stdout:2/826: rename d0/d2/d18/d19/d40/l7f to d0/d2/d2e/d82/da1/d10d/l115 0 2024-02-16T19:29:22.958 INFO:tasks.workunit.client.1.smithi117.stdout:1/827: creat d2/d19/d59/d89/d69/dad/d100/f11d x:0 0 0 2024-02-16T19:29:22.959 INFO:tasks.workunit.client.1.smithi117.stdout:1/828: chown d2/d19/d59/d89/d8d/f117 105045760 1 2024-02-16T19:29:22.961 INFO:tasks.workunit.client.1.smithi117.stdout:0/923: truncate d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/d27/d4e/f62 165675 0 2024-02-16T19:29:22.964 INFO:tasks.workunit.client.0.smithi022.stdout:8/940: dread d3/d12/da6/f4c [0,4194304] 0 2024-02-16T19:29:22.964 INFO:tasks.workunit.client.0.smithi022.stdout:4/808: getdents d4/d6/dd/d22/da8/dc4 0 2024-02-16T19:29:22.968 INFO:tasks.workunit.client.0.smithi022.stdout:7/899: dwrite d9/d5d/dc7/d118/f96 [0,4194304] 0 2024-02-16T19:29:22.969 INFO:tasks.workunit.client.0.smithi022.stdout:7/900: chown d9/f124 99 1 2024-02-16T19:29:22.973 INFO:tasks.workunit.client.0.smithi022.stdout:8/941: dread d3/d12/d2b/dca/fdf [0,4194304] 0 2024-02-16T19:29:22.973 INFO:tasks.workunit.client.1.smithi117.stdout:4/948: creat d0/da/d21/db4/f133 x:0 0 0 2024-02-16T19:29:22.974 INFO:tasks.workunit.client.0.smithi022.stdout:8/942: dread - d3/d12/d2b/fa5 zero size 2024-02-16T19:29:22.975 INFO:tasks.workunit.client.0.smithi022.stdout:8/943: dread d3/dd3/fd7 [0,4194304] 0 2024-02-16T19:29:22.976 INFO:tasks.workunit.client.0.smithi022.stdout:8/944: dread - d3/d12/d6a/fb2 zero size 2024-02-16T19:29:22.984 INFO:tasks.workunit.client.1.smithi117.stdout:2/953: write d0/d3/d10/f95 [3069237,9017] 0 2024-02-16T19:29:22.988 INFO:tasks.workunit.client.0.smithi022.stdout:1/879: dwrite d6/d8/d22/d3f/f84 [0,4194304] 0 2024-02-16T19:29:22.988 INFO:tasks.workunit.client.1.smithi117.stdout:8/975: creat d4/d7/dfa/f139 x:0 0 0 2024-02-16T19:29:22.989 INFO:tasks.workunit.client.1.smithi117.stdout:5/962: dread d2/d5d/fc9 [0,4194304] 0 2024-02-16T19:29:22.990 INFO:tasks.workunit.client.1.smithi117.stdout:8/976: chown d4/d12/d2a/d64/dec/cb8 1 1 2024-02-16T19:29:22.990 INFO:tasks.workunit.client.0.smithi022.stdout:2/827: link d0/d2/d2e/d73/c89 d0/d32/d3b/c116 0 2024-02-16T19:29:22.991 INFO:tasks.workunit.client.1.smithi117.stdout:8/977: chown d4/d73/dff 731122767 1 2024-02-16T19:29:22.992 INFO:tasks.workunit.client.1.smithi117.stdout:2/954: dwrite d0/d13/d4d/d80/f108 [0,4194304] 0 2024-02-16T19:29:22.993 INFO:tasks.workunit.client.0.smithi022.stdout:4/809: creat d4/d6/d14/d64/d4e/d54/fff x:0 0 0 2024-02-16T19:29:22.994 INFO:tasks.workunit.client.0.smithi022.stdout:4/810: read - d4/d6/dd/df/d10/fd8 zero size 2024-02-16T19:29:22.999 INFO:tasks.workunit.client.1.smithi117.stdout:9/847: dwrite dd/d47/d5b/fc6 [0,4194304] 0 2024-02-16T19:29:23.001 INFO:tasks.workunit.client.0.smithi022.stdout:1/880: write d6/d8/d22/d16/d4f/ffd [118048,126928] 0 2024-02-16T19:29:23.002 INFO:tasks.workunit.client.1.smithi117.stdout:9/848: stat dd/d47/d5b/d5f 0 2024-02-16T19:29:23.002 INFO:tasks.workunit.client.0.smithi022.stdout:2/828: rmdir d0/d2/d18/d19/d40 39 2024-02-16T19:29:23.004 INFO:tasks.workunit.client.1.smithi117.stdout:7/904: creat d4/d1c/d39/d9f/f151 x:0 0 0 2024-02-16T19:29:23.005 INFO:tasks.workunit.client.1.smithi117.stdout:5/963: rmdir d2/d9/d3c/d42 39 2024-02-16T19:29:23.007 INFO:tasks.workunit.client.1.smithi117.stdout:4/949: dread d0/da/d21/f3f [0,4194304] 0 2024-02-16T19:29:23.008 INFO:tasks.workunit.client.1.smithi117.stdout:6/919: truncate d5/d9c/daa/fb2 960835 0 2024-02-16T19:29:23.008 INFO:tasks.workunit.client.1.smithi117.stdout:4/950: chown d0/f25 20 1 2024-02-16T19:29:23.008 INFO:tasks.workunit.client.0.smithi022.stdout:7/901: link d9/da/d1f/l12 d9/da/l135 0 2024-02-16T19:29:23.009 INFO:tasks.workunit.client.1.smithi117.stdout:2/955: read d0/d13/d4d/d53/fa4 [188992,58498] 0 2024-02-16T19:29:23.010 INFO:tasks.workunit.client.1.smithi117.stdout:6/920: write d5/d9c/f9e [3812799,77926] 0 2024-02-16T19:29:23.011 INFO:tasks.workunit.client.1.smithi117.stdout:8/978: fsync d4/d12/d2a/d64/dec/d76/d121/d12e/f49 0 2024-02-16T19:29:23.012 INFO:tasks.workunit.client.1.smithi117.stdout:4/951: chown d0/da/d21/d3d/d28/d64/d73/d91/d9e/cd4 1 1 2024-02-16T19:29:23.012 INFO:tasks.workunit.client.1.smithi117.stdout:2/956: readlink d0/d13/d4d/d53/d58/dbc/lf4 0 2024-02-16T19:29:23.013 INFO:tasks.workunit.client.0.smithi022.stdout:7/902: creat d9/da/d2c/d46/f136 x:0 0 0 2024-02-16T19:29:23.013 INFO:tasks.workunit.client.1.smithi117.stdout:6/921: write d5/d4b/d20/d69/fc2 [832545,28968] 0 2024-02-16T19:29:23.015 INFO:tasks.workunit.client.1.smithi117.stdout:6/922: chown d5/db/d7d/d32/da8/ce9 3450520 1 2024-02-16T19:29:23.018 INFO:tasks.workunit.client.0.smithi022.stdout:4/811: rename d4/d6/dd/d22/d30/fcf to d4/d6/f100 0 2024-02-16T19:29:23.018 INFO:tasks.workunit.client.1.smithi117.stdout:9/849: dread dd/d45/d73/fee [0,4194304] 0 2024-02-16T19:29:23.019 INFO:tasks.workunit.client.1.smithi117.stdout:5/964: creat d2/d3/d21/d49/d59/daa/f133 x:0 0 0 2024-02-16T19:29:23.020 INFO:tasks.workunit.client.1.smithi117.stdout:9/850: write dd/d47/d5b/d5f/fc5 [909538,4855] 0 2024-02-16T19:29:23.020 INFO:tasks.workunit.client.1.smithi117.stdout:5/965: chown d2/d9/d12e/d96/dbc/d30/c83 29096 1 2024-02-16T19:29:23.021 INFO:tasks.workunit.client.1.smithi117.stdout:5/966: read - d2/d3/d21/d2a/f93 zero size 2024-02-16T19:29:23.022 INFO:tasks.workunit.client.0.smithi022.stdout:4/812: symlink d4/d6/dd/df/d10/ddc/l101 0 2024-02-16T19:29:23.023 INFO:tasks.workunit.client.1.smithi117.stdout:5/967: write d2/d4c/d4f/d67/d11e/d62/fdb [5179645,63423] 0 2024-02-16T19:29:23.023 INFO:tasks.workunit.client.0.smithi022.stdout:4/813: fdatasync d4/d6/d14/dc7/d8d/f99 0 2024-02-16T19:29:23.025 INFO:tasks.workunit.client.1.smithi117.stdout:1/829: getdents d2/d4/dbb 0 2024-02-16T19:29:23.025 INFO:tasks.workunit.client.0.smithi022.stdout:7/903: mkdir d9/da/d2c/d2f/d137 0 2024-02-16T19:29:23.027 INFO:tasks.workunit.client.0.smithi022.stdout:8/945: write d3/d12/fb7 [3745637,16742] 0 2024-02-16T19:29:23.027 INFO:tasks.workunit.client.1.smithi117.stdout:8/979: fsync d4/d7/da6/d3a/d65/d96/fce 0 2024-02-16T19:29:23.027 INFO:tasks.workunit.client.1.smithi117.stdout:8/980: chown d4/d73/dff 0 1 2024-02-16T19:29:23.028 INFO:tasks.workunit.client.0.smithi022.stdout:1/881: rename d6/d66/fb5 to d6/f113 0 2024-02-16T19:29:23.034 INFO:tasks.workunit.client.0.smithi022.stdout:8/946: dwrite d3/d12/d17/d64/d8f/fec [0,4194304] 0 2024-02-16T19:29:23.034 INFO:tasks.workunit.client.1.smithi117.stdout:6/923: mknod d5/d4b/d20/da4/c131 0 2024-02-16T19:29:23.036 INFO:tasks.workunit.client.0.smithi022.stdout:5/900: sync 2024-02-16T19:29:23.039 INFO:tasks.workunit.client.0.smithi022.stdout:4/814: rename d4/d6/l52 to d4/d6/d14/d64/d4e/d54/dba/l102 0 2024-02-16T19:29:23.041 INFO:tasks.workunit.client.0.smithi022.stdout:1/882: unlink d6/d8/d22/d47/d54/c60 0 2024-02-16T19:29:23.043 INFO:tasks.workunit.client.1.smithi117.stdout:9/851: creat dd/d47/d5b/d5f/d77/d7c/f11f x:0 0 0 2024-02-16T19:29:23.043 INFO:tasks.workunit.client.0.smithi022.stdout:8/947: dwrite d3/d12/d2b/d54/d74/f128 [0,4194304] 0 2024-02-16T19:29:23.043 INFO:tasks.workunit.client.0.smithi022.stdout:1/883: write d6/d8/d22/d47/fd3 [3757759,92867] 0 2024-02-16T19:29:23.045 INFO:tasks.workunit.client.0.smithi022.stdout:1/884: write d6/d8/d22/f1b [1075975,75616] 0 2024-02-16T19:29:23.048 INFO:tasks.workunit.client.0.smithi022.stdout:1/885: dread - d6/d8/d6c/dc9/fec zero size 2024-02-16T19:29:23.050 INFO:tasks.workunit.client.0.smithi022.stdout:5/901: fdatasync dd/d1f/d48/d21/d64/d11d/d6a/f7f 0 2024-02-16T19:29:23.050 INFO:tasks.workunit.client.1.smithi117.stdout:9/852: dwrite dd/d45/d31/d63/dba/df0/daf/df4/d10d/f11b [0,4194304] 0 2024-02-16T19:29:23.053 INFO:tasks.workunit.client.0.smithi022.stdout:2/829: dwrite d0/d2/d18/d19/d40/fa5 [0,4194304] 0 2024-02-16T19:29:23.053 INFO:tasks.workunit.client.1.smithi117.stdout:0/924: sync 2024-02-16T19:29:23.054 INFO:tasks.workunit.client.1.smithi117.stdout:7/905: dwrite d4/d1c/d87/d89/faa [0,4194304] 0 2024-02-16T19:29:23.059 INFO:tasks.workunit.client.0.smithi022.stdout:7/904: rename d9/da/d1f/d40/dde/f11d to d9/da/d2c/d46/d73/f138 0 2024-02-16T19:29:23.059 INFO:tasks.workunit.client.1.smithi117.stdout:1/830: creat d2/d4/dbb/d10b/dcc/f11e x:0 0 0 2024-02-16T19:29:23.060 INFO:tasks.workunit.client.0.smithi022.stdout:7/905: stat d9/da/d1f/d40/fb9 0 2024-02-16T19:29:23.061 INFO:tasks.workunit.client.0.smithi022.stdout:4/815: creat d4/d6/d14/d64/d4e/d54/d5a/f103 x:0 0 0 2024-02-16T19:29:23.062 INFO:tasks.workunit.client.0.smithi022.stdout:4/816: fdatasync d4/d6/dd/d22/f59 0 2024-02-16T19:29:23.064 INFO:tasks.workunit.client.1.smithi117.stdout:8/981: rmdir d4/d7/d6a/d72 39 2024-02-16T19:29:23.068 INFO:tasks.workunit.client.0.smithi022.stdout:8/948: mknod d3/d12/d117/c145 0 2024-02-16T19:29:23.070 INFO:tasks.workunit.client.1.smithi117.stdout:4/952: creat d0/da/d21/d3d/d28/d64/d8a/d92/de8/d113/f134 x:0 0 0 2024-02-16T19:29:23.075 INFO:tasks.workunit.client.0.smithi022.stdout:4/817: dread d4/d6/dd/d22/da8/dc4/fc3 [0,4194304] 0 2024-02-16T19:29:23.081 INFO:tasks.workunit.client.0.smithi022.stdout:2/830: readlink d0/lc 0 2024-02-16T19:29:23.081 INFO:tasks.workunit.client.0.smithi022.stdout:2/831: chown d0/d2/d2e/d73 11019 1 2024-02-16T19:29:23.084 INFO:tasks.workunit.client.1.smithi117.stdout:5/968: dwrite d2/d3/d21/d2a/fa6 [0,4194304] 0 2024-02-16T19:29:23.084 INFO:tasks.workunit.client.1.smithi117.stdout:2/957: creat d0/f130 x:0 0 0 2024-02-16T19:29:23.087 INFO:tasks.workunit.client.1.smithi117.stdout:7/906: read d4/f11 [3887291,27124] 0 2024-02-16T19:29:23.088 INFO:tasks.workunit.client.0.smithi022.stdout:1/886: dread d6/d8/d22/d16/d17/d5d/f95 [0,4194304] 0 2024-02-16T19:29:23.089 INFO:tasks.workunit.client.1.smithi117.stdout:0/925: creat d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/d27/df9/f139 x:0 0 0 2024-02-16T19:29:23.091 INFO:tasks.workunit.client.0.smithi022.stdout:8/949: read - d3/d12/f9c zero size 2024-02-16T19:29:23.092 INFO:tasks.workunit.client.0.smithi022.stdout:8/950: readlink d3/d25/d33/l44 0 2024-02-16T19:29:23.092 INFO:tasks.workunit.client.1.smithi117.stdout:1/831: read - d2/d19/d59/d89/df4/dfc/fbc zero size 2024-02-16T19:29:23.093 INFO:tasks.workunit.client.0.smithi022.stdout:7/906: dread d9/da/d1f/d44/d5b/d95/fbf [0,4194304] 0 2024-02-16T19:29:23.093 INFO:tasks.workunit.client.0.smithi022.stdout:8/951: write d3/d25/f5c [2098304,49476] 0 2024-02-16T19:29:23.095 INFO:tasks.workunit.client.0.smithi022.stdout:8/952: chown d3/d12/d2b/f3f 54827 1 2024-02-16T19:29:23.096 INFO:tasks.workunit.client.0.smithi022.stdout:5/902: mknod dd/d1f/d48/d21/d2f/d52/d5c/c127 0 2024-02-16T19:29:23.096 INFO:tasks.workunit.client.0.smithi022.stdout:8/953: stat d3/d12/d2b/d54/db8/de5/l113 0 2024-02-16T19:29:23.099 INFO:tasks.workunit.client.0.smithi022.stdout:7/907: dread d9/da/f62 [0,4194304] 0 2024-02-16T19:29:23.100 INFO:tasks.workunit.client.1.smithi117.stdout:6/924: rename d5/d4b/d20/ff0 to d5/d4b/d71/f132 0 2024-02-16T19:29:23.102 INFO:tasks.workunit.client.0.smithi022.stdout:1/887: creat d6/d8/d22/d3f/dc5/f114 x:0 0 0 2024-02-16T19:29:23.103 INFO:tasks.workunit.client.1.smithi117.stdout:2/958: symlink d0/d3/d34/d94/d46/d68/d111/l131 0 2024-02-16T19:29:23.107 INFO:tasks.workunit.client.0.smithi022.stdout:2/832: write d0/d2/d18/d19/fab [836743,99256] 0 2024-02-16T19:29:23.109 INFO:tasks.workunit.client.1.smithi117.stdout:8/982: chown d4/d7/d6a/d72 1 1 2024-02-16T19:29:23.109 INFO:tasks.workunit.client.1.smithi117.stdout:0/926: dread d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/d27/f5f [0,4194304] 0 2024-02-16T19:29:23.113 INFO:tasks.workunit.client.0.smithi022.stdout:5/903: unlink dd/d1f/d48/d21/d2f/d40/d49/fa1 0 2024-02-16T19:29:23.114 INFO:tasks.workunit.client.1.smithi117.stdout:5/969: dread d2/d9/d12e/d52/d74/f11c [0,4194304] 0 2024-02-16T19:29:23.116 INFO:tasks.workunit.client.1.smithi117.stdout:5/970: dread d2/d9/d12e/d52/d74/fab [0,4194304] 0 2024-02-16T19:29:23.118 INFO:tasks.workunit.client.1.smithi117.stdout:2/959: creat d0/d3/d40/f132 x:0 0 0 2024-02-16T19:29:23.118 INFO:tasks.workunit.client.1.smithi117.stdout:4/953: truncate d0/da/d21/d3d/d28/d64/f67 1440773 0 2024-02-16T19:29:23.119 INFO:tasks.workunit.client.1.smithi117.stdout:6/925: dread d5/d9c/daa/f9a [4194304,4194304] 0 2024-02-16T19:29:23.120 INFO:tasks.workunit.client.0.smithi022.stdout:8/954: rename d3/d12/da6/f27 to d3/d12/d2b/d46/f146 0 2024-02-16T19:29:23.120 INFO:tasks.workunit.client.1.smithi117.stdout:2/960: fdatasync d0/d3/d34/d94/d46/d68/d93/ddc/feb 0 2024-02-16T19:29:23.121 INFO:tasks.workunit.client.1.smithi117.stdout:7/907: symlink d4/d1c/d39/l152 0 2024-02-16T19:29:23.122 INFO:tasks.workunit.client.1.smithi117.stdout:7/908: fdatasync d4/d1c/d39/d46/f57 0 2024-02-16T19:29:23.122 INFO:tasks.workunit.client.1.smithi117.stdout:1/832: mknod d2/d19/d59/d89/d7b/c11f 0 2024-02-16T19:29:23.124 INFO:tasks.workunit.client.1.smithi117.stdout:8/983: mkdir d4/d7/da6/d3a/d43/d13a 0 2024-02-16T19:29:23.127 INFO:tasks.workunit.client.0.smithi022.stdout:2/833: mkdir d0/d2/d18/d19/ddf/d10c/d117 0 2024-02-16T19:29:23.135 INFO:tasks.workunit.client.0.smithi022.stdout:8/955: mkdir d3/d12/d17/d10d/d147 0 2024-02-16T19:29:23.136 INFO:tasks.workunit.client.1.smithi117.stdout:2/961: creat d0/d13/d4d/d53/d58/f133 x:0 0 0 2024-02-16T19:29:23.138 INFO:tasks.workunit.client.1.smithi117.stdout:6/926: dwrite d5/db/d7d/d32/f5b [0,4194304] 0 2024-02-16T19:29:23.139 INFO:tasks.workunit.client.1.smithi117.stdout:4/954: rename d0/da/d21/d3d/d28/d66/d97/d6d/f7e to d0/da/d21/d3d/d28/d66/dd9/f135 0 2024-02-16T19:29:23.142 INFO:tasks.workunit.client.0.smithi022.stdout:4/818: getdents d4/d6/d14 0 2024-02-16T19:29:23.143 INFO:tasks.workunit.client.1.smithi117.stdout:7/909: truncate d4/d1c/d39/d46/d55/db8/fc5 506360 0 2024-02-16T19:29:23.147 INFO:tasks.workunit.client.1.smithi117.stdout:8/984: fsync d4/d7/da6/fee 0 2024-02-16T19:29:23.148 INFO:tasks.workunit.client.1.smithi117.stdout:8/985: chown d4/d12/c2f 26885 1 2024-02-16T19:29:23.148 INFO:tasks.workunit.client.0.smithi022.stdout:4/819: dwrite d4/d6/d14/dc7/fc0 [0,4194304] 0 2024-02-16T19:29:23.153 INFO:tasks.workunit.client.1.smithi117.stdout:0/927: dwrite d0/d9/d14/d1d/d60/d76/ffb [0,4194304] 0 2024-02-16T19:29:23.153 INFO:tasks.workunit.client.1.smithi117.stdout:2/962: rmdir d0/d3/d22 39 2024-02-16T19:29:23.155 INFO:tasks.workunit.client.0.smithi022.stdout:5/904: creat dd/d1f/d48/d21/d64/d11d/f128 x:0 0 0 2024-02-16T19:29:23.156 INFO:tasks.workunit.client.0.smithi022.stdout:5/905: chown dd/d1f/d48/d21/d2f/d40/d49/d4b/d119 1 1 2024-02-16T19:29:23.158 INFO:tasks.workunit.client.0.smithi022.stdout:7/908: truncate d9/da/f3a 1541127 0 2024-02-16T19:29:23.159 INFO:tasks.workunit.client.0.smithi022.stdout:8/956: symlink d3/d12/db0/l148 0 2024-02-16T19:29:23.161 INFO:tasks.workunit.client.0.smithi022.stdout:1/888: write d6/d8/d22/d3f/d99/fb0 [1808043,126937] 0 2024-02-16T19:29:23.163 INFO:tasks.workunit.client.0.smithi022.stdout:5/906: symlink dd/d1f/d48/d21/d2f/d52/d77/d95/l129 0 2024-02-16T19:29:23.167 INFO:tasks.workunit.client.0.smithi022.stdout:7/909: symlink d9/da/d2c/ded/l139 0 2024-02-16T19:29:23.167 INFO:tasks.workunit.client.0.smithi022.stdout:2/834: write d0/d2/d18/d19/d40/fcb [475849,8770] 0 2024-02-16T19:29:23.169 INFO:tasks.workunit.client.0.smithi022.stdout:2/835: dread - d0/d5d/dc4/f105 zero size 2024-02-16T19:29:23.170 INFO:tasks.workunit.client.1.smithi117.stdout:0/928: write d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/d27/df9/f139 [1672,74496] 0 2024-02-16T19:29:23.171 INFO:tasks.workunit.client.0.smithi022.stdout:8/957: creat d3/d12/d6a/f149 x:0 0 0 2024-02-16T19:29:23.171 INFO:tasks.workunit.client.0.smithi022.stdout:2/836: chown d0/d2/d2e/d73/fd8 94228 1 2024-02-16T19:29:23.171 INFO:tasks.workunit.client.0.smithi022.stdout:5/907: dread dd/d1f/d48/d21/d64/d11d/d6a/db0/fdd [0,4194304] 0 2024-02-16T19:29:23.172 INFO:tasks.workunit.client.1.smithi117.stdout:0/929: chown d0/d9/d14/d1d/d1e/c20 4387544 1 2024-02-16T19:29:23.172 INFO:tasks.workunit.client.0.smithi022.stdout:8/958: chown d3/d25/ddb 985889 1 2024-02-16T19:29:23.173 INFO:tasks.workunit.client.0.smithi022.stdout:5/908: chown dd/d1f/d2e/dd4/ddf/f114 218142 1 2024-02-16T19:29:23.173 INFO:tasks.workunit.client.0.smithi022.stdout:2/837: chown d0/d2/d18/d19/d40/d55/d59/l80 280416886 1 2024-02-16T19:29:23.175 INFO:tasks.workunit.client.1.smithi117.stdout:8/986: mknod d4/d12/d31/c13b 0 2024-02-16T19:29:23.176 INFO:tasks.workunit.client.1.smithi117.stdout:8/987: chown d4/d12/d2a/d64/dec/d76/d8c/fa4 1 1 2024-02-16T19:29:23.177 INFO:tasks.workunit.client.1.smithi117.stdout:8/988: chown d4/d12/d31/d8e/lba 0 1 2024-02-16T19:29:23.179 INFO:tasks.workunit.client.1.smithi117.stdout:6/927: write d5/f7 [3523545,70062] 0 2024-02-16T19:29:23.184 INFO:tasks.workunit.client.1.smithi117.stdout:7/910: symlink d4/d1c/d98/def/d148/l153 0 2024-02-16T19:29:23.186 INFO:tasks.workunit.client.0.smithi022.stdout:5/909: rename dd/d1f/d2e/dd4/c107 to dd/d1f/d48/d21/d2f/d40/d71/c12a 0 2024-02-16T19:29:23.187 INFO:tasks.workunit.client.1.smithi117.stdout:1/833: link d2/d1d/d60/df2/c114 d2/d19/d59/d89/d69/d80/d8e/c120 0 2024-02-16T19:29:23.188 INFO:tasks.workunit.client.0.smithi022.stdout:7/910: mkdir d9/da/d1f/d67/ddb/dd4/d13a 0 2024-02-16T19:29:23.192 INFO:tasks.workunit.client.0.smithi022.stdout:7/911: fdatasync d9/da9/dbc/fee 0 2024-02-16T19:29:23.192 INFO:tasks.workunit.client.0.smithi022.stdout:2/838: dread d0/d2/d2e/d82/df0/f8b [0,4194304] 0 2024-02-16T19:29:23.192 INFO:tasks.workunit.client.0.smithi022.stdout:7/912: truncate d9/da/d2c/d2f/d21/fec 232200 0 2024-02-16T19:29:23.192 INFO:tasks.workunit.client.0.smithi022.stdout:1/889: creat d6/d8/d22/d16/d17/f115 x:0 0 0 2024-02-16T19:29:23.192 INFO:tasks.workunit.client.0.smithi022.stdout:7/913: readlink d9/d5d/dc7/d118/la0 0 2024-02-16T19:29:23.193 INFO:tasks.workunit.client.0.smithi022.stdout:1/890: readlink d6/d8/d22/d16/d17/d26/l64 0 2024-02-16T19:29:23.193 INFO:tasks.workunit.client.1.smithi117.stdout:7/911: dwrite d4/d13/d63/fa2 [4194304,4194304] 0 2024-02-16T19:29:23.193 INFO:tasks.workunit.client.1.smithi117.stdout:7/912: readlink d4/d1c/d87/d89/l76 0 2024-02-16T19:29:23.194 INFO:tasks.workunit.client.0.smithi022.stdout:8/959: creat d3/d25/d138/f14a x:0 0 0 2024-02-16T19:29:23.196 INFO:tasks.workunit.client.1.smithi117.stdout:6/928: creat d5/d4b/d20/d69/f133 x:0 0 0 2024-02-16T19:29:23.199 INFO:tasks.workunit.client.1.smithi117.stdout:1/834: rename d2/d19/d59/d89/d69/d80/d8e/dea to d2/d19/d59/d89/d69/d80/d8e/d121 0 2024-02-16T19:29:23.200 INFO:tasks.workunit.client.1.smithi117.stdout:1/835: read - d2/d19/d59/d76/d91/f111 zero size 2024-02-16T19:29:23.200 INFO:tasks.workunit.client.1.smithi117.stdout:9/853: sync 2024-02-16T19:29:23.200 INFO:tasks.workunit.client.1.smithi117.stdout:7/913: mkdir d4/d1c/d87/df0/dfe/d154 0 2024-02-16T19:29:23.201 INFO:tasks.workunit.client.0.smithi022.stdout:7/914: chown d9/da/d1f/d44/d5b/d95/d10a/d113/c3f 24 1 2024-02-16T19:29:23.202 INFO:tasks.workunit.client.1.smithi117.stdout:9/854: chown dd/d45/d31/la3 0 1 2024-02-16T19:29:23.202 INFO:tasks.workunit.client.1.smithi117.stdout:7/914: write d4/d13/da3/fa7 [3827761,73843] 0 2024-02-16T19:29:23.208 INFO:tasks.workunit.client.1.smithi117.stdout:8/989: write d4/d7/da6/d3a/d65/d96/fce [1459565,59028] 0 2024-02-16T19:29:23.209 INFO:tasks.workunit.client.0.smithi022.stdout:5/910: write dd/d1f/d48/d21/d64/d11d/fda [102870,50002] 0 2024-02-16T19:29:23.212 INFO:tasks.workunit.client.1.smithi117.stdout:2/963: link d0/d3/d40/cec d0/d13/dc4/d10f/c134 0 2024-02-16T19:29:23.214 INFO:tasks.workunit.client.1.smithi117.stdout:6/929: dread d5/db/f4a [0,4194304] 0 2024-02-16T19:29:23.215 INFO:tasks.workunit.client.0.smithi022.stdout:1/891: getdents d6/d8 0 2024-02-16T19:29:23.215 INFO:tasks.workunit.client.0.smithi022.stdout:7/915: dwrite d9/da9/dbc/fef [0,4194304] 0 2024-02-16T19:29:23.215 INFO:tasks.workunit.client.1.smithi117.stdout:6/930: chown d5/db/d7d/d32/da8/de1/lbb 12681317 1 2024-02-16T19:29:23.216 INFO:tasks.workunit.client.0.smithi022.stdout:8/960: write d3/d12/d2b/d54/d97/f139 [180868,45358] 0 2024-02-16T19:29:23.216 INFO:tasks.workunit.client.1.smithi117.stdout:0/930: getdents d0/d9/d14/d1d/d1e 0 2024-02-16T19:29:23.219 INFO:tasks.workunit.client.0.smithi022.stdout:8/961: chown d3/d25/c52 1454 1 2024-02-16T19:29:23.219 INFO:tasks.workunit.client.1.smithi117.stdout:1/836: dread d2/f57 [0,4194304] 0 2024-02-16T19:29:23.220 INFO:tasks.workunit.client.0.smithi022.stdout:2/839: dwrite d0/d2/d18/d1f/d6c/d99/fdd [4194304,4194304] 0 2024-02-16T19:29:23.220 INFO:tasks.workunit.client.1.smithi117.stdout:8/990: rename d4/d7/da6/d3a/d43/c4f to d4/d7/da6/d50/c13c 0 2024-02-16T19:29:23.221 INFO:tasks.workunit.client.1.smithi117.stdout:8/991: write d4/d12/d2a/f48 [2495565,61225] 0 2024-02-16T19:29:23.222 INFO:tasks.workunit.client.1.smithi117.stdout:7/915: unlink d4/d1c/d39/d46/d55/ce6 0 2024-02-16T19:29:23.223 INFO:tasks.workunit.client.1.smithi117.stdout:8/992: chown d4/d12/d2a/d64/dc6/l11d 484290191 1 2024-02-16T19:29:23.223 INFO:tasks.workunit.client.0.smithi022.stdout:5/911: truncate dd/d1f/d48/d21/d2f/d40/d4e/d56/fa5 809811 0 2024-02-16T19:29:23.225 INFO:tasks.workunit.client.0.smithi022.stdout:1/892: read - d6/d8/d22/d67/d7e/fa5 zero size 2024-02-16T19:29:23.225 INFO:tasks.workunit.client.1.smithi117.stdout:2/964: chown d0/d3/d22/c24 3 1 2024-02-16T19:29:23.226 INFO:tasks.workunit.client.1.smithi117.stdout:6/931: creat d5/db/d7d/d32/da8/f134 x:0 0 0 2024-02-16T19:29:23.228 INFO:tasks.workunit.client.0.smithi022.stdout:7/916: rename d9/da/d2c/d46/f6f to d9/da9/f13b 0 2024-02-16T19:29:23.230 INFO:tasks.workunit.client.1.smithi117.stdout:9/855: creat dd/d47/d5b/d5f/d77/f120 x:0 0 0 2024-02-16T19:29:23.231 INFO:tasks.workunit.client.1.smithi117.stdout:1/837: symlink d2/d4/dbb/d7f/l122 0 2024-02-16T19:29:23.233 INFO:tasks.workunit.client.0.smithi022.stdout:5/912: rmdir dd/d1f/d48/d21/d2f/d52/d77 39 2024-02-16T19:29:23.234 INFO:tasks.workunit.client.0.smithi022.stdout:1/893: mknod d6/d8/d22/d16/d17/d1d/c116 0 2024-02-16T19:29:23.237 INFO:tasks.workunit.client.1.smithi117.stdout:2/965: creat d0/d3/d34/d94/d46/f135 x:0 0 0 2024-02-16T19:29:23.238 INFO:tasks.workunit.client.0.smithi022.stdout:7/917: fsync d9/da/d1f/f22 0 2024-02-16T19:29:23.239 INFO:tasks.workunit.client.0.smithi022.stdout:8/962: truncate d3/d12/d17/f3b 3672705 0 2024-02-16T19:29:23.241 INFO:tasks.workunit.client.0.smithi022.stdout:5/913: dwrite dd/d1f/d48/d21/d2f/ff8 [0,4194304] 0 2024-02-16T19:29:23.242 INFO:tasks.workunit.client.0.smithi022.stdout:1/894: dwrite d6/d8/d22/d47/fd3 [0,4194304] 0 2024-02-16T19:29:23.242 INFO:tasks.workunit.client.0.smithi022.stdout:5/914: readlink dd/d1f/d48/d21/d2f/d52/l106 0 2024-02-16T19:29:23.244 INFO:tasks.workunit.client.0.smithi022.stdout:1/895: chown d6/d8/d22/d16/d17/d26/l64 2 1 2024-02-16T19:29:23.245 INFO:tasks.workunit.client.0.smithi022.stdout:2/840: getdents d0/d2/dde 0 2024-02-16T19:29:23.246 INFO:tasks.workunit.client.1.smithi117.stdout:9/856: rename dd/d47/d5e to dd/d45/d31/d63/d121 0 2024-02-16T19:29:23.246 INFO:tasks.workunit.client.0.smithi022.stdout:1/896: read d6/d8/d22/d67/f9f [416318,15115] 0 2024-02-16T19:29:23.249 INFO:tasks.workunit.client.0.smithi022.stdout:2/841: truncate d0/d2/d18/d19/d40/fcb 990399 0 2024-02-16T19:29:23.258 INFO:tasks.workunit.client.1.smithi117.stdout:2/966: mkdir d0/d3/d34/d94/d46/d68/d76/da1/d56/d60/d136 0 2024-02-16T19:29:23.260 INFO:tasks.workunit.client.0.smithi022.stdout:3/832: sync 2024-02-16T19:29:23.261 INFO:tasks.workunit.client.1.smithi117.stdout:0/931: creat d0/d9/d14/d23/d33/f13a x:0 0 0 2024-02-16T19:29:23.265 INFO:tasks.workunit.client.1.smithi117.stdout:2/967: dwrite d0/d13/dc4/ffb [0,4194304] 0 2024-02-16T19:29:23.265 INFO:tasks.workunit.client.0.smithi022.stdout:5/915: creat dd/d1f/d48/d21/d2f/d40/d4e/d56/d9d/f12b x:0 0 0 2024-02-16T19:29:23.267 INFO:tasks.workunit.client.0.smithi022.stdout:1/897: fdatasync d6/d66/fb3 0 2024-02-16T19:29:23.269 INFO:tasks.workunit.client.0.smithi022.stdout:2/842: unlink d0/d5d/dc4/f105 0 2024-02-16T19:29:23.269 INFO:tasks.workunit.client.1.smithi117.stdout:1/838: dwrite d2/d1d/d1f/f44 [4194304,4194304] 0 2024-02-16T19:29:23.269 INFO:tasks.workunit.client.1.smithi117.stdout:5/971: sync 2024-02-16T19:29:23.270 INFO:tasks.workunit.client.1.smithi117.stdout:4/955: sync 2024-02-16T19:29:23.270 INFO:tasks.workunit.client.1.smithi117.stdout:7/916: link d4/d1c/d39/d46/d47/l74 d4/dd3/l155 0 2024-02-16T19:29:23.273 INFO:tasks.workunit.client.1.smithi117.stdout:8/993: creat d4/d7/d6a/f13d x:0 0 0 2024-02-16T19:29:23.278 INFO:tasks.workunit.client.0.smithi022.stdout:5/916: chown dd/d1f/d48/d21/d2f/d40/d4e/d56/fa5 77 1 2024-02-16T19:29:23.280 INFO:tasks.workunit.client.1.smithi117.stdout:2/968: unlink d0/d3/d22/d37/d9d/ff6 0 2024-02-16T19:29:23.286 INFO:tasks.workunit.client.1.smithi117.stdout:2/969: dwrite d0/d3/f126 [0,4194304] 0 2024-02-16T19:29:23.286 INFO:tasks.workunit.client.1.smithi117.stdout:4/956: creat d0/da/d21/d3d/d28/f136 x:0 0 0 2024-02-16T19:29:23.286 INFO:tasks.workunit.client.0.smithi022.stdout:2/843: symlink d0/d5d/l118 0 2024-02-16T19:29:23.288 INFO:tasks.workunit.client.0.smithi022.stdout:2/844: write d0/d2/d18/d104/f9b [799376,43112] 0 2024-02-16T19:29:23.288 INFO:tasks.workunit.client.0.smithi022.stdout:7/918: creat d9/f13c x:0 0 0 2024-02-16T19:29:23.288 INFO:tasks.workunit.client.1.smithi117.stdout:9/857: dwrite dd/d45/d73/f94 [0,4194304] 0 2024-02-16T19:29:23.289 INFO:tasks.workunit.client.0.smithi022.stdout:2/845: read - d0/d32/d3b/de5/f106 zero size 2024-02-16T19:29:23.289 INFO:tasks.workunit.client.0.smithi022.stdout:7/919: stat d9/da/d1f/d44/d5b/dfc 0 2024-02-16T19:29:23.292 INFO:tasks.workunit.client.1.smithi117.stdout:6/932: getdents d5/d4b/d44/d52/d81 0 2024-02-16T19:29:23.297 INFO:tasks.workunit.client.1.smithi117.stdout:8/994: truncate d4/d12/d2a/d64/dec/d76/d121/d12e/f60 46180 0 2024-02-16T19:29:23.298 INFO:tasks.workunit.client.0.smithi022.stdout:2/846: dwrite d0/d2/d18/d1f/d6c/d99/ff6 [0,4194304] 0 2024-02-16T19:29:23.300 INFO:tasks.workunit.client.0.smithi022.stdout:2/847: chown d0/d2/d2e/f2a 241577823 1 2024-02-16T19:29:23.306 INFO:tasks.workunit.client.0.smithi022.stdout:8/963: getdents d3/dd3 0 2024-02-16T19:29:23.306 INFO:tasks.workunit.client.1.smithi117.stdout:7/917: write d4/d1c/d39/d13e/f14a [273883,9238] 0 2024-02-16T19:29:23.307 INFO:tasks.workunit.client.0.smithi022.stdout:1/898: dread d6/d8/d22/d47/d54/f8a [0,4194304] 0 2024-02-16T19:29:23.312 INFO:tasks.workunit.client.0.smithi022.stdout:3/833: creat d7/d12/d18/dd/d24/d93/f11e x:0 0 0 2024-02-16T19:29:23.317 INFO:tasks.workunit.client.1.smithi117.stdout:4/957: rename d0/da/d21/d3d/c7d to d0/da/d21/db4/de4/c137 0 2024-02-16T19:29:23.318 INFO:tasks.workunit.client.0.smithi022.stdout:2/848: dread - d0/d2/d18/d1f/d6c/d99/fc7 zero size 2024-02-16T19:29:23.319 INFO:tasks.workunit.client.1.smithi117.stdout:4/958: truncate d0/da/d21/d3d/d28/d64/d73/d91/d37/f107 366196 0 2024-02-16T19:29:23.320 INFO:tasks.workunit.client.1.smithi117.stdout:4/959: write d0/da/d21/d3d/d28/d66/da2/fe1 [600358,115442] 0 2024-02-16T19:29:23.321 INFO:tasks.workunit.client.1.smithi117.stdout:0/932: creat d0/d9/d14/f13b x:0 0 0 2024-02-16T19:29:23.322 INFO:tasks.workunit.client.0.smithi022.stdout:7/920: write d9/da/d1f/d67/fcf [539688,82384] 0 2024-02-16T19:29:23.322 INFO:tasks.workunit.client.1.smithi117.stdout:0/933: write d0/d9/d14/d1d/d60/d76/d7b/d8d/f48 [3597731,119484] 0 2024-02-16T19:29:23.324 INFO:tasks.workunit.client.1.smithi117.stdout:8/995: symlink d4/d73/dff/l13e 0 2024-02-16T19:29:23.327 INFO:tasks.workunit.client.1.smithi117.stdout:2/970: unlink d0/d13/dc4/d10f/c134 0 2024-02-16T19:29:23.329 INFO:tasks.workunit.client.0.smithi022.stdout:5/917: truncate dd/d1f/d48/d21/d5d/fb3 3790702 0 2024-02-16T19:29:23.331 INFO:tasks.workunit.client.0.smithi022.stdout:2/849: mkdir d0/d2/d18/d19/d40/d55/d59/d119 0 2024-02-16T19:29:23.331 INFO:tasks.workunit.client.0.smithi022.stdout:9/848: sync 2024-02-16T19:29:23.331 INFO:tasks.workunit.client.0.smithi022.stdout:8/964: write d3/f2d [883321,4994] 0 2024-02-16T19:29:23.332 INFO:tasks.workunit.client.1.smithi117.stdout:2/971: dwrite d0/d13/dc4/ffb [4194304,4194304] 0 2024-02-16T19:29:23.332 INFO:tasks.workunit.client.1.smithi117.stdout:1/839: link d2/d1d/d1f/c104 d2/d19/d59/d76/d8c/dc7/dd2/c123 0 2024-02-16T19:29:23.333 INFO:tasks.workunit.client.1.smithi117.stdout:5/972: link d2/db3/db0/d9b/le5 d2/d9/d12e/d96/dbc/l134 0 2024-02-16T19:29:23.335 INFO:tasks.workunit.client.1.smithi117.stdout:9/858: stat dd/d47/d48/dc0/c51 0 2024-02-16T19:29:23.335 INFO:tasks.workunit.client.0.smithi022.stdout:3/834: creat d7/d12/d18/dd/d24/de5/f11f x:0 0 0 2024-02-16T19:29:23.336 INFO:tasks.workunit.client.0.smithi022.stdout:1/899: dread d6/d8/d22/f82 [0,4194304] 0 2024-02-16T19:29:23.337 INFO:tasks.workunit.client.1.smithi117.stdout:6/933: mkdir d5/d4b/d71/dc7/dce/ded/df3/d135 0 2024-02-16T19:29:23.337 INFO:tasks.workunit.client.0.smithi022.stdout:3/835: chown d7/d12/d18/dd/d3c/f115 968 1 2024-02-16T19:29:23.339 INFO:tasks.workunit.client.1.smithi117.stdout:4/960: dread - d0/da/d21/d3d/d28/d66/da2/dac/fc7 zero size 2024-02-16T19:29:23.340 INFO:tasks.workunit.client.1.smithi117.stdout:4/961: readlink d0/da/d21/d3d/d28/d64/d73/le5 0 2024-02-16T19:29:23.341 INFO:tasks.workunit.client.0.smithi022.stdout:8/965: dread - d3/d12/d92/d135/dc0/fd0 zero size 2024-02-16T19:29:23.343 INFO:tasks.workunit.client.0.smithi022.stdout:7/921: write d9/da/d2c/d46/de3/f11e [384996,43897] 0 2024-02-16T19:29:23.345 INFO:tasks.workunit.client.0.smithi022.stdout:2/850: symlink d0/d2/d2e/l11a 0 2024-02-16T19:29:23.346 INFO:tasks.workunit.client.1.smithi117.stdout:8/996: mkdir d4/d12/d2a/d6d/de3/d13f 0 2024-02-16T19:29:23.347 INFO:tasks.workunit.client.1.smithi117.stdout:8/997: chown d4/d7/da6/d3a/d65/c66 3 1 2024-02-16T19:29:23.347 INFO:tasks.workunit.client.0.smithi022.stdout:1/900: mknod d6/d8/d6c/d96/c117 0 2024-02-16T19:29:23.348 INFO:tasks.workunit.client.0.smithi022.stdout:1/901: read - d6/d8/dba/fce zero size 2024-02-16T19:29:23.349 INFO:tasks.workunit.client.0.smithi022.stdout:1/902: chown d6/d8/d22/d59/da7/fd1 1326554 1 2024-02-16T19:29:23.350 INFO:tasks.workunit.client.0.smithi022.stdout:3/836: truncate d7/d12/d18/dd/d31/d7f/fa5 2158478 0 2024-02-16T19:29:23.351 INFO:tasks.workunit.client.1.smithi117.stdout:2/972: mknod d0/d13/dc4/d10f/c137 0 2024-02-16T19:29:23.352 INFO:tasks.workunit.client.1.smithi117.stdout:2/973: chown d0/d3/d34/d94/d91/f107 15576 1 2024-02-16T19:29:23.353 INFO:tasks.workunit.client.1.smithi117.stdout:1/840: fdatasync d2/d1d/d60/d68/f8a 0 2024-02-16T19:29:23.355 INFO:tasks.workunit.client.1.smithi117.stdout:5/973: mknod d2/d4c/d4f/d67/c135 0 2024-02-16T19:29:23.358 INFO:tasks.workunit.client.0.smithi022.stdout:2/851: rename d0/d2/d18/d38/db6/cc5 to d0/d2/d2e/d82/df0/df3/c11b 0 2024-02-16T19:29:23.359 INFO:tasks.workunit.client.0.smithi022.stdout:2/852: dread - d0/d2/d18/d38/db6/fe4 zero size 2024-02-16T19:29:23.363 INFO:tasks.workunit.client.1.smithi117.stdout:7/918: dwrite d4/d1c/f37 [0,4194304] 0 2024-02-16T19:29:23.365 INFO:tasks.workunit.client.0.smithi022.stdout:3/837: fdatasync d7/d12/d18/dac/dd7/fdb 0 2024-02-16T19:29:23.367 INFO:tasks.workunit.client.1.smithi117.stdout:8/998: truncate d4/d7/fca 518558 0 2024-02-16T19:29:23.370 INFO:tasks.workunit.client.0.smithi022.stdout:7/922: creat d9/da/d2c/d2f/d137/f13d x:0 0 0 2024-02-16T19:29:23.370 INFO:tasks.workunit.client.1.smithi117.stdout:4/962: write d0/da/d21/d3d/d28/d64/d73/d91/f62 [561169,102017] 0 2024-02-16T19:29:23.371 INFO:tasks.workunit.client.1.smithi117.stdout:0/934: write d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/d27/d4e/f62 [830462,115928] 0 2024-02-16T19:29:23.374 INFO:tasks.workunit.client.1.smithi117.stdout:1/841: truncate d2/d1d/d60/f66 2964525 0 2024-02-16T19:29:23.374 INFO:tasks.workunit.client.1.smithi117.stdout:6/934: dwrite d5/d4b/d20/fb5 [0,4194304] 0 2024-02-16T19:29:23.374 INFO:tasks.workunit.client.0.smithi022.stdout:9/849: getdents db/d14/d16/d5d 0 2024-02-16T19:29:23.376 INFO:tasks.workunit.client.1.smithi117.stdout:9/859: mknod dd/d45/df2/c122 0 2024-02-16T19:29:23.377 INFO:tasks.workunit.client.0.smithi022.stdout:7/923: dwrite d9/da/d2c/d46/de3/f11e [0,4194304] 0 2024-02-16T19:29:23.378 INFO:tasks.workunit.client.0.smithi022.stdout:3/838: stat d7/d12/d18/dd/d24/d93/cfa 0 2024-02-16T19:29:23.380 INFO:tasks.workunit.client.1.smithi117.stdout:4/963: dwrite d0/da/d21/d3d/d28/d64/d73/d91/f62 [4194304,4194304] 0 2024-02-16T19:29:23.380 INFO:tasks.workunit.client.1.smithi117.stdout:9/860: read dd/d45/d31/d63/d121/f87 [174683,67315] 0 2024-02-16T19:29:23.380 INFO:tasks.workunit.client.0.smithi022.stdout:8/966: creat d3/d12/db0/f14b x:0 0 0 2024-02-16T19:29:23.381 INFO:tasks.workunit.client.1.smithi117.stdout:9/861: chown dd/d45/d31/d63/lf6 716224 1 2024-02-16T19:29:23.384 INFO:tasks.workunit.client.0.smithi022.stdout:2/853: mkdir d0/d2/dfe/d11c 0 2024-02-16T19:29:23.384 INFO:tasks.workunit.client.0.smithi022.stdout:3/839: dread d7/f105 [0,4194304] 0 2024-02-16T19:29:23.385 INFO:tasks.workunit.client.1.smithi117.stdout:2/974: mkdir d0/d3/d34/d94/d46/d68/d76/da1/d56/d60/d12b/d138 0 2024-02-16T19:29:23.387 INFO:tasks.workunit.client.0.smithi022.stdout:9/850: creat db/d14/d19/f122 x:0 0 0 2024-02-16T19:29:23.387 INFO:tasks.workunit.client.0.smithi022.stdout:2/854: stat d0/d2/d18/d19/f2f 0 2024-02-16T19:29:23.387 INFO:tasks.workunit.client.0.smithi022.stdout:3/840: chown d7/d12/d18/d4b/fff 1164820 1 2024-02-16T19:29:23.388 INFO:tasks.workunit.client.0.smithi022.stdout:1/903: link d6/d8/d6c/cf8 d6/d8/d22/d16/d17/d63/dee/d110/c118 0 2024-02-16T19:29:23.389 INFO:tasks.workunit.client.0.smithi022.stdout:7/924: dwrite d9/da/f60 [4194304,4194304] 0 2024-02-16T19:29:23.397 INFO:tasks.workunit.client.1.smithi117.stdout:6/935: rmdir d5/d4b/d20/da4 39 2024-02-16T19:29:23.401 INFO:tasks.workunit.client.0.smithi022.stdout:7/925: symlink d9/da/d1f/l13e 0 2024-02-16T19:29:23.402 INFO:tasks.workunit.client.1.smithi117.stdout:4/964: mknod d0/da/d21/d3d/d28/d66/d97/d6d/c138 0 2024-02-16T19:29:23.403 INFO:tasks.workunit.client.0.smithi022.stdout:2/855: truncate d0/d5d/dc4/fdc 612651 0 2024-02-16T19:29:23.404 INFO:tasks.workunit.client.0.smithi022.stdout:1/904: truncate d6/d8/d22/d67/fd6 63483 0 2024-02-16T19:29:23.404 INFO:tasks.workunit.client.1.smithi117.stdout:4/965: dread d0/da/d21/f3f [0,4194304] 0 2024-02-16T19:29:23.404 INFO:tasks.workunit.client.0.smithi022.stdout:2/856: write d0/d2/d18/f65 [2724724,119843] 0 2024-02-16T19:29:23.405 INFO:tasks.workunit.client.1.smithi117.stdout:8/999: getdents d4/d7/da6/d50/d10d 0 2024-02-16T19:29:23.405 INFO:tasks.workunit.client.0.smithi022.stdout:2/857: fsync d0/d2/d18/d1f/f21 0 2024-02-16T19:29:23.407 INFO:tasks.workunit.client.0.smithi022.stdout:8/967: rename d3/d12/da6/l30 to d3/d12/d17/d10d/d110/l14c 0 2024-02-16T19:29:23.408 INFO:tasks.workunit.client.1.smithi117.stdout:2/975: rename d0/d3/d34/d94/d46/d68/d111/l131 to d0/d13/dc4/d10f/l139 0 2024-02-16T19:29:23.409 INFO:tasks.workunit.client.0.smithi022.stdout:3/841: symlink d7/d12/d18/l120 0 2024-02-16T19:29:23.410 INFO:tasks.workunit.client.1.smithi117.stdout:1/842: fdatasync d2/ff 0 2024-02-16T19:29:23.412 INFO:tasks.workunit.client.0.smithi022.stdout:7/926: unlink d9/da/d1f/d40/c7b 0 2024-02-16T19:29:23.414 INFO:tasks.workunit.client.1.smithi117.stdout:6/936: creat d5/f136 x:0 0 0 2024-02-16T19:29:23.416 INFO:tasks.workunit.client.0.smithi022.stdout:1/905: dread - d6/d8/d22/d16/d17/d26/f91 zero size 2024-02-16T19:29:23.417 INFO:tasks.workunit.client.1.smithi117.stdout:4/966: dread d0/f5c [0,4194304] 0 2024-02-16T19:29:23.417 INFO:tasks.workunit.client.0.smithi022.stdout:7/927: dread d9/da/d2c/d46/f9a [0,4194304] 0 2024-02-16T19:29:23.420 INFO:tasks.workunit.client.1.smithi117.stdout:9/862: symlink dd/d45/d31/d63/dba/d11e/l123 0 2024-02-16T19:29:23.423 INFO:tasks.workunit.client.1.smithi117.stdout:7/919: write d4/dd3/fde [1370303,68731] 0 2024-02-16T19:29:23.428 INFO:tasks.workunit.client.0.smithi022.stdout:8/968: truncate d3/d12/d2b/fbf 453502 0 2024-02-16T19:29:23.429 INFO:tasks.workunit.client.1.smithi117.stdout:5/974: getdents d2/db3/db0/d9b 0 2024-02-16T19:29:23.431 INFO:tasks.workunit.client.0.smithi022.stdout:1/906: fsync d6/d8/d22/d3a/f44 0 2024-02-16T19:29:23.432 INFO:tasks.workunit.client.0.smithi022.stdout:1/907: chown d6/d66/fdf 4761 1 2024-02-16T19:29:23.435 INFO:tasks.workunit.client.1.smithi117.stdout:0/935: write d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/d27/f4d [3064286,77015] 0 2024-02-16T19:29:23.440 INFO:tasks.workunit.client.1.smithi117.stdout:1/843: symlink d2/d4/dbb/d7f/dc8/l124 0 2024-02-16T19:29:23.443 INFO:tasks.workunit.client.1.smithi117.stdout:5/975: dread d2/d36/f10d [0,4194304] 0 2024-02-16T19:29:23.443 INFO:tasks.workunit.client.0.smithi022.stdout:9/851: write db/d14/dc8/dc9/d105/f4f [504006,96255] 0 2024-02-16T19:29:23.444 INFO:tasks.workunit.client.1.smithi117.stdout:9/863: mknod dd/d47/d5b/d5f/c124 0 2024-02-16T19:29:23.445 INFO:tasks.workunit.client.1.smithi117.stdout:5/976: write d2/d4c/d4f/d67/fc5 [4289260,69866] 0 2024-02-16T19:29:23.446 INFO:tasks.workunit.client.1.smithi117.stdout:1/844: dwrite d2/d19/d59/d76/f109 [0,4194304] 0 2024-02-16T19:29:23.451 INFO:tasks.workunit.client.1.smithi117.stdout:0/936: rename d0/lc2 to d0/d9/d14/d1d/d60/dc3/l13c 0 2024-02-16T19:29:23.456 INFO:tasks.workunit.client.1.smithi117.stdout:7/920: symlink d4/d13/d63/ddd/d117/d11a/l156 0 2024-02-16T19:29:23.459 INFO:tasks.workunit.client.0.smithi022.stdout:2/858: write d0/d2/f85 [1473046,88089] 0 2024-02-16T19:29:23.463 INFO:tasks.workunit.client.1.smithi117.stdout:9/864: symlink dd/d45/d31/l125 0 2024-02-16T19:29:23.465 INFO:tasks.workunit.client.1.smithi117.stdout:5/977: write d2/d5d/fa9 [2504929,69620] 0 2024-02-16T19:29:23.466 INFO:tasks.workunit.client.1.smithi117.stdout:2/976: write d0/d13/dc4/de4/d12a/f12c [4502151,49536] 0 2024-02-16T19:29:23.466 INFO:tasks.workunit.client.1.smithi117.stdout:1/845: mkdir d2/d1d/d77/d125 0 2024-02-16T19:29:23.468 INFO:tasks.workunit.client.1.smithi117.stdout:2/977: fdatasync d0/d3/d34/d94/d46/f105 0 2024-02-16T19:29:23.468 INFO:tasks.workunit.client.0.smithi022.stdout:7/928: dwrite d9/da/f62 [0,4194304] 0 2024-02-16T19:29:23.469 INFO:tasks.workunit.client.1.smithi117.stdout:4/967: dwrite d0/da/f5b [4194304,4194304] 0 2024-02-16T19:29:23.470 INFO:tasks.workunit.client.1.smithi117.stdout:1/846: chown d2/d19/d59/d89/d69/dad/d100/f11d 228274 1 2024-02-16T19:29:23.473 INFO:tasks.workunit.client.1.smithi117.stdout:4/968: chown d0/da/d21/d3d/d28/d66/d97/la0 251 1 2024-02-16T19:29:23.476 INFO:tasks.workunit.client.0.smithi022.stdout:4/820: sync 2024-02-16T19:29:23.476 INFO:tasks.workunit.client.0.smithi022.stdout:6/864: sync 2024-02-16T19:29:23.478 INFO:tasks.workunit.client.0.smithi022.stdout:4/821: dread - d4/d6/dd/d22/da8/dc4/d86/dfd/df2/ffe zero size 2024-02-16T19:29:23.479 INFO:tasks.workunit.client.1.smithi117.stdout:1/847: dwrite d2/d19/d59/f97 [0,4194304] 0 2024-02-16T19:29:23.479 INFO:tasks.workunit.client.1.smithi117.stdout:9/865: rename dd/d45/d31/d63/d121/lae to dd/db5/dc4/dcf/dda/l126 0 2024-02-16T19:29:23.482 INFO:tasks.workunit.client.1.smithi117.stdout:1/848: chown d2/d19/d59/d89/d69/f115 30167583 1 2024-02-16T19:29:23.485 INFO:tasks.workunit.client.0.smithi022.stdout:4/822: dwrite d4/d6/d14/d64/d4e/d54/d5a/f5f [0,4194304] 0 2024-02-16T19:29:23.488 INFO:tasks.workunit.client.0.smithi022.stdout:4/823: truncate d4/d6/dd/d22/d83/dc5/fdb 961887 0 2024-02-16T19:29:23.489 INFO:tasks.workunit.client.0.smithi022.stdout:4/824: dread - d4/fe7 zero size 2024-02-16T19:29:23.493 INFO:tasks.workunit.client.1.smithi117.stdout:6/937: creat d5/db/d7d/d32/da8/de1/f137 x:0 0 0 2024-02-16T19:29:23.495 INFO:tasks.workunit.client.1.smithi117.stdout:1/849: dread d2/d1d/f29 [0,4194304] 0 2024-02-16T19:29:23.498 INFO:tasks.workunit.client.1.smithi117.stdout:7/921: creat d4/d13/dfd/f157 x:0 0 0 2024-02-16T19:29:23.499 INFO:tasks.workunit.client.1.smithi117.stdout:4/969: symlink d0/da/d21/d3d/d28/d64/d73/d91/d37/d102/l139 0 2024-02-16T19:29:23.506 INFO:tasks.workunit.client.1.smithi117.stdout:1/850: dwrite d2/d19/d59/d89/d69/f115 [0,4194304] 0 2024-02-16T19:29:23.507 INFO:tasks.workunit.client.1.smithi117.stdout:0/937: dwrite d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/d27/d31/fe2 [0,4194304] 0 2024-02-16T19:29:23.508 INFO:tasks.workunit.client.1.smithi117.stdout:0/938: chown d0/d9/d2c/d8b/de0/f12f 56219684 1 2024-02-16T19:29:23.513 INFO:tasks.workunit.client.1.smithi117.stdout:4/970: rmdir d0/d116 39 2024-02-16T19:29:23.513 INFO:tasks.workunit.client.1.smithi117.stdout:2/978: dwrite d0/d3/d34/d94/d46/d68/d93/da9/fcd [0,4194304] 0 2024-02-16T19:29:23.515 INFO:tasks.workunit.client.1.smithi117.stdout:2/979: read d0/d3/d34/d94/d46/d68/d76/fd2 [485618,39882] 0 2024-02-16T19:29:23.516 INFO:tasks.workunit.client.1.smithi117.stdout:4/971: dread d0/da/d21/d3d/d28/fad [0,4194304] 0 2024-02-16T19:29:23.516 INFO:tasks.workunit.client.0.smithi022.stdout:5/918: sync 2024-02-16T19:29:23.519 INFO:tasks.workunit.client.1.smithi117.stdout:9/866: link dd/d45/d31/d63/dba/df0/daf/df4/dd7/ffb dd/db5/dc4/dcf/f127 0 2024-02-16T19:29:23.520 INFO:tasks.workunit.client.1.smithi117.stdout:9/867: chown dd/d45/d31/d63/d121/f7f 49626 1 2024-02-16T19:29:23.523 INFO:tasks.workunit.client.1.smithi117.stdout:1/851: fdatasync d2/d19/f54 0 2024-02-16T19:29:23.524 INFO:tasks.workunit.client.1.smithi117.stdout:7/922: creat d4/d1c/d87/d89/d5f/dae/f158 x:0 0 0 2024-02-16T19:29:23.525 INFO:tasks.workunit.client.0.smithi022.stdout:1/908: chown d6/d8/d22/d67/f72 0 1 2024-02-16T19:29:23.526 INFO:tasks.workunit.client.0.smithi022.stdout:1/909: chown d6/d8/d22/l1f 1392 1 2024-02-16T19:29:23.527 INFO:tasks.workunit.client.0.smithi022.stdout:1/910: chown d6/d8/d22/d16/d17/d26/l38 89772 1 2024-02-16T19:29:23.529 INFO:tasks.workunit.client.0.smithi022.stdout:9/852: rename db/d14/dc8/dc9/d105/fbd to db/d14/d16/dec/d119/f123 0 2024-02-16T19:29:23.530 INFO:tasks.workunit.client.0.smithi022.stdout:9/853: dread - db/d14/d16/d28/f113 zero size 2024-02-16T19:29:23.532 INFO:tasks.workunit.client.1.smithi117.stdout:4/972: mknod d0/da/d21/d3d/d28/d64/d8a/c13a 0 2024-02-16T19:29:23.532 INFO:tasks.workunit.client.0.smithi022.stdout:2/859: dread d0/d5d/dc4/fdc [0,4194304] 0 2024-02-16T19:29:23.533 INFO:tasks.workunit.client.1.smithi117.stdout:9/868: mknod dd/db5/dc4/dcf/c128 0 2024-02-16T19:29:23.535 INFO:tasks.workunit.client.0.smithi022.stdout:6/865: creat d1/d8/d3c/d4d/f126 x:0 0 0 2024-02-16T19:29:23.537 INFO:tasks.workunit.client.0.smithi022.stdout:6/866: read d1/d8/da/d7b/dae/db6/f100 [453962,91874] 0 2024-02-16T19:29:23.538 INFO:tasks.workunit.client.0.smithi022.stdout:6/867: stat d1/d8/d81/f11d 0 2024-02-16T19:29:23.538 INFO:tasks.workunit.client.1.smithi117.stdout:9/869: dwrite dd/d45/f42 [0,4194304] 0 2024-02-16T19:29:23.540 INFO:tasks.workunit.client.1.smithi117.stdout:1/852: creat d2/d1d/d60/df2/f126 x:0 0 0 2024-02-16T19:29:23.544 INFO:tasks.workunit.client.0.smithi022.stdout:9/854: dread db/d14/d16/d27/d8b/fea [0,4194304] 0 2024-02-16T19:29:23.544 INFO:tasks.workunit.client.1.smithi117.stdout:7/923: creat d4/d1c/d39/d46/d55/f159 x:0 0 0 2024-02-16T19:29:23.545 INFO:tasks.workunit.client.0.smithi022.stdout:4/825: dwrite d4/d6/dd/df/f29 [4194304,4194304] 0 2024-02-16T19:29:23.546 INFO:tasks.workunit.client.0.smithi022.stdout:3/842: link d7/d12/d18/f1a d7/d12/d18/f121 0 2024-02-16T19:29:23.547 INFO:tasks.workunit.client.0.smithi022.stdout:6/868: dread d1/d8/da/d7b/dae/dbb/fbd [0,4194304] 0 2024-02-16T19:29:23.550 INFO:tasks.workunit.client.1.smithi117.stdout:0/939: rename d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/d27/c111 to d0/d9/d14/d1d/d60/d76/c13d 0 2024-02-16T19:29:23.556 INFO:tasks.workunit.client.0.smithi022.stdout:3/843: dwrite d7/d12/d18/f57 [8388608,4194304] 0 2024-02-16T19:29:23.564 INFO:tasks.workunit.client.0.smithi022.stdout:1/911: dwrite d6/d8/d22/d3a/f44 [0,4194304] 0 2024-02-16T19:29:23.565 INFO:tasks.workunit.client.1.smithi117.stdout:0/940: dread d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/d27/d83/f67 [0,4194304] 0 2024-02-16T19:29:23.565 INFO:tasks.workunit.client.1.smithi117.stdout:9/870: mknod dd/d47/d5b/d5f/de6/c129 0 2024-02-16T19:29:23.568 INFO:tasks.workunit.client.1.smithi117.stdout:0/941: write d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/d27/d83/f67 [593931,120021] 0 2024-02-16T19:29:23.568 INFO:tasks.workunit.client.1.smithi117.stdout:1/853: creat d2/d19/d59/d76/f127 x:0 0 0 2024-02-16T19:29:23.574 INFO:tasks.workunit.client.0.smithi022.stdout:5/919: read dd/d1f/d48/d21/d2f/d52/d77/d95/fec [694525,99648] 0 2024-02-16T19:29:23.576 INFO:tasks.workunit.client.1.smithi117.stdout:6/938: write d5/d4b/d71/f132 [487168,41356] 0 2024-02-16T19:29:23.577 INFO:tasks.workunit.client.1.smithi117.stdout:6/939: fsync d5/d4b/d44/d7c/d125/f10c 0 2024-02-16T19:29:23.578 INFO:tasks.workunit.client.1.smithi117.stdout:9/871: rmdir dd/db5 39 2024-02-16T19:29:23.579 INFO:tasks.workunit.client.1.smithi117.stdout:6/940: write d5/d4b/f46 [2117594,130513] 0 2024-02-16T19:29:23.580 INFO:tasks.workunit.client.1.smithi117.stdout:6/941: dread - d5/db/f108 zero size 2024-02-16T19:29:23.580 INFO:tasks.workunit.client.1.smithi117.stdout:0/942: readlink d0/d9/d14/d1d/d60/d76/l79 0 2024-02-16T19:29:23.585 INFO:tasks.workunit.client.1.smithi117.stdout:9/872: dwrite dd/d47/d5b/d5f/fc5 [0,4194304] 0 2024-02-16T19:29:23.586 INFO:tasks.workunit.client.1.smithi117.stdout:9/873: write dd/d47/d48/f50 [3664313,5918] 0 2024-02-16T19:29:23.586 INFO:tasks.workunit.client.1.smithi117.stdout:5/978: sync 2024-02-16T19:29:23.592 INFO:tasks.workunit.client.0.smithi022.stdout:8/969: creat d3/f14d x:0 0 0 2024-02-16T19:29:23.593 INFO:tasks.workunit.client.1.smithi117.stdout:1/854: creat d2/d4/dbb/d7f/dc8/f128 x:0 0 0 2024-02-16T19:29:23.593 INFO:tasks.workunit.client.1.smithi117.stdout:1/855: readlink d2/d1d/d1f/d9c/l110 0 2024-02-16T19:29:23.597 INFO:tasks.workunit.client.0.smithi022.stdout:7/929: getdents d9/da/d2c 0 2024-02-16T19:29:23.599 INFO:tasks.workunit.client.1.smithi117.stdout:2/980: dwrite d0/d13/f36 [0,4194304] 0 2024-02-16T19:29:23.599 INFO:tasks.workunit.client.0.smithi022.stdout:1/912: chown d6/d8/d6c/cf8 6 1 2024-02-16T19:29:23.601 INFO:tasks.workunit.client.0.smithi022.stdout:2/860: creat d0/d2/d2e/d73/de3/de6/f11d x:0 0 0 2024-02-16T19:29:23.604 INFO:tasks.workunit.client.0.smithi022.stdout:2/861: read d0/d2/d2e/d82/fc3 [1511480,101768] 0 2024-02-16T19:29:23.607 INFO:tasks.workunit.client.1.smithi117.stdout:7/924: dwrite d4/f4d [0,4194304] 0 2024-02-16T19:29:23.609 INFO:tasks.workunit.client.1.smithi117.stdout:7/925: readlink d4/d1c/d87/d89/d60/lf7 0 2024-02-16T19:29:23.610 INFO:tasks.workunit.client.1.smithi117.stdout:7/926: write d4/d1c/d87/fff [156099,85586] 0 2024-02-16T19:29:23.611 INFO:tasks.workunit.client.1.smithi117.stdout:4/973: dwrite d0/da/d21/d3d/d28/d64/d73/d91/d37/d98/fee [0,4194304] 0 2024-02-16T19:29:23.612 INFO:tasks.workunit.client.0.smithi022.stdout:3/844: write d7/f1e [1683461,110604] 0 2024-02-16T19:29:23.615 INFO:tasks.workunit.client.0.smithi022.stdout:8/970: mkdir d3/d12/d2b/da9/d14e 0 2024-02-16T19:29:23.617 INFO:tasks.workunit.client.1.smithi117.stdout:5/979: rmdir d2 39 2024-02-16T19:29:23.619 INFO:tasks.workunit.client.1.smithi117.stdout:9/874: dread - dd/d47/d5b/d5f/d77/d7c/fca zero size 2024-02-16T19:29:23.621 INFO:tasks.workunit.client.1.smithi117.stdout:9/875: write dd/d47/d5b/d5f/d77/f120 [771891,60459] 0 2024-02-16T19:29:23.622 INFO:tasks.workunit.client.0.smithi022.stdout:1/913: unlink d6/d8/d22/d16/c3c 0 2024-02-16T19:29:23.624 INFO:tasks.workunit.client.0.smithi022.stdout:1/914: read d6/d8/d22/d16/d4f/f8c [2367679,125662] 0 2024-02-16T19:29:23.625 INFO:tasks.workunit.client.1.smithi117.stdout:6/942: mknod d5/d9c/daa/c138 0 2024-02-16T19:29:23.626 INFO:tasks.workunit.client.1.smithi117.stdout:6/943: dread - d5/d4b/d71/dc7/dce/ded/d100/f123 zero size 2024-02-16T19:29:23.626 INFO:tasks.workunit.client.0.smithi022.stdout:2/862: creat d0/d2/d18/d19/d90/d100/f11e x:0 0 0 2024-02-16T19:29:23.627 INFO:tasks.workunit.client.0.smithi022.stdout:9/855: link db/d14/d16/d27/c112 db/d14/d16/d5d/d65/d6b/dd5/dd9/ddd/c124 0 2024-02-16T19:29:23.628 INFO:tasks.workunit.client.0.smithi022.stdout:9/856: readlink db/d14/d19/d1a/l47 0 2024-02-16T19:29:23.632 INFO:tasks.workunit.client.1.smithi117.stdout:4/974: mknod d0/da/d21/d3d/d28/d64/d8a/c13b 0 2024-02-16T19:29:23.633 INFO:tasks.workunit.client.0.smithi022.stdout:7/930: link d9/da/d3b/dba/ff5 d9/da/d3b/f13f 0 2024-02-16T19:29:23.634 INFO:tasks.workunit.client.0.smithi022.stdout:7/931: chown d9/da/d2c/d6d/f8b 22 1 2024-02-16T19:29:23.635 INFO:tasks.workunit.client.0.smithi022.stdout:6/869: truncate d1/d8/ff 607344 0 2024-02-16T19:29:23.636 INFO:tasks.workunit.client.0.smithi022.stdout:6/870: chown d1/d8/dac 2 1 2024-02-16T19:29:23.636 INFO:tasks.workunit.client.1.smithi117.stdout:1/856: mknod d2/d19/d59/d89/c129 0 2024-02-16T19:29:23.637 INFO:tasks.workunit.client.0.smithi022.stdout:9/857: read - db/d14/d16/d27/d55/f99 zero size 2024-02-16T19:29:23.639 INFO:tasks.workunit.client.1.smithi117.stdout:9/876: dread dd/d47/f7b [0,4194304] 0 2024-02-16T19:29:23.639 INFO:tasks.workunit.client.0.smithi022.stdout:4/826: truncate d4/d6/dd/df/d10/f25 2604769 0 2024-02-16T19:29:23.640 INFO:tasks.workunit.client.1.smithi117.stdout:9/877: write dd/d45/d31/d63/dba/df0/f1a [1124680,124306] 0 2024-02-16T19:29:23.640 INFO:tasks.workunit.client.0.smithi022.stdout:3/845: mkdir d7/d12/d18/dd/d24/d93/d122 0 2024-02-16T19:29:23.644 INFO:tasks.workunit.client.0.smithi022.stdout:4/827: dread d4/d6/d14/d64/d4e/d54/d5a/f5f [0,4194304] 0 2024-02-16T19:29:23.644 INFO:tasks.workunit.client.0.smithi022.stdout:8/971: rename d3/d12/d2b/d2e/l101 to d3/d12/d6a/l14f 0 2024-02-16T19:29:23.645 INFO:tasks.workunit.client.0.smithi022.stdout:8/972: stat d3/d12/d2b/da9/fbd 0 2024-02-16T19:29:23.645 INFO:tasks.workunit.client.0.smithi022.stdout:8/973: fdatasync d3/f14d 0 2024-02-16T19:29:23.647 INFO:tasks.workunit.client.1.smithi117.stdout:7/927: truncate d4/d1c/d112/f125 437241 0 2024-02-16T19:29:23.648 INFO:tasks.workunit.client.1.smithi117.stdout:7/928: chown d4/d1c/d87/d89/l61 18752 1 2024-02-16T19:29:23.649 INFO:tasks.workunit.client.0.smithi022.stdout:6/871: creat d1/d8/da/d29/deb/dfd/d85/d56/f127 x:0 0 0 2024-02-16T19:29:23.650 INFO:tasks.workunit.client.1.smithi117.stdout:0/943: truncate d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/ffc 133244 0 2024-02-16T19:29:23.656 INFO:tasks.workunit.client.0.smithi022.stdout:3/846: unlink d7/d12/d18/dd/d24/lc0 0 2024-02-16T19:29:23.657 INFO:tasks.workunit.client.0.smithi022.stdout:3/847: truncate d7/f1e 4523866 0 2024-02-16T19:29:23.658 INFO:tasks.workunit.client.0.smithi022.stdout:1/915: rename d6/d8/d22/d16/d17/d1d/c23 to d6/d8/d22/d16/d17/d1d/df9/c119 0 2024-02-16T19:29:23.662 INFO:tasks.workunit.client.1.smithi117.stdout:2/981: truncate d0/d3/d22/d37/d109/f11b 229906 0 2024-02-16T19:29:23.665 INFO:tasks.workunit.client.1.smithi117.stdout:6/944: mknod d5/d4b/d71/dc7/c139 0 2024-02-16T19:29:23.665 INFO:tasks.workunit.client.1.smithi117.stdout:6/945: write d5/db/f1f [6802388,69526] 0 2024-02-16T19:29:23.668 INFO:tasks.workunit.client.0.smithi022.stdout:6/872: fdatasync d1/d8/d81/fd9 0 2024-02-16T19:29:23.670 INFO:tasks.workunit.client.0.smithi022.stdout:6/873: dread - d1/d8/d81/f11d zero size 2024-02-16T19:29:23.670 INFO:tasks.workunit.client.1.smithi117.stdout:4/975: unlink d0/da/d21/d3d/laf 0 2024-02-16T19:29:23.670 INFO:tasks.workunit.client.0.smithi022.stdout:7/932: dwrite d9/da/d1f/d44/f54 [0,4194304] 0 2024-02-16T19:29:23.672 INFO:tasks.workunit.client.0.smithi022.stdout:7/933: readlink d9/da/d2c/d6d/l82 0 2024-02-16T19:29:23.672 INFO:tasks.workunit.client.0.smithi022.stdout:6/874: chown d1/d8/da/d29/deb/dfd/d53/f91 128 1 2024-02-16T19:29:23.676 INFO:tasks.workunit.client.1.smithi117.stdout:5/980: creat d2/d3/d21/d49/d59/d97/f136 x:0 0 0 2024-02-16T19:29:23.677 INFO:tasks.workunit.client.1.smithi117.stdout:5/981: stat d2/d3/d21/d49/d59/c80 0 2024-02-16T19:29:23.678 INFO:tasks.workunit.client.0.smithi022.stdout:7/934: dread d9/f124 [0,4194304] 0 2024-02-16T19:29:23.679 INFO:tasks.workunit.client.0.smithi022.stdout:7/935: fsync d9/f13c 0 2024-02-16T19:29:23.680 INFO:tasks.workunit.client.0.smithi022.stdout:2/863: rename d0/d2/d2e/fd2 to d0/d5d/dc4/f11f 0 2024-02-16T19:29:23.681 INFO:tasks.workunit.client.0.smithi022.stdout:8/974: write d3/d12/da6/f28 [844949,2227] 0 2024-02-16T19:29:23.688 INFO:tasks.workunit.client.1.smithi117.stdout:6/946: fsync d5/db/f4a 0 2024-02-16T19:29:23.688 INFO:tasks.workunit.client.1.smithi117.stdout:6/947: read - d5/d4b/d71/f121 zero size 2024-02-16T19:29:23.694 INFO:tasks.workunit.client.1.smithi117.stdout:0/944: rename d0/d9/d14/d1d/d60/d76/d7b/d8d/l136 to d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/d27/d10a/d115/l13e 0 2024-02-16T19:29:23.699 INFO:tasks.workunit.client.1.smithi117.stdout:2/982: dwrite d0/d3/d34/d94/d91/f98 [0,4194304] 0 2024-02-16T19:29:23.700 INFO:tasks.workunit.client.1.smithi117.stdout:2/983: chown d0/d3/f7c 153 1 2024-02-16T19:29:23.701 INFO:tasks.workunit.client.1.smithi117.stdout:7/929: dwrite d4/d1c/d39/d46/f48 [0,4194304] 0 2024-02-16T19:29:23.702 INFO:tasks.workunit.client.0.smithi022.stdout:7/936: creat d9/da/d3b/dba/f140 x:0 0 0 2024-02-16T19:29:23.703 INFO:tasks.workunit.client.1.smithi117.stdout:9/878: creat dd/d45/f12a x:0 0 0 2024-02-16T19:29:23.705 INFO:tasks.workunit.client.0.smithi022.stdout:9/858: rename db/d14/d16/d87/f108 to db/d14/d19/da8/f125 0 2024-02-16T19:29:23.708 INFO:tasks.workunit.client.1.smithi117.stdout:2/984: dwrite d0/d3/d34/d94/d46/f135 [0,4194304] 0 2024-02-16T19:29:23.708 INFO:tasks.workunit.client.1.smithi117.stdout:6/948: chown d5/d4b/d44/c11e 1 1 2024-02-16T19:29:23.709 INFO:tasks.workunit.client.1.smithi117.stdout:6/949: readlink d5/d4b/d20/d69/l6b 0 2024-02-16T19:29:23.709 INFO:tasks.workunit.client.1.smithi117.stdout:2/985: write d0/d3b/f51 [1420366,12750] 0 2024-02-16T19:29:23.711 INFO:tasks.workunit.client.1.smithi117.stdout:6/950: truncate d5/db/d7d/d32/f48 4914533 0 2024-02-16T19:29:23.712 INFO:tasks.workunit.client.0.smithi022.stdout:4/828: creat d4/d6/dd/df/d10/f104 x:0 0 0 2024-02-16T19:29:23.724 INFO:tasks.workunit.client.1.smithi117.stdout:1/857: getdents d2/d4 0 2024-02-16T19:29:23.725 INFO:tasks.workunit.client.0.smithi022.stdout:8/975: dwrite d3/d12/d92/fad [0,4194304] 0 2024-02-16T19:29:23.725 INFO:tasks.workunit.client.0.smithi022.stdout:7/937: mkdir d9/da/d1f/d44/d5b/d141 0 2024-02-16T19:29:23.727 INFO:tasks.workunit.client.1.smithi117.stdout:5/982: symlink d2/d9/d3c/d42/l137 0 2024-02-16T19:29:23.728 INFO:tasks.workunit.client.1.smithi117.stdout:5/983: stat d2/d9/d3c/f8d 0 2024-02-16T19:29:23.729 INFO:tasks.workunit.client.0.smithi022.stdout:9/859: dread - db/d14/d16/d21/d37/fb1 zero size 2024-02-16T19:29:23.733 INFO:tasks.workunit.client.1.smithi117.stdout:9/879: truncate dd/d45/d73/f8a 1614432 0 2024-02-16T19:29:23.735 INFO:tasks.workunit.client.1.smithi117.stdout:2/986: rmdir d0/d13/dc4/d10f 39 2024-02-16T19:29:23.735 INFO:tasks.workunit.client.0.smithi022.stdout:6/875: link d1/d8/d3c/d4d/fff d1/d8/da/d29/deb/dfd/d53/d103/f128 0 2024-02-16T19:29:23.737 INFO:tasks.workunit.client.1.smithi117.stdout:6/951: rmdir d5/d4b/d71/dc7/d72 39 2024-02-16T19:29:23.740 INFO:tasks.workunit.client.0.smithi022.stdout:7/938: unlink d9/da/d1f/d93/c105 0 2024-02-16T19:29:23.741 INFO:tasks.workunit.client.0.smithi022.stdout:2/864: truncate d0/d2/d2e/f2a 202404 0 2024-02-16T19:29:23.742 INFO:tasks.workunit.client.1.smithi117.stdout:1/858: truncate d2/d4/dbb/d7f/f94 1619653 0 2024-02-16T19:29:23.743 INFO:tasks.workunit.client.0.smithi022.stdout:9/860: symlink db/d14/d16/d49/l126 0 2024-02-16T19:29:23.744 INFO:tasks.workunit.client.0.smithi022.stdout:1/916: getdents d6/d8/d22/d3f 0 2024-02-16T19:29:23.745 INFO:tasks.workunit.client.0.smithi022.stdout:1/917: fsync d6/d8/d22/d16/d4f/ffd 0 2024-02-16T19:29:23.746 INFO:tasks.workunit.client.1.smithi117.stdout:7/930: creat d4/d1c/d39/d46/d136/f15a x:0 0 0 2024-02-16T19:29:23.748 INFO:tasks.workunit.client.0.smithi022.stdout:6/876: creat d1/d8/dac/d101/db3/f129 x:0 0 0 2024-02-16T19:29:23.750 INFO:tasks.workunit.client.0.smithi022.stdout:2/865: mkdir d0/d2/d2e/d82/df0/d120 0 2024-02-16T19:29:23.754 INFO:tasks.workunit.client.1.smithi117.stdout:7/931: mkdir d4/d1c/d87/df0/dfe/d15b 0 2024-02-16T19:29:23.755 INFO:tasks.workunit.client.0.smithi022.stdout:2/866: dread d0/d2/d18/d19/fab [0,4194304] 0 2024-02-16T19:29:23.755 INFO:tasks.workunit.client.0.smithi022.stdout:4/829: rename d4/d6/dd/df/dc1 to d4/d6/dd/d22/d105 0 2024-02-16T19:29:23.756 INFO:tasks.workunit.client.0.smithi022.stdout:6/877: rmdir d1/d8/da/d29/deb/dfd/d85/d9c 39 2024-02-16T19:29:23.757 INFO:tasks.workunit.client.0.smithi022.stdout:6/878: dread - d1/d8/da/d29/f110 zero size 2024-02-16T19:29:23.757 INFO:tasks.workunit.client.0.smithi022.stdout:6/879: readlink d1/d8/d81/lec 0 2024-02-16T19:29:23.758 INFO:tasks.workunit.client.0.smithi022.stdout:6/880: read - d1/d8/da/d29/deb/f114 zero size 2024-02-16T19:29:23.759 INFO:tasks.workunit.client.0.smithi022.stdout:6/881: dread - d1/d8/da/d29/deb/dfd/d48/de4/dfb/f10c zero size 2024-02-16T19:29:23.759 INFO:tasks.workunit.client.0.smithi022.stdout:9/861: unlink db/d14/d16/d27/c112 0 2024-02-16T19:29:23.760 INFO:tasks.workunit.client.1.smithi117.stdout:7/932: fdatasync d4/d13/da3/fdc 0 2024-02-16T19:29:23.762 INFO:tasks.workunit.client.1.smithi117.stdout:4/976: sync 2024-02-16T19:29:23.764 INFO:tasks.workunit.client.1.smithi117.stdout:9/880: getdents dd/d47 0 2024-02-16T19:29:23.766 INFO:tasks.workunit.client.1.smithi117.stdout:4/977: unlink d0/da/d21/d3d/l7a 0 2024-02-16T19:29:23.768 INFO:tasks.workunit.client.1.smithi117.stdout:7/933: dread d4/d1c/d39/f54 [0,4194304] 0 2024-02-16T19:29:23.770 INFO:tasks.workunit.client.1.smithi117.stdout:9/881: fdatasync dd/db5/dc4/feb 0 2024-02-16T19:29:23.771 INFO:tasks.workunit.client.0.smithi022.stdout:1/918: creat d6/d8/d22/f11a x:0 0 0 2024-02-16T19:29:23.772 INFO:tasks.workunit.client.0.smithi022.stdout:2/867: mknod d0/d2/c121 0 2024-02-16T19:29:23.772 INFO:tasks.workunit.client.0.smithi022.stdout:1/919: write d6/d8/d22/d3a/d80/f10c [261624,2575] 0 2024-02-16T19:29:23.774 INFO:tasks.workunit.client.0.smithi022.stdout:4/830: creat d4/d6/d14/d64/de0/f106 x:0 0 0 2024-02-16T19:29:23.775 INFO:tasks.workunit.client.0.smithi022.stdout:4/831: chown d4/d6/dd/d22/d83/dc5/fdb 432160 1 2024-02-16T19:29:23.775 INFO:tasks.workunit.client.0.smithi022.stdout:7/939: link d9/da/d4e/cfa d9/da/d1f/d40/c142 0 2024-02-16T19:29:23.779 INFO:tasks.workunit.client.1.smithi117.stdout:4/978: unlink d0/da/d21/d3d/d28/d64/d8a/d92/de8/l11e 0 2024-02-16T19:29:23.781 INFO:tasks.workunit.client.0.smithi022.stdout:6/882: mkdir d1/d8/da/d29/deb/dfd/d85/d9c/d12a 0 2024-02-16T19:29:23.781 INFO:tasks.workunit.client.1.smithi117.stdout:7/934: dread d4/dd3/f99 [0,4194304] 0 2024-02-16T19:29:23.784 INFO:tasks.workunit.client.0.smithi022.stdout:1/920: unlink d6/f6b 0 2024-02-16T19:29:23.784 INFO:tasks.workunit.client.0.smithi022.stdout:4/832: dread d4/d6/db6/fb5 [0,4194304] 0 2024-02-16T19:29:23.785 INFO:tasks.workunit.client.0.smithi022.stdout:4/833: chown d4/d6/d3a/l75 2994084 1 2024-02-16T19:29:23.786 INFO:tasks.workunit.client.0.smithi022.stdout:2/868: read d0/d2/d18/d104/fac [206687,112357] 0 2024-02-16T19:29:23.787 INFO:tasks.workunit.client.0.smithi022.stdout:2/869: write d0/d2/d18/d19/d40/dc9/ff9 [5146,73262] 0 2024-02-16T19:29:23.790 INFO:tasks.workunit.client.1.smithi117.stdout:7/935: rename d4/f4d to d4/d13/d63/ddd/d117/f15c 0 2024-02-16T19:29:23.791 INFO:tasks.workunit.client.0.smithi022.stdout:7/940: mkdir d9/da/d3b/d143 0 2024-02-16T19:29:23.792 INFO:tasks.workunit.client.1.smithi117.stdout:9/882: rmdir dd/db5/dc4/dcf/d11d 0 2024-02-16T19:29:23.794 INFO:tasks.workunit.client.0.smithi022.stdout:9/862: creat db/d14/d16/f127 x:0 0 0 2024-02-16T19:29:23.797 INFO:tasks.workunit.client.1.smithi117.stdout:9/883: dwrite dd/d47/d48/dc0/fbf [0,4194304] 0 2024-02-16T19:29:23.799 INFO:tasks.workunit.client.1.smithi117.stdout:7/936: read - d4/d1c/d39/d9f/fca zero size 2024-02-16T19:29:23.801 INFO:tasks.workunit.client.0.smithi022.stdout:1/921: rename d6/d8/d22/d16/d17/d5d/df3 to d6/d8/d22/d3f/d11b 0 2024-02-16T19:29:23.803 INFO:tasks.workunit.client.0.smithi022.stdout:4/834: fdatasync d4/d6/dd/d22/f51 0 2024-02-16T19:29:23.804 INFO:tasks.workunit.client.1.smithi117.stdout:9/884: dwrite dd/f19 [4194304,4194304] 0 2024-02-16T19:29:23.806 INFO:tasks.workunit.client.1.smithi117.stdout:7/937: mkdir d4/d1c/d39/d46/da4/d15d 0 2024-02-16T19:29:23.809 INFO:tasks.workunit.client.1.smithi117.stdout:9/885: mkdir dd/d45/d31/d63/dba/df0/daf/d12b 0 2024-02-16T19:29:23.812 INFO:tasks.workunit.client.0.smithi022.stdout:7/941: mkdir d9/da/d2c/d46/d73/dcc/d11f/d144 0 2024-02-16T19:29:23.817 INFO:tasks.workunit.client.0.smithi022.stdout:6/883: rmdir d1/d8/dac/d101/da3/d109 0 2024-02-16T19:29:23.819 INFO:tasks.workunit.client.1.smithi117.stdout:0/945: write d0/d9/f7f [29609,3985] 0 2024-02-16T19:29:23.821 INFO:tasks.workunit.client.0.smithi022.stdout:7/942: read d9/d5d/fc8 [4181425,84758] 0 2024-02-16T19:29:23.826 INFO:tasks.workunit.client.0.smithi022.stdout:6/884: rename d1/d8/f11 to d1/d8/d81/f12b 0 2024-02-16T19:29:23.827 INFO:tasks.workunit.client.0.smithi022.stdout:6/885: truncate d1/d8/d3c/d4d/f126 34150 0 2024-02-16T19:29:23.828 INFO:tasks.workunit.client.1.smithi117.stdout:0/946: unlink d0/d9/d14/d1d/d1e/f2d 0 2024-02-16T19:29:23.828 INFO:tasks.workunit.client.1.smithi117.stdout:6/952: dread d5/f9 [0,4194304] 0 2024-02-16T19:29:23.829 INFO:tasks.workunit.client.0.smithi022.stdout:9/863: getdents db/d14/d16/d27/d55 0 2024-02-16T19:29:23.829 INFO:tasks.workunit.client.1.smithi117.stdout:5/984: write d2/d4c/d4f/f6c [77507,88406] 0 2024-02-16T19:29:23.829 INFO:tasks.workunit.client.1.smithi117.stdout:7/938: dread d4/d13/d63/d71/f90 [0,4194304] 0 2024-02-16T19:29:23.829 INFO:tasks.workunit.client.0.smithi022.stdout:9/864: stat db/d14/d16/d28/fd4 0 2024-02-16T19:29:23.830 INFO:tasks.workunit.client.1.smithi117.stdout:6/953: chown d5/db/d7d/d32/da8/de1/lbb 94 1 2024-02-16T19:29:23.833 INFO:tasks.workunit.client.1.smithi117.stdout:1/859: write d2/d1d/d60/d68/f8a [317199,110130] 0 2024-02-16T19:29:23.837 INFO:tasks.workunit.client.1.smithi117.stdout:0/947: truncate d0/d9/d2c/da9/faa 1855465 0 2024-02-16T19:29:23.838 INFO:tasks.workunit.client.1.smithi117.stdout:0/948: chown d0/d9/d14/d1d/d125 402054329 1 2024-02-16T19:29:23.838 INFO:tasks.workunit.client.1.smithi117.stdout:2/987: truncate d0/d3/d34/d94/d46/d68/d76/da1/fe9 3722330 0 2024-02-16T19:29:23.838 INFO:tasks.workunit.client.1.smithi117.stdout:7/939: symlink d4/d13/l15e 0 2024-02-16T19:29:23.840 INFO:tasks.workunit.client.1.smithi117.stdout:1/860: mkdir d2/d19/d59/d76/d12a 0 2024-02-16T19:29:23.840 INFO:tasks.workunit.client.1.smithi117.stdout:7/940: truncate d4/d1c/d87/d89/d5f/dae/df3/f12f 740872 0 2024-02-16T19:29:23.842 INFO:tasks.workunit.client.1.smithi117.stdout:2/988: rmdir d0/d13/d4d/d53/d58/dbc 39 2024-02-16T19:29:23.853 INFO:tasks.workunit.client.1.smithi117.stdout:7/941: fsync d4/d13/d63/ddd/d117/f15c 0 2024-02-16T19:29:23.858 INFO:tasks.workunit.client.1.smithi117.stdout:4/979: write d0/da/d21/d3d/d28/d66/da2/dac/ff2 [895480,86733] 0 2024-02-16T19:29:23.860 INFO:tasks.workunit.client.1.smithi117.stdout:2/989: dread d0/d3/d10/f1d [0,4194304] 0 2024-02-16T19:29:23.861 INFO:tasks.workunit.client.1.smithi117.stdout:2/990: chown d0/d13/d4d/d53/d58/l12d 392824265 1 2024-02-16T19:29:23.873 INFO:tasks.workunit.client.1.smithi117.stdout:9/886: truncate dd/d47/d5b/f8c 1446066 0 2024-02-16T19:29:23.883 INFO:tasks.workunit.client.1.smithi117.stdout:5/985: write d2/d9/d12e/d96/dbc/d30/d95/f9e [438109,51321] 0 2024-02-16T19:29:23.886 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:23 smithi117 ceph-mon[35309]: Adjusting osd_memory_target on smithi117 to 2240M 2024-02-16T19:29:23.900 INFO:tasks.workunit.client.0.smithi022.stdout:6/886: creat d1/d8/dac/d101/db3/f12c x:0 0 0 2024-02-16T19:29:23.900 INFO:tasks.workunit.client.0.smithi022.stdout:6/887: readlink d1/l5 0 2024-02-16T19:29:23.904 INFO:tasks.workunit.client.0.smithi022.stdout:7/943: write d9/d5d/dc7/d118/fd5 [319018,109713] 0 2024-02-16T19:29:23.908 INFO:tasks.workunit.client.0.smithi022.stdout:9/865: dwrite db/d14/d16/dec/dee/f118 [0,4194304] 0 2024-02-16T19:29:23.909 INFO:tasks.workunit.client.0.smithi022.stdout:4/835: dwrite d4/d6/dd/d22/f51 [0,4194304] 0 2024-02-16T19:29:23.964 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:23 smithi022 ceph-mon[29221]: Adjusting osd_memory_target on smithi117 to 2240M 2024-02-16T19:29:23.970 INFO:tasks.workunit.client.0.smithi022.stdout:5/920: sync 2024-02-16T19:29:23.972 INFO:tasks.workunit.client.0.smithi022.stdout:5/921: write dd/d1f/d48/d21/d2f/d46/d5e/f7b [471356,45902] 0 2024-02-16T19:29:23.978 INFO:tasks.workunit.client.1.smithi117.stdout:6/954: truncate d5/d9c/daa/d85/f8b 2689352 0 2024-02-16T19:29:23.984 INFO:tasks.workunit.client.1.smithi117.stdout:6/955: dwrite d5/d9c/f116 [0,4194304] 0 2024-02-16T19:29:23.996 INFO:tasks.workunit.client.1.smithi117.stdout:0/949: truncate d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/ffc 278766 0 2024-02-16T19:29:23.997 INFO:tasks.workunit.client.1.smithi117.stdout:0/950: chown d0/d9/d14/d1d/d60/d76/ffb 2966 1 2024-02-16T19:29:24.001 INFO:tasks.workunit.client.0.smithi022.stdout:6/888: creat d1/d8/da/d29/deb/dfd/d48/de4/dfb/f12d x:0 0 0 2024-02-16T19:29:24.003 INFO:tasks.workunit.client.0.smithi022.stdout:6/889: read - d1/d8/da/d29/deb/dfd/d53/d103/f128 zero size 2024-02-16T19:29:24.007 INFO:tasks.workunit.client.0.smithi022.stdout:6/890: write d1/d8/dac/d101/f10a [1520783,48321] 0 2024-02-16T19:29:24.007 INFO:tasks.workunit.client.0.smithi022.stdout:7/944: creat d9/da/d1f/d93/f145 x:0 0 0 2024-02-16T19:29:24.008 INFO:tasks.workunit.client.1.smithi117.stdout:2/991: creat d0/d3/d22/f13a x:0 0 0 2024-02-16T19:29:24.009 INFO:tasks.workunit.client.1.smithi117.stdout:2/992: chown d0/d13/dc4/de4/ff2 5 1 2024-02-16T19:29:24.011 INFO:tasks.workunit.client.1.smithi117.stdout:9/887: mkdir dd/d47/d12c 0 2024-02-16T19:29:24.018 INFO:tasks.workunit.client.0.smithi022.stdout:5/922: chown dd/d1f/d48/l8d 3023643 1 2024-02-16T19:29:24.019 INFO:tasks.workunit.client.0.smithi022.stdout:5/923: chown dd/d1f/d48/d21/d2f/d40/d71/lc4 3631837 1 2024-02-16T19:29:24.027 INFO:tasks.workunit.client.0.smithi022.stdout:6/891: rmdir d1/d8/da/d7b/dae/dbb 39 2024-02-16T19:29:24.029 INFO:tasks.workunit.client.1.smithi117.stdout:1/861: link d2/d19/d59/d89/d69/d80/d8e/c10e d2/d19/d59/d89/d8d/c12b 0 2024-02-16T19:29:24.030 INFO:tasks.workunit.client.1.smithi117.stdout:1/862: write d2/d1d/d1f/f44 [4893488,117146] 0 2024-02-16T19:29:24.031 INFO:tasks.workunit.client.1.smithi117.stdout:1/863: chown d2/d4/dbb/d7f/dc8/f128 5 1 2024-02-16T19:29:24.032 INFO:tasks.workunit.client.0.smithi022.stdout:6/892: fsync d1/d8/da/d29/deb/dfd/d85/fe6 0 2024-02-16T19:29:24.033 INFO:tasks.workunit.client.1.smithi117.stdout:1/864: fdatasync d2/d19/d59/d89/d69/ffe 0 2024-02-16T19:29:24.036 INFO:tasks.workunit.client.0.smithi022.stdout:6/893: dread d1/d8/da/d29/deb/dfd/d48/d51/f9b [0,4194304] 0 2024-02-16T19:29:24.038 INFO:tasks.workunit.client.0.smithi022.stdout:4/836: creat d4/d6/d14/d64/d4e/f107 x:0 0 0 2024-02-16T19:29:24.039 INFO:tasks.workunit.client.1.smithi117.stdout:4/980: fdatasync d0/da/d21/d3d/d28/fad 0 2024-02-16T19:29:24.040 INFO:tasks.workunit.client.0.smithi022.stdout:4/837: write d4/d6/d14/d64/d4e/d54/ff3 [382362,117118] 0 2024-02-16T19:29:24.044 INFO:tasks.workunit.client.1.smithi117.stdout:2/993: symlink d0/d3/d34/d94/d46/d68/d76/da1/d2f/l13b 0 2024-02-16T19:29:24.044 INFO:tasks.workunit.client.1.smithi117.stdout:2/994: fsync d0/d3/d40/f132 0 2024-02-16T19:29:24.045 INFO:tasks.workunit.client.0.smithi022.stdout:9/866: mkdir db/d14/dc8/d128 0 2024-02-16T19:29:24.046 INFO:tasks.workunit.client.0.smithi022.stdout:6/894: fsync d1/d8/d3c/d4d/fff 0 2024-02-16T19:29:24.048 INFO:tasks.workunit.client.1.smithi117.stdout:9/888: symlink dd/d45/d31/d63/dba/df0/daf/d106/l12d 0 2024-02-16T19:29:24.050 INFO:tasks.workunit.client.0.smithi022.stdout:4/838: chown d4/d6/dd/d22/da8/dc4/d86/cd9 2519 1 2024-02-16T19:29:24.052 INFO:tasks.workunit.client.0.smithi022.stdout:9/867: chown db/d14/d16/d21/d37/la5 3266 1 2024-02-16T19:29:24.054 INFO:tasks.workunit.client.1.smithi117.stdout:9/889: dwrite dd/d45/d31/d63/d121/f110 [0,4194304] 0 2024-02-16T19:29:24.058 INFO:tasks.workunit.client.0.smithi022.stdout:6/895: mknod d1/d8/d23/db1/c12e 0 2024-02-16T19:29:24.062 INFO:tasks.workunit.client.1.smithi117.stdout:1/865: mknod d2/d1d/d60/d7c/c12c 0 2024-02-16T19:29:24.064 INFO:tasks.workunit.client.0.smithi022.stdout:4/839: mkdir d4/d6/dd/df/d10/d55/d108 0 2024-02-16T19:29:24.071 INFO:tasks.workunit.client.0.smithi022.stdout:6/896: read d1/d8/da/d29/deb/dfd/f99 [2683731,62966] 0 2024-02-16T19:29:24.081 INFO:tasks.workunit.client.1.smithi117.stdout:2/995: dwrite d0/d3/d34/d94/d46/d68/d76/da1/f2d [0,4194304] 0 2024-02-16T19:29:24.081 INFO:tasks.workunit.client.1.smithi117.stdout:9/890: fsync dd/fa9 0 2024-02-16T19:29:24.085 INFO:tasks.workunit.client.0.smithi022.stdout:6/897: dwrite d1/d8/da/d29/deb/dfd/d85/d5b/fc9 [0,4194304] 0 2024-02-16T19:29:24.092 INFO:tasks.workunit.client.1.smithi117.stdout:9/891: rename dd/db5/dc4/dd3/ld9 to dd/d45/d31/d63/dba/df0/daf/df4/d10d/l12e 0 2024-02-16T19:29:24.094 INFO:tasks.workunit.client.0.smithi022.stdout:6/898: creat d1/d8/da/d7b/f12f x:0 0 0 2024-02-16T19:29:24.099 INFO:tasks.workunit.client.0.smithi022.stdout:6/899: creat d1/d8/dac/d101/da3/dad/f130 x:0 0 0 2024-02-16T19:29:24.103 INFO:tasks.workunit.client.0.smithi022.stdout:6/900: mkdir d1/d8/da/d29/deb/dfd/d85/d56/d131 0 2024-02-16T19:29:24.105 INFO:tasks.workunit.client.0.smithi022.stdout:6/901: write d1/d8/d3c/d77/fe5 [2105105,91111] 0 2024-02-16T19:29:24.105 INFO:tasks.workunit.client.0.smithi022.stdout:6/902: readlink d1/d8/d81/lf2 0 2024-02-16T19:29:24.106 INFO:tasks.workunit.client.0.smithi022.stdout:6/903: chown d1/f2 3 1 2024-02-16T19:29:24.111 INFO:tasks.workunit.client.0.smithi022.stdout:6/904: getdents d1/d8/dac 0 2024-02-16T19:29:24.112 INFO:tasks.workunit.client.0.smithi022.stdout:6/905: dread - d1/d8/da/d29/deb/f114 zero size 2024-02-16T19:29:24.118 INFO:tasks.workunit.client.1.smithi117.stdout:4/981: mkdir d0/da/d21/d3d/d28/d64/d90/dc0/d13c 0 2024-02-16T19:29:24.147 INFO:tasks.workunit.client.0.smithi022.stdout:3/848: sync 2024-02-16T19:29:24.167 INFO:tasks.workunit.client.0.smithi022.stdout:3/849: dread d7/d12/d18/dd/d3c/fa9 [0,4194304] 0 2024-02-16T19:29:24.174 INFO:tasks.workunit.client.0.smithi022.stdout:3/850: dwrite d7/d12/d18/dd/d31/dbf/f103 [0,4194304] 0 2024-02-16T19:29:24.179 INFO:tasks.workunit.client.0.smithi022.stdout:1/922: dwrite d6/d8/d22/d47/fcd [0,4194304] 0 2024-02-16T19:29:24.181 INFO:tasks.workunit.client.0.smithi022.stdout:3/851: symlink d7/d12/d18/dd/d24/d93/d122/l123 0 2024-02-16T19:29:24.184 INFO:tasks.workunit.client.0.smithi022.stdout:1/923: rmdir d6/d8/d22/d59/da7 39 2024-02-16T19:29:24.186 INFO:tasks.workunit.client.1.smithi117.stdout:7/942: write d4/dd3/f99 [726195,22588] 0 2024-02-16T19:29:24.189 INFO:tasks.workunit.client.0.smithi022.stdout:3/852: dwrite d7/d12/d18/dd/d24/d53/fe4 [0,4194304] 0 2024-02-16T19:29:24.190 INFO:tasks.workunit.client.1.smithi117.stdout:7/943: truncate d4/d1c/f3c 1744951 0 2024-02-16T19:29:24.190 INFO:tasks.workunit.client.0.smithi022.stdout:3/853: chown d7/d12/d18/f91 1440497 1 2024-02-16T19:29:24.193 INFO:tasks.workunit.client.1.smithi117.stdout:7/944: creat d4/d13/dfd/f15f x:0 0 0 2024-02-16T19:29:24.193 INFO:tasks.workunit.client.1.smithi117.stdout:7/945: readlink d4/d13/d63/d71/l115 0 2024-02-16T19:29:24.199 INFO:tasks.workunit.client.0.smithi022.stdout:3/854: rename d7/d12/d18/dd/d24/d53/ld8 to d7/d12/d18/dd/d24/d93/de9/dc4/l124 0 2024-02-16T19:29:24.203 INFO:tasks.workunit.client.1.smithi117.stdout:9/892: write dd/d47/d5b/d5f/f6f [4326140,52850] 0 2024-02-16T19:29:24.204 INFO:tasks.workunit.client.0.smithi022.stdout:3/855: mkdir d7/d12/d18/dd/d3c/d11d/d125 0 2024-02-16T19:29:24.208 INFO:tasks.workunit.client.1.smithi117.stdout:5/986: mkdir d2/d9/d12e/d52/d138 0 2024-02-16T19:29:24.210 INFO:tasks.workunit.client.0.smithi022.stdout:3/856: dwrite f2 [8388608,4194304] 0 2024-02-16T19:29:24.211 INFO:tasks.workunit.client.0.smithi022.stdout:3/857: chown d7/d12/d18/dd/d24/d29/cd5 11 1 2024-02-16T19:29:24.213 INFO:tasks.workunit.client.1.smithi117.stdout:5/987: creat d2/d9/d12e/f139 x:0 0 0 2024-02-16T19:29:24.215 INFO:tasks.workunit.client.1.smithi117.stdout:5/988: mknod d2/d36/d6f/c13a 0 2024-02-16T19:29:24.218 INFO:tasks.workunit.client.0.smithi022.stdout:3/858: mkdir d7/d12/d18/dd/d24/d53/d116/de6/d126 0 2024-02-16T19:29:24.220 INFO:tasks.workunit.client.0.smithi022.stdout:3/859: write d7/d12/d18/dd/d3c/d5a/d67/f10d [914515,119331] 0 2024-02-16T19:29:24.220 INFO:tasks.workunit.client.1.smithi117.stdout:5/989: getdents d2/d9/d12e/d96/dc4/dca/df3/d128 0 2024-02-16T19:29:24.224 INFO:tasks.workunit.client.1.smithi117.stdout:0/951: write d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/d27/d31/fb2 [158504,82905] 0 2024-02-16T19:29:24.224 INFO:tasks.workunit.client.1.smithi117.stdout:0/952: chown d0/d9/d14/d23/f56 7842499 1 2024-02-16T19:29:24.225 INFO:tasks.workunit.client.0.smithi022.stdout:3/860: creat d7/d12/d18/dd/d3c/d5a/f127 x:0 0 0 2024-02-16T19:29:24.229 INFO:tasks.workunit.client.1.smithi117.stdout:0/953: mknod d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/d27/d31/c13f 0 2024-02-16T19:29:24.231 INFO:tasks.workunit.client.0.smithi022.stdout:3/861: creat d7/d12/d18/dd/d24/d93/de9/dc4/f128 x:0 0 0 2024-02-16T19:29:24.236 INFO:tasks.workunit.client.0.smithi022.stdout:3/862: rename d7/d12/d18/dd/d3c/d5a/ff6 to d7/d12/d18/dd/d3c/f129 0 2024-02-16T19:29:24.238 INFO:tasks.workunit.client.0.smithi022.stdout:3/863: mkdir d7/d12/d18/dac/d12a 0 2024-02-16T19:29:24.240 INFO:tasks.workunit.client.0.smithi022.stdout:3/864: fdatasync d7/d12/d18/dd/d3c/d5a/f69 0 2024-02-16T19:29:24.252 INFO:tasks.workunit.client.0.smithi022.stdout:9/868: dwrite db/d14/d16/d5d/d65/d6b/f9b [0,4194304] 0 2024-02-16T19:29:24.255 INFO:tasks.workunit.client.0.smithi022.stdout:4/840: dwrite d4/d6/dd/d22/da8/dc4/d86/dfd/faa [0,4194304] 0 2024-02-16T19:29:24.256 INFO:tasks.workunit.client.1.smithi117.stdout:2/996: write d0/f4c [1112183,6777] 0 2024-02-16T19:29:24.257 INFO:tasks.workunit.client.0.smithi022.stdout:4/841: fdatasync d4/d6/dd/d22/f51 0 2024-02-16T19:29:24.257 INFO:tasks.workunit.client.1.smithi117.stdout:5/990: mkdir d2/d9/d3c/d24/d13b 0 2024-02-16T19:29:24.258 INFO:tasks.workunit.client.0.smithi022.stdout:4/842: chown d4/d6/db6/f78 0 1 2024-02-16T19:29:24.258 INFO:tasks.workunit.client.1.smithi117.stdout:5/991: fdatasync d2/d4c/d4f/d67/fc5 0 2024-02-16T19:29:24.261 INFO:tasks.workunit.client.1.smithi117.stdout:1/866: write d2/d19/d59/d89/f64 [1662611,5277] 0 2024-02-16T19:29:24.268 INFO:tasks.workunit.client.1.smithi117.stdout:5/992: creat d2/d9/d12e/d52/d74/d8c/dac/f13c x:0 0 0 2024-02-16T19:29:24.270 INFO:tasks.workunit.client.0.smithi022.stdout:4/843: mknod d4/d6/d14/dc7/d8d/dd1/c109 0 2024-02-16T19:29:24.270 INFO:tasks.workunit.client.0.smithi022.stdout:4/844: readlink d4/d6/dd/df/d10/d55/lf8 0 2024-02-16T19:29:24.271 INFO:tasks.workunit.client.0.smithi022.stdout:4/845: stat d4/d6/dd/d22/c57 0 2024-02-16T19:29:24.272 INFO:tasks.workunit.client.1.smithi117.stdout:2/997: rename d0/d3/d40/cec to d0/d3/d40/c13c 0 2024-02-16T19:29:24.273 INFO:tasks.workunit.client.0.smithi022.stdout:9/869: link db/d14/dc8/dc9/d105/l54 db/d14/d16/d5d/d65/d6b/l129 0 2024-02-16T19:29:24.275 INFO:tasks.workunit.client.0.smithi022.stdout:9/870: read db/d18/f73 [5577030,18600] 0 2024-02-16T19:29:24.275 INFO:tasks.workunit.client.1.smithi117.stdout:5/993: creat d2/d3/d21/d49/d59/daa/f13d x:0 0 0 2024-02-16T19:29:24.276 INFO:tasks.workunit.client.0.smithi022.stdout:9/871: stat db/d14/d16/d21/d37/f10c 0 2024-02-16T19:29:24.281 INFO:tasks.workunit.client.1.smithi117.stdout:2/998: chown d0/d13/d4d/d53/d58/l129 39 1 2024-02-16T19:29:24.282 INFO:tasks.workunit.client.1.smithi117.stdout:2/999: write d0/d3/d34/d94/d46/d68/d76/da1/d2f/f10e [780435,68632] 0 2024-02-16T19:29:24.282 INFO:tasks.workunit.client.0.smithi022.stdout:9/872: dwrite db/d14/d19/d1a/f50 [4194304,4194304] 0 2024-02-16T19:29:24.289 INFO:tasks.workunit.client.0.smithi022.stdout:4/846: truncate d4/d6/d14/dc7/d6a/fea 540661 0 2024-02-16T19:29:24.291 INFO:tasks.workunit.client.0.smithi022.stdout:1/924: write d6/d8/d22/d47/d54/f65 [6284947,2081] 0 2024-02-16T19:29:24.291 INFO:tasks.workunit.client.0.smithi022.stdout:6/906: dwrite d1/d8/dac/ff9 [0,4194304] 0 2024-02-16T19:29:24.298 INFO:tasks.workunit.client.1.smithi117.stdout:4/982: truncate d0/da/d21/f22 1445887 0 2024-02-16T19:29:24.299 INFO:tasks.workunit.client.0.smithi022.stdout:1/925: creat d6/d8/d22/d83/f11c x:0 0 0 2024-02-16T19:29:24.300 INFO:tasks.workunit.client.1.smithi117.stdout:4/983: read d0/da/d21/d3d/d28/d64/d73/d91/d37/d98/fe7 [126476,127030] 0 2024-02-16T19:29:24.301 INFO:tasks.workunit.client.1.smithi117.stdout:4/984: readlink d0/da/d21/d3d/d28/d64/d8a/ld0 0 2024-02-16T19:29:24.303 INFO:tasks.workunit.client.0.smithi022.stdout:6/907: rename d1/d8/da/d29/deb/dfd/lbc to d1/d8/da/d29/deb/dfd/d85/l132 0 2024-02-16T19:29:24.304 INFO:tasks.workunit.client.1.smithi117.stdout:4/985: dread - d0/da/d21/d3d/d28/d66/da2/dac/fec zero size 2024-02-16T19:29:24.305 INFO:tasks.workunit.client.0.smithi022.stdout:1/926: fsync d6/d8/d22/d67/fd6 0 2024-02-16T19:29:24.307 INFO:tasks.workunit.client.0.smithi022.stdout:1/927: mknod d6/d8/dba/c11d 0 2024-02-16T19:29:24.310 INFO:tasks.workunit.client.0.smithi022.stdout:6/908: link d1/d8/dac/d101/da3/c11c d1/d8/da/d29/deb/dfd/d53/d103/c133 0 2024-02-16T19:29:24.311 INFO:tasks.workunit.client.0.smithi022.stdout:6/909: truncate d1/d8/da/d29/deb/f114 200878 0 2024-02-16T19:29:24.313 INFO:tasks.workunit.client.0.smithi022.stdout:1/928: link d6/d8/d6c/lda d6/d8/d22/d16/d17/d63/dee/l11e 0 2024-02-16T19:29:24.313 INFO:tasks.workunit.client.0.smithi022.stdout:6/910: write d1/d8/da/d29/deb/dfd/d85/d9c/ff6 [110710,97151] 0 2024-02-16T19:29:24.315 INFO:tasks.workunit.client.1.smithi117.stdout:4/986: dread d0/da/d21/d3d/d28/d66/f7f [4194304,4194304] 0 2024-02-16T19:29:24.315 INFO:tasks.workunit.client.0.smithi022.stdout:1/929: fdatasync d6/d8/f7d 0 2024-02-16T19:29:24.316 INFO:tasks.workunit.client.1.smithi117.stdout:4/987: chown d0/da/d21/d3d/d28/d64/d73/d91/d37/d98/f99 1689240055 1 2024-02-16T19:29:24.319 INFO:tasks.workunit.client.1.smithi117.stdout:4/988: mkdir d0/da/d21/d3d/d28/d64/d8a/d92/de8/d13d 0 2024-02-16T19:29:24.324 INFO:tasks.workunit.client.0.smithi022.stdout:1/930: getdents d6/d8/d22/d3a/d7a 0 2024-02-16T19:29:24.326 INFO:tasks.workunit.client.0.smithi022.stdout:1/931: chown d6/d8/d22/d67/d7e/l106 11 1 2024-02-16T19:29:24.328 INFO:tasks.workunit.client.0.smithi022.stdout:1/932: chown d6/d8/d22/d3f/f84 855 1 2024-02-16T19:29:24.329 INFO:tasks.workunit.client.0.smithi022.stdout:1/933: readlink d6/la 0 2024-02-16T19:29:24.334 INFO:tasks.workunit.client.0.smithi022.stdout:1/934: rename d6/d8/d22/d16/d17/d26/l4d to d6/d8/d22/d3a/d80/l11f 0 2024-02-16T19:29:24.335 INFO:tasks.workunit.client.0.smithi022.stdout:1/935: readlink d6/l5c 0 2024-02-16T19:29:24.337 INFO:tasks.workunit.client.0.smithi022.stdout:1/936: creat d6/d8/d22/d3a/d80/f120 x:0 0 0 2024-02-16T19:29:24.338 INFO:tasks.workunit.client.1.smithi117.stdout:0/954: rmdir d0/d9/d14/d1d/d60/d76 39 2024-02-16T19:29:24.338 INFO:tasks.workunit.client.0.smithi022.stdout:1/937: dread - d6/d8/d22/d67/d7e/fa5 zero size 2024-02-16T19:29:24.340 INFO:tasks.workunit.client.0.smithi022.stdout:1/938: readlink d6/l27 0 2024-02-16T19:29:24.341 INFO:tasks.workunit.client.0.smithi022.stdout:1/939: chown d6/d8/d22/d16/c6a 6047 1 2024-02-16T19:29:24.343 INFO:tasks.workunit.client.1.smithi117.stdout:0/955: mknod d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/dd4/c140 0 2024-02-16T19:29:24.345 INFO:tasks.workunit.client.1.smithi117.stdout:7/946: dwrite d4/d1c/d39/d9f/fca [0,4194304] 0 2024-02-16T19:29:24.346 INFO:tasks.workunit.client.1.smithi117.stdout:9/893: write dd/d45/d73/f82 [841208,84817] 0 2024-02-16T19:29:24.347 INFO:tasks.workunit.client.1.smithi117.stdout:7/947: rmdir d4/d13/da3 39 2024-02-16T19:29:24.347 INFO:tasks.workunit.client.0.smithi022.stdout:1/940: dwrite d6/d8/d22/f1b [0,4194304] 0 2024-02-16T19:29:24.347 INFO:tasks.workunit.client.1.smithi117.stdout:0/956: dread d0/d9/d2c/da9/faa [0,4194304] 0 2024-02-16T19:29:24.349 INFO:tasks.workunit.client.0.smithi022.stdout:1/941: dread - d6/d8/d22/f11a zero size 2024-02-16T19:29:24.350 INFO:tasks.workunit.client.1.smithi117.stdout:7/948: truncate d4/d1c/d87/df0/dfe/f13a 616095 0 2024-02-16T19:29:24.354 INFO:tasks.workunit.client.1.smithi117.stdout:9/894: fdatasync dd/d45/d31/d63/d8e/f96 0 2024-02-16T19:29:24.357 INFO:tasks.workunit.client.1.smithi117.stdout:7/949: truncate d4/d1c/d39/d9f/fd0 1042974 0 2024-02-16T19:29:24.359 INFO:tasks.workunit.client.0.smithi022.stdout:8/976: sync 2024-02-16T19:29:24.362 INFO:tasks.workunit.client.1.smithi117.stdout:6/956: creat d5/db/d7d/f13a x:0 0 0 2024-02-16T19:29:24.366 INFO:tasks.workunit.client.1.smithi117.stdout:7/950: mknod d4/d1c/d39/d46/da4/d15d/c160 0 2024-02-16T19:29:24.368 INFO:tasks.workunit.client.1.smithi117.stdout:6/957: chown d5/d4b/d20/d2d/l33 344259 1 2024-02-16T19:29:24.368 INFO:tasks.workunit.client.1.smithi117.stdout:7/951: readlink d4/d1c/dc0/ldb 0 2024-02-16T19:29:24.371 INFO:tasks.workunit.client.1.smithi117.stdout:6/958: getdents d5/d4b/d44/d52/d81/dfc/d106 0 2024-02-16T19:29:24.375 INFO:tasks.workunit.client.1.smithi117.stdout:6/959: creat d5/d4b/f13b x:0 0 0 2024-02-16T19:29:24.376 INFO:tasks.workunit.client.1.smithi117.stdout:7/952: dwrite d4/d13/da3/f124 [0,4194304] 0 2024-02-16T19:29:24.377 INFO:tasks.workunit.client.1.smithi117.stdout:7/953: readlink d4/d1c/d87/d89/d60/l8f 0 2024-02-16T19:29:24.379 INFO:tasks.workunit.client.1.smithi117.stdout:5/994: rename d2/d3/d21/d49/d59 to d2/d9/d12e/d52/d74/d8f/d13e 0 2024-02-16T19:29:24.381 INFO:tasks.workunit.client.1.smithi117.stdout:6/960: mknod d5/d4b/d71/dc7/d72/c13c 0 2024-02-16T19:29:24.382 INFO:tasks.workunit.client.0.smithi022.stdout:9/873: write db/d14/d16/d21/fb6 [3777563,91403] 0 2024-02-16T19:29:24.382 INFO:tasks.workunit.client.1.smithi117.stdout:6/961: stat d5/d4b/d44/d52/lb7 0 2024-02-16T19:29:24.383 INFO:tasks.workunit.client.1.smithi117.stdout:6/962: chown d5/db/d7d/d32/da8 161805130 1 2024-02-16T19:29:24.384 INFO:tasks.workunit.client.1.smithi117.stdout:9/895: rename dd/d45/c92 to dd/d45/d31/d63/dba/df0/daf/df4/dd7/c12f 0 2024-02-16T19:29:24.385 INFO:tasks.workunit.client.0.smithi022.stdout:7/945: sync 2024-02-16T19:29:24.385 INFO:tasks.workunit.client.0.smithi022.stdout:5/924: sync 2024-02-16T19:29:24.386 INFO:tasks.workunit.client.0.smithi022.stdout:2/870: sync 2024-02-16T19:29:24.386 INFO:tasks.workunit.client.1.smithi117.stdout:9/896: chown dd/d45/d31/d63/dba/df0/f1f 48 1 2024-02-16T19:29:24.387 INFO:tasks.workunit.client.1.smithi117.stdout:9/897: fdatasync dd/d45/d73/f94 0 2024-02-16T19:29:24.391 INFO:tasks.workunit.client.0.smithi022.stdout:4/847: dwrite d4/d6/dd/d22/da8/dc4/d86/dfd/f7d [0,4194304] 0 2024-02-16T19:29:24.397 INFO:tasks.workunit.client.1.smithi117.stdout:7/954: dread d4/d1c/d87/d89/d60/d79/fd7 [0,4194304] 0 2024-02-16T19:29:24.398 INFO:tasks.workunit.client.1.smithi117.stdout:6/963: rename d5/db/d7d/d32/c80 to d5/d4b/d20/d69/d115/c13d 0 2024-02-16T19:29:24.401 INFO:tasks.workunit.client.0.smithi022.stdout:7/946: rename d9/d5d/dc7/d118 to d9/da/d2c/d6d/d146 0 2024-02-16T19:29:24.405 INFO:tasks.workunit.client.1.smithi117.stdout:5/995: creat d2/d9/d12e/d96/dc4/dca/df3/d128/f13f x:0 0 0 2024-02-16T19:29:24.407 INFO:tasks.workunit.client.1.smithi117.stdout:4/989: write d0/da/d21/d3d/d28/d64/d73/d91/d9e/ff4 [967304,742] 0 2024-02-16T19:29:24.412 INFO:tasks.workunit.client.0.smithi022.stdout:6/911: dwrite d1/d8/ff [0,4194304] 0 2024-02-16T19:29:24.413 INFO:tasks.workunit.client.1.smithi117.stdout:4/990: dwrite d0/f72 [0,4194304] 0 2024-02-16T19:29:24.414 INFO:tasks.workunit.client.0.smithi022.stdout:4/848: creat d4/d6/dd/df/d10/d55/d108/f10a x:0 0 0 2024-02-16T19:29:24.417 INFO:tasks.workunit.client.1.smithi117.stdout:9/898: truncate dd/d47/d48/f98 3197961 0 2024-02-16T19:29:24.420 INFO:tasks.workunit.client.1.smithi117.stdout:5/996: creat d2/d9/d3c/d24/f140 x:0 0 0 2024-02-16T19:29:24.422 INFO:tasks.workunit.client.1.smithi117.stdout:5/997: write d2/d9/d12e/d52/d74/d8f/d13e/daa/f133 [804081,73707] 0 2024-02-16T19:29:24.423 INFO:tasks.workunit.client.1.smithi117.stdout:7/955: creat d4/d1c/d87/df0/dfe/d15b/f161 x:0 0 0 2024-02-16T19:29:24.425 INFO:tasks.workunit.client.0.smithi022.stdout:2/871: dread d0/d32/f62 [0,4194304] 0 2024-02-16T19:29:24.426 INFO:tasks.workunit.client.0.smithi022.stdout:6/912: chown d1/d8/da/d7b/dae/dbb/fbd 28922821 1 2024-02-16T19:29:24.426 INFO:tasks.workunit.client.1.smithi117.stdout:0/957: write d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/d27/d31/f38 [2493830,48097] 0 2024-02-16T19:29:24.433 INFO:tasks.workunit.client.1.smithi117.stdout:4/991: symlink d0/da/d21/d3d/d28/d64/d8a/d92/l13e 0 2024-02-16T19:29:24.434 INFO:tasks.workunit.client.0.smithi022.stdout:4/849: rename d4/d6/dd/d22/d30/l69 to d4/d6/dd/d22/d83/d9e/l10b 0 2024-02-16T19:29:24.434 INFO:tasks.workunit.client.0.smithi022.stdout:6/913: chown d1/d8/da/d29/deb/dfd/d53/c107 1252169 1 2024-02-16T19:29:24.437 INFO:tasks.workunit.client.0.smithi022.stdout:2/872: creat d0/d5d/f122 x:0 0 0 2024-02-16T19:29:24.438 INFO:tasks.workunit.client.1.smithi117.stdout:1/867: sync 2024-02-16T19:29:24.439 INFO:tasks.workunit.client.0.smithi022.stdout:4/850: rename d4 to d4/d6/dd/d22/d83/d10c 22 2024-02-16T19:29:24.440 INFO:tasks.workunit.client.0.smithi022.stdout:1/942: dwrite d6/d8/d22/d47/f4e [0,4194304] 0 2024-02-16T19:29:24.443 INFO:tasks.workunit.client.1.smithi117.stdout:7/956: unlink d4/f1a 0 2024-02-16T19:29:24.444 INFO:tasks.workunit.client.1.smithi117.stdout:7/957: chown d4/d1c/d39/d46/f48 216499295 1 2024-02-16T19:29:24.444 INFO:tasks.workunit.client.0.smithi022.stdout:8/977: dwrite d3/d12/d17/d64/d8f/f109 [0,4194304] 0 2024-02-16T19:29:24.444 INFO:tasks.workunit.client.1.smithi117.stdout:7/958: dread - d4/d1c/d39/d9f/f151 zero size 2024-02-16T19:29:24.447 INFO:tasks.workunit.client.0.smithi022.stdout:5/925: write dd/d1f/f9f [371118,102054] 0 2024-02-16T19:29:24.449 INFO:tasks.workunit.client.1.smithi117.stdout:0/958: symlink d0/d78/l141 0 2024-02-16T19:29:24.453 INFO:tasks.workunit.client.0.smithi022.stdout:9/874: dwrite db/d14/d16/d27/d55/fb8 [0,4194304] 0 2024-02-16T19:29:24.454 INFO:tasks.workunit.client.1.smithi117.stdout:5/998: dread d2/d9/d12e/d52/d74/d8f/fc7 [0,4194304] 0 2024-02-16T19:29:24.460 INFO:tasks.workunit.client.1.smithi117.stdout:4/992: mkdir d0/da/d21/db4/de4/d13f 0 2024-02-16T19:29:24.462 INFO:tasks.workunit.client.0.smithi022.stdout:3/865: sync 2024-02-16T19:29:24.463 INFO:tasks.workunit.client.1.smithi117.stdout:6/964: write d5/d4b/d44/d7c/d125/fd3 [424120,37186] 0 2024-02-16T19:29:24.464 INFO:tasks.workunit.client.1.smithi117.stdout:6/965: readlink d5/db/d7d/d32/l7b 0 2024-02-16T19:29:24.471 INFO:tasks.workunit.client.1.smithi117.stdout:7/959: creat d4/d1c/d39/d46/d47/f162 x:0 0 0 2024-02-16T19:29:24.479 INFO:tasks.workunit.client.1.smithi117.stdout:5/999: unlink d2/d9/d12e/d96/dbc/d30/d38/d51/f7e 0 2024-02-16T19:29:24.481 INFO:tasks.workunit.client.1.smithi117.stdout:4/993: creat d0/da/d4a/f140 x:0 0 0 2024-02-16T19:29:24.485 INFO:tasks.workunit.client.1.smithi117.stdout:9/899: dwrite dd/d47/d5b/d5f/d77/d7c/fa2 [0,4194304] 0 2024-02-16T19:29:24.487 INFO:tasks.workunit.client.1.smithi117.stdout:0/959: symlink d0/d9/d14/d1d/d125/l142 0 2024-02-16T19:29:24.488 INFO:tasks.workunit.client.1.smithi117.stdout:6/966: mknod d5/c13e 0 2024-02-16T19:29:24.491 INFO:tasks.workunit.client.1.smithi117.stdout:4/994: dread d0/da/d21/d3d/d28/d64/d8a/fea [0,4194304] 0 2024-02-16T19:29:24.492 INFO:tasks.workunit.client.1.smithi117.stdout:7/960: symlink d4/d1c/d87/d89/l163 0 2024-02-16T19:29:24.494 INFO:tasks.workunit.client.1.smithi117.stdout:1/868: dwrite d2/f53 [0,4194304] 0 2024-02-16T19:29:24.496 INFO:tasks.workunit.client.1.smithi117.stdout:0/960: symlink d0/d9/d14/d1d/d60/dc3/d109/d132/l143 0 2024-02-16T19:29:24.498 INFO:tasks.workunit.client.1.smithi117.stdout:9/900: mkdir dd/db5/d117/d130 0 2024-02-16T19:29:24.498 INFO:tasks.workunit.client.1.smithi117.stdout:9/901: chown dd/ff5 176890 1 2024-02-16T19:29:24.501 INFO:tasks.workunit.client.1.smithi117.stdout:1/869: creat d2/d19/d59/d76/d8c/f12d x:0 0 0 2024-02-16T19:29:24.507 INFO:tasks.workunit.client.1.smithi117.stdout:9/902: symlink dd/d45/d31/d63/d8e/l131 0 2024-02-16T19:29:24.513 INFO:tasks.workunit.client.1.smithi117.stdout:6/967: dread d5/db/d7d/d32/d37/f41 [0,4194304] 0 2024-02-16T19:29:24.514 INFO:tasks.workunit.client.0.smithi022.stdout:7/947: link d9/da/f133 d9/da/d2c/d46/f147 0 2024-02-16T19:29:24.516 INFO:tasks.workunit.client.1.smithi117.stdout:9/903: mknod dd/d47/c132 0 2024-02-16T19:29:24.520 INFO:tasks.workunit.client.1.smithi117.stdout:9/904: creat dd/d45/d31/d63/d8e/dbb/f133 x:0 0 0 2024-02-16T19:29:24.524 INFO:tasks.workunit.client.0.smithi022.stdout:4/851: rmdir d4/d6/dd 39 2024-02-16T19:29:24.525 INFO:tasks.workunit.client.0.smithi022.stdout:4/852: chown d4/d6/d14/d64/d4e/f8b 971 1 2024-02-16T19:29:24.525 INFO:tasks.workunit.client.1.smithi117.stdout:9/905: dwrite dd/d45/d73/f94 [0,4194304] 0 2024-02-16T19:29:24.526 INFO:tasks.workunit.client.0.smithi022.stdout:1/943: rename d6/d8/d22/d16/d17/d26/lb2 to d6/d8/d22/d16/l121 0 2024-02-16T19:29:24.528 INFO:tasks.workunit.client.0.smithi022.stdout:8/978: creat d3/d12/d2b/d54/db8/f150 x:0 0 0 2024-02-16T19:29:24.531 INFO:tasks.workunit.client.0.smithi022.stdout:5/926: mknod dd/d1f/d48/d21/d2f/d46/d5e/c12c 0 2024-02-16T19:29:24.531 INFO:tasks.workunit.client.0.smithi022.stdout:1/944: write d6/f104 [2585,26518] 0 2024-02-16T19:29:24.533 INFO:tasks.workunit.client.1.smithi117.stdout:4/995: write d0/da/d21/d3d/d28/d64/f1f [3811032,39868] 0 2024-02-16T19:29:24.536 INFO:tasks.workunit.client.0.smithi022.stdout:3/866: creat d7/d21/f12b x:0 0 0 2024-02-16T19:29:24.536 INFO:tasks.workunit.client.0.smithi022.stdout:6/914: mkdir d1/d8/da/d29/deb/dfd/d85/d9c/d12a/d134 0 2024-02-16T19:29:24.540 INFO:tasks.workunit.client.1.smithi117.stdout:7/961: dwrite d4/d1c/d87/d89/fd4 [4194304,4194304] 0 2024-02-16T19:29:24.540 INFO:tasks.workunit.client.1.smithi117.stdout:4/996: dread d0/da/d21/d3d/d28/d66/d97/d6d/fda [0,4194304] 0 2024-02-16T19:29:24.540 INFO:tasks.workunit.client.0.smithi022.stdout:7/948: rename d9/da/d2c/d6d/d146/la0 to d9/da/d1f/d44/d5b/df4/l148 0 2024-02-16T19:29:24.541 INFO:tasks.workunit.client.0.smithi022.stdout:8/979: rmdir d3/d12/d2b 39 2024-02-16T19:29:24.542 INFO:tasks.workunit.client.1.smithi117.stdout:4/997: write d0/da/d21/d3d/d28/d64/d8a/d92/f89 [1489169,49721] 0 2024-02-16T19:29:24.544 INFO:tasks.workunit.client.0.smithi022.stdout:1/945: fdatasync d6/d8/d22/d3f/d99/f9b 0 2024-02-16T19:29:24.545 INFO:tasks.workunit.client.1.smithi117.stdout:7/962: mknod d4/d1c/d39/d46/d136/c164 0 2024-02-16T19:29:24.546 INFO:tasks.workunit.client.0.smithi022.stdout:5/927: mkdir dd/d1f/d48/d21/d64/d12d 0 2024-02-16T19:29:24.547 INFO:tasks.workunit.client.0.smithi022.stdout:9/875: dread db/f7b [0,4194304] 0 2024-02-16T19:29:24.548 INFO:tasks.workunit.client.1.smithi117.stdout:7/963: truncate d4/d1c/d87/d89/d5f/dae/f158 796851 0 2024-02-16T19:29:24.549 INFO:tasks.workunit.client.0.smithi022.stdout:6/915: mknod d1/d8/da/d29/deb/c135 0 2024-02-16T19:29:24.550 INFO:tasks.workunit.client.1.smithi117.stdout:7/964: read d4/d1c/d87/f66 [1236633,43485] 0 2024-02-16T19:29:24.551 INFO:tasks.workunit.client.0.smithi022.stdout:4/853: symlink d4/d6/l10d 0 2024-02-16T19:29:24.552 INFO:tasks.workunit.client.1.smithi117.stdout:1/870: dwrite d2/d19/d59/d89/d69/d80/d8e/f90 [0,4194304] 0 2024-02-16T19:29:24.554 INFO:tasks.workunit.client.0.smithi022.stdout:3/867: creat d7/d12/d18/dac/dd7/f12c x:0 0 0 2024-02-16T19:29:24.557 INFO:tasks.workunit.client.0.smithi022.stdout:2/873: dwrite d0/d2/f3d [0,4194304] 0 2024-02-16T19:29:24.558 INFO:tasks.workunit.client.1.smithi117.stdout:6/968: write d5/d4b/f1e [1059784,91276] 0 2024-02-16T19:29:24.559 INFO:tasks.workunit.client.1.smithi117.stdout:7/965: read d4/d13/d63/d71/f150 [52585,80725] 0 2024-02-16T19:29:24.564 INFO:tasks.workunit.client.1.smithi117.stdout:1/871: write d2/d19/d59/d89/d69/fe0 [4390401,28747] 0 2024-02-16T19:29:24.566 INFO:tasks.workunit.client.0.smithi022.stdout:9/876: unlink db/d14/d16/d5d/d65/f11c 0 2024-02-16T19:29:24.570 INFO:tasks.workunit.client.0.smithi022.stdout:6/916: creat d1/d8/da/d29/deb/dfd/d85/d9c/d12a/f136 x:0 0 0 2024-02-16T19:29:24.571 INFO:tasks.workunit.client.1.smithi117.stdout:9/906: dwrite dd/d45/d31/d63/d8e/dbb/fc9 [0,4194304] 0 2024-02-16T19:29:24.571 INFO:tasks.workunit.client.1.smithi117.stdout:0/961: sync 2024-02-16T19:29:24.574 INFO:tasks.workunit.client.1.smithi117.stdout:6/969: dread d5/d4b/d71/dc7/dce/fad [0,4194304] 0 2024-02-16T19:29:24.577 INFO:tasks.workunit.client.1.smithi117.stdout:1/872: rename d2/d19/d59/d76/d8c/l10a to d2/d19/d59/d89/d69/l12e 0 2024-02-16T19:29:24.580 INFO:tasks.workunit.client.0.smithi022.stdout:2/874: rename d0/d2/d18/c84 to d0/d2/d18/d104/c123 0 2024-02-16T19:29:24.581 INFO:tasks.workunit.client.1.smithi117.stdout:4/998: dread d0/da/d21/f65 [4194304,4194304] 0 2024-02-16T19:29:24.583 INFO:tasks.workunit.client.1.smithi117.stdout:0/962: mknod d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/c144 0 2024-02-16T19:29:24.584 INFO:tasks.workunit.client.0.smithi022.stdout:8/980: write d3/f107 [310787,10384] 0 2024-02-16T19:29:24.586 INFO:tasks.workunit.client.1.smithi117.stdout:6/970: creat d5/db/d7d/d32/f13f x:0 0 0 2024-02-16T19:29:24.588 INFO:tasks.workunit.client.1.smithi117.stdout:1/873: chown d2/d19/d59/d89/ca1 102771236 1 2024-02-16T19:29:24.589 INFO:tasks.workunit.client.0.smithi022.stdout:1/946: dwrite d6/d8/f111 [4194304,4194304] 0 2024-02-16T19:29:24.590 INFO:tasks.workunit.client.1.smithi117.stdout:1/874: read d2/d1d/d60/fd3 [383072,115196] 0 2024-02-16T19:29:24.590 INFO:tasks.workunit.client.1.smithi117.stdout:7/966: getdents d4/d1c/d87/d89/d60/d11f 0 2024-02-16T19:29:24.592 INFO:tasks.workunit.client.0.smithi022.stdout:5/928: dwrite dd/d1f/d48/d21/d2f/d40/d4e/d56/f59 [0,4194304] 0 2024-02-16T19:29:24.593 INFO:tasks.workunit.client.0.smithi022.stdout:4/854: mknod d4/d6/dd/d22/da8/dc4/d86/c10e 0 2024-02-16T19:29:24.595 INFO:tasks.workunit.client.1.smithi117.stdout:4/999: mknod d0/da/d21/dc2/c141 0 2024-02-16T19:29:24.599 INFO:tasks.workunit.client.1.smithi117.stdout:6/971: unlink d5/db/d7d/cbf 0 2024-02-16T19:29:24.600 INFO:tasks.workunit.client.1.smithi117.stdout:6/972: write d5/d4b/d71/dc7/d49/fbc [497108,78643] 0 2024-02-16T19:29:24.606 INFO:tasks.workunit.client.1.smithi117.stdout:0/963: mkdir d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/d27/d145 0 2024-02-16T19:29:24.610 INFO:tasks.workunit.client.0.smithi022.stdout:4/855: unlink d4/d6/d14/d64/c48 0 2024-02-16T19:29:24.613 INFO:tasks.workunit.client.0.smithi022.stdout:3/868: creat d7/d12/d18/f12d x:0 0 0 2024-02-16T19:29:24.614 INFO:tasks.workunit.client.0.smithi022.stdout:1/947: dread d6/f55 [0,4194304] 0 2024-02-16T19:29:24.615 INFO:tasks.workunit.client.1.smithi117.stdout:1/875: mknod d2/d19/d59/d89/c12f 0 2024-02-16T19:29:24.615 INFO:tasks.workunit.client.1.smithi117.stdout:1/876: chown d2/d19/d59/d89/d69/d80/d8e/df5 48920 1 2024-02-16T19:29:24.616 INFO:tasks.workunit.client.0.smithi022.stdout:1/948: stat d6/d8/d22/d16/d17/d5d/cb4 0 2024-02-16T19:29:24.616 INFO:tasks.workunit.client.0.smithi022.stdout:7/949: link d9/da/d2c/d46/d73/fd0 d9/da/d2c/d46/d73/dca/f149 0 2024-02-16T19:29:24.617 INFO:tasks.workunit.client.0.smithi022.stdout:7/950: fsync d9/da/d2c/d46/d10b/f121 0 2024-02-16T19:29:24.618 INFO:tasks.workunit.client.0.smithi022.stdout:2/875: mknod d0/d2/d18/d19/ddf/d10c/d117/c124 0 2024-02-16T19:29:24.618 INFO:tasks.workunit.client.0.smithi022.stdout:2/876: stat d0/d2/d2e/laa 0 2024-02-16T19:29:24.619 INFO:tasks.workunit.client.1.smithi117.stdout:9/907: getdents dd/d45/d31/d63/d8e 0 2024-02-16T19:29:24.623 INFO:tasks.workunit.client.0.smithi022.stdout:9/877: link db/f12 db/d14/dc8/f12a 0 2024-02-16T19:29:24.624 INFO:tasks.workunit.client.1.smithi117.stdout:6/973: dread d5/d9c/daa/f9a [4194304,4194304] 0 2024-02-16T19:29:24.624 INFO:tasks.workunit.client.1.smithi117.stdout:1/877: read d2/d1d/d60/fb2 [228650,49293] 0 2024-02-16T19:29:24.625 INFO:tasks.workunit.client.0.smithi022.stdout:6/917: creat d1/d8/da/d29/deb/dfd/f137 x:0 0 0 2024-02-16T19:29:24.625 INFO:tasks.workunit.client.1.smithi117.stdout:7/967: write d4/d1c/d39/f54 [1425154,9784] 0 2024-02-16T19:29:24.627 INFO:tasks.workunit.client.1.smithi117.stdout:0/964: mkdir d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/d27/d4e/d11b/d146 0 2024-02-16T19:29:24.628 INFO:tasks.workunit.client.1.smithi117.stdout:0/965: stat d0/d9/d14/d1d/d60/dc3/de9/lfe 0 2024-02-16T19:29:24.629 INFO:tasks.workunit.client.0.smithi022.stdout:4/856: symlink d4/d6/d14/d64/d4e/d54/d5a/l10f 0 2024-02-16T19:29:24.632 INFO:tasks.workunit.client.0.smithi022.stdout:3/869: rename d7/d12/d18/dd/d24/d29/c66 to d7/d12/d18/dd/d98/c12e 0 2024-02-16T19:29:24.639 INFO:tasks.workunit.client.0.smithi022.stdout:8/981: dwrite d3/f55 [0,4194304] 0 2024-02-16T19:29:24.641 INFO:tasks.workunit.client.1.smithi117.stdout:0/966: rename d0/d9/d14/d1d/d60/dc3/fcc to d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/d27/df9/f147 0 2024-02-16T19:29:24.642 INFO:tasks.workunit.client.0.smithi022.stdout:1/949: dwrite d6/d8/d22/d3a/d3e/ff6 [0,4194304] 0 2024-02-16T19:29:24.643 INFO:tasks.workunit.client.0.smithi022.stdout:6/918: symlink d1/d8/da/d29/deb/dfd/d48/de4/dfb/l138 0 2024-02-16T19:29:24.647 INFO:tasks.workunit.client.1.smithi117.stdout:9/908: creat dd/d47/f134 x:0 0 0 2024-02-16T19:29:24.648 INFO:tasks.workunit.client.0.smithi022.stdout:2/877: write d0/d2/d18/f34 [4519965,111624] 0 2024-02-16T19:29:24.651 INFO:tasks.workunit.client.1.smithi117.stdout:6/974: creat d5/f140 x:0 0 0 2024-02-16T19:29:24.652 INFO:tasks.workunit.client.1.smithi117.stdout:6/975: write d5/d4b/d44/f75 [2338945,20140] 0 2024-02-16T19:29:24.653 INFO:tasks.workunit.client.0.smithi022.stdout:7/951: dwrite d9/da/d1f/f22 [0,4194304] 0 2024-02-16T19:29:24.655 INFO:tasks.workunit.client.1.smithi117.stdout:7/968: truncate d4/d13/fb0 1313299 0 2024-02-16T19:29:24.659 INFO:tasks.workunit.client.1.smithi117.stdout:7/969: dread d4/dd3/f99 [0,4194304] 0 2024-02-16T19:29:24.662 INFO:tasks.workunit.client.1.smithi117.stdout:0/967: creat d0/d9/d14/d1d/d60/dc3/d109/d132/f148 x:0 0 0 2024-02-16T19:29:24.663 INFO:tasks.workunit.client.1.smithi117.stdout:0/968: chown d0/d9/d14/d1d/d60/dc3/dfd/f10b 1 1 2024-02-16T19:29:24.664 INFO:tasks.workunit.client.0.smithi022.stdout:5/929: getdents dd/d1f/d48/d21/d2f/d52/d77/da9 0 2024-02-16T19:29:24.668 INFO:tasks.workunit.client.0.smithi022.stdout:8/982: dread - d3/d12/d2b/d54/fcb zero size 2024-02-16T19:29:24.669 INFO:tasks.workunit.client.1.smithi117.stdout:6/976: symlink d5/d4b/d71/dc7/d72/l141 0 2024-02-16T19:29:24.670 INFO:tasks.workunit.client.0.smithi022.stdout:5/930: dwrite dd/d1f/f31 [0,4194304] 0 2024-02-16T19:29:24.670 INFO:tasks.workunit.client.0.smithi022.stdout:4/857: rename d4/d6/db6/f78 to d4/d6/dd/df/d10/de8/f110 0 2024-02-16T19:29:24.676 INFO:tasks.workunit.client.1.smithi117.stdout:1/878: getdents d2/d19/d59/d89/d69/d80/d8e/df5 0 2024-02-16T19:29:24.680 INFO:tasks.workunit.client.1.smithi117.stdout:0/969: rename d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/d27/d4e/c5a to d0/d9/d14/d1d/d60/dc3/dfd/c149 0 2024-02-16T19:29:24.682 INFO:tasks.workunit.client.1.smithi117.stdout:9/909: read - dd/d47/d5b/d5f/de6/f101 zero size 2024-02-16T19:29:24.688 INFO:tasks.workunit.client.1.smithi117.stdout:1/879: truncate d2/f57 508864 0 2024-02-16T19:29:24.689 INFO:tasks.workunit.client.1.smithi117.stdout:0/970: fsync d0/d78/d135/f64 0 2024-02-16T19:29:24.691 INFO:tasks.workunit.client.1.smithi117.stdout:9/910: mknod dd/d45/d31/d63/dba/d11e/c135 0 2024-02-16T19:29:24.695 INFO:tasks.workunit.client.1.smithi117.stdout:9/911: creat dd/db5/dc4/dd3/f136 x:0 0 0 2024-02-16T19:29:24.696 INFO:tasks.workunit.client.1.smithi117.stdout:7/970: getdents d4/d1c/d39/d46/dc4 0 2024-02-16T19:29:24.697 INFO:tasks.workunit.client.1.smithi117.stdout:6/977: getdents d5/db/d7d/d32/da8 0 2024-02-16T19:29:24.697 INFO:tasks.workunit.client.1.smithi117.stdout:6/978: readlink d5/d4b/d71/l7f 0 2024-02-16T19:29:24.699 INFO:tasks.workunit.client.1.smithi117.stdout:9/912: truncate dd/d45/d31/d63/dba/df0/fce 235073 0 2024-02-16T19:29:24.700 INFO:tasks.workunit.client.1.smithi117.stdout:6/979: chown d5/d9c/daa/d85/f8b 14074 1 2024-02-16T19:29:24.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:24 smithi022 ceph-mon[29221]: pgmap v15: 65 pgs: 65 active+clean; 3.5 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 79 MiB/s rd, 183 MiB/s wr, 404 op/s 2024-02-16T19:29:24.781 INFO:tasks.workunit.client.0.smithi022.stdout:7/952: unlink d9/faf 0 2024-02-16T19:29:24.783 INFO:tasks.workunit.client.0.smithi022.stdout:1/950: rename d6/d8/d22/d16/d17/d63/f68 to d6/d8/d22/d3a/d7a/f122 0 2024-02-16T19:29:24.783 INFO:tasks.workunit.client.0.smithi022.stdout:7/953: chown d9/da/d1f/d40/fa5 2375 1 2024-02-16T19:29:24.785 INFO:tasks.workunit.client.1.smithi117.stdout:0/971: dread d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/dd4/fd7 [0,4194304] 0 2024-02-16T19:29:24.786 INFO:tasks.workunit.client.1.smithi117.stdout:6/980: unlink d5/d4b/d44/d7c/d125/f10c 0 2024-02-16T19:29:24.786 INFO:tasks.workunit.client.0.smithi022.stdout:1/951: truncate d6/d8/d22/d3a/d80/f120 149010 0 2024-02-16T19:29:24.786 INFO:tasks.workunit.client.1.smithi117.stdout:0/972: fsync d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/d27/f4d 0 2024-02-16T19:29:24.790 INFO:tasks.workunit.client.1.smithi117.stdout:9/913: fsync dd/d45/d31/d63/d121/d9a/fec 0 2024-02-16T19:29:24.795 INFO:tasks.workunit.client.0.smithi022.stdout:6/919: rename d1/d8/da/d29/deb/dfd/d53/f5c to d1/d8/da/d29/d61/d57/d121/f139 0 2024-02-16T19:29:24.798 INFO:tasks.workunit.client.1.smithi117.stdout:0/973: creat d0/d9/d14/d23/f14a x:0 0 0 2024-02-16T19:29:24.799 INFO:tasks.workunit.client.0.smithi022.stdout:8/983: link d3/d12/d2b/d2e/l6f d3/d25/ddb/l151 0 2024-02-16T19:29:24.801 INFO:tasks.workunit.client.0.smithi022.stdout:3/870: dwrite d7/d12/d18/dd/d31/d7f/faf [0,4194304] 0 2024-02-16T19:29:24.803 INFO:tasks.workunit.client.1.smithi117.stdout:9/914: creat dd/d45/d31/d63/d8e/f137 x:0 0 0 2024-02-16T19:29:24.804 INFO:tasks.workunit.client.0.smithi022.stdout:4/858: getdents d4/d6/dd/d22/da8/dc4 0 2024-02-16T19:29:24.810 INFO:tasks.workunit.client.0.smithi022.stdout:6/920: creat d1/d8/da/d7b/f13a x:0 0 0 2024-02-16T19:29:24.812 INFO:tasks.workunit.client.1.smithi117.stdout:6/981: rename d5/db/d7d/d32/da8/de1/ld9 to d5/d4b/d20/l142 0 2024-02-16T19:29:24.812 INFO:tasks.workunit.client.1.smithi117.stdout:0/974: dread d0/d9/d14/d1d/d60/d76/d7b/fb1 [0,4194304] 0 2024-02-16T19:29:24.812 INFO:tasks.workunit.client.0.smithi022.stdout:7/954: truncate d9/da/d2c/d6d/d146/f123 4555488 0 2024-02-16T19:29:24.813 INFO:tasks.workunit.client.1.smithi117.stdout:0/975: stat d0/f10 0 2024-02-16T19:29:24.813 INFO:tasks.workunit.client.1.smithi117.stdout:1/880: truncate d2/d19/d59/d89/d69/d80/d8e/f90 2507365 0 2024-02-16T19:29:24.813 INFO:tasks.workunit.client.0.smithi022.stdout:7/955: read d9/da/d1f/d67/f12f [132597,15754] 0 2024-02-16T19:29:24.814 INFO:tasks.workunit.client.1.smithi117.stdout:6/982: dread d5/d4b/d71/dc7/dce/fad [0,4194304] 0 2024-02-16T19:29:24.816 INFO:tasks.workunit.client.0.smithi022.stdout:8/984: mkdir d3/d25/d138/d152 0 2024-02-16T19:29:24.817 INFO:tasks.workunit.client.1.smithi117.stdout:7/971: truncate d4/d1c/d87/d89/d5f/dae/f158 673456 0 2024-02-16T19:29:24.818 INFO:tasks.workunit.client.1.smithi117.stdout:9/915: rename dd/d45/d31/c53 to dd/d47/d12c/c138 0 2024-02-16T19:29:24.818 INFO:tasks.workunit.client.0.smithi022.stdout:7/956: dread d9/da/d3b/dba/ff5 [0,4194304] 0 2024-02-16T19:29:24.819 INFO:tasks.workunit.client.1.smithi117.stdout:9/916: chown dd/d45/d31/d63/dba/df0/daf/df4/d10d/f11b 537309 1 2024-02-16T19:29:24.820 INFO:tasks.workunit.client.0.smithi022.stdout:7/957: write d9/da/d1f/d40/fa5 [2282756,11253] 0 2024-02-16T19:29:24.820 INFO:tasks.workunit.client.0.smithi022.stdout:1/952: mkdir d6/d8/d22/d47/d51/d8d/d123 0 2024-02-16T19:29:24.821 INFO:tasks.workunit.client.0.smithi022.stdout:7/958: chown d9/da/d2c/d46/f12a 0 1 2024-02-16T19:29:24.822 INFO:tasks.workunit.client.0.smithi022.stdout:3/871: creat d7/d12/d18/dd/d24/d58/d70/f12f x:0 0 0 2024-02-16T19:29:24.827 INFO:tasks.workunit.client.0.smithi022.stdout:6/921: unlink d1/d8/da/d29/deb/dfd/d85/d9c/ff6 0 2024-02-16T19:29:24.828 INFO:tasks.workunit.client.1.smithi117.stdout:0/976: creat d0/d9/d14/d1d/d60/d76/d7b/d8d/d55/f14b x:0 0 0 2024-02-16T19:29:24.828 INFO:tasks.workunit.client.1.smithi117.stdout:0/977: chown d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/d27/d31/cdd 1414421 1 2024-02-16T19:29:24.828 INFO:tasks.workunit.client.1.smithi117.stdout:0/978: write d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/d27/d31/d94/f10d [2588854,65152] 0 2024-02-16T19:29:24.831 INFO:tasks.workunit.client.1.smithi117.stdout:7/972: fdatasync d4/f5 0 2024-02-16T19:29:24.832 INFO:tasks.workunit.client.1.smithi117.stdout:7/973: chown d4/d1c/d39/d46/dc4 300943321 1 2024-02-16T19:29:24.835 INFO:tasks.workunit.client.1.smithi117.stdout:1/881: rename d2/d1d/l2e to d2/d19/d59/d89/d8d/l130 0 2024-02-16T19:29:24.838 INFO:tasks.workunit.client.0.smithi022.stdout:6/922: dread d1/d8/da/d29/deb/dfd/f99 [0,4194304] 0 2024-02-16T19:29:24.842 INFO:tasks.workunit.client.1.smithi117.stdout:1/882: dwrite d2/d19/d59/d89/f64 [0,4194304] 0 2024-02-16T19:29:24.842 INFO:tasks.workunit.client.0.smithi022.stdout:1/953: write d6/d8/d22/d47/fd3 [2517024,84867] 0 2024-02-16T19:29:24.843 INFO:tasks.workunit.client.1.smithi117.stdout:1/883: chown d2/d19/d59/d76/d8c 5265678 1 2024-02-16T19:29:24.844 INFO:tasks.workunit.client.0.smithi022.stdout:7/959: symlink d9/da/d1f/d67/ddb/dd4/d122/l14a 0 2024-02-16T19:29:24.844 INFO:tasks.workunit.client.1.smithi117.stdout:0/979: dread d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/d27/f42 [0,4194304] 0 2024-02-16T19:29:24.846 INFO:tasks.workunit.client.0.smithi022.stdout:3/872: symlink d7/d12/d18/dd/d31/dbf/dc5/l130 0 2024-02-16T19:29:24.853 INFO:tasks.workunit.client.1.smithi117.stdout:7/974: mknod d4/d13/d63/ddd/d117/c165 0 2024-02-16T19:29:24.854 INFO:tasks.workunit.client.0.smithi022.stdout:8/985: mknod d3/d12/da6/c153 0 2024-02-16T19:29:24.855 INFO:tasks.workunit.client.0.smithi022.stdout:8/986: chown d3/d12/d2b/d54/d97/ff7 100330675 1 2024-02-16T19:29:24.860 INFO:tasks.workunit.client.0.smithi022.stdout:3/873: creat d7/d12/d18/dac/f131 x:0 0 0 2024-02-16T19:29:24.860 INFO:tasks.workunit.client.0.smithi022.stdout:3/874: chown d7/d12/d18/lb7 18332 1 2024-02-16T19:29:24.861 INFO:tasks.workunit.client.0.smithi022.stdout:3/875: chown d7/d12/d18/dd/d31/d7f/faf 0 1 2024-02-16T19:29:24.862 INFO:tasks.workunit.client.0.smithi022.stdout:4/859: rename d4/d6/db6/fb5 to d4/d6/dd/d22/da8/dc4/d86/dfd/df2/f111 0 2024-02-16T19:29:24.863 INFO:tasks.workunit.client.0.smithi022.stdout:4/860: readlink d4/d6/d14/d64/d4e/d54/d5a/l10f 0 2024-02-16T19:29:24.866 INFO:tasks.workunit.client.1.smithi117.stdout:0/980: dread d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/d27/f42 [0,4194304] 0 2024-02-16T19:29:24.868 INFO:tasks.workunit.client.0.smithi022.stdout:3/876: rename d7/d12/d18/dd/d24/d53/d116/f78 to d7/d12/d18/dac/d12a/f132 0 2024-02-16T19:29:24.870 INFO:tasks.workunit.client.1.smithi117.stdout:9/917: truncate dd/d45/d31/d63/dba/df0/daf/fe1 154885 0 2024-02-16T19:29:24.872 INFO:tasks.workunit.client.0.smithi022.stdout:7/960: write d9/da9/f13b [1459345,117188] 0 2024-02-16T19:29:24.873 INFO:tasks.workunit.client.0.smithi022.stdout:7/961: write d9/da/d2c/d46/f136 [126842,122] 0 2024-02-16T19:29:24.876 INFO:tasks.workunit.client.0.smithi022.stdout:8/987: write d3/d12/d2b/d54/d97/ffc [161820,127077] 0 2024-02-16T19:29:24.882 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:24 smithi117 ceph-mon[35309]: pgmap v15: 65 pgs: 65 active+clean; 3.5 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 79 MiB/s rd, 183 MiB/s wr, 404 op/s 2024-02-16T19:29:24.883 INFO:tasks.workunit.client.0.smithi022.stdout:4/861: link d4/d6/dd/d22/f68 d4/f112 0 2024-02-16T19:29:24.885 INFO:tasks.workunit.client.0.smithi022.stdout:8/988: stat d3/d12/c49 0 2024-02-16T19:29:24.885 INFO:tasks.workunit.client.0.smithi022.stdout:3/877: write d7/d12/d18/dd/d24/fd0 [1209182,103948] 0 2024-02-16T19:29:24.887 INFO:tasks.workunit.client.0.smithi022.stdout:3/878: dread - d7/d12/d18/dd/d3c/d5a/f111 zero size 2024-02-16T19:29:24.888 INFO:tasks.workunit.client.0.smithi022.stdout:4/862: mkdir d4/d6/dd/d22/d83/d113 0 2024-02-16T19:29:24.892 INFO:tasks.workunit.client.0.smithi022.stdout:4/863: stat d4/d6/d14/d64/d4e/d54/dba 0 2024-02-16T19:29:24.892 INFO:tasks.workunit.client.0.smithi022.stdout:8/989: rename d3/d12/d2b/d54/d74/f82 to d3/d25/d138/d152/f154 0 2024-02-16T19:29:24.894 INFO:tasks.workunit.client.0.smithi022.stdout:4/864: readlink d4/d6/dd/d22/d30/l9d 0 2024-02-16T19:29:24.896 INFO:tasks.workunit.client.0.smithi022.stdout:7/962: dwrite d9/da/f29 [4194304,4194304] 0 2024-02-16T19:29:24.896 INFO:tasks.workunit.client.0.smithi022.stdout:3/879: dwrite d7/d12/d18/dd/d3c/f115 [0,4194304] 0 2024-02-16T19:29:24.898 INFO:tasks.workunit.client.0.smithi022.stdout:8/990: getdents d3/dd3 0 2024-02-16T19:29:24.903 INFO:tasks.workunit.client.0.smithi022.stdout:4/865: symlink d4/d6/dd/df/d10/de8/l114 0 2024-02-16T19:29:24.907 INFO:tasks.workunit.client.0.smithi022.stdout:7/963: creat d9/d5d/dc3/f14b x:0 0 0 2024-02-16T19:29:24.907 INFO:tasks.workunit.client.0.smithi022.stdout:7/964: chown d9/da/d1f/d40/dfb 12962399 1 2024-02-16T19:29:24.909 INFO:tasks.workunit.client.0.smithi022.stdout:7/965: write d9/da/d2c/d46/f12a [558429,11997] 0 2024-02-16T19:29:24.909 INFO:tasks.workunit.client.0.smithi022.stdout:3/880: mkdir d7/d12/d18/dd/d24/d93/de9/dc4/d133 0 2024-02-16T19:29:24.910 INFO:tasks.workunit.client.0.smithi022.stdout:3/881: readlink l3 0 2024-02-16T19:29:24.910 INFO:tasks.workunit.client.0.smithi022.stdout:7/966: chown d9/da/d1f/d40/fa5 807116621 1 2024-02-16T19:29:24.912 INFO:tasks.workunit.client.0.smithi022.stdout:8/991: truncate d3/d12/d2b/da9/fe4 4252395 0 2024-02-16T19:29:24.913 INFO:tasks.workunit.client.0.smithi022.stdout:7/967: dread d9/da/d1f/d44/d5b/d95/fbf [0,4194304] 0 2024-02-16T19:29:24.914 INFO:tasks.workunit.client.0.smithi022.stdout:8/992: chown d3/d12/d2b/d2e/c90 955223853 1 2024-02-16T19:29:24.920 INFO:tasks.workunit.client.0.smithi022.stdout:7/968: dwrite d9/da/d1f/d40/dde/fe7 [0,4194304] 0 2024-02-16T19:29:24.925 INFO:tasks.workunit.client.1.smithi117.stdout:6/983: sync 2024-02-16T19:29:24.925 INFO:tasks.workunit.client.1.smithi117.stdout:7/975: creat d4/d1c/d112/f166 x:0 0 0 2024-02-16T19:29:24.927 INFO:tasks.workunit.client.0.smithi022.stdout:7/969: dwrite d9/da/d1f/fcd [0,4194304] 0 2024-02-16T19:29:24.928 INFO:tasks.workunit.client.1.smithi117.stdout:6/984: write d5/d4b/d20/de7/f10d [29625,76980] 0 2024-02-16T19:29:24.929 INFO:tasks.workunit.client.0.smithi022.stdout:8/993: link d3/d12/f51 d3/d12/d17/d64/f155 0 2024-02-16T19:29:24.929 INFO:tasks.workunit.client.1.smithi117.stdout:0/981: chown d0/d9/l81 10 1 2024-02-16T19:29:24.932 INFO:tasks.workunit.client.1.smithi117.stdout:1/884: truncate d2/d4/dbb/d7f/f84 7523966 0 2024-02-16T19:29:24.932 INFO:tasks.workunit.client.0.smithi022.stdout:8/994: rmdir d3/d12/d2b/d54/db8/de5 39 2024-02-16T19:29:24.935 INFO:tasks.workunit.client.0.smithi022.stdout:7/970: mkdir d9/da/d14c 0 2024-02-16T19:29:24.940 INFO:tasks.workunit.client.1.smithi117.stdout:0/982: truncate d0/d9/d2c/da9/fe5 679176 0 2024-02-16T19:29:24.941 INFO:tasks.workunit.client.0.smithi022.stdout:8/995: unlink d3/c1d 0 2024-02-16T19:29:24.941 INFO:tasks.workunit.client.0.smithi022.stdout:3/882: dread d7/d12/d18/dd/fcb [0,4194304] 0 2024-02-16T19:29:24.942 INFO:tasks.workunit.client.1.smithi117.stdout:9/918: truncate dd/d47/d48/dc0/fe7 599189 0 2024-02-16T19:29:24.943 INFO:tasks.workunit.client.1.smithi117.stdout:9/919: stat dd/d45/d31/d63/d8e/l114 0 2024-02-16T19:29:24.944 INFO:tasks.workunit.client.1.smithi117.stdout:9/920: chown dd/d45/d31/d63/dba/df0/daf/df4/fea 851226 1 2024-02-16T19:29:24.948 INFO:tasks.workunit.client.1.smithi117.stdout:9/921: dread dd/d47/d5b/d5f/fc5 [0,4194304] 0 2024-02-16T19:29:24.950 INFO:tasks.workunit.client.1.smithi117.stdout:1/885: truncate d2/f57 404916 0 2024-02-16T19:29:24.954 INFO:tasks.workunit.client.0.smithi022.stdout:8/996: rename d3/d12/d2b/d2e/l6f to d3/d12/d2b/d54/d74/l156 0 2024-02-16T19:29:24.956 INFO:tasks.workunit.client.1.smithi117.stdout:1/886: dwrite d2/d19/d59/d89/d69/f115 [4194304,4194304] 0 2024-02-16T19:29:24.957 INFO:tasks.workunit.client.1.smithi117.stdout:9/922: mknod dd/d47/d48/dc0/c139 0 2024-02-16T19:29:24.964 INFO:tasks.workunit.client.0.smithi022.stdout:8/997: symlink d3/d12/d17/d10d/d147/l157 0 2024-02-16T19:29:24.966 INFO:tasks.workunit.client.0.smithi022.stdout:3/883: rename d7/d12/d18/dd/d24/d53/d116/f89 to d7/d12/f134 0 2024-02-16T19:29:24.967 INFO:tasks.workunit.client.1.smithi117.stdout:7/976: write d4/d1c/d112/f125 [168025,57009] 0 2024-02-16T19:29:24.969 INFO:tasks.workunit.client.0.smithi022.stdout:8/998: creat d3/d12/d2b/da9/f158 x:0 0 0 2024-02-16T19:29:24.972 INFO:tasks.workunit.client.0.smithi022.stdout:3/884: dwrite d7/f118 [0,4194304] 0 2024-02-16T19:29:24.974 INFO:tasks.workunit.client.1.smithi117.stdout:7/977: dwrite d4/d13/d63/ddd/f121 [0,4194304] 0 2024-02-16T19:29:24.974 INFO:tasks.workunit.client.1.smithi117.stdout:0/983: getdents d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/d27/d83/d7c 0 2024-02-16T19:29:24.975 INFO:tasks.workunit.client.1.smithi117.stdout:0/984: chown d0/d9/d14/d23/d33/cf4 0 1 2024-02-16T19:29:24.981 INFO:tasks.workunit.client.1.smithi117.stdout:6/985: link d5/c17 d5/d9c/daa/d85/db0/c143 0 2024-02-16T19:29:24.982 INFO:tasks.workunit.client.1.smithi117.stdout:6/986: write d5/db/d7d/d32/d37/fa1 [170902,75725] 0 2024-02-16T19:29:24.982 INFO:tasks.workunit.client.1.smithi117.stdout:0/985: truncate d0/d78/d135/fd2 24768 0 2024-02-16T19:29:24.983 INFO:tasks.workunit.client.1.smithi117.stdout:1/887: link d2/d4/dbb/d7f/l122 d2/d4/l131 0 2024-02-16T19:29:24.995 INFO:tasks.workunit.client.1.smithi117.stdout:7/978: dread d4/d1c/fbe [0,4194304] 0 2024-02-16T19:29:25.026 INFO:tasks.workunit.client.0.smithi022.stdout:3/885: write d7/d21/f5d [4980569,93229] 0 2024-02-16T19:29:25.027 INFO:tasks.workunit.client.0.smithi022.stdout:8/999: truncate d3/d12/d17/f9d 2967557 0 2024-02-16T19:29:25.027 INFO:tasks.workunit.client.1.smithi117.stdout:6/987: read d5/fa [3099750,99689] 0 2024-02-16T19:29:25.031 INFO:tasks.workunit.client.1.smithi117.stdout:0/986: rmdir d0/d9 39 2024-02-16T19:29:25.031 INFO:tasks.workunit.client.1.smithi117.stdout:7/979: mknod d4/d1c/d39/d46/d47/c167 0 2024-02-16T19:29:25.032 INFO:tasks.workunit.client.0.smithi022.stdout:3/886: readlink d7/d12/d18/d4b/lbc 0 2024-02-16T19:29:25.032 INFO:tasks.workunit.client.1.smithi117.stdout:7/980: write d4/d85/fcd [3236334,28089] 0 2024-02-16T19:29:25.036 INFO:tasks.workunit.client.1.smithi117.stdout:9/923: dread dd/d45/d31/d63/d121/d9a/fec [0,4194304] 0 2024-02-16T19:29:25.037 INFO:tasks.workunit.client.1.smithi117.stdout:9/924: dread - dd/d45/d73/f88 zero size 2024-02-16T19:29:25.041 INFO:tasks.workunit.client.0.smithi022.stdout:3/887: symlink d7/d12/d18/d4b/db6/l135 0 2024-02-16T19:29:25.042 INFO:tasks.workunit.client.1.smithi117.stdout:7/981: mknod d4/d1c/d112/c168 0 2024-02-16T19:29:25.049 INFO:tasks.workunit.client.1.smithi117.stdout:0/987: rename d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/d27 to d0/d9/d14/d1d/d60/dc3/d109/d14c 0 2024-02-16T19:29:25.052 INFO:tasks.workunit.client.1.smithi117.stdout:7/982: rmdir d4/d1c/d87/d89/d5f/dae/df3 39 2024-02-16T19:29:25.053 INFO:tasks.workunit.client.0.smithi022.stdout:3/888: creat d7/d12/d18/dd/d3c/d5a/f136 x:0 0 0 2024-02-16T19:29:25.055 INFO:tasks.workunit.client.0.smithi022.stdout:3/889: write d7/d12/d18/dd/d98/f110 [301279,8049] 0 2024-02-16T19:29:25.057 INFO:tasks.workunit.client.0.smithi022.stdout:3/890: rmdir d7/d12/d18/dd/d24/d53/d116 39 2024-02-16T19:29:25.058 INFO:tasks.workunit.client.1.smithi117.stdout:0/988: rmdir d0/d9/d14/d1d/d60/d76/d7b/d8d/d5b/d102/db7 39 2024-02-16T19:29:25.058 INFO:tasks.workunit.client.1.smithi117.stdout:9/925: write dd/d45/f4f [1225274,49231] 0 2024-02-16T19:29:25.059 INFO:tasks.workunit.client.1.smithi117.stdout:6/988: getdents d5/d4b/d20/d2d 0 2024-02-16T19:29:25.060 INFO:tasks.workunit.client.0.smithi022.stdout:3/891: chown d7/l11 48369 1 2024-02-16T19:29:25.060 INFO:tasks.workunit.client.1.smithi117.stdout:9/926: readlink dd/d45/d31/d63/dba/df0/daf/lc3 0 2024-02-16T19:29:25.060 INFO:tasks.workunit.client.1.smithi117.stdout:7/983: mknod d4/d1c/d39/d46/da4/d142/c169 0 2024-02-16T19:29:25.061 INFO:tasks.workunit.client.0.smithi022.stdout:3/892: write d7/d12/d18/f42 [3111095,129638] 0 2024-02-16T19:29:25.062 INFO:tasks.workunit.client.1.smithi117.stdout:1/888: link d2/d19/d59/d89/d69/dad/d100/f11d d2/d4/dbb/f132 0 2024-02-16T19:29:25.063 INFO:tasks.workunit.client.1.smithi117.stdout:0/989: read d0/d9/d14/d1d/d60/dc3/d109/d14c/d83/f95 [259102,110867] 0 2024-02-16T19:29:25.064 INFO:tasks.workunit.client.0.smithi022.stdout:3/893: symlink d7/d12/d18/dd/d3c/d11d/l137 0 2024-02-16T19:29:25.064 INFO:tasks.workunit.client.0.smithi022.stdout:3/894: fdatasync d7/d12/d18/d4b/f4d 0 2024-02-16T19:29:25.066 INFO:tasks.workunit.client.1.smithi117.stdout:1/889: creat d2/d4/dbb/d7f/f133 x:0 0 0 2024-02-16T19:29:25.067 INFO:tasks.workunit.client.1.smithi117.stdout:1/890: chown d2/d19/d59/d89/d69/d9a 631239 1 2024-02-16T19:29:25.070 INFO:tasks.workunit.client.1.smithi117.stdout:0/990: dread - d0/d9/d14/d1d/d60/dc3/dfd/f10b zero size 2024-02-16T19:29:25.071 INFO:tasks.workunit.client.0.smithi022.stdout:3/895: mknod d7/d12/d18/dd/d24/d53/d116/de6/d11a/c138 0 2024-02-16T19:29:25.075 INFO:tasks.workunit.client.1.smithi117.stdout:6/989: rmdir d5/d4b/d71/dc7/dce/ded/df3/d135 0 2024-02-16T19:29:25.076 INFO:tasks.workunit.client.1.smithi117.stdout:6/990: chown d5/db/d7d/d32/l8a 0 1 2024-02-16T19:29:25.077 INFO:tasks.workunit.client.1.smithi117.stdout:1/891: mknod d2/d1d/d118/c134 0 2024-02-16T19:29:25.078 INFO:tasks.workunit.client.1.smithi117.stdout:9/927: truncate dd/d47/d48/dc0/fbf 2213693 0 2024-02-16T19:29:25.078 INFO:tasks.workunit.client.1.smithi117.stdout:7/984: dread d4/f34 [4194304,4194304] 0 2024-02-16T19:29:25.080 INFO:tasks.workunit.client.1.smithi117.stdout:6/991: rmdir d5/d4b/d20/d2d 39 2024-02-16T19:29:25.080 INFO:tasks.workunit.client.1.smithi117.stdout:0/991: write d0/f1a [2257882,37031] 0 2024-02-16T19:29:25.080 INFO:tasks.workunit.client.0.smithi022.stdout:3/896: link d7/d12/d18/dd/d24/f9f d7/d12/d18/dd/d31/dbf/dc5/f139 0 2024-02-16T19:29:25.082 INFO:tasks.workunit.client.1.smithi117.stdout:1/892: chown d2/d4/dbb/d7f/ce8 672988 1 2024-02-16T19:29:25.082 INFO:tasks.workunit.client.0.smithi022.stdout:3/897: write d7/d12/d18/dd/d24/d58/d70/f7b [3099329,119790] 0 2024-02-16T19:29:25.083 INFO:tasks.workunit.client.1.smithi117.stdout:1/893: write d2/d19/d59/d76/d91/f11a [347947,8505] 0 2024-02-16T19:29:25.084 INFO:tasks.workunit.client.1.smithi117.stdout:1/894: chown d2/d19/d59/d89/d69 2135468 1 2024-02-16T19:29:25.085 INFO:tasks.workunit.client.1.smithi117.stdout:7/985: read d4/d1c/d39/d46/d55/db8/fd5 [156884,129608] 0 2024-02-16T19:29:25.086 INFO:tasks.workunit.client.1.smithi117.stdout:7/986: chown d4/d1c/d87/d89/d60/c67 99 1 2024-02-16T19:29:25.090 INFO:tasks.workunit.client.0.smithi022.stdout:3/898: dwrite d7/d12/d18/dd/d24/fd0 [0,4194304] 0 2024-02-16T19:29:25.090 INFO:tasks.workunit.client.1.smithi117.stdout:9/928: write dd/db5/fa0 [2409007,9949] 0 2024-02-16T19:29:25.092 INFO:tasks.workunit.client.1.smithi117.stdout:7/987: rmdir d4/d13 39 2024-02-16T19:29:25.093 INFO:tasks.workunit.client.1.smithi117.stdout:0/992: dread d0/d9/d14/f92 [0,4194304] 0 2024-02-16T19:29:25.094 INFO:tasks.workunit.client.0.smithi022.stdout:3/899: creat d7/d21/f13a x:0 0 0 2024-02-16T19:29:25.096 INFO:tasks.workunit.client.1.smithi117.stdout:1/895: rename d2/d19/d59/d76/d91/f11a to d2/d19/d59/d89/f135 0 2024-02-16T19:29:25.098 INFO:tasks.workunit.client.1.smithi117.stdout:0/993: creat d0/d9/d14/d1d/d60/dc3/d109/f14d x:0 0 0 2024-02-16T19:29:25.098 INFO:tasks.workunit.client.1.smithi117.stdout:6/992: dwrite d5/db/d7d/f2b [0,4194304] 0 2024-02-16T19:29:25.123 INFO:tasks.workunit.client.0.smithi022.stdout:3/900: dread d7/d12/d18/dd/d24/d93/de9/f60 [0,4194304] 0 2024-02-16T19:29:25.129 INFO:tasks.workunit.client.0.smithi022.stdout:3/901: dwrite d7/d12/d18/dd/d24/de5/f11f [0,4194304] 0 2024-02-16T19:29:25.159 INFO:tasks.workunit.client.1.smithi117.stdout:7/988: mkdir d4/d13/d63/ddd/d117/d11a/d16a 0 2024-02-16T19:29:25.163 INFO:tasks.workunit.client.1.smithi117.stdout:6/993: creat d5/d4b/d20/db3/f144 x:0 0 0 2024-02-16T19:29:25.219 INFO:tasks.workunit.client.1.smithi117.stdout:0/994: rmdir d0/d9/d14/d1d/d60/dc3/d109/d14c/d83/d7c 39 2024-02-16T19:29:25.224 INFO:tasks.workunit.client.1.smithi117.stdout:6/994: creat d5/db/d7d/d32/da8/f145 x:0 0 0 2024-02-16T19:29:25.226 INFO:tasks.workunit.client.1.smithi117.stdout:0/995: unlink d0/fe 0 2024-02-16T19:29:25.227 INFO:tasks.workunit.client.1.smithi117.stdout:7/989: mkdir d4/d1c/d16b 0 2024-02-16T19:29:25.228 INFO:tasks.workunit.client.1.smithi117.stdout:6/995: readlink d5/db/l45 0 2024-02-16T19:29:25.228 INFO:tasks.workunit.client.1.smithi117.stdout:6/996: chown d5/db/c8e 2143935772 1 2024-02-16T19:29:25.231 INFO:tasks.workunit.client.1.smithi117.stdout:9/929: rename dd/d45/d31/d63/dba/df0/l36 to dd/d45/l13a 0 2024-02-16T19:29:25.233 INFO:tasks.workunit.client.1.smithi117.stdout:1/896: dwrite d2/d1d/f4f [0,4194304] 0 2024-02-16T19:29:25.233 INFO:tasks.workunit.client.1.smithi117.stdout:0/996: link d0/f1a d0/d9/d2c/d8b/f14e 0 2024-02-16T19:29:25.236 INFO:tasks.workunit.client.1.smithi117.stdout:1/897: creat d2/d4/dbb/d10b/dcc/f136 x:0 0 0 2024-02-16T19:29:25.238 INFO:tasks.workunit.client.1.smithi117.stdout:1/898: truncate d2/d19/d59/d89/d7b/fd9 1935146 0 2024-02-16T19:29:25.238 INFO:tasks.workunit.client.0.smithi022.stdout:3/902: dwrite d7/d12/d18/dd/d24/d93/de9/f71 [0,4194304] 0 2024-02-16T19:29:25.240 INFO:tasks.workunit.client.0.smithi022.stdout:3/903: chown d7/d12/d18/dd/d31/f47 25 1 2024-02-16T19:29:25.241 INFO:tasks.workunit.client.0.smithi022.stdout:3/904: fdatasync d7/d21/f96 0 2024-02-16T19:29:25.242 INFO:tasks.workunit.client.1.smithi117.stdout:6/997: rename d5/d9c/daa/d85/db0/f12f to d5/d4b/d20/db3/f146 0 2024-02-16T19:29:25.243 INFO:tasks.workunit.client.1.smithi117.stdout:7/990: write d4/d13/da3/fdc [562531,117768] 0 2024-02-16T19:29:25.244 INFO:tasks.workunit.client.1.smithi117.stdout:6/998: dread - d5/d4b/d44/d52/d81/dd7/f126 zero size 2024-02-16T19:29:25.247 INFO:tasks.workunit.client.1.smithi117.stdout:0/997: creat d0/d9/d14/d1d/f14f x:0 0 0 2024-02-16T19:29:25.248 INFO:tasks.workunit.client.0.smithi022.stdout:3/905: creat d7/d12/d18/dd/d3c/d11d/d125/f13b x:0 0 0 2024-02-16T19:29:25.252 INFO:tasks.workunit.client.1.smithi117.stdout:1/899: dread d2/d19/d59/d89/d69/dad/fdb [0,4194304] 0 2024-02-16T19:29:25.252 INFO:tasks.workunit.client.1.smithi117.stdout:6/999: rename d5/db/d7d/d32/da8/f145 to d5/f147 0 2024-02-16T19:29:25.254 INFO:tasks.workunit.client.0.smithi022.stdout:3/906: link d7/d12/d18/f4e d7/d12/d18/dd/d24/d93/de9/def/f13c 0 2024-02-16T19:29:25.266 INFO:tasks.workunit.client.0.smithi022.stdout:9/878: sync 2024-02-16T19:29:25.269 INFO:tasks.workunit.client.0.smithi022.stdout:3/907: truncate d7/d21/f5d 2268605 0 2024-02-16T19:29:25.270 INFO:tasks.workunit.client.0.smithi022.stdout:9/879: dread db/d14/d19/fcf [0,4194304] 0 2024-02-16T19:29:25.271 INFO:tasks.workunit.client.0.smithi022.stdout:3/908: stat d7/d12/d18/dd/d98/c12e 0 2024-02-16T19:29:25.275 INFO:tasks.workunit.client.1.smithi117.stdout:7/991: rename d4/d1c/d87/d89/d60/d11f/fb2 to d4/d1c/d98/f16c 0 2024-02-16T19:29:25.279 INFO:tasks.workunit.client.0.smithi022.stdout:3/909: dwrite d7/d12/d18/dd/d3c/f65 [4194304,4194304] 0 2024-02-16T19:29:25.283 INFO:tasks.workunit.client.1.smithi117.stdout:7/992: write d4/d1c/d87/d89/d5f/dae/f105 [388114,127693] 0 2024-02-16T19:29:25.283 INFO:tasks.workunit.client.0.smithi022.stdout:3/910: mknod d7/d12/d18/dd/d24/d53/d116/de6/c13d 0 2024-02-16T19:29:25.284 INFO:tasks.workunit.client.0.smithi022.stdout:9/880: dwrite db/d14/d16/d5d/d65/d6b/fcb [0,4194304] 0 2024-02-16T19:29:25.287 INFO:tasks.workunit.client.1.smithi117.stdout:7/993: dread - d4/d1c/d39/d9f/de1/f123 zero size 2024-02-16T19:29:25.288 INFO:tasks.workunit.client.1.smithi117.stdout:0/998: dread d0/d9/d14/d1d/d60/dc3/d109/d14c/df9/f147 [0,4194304] 0 2024-02-16T19:29:25.289 INFO:tasks.workunit.client.1.smithi117.stdout:1/900: dwrite d2/d19/d59/d89/d69/dad/d100/f11d [0,4194304] 0 2024-02-16T19:29:25.290 INFO:tasks.workunit.client.1.smithi117.stdout:7/994: chown d4/d13/d63/d71/l7d 2 1 2024-02-16T19:29:25.290 INFO:tasks.workunit.client.0.smithi022.stdout:9/881: getdents db/d14/d16/d49/d90 0 2024-02-16T19:29:25.293 INFO:tasks.workunit.client.0.smithi022.stdout:3/911: dwrite d7/d12/d18/dac/f131 [0,4194304] 0 2024-02-16T19:29:25.294 INFO:tasks.workunit.client.0.smithi022.stdout:9/882: rmdir db/d14/d16/d5d/d65/d6b 39 2024-02-16T19:29:25.294 INFO:tasks.workunit.client.1.smithi117.stdout:1/901: rmdir d2/d1d/d60/d7c 39 2024-02-16T19:29:25.295 INFO:tasks.workunit.client.0.smithi022.stdout:9/883: stat db/d14/d16/dec/cef 0 2024-02-16T19:29:25.295 INFO:tasks.workunit.client.1.smithi117.stdout:1/902: chown d2/f4d 449 1 2024-02-16T19:29:25.296 INFO:tasks.workunit.client.1.smithi117.stdout:0/999: truncate d0/d9/d14/d1d/d60/d65/f103 1698024 0 2024-02-16T19:29:25.298 INFO:tasks.workunit.client.0.smithi022.stdout:2/878: sync 2024-02-16T19:29:25.298 INFO:tasks.workunit.client.0.smithi022.stdout:6/923: sync 2024-02-16T19:29:25.298 INFO:tasks.workunit.client.0.smithi022.stdout:5/931: sync 2024-02-16T19:29:25.301 INFO:tasks.workunit.client.0.smithi022.stdout:3/912: rename d7/d12/d18/d4b/f4d to d7/d21/dd9/d11b/f13e 0 2024-02-16T19:29:25.302 INFO:tasks.workunit.client.1.smithi117.stdout:1/903: mknod d2/d1d/d77/c137 0 2024-02-16T19:29:25.306 INFO:tasks.workunit.client.1.smithi117.stdout:7/995: link d4/f118 d4/d13/f16d 0 2024-02-16T19:29:25.307 INFO:tasks.workunit.client.1.smithi117.stdout:7/996: chown d4/d13/dfd/c114 146719 1 2024-02-16T19:29:25.308 INFO:tasks.workunit.client.0.smithi022.stdout:5/932: fsync dd/f1d 0 2024-02-16T19:29:25.309 INFO:tasks.workunit.client.1.smithi117.stdout:1/904: truncate d2/d4/dbb/d7f/f94 2450740 0 2024-02-16T19:29:25.309 INFO:tasks.workunit.client.0.smithi022.stdout:2/879: rmdir d0/d2/d18/d1f/d6c/df1 39 2024-02-16T19:29:25.311 INFO:tasks.workunit.client.0.smithi022.stdout:3/913: creat d7/d12/d18/d4b/db6/f13f x:0 0 0 2024-02-16T19:29:25.314 INFO:tasks.workunit.client.0.smithi022.stdout:5/933: creat dd/d1f/d2e/f12e x:0 0 0 2024-02-16T19:29:25.316 INFO:tasks.workunit.client.0.smithi022.stdout:3/914: readlink d7/d12/d18/dd/d24/d58/l72 0 2024-02-16T19:29:25.318 INFO:tasks.workunit.client.0.smithi022.stdout:9/884: creat db/d14/d16/d21/d37/f12b x:0 0 0 2024-02-16T19:29:25.321 INFO:tasks.workunit.client.0.smithi022.stdout:5/934: rename l2 to dd/d1f/d48/d21/d5d/dc0/l12f 0 2024-02-16T19:29:25.324 INFO:tasks.workunit.client.0.smithi022.stdout:3/915: mkdir d7/d12/d18/dd/d31/dbf/dc5/d140 0 2024-02-16T19:29:25.326 INFO:tasks.workunit.client.0.smithi022.stdout:6/924: write d1/fba [701328,91672] 0 2024-02-16T19:29:25.332 INFO:tasks.workunit.client.0.smithi022.stdout:5/935: rename dd/d1f/d48/d21/d2f/d40/d49/f4c to dd/d1f/d48/d21/d64/f130 0 2024-02-16T19:29:25.332 INFO:tasks.workunit.client.1.smithi117.stdout:7/997: write d4/d1c/d87/df0/dfe/f116 [99458,70512] 0 2024-02-16T19:29:25.335 INFO:tasks.workunit.client.0.smithi022.stdout:6/925: dwrite d1/d8/da/d29/deb/dfd/d85/d9c/d12a/f136 [0,4194304] 0 2024-02-16T19:29:25.338 INFO:tasks.workunit.client.1.smithi117.stdout:1/905: dwrite d2/f4d [4194304,4194304] 0 2024-02-16T19:29:25.341 INFO:tasks.workunit.client.0.smithi022.stdout:2/880: dwrite d0/f37 [0,4194304] 0 2024-02-16T19:29:25.342 INFO:tasks.workunit.client.0.smithi022.stdout:2/881: stat d0/d2/d18/c64 0 2024-02-16T19:29:25.343 INFO:tasks.workunit.client.0.smithi022.stdout:2/882: chown d0/d5d/f122 248728368 1 2024-02-16T19:29:25.344 INFO:tasks.workunit.client.0.smithi022.stdout:9/885: write db/d18/f73 [5901982,100235] 0 2024-02-16T19:29:25.345 INFO:tasks.workunit.client.0.smithi022.stdout:2/883: chown d0/d2/d18/d19/d40/d55/d59 45154 1 2024-02-16T19:29:25.346 INFO:tasks.workunit.client.1.smithi117.stdout:1/906: rmdir d2/d19/d59/d89/d69/dad/d100 39 2024-02-16T19:29:25.347 INFO:tasks.workunit.client.0.smithi022.stdout:3/916: write d7/d12/f19 [2769269,22103] 0 2024-02-16T19:29:25.348 INFO:tasks.workunit.client.1.smithi117.stdout:7/998: link d4/d1c/d39/d9f/fd0 d4/d1c/d87/d89/d60/d79/dc3/d11d/f16e 0 2024-02-16T19:29:25.350 INFO:tasks.workunit.client.0.smithi022.stdout:9/886: chown db/d14/d16/d5d/d65/d6b 83973394 1 2024-02-16T19:29:25.356 INFO:tasks.workunit.client.1.smithi117.stdout:1/907: mkdir d2/d19/d59/d89/d7b/d86/d138 0 2024-02-16T19:29:25.356 INFO:tasks.workunit.client.1.smithi117.stdout:9/930: sync 2024-02-16T19:29:25.361 INFO:tasks.workunit.client.0.smithi022.stdout:9/887: link db/c3f db/d14/d19/d1a/c12c 0 2024-02-16T19:29:25.363 INFO:tasks.workunit.client.1.smithi117.stdout:1/908: symlink d2/d1d/d60/d7c/d9e/da5/l139 0 2024-02-16T19:29:25.364 INFO:tasks.workunit.client.0.smithi022.stdout:9/888: dread - db/d14/ff4 zero size 2024-02-16T19:29:25.365 INFO:tasks.workunit.client.0.smithi022.stdout:5/936: dread dd/d1f/d48/d21/d2f/ff8 [0,4194304] 0 2024-02-16T19:29:25.366 INFO:tasks.workunit.client.1.smithi117.stdout:9/931: getdents dd/d47/d48 0 2024-02-16T19:29:25.368 INFO:tasks.workunit.client.0.smithi022.stdout:5/937: unlink dd/f1d 0 2024-02-16T19:29:25.370 INFO:tasks.workunit.client.0.smithi022.stdout:5/938: mkdir dd/d1f/d2e/dd4/d131 0 2024-02-16T19:29:25.371 INFO:tasks.workunit.client.1.smithi117.stdout:9/932: creat dd/f13b x:0 0 0 2024-02-16T19:29:25.371 INFO:tasks.workunit.client.1.smithi117.stdout:9/933: chown dd/ff5 93576 1 2024-02-16T19:29:25.372 INFO:tasks.workunit.client.0.smithi022.stdout:5/939: read dd/f124 [1289448,23321] 0 2024-02-16T19:29:25.377 INFO:tasks.workunit.client.0.smithi022.stdout:2/884: truncate d0/d2/d2e/fba 5761164 0 2024-02-16T19:29:25.378 INFO:tasks.workunit.client.0.smithi022.stdout:5/940: dread dd/d1f/d48/d21/d2f/d40/d4e/f6d [0,4194304] 0 2024-02-16T19:29:25.379 INFO:tasks.workunit.client.1.smithi117.stdout:7/999: dwrite d4/d1c/d87/d89/d5f/dae/f158 [0,4194304] 0 2024-02-16T19:29:25.380 INFO:tasks.workunit.client.0.smithi022.stdout:3/917: dwrite d7/d12/d18/dd/d24/d53/d116/fe1 [0,4194304] 0 2024-02-16T19:29:25.381 INFO:tasks.workunit.client.1.smithi117.stdout:1/909: write d2/d19/d59/d89/d49/fcd [226389,115817] 0 2024-02-16T19:29:25.383 INFO:tasks.workunit.client.1.smithi117.stdout:1/910: write d2/d4/dbb/d10b/dcc/f136 [620446,73773] 0 2024-02-16T19:29:25.386 INFO:tasks.workunit.client.0.smithi022.stdout:3/918: mkdir d7/d12/d18/dd/d24/d93/de9/dc4/d141 0 2024-02-16T19:29:25.390 INFO:tasks.workunit.client.0.smithi022.stdout:5/941: rename dd/d1f/d48/d21/d2f/d52/d5c/f63 to dd/d1f/d48/d21/d2f/d40/d49/d4b/db5/f132 0 2024-02-16T19:29:25.390 INFO:tasks.workunit.client.0.smithi022.stdout:9/889: dwrite db/d14/d16/d49/fe7 [0,4194304] 0 2024-02-16T19:29:25.392 INFO:tasks.workunit.client.1.smithi117.stdout:1/911: creat d2/d1d/f13a x:0 0 0 2024-02-16T19:29:25.397 INFO:tasks.workunit.client.1.smithi117.stdout:1/912: read d2/d19/d59/d89/d49/f4a [2285928,86949] 0 2024-02-16T19:29:25.398 INFO:tasks.workunit.client.0.smithi022.stdout:5/942: creat dd/d1f/d48/d21/d64/d11d/d6a/dfb/f133 x:0 0 0 2024-02-16T19:29:25.400 INFO:tasks.workunit.client.1.smithi117.stdout:9/934: dwrite dd/d45/d31/d63/dba/df0/f95 [0,4194304] 0 2024-02-16T19:29:25.402 INFO:tasks.workunit.client.0.smithi022.stdout:5/943: mknod dd/d1f/d48/d21/d2f/d40/c134 0 2024-02-16T19:29:25.402 INFO:tasks.workunit.client.0.smithi022.stdout:3/919: truncate d7/d12/d18/dd/d24/d29/fc3 151824 0 2024-02-16T19:29:25.404 INFO:tasks.workunit.client.0.smithi022.stdout:5/944: write dd/d1f/d48/d21/d2f/d46/d5e/f7b [4796272,67683] 0 2024-02-16T19:29:25.404 INFO:tasks.workunit.client.1.smithi117.stdout:9/935: symlink dd/d45/d31/d63/dba/df0/daf/l13c 0 2024-02-16T19:29:25.404 INFO:tasks.workunit.client.1.smithi117.stdout:9/936: stat l8 0 2024-02-16T19:29:25.406 INFO:tasks.workunit.client.1.smithi117.stdout:9/937: dread dd/f4a [0,4194304] 0 2024-02-16T19:29:25.408 INFO:tasks.workunit.client.1.smithi117.stdout:1/913: rename d2/d1d/d60/df2/c114 to d2/d19/d59/c13b 0 2024-02-16T19:29:25.409 INFO:tasks.workunit.client.0.smithi022.stdout:5/945: chown dd/d1f/d48/d21/d2f/d46/d5e/d84/c115 589237 1 2024-02-16T19:29:25.414 INFO:tasks.workunit.client.0.smithi022.stdout:3/920: rename d7/d12/d18/dd/d24/cb2 to d7/c142 0 2024-02-16T19:29:25.417 INFO:tasks.workunit.client.0.smithi022.stdout:5/946: creat dd/d1f/d48/d21/d64/d11d/f135 x:0 0 0 2024-02-16T19:29:25.418 INFO:tasks.workunit.client.1.smithi117.stdout:1/914: getdents d2/d1d/d1f/d9c 0 2024-02-16T19:29:25.420 INFO:tasks.workunit.client.0.smithi022.stdout:5/947: stat dd/d1f/d48/d21/d2f/d46/d5e/d84/db4/fb9 0 2024-02-16T19:29:25.420 INFO:tasks.workunit.client.0.smithi022.stdout:3/921: creat d7/d12/d18/dd/d24/d58/d70/f143 x:0 0 0 2024-02-16T19:29:25.421 INFO:tasks.workunit.client.0.smithi022.stdout:2/885: dwrite d0/d2/d18/d19/fc1 [0,4194304] 0 2024-02-16T19:29:25.422 INFO:tasks.workunit.client.1.smithi117.stdout:1/915: creat d2/d19/d59/d89/d69/d9a/f13c x:0 0 0 2024-02-16T19:29:25.426 INFO:tasks.workunit.client.0.smithi022.stdout:5/948: rmdir dd/d1f/d48/d21/d2f/d46/d5e/d84/d8b 39 2024-02-16T19:29:25.426 INFO:tasks.workunit.client.0.smithi022.stdout:2/886: dread d0/d2/d2e/d82/df0/f76 [0,4194304] 0 2024-02-16T19:29:25.426 INFO:tasks.workunit.client.0.smithi022.stdout:9/890: write db/d14/d16/d21/d37/f4e [2447251,67635] 0 2024-02-16T19:29:25.429 INFO:tasks.workunit.client.0.smithi022.stdout:9/891: write db/d14/f5e [6973997,119651] 0 2024-02-16T19:29:25.430 INFO:tasks.workunit.client.0.smithi022.stdout:3/922: dread d7/d21/fbd [0,4194304] 0 2024-02-16T19:29:25.431 INFO:tasks.workunit.client.1.smithi117.stdout:1/916: creat d2/d1d/d60/d7c/d9e/da5/db0/dba/df3/f13d x:0 0 0 2024-02-16T19:29:25.432 INFO:tasks.workunit.client.0.smithi022.stdout:2/887: mkdir d0/d5d/dc4/d125 0 2024-02-16T19:29:25.435 INFO:tasks.workunit.client.1.smithi117.stdout:1/917: read - d2/f9d zero size 2024-02-16T19:29:25.435 INFO:tasks.workunit.client.1.smithi117.stdout:9/938: write dd/d45/d31/d63/d8e/f96 [529030,48458] 0 2024-02-16T19:29:25.436 INFO:tasks.workunit.client.1.smithi117.stdout:9/939: chown dd/f4a 86 1 2024-02-16T19:29:25.440 INFO:tasks.workunit.client.0.smithi022.stdout:5/949: fsync dd/d1f/d48/d21/d5d/f112 0 2024-02-16T19:29:25.440 INFO:tasks.workunit.client.0.smithi022.stdout:5/950: chown dd/d1f/d48/d21/d64/f116 106945733 1 2024-02-16T19:29:25.440 INFO:tasks.workunit.client.1.smithi117.stdout:1/918: fsync d2/d19/d59/d89/df4/dfc/f75 0 2024-02-16T19:29:25.441 INFO:tasks.workunit.client.0.smithi022.stdout:9/892: symlink db/d14/d16/d28/daa/l12d 0 2024-02-16T19:29:25.442 INFO:tasks.workunit.client.1.smithi117.stdout:9/940: symlink dd/d45/d31/d63/dba/d11e/l13d 0 2024-02-16T19:29:25.443 INFO:tasks.workunit.client.0.smithi022.stdout:2/888: rmdir d0/d2/d18/d19/d90/d100 39 2024-02-16T19:29:25.446 INFO:tasks.workunit.client.0.smithi022.stdout:1/954: sync 2024-02-16T19:29:25.446 INFO:tasks.workunit.client.0.smithi022.stdout:7/971: sync 2024-02-16T19:29:25.446 INFO:tasks.workunit.client.0.smithi022.stdout:4/866: sync 2024-02-16T19:29:25.450 INFO:tasks.workunit.client.0.smithi022.stdout:2/889: dwrite d0/d2/d18/d1f/d6c/d99/fdd [4194304,4194304] 0 2024-02-16T19:29:25.453 INFO:tasks.workunit.client.1.smithi117.stdout:9/941: link dd/d45/d73/ff7 dd/d45/d31/d63/dba/df0/daf/df4/d10d/f13e 0 2024-02-16T19:29:25.455 INFO:tasks.workunit.client.1.smithi117.stdout:1/919: creat d2/d4/f13e x:0 0 0 2024-02-16T19:29:25.456 INFO:tasks.workunit.client.1.smithi117.stdout:1/920: write d2/d19/d59/d89/d49/ffb [1244993,100314] 0 2024-02-16T19:29:25.456 INFO:tasks.workunit.client.1.smithi117.stdout:9/942: creat dd/d45/d73/f13f x:0 0 0 2024-02-16T19:29:25.460 INFO:tasks.workunit.client.1.smithi117.stdout:9/943: truncate dd/d45/d31/d63/dba/df0/f25 2603792 0 2024-02-16T19:29:25.480 INFO:tasks.workunit.client.1.smithi117.stdout:1/921: dwrite d2/d19/d59/d89/d8d/f117 [0,4194304] 0 2024-02-16T19:29:25.492 INFO:tasks.workunit.client.0.smithi022.stdout:5/951: dwrite dd/d1f/d48/f60 [0,4194304] 0 2024-02-16T19:29:25.492 INFO:tasks.workunit.client.0.smithi022.stdout:1/955: mkdir d6/d8/d22/d16/d17/d63/dee/d124 0 2024-02-16T19:29:25.497 INFO:tasks.workunit.client.0.smithi022.stdout:2/890: unlink d0/d2/d18/d38/ld4 0 2024-02-16T19:29:25.500 INFO:tasks.workunit.client.0.smithi022.stdout:4/867: symlink d4/d6/dd/df/l115 0 2024-02-16T19:29:25.506 INFO:tasks.workunit.client.0.smithi022.stdout:5/952: rename dd/d1f/d48/d21/d2f/d46 to dd/d1f/d48/d21/d64/d11d/d10a/d136 0 2024-02-16T19:29:25.508 INFO:tasks.workunit.client.0.smithi022.stdout:5/953: write dd/d1f/d48/d21/d2f/d40/d49/d4b/f7e [678303,45852] 0 2024-02-16T19:29:25.509 INFO:tasks.workunit.client.0.smithi022.stdout:4/868: symlink d4/d6/dd/d22/d83/dc5/l116 0 2024-02-16T19:29:25.510 INFO:tasks.workunit.client.0.smithi022.stdout:1/956: truncate d6/f8e 1134789 0 2024-02-16T19:29:25.511 INFO:tasks.workunit.client.0.smithi022.stdout:2/891: getdents d0/d2/dde 0 2024-02-16T19:29:25.514 INFO:tasks.workunit.client.0.smithi022.stdout:5/954: creat dd/d1f/d48/d21/d64/f137 x:0 0 0 2024-02-16T19:29:25.518 INFO:tasks.workunit.client.0.smithi022.stdout:1/957: unlink d6/d8/d22/d3a/d3e/fa1 0 2024-02-16T19:29:25.519 INFO:tasks.workunit.client.0.smithi022.stdout:2/892: chown d0/d32/d3b/c7e 201663 1 2024-02-16T19:29:25.520 INFO:tasks.workunit.client.0.smithi022.stdout:5/955: creat dd/d1f/d48/d21/d2f/d40/d71/f138 x:0 0 0 2024-02-16T19:29:25.521 INFO:tasks.workunit.client.0.smithi022.stdout:5/956: dread - dd/d1f/d48/d21/d64/f137 zero size 2024-02-16T19:29:25.523 INFO:tasks.workunit.client.0.smithi022.stdout:5/957: write dd/d1f/d48/d21/d64/d11d/d10a/d136/d5e/d84/fa3 [5282196,70932] 0 2024-02-16T19:29:25.524 INFO:tasks.workunit.client.0.smithi022.stdout:2/893: unlink d0/d2/d2e/l103 0 2024-02-16T19:29:25.524 INFO:tasks.workunit.client.1.smithi117.stdout:1/922: dwrite d2/d1d/d1f/f56 [0,4194304] 0 2024-02-16T19:29:25.528 INFO:tasks.workunit.client.0.smithi022.stdout:9/893: truncate db/d18/f73 2116528 0 2024-02-16T19:29:25.530 INFO:tasks.workunit.client.0.smithi022.stdout:5/958: creat dd/d1f/d48/d21/d2f/d40/d49/d4b/d119/f139 x:0 0 0 2024-02-16T19:29:25.531 INFO:tasks.workunit.client.0.smithi022.stdout:7/972: truncate d9/da/d1f/d67/ddb/fd3 143944 0 2024-02-16T19:29:25.533 INFO:tasks.workunit.client.1.smithi117.stdout:1/923: mkdir d2/d13f 0 2024-02-16T19:29:25.534 INFO:tasks.workunit.client.1.smithi117.stdout:1/924: write d2/d19/d59/d89/d49/fcd [85570,54496] 0 2024-02-16T19:29:25.538 INFO:tasks.workunit.client.0.smithi022.stdout:5/959: creat dd/d1f/d48/d21/d2f/d52/d77/d95/ded/f13a x:0 0 0 2024-02-16T19:29:25.540 INFO:tasks.workunit.client.1.smithi117.stdout:1/925: dwrite d2/d4/f13e [0,4194304] 0 2024-02-16T19:29:25.543 INFO:tasks.workunit.client.0.smithi022.stdout:2/894: fdatasync d0/d2/d18/d1f/d6c/df1/fa9 0 2024-02-16T19:29:25.546 INFO:tasks.workunit.client.0.smithi022.stdout:1/958: write d6/d8/d22/d67/d7e/fa5 [746681,65339] 0 2024-02-16T19:29:25.546 INFO:tasks.workunit.client.0.smithi022.stdout:4/869: dwrite d4/d6/d14/d64/f34 [0,4194304] 0 2024-02-16T19:29:25.553 INFO:tasks.workunit.client.0.smithi022.stdout:9/894: rename db/c11 to db/d14/c12e 0 2024-02-16T19:29:25.555 INFO:tasks.workunit.client.1.smithi117.stdout:9/944: sync 2024-02-16T19:29:25.555 INFO:tasks.workunit.client.0.smithi022.stdout:2/895: dwrite d0/d2/d18/d38/db6/fe4 [0,4194304] 0 2024-02-16T19:29:25.558 INFO:tasks.workunit.client.0.smithi022.stdout:4/870: creat d4/d6/dd/d22/d83/d9e/db7/f117 x:0 0 0 2024-02-16T19:29:25.562 INFO:tasks.workunit.client.0.smithi022.stdout:2/896: stat d0/d5d/fb0 0 2024-02-16T19:29:25.566 INFO:tasks.workunit.client.0.smithi022.stdout:5/960: write dd/d1f/d48/d21/d2f/d52/d5c/fce [2983864,103325] 0 2024-02-16T19:29:25.569 INFO:tasks.workunit.client.0.smithi022.stdout:7/973: dwrite d9/da/d1f/d40/fb5 [0,4194304] 0 2024-02-16T19:29:25.571 INFO:tasks.workunit.client.0.smithi022.stdout:4/871: getdents d4/d6/d14/d64/d4e/d54/dba 0 2024-02-16T19:29:25.571 INFO:tasks.workunit.client.0.smithi022.stdout:5/961: read - dd/d1f/d48/d21/d64/d11d/d6a/dfb/f133 zero size 2024-02-16T19:29:25.572 INFO:tasks.workunit.client.0.smithi022.stdout:6/926: sync 2024-02-16T19:29:25.575 INFO:tasks.workunit.client.1.smithi117.stdout:9/945: dread dd/d45/d31/d63/dba/df0/f1f [0,4194304] 0 2024-02-16T19:29:25.576 INFO:tasks.workunit.client.0.smithi022.stdout:4/872: chown d4/d6/dd/f1e 9909944 1 2024-02-16T19:29:25.577 INFO:tasks.workunit.client.0.smithi022.stdout:1/959: dread d6/d8/d22/d47/d54/f8a [0,4194304] 0 2024-02-16T19:29:25.578 INFO:tasks.workunit.client.0.smithi022.stdout:6/927: creat d1/ddc/f13b x:0 0 0 2024-02-16T19:29:25.578 INFO:tasks.workunit.client.0.smithi022.stdout:5/962: fdatasync dd/d1f/d48/d21/d5d/dc0/fe9 0 2024-02-16T19:29:25.581 INFO:tasks.workunit.client.0.smithi022.stdout:4/873: mkdir d4/d6/dd/d22/d30/d118 0 2024-02-16T19:29:25.586 INFO:tasks.workunit.client.1.smithi117.stdout:1/926: dwrite d2/f9b [0,4194304] 0 2024-02-16T19:29:25.586 INFO:tasks.workunit.client.0.smithi022.stdout:2/897: dread d0/d2/d18/d1f/d6c/fa2 [0,4194304] 0 2024-02-16T19:29:25.587 INFO:tasks.workunit.client.0.smithi022.stdout:5/963: mknod dd/d1f/d48/d21/d2f/d40/d4e/d56/c13b 0 2024-02-16T19:29:25.588 INFO:tasks.workunit.client.1.smithi117.stdout:1/927: chown d2/d19/d59/d89/d69/d80/lf8 0 1 2024-02-16T19:29:25.590 INFO:tasks.workunit.client.0.smithi022.stdout:4/874: rename d4/d6/dd/df/d10/d55/d108/f10a to d4/d6/d3a/f119 0 2024-02-16T19:29:25.591 INFO:tasks.workunit.client.1.smithi117.stdout:1/928: rmdir d2/d1d/d60/d7c 39 2024-02-16T19:29:25.598 INFO:tasks.workunit.client.0.smithi022.stdout:2/898: symlink d0/d2/d2e/d82/df0/l126 0 2024-02-16T19:29:25.598 INFO:tasks.workunit.client.0.smithi022.stdout:4/875: dwrite d4/d6/dd/df/d10/d24/f8c [0,4194304] 0 2024-02-16T19:29:25.599 INFO:tasks.workunit.client.0.smithi022.stdout:5/964: unlink dd/d1f/d48/d21/d2f/fe2 0 2024-02-16T19:29:25.600 INFO:tasks.workunit.client.0.smithi022.stdout:7/974: write d9/da/d1f/d44/d5b/dfc/fe2 [908268,64292] 0 2024-02-16T19:29:25.601 INFO:tasks.workunit.client.1.smithi117.stdout:1/929: mkdir d2/d4/dbb/d10b/d140 0 2024-02-16T19:29:25.603 INFO:tasks.workunit.client.0.smithi022.stdout:6/928: write d1/d8/dac/d101/fd3 [3687300,17052] 0 2024-02-16T19:29:25.607 INFO:tasks.workunit.client.0.smithi022.stdout:9/895: unlink db/f12 0 2024-02-16T19:29:25.609 INFO:tasks.workunit.client.1.smithi117.stdout:1/930: dwrite d2/f53 [4194304,4194304] 0 2024-02-16T19:29:25.610 INFO:tasks.workunit.client.0.smithi022.stdout:1/960: write d6/d8/d22/d16/d17/d63/d6f/d8b/fa3 [1268851,92466] 0 2024-02-16T19:29:25.618 INFO:tasks.workunit.client.0.smithi022.stdout:9/896: write db/d14/d16/d5d/d65/d6b/dd5/f116 [3668593,46817] 0 2024-02-16T19:29:25.619 INFO:tasks.workunit.client.0.smithi022.stdout:1/961: rename d6/d8/d22/d3a/d80/f120 to d6/d8/dba/dc7/f125 0 2024-02-16T19:29:25.622 INFO:tasks.workunit.client.1.smithi117.stdout:1/931: rmdir d2/d4 39 2024-02-16T19:29:25.625 INFO:tasks.workunit.client.0.smithi022.stdout:9/897: dwrite db/d14/dc8/dc9/d105/fa7 [4194304,4194304] 0 2024-02-16T19:29:25.626 INFO:tasks.workunit.client.0.smithi022.stdout:1/962: creat d6/d8/d22/d16/d17/d5d/de9/f126 x:0 0 0 2024-02-16T19:29:25.628 INFO:tasks.workunit.client.0.smithi022.stdout:9/898: creat db/d14/d19/d1a/de5/f12f x:0 0 0 2024-02-16T19:29:25.632 INFO:tasks.workunit.client.0.smithi022.stdout:1/963: chown d6/d8/d22/d59/da7/fd1 17 1 2024-02-16T19:29:25.633 INFO:tasks.workunit.client.0.smithi022.stdout:9/899: chown db/d14/d16/d49/d90/lc2 8842 1 2024-02-16T19:29:25.636 INFO:tasks.workunit.client.0.smithi022.stdout:7/975: dread d9/da9/dbc/fef [0,4194304] 0 2024-02-16T19:29:25.638 INFO:tasks.workunit.client.0.smithi022.stdout:1/964: creat d6/d66/f127 x:0 0 0 2024-02-16T19:29:25.641 INFO:tasks.workunit.client.0.smithi022.stdout:5/965: dwrite dd/d1f/d48/d21/d2f/d52/f10d [0,4194304] 0 2024-02-16T19:29:25.642 INFO:tasks.workunit.client.0.smithi022.stdout:7/976: readlink d9/da/d3b/lf1 0 2024-02-16T19:29:25.642 INFO:tasks.workunit.client.0.smithi022.stdout:2/899: dwrite d0/d5d/dc4/fdc [0,4194304] 0 2024-02-16T19:29:25.651 INFO:tasks.workunit.client.0.smithi022.stdout:6/929: dwrite d1/d8/da/d29/deb/dfd/d85/d5b/f63 [4194304,4194304] 0 2024-02-16T19:29:25.654 INFO:tasks.workunit.client.0.smithi022.stdout:9/900: dwrite db/d14/dc8/dc9/d105/fa7 [0,4194304] 0 2024-02-16T19:29:25.656 INFO:tasks.workunit.client.0.smithi022.stdout:5/966: fdatasync f5 0 2024-02-16T19:29:25.662 INFO:tasks.workunit.client.0.smithi022.stdout:1/965: rmdir d6/d8/d6c/df5 0 2024-02-16T19:29:25.664 INFO:tasks.workunit.client.0.smithi022.stdout:7/977: write d9/d5d/f97 [168519,9452] 0 2024-02-16T19:29:25.666 INFO:tasks.workunit.client.0.smithi022.stdout:5/967: unlink dd/d1f/d48/d21/d2f/d40/d49/d4b/f66 0 2024-02-16T19:29:25.666 INFO:tasks.workunit.client.0.smithi022.stdout:2/900: write d0/d2/d18/d19/d90/fa0 [3297689,4053] 0 2024-02-16T19:29:25.667 INFO:tasks.workunit.client.0.smithi022.stdout:1/966: dread d6/d8/d22/d3a/d7a/f122 [0,4194304] 0 2024-02-16T19:29:25.670 INFO:tasks.workunit.client.0.smithi022.stdout:6/930: write d1/d8/da/d29/deb/dfd/d85/fe6 [878242,84493] 0 2024-02-16T19:29:25.671 INFO:tasks.workunit.client.0.smithi022.stdout:7/978: fsync d9/da/d2c/d6d/d146/fd5 0 2024-02-16T19:29:25.676 INFO:tasks.workunit.client.0.smithi022.stdout:9/901: write db/d14/d19/d1a/f24 [679128,45845] 0 2024-02-16T19:29:25.676 INFO:tasks.workunit.client.0.smithi022.stdout:5/968: dread dd/d1f/d76/fab [0,4194304] 0 2024-02-16T19:29:25.678 INFO:tasks.workunit.client.0.smithi022.stdout:6/931: mkdir d1/d8/da/d29/deb/dfd/d85/d56/d131/d13c 0 2024-02-16T19:29:25.680 INFO:tasks.workunit.client.0.smithi022.stdout:7/979: dwrite d9/da/d1f/d93/f145 [0,4194304] 0 2024-02-16T19:29:25.683 INFO:tasks.workunit.client.0.smithi022.stdout:5/969: mknod dd/d1f/d48/d21/d2f/d40/d49/c13c 0 2024-02-16T19:29:25.684 INFO:tasks.workunit.client.0.smithi022.stdout:6/932: creat d1/d8/da/d29/deb/dfd/d85/d56/d131/f13d x:0 0 0 2024-02-16T19:29:25.684 INFO:tasks.workunit.client.0.smithi022.stdout:7/980: creat d9/da/d1f/d93/f14d x:0 0 0 2024-02-16T19:29:25.684 INFO:tasks.workunit.client.0.smithi022.stdout:1/967: getdents d6/d8/d22/d16/d17/d1d/df9 0 2024-02-16T19:29:25.687 INFO:tasks.workunit.client.0.smithi022.stdout:9/902: chown db/d14/dc8/f12a 1987542 1 2024-02-16T19:29:25.687 INFO:tasks.workunit.client.0.smithi022.stdout:7/981: truncate d9/da/d1f/d40/fb5 4759490 0 2024-02-16T19:29:25.689 INFO:tasks.workunit.client.0.smithi022.stdout:1/968: write d6/d8/d22/d3f/d99/fb0 [681288,89962] 0 2024-02-16T19:29:25.690 INFO:tasks.workunit.client.0.smithi022.stdout:7/982: stat d9/da/d1f/d93/fb8 0 2024-02-16T19:29:25.693 INFO:tasks.workunit.client.0.smithi022.stdout:7/983: chown d9/da/d1f/d44/f54 1 1 2024-02-16T19:29:25.693 INFO:tasks.workunit.client.0.smithi022.stdout:9/903: unlink db/d14/d16/d28/l40 0 2024-02-16T19:29:25.694 INFO:tasks.workunit.client.0.smithi022.stdout:1/969: dread - d6/d8/d22/d16/d17/d26/f91 zero size 2024-02-16T19:29:25.694 INFO:tasks.workunit.client.0.smithi022.stdout:7/984: readlink d9/da/d4e/d7a/l9b 0 2024-02-16T19:29:25.698 INFO:tasks.workunit.client.0.smithi022.stdout:2/901: dwrite d0/d32/fbb [4194304,4194304] 0 2024-02-16T19:29:25.699 INFO:tasks.workunit.client.0.smithi022.stdout:5/970: dread dd/d1f/d48/d21/d64/d11d/d10a/d136/d5e/f7b [0,4194304] 0 2024-02-16T19:29:25.700 INFO:tasks.workunit.client.0.smithi022.stdout:5/971: truncate dd/ff9 2173807 0 2024-02-16T19:29:25.701 INFO:tasks.workunit.client.0.smithi022.stdout:9/904: mkdir db/d14/d16/d27/d8b/d130 0 2024-02-16T19:29:25.704 INFO:tasks.workunit.client.0.smithi022.stdout:6/933: write d1/d8/da/d29/deb/dfd/d85/d56/f6a [1476215,130746] 0 2024-02-16T19:29:25.705 INFO:tasks.workunit.client.1.smithi117.stdout:9/946: sync 2024-02-16T19:29:25.707 INFO:tasks.workunit.client.0.smithi022.stdout:2/902: dwrite d0/d5d/dc4/fdc [0,4194304] 0 2024-02-16T19:29:25.710 INFO:tasks.workunit.client.0.smithi022.stdout:1/970: write d6/d8/d22/f28 [3976716,42382] 0 2024-02-16T19:29:25.711 INFO:tasks.workunit.client.0.smithi022.stdout:5/972: mkdir dd/d1f/d48/d21/d2f/d52/d5c/d13d 0 2024-02-16T19:29:25.716 INFO:tasks.workunit.client.0.smithi022.stdout:9/905: creat db/d14/d16/d5d/d65/f131 x:0 0 0 2024-02-16T19:29:25.717 INFO:tasks.workunit.client.0.smithi022.stdout:7/985: dwrite d9/da/d1f/d67/f6b [4194304,4194304] 0 2024-02-16T19:29:25.717 INFO:tasks.workunit.client.0.smithi022.stdout:5/973: creat dd/d1f/d48/d21/dc9/f13e x:0 0 0 2024-02-16T19:29:25.720 INFO:tasks.workunit.client.1.smithi117.stdout:9/947: rename dd/d45/d31/d63/dba/df0/l33 to dd/d45/d31/d63/dba/df0/l140 0 2024-02-16T19:29:25.720 INFO:tasks.workunit.client.1.smithi117.stdout:9/948: dread - dd/d47/d48/dc0/f44 zero size 2024-02-16T19:29:25.722 INFO:tasks.workunit.client.0.smithi022.stdout:2/903: creat d0/d2/dde/f127 x:0 0 0 2024-02-16T19:29:25.723 INFO:tasks.workunit.client.0.smithi022.stdout:6/934: getdents d1/d8/dac 0 2024-02-16T19:29:25.729 INFO:tasks.workunit.client.0.smithi022.stdout:5/974: creat dd/d1f/d48/d21/d2f/d40/d49/d4b/db5/f13f x:0 0 0 2024-02-16T19:29:25.730 INFO:tasks.workunit.client.0.smithi022.stdout:5/975: readlink dd/d1f/d48/d21/d2f/d40/l90 0 2024-02-16T19:29:25.730 INFO:tasks.workunit.client.1.smithi117.stdout:9/949: dread dd/d45/f4f [0,4194304] 0 2024-02-16T19:29:25.732 INFO:tasks.workunit.client.0.smithi022.stdout:5/976: chown dd/d1f/d48/d21/d2f/d52/d77/d95/fec 31854 1 2024-02-16T19:29:25.732 INFO:tasks.workunit.client.1.smithi117.stdout:9/950: write dd/d47/d48/f50 [2867632,37477] 0 2024-02-16T19:29:25.738 INFO:tasks.workunit.client.0.smithi022.stdout:7/986: mkdir d9/da/d74/d131/d14e 0 2024-02-16T19:29:25.738 INFO:tasks.workunit.client.0.smithi022.stdout:5/977: rmdir dd/d1f/d48/d21/d64/d11d/d10a/d136/d5e/d84 39 2024-02-16T19:29:25.739 INFO:tasks.workunit.client.0.smithi022.stdout:9/906: write db/d14/d19/f5f [1243449,35320] 0 2024-02-16T19:29:25.739 INFO:tasks.workunit.client.0.smithi022.stdout:1/971: dwrite d6/d8/f2d [0,4194304] 0 2024-02-16T19:29:25.742 INFO:tasks.workunit.client.0.smithi022.stdout:6/935: write d1/d8/da/d29/deb/dfd/d53/f7d [1184068,130266] 0 2024-02-16T19:29:25.743 INFO:tasks.workunit.client.0.smithi022.stdout:3/923: sync 2024-02-16T19:29:25.744 INFO:tasks.workunit.client.0.smithi022.stdout:9/907: readlink db/l2c 0 2024-02-16T19:29:25.745 INFO:tasks.workunit.client.0.smithi022.stdout:1/972: fdatasync d6/d8/d22/d67/d7e/fa4 0 2024-02-16T19:29:25.746 INFO:tasks.workunit.client.0.smithi022.stdout:7/987: dread d9/da/d1f/d67/fcf [0,4194304] 0 2024-02-16T19:29:25.747 INFO:tasks.workunit.client.0.smithi022.stdout:3/924: chown d7/d12/d18/dd/d24/de5 13 1 2024-02-16T19:29:25.748 INFO:tasks.workunit.client.0.smithi022.stdout:5/978: truncate dd/d1f/d48/d21/d2f/d52/d77/d95/ded/f13a 784795 0 2024-02-16T19:29:25.749 INFO:tasks.workunit.client.0.smithi022.stdout:9/908: readlink db/d14/dc8/dc9/d105/le6 0 2024-02-16T19:29:25.750 INFO:tasks.workunit.client.0.smithi022.stdout:3/925: stat d7/f1c 0 2024-02-16T19:29:25.750 INFO:tasks.workunit.client.0.smithi022.stdout:2/904: dwrite d0/d32/f94 [0,4194304] 0 2024-02-16T19:29:25.756 INFO:tasks.workunit.client.1.smithi117.stdout:1/932: sync 2024-02-16T19:29:25.758 INFO:tasks.workunit.client.1.smithi117.stdout:9/951: dread dd/d47/d5b/fc6 [0,4194304] 0 2024-02-16T19:29:25.759 INFO:tasks.workunit.client.1.smithi117.stdout:9/952: dread - dd/d45/d31/d63/d121/f113 zero size 2024-02-16T19:29:25.759 INFO:tasks.workunit.client.0.smithi022.stdout:6/936: dread d1/d8/da/d29/deb/dfd/d85/f36 [0,4194304] 0 2024-02-16T19:29:25.760 INFO:tasks.workunit.client.1.smithi117.stdout:9/953: chown dd/d47/d48/c6c 2025567 1 2024-02-16T19:29:25.761 INFO:tasks.workunit.client.1.smithi117.stdout:1/933: unlink d2/d19/d59/d76/d8c/fca 0 2024-02-16T19:29:25.761 INFO:tasks.workunit.client.0.smithi022.stdout:1/973: write d6/d8/dba/fce [260299,84966] 0 2024-02-16T19:29:25.762 INFO:tasks.workunit.client.0.smithi022.stdout:3/926: rename d7/d12/d18/dd/cbb to d7/d12/d18/dd/c144 0 2024-02-16T19:29:25.763 INFO:tasks.workunit.client.0.smithi022.stdout:5/979: write dd/d1f/d48/d21/d5d/dc0/ffc [839588,124205] 0 2024-02-16T19:29:25.765 INFO:tasks.workunit.client.0.smithi022.stdout:7/988: truncate d9/da/d1f/f22 3468724 0 2024-02-16T19:29:25.767 INFO:tasks.workunit.client.1.smithi117.stdout:9/954: dwrite dd/d47/d5b/fbd [0,4194304] 0 2024-02-16T19:29:25.768 INFO:tasks.workunit.client.0.smithi022.stdout:1/974: mkdir d6/d8/d22/d16/d17/d1d/df9/d128 0 2024-02-16T19:29:25.768 INFO:tasks.workunit.client.0.smithi022.stdout:9/909: dread db/d18/f74 [0,4194304] 0 2024-02-16T19:29:25.769 INFO:tasks.workunit.client.0.smithi022.stdout:6/937: mknod d1/d8/da/d29/da1/c13e 0 2024-02-16T19:29:25.773 INFO:tasks.workunit.client.0.smithi022.stdout:7/989: symlink d9/da/d3b/dba/l14f 0 2024-02-16T19:29:25.775 INFO:tasks.workunit.client.0.smithi022.stdout:5/980: rename dd/d1f/d48/d21/d2f/d40/d49/d4b/db5/f13f to dd/d1f/d48/d21/dc9/f140 0 2024-02-16T19:29:25.778 INFO:tasks.workunit.client.0.smithi022.stdout:1/975: write d6/d8/d22/d47/db7/fdb [1036073,21095] 0 2024-02-16T19:29:25.782 INFO:tasks.workunit.client.0.smithi022.stdout:3/927: mknod d7/d12/d18/dd/d31/dbf/dc5/d140/c145 0 2024-02-16T19:29:25.784 INFO:tasks.workunit.client.0.smithi022.stdout:7/990: rename d9/da/d1f/f81 to d9/da/d1f/d44/d5b/d95/f150 0 2024-02-16T19:29:25.788 INFO:tasks.workunit.client.0.smithi022.stdout:9/910: link db/d14/d16/d28/fd4 db/d14/d16/d27/d2e/f132 0 2024-02-16T19:29:25.789 INFO:tasks.workunit.client.1.smithi117.stdout:1/934: dwrite d2/d19/d59/d89/d69/d80/d8e/f90 [0,4194304] 0 2024-02-16T19:29:25.789 INFO:tasks.workunit.client.0.smithi022.stdout:6/938: write d1/d8/da/d29/deb/dfd/d48/d51/f9b [966296,20883] 0 2024-02-16T19:29:25.792 INFO:tasks.workunit.client.0.smithi022.stdout:1/976: truncate d6/d8/d22/d3f/dc5/d100/f10e 333088 0 2024-02-16T19:29:25.794 INFO:tasks.workunit.client.0.smithi022.stdout:5/981: write dd/d1f/d48/d21/d64/d11d/d10a/d136/d5e/fa6 [4917167,52075] 0 2024-02-16T19:29:25.794 INFO:tasks.workunit.client.1.smithi117.stdout:9/955: dread dd/d45/d31/d63/dba/df0/daf/df4/f9f [0,4194304] 0 2024-02-16T19:29:25.796 INFO:tasks.workunit.client.1.smithi117.stdout:1/935: chown d2/d4/l3b 0 1 2024-02-16T19:29:25.797 INFO:tasks.workunit.client.0.smithi022.stdout:9/911: unlink db/d14/d16/d21/d4a/lc4 0 2024-02-16T19:29:25.798 INFO:tasks.workunit.client.0.smithi022.stdout:6/939: creat d1/d8/da/d29/deb/dfd/d85/d9c/f13f x:0 0 0 2024-02-16T19:29:25.799 INFO:tasks.workunit.client.1.smithi117.stdout:9/956: dread dd/d45/d31/d63/dba/df0/daf/df4/f61 [0,4194304] 0 2024-02-16T19:29:25.800 INFO:tasks.workunit.client.0.smithi022.stdout:6/940: fsync d1/d8/da/d29/d61/f9e 0 2024-02-16T19:29:25.805 INFO:tasks.workunit.client.0.smithi022.stdout:9/912: rename db/d14/d19/d1a/f50 to db/d14/d16/d27/d2e/f133 0 2024-02-16T19:29:25.807 INFO:tasks.workunit.client.0.smithi022.stdout:3/928: dwrite d7/d12/d18/f1b [4194304,4194304] 0 2024-02-16T19:29:25.808 INFO:tasks.workunit.client.1.smithi117.stdout:9/957: mknod dd/d45/d31/d63/dba/df0/daf/d12b/c141 0 2024-02-16T19:29:25.809 INFO:tasks.workunit.client.0.smithi022.stdout:1/977: write d6/d66/f9c [552873,20396] 0 2024-02-16T19:29:25.810 INFO:tasks.workunit.client.0.smithi022.stdout:5/982: creat dd/d1f/d48/d21/d2f/d52/d5c/d13d/f141 x:0 0 0 2024-02-16T19:29:25.813 INFO:tasks.workunit.client.0.smithi022.stdout:9/913: dread - db/d14/d16/d5d/d65/d6b/d89/dd2/ff0 zero size 2024-02-16T19:29:25.813 INFO:tasks.workunit.client.0.smithi022.stdout:3/929: mkdir d7/d12/d18/dd/d24/d53/d116/de6/d11a/d146 0 2024-02-16T19:29:25.815 INFO:tasks.workunit.client.0.smithi022.stdout:5/983: truncate f5 1000378 0 2024-02-16T19:29:25.816 INFO:tasks.workunit.client.1.smithi117.stdout:1/936: dread d2/d1d/f25 [4194304,4194304] 0 2024-02-16T19:29:25.817 INFO:tasks.workunit.client.0.smithi022.stdout:1/978: rmdir d6/d8/d22/d16/d17/d63 39 2024-02-16T19:29:25.820 INFO:tasks.workunit.client.0.smithi022.stdout:9/914: stat db/d14/d16/d5d/d65/d6b/d89/dd2/lde 0 2024-02-16T19:29:25.822 INFO:tasks.workunit.client.1.smithi117.stdout:1/937: creat d2/d19/d59/d76/d8c/dc7/f141 x:0 0 0 2024-02-16T19:29:25.823 INFO:tasks.workunit.client.1.smithi117.stdout:1/938: chown d2/d4/dbb/le4 35 1 2024-02-16T19:29:25.824 INFO:tasks.workunit.client.1.smithi117.stdout:9/958: getdents dd/d47/d5b/d5f/d77/d7c 0 2024-02-16T19:29:25.827 INFO:tasks.workunit.client.0.smithi022.stdout:5/984: symlink dd/d1f/d48/d21/d64/d11d/d10a/l142 0 2024-02-16T19:29:25.828 INFO:tasks.workunit.client.0.smithi022.stdout:1/979: symlink d6/d8/d22/d67/d7e/l129 0 2024-02-16T19:29:25.830 INFO:tasks.workunit.client.0.smithi022.stdout:1/980: dread d6/d8/d22/d67/fd6 [0,4194304] 0 2024-02-16T19:29:25.830 INFO:tasks.workunit.client.1.smithi117.stdout:9/959: mkdir dd/d45/d31/d63/dba/df0/daf/d142 0 2024-02-16T19:29:25.833 INFO:tasks.workunit.client.1.smithi117.stdout:9/960: unlink dd/d45/d31/d63/lf6 0 2024-02-16T19:29:25.834 INFO:tasks.workunit.client.0.smithi022.stdout:5/985: mkdir dd/d1f/d48/d21/d2f/d40/d4e/d56/d9d/d143 0 2024-02-16T19:29:25.838 INFO:tasks.workunit.client.0.smithi022.stdout:9/915: mkdir db/d14/d10a/d134 0 2024-02-16T19:29:25.838 INFO:tasks.workunit.client.0.smithi022.stdout:3/930: link d7/d12/f14 d7/d12/d18/dac/f147 0 2024-02-16T19:29:25.839 INFO:tasks.workunit.client.0.smithi022.stdout:1/981: rmdir d6/d8/d22/d67 39 2024-02-16T19:29:25.840 INFO:tasks.workunit.client.1.smithi117.stdout:1/939: dread d2/d4/f11 [0,4194304] 0 2024-02-16T19:29:25.845 INFO:tasks.workunit.client.1.smithi117.stdout:9/961: dread dd/d47/d48/f50 [0,4194304] 0 2024-02-16T19:29:25.846 INFO:tasks.workunit.client.0.smithi022.stdout:3/931: dwrite d7/d12/d18/dac/faa [4194304,4194304] 0 2024-02-16T19:29:25.846 INFO:tasks.workunit.client.0.smithi022.stdout:9/916: rename db/d14/d16/d5d/d65/d6b/df6 to db/d14/d10a/d135 0 2024-02-16T19:29:25.847 INFO:tasks.workunit.client.1.smithi117.stdout:1/940: fsync d2/d19/d59/d76/d8c/fb1 0 2024-02-16T19:29:25.851 INFO:tasks.workunit.client.0.smithi022.stdout:1/982: truncate d6/d8/d22/d67/d7e/fdc 673395 0 2024-02-16T19:29:25.854 INFO:tasks.workunit.client.0.smithi022.stdout:9/917: read db/d18/f93 [255364,98938] 0 2024-02-16T19:29:25.858 INFO:tasks.workunit.client.0.smithi022.stdout:1/983: creat d6/d8/d22/d47/d51/f12a x:0 0 0 2024-02-16T19:29:25.858 INFO:tasks.workunit.client.0.smithi022.stdout:5/986: dwrite dd/d1f/d48/d21/d2f/d40/fe8 [0,4194304] 0 2024-02-16T19:29:25.860 INFO:tasks.workunit.client.1.smithi117.stdout:9/962: write dd/d47/d48/dc0/fe7 [1447216,108650] 0 2024-02-16T19:29:25.865 INFO:tasks.workunit.client.0.smithi022.stdout:3/932: mkdir d7/d12/d18/dd/d3c/d5a/d148 0 2024-02-16T19:29:25.866 INFO:tasks.workunit.client.0.smithi022.stdout:3/933: dread - d7/d12/d18/dd/d3c/d5a/f69 zero size 2024-02-16T19:29:25.871 INFO:tasks.workunit.client.0.smithi022.stdout:5/987: dread dd/d1f/d48/d21/d64/d11d/d6a/dfb/ffe [0,4194304] 0 2024-02-16T19:29:25.872 INFO:tasks.workunit.client.0.smithi022.stdout:9/918: dwrite db/d18/f74 [4194304,4194304] 0 2024-02-16T19:29:25.873 INFO:tasks.workunit.client.0.smithi022.stdout:9/919: stat db/d14/d16/d27/fd1 0 2024-02-16T19:29:25.877 INFO:tasks.workunit.client.0.smithi022.stdout:9/920: dread db/d14/d16/d5d/f7e [4194304,4194304] 0 2024-02-16T19:29:25.882 INFO:tasks.workunit.client.0.smithi022.stdout:4/876: sync 2024-02-16T19:29:25.886 INFO:tasks.workunit.client.0.smithi022.stdout:5/988: readlink dd/d1f/d48/d21/d64/d11d/d10a/d136/d5e/d84/l102 0 2024-02-16T19:29:25.893 INFO:tasks.workunit.client.0.smithi022.stdout:4/877: fsync d4/d6/dd/f37 0 2024-02-16T19:29:25.894 INFO:tasks.workunit.client.0.smithi022.stdout:1/984: getdents d6/d8 0 2024-02-16T19:29:25.897 INFO:tasks.workunit.client.0.smithi022.stdout:5/989: creat dd/d1f/d48/d21/dc9/f144 x:0 0 0 2024-02-16T19:29:25.900 INFO:tasks.workunit.client.0.smithi022.stdout:3/934: getdents d7/d12/d18/dd/d24/d93/de9/dc4 0 2024-02-16T19:29:25.901 INFO:tasks.workunit.client.0.smithi022.stdout:4/878: creat d4/d6/d14/d64/de0/f11a x:0 0 0 2024-02-16T19:29:25.903 INFO:tasks.workunit.client.0.smithi022.stdout:4/879: read - d4/d6/dd/d22/d83/d9e/db7/f117 zero size 2024-02-16T19:29:25.903 INFO:tasks.workunit.client.0.smithi022.stdout:5/990: fsync dd/d1f/d48/d21/d2f/d40/d71/fe4 0 2024-02-16T19:29:25.905 INFO:tasks.workunit.client.0.smithi022.stdout:9/921: link db/d14/d16/fbf db/d18/d103/f136 0 2024-02-16T19:29:25.905 INFO:tasks.workunit.client.0.smithi022.stdout:5/991: write dd/d1f/d48/d21/d2f/d40/d4e/f98 [2138437,78360] 0 2024-02-16T19:29:25.909 INFO:tasks.workunit.client.0.smithi022.stdout:1/985: rename d6/d8/d22/d16/d17/d5d/c6d to d6/d8/d22/d16/d17/d5d/de9/c12b 0 2024-02-16T19:29:25.911 INFO:tasks.workunit.client.0.smithi022.stdout:4/880: chown d4/d6/d14/d64/l67 509 1 2024-02-16T19:29:25.911 INFO:tasks.workunit.client.0.smithi022.stdout:4/881: readlink d4/d6/dd/d22/da8/dc4/l76 0 2024-02-16T19:29:25.916 INFO:tasks.workunit.client.0.smithi022.stdout:9/922: mkdir db/d14/d16/d28/daa/d137 0 2024-02-16T19:29:25.921 INFO:tasks.workunit.client.0.smithi022.stdout:4/882: rmdir d4/d6/dd/d22/d105 39 2024-02-16T19:29:25.925 INFO:tasks.workunit.client.1.smithi117.stdout:1/941: creat d2/d4/dbb/d10b/d140/f142 x:0 0 0 2024-02-16T19:29:25.925 INFO:tasks.workunit.client.0.smithi022.stdout:5/992: mknod dd/d1f/d48/d21/d2f/d52/d5c/c145 0 2024-02-16T19:29:25.926 INFO:tasks.workunit.client.0.smithi022.stdout:5/993: write dd/ff9 [2264422,80059] 0 2024-02-16T19:29:25.929 INFO:tasks.workunit.client.1.smithi117.stdout:1/942: fdatasync d2/d1d/f25 0 2024-02-16T19:29:25.931 INFO:tasks.workunit.client.1.smithi117.stdout:9/963: link dd/d47/d5b/fbd dd/d45/d31/d63/dba/df0/daf/f143 0 2024-02-16T19:29:25.933 INFO:tasks.workunit.client.0.smithi022.stdout:3/935: dwrite d7/d12/d18/dac/f147 [0,4194304] 0 2024-02-16T19:29:25.937 INFO:tasks.workunit.client.0.smithi022.stdout:3/936: chown d7/d12/d18/dd/d24/d53/lc1 1797 1 2024-02-16T19:29:25.938 INFO:tasks.workunit.client.0.smithi022.stdout:4/883: dwrite d4/d6/d3a/f119 [0,4194304] 0 2024-02-16T19:29:25.938 INFO:tasks.workunit.client.0.smithi022.stdout:5/994: mknod dd/d1f/d48/d21/d5d/c146 0 2024-02-16T19:29:25.940 INFO:tasks.workunit.client.1.smithi117.stdout:9/964: fsync dd/d45/d31/d63/dba/df0/daf/df4/f5a 0 2024-02-16T19:29:25.941 INFO:tasks.workunit.client.0.smithi022.stdout:9/923: dwrite db/d14/d16/d28/fd4 [0,4194304] 0 2024-02-16T19:29:25.944 INFO:tasks.workunit.client.0.smithi022.stdout:1/986: creat d6/d8/d22/d16/d17/f12c x:0 0 0 2024-02-16T19:29:25.945 INFO:tasks.workunit.client.1.smithi117.stdout:1/943: creat d2/d1d/d77/d125/f143 x:0 0 0 2024-02-16T19:29:25.947 INFO:tasks.workunit.client.1.smithi117.stdout:1/944: dread - d2/d19/d59/d76/d8c/dc7/f141 zero size 2024-02-16T19:29:25.949 INFO:tasks.workunit.client.0.smithi022.stdout:1/987: dread d6/d8/d22/d16/d17/d5d/fe1 [0,4194304] 0 2024-02-16T19:29:25.950 INFO:tasks.workunit.client.0.smithi022.stdout:3/937: symlink d7/d12/d18/dac/l149 0 2024-02-16T19:29:25.950 INFO:tasks.workunit.client.1.smithi117.stdout:1/945: rmdir d2/d19/d59/d89/d8d 39 2024-02-16T19:29:25.955 INFO:tasks.workunit.client.1.smithi117.stdout:1/946: creat d2/d19/d59/d89/d69/d9a/f144 x:0 0 0 2024-02-16T19:29:25.957 INFO:tasks.workunit.client.0.smithi022.stdout:4/884: dwrite d4/d6/db6/fa4 [8388608,4194304] 0 2024-02-16T19:29:25.957 INFO:tasks.workunit.client.0.smithi022.stdout:1/988: dread d6/d8/d22/d67/d7e/fa5 [0,4194304] 0 2024-02-16T19:29:25.958 INFO:tasks.workunit.client.1.smithi117.stdout:9/965: link dd/d45/c9d dd/d45/d31/d63/dba/df0/daf/df4/d10d/c144 0 2024-02-16T19:29:25.959 INFO:tasks.workunit.client.0.smithi022.stdout:1/989: fsync d6/d8/f111 0 2024-02-16T19:29:25.962 INFO:tasks.workunit.client.1.smithi117.stdout:1/947: dwrite d2/d4/dbb/f132 [0,4194304] 0 2024-02-16T19:29:25.964 INFO:tasks.workunit.client.0.smithi022.stdout:3/938: rmdir d7/d12/d18/dd/d24 39 2024-02-16T19:29:25.966 INFO:tasks.workunit.client.1.smithi117.stdout:1/948: rmdir d2/d19/d59/d76/d91 39 2024-02-16T19:29:25.969 INFO:tasks.workunit.client.1.smithi117.stdout:1/949: write d2/d1d/d1f/f56 [1539488,33575] 0 2024-02-16T19:29:25.969 INFO:tasks.workunit.client.0.smithi022.stdout:4/885: creat d4/d6/d14/dc7/d6a/f11b x:0 0 0 2024-02-16T19:29:25.975 INFO:tasks.workunit.client.0.smithi022.stdout:5/995: creat dd/d1f/d48/d21/d2f/d52/f147 x:0 0 0 2024-02-16T19:29:25.977 INFO:tasks.workunit.client.1.smithi117.stdout:9/966: rename dd/d45/d31/d63/dba/df0/daf/df4/c4b to dd/d47/d5b/d5f/d77/c145 0 2024-02-16T19:29:25.979 INFO:tasks.workunit.client.1.smithi117.stdout:1/950: mknod d2/d19/d59/d76/d12a/c145 0 2024-02-16T19:29:25.986 INFO:tasks.workunit.client.0.smithi022.stdout:4/886: mkdir d4/d6/db6/d11c 0 2024-02-16T19:29:25.987 INFO:tasks.workunit.client.0.smithi022.stdout:9/924: dwrite db/d14/ff4 [0,4194304] 0 2024-02-16T19:29:25.988 INFO:tasks.workunit.client.1.smithi117.stdout:1/951: truncate d2/d19/d59/d89/f32 4992049 0 2024-02-16T19:29:25.988 INFO:tasks.workunit.client.0.smithi022.stdout:4/887: fsync d4/d6/d14/dd2/fec 0 2024-02-16T19:29:25.990 INFO:tasks.workunit.client.0.smithi022.stdout:5/996: mknod dd/d1f/d48/d21/d2f/d40/d11a/c148 0 2024-02-16T19:29:25.994 INFO:tasks.workunit.client.1.smithi117.stdout:9/967: rmdir dd/d47 39 2024-02-16T19:29:25.995 INFO:tasks.workunit.client.1.smithi117.stdout:9/968: readlink dd/l10 0 2024-02-16T19:29:25.995 INFO:tasks.workunit.client.1.smithi117.stdout:1/952: dread d2/d19/d59/d89/d69/d9a/fd1 [0,4194304] 0 2024-02-16T19:29:25.996 INFO:tasks.workunit.client.0.smithi022.stdout:9/925: dwrite db/d14/d16/d27/d2e/f132 [0,4194304] 0 2024-02-16T19:29:25.996 INFO:tasks.workunit.client.0.smithi022.stdout:5/997: mkdir dd/d1f/d48/d21/d64/d11d/d6a/d149 0 2024-02-16T19:29:25.997 INFO:tasks.workunit.client.0.smithi022.stdout:9/926: fsync db/d14/d10a/d135/f11e 0 2024-02-16T19:29:25.997 INFO:tasks.workunit.client.0.smithi022.stdout:1/990: write d6/d8/d22/d16/d17/f98 [12099,42001] 0 2024-02-16T19:29:26.002 INFO:tasks.workunit.client.1.smithi117.stdout:9/969: readlink dd/d45/d31/d63/dba/df0/daf/ldd 0 2024-02-16T19:29:26.003 INFO:tasks.workunit.client.0.smithi022.stdout:1/991: stat d6/f8e 0 2024-02-16T19:29:26.004 INFO:tasks.workunit.client.0.smithi022.stdout:1/992: fdatasync d6/d8/d22/d3f/f84 0 2024-02-16T19:29:26.004 INFO:tasks.workunit.client.0.smithi022.stdout:5/998: truncate dd/d1f/d48/d21/d64/d11d/f101 1734576 0 2024-02-16T19:29:26.005 INFO:tasks.workunit.client.0.smithi022.stdout:4/888: dread d4/d6/dd/d22/da8/dc4/f82 [0,4194304] 0 2024-02-16T19:29:26.008 INFO:tasks.workunit.client.1.smithi117.stdout:1/953: creat d2/d1d/d60/d7c/d9e/da5/db0/f146 x:0 0 0 2024-02-16T19:29:26.009 INFO:tasks.workunit.client.0.smithi022.stdout:1/993: creat d6/d8/dba/f12d x:0 0 0 2024-02-16T19:29:26.010 INFO:tasks.workunit.client.0.smithi022.stdout:3/939: dwrite d7/d12/d18/dd/d24/d29/f2c [0,4194304] 0 2024-02-16T19:29:26.011 INFO:tasks.workunit.client.0.smithi022.stdout:4/889: read d4/f47 [422343,35548] 0 2024-02-16T19:29:26.012 INFO:tasks.workunit.client.0.smithi022.stdout:1/994: chown d6/d66/f9c 0 1 2024-02-16T19:29:26.012 INFO:tasks.workunit.client.1.smithi117.stdout:1/954: rmdir d2/d19/d59/d89/d49 39 2024-02-16T19:29:26.012 INFO:tasks.workunit.client.0.smithi022.stdout:4/890: write d4/d6/dd/df/d10/fd8 [734346,3715] 0 2024-02-16T19:29:26.014 INFO:tasks.workunit.client.1.smithi117.stdout:9/970: creat dd/d47/d5b/d5f/de6/f146 x:0 0 0 2024-02-16T19:29:26.015 INFO:tasks.workunit.client.0.smithi022.stdout:3/940: unlink d7/d12/d18/d4b/f6f 0 2024-02-16T19:29:26.017 INFO:tasks.workunit.client.0.smithi022.stdout:3/941: chown d7/d12/d18/dd/cfb 197939 1 2024-02-16T19:29:26.019 INFO:tasks.workunit.client.0.smithi022.stdout:5/999: creat dd/d1f/d48/d21/d64/d11d/d6a/d149/f14a x:0 0 0 2024-02-16T19:29:26.021 INFO:tasks.workunit.client.1.smithi117.stdout:9/971: unlink dd/d47/d5b/d5f/d77/f120 0 2024-02-16T19:29:26.022 INFO:tasks.workunit.client.0.smithi022.stdout:1/995: unlink d6/d8/d22/d16/l74 0 2024-02-16T19:29:26.024 INFO:tasks.workunit.client.0.smithi022.stdout:4/891: creat d4/d6/dd/d22/d83/dc5/f11d x:0 0 0 2024-02-16T19:29:26.025 INFO:tasks.workunit.client.0.smithi022.stdout:4/892: stat d4/d6/d3a/l75 0 2024-02-16T19:29:26.027 INFO:tasks.workunit.client.0.smithi022.stdout:3/942: mkdir d7/d12/d18/dd/d31/dbf/dc5/d140/d14a 0 2024-02-16T19:29:26.029 INFO:tasks.workunit.client.0.smithi022.stdout:1/996: mkdir d6/d8/d22/d16/d17/d26/d12e 0 2024-02-16T19:29:26.029 INFO:tasks.workunit.client.0.smithi022.stdout:9/927: dread db/d14/d16/d27/d8b/f100 [0,4194304] 0 2024-02-16T19:29:26.031 INFO:tasks.workunit.client.0.smithi022.stdout:4/893: chown d4/d6/dd/d22/da8/dc4/d86/dfd/df2/f111 503255639 1 2024-02-16T19:29:26.032 INFO:tasks.workunit.client.1.smithi117.stdout:1/955: dwrite d2/d1d/d77/f103 [0,4194304] 0 2024-02-16T19:29:26.033 INFO:tasks.workunit.client.0.smithi022.stdout:1/997: write d6/f113 [1902314,65247] 0 2024-02-16T19:29:26.034 INFO:tasks.workunit.client.0.smithi022.stdout:4/894: truncate d4/d6/d14/d64/de0/f11a 965533 0 2024-02-16T19:29:26.038 INFO:tasks.workunit.client.0.smithi022.stdout:3/943: rename d7/d12/d18/dd/d3c to d7/d12/d18/dd/d24/d58/d14b 0 2024-02-16T19:29:26.041 INFO:tasks.workunit.client.0.smithi022.stdout:9/928: mkdir db/d14/d16/d21/d138 0 2024-02-16T19:29:26.043 INFO:tasks.workunit.client.1.smithi117.stdout:1/956: creat d2/d19/d59/d76/d91/f147 x:0 0 0 2024-02-16T19:29:26.045 INFO:tasks.workunit.client.0.smithi022.stdout:1/998: fdatasync d6/d8/d22/d16/d17/d5d/fe1 0 2024-02-16T19:29:26.045 INFO:tasks.workunit.client.0.smithi022.stdout:3/944: dwrite d7/d12/d18/dd/d24/d53/fe4 [4194304,4194304] 0 2024-02-16T19:29:26.049 INFO:tasks.workunit.client.1.smithi117.stdout:9/972: write dd/d47/d5b/d5f/f9b [672919,975] 0 2024-02-16T19:29:26.049 INFO:tasks.workunit.client.0.smithi022.stdout:9/929: creat db/d14/d16/d5d/d65/d6b/d89/dd2/f139 x:0 0 0 2024-02-16T19:29:26.052 INFO:tasks.workunit.client.1.smithi117.stdout:1/957: unlink d2/d4/dbb/d7f/f84 0 2024-02-16T19:29:26.058 INFO:tasks.workunit.client.0.smithi022.stdout:9/930: creat db/d14/d16/d5d/d65/d6b/d89/dd2/f13a x:0 0 0 2024-02-16T19:29:26.059 INFO:tasks.workunit.client.0.smithi022.stdout:4/895: getdents d4/d6/dd/d22/d105 0 2024-02-16T19:29:26.060 INFO:tasks.workunit.client.0.smithi022.stdout:9/931: creat db/d14/d16/d5d/d65/d6b/d11f/f13b x:0 0 0 2024-02-16T19:29:26.062 INFO:tasks.workunit.client.0.smithi022.stdout:9/932: write db/d14/d16/d28/daa/f111 [315985,52352] 0 2024-02-16T19:29:26.063 INFO:tasks.workunit.client.0.smithi022.stdout:4/896: mkdir d4/d6/dd/df/d10/d11e 0 2024-02-16T19:29:26.064 INFO:tasks.workunit.client.0.smithi022.stdout:4/897: stat d4/d6/dd/df/d10/de8/f110 0 2024-02-16T19:29:26.065 INFO:tasks.workunit.client.0.smithi022.stdout:9/933: fsync db/d14/d16/d28/f9a 0 2024-02-16T19:29:26.065 INFO:tasks.workunit.client.1.smithi117.stdout:1/958: write d2/d4/dbb/d7f/f85 [103516,110809] 0 2024-02-16T19:29:26.068 INFO:tasks.workunit.client.1.smithi117.stdout:9/973: dread dd/d47/d5b/d5f/f6f [4194304,4194304] 0 2024-02-16T19:29:26.068 INFO:tasks.workunit.client.0.smithi022.stdout:3/945: truncate d7/d12/d18/d4b/f9e 2471442 0 2024-02-16T19:29:26.068 INFO:tasks.workunit.client.0.smithi022.stdout:4/898: mkdir d4/d6/dd/d22/d83/d113/d11f 0 2024-02-16T19:29:26.069 INFO:tasks.workunit.client.1.smithi117.stdout:9/974: write dd/d47/d48/dc0/fe7 [1926675,40512] 0 2024-02-16T19:29:26.069 INFO:tasks.workunit.client.0.smithi022.stdout:9/934: write db/d14/d16/d27/f68 [3030539,115601] 0 2024-02-16T19:29:26.072 INFO:tasks.workunit.client.0.smithi022.stdout:9/935: read db/d14/d16/d87/f8f [2598968,29150] 0 2024-02-16T19:29:26.073 INFO:tasks.workunit.client.1.smithi117.stdout:9/975: unlink dd/db5/dc4/dcf/dda/ldf 0 2024-02-16T19:29:26.074 INFO:tasks.workunit.client.1.smithi117.stdout:9/976: truncate dd/d45/d73/ff1 1054866 0 2024-02-16T19:29:26.074 INFO:tasks.workunit.client.1.smithi117.stdout:1/959: symlink d2/d19/d59/d89/d69/dad/l148 0 2024-02-16T19:29:26.075 INFO:tasks.workunit.client.1.smithi117.stdout:1/960: truncate d2/d1d/f13a 201732 0 2024-02-16T19:29:26.076 INFO:tasks.workunit.client.0.smithi022.stdout:9/936: fsync db/f79 0 2024-02-16T19:29:26.077 INFO:tasks.workunit.client.0.smithi022.stdout:9/937: chown db/d14/d19/c46 0 1 2024-02-16T19:29:26.080 INFO:tasks.workunit.client.1.smithi117.stdout:9/977: rename dd/db5/fa0 to dd/d45/d31/d63/dba/f147 0 2024-02-16T19:29:26.081 INFO:tasks.workunit.client.1.smithi117.stdout:9/978: read dd/d45/d31/d63/d8e/fc8 [1373213,44925] 0 2024-02-16T19:29:26.083 INFO:tasks.workunit.client.0.smithi022.stdout:4/899: write d4/d6/dd/d22/d30/f26 [2196976,88325] 0 2024-02-16T19:29:26.085 INFO:tasks.workunit.client.1.smithi117.stdout:1/961: rename d2/d19/d59/d89/d69/f95 to d2/d4/dbb/d10b/dcc/f149 0 2024-02-16T19:29:26.085 INFO:tasks.workunit.client.0.smithi022.stdout:4/900: truncate d4/d6/dd/d22/d83/dc5/f11d 785066 0 2024-02-16T19:29:26.087 INFO:tasks.workunit.client.1.smithi117.stdout:9/979: fsync dd/d47/d48/f50 0 2024-02-16T19:29:26.088 INFO:tasks.workunit.client.0.smithi022.stdout:4/901: read d4/d6/d14/d64/d4e/d54/fb4 [106790,114179] 0 2024-02-16T19:29:26.090 INFO:tasks.workunit.client.1.smithi117.stdout:1/962: rename d2/d1d/d1f/ce2 to d2/d1d/d60/df2/c14a 0 2024-02-16T19:29:26.090 INFO:tasks.workunit.client.0.smithi022.stdout:3/946: dwrite d7/d12/d18/dd/d24/d58/d14b/f49 [0,4194304] 0 2024-02-16T19:29:26.092 INFO:tasks.workunit.client.0.smithi022.stdout:3/947: chown d7/d12/d18/dd/d24/d58/d14b/d5a/f111 27913 1 2024-02-16T19:29:26.094 INFO:tasks.workunit.client.0.smithi022.stdout:4/902: dread - d4/d6/dd/df/d10/ddc/fdf zero size 2024-02-16T19:29:26.096 INFO:tasks.workunit.client.1.smithi117.stdout:1/963: link d2/f53 d2/d19/d59/d76/d91/dcf/f14b 0 2024-02-16T19:29:26.098 INFO:tasks.workunit.client.0.smithi022.stdout:3/948: fdatasync d7/d12/d18/dd/d24/d53/d116/f94 0 2024-02-16T19:29:26.100 INFO:tasks.workunit.client.0.smithi022.stdout:9/938: creat db/d14/d16/d21/f13c x:0 0 0 2024-02-16T19:29:26.100 INFO:tasks.workunit.client.0.smithi022.stdout:9/939: chown db/d14/d16/lba 3635946 1 2024-02-16T19:29:26.102 INFO:tasks.workunit.client.0.smithi022.stdout:4/903: rename d4/d6/db6 to d4/d6/d14/dc7/d6a/d120 0 2024-02-16T19:29:26.103 INFO:tasks.workunit.client.1.smithi117.stdout:1/964: dwrite d2/d19/d59/fd5 [0,4194304] 0 2024-02-16T19:29:26.107 INFO:tasks.workunit.client.1.smithi117.stdout:1/965: symlink d2/d1d/d60/d7c/d9e/da5/l14c 0 2024-02-16T19:29:26.107 INFO:tasks.workunit.client.0.smithi022.stdout:4/904: mkdir d4/d6/d14/d64/d4e/d54/d121 0 2024-02-16T19:29:26.109 INFO:tasks.workunit.client.0.smithi022.stdout:3/949: rename d7/d12/d18/dd/d31/dbf/dc5/d140 to d7/d12/d14c 0 2024-02-16T19:29:26.110 INFO:tasks.workunit.client.1.smithi117.stdout:9/980: dwrite dd/db5/dc4/dcf/f127 [0,4194304] 0 2024-02-16T19:29:26.114 INFO:tasks.workunit.client.1.smithi117.stdout:1/966: rename d2/d19/d59/d89/d7b/d86/l10f to d2/d4/dbb/d7f/dc8/l14d 0 2024-02-16T19:29:26.115 INFO:tasks.workunit.client.1.smithi117.stdout:9/981: unlink dd/d45/d31/d63/dba/df0/fa8 0 2024-02-16T19:29:26.116 INFO:tasks.workunit.client.1.smithi117.stdout:1/967: mknod d2/d19/d59/d89/d69/d9a/c14e 0 2024-02-16T19:29:26.116 INFO:tasks.workunit.client.1.smithi117.stdout:9/982: dread - dd/d45/d31/d63/dba/df0/f105 zero size 2024-02-16T19:29:26.118 INFO:tasks.workunit.client.1.smithi117.stdout:1/968: stat d2/d19/d59/d89/d69/l12e 0 2024-02-16T19:29:26.120 INFO:tasks.workunit.client.1.smithi117.stdout:9/983: creat dd/db5/d117/d130/f148 x:0 0 0 2024-02-16T19:29:26.120 INFO:tasks.workunit.client.1.smithi117.stdout:1/969: mknod d2/d19/d59/d89/d7b/d86/d138/c14f 0 2024-02-16T19:29:26.121 INFO:tasks.workunit.client.1.smithi117.stdout:1/970: fdatasync d2/d1d/d1f/f56 0 2024-02-16T19:29:26.121 INFO:tasks.workunit.client.1.smithi117.stdout:9/984: symlink dd/d45/d31/d63/dba/l149 0 2024-02-16T19:29:26.121 INFO:tasks.workunit.client.1.smithi117.stdout:1/971: fsync d2/d1d/d77/f103 0 2024-02-16T19:29:26.123 INFO:tasks.workunit.client.1.smithi117.stdout:1/972: fdatasync d2/f9d 0 2024-02-16T19:29:26.123 INFO:tasks.workunit.client.1.smithi117.stdout:9/985: truncate dd/d45/f3f 2821795 0 2024-02-16T19:29:26.124 INFO:tasks.workunit.client.1.smithi117.stdout:9/986: fsync dd/f13b 0 2024-02-16T19:29:26.125 INFO:tasks.workunit.client.1.smithi117.stdout:1/973: truncate d2/d19/d59/d76/f127 379493 0 2024-02-16T19:29:26.131 INFO:tasks.workunit.client.1.smithi117.stdout:9/987: dwrite dd/d47/d5b/d5f/d77/d7c/f11f [0,4194304] 0 2024-02-16T19:29:26.137 INFO:tasks.workunit.client.1.smithi117.stdout:1/974: write d2/d4/dbb/d10b/dcc/f149 [1257100,86918] 0 2024-02-16T19:29:26.138 INFO:tasks.workunit.client.1.smithi117.stdout:1/975: chown d2/d1d/d60/f5f 439 1 2024-02-16T19:29:26.139 INFO:tasks.workunit.client.1.smithi117.stdout:9/988: truncate dd/d47/d5b/fc6 767707 0 2024-02-16T19:29:26.139 INFO:tasks.workunit.client.1.smithi117.stdout:1/976: dread - d2/d19/d59/d89/d69/ffe zero size 2024-02-16T19:29:26.140 INFO:tasks.workunit.client.1.smithi117.stdout:9/989: readlink dd/d47/d5b/d5f/de6/le8 0 2024-02-16T19:29:26.141 INFO:tasks.workunit.client.1.smithi117.stdout:9/990: dread - dd/db5/dc4/feb zero size 2024-02-16T19:29:26.142 INFO:tasks.workunit.client.1.smithi117.stdout:1/977: creat d2/d19/d59/d76/f150 x:0 0 0 2024-02-16T19:29:26.143 INFO:tasks.workunit.client.1.smithi117.stdout:9/991: symlink dd/d45/d31/d63/d121/d9a/l14a 0 2024-02-16T19:29:26.145 INFO:tasks.workunit.client.1.smithi117.stdout:1/978: truncate d2/d1d/d1f/fc4 3255882 0 2024-02-16T19:29:26.148 INFO:tasks.workunit.client.0.smithi022.stdout:9/940: getdents db/d14/d19/d1a 0 2024-02-16T19:29:26.149 INFO:tasks.workunit.client.1.smithi117.stdout:9/992: dwrite dd/d47/f10c [0,4194304] 0 2024-02-16T19:29:26.150 INFO:tasks.workunit.client.0.smithi022.stdout:9/941: read - db/d14/d16/d5d/d65/d6b/d89/dd2/f13a zero size 2024-02-16T19:29:26.158 INFO:tasks.workunit.client.0.smithi022.stdout:3/950: rename d7/d12/d18/dd/d24/d29/fa3 to d7/d12/d18/dd/d24/d93/de9/def/f14d 0 2024-02-16T19:29:26.160 INFO:tasks.workunit.client.0.smithi022.stdout:3/951: write d7/d12/d18/dd/d24/d29/f2c [5092031,100331] 0 2024-02-16T19:29:26.162 INFO:tasks.workunit.client.1.smithi117.stdout:9/993: write dd/f4a [83433,33228] 0 2024-02-16T19:29:26.164 INFO:tasks.workunit.client.0.smithi022.stdout:9/942: creat db/d14/dc8/d128/f13d x:0 0 0 2024-02-16T19:29:26.166 INFO:tasks.workunit.client.0.smithi022.stdout:4/905: dwrite d4/d6/d14/dc7/f79 [0,4194304] 0 2024-02-16T19:29:26.170 INFO:tasks.workunit.client.0.smithi022.stdout:9/943: rename db/d18/c43 to db/d14/d19/c13e 0 2024-02-16T19:29:26.179 INFO:tasks.workunit.client.0.smithi022.stdout:4/906: creat d4/d6/d14/dc7/f122 x:0 0 0 2024-02-16T19:29:26.180 INFO:tasks.workunit.client.1.smithi117.stdout:9/994: write dd/d45/f3f [2042261,109007] 0 2024-02-16T19:29:26.180 INFO:tasks.workunit.client.1.smithi117.stdout:9/995: stat dd/d47/d48/ccd 0 2024-02-16T19:29:26.181 INFO:tasks.workunit.client.0.smithi022.stdout:4/907: truncate d4/d6/d14/d64/d4e/ffa 796274 0 2024-02-16T19:29:26.183 INFO:tasks.workunit.client.0.smithi022.stdout:4/908: write d4/d6/dd/f2c [4221409,31743] 0 2024-02-16T19:29:26.190 INFO:tasks.workunit.client.0.smithi022.stdout:9/944: dwrite db/d14/d16/d87/f8f [0,4194304] 0 2024-02-16T19:29:26.190 INFO:tasks.workunit.client.0.smithi022.stdout:4/909: dwrite d4/d6/dd/df/d10/d24/fbc [0,4194304] 0 2024-02-16T19:29:26.192 INFO:tasks.workunit.client.0.smithi022.stdout:9/945: stat db/d18/fb4 0 2024-02-16T19:29:26.193 INFO:tasks.workunit.client.1.smithi117.stdout:9/996: dwrite dd/d45/d73/f99 [4194304,4194304] 0 2024-02-16T19:29:26.193 INFO:tasks.workunit.client.0.smithi022.stdout:3/952: dread d7/d12/d18/dd/ff [0,4194304] 0 2024-02-16T19:29:26.194 INFO:tasks.workunit.client.0.smithi022.stdout:4/910: creat d4/d6/dd/df/d10/d55/f123 x:0 0 0 2024-02-16T19:29:26.197 INFO:tasks.workunit.client.0.smithi022.stdout:3/953: creat d7/d12/d18/dd/d24/d93/d122/f14e x:0 0 0 2024-02-16T19:29:26.198 INFO:tasks.workunit.client.0.smithi022.stdout:4/911: symlink d4/d6/dd/df/d10/d24/l124 0 2024-02-16T19:29:26.199 INFO:tasks.workunit.client.0.smithi022.stdout:3/954: fdatasync d7/d21/fc2 0 2024-02-16T19:29:26.201 INFO:tasks.workunit.client.0.smithi022.stdout:4/912: fsync d4/d6/dd/d22/da8/dc4/d86/dfd/f7e 0 2024-02-16T19:29:26.203 INFO:tasks.workunit.client.0.smithi022.stdout:4/913: dread - d4/d6/d14/dd2/fec zero size 2024-02-16T19:29:26.203 INFO:tasks.workunit.client.1.smithi117.stdout:9/997: truncate dd/d47/d5b/d5f/de6/f101 670362 0 2024-02-16T19:29:26.205 INFO:tasks.workunit.client.1.smithi117.stdout:9/998: dread - dd/d47/d5b/d5f/de6/f146 zero size 2024-02-16T19:29:26.205 INFO:tasks.workunit.client.0.smithi022.stdout:3/955: symlink d7/d12/d18/dd/d24/d58/d14b/d5a/d148/l14f 0 2024-02-16T19:29:26.207 INFO:tasks.workunit.client.0.smithi022.stdout:4/914: mknod d4/d6/dd/d22/da8/dc4/c125 0 2024-02-16T19:29:26.214 INFO:tasks.workunit.client.1.smithi117.stdout:9/999: creat dd/d45/d31/d63/d121/d9a/f14b x:0 0 0 2024-02-16T19:29:26.222 INFO:tasks.workunit.client.0.smithi022.stdout:4/915: write d4/d6/d14/d64/d4e/f8b [3658835,120136] 0 2024-02-16T19:29:26.224 INFO:tasks.workunit.client.0.smithi022.stdout:4/916: fdatasync d4/d6/dd/df/d10/de8/f110 0 2024-02-16T19:29:26.226 INFO:tasks.workunit.client.0.smithi022.stdout:3/956: dwrite d7/d12/d18/dd/d24/d29/f3f [0,4194304] 0 2024-02-16T19:29:26.228 INFO:tasks.workunit.client.0.smithi022.stdout:4/917: mkdir d4/d6/d14/d64/d4e/d54/d5a/d126 0 2024-02-16T19:29:26.230 INFO:tasks.workunit.client.0.smithi022.stdout:3/957: mkdir d7/d12/d18/dd/d24/d93/d150 0 2024-02-16T19:29:26.231 INFO:tasks.workunit.client.0.smithi022.stdout:3/958: rmdir d7/d12/d18/dd/d24/d58/d70 39 2024-02-16T19:29:26.237 INFO:tasks.workunit.client.0.smithi022.stdout:4/918: truncate d4/d6/dd/df/d10/d24/f8c 7488225 0 2024-02-16T19:29:26.238 INFO:tasks.workunit.client.0.smithi022.stdout:4/919: stat d4/d6/d14/f1d 0 2024-02-16T19:29:26.240 INFO:tasks.workunit.client.0.smithi022.stdout:4/920: chown d4/d6/d14/d64/d4e/ffa 830157504 1 2024-02-16T19:29:26.242 INFO:tasks.workunit.client.0.smithi022.stdout:3/959: dwrite d7/d12/d18/dac/f9d [0,4194304] 0 2024-02-16T19:29:26.243 INFO:tasks.workunit.client.0.smithi022.stdout:4/921: mkdir d4/d6/dd/d22/d83/dc5/d127 0 2024-02-16T19:29:26.248 INFO:tasks.workunit.client.0.smithi022.stdout:4/922: mkdir d4/d6/dd/df/d10/de8/d128 0 2024-02-16T19:29:26.251 INFO:tasks.workunit.client.0.smithi022.stdout:3/960: link d7/d12/d18/dd/d24/d58/lee d7/d12/d18/dd/d31/dbf/l151 0 2024-02-16T19:29:26.253 INFO:tasks.workunit.client.0.smithi022.stdout:4/923: rename d4/d6/d14/d64/d4e/d54/d5a/c9b to d4/d6/d14/dc7/d8d/dd1/c129 0 2024-02-16T19:29:26.254 INFO:tasks.workunit.client.0.smithi022.stdout:4/924: chown d4/d6/f100 5 1 2024-02-16T19:29:26.255 INFO:tasks.workunit.client.0.smithi022.stdout:4/925: chown d4/d6/dd/df/d10/d55/l97 1 1 2024-02-16T19:29:26.259 INFO:tasks.workunit.client.0.smithi022.stdout:4/926: link d4/d6/dd/df/d10/d24/d6b/le9 d4/d6/dd/d22/d83/d9e/db7/l12a 0 2024-02-16T19:29:26.260 INFO:tasks.workunit.client.0.smithi022.stdout:3/961: write d7/d12/d18/dd/d24/d29/fb5 [1974219,37149] 0 2024-02-16T19:29:26.266 INFO:tasks.workunit.client.0.smithi022.stdout:3/962: fsync d7/d12/d18/dd/d31/dbf/dc5/fe8 0 2024-02-16T19:29:26.267 INFO:tasks.workunit.client.0.smithi022.stdout:3/963: fdatasync d7/d12/d18/dd/d24/fb4 0 2024-02-16T19:29:26.269 INFO:tasks.workunit.client.0.smithi022.stdout:3/964: mknod d7/d12/d18/dd/d24/d93/de9/dc4/d141/c152 0 2024-02-16T19:29:26.277 INFO:tasks.workunit.client.0.smithi022.stdout:3/965: dwrite d7/d12/d18/dd/d24/d93/de9/f6c [0,4194304] 0 2024-02-16T19:29:26.280 INFO:tasks.workunit.client.0.smithi022.stdout:3/966: rmdir d7/d12/d18/dd/d31/dbf/dc5 39 2024-02-16T19:29:26.282 INFO:tasks.workunit.client.1.smithi117.stdout:1/979: sync 2024-02-16T19:29:26.284 INFO:tasks.workunit.client.0.smithi022.stdout:3/967: creat d7/d12/f153 x:0 0 0 2024-02-16T19:29:26.284 INFO:tasks.workunit.client.1.smithi117.stdout:1/980: readlink d2/d19/d59/d89/d49/l6c 0 2024-02-16T19:29:26.285 INFO:tasks.workunit.client.0.smithi022.stdout:3/968: write d7/d12/d18/dd/d24/d93/d122/f14e [425063,40427] 0 2024-02-16T19:29:26.287 INFO:tasks.workunit.client.1.smithi117.stdout:1/981: creat d2/d19/d59/d89/d7b/d86/f151 x:0 0 0 2024-02-16T19:29:26.288 INFO:tasks.workunit.client.0.smithi022.stdout:3/969: fdatasync d7/d12/d18/dd/d24/d58/d14b/f92 0 2024-02-16T19:29:26.291 INFO:tasks.workunit.client.0.smithi022.stdout:3/970: unlink d7/d12/d18/dd/d24/d58/d14b/f92 0 2024-02-16T19:29:26.292 INFO:tasks.workunit.client.1.smithi117.stdout:1/982: write d2/d1d/d60/fd3 [380665,91750] 0 2024-02-16T19:29:26.293 INFO:tasks.workunit.client.0.smithi022.stdout:3/971: mkdir d7/d12/d18/dd/d31/d7f/d154 0 2024-02-16T19:29:26.294 INFO:tasks.workunit.client.1.smithi117.stdout:1/983: symlink d2/d19/d59/d89/d69/d80/l152 0 2024-02-16T19:29:26.296 INFO:tasks.workunit.client.0.smithi022.stdout:3/972: link d7/d12/f153 d7/d12/d18/dd/d24/d93/de9/f155 0 2024-02-16T19:29:26.298 INFO:tasks.workunit.client.0.smithi022.stdout:3/973: fsync d7/d12/d18/dd/d24/d58/d70/f8e 0 2024-02-16T19:29:26.298 INFO:tasks.workunit.client.0.smithi022.stdout:3/974: stat d7 0 2024-02-16T19:29:26.299 INFO:tasks.workunit.client.1.smithi117.stdout:1/984: getdents d2/d4/dbb/d7f/dc8 0 2024-02-16T19:29:26.301 INFO:tasks.workunit.client.0.smithi022.stdout:3/975: truncate d7/d12/d18/dd/d24/d53/d116/f94 2065407 0 2024-02-16T19:29:26.304 INFO:tasks.workunit.client.1.smithi117.stdout:1/985: write d2/d1d/d1f/f4e [921527,9770] 0 2024-02-16T19:29:26.305 INFO:tasks.workunit.client.0.smithi022.stdout:3/976: write d7/f105 [1255707,44441] 0 2024-02-16T19:29:26.308 INFO:tasks.workunit.client.0.smithi022.stdout:3/977: mknod d7/d12/d18/dd/d24/d53/d116/de6/d11a/c156 0 2024-02-16T19:29:26.311 INFO:tasks.workunit.client.1.smithi117.stdout:1/986: dwrite d2/d19/d59/d89/d69/d80/d8e/f90 [0,4194304] 0 2024-02-16T19:29:26.313 INFO:tasks.workunit.client.1.smithi117.stdout:1/987: truncate d2/d4/dbb/d10b/dcc/f149 1946811 0 2024-02-16T19:29:26.314 INFO:tasks.workunit.client.0.smithi022.stdout:3/978: write d7/d12/d18/dd/d31/dbf/dc5/f117 [704807,101067] 0 2024-02-16T19:29:26.315 INFO:tasks.workunit.client.0.smithi022.stdout:2/905: sync 2024-02-16T19:29:26.316 INFO:tasks.workunit.client.0.smithi022.stdout:3/979: creat d7/d21/f157 x:0 0 0 2024-02-16T19:29:26.316 INFO:tasks.workunit.client.1.smithi117.stdout:1/988: dread d2/d19/d59/d89/d49/fcd [0,4194304] 0 2024-02-16T19:29:26.319 INFO:tasks.workunit.client.0.smithi022.stdout:3/980: creat d7/d12/d18/dac/dd7/f158 x:0 0 0 2024-02-16T19:29:26.326 INFO:tasks.workunit.client.0.smithi022.stdout:2/906: write d0/d2/d18/d1f/d6c/f9e [627935,123766] 0 2024-02-16T19:29:26.327 INFO:tasks.workunit.client.1.smithi117.stdout:1/989: truncate d2/d4/dbb/d10b/dcc/f136 231446 0 2024-02-16T19:29:26.328 INFO:tasks.workunit.client.0.smithi022.stdout:2/907: write d0/d32/f94 [3298325,23813] 0 2024-02-16T19:29:26.328 INFO:tasks.workunit.client.0.smithi022.stdout:3/981: write d7/d12/d18/dd/d31/d7f/fa5 [1897578,76589] 0 2024-02-16T19:29:26.330 INFO:tasks.workunit.client.1.smithi117.stdout:1/990: creat d2/d19/d59/d89/d7b/d86/d138/f153 x:0 0 0 2024-02-16T19:29:26.331 INFO:tasks.workunit.client.0.smithi022.stdout:2/908: truncate d0/d2/d18/d1f/d6c/d99/fc7 388048 0 2024-02-16T19:29:26.332 INFO:tasks.workunit.client.1.smithi117.stdout:1/991: mknod d2/dab/c154 0 2024-02-16T19:29:26.334 INFO:tasks.workunit.client.1.smithi117.stdout:1/992: creat d2/dab/f155 x:0 0 0 2024-02-16T19:29:26.334 INFO:tasks.workunit.client.0.smithi022.stdout:2/909: fdatasync d0/d2/d18/d1f/d6c/d99/ff6 0 2024-02-16T19:29:26.335 INFO:tasks.workunit.client.0.smithi022.stdout:3/982: dwrite d7/d12/d18/d4b/db6/f13f [0,4194304] 0 2024-02-16T19:29:26.338 INFO:tasks.workunit.client.0.smithi022.stdout:2/910: creat d0/d2/dce/f128 x:0 0 0 2024-02-16T19:29:26.338 INFO:tasks.workunit.client.1.smithi117.stdout:1/993: chown d2/d19/d59/d89/d8d/l130 1 1 2024-02-16T19:29:26.340 INFO:tasks.workunit.client.0.smithi022.stdout:3/983: unlink d7/d12/d18/dd/d24/d58/d70/l86 0 2024-02-16T19:29:26.344 INFO:tasks.workunit.client.1.smithi117.stdout:1/994: mknod d2/d4/dbb/d7f/c156 0 2024-02-16T19:29:26.345 INFO:tasks.workunit.client.1.smithi117.stdout:1/995: write d2/d1d/d77/f103 [1824818,70300] 0 2024-02-16T19:29:26.346 INFO:tasks.workunit.client.0.smithi022.stdout:3/984: rename d7/d12/d18/dd/d31/cca to d7/d12/d18/dd/d31/dbf/dc5/c159 0 2024-02-16T19:29:26.346 INFO:tasks.workunit.client.1.smithi117.stdout:1/996: write d2/d4/f13e [1790373,52959] 0 2024-02-16T19:29:26.349 INFO:tasks.workunit.client.0.smithi022.stdout:2/911: creat d0/d2/d18/d19/d90/f129 x:0 0 0 2024-02-16T19:29:26.350 INFO:tasks.workunit.client.0.smithi022.stdout:3/985: dread d7/d12/d18/dd/d31/fed [0,4194304] 0 2024-02-16T19:29:26.350 INFO:tasks.workunit.client.1.smithi117.stdout:1/997: creat d2/d19/d59/d89/d69/d9a/f157 x:0 0 0 2024-02-16T19:29:26.350 INFO:tasks.workunit.client.1.smithi117.stdout:1/998: fsync d2/d19/d59/f97 0 2024-02-16T19:29:26.352 INFO:tasks.workunit.client.1.smithi117.stdout:1/999: symlink d2/d19/d59/d89/d69/l158 0 2024-02-16T19:29:26.356 INFO:tasks.workunit.client.0.smithi022.stdout:2/912: dwrite d0/f10b [0,4194304] 0 2024-02-16T19:29:26.356 INFO:tasks.workunit.client.1.smithi117.stderr:+ rm -rf -- ./tmp.hytcJGc2wO 2024-02-16T19:29:26.358 INFO:tasks.workunit.client.0.smithi022.stdout:3/986: dwrite d7/d12/d18/dd/d24/d58/d70/f8e [0,4194304] 0 2024-02-16T19:29:26.360 INFO:tasks.workunit.client.0.smithi022.stdout:3/987: chown d7/d12/d18/dd/d24/d93/f11e 837048 1 2024-02-16T19:29:26.365 INFO:tasks.workunit.client.0.smithi022.stdout:2/913: creat d0/d2/d18/d19/ddf/d10c/d117/f12a x:0 0 0 2024-02-16T19:29:26.368 INFO:tasks.workunit.client.0.smithi022.stdout:3/988: symlink d7/d12/d18/dd/d31/l15a 0 2024-02-16T19:29:26.370 INFO:tasks.workunit.client.0.smithi022.stdout:2/914: mknod d0/d2/d2e/d82/df0/d120/c12b 0 2024-02-16T19:29:26.372 INFO:tasks.workunit.client.0.smithi022.stdout:3/989: mkdir d7/d12/d18/dd/d24/d93/de9/dc4/d15b 0 2024-02-16T19:29:26.378 INFO:tasks.workunit.client.0.smithi022.stdout:2/915: write d0/d2/d18/d38/db6/fd9 [68225,62114] 0 2024-02-16T19:29:26.382 INFO:tasks.workunit.client.0.smithi022.stdout:2/916: truncate d0/d2/d18/d19/ddf/feb 1574121 0 2024-02-16T19:29:26.384 INFO:tasks.workunit.client.0.smithi022.stdout:3/990: dwrite d7/d12/d18/dac/f4c [4194304,4194304] 0 2024-02-16T19:29:26.385 INFO:tasks.workunit.client.0.smithi022.stdout:2/917: creat d0/d2/d2e/d73/f12c x:0 0 0 2024-02-16T19:29:26.389 INFO:tasks.workunit.client.0.smithi022.stdout:2/918: mkdir d0/d2/d18/d1f/d6c/d99/d12d 0 2024-02-16T19:29:26.389 INFO:tasks.workunit.client.0.smithi022.stdout:3/991: dread d7/d12/d18/dd/d24/d93/de9/f6c [0,4194304] 0 2024-02-16T19:29:26.392 INFO:tasks.workunit.client.0.smithi022.stdout:3/992: rmdir d7/d21 39 2024-02-16T19:29:26.396 INFO:tasks.workunit.client.0.smithi022.stdout:3/993: creat d7/d12/d18/dd/d24/d93/de9/dc4/f15c x:0 0 0 2024-02-16T19:29:26.396 INFO:tasks.workunit.client.0.smithi022.stdout:3/994: readlink d7/d12/d18/d4b/lbc 0 2024-02-16T19:29:26.401 INFO:tasks.workunit.client.0.smithi022.stdout:3/995: mknod d7/d21/dd9/d11b/c15d 0 2024-02-16T19:29:26.415 INFO:tasks.workunit.client.0.smithi022.stdout:2/919: dread d0/d2/d18/d19/d90/fa3 [4194304,4194304] 0 2024-02-16T19:29:26.417 INFO:tasks.workunit.client.0.smithi022.stdout:3/996: dread d7/d21/dd9/fe3 [0,4194304] 0 2024-02-16T19:29:26.418 INFO:tasks.workunit.client.0.smithi022.stdout:2/920: truncate d0/d2/d18/f28 108007 0 2024-02-16T19:29:26.422 INFO:tasks.workunit.client.0.smithi022.stdout:3/997: unlink d7/d12/d18/dd/d24/d58/d70/cfc 0 2024-02-16T19:29:26.427 INFO:tasks.workunit.client.0.smithi022.stdout:3/998: dwrite d7/d12/d18/dd/f7d [0,4194304] 0 2024-02-16T19:29:26.433 INFO:tasks.workunit.client.0.smithi022.stdout:2/921: truncate d0/d2/d2e/d73/fd8 8891 0 2024-02-16T19:29:26.434 INFO:tasks.workunit.client.0.smithi022.stdout:3/999: dwrite d7/d12/f14 [4194304,4194304] 0 2024-02-16T19:29:26.449 INFO:tasks.workunit.client.0.smithi022.stdout:2/922: dwrite d0/d2/d18/d1f/f7b [0,4194304] 0 2024-02-16T19:29:26.457 INFO:tasks.workunit.client.0.smithi022.stdout:7/991: sync 2024-02-16T19:29:26.457 INFO:tasks.workunit.client.0.smithi022.stdout:2/923: dwrite d0/d2/dde/f127 [0,4194304] 0 2024-02-16T19:29:26.460 INFO:tasks.workunit.client.0.smithi022.stdout:2/924: readlink d0/d2/d2e/d82/da1/l113 0 2024-02-16T19:29:26.492 INFO:tasks.workunit.client.0.smithi022.stdout:7/992: mkdir d9/da/d74/d131/d14e/d151 0 2024-02-16T19:29:26.495 INFO:tasks.workunit.client.0.smithi022.stdout:7/993: truncate d9/da/d1f/d93/fa2 702289 0 2024-02-16T19:29:26.501 INFO:tasks.workunit.client.0.smithi022.stdout:7/994: symlink d9/da/d14c/l152 0 2024-02-16T19:29:26.501 INFO:tasks.workunit.client.0.smithi022.stdout:2/925: dread d0/d2/d18/d1f/f4d [0,4194304] 0 2024-02-16T19:29:26.505 INFO:tasks.workunit.client.0.smithi022.stdout:2/926: fsync d0/d2/d18/d1f/f4d 0 2024-02-16T19:29:26.507 INFO:tasks.workunit.client.0.smithi022.stdout:2/927: readlink d0/d2/d2e/d82/df0/df3/l107 0 2024-02-16T19:29:26.507 INFO:tasks.workunit.client.0.smithi022.stdout:4/927: sync 2024-02-16T19:29:26.508 INFO:tasks.workunit.client.0.smithi022.stdout:6/941: sync 2024-02-16T19:29:26.508 INFO:tasks.workunit.client.0.smithi022.stdout:1/999: sync 2024-02-16T19:29:26.508 INFO:tasks.workunit.client.0.smithi022.stdout:9/946: sync 2024-02-16T19:29:26.508 INFO:tasks.workunit.client.0.smithi022.stdout:7/995: dwrite d9/f13c [0,4194304] 0 2024-02-16T19:29:26.510 INFO:tasks.workunit.client.0.smithi022.stdout:6/942: dread - d1/d8/da/d29/deb/dfd/f62 zero size 2024-02-16T19:29:26.510 INFO:tasks.workunit.client.0.smithi022.stdout:2/928: rmdir d0/d2/d2e 39 2024-02-16T19:29:26.513 INFO:tasks.workunit.client.0.smithi022.stdout:9/947: creat db/d14/d16/dec/dee/f13f x:0 0 0 2024-02-16T19:29:26.515 INFO:tasks.workunit.client.0.smithi022.stdout:7/996: creat d9/da/d3b/f153 x:0 0 0 2024-02-16T19:29:26.517 INFO:tasks.workunit.client.0.smithi022.stdout:2/929: chown d0/d2/d2e/d82/cbf 15 1 2024-02-16T19:29:26.518 INFO:tasks.workunit.client.0.smithi022.stdout:4/928: symlink d4/d6/d14/d64/d4e/d54/d121/l12b 0 2024-02-16T19:29:26.520 INFO:tasks.workunit.client.0.smithi022.stdout:7/997: creat d9/da/d1f/f154 x:0 0 0 2024-02-16T19:29:26.520 INFO:tasks.workunit.client.0.smithi022.stdout:7/998: write d9/da/f62 [5062459,11505] 0 2024-02-16T19:29:26.521 INFO:tasks.workunit.client.0.smithi022.stdout:6/943: creat d1/d8/dac/d101/db3/db0/d11e/f140 x:0 0 0 2024-02-16T19:29:26.525 INFO:tasks.workunit.client.0.smithi022.stdout:2/930: symlink d0/d5d/l12e 0 2024-02-16T19:29:26.526 INFO:tasks.workunit.client.0.smithi022.stdout:2/931: chown d0/d2/d2e/d82/da1/d10d 2095 1 2024-02-16T19:29:26.531 INFO:tasks.workunit.client.0.smithi022.stdout:2/932: dwrite d0/d2/d18/f34 [0,4194304] 0 2024-02-16T19:29:26.531 INFO:tasks.workunit.client.0.smithi022.stdout:6/944: truncate d1/d8/d81/fd9 4682478 0 2024-02-16T19:29:26.533 INFO:tasks.workunit.client.0.smithi022.stdout:2/933: write d0/d2/d18/d19/d40/fa5 [3448167,79989] 0 2024-02-16T19:29:26.535 INFO:tasks.workunit.client.0.smithi022.stdout:4/929: mkdir d4/d6/d14/d64/d4e/d12c 0 2024-02-16T19:29:26.538 INFO:tasks.workunit.client.0.smithi022.stdout:7/999: mkdir d9/da/d74/d131/d14e/d151/d155 0 2024-02-16T19:29:26.540 INFO:tasks.workunit.client.0.smithi022.stdout:6/945: creat d1/d8/da/d29/deb/dfd/d8b/f141 x:0 0 0 2024-02-16T19:29:26.544 INFO:tasks.workunit.client.0.smithi022.stdout:9/948: rename db/d14/d16/d27/d2e/f132 to db/d14/d16/d5d/d65/d6b/dd5/dd9/ddd/f140 0 2024-02-16T19:29:26.545 INFO:tasks.workunit.client.0.smithi022.stdout:2/934: symlink d0/d2/d18/d19/d40/l12f 0 2024-02-16T19:29:26.545 INFO:tasks.workunit.client.0.smithi022.stdout:2/935: stat d0/d2/d18/d19 0 2024-02-16T19:29:26.558 INFO:tasks.workunit.client.0.smithi022.stdout:9/949: rmdir db/d14/d16/d28/daa/d137 0 2024-02-16T19:29:26.561 INFO:tasks.workunit.client.0.smithi022.stdout:9/950: truncate db/d14/d16/d21/d4a/fed 600559 0 2024-02-16T19:29:26.564 INFO:tasks.workunit.client.0.smithi022.stdout:9/951: truncate db/d14/dc8/f12a 7755241 0 2024-02-16T19:29:26.565 INFO:tasks.workunit.client.0.smithi022.stdout:4/930: dwrite d4/d6/d14/f2e [4194304,4194304] 0 2024-02-16T19:29:26.566 INFO:tasks.workunit.client.0.smithi022.stdout:9/952: readlink db/d14/d19/da8/l75 0 2024-02-16T19:29:26.566 INFO:tasks.workunit.client.0.smithi022.stdout:6/946: write d1/d8/d3c/d77/fd1 [162621,22501] 0 2024-02-16T19:29:26.567 INFO:tasks.workunit.client.0.smithi022.stdout:4/931: dread - d4/d6/dd/df/d10/d24/fa6 zero size 2024-02-16T19:29:26.569 INFO:tasks.workunit.client.0.smithi022.stdout:2/936: write d0/d5d/f7a [515216,16413] 0 2024-02-16T19:29:26.573 INFO:tasks.workunit.client.0.smithi022.stdout:4/932: mkdir d4/d6/d14/dc7/d6a/d12d 0 2024-02-16T19:29:26.575 INFO:tasks.workunit.client.0.smithi022.stdout:9/953: mkdir db/d14/d16/d21/d141 0 2024-02-16T19:29:26.575 INFO:tasks.workunit.client.0.smithi022.stdout:4/933: dread - d4/d6/dd/d22/d83/d9e/db7/f117 zero size 2024-02-16T19:29:26.578 INFO:tasks.workunit.client.0.smithi022.stdout:6/947: symlink d1/d8/da/d29/l142 0 2024-02-16T19:29:26.579 INFO:tasks.workunit.client.0.smithi022.stdout:2/937: truncate d0/d32/d3b/f9a 194134 0 2024-02-16T19:29:26.580 INFO:tasks.workunit.client.0.smithi022.stdout:4/934: dread d4/d6/dd/df/d10/d24/fbc [0,4194304] 0 2024-02-16T19:29:26.582 INFO:tasks.workunit.client.0.smithi022.stdout:4/935: dread d4/f47 [0,4194304] 0 2024-02-16T19:29:26.582 INFO:tasks.workunit.client.0.smithi022.stdout:9/954: creat db/d14/d16/d27/d8b/f142 x:0 0 0 2024-02-16T19:29:26.585 INFO:tasks.workunit.client.0.smithi022.stdout:2/938: creat d0/d2/dde/f130 x:0 0 0 2024-02-16T19:29:26.586 INFO:tasks.workunit.client.0.smithi022.stdout:2/939: chown d0/d2/d18/d19/f2f 3729683 1 2024-02-16T19:29:26.590 INFO:tasks.workunit.client.0.smithi022.stdout:9/955: readlink db/d14/dc8/dc9/d105/l54 0 2024-02-16T19:29:26.591 INFO:tasks.workunit.client.0.smithi022.stdout:2/940: dread d0/d2/d18/d19/d90/fa3 [8388608,4194304] 0 2024-02-16T19:29:26.591 INFO:tasks.workunit.client.0.smithi022.stdout:4/936: fsync d4/d6/dd/df/d10/de8/fee 0 2024-02-16T19:29:26.592 INFO:tasks.workunit.client.0.smithi022.stdout:2/941: read d0/d32/f94 [246176,76380] 0 2024-02-16T19:29:26.594 INFO:tasks.workunit.client.0.smithi022.stdout:4/937: dread - d4/d6/d14/d64/d4e/d54/fff zero size 2024-02-16T19:29:26.598 INFO:tasks.workunit.client.0.smithi022.stdout:6/948: dwrite d1/d8/f8f [0,4194304] 0 2024-02-16T19:29:26.599 INFO:tasks.workunit.client.0.smithi022.stdout:2/942: creat d0/d32/d3b/de5/f131 x:0 0 0 2024-02-16T19:29:26.602 INFO:tasks.workunit.client.0.smithi022.stdout:6/949: truncate d1/d8/d3c/f44 4680552 0 2024-02-16T19:29:26.603 INFO:tasks.workunit.client.0.smithi022.stdout:2/943: fsync d0/d5d/f110 0 2024-02-16T19:29:26.604 INFO:tasks.workunit.client.0.smithi022.stdout:9/956: dwrite db/d14/d16/d21/fcc [0,4194304] 0 2024-02-16T19:29:26.606 INFO:tasks.workunit.client.0.smithi022.stdout:9/957: dread - db/d14/d16/f127 zero size 2024-02-16T19:29:26.608 INFO:tasks.workunit.client.0.smithi022.stdout:4/938: truncate d4/d6/d14/dc7/d6a/fea 615979 0 2024-02-16T19:29:26.612 INFO:tasks.workunit.client.0.smithi022.stdout:6/950: creat d1/d8/d23/dc6/ddb/f143 x:0 0 0 2024-02-16T19:29:26.613 INFO:tasks.workunit.client.0.smithi022.stdout:6/951: fdatasync d1/d8/dac/d101/db3/f129 0 2024-02-16T19:29:26.615 INFO:tasks.workunit.client.0.smithi022.stdout:9/958: creat db/d14/d16/dec/d119/f143 x:0 0 0 2024-02-16T19:29:26.615 INFO:tasks.workunit.client.0.smithi022.stdout:6/952: chown d1/l26 17 1 2024-02-16T19:29:26.615 INFO:tasks.workunit.client.0.smithi022.stdout:4/939: dread d4/d6/d14/dc7/f79 [0,4194304] 0 2024-02-16T19:29:26.616 INFO:tasks.workunit.client.0.smithi022.stdout:9/959: fsync db/d14/d16/d5d/d65/f131 0 2024-02-16T19:29:26.619 INFO:tasks.workunit.client.0.smithi022.stdout:2/944: truncate d0/f20 4624213 0 2024-02-16T19:29:26.621 INFO:tasks.workunit.client.0.smithi022.stdout:6/953: symlink d1/d8/da/d7b/dae/db6/l144 0 2024-02-16T19:29:26.625 INFO:tasks.workunit.client.0.smithi022.stdout:4/940: truncate d4/d6/dd/d22/da8/dc4/f82 1753482 0 2024-02-16T19:29:26.628 INFO:tasks.workunit.client.0.smithi022.stdout:6/954: dread d1/d8/da/d29/d61/f9e [0,4194304] 0 2024-02-16T19:29:26.629 INFO:tasks.workunit.client.0.smithi022.stdout:9/960: creat db/d14/d16/d21/d37/f144 x:0 0 0 2024-02-16T19:29:26.631 INFO:tasks.workunit.client.0.smithi022.stdout:4/941: chown d4/d6/d14/dc7/d6a/d120/ccc 31 1 2024-02-16T19:29:26.635 INFO:tasks.workunit.client.0.smithi022.stdout:2/945: rename d0/d2/d2e/d82/df0/dff to d0/d2/d2e/d132 0 2024-02-16T19:29:26.640 INFO:tasks.workunit.client.0.smithi022.stdout:2/946: dread d0/d2/d18/d19/d90/fbd [8388608,4194304] 0 2024-02-16T19:29:26.644 INFO:tasks.workunit.client.0.smithi022.stdout:9/961: dread db/d18/f73 [0,4194304] 0 2024-02-16T19:29:26.644 INFO:tasks.workunit.client.0.smithi022.stdout:4/942: mkdir d4/d6/dd/df/d10/de8/d128/d12e 0 2024-02-16T19:29:26.646 INFO:tasks.workunit.client.0.smithi022.stdout:2/947: creat d0/d2/d2e/d82/df0/d120/f133 x:0 0 0 2024-02-16T19:29:26.649 INFO:tasks.workunit.client.0.smithi022.stdout:9/962: fdatasync db/d14/d16/d5d/f7e 0 2024-02-16T19:29:26.652 INFO:tasks.workunit.client.0.smithi022.stdout:4/943: truncate d4/d6/dd/d22/d30/fd0 5169779 0 2024-02-16T19:29:26.654 INFO:tasks.workunit.client.0.smithi022.stdout:2/948: mknod d0/d2/d2e/d82/c134 0 2024-02-16T19:29:26.655 INFO:tasks.workunit.client.0.smithi022.stdout:2/949: write d0/d2/d2e/d73/f12c [473121,104485] 0 2024-02-16T19:29:26.658 INFO:tasks.workunit.client.0.smithi022.stdout:6/955: getdents d1/d8/da/d29/deb 0 2024-02-16T19:29:26.659 INFO:tasks.workunit.client.0.smithi022.stdout:6/956: fsync d1/d8/da/d29/deb/dfd/f2c 0 2024-02-16T19:29:26.664 INFO:tasks.workunit.client.0.smithi022.stdout:4/944: dread d4/f112 [0,4194304] 0 2024-02-16T19:29:26.665 INFO:tasks.workunit.client.0.smithi022.stdout:4/945: fsync d4/f23 0 2024-02-16T19:29:26.666 INFO:tasks.workunit.client.0.smithi022.stdout:2/950: write d0/d2/dce/ff8 [135364,9078] 0 2024-02-16T19:29:26.671 INFO:tasks.workunit.client.0.smithi022.stdout:9/963: mknod db/d14/c145 0 2024-02-16T19:29:26.673 INFO:tasks.workunit.client.0.smithi022.stdout:6/957: dread d1/d8/da/d7b/dae/dbb/fbd [0,4194304] 0 2024-02-16T19:29:26.674 INFO:tasks.workunit.client.0.smithi022.stdout:2/951: dwrite d0/d2/dde/f130 [0,4194304] 0 2024-02-16T19:29:26.675 INFO:tasks.workunit.client.0.smithi022.stdout:6/958: chown d1/d8/da/d29/deb/dfd/d48/f74 0 1 2024-02-16T19:29:26.677 INFO:tasks.workunit.client.0.smithi022.stdout:9/964: mkdir db/d14/dc8/dc9/d105/d146 0 2024-02-16T19:29:26.681 INFO:tasks.workunit.client.0.smithi022.stdout:6/959: symlink d1/d8/da/d29/deb/dfd/d85/d9c/d12a/l145 0 2024-02-16T19:29:26.682 INFO:tasks.workunit.client.0.smithi022.stdout:6/960: readlink d1/d8/d23/db1/l108 0 2024-02-16T19:29:26.682 INFO:tasks.workunit.client.0.smithi022.stdout:9/965: unlink db/d18/d103/c106 0 2024-02-16T19:29:26.684 INFO:tasks.workunit.client.0.smithi022.stdout:4/946: dwrite d4/d6/dd/f1e [0,4194304] 0 2024-02-16T19:29:26.686 INFO:tasks.workunit.client.0.smithi022.stdout:2/952: creat d0/d2/d2e/d82/d8a/dfd/f135 x:0 0 0 2024-02-16T19:29:26.686 INFO:tasks.workunit.client.0.smithi022.stdout:6/961: rmdir d1/d8/da/d29 39 2024-02-16T19:29:26.693 INFO:tasks.workunit.client.0.smithi022.stdout:2/953: dread d0/d2/d18/d1f/f4d [0,4194304] 0 2024-02-16T19:29:26.694 INFO:tasks.workunit.client.0.smithi022.stdout:4/947: mkdir d4/d6/dd/d22/d83/d113/d11f/d12f 0 2024-02-16T19:29:26.695 INFO:tasks.workunit.client.0.smithi022.stdout:2/954: write d0/d2/d18/d104/f9b [1295818,102100] 0 2024-02-16T19:29:26.697 INFO:tasks.workunit.client.0.smithi022.stdout:2/955: write d0/d2/d18/d19/d40/dc9/ff9 [352690,62229] 0 2024-02-16T19:29:26.709 INFO:tasks.workunit.client.0.smithi022.stdout:9/966: truncate db/d14/d19/fcf 720428 0 2024-02-16T19:29:26.709 INFO:tasks.workunit.client.0.smithi022.stdout:6/962: dread d1/d8/d3c/f44 [0,4194304] 0 2024-02-16T19:29:26.712 INFO:tasks.workunit.client.0.smithi022.stdout:6/963: stat d1/l15 0 2024-02-16T19:29:26.712 INFO:tasks.workunit.client.0.smithi022.stdout:2/956: dread d0/d2/d18/d19/d40/fa5 [0,4194304] 0 2024-02-16T19:29:26.715 INFO:tasks.workunit.client.0.smithi022.stdout:6/964: mknod d1/d8/da/d7b/dae/db6/c146 0 2024-02-16T19:29:26.716 INFO:tasks.workunit.client.0.smithi022.stdout:4/948: dwrite d4/d6/dd/d22/d30/fd0 [0,4194304] 0 2024-02-16T19:29:26.717 INFO:tasks.workunit.client.0.smithi022.stdout:6/965: write d1/d8/dac/d101/db3/db0/d11e/f140 [252615,15994] 0 2024-02-16T19:29:26.717 INFO:tasks.workunit.client.0.smithi022.stdout:2/957: unlink d0/d2/d18/d38/lad 0 2024-02-16T19:29:26.718 INFO:tasks.workunit.client.0.smithi022.stdout:4/949: stat d4/d6/d3a/l8e 0 2024-02-16T19:29:26.719 INFO:tasks.workunit.client.0.smithi022.stdout:2/958: chown d0/d2/d18/d1f/d6c 7650 1 2024-02-16T19:29:26.720 INFO:tasks.workunit.client.0.smithi022.stdout:6/966: symlink d1/d8/d81/l147 0 2024-02-16T19:29:26.723 INFO:tasks.workunit.client.0.smithi022.stdout:2/959: rmdir d0/d2/d18/d1f/d6c 39 2024-02-16T19:29:26.728 INFO:tasks.workunit.client.0.smithi022.stdout:2/960: mknod d0/d2/d2e/d82/df0/d120/c136 0 2024-02-16T19:29:26.729 INFO:tasks.workunit.client.0.smithi022.stdout:2/961: fsync d0/d2/d18/d19/d40/fcb 0 2024-02-16T19:29:26.734 INFO:tasks.workunit.client.0.smithi022.stdout:2/962: read d0/f20 [3753903,106835] 0 2024-02-16T19:29:26.735 INFO:tasks.workunit.client.0.smithi022.stdout:2/963: fdatasync d0/d5d/dc4/fdc 0 2024-02-16T19:29:26.740 INFO:tasks.workunit.client.0.smithi022.stdout:4/950: write d4/d6/dd/f2c [640311,123764] 0 2024-02-16T19:29:26.742 INFO:tasks.workunit.client.0.smithi022.stdout:2/964: rename d0/f37 to d0/d2/d18/d19/d40/d55/f137 0 2024-02-16T19:29:26.744 INFO:tasks.workunit.client.0.smithi022.stdout:6/967: write d1/d8/da/d29/deb/dfd/d85/f50 [850482,28396] 0 2024-02-16T19:29:26.745 INFO:tasks.workunit.client.0.smithi022.stdout:6/968: chown d1/d8/da/d29/deb/dfd/d48/de4 498074463 1 2024-02-16T19:29:26.747 INFO:tasks.workunit.client.0.smithi022.stdout:4/951: rename d4/d6/d14/d64/d4e to d4/d6/d14/dc7/d8d/d130 0 2024-02-16T19:29:26.749 INFO:tasks.workunit.client.0.smithi022.stdout:6/969: mkdir d1/d8/d23/d148 0 2024-02-16T19:29:26.757 INFO:tasks.workunit.client.0.smithi022.stdout:4/952: link d4/d6/dd/d22/d30/l9d d4/d6/d14/dc7/d8d/d130/d54/d5a/d126/l131 0 2024-02-16T19:29:26.767 INFO:tasks.workunit.client.0.smithi022.stdout:4/953: mkdir d4/d6/d14/d64/d132 0 2024-02-16T19:29:26.770 INFO:tasks.workunit.client.0.smithi022.stdout:6/970: dwrite d1/d8/d3c/d4d/fc2 [0,4194304] 0 2024-02-16T19:29:26.770 INFO:tasks.workunit.client.0.smithi022.stdout:4/954: fsync d4/d6/d14/dc7/f79 0 2024-02-16T19:29:26.778 INFO:tasks.workunit.client.0.smithi022.stdout:2/965: truncate d0/d2/d18/d19/d40/fa5 3666928 0 2024-02-16T19:29:26.779 INFO:tasks.workunit.client.0.smithi022.stdout:6/971: creat d1/d8/da/d29/deb/dfd/d85/d56/f149 x:0 0 0 2024-02-16T19:29:26.782 INFO:tasks.workunit.client.0.smithi022.stdout:6/972: mkdir d1/ddc/d14a 0 2024-02-16T19:29:26.783 INFO:tasks.workunit.client.0.smithi022.stdout:6/973: chown d1/l26 23104707 1 2024-02-16T19:29:26.784 INFO:tasks.workunit.client.0.smithi022.stdout:2/966: symlink d0/d2/d18/d19/d40/d55/l138 0 2024-02-16T19:29:26.787 INFO:tasks.workunit.client.0.smithi022.stdout:4/955: dwrite d4/f47 [0,4194304] 0 2024-02-16T19:29:26.791 INFO:tasks.workunit.client.0.smithi022.stdout:2/967: unlink d0/d2/d2e/fba 0 2024-02-16T19:29:26.800 INFO:tasks.workunit.client.0.smithi022.stdout:6/974: getdents d1/d8/dac/d101/db3/db0/d11e 0 2024-02-16T19:29:26.800 INFO:tasks.workunit.client.0.smithi022.stdout:6/975: readlink d1/ld2 0 2024-02-16T19:29:26.810 INFO:tasks.workunit.client.0.smithi022.stdout:4/956: dwrite d4/d6/d14/dc7/d6a/d120/fe5 [0,4194304] 0 2024-02-16T19:29:26.813 INFO:tasks.workunit.client.0.smithi022.stdout:2/968: dwrite d0/d2/d18/d1f/d6c/fe2 [0,4194304] 0 2024-02-16T19:29:26.822 INFO:tasks.workunit.client.0.smithi022.stdout:6/976: dwrite d1/d8/da/d29/deb/dfd/d48/d51/f59 [4194304,4194304] 0 2024-02-16T19:29:26.823 INFO:tasks.workunit.client.0.smithi022.stdout:2/969: rename d0/d2/d2e/d82/df0/d120/c136 to d0/d2/d18/d38/db6/c139 0 2024-02-16T19:29:26.826 INFO:tasks.workunit.client.0.smithi022.stdout:2/970: dread d0/d2/f54 [0,4194304] 0 2024-02-16T19:29:26.829 INFO:tasks.workunit.client.0.smithi022.stdout:4/957: truncate d4/d6/dd/d22/da8/dc4/d86/dfd/f7d 1326227 0 2024-02-16T19:29:26.833 INFO:tasks.workunit.client.0.smithi022.stdout:2/971: mkdir d0/d2/d18/d19/ddf/d10c/d13a 0 2024-02-16T19:29:26.836 INFO:tasks.workunit.client.0.smithi022.stdout:4/958: getdents d4/d6/d14/dc7/d8d/d130/d54/dba 0 2024-02-16T19:29:26.842 INFO:tasks.workunit.client.0.smithi022.stdout:4/959: dwrite d4/d6/dd/f2c [0,4194304] 0 2024-02-16T19:29:26.852 INFO:tasks.workunit.client.0.smithi022.stdout:2/972: write d0/d5d/fb0 [765749,125594] 0 2024-02-16T19:29:26.855 INFO:tasks.workunit.client.0.smithi022.stdout:2/973: chown d0/d32/d3b/fcd 35318645 1 2024-02-16T19:29:26.856 INFO:tasks.workunit.client.0.smithi022.stdout:4/960: write d4/d6/dd/df/d10/f25 [2624643,127539] 0 2024-02-16T19:29:26.857 INFO:tasks.workunit.client.0.smithi022.stdout:4/961: fdatasync d4/f23 0 2024-02-16T19:29:26.858 INFO:tasks.workunit.client.0.smithi022.stdout:4/962: readlink d4/d6/l5c 0 2024-02-16T19:29:26.858 INFO:tasks.workunit.client.0.smithi022.stdout:6/977: dread d1/d8/dac/d101/fd3 [0,4194304] 0 2024-02-16T19:29:26.861 INFO:tasks.workunit.client.0.smithi022.stdout:2/974: dread - d0/d2/d18/d19/d90/d100/f102 zero size 2024-02-16T19:29:26.862 INFO:tasks.workunit.client.0.smithi022.stdout:4/963: mknod d4/d6/dd/df/d10/de8/d128/c133 0 2024-02-16T19:29:26.867 INFO:tasks.workunit.client.0.smithi022.stdout:2/975: dread - d0/fbe zero size 2024-02-16T19:29:26.868 INFO:tasks.workunit.client.0.smithi022.stdout:4/964: symlink d4/d6/dd/d22/d30/d118/l134 0 2024-02-16T19:29:26.872 INFO:tasks.workunit.client.0.smithi022.stdout:6/978: rename d1/d8/da/d29/deb/dfd/d53/d103 to d1/d8/d23/d14b 0 2024-02-16T19:29:26.884 INFO:tasks.workunit.client.0.smithi022.stdout:6/979: mknod d1/d8/d3c/d4d/ddd/c14c 0 2024-02-16T19:29:26.885 INFO:tasks.workunit.client.0.smithi022.stdout:6/980: read - d1/d8/da/d29/deb/dfd/d85/d56/fab zero size 2024-02-16T19:29:26.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:26 smithi117 ceph-mon[35309]: pgmap v16: 65 pgs: 65 active+clean; 3.6 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 86 MiB/s rd, 196 MiB/s wr, 454 op/s 2024-02-16T19:29:26.891 INFO:tasks.workunit.client.0.smithi022.stdout:4/965: write d4/d6/d3a/f41 [4161968,80508] 0 2024-02-16T19:29:26.895 INFO:tasks.workunit.client.0.smithi022.stdout:2/976: write d0/d2/d2e/f2a [1242706,14607] 0 2024-02-16T19:29:26.899 INFO:tasks.workunit.client.0.smithi022.stdout:2/977: mkdir d0/d5d/dd0/d13b 0 2024-02-16T19:29:26.901 INFO:tasks.workunit.client.0.smithi022.stdout:2/978: write d0/d2/d18/f34 [5595823,89846] 0 2024-02-16T19:29:26.901 INFO:tasks.workunit.client.0.smithi022.stdout:6/981: link d1/d8/da/d29/deb/dfd/d53/c7a d1/d8/da/d29/deb/dfd/d85/d56/c14d 0 2024-02-16T19:29:26.902 INFO:tasks.workunit.client.0.smithi022.stdout:2/979: dread - d0/d2/d2e/d82/df0/d120/f133 zero size 2024-02-16T19:29:26.912 INFO:tasks.workunit.client.0.smithi022.stdout:6/982: mkdir d1/d8/d23/d14e 0 2024-02-16T19:29:26.914 INFO:tasks.workunit.client.0.smithi022.stdout:2/980: mkdir d0/d2/d13c 0 2024-02-16T19:29:26.915 INFO:tasks.workunit.client.0.smithi022.stdout:4/966: write d4/d6/d14/dc7/d8d/d130/d54/d5a/dce/dda/ff1 [902205,52792] 0 2024-02-16T19:29:26.915 INFO:tasks.workunit.client.0.smithi022.stdout:2/981: fsync d0/d2/d18/d38/db6/fe4 0 2024-02-16T19:29:26.917 INFO:tasks.workunit.client.0.smithi022.stdout:6/983: rename d1/d8/d3c/d77/f80 to d1/d8/f14f 0 2024-02-16T19:29:26.917 INFO:tasks.workunit.client.0.smithi022.stdout:2/982: write d0/d2/dce/ff8 [861287,2597] 0 2024-02-16T19:29:26.922 INFO:tasks.workunit.client.0.smithi022.stdout:4/967: dwrite d4/d6/dd/d22/d105/fd5 [0,4194304] 0 2024-02-16T19:29:26.930 INFO:tasks.workunit.client.0.smithi022.stdout:6/984: rename d1/d8/da/d7b/dae/db6/l144 to d1/d8/da/d7b/l150 0 2024-02-16T19:29:26.939 INFO:tasks.workunit.client.0.smithi022.stdout:4/968: creat d4/d6/d14/dc7/d6a/d12d/f135 x:0 0 0 2024-02-16T19:29:26.941 INFO:tasks.workunit.client.0.smithi022.stdout:4/969: chown d4/d6/d14/dc7/d8d/d130/d54/dba/l102 335226 1 2024-02-16T19:29:26.942 INFO:tasks.workunit.client.0.smithi022.stdout:6/985: dread d1/d8/da/d7b/fb2 [0,4194304] 0 2024-02-16T19:29:26.944 INFO:tasks.workunit.client.0.smithi022.stdout:6/986: fsync d1/d8/da/d29/deb/dfd/d85/d56/f149 0 2024-02-16T19:29:26.947 INFO:tasks.workunit.client.0.smithi022.stdout:6/987: fdatasync d1/d8/d23/dc6/ddb/ffe 0 2024-02-16T19:29:26.950 INFO:tasks.workunit.client.0.smithi022.stdout:4/970: mkdir d4/d6/dd/df/d10/de8/d128/d12e/d136 0 2024-02-16T19:29:26.951 INFO:tasks.workunit.client.0.smithi022.stdout:4/971: chown d4/d6/d14/dc7/d8d/d130/d54/d5a/d126 4 1 2024-02-16T19:29:26.956 INFO:tasks.workunit.client.0.smithi022.stdout:6/988: unlink d1/d8/da/d29/deb/dfd/d85/d56/fab 0 2024-02-16T19:29:26.957 INFO:tasks.workunit.client.0.smithi022.stdout:2/983: dwrite d0/f2d [0,4194304] 0 2024-02-16T19:29:26.964 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:26 smithi022 ceph-mon[29221]: pgmap v16: 65 pgs: 65 active+clean; 3.6 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 86 MiB/s rd, 196 MiB/s wr, 454 op/s 2024-02-16T19:29:26.965 INFO:tasks.workunit.client.0.smithi022.stdout:6/989: link d1/d8/da/d29/deb/dfd/d48/d51/f59 d1/d8/da/d7b/dae/dbb/f151 0 2024-02-16T19:29:26.967 INFO:tasks.workunit.client.0.smithi022.stdout:6/990: dread d1/d8/d3c/d4d/f126 [0,4194304] 0 2024-02-16T19:29:26.969 INFO:tasks.workunit.client.0.smithi022.stdout:6/991: rmdir d1/d8/dac 39 2024-02-16T19:29:26.972 INFO:tasks.workunit.client.0.smithi022.stdout:6/992: read d1/d8/da/d29/deb/dfd/f2c [67748,86893] 0 2024-02-16T19:29:27.003 INFO:tasks.workunit.client.0.smithi022.stdout:6/993: dwrite d1/d8/d3c/d4d/fff [0,4194304] 0 2024-02-16T19:29:27.007 INFO:tasks.workunit.client.0.smithi022.stdout:6/994: symlink d1/d8/da/d29/deb/dfd/d85/d56/l152 0 2024-02-16T19:29:27.020 INFO:tasks.workunit.client.0.smithi022.stdout:6/995: write d1/d8/da/d29/deb/dfd/d53/f91 [220252,97390] 0 2024-02-16T19:29:27.120 INFO:tasks.workunit.client.0.smithi022.stdout:6/996: dread d1/d8/ff [0,4194304] 0 2024-02-16T19:29:27.126 INFO:tasks.workunit.client.0.smithi022.stdout:6/997: dwrite d1/d8/d3c/f44 [0,4194304] 0 2024-02-16T19:29:27.130 INFO:tasks.workunit.client.0.smithi022.stdout:6/998: creat d1/d8/d3c/d77/f153 x:0 0 0 2024-02-16T19:29:27.132 INFO:tasks.workunit.client.0.smithi022.stdout:6/999: mknod d1/d8/da/d29/deb/dfd/d85/d56/d131/c154 0 2024-02-16T19:29:27.139 INFO:tasks.workunit.client.0.smithi022.stdout:9/967: sync 2024-02-16T19:29:27.146 INFO:tasks.workunit.client.0.smithi022.stdout:9/968: write db/d14/d16/d21/f4c [2578855,54478] 0 2024-02-16T19:29:27.148 INFO:tasks.workunit.client.0.smithi022.stdout:9/969: dread - db/d14/d16/d5d/d65/d6b/d89/dd2/ff0 zero size 2024-02-16T19:29:27.165 INFO:tasks.workunit.client.0.smithi022.stdout:9/970: dread db/d14/d16/d87/fb0 [0,4194304] 0 2024-02-16T19:29:27.183 INFO:tasks.workunit.client.0.smithi022.stdout:9/971: dwrite db/d14/d16/d5d/d65/d6b/d89/dd2/ff0 [0,4194304] 0 2024-02-16T19:29:27.185 INFO:tasks.workunit.client.0.smithi022.stdout:9/972: truncate db/d14/d16/dec/dee/f118 4630226 0 2024-02-16T19:29:27.192 INFO:tasks.workunit.client.0.smithi022.stdout:9/973: mknod db/c147 0 2024-02-16T19:29:27.201 INFO:tasks.workunit.client.0.smithi022.stdout:9/974: write db/d14/d19/d1a/de5/f107 [782383,39307] 0 2024-02-16T19:29:27.212 INFO:tasks.workunit.client.0.smithi022.stdout:9/975: dwrite db/d14/f60 [0,4194304] 0 2024-02-16T19:29:27.220 INFO:tasks.workunit.client.0.smithi022.stdout:9/976: read db/d14/d16/d21/d37/f63 [270930,78120] 0 2024-02-16T19:29:27.222 INFO:tasks.workunit.client.0.smithi022.stdout:9/977: rmdir db/d14/d16/dec 39 2024-02-16T19:29:27.224 INFO:tasks.workunit.client.0.smithi022.stdout:9/978: symlink db/d14/d16/d5d/d65/d6b/dd5/dd9/l148 0 2024-02-16T19:29:27.230 INFO:tasks.workunit.client.0.smithi022.stdout:9/979: truncate db/d14/d16/d21/d4a/fed 455922 0 2024-02-16T19:29:27.237 INFO:tasks.workunit.client.0.smithi022.stdout:9/980: dwrite db/d14/d16/d21/d37/f12b [0,4194304] 0 2024-02-16T19:29:27.244 INFO:tasks.workunit.client.0.smithi022.stdout:9/981: getdents db/d14/d16/d27/d55 0 2024-02-16T19:29:27.246 INFO:tasks.workunit.client.0.smithi022.stdout:9/982: truncate db/d14/d16/d27/d8b/f142 956188 0 2024-02-16T19:29:27.250 INFO:tasks.workunit.client.0.smithi022.stdout:9/983: chown db/d14/d16/dec/dee/f118 0 1 2024-02-16T19:29:27.257 INFO:tasks.workunit.client.0.smithi022.stdout:9/984: dwrite db/d14/d16/d5d/d65/d6b/d89/f10e [0,4194304] 0 2024-02-16T19:29:27.265 INFO:tasks.workunit.client.0.smithi022.stdout:9/985: getdents db/d18/d103 0 2024-02-16T19:29:27.267 INFO:tasks.workunit.client.0.smithi022.stdout:4/972: sync 2024-02-16T19:29:27.268 INFO:tasks.workunit.client.0.smithi022.stdout:4/973: write d4/d6/dd/d22/f51 [3434914,70674] 0 2024-02-16T19:29:27.271 INFO:tasks.workunit.client.0.smithi022.stdout:4/974: creat d4/d6/dd/d22/d83/d113/d11f/f137 x:0 0 0 2024-02-16T19:29:27.271 INFO:tasks.workunit.client.0.smithi022.stdout:9/986: write db/d14/dc8/dc9/d105/fa7 [3167285,79825] 0 2024-02-16T19:29:27.275 INFO:tasks.workunit.client.0.smithi022.stdout:9/987: mkdir db/d14/d16/d87/d149 0 2024-02-16T19:29:27.275 INFO:tasks.workunit.client.0.smithi022.stdout:9/988: chown db/d14/d16/lba 42 1 2024-02-16T19:29:27.279 INFO:tasks.workunit.client.0.smithi022.stdout:4/975: truncate d4/d6/d14/dc7/d8d/f99 10414147 0 2024-02-16T19:29:27.280 INFO:tasks.workunit.client.0.smithi022.stdout:4/976: chown d4/d6/dd/df/d10/ddc/fdf 25291 1 2024-02-16T19:29:27.280 INFO:tasks.workunit.client.0.smithi022.stdout:9/989: creat db/d14/dc8/dc9/f14a x:0 0 0 2024-02-16T19:29:27.283 INFO:tasks.workunit.client.0.smithi022.stdout:9/990: creat db/d18/d103/f14b x:0 0 0 2024-02-16T19:29:27.286 INFO:tasks.workunit.client.0.smithi022.stdout:9/991: unlink db/d14/d19/d1a/c6a 0 2024-02-16T19:29:27.286 INFO:tasks.workunit.client.0.smithi022.stdout:9/992: stat db/d14/d16/d27/d55/fc3 0 2024-02-16T19:29:27.288 INFO:tasks.workunit.client.0.smithi022.stdout:9/993: truncate db/d14/dc8/dc9/f14a 389865 0 2024-02-16T19:29:27.291 INFO:tasks.workunit.client.0.smithi022.stdout:9/994: chown db/d14/d16/d21/d4a/fed 466617 1 2024-02-16T19:29:27.294 INFO:tasks.workunit.client.0.smithi022.stdout:4/977: write d4/d6/dd/d22/da8/dc4/d86/dfd/f7d [160895,44839] 0 2024-02-16T19:29:27.295 INFO:tasks.workunit.client.0.smithi022.stdout:9/995: read db/d14/d16/d49/fe7 [2860586,107718] 0 2024-02-16T19:29:27.300 INFO:tasks.workunit.client.0.smithi022.stdout:4/978: link d4/d6/dd/d22/f51 d4/d6/d14/dc7/d8d/f138 0 2024-02-16T19:29:27.302 INFO:tasks.workunit.client.0.smithi022.stdout:4/979: stat d4/d6/d14/dc7/d8d/d130/d54/d5a/d126 0 2024-02-16T19:29:27.303 INFO:tasks.workunit.client.0.smithi022.stdout:9/996: link db/d14/c145 db/d14/d16/d21/c14c 0 2024-02-16T19:29:27.304 INFO:tasks.workunit.client.0.smithi022.stdout:4/980: chown d4/d6/dd/d22/f59 0 1 2024-02-16T19:29:27.306 INFO:tasks.workunit.client.0.smithi022.stdout:9/997: creat db/d14/d10a/d135/f14d x:0 0 0 2024-02-16T19:29:27.308 INFO:tasks.workunit.client.0.smithi022.stdout:4/981: fdatasync d4/d6/d14/dc7/d8d/f138 0 2024-02-16T19:29:27.309 INFO:tasks.workunit.client.0.smithi022.stdout:9/998: mknod db/d14/d16/d27/d2e/c14e 0 2024-02-16T19:29:27.310 INFO:tasks.workunit.client.0.smithi022.stdout:9/999: dread - db/d14/d16/d21/d37/f144 zero size 2024-02-16T19:29:27.311 INFO:tasks.workunit.client.0.smithi022.stdout:4/982: mkdir d4/d6/d14/dc7/d8d/dd1/d139 0 2024-02-16T19:29:27.314 INFO:tasks.workunit.client.0.smithi022.stdout:4/983: dread d4/d6/dd/f2c [0,4194304] 0 2024-02-16T19:29:27.318 INFO:tasks.workunit.client.0.smithi022.stdout:4/984: read d4/d6/dd/d22/da8/dc4/d86/dfd/f7d [704386,128138] 0 2024-02-16T19:29:27.322 INFO:tasks.workunit.client.0.smithi022.stdout:4/985: creat d4/d6/d14/dc7/d8d/dd1/f13a x:0 0 0 2024-02-16T19:29:27.336 INFO:tasks.workunit.client.0.smithi022.stdout:4/986: dread d4/d6/dd/d22/f51 [0,4194304] 0 2024-02-16T19:29:27.342 INFO:tasks.workunit.client.0.smithi022.stdout:2/984: sync 2024-02-16T19:29:27.342 INFO:tasks.workunit.client.0.smithi022.stdout:4/987: dread d4/d6/dd/df/d10/d24/f88 [0,4194304] 0 2024-02-16T19:29:27.344 INFO:tasks.workunit.client.0.smithi022.stdout:2/985: truncate d0/d2/d18/d19/ddf/d10c/d117/f12a 515124 0 2024-02-16T19:29:27.349 INFO:tasks.workunit.client.0.smithi022.stdout:2/986: mkdir d0/d2/d18/d13d 0 2024-02-16T19:29:27.349 INFO:tasks.workunit.client.0.smithi022.stdout:4/988: dwrite d4/d6/d14/dd2/fec [0,4194304] 0 2024-02-16T19:29:27.364 INFO:tasks.workunit.client.0.smithi022.stdout:4/989: write d4/d6/dd/df/d10/de8/fee [394844,2282] 0 2024-02-16T19:29:27.366 INFO:tasks.workunit.client.0.smithi022.stdout:4/990: truncate d4/d6/d14/d64/de0/f106 182807 0 2024-02-16T19:29:27.368 INFO:tasks.workunit.client.0.smithi022.stdout:4/991: rmdir d4/d6/dd/df 39 2024-02-16T19:29:27.369 INFO:tasks.workunit.client.0.smithi022.stdout:2/987: dread d0/d5d/dc4/f11f [0,4194304] 0 2024-02-16T19:29:27.372 INFO:tasks.workunit.client.0.smithi022.stdout:2/988: dread d0/d2/d18/d1f/d6c/d99/fc7 [0,4194304] 0 2024-02-16T19:29:27.373 INFO:tasks.workunit.client.0.smithi022.stdout:4/992: mknod d4/d6/d14/dc7/d8d/d130/d54/d5a/dce/dda/c13b 0 2024-02-16T19:29:27.375 INFO:tasks.workunit.client.0.smithi022.stdout:4/993: truncate d4/d6/d14/dc7/d6a/f11b 495341 0 2024-02-16T19:29:27.375 INFO:tasks.workunit.client.0.smithi022.stdout:2/989: truncate d0/d2/d18/f22 2157583 0 2024-02-16T19:29:27.380 INFO:tasks.workunit.client.0.smithi022.stdout:2/990: rename d0/d2/d18/d38/l4b to d0/d2/d13c/l13e 0 2024-02-16T19:29:27.381 INFO:tasks.workunit.client.0.smithi022.stdout:2/991: chown d0/d2/d18/d19/d90/c10a 24604142 1 2024-02-16T19:29:27.384 INFO:tasks.workunit.client.0.smithi022.stdout:2/992: rename d0/d2/dce/f128 to d0/d2/d18/d19/d90/d100/f13f 0 2024-02-16T19:29:27.391 INFO:tasks.workunit.client.0.smithi022.stdout:2/993: getdents d0/d2/d13c 0 2024-02-16T19:29:27.393 INFO:tasks.workunit.client.0.smithi022.stdout:2/994: write d0/d2/d18/d19/d40/dc9/ff9 [1000717,95864] 0 2024-02-16T19:29:27.395 INFO:tasks.workunit.client.0.smithi022.stdout:2/995: write d0/d2/d18/d1f/d6c/d99/fc6 [4297178,102966] 0 2024-02-16T19:29:27.397 INFO:tasks.workunit.client.0.smithi022.stdout:2/996: truncate d0/d2/d18/d19/d90/d100/f13f 725948 0 2024-02-16T19:29:27.402 INFO:tasks.workunit.client.0.smithi022.stdout:2/997: dread d0/d2/d2e/f2a [0,4194304] 0 2024-02-16T19:29:27.404 INFO:tasks.workunit.client.0.smithi022.stdout:2/998: truncate d0/d2/dde/f130 4483273 0 2024-02-16T19:29:27.412 INFO:tasks.workunit.client.0.smithi022.stdout:2/999: dwrite d0/d2/d2e/f2a [0,4194304] 0 2024-02-16T19:29:27.569 INFO:tasks.workunit.client.0.smithi022.stdout:4/994: sync 2024-02-16T19:29:27.569 INFO:tasks.workunit.client.0.smithi022.stdout:4/995: readlink d4/lb2 0 2024-02-16T19:29:27.582 INFO:tasks.workunit.client.0.smithi022.stdout:4/996: dwrite d4/d6/dd/df/d10/f50 [0,4194304] 0 2024-02-16T19:29:27.586 INFO:tasks.workunit.client.0.smithi022.stdout:4/997: dread d4/d6/d14/dc7/d8d/f138 [0,4194304] 0 2024-02-16T19:29:27.590 INFO:tasks.workunit.client.0.smithi022.stdout:4/998: creat d4/d6/d14/d64/f13c x:0 0 0 2024-02-16T19:29:27.592 INFO:tasks.workunit.client.0.smithi022.stdout:4/999: chown d4/d6/d14/dc7/d8d/d130/d54/d5a/dce/dda/feb 164 1 2024-02-16T19:29:27.596 INFO:tasks.workunit.client.0.smithi022.stderr:+ rm -rf -- ./tmp.6udblTWqFb 2024-02-16T19:29:28.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:27 smithi117 ceph-mon[35309]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' 2024-02-16T19:29:28.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:27 smithi117 ceph-mon[35309]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' 2024-02-16T19:29:28.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:27 smithi117 ceph-mon[35309]: pgmap v17: 65 pgs: 65 active+clean; 4.0 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 105 MiB/s rd, 250 MiB/s wr, 582 op/s 2024-02-16T19:29:28.464 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:27 smithi022 ceph-mon[29221]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' 2024-02-16T19:29:28.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:27 smithi022 ceph-mon[29221]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' 2024-02-16T19:29:28.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:27 smithi022 ceph-mon[29221]: pgmap v17: 65 pgs: 65 active+clean; 4.0 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 105 MiB/s rd, 250 MiB/s wr, 582 op/s 2024-02-16T19:29:29.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:29 smithi117 ceph-mon[35309]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' 2024-02-16T19:29:29.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:29 smithi117 ceph-mon[35309]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' 2024-02-16T19:29:29.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:29 smithi117 ceph-mon[35309]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:29:29.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:29 smithi117 ceph-mon[35309]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:29:29.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:29 smithi117 ceph-mon[35309]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:29:29.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:29 smithi117 ceph-mon[35309]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:29:29.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:29 smithi117 ceph-mon[35309]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:29:29.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:29 smithi117 ceph-mon[35309]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:29:29.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:29 smithi117 ceph-mon[35309]: Adjusting osd_memory_target on smithi022 to 1217M 2024-02-16T19:29:29.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:29 smithi117 ceph-mon[35309]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' 2024-02-16T19:29:29.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:29 smithi117 ceph-mon[35309]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:29:29.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:29 smithi117 ceph-mon[35309]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:29:29.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:29 smithi117 ceph-mon[35309]: Updating smithi022:/etc/ceph/ceph.conf 2024-02-16T19:29:29.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:29 smithi117 ceph-mon[35309]: Updating smithi117:/etc/ceph/ceph.conf 2024-02-16T19:29:29.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:29 smithi022 ceph-mon[29221]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' 2024-02-16T19:29:29.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:29 smithi022 ceph-mon[29221]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' 2024-02-16T19:29:29.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:29 smithi022 ceph-mon[29221]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:29:29.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:29 smithi022 ceph-mon[29221]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:29:29.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:29 smithi022 ceph-mon[29221]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:29:29.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:29 smithi022 ceph-mon[29221]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:29:29.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:29 smithi022 ceph-mon[29221]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:29:29.716 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:29 smithi022 ceph-mon[29221]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:29:29.716 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:29 smithi022 ceph-mon[29221]: Adjusting osd_memory_target on smithi022 to 1217M 2024-02-16T19:29:29.716 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:29 smithi022 ceph-mon[29221]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' 2024-02-16T19:29:29.716 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:29 smithi022 ceph-mon[29221]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:29:29.716 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:29 smithi022 ceph-mon[29221]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:29:29.716 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:29 smithi022 ceph-mon[29221]: Updating smithi022:/etc/ceph/ceph.conf 2024-02-16T19:29:29.716 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:29 smithi022 ceph-mon[29221]: Updating smithi117:/etc/ceph/ceph.conf 2024-02-16T19:29:30.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:30 smithi022 ceph-mon[29221]: Updating smithi022:/var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T19:29:30.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:30 smithi022 ceph-mon[29221]: Updating smithi117:/var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T19:29:30.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:30 smithi022 ceph-mon[29221]: Updating smithi022:/etc/ceph/ceph.client.admin.keyring 2024-02-16T19:29:30.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:30 smithi022 ceph-mon[29221]: Updating smithi117:/etc/ceph/ceph.client.admin.keyring 2024-02-16T19:29:30.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:30 smithi022 ceph-mon[29221]: Updating smithi117:/var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-16T19:29:30.716 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:30 smithi022 ceph-mon[29221]: Updating smithi022:/var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-16T19:29:30.716 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:30 smithi022 ceph-mon[29221]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' 2024-02-16T19:29:30.716 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:30 smithi022 ceph-mon[29221]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' 2024-02-16T19:29:30.716 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:30 smithi022 ceph-mon[29221]: pgmap v18: 65 pgs: 65 active+clean; 4.0 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 69 MiB/s rd, 179 MiB/s wr, 400 op/s 2024-02-16T19:29:30.716 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:30 smithi022 ceph-mon[29221]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' 2024-02-16T19:29:30.716 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:30 smithi022 ceph-mon[29221]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' 2024-02-16T19:29:30.716 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:30 smithi022 ceph-mon[29221]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' 2024-02-16T19:29:30.716 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:30 smithi022 ceph-mon[29221]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:29:30.716 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:30 smithi022 ceph-mon[29221]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' 2024-02-16T19:29:30.716 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:30 smithi022 ceph-mon[29221]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' 2024-02-16T19:29:30.716 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:30 smithi022 ceph-mon[29221]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' 2024-02-16T19:29:30.716 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:30 smithi022 ceph-mon[29221]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' 2024-02-16T19:29:30.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:30 smithi117 ceph-mon[35309]: Updating smithi022:/var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T19:29:30.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:30 smithi117 ceph-mon[35309]: Updating smithi117:/var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T19:29:30.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:30 smithi117 ceph-mon[35309]: Updating smithi022:/etc/ceph/ceph.client.admin.keyring 2024-02-16T19:29:30.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:30 smithi117 ceph-mon[35309]: Updating smithi117:/etc/ceph/ceph.client.admin.keyring 2024-02-16T19:29:30.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:30 smithi117 ceph-mon[35309]: Updating smithi117:/var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-16T19:29:30.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:30 smithi117 ceph-mon[35309]: Updating smithi022:/var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-16T19:29:30.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:30 smithi117 ceph-mon[35309]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' 2024-02-16T19:29:30.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:30 smithi117 ceph-mon[35309]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' 2024-02-16T19:29:30.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:30 smithi117 ceph-mon[35309]: pgmap v18: 65 pgs: 65 active+clean; 4.0 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 69 MiB/s rd, 179 MiB/s wr, 400 op/s 2024-02-16T19:29:30.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:30 smithi117 ceph-mon[35309]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' 2024-02-16T19:29:30.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:30 smithi117 ceph-mon[35309]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' 2024-02-16T19:29:30.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:30 smithi117 ceph-mon[35309]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' 2024-02-16T19:29:30.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:30 smithi117 ceph-mon[35309]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:29:30.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:30 smithi117 ceph-mon[35309]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' 2024-02-16T19:29:30.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:30 smithi117 ceph-mon[35309]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' 2024-02-16T19:29:30.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:30 smithi117 ceph-mon[35309]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' 2024-02-16T19:29:30.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:30 smithi117 ceph-mon[35309]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' 2024-02-16T19:29:31.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:31 smithi022 ceph-mon[29221]: Reconfiguring prometheus.smithi022 (dependencies changed)... 2024-02-16T19:29:31.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:31 smithi022 ceph-mon[29221]: Reconfiguring daemon prometheus.smithi022 on smithi022 2024-02-16T19:29:31.886 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:31 smithi117 ceph-mon[35309]: Reconfiguring prometheus.smithi022 (dependencies changed)... 2024-02-16T19:29:31.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:31 smithi117 ceph-mon[35309]: Reconfiguring daemon prometheus.smithi022 on smithi022 2024-02-16T19:29:32.694 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:32 smithi022 ceph-mon[29221]: pgmap v19: 65 pgs: 65 active+clean; 3.9 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 81 MiB/s rd, 208 MiB/s wr, 513 op/s 2024-02-16T19:29:32.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:32 smithi117 ceph-mon[35309]: pgmap v19: 65 pgs: 65 active+clean; 3.9 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 81 MiB/s rd, 208 MiB/s wr, 513 op/s 2024-02-16T19:29:34.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:34 smithi022 ceph-mon[29221]: pgmap v20: 65 pgs: 65 active+clean; 3.9 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 46 MiB/s rd, 116 MiB/s wr, 315 op/s 2024-02-16T19:29:34.886 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:34 smithi117 ceph-mon[35309]: pgmap v20: 65 pgs: 65 active+clean; 3.9 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 46 MiB/s rd, 116 MiB/s wr, 315 op/s 2024-02-16T19:29:36.431 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:36 smithi022 ceph-mon[29221]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' 2024-02-16T19:29:36.431 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:36 smithi022 ceph-mon[29221]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' 2024-02-16T19:29:36.431 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:36 smithi022 ceph-mon[29221]: pgmap v21: 65 pgs: 65 active+clean; 3.7 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 46 MiB/s rd, 116 MiB/s wr, 340 op/s 2024-02-16T19:29:36.431 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:36 smithi022 ceph-mon[29221]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-16T19:29:36.431 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:36 smithi022 ceph-mon[29221]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-16T19:29:36.432 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:36 smithi022 ceph-mon[29221]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:29:36.432 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:36 smithi022 ceph-mon[29221]: Upgrade: Need to upgrade myself (mgr.smithi117.cohbtv) 2024-02-16T19:29:36.432 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:36 smithi022 ceph-mon[29221]: Upgrade: Need to upgrade myself (mgr.smithi117.cohbtv) 2024-02-16T19:29:36.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:36 smithi117 ceph-mon[35309]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' 2024-02-16T19:29:36.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:36 smithi117 ceph-mon[35309]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' 2024-02-16T19:29:36.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:36 smithi117 ceph-mon[35309]: pgmap v21: 65 pgs: 65 active+clean; 3.7 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 46 MiB/s rd, 116 MiB/s wr, 340 op/s 2024-02-16T19:29:36.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:36 smithi117 ceph-mon[35309]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-16T19:29:36.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:36 smithi117 ceph-mon[35309]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-16T19:29:36.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:36 smithi117 ceph-mon[35309]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:29:36.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:36 smithi117 ceph-mon[35309]: Upgrade: Need to upgrade myself (mgr.smithi117.cohbtv) 2024-02-16T19:29:36.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:36 smithi117 ceph-mon[35309]: Upgrade: Need to upgrade myself (mgr.smithi117.cohbtv) 2024-02-16T19:29:38.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:38 smithi117 ceph-mon[35309]: Upgrade: Updating mgr.smithi022.gmdcch 2024-02-16T19:29:38.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:38 smithi117 ceph-mon[35309]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' 2024-02-16T19:29:38.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:38 smithi117 ceph-mon[35309]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi022.gmdcch", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-16T19:29:38.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:38 smithi117 ceph-mon[35309]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi022.gmdcch", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-16T19:29:38.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:38 smithi117 ceph-mon[35309]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-16T19:29:38.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:38 smithi117 ceph-mon[35309]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:29:38.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:38 smithi117 ceph-mon[35309]: Deploying daemon mgr.smithi022.gmdcch on smithi022 2024-02-16T19:29:38.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:38 smithi117 ceph-mon[35309]: pgmap v22: 65 pgs: 65 active+clean; 3.0 GiB data, 9.5 GiB used, 527 GiB / 536 GiB avail; 39 MiB/s rd, 103 MiB/s wr, 339 op/s 2024-02-16T19:29:38.714 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:38 smithi022 ceph-mon[29221]: Upgrade: Updating mgr.smithi022.gmdcch 2024-02-16T19:29:38.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:38 smithi022 ceph-mon[29221]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' 2024-02-16T19:29:38.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:38 smithi022 ceph-mon[29221]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi022.gmdcch", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-16T19:29:38.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:38 smithi022 ceph-mon[29221]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi022.gmdcch", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-16T19:29:38.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:38 smithi022 ceph-mon[29221]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-16T19:29:38.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:38 smithi022 ceph-mon[29221]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:29:38.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:38 smithi022 ceph-mon[29221]: Deploying daemon mgr.smithi022.gmdcch on smithi022 2024-02-16T19:29:38.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:38 smithi022 ceph-mon[29221]: pgmap v22: 65 pgs: 65 active+clean; 3.0 GiB data, 9.5 GiB used, 527 GiB / 536 GiB avail; 39 MiB/s rd, 103 MiB/s wr, 339 op/s 2024-02-16T19:29:40.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:40 smithi022 ceph-mon[29221]: pgmap v23: 65 pgs: 65 active+clean; 3.0 GiB data, 9.5 GiB used, 527 GiB / 536 GiB avail; 11 MiB/s rd, 30 MiB/s wr, 187 op/s 2024-02-16T19:29:40.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:40 smithi117 ceph-mon[35309]: pgmap v23: 65 pgs: 65 active+clean; 3.0 GiB data, 9.5 GiB used, 527 GiB / 536 GiB avail; 11 MiB/s rd, 30 MiB/s wr, 187 op/s 2024-02-16T19:29:41.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:41.770+0000 7f14b867d700 1 -- 172.21.15.22:0/397855051 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14b00fffb0 msgr2=0x7f14b0100440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:29:41.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:41.770+0000 7f14b867d700 1 --2- 172.21.15.22:0/397855051 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14b00fffb0 0x7f14b0100440 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f14a0009910 tx=0x7f14a0009c20 comp rx=0 tx=0).stop 2024-02-16T19:29:41.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:41.770+0000 7f14b867d700 1 -- 172.21.15.22:0/397855051 shutdown_connections 2024-02-16T19:29:41.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:41.770+0000 7f14b867d700 1 --2- 172.21.15.22:0/397855051 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14b00fffb0 0x7f14b0100440 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:41.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:41.770+0000 7f14b867d700 1 --2- 172.21.15.22:0/397855051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14b00ff660 0x7f14b00ffa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:41.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:41.770+0000 7f14b867d700 1 -- 172.21.15.22:0/397855051 >> 172.21.15.22:0/397855051 conn(0x7f14b00f9f10 msgr2=0x7f14b00fc340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:29:41.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:41.770+0000 7f14b867d700 1 -- 172.21.15.22:0/397855051 shutdown_connections 2024-02-16T19:29:41.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:41.770+0000 7f14b867d700 1 -- 172.21.15.22:0/397855051 wait complete. 2024-02-16T19:29:41.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:41.771+0000 7f14b867d700 1 Processor -- start 2024-02-16T19:29:41.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:41.771+0000 7f14b867d700 1 -- start start 2024-02-16T19:29:41.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:41.771+0000 7f14b867d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14b00ff660 0x7f14b0195af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:29:41.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:41.771+0000 7f14b867d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14b00fffb0 0x7f14b0196030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:29:41.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:41.771+0000 7f14b867d700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b0196650 con 0x7f14b00fffb0 2024-02-16T19:29:41.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:41.771+0000 7f14b867d700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b0196790 con 0x7f14b00ff660 2024-02-16T19:29:41.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:41.772+0000 7f14b5c18700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14b00fffb0 0x7f14b0196030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:29:41.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:41.772+0000 7f14b5c18700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14b00fffb0 0x7f14b0196030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59020/0 (socket says 172.21.15.22:59020) 2024-02-16T19:29:41.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:41.772+0000 7f14b5c18700 1 -- 172.21.15.22:0/2571243332 learned_addr learned my addr 172.21.15.22:0/2571243332 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:29:41.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:41.772+0000 7f14b6419700 1 --2- 172.21.15.22:0/2571243332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14b00ff660 0x7f14b0195af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:29:41.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:41.772+0000 7f14b6419700 1 -- 172.21.15.22:0/2571243332 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14b00fffb0 msgr2=0x7f14b0196030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:29:41.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:41.772+0000 7f14b6419700 1 --2- 172.21.15.22:0/2571243332 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14b00fffb0 0x7f14b0196030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:41.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:41.772+0000 7f14b6419700 1 -- 172.21.15.22:0/2571243332 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14a00095f0 con 0x7f14b00ff660 2024-02-16T19:29:41.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:41.773+0000 7f14b6419700 1 --2- 172.21.15.22:0/2571243332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14b00ff660 0x7f14b0195af0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f14ac00e940 tx=0x7f14ac00ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:29:41.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:41.773+0000 7f14a77fe700 1 -- 172.21.15.22:0/2571243332 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f14ac0041f0 con 0x7f14b00ff660 2024-02-16T19:29:41.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:41.773+0000 7f14a77fe700 1 -- 172.21.15.22:0/2571243332 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f14ac004350 con 0x7f14b00ff660 2024-02-16T19:29:41.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:41.773+0000 7f14a77fe700 1 -- 172.21.15.22:0/2571243332 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f14ac003c80 con 0x7f14b00ff660 2024-02-16T19:29:41.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:41.773+0000 7f14b867d700 1 -- 172.21.15.22:0/2571243332 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14b019b240 con 0x7f14b00ff660 2024-02-16T19:29:41.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:41.773+0000 7f14b867d700 1 -- 172.21.15.22:0/2571243332 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14b019b740 con 0x7f14b00ff660 2024-02-16T19:29:41.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:41.774+0000 7f14b867d700 1 -- 172.21.15.22:0/2571243332 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14b0107c00 con 0x7f14b00ff660 2024-02-16T19:29:41.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:41.776+0000 7f14a77fe700 1 -- 172.21.15.22:0/2571243332 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 25) v1 ==== 90734+0+0 (secure 0 0 0) 0x7f14ac00c7c0 con 0x7f14b00ff660 2024-02-16T19:29:41.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:41.776+0000 7f14a77fe700 1 --2- 172.21.15.22:0/2571243332 >> [v2:172.21.15.117:6828/10500008,v1:172.21.15.117:6829/10500008] conn(0x7f149c06cf10 0x7f149c06f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:29:41.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:41.777+0000 7f14a77fe700 1 -- 172.21.15.22:0/2571243332 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f14ac014070 con 0x7f14b00ff660 2024-02-16T19:29:41.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:41.777+0000 7f14b5c18700 1 --2- 172.21.15.22:0/2571243332 >> [v2:172.21.15.117:6828/10500008,v1:172.21.15.117:6829/10500008] conn(0x7f149c06cf10 0x7f149c06f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:29:41.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:41.777+0000 7f14b5c18700 1 --2- 172.21.15.22:0/2571243332 >> [v2:172.21.15.117:6828/10500008,v1:172.21.15.117:6829/10500008] conn(0x7f149c06cf10 0x7f149c06f3c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f14a0006010 tx=0x7f14a0005a90 comp rx=0 tx=0).ready entity=mgr.24349 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:29:41.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:41.780+0000 7f14a77fe700 1 -- 172.21.15.22:0/2571243332 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f14ac051a50 con 0x7f14b00ff660 2024-02-16T19:29:42.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.123+0000 7f14b867d700 1 -- 172.21.15.22:0/2571243332 --> [v2:172.21.15.117:6828/10500008,v1:172.21.15.117:6829/10500008] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f14b00fe970 con 0x7f149c06cf10 2024-02-16T19:29:42.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.124+0000 7f14a77fe700 1 -- 172.21.15.22:0/2571243332 <== mgr.24349 v2:172.21.15.117:6828/10500008 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+321 (secure 0 0 0) 0x7f14b00fe970 con 0x7f149c06cf10 2024-02-16T19:29:42.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.127+0000 7f14b867d700 1 -- 172.21.15.22:0/2571243332 >> [v2:172.21.15.117:6828/10500008,v1:172.21.15.117:6829/10500008] conn(0x7f149c06cf10 msgr2=0x7f149c06f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:29:42.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.127+0000 7f14b867d700 1 --2- 172.21.15.22:0/2571243332 >> [v2:172.21.15.117:6828/10500008,v1:172.21.15.117:6829/10500008] conn(0x7f149c06cf10 0x7f149c06f3c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f14a0006010 tx=0x7f14a0005a90 comp rx=0 tx=0).stop 2024-02-16T19:29:42.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.127+0000 7f14b867d700 1 -- 172.21.15.22:0/2571243332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14b00ff660 msgr2=0x7f14b0195af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:29:42.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.127+0000 7f14b867d700 1 --2- 172.21.15.22:0/2571243332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14b00ff660 0x7f14b0195af0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f14ac00e940 tx=0x7f14ac00ec50 comp rx=0 tx=0).stop 2024-02-16T19:29:42.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.127+0000 7f14b867d700 1 -- 172.21.15.22:0/2571243332 shutdown_connections 2024-02-16T19:29:42.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.127+0000 7f14b867d700 1 --2- 172.21.15.22:0/2571243332 >> [v2:172.21.15.117:6828/10500008,v1:172.21.15.117:6829/10500008] conn(0x7f149c06cf10 0x7f149c06f3c0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:42.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.127+0000 7f14b867d700 1 --2- 172.21.15.22:0/2571243332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14b00ff660 0x7f14b0195af0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:42.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.127+0000 7f14b867d700 1 --2- 172.21.15.22:0/2571243332 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14b00fffb0 0x7f14b0196030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:42.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.127+0000 7f14b867d700 1 -- 172.21.15.22:0/2571243332 >> 172.21.15.22:0/2571243332 conn(0x7f14b00f9f10 msgr2=0x7f14b00fe260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:29:42.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.128+0000 7f14b867d700 1 -- 172.21.15.22:0/2571243332 shutdown_connections 2024-02-16T19:29:42.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.128+0000 7f14b867d700 1 -- 172.21.15.22:0/2571243332 wait complete. 2024-02-16T19:29:42.141 INFO:teuthology.orchestra.run.smithi022.stdout:true 2024-02-16T19:29:42.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.273+0000 7f0e1118c700 1 -- 172.21.15.22:0/2321412127 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0dfc004a00 con 0x7f0e0c1003d0 2024-02-16T19:29:42.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.273+0000 7f0e143f2700 1 -- 172.21.15.22:0/2321412127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e0c1003d0 msgr2=0x7f0e0c100840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:29:42.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.274+0000 7f0e143f2700 1 --2- 172.21.15.22:0/2321412127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e0c1003d0 0x7f0e0c100840 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc009910 tx=0x7f0dfc009c20 comp rx=0 tx=0).stop 2024-02-16T19:29:42.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.274+0000 7f0e143f2700 1 -- 172.21.15.22:0/2321412127 shutdown_connections 2024-02-16T19:29:42.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.274+0000 7f0e143f2700 1 --2- 172.21.15.22:0/2321412127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e0c1003d0 0x7f0e0c100840 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:42.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.274+0000 7f0e143f2700 1 --2- 172.21.15.22:0/2321412127 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e0c0ff130 0x7f0e0c0ff540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:42.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.274+0000 7f0e143f2700 1 -- 172.21.15.22:0/2321412127 >> 172.21.15.22:0/2321412127 conn(0x7f0e0c0fa780 msgr2=0x7f0e0c0fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:29:42.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.274+0000 7f0e143f2700 1 -- 172.21.15.22:0/2321412127 shutdown_connections 2024-02-16T19:29:42.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.274+0000 7f0e143f2700 1 -- 172.21.15.22:0/2321412127 wait complete. 2024-02-16T19:29:42.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.275+0000 7f0e143f2700 1 Processor -- start 2024-02-16T19:29:42.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.275+0000 7f0e143f2700 1 -- start start 2024-02-16T19:29:42.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.276+0000 7f0e143f2700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e0c0ff130 0x7f0e0c194930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:29:42.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.276+0000 7f0e143f2700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e0c1003d0 0x7f0e0c194e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:29:42.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.276+0000 7f0e143f2700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e0c195490 con 0x7f0e0c1003d0 2024-02-16T19:29:42.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.276+0000 7f0e143f2700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e0c1955d0 con 0x7f0e0c0ff130 2024-02-16T19:29:42.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.276+0000 7f0e1218e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e0c0ff130 0x7f0e0c194930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:29:42.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.276+0000 7f0e1198d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e0c1003d0 0x7f0e0c194e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:29:42.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.276+0000 7f0e1218e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e0c0ff130 0x7f0e0c194930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.22:49262/0 (socket says 172.21.15.22:49262) 2024-02-16T19:29:42.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.276+0000 7f0e1198d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e0c1003d0 0x7f0e0c194e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59040/0 (socket says 172.21.15.22:59040) 2024-02-16T19:29:42.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.276+0000 7f0e1218e700 1 -- 172.21.15.22:0/1365070453 learned_addr learned my addr 172.21.15.22:0/1365070453 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:29:42.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.276+0000 7f0e1218e700 1 -- 172.21.15.22:0/1365070453 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e0c1003d0 msgr2=0x7f0e0c194e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:29:42.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.276+0000 7f0e1218e700 1 --2- 172.21.15.22:0/1365070453 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e0c1003d0 0x7f0e0c194e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:42.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.276+0000 7f0e1218e700 1 -- 172.21.15.22:0/1365070453 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dfc0095f0 con 0x7f0e0c0ff130 2024-02-16T19:29:42.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.276+0000 7f0e1198d700 1 --2- 172.21.15.22:0/1365070453 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e0c1003d0 0x7f0e0c194e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T19:29:42.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.277+0000 7f0e1218e700 1 --2- 172.21.15.22:0/1365070453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e0c0ff130 0x7f0e0c194930 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0e08007e80 tx=0x7f0e0800dbe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:29:42.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.277+0000 7f0e037fe700 1 -- 172.21.15.22:0/1365070453 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e080041f0 con 0x7f0e0c0ff130 2024-02-16T19:29:42.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.277+0000 7f0e037fe700 1 -- 172.21.15.22:0/1365070453 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0e08014070 con 0x7f0e0c0ff130 2024-02-16T19:29:42.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.277+0000 7f0e143f2700 1 -- 172.21.15.22:0/1365070453 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e0c0685a0 con 0x7f0e0c0ff130 2024-02-16T19:29:42.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.277+0000 7f0e037fe700 1 -- 172.21.15.22:0/1365070453 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e080041f0 con 0x7f0e0c0ff130 2024-02-16T19:29:42.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.277+0000 7f0e143f2700 1 -- 172.21.15.22:0/1365070453 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e0c068aa0 con 0x7f0e0c0ff130 2024-02-16T19:29:42.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.279+0000 7f0e037fe700 1 -- 172.21.15.22:0/1365070453 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 25) v1 ==== 90734+0+0 (secure 0 0 0) 0x7f0e08005020 con 0x7f0e0c0ff130 2024-02-16T19:29:42.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.279+0000 7f0e143f2700 1 -- 172.21.15.22:0/1365070453 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0df0005320 con 0x7f0e0c0ff130 2024-02-16T19:29:42.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.279+0000 7f0e037fe700 1 --2- 172.21.15.22:0/1365070453 >> [v2:172.21.15.117:6828/10500008,v1:172.21.15.117:6829/10500008] conn(0x7f0df806cf60 0x7f0df806f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:29:42.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.279+0000 7f0e037fe700 1 -- 172.21.15.22:0/1365070453 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f0e08016070 con 0x7f0e0c0ff130 2024-02-16T19:29:42.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.280+0000 7f0e1198d700 1 --2- 172.21.15.22:0/1365070453 >> [v2:172.21.15.117:6828/10500008,v1:172.21.15.117:6829/10500008] conn(0x7f0df806cf60 0x7f0df806f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:29:42.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.280+0000 7f0e1198d700 1 --2- 172.21.15.22:0/1365070453 >> [v2:172.21.15.117:6828/10500008,v1:172.21.15.117:6829/10500008] conn(0x7f0df806cf60 0x7f0df806f410 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc006010 tx=0x7f0dfc00b580 comp rx=0 tx=0).ready entity=mgr.24349 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:29:42.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.284+0000 7f0e037fe700 1 -- 172.21.15.22:0/1365070453 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f0e0805c010 con 0x7f0e0c0ff130 2024-02-16T19:29:42.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.680+0000 7f0e143f2700 1 -- 172.21.15.22:0/1365070453 --> [v2:172.21.15.117:6828/10500008,v1:172.21.15.117:6829/10500008] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0df0000bf0 con 0x7f0df806cf60 2024-02-16T19:29:42.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.681+0000 7f0e037fe700 1 -- 172.21.15.22:0/1365070453 <== mgr.24349 v2:172.21.15.117:6828/10500008 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+321 (secure 0 0 0) 0x7f0df0000bf0 con 0x7f0df806cf60 2024-02-16T19:29:42.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.684+0000 7f0e143f2700 1 -- 172.21.15.22:0/1365070453 >> [v2:172.21.15.117:6828/10500008,v1:172.21.15.117:6829/10500008] conn(0x7f0df806cf60 msgr2=0x7f0df806f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:29:42.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.684+0000 7f0e143f2700 1 --2- 172.21.15.22:0/1365070453 >> [v2:172.21.15.117:6828/10500008,v1:172.21.15.117:6829/10500008] conn(0x7f0df806cf60 0x7f0df806f410 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc006010 tx=0x7f0dfc00b580 comp rx=0 tx=0).stop 2024-02-16T19:29:42.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.684+0000 7f0e143f2700 1 -- 172.21.15.22:0/1365070453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e0c0ff130 msgr2=0x7f0e0c194930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:29:42.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.684+0000 7f0e143f2700 1 --2- 172.21.15.22:0/1365070453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e0c0ff130 0x7f0e0c194930 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0e08007e80 tx=0x7f0e0800dbe0 comp rx=0 tx=0).stop 2024-02-16T19:29:42.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.684+0000 7f0e143f2700 1 -- 172.21.15.22:0/1365070453 shutdown_connections 2024-02-16T19:29:42.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.684+0000 7f0e143f2700 1 --2- 172.21.15.22:0/1365070453 >> [v2:172.21.15.117:6828/10500008,v1:172.21.15.117:6829/10500008] conn(0x7f0df806cf60 0x7f0df806f410 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:42.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.684+0000 7f0e143f2700 1 --2- 172.21.15.22:0/1365070453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e0c0ff130 0x7f0e0c194930 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:42.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.684+0000 7f0e143f2700 1 --2- 172.21.15.22:0/1365070453 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e0c1003d0 0x7f0e0c194e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:42.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.685+0000 7f0e143f2700 1 -- 172.21.15.22:0/1365070453 >> 172.21.15.22:0/1365070453 conn(0x7f0e0c0fa780 msgr2=0x7f0e0c103600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:29:42.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.685+0000 7f0e143f2700 1 -- 172.21.15.22:0/1365070453 shutdown_connections 2024-02-16T19:29:42.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.685+0000 7f0e143f2700 1 -- 172.21.15.22:0/1365070453 wait complete. 2024-02-16T19:29:42.752 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:42 smithi022 ceph-mon[29221]: pgmap v24: 65 pgs: 65 active+clean; 2.1 GiB data, 6.5 GiB used, 530 GiB / 536 GiB avail; 11 MiB/s rd, 31 MiB/s wr, 258 op/s 2024-02-16T19:29:42.752 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:42 smithi022 ceph-mon[29221]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' 2024-02-16T19:29:42.752 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:42 smithi022 ceph-mon[29221]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' 2024-02-16T19:29:42.752 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:42 smithi022 ceph-mon[29221]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:29:42.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.830+0000 7f6200087700 1 -- 172.21.15.22:0/1214818369 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61f8100d60 msgr2=0x7f61f8103140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:29:42.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.830+0000 7f6200087700 1 --2- 172.21.15.22:0/1214818369 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61f8100d60 0x7f61f8103140 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f61f4009910 tx=0x7f61f4009c20 comp rx=0 tx=0).stop 2024-02-16T19:29:42.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.830+0000 7f6200087700 1 -- 172.21.15.22:0/1214818369 shutdown_connections 2024-02-16T19:29:42.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.830+0000 7f6200087700 1 --2- 172.21.15.22:0/1214818369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61f8103680 0x7f61f8105a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:42.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.830+0000 7f6200087700 1 --2- 172.21.15.22:0/1214818369 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61f8100d60 0x7f61f8103140 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:42.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.830+0000 7f6200087700 1 -- 172.21.15.22:0/1214818369 >> 172.21.15.22:0/1214818369 conn(0x7f61f80fa780 msgr2=0x7f61f80fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:29:42.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.830+0000 7f6200087700 1 -- 172.21.15.22:0/1214818369 shutdown_connections 2024-02-16T19:29:42.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.830+0000 7f6200087700 1 -- 172.21.15.22:0/1214818369 wait complete. 2024-02-16T19:29:42.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.831+0000 7f6200087700 1 Processor -- start 2024-02-16T19:29:42.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.831+0000 7f6200087700 1 -- start start 2024-02-16T19:29:42.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.831+0000 7f6200087700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61f8100d60 0x7f61f819c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:29:42.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.831+0000 7f6200087700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61f8103680 0x7f61f819c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:29:42.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.831+0000 7f6200087700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f819cc20 con 0x7f61f8103680 2024-02-16T19:29:42.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.831+0000 7f6200087700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f819cd60 con 0x7f61f8100d60 2024-02-16T19:29:42.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.832+0000 7f61fde23700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61f8100d60 0x7f61f819c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:29:42.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.832+0000 7f61fde23700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61f8100d60 0x7f61f819c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.22:49274/0 (socket says 172.21.15.22:49274) 2024-02-16T19:29:42.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.832+0000 7f61fd622700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61f8103680 0x7f61f819c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:29:42.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.832+0000 7f61fde23700 1 -- 172.21.15.22:0/3876275041 learned_addr learned my addr 172.21.15.22:0/3876275041 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:29:42.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.832+0000 7f61fde23700 1 -- 172.21.15.22:0/3876275041 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61f8103680 msgr2=0x7f61f819c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:29:42.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.832+0000 7f61fde23700 1 --2- 172.21.15.22:0/3876275041 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61f8103680 0x7f61f819c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:42.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.832+0000 7f61fde23700 1 -- 172.21.15.22:0/3876275041 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61f40095f0 con 0x7f61f8100d60 2024-02-16T19:29:42.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.832+0000 7f61fde23700 1 --2- 172.21.15.22:0/3876275041 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61f8100d60 0x7f61f819c0c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f61f4000c00 tx=0x7f61f400f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:29:42.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.832+0000 7f61eeffd700 1 -- 172.21.15.22:0/3876275041 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f61f401d070 con 0x7f61f8100d60 2024-02-16T19:29:42.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.833+0000 7f61eeffd700 1 -- 172.21.15.22:0/3876275041 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f61f4005dc0 con 0x7f61f8100d60 2024-02-16T19:29:42.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.833+0000 7f61eeffd700 1 -- 172.21.15.22:0/3876275041 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f61f4022470 con 0x7f61f8100d60 2024-02-16T19:29:42.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.833+0000 7f6200087700 1 -- 172.21.15.22:0/3876275041 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61f81a17b0 con 0x7f61f8100d60 2024-02-16T19:29:42.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.833+0000 7f6200087700 1 -- 172.21.15.22:0/3876275041 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61f81a1be0 con 0x7f61f8100d60 2024-02-16T19:29:42.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.834+0000 7f6200087700 1 -- 172.21.15.22:0/3876275041 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61f8108e30 con 0x7f61f8100d60 2024-02-16T19:29:42.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.835+0000 7f61eeffd700 1 -- 172.21.15.22:0/3876275041 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 25) v1 ==== 90734+0+0 (secure 0 0 0) 0x7f61f400fa70 con 0x7f61f8100d60 2024-02-16T19:29:42.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.835+0000 7f61eeffd700 1 --2- 172.21.15.22:0/3876275041 >> [v2:172.21.15.117:6828/10500008,v1:172.21.15.117:6829/10500008] conn(0x7f61e406cf60 0x7f61e406f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:29:42.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.835+0000 7f61eeffd700 1 -- 172.21.15.22:0/3876275041 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f61f408db70 con 0x7f61f8100d60 2024-02-16T19:29:42.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.836+0000 7f61fd622700 1 --2- 172.21.15.22:0/3876275041 >> [v2:172.21.15.117:6828/10500008,v1:172.21.15.117:6829/10500008] conn(0x7f61e406cf60 0x7f61e406f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:29:42.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.836+0000 7f61fd622700 1 --2- 172.21.15.22:0/3876275041 >> [v2:172.21.15.117:6828/10500008,v1:172.21.15.117:6829/10500008] conn(0x7f61e406cf60 0x7f61e406f410 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f61e8009c60 tx=0x7f61e8009260 comp rx=0 tx=0).ready entity=mgr.24349 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:29:42.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:42.840+0000 7f61eeffd700 1 -- 172.21.15.22:0/3876275041 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f61f405b290 con 0x7f61f8100d60 2024-02-16T19:29:42.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:42 smithi117 ceph-mon[35309]: pgmap v24: 65 pgs: 65 active+clean; 2.1 GiB data, 6.5 GiB used, 530 GiB / 536 GiB avail; 11 MiB/s rd, 31 MiB/s wr, 258 op/s 2024-02-16T19:29:42.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:42 smithi117 ceph-mon[35309]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' 2024-02-16T19:29:42.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:42 smithi117 ceph-mon[35309]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' 2024-02-16T19:29:42.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:42 smithi117 ceph-mon[35309]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:29:43.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.183+0000 7f6200087700 1 -- 172.21.15.22:0/3876275041 --> [v2:172.21.15.117:6828/10500008,v1:172.21.15.117:6829/10500008] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f61f8061190 con 0x7f61e406cf60 2024-02-16T19:29:43.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.194+0000 7f61eeffd700 1 -- 172.21.15.22:0/3876275041 <== mgr.24349 v2:172.21.15.117:6828/10500008 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f61f8061190 con 0x7f61e406cf60 2024-02-16T19:29:43.195 INFO:teuthology.orchestra.run.smithi022.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T19:29:43.195 INFO:teuthology.orchestra.run.smithi022.stdout:alertmanager.smithi022 smithi022 *:9093,9094 running (7m) 16s ago 8m 26.6M - 0.25.0 c8568f914cd2 4dac585d8372 2024-02-16T19:29:43.195 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-exporter.smithi022 smithi022 running (9m) 16s ago 9m 7638k - 18.2.0 dc2bc1663786 3f8bef4349ba 2024-02-16T19:29:43.195 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-exporter.smithi117 smithi117 running (8m) 30s ago 8m 7352k - 18.2.0 dc2bc1663786 59b65a7e309f 2024-02-16T19:29:43.195 INFO:teuthology.orchestra.run.smithi022.stdout:crash.smithi022 smithi022 running (9m) 16s ago 9m 6669k - 18.2.0 dc2bc1663786 efef476aa486 2024-02-16T19:29:43.195 INFO:teuthology.orchestra.run.smithi022.stdout:crash.smithi117 smithi117 running (8m) 30s ago 8m 6660k - 18.2.0 dc2bc1663786 5466453eb149 2024-02-16T19:29:43.195 INFO:teuthology.orchestra.run.smithi022.stdout:grafana.smithi022 smithi022 *:3000 running (7m) 16s ago 8m 85.0M - 9.4.7 954c08fa6188 a48872ecbbf2 2024-02-16T19:29:43.195 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi022.btppwm smithi022 running (3m) 16s ago 3m 228M - 18.2.0 dc2bc1663786 22971fb3a907 2024-02-16T19:29:43.195 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi022.ocukvs smithi022 running (3m) 16s ago 3m 13.1M - 18.2.0 dc2bc1663786 81ee7bd5fd39 2024-02-16T19:29:43.196 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi117.ibmppn smithi117 running (3m) 30s ago 3m 14.8M - 18.2.0 dc2bc1663786 8c606cbeeb86 2024-02-16T19:29:43.196 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi117.orzors smithi117 running (3m) 30s ago 3m 13.9M - 18.2.0 dc2bc1663786 7d2bc4df3feb 2024-02-16T19:29:43.196 INFO:teuthology.orchestra.run.smithi022.stdout:mgr.smithi022.gmdcch smithi022 *:8443,9283,8765 starting - - - - 2024-02-16T19:29:43.196 INFO:teuthology.orchestra.run.smithi022.stdout:mgr.smithi117.cohbtv smithi117 *:8443,9283,8765 running (67s) 30s ago 7m 486M - 19.0.0-1450-gf78a58c0 f081f847070a 688882bc227b 2024-02-16T19:29:43.196 INFO:teuthology.orchestra.run.smithi022.stdout:mon.smithi022 smithi022 running (10m) 16s ago 10m 49.6M 2048M 18.2.0 dc2bc1663786 b4ffc3032e12 2024-02-16T19:29:43.196 INFO:teuthology.orchestra.run.smithi022.stdout:mon.smithi117 smithi117 running (7m) 30s ago 7m 39.2M 2048M 18.2.0 dc2bc1663786 aff3e21a73a0 2024-02-16T19:29:43.196 INFO:teuthology.orchestra.run.smithi022.stdout:node-exporter.smithi022 smithi022 *:9100 running (9m) 16s ago 9m 15.7M - 1.5.0 0da6a335fe13 d727f51df007 2024-02-16T19:29:43.196 INFO:teuthology.orchestra.run.smithi022.stdout:node-exporter.smithi117 smithi117 *:9100 running (7m) 30s ago 7m 16.9M - 1.5.0 0da6a335fe13 fc8b2df5a29d 2024-02-16T19:29:43.196 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0 smithi022 running (7m) 16s ago 7m 165M 1217M 18.2.0 dc2bc1663786 0027f86f16ce 2024-02-16T19:29:43.196 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1 smithi022 running (6m) 16s ago 6m 163M 1217M 18.2.0 dc2bc1663786 87300382b8a7 2024-02-16T19:29:43.196 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2 smithi022 running (6m) 16s ago 6m 153M 1217M 18.2.0 dc2bc1663786 be8a631b4f93 2024-02-16T19:29:43.196 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3 smithi117 running (5m) 30s ago 5m 202M 2240M 18.2.0 dc2bc1663786 9c0ce9a981cf 2024-02-16T19:29:43.196 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4 smithi117 running (5m) 30s ago 5m 170M 2240M 18.2.0 dc2bc1663786 1779c3496f4d 2024-02-16T19:29:43.197 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5 smithi117 running (4m) 30s ago 4m 192M 2240M 18.2.0 dc2bc1663786 fefaabca0638 2024-02-16T19:29:43.197 INFO:teuthology.orchestra.run.smithi022.stdout:prometheus.smithi022 smithi022 *:9095 starting - - - - 2024-02-16T19:29:43.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.198+0000 7f6200087700 1 -- 172.21.15.22:0/3876275041 >> [v2:172.21.15.117:6828/10500008,v1:172.21.15.117:6829/10500008] conn(0x7f61e406cf60 msgr2=0x7f61e406f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:29:43.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.198+0000 7f6200087700 1 --2- 172.21.15.22:0/3876275041 >> [v2:172.21.15.117:6828/10500008,v1:172.21.15.117:6829/10500008] conn(0x7f61e406cf60 0x7f61e406f410 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f61e8009c60 tx=0x7f61e8009260 comp rx=0 tx=0).stop 2024-02-16T19:29:43.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.198+0000 7f6200087700 1 -- 172.21.15.22:0/3876275041 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61f8100d60 msgr2=0x7f61f819c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:29:43.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.198+0000 7f6200087700 1 --2- 172.21.15.22:0/3876275041 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61f8100d60 0x7f61f819c0c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f61f4000c00 tx=0x7f61f400f690 comp rx=0 tx=0).stop 2024-02-16T19:29:43.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.198+0000 7f6200087700 1 -- 172.21.15.22:0/3876275041 shutdown_connections 2024-02-16T19:29:43.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.198+0000 7f6200087700 1 --2- 172.21.15.22:0/3876275041 >> [v2:172.21.15.117:6828/10500008,v1:172.21.15.117:6829/10500008] conn(0x7f61e406cf60 0x7f61e406f410 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:43.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.198+0000 7f6200087700 1 --2- 172.21.15.22:0/3876275041 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61f8100d60 0x7f61f819c0c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:43.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.198+0000 7f6200087700 1 --2- 172.21.15.22:0/3876275041 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61f8103680 0x7f61f819c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:43.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.198+0000 7f6200087700 1 -- 172.21.15.22:0/3876275041 >> 172.21.15.22:0/3876275041 conn(0x7f61f80fa780 msgr2=0x7f61f80fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:29:43.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.198+0000 7f6200087700 1 -- 172.21.15.22:0/3876275041 shutdown_connections 2024-02-16T19:29:43.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.198+0000 7f6200087700 1 -- 172.21.15.22:0/3876275041 wait complete. 2024-02-16T19:29:43.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.319+0000 7f34e21eb700 1 -- 172.21.15.22:0/3671026019 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34dc0ff150 msgr2=0x7f34dc0ff560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:29:43.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.319+0000 7f34e21eb700 1 --2- 172.21.15.22:0/3671026019 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34dc0ff150 0x7f34dc0ff560 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f34cc009870 tx=0x7f34cc009b80 comp rx=0 tx=0).stop 2024-02-16T19:29:43.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.319+0000 7f34e21eb700 1 -- 172.21.15.22:0/3671026019 shutdown_connections 2024-02-16T19:29:43.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.319+0000 7f34e21eb700 1 --2- 172.21.15.22:0/3671026019 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34dc1003f0 0x7f34dc100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:43.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.319+0000 7f34e21eb700 1 --2- 172.21.15.22:0/3671026019 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34dc0ff150 0x7f34dc0ff560 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:43.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.319+0000 7f34e21eb700 1 -- 172.21.15.22:0/3671026019 >> 172.21.15.22:0/3671026019 conn(0x7f34dc0fa7a0 msgr2=0x7f34dc0fcbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:29:43.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.320+0000 7f34e21eb700 1 -- 172.21.15.22:0/3671026019 shutdown_connections 2024-02-16T19:29:43.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.320+0000 7f34e21eb700 1 -- 172.21.15.22:0/3671026019 wait complete. 2024-02-16T19:29:43.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.321+0000 7f34e21eb700 1 Processor -- start 2024-02-16T19:29:43.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.321+0000 7f34e21eb700 1 -- start start 2024-02-16T19:29:43.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.322+0000 7f34e21eb700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34dc0ff150 0x7f34dc195c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:29:43.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.322+0000 7f34e21eb700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34dc1003f0 0x7f34dc196150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:29:43.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.322+0000 7f34e21eb700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34dc196770 con 0x7f34dc1003f0 2024-02-16T19:29:43.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.322+0000 7f34e21eb700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34dc19b180 con 0x7f34dc0ff150 2024-02-16T19:29:43.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.322+0000 7f34daffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34dc1003f0 0x7f34dc196150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:29:43.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.322+0000 7f34daffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34dc1003f0 0x7f34dc196150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59090/0 (socket says 172.21.15.22:59090) 2024-02-16T19:29:43.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.322+0000 7f34daffd700 1 -- 172.21.15.22:0/3925605687 learned_addr learned my addr 172.21.15.22:0/3925605687 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:29:43.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.322+0000 7f34db7fe700 1 --2- 172.21.15.22:0/3925605687 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34dc0ff150 0x7f34dc195c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:29:43.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.322+0000 7f34db7fe700 1 -- 172.21.15.22:0/3925605687 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34dc1003f0 msgr2=0x7f34dc196150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:29:43.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.322+0000 7f34db7fe700 1 --2- 172.21.15.22:0/3925605687 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34dc1003f0 0x7f34dc196150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:43.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.322+0000 7f34db7fe700 1 -- 172.21.15.22:0/3925605687 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34d00095f0 con 0x7f34dc0ff150 2024-02-16T19:29:43.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.322+0000 7f34daffd700 1 --2- 172.21.15.22:0/3925605687 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34dc1003f0 0x7f34dc196150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T19:29:43.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.323+0000 7f34db7fe700 1 --2- 172.21.15.22:0/3925605687 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34dc0ff150 0x7f34dc195c10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f34cc009870 tx=0x7f34cc004700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:29:43.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.323+0000 7f34d8ff9700 1 -- 172.21.15.22:0/3925605687 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34cc01c070 con 0x7f34dc0ff150 2024-02-16T19:29:43.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.323+0000 7f34e21eb700 1 -- 172.21.15.22:0/3925605687 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34cc009520 con 0x7f34dc0ff150 2024-02-16T19:29:43.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.323+0000 7f34d8ff9700 1 -- 172.21.15.22:0/3925605687 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f34cc003d00 con 0x7f34dc0ff150 2024-02-16T19:29:43.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.323+0000 7f34d8ff9700 1 -- 172.21.15.22:0/3925605687 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34cc022420 con 0x7f34dc0ff150 2024-02-16T19:29:43.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.323+0000 7f34e21eb700 1 -- 172.21.15.22:0/3925605687 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34dc101cc0 con 0x7f34dc0ff150 2024-02-16T19:29:43.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.324+0000 7f34e21eb700 1 -- 172.21.15.22:0/3925605687 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34dc068ab0 con 0x7f34dc0ff150 2024-02-16T19:29:43.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.325+0000 7f34d8ff9700 1 -- 172.21.15.22:0/3925605687 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 25) v1 ==== 90734+0+0 (secure 0 0 0) 0x7f34cc020070 con 0x7f34dc0ff150 2024-02-16T19:29:43.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.325+0000 7f34d8ff9700 1 --2- 172.21.15.22:0/3925605687 >> [v2:172.21.15.117:6828/10500008,v1:172.21.15.117:6829/10500008] conn(0x7f34c406cf00 0x7f34c406f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:29:43.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.325+0000 7f34d8ff9700 1 -- 172.21.15.22:0/3925605687 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f34cc08da10 con 0x7f34dc0ff150 2024-02-16T19:29:43.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.326+0000 7f34daffd700 1 --2- 172.21.15.22:0/3925605687 >> [v2:172.21.15.117:6828/10500008,v1:172.21.15.117:6829/10500008] conn(0x7f34c406cf00 0x7f34c406f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:29:43.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.326+0000 7f34daffd700 1 --2- 172.21.15.22:0/3925605687 >> [v2:172.21.15.117:6828/10500008,v1:172.21.15.117:6829/10500008] conn(0x7f34c406cf00 0x7f34c406f3b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f34d0005d10 tx=0x7f34d0009310 comp rx=0 tx=0).ready entity=mgr.24349 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:29:43.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.330+0000 7f34d8ff9700 1 -- 172.21.15.22:0/3925605687 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f34cc057580 con 0x7f34dc0ff150 2024-02-16T19:29:43.626 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:43 smithi022 ceph-mon[29221]: from='client.24411 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:29:43.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.808+0000 7f34e21eb700 1 -- 172.21.15.22:0/3925605687 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f34dc101ee0 con 0x7f34dc0ff150 2024-02-16T19:29:43.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.809+0000 7f34d8ff9700 1 -- 172.21.15.22:0/3925605687 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+766 (secure 0 0 0) 0x7f34cc05ac40 con 0x7f34dc0ff150 2024-02-16T19:29:43.809 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-02-16T19:29:43.809 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": { 2024-02-16T19:29:43.809 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-02-16T19:29:43.809 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:29:43.809 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": { 2024-02-16T19:29:43.809 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-16T19:29:43.809 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 1 2024-02-16T19:29:43.809 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:29:43.809 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": { 2024-02-16T19:29:43.810 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-02-16T19:29:43.810 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:29:43.810 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": { 2024-02-16T19:29:43.810 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-02-16T19:29:43.810 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:29:43.810 INFO:teuthology.orchestra.run.smithi022.stdout: "overall": { 2024-02-16T19:29:43.810 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 13, 2024-02-16T19:29:43.810 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 1 2024-02-16T19:29:43.810 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-02-16T19:29:43.810 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-02-16T19:29:43.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.812+0000 7f34e21eb700 1 -- 172.21.15.22:0/3925605687 >> [v2:172.21.15.117:6828/10500008,v1:172.21.15.117:6829/10500008] conn(0x7f34c406cf00 msgr2=0x7f34c406f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:29:43.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.812+0000 7f34e21eb700 1 --2- 172.21.15.22:0/3925605687 >> [v2:172.21.15.117:6828/10500008,v1:172.21.15.117:6829/10500008] conn(0x7f34c406cf00 0x7f34c406f3b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f34d0005d10 tx=0x7f34d0009310 comp rx=0 tx=0).stop 2024-02-16T19:29:43.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.812+0000 7f34e21eb700 1 -- 172.21.15.22:0/3925605687 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34dc0ff150 msgr2=0x7f34dc195c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:29:43.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.812+0000 7f34e21eb700 1 --2- 172.21.15.22:0/3925605687 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34dc0ff150 0x7f34dc195c10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f34cc009870 tx=0x7f34cc004700 comp rx=0 tx=0).stop 2024-02-16T19:29:43.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.812+0000 7f34e21eb700 1 -- 172.21.15.22:0/3925605687 shutdown_connections 2024-02-16T19:29:43.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.812+0000 7f34e21eb700 1 --2- 172.21.15.22:0/3925605687 >> [v2:172.21.15.117:6828/10500008,v1:172.21.15.117:6829/10500008] conn(0x7f34c406cf00 0x7f34c406f3b0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:43.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.812+0000 7f34e21eb700 1 --2- 172.21.15.22:0/3925605687 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34dc0ff150 0x7f34dc195c10 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:43.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.812+0000 7f34e21eb700 1 --2- 172.21.15.22:0/3925605687 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34dc1003f0 0x7f34dc196150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:43.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.812+0000 7f34e21eb700 1 -- 172.21.15.22:0/3925605687 >> 172.21.15.22:0/3925605687 conn(0x7f34dc0fa7a0 msgr2=0x7f34dc068dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:29:43.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.813+0000 7f34e21eb700 1 -- 172.21.15.22:0/3925605687 shutdown_connections 2024-02-16T19:29:43.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.813+0000 7f34e21eb700 1 -- 172.21.15.22:0/3925605687 wait complete. 2024-02-16T19:29:43.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:43 smithi117 ceph-mon[35309]: from='client.24411 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:29:43.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.944+0000 7fa373049700 1 -- 172.21.15.22:0/3972230318 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa36c0ff150 msgr2=0x7fa36c0ff560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:29:43.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.944+0000 7fa373049700 1 --2- 172.21.15.22:0/3972230318 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa36c0ff150 0x7fa36c0ff560 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa360009910 tx=0x7fa360009c20 comp rx=0 tx=0).stop 2024-02-16T19:29:43.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.944+0000 7fa373049700 1 -- 172.21.15.22:0/3972230318 shutdown_connections 2024-02-16T19:29:43.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.944+0000 7fa373049700 1 --2- 172.21.15.22:0/3972230318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa36c100350 0x7fa36c1007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:43.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.944+0000 7fa373049700 1 --2- 172.21.15.22:0/3972230318 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa36c0ff150 0x7fa36c0ff560 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:43.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.944+0000 7fa373049700 1 -- 172.21.15.22:0/3972230318 >> 172.21.15.22:0/3972230318 conn(0x7fa36c0fa700 msgr2=0x7fa36c0fcb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:29:43.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.945+0000 7fa373049700 1 -- 172.21.15.22:0/3972230318 shutdown_connections 2024-02-16T19:29:43.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.945+0000 7fa373049700 1 -- 172.21.15.22:0/3972230318 wait complete. 2024-02-16T19:29:43.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.946+0000 7fa373049700 1 Processor -- start 2024-02-16T19:29:43.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.946+0000 7fa373049700 1 -- start start 2024-02-16T19:29:43.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.946+0000 7fa373049700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa36c0ff150 0x7fa36c1949e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:29:43.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.946+0000 7fa373049700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa36c100350 0x7fa36c194f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:29:43.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.946+0000 7fa373049700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa36c195540 con 0x7fa36c0ff150 2024-02-16T19:29:43.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.946+0000 7fa373049700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa36c195680 con 0x7fa36c100350 2024-02-16T19:29:43.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.946+0000 7fa370de5700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa36c0ff150 0x7fa36c1949e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:29:43.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.946+0000 7fa36bfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa36c100350 0x7fa36c194f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:29:43.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.946+0000 7fa370de5700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa36c0ff150 0x7fa36c1949e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59110/0 (socket says 172.21.15.22:59110) 2024-02-16T19:29:43.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.946+0000 7fa36bfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa36c100350 0x7fa36c194f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.22:49302/0 (socket says 172.21.15.22:49302) 2024-02-16T19:29:43.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.946+0000 7fa370de5700 1 -- 172.21.15.22:0/2620846427 learned_addr learned my addr 172.21.15.22:0/2620846427 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:29:43.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.947+0000 7fa36bfff700 1 -- 172.21.15.22:0/2620846427 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa36c0ff150 msgr2=0x7fa36c1949e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:29:43.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.947+0000 7fa36bfff700 1 --2- 172.21.15.22:0/2620846427 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa36c0ff150 0x7fa36c1949e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:43.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.947+0000 7fa36bfff700 1 -- 172.21.15.22:0/2620846427 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3600095f0 con 0x7fa36c100350 2024-02-16T19:29:43.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.947+0000 7fa370de5700 1 --2- 172.21.15.22:0/2620846427 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa36c0ff150 0x7fa36c1949e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T19:29:43.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.947+0000 7fa36bfff700 1 --2- 172.21.15.22:0/2620846427 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa36c100350 0x7fa36c194f20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa35800e970 tx=0x7fa35800ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:29:43.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.947+0000 7fa369ffb700 1 -- 172.21.15.22:0/2620846427 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa35800c9d0 con 0x7fa36c100350 2024-02-16T19:29:43.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.947+0000 7fa373049700 1 -- 172.21.15.22:0/2620846427 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa36c068f70 con 0x7fa36c100350 2024-02-16T19:29:43.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.947+0000 7fa369ffb700 1 -- 172.21.15.22:0/2620846427 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa358010430 con 0x7fa36c100350 2024-02-16T19:29:43.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.947+0000 7fa369ffb700 1 -- 172.21.15.22:0/2620846427 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa358018650 con 0x7fa36c100350 2024-02-16T19:29:43.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.948+0000 7fa373049700 1 -- 172.21.15.22:0/2620846427 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa36c0694c0 con 0x7fa36c100350 2024-02-16T19:29:43.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.949+0000 7fa373049700 1 -- 172.21.15.22:0/2620846427 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa36c107ca0 con 0x7fa36c100350 2024-02-16T19:29:43.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.950+0000 7fa369ffb700 1 -- 172.21.15.22:0/2620846427 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 25) v1 ==== 90734+0+0 (secure 0 0 0) 0x7fa35800cb30 con 0x7fa36c100350 2024-02-16T19:29:43.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.950+0000 7fa369ffb700 1 --2- 172.21.15.22:0/2620846427 >> [v2:172.21.15.117:6828/10500008,v1:172.21.15.117:6829/10500008] conn(0x7fa35c06cf10 0x7fa35c06f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:29:43.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.950+0000 7fa369ffb700 1 -- 172.21.15.22:0/2620846427 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fa358014070 con 0x7fa36c100350 2024-02-16T19:29:43.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.950+0000 7fa370de5700 1 --2- 172.21.15.22:0/2620846427 >> [v2:172.21.15.117:6828/10500008,v1:172.21.15.117:6829/10500008] conn(0x7fa35c06cf10 0x7fa35c06f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:29:43.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.951+0000 7fa370de5700 1 --2- 172.21.15.22:0/2620846427 >> [v2:172.21.15.117:6828/10500008,v1:172.21.15.117:6829/10500008] conn(0x7fa35c06cf10 0x7fa35c06f3c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa360009fd0 tx=0x7fa360005940 comp rx=0 tx=0).ready entity=mgr.24349 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:29:43.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:43.955+0000 7fa369ffb700 1 -- 172.21.15.22:0/2620846427 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7fa358052a50 con 0x7fa36c100350 2024-02-16T19:29:44.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:44.255+0000 7fa373049700 1 -- 172.21.15.22:0/2620846427 --> [v2:172.21.15.117:6828/10500008,v1:172.21.15.117:6829/10500008] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa36c104ca0 con 0x7fa35c06cf10 2024-02-16T19:29:44.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:44.257+0000 7fa369ffb700 1 -- 172.21.15.22:0/2620846427 <== mgr.24349 v2:172.21.15.117:6828/10500008 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+321 (secure 0 0 0) 0x7fa36c104ca0 con 0x7fa35c06cf10 2024-02-16T19:29:44.257 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-02-16T19:29:44.257 INFO:teuthology.orchestra.run.smithi022.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T19:29:44.257 INFO:teuthology.orchestra.run.smithi022.stdout: "in_progress": true, 2024-02-16T19:29:44.257 INFO:teuthology.orchestra.run.smithi022.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-02-16T19:29:44.257 INFO:teuthology.orchestra.run.smithi022.stdout: "services_complete": [], 2024-02-16T19:29:44.257 INFO:teuthology.orchestra.run.smithi022.stdout: "progress": "", 2024-02-16T19:29:44.258 INFO:teuthology.orchestra.run.smithi022.stdout: "message": "Currently upgrading mgr daemons", 2024-02-16T19:29:44.258 INFO:teuthology.orchestra.run.smithi022.stdout: "is_paused": false 2024-02-16T19:29:44.258 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-02-16T19:29:44.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:44.259+0000 7fa373049700 1 -- 172.21.15.22:0/2620846427 >> [v2:172.21.15.117:6828/10500008,v1:172.21.15.117:6829/10500008] conn(0x7fa35c06cf10 msgr2=0x7fa35c06f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:29:44.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:44.259+0000 7fa373049700 1 --2- 172.21.15.22:0/2620846427 >> [v2:172.21.15.117:6828/10500008,v1:172.21.15.117:6829/10500008] conn(0x7fa35c06cf10 0x7fa35c06f3c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa360009fd0 tx=0x7fa360005940 comp rx=0 tx=0).stop 2024-02-16T19:29:44.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:44.259+0000 7fa373049700 1 -- 172.21.15.22:0/2620846427 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa36c100350 msgr2=0x7fa36c194f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:29:44.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:44.259+0000 7fa373049700 1 --2- 172.21.15.22:0/2620846427 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa36c100350 0x7fa36c194f20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa35800e970 tx=0x7fa35800ec80 comp rx=0 tx=0).stop 2024-02-16T19:29:44.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:44.259+0000 7fa373049700 1 -- 172.21.15.22:0/2620846427 shutdown_connections 2024-02-16T19:29:44.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:44.259+0000 7fa373049700 1 --2- 172.21.15.22:0/2620846427 >> [v2:172.21.15.117:6828/10500008,v1:172.21.15.117:6829/10500008] conn(0x7fa35c06cf10 0x7fa35c06f3c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:44.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:44.259+0000 7fa373049700 1 --2- 172.21.15.22:0/2620846427 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa36c0ff150 0x7fa36c1949e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:44.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:44.260+0000 7fa373049700 1 --2- 172.21.15.22:0/2620846427 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa36c100350 0x7fa36c194f20 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:44.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:44.260+0000 7fa373049700 1 -- 172.21.15.22:0/2620846427 >> 172.21.15.22:0/2620846427 conn(0x7fa36c0fa700 msgr2=0x7fa36c103580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:29:44.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:44.260+0000 7fa373049700 1 -- 172.21.15.22:0/2620846427 shutdown_connections 2024-02-16T19:29:44.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:29:44.260+0000 7fa373049700 1 -- 172.21.15.22:0/2620846427 wait complete. 2024-02-16T19:29:44.546 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:44 smithi022 ceph-mon[29221]: from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:29:44.546 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:44 smithi022 ceph-mon[29221]: from='client.24419 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:29:44.546 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:44 smithi022 ceph-mon[29221]: pgmap v25: 65 pgs: 65 active+clean; 2.1 GiB data, 6.5 GiB used, 530 GiB / 536 GiB avail; 22 KiB/s rd, 1.2 MiB/s wr, 144 op/s 2024-02-16T19:29:44.546 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:44 smithi022 ceph-mon[29221]: from='client.? 172.21.15.22:0/3925605687' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:29:44.886 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:44 smithi117 ceph-mon[35309]: from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:29:44.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:44 smithi117 ceph-mon[35309]: from='client.24419 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:29:44.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:44 smithi117 ceph-mon[35309]: pgmap v25: 65 pgs: 65 active+clean; 2.1 GiB data, 6.5 GiB used, 530 GiB / 536 GiB avail; 22 KiB/s rd, 1.2 MiB/s wr, 144 op/s 2024-02-16T19:29:44.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:44 smithi117 ceph-mon[35309]: from='client.? 172.21.15.22:0/3925605687' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:29:45.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:45 smithi117 ceph-mon[35309]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:29:45.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:45 smithi117 ceph-mon[35309]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:29:45.964 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:45 smithi022 ceph-mon[29221]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:29:45.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:45 smithi022 ceph-mon[29221]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:29:46.128 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-02-16T19:29:46.128 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-02-16T19:29:46.509 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:46 smithi022 ceph-mon[29221]: pgmap v26: 65 pgs: 65 active+clean; 1.9 GiB data, 5.8 GiB used, 531 GiB / 536 GiB avail; 22 KiB/s rd, 1.2 MiB/s wr, 149 op/s 2024-02-16T19:29:46.787 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-02-16T19:29:46.788 DEBUG:teuthology.orchestra.run.smithi022:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-02-16T19:29:46.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:46 smithi117 ceph-mon[35309]: pgmap v26: 65 pgs: 65 active+clean; 1.9 GiB data, 5.8 GiB used, 531 GiB / 536 GiB avail; 22 KiB/s rd, 1.2 MiB/s wr, 149 op/s 2024-02-16T19:29:48.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:47 smithi022 ceph-mon[29221]: pgmap v27: 65 pgs: 65 active+clean; 1002 MiB data, 3.3 GiB used, 533 GiB / 536 GiB avail; 32 KiB/s rd, 1.5 MiB/s wr, 183 op/s 2024-02-16T19:29:48.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:47 smithi117 ceph-mon[35309]: pgmap v27: 65 pgs: 65 active+clean; 1002 MiB data, 3.3 GiB used, 533 GiB / 536 GiB avail; 32 KiB/s rd, 1.5 MiB/s wr, 183 op/s 2024-02-16T19:29:49.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:48 smithi022 ceph-mon[29221]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' 2024-02-16T19:29:49.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:48 smithi022 ceph-mon[29221]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' 2024-02-16T19:29:49.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:48 smithi117 ceph-mon[35309]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' 2024-02-16T19:29:49.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:48 smithi117 ceph-mon[35309]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' 2024-02-16T19:29:50.590 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:50 smithi022 ceph-mon[29221]: pgmap v28: 65 pgs: 65 active+clean; 1002 MiB data, 3.3 GiB used, 533 GiB / 536 GiB avail; 22 KiB/s rd, 1.0 MiB/s wr, 135 op/s 2024-02-16T19:29:50.590 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:50 smithi022 ceph-mon[29221]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' 2024-02-16T19:29:50.590 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:50 smithi022 ceph-mon[29221]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' 2024-02-16T19:29:50.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:50 smithi117 ceph-mon[35309]: pgmap v28: 65 pgs: 65 active+clean; 1002 MiB data, 3.3 GiB used, 533 GiB / 536 GiB avail; 22 KiB/s rd, 1.0 MiB/s wr, 135 op/s 2024-02-16T19:29:50.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:50 smithi117 ceph-mon[35309]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' 2024-02-16T19:29:50.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:50 smithi117 ceph-mon[35309]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' 2024-02-16T19:29:51.712 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:51 smithi022 ceph-mon[29221]: Standby manager daemon smithi022.gmdcch restarted 2024-02-16T19:29:51.713 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:51 smithi022 ceph-mon[29221]: Standby manager daemon smithi022.gmdcch started 2024-02-16T19:29:51.713 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:51 smithi022 ceph-mon[29221]: from='mgr.? 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi022.gmdcch/crt"}]: dispatch 2024-02-16T19:29:51.713 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:51 smithi022 ceph-mon[29221]: from='mgr.? 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-16T19:29:51.713 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:51 smithi022 ceph-mon[29221]: from='mgr.? 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi022.gmdcch/key"}]: dispatch 2024-02-16T19:29:51.713 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:51 smithi022 ceph-mon[29221]: from='mgr.? 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-16T19:29:51.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:51 smithi117 ceph-mon[35309]: Standby manager daemon smithi022.gmdcch restarted 2024-02-16T19:29:51.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:51 smithi117 ceph-mon[35309]: Standby manager daemon smithi022.gmdcch started 2024-02-16T19:29:51.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:51 smithi117 ceph-mon[35309]: from='mgr.? 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi022.gmdcch/crt"}]: dispatch 2024-02-16T19:29:51.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:51 smithi117 ceph-mon[35309]: from='mgr.? 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-16T19:29:51.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:51 smithi117 ceph-mon[35309]: from='mgr.? 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi022.gmdcch/key"}]: dispatch 2024-02-16T19:29:51.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:51 smithi117 ceph-mon[35309]: from='mgr.? 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-16T19:29:52.662 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:52 smithi022 ceph-mon[29221]: pgmap v29: 65 pgs: 65 active+clean; 282 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 31 KiB/s rd, 1.5 MiB/s wr, 194 op/s 2024-02-16T19:29:52.663 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:52 smithi022 ceph-mon[29221]: mgrmap e26: smithi117.cohbtv(active, since 52s), standbys: smithi022.gmdcch 2024-02-16T19:29:52.787 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:52 smithi117 ceph-mon[35309]: pgmap v29: 65 pgs: 65 active+clean; 282 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 31 KiB/s rd, 1.5 MiB/s wr, 194 op/s 2024-02-16T19:29:52.788 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:52 smithi117 ceph-mon[35309]: mgrmap e26: smithi117.cohbtv(active, since 52s), standbys: smithi022.gmdcch 2024-02-16T19:29:54.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:54 smithi117 ceph-mon[35309]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' 2024-02-16T19:29:54.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:54 smithi117 ceph-mon[35309]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' 2024-02-16T19:29:54.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:54 smithi117 ceph-mon[35309]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:29:54.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:54 smithi117 ceph-mon[35309]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:29:54.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:54 smithi117 ceph-mon[35309]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' 2024-02-16T19:29:54.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:54 smithi117 ceph-mon[35309]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:29:54.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:54 smithi117 ceph-mon[35309]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "mgr fail", "who": "smithi117.cohbtv"}]: dispatch 2024-02-16T19:29:54.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:54 smithi117 ceph-mon[35309]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "mgr fail", "who": "smithi117.cohbtv"}]: dispatch 2024-02-16T19:29:54.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:54 smithi117 ceph-mon[35309]: osdmap e49: 6 total, 6 up, 6 in 2024-02-16T19:29:54.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:54 smithi117 ceph-mon[35309]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' cmd='[{"prefix": "mgr fail", "who": "smithi117.cohbtv"}]': finished 2024-02-16T19:29:54.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:54 smithi117 ceph-mon[35309]: mgrmap e27: smithi022.gmdcch(active, starting, since 0.00771266s) 2024-02-16T19:29:54.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:54 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mon metadata", "id": "smithi022"}]: dispatch 2024-02-16T19:29:54.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:54 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mon metadata", "id": "smithi117"}]: dispatch 2024-02-16T19:29:54.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:54 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi022.btppwm"}]: dispatch 2024-02-16T19:29:54.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:54 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi022.ocukvs"}]: dispatch 2024-02-16T19:29:54.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:54 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi117.ibmppn"}]: dispatch 2024-02-16T19:29:54.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:54 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi117.orzors"}]: dispatch 2024-02-16T19:29:54.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:54 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mgr metadata", "who": "smithi022.gmdcch", "id": "smithi022.gmdcch"}]: dispatch 2024-02-16T19:29:54.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:54 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-16T19:29:54.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:54 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-16T19:29:54.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:54 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-16T19:29:54.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:54 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-16T19:29:54.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:54 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-16T19:29:54.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:54 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-16T19:29:54.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:54 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-16T19:29:54.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:54 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-16T19:29:54.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:54 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-16T19:29:54.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:54 smithi117 ceph-mon[35309]: Manager daemon smithi022.gmdcch is now available 2024-02-16T19:29:54.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:54 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:29:54.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:54 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:29:54.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:54 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:29:54.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:54 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi022.gmdcch/mirror_snapshot_schedule"}]: dispatch 2024-02-16T19:29:54.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:54 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi022.gmdcch/trash_purge_schedule"}]: dispatch 2024-02-16T19:29:54.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:54 smithi022 ceph-mon[29221]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' 2024-02-16T19:29:54.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:54 smithi022 ceph-mon[29221]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' 2024-02-16T19:29:54.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:54 smithi022 ceph-mon[29221]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:29:54.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:54 smithi022 ceph-mon[29221]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:29:54.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:54 smithi022 ceph-mon[29221]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' 2024-02-16T19:29:54.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:54 smithi022 ceph-mon[29221]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:29:54.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:54 smithi022 ceph-mon[29221]: from='mgr.24349 172.21.15.117:0/2714578038' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "mgr fail", "who": "smithi117.cohbtv"}]: dispatch 2024-02-16T19:29:54.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:54 smithi022 ceph-mon[29221]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "mgr fail", "who": "smithi117.cohbtv"}]: dispatch 2024-02-16T19:29:54.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:54 smithi022 ceph-mon[29221]: osdmap e49: 6 total, 6 up, 6 in 2024-02-16T19:29:54.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:54 smithi022 ceph-mon[29221]: from='mgr.24349 ' entity='mgr.smithi117.cohbtv' cmd='[{"prefix": "mgr fail", "who": "smithi117.cohbtv"}]': finished 2024-02-16T19:29:54.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:54 smithi022 ceph-mon[29221]: mgrmap e27: smithi022.gmdcch(active, starting, since 0.00771266s) 2024-02-16T19:29:54.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:54 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mon metadata", "id": "smithi022"}]: dispatch 2024-02-16T19:29:54.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:54 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mon metadata", "id": "smithi117"}]: dispatch 2024-02-16T19:29:54.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:54 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi022.btppwm"}]: dispatch 2024-02-16T19:29:54.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:54 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi022.ocukvs"}]: dispatch 2024-02-16T19:29:54.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:54 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi117.ibmppn"}]: dispatch 2024-02-16T19:29:54.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:54 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi117.orzors"}]: dispatch 2024-02-16T19:29:54.467 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:54 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mgr metadata", "who": "smithi022.gmdcch", "id": "smithi022.gmdcch"}]: dispatch 2024-02-16T19:29:54.467 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:54 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-16T19:29:54.467 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:54 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-16T19:29:54.467 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:54 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-16T19:29:54.467 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:54 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-16T19:29:54.467 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:54 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-16T19:29:54.467 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:54 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-16T19:29:54.467 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:54 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-16T19:29:54.467 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:54 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-16T19:29:54.467 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:54 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-16T19:29:54.467 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:54 smithi022 ceph-mon[29221]: Manager daemon smithi022.gmdcch is now available 2024-02-16T19:29:54.467 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:54 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:29:54.467 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:54 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:29:54.467 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:54 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:29:54.467 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:54 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi022.gmdcch/mirror_snapshot_schedule"}]: dispatch 2024-02-16T19:29:54.467 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:54 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi022.gmdcch/trash_purge_schedule"}]: dispatch 2024-02-16T19:29:55.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:55 smithi022 ceph-mon[29221]: [16/Feb/2024:19:29:54] ENGINE Bus STARTING 2024-02-16T19:29:55.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:55 smithi022 ceph-mon[29221]: mgrmap e28: smithi022.gmdcch(active, since 1.01036s) 2024-02-16T19:29:55.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:55 smithi022 ceph-mon[29221]: pgmap v3: 65 pgs: 65 active+clean; 286 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:29:55.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:55 smithi022 ceph-mon[29221]: [16/Feb/2024:19:29:54] ENGINE Serving on https://172.21.15.22:7150 2024-02-16T19:29:55.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:55 smithi022 ceph-mon[29221]: [16/Feb/2024:19:29:54] ENGINE Serving on http://172.21.15.22:8765 2024-02-16T19:29:55.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:55 smithi022 ceph-mon[29221]: [16/Feb/2024:19:29:54] ENGINE Bus STARTED 2024-02-16T19:29:55.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:55 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:29:55.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:55 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:29:55.492 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:55 smithi117 ceph-mon[35309]: [16/Feb/2024:19:29:54] ENGINE Bus STARTING 2024-02-16T19:29:55.492 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:55 smithi117 ceph-mon[35309]: mgrmap e28: smithi022.gmdcch(active, since 1.01036s) 2024-02-16T19:29:55.492 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:55 smithi117 ceph-mon[35309]: pgmap v3: 65 pgs: 65 active+clean; 286 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:29:55.493 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:55 smithi117 ceph-mon[35309]: [16/Feb/2024:19:29:54] ENGINE Serving on https://172.21.15.22:7150 2024-02-16T19:29:55.493 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:55 smithi117 ceph-mon[35309]: [16/Feb/2024:19:29:54] ENGINE Serving on http://172.21.15.22:8765 2024-02-16T19:29:55.493 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:55 smithi117 ceph-mon[35309]: [16/Feb/2024:19:29:54] ENGINE Bus STARTED 2024-02-16T19:29:55.493 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:55 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:29:55.493 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:55 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:29:56.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:56 smithi022 ceph-mon[29221]: pgmap v4: 65 pgs: 65 active+clean; 286 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:29:56.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:56 smithi022 ceph-mon[29221]: mgrmap e29: smithi022.gmdcch(active, since 2s) 2024-02-16T19:29:56.886 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:56 smithi117 ceph-mon[35309]: pgmap v4: 65 pgs: 65 active+clean; 286 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:29:56.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:56 smithi117 ceph-mon[35309]: mgrmap e29: smithi022.gmdcch(active, since 2s) 2024-02-16T19:29:58.087 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:57 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:29:58.087 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:57 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:29:58.087 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:57 smithi117 ceph-mon[35309]: pgmap v5: 65 pgs: 65 active+clean; 286 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:29:58.153 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:57 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:29:58.153 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:57 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:29:58.153 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:57 smithi022 ceph-mon[29221]: pgmap v5: 65 pgs: 65 active+clean; 286 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:29:59.151 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:58 smithi117 ceph-mon[35309]: mgrmap e30: smithi022.gmdcch(active, since 4s) 2024-02-16T19:29:59.151 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:58 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:29:59.151 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:58 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:29:59.168 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:58 smithi022 ceph-mon[29221]: mgrmap e30: smithi022.gmdcch(active, since 4s) 2024-02-16T19:29:59.168 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:58 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:29:59.168 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:58 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:30:00.200 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:29:59 smithi117 ceph-mon[35309]: pgmap v6: 65 pgs: 65 active+clean; 287 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 34 KiB/s rd, 263 KiB/s wr, 25 op/s 2024-02-16T19:30:00.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:29:59 smithi022 ceph-mon[29221]: pgmap v6: 65 pgs: 65 active+clean; 287 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 34 KiB/s rd, 263 KiB/s wr, 25 op/s 2024-02-16T19:30:01.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:00 smithi022 ceph-mon[29221]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-02-16T19:30:01.216 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:00 smithi022 ceph-mon[29221]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-02-16T19:30:01.216 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:00 smithi022 ceph-mon[29221]: fs cephfs has deprecated feature inline_data enabled. 2024-02-16T19:30:01.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:00 smithi117 ceph-mon[35309]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-02-16T19:30:01.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:00 smithi117 ceph-mon[35309]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-02-16T19:30:01.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:00 smithi117 ceph-mon[35309]: fs cephfs has deprecated feature inline_data enabled. 2024-02-16T19:30:02.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:01 smithi022 ceph-mon[29221]: Standby manager daemon smithi117.cohbtv started 2024-02-16T19:30:02.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:01 smithi022 ceph-mon[29221]: from='mgr.? 172.21.15.117:0/2664596086' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi117.cohbtv/crt"}]: dispatch 2024-02-16T19:30:02.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:01 smithi022 ceph-mon[29221]: from='mgr.? 172.21.15.117:0/2664596086' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-16T19:30:02.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:01 smithi022 ceph-mon[29221]: from='mgr.? 172.21.15.117:0/2664596086' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi117.cohbtv/key"}]: dispatch 2024-02-16T19:30:02.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:01 smithi022 ceph-mon[29221]: from='mgr.? 172.21.15.117:0/2664596086' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-16T19:30:02.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:01 smithi022 ceph-mon[29221]: pgmap v7: 65 pgs: 65 active+clean; 293 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 54 KiB/s rd, 1.1 MiB/s wr, 82 op/s 2024-02-16T19:30:02.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:01 smithi117 ceph-mon[35309]: Standby manager daemon smithi117.cohbtv started 2024-02-16T19:30:02.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:01 smithi117 ceph-mon[35309]: from='mgr.? 172.21.15.117:0/2664596086' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi117.cohbtv/crt"}]: dispatch 2024-02-16T19:30:02.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:01 smithi117 ceph-mon[35309]: from='mgr.? 172.21.15.117:0/2664596086' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-16T19:30:02.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:01 smithi117 ceph-mon[35309]: from='mgr.? 172.21.15.117:0/2664596086' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi117.cohbtv/key"}]: dispatch 2024-02-16T19:30:02.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:01 smithi117 ceph-mon[35309]: from='mgr.? 172.21.15.117:0/2664596086' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-16T19:30:02.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:01 smithi117 ceph-mon[35309]: pgmap v7: 65 pgs: 65 active+clean; 293 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 54 KiB/s rd, 1.1 MiB/s wr, 82 op/s 2024-02-16T19:30:03.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:02 smithi022 ceph-mon[29221]: mgrmap e31: smithi022.gmdcch(active, since 8s), standbys: smithi117.cohbtv 2024-02-16T19:30:03.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:02 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mgr metadata", "who": "smithi117.cohbtv", "id": "smithi117.cohbtv"}]: dispatch 2024-02-16T19:30:03.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:02 smithi117 ceph-mon[35309]: mgrmap e31: smithi022.gmdcch(active, since 8s), standbys: smithi117.cohbtv 2024-02-16T19:30:03.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:02 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mgr metadata", "who": "smithi117.cohbtv", "id": "smithi117.cohbtv"}]: dispatch 2024-02-16T19:30:03.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:03 smithi022 ceph-mon[29221]: pgmap v8: 65 pgs: 65 active+clean; 293 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 42 KiB/s rd, 905 KiB/s wr, 63 op/s 2024-02-16T19:30:04.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:03 smithi117 ceph-mon[35309]: pgmap v8: 65 pgs: 65 active+clean; 293 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 42 KiB/s rd, 905 KiB/s wr, 63 op/s 2024-02-16T19:30:06.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:06 smithi117 ceph-mon[35309]: pgmap v9: 65 pgs: 65 active+clean; 301 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 50 KiB/s rd, 1.5 MiB/s wr, 88 op/s 2024-02-16T19:30:06.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:06 smithi022 ceph-mon[29221]: pgmap v9: 65 pgs: 65 active+clean; 301 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 50 KiB/s rd, 1.5 MiB/s wr, 88 op/s 2024-02-16T19:30:08.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:07 smithi022 ceph-mon[29221]: pgmap v10: 65 pgs: 65 active+clean; 301 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 46 KiB/s rd, 1.4 MiB/s wr, 81 op/s 2024-02-16T19:30:08.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:07 smithi117 ceph-mon[35309]: pgmap v10: 65 pgs: 65 active+clean; 301 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 46 KiB/s rd, 1.4 MiB/s wr, 81 op/s 2024-02-16T19:30:09.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:08 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:30:09.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:08 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:30:10.161 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:09 smithi022 ceph-mon[29221]: pgmap v11: 65 pgs: 65 active+clean; 300 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 60 KiB/s rd, 1.4 MiB/s wr, 93 op/s 2024-02-16T19:30:10.305 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:09 smithi117 ceph-mon[35309]: pgmap v11: 65 pgs: 65 active+clean; 300 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 60 KiB/s rd, 1.4 MiB/s wr, 93 op/s 2024-02-16T19:30:12.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:12 smithi117 ceph-mon[35309]: pgmap v12: 65 pgs: 65 active+clean; 307 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 46 KiB/s rd, 1.9 MiB/s wr, 114 op/s 2024-02-16T19:30:12.714 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:12 smithi022 ceph-mon[29221]: pgmap v12: 65 pgs: 65 active+clean; 307 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 46 KiB/s rd, 1.9 MiB/s wr, 114 op/s 2024-02-16T19:30:14.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.407+0000 7f23945bb700 1 -- 172.21.15.22:0/2527356474 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f238c100240 msgr2=0x7f238c100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:14.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.407+0000 7f23945bb700 1 --2- 172.21.15.22:0/2527356474 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f238c100240 0x7f238c100650 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2388009910 tx=0x7f2388009c20 comp rx=0 tx=0).stop 2024-02-16T19:30:14.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.407+0000 7f23945bb700 1 -- 172.21.15.22:0/2527356474 shutdown_connections 2024-02-16T19:30:14.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.407+0000 7f23945bb700 1 --2- 172.21.15.22:0/2527356474 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f238c1014e0 0x7f238c101950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:14.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.407+0000 7f23945bb700 1 --2- 172.21.15.22:0/2527356474 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f238c100240 0x7f238c100650 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:14.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.407+0000 7f23945bb700 1 -- 172.21.15.22:0/2527356474 >> 172.21.15.22:0/2527356474 conn(0x7f238c0fb890 msgr2=0x7f238c0fdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:30:14.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.407+0000 7f23945bb700 1 -- 172.21.15.22:0/2527356474 shutdown_connections 2024-02-16T19:30:14.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.407+0000 7f23945bb700 1 -- 172.21.15.22:0/2527356474 wait complete. 2024-02-16T19:30:14.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.408+0000 7f23945bb700 1 Processor -- start 2024-02-16T19:30:14.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.408+0000 7f23945bb700 1 -- start start 2024-02-16T19:30:14.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.408+0000 7f23945bb700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f238c100240 0x7f238c197ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:30:14.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.408+0000 7f23945bb700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f238c1014e0 0x7f238c198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:30:14.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.408+0000 7f23945bb700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f238c198840 con 0x7f238c1014e0 2024-02-16T19:30:14.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.408+0000 7f23945bb700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f238c19d250 con 0x7f238c100240 2024-02-16T19:30:14.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.408+0000 7f2391b56700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f238c1014e0 0x7f238c198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:30:14.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.408+0000 7f2391b56700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f238c1014e0 0x7f238c198220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49078/0 (socket says 172.21.15.22:49078) 2024-02-16T19:30:14.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.408+0000 7f2391b56700 1 -- 172.21.15.22:0/3754448407 learned_addr learned my addr 172.21.15.22:0/3754448407 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:30:14.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.408+0000 7f2392357700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f238c100240 0x7f238c197ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:30:14.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.408+0000 7f2392357700 1 -- 172.21.15.22:0/3754448407 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f238c1014e0 msgr2=0x7f238c198220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:14.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.408+0000 7f2392357700 1 --2- 172.21.15.22:0/3754448407 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f238c1014e0 0x7f238c198220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:14.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.408+0000 7f2392357700 1 -- 172.21.15.22:0/3754448407 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23880095f0 con 0x7f238c100240 2024-02-16T19:30:14.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.408+0000 7f2391b56700 1 --2- 172.21.15.22:0/3754448407 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f238c1014e0 0x7f238c198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T19:30:14.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.409+0000 7f2392357700 1 --2- 172.21.15.22:0/3754448407 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f238c100240 0x7f238c197ce0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2388000c00 tx=0x7f238800f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:30:14.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.409+0000 7f23837fe700 1 -- 172.21.15.22:0/3754448407 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f238801d070 con 0x7f238c100240 2024-02-16T19:30:14.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.409+0000 7f23945bb700 1 -- 172.21.15.22:0/3754448407 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f238c19d3f0 con 0x7f238c100240 2024-02-16T19:30:14.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.409+0000 7f23837fe700 1 -- 172.21.15.22:0/3754448407 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2388005dc0 con 0x7f238c100240 2024-02-16T19:30:14.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.409+0000 7f23837fe700 1 -- 172.21.15.22:0/3754448407 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2388017400 con 0x7f238c100240 2024-02-16T19:30:14.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.409+0000 7f23945bb700 1 -- 172.21.15.22:0/3754448407 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f238c19d8b0 con 0x7f238c100240 2024-02-16T19:30:14.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.410+0000 7f23945bb700 1 -- 172.21.15.22:0/3754448407 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f238c10af00 con 0x7f238c100240 2024-02-16T19:30:14.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.411+0000 7f23837fe700 1 -- 172.21.15.22:0/3754448407 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 31) v1 ==== 91448+0+0 (secure 0 0 0) 0x7f2388003810 con 0x7f238c100240 2024-02-16T19:30:14.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.411+0000 7f23837fe700 1 --2- 172.21.15.22:0/3754448407 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f237806d660 0x7f237806fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:30:14.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.411+0000 7f23837fe700 1 -- 172.21.15.22:0/3754448407 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f238808e580 con 0x7f238c100240 2024-02-16T19:30:14.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.411+0000 7f2391b56700 1 --2- 172.21.15.22:0/3754448407 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f237806d660 0x7f237806fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:30:14.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.412+0000 7f2391b56700 1 --2- 172.21.15.22:0/3754448407 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f237806d660 0x7f237806fb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f237c011550 tx=0x7f237c009260 comp rx=0 tx=0).ready entity=mgr.24407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:30:14.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.416+0000 7f23837fe700 1 -- 172.21.15.22:0/3754448407 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f2388058040 con 0x7f238c100240 2024-02-16T19:30:14.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:14 smithi117 ceph-mon[35309]: pgmap v13: 65 pgs: 65 active+clean; 307 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 29 KiB/s rd, 1.4 MiB/s wr, 77 op/s 2024-02-16T19:30:14.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.691+0000 7f23945bb700 1 -- 172.21.15.22:0/3754448407 --> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f238c19dec0 con 0x7f237806d660 2024-02-16T19:30:14.692 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:14 smithi022 ceph-mon[29221]: pgmap v13: 65 pgs: 65 active+clean; 307 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 29 KiB/s rd, 1.4 MiB/s wr, 77 op/s 2024-02-16T19:30:14.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.693+0000 7f23837fe700 1 -- 172.21.15.22:0/3754448407 <== mgr.24407 v2:172.21.15.22:6800/3703806543 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7f238c19dec0 con 0x7f237806d660 2024-02-16T19:30:14.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.696+0000 7f23945bb700 1 -- 172.21.15.22:0/3754448407 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f237806d660 msgr2=0x7f237806fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:14.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.696+0000 7f23945bb700 1 --2- 172.21.15.22:0/3754448407 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f237806d660 0x7f237806fb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f237c011550 tx=0x7f237c009260 comp rx=0 tx=0).stop 2024-02-16T19:30:14.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.696+0000 7f23945bb700 1 -- 172.21.15.22:0/3754448407 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f238c100240 msgr2=0x7f238c197ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:14.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.696+0000 7f23945bb700 1 --2- 172.21.15.22:0/3754448407 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f238c100240 0x7f238c197ce0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2388000c00 tx=0x7f238800f690 comp rx=0 tx=0).stop 2024-02-16T19:30:14.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.696+0000 7f23945bb700 1 -- 172.21.15.22:0/3754448407 shutdown_connections 2024-02-16T19:30:14.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.696+0000 7f23945bb700 1 --2- 172.21.15.22:0/3754448407 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f237806d660 0x7f237806fb10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:14.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.696+0000 7f23945bb700 1 --2- 172.21.15.22:0/3754448407 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f238c100240 0x7f238c197ce0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:14.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.696+0000 7f23945bb700 1 --2- 172.21.15.22:0/3754448407 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f238c1014e0 0x7f238c198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:14.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.696+0000 7f23945bb700 1 -- 172.21.15.22:0/3754448407 >> 172.21.15.22:0/3754448407 conn(0x7f238c0fb890 msgr2=0x7f238c104710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:30:14.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.696+0000 7f23945bb700 1 -- 172.21.15.22:0/3754448407 shutdown_connections 2024-02-16T19:30:14.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.696+0000 7f23945bb700 1 -- 172.21.15.22:0/3754448407 wait complete. 2024-02-16T19:30:14.708 INFO:teuthology.orchestra.run.smithi022.stdout:true 2024-02-16T19:30:14.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.822+0000 7f583157b700 1 -- 172.21.15.22:0/2272014077 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f582c0ff150 msgr2=0x7f582c0ff560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:14.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.822+0000 7f583157b700 1 --2- 172.21.15.22:0/2272014077 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f582c0ff150 0x7f582c0ff560 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f581c009870 tx=0x7f581c009b80 comp rx=0 tx=0).stop 2024-02-16T19:30:14.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.822+0000 7f583157b700 1 -- 172.21.15.22:0/2272014077 shutdown_connections 2024-02-16T19:30:14.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.822+0000 7f583157b700 1 --2- 172.21.15.22:0/2272014077 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f582c100350 0x7f582c1007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:14.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.822+0000 7f583157b700 1 --2- 172.21.15.22:0/2272014077 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f582c0ff150 0x7f582c0ff560 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:14.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.822+0000 7f583157b700 1 -- 172.21.15.22:0/2272014077 >> 172.21.15.22:0/2272014077 conn(0x7f582c0fa700 msgr2=0x7f582c0fcb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:30:14.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.823+0000 7f583157b700 1 -- 172.21.15.22:0/2272014077 shutdown_connections 2024-02-16T19:30:14.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.823+0000 7f583157b700 1 -- 172.21.15.22:0/2272014077 wait complete. 2024-02-16T19:30:14.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.823+0000 7f583157b700 1 Processor -- start 2024-02-16T19:30:14.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.823+0000 7f583157b700 1 -- start start 2024-02-16T19:30:14.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.824+0000 7f583157b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f582c0ff150 0x7f582c1030e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:30:14.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.824+0000 7f583157b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f582c100350 0x7f582c101730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:30:14.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.824+0000 7f583157b700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f582c101c70 con 0x7f582c0ff150 2024-02-16T19:30:14.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.824+0000 7f583157b700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f582c101db0 con 0x7f582c100350 2024-02-16T19:30:14.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.824+0000 7f582affd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f582c0ff150 0x7f582c1030e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:30:14.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.824+0000 7f582affd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f582c0ff150 0x7f582c1030e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49108/0 (socket says 172.21.15.22:49108) 2024-02-16T19:30:14.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.824+0000 7f582affd700 1 -- 172.21.15.22:0/2372457306 learned_addr learned my addr 172.21.15.22:0/2372457306 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:30:14.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.824+0000 7f582a7fc700 1 --2- 172.21.15.22:0/2372457306 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f582c100350 0x7f582c101730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:30:14.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.824+0000 7f582affd700 1 -- 172.21.15.22:0/2372457306 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f582c100350 msgr2=0x7f582c101730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:14.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.824+0000 7f582affd700 1 --2- 172.21.15.22:0/2372457306 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f582c100350 0x7f582c101730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:14.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.824+0000 7f582affd700 1 -- 172.21.15.22:0/2372457306 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58200095f0 con 0x7f582c0ff150 2024-02-16T19:30:14.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.824+0000 7f582affd700 1 --2- 172.21.15.22:0/2372457306 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f582c0ff150 0x7f582c1030e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f581c009230 tx=0x7f581c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:30:14.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.825+0000 7f5813fff700 1 -- 172.21.15.22:0/2372457306 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f581c01c070 con 0x7f582c0ff150 2024-02-16T19:30:14.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.825+0000 7f5813fff700 1 -- 172.21.15.22:0/2372457306 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f581c00fd60 con 0x7f582c0ff150 2024-02-16T19:30:14.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.825+0000 7f5813fff700 1 -- 172.21.15.22:0/2372457306 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f581c022ea0 con 0x7f582c0ff150 2024-02-16T19:30:14.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.825+0000 7f583157b700 1 -- 172.21.15.22:0/2372457306 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f581c009520 con 0x7f582c0ff150 2024-02-16T19:30:14.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.825+0000 7f583157b700 1 -- 172.21.15.22:0/2372457306 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f582c102340 con 0x7f582c0ff150 2024-02-16T19:30:14.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.826+0000 7f583157b700 1 -- 172.21.15.22:0/2372457306 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f582c04f2a0 con 0x7f582c0ff150 2024-02-16T19:30:14.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.826+0000 7f5813fff700 1 -- 172.21.15.22:0/2372457306 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 31) v1 ==== 91448+0+0 (secure 0 0 0) 0x7f581c020070 con 0x7f582c0ff150 2024-02-16T19:30:14.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.827+0000 7f5813fff700 1 --2- 172.21.15.22:0/2372457306 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f581406d660 0x7f581406fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:30:14.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.827+0000 7f5813fff700 1 -- 172.21.15.22:0/2372457306 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f581c08e2f0 con 0x7f582c0ff150 2024-02-16T19:30:14.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.827+0000 7f582a7fc700 1 --2- 172.21.15.22:0/2372457306 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f581406d660 0x7f581406fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:30:14.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.828+0000 7f582a7fc700 1 --2- 172.21.15.22:0/2372457306 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f581406d660 0x7f581406fb10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5820005fd0 tx=0x7f5820009310 comp rx=0 tx=0).ready entity=mgr.24407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:30:14.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:14.833+0000 7f5813fff700 1 -- 172.21.15.22:0/2372457306 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f581c05b990 con 0x7f582c0ff150 2024-02-16T19:30:15.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.108+0000 7f583157b700 1 -- 172.21.15.22:0/2372457306 --> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f582c0694e0 con 0x7f581406d660 2024-02-16T19:30:15.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.110+0000 7f5813fff700 1 -- 172.21.15.22:0/2372457306 <== mgr.24407 v2:172.21.15.22:6800/3703806543 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7f582c0694e0 con 0x7f581406d660 2024-02-16T19:30:15.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.112+0000 7f583157b700 1 -- 172.21.15.22:0/2372457306 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f581406d660 msgr2=0x7f581406fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:15.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.112+0000 7f583157b700 1 --2- 172.21.15.22:0/2372457306 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f581406d660 0x7f581406fb10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5820005fd0 tx=0x7f5820009310 comp rx=0 tx=0).stop 2024-02-16T19:30:15.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.112+0000 7f583157b700 1 -- 172.21.15.22:0/2372457306 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f582c0ff150 msgr2=0x7f582c1030e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:15.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.112+0000 7f583157b700 1 --2- 172.21.15.22:0/2372457306 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f582c0ff150 0x7f582c1030e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f581c009230 tx=0x7f581c00f690 comp rx=0 tx=0).stop 2024-02-16T19:30:15.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.112+0000 7f583157b700 1 -- 172.21.15.22:0/2372457306 shutdown_connections 2024-02-16T19:30:15.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.112+0000 7f583157b700 1 --2- 172.21.15.22:0/2372457306 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f581406d660 0x7f581406fb10 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:15.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.112+0000 7f583157b700 1 --2- 172.21.15.22:0/2372457306 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f582c0ff150 0x7f582c1030e0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:15.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.112+0000 7f583157b700 1 --2- 172.21.15.22:0/2372457306 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f582c100350 0x7f582c101730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:15.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.112+0000 7f583157b700 1 -- 172.21.15.22:0/2372457306 >> 172.21.15.22:0/2372457306 conn(0x7f582c0fa700 msgr2=0x7f582c068dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:30:15.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.112+0000 7f583157b700 1 -- 172.21.15.22:0/2372457306 shutdown_connections 2024-02-16T19:30:15.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.112+0000 7f583157b700 1 -- 172.21.15.22:0/2372457306 wait complete. 2024-02-16T19:30:15.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.245+0000 7f2d20e7e700 1 -- 172.21.15.22:0/3464287950 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d1c0ffb60 msgr2=0x7f2d1c0fffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:15.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.245+0000 7f2d20e7e700 1 --2- 172.21.15.22:0/3464287950 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d1c0ffb60 0x7f2d1c0fffd0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c009910 tx=0x7f2d0c009c20 comp rx=0 tx=0).stop 2024-02-16T19:30:15.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.247+0000 7f2d20e7e700 1 -- 172.21.15.22:0/3464287950 shutdown_connections 2024-02-16T19:30:15.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.247+0000 7f2d20e7e700 1 --2- 172.21.15.22:0/3464287950 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d1c0ffb60 0x7f2d1c0fffd0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:15.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.247+0000 7f2d20e7e700 1 --2- 172.21.15.22:0/3464287950 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d1c0fe8c0 0x7f2d1c0fecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:15.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.247+0000 7f2d20e7e700 1 -- 172.21.15.22:0/3464287950 >> 172.21.15.22:0/3464287950 conn(0x7f2d1c0f9f10 msgr2=0x7f2d1c0fc340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:30:15.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.247+0000 7f2d20e7e700 1 -- 172.21.15.22:0/3464287950 shutdown_connections 2024-02-16T19:30:15.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.247+0000 7f2d20e7e700 1 -- 172.21.15.22:0/3464287950 wait complete. 2024-02-16T19:30:15.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.248+0000 7f2d20e7e700 1 Processor -- start 2024-02-16T19:30:15.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.248+0000 7f2d20e7e700 1 -- start start 2024-02-16T19:30:15.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.249+0000 7f2d20e7e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d1c0fe8c0 0x7f2d1c071dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:30:15.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.249+0000 7f2d20e7e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d1c0ffb60 0x7f2d1c072300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:30:15.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.249+0000 7f2d20e7e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d1c072920 con 0x7f2d1c0fe8c0 2024-02-16T19:30:15.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.249+0000 7f2d20e7e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d1c072a60 con 0x7f2d1c0ffb60 2024-02-16T19:30:15.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.249+0000 7f2d19d9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d1c0ffb60 0x7f2d1c072300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:30:15.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.249+0000 7f2d1a59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d1c0fe8c0 0x7f2d1c071dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:30:15.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.249+0000 7f2d1a59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d1c0fe8c0 0x7f2d1c071dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49128/0 (socket says 172.21.15.22:49128) 2024-02-16T19:30:15.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.249+0000 7f2d1a59c700 1 -- 172.21.15.22:0/2892411161 learned_addr learned my addr 172.21.15.22:0/2892411161 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:30:15.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.249+0000 7f2d1a59c700 1 -- 172.21.15.22:0/2892411161 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d1c0ffb60 msgr2=0x7f2d1c072300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:15.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.249+0000 7f2d1a59c700 1 --2- 172.21.15.22:0/2892411161 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d1c0ffb60 0x7f2d1c072300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:15.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.249+0000 7f2d1a59c700 1 -- 172.21.15.22:0/2892411161 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d0c0095f0 con 0x7f2d1c0fe8c0 2024-02-16T19:30:15.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.250+0000 7f2d1a59c700 1 --2- 172.21.15.22:0/2892411161 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d1c0fe8c0 0x7f2d1c071dc0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f2d1400b840 tx=0x7f2d1400bb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:30:15.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.250+0000 7f2d0b7fe700 1 -- 172.21.15.22:0/2892411161 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d14010830 con 0x7f2d1c0fe8c0 2024-02-16T19:30:15.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.250+0000 7f2d0b7fe700 1 -- 172.21.15.22:0/2892411161 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2d14010e40 con 0x7f2d1c0fe8c0 2024-02-16T19:30:15.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.250+0000 7f2d20e7e700 1 -- 172.21.15.22:0/2892411161 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d1c1a36c0 con 0x7f2d1c0fe8c0 2024-02-16T19:30:15.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.250+0000 7f2d20e7e700 1 -- 172.21.15.22:0/2892411161 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d1c106590 con 0x7f2d1c0fe8c0 2024-02-16T19:30:15.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.250+0000 7f2d0b7fe700 1 -- 172.21.15.22:0/2892411161 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d14019620 con 0x7f2d1c0fe8c0 2024-02-16T19:30:15.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.252+0000 7f2d20e7e700 1 -- 172.21.15.22:0/2892411161 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cfc005320 con 0x7f2d1c0fe8c0 2024-02-16T19:30:15.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.252+0000 7f2d0b7fe700 1 -- 172.21.15.22:0/2892411161 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 31) v1 ==== 91448+0+0 (secure 0 0 0) 0x7f2d14019780 con 0x7f2d1c0fe8c0 2024-02-16T19:30:15.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.253+0000 7f2d0b7fe700 1 --2- 172.21.15.22:0/2892411161 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f2d0406d6a0 0x7f2d0406fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:30:15.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.253+0000 7f2d0b7fe700 1 -- 172.21.15.22:0/2892411161 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f2d1408d4a0 con 0x7f2d1c0fe8c0 2024-02-16T19:30:15.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.253+0000 7f2d19d9b700 1 --2- 172.21.15.22:0/2892411161 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f2d0406d6a0 0x7f2d0406fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:30:15.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.254+0000 7f2d19d9b700 1 --2- 172.21.15.22:0/2892411161 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f2d0406d6a0 0x7f2d0406fb50 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c009fd0 tx=0x7f2d0c00b540 comp rx=0 tx=0).ready entity=mgr.24407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:30:15.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.262+0000 7f2d0b7fe700 1 -- 172.21.15.22:0/2892411161 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f2d1405a9f0 con 0x7f2d1c0fe8c0 2024-02-16T19:30:15.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.638+0000 7f2d20e7e700 1 -- 172.21.15.22:0/2892411161 --> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f2cfc000bf0 con 0x7f2d0406d6a0 2024-02-16T19:30:15.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.650+0000 7f2d0b7fe700 1 -- 172.21.15.22:0/2892411161 <== mgr.24407 v2:172.21.15.22:6800/3703806543 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f2cfc000bf0 con 0x7f2d0406d6a0 2024-02-16T19:30:15.652 INFO:teuthology.orchestra.run.smithi022.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T19:30:15.652 INFO:teuthology.orchestra.run.smithi022.stdout:alertmanager.smithi022 smithi022 *:9093,9094 running (7m) 27s ago 9m 26.6M - 0.25.0 c8568f914cd2 4dac585d8372 2024-02-16T19:30:15.652 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-exporter.smithi022 smithi022 running (9m) 27s ago 9m 7688k - 18.2.0 dc2bc1663786 3f8bef4349ba 2024-02-16T19:30:15.652 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-exporter.smithi117 smithi117 running (8m) 18s ago 8m 7415k - 18.2.0 dc2bc1663786 59b65a7e309f 2024-02-16T19:30:15.652 INFO:teuthology.orchestra.run.smithi022.stdout:crash.smithi022 smithi022 running (9m) 27s ago 9m 6669k - 18.2.0 dc2bc1663786 efef476aa486 2024-02-16T19:30:15.652 INFO:teuthology.orchestra.run.smithi022.stdout:crash.smithi117 smithi117 running (8m) 18s ago 8m 6660k - 18.2.0 dc2bc1663786 5466453eb149 2024-02-16T19:30:15.652 INFO:teuthology.orchestra.run.smithi022.stdout:grafana.smithi022 smithi022 *:3000 running (7m) 27s ago 9m 85.0M - 9.4.7 954c08fa6188 a48872ecbbf2 2024-02-16T19:30:15.652 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi022.btppwm smithi022 running (3m) 27s ago 3m 257M - 18.2.0 dc2bc1663786 22971fb3a907 2024-02-16T19:30:15.652 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi022.ocukvs smithi022 running (3m) 27s ago 3m 13.5M - 18.2.0 dc2bc1663786 81ee7bd5fd39 2024-02-16T19:30:15.652 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi117.ibmppn smithi117 running (3m) 18s ago 3m 15.4M - 18.2.0 dc2bc1663786 8c606cbeeb86 2024-02-16T19:30:15.653 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi117.orzors smithi117 running (3m) 18s ago 3m 14.5M - 18.2.0 dc2bc1663786 7d2bc4df3feb 2024-02-16T19:30:15.653 INFO:teuthology.orchestra.run.smithi022.stdout:mgr.smithi022.gmdcch smithi022 *:8443,9283,8765 running (34s) 27s ago 10m 63.5M - 19.0.0-1450-gf78a58c0 f081f847070a 7cb74cc9deea 2024-02-16T19:30:15.653 INFO:teuthology.orchestra.run.smithi022.stdout:mgr.smithi117.cohbtv smithi117 *:8443,9283,8765 running (99s) 18s ago 8m 76.3M - 19.0.0-1450-gf78a58c0 f081f847070a 688882bc227b 2024-02-16T19:30:15.653 INFO:teuthology.orchestra.run.smithi022.stdout:mon.smithi022 smithi022 running (10m) 27s ago 11m 50.3M 2048M 18.2.0 dc2bc1663786 b4ffc3032e12 2024-02-16T19:30:15.653 INFO:teuthology.orchestra.run.smithi022.stdout:mon.smithi117 smithi117 running (8m) 18s ago 8m 41.7M 2048M 18.2.0 dc2bc1663786 aff3e21a73a0 2024-02-16T19:30:15.653 INFO:teuthology.orchestra.run.smithi022.stdout:node-exporter.smithi022 smithi022 *:9100 running (9m) 27s ago 9m 15.6M - 1.5.0 0da6a335fe13 d727f51df007 2024-02-16T19:30:15.653 INFO:teuthology.orchestra.run.smithi022.stdout:node-exporter.smithi117 smithi117 *:9100 running (8m) 18s ago 8m 16.6M - 1.5.0 0da6a335fe13 fc8b2df5a29d 2024-02-16T19:30:15.653 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0 smithi022 running (7m) 27s ago 7m 191M 1217M 18.2.0 dc2bc1663786 0027f86f16ce 2024-02-16T19:30:15.653 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1 smithi022 running (7m) 27s ago 7m 196M 1217M 18.2.0 dc2bc1663786 87300382b8a7 2024-02-16T19:30:15.653 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2 smithi022 running (6m) 27s ago 6m 163M 1217M 18.2.0 dc2bc1663786 be8a631b4f93 2024-02-16T19:30:15.653 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3 smithi117 running (6m) 18s ago 6m 204M 2240M 18.2.0 dc2bc1663786 9c0ce9a981cf 2024-02-16T19:30:15.653 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4 smithi117 running (5m) 18s ago 5m 178M 2240M 18.2.0 dc2bc1663786 1779c3496f4d 2024-02-16T19:30:15.653 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5 smithi117 running (5m) 18s ago 5m 162M 2240M 18.2.0 dc2bc1663786 fefaabca0638 2024-02-16T19:30:15.653 INFO:teuthology.orchestra.run.smithi022.stdout:prometheus.smithi022 smithi022 *:9095 running (40s) 27s ago 8m 40.4M - 2.43.0 a07b618ecd1d 8885719da229 2024-02-16T19:30:15.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.654+0000 7f2d20e7e700 1 -- 172.21.15.22:0/2892411161 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f2d0406d6a0 msgr2=0x7f2d0406fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:15.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.654+0000 7f2d20e7e700 1 --2- 172.21.15.22:0/2892411161 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f2d0406d6a0 0x7f2d0406fb50 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c009fd0 tx=0x7f2d0c00b540 comp rx=0 tx=0).stop 2024-02-16T19:30:15.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.654+0000 7f2d20e7e700 1 -- 172.21.15.22:0/2892411161 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d1c0fe8c0 msgr2=0x7f2d1c071dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:15.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.654+0000 7f2d20e7e700 1 --2- 172.21.15.22:0/2892411161 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d1c0fe8c0 0x7f2d1c071dc0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f2d1400b840 tx=0x7f2d1400bb50 comp rx=0 tx=0).stop 2024-02-16T19:30:15.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.654+0000 7f2d20e7e700 1 -- 172.21.15.22:0/2892411161 shutdown_connections 2024-02-16T19:30:15.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.654+0000 7f2d20e7e700 1 --2- 172.21.15.22:0/2892411161 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f2d0406d6a0 0x7f2d0406fb50 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:15.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.654+0000 7f2d20e7e700 1 --2- 172.21.15.22:0/2892411161 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d1c0fe8c0 0x7f2d1c071dc0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:15.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.654+0000 7f2d20e7e700 1 --2- 172.21.15.22:0/2892411161 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d1c0ffb60 0x7f2d1c072300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:15.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.654+0000 7f2d20e7e700 1 -- 172.21.15.22:0/2892411161 >> 172.21.15.22:0/2892411161 conn(0x7f2d1c0f9f10 msgr2=0x7f2d1c102d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:30:15.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.654+0000 7f2d20e7e700 1 -- 172.21.15.22:0/2892411161 shutdown_connections 2024-02-16T19:30:15.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.654+0000 7f2d20e7e700 1 -- 172.21.15.22:0/2892411161 wait complete. 2024-02-16T19:30:15.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.788+0000 7f3a8307e700 1 -- 172.21.15.22:0/3337438505 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a7c105710 msgr2=0x7f3a7c107af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:15.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.788+0000 7f3a8307e700 1 --2- 172.21.15.22:0/3337438505 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a7c105710 0x7f3a7c107af0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f3a68009910 tx=0x7f3a68009c20 comp rx=0 tx=0).stop 2024-02-16T19:30:15.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.788+0000 7f3a8307e700 1 -- 172.21.15.22:0/3337438505 shutdown_connections 2024-02-16T19:30:15.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.788+0000 7f3a8307e700 1 --2- 172.21.15.22:0/3337438505 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a7c105710 0x7f3a7c107af0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:15.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.788+0000 7f3a8307e700 1 --2- 172.21.15.22:0/3337438505 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a7c069270 0x7f3a7c1051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:15.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.788+0000 7f3a8307e700 1 -- 172.21.15.22:0/3337438505 >> 172.21.15.22:0/3337438505 conn(0x7f3a7c0fa780 msgr2=0x7f3a7c0fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:30:15.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.788+0000 7f3a8307e700 1 -- 172.21.15.22:0/3337438505 shutdown_connections 2024-02-16T19:30:15.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.789+0000 7f3a8307e700 1 -- 172.21.15.22:0/3337438505 wait complete. 2024-02-16T19:30:15.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.789+0000 7f3a8307e700 1 Processor -- start 2024-02-16T19:30:15.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.789+0000 7f3a8307e700 1 -- start start 2024-02-16T19:30:15.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.789+0000 7f3a8307e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a7c105710 0x7f3a7c197cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:30:15.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.789+0000 7f3a8307e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a7c198230 0x7f3a7c19d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:30:15.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.789+0000 7f3a8307e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7c1986a0 con 0x7f3a7c198230 2024-02-16T19:30:15.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.789+0000 7f3a8307e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7c198810 con 0x7f3a7c105710 2024-02-16T19:30:15.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.790+0000 7f3a7bfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a7c198230 0x7f3a7c19d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:30:15.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.790+0000 7f3a7bfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a7c198230 0x7f3a7c19d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49144/0 (socket says 172.21.15.22:49144) 2024-02-16T19:30:15.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.790+0000 7f3a7bfff700 1 -- 172.21.15.22:0/3408910739 learned_addr learned my addr 172.21.15.22:0/3408910739 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:30:15.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.790+0000 7f3a80e1a700 1 --2- 172.21.15.22:0/3408910739 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a7c105710 0x7f3a7c197cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:30:15.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.790+0000 7f3a7bfff700 1 -- 172.21.15.22:0/3408910739 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a7c105710 msgr2=0x7f3a7c197cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:15.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.790+0000 7f3a7bfff700 1 --2- 172.21.15.22:0/3408910739 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a7c105710 0x7f3a7c197cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:15.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.790+0000 7f3a7bfff700 1 -- 172.21.15.22:0/3408910739 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a680095f0 con 0x7f3a7c198230 2024-02-16T19:30:15.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.790+0000 7f3a7bfff700 1 --2- 172.21.15.22:0/3408910739 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a7c198230 0x7f3a7c19d260 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f3a680092e0 tx=0x7f3a6800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:30:15.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.790+0000 7f3a79ffb700 1 -- 172.21.15.22:0/3408910739 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a6801d070 con 0x7f3a7c198230 2024-02-16T19:30:15.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.790+0000 7f3a8307e700 1 -- 172.21.15.22:0/3408910739 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a7c19d7a0 con 0x7f3a7c198230 2024-02-16T19:30:15.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.790+0000 7f3a79ffb700 1 -- 172.21.15.22:0/3408910739 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3a68005dc0 con 0x7f3a7c198230 2024-02-16T19:30:15.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.790+0000 7f3a79ffb700 1 -- 172.21.15.22:0/3408910739 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a68017630 con 0x7f3a7c198230 2024-02-16T19:30:15.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.791+0000 7f3a8307e700 1 -- 172.21.15.22:0/3408910739 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a7c19dc60 con 0x7f3a7c198230 2024-02-16T19:30:15.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.791+0000 7f3a8307e700 1 -- 172.21.15.22:0/3408910739 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a7c191df0 con 0x7f3a7c198230 2024-02-16T19:30:15.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.792+0000 7f3a79ffb700 1 -- 172.21.15.22:0/3408910739 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 31) v1 ==== 91448+0+0 (secure 0 0 0) 0x7f3a6800faf0 con 0x7f3a7c198230 2024-02-16T19:30:15.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.792+0000 7f3a79ffb700 1 --2- 172.21.15.22:0/3408910739 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f3a6c06d650 0x7f3a6c06fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:30:15.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.793+0000 7f3a79ffb700 1 -- 172.21.15.22:0/3408910739 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f3a6808e520 con 0x7f3a7c198230 2024-02-16T19:30:15.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.793+0000 7f3a80e1a700 1 --2- 172.21.15.22:0/3408910739 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f3a6c06d650 0x7f3a6c06fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:30:15.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.793+0000 7f3a80e1a700 1 --2- 172.21.15.22:0/3408910739 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f3a6c06d650 0x7f3a6c06fb00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c1991d0 tx=0x7f3a70009260 comp rx=0 tx=0).ready entity=mgr.24407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:30:15.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:15.798+0000 7f3a79ffb700 1 -- 172.21.15.22:0/3408910739 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f3a68058060 con 0x7f3a7c198230 2024-02-16T19:30:16.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:16.190+0000 7f3a8307e700 1 -- 172.21.15.22:0/3408910739 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f3a7c069270 con 0x7f3a7c198230 2024-02-16T19:30:16.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:16.190+0000 7f3a79ffb700 1 -- 172.21.15.22:0/3408910739 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7f3a6805b720 con 0x7f3a7c198230 2024-02-16T19:30:16.192 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-02-16T19:30:16.192 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": { 2024-02-16T19:30:16.192 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-02-16T19:30:16.192 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:30:16.192 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": { 2024-02-16T19:30:16.192 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T19:30:16.192 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:30:16.192 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": { 2024-02-16T19:30:16.192 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-02-16T19:30:16.193 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:30:16.193 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": { 2024-02-16T19:30:16.193 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-02-16T19:30:16.193 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:30:16.193 INFO:teuthology.orchestra.run.smithi022.stdout: "overall": { 2024-02-16T19:30:16.193 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 12, 2024-02-16T19:30:16.193 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T19:30:16.193 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-02-16T19:30:16.193 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-02-16T19:30:16.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:16.193+0000 7f3a8307e700 1 -- 172.21.15.22:0/3408910739 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f3a6c06d650 msgr2=0x7f3a6c06fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:16.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:16.193+0000 7f3a8307e700 1 --2- 172.21.15.22:0/3408910739 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f3a6c06d650 0x7f3a6c06fb00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c1991d0 tx=0x7f3a70009260 comp rx=0 tx=0).stop 2024-02-16T19:30:16.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:16.193+0000 7f3a8307e700 1 -- 172.21.15.22:0/3408910739 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a7c198230 msgr2=0x7f3a7c19d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:16.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:16.193+0000 7f3a8307e700 1 --2- 172.21.15.22:0/3408910739 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a7c198230 0x7f3a7c19d260 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f3a680092e0 tx=0x7f3a6800f690 comp rx=0 tx=0).stop 2024-02-16T19:30:16.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:16.193+0000 7f3a8307e700 1 -- 172.21.15.22:0/3408910739 shutdown_connections 2024-02-16T19:30:16.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:16.193+0000 7f3a8307e700 1 --2- 172.21.15.22:0/3408910739 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f3a6c06d650 0x7f3a6c06fb00 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:16.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:16.193+0000 7f3a8307e700 1 --2- 172.21.15.22:0/3408910739 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a7c105710 0x7f3a7c197cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:16.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:16.193+0000 7f3a8307e700 1 --2- 172.21.15.22:0/3408910739 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a7c198230 0x7f3a7c19d260 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:16.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:16.193+0000 7f3a8307e700 1 -- 172.21.15.22:0/3408910739 >> 172.21.15.22:0/3408910739 conn(0x7f3a7c0fa780 msgr2=0x7f3a7c0fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:30:16.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:16.193+0000 7f3a8307e700 1 -- 172.21.15.22:0/3408910739 shutdown_connections 2024-02-16T19:30:16.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:16.194+0000 7f3a8307e700 1 -- 172.21.15.22:0/3408910739 wait complete. 2024-02-16T19:30:16.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:16.362+0000 7ff18e3e6700 1 -- 172.21.15.22:0/2860529166 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1880723e0 msgr2=0x7ff18810be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:16.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:16.363+0000 7ff18e3e6700 1 --2- 172.21.15.22:0/2860529166 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1880723e0 0x7ff18810be10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff180009f90 tx=0x7ff18000b330 comp rx=0 tx=0).stop 2024-02-16T19:30:16.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:16.363+0000 7ff18e3e6700 1 -- 172.21.15.22:0/2860529166 shutdown_connections 2024-02-16T19:30:16.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:16.363+0000 7ff18e3e6700 1 --2- 172.21.15.22:0/2860529166 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1880723e0 0x7ff18810be10 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:16.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:16.363+0000 7ff18e3e6700 1 --2- 172.21.15.22:0/2860529166 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff188071a00 0x7ff188071e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:16.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:16.363+0000 7ff18e3e6700 1 -- 172.21.15.22:0/2860529166 >> 172.21.15.22:0/2860529166 conn(0x7ff18806d110 msgr2=0x7ff18806f560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:30:16.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:16.364+0000 7ff18e3e6700 1 -- 172.21.15.22:0/2860529166 shutdown_connections 2024-02-16T19:30:16.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:16.364+0000 7ff18e3e6700 1 -- 172.21.15.22:0/2860529166 wait complete. 2024-02-16T19:30:16.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:16.365+0000 7ff18e3e6700 1 Processor -- start 2024-02-16T19:30:16.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:16.365+0000 7ff18e3e6700 1 -- start start 2024-02-16T19:30:16.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:16.366+0000 7ff18e3e6700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff188071a00 0x7ff1881af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:30:16.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:16.366+0000 7ff18e3e6700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1880723e0 0x7ff1881b1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:30:16.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:16.366+0000 7ff18e3e6700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1881b2450 con 0x7ff188071a00 2024-02-16T19:30:16.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:16.366+0000 7ff187fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff188071a00 0x7ff1881af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:30:16.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:16.366+0000 7ff187fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff188071a00 0x7ff1881af9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49158/0 (socket says 172.21.15.22:49158) 2024-02-16T19:30:16.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:16.366+0000 7ff187fff700 1 -- 172.21.15.22:0/3891371678 learned_addr learned my addr 172.21.15.22:0/3891371678 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:30:16.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:16.366+0000 7ff18e3e6700 1 -- 172.21.15.22:0/3891371678 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1881b2590 con 0x7ff1880723e0 2024-02-16T19:30:16.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:16.366+0000 7ff1877fe700 1 --2- 172.21.15.22:0/3891371678 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1880723e0 0x7ff1881b1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:30:16.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:16.366+0000 7ff187fff700 1 -- 172.21.15.22:0/3891371678 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1880723e0 msgr2=0x7ff1881b1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:16.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:16.366+0000 7ff187fff700 1 --2- 172.21.15.22:0/3891371678 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1880723e0 0x7ff1881b1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:16.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:16.366+0000 7ff187fff700 1 -- 172.21.15.22:0/3891371678 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff180009c40 con 0x7ff188071a00 2024-02-16T19:30:16.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:16.367+0000 7ff187fff700 1 --2- 172.21.15.22:0/3891371678 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff188071a00 0x7ff1881af9c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff17800e940 tx=0x7ff17800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:30:16.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:16.367+0000 7ff1857fa700 1 -- 172.21.15.22:0/3891371678 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff178004a00 con 0x7ff188071a00 2024-02-16T19:30:16.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:16.367+0000 7ff1857fa700 1 -- 172.21.15.22:0/3891371678 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff17800cc70 con 0x7ff188071a00 2024-02-16T19:30:16.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:16.367+0000 7ff1857fa700 1 -- 172.21.15.22:0/3891371678 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff1780055b0 con 0x7ff188071a00 2024-02-16T19:30:16.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:16.367+0000 7ff18e3e6700 1 -- 172.21.15.22:0/3891371678 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1881b27f0 con 0x7ff188071a00 2024-02-16T19:30:16.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:16.367+0000 7ff18e3e6700 1 -- 172.21.15.22:0/3891371678 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1881b2c60 con 0x7ff188071a00 2024-02-16T19:30:16.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:16.369+0000 7ff18e3e6700 1 -- 172.21.15.22:0/3891371678 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff188111540 con 0x7ff188071a00 2024-02-16T19:30:16.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:16.370+0000 7ff1857fa700 1 -- 172.21.15.22:0/3891371678 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 31) v1 ==== 91448+0+0 (secure 0 0 0) 0x7ff17800c7c0 con 0x7ff188071a00 2024-02-16T19:30:16.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:16.370+0000 7ff1857fa700 1 --2- 172.21.15.22:0/3891371678 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7ff17006d6e0 0x7ff17006fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:30:16.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:16.370+0000 7ff1857fa700 1 -- 172.21.15.22:0/3891371678 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7ff178014070 con 0x7ff188071a00 2024-02-16T19:30:16.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:16.372+0000 7ff1877fe700 1 --2- 172.21.15.22:0/3891371678 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7ff17006d6e0 0x7ff17006fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:30:16.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:16.373+0000 7ff1877fe700 1 --2- 172.21.15.22:0/3891371678 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7ff17006d6e0 0x7ff17006fb90 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff1800060b0 tx=0x7ff180006020 comp rx=0 tx=0).ready entity=mgr.24407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:30:16.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:16.379+0000 7ff1857fa700 1 -- 172.21.15.22:0/3891371678 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7ff17805aa50 con 0x7ff188071a00 2024-02-16T19:30:16.464 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:16 smithi022 ceph-mon[29221]: from='client.24455 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:30:16.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:16 smithi022 ceph-mon[29221]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:30:16.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:16 smithi022 ceph-mon[29221]: pgmap v14: 65 pgs: 65 active+clean; 312 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 43 KiB/s rd, 2.1 MiB/s wr, 112 op/s 2024-02-16T19:30:16.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:16 smithi022 ceph-mon[29221]: from='client.? 172.21.15.22:0/3408910739' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:30:16.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:16 smithi117 ceph-mon[35309]: from='client.24455 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:30:16.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:16 smithi117 ceph-mon[35309]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:30:16.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:16 smithi117 ceph-mon[35309]: pgmap v14: 65 pgs: 65 active+clean; 312 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 43 KiB/s rd, 2.1 MiB/s wr, 112 op/s 2024-02-16T19:30:16.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:16 smithi117 ceph-mon[35309]: from='client.? 172.21.15.22:0/3408910739' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:30:16.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:16.718+0000 7ff18e3e6700 1 -- 172.21.15.22:0/3891371678 --> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff188061190 con 0x7ff17006d6e0 2024-02-16T19:30:16.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:16.719+0000 7ff1857fa700 1 -- 172.21.15.22:0/3891371678 <== mgr.24407 v2:172.21.15.22:6800/3703806543 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7ff188061190 con 0x7ff17006d6e0 2024-02-16T19:30:16.721 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-02-16T19:30:16.721 INFO:teuthology.orchestra.run.smithi022.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T19:30:16.721 INFO:teuthology.orchestra.run.smithi022.stdout: "in_progress": true, 2024-02-16T19:30:16.721 INFO:teuthology.orchestra.run.smithi022.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-02-16T19:30:16.721 INFO:teuthology.orchestra.run.smithi022.stdout: "services_complete": [ 2024-02-16T19:30:16.721 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr" 2024-02-16T19:30:16.721 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-02-16T19:30:16.721 INFO:teuthology.orchestra.run.smithi022.stdout: "progress": "2/2 daemons upgraded", 2024-02-16T19:30:16.721 INFO:teuthology.orchestra.run.smithi022.stdout: "message": "", 2024-02-16T19:30:16.721 INFO:teuthology.orchestra.run.smithi022.stdout: "is_paused": false 2024-02-16T19:30:16.721 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-02-16T19:30:16.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:16.723+0000 7ff18e3e6700 1 -- 172.21.15.22:0/3891371678 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7ff17006d6e0 msgr2=0x7ff17006fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:16.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:16.723+0000 7ff18e3e6700 1 --2- 172.21.15.22:0/3891371678 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7ff17006d6e0 0x7ff17006fb90 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff1800060b0 tx=0x7ff180006020 comp rx=0 tx=0).stop 2024-02-16T19:30:16.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:16.723+0000 7ff18e3e6700 1 -- 172.21.15.22:0/3891371678 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff188071a00 msgr2=0x7ff1881af9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:16.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:16.723+0000 7ff18e3e6700 1 --2- 172.21.15.22:0/3891371678 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff188071a00 0x7ff1881af9c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff17800e940 tx=0x7ff17800ec50 comp rx=0 tx=0).stop 2024-02-16T19:30:16.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:16.724+0000 7ff18e3e6700 1 -- 172.21.15.22:0/3891371678 shutdown_connections 2024-02-16T19:30:16.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:16.724+0000 7ff18e3e6700 1 --2- 172.21.15.22:0/3891371678 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7ff17006d6e0 0x7ff17006fb90 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:16.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:16.724+0000 7ff18e3e6700 1 --2- 172.21.15.22:0/3891371678 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff188071a00 0x7ff1881af9c0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:16.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:16.724+0000 7ff18e3e6700 1 --2- 172.21.15.22:0/3891371678 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1880723e0 0x7ff1881b1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:16.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:16.724+0000 7ff18e3e6700 1 -- 172.21.15.22:0/3891371678 >> 172.21.15.22:0/3891371678 conn(0x7ff18806d110 msgr2=0x7ff18810b460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:30:16.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:16.724+0000 7ff18e3e6700 1 -- 172.21.15.22:0/3891371678 shutdown_connections 2024-02-16T19:30:16.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:16.724+0000 7ff18e3e6700 1 -- 172.21.15.22:0/3891371678 wait complete. 2024-02-16T19:30:17.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:17 smithi117 ceph-mon[35309]: from='client.14704 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:30:17.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:17 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:30:17.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:17 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:30:17.714 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:17 smithi022 ceph-mon[29221]: from='client.14704 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:30:17.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:17 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:30:17.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:17 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:30:18.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:18 smithi022 ceph-mon[29221]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:30:18.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:18 smithi022 ceph-mon[29221]: pgmap v15: 65 pgs: 65 active+clean; 312 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 29 KiB/s rd, 1.4 MiB/s wr, 79 op/s 2024-02-16T19:30:18.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:18 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:30:18.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:18 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:30:18.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:18 smithi117 ceph-mon[35309]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:30:18.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:18 smithi117 ceph-mon[35309]: pgmap v15: 65 pgs: 65 active+clean; 312 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 29 KiB/s rd, 1.4 MiB/s wr, 79 op/s 2024-02-16T19:30:18.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:18 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:30:18.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:18 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:30:20.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:20 smithi117 ceph-mon[35309]: pgmap v16: 65 pgs: 65 active+clean; 307 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 47 KiB/s rd, 1.6 MiB/s wr, 102 op/s 2024-02-16T19:30:20.714 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:20 smithi022 ceph-mon[29221]: pgmap v16: 65 pgs: 65 active+clean; 307 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 47 KiB/s rd, 1.6 MiB/s wr, 102 op/s 2024-02-16T19:30:22.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:22 smithi022 ceph-mon[29221]: pgmap v17: 65 pgs: 65 active+clean; 300 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 32 KiB/s rd, 2.2 MiB/s wr, 124 op/s 2024-02-16T19:30:22.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:22 smithi117 ceph-mon[35309]: pgmap v17: 65 pgs: 65 active+clean; 300 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 32 KiB/s rd, 2.2 MiB/s wr, 124 op/s 2024-02-16T19:30:24.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:24 smithi117 ceph-mon[35309]: pgmap v18: 65 pgs: 65 active+clean; 300 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 32 KiB/s rd, 1.5 MiB/s wr, 93 op/s 2024-02-16T19:30:24.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:24 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:30:24.646 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:24 smithi022 ceph-mon[29221]: pgmap v18: 65 pgs: 65 active+clean; 300 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 32 KiB/s rd, 1.5 MiB/s wr, 93 op/s 2024-02-16T19:30:24.646 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:24 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:30:26.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:26 smithi117 ceph-mon[35309]: pgmap v19: 65 pgs: 65 active+clean; 304 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 44 KiB/s rd, 2.3 MiB/s wr, 123 op/s 2024-02-16T19:30:26.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:26 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:30:26.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:26 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:30:26.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:26 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:30:26.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:26 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:30:26.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:26 smithi117 ceph-mon[35309]: Updating smithi022:/etc/ceph/ceph.conf 2024-02-16T19:30:26.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:26 smithi117 ceph-mon[35309]: Updating smithi117:/etc/ceph/ceph.conf 2024-02-16T19:30:26.714 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:26 smithi022 ceph-mon[29221]: pgmap v19: 65 pgs: 65 active+clean; 304 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 44 KiB/s rd, 2.3 MiB/s wr, 123 op/s 2024-02-16T19:30:26.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:26 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:30:26.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:26 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:30:26.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:26 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:30:26.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:26 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:30:26.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:26 smithi022 ceph-mon[29221]: Updating smithi022:/etc/ceph/ceph.conf 2024-02-16T19:30:26.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:26 smithi022 ceph-mon[29221]: Updating smithi117:/etc/ceph/ceph.conf 2024-02-16T19:30:27.620 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:27 smithi022 ceph-mon[29221]: Updating smithi117:/var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T19:30:27.620 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:27 smithi022 ceph-mon[29221]: Updating smithi022:/var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T19:30:27.620 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:27 smithi022 ceph-mon[29221]: Updating smithi117:/etc/ceph/ceph.client.admin.keyring 2024-02-16T19:30:27.621 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:27 smithi022 ceph-mon[29221]: Updating smithi022:/etc/ceph/ceph.client.admin.keyring 2024-02-16T19:30:27.621 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:27 smithi022 ceph-mon[29221]: Updating smithi117:/var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-16T19:30:27.621 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:27 smithi022 ceph-mon[29221]: Updating smithi022:/var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-16T19:30:27.621 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:27 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:30:27.621 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:27 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:30:27.621 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:27 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:30:27.621 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:27 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:30:27.621 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:27 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:30:27.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:27 smithi117 ceph-mon[35309]: Updating smithi117:/var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T19:30:27.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:27 smithi117 ceph-mon[35309]: Updating smithi022:/var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T19:30:27.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:27 smithi117 ceph-mon[35309]: Updating smithi117:/etc/ceph/ceph.client.admin.keyring 2024-02-16T19:30:27.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:27 smithi117 ceph-mon[35309]: Updating smithi022:/etc/ceph/ceph.client.admin.keyring 2024-02-16T19:30:27.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:27 smithi117 ceph-mon[35309]: Updating smithi117:/var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-16T19:30:27.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:27 smithi117 ceph-mon[35309]: Updating smithi022:/var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-16T19:30:27.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:27 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:30:27.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:27 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:30:27.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:27 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:30:27.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:27 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:30:27.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:27 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:30:28.464 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:28 smithi022 ceph-mon[29221]: Reconfiguring prometheus.smithi022 (dependencies changed)... 2024-02-16T19:30:28.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:28 smithi022 ceph-mon[29221]: Reconfiguring daemon prometheus.smithi022 on smithi022 2024-02-16T19:30:28.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:28 smithi022 ceph-mon[29221]: pgmap v20: 65 pgs: 65 active+clean; 304 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 30 KiB/s rd, 1.6 MiB/s wr, 87 op/s 2024-02-16T19:30:28.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:28 smithi117 ceph-mon[35309]: Reconfiguring prometheus.smithi022 (dependencies changed)... 2024-02-16T19:30:28.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:28 smithi117 ceph-mon[35309]: Reconfiguring daemon prometheus.smithi022 on smithi022 2024-02-16T19:30:28.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:28 smithi117 ceph-mon[35309]: pgmap v20: 65 pgs: 65 active+clean; 304 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 30 KiB/s rd, 1.6 MiB/s wr, 87 op/s 2024-02-16T19:30:30.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:30 smithi117 ceph-mon[35309]: pgmap v21: 65 pgs: 65 active+clean; 305 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 30 KiB/s rd, 1.7 MiB/s wr, 97 op/s 2024-02-16T19:30:30.714 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:30 smithi022 ceph-mon[29221]: pgmap v21: 65 pgs: 65 active+clean; 305 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 30 KiB/s rd, 1.7 MiB/s wr, 97 op/s 2024-02-16T19:30:32.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:32 smithi117 ceph-mon[35309]: pgmap v22: 65 pgs: 65 active+clean; 308 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 28 KiB/s rd, 2.3 MiB/s wr, 199 op/s 2024-02-16T19:30:32.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:32 smithi022 ceph-mon[29221]: pgmap v22: 65 pgs: 65 active+clean; 308 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 28 KiB/s rd, 2.3 MiB/s wr, 199 op/s 2024-02-16T19:30:34.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:34 smithi022 ceph-mon[29221]: pgmap v23: 65 pgs: 65 active+clean; 308 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 28 KiB/s rd, 1.7 MiB/s wr, 164 op/s 2024-02-16T19:30:34.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:34 smithi117 ceph-mon[35309]: pgmap v23: 65 pgs: 65 active+clean; 308 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 28 KiB/s rd, 1.7 MiB/s wr, 164 op/s 2024-02-16T19:30:36.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:36 smithi117 ceph-mon[35309]: pgmap v24: 65 pgs: 65 active+clean; 305 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 40 KiB/s rd, 2.4 MiB/s wr, 353 op/s 2024-02-16T19:30:36.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:36 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:30:36.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:36 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:30:36.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:36 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-16T19:30:36.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:36 smithi117 ceph-mon[35309]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-16T19:30:36.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:36 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:30:36.714 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:36 smithi022 ceph-mon[29221]: pgmap v24: 65 pgs: 65 active+clean; 305 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 40 KiB/s rd, 2.4 MiB/s wr, 353 op/s 2024-02-16T19:30:36.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:36 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:30:36.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:36 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:30:36.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:36 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-16T19:30:36.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:36 smithi022 ceph-mon[29221]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-16T19:30:36.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:36 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:30:38.104 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:37 smithi117 ceph-mon[35309]: pgmap v25: 65 pgs: 65 active+clean; 305 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 28 KiB/s rd, 1.5 MiB/s wr, 323 op/s 2024-02-16T19:30:38.104 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:37 smithi117 ceph-mon[35309]: Upgrade: Updating mgr.smithi117.cohbtv 2024-02-16T19:30:38.104 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:37 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi117.cohbtv", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-16T19:30:38.104 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:37 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-16T19:30:38.104 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:37 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:30:38.104 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:37 smithi117 ceph-mon[35309]: Deploying daemon mgr.smithi117.cohbtv on smithi117 2024-02-16T19:30:38.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:37 smithi022 ceph-mon[29221]: pgmap v25: 65 pgs: 65 active+clean; 305 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 28 KiB/s rd, 1.5 MiB/s wr, 323 op/s 2024-02-16T19:30:38.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:37 smithi022 ceph-mon[29221]: Upgrade: Updating mgr.smithi117.cohbtv 2024-02-16T19:30:38.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:37 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi117.cohbtv", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-16T19:30:38.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:37 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-16T19:30:38.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:37 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:30:38.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:37 smithi022 ceph-mon[29221]: Deploying daemon mgr.smithi117.cohbtv on smithi117 2024-02-16T19:30:39.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:38 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:30:39.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:38 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:30:40.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:39 smithi022 ceph-mon[29221]: pgmap v26: 65 pgs: 65 active+clean; 304 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 28 KiB/s rd, 1.5 MiB/s wr, 329 op/s 2024-02-16T19:30:40.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:39 smithi117 ceph-mon[35309]: pgmap v26: 65 pgs: 65 active+clean; 304 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 28 KiB/s rd, 1.5 MiB/s wr, 329 op/s 2024-02-16T19:30:42.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:42 smithi117 ceph-mon[35309]: pgmap v27: 65 pgs: 65 active+clean; 306 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 41 KiB/s rd, 2.1 MiB/s wr, 407 op/s 2024-02-16T19:30:42.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:42 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:30:42.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:42 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:30:42.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:42 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:30:42.629 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:42 smithi022 ceph-mon[29221]: pgmap v27: 65 pgs: 65 active+clean; 306 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 41 KiB/s rd, 2.1 MiB/s wr, 407 op/s 2024-02-16T19:30:42.629 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:42 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:30:42.629 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:42 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:30:42.630 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:42 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:30:44.380 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:44 smithi117 ceph-mon[35309]: pgmap v28: 65 pgs: 65 active+clean; 306 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 25 KiB/s rd, 1.3 MiB/s wr, 282 op/s 2024-02-16T19:30:44.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:44 smithi022 ceph-mon[29221]: pgmap v28: 65 pgs: 65 active+clean; 306 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 25 KiB/s rd, 1.3 MiB/s wr, 282 op/s 2024-02-16T19:30:46.275 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:46 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:30:46.275 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:46 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:30:46.276 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:46 smithi117 ceph-mon[35309]: pgmap v29: 65 pgs: 65 active+clean; 294 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 40 KiB/s rd, 2.0 MiB/s wr, 350 op/s 2024-02-16T19:30:46.464 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:46 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:30:46.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:46 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:30:46.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:46 smithi022 ceph-mon[29221]: pgmap v29: 65 pgs: 65 active+clean; 294 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 40 KiB/s rd, 2.0 MiB/s wr, 350 op/s 2024-02-16T19:30:46.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:46.850+0000 7f599812c700 1 -- 172.21.15.22:0/4063531068 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5990103680 msgr2=0x7f5990105a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:46.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:46.850+0000 7f599812c700 1 --2- 172.21.15.22:0/4063531068 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5990103680 0x7f5990105a60 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f5980009910 tx=0x7f5980009c20 comp rx=0 tx=0).stop 2024-02-16T19:30:46.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:46.850+0000 7f599812c700 1 -- 172.21.15.22:0/4063531068 shutdown_connections 2024-02-16T19:30:46.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:46.850+0000 7f599812c700 1 --2- 172.21.15.22:0/4063531068 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5990103680 0x7f5990105a60 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:46.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:46.850+0000 7f599812c700 1 --2- 172.21.15.22:0/4063531068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5990100d60 0x7f5990103140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:46.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:46.850+0000 7f599812c700 1 -- 172.21.15.22:0/4063531068 >> 172.21.15.22:0/4063531068 conn(0x7f59900fa780 msgr2=0x7f59900fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:30:46.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:46.850+0000 7f599812c700 1 -- 172.21.15.22:0/4063531068 shutdown_connections 2024-02-16T19:30:46.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:46.850+0000 7f599812c700 1 -- 172.21.15.22:0/4063531068 wait complete. 2024-02-16T19:30:46.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:46.851+0000 7f599812c700 1 Processor -- start 2024-02-16T19:30:46.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:46.851+0000 7f599812c700 1 -- start start 2024-02-16T19:30:46.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:46.851+0000 7f599812c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5990100d60 0x7f5990195ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:30:46.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:46.851+0000 7f599812c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5990103680 0x7f5990196020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:30:46.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:46.851+0000 7f599812c700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5990196640 con 0x7f5990100d60 2024-02-16T19:30:46.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:46.851+0000 7f599812c700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5990196780 con 0x7f5990103680 2024-02-16T19:30:46.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:46.851+0000 7f5995ec8700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5990100d60 0x7f5990195ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:30:46.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:46.851+0000 7f5995ec8700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5990100d60 0x7f5990195ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57206/0 (socket says 172.21.15.22:57206) 2024-02-16T19:30:46.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:46.851+0000 7f5995ec8700 1 -- 172.21.15.22:0/1805657721 learned_addr learned my addr 172.21.15.22:0/1805657721 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:30:46.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:46.851+0000 7f59956c7700 1 --2- 172.21.15.22:0/1805657721 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5990103680 0x7f5990196020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:30:46.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:46.851+0000 7f5995ec8700 1 -- 172.21.15.22:0/1805657721 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5990103680 msgr2=0x7f5990196020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:46.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:46.851+0000 7f5995ec8700 1 --2- 172.21.15.22:0/1805657721 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5990103680 0x7f5990196020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:46.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:46.851+0000 7f5995ec8700 1 -- 172.21.15.22:0/1805657721 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59800095f0 con 0x7f5990100d60 2024-02-16T19:30:46.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:46.852+0000 7f5995ec8700 1 --2- 172.21.15.22:0/1805657721 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5990100d60 0x7f5990195ae0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f598c00d900 tx=0x7f598c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:30:46.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:46.852+0000 7f5986ffd700 1 -- 172.21.15.22:0/1805657721 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f598c009900 con 0x7f5990100d60 2024-02-16T19:30:46.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:46.852+0000 7f599812c700 1 -- 172.21.15.22:0/1805657721 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f599019b230 con 0x7f5990100d60 2024-02-16T19:30:46.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:46.852+0000 7f5986ffd700 1 -- 172.21.15.22:0/1805657721 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f598c010460 con 0x7f5990100d60 2024-02-16T19:30:46.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:46.852+0000 7f5986ffd700 1 -- 172.21.15.22:0/1805657721 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f598c00f5f0 con 0x7f5990100d60 2024-02-16T19:30:46.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:46.852+0000 7f599812c700 1 -- 172.21.15.22:0/1805657721 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f599019b780 con 0x7f5990100d60 2024-02-16T19:30:46.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:46.853+0000 7f599812c700 1 -- 172.21.15.22:0/1805657721 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f599018fbe0 con 0x7f5990100d60 2024-02-16T19:30:46.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:46.853+0000 7f5986ffd700 1 -- 172.21.15.22:0/1805657721 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 31) v1 ==== 91448+0+0 (secure 0 0 0) 0x7f598c0105d0 con 0x7f5990100d60 2024-02-16T19:30:46.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:46.854+0000 7f5986ffd700 1 --2- 172.21.15.22:0/1805657721 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f597c06d660 0x7f597c06fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:30:46.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:46.854+0000 7f5986ffd700 1 -- 172.21.15.22:0/1805657721 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f598c08cda0 con 0x7f5990100d60 2024-02-16T19:30:46.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:46.854+0000 7f59956c7700 1 --2- 172.21.15.22:0/1805657721 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f597c06d660 0x7f597c06fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:30:46.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:46.854+0000 7f59956c7700 1 --2- 172.21.15.22:0/1805657721 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f597c06d660 0x7f597c06fb10 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5980006010 tx=0x7f598000b540 comp rx=0 tx=0).ready entity=mgr.24407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:30:46.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:46.861+0000 7f5986ffd700 1 -- 172.21.15.22:0/1805657721 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f598c05a390 con 0x7f5990100d60 2024-02-16T19:30:47.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.206+0000 7f599812c700 1 -- 172.21.15.22:0/1805657721 --> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5990061190 con 0x7f597c06d660 2024-02-16T19:30:47.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.208+0000 7f5986ffd700 1 -- 172.21.15.22:0/1805657721 <== mgr.24407 v2:172.21.15.22:6800/3703806543 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7f5990061190 con 0x7f597c06d660 2024-02-16T19:30:47.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.210+0000 7f599812c700 1 -- 172.21.15.22:0/1805657721 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f597c06d660 msgr2=0x7f597c06fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:47.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.210+0000 7f599812c700 1 --2- 172.21.15.22:0/1805657721 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f597c06d660 0x7f597c06fb10 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5980006010 tx=0x7f598000b540 comp rx=0 tx=0).stop 2024-02-16T19:30:47.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.210+0000 7f599812c700 1 -- 172.21.15.22:0/1805657721 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5990100d60 msgr2=0x7f5990195ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:47.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.210+0000 7f599812c700 1 --2- 172.21.15.22:0/1805657721 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5990100d60 0x7f5990195ae0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f598c00d900 tx=0x7f598c00dc10 comp rx=0 tx=0).stop 2024-02-16T19:30:47.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.210+0000 7f599812c700 1 -- 172.21.15.22:0/1805657721 shutdown_connections 2024-02-16T19:30:47.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.210+0000 7f599812c700 1 --2- 172.21.15.22:0/1805657721 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f597c06d660 0x7f597c06fb10 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:47.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.210+0000 7f599812c700 1 --2- 172.21.15.22:0/1805657721 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5990100d60 0x7f5990195ae0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:47.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.210+0000 7f599812c700 1 --2- 172.21.15.22:0/1805657721 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5990103680 0x7f5990196020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:47.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.210+0000 7f599812c700 1 -- 172.21.15.22:0/1805657721 >> 172.21.15.22:0/1805657721 conn(0x7f59900fa780 msgr2=0x7f59900fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:30:47.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.210+0000 7f599812c700 1 -- 172.21.15.22:0/1805657721 shutdown_connections 2024-02-16T19:30:47.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.210+0000 7f599812c700 1 -- 172.21.15.22:0/1805657721 wait complete. 2024-02-16T19:30:47.222 INFO:teuthology.orchestra.run.smithi022.stdout:true 2024-02-16T19:30:47.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.336+0000 7fb184e3a700 1 -- 172.21.15.22:0/4289463211 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1800ff130 msgr2=0x7fb1800ff540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:47.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.336+0000 7fb184e3a700 1 --2- 172.21.15.22:0/4289463211 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1800ff130 0x7fb1800ff540 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb168009910 tx=0x7fb168009c20 comp rx=0 tx=0).stop 2024-02-16T19:30:47.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.336+0000 7fb184e3a700 1 -- 172.21.15.22:0/4289463211 shutdown_connections 2024-02-16T19:30:47.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.336+0000 7fb184e3a700 1 --2- 172.21.15.22:0/4289463211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1801003d0 0x7fb180100840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:47.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.336+0000 7fb184e3a700 1 --2- 172.21.15.22:0/4289463211 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1800ff130 0x7fb1800ff540 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:47.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.336+0000 7fb184e3a700 1 -- 172.21.15.22:0/4289463211 >> 172.21.15.22:0/4289463211 conn(0x7fb1800fa780 msgr2=0x7fb1800fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:30:47.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.336+0000 7fb184e3a700 1 -- 172.21.15.22:0/4289463211 shutdown_connections 2024-02-16T19:30:47.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.336+0000 7fb184e3a700 1 -- 172.21.15.22:0/4289463211 wait complete. 2024-02-16T19:30:47.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.337+0000 7fb184e3a700 1 Processor -- start 2024-02-16T19:30:47.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.337+0000 7fb184e3a700 1 -- start start 2024-02-16T19:30:47.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.337+0000 7fb184e3a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1800ff130 0x7fb180195aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:30:47.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.337+0000 7fb184e3a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1801003d0 0x7fb180195fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:30:47.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.337+0000 7fb184e3a700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb180196600 con 0x7fb1801003d0 2024-02-16T19:30:47.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.337+0000 7fb184e3a700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb180196740 con 0x7fb1800ff130 2024-02-16T19:30:47.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.337+0000 7fb17dd9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1801003d0 0x7fb180195fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:30:47.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.337+0000 7fb17dd9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1801003d0 0x7fb180195fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57230/0 (socket says 172.21.15.22:57230) 2024-02-16T19:30:47.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.337+0000 7fb17dd9b700 1 -- 172.21.15.22:0/1077417548 learned_addr learned my addr 172.21.15.22:0/1077417548 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:30:47.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.337+0000 7fb17e59c700 1 --2- 172.21.15.22:0/1077417548 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1800ff130 0x7fb180195aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:30:47.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.337+0000 7fb17dd9b700 1 -- 172.21.15.22:0/1077417548 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1800ff130 msgr2=0x7fb180195aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:47.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.337+0000 7fb17dd9b700 1 --2- 172.21.15.22:0/1077417548 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1800ff130 0x7fb180195aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:47.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.337+0000 7fb17dd9b700 1 -- 172.21.15.22:0/1077417548 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1680095f0 con 0x7fb1801003d0 2024-02-16T19:30:47.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.337+0000 7fb17dd9b700 1 --2- 172.21.15.22:0/1077417548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1801003d0 0x7fb180195fe0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb17000e940 tx=0x7fb17000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:30:47.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.337+0000 7fb1777fe700 1 -- 172.21.15.22:0/1077417548 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb17000cba0 con 0x7fb1801003d0 2024-02-16T19:30:47.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.337+0000 7fb1777fe700 1 -- 172.21.15.22:0/1077417548 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb17000ee90 con 0x7fb1801003d0 2024-02-16T19:30:47.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.338+0000 7fb1777fe700 1 -- 172.21.15.22:0/1077417548 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb170005650 con 0x7fb1801003d0 2024-02-16T19:30:47.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.338+0000 7fb184e3a700 1 -- 172.21.15.22:0/1077417548 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb18019b1f0 con 0x7fb1801003d0 2024-02-16T19:30:47.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.338+0000 7fb184e3a700 1 -- 172.21.15.22:0/1077417548 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb18019b740 con 0x7fb1801003d0 2024-02-16T19:30:47.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.339+0000 7fb184e3a700 1 -- 172.21.15.22:0/1077417548 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb180107c30 con 0x7fb1801003d0 2024-02-16T19:30:47.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.339+0000 7fb1777fe700 1 -- 172.21.15.22:0/1077417548 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 31) v1 ==== 91448+0+0 (secure 0 0 0) 0x7fb170004750 con 0x7fb1801003d0 2024-02-16T19:30:47.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.339+0000 7fb1777fe700 1 --2- 172.21.15.22:0/1077417548 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7fb16c06d660 0x7fb16c06fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:30:47.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.340+0000 7fb1777fe700 1 -- 172.21.15.22:0/1077417548 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fb170014070 con 0x7fb1801003d0 2024-02-16T19:30:47.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.340+0000 7fb17e59c700 1 --2- 172.21.15.22:0/1077417548 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7fb16c06d660 0x7fb16c06fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:30:47.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.340+0000 7fb17e59c700 1 --2- 172.21.15.22:0/1077417548 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7fb16c06d660 0x7fb16c06fb10 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb168009fd0 tx=0x7fb168005940 comp rx=0 tx=0).ready entity=mgr.24407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:30:47.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.344+0000 7fb1777fe700 1 -- 172.21.15.22:0/1077417548 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7fb17005aaa0 con 0x7fb1801003d0 2024-02-16T19:30:47.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:47 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:30:47.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:47 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:30:47.464 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:47 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:30:47.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:47 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:30:47.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.682+0000 7fb184e3a700 1 -- 172.21.15.22:0/1077417548 --> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb18019bde0 con 0x7fb16c06d660 2024-02-16T19:30:47.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.685+0000 7fb1777fe700 1 -- 172.21.15.22:0/1077417548 <== mgr.24407 v2:172.21.15.22:6800/3703806543 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7fb18019bde0 con 0x7fb16c06d660 2024-02-16T19:30:47.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.689+0000 7fb184e3a700 1 -- 172.21.15.22:0/1077417548 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7fb16c06d660 msgr2=0x7fb16c06fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:47.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.689+0000 7fb184e3a700 1 --2- 172.21.15.22:0/1077417548 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7fb16c06d660 0x7fb16c06fb10 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb168009fd0 tx=0x7fb168005940 comp rx=0 tx=0).stop 2024-02-16T19:30:47.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.689+0000 7fb184e3a700 1 -- 172.21.15.22:0/1077417548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1801003d0 msgr2=0x7fb180195fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:47.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.689+0000 7fb184e3a700 1 --2- 172.21.15.22:0/1077417548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1801003d0 0x7fb180195fe0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb17000e940 tx=0x7fb17000ec50 comp rx=0 tx=0).stop 2024-02-16T19:30:47.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.690+0000 7fb184e3a700 1 -- 172.21.15.22:0/1077417548 shutdown_connections 2024-02-16T19:30:47.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.690+0000 7fb184e3a700 1 --2- 172.21.15.22:0/1077417548 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7fb16c06d660 0x7fb16c06fb10 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:47.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.690+0000 7fb184e3a700 1 --2- 172.21.15.22:0/1077417548 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1800ff130 0x7fb180195aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:47.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.690+0000 7fb184e3a700 1 --2- 172.21.15.22:0/1077417548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1801003d0 0x7fb180195fe0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:47.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.690+0000 7fb184e3a700 1 -- 172.21.15.22:0/1077417548 >> 172.21.15.22:0/1077417548 conn(0x7fb1800fa780 msgr2=0x7fb180068540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:30:47.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.690+0000 7fb184e3a700 1 -- 172.21.15.22:0/1077417548 shutdown_connections 2024-02-16T19:30:47.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.690+0000 7fb184e3a700 1 -- 172.21.15.22:0/1077417548 wait complete. 2024-02-16T19:30:47.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.834+0000 7f6cd223f700 1 -- 172.21.15.22:0/793094824 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ccc100d60 msgr2=0x7f6ccc103140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:47.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.834+0000 7f6cd223f700 1 --2- 172.21.15.22:0/793094824 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ccc100d60 0x7f6ccc103140 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc009910 tx=0x7f6cbc009c20 comp rx=0 tx=0).stop 2024-02-16T19:30:47.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.834+0000 7f6cd223f700 1 -- 172.21.15.22:0/793094824 shutdown_connections 2024-02-16T19:30:47.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.834+0000 7f6cd223f700 1 --2- 172.21.15.22:0/793094824 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ccc103680 0x7f6ccc105a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:47.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.834+0000 7f6cd223f700 1 --2- 172.21.15.22:0/793094824 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ccc100d60 0x7f6ccc103140 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:47.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.834+0000 7f6cd223f700 1 -- 172.21.15.22:0/793094824 >> 172.21.15.22:0/793094824 conn(0x7f6ccc0fa780 msgr2=0x7f6ccc0fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:30:47.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.834+0000 7f6cd223f700 1 -- 172.21.15.22:0/793094824 shutdown_connections 2024-02-16T19:30:47.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.834+0000 7f6cd223f700 1 -- 172.21.15.22:0/793094824 wait complete. 2024-02-16T19:30:47.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.835+0000 7f6cd223f700 1 Processor -- start 2024-02-16T19:30:47.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.835+0000 7f6cd223f700 1 -- start start 2024-02-16T19:30:47.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.835+0000 7f6cd223f700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ccc100d60 0x7f6ccc197d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:30:47.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.835+0000 7f6cd223f700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ccc103680 0x7f6ccc198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:30:47.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.835+0000 7f6cd223f700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ccc198860 con 0x7f6ccc100d60 2024-02-16T19:30:47.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.835+0000 7f6cd223f700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ccc19d270 con 0x7f6ccc103680 2024-02-16T19:30:47.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.835+0000 7f6ccb7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ccc100d60 0x7f6ccc197d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:30:47.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.835+0000 7f6ccb7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ccc100d60 0x7f6ccc197d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57256/0 (socket says 172.21.15.22:57256) 2024-02-16T19:30:47.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.835+0000 7f6ccb7fe700 1 -- 172.21.15.22:0/2559300452 learned_addr learned my addr 172.21.15.22:0/2559300452 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:30:47.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.836+0000 7f6ccaffd700 1 --2- 172.21.15.22:0/2559300452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ccc103680 0x7f6ccc198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:30:47.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.836+0000 7f6ccb7fe700 1 -- 172.21.15.22:0/2559300452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ccc103680 msgr2=0x7f6ccc198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:47.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.836+0000 7f6ccb7fe700 1 --2- 172.21.15.22:0/2559300452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ccc103680 0x7f6ccc198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:47.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.836+0000 7f6ccb7fe700 1 -- 172.21.15.22:0/2559300452 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cbc0095f0 con 0x7f6ccc100d60 2024-02-16T19:30:47.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.836+0000 7f6ccb7fe700 1 --2- 172.21.15.22:0/2559300452 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ccc100d60 0x7f6ccc197d00 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc000c00 tx=0x7f6cbc00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:30:47.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.836+0000 7f6cc8ff9700 1 -- 172.21.15.22:0/2559300452 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6cbc01d070 con 0x7f6ccc100d60 2024-02-16T19:30:47.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.836+0000 7f6cd223f700 1 -- 172.21.15.22:0/2559300452 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ccc19d410 con 0x7f6ccc100d60 2024-02-16T19:30:47.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.836+0000 7f6cc8ff9700 1 -- 172.21.15.22:0/2559300452 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6cbc005dc0 con 0x7f6ccc100d60 2024-02-16T19:30:47.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.836+0000 7f6cc8ff9700 1 -- 172.21.15.22:0/2559300452 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6cbc022470 con 0x7f6ccc100d60 2024-02-16T19:30:47.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.836+0000 7f6cd223f700 1 -- 172.21.15.22:0/2559300452 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ccc19d8d0 con 0x7f6ccc100d60 2024-02-16T19:30:47.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.838+0000 7f6cd223f700 1 -- 172.21.15.22:0/2559300452 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ccc0fc340 con 0x7f6ccc100d60 2024-02-16T19:30:47.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.838+0000 7f6cc8ff9700 1 -- 172.21.15.22:0/2559300452 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 31) v1 ==== 91448+0+0 (secure 0 0 0) 0x7f6cbc003810 con 0x7f6ccc100d60 2024-02-16T19:30:47.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.838+0000 7f6cc8ff9700 1 --2- 172.21.15.22:0/2559300452 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f6cb406d660 0x7f6cb406fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:30:47.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.838+0000 7f6cc8ff9700 1 -- 172.21.15.22:0/2559300452 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f6cbc08e600 con 0x7f6ccc100d60 2024-02-16T19:30:47.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.839+0000 7f6ccaffd700 1 --2- 172.21.15.22:0/2559300452 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f6cb406d660 0x7f6cb406fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:30:47.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.839+0000 7f6ccaffd700 1 --2- 172.21.15.22:0/2559300452 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f6cb406d660 0x7f6cb406fb10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6cc0009c60 tx=0x7f6cc0009260 comp rx=0 tx=0).ready entity=mgr.24407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:30:47.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:47.844+0000 7f6cc8ff9700 1 -- 172.21.15.22:0/2559300452 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f6cbc058140 con 0x7f6ccc100d60 2024-02-16T19:30:48.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.160+0000 7f6cd223f700 1 -- 172.21.15.22:0/2559300452 --> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6ccc061190 con 0x7f6cb406d660 2024-02-16T19:30:48.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.172+0000 7f6cc8ff9700 1 -- 172.21.15.22:0/2559300452 <== mgr.24407 v2:172.21.15.22:6800/3703806543 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f6ccc061190 con 0x7f6cb406d660 2024-02-16T19:30:48.174 INFO:teuthology.orchestra.run.smithi022.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T19:30:48.174 INFO:teuthology.orchestra.run.smithi022.stdout:alertmanager.smithi022 smithi022 *:9093,9094 running (8m) 31s ago 10m 26.6M - 0.25.0 c8568f914cd2 4dac585d8372 2024-02-16T19:30:48.174 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-exporter.smithi022 smithi022 running (10m) 31s ago 10m 7704k - 18.2.0 dc2bc1663786 3f8bef4349ba 2024-02-16T19:30:48.174 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-exporter.smithi117 smithi117 running (9m) 3s ago 9m 7478k - 18.2.0 dc2bc1663786 59b65a7e309f 2024-02-16T19:30:48.174 INFO:teuthology.orchestra.run.smithi022.stdout:crash.smithi022 smithi022 running (10m) 31s ago 10m 6669k - 18.2.0 dc2bc1663786 efef476aa486 2024-02-16T19:30:48.174 INFO:teuthology.orchestra.run.smithi022.stdout:crash.smithi117 smithi117 running (9m) 3s ago 9m 6660k - 18.2.0 dc2bc1663786 5466453eb149 2024-02-16T19:30:48.174 INFO:teuthology.orchestra.run.smithi022.stdout:grafana.smithi022 smithi022 *:3000 running (8m) 31s ago 9m 85.9M - 9.4.7 954c08fa6188 a48872ecbbf2 2024-02-16T19:30:48.174 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi022.btppwm smithi022 running (4m) 31s ago 4m 258M - 18.2.0 dc2bc1663786 22971fb3a907 2024-02-16T19:30:48.174 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi022.ocukvs smithi022 running (4m) 31s ago 4m 13.7M - 18.2.0 dc2bc1663786 81ee7bd5fd39 2024-02-16T19:30:48.174 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi117.ibmppn smithi117 running (4m) 3s ago 4m 15.9M - 18.2.0 dc2bc1663786 8c606cbeeb86 2024-02-16T19:30:48.174 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi117.orzors smithi117 running (4m) 3s ago 4m 14.8M - 18.2.0 dc2bc1663786 7d2bc4df3feb 2024-02-16T19:30:48.174 INFO:teuthology.orchestra.run.smithi022.stdout:mgr.smithi022.gmdcch smithi022 *:8443,9283,8765 running (67s) 31s ago 11m 477M - 19.0.0-1450-gf78a58c0 f081f847070a 7cb74cc9deea 2024-02-16T19:30:48.174 INFO:teuthology.orchestra.run.smithi022.stdout:mgr.smithi117.cohbtv smithi117 *:8443,9283,8765 running (7s) 3s ago 9m 67.7M - 19.0.0-1450-gf78a58c0 f081f847070a c825b1152189 2024-02-16T19:30:48.174 INFO:teuthology.orchestra.run.smithi022.stdout:mon.smithi022 smithi022 running (11m) 31s ago 11m 54.0M 2048M 18.2.0 dc2bc1663786 b4ffc3032e12 2024-02-16T19:30:48.175 INFO:teuthology.orchestra.run.smithi022.stdout:mon.smithi117 smithi117 running (8m) 3s ago 8m 42.4M 2048M 18.2.0 dc2bc1663786 aff3e21a73a0 2024-02-16T19:30:48.175 INFO:teuthology.orchestra.run.smithi022.stdout:node-exporter.smithi022 smithi022 *:9100 running (10m) 31s ago 10m 16.1M - 1.5.0 0da6a335fe13 d727f51df007 2024-02-16T19:30:48.175 INFO:teuthology.orchestra.run.smithi022.stdout:node-exporter.smithi117 smithi117 *:9100 running (9m) 3s ago 9m 17.2M - 1.5.0 0da6a335fe13 fc8b2df5a29d 2024-02-16T19:30:48.175 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0 smithi022 running (8m) 31s ago 8m 180M 1217M 18.2.0 dc2bc1663786 0027f86f16ce 2024-02-16T19:30:48.175 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1 smithi022 running (7m) 31s ago 7m 187M 1217M 18.2.0 dc2bc1663786 87300382b8a7 2024-02-16T19:30:48.175 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2 smithi022 running (7m) 31s ago 7m 153M 1217M 18.2.0 dc2bc1663786 be8a631b4f93 2024-02-16T19:30:48.175 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3 smithi117 running (6m) 3s ago 6m 233M 2240M 18.2.0 dc2bc1663786 9c0ce9a981cf 2024-02-16T19:30:48.175 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4 smithi117 running (6m) 3s ago 6m 202M 2240M 18.2.0 dc2bc1663786 1779c3496f4d 2024-02-16T19:30:48.175 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5 smithi117 running (5m) 3s ago 5m 185M 2240M 18.2.0 dc2bc1663786 fefaabca0638 2024-02-16T19:30:48.175 INFO:teuthology.orchestra.run.smithi022.stdout:prometheus.smithi022 smithi022 *:9095 starting - - - - 2024-02-16T19:30:48.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.175+0000 7f6cd223f700 1 -- 172.21.15.22:0/2559300452 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f6cb406d660 msgr2=0x7f6cb406fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:48.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.175+0000 7f6cd223f700 1 --2- 172.21.15.22:0/2559300452 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f6cb406d660 0x7f6cb406fb10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6cc0009c60 tx=0x7f6cc0009260 comp rx=0 tx=0).stop 2024-02-16T19:30:48.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.175+0000 7f6cd223f700 1 -- 172.21.15.22:0/2559300452 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ccc100d60 msgr2=0x7f6ccc197d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:48.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.175+0000 7f6cd223f700 1 --2- 172.21.15.22:0/2559300452 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ccc100d60 0x7f6ccc197d00 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc000c00 tx=0x7f6cbc00f690 comp rx=0 tx=0).stop 2024-02-16T19:30:48.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.175+0000 7f6cd223f700 1 -- 172.21.15.22:0/2559300452 shutdown_connections 2024-02-16T19:30:48.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.175+0000 7f6cd223f700 1 --2- 172.21.15.22:0/2559300452 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f6cb406d660 0x7f6cb406fb10 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:48.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.175+0000 7f6cd223f700 1 --2- 172.21.15.22:0/2559300452 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ccc100d60 0x7f6ccc197d00 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:48.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.175+0000 7f6cd223f700 1 --2- 172.21.15.22:0/2559300452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ccc103680 0x7f6ccc198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:48.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.175+0000 7f6cd223f700 1 -- 172.21.15.22:0/2559300452 >> 172.21.15.22:0/2559300452 conn(0x7f6ccc0fa780 msgr2=0x7f6ccc100610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:30:48.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.175+0000 7f6cd223f700 1 -- 172.21.15.22:0/2559300452 shutdown_connections 2024-02-16T19:30:48.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.175+0000 7f6cd223f700 1 -- 172.21.15.22:0/2559300452 wait complete. 2024-02-16T19:30:48.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.320+0000 7f0ce096f700 1 -- 172.21.15.22:0/2699867177 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0cdc0ff150 msgr2=0x7f0cdc0ff560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:48.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.320+0000 7f0ce096f700 1 --2- 172.21.15.22:0/2699867177 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0cdc0ff150 0x7f0cdc0ff560 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc009910 tx=0x7f0ccc009c20 comp rx=0 tx=0).stop 2024-02-16T19:30:48.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.320+0000 7f0ce096f700 1 -- 172.21.15.22:0/2699867177 shutdown_connections 2024-02-16T19:30:48.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.320+0000 7f0ce096f700 1 --2- 172.21.15.22:0/2699867177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cdc1003f0 0x7f0cdc100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:48.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.320+0000 7f0ce096f700 1 --2- 172.21.15.22:0/2699867177 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0cdc0ff150 0x7f0cdc0ff560 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:48.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.320+0000 7f0ce096f700 1 -- 172.21.15.22:0/2699867177 >> 172.21.15.22:0/2699867177 conn(0x7f0cdc0fa7a0 msgr2=0x7f0cdc0fcbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:30:48.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.320+0000 7f0ce096f700 1 -- 172.21.15.22:0/2699867177 shutdown_connections 2024-02-16T19:30:48.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.320+0000 7f0ce096f700 1 -- 172.21.15.22:0/2699867177 wait complete. 2024-02-16T19:30:48.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.321+0000 7f0ce096f700 1 Processor -- start 2024-02-16T19:30:48.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.321+0000 7f0ce096f700 1 -- start start 2024-02-16T19:30:48.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.321+0000 7f0ce096f700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0cdc0ff150 0x7f0cdc069990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:30:48.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.322+0000 7f0cda59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0cdc0ff150 0x7f0cdc069990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:30:48.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.322+0000 7f0cda59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0cdc0ff150 0x7f0cdc069990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57272/0 (socket says 172.21.15.22:57272) 2024-02-16T19:30:48.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.322+0000 7f0cda59c700 1 -- 172.21.15.22:0/1775539425 learned_addr learned my addr 172.21.15.22:0/1775539425 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:30:48.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.322+0000 7f0ce096f700 1 --2- 172.21.15.22:0/1775539425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cdc1003f0 0x7f0cdc19aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:30:48.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.322+0000 7f0ce096f700 1 -- 172.21.15.22:0/1775539425 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cdc19b110 con 0x7f0cdc0ff150 2024-02-16T19:30:48.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.322+0000 7f0ce096f700 1 -- 172.21.15.22:0/1775539425 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cdc19b250 con 0x7f0cdc1003f0 2024-02-16T19:30:48.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.322+0000 7f0cd9d9b700 1 --2- 172.21.15.22:0/1775539425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cdc1003f0 0x7f0cdc19aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:30:48.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.322+0000 7f0cda59c700 1 -- 172.21.15.22:0/1775539425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cdc1003f0 msgr2=0x7f0cdc19aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:48.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.322+0000 7f0cda59c700 1 --2- 172.21.15.22:0/1775539425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cdc1003f0 0x7f0cdc19aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:48.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.322+0000 7f0cda59c700 1 -- 172.21.15.22:0/1775539425 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ccc0095f0 con 0x7f0cdc0ff150 2024-02-16T19:30:48.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.322+0000 7f0cda59c700 1 --2- 172.21.15.22:0/1775539425 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0cdc0ff150 0x7f0cdc069990 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc000c00 tx=0x7f0ccc00f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:30:48.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.325+0000 7f0ccb7fe700 1 -- 172.21.15.22:0/1775539425 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ccc01d070 con 0x7f0cdc0ff150 2024-02-16T19:30:48.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.325+0000 7f0ce096f700 1 -- 172.21.15.22:0/1775539425 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cdc19b4d0 con 0x7f0cdc0ff150 2024-02-16T19:30:48.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.325+0000 7f0ce096f700 1 -- 172.21.15.22:0/1775539425 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cdc19b900 con 0x7f0cdc0ff150 2024-02-16T19:30:48.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.325+0000 7f0ccb7fe700 1 -- 172.21.15.22:0/1775539425 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0ccc005dc0 con 0x7f0cdc0ff150 2024-02-16T19:30:48.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.325+0000 7f0ccb7fe700 1 -- 172.21.15.22:0/1775539425 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ccc022640 con 0x7f0cdc0ff150 2024-02-16T19:30:48.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.326+0000 7f0ce096f700 1 -- 172.21.15.22:0/1775539425 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cbc005320 con 0x7f0cdc0ff150 2024-02-16T19:30:48.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.327+0000 7f0ccb7fe700 1 -- 172.21.15.22:0/1775539425 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 31) v1 ==== 91448+0+0 (secure 0 0 0) 0x7f0ccc0227a0 con 0x7f0cdc0ff150 2024-02-16T19:30:48.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.327+0000 7f0ccb7fe700 1 --2- 172.21.15.22:0/1775539425 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f0cc406d660 0x7f0cc406fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:30:48.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.327+0000 7f0ccb7fe700 1 -- 172.21.15.22:0/1775539425 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f0ccc08e7b0 con 0x7f0cdc0ff150 2024-02-16T19:30:48.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.328+0000 7f0cd9d9b700 1 --2- 172.21.15.22:0/1775539425 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f0cc406d660 0x7f0cc406fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:30:48.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.328+0000 7f0cd9d9b700 1 --2- 172.21.15.22:0/1775539425 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f0cc406d660 0x7f0cc406fb10 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0009bf0 tx=0x7f0cd0009260 comp rx=0 tx=0).ready entity=mgr.24407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:30:48.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.334+0000 7f0ccb7fe700 1 -- 172.21.15.22:0/1775539425 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f0ccc05bda0 con 0x7f0cdc0ff150 2024-02-16T19:30:48.418 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:48 smithi022 ceph-mon[29221]: from='client.14720 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:30:48.419 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:48 smithi022 ceph-mon[29221]: pgmap v30: 65 pgs: 65 active+clean; 294 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 29 KiB/s rd, 1.4 MiB/s wr, 161 op/s 2024-02-16T19:30:48.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:48 smithi117 ceph-mon[35309]: from='client.14720 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:30:48.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:48 smithi117 ceph-mon[35309]: pgmap v30: 65 pgs: 65 active+clean; 294 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 29 KiB/s rd, 1.4 MiB/s wr, 161 op/s 2024-02-16T19:30:48.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.760+0000 7f0ce096f700 1 -- 172.21.15.22:0/1775539425 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f0cbc005cc0 con 0x7f0cdc0ff150 2024-02-16T19:30:48.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.762+0000 7f0ccb7fe700 1 -- 172.21.15.22:0/1775539425 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7f0ccc05b930 con 0x7f0cdc0ff150 2024-02-16T19:30:48.766 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-02-16T19:30:48.767 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": { 2024-02-16T19:30:48.767 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-02-16T19:30:48.767 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:30:48.767 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": { 2024-02-16T19:30:48.767 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T19:30:48.767 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:30:48.767 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": { 2024-02-16T19:30:48.767 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-02-16T19:30:48.767 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:30:48.767 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": { 2024-02-16T19:30:48.767 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-02-16T19:30:48.767 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:30:48.767 INFO:teuthology.orchestra.run.smithi022.stdout: "overall": { 2024-02-16T19:30:48.767 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 12, 2024-02-16T19:30:48.768 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T19:30:48.768 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-02-16T19:30:48.768 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-02-16T19:30:48.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.769+0000 7f0ce096f700 1 -- 172.21.15.22:0/1775539425 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f0cc406d660 msgr2=0x7f0cc406fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:48.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.769+0000 7f0ce096f700 1 --2- 172.21.15.22:0/1775539425 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f0cc406d660 0x7f0cc406fb10 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0009bf0 tx=0x7f0cd0009260 comp rx=0 tx=0).stop 2024-02-16T19:30:48.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.769+0000 7f0ce096f700 1 -- 172.21.15.22:0/1775539425 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0cdc0ff150 msgr2=0x7f0cdc069990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:48.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.769+0000 7f0ce096f700 1 --2- 172.21.15.22:0/1775539425 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0cdc0ff150 0x7f0cdc069990 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc000c00 tx=0x7f0ccc00f740 comp rx=0 tx=0).stop 2024-02-16T19:30:48.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.770+0000 7f0ce096f700 1 -- 172.21.15.22:0/1775539425 shutdown_connections 2024-02-16T19:30:48.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.770+0000 7f0ce096f700 1 --2- 172.21.15.22:0/1775539425 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f0cc406d660 0x7f0cc406fb10 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:48.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.770+0000 7f0ce096f700 1 --2- 172.21.15.22:0/1775539425 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0cdc0ff150 0x7f0cdc069990 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:48.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.770+0000 7f0ce096f700 1 --2- 172.21.15.22:0/1775539425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cdc1003f0 0x7f0cdc19aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:48.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.770+0000 7f0ce096f700 1 -- 172.21.15.22:0/1775539425 >> 172.21.15.22:0/1775539425 conn(0x7f0cdc0fa7a0 msgr2=0x7f0cdc1036c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:30:48.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.770+0000 7f0ce096f700 1 -- 172.21.15.22:0/1775539425 shutdown_connections 2024-02-16T19:30:48.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.770+0000 7f0ce096f700 1 -- 172.21.15.22:0/1775539425 wait complete. 2024-02-16T19:30:48.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.920+0000 7fe46bafb700 1 -- 172.21.15.22:0/230759921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe464100250 msgr2=0x7fe464100660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:48.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.920+0000 7fe46bafb700 1 --2- 172.21.15.22:0/230759921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe464100250 0x7fe464100660 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe460009870 tx=0x7fe460009b80 comp rx=0 tx=0).stop 2024-02-16T19:30:48.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.920+0000 7fe46bafb700 1 -- 172.21.15.22:0/230759921 shutdown_connections 2024-02-16T19:30:48.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.920+0000 7fe46bafb700 1 --2- 172.21.15.22:0/230759921 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe464101450 0x7fe4641018a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:48.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.920+0000 7fe46bafb700 1 --2- 172.21.15.22:0/230759921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe464100250 0x7fe464100660 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:48.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.920+0000 7fe46bafb700 1 -- 172.21.15.22:0/230759921 >> 172.21.15.22:0/230759921 conn(0x7fe4640fb7e0 msgr2=0x7fe4640fdc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:30:48.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.921+0000 7fe46bafb700 1 -- 172.21.15.22:0/230759921 shutdown_connections 2024-02-16T19:30:48.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.921+0000 7fe46bafb700 1 -- 172.21.15.22:0/230759921 wait complete. 2024-02-16T19:30:48.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.921+0000 7fe46bafb700 1 Processor -- start 2024-02-16T19:30:48.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.922+0000 7fe46bafb700 1 -- start start 2024-02-16T19:30:48.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.922+0000 7fe46bafb700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe464100250 0x7fe464195ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:30:48.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.922+0000 7fe46bafb700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe464101450 0x7fe464196000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:30:48.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.922+0000 7fe46bafb700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe464196620 con 0x7fe464101450 2024-02-16T19:30:48.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.922+0000 7fe46bafb700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe464196760 con 0x7fe464100250 2024-02-16T19:30:48.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.922+0000 7fe469096700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe464101450 0x7fe464196000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:30:48.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.922+0000 7fe469096700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe464101450 0x7fe464196000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57300/0 (socket says 172.21.15.22:57300) 2024-02-16T19:30:48.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.922+0000 7fe469897700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe464100250 0x7fe464195ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:30:48.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.922+0000 7fe469096700 1 -- 172.21.15.22:0/3330931695 learned_addr learned my addr 172.21.15.22:0/3330931695 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:30:48.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.922+0000 7fe469897700 1 -- 172.21.15.22:0/3330931695 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe464101450 msgr2=0x7fe464196000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:48.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.922+0000 7fe469897700 1 --2- 172.21.15.22:0/3330931695 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe464101450 0x7fe464196000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:48.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.922+0000 7fe469897700 1 -- 172.21.15.22:0/3330931695 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe460009520 con 0x7fe464100250 2024-02-16T19:30:48.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.922+0000 7fe469096700 1 --2- 172.21.15.22:0/3330931695 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe464101450 0x7fe464196000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T19:30:48.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.923+0000 7fe469897700 1 --2- 172.21.15.22:0/3330931695 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe464100250 0x7fe464195ac0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe460000c00 tx=0x7fe46000f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:30:48.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.923+0000 7fe45affd700 1 -- 172.21.15.22:0/3330931695 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe46001c070 con 0x7fe464100250 2024-02-16T19:30:48.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.923+0000 7fe45affd700 1 -- 172.21.15.22:0/3330931695 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe46000fd60 con 0x7fe464100250 2024-02-16T19:30:48.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.923+0000 7fe45affd700 1 -- 172.21.15.22:0/3330931695 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe460022da0 con 0x7fe464100250 2024-02-16T19:30:48.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.923+0000 7fe46bafb700 1 -- 172.21.15.22:0/3330931695 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe46419b1b0 con 0x7fe464100250 2024-02-16T19:30:48.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.923+0000 7fe46bafb700 1 -- 172.21.15.22:0/3330931695 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe46419b6a0 con 0x7fe464100250 2024-02-16T19:30:48.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.924+0000 7fe46bafb700 1 -- 172.21.15.22:0/3330931695 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe464108da0 con 0x7fe464100250 2024-02-16T19:30:48.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.925+0000 7fe45affd700 1 -- 172.21.15.22:0/3330931695 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 31) v1 ==== 91448+0+0 (secure 0 0 0) 0x7fe460020070 con 0x7fe464100250 2024-02-16T19:30:48.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.926+0000 7fe45affd700 1 --2- 172.21.15.22:0/3330931695 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7fe45006d6b0 0x7fe45006fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:30:48.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.926+0000 7fe45affd700 1 -- 172.21.15.22:0/3330931695 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fe46008e3a0 con 0x7fe464100250 2024-02-16T19:30:48.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.926+0000 7fe469096700 1 --2- 172.21.15.22:0/3330931695 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7fe45006d6b0 0x7fe45006fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:30:48.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.927+0000 7fe469096700 1 --2- 172.21.15.22:0/3330931695 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7fe45006d6b0 0x7fe45006fb60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe454005950 tx=0x7fe454009310 comp rx=0 tx=0).ready entity=mgr.24407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:30:48.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:48.931+0000 7fe45affd700 1 -- 172.21.15.22:0/3330931695 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7fe46005ba90 con 0x7fe464100250 2024-02-16T19:30:49.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:49.269+0000 7fe46bafb700 1 -- 172.21.15.22:0/3330931695 --> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe464105da0 con 0x7fe45006d6b0 2024-02-16T19:30:49.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:49.270+0000 7fe45affd700 1 -- 172.21.15.22:0/3330931695 <== mgr.24407 v2:172.21.15.22:6800/3703806543 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7fe464105da0 con 0x7fe45006d6b0 2024-02-16T19:30:49.272 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-02-16T19:30:49.272 INFO:teuthology.orchestra.run.smithi022.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T19:30:49.272 INFO:teuthology.orchestra.run.smithi022.stdout: "in_progress": true, 2024-02-16T19:30:49.272 INFO:teuthology.orchestra.run.smithi022.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-02-16T19:30:49.272 INFO:teuthology.orchestra.run.smithi022.stdout: "services_complete": [ 2024-02-16T19:30:49.272 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr" 2024-02-16T19:30:49.272 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-02-16T19:30:49.272 INFO:teuthology.orchestra.run.smithi022.stdout: "progress": "2/2 daemons upgraded", 2024-02-16T19:30:49.272 INFO:teuthology.orchestra.run.smithi022.stdout: "message": "Currently upgrading mgr daemons", 2024-02-16T19:30:49.272 INFO:teuthology.orchestra.run.smithi022.stdout: "is_paused": false 2024-02-16T19:30:49.272 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-02-16T19:30:49.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:49.273+0000 7fe46bafb700 1 -- 172.21.15.22:0/3330931695 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7fe45006d6b0 msgr2=0x7fe45006fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:49.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:49.273+0000 7fe46bafb700 1 --2- 172.21.15.22:0/3330931695 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7fe45006d6b0 0x7fe45006fb60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe454005950 tx=0x7fe454009310 comp rx=0 tx=0).stop 2024-02-16T19:30:49.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:49.273+0000 7fe46bafb700 1 -- 172.21.15.22:0/3330931695 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe464100250 msgr2=0x7fe464195ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:49.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:49.273+0000 7fe46bafb700 1 --2- 172.21.15.22:0/3330931695 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe464100250 0x7fe464195ac0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe460000c00 tx=0x7fe46000f690 comp rx=0 tx=0).stop 2024-02-16T19:30:49.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:49.273+0000 7fe46bafb700 1 -- 172.21.15.22:0/3330931695 shutdown_connections 2024-02-16T19:30:49.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:49.273+0000 7fe46bafb700 1 --2- 172.21.15.22:0/3330931695 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7fe45006d6b0 0x7fe45006fb60 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:49.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:49.273+0000 7fe46bafb700 1 --2- 172.21.15.22:0/3330931695 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe464100250 0x7fe464195ac0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:49.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:49.273+0000 7fe46bafb700 1 --2- 172.21.15.22:0/3330931695 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe464101450 0x7fe464196000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:49.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:49.273+0000 7fe46bafb700 1 -- 172.21.15.22:0/3330931695 >> 172.21.15.22:0/3330931695 conn(0x7fe4640fb7e0 msgr2=0x7fe464104680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:30:49.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:49.273+0000 7fe46bafb700 1 -- 172.21.15.22:0/3330931695 shutdown_connections 2024-02-16T19:30:49.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:30:49.273+0000 7fe46bafb700 1 -- 172.21.15.22:0/3330931695 wait complete. 2024-02-16T19:30:49.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:49 smithi022 ceph-mon[29221]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:30:49.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:49 smithi022 ceph-mon[29221]: from='client.14728 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:30:49.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:49 smithi022 ceph-mon[29221]: from='client.? 172.21.15.22:0/1775539425' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:30:49.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:49 smithi022 ceph-mon[29221]: Standby manager daemon smithi117.cohbtv restarted 2024-02-16T19:30:49.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:49 smithi022 ceph-mon[29221]: Standby manager daemon smithi117.cohbtv started 2024-02-16T19:30:49.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:49 smithi022 ceph-mon[29221]: from='mgr.? 172.21.15.117:0/649321620' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi117.cohbtv/crt"}]: dispatch 2024-02-16T19:30:49.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:49 smithi022 ceph-mon[29221]: from='mgr.? 172.21.15.117:0/649321620' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-16T19:30:49.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:49 smithi022 ceph-mon[29221]: from='mgr.? 172.21.15.117:0/649321620' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi117.cohbtv/key"}]: dispatch 2024-02-16T19:30:49.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:49 smithi022 ceph-mon[29221]: from='mgr.? 172.21.15.117:0/649321620' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-16T19:30:49.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:49 smithi117 ceph-mon[35309]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:30:49.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:49 smithi117 ceph-mon[35309]: from='client.14728 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:30:49.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:49 smithi117 ceph-mon[35309]: from='client.? 172.21.15.22:0/1775539425' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:30:49.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:49 smithi117 ceph-mon[35309]: Standby manager daemon smithi117.cohbtv restarted 2024-02-16T19:30:49.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:49 smithi117 ceph-mon[35309]: Standby manager daemon smithi117.cohbtv started 2024-02-16T19:30:49.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:49 smithi117 ceph-mon[35309]: from='mgr.? 172.21.15.117:0/649321620' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi117.cohbtv/crt"}]: dispatch 2024-02-16T19:30:49.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:49 smithi117 ceph-mon[35309]: from='mgr.? 172.21.15.117:0/649321620' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-16T19:30:49.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:49 smithi117 ceph-mon[35309]: from='mgr.? 172.21.15.117:0/649321620' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi117.cohbtv/key"}]: dispatch 2024-02-16T19:30:49.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:49 smithi117 ceph-mon[35309]: from='mgr.? 172.21.15.117:0/649321620' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-16T19:30:50.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:50 smithi117 ceph-mon[35309]: pgmap v31: 65 pgs: 65 active+clean; 293 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 29 KiB/s rd, 1.4 MiB/s wr, 164 op/s 2024-02-16T19:30:50.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:50 smithi117 ceph-mon[35309]: from='client.24493 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:30:50.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:50 smithi117 ceph-mon[35309]: mgrmap e32: smithi022.gmdcch(active, since 56s), standbys: smithi117.cohbtv 2024-02-16T19:30:50.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:50 smithi022 ceph-mon[29221]: pgmap v31: 65 pgs: 65 active+clean; 293 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 29 KiB/s rd, 1.4 MiB/s wr, 164 op/s 2024-02-16T19:30:50.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:50 smithi022 ceph-mon[29221]: from='client.24493 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:30:50.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:50 smithi022 ceph-mon[29221]: mgrmap e32: smithi022.gmdcch(active, since 56s), standbys: smithi117.cohbtv 2024-02-16T19:30:51.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:51 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:30:51.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:51 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:30:51.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:51 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:30:51.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:51 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:30:51.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:51 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:30:51.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:51 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:30:51.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:51 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:30:51.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:51 smithi117 ceph-mon[35309]: pgmap v32: 65 pgs: 65 active+clean; 287 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 42 KiB/s rd, 1.9 MiB/s wr, 227 op/s 2024-02-16T19:30:51.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:51 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:30:51.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:51 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:30:51.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:51 smithi117 ceph-mon[35309]: Upgrade: Setting container_image for all mgr 2024-02-16T19:30:51.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:51 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:30:51.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:51 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi022.gmdcch"}]: dispatch 2024-02-16T19:30:51.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:51 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi022.gmdcch"}]': finished 2024-02-16T19:30:51.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:51 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi117.cohbtv"}]: dispatch 2024-02-16T19:30:51.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:51 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi117.cohbtv"}]': finished 2024-02-16T19:30:51.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:51 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:30:51.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:51 smithi117 ceph-mon[35309]: Upgrade: Setting container_image for all rgw 2024-02-16T19:30:51.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:51 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:30:51.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:51 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:30:51.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:51 smithi117 ceph-mon[35309]: Upgrade: Setting container_image for all rbd-mirror 2024-02-16T19:30:51.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:51 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:30:51.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:51 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:30:51.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:51 smithi117 ceph-mon[35309]: Upgrade: Setting container_image for all cephfs-mirror 2024-02-16T19:30:51.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:51 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:30:51.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:51 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:30:51.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:51 smithi117 ceph-mon[35309]: Upgrade: Setting container_image for all iscsi 2024-02-16T19:30:51.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:51 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:30:51.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:51 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:30:51.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:51 smithi117 ceph-mon[35309]: Upgrade: Setting container_image for all nfs 2024-02-16T19:30:51.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:51 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:30:51.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:51 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:30:51.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:51 smithi117 ceph-mon[35309]: Upgrade: Setting container_image for all nvmeof 2024-02-16T19:30:51.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:51 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:30:51.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:51 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:30:51.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:51 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:30:51.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:51 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:30:51.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:51 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:30:51.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:51 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:30:51.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:51 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:30:51.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:51 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:30:51.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:51 smithi022 ceph-mon[29221]: pgmap v32: 65 pgs: 65 active+clean; 287 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 42 KiB/s rd, 1.9 MiB/s wr, 227 op/s 2024-02-16T19:30:51.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:51 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:30:51.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:51 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:30:51.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:51 smithi022 ceph-mon[29221]: Upgrade: Setting container_image for all mgr 2024-02-16T19:30:51.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:51 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:30:51.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:51 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi022.gmdcch"}]: dispatch 2024-02-16T19:30:51.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:51 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi022.gmdcch"}]': finished 2024-02-16T19:30:51.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:51 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi117.cohbtv"}]: dispatch 2024-02-16T19:30:51.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:51 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi117.cohbtv"}]': finished 2024-02-16T19:30:51.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:51 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:30:51.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:51 smithi022 ceph-mon[29221]: Upgrade: Setting container_image for all rgw 2024-02-16T19:30:51.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:51 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:30:51.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:51 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:30:51.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:51 smithi022 ceph-mon[29221]: Upgrade: Setting container_image for all rbd-mirror 2024-02-16T19:30:51.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:51 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:30:51.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:51 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:30:51.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:51 smithi022 ceph-mon[29221]: Upgrade: Setting container_image for all cephfs-mirror 2024-02-16T19:30:51.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:51 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:30:51.967 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:51 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:30:51.967 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:51 smithi022 ceph-mon[29221]: Upgrade: Setting container_image for all iscsi 2024-02-16T19:30:51.967 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:51 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:30:51.967 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:51 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:30:51.967 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:51 smithi022 ceph-mon[29221]: Upgrade: Setting container_image for all nfs 2024-02-16T19:30:51.967 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:51 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:30:51.967 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:51 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:30:51.967 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:51 smithi022 ceph-mon[29221]: Upgrade: Setting container_image for all nvmeof 2024-02-16T19:30:51.967 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:51 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:30:54.546 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:54 smithi117 ceph-mon[35309]: pgmap v33: 65 pgs: 65 active+clean; 287 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 28 KiB/s rd, 1.2 MiB/s wr, 140 op/s 2024-02-16T19:30:54.546 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:54 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:30:54.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:54 smithi022 ceph-mon[29221]: pgmap v33: 65 pgs: 65 active+clean; 287 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 28 KiB/s rd, 1.2 MiB/s wr, 140 op/s 2024-02-16T19:30:54.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:54 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:30:56.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:56 smithi117 ceph-mon[35309]: pgmap v34: 65 pgs: 65 active+clean; 277 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 40 KiB/s rd, 1.7 MiB/s wr, 237 op/s 2024-02-16T19:30:56.714 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:56 smithi022 ceph-mon[29221]: pgmap v34: 65 pgs: 65 active+clean; 277 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 40 KiB/s rd, 1.7 MiB/s wr, 237 op/s 2024-02-16T19:30:57.591 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:57 smithi022 ceph-mon[29221]: Upgrade: Updating node-exporter.smithi022 (1/2) 2024-02-16T19:30:57.591 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:57 smithi022 ceph-mon[29221]: Deploying daemon node-exporter.smithi022 on smithi022 2024-02-16T19:30:57.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:57 smithi117 ceph-mon[35309]: Upgrade: Updating node-exporter.smithi022 (1/2) 2024-02-16T19:30:57.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:57 smithi117 ceph-mon[35309]: Deploying daemon node-exporter.smithi022 on smithi022 2024-02-16T19:30:58.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:30:58 smithi022 ceph-mon[29221]: pgmap v35: 65 pgs: 65 active+clean; 277 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 25 KiB/s rd, 1.0 MiB/s wr, 169 op/s 2024-02-16T19:30:58.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:30:58 smithi117 ceph-mon[35309]: pgmap v35: 65 pgs: 65 active+clean; 277 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 25 KiB/s rd, 1.0 MiB/s wr, 169 op/s 2024-02-16T19:31:00.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:00 smithi117 ceph-mon[35309]: pgmap v36: 65 pgs: 65 active+clean; 276 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 25 KiB/s rd, 1.0 MiB/s wr, 174 op/s 2024-02-16T19:31:00.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:00 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:31:00.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:00 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:31:00.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:00 smithi022 ceph-mon[29221]: pgmap v36: 65 pgs: 65 active+clean; 276 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 25 KiB/s rd, 1.0 MiB/s wr, 174 op/s 2024-02-16T19:31:00.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:00 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:31:00.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:00 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:31:01.200 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.0... 2024-02-16T19:31:01.201 DEBUG:teuthology.orchestra.run.smithi022:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-02-16T19:31:01.537 DEBUG:teuthology.parallel:result is None 2024-02-16T19:31:01.728 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.1... 2024-02-16T19:31:01.728 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.1 /home/ubuntu/cephtest/clone.client.1 2024-02-16T19:31:02.071 DEBUG:teuthology.parallel:result is None 2024-02-16T19:31:02.072 DEBUG:teuthology.orchestra.run.smithi022:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-02-16T19:31:02.107 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-02-16T19:31:02.108 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1 2024-02-16T19:31:02.146 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.1/client.1 2024-02-16T19:31:02.146 DEBUG:teuthology.parallel:result is None 2024-02-16T19:31:02.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:02 smithi117 ceph-mon[35309]: Upgrade: Updating node-exporter.smithi117 (2/2) 2024-02-16T19:31:02.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:02 smithi117 ceph-mon[35309]: Deploying daemon node-exporter.smithi117 on smithi117 2024-02-16T19:31:02.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:02 smithi117 ceph-mon[35309]: pgmap v37: 65 pgs: 65 active+clean; 273 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 37 KiB/s rd, 1.6 MiB/s wr, 257 op/s 2024-02-16T19:31:02.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:02 smithi022 ceph-mon[29221]: Upgrade: Updating node-exporter.smithi117 (2/2) 2024-02-16T19:31:02.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:02 smithi022 ceph-mon[29221]: Deploying daemon node-exporter.smithi117 on smithi117 2024-02-16T19:31:02.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:02 smithi022 ceph-mon[29221]: pgmap v37: 65 pgs: 65 active+clean; 273 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 37 KiB/s rd, 1.6 MiB/s wr, 257 op/s 2024-02-16T19:31:04.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:04 smithi117 ceph-mon[35309]: pgmap v38: 65 pgs: 65 active+clean; 273 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 23 KiB/s rd, 1.0 MiB/s wr, 188 op/s 2024-02-16T19:31:04.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:04 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:31:04.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:04 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:31:04.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:04 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:31:04.596 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:04 smithi022 ceph-mon[29221]: pgmap v38: 65 pgs: 65 active+clean; 273 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 23 KiB/s rd, 1.0 MiB/s wr, 188 op/s 2024-02-16T19:31:04.596 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:04 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:31:04.596 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:04 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:31:04.597 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:04 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:31:06.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:06 smithi117 ceph-mon[35309]: pgmap v39: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 38 KiB/s rd, 1.4 MiB/s wr, 260 op/s 2024-02-16T19:31:06.716 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:06 smithi022 ceph-mon[29221]: pgmap v39: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 38 KiB/s rd, 1.4 MiB/s wr, 260 op/s 2024-02-16T19:31:07.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:07 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:31:07.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:07 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:31:07.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:07 smithi022 ceph-mon[29221]: pgmap v40: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 26 KiB/s rd, 941 KiB/s wr, 163 op/s 2024-02-16T19:31:08.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:07 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:31:08.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:07 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:31:08.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:07 smithi117 ceph-mon[35309]: pgmap v40: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 26 KiB/s rd, 941 KiB/s wr, 163 op/s 2024-02-16T19:31:09.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:08 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:31:09.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:08 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:31:09.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:08 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:31:09.464 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:08 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:31:09.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:08 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:31:09.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:08 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:31:10.532 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:10 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:31:10.533 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:10 smithi117 ceph-mon[35309]: pgmap v41: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 26 KiB/s rd, 941 KiB/s wr, 163 op/s 2024-02-16T19:31:10.533 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:10 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:31:10.714 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:10 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:31:10.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:10 smithi022 ceph-mon[29221]: pgmap v41: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 26 KiB/s rd, 941 KiB/s wr, 163 op/s 2024-02-16T19:31:10.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:10 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:31:11.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:11 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:31:11.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:11 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:31:11.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:11 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:31:11.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:11 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:31:11.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:11 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:31:11.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:11 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:31:11.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:11 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:31:11.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:11 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:31:12.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:12 smithi117 ceph-mon[35309]: pgmap v42: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 26 KiB/s rd, 941 KiB/s wr, 158 op/s 2024-02-16T19:31:12.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:12 smithi022 ceph-mon[29221]: pgmap v42: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 26 KiB/s rd, 941 KiB/s wr, 158 op/s 2024-02-16T19:31:14.621 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:14 smithi022 ceph-mon[29221]: pgmap v43: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 14 KiB/s rd, 394 KiB/s wr, 72 op/s 2024-02-16T19:31:14.621 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:14 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:31:14.621 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:14 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:31:14.621 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:14 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:31:14.621 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:14 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:31:14.621 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:14 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:31:14.622 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:14 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:31:14.622 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:14 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:14.622 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:14 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:14.622 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:14 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:14.622 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:14 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:14.622 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:14 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:14.622 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:14 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:14.622 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:14 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:14.622 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:14 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:14.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:14 smithi117 ceph-mon[35309]: pgmap v43: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 14 KiB/s rd, 394 KiB/s wr, 72 op/s 2024-02-16T19:31:14.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:14 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:31:14.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:14 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:31:14.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:14 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:31:14.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:14 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:31:14.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:14 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:31:14.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:14 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:31:14.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:14 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:14.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:14 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:14.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:14 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:14.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:14 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:14.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:14 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:14.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:14 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:14.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:14 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:14.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:14 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:16.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:16 smithi117 ceph-mon[35309]: pgmap v44: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 14 KiB/s rd, 394 KiB/s wr, 72 op/s 2024-02-16T19:31:16.714 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:16 smithi022 ceph-mon[29221]: pgmap v44: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 14 KiB/s rd, 394 KiB/s wr, 72 op/s 2024-02-16T19:31:17.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:17 smithi117 ceph-mon[35309]: Upgrade: Updating prometheus.smithi022 2024-02-16T19:31:17.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:17 smithi117 ceph-mon[35309]: Deploying daemon prometheus.smithi022 on smithi022 2024-02-16T19:31:17.694 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:17 smithi022 ceph-mon[29221]: Upgrade: Updating prometheus.smithi022 2024-02-16T19:31:17.694 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:17 smithi022 ceph-mon[29221]: Deploying daemon prometheus.smithi022 on smithi022 2024-02-16T19:31:18.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:18 smithi117 ceph-mon[35309]: pgmap v45: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 85 B/s rd, 426 B/s wr, 0 op/s 2024-02-16T19:31:18.714 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:18 smithi022 ceph-mon[29221]: pgmap v45: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 85 B/s rd, 426 B/s wr, 0 op/s 2024-02-16T19:31:19.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.407+0000 7fecd4623700 1 -- 172.21.15.22:0/1636603521 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feccc0ff430 msgr2=0x7feccc0ff8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:19.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.407+0000 7fecd4623700 1 --2- 172.21.15.22:0/1636603521 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feccc0ff430 0x7feccc0ff8a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fecbc009910 tx=0x7fecbc009c20 comp rx=0 tx=0).stop 2024-02-16T19:31:19.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.408+0000 7fecd4623700 1 -- 172.21.15.22:0/1636603521 shutdown_connections 2024-02-16T19:31:19.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.408+0000 7fecd4623700 1 --2- 172.21.15.22:0/1636603521 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feccc0ff430 0x7feccc0ff8a0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:19.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.408+0000 7fecd4623700 1 --2- 172.21.15.22:0/1636603521 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feccc100730 0x7feccc100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:19.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.408+0000 7fecd4623700 1 -- 172.21.15.22:0/1636603521 >> 172.21.15.22:0/1636603521 conn(0x7feccc0f9f10 msgr2=0x7feccc0fc340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:31:19.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.408+0000 7fecd4623700 1 -- 172.21.15.22:0/1636603521 shutdown_connections 2024-02-16T19:31:19.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.408+0000 7fecd4623700 1 -- 172.21.15.22:0/1636603521 wait complete. 2024-02-16T19:31:19.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.409+0000 7fecd4623700 1 Processor -- start 2024-02-16T19:31:19.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.409+0000 7fecd4623700 1 -- start start 2024-02-16T19:31:19.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.409+0000 7fecd4623700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feccc0ff430 0x7feccc0fee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:31:19.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.409+0000 7fecd4623700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feccc100730 0x7feccc19a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:31:19.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.409+0000 7fecd4623700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feccc19ae70 con 0x7feccc100730 2024-02-16T19:31:19.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.409+0000 7fecd4623700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feccc19afb0 con 0x7feccc0ff430 2024-02-16T19:31:19.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.409+0000 7fecd1bbe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feccc100730 0x7feccc19a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:31:19.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.409+0000 7fecd1bbe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feccc100730 0x7feccc19a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51832/0 (socket says 172.21.15.22:51832) 2024-02-16T19:31:19.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.409+0000 7fecd1bbe700 1 -- 172.21.15.22:0/4064818078 learned_addr learned my addr 172.21.15.22:0/4064818078 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:31:19.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.409+0000 7fecd23bf700 1 --2- 172.21.15.22:0/4064818078 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feccc0ff430 0x7feccc0fee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:31:19.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.409+0000 7fecd1bbe700 1 -- 172.21.15.22:0/4064818078 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feccc0ff430 msgr2=0x7feccc0fee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:19.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.409+0000 7fecd1bbe700 1 --2- 172.21.15.22:0/4064818078 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feccc0ff430 0x7feccc0fee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:19.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.409+0000 7fecd1bbe700 1 -- 172.21.15.22:0/4064818078 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecbc0095f0 con 0x7feccc100730 2024-02-16T19:31:19.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.410+0000 7fecd1bbe700 1 --2- 172.21.15.22:0/4064818078 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feccc100730 0x7feccc19a8a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fecbc006010 tx=0x7fecbc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:31:19.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.410+0000 7fecc37fe700 1 -- 172.21.15.22:0/4064818078 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fecbc01d070 con 0x7feccc100730 2024-02-16T19:31:19.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.410+0000 7fecd4623700 1 -- 172.21.15.22:0/4064818078 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feccc19b230 con 0x7feccc100730 2024-02-16T19:31:19.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.410+0000 7fecc37fe700 1 -- 172.21.15.22:0/4064818078 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fecbc00f460 con 0x7feccc100730 2024-02-16T19:31:19.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.410+0000 7fecc37fe700 1 -- 172.21.15.22:0/4064818078 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fecbc021640 con 0x7feccc100730 2024-02-16T19:31:19.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.410+0000 7fecd4623700 1 -- 172.21.15.22:0/4064818078 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feccc19b660 con 0x7feccc100730 2024-02-16T19:31:19.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.411+0000 7fecd4623700 1 -- 172.21.15.22:0/4064818078 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feccc107c70 con 0x7feccc100730 2024-02-16T19:31:19.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.412+0000 7fecc37fe700 1 -- 172.21.15.22:0/4064818078 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 32) v1 ==== 91448+0+0 (secure 0 0 0) 0x7fecbc004b20 con 0x7feccc100730 2024-02-16T19:31:19.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.412+0000 7fecc37fe700 1 --2- 172.21.15.22:0/4064818078 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7fecb806d660 0x7fecb806fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:31:19.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.412+0000 7fecc37fe700 1 -- 172.21.15.22:0/4064818078 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fecbc08e5c0 con 0x7feccc100730 2024-02-16T19:31:19.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.412+0000 7fecd23bf700 1 --2- 172.21.15.22:0/4064818078 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7fecb806d660 0x7fecb806fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:31:19.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.413+0000 7fecd23bf700 1 --2- 172.21.15.22:0/4064818078 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7fecb806d660 0x7fecb806fb10 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fecc8006fd0 tx=0x7fecc8008040 comp rx=0 tx=0).ready entity=mgr.24407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:31:19.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.417+0000 7fecc37fe700 1 -- 172.21.15.22:0/4064818078 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7fecbc05bc30 con 0x7feccc100730 2024-02-16T19:31:19.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.649+0000 7fecd4623700 1 -- 172.21.15.22:0/4064818078 --> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7feccc104d90 con 0x7fecb806d660 2024-02-16T19:31:19.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.651+0000 7fecc37fe700 1 -- 172.21.15.22:0/4064818078 <== mgr.24407 v2:172.21.15.22:6800/3703806543 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+367 (secure 0 0 0) 0x7feccc104d90 con 0x7fecb806d660 2024-02-16T19:31:19.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.653+0000 7fecd4623700 1 -- 172.21.15.22:0/4064818078 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7fecb806d660 msgr2=0x7fecb806fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:19.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.653+0000 7fecd4623700 1 --2- 172.21.15.22:0/4064818078 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7fecb806d660 0x7fecb806fb10 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fecc8006fd0 tx=0x7fecc8008040 comp rx=0 tx=0).stop 2024-02-16T19:31:19.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.653+0000 7fecd4623700 1 -- 172.21.15.22:0/4064818078 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feccc100730 msgr2=0x7feccc19a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:19.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.653+0000 7fecd4623700 1 --2- 172.21.15.22:0/4064818078 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feccc100730 0x7feccc19a8a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fecbc006010 tx=0x7fecbc00be20 comp rx=0 tx=0).stop 2024-02-16T19:31:19.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.653+0000 7fecd4623700 1 -- 172.21.15.22:0/4064818078 shutdown_connections 2024-02-16T19:31:19.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.653+0000 7fecd4623700 1 --2- 172.21.15.22:0/4064818078 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7fecb806d660 0x7fecb806fb10 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:19.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.653+0000 7fecd4623700 1 --2- 172.21.15.22:0/4064818078 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feccc0ff430 0x7feccc0fee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:19.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.653+0000 7fecd4623700 1 --2- 172.21.15.22:0/4064818078 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feccc100730 0x7feccc19a8a0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:19.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.653+0000 7fecd4623700 1 -- 172.21.15.22:0/4064818078 >> 172.21.15.22:0/4064818078 conn(0x7feccc0f9f10 msgr2=0x7feccc103670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:31:19.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.653+0000 7fecd4623700 1 -- 172.21.15.22:0/4064818078 shutdown_connections 2024-02-16T19:31:19.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.653+0000 7fecd4623700 1 -- 172.21.15.22:0/4064818078 wait complete. 2024-02-16T19:31:19.666 INFO:teuthology.orchestra.run.smithi022.stdout:true 2024-02-16T19:31:19.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.765+0000 7f3454099700 1 -- 172.21.15.22:0/420741514 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f344c100d80 msgr2=0x7f344c103160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:19.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.765+0000 7f3454099700 1 --2- 172.21.15.22:0/420741514 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f344c100d80 0x7f344c103160 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f344800b0a0 tx=0x7f344800b3b0 comp rx=0 tx=0).stop 2024-02-16T19:31:19.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.765+0000 7f3454099700 1 -- 172.21.15.22:0/420741514 shutdown_connections 2024-02-16T19:31:19.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.765+0000 7f3454099700 1 --2- 172.21.15.22:0/420741514 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f344c1036a0 0x7f344c105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:19.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.765+0000 7f3454099700 1 --2- 172.21.15.22:0/420741514 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f344c100d80 0x7f344c103160 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:19.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.765+0000 7f3454099700 1 -- 172.21.15.22:0/420741514 >> 172.21.15.22:0/420741514 conn(0x7f344c0fa7a0 msgr2=0x7f344c0fcbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:31:19.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.766+0000 7f3454099700 1 -- 172.21.15.22:0/420741514 shutdown_connections 2024-02-16T19:31:19.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.766+0000 7f3454099700 1 -- 172.21.15.22:0/420741514 wait complete. 2024-02-16T19:31:19.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.766+0000 7f3454099700 1 Processor -- start 2024-02-16T19:31:19.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.766+0000 7f3454099700 1 -- start start 2024-02-16T19:31:19.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.766+0000 7f3454099700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f344c100d80 0x7f344c1006e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:31:19.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.766+0000 7f3454099700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f344c1036a0 0x7f344c0fed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:31:19.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.766+0000 7f3454099700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f344c0ff270 con 0x7f344c1036a0 2024-02-16T19:31:19.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.766+0000 7f3454099700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f344c0ff3b0 con 0x7f344c100d80 2024-02-16T19:31:19.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.767+0000 7f3451634700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f344c1036a0 0x7f344c0fed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:31:19.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.767+0000 7f3451634700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f344c1036a0 0x7f344c0fed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51850/0 (socket says 172.21.15.22:51850) 2024-02-16T19:31:19.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.767+0000 7f3451634700 1 -- 172.21.15.22:0/2008192579 learned_addr learned my addr 172.21.15.22:0/2008192579 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:31:19.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.767+0000 7f3451e35700 1 --2- 172.21.15.22:0/2008192579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f344c100d80 0x7f344c1006e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:31:19.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.767+0000 7f3451634700 1 -- 172.21.15.22:0/2008192579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f344c100d80 msgr2=0x7f344c1006e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:19.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.767+0000 7f3451634700 1 --2- 172.21.15.22:0/2008192579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f344c100d80 0x7f344c1006e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:19.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.767+0000 7f3451634700 1 -- 172.21.15.22:0/2008192579 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3448009d00 con 0x7f344c1036a0 2024-02-16T19:31:19.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.767+0000 7f3451634700 1 --2- 172.21.15.22:0/2008192579 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f344c1036a0 0x7f344c0fed30 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f343c00c930 tx=0x7f343c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:31:19.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.767+0000 7f3442ffd700 1 -- 172.21.15.22:0/2008192579 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f343c007ad0 con 0x7f344c1036a0 2024-02-16T19:31:19.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.767+0000 7f3454099700 1 -- 172.21.15.22:0/2008192579 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f344c0ff690 con 0x7f344c1036a0 2024-02-16T19:31:19.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.767+0000 7f3442ffd700 1 -- 172.21.15.22:0/2008192579 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f343c00ce80 con 0x7f344c1036a0 2024-02-16T19:31:19.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.767+0000 7f3442ffd700 1 -- 172.21.15.22:0/2008192579 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f343c018630 con 0x7f344c1036a0 2024-02-16T19:31:19.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.767+0000 7f3454099700 1 -- 172.21.15.22:0/2008192579 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f344c19de50 con 0x7f344c1036a0 2024-02-16T19:31:19.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.768+0000 7f3454099700 1 -- 172.21.15.22:0/2008192579 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f344c18fbe0 con 0x7f344c1036a0 2024-02-16T19:31:19.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.769+0000 7f3442ffd700 1 -- 172.21.15.22:0/2008192579 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 32) v1 ==== 91448+0+0 (secure 0 0 0) 0x7f343c007c30 con 0x7f344c1036a0 2024-02-16T19:31:19.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.769+0000 7f3442ffd700 1 --2- 172.21.15.22:0/2008192579 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f343806d660 0x7f343806fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:31:19.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.769+0000 7f3442ffd700 1 -- 172.21.15.22:0/2008192579 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f343c08d1d0 con 0x7f344c1036a0 2024-02-16T19:31:19.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.769+0000 7f3451e35700 1 --2- 172.21.15.22:0/2008192579 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f343806d660 0x7f343806fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:31:19.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.770+0000 7f3451e35700 1 --2- 172.21.15.22:0/2008192579 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f343806d660 0x7f343806fb10 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f3448009fd0 tx=0x7f34480060c0 comp rx=0 tx=0).ready entity=mgr.24407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:31:19.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:19.774+0000 7f3442ffd700 1 -- 172.21.15.22:0/2008192579 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f343c05a7c0 con 0x7f344c1036a0 2024-02-16T19:31:20.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.008+0000 7f3454099700 1 -- 172.21.15.22:0/2008192579 --> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f344c061190 con 0x7f343806d660 2024-02-16T19:31:20.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.009+0000 7f3442ffd700 1 -- 172.21.15.22:0/2008192579 <== mgr.24407 v2:172.21.15.22:6800/3703806543 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+367 (secure 0 0 0) 0x7f344c061190 con 0x7f343806d660 2024-02-16T19:31:20.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.011+0000 7f3454099700 1 -- 172.21.15.22:0/2008192579 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f343806d660 msgr2=0x7f343806fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:20.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.011+0000 7f3454099700 1 --2- 172.21.15.22:0/2008192579 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f343806d660 0x7f343806fb10 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f3448009fd0 tx=0x7f34480060c0 comp rx=0 tx=0).stop 2024-02-16T19:31:20.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.011+0000 7f3454099700 1 -- 172.21.15.22:0/2008192579 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f344c1036a0 msgr2=0x7f344c0fed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:20.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.011+0000 7f3454099700 1 --2- 172.21.15.22:0/2008192579 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f344c1036a0 0x7f344c0fed30 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f343c00c930 tx=0x7f343c00cc40 comp rx=0 tx=0).stop 2024-02-16T19:31:20.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.011+0000 7f3454099700 1 -- 172.21.15.22:0/2008192579 shutdown_connections 2024-02-16T19:31:20.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.011+0000 7f3454099700 1 --2- 172.21.15.22:0/2008192579 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f343806d660 0x7f343806fb10 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:20.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.011+0000 7f3454099700 1 --2- 172.21.15.22:0/2008192579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f344c100d80 0x7f344c1006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:20.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.011+0000 7f3454099700 1 --2- 172.21.15.22:0/2008192579 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f344c1036a0 0x7f344c0fed30 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:20.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.011+0000 7f3454099700 1 -- 172.21.15.22:0/2008192579 >> 172.21.15.22:0/2008192579 conn(0x7f344c0fa7a0 msgr2=0x7f344c0fcbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:31:20.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.011+0000 7f3454099700 1 -- 172.21.15.22:0/2008192579 shutdown_connections 2024-02-16T19:31:20.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.011+0000 7f3454099700 1 -- 172.21.15.22:0/2008192579 wait complete. 2024-02-16T19:31:20.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.118+0000 7fbc1ac15700 1 -- 172.21.15.22:0/1030728819 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbc140ff3b0 msgr2=0x7fbc140ff820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:20.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.118+0000 7fbc1ac15700 1 --2- 172.21.15.22:0/1030728819 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbc140ff3b0 0x7fbc140ff820 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fbc00009910 tx=0x7fbc00009c20 comp rx=0 tx=0).stop 2024-02-16T19:31:20.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.118+0000 7fbc1ac15700 1 -- 172.21.15.22:0/1030728819 shutdown_connections 2024-02-16T19:31:20.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.118+0000 7fbc1ac15700 1 --2- 172.21.15.22:0/1030728819 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbc140ff3b0 0x7fbc140ff820 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:20.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.118+0000 7fbc1ac15700 1 --2- 172.21.15.22:0/1030728819 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc141007e0 0x7fbc14100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:20.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.118+0000 7fbc1ac15700 1 -- 172.21.15.22:0/1030728819 >> 172.21.15.22:0/1030728819 conn(0x7fbc140f9e70 msgr2=0x7fbc140fc2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:31:20.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.118+0000 7fbc1ac15700 1 -- 172.21.15.22:0/1030728819 shutdown_connections 2024-02-16T19:31:20.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.118+0000 7fbc1ac15700 1 -- 172.21.15.22:0/1030728819 wait complete. 2024-02-16T19:31:20.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.119+0000 7fbc1ac15700 1 Processor -- start 2024-02-16T19:31:20.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.119+0000 7fbc1ac15700 1 -- start start 2024-02-16T19:31:20.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.119+0000 7fbc1ac15700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbc140ff3b0 0x7fbc14103150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:31:20.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.119+0000 7fbc1ac15700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc141007e0 0x7fbc141017a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:31:20.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.119+0000 7fbc1ac15700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc14101ce0 con 0x7fbc140ff3b0 2024-02-16T19:31:20.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.119+0000 7fbc1ac15700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc14101e20 con 0x7fbc141007e0 2024-02-16T19:31:20.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.119+0000 7fbc189b1700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbc140ff3b0 0x7fbc14103150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:31:20.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.119+0000 7fbc189b1700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbc140ff3b0 0x7fbc14103150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51854/0 (socket says 172.21.15.22:51854) 2024-02-16T19:31:20.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.119+0000 7fbc189b1700 1 -- 172.21.15.22:0/1809403484 learned_addr learned my addr 172.21.15.22:0/1809403484 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:31:20.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.119+0000 7fbc13fff700 1 --2- 172.21.15.22:0/1809403484 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc141007e0 0x7fbc141017a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:31:20.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.119+0000 7fbc189b1700 1 -- 172.21.15.22:0/1809403484 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc141007e0 msgr2=0x7fbc141017a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:20.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.119+0000 7fbc189b1700 1 --2- 172.21.15.22:0/1809403484 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc141007e0 0x7fbc141017a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:20.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.119+0000 7fbc189b1700 1 -- 172.21.15.22:0/1809403484 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc000095f0 con 0x7fbc140ff3b0 2024-02-16T19:31:20.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.119+0000 7fbc189b1700 1 --2- 172.21.15.22:0/1809403484 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbc140ff3b0 0x7fbc14103150 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fbc0800c930 tx=0x7fbc0800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:31:20.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.120+0000 7fbc11ffb700 1 -- 172.21.15.22:0/1809403484 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc08007ad0 con 0x7fbc140ff3b0 2024-02-16T19:31:20.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.120+0000 7fbc11ffb700 1 -- 172.21.15.22:0/1809403484 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fbc08007c30 con 0x7fbc140ff3b0 2024-02-16T19:31:20.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.120+0000 7fbc1ac15700 1 -- 172.21.15.22:0/1809403484 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc14102100 con 0x7fbc140ff3b0 2024-02-16T19:31:20.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.120+0000 7fbc11ffb700 1 -- 172.21.15.22:0/1809403484 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc08018630 con 0x7fbc140ff3b0 2024-02-16T19:31:20.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.120+0000 7fbc1ac15700 1 -- 172.21.15.22:0/1809403484 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc1419db60 con 0x7fbc140ff3b0 2024-02-16T19:31:20.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.121+0000 7fbc1ac15700 1 -- 172.21.15.22:0/1809403484 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc14107d50 con 0x7fbc140ff3b0 2024-02-16T19:31:20.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.121+0000 7fbc11ffb700 1 -- 172.21.15.22:0/1809403484 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 32) v1 ==== 91448+0+0 (secure 0 0 0) 0x7fbc0800f450 con 0x7fbc140ff3b0 2024-02-16T19:31:20.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.122+0000 7fbc11ffb700 1 --2- 172.21.15.22:0/1809403484 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7fbc0406d660 0x7fbc0406fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:31:20.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.122+0000 7fbc11ffb700 1 -- 172.21.15.22:0/1809403484 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fbc0808d490 con 0x7fbc140ff3b0 2024-02-16T19:31:20.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.122+0000 7fbc13fff700 1 --2- 172.21.15.22:0/1809403484 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7fbc0406d660 0x7fbc0406fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:31:20.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.122+0000 7fbc13fff700 1 --2- 172.21.15.22:0/1809403484 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7fbc0406d660 0x7fbc0406fb10 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fbc00005090 tx=0x7fbc0001a040 comp rx=0 tx=0).ready entity=mgr.24407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:31:20.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.126+0000 7fbc11ffb700 1 -- 172.21.15.22:0/1809403484 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7fbc0805a9e0 con 0x7fbc140ff3b0 2024-02-16T19:31:20.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.362+0000 7fbc1ac15700 1 -- 172.21.15.22:0/1809403484 --> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fbc140fe9b0 con 0x7fbc0406d660 2024-02-16T19:31:20.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.373+0000 7fbc11ffb700 1 -- 172.21.15.22:0/1809403484 <== mgr.24407 v2:172.21.15.22:6800/3703806543 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fbc140fe9b0 con 0x7fbc0406d660 2024-02-16T19:31:20.375 INFO:teuthology.orchestra.run.smithi022.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T19:31:20.375 INFO:teuthology.orchestra.run.smithi022.stdout:alertmanager.smithi022 smithi022 *:9093,9094 running (9m) 11s ago 10m 26.6M - 0.25.0 c8568f914cd2 4dac585d8372 2024-02-16T19:31:20.375 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-exporter.smithi022 smithi022 running (10m) 11s ago 10m 7830k - 18.2.0 dc2bc1663786 3f8bef4349ba 2024-02-16T19:31:20.375 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-exporter.smithi117 smithi117 running (9m) 13s ago 9m 7507k - 18.2.0 dc2bc1663786 59b65a7e309f 2024-02-16T19:31:20.375 INFO:teuthology.orchestra.run.smithi022.stdout:crash.smithi022 smithi022 running (10m) 11s ago 10m 6669k - 18.2.0 dc2bc1663786 efef476aa486 2024-02-16T19:31:20.375 INFO:teuthology.orchestra.run.smithi022.stdout:crash.smithi117 smithi117 running (9m) 13s ago 9m 6660k - 18.2.0 dc2bc1663786 5466453eb149 2024-02-16T19:31:20.375 INFO:teuthology.orchestra.run.smithi022.stdout:grafana.smithi022 smithi022 *:3000 running (8m) 11s ago 10m 84.7M - 9.4.7 954c08fa6188 a48872ecbbf2 2024-02-16T19:31:20.375 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi022.btppwm smithi022 running (4m) 11s ago 4m 183M - 18.2.0 dc2bc1663786 22971fb3a907 2024-02-16T19:31:20.375 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi022.ocukvs smithi022 running (4m) 11s ago 4m 14.3M - 18.2.0 dc2bc1663786 81ee7bd5fd39 2024-02-16T19:31:20.375 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi117.ibmppn smithi117 running (4m) 13s ago 4m 16.1M - 18.2.0 dc2bc1663786 8c606cbeeb86 2024-02-16T19:31:20.376 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi117.orzors smithi117 running (4m) 13s ago 4m 15.1M - 18.2.0 dc2bc1663786 7d2bc4df3feb 2024-02-16T19:31:20.376 INFO:teuthology.orchestra.run.smithi022.stdout:mgr.smithi022.gmdcch smithi022 *:8443,9283,8765 running (99s) 11s ago 12m 493M - 19.0.0-1450-gf78a58c0 f081f847070a 7cb74cc9deea 2024-02-16T19:31:20.376 INFO:teuthology.orchestra.run.smithi022.stdout:mgr.smithi117.cohbtv smithi117 *:8443,9283,8765 running (39s) 13s ago 9m 450M - 19.0.0-1450-gf78a58c0 f081f847070a c825b1152189 2024-02-16T19:31:20.376 INFO:teuthology.orchestra.run.smithi022.stdout:mon.smithi022 smithi022 running (12m) 11s ago 12m 54.9M 2048M 18.2.0 dc2bc1663786 b4ffc3032e12 2024-02-16T19:31:20.376 INFO:teuthology.orchestra.run.smithi022.stdout:mon.smithi117 smithi117 running (9m) 13s ago 9m 44.6M 2048M 18.2.0 dc2bc1663786 aff3e21a73a0 2024-02-16T19:31:20.376 INFO:teuthology.orchestra.run.smithi022.stdout:node-exporter.smithi022 smithi022 *:9100 running (21s) 11s ago 10m 8082k - 1.5.0 0da6a335fe13 1f5b58292d27 2024-02-16T19:31:20.376 INFO:teuthology.orchestra.run.smithi022.stdout:node-exporter.smithi117 smithi117 *:9100 running (17s) 13s ago 9m 3275k - 1.5.0 0da6a335fe13 bb5e8491d5a6 2024-02-16T19:31:20.376 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0 smithi022 running (8m) 11s ago 8m 211M 1217M 18.2.0 dc2bc1663786 0027f86f16ce 2024-02-16T19:31:20.376 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1 smithi022 running (8m) 11s ago 8m 226M 1217M 18.2.0 dc2bc1663786 87300382b8a7 2024-02-16T19:31:20.376 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2 smithi022 running (7m) 11s ago 7m 184M 1217M 18.2.0 dc2bc1663786 be8a631b4f93 2024-02-16T19:31:20.376 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3 smithi117 running (7m) 13s ago 7m 242M 2240M 18.2.0 dc2bc1663786 9c0ce9a981cf 2024-02-16T19:31:20.376 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4 smithi117 running (6m) 13s ago 6m 208M 2240M 18.2.0 dc2bc1663786 1779c3496f4d 2024-02-16T19:31:20.376 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5 smithi117 running (6m) 13s ago 6m 194M 2240M 18.2.0 dc2bc1663786 fefaabca0638 2024-02-16T19:31:20.377 INFO:teuthology.orchestra.run.smithi022.stdout:prometheus.smithi022 smithi022 *:9095 running (45s) 11s ago 10m 51.3M - 2.43.0 a07b618ecd1d b3f0b29c20d1 2024-02-16T19:31:20.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.376+0000 7fbc1ac15700 1 -- 172.21.15.22:0/1809403484 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7fbc0406d660 msgr2=0x7fbc0406fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:20.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.376+0000 7fbc1ac15700 1 --2- 172.21.15.22:0/1809403484 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7fbc0406d660 0x7fbc0406fb10 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fbc00005090 tx=0x7fbc0001a040 comp rx=0 tx=0).stop 2024-02-16T19:31:20.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.376+0000 7fbc1ac15700 1 -- 172.21.15.22:0/1809403484 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbc140ff3b0 msgr2=0x7fbc14103150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:20.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.376+0000 7fbc1ac15700 1 --2- 172.21.15.22:0/1809403484 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbc140ff3b0 0x7fbc14103150 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fbc0800c930 tx=0x7fbc0800ccf0 comp rx=0 tx=0).stop 2024-02-16T19:31:20.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.376+0000 7fbc1ac15700 1 -- 172.21.15.22:0/1809403484 shutdown_connections 2024-02-16T19:31:20.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.376+0000 7fbc1ac15700 1 --2- 172.21.15.22:0/1809403484 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7fbc0406d660 0x7fbc0406fb10 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:20.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.376+0000 7fbc1ac15700 1 --2- 172.21.15.22:0/1809403484 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbc140ff3b0 0x7fbc14103150 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:20.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.376+0000 7fbc1ac15700 1 --2- 172.21.15.22:0/1809403484 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc141007e0 0x7fbc141017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:20.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.376+0000 7fbc1ac15700 1 -- 172.21.15.22:0/1809403484 >> 172.21.15.22:0/1809403484 conn(0x7fbc140f9e70 msgr2=0x7fbc140fe2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:31:20.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.376+0000 7fbc1ac15700 1 -- 172.21.15.22:0/1809403484 shutdown_connections 2024-02-16T19:31:20.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.376+0000 7fbc1ac15700 1 -- 172.21.15.22:0/1809403484 wait complete. 2024-02-16T19:31:20.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.483+0000 7fda2ee69700 1 -- 172.21.15.22:0/3434914061 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda28103330 msgr2=0x7fda281057b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:20.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.483+0000 7fda2ee69700 1 --2- 172.21.15.22:0/3434914061 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda28103330 0x7fda281057b0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fda14009910 tx=0x7fda14009c20 comp rx=0 tx=0).stop 2024-02-16T19:31:20.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.483+0000 7fda2ee69700 1 -- 172.21.15.22:0/3434914061 shutdown_connections 2024-02-16T19:31:20.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.483+0000 7fda2ee69700 1 --2- 172.21.15.22:0/3434914061 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda28103330 0x7fda281057b0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:20.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.483+0000 7fda2ee69700 1 --2- 172.21.15.22:0/3434914061 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda28069b00 0x7fda28102df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:20.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.483+0000 7fda2ee69700 1 -- 172.21.15.22:0/3434914061 >> 172.21.15.22:0/3434914061 conn(0x7fda280fa7a0 msgr2=0x7fda280fcbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:31:20.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.483+0000 7fda2ee69700 1 -- 172.21.15.22:0/3434914061 shutdown_connections 2024-02-16T19:31:20.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.483+0000 7fda2ee69700 1 -- 172.21.15.22:0/3434914061 wait complete. 2024-02-16T19:31:20.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.483+0000 7fda2ee69700 1 Processor -- start 2024-02-16T19:31:20.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.483+0000 7fda2ee69700 1 -- start start 2024-02-16T19:31:20.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.483+0000 7fda2ee69700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda28103330 0x7fda28100740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:31:20.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.483+0000 7fda2ee69700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda280fed90 0x7fda280ff200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:31:20.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.483+0000 7fda2ee69700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda280ff740 con 0x7fda28103330 2024-02-16T19:31:20.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.483+0000 7fda2ee69700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda280ff8b0 con 0x7fda280fed90 2024-02-16T19:31:20.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.484+0000 7fda2cc05700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda28103330 0x7fda28100740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:31:20.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.484+0000 7fda2cc05700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda28103330 0x7fda28100740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51862/0 (socket says 172.21.15.22:51862) 2024-02-16T19:31:20.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.484+0000 7fda2cc05700 1 -- 172.21.15.22:0/1614335889 learned_addr learned my addr 172.21.15.22:0/1614335889 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:31:20.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.484+0000 7fda27fff700 1 --2- 172.21.15.22:0/1614335889 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda280fed90 0x7fda280ff200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:31:20.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.484+0000 7fda2cc05700 1 -- 172.21.15.22:0/1614335889 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda280fed90 msgr2=0x7fda280ff200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:20.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.484+0000 7fda2cc05700 1 --2- 172.21.15.22:0/1614335889 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda280fed90 0x7fda280ff200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:20.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.484+0000 7fda2cc05700 1 -- 172.21.15.22:0/1614335889 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda140095f0 con 0x7fda28103330 2024-02-16T19:31:20.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.484+0000 7fda2cc05700 1 --2- 172.21.15.22:0/1614335889 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda28103330 0x7fda28100740 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fda1c00d900 tx=0x7fda1c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:31:20.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.484+0000 7fda25ffb700 1 -- 172.21.15.22:0/1614335889 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda1c009900 con 0x7fda28103330 2024-02-16T19:31:20.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.484+0000 7fda25ffb700 1 -- 172.21.15.22:0/1614335889 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fda1c010460 con 0x7fda28103330 2024-02-16T19:31:20.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.484+0000 7fda2ee69700 1 -- 172.21.15.22:0/1614335889 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda280ffb90 con 0x7fda28103330 2024-02-16T19:31:20.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.484+0000 7fda25ffb700 1 -- 172.21.15.22:0/1614335889 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda1c00f5f0 con 0x7fda28103330 2024-02-16T19:31:20.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.485+0000 7fda2ee69700 1 -- 172.21.15.22:0/1614335889 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda2819e070 con 0x7fda28103330 2024-02-16T19:31:20.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.485+0000 7fda2ee69700 1 -- 172.21.15.22:0/1614335889 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda2818fc10 con 0x7fda28103330 2024-02-16T19:31:20.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.486+0000 7fda25ffb700 1 -- 172.21.15.22:0/1614335889 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 32) v1 ==== 91448+0+0 (secure 0 0 0) 0x7fda1c0105d0 con 0x7fda28103330 2024-02-16T19:31:20.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.486+0000 7fda25ffb700 1 --2- 172.21.15.22:0/1614335889 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7fda1806d650 0x7fda1806fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:31:20.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.486+0000 7fda25ffb700 1 -- 172.21.15.22:0/1614335889 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fda1c08d020 con 0x7fda28103330 2024-02-16T19:31:20.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.486+0000 7fda27fff700 1 --2- 172.21.15.22:0/1614335889 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7fda1806d650 0x7fda1806fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:31:20.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.487+0000 7fda27fff700 1 --2- 172.21.15.22:0/1614335889 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7fda1806d650 0x7fda1806fb00 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fda14006010 tx=0x7fda1400b540 comp rx=0 tx=0).ready entity=mgr.24407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:31:20.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.491+0000 7fda25ffb700 1 -- 172.21.15.22:0/1614335889 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7fda1c05a610 con 0x7fda28103330 2024-02-16T19:31:20.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:20 smithi117 ceph-mon[35309]: pgmap v46: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 85 B/s rd, 426 B/s wr, 0 op/s 2024-02-16T19:31:20.688 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:20 smithi022 ceph-mon[29221]: pgmap v46: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 85 B/s rd, 426 B/s wr, 0 op/s 2024-02-16T19:31:20.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.812+0000 7fda2ee69700 1 -- 172.21.15.22:0/1614335889 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fda28068ab0 con 0x7fda28103330 2024-02-16T19:31:20.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.813+0000 7fda25ffb700 1 -- 172.21.15.22:0/1614335889 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7fda1c05a1a0 con 0x7fda28103330 2024-02-16T19:31:20.815 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-02-16T19:31:20.815 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": { 2024-02-16T19:31:20.815 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-02-16T19:31:20.815 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:31:20.815 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": { 2024-02-16T19:31:20.815 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T19:31:20.815 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:31:20.815 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": { 2024-02-16T19:31:20.815 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-02-16T19:31:20.815 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:31:20.815 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": { 2024-02-16T19:31:20.815 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-02-16T19:31:20.815 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:31:20.816 INFO:teuthology.orchestra.run.smithi022.stdout: "overall": { 2024-02-16T19:31:20.816 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 12, 2024-02-16T19:31:20.816 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T19:31:20.816 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-02-16T19:31:20.816 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-02-16T19:31:20.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.816+0000 7fda2ee69700 1 -- 172.21.15.22:0/1614335889 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7fda1806d650 msgr2=0x7fda1806fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:20.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.816+0000 7fda2ee69700 1 --2- 172.21.15.22:0/1614335889 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7fda1806d650 0x7fda1806fb00 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fda14006010 tx=0x7fda1400b540 comp rx=0 tx=0).stop 2024-02-16T19:31:20.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.816+0000 7fda2ee69700 1 -- 172.21.15.22:0/1614335889 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda28103330 msgr2=0x7fda28100740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:20.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.816+0000 7fda2ee69700 1 --2- 172.21.15.22:0/1614335889 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda28103330 0x7fda28100740 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fda1c00d900 tx=0x7fda1c00dcc0 comp rx=0 tx=0).stop 2024-02-16T19:31:20.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.816+0000 7fda2ee69700 1 -- 172.21.15.22:0/1614335889 shutdown_connections 2024-02-16T19:31:20.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.816+0000 7fda2ee69700 1 --2- 172.21.15.22:0/1614335889 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7fda1806d650 0x7fda1806fb00 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:20.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.816+0000 7fda2ee69700 1 --2- 172.21.15.22:0/1614335889 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda28103330 0x7fda28100740 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:20.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.816+0000 7fda2ee69700 1 --2- 172.21.15.22:0/1614335889 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda280fed90 0x7fda280ff200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:20.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.816+0000 7fda2ee69700 1 -- 172.21.15.22:0/1614335889 >> 172.21.15.22:0/1614335889 conn(0x7fda280fa7a0 msgr2=0x7fda280fcbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:31:20.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.816+0000 7fda2ee69700 1 -- 172.21.15.22:0/1614335889 shutdown_connections 2024-02-16T19:31:20.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.816+0000 7fda2ee69700 1 -- 172.21.15.22:0/1614335889 wait complete. 2024-02-16T19:31:20.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.924+0000 7f56c8428700 1 -- 172.21.15.22:0/2550256559 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f56c00ff130 msgr2=0x7f56c00ff540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:20.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.924+0000 7f56c8428700 1 --2- 172.21.15.22:0/2550256559 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f56c00ff130 0x7f56c00ff540 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f56b800b0a0 tx=0x7f56b800b3b0 comp rx=0 tx=0).stop 2024-02-16T19:31:20.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.924+0000 7f56c8428700 1 -- 172.21.15.22:0/2550256559 shutdown_connections 2024-02-16T19:31:20.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.924+0000 7f56c8428700 1 --2- 172.21.15.22:0/2550256559 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56c01003d0 0x7f56c0100840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:20.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.924+0000 7f56c8428700 1 --2- 172.21.15.22:0/2550256559 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f56c00ff130 0x7f56c00ff540 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:20.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.924+0000 7f56c8428700 1 -- 172.21.15.22:0/2550256559 >> 172.21.15.22:0/2550256559 conn(0x7f56c00fa780 msgr2=0x7f56c00fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:31:20.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.924+0000 7f56c8428700 1 -- 172.21.15.22:0/2550256559 shutdown_connections 2024-02-16T19:31:20.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.924+0000 7f56c8428700 1 -- 172.21.15.22:0/2550256559 wait complete. 2024-02-16T19:31:20.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.925+0000 7f56c8428700 1 Processor -- start 2024-02-16T19:31:20.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.925+0000 7f56c8428700 1 -- start start 2024-02-16T19:31:20.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.925+0000 7f56c8428700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56c00ff130 0x7f56c0103160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:31:20.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.925+0000 7f56c8428700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f56c01003d0 0x7f56c01017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:31:20.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.925+0000 7f56c8428700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c0101cf0 con 0x7f56c01003d0 2024-02-16T19:31:20.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.925+0000 7f56c8428700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c0101e30 con 0x7f56c00ff130 2024-02-16T19:31:20.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.925+0000 7f56c59c3700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f56c01003d0 0x7f56c01017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:31:20.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.925+0000 7f56c59c3700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f56c01003d0 0x7f56c01017b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51892/0 (socket says 172.21.15.22:51892) 2024-02-16T19:31:20.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.925+0000 7f56c59c3700 1 -- 172.21.15.22:0/3599989621 learned_addr learned my addr 172.21.15.22:0/3599989621 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:31:20.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.925+0000 7f56c61c4700 1 --2- 172.21.15.22:0/3599989621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56c00ff130 0x7f56c0103160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:31:20.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.926+0000 7f56c59c3700 1 -- 172.21.15.22:0/3599989621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56c00ff130 msgr2=0x7f56c0103160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:20.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.926+0000 7f56c59c3700 1 --2- 172.21.15.22:0/3599989621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56c00ff130 0x7f56c0103160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:20.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.926+0000 7f56c59c3700 1 -- 172.21.15.22:0/3599989621 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56b8009d00 con 0x7f56c01003d0 2024-02-16T19:31:20.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.926+0000 7f56c59c3700 1 --2- 172.21.15.22:0/3599989621 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f56c01003d0 0x7f56c01017b0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f56b400d900 tx=0x7f56b400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:31:20.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.926+0000 7f56bf7fe700 1 -- 172.21.15.22:0/3599989621 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f56b40041f0 con 0x7f56c01003d0 2024-02-16T19:31:20.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.926+0000 7f56bf7fe700 1 -- 172.21.15.22:0/3599989621 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f56b4004ce0 con 0x7f56c01003d0 2024-02-16T19:31:20.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.926+0000 7f56bf7fe700 1 -- 172.21.15.22:0/3599989621 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f56b400b750 con 0x7f56c01003d0 2024-02-16T19:31:20.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.926+0000 7f56c8428700 1 -- 172.21.15.22:0/3599989621 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56c0102110 con 0x7f56c01003d0 2024-02-16T19:31:20.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.926+0000 7f56c8428700 1 -- 172.21.15.22:0/3599989621 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56c019db10 con 0x7f56c01003d0 2024-02-16T19:31:20.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.927+0000 7f56c8428700 1 -- 172.21.15.22:0/3599989621 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56c0107c10 con 0x7f56c01003d0 2024-02-16T19:31:20.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.928+0000 7f56bf7fe700 1 -- 172.21.15.22:0/3599989621 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 32) v1 ==== 91448+0+0 (secure 0 0 0) 0x7f56b4004e50 con 0x7f56c01003d0 2024-02-16T19:31:20.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.928+0000 7f56bf7fe700 1 --2- 172.21.15.22:0/3599989621 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f56a806d650 0x7f56a806fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:31:20.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.929+0000 7f56bf7fe700 1 -- 172.21.15.22:0/3599989621 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f56b408d020 con 0x7f56c01003d0 2024-02-16T19:31:20.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.929+0000 7f56c61c4700 1 --2- 172.21.15.22:0/3599989621 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f56a806d650 0x7f56a806fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:31:20.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.929+0000 7f56c61c4700 1 --2- 172.21.15.22:0/3599989621 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f56a806d650 0x7f56a806fb00 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f56b800b830 tx=0x7f56b8000f40 comp rx=0 tx=0).ready entity=mgr.24407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:31:20.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:20.933+0000 7f56bf7fe700 1 -- 172.21.15.22:0/3599989621 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f56b405a690 con 0x7f56c01003d0 2024-02-16T19:31:21.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:21.177+0000 7f56c8428700 1 -- 172.21.15.22:0/3599989621 --> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f56c019e0c0 con 0x7f56a806d650 2024-02-16T19:31:21.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:21.178+0000 7f56bf7fe700 1 -- 172.21.15.22:0/3599989621 <== mgr.24407 v2:172.21.15.22:6800/3703806543 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+367 (secure 0 0 0) 0x7f56c019e0c0 con 0x7f56a806d650 2024-02-16T19:31:21.179 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-02-16T19:31:21.179 INFO:teuthology.orchestra.run.smithi022.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T19:31:21.179 INFO:teuthology.orchestra.run.smithi022.stdout: "in_progress": true, 2024-02-16T19:31:21.180 INFO:teuthology.orchestra.run.smithi022.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-02-16T19:31:21.180 INFO:teuthology.orchestra.run.smithi022.stdout: "services_complete": [ 2024-02-16T19:31:21.180 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr" 2024-02-16T19:31:21.180 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-02-16T19:31:21.180 INFO:teuthology.orchestra.run.smithi022.stdout: "progress": "2/2 daemons upgraded", 2024-02-16T19:31:21.180 INFO:teuthology.orchestra.run.smithi022.stdout: "message": "Currently upgrading prometheus daemons", 2024-02-16T19:31:21.180 INFO:teuthology.orchestra.run.smithi022.stdout: "is_paused": false 2024-02-16T19:31:21.180 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-02-16T19:31:21.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:21.180+0000 7f56c8428700 1 -- 172.21.15.22:0/3599989621 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f56a806d650 msgr2=0x7f56a806fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:21.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:21.180+0000 7f56c8428700 1 --2- 172.21.15.22:0/3599989621 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f56a806d650 0x7f56a806fb00 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f56b800b830 tx=0x7f56b8000f40 comp rx=0 tx=0).stop 2024-02-16T19:31:21.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:21.180+0000 7f56c8428700 1 -- 172.21.15.22:0/3599989621 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f56c01003d0 msgr2=0x7f56c01017b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:21.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:21.180+0000 7f56c8428700 1 --2- 172.21.15.22:0/3599989621 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f56c01003d0 0x7f56c01017b0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f56b400d900 tx=0x7f56b400dc10 comp rx=0 tx=0).stop 2024-02-16T19:31:21.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:21.180+0000 7f56c8428700 1 -- 172.21.15.22:0/3599989621 shutdown_connections 2024-02-16T19:31:21.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:21.180+0000 7f56c8428700 1 --2- 172.21.15.22:0/3599989621 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f56a806d650 0x7f56a806fb00 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:21.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:21.180+0000 7f56c8428700 1 --2- 172.21.15.22:0/3599989621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56c00ff130 0x7f56c0103160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:21.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:21.180+0000 7f56c8428700 1 --2- 172.21.15.22:0/3599989621 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f56c01003d0 0x7f56c01017b0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:21.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:21.180+0000 7f56c8428700 1 -- 172.21.15.22:0/3599989621 >> 172.21.15.22:0/3599989621 conn(0x7f56c00fa780 msgr2=0x7f56c0068540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:31:21.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:21.181+0000 7f56c8428700 1 -- 172.21.15.22:0/3599989621 shutdown_connections 2024-02-16T19:31:21.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:21.181+0000 7f56c8428700 1 -- 172.21.15.22:0/3599989621 wait complete. 2024-02-16T19:31:21.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:21 smithi117 ceph-mon[35309]: from='client.14740 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:31:21.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:21 smithi117 ceph-mon[35309]: from='client.14744 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:31:21.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:21 smithi117 ceph-mon[35309]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:31:21.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:21 smithi117 ceph-mon[35309]: from='client.? 172.21.15.22:0/1614335889' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:21.714 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:21 smithi022 ceph-mon[29221]: from='client.14740 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:31:21.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:21 smithi022 ceph-mon[29221]: from='client.14744 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:31:21.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:21 smithi022 ceph-mon[29221]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:31:21.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:21 smithi022 ceph-mon[29221]: from='client.? 172.21.15.22:0/1614335889' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:22.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:22 smithi022 ceph-mon[29221]: from='client.14756 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:31:22.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:22 smithi022 ceph-mon[29221]: pgmap v47: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 85 B/s rd, 426 B/s wr, 0 op/s 2024-02-16T19:31:22.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:22 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:31:22.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:22 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:31:22.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:22 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:31:22.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:22 smithi117 ceph-mon[35309]: from='client.14756 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:31:22.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:22 smithi117 ceph-mon[35309]: pgmap v47: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 85 B/s rd, 426 B/s wr, 0 op/s 2024-02-16T19:31:22.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:22 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:31:22.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:22 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:31:22.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:22 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:31:23.772 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:23 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:31:23.788 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:23 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:31:24.777 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:24 smithi022 ceph-mon[29221]: pgmap v48: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:31:24.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:24 smithi117 ceph-mon[35309]: pgmap v48: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:31:25.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:25 smithi117 ceph-mon[35309]: mgrmap e33: smithi022.gmdcch(active, since 91s), standbys: smithi117.cohbtv 2024-02-16T19:31:25.932 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:25 smithi022 ceph-mon[29221]: mgrmap e33: smithi022.gmdcch(active, since 91s), standbys: smithi117.cohbtv 2024-02-16T19:31:26.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:26 smithi117 ceph-mon[35309]: pgmap v49: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-02-16T19:31:26.964 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:26 smithi022 ceph-mon[29221]: pgmap v49: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-02-16T19:31:27.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:27 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:31:27.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:27 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:31:27.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:27 smithi117 ceph-mon[35309]: pgmap v50: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-02-16T19:31:27.964 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:27 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:31:27.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:27 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:31:27.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:27 smithi022 ceph-mon[29221]: pgmap v50: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-02-16T19:31:28.817 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:28 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:31:28.818 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:28 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:31:29.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:28 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:31:29.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:28 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:31:29.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:29 smithi022 ceph-mon[29221]: pgmap v51: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-02-16T19:31:30.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:29 smithi117 ceph-mon[35309]: pgmap v51: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-02-16T19:31:32.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:32 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:31:32.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:32 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:31:32.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:32 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:31:32.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:32 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:31:32.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:32 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:31:32.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:32 smithi117 ceph-mon[35309]: pgmap v52: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-02-16T19:31:32.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:32 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-16T19:31:32.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:32 smithi117 ceph-mon[35309]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-16T19:31:32.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:32 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:31:32.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:32 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:32.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:32 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:32.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:32 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:32.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:32 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:32.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:32 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:32.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:32 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:32.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:32 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:32.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:32 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:32.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:32 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:32.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:32 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:31:32.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:32 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:31:32.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:32 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:31:32.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:32 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:31:32.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:32 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:31:32.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:32 smithi022 ceph-mon[29221]: pgmap v52: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-02-16T19:31:32.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:32 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-16T19:31:32.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:32 smithi022 ceph-mon[29221]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-16T19:31:32.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:32 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:31:32.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:32 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:32.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:32 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:32.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:32 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:32.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:32 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:32.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:32 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:32.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:32 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:32.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:32 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:32.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:32 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:32.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:32 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:34.616 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:34 smithi022 ceph-mon[29221]: Upgrade: Updating alertmanager.smithi022 2024-02-16T19:31:34.616 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:34 smithi022 ceph-mon[29221]: Deploying daemon alertmanager.smithi022 on smithi022 2024-02-16T19:31:34.616 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:34 smithi022 ceph-mon[29221]: pgmap v53: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-02-16T19:31:34.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:34 smithi117 ceph-mon[35309]: Upgrade: Updating alertmanager.smithi022 2024-02-16T19:31:34.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:34 smithi117 ceph-mon[35309]: Deploying daemon alertmanager.smithi022 on smithi022 2024-02-16T19:31:34.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:34 smithi117 ceph-mon[35309]: pgmap v53: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-02-16T19:31:36.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:36 smithi022 ceph-mon[29221]: pgmap v54: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-02-16T19:31:36.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:36 smithi117 ceph-mon[35309]: pgmap v54: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-02-16T19:31:38.124 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:37 smithi022 ceph-mon[29221]: pgmap v55: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:31:38.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:37 smithi117 ceph-mon[35309]: pgmap v55: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:31:39.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:38 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:31:39.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:38 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:31:39.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:38 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:31:39.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:38 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:31:39.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:38 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:31:39.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:38 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:31:39.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:38 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:31:39.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:38 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:31:40.117 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:40 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:31:40.117 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:40 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:31:40.117 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:40 smithi022 ceph-mon[29221]: pgmap v56: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:31:40.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:40 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:31:40.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:40 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:31:40.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:40 smithi117 ceph-mon[35309]: pgmap v56: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:31:42.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:42 smithi117 ceph-mon[35309]: pgmap v57: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:31:42.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:42 smithi022 ceph-mon[29221]: pgmap v57: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:31:44.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:44 smithi117 ceph-mon[35309]: pgmap v58: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:31:44.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:44 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:31:44.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:44 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:31:44.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:44 smithi022 ceph-mon[29221]: pgmap v58: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:31:44.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:44 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:31:44.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:44 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:31:45.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:45 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:31:45.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:45 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:31:45.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:45 smithi022 ceph-mon[29221]: pgmap v59: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:31:46.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:45 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:31:46.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:45 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:31:46.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:45 smithi117 ceph-mon[35309]: pgmap v59: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:31:48.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:47 smithi022 ceph-mon[29221]: pgmap v60: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:31:48.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:47 smithi117 ceph-mon[35309]: pgmap v60: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:31:49.029 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:49 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:31:49.029 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:49 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:31:49.029 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:49 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:31:49.030 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:49 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:31:49.030 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:49 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:31:49.030 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:49 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-16T19:31:49.030 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:49 smithi022 ceph-mon[29221]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-16T19:31:49.030 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:49 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:31:49.030 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:49 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:49.030 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:49 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:49.030 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:49 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:49.030 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:49 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:49.030 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:49 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:49.030 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:49 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:49.030 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:49 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:49.030 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:49 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:49.030 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:49 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:49.031 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:49 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:49.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:49 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:31:49.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:49 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:31:49.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:49 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:31:49.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:49 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:31:49.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:49 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:31:49.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:49 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-16T19:31:49.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:49 smithi117 ceph-mon[35309]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-16T19:31:49.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:49 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:31:49.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:49 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:49.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:49 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:49.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:49 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:49.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:49 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:49.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:49 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:49.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:49 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:49.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:49 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:49.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:49 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:49.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:49 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:49.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:49 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:50.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:50 smithi022 ceph-mon[29221]: pgmap v61: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:31:50.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:50 smithi117 ceph-mon[35309]: pgmap v61: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:31:51.302 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:51 smithi117 ceph-mon[35309]: Upgrade: Updating grafana.smithi022 2024-02-16T19:31:51.302 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:51 smithi117 ceph-mon[35309]: Deploying daemon grafana.smithi022 on smithi022 2024-02-16T19:31:51.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.304+0000 7f06220f3700 1 -- 172.21.15.22:0/3400151470 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f061c069b00 msgr2=0x7f061c1051f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:51.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.304+0000 7f06220f3700 1 --2- 172.21.15.22:0/3400151470 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f061c069b00 0x7f061c1051f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f060c009910 tx=0x7f060c009c20 comp rx=0 tx=0).stop 2024-02-16T19:31:51.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.304+0000 7f06220f3700 1 -- 172.21.15.22:0/3400151470 shutdown_connections 2024-02-16T19:31:51.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.304+0000 7f06220f3700 1 --2- 172.21.15.22:0/3400151470 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f061c105730 0x7f061c107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:51.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.304+0000 7f06220f3700 1 --2- 172.21.15.22:0/3400151470 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f061c069b00 0x7f061c1051f0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:51.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.304+0000 7f06220f3700 1 -- 172.21.15.22:0/3400151470 >> 172.21.15.22:0/3400151470 conn(0x7f061c0fa7a0 msgr2=0x7f061c0fcbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:31:51.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.304+0000 7f06220f3700 1 -- 172.21.15.22:0/3400151470 shutdown_connections 2024-02-16T19:31:51.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.304+0000 7f06220f3700 1 -- 172.21.15.22:0/3400151470 wait complete. 2024-02-16T19:31:51.306 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:51 smithi022 ceph-mon[29221]: Upgrade: Updating grafana.smithi022 2024-02-16T19:31:51.306 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:51 smithi022 ceph-mon[29221]: Deploying daemon grafana.smithi022 on smithi022 2024-02-16T19:31:51.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.305+0000 7f06220f3700 1 Processor -- start 2024-02-16T19:31:51.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.305+0000 7f06220f3700 1 -- start start 2024-02-16T19:31:51.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.305+0000 7f06220f3700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f061c105730 0x7f061c198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:31:51.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.305+0000 7f06220f3700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f061c198840 0x7f061c19d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:31:51.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.305+0000 7f06220f3700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f061c198d40 con 0x7f061c105730 2024-02-16T19:31:51.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.305+0000 7f06220f3700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f061c198eb0 con 0x7f061c198840 2024-02-16T19:31:51.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.305+0000 7f061b7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f061c105730 0x7f061c198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:31:51.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.305+0000 7f061b7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f061c105730 0x7f061c198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55434/0 (socket says 172.21.15.22:55434) 2024-02-16T19:31:51.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.305+0000 7f061b7fe700 1 -- 172.21.15.22:0/1052385556 learned_addr learned my addr 172.21.15.22:0/1052385556 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:31:51.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.305+0000 7f061affd700 1 --2- 172.21.15.22:0/1052385556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f061c198840 0x7f061c19d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:31:51.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.306+0000 7f061b7fe700 1 -- 172.21.15.22:0/1052385556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f061c198840 msgr2=0x7f061c19d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:51.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.306+0000 7f061b7fe700 1 --2- 172.21.15.22:0/1052385556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f061c198840 0x7f061c19d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:51.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.306+0000 7f061b7fe700 1 -- 172.21.15.22:0/1052385556 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f060c0095f0 con 0x7f061c105730 2024-02-16T19:31:51.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.306+0000 7f061b7fe700 1 --2- 172.21.15.22:0/1052385556 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f061c105730 0x7f061c198300 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f060c009910 tx=0x7f060c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:31:51.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.306+0000 7f0618ff9700 1 -- 172.21.15.22:0/1052385556 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f060c01d070 con 0x7f061c105730 2024-02-16T19:31:51.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.306+0000 7f0618ff9700 1 -- 172.21.15.22:0/1052385556 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f060c005dc0 con 0x7f061c105730 2024-02-16T19:31:51.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.306+0000 7f06220f3700 1 -- 172.21.15.22:0/1052385556 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f061c19ddf0 con 0x7f061c105730 2024-02-16T19:31:51.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.306+0000 7f0618ff9700 1 -- 172.21.15.22:0/1052385556 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f060c021410 con 0x7f061c105730 2024-02-16T19:31:51.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.306+0000 7f06220f3700 1 -- 172.21.15.22:0/1052385556 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f061c19e250 con 0x7f061c105730 2024-02-16T19:31:51.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.307+0000 7f06220f3700 1 -- 172.21.15.22:0/1052385556 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f061c191df0 con 0x7f061c105730 2024-02-16T19:31:51.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.308+0000 7f0618ff9700 1 -- 172.21.15.22:0/1052385556 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 33) v1 ==== 91346+0+0 (secure 0 0 0) 0x7f060c00f460 con 0x7f061c105730 2024-02-16T19:31:51.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.308+0000 7f0618ff9700 1 --2- 172.21.15.22:0/1052385556 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f060406d520 0x7f060406f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:31:51.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.308+0000 7f0618ff9700 1 -- 172.21.15.22:0/1052385556 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f060c08dfc0 con 0x7f061c105730 2024-02-16T19:31:51.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.308+0000 7f061affd700 1 --2- 172.21.15.22:0/1052385556 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f060406d520 0x7f060406f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:31:51.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.309+0000 7f061affd700 1 --2- 172.21.15.22:0/1052385556 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f060406d520 0x7f060406f9d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0610005950 tx=0x7f061000b410 comp rx=0 tx=0).ready entity=mgr.24407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:31:51.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.313+0000 7f0618ff9700 1 -- 172.21.15.22:0/1052385556 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f060c05b850 con 0x7f061c105730 2024-02-16T19:31:51.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.611+0000 7f06220f3700 1 -- 172.21.15.22:0/1052385556 --> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f061c02cfa0 con 0x7f060406d520 2024-02-16T19:31:51.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.612+0000 7f0618ff9700 1 -- 172.21.15.22:0/1052385556 <== mgr.24407 v2:172.21.15.22:6800/3703806543 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7f061c02cfa0 con 0x7f060406d520 2024-02-16T19:31:51.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.615+0000 7f06220f3700 1 -- 172.21.15.22:0/1052385556 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f060406d520 msgr2=0x7f060406f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:51.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.615+0000 7f06220f3700 1 --2- 172.21.15.22:0/1052385556 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f060406d520 0x7f060406f9d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0610005950 tx=0x7f061000b410 comp rx=0 tx=0).stop 2024-02-16T19:31:51.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.615+0000 7f06220f3700 1 -- 172.21.15.22:0/1052385556 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f061c105730 msgr2=0x7f061c198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:51.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.615+0000 7f06220f3700 1 --2- 172.21.15.22:0/1052385556 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f061c105730 0x7f061c198300 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f060c009910 tx=0x7f060c00bd70 comp rx=0 tx=0).stop 2024-02-16T19:31:51.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.615+0000 7f06220f3700 1 -- 172.21.15.22:0/1052385556 shutdown_connections 2024-02-16T19:31:51.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.615+0000 7f06220f3700 1 --2- 172.21.15.22:0/1052385556 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f060406d520 0x7f060406f9d0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:51.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.615+0000 7f06220f3700 1 --2- 172.21.15.22:0/1052385556 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f061c105730 0x7f061c198300 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:51.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.615+0000 7f06220f3700 1 --2- 172.21.15.22:0/1052385556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f061c198840 0x7f061c19d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:51.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.615+0000 7f06220f3700 1 -- 172.21.15.22:0/1052385556 >> 172.21.15.22:0/1052385556 conn(0x7f061c0fa7a0 msgr2=0x7f061c0fcbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:31:51.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.615+0000 7f06220f3700 1 -- 172.21.15.22:0/1052385556 shutdown_connections 2024-02-16T19:31:51.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.615+0000 7f06220f3700 1 -- 172.21.15.22:0/1052385556 wait complete. 2024-02-16T19:31:51.631 INFO:teuthology.orchestra.run.smithi022.stdout:true 2024-02-16T19:31:51.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.734+0000 7fa1035eb700 1 -- 172.21.15.22:0/3022869722 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0fc100750 msgr2=0x7fa0fc100b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:51.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.734+0000 7fa1035eb700 1 --2- 172.21.15.22:0/3022869722 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0fc100750 0x7fa0fc100b60 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8009910 tx=0x7fa0f8009c20 comp rx=0 tx=0).stop 2024-02-16T19:31:51.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.734+0000 7fa1035eb700 1 -- 172.21.15.22:0/3022869722 shutdown_connections 2024-02-16T19:31:51.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.734+0000 7fa1035eb700 1 --2- 172.21.15.22:0/3022869722 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0fc1010a0 0x7fa0fc0ff4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:51.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.734+0000 7fa1035eb700 1 --2- 172.21.15.22:0/3022869722 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0fc100750 0x7fa0fc100b60 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:51.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.734+0000 7fa1035eb700 1 -- 172.21.15.22:0/3022869722 >> 172.21.15.22:0/3022869722 conn(0x7fa0fc0f9f10 msgr2=0x7fa0fc0fc340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:31:51.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.734+0000 7fa1035eb700 1 -- 172.21.15.22:0/3022869722 shutdown_connections 2024-02-16T19:31:51.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.734+0000 7fa1035eb700 1 -- 172.21.15.22:0/3022869722 wait complete. 2024-02-16T19:31:51.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.735+0000 7fa1035eb700 1 Processor -- start 2024-02-16T19:31:51.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.735+0000 7fa1035eb700 1 -- start start 2024-02-16T19:31:51.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.735+0000 7fa1035eb700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0fc100750 0x7fa0fc195af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:31:51.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.735+0000 7fa1035eb700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0fc1010a0 0x7fa0fc196030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:31:51.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.735+0000 7fa1035eb700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0fc196650 con 0x7fa0fc1010a0 2024-02-16T19:31:51.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.735+0000 7fa1035eb700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0fc196790 con 0x7fa0fc100750 2024-02-16T19:31:51.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.735+0000 7fa100b86700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0fc1010a0 0x7fa0fc196030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:31:51.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.735+0000 7fa100b86700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0fc1010a0 0x7fa0fc196030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55458/0 (socket says 172.21.15.22:55458) 2024-02-16T19:31:51.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.735+0000 7fa100b86700 1 -- 172.21.15.22:0/4249020940 learned_addr learned my addr 172.21.15.22:0/4249020940 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:31:51.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.735+0000 7fa101387700 1 --2- 172.21.15.22:0/4249020940 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0fc100750 0x7fa0fc195af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:31:51.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.736+0000 7fa100b86700 1 -- 172.21.15.22:0/4249020940 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0fc100750 msgr2=0x7fa0fc195af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:51.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.736+0000 7fa100b86700 1 --2- 172.21.15.22:0/4249020940 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0fc100750 0x7fa0fc195af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:51.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.736+0000 7fa100b86700 1 -- 172.21.15.22:0/4249020940 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0f80095f0 con 0x7fa0fc1010a0 2024-02-16T19:31:51.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.736+0000 7fa100b86700 1 --2- 172.21.15.22:0/4249020940 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0fc1010a0 0x7fa0fc196030 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec00d8d0 tx=0x7fa0ec00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:31:51.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.736+0000 7fa0f27fc700 1 -- 172.21.15.22:0/4249020940 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa0ec0098a0 con 0x7fa0fc1010a0 2024-02-16T19:31:51.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.736+0000 7fa1035eb700 1 -- 172.21.15.22:0/4249020940 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0fc19b240 con 0x7fa0fc1010a0 2024-02-16T19:31:51.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.736+0000 7fa0f27fc700 1 -- 172.21.15.22:0/4249020940 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa0ec010460 con 0x7fa0fc1010a0 2024-02-16T19:31:51.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.736+0000 7fa0f27fc700 1 -- 172.21.15.22:0/4249020940 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa0ec00f5f0 con 0x7fa0fc1010a0 2024-02-16T19:31:51.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.736+0000 7fa1035eb700 1 -- 172.21.15.22:0/4249020940 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0fc19b790 con 0x7fa0fc1010a0 2024-02-16T19:31:51.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.737+0000 7fa1035eb700 1 -- 172.21.15.22:0/4249020940 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0fc107c00 con 0x7fa0fc1010a0 2024-02-16T19:31:51.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.738+0000 7fa0f27fc700 1 -- 172.21.15.22:0/4249020940 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 33) v1 ==== 91346+0+0 (secure 0 0 0) 0x7fa0ec009a00 con 0x7fa0fc1010a0 2024-02-16T19:31:51.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.738+0000 7fa0f27fc700 1 --2- 172.21.15.22:0/4249020940 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7fa0e806d520 0x7fa0e806f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:31:51.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.738+0000 7fa0f27fc700 1 -- 172.21.15.22:0/4249020940 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fa0ec08ce40 con 0x7fa0fc1010a0 2024-02-16T19:31:51.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.738+0000 7fa101387700 1 --2- 172.21.15.22:0/4249020940 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7fa0e806d520 0x7fa0e806f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:31:51.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.739+0000 7fa101387700 1 --2- 172.21.15.22:0/4249020940 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7fa0e806d520 0x7fa0e806f9d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8006010 tx=0x7fa0f800b540 comp rx=0 tx=0).ready entity=mgr.24407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:31:51.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.743+0000 7fa0f27fc700 1 -- 172.21.15.22:0/4249020940 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7fa0ec05a650 con 0x7fa0fc1010a0 2024-02-16T19:31:51.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.994+0000 7fa1035eb700 1 -- 172.21.15.22:0/4249020940 --> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa0fc0fea50 con 0x7fa0e806d520 2024-02-16T19:31:51.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.995+0000 7fa0f27fc700 1 -- 172.21.15.22:0/4249020940 <== mgr.24407 v2:172.21.15.22:6800/3703806543 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7fa0fc0fea50 con 0x7fa0e806d520 2024-02-16T19:31:51.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.997+0000 7fa1035eb700 1 -- 172.21.15.22:0/4249020940 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7fa0e806d520 msgr2=0x7fa0e806f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:51.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.997+0000 7fa1035eb700 1 --2- 172.21.15.22:0/4249020940 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7fa0e806d520 0x7fa0e806f9d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8006010 tx=0x7fa0f800b540 comp rx=0 tx=0).stop 2024-02-16T19:31:51.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.997+0000 7fa1035eb700 1 -- 172.21.15.22:0/4249020940 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0fc1010a0 msgr2=0x7fa0fc196030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:51.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.997+0000 7fa1035eb700 1 --2- 172.21.15.22:0/4249020940 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0fc1010a0 0x7fa0fc196030 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec00d8d0 tx=0x7fa0ec00dbe0 comp rx=0 tx=0).stop 2024-02-16T19:31:51.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.997+0000 7fa1035eb700 1 -- 172.21.15.22:0/4249020940 shutdown_connections 2024-02-16T19:31:51.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.997+0000 7fa1035eb700 1 --2- 172.21.15.22:0/4249020940 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7fa0e806d520 0x7fa0e806f9d0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:51.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.997+0000 7fa1035eb700 1 --2- 172.21.15.22:0/4249020940 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0fc100750 0x7fa0fc195af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:51.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.997+0000 7fa1035eb700 1 --2- 172.21.15.22:0/4249020940 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0fc1010a0 0x7fa0fc196030 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:51.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.997+0000 7fa1035eb700 1 -- 172.21.15.22:0/4249020940 >> 172.21.15.22:0/4249020940 conn(0x7fa0fc0f9f10 msgr2=0x7fa0fc0fe340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:31:51.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.997+0000 7fa1035eb700 1 -- 172.21.15.22:0/4249020940 shutdown_connections 2024-02-16T19:31:51.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:51.997+0000 7fa1035eb700 1 -- 172.21.15.22:0/4249020940 wait complete. 2024-02-16T19:31:52.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.110+0000 7f5829338700 1 -- 172.21.15.22:0/722607566 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58240fee50 msgr2=0x7f5824101580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:52.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.110+0000 7f5829338700 1 --2- 172.21.15.22:0/722607566 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58240fee50 0x7f5824101580 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f5818009910 tx=0x7f5818009c20 comp rx=0 tx=0).stop 2024-02-16T19:31:52.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.110+0000 7f5829338700 1 -- 172.21.15.22:0/722607566 shutdown_connections 2024-02-16T19:31:52.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.110+0000 7f5829338700 1 --2- 172.21.15.22:0/722607566 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58240fee50 0x7f5824101580 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:52.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.110+0000 7f5829338700 1 --2- 172.21.15.22:0/722607566 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58240fe500 0x7f58240fe910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:52.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.110+0000 7f5829338700 1 -- 172.21.15.22:0/722607566 >> 172.21.15.22:0/722607566 conn(0x7f58240f9f10 msgr2=0x7f58240fc340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:31:52.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.110+0000 7f5829338700 1 -- 172.21.15.22:0/722607566 shutdown_connections 2024-02-16T19:31:52.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.110+0000 7f5829338700 1 -- 172.21.15.22:0/722607566 wait complete. 2024-02-16T19:31:52.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.111+0000 7f5829338700 1 Processor -- start 2024-02-16T19:31:52.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.111+0000 7f5829338700 1 -- start start 2024-02-16T19:31:52.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.111+0000 7f5829338700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58240fe500 0x7f5824195b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:31:52.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.111+0000 7f5829338700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58240fee50 0x7f5824196070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:31:52.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.111+0000 7f5829338700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5824196690 con 0x7f58240fee50 2024-02-16T19:31:52.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.111+0000 7f5829338700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58241967d0 con 0x7f58240fe500 2024-02-16T19:31:52.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.111+0000 7f58227fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58240fee50 0x7f5824196070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:31:52.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.111+0000 7f58227fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58240fee50 0x7f5824196070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55470/0 (socket says 172.21.15.22:55470) 2024-02-16T19:31:52.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.111+0000 7f58227fc700 1 -- 172.21.15.22:0/3089716765 learned_addr learned my addr 172.21.15.22:0/3089716765 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:31:52.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.111+0000 7f5822ffd700 1 --2- 172.21.15.22:0/3089716765 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58240fe500 0x7f5824195b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:31:52.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.111+0000 7f58227fc700 1 -- 172.21.15.22:0/3089716765 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58240fe500 msgr2=0x7f5824195b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:52.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.111+0000 7f58227fc700 1 --2- 172.21.15.22:0/3089716765 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58240fe500 0x7f5824195b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:52.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.111+0000 7f58227fc700 1 -- 172.21.15.22:0/3089716765 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58180095f0 con 0x7f58240fee50 2024-02-16T19:31:52.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.112+0000 7f58227fc700 1 --2- 172.21.15.22:0/3089716765 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58240fee50 0x7f5824196070 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f581800bd70 tx=0x7f581800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:31:52.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.112+0000 7f580bfff700 1 -- 172.21.15.22:0/3089716765 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f581801d070 con 0x7f58240fee50 2024-02-16T19:31:52.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.112+0000 7f580bfff700 1 -- 172.21.15.22:0/3089716765 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5818022470 con 0x7f58240fee50 2024-02-16T19:31:52.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.112+0000 7f5829338700 1 -- 172.21.15.22:0/3089716765 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f582419b220 con 0x7f58240fee50 2024-02-16T19:31:52.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.112+0000 7f580bfff700 1 -- 172.21.15.22:0/3089716765 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f581800f670 con 0x7f58240fee50 2024-02-16T19:31:52.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.112+0000 7f5829338700 1 -- 172.21.15.22:0/3089716765 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f582419b710 con 0x7f58240fee50 2024-02-16T19:31:52.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.113+0000 7f5829338700 1 -- 172.21.15.22:0/3089716765 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5824108d10 con 0x7f58240fee50 2024-02-16T19:31:52.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.114+0000 7f580bfff700 1 -- 172.21.15.22:0/3089716765 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 33) v1 ==== 91346+0+0 (secure 0 0 0) 0x7f5818022a80 con 0x7f58240fee50 2024-02-16T19:31:52.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.114+0000 7f580bfff700 1 --2- 172.21.15.22:0/3089716765 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f580c06d520 0x7f580c06f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:31:52.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.114+0000 7f580bfff700 1 -- 172.21.15.22:0/3089716765 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f581808e570 con 0x7f58240fee50 2024-02-16T19:31:52.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.114+0000 7f5822ffd700 1 --2- 172.21.15.22:0/3089716765 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f580c06d520 0x7f580c06f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:31:52.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.115+0000 7f5822ffd700 1 --2- 172.21.15.22:0/3089716765 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f580c06d520 0x7f580c06f9d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5814005fd0 tx=0x7f581400a560 comp rx=0 tx=0).ready entity=mgr.24407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:31:52.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.119+0000 7f580bfff700 1 -- 172.21.15.22:0/3089716765 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f581805bf50 con 0x7f58240fee50 2024-02-16T19:31:52.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.364+0000 7f5829338700 1 -- 172.21.15.22:0/3089716765 --> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f582419bd50 con 0x7f580c06d520 2024-02-16T19:31:52.366 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:52 smithi022 ceph-mon[29221]: pgmap v62: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:31:52.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.376+0000 7f580bfff700 1 -- 172.21.15.22:0/3089716765 <== mgr.24407 v2:172.21.15.22:6800/3703806543 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f582419bd50 con 0x7f580c06d520 2024-02-16T19:31:52.377 INFO:teuthology.orchestra.run.smithi022.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T19:31:52.377 INFO:teuthology.orchestra.run.smithi022.stdout:alertmanager.smithi022 smithi022 *:9093,9094 running (14s) 8s ago 11m 15.6M - 0.25.0 c8568f914cd2 c89bc76be374 2024-02-16T19:31:52.378 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-exporter.smithi022 smithi022 running (11m) 8s ago 11m 7876k - 18.2.0 dc2bc1663786 3f8bef4349ba 2024-02-16T19:31:52.378 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-exporter.smithi117 smithi117 running (10m) 45s ago 10m 7507k - 18.2.0 dc2bc1663786 59b65a7e309f 2024-02-16T19:31:52.378 INFO:teuthology.orchestra.run.smithi022.stdout:crash.smithi022 smithi022 running (11m) 8s ago 11m 6669k - 18.2.0 dc2bc1663786 efef476aa486 2024-02-16T19:31:52.378 INFO:teuthology.orchestra.run.smithi022.stdout:crash.smithi117 smithi117 running (10m) 45s ago 10m 6660k - 18.2.0 dc2bc1663786 5466453eb149 2024-02-16T19:31:52.378 INFO:teuthology.orchestra.run.smithi022.stdout:grafana.smithi022 smithi022 *:3000 running (9m) 8s ago 10m 83.6M - 9.4.7 954c08fa6188 a48872ecbbf2 2024-02-16T19:31:52.378 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi022.btppwm smithi022 running (5m) 8s ago 5m 158M - 18.2.0 dc2bc1663786 22971fb3a907 2024-02-16T19:31:52.378 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi022.ocukvs smithi022 running (5m) 8s ago 5m 14.5M - 18.2.0 dc2bc1663786 81ee7bd5fd39 2024-02-16T19:31:52.378 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi117.ibmppn smithi117 running (5m) 45s ago 5m 16.1M - 18.2.0 dc2bc1663786 8c606cbeeb86 2024-02-16T19:31:52.378 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi117.orzors smithi117 running (5m) 45s ago 5m 15.1M - 18.2.0 dc2bc1663786 7d2bc4df3feb 2024-02-16T19:31:52.378 INFO:teuthology.orchestra.run.smithi022.stdout:mgr.smithi022.gmdcch smithi022 *:8443,9283,8765 running (2m) 8s ago 12m 497M - 19.0.0-1450-gf78a58c0 f081f847070a 7cb74cc9deea 2024-02-16T19:31:52.378 INFO:teuthology.orchestra.run.smithi022.stdout:mgr.smithi117.cohbtv smithi117 *:8443,9283,8765 running (71s) 45s ago 10m 450M - 19.0.0-1450-gf78a58c0 f081f847070a c825b1152189 2024-02-16T19:31:52.378 INFO:teuthology.orchestra.run.smithi022.stdout:mon.smithi022 smithi022 running (12m) 8s ago 12m 57.2M 2048M 18.2.0 dc2bc1663786 b4ffc3032e12 2024-02-16T19:31:52.378 INFO:teuthology.orchestra.run.smithi022.stdout:mon.smithi117 smithi117 running (9m) 45s ago 9m 44.6M 2048M 18.2.0 dc2bc1663786 aff3e21a73a0 2024-02-16T19:31:52.378 INFO:teuthology.orchestra.run.smithi022.stdout:node-exporter.smithi022 smithi022 *:9100 running (53s) 8s ago 11m 8702k - 1.5.0 0da6a335fe13 1f5b58292d27 2024-02-16T19:31:52.378 INFO:teuthology.orchestra.run.smithi022.stdout:node-exporter.smithi117 smithi117 *:9100 running (49s) 45s ago 10m 3275k - 1.5.0 0da6a335fe13 bb5e8491d5a6 2024-02-16T19:31:52.378 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0 smithi022 running (9m) 8s ago 9m 210M 1217M 18.2.0 dc2bc1663786 0027f86f16ce 2024-02-16T19:31:52.379 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1 smithi022 running (8m) 8s ago 8m 225M 1217M 18.2.0 dc2bc1663786 87300382b8a7 2024-02-16T19:31:52.379 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2 smithi022 running (8m) 8s ago 8m 185M 1217M 18.2.0 dc2bc1663786 be8a631b4f93 2024-02-16T19:31:52.379 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3 smithi117 running (7m) 45s ago 7m 242M 2240M 18.2.0 dc2bc1663786 9c0ce9a981cf 2024-02-16T19:31:52.379 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4 smithi117 running (7m) 45s ago 7m 208M 2240M 18.2.0 dc2bc1663786 1779c3496f4d 2024-02-16T19:31:52.379 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5 smithi117 running (6m) 45s ago 6m 194M 2240M 18.2.0 dc2bc1663786 fefaabca0638 2024-02-16T19:31:52.379 INFO:teuthology.orchestra.run.smithi022.stdout:prometheus.smithi022 smithi022 *:9095 running (31s) 8s ago 10m 46.4M - 2.43.0 a07b618ecd1d c3b8689d0c97 2024-02-16T19:31:52.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.378+0000 7f5829338700 1 -- 172.21.15.22:0/3089716765 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f580c06d520 msgr2=0x7f580c06f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:52.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.378+0000 7f5829338700 1 --2- 172.21.15.22:0/3089716765 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f580c06d520 0x7f580c06f9d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5814005fd0 tx=0x7f581400a560 comp rx=0 tx=0).stop 2024-02-16T19:31:52.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.378+0000 7f5829338700 1 -- 172.21.15.22:0/3089716765 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58240fee50 msgr2=0x7f5824196070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:52.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.378+0000 7f5829338700 1 --2- 172.21.15.22:0/3089716765 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58240fee50 0x7f5824196070 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f581800bd70 tx=0x7f581800be50 comp rx=0 tx=0).stop 2024-02-16T19:31:52.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.378+0000 7f5829338700 1 -- 172.21.15.22:0/3089716765 shutdown_connections 2024-02-16T19:31:52.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.378+0000 7f5829338700 1 --2- 172.21.15.22:0/3089716765 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f580c06d520 0x7f580c06f9d0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:52.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.378+0000 7f5829338700 1 --2- 172.21.15.22:0/3089716765 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58240fe500 0x7f5824195b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:52.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.378+0000 7f5829338700 1 --2- 172.21.15.22:0/3089716765 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58240fee50 0x7f5824196070 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:52.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.378+0000 7f5829338700 1 -- 172.21.15.22:0/3089716765 >> 172.21.15.22:0/3089716765 conn(0x7f58240f9f10 msgr2=0x7f58241047b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:31:52.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.379+0000 7f5829338700 1 -- 172.21.15.22:0/3089716765 shutdown_connections 2024-02-16T19:31:52.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.379+0000 7f5829338700 1 -- 172.21.15.22:0/3089716765 wait complete. 2024-02-16T19:31:52.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:52 smithi117 ceph-mon[35309]: pgmap v62: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:31:52.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.496+0000 7fa99806b700 1 -- 172.21.15.22:0/4255791395 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa990103600 msgr2=0x7fa9901059e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:52.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.496+0000 7fa99806b700 1 --2- 172.21.15.22:0/4255791395 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa990103600 0x7fa9901059e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa980009910 tx=0x7fa980009c20 comp rx=0 tx=0).stop 2024-02-16T19:31:52.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.496+0000 7fa99806b700 1 -- 172.21.15.22:0/4255791395 shutdown_connections 2024-02-16T19:31:52.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.496+0000 7fa99806b700 1 --2- 172.21.15.22:0/4255791395 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa990103600 0x7fa9901059e0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:52.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.496+0000 7fa99806b700 1 --2- 172.21.15.22:0/4255791395 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa990100ce0 0x7fa9901030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:52.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.496+0000 7fa99806b700 1 -- 172.21.15.22:0/4255791395 >> 172.21.15.22:0/4255791395 conn(0x7fa9900fa700 msgr2=0x7fa9900fcb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:31:52.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.496+0000 7fa99806b700 1 -- 172.21.15.22:0/4255791395 shutdown_connections 2024-02-16T19:31:52.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.496+0000 7fa99806b700 1 -- 172.21.15.22:0/4255791395 wait complete. 2024-02-16T19:31:52.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.497+0000 7fa99806b700 1 Processor -- start 2024-02-16T19:31:52.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.497+0000 7fa99806b700 1 -- start start 2024-02-16T19:31:52.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.497+0000 7fa99806b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa990100ce0 0x7fa9901005f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:31:52.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.497+0000 7fa99806b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa990103600 0x7fa9900fec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:31:52.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.497+0000 7fa99806b700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9900ff1d0 con 0x7fa990100ce0 2024-02-16T19:31:52.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.497+0000 7fa99806b700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9900ff310 con 0x7fa990103600 2024-02-16T19:31:52.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.497+0000 7fa995e07700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa990100ce0 0x7fa9901005f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:31:52.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.497+0000 7fa995e07700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa990100ce0 0x7fa9901005f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55484/0 (socket says 172.21.15.22:55484) 2024-02-16T19:31:52.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.497+0000 7fa995e07700 1 -- 172.21.15.22:0/411036178 learned_addr learned my addr 172.21.15.22:0/411036178 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:31:52.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.497+0000 7fa995606700 1 --2- 172.21.15.22:0/411036178 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa990103600 0x7fa9900fec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:31:52.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.498+0000 7fa995e07700 1 -- 172.21.15.22:0/411036178 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa990103600 msgr2=0x7fa9900fec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:52.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.498+0000 7fa995e07700 1 --2- 172.21.15.22:0/411036178 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa990103600 0x7fa9900fec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:52.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.498+0000 7fa995e07700 1 -- 172.21.15.22:0/411036178 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9800095f0 con 0x7fa990100ce0 2024-02-16T19:31:52.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.498+0000 7fa995e07700 1 --2- 172.21.15.22:0/411036178 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa990100ce0 0x7fa9901005f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fa98c00c8a0 tx=0x7fa98c00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:31:52.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.498+0000 7fa986ffd700 1 -- 172.21.15.22:0/411036178 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa98c0078e0 con 0x7fa990100ce0 2024-02-16T19:31:52.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.498+0000 7fa986ffd700 1 -- 172.21.15.22:0/411036178 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa98c00f450 con 0x7fa990100ce0 2024-02-16T19:31:52.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.498+0000 7fa99806b700 1 -- 172.21.15.22:0/411036178 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9900ff5f0 con 0x7fa990100ce0 2024-02-16T19:31:52.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.498+0000 7fa986ffd700 1 -- 172.21.15.22:0/411036178 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa98c00e5e0 con 0x7fa990100ce0 2024-02-16T19:31:52.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.498+0000 7fa99806b700 1 -- 172.21.15.22:0/411036178 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa99019db20 con 0x7fa990100ce0 2024-02-16T19:31:52.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.499+0000 7fa99806b700 1 -- 172.21.15.22:0/411036178 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa99018fc70 con 0x7fa990100ce0 2024-02-16T19:31:52.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.499+0000 7fa986ffd700 1 -- 172.21.15.22:0/411036178 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 33) v1 ==== 91346+0+0 (secure 0 0 0) 0x7fa98c00f5c0 con 0x7fa990100ce0 2024-02-16T19:31:52.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.500+0000 7fa986ffd700 1 --2- 172.21.15.22:0/411036178 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7fa97c06d520 0x7fa97c06f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:31:52.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.500+0000 7fa986ffd700 1 -- 172.21.15.22:0/411036178 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fa98c08cf80 con 0x7fa990100ce0 2024-02-16T19:31:52.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.500+0000 7fa995606700 1 --2- 172.21.15.22:0/411036178 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7fa97c06d520 0x7fa97c06f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:31:52.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.501+0000 7fa995606700 1 --2- 172.21.15.22:0/411036178 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7fa97c06d520 0x7fa97c06f9d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa980006010 tx=0x7fa98000b540 comp rx=0 tx=0).ready entity=mgr.24407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:31:52.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.505+0000 7fa986ffd700 1 -- 172.21.15.22:0/411036178 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7fa98c05a790 con 0x7fa990100ce0 2024-02-16T19:31:52.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.820+0000 7fa99806b700 1 -- 172.21.15.22:0/411036178 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa99002cc30 con 0x7fa990100ce0 2024-02-16T19:31:52.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.820+0000 7fa986ffd700 1 -- 172.21.15.22:0/411036178 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7fa98c05a320 con 0x7fa990100ce0 2024-02-16T19:31:52.822 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-02-16T19:31:52.822 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": { 2024-02-16T19:31:52.822 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-02-16T19:31:52.822 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:31:52.822 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": { 2024-02-16T19:31:52.822 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T19:31:52.822 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:31:52.822 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": { 2024-02-16T19:31:52.822 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-02-16T19:31:52.822 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:31:52.822 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": { 2024-02-16T19:31:52.822 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-02-16T19:31:52.823 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:31:52.823 INFO:teuthology.orchestra.run.smithi022.stdout: "overall": { 2024-02-16T19:31:52.823 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 12, 2024-02-16T19:31:52.823 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T19:31:52.823 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-02-16T19:31:52.823 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-02-16T19:31:52.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.823+0000 7fa99806b700 1 -- 172.21.15.22:0/411036178 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7fa97c06d520 msgr2=0x7fa97c06f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:52.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.823+0000 7fa99806b700 1 --2- 172.21.15.22:0/411036178 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7fa97c06d520 0x7fa97c06f9d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa980006010 tx=0x7fa98000b540 comp rx=0 tx=0).stop 2024-02-16T19:31:52.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.823+0000 7fa99806b700 1 -- 172.21.15.22:0/411036178 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa990100ce0 msgr2=0x7fa9901005f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:52.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.823+0000 7fa99806b700 1 --2- 172.21.15.22:0/411036178 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa990100ce0 0x7fa9901005f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fa98c00c8a0 tx=0x7fa98c00cbb0 comp rx=0 tx=0).stop 2024-02-16T19:31:52.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.823+0000 7fa99806b700 1 -- 172.21.15.22:0/411036178 shutdown_connections 2024-02-16T19:31:52.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.823+0000 7fa99806b700 1 --2- 172.21.15.22:0/411036178 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7fa97c06d520 0x7fa97c06f9d0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:52.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.823+0000 7fa99806b700 1 --2- 172.21.15.22:0/411036178 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa990100ce0 0x7fa9901005f0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:52.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.823+0000 7fa99806b700 1 --2- 172.21.15.22:0/411036178 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa990103600 0x7fa9900fec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:52.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.823+0000 7fa99806b700 1 -- 172.21.15.22:0/411036178 >> 172.21.15.22:0/411036178 conn(0x7fa9900fa700 msgr2=0x7fa9900fcb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:31:52.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.823+0000 7fa99806b700 1 -- 172.21.15.22:0/411036178 shutdown_connections 2024-02-16T19:31:52.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.823+0000 7fa99806b700 1 -- 172.21.15.22:0/411036178 wait complete. 2024-02-16T19:31:52.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.956+0000 7f6bf2751700 1 -- 172.21.15.22:0/679901877 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6bec100240 msgr2=0x7f6bec100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:52.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.956+0000 7f6bf2751700 1 --2- 172.21.15.22:0/679901877 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6bec100240 0x7f6bec100650 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc009910 tx=0x7f6bdc009c20 comp rx=0 tx=0).stop 2024-02-16T19:31:52.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.956+0000 7f6bf2751700 1 -- 172.21.15.22:0/679901877 shutdown_connections 2024-02-16T19:31:52.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.956+0000 7f6bf2751700 1 --2- 172.21.15.22:0/679901877 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bec101440 0x7f6bec101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:52.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.956+0000 7f6bf2751700 1 --2- 172.21.15.22:0/679901877 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6bec100240 0x7f6bec100650 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:52.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.956+0000 7f6bf2751700 1 -- 172.21.15.22:0/679901877 >> 172.21.15.22:0/679901877 conn(0x7f6bec0fb7f0 msgr2=0x7f6bec0fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:31:52.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.957+0000 7f6bf2751700 1 -- 172.21.15.22:0/679901877 shutdown_connections 2024-02-16T19:31:52.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.957+0000 7f6bf2751700 1 -- 172.21.15.22:0/679901877 wait complete. 2024-02-16T19:31:52.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.957+0000 7f6bf2751700 1 Processor -- start 2024-02-16T19:31:52.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.957+0000 7f6bf2751700 1 -- start start 2024-02-16T19:31:52.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.958+0000 7f6bf2751700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6bec100240 0x7f6bec195ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:31:52.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.958+0000 7f6bf2751700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bec101440 0x7f6bec196010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:31:52.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.958+0000 7f6bf2751700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bec1965a0 con 0x7f6bec100240 2024-02-16T19:31:52.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.958+0000 7f6bf2751700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bec1966e0 con 0x7f6bec101440 2024-02-16T19:31:52.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.958+0000 7f6bebfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6bec100240 0x7f6bec195ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:31:52.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.958+0000 7f6bebfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6bec100240 0x7f6bec195ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55498/0 (socket says 172.21.15.22:55498) 2024-02-16T19:31:52.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.958+0000 7f6bebfff700 1 -- 172.21.15.22:0/2835499199 learned_addr learned my addr 172.21.15.22:0/2835499199 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:31:52.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.958+0000 7f6beb7fe700 1 --2- 172.21.15.22:0/2835499199 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bec101440 0x7f6bec196010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:31:52.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.958+0000 7f6bebfff700 1 -- 172.21.15.22:0/2835499199 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bec101440 msgr2=0x7f6bec196010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:52.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.958+0000 7f6bebfff700 1 --2- 172.21.15.22:0/2835499199 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bec101440 0x7f6bec196010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:52.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.958+0000 7f6bebfff700 1 -- 172.21.15.22:0/2835499199 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bdc0095f0 con 0x7f6bec100240 2024-02-16T19:31:52.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.958+0000 7f6bebfff700 1 --2- 172.21.15.22:0/2835499199 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6bec100240 0x7f6bec195ad0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc00bda0 tx=0x7f6bdc00bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:31:52.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.959+0000 7f6be97fa700 1 -- 172.21.15.22:0/2835499199 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6bdc01d070 con 0x7f6bec100240 2024-02-16T19:31:52.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.959+0000 7f6be97fa700 1 -- 172.21.15.22:0/2835499199 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6bdc022470 con 0x7f6bec100240 2024-02-16T19:31:52.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.959+0000 7f6bf2751700 1 -- 172.21.15.22:0/2835499199 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bec19b140 con 0x7f6bec100240 2024-02-16T19:31:52.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.959+0000 7f6be97fa700 1 -- 172.21.15.22:0/2835499199 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6bdc00f670 con 0x7f6bec100240 2024-02-16T19:31:52.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.959+0000 7f6bf2751700 1 -- 172.21.15.22:0/2835499199 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bec19b600 con 0x7f6bec100240 2024-02-16T19:31:52.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.960+0000 7f6bf2751700 1 -- 172.21.15.22:0/2835499199 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bec108d90 con 0x7f6bec100240 2024-02-16T19:31:52.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.960+0000 7f6be97fa700 1 -- 172.21.15.22:0/2835499199 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 33) v1 ==== 91346+0+0 (secure 0 0 0) 0x7f6bdc004af0 con 0x7f6bec100240 2024-02-16T19:31:52.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.961+0000 7f6be97fa700 1 --2- 172.21.15.22:0/2835499199 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f6bd406d520 0x7f6bd406f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:31:52.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.961+0000 7f6be97fa700 1 -- 172.21.15.22:0/2835499199 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f6bdc08e560 con 0x7f6bec100240 2024-02-16T19:31:52.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.961+0000 7f6beb7fe700 1 --2- 172.21.15.22:0/2835499199 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f6bd406d520 0x7f6bd406f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:31:52.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.961+0000 7f6beb7fe700 1 --2- 172.21.15.22:0/2835499199 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f6bd406d520 0x7f6bd406f9d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6be000ac50 tx=0x7f6be000a380 comp rx=0 tx=0).ready entity=mgr.24407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:31:52.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:52.966+0000 7f6be97fa700 1 -- 172.21.15.22:0/2835499199 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f6bdc05be20 con 0x7f6bec100240 2024-02-16T19:31:53.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:53.196+0000 7f6bf2751700 1 -- 172.21.15.22:0/2835499199 --> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6bec19bc40 con 0x7f6bd406d520 2024-02-16T19:31:53.198 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:53 smithi022 ceph-mon[29221]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:31:53.198 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:53 smithi022 ceph-mon[29221]: from='client.14764 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:31:53.198 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:53 smithi022 ceph-mon[29221]: from='client.14768 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:31:53.198 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:53 smithi022 ceph-mon[29221]: from='client.? 172.21.15.22:0/411036178' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:53.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:53.197+0000 7f6be97fa700 1 -- 172.21.15.22:0/2835499199 <== mgr.24407 v2:172.21.15.22:6800/3703806543 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7f6bec19bc40 con 0x7f6bd406d520 2024-02-16T19:31:53.199 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-02-16T19:31:53.199 INFO:teuthology.orchestra.run.smithi022.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T19:31:53.199 INFO:teuthology.orchestra.run.smithi022.stdout: "in_progress": true, 2024-02-16T19:31:53.199 INFO:teuthology.orchestra.run.smithi022.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-02-16T19:31:53.199 INFO:teuthology.orchestra.run.smithi022.stdout: "services_complete": [ 2024-02-16T19:31:53.199 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr" 2024-02-16T19:31:53.199 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-02-16T19:31:53.199 INFO:teuthology.orchestra.run.smithi022.stdout: "progress": "2/2 daemons upgraded", 2024-02-16T19:31:53.199 INFO:teuthology.orchestra.run.smithi022.stdout: "message": "Currently upgrading grafana daemons", 2024-02-16T19:31:53.199 INFO:teuthology.orchestra.run.smithi022.stdout: "is_paused": false 2024-02-16T19:31:53.199 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-02-16T19:31:53.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:53.200+0000 7f6bf2751700 1 -- 172.21.15.22:0/2835499199 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f6bd406d520 msgr2=0x7f6bd406f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:53.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:53.200+0000 7f6bf2751700 1 --2- 172.21.15.22:0/2835499199 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f6bd406d520 0x7f6bd406f9d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6be000ac50 tx=0x7f6be000a380 comp rx=0 tx=0).stop 2024-02-16T19:31:53.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:53.200+0000 7f6bf2751700 1 -- 172.21.15.22:0/2835499199 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6bec100240 msgr2=0x7f6bec195ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:53.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:53.200+0000 7f6bf2751700 1 --2- 172.21.15.22:0/2835499199 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6bec100240 0x7f6bec195ad0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc00bda0 tx=0x7f6bdc00bdd0 comp rx=0 tx=0).stop 2024-02-16T19:31:53.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:53.201+0000 7f6bf2751700 1 -- 172.21.15.22:0/2835499199 shutdown_connections 2024-02-16T19:31:53.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:53.201+0000 7f6bf2751700 1 --2- 172.21.15.22:0/2835499199 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f6bd406d520 0x7f6bd406f9d0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:53.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:53.201+0000 7f6bf2751700 1 --2- 172.21.15.22:0/2835499199 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6bec100240 0x7f6bec195ad0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:53.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:53.201+0000 7f6bf2751700 1 --2- 172.21.15.22:0/2835499199 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bec101440 0x7f6bec196010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:53.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:53.201+0000 7f6bf2751700 1 -- 172.21.15.22:0/2835499199 >> 172.21.15.22:0/2835499199 conn(0x7f6bec0fb7f0 msgr2=0x7f6bec104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:31:53.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:53.201+0000 7f6bf2751700 1 -- 172.21.15.22:0/2835499199 shutdown_connections 2024-02-16T19:31:53.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:31:53.201+0000 7f6bf2751700 1 -- 172.21.15.22:0/2835499199 wait complete. 2024-02-16T19:31:53.288 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:53 smithi117 ceph-mon[35309]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:31:53.288 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:53 smithi117 ceph-mon[35309]: from='client.14764 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:31:53.288 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:53 smithi117 ceph-mon[35309]: from='client.14768 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:31:53.288 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:53 smithi117 ceph-mon[35309]: from='client.? 172.21.15.22:0/411036178' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:54.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:54 smithi117 ceph-mon[35309]: from='client.14776 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:31:54.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:54 smithi117 ceph-mon[35309]: pgmap v63: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:31:54.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:54 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:31:54.464 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:54 smithi022 ceph-mon[29221]: from='client.14776 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:31:54.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:54 smithi022 ceph-mon[29221]: pgmap v63: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:31:54.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:54 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:31:56.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:56 smithi117 ceph-mon[35309]: pgmap v64: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:31:56.714 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:56 smithi022 ceph-mon[29221]: pgmap v64: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:31:58.214 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:31:57 smithi022 ceph-mon[29221]: pgmap v65: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:31:58.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:31:57 smithi117 ceph-mon[35309]: pgmap v65: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:32:00.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:00 smithi022 ceph-mon[29221]: pgmap v66: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:32:00.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:00 smithi117 ceph-mon[35309]: pgmap v66: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:32:02.515 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:02 smithi022 ceph-mon[29221]: pgmap v67: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:32:02.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:02 smithi117 ceph-mon[35309]: pgmap v67: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:32:04.430 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:04 smithi022 ceph-mon[29221]: pgmap v68: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:32:04.549 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:04 smithi117 ceph-mon[35309]: pgmap v68: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:32:05.840 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:05 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:32:05.840 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:05 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:32:05.840 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:05 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:32:05.840 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:05 smithi022 ceph-mon[29221]: pgmap v69: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:32:06.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:05 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:32:06.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:05 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:32:06.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:05 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:32:06.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:05 smithi117 ceph-mon[35309]: pgmap v69: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:32:08.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:07 smithi022 ceph-mon[29221]: pgmap v70: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:32:08.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:07 smithi117 ceph-mon[35309]: pgmap v70: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:32:09.189 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:08 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:32:09.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:08 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:32:10.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:09 smithi022 ceph-mon[29221]: pgmap v71: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:32:10.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:09 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:32:10.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:09 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:32:10.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:09 smithi117 ceph-mon[35309]: pgmap v71: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:32:10.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:09 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:32:10.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:09 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:32:12.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:12 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:32:12.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:12 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:32:12.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:12 smithi117 ceph-mon[35309]: pgmap v72: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:32:12.464 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:12 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:32:12.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:12 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:32:12.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:12 smithi022 ceph-mon[29221]: pgmap v72: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:32:14.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:14 smithi022 ceph-mon[29221]: pgmap v73: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:32:14.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:14 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:32:14.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:14 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:32:14.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:14 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:32:14.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:14 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:32:14.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:14 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:32:14.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:14 smithi117 ceph-mon[35309]: pgmap v73: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:32:14.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:14 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:32:14.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:14 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:32:14.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:14 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:32:14.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:14 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:32:14.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:14 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:32:15.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:15 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-16T19:32:15.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:15 smithi117 ceph-mon[35309]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-16T19:32:15.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:15 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:32:15.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:15 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:32:15.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:15 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:32:15.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:15 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:32:15.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:15 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:32:15.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:15 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:32:15.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:15 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:32:15.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:15 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:32:15.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:15 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:32:15.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:15 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:32:15.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:15 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:32:15.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:15 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:32:15.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:15 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:32:15.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:15 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:32:15.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:15 smithi117 ceph-mon[35309]: Upgrade: Finalizing container_image settings 2024-02-16T19:32:15.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:15 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:32:15.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:15 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-02-16T19:32:15.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:15 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-02-16T19:32:15.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:15 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T19:32:15.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:15 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-02-16T19:32:15.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:15 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-02-16T19:32:15.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:15 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-02-16T19:32:15.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:15 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-02-16T19:32:15.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:15 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-02-16T19:32:15.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:15 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-02-16T19:32:15.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:15 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-02-16T19:32:15.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:15 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-02-16T19:32:15.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:15 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T19:32:15.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:15 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-02-16T19:32:15.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:15 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-02-16T19:32:15.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:15 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-02-16T19:32:15.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:15 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-02-16T19:32:15.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:15 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-02-16T19:32:15.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:15 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-02-16T19:32:15.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:15 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-02-16T19:32:15.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:15 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T19:32:15.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:15 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T19:32:15.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:15 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T19:32:15.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:15 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T19:32:15.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:15 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T19:32:15.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:15 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T19:32:15.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:15 smithi117 ceph-mon[35309]: Upgrade: Complete! 2024-02-16T19:32:15.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:15 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-02-16T19:32:15.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:15 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-02-16T19:32:15.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:15 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:32:15.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:15 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:32:15.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:15 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:32:15.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:15 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:32:15.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:15 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:32:15.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:15 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:32:15.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:15 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:32:15.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:15 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:32:15.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:15 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-16T19:32:15.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:15 smithi022 ceph-mon[29221]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-16T19:32:15.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:15 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:32:15.716 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:15 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:32:15.716 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:15 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:32:15.716 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:15 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:32:15.716 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:15 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:32:15.716 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:15 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:32:15.716 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:15 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:32:15.716 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:15 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:32:15.716 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:15 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:32:15.716 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:15 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:32:15.716 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:15 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:32:15.716 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:15 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:32:15.716 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:15 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:32:15.716 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:15 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:32:15.717 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:15 smithi022 ceph-mon[29221]: Upgrade: Finalizing container_image settings 2024-02-16T19:32:15.717 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:15 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:32:15.717 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:15 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-02-16T19:32:15.717 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:15 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-02-16T19:32:15.717 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:15 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T19:32:15.717 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:15 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-02-16T19:32:15.717 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:15 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-02-16T19:32:15.717 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:15 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-02-16T19:32:15.717 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:15 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-02-16T19:32:15.717 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:15 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-02-16T19:32:15.717 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:15 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-02-16T19:32:15.717 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:15 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-02-16T19:32:15.717 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:15 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-02-16T19:32:15.717 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:15 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T19:32:15.718 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:15 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-02-16T19:32:15.718 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:15 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-02-16T19:32:15.718 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:15 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-02-16T19:32:15.718 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:15 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-02-16T19:32:15.718 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:15 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-02-16T19:32:15.718 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:15 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-02-16T19:32:15.718 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:15 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-02-16T19:32:15.718 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:15 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T19:32:15.718 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:15 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T19:32:15.718 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:15 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T19:32:15.718 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:15 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T19:32:15.718 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:15 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T19:32:15.718 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:15 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T19:32:15.718 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:15 smithi022 ceph-mon[29221]: Upgrade: Complete! 2024-02-16T19:32:15.718 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:15 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-02-16T19:32:15.718 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:15 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-02-16T19:32:15.719 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:15 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:32:15.719 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:15 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:32:15.719 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:15 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:32:15.719 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:15 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:32:15.719 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:15 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:32:15.719 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:15 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:32:15.719 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:15 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:32:15.719 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:15 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:32:16.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:16 smithi117 ceph-mon[35309]: pgmap v74: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:32:16.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:16 smithi022 ceph-mon[29221]: pgmap v74: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:32:18.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:17 smithi022 ceph-mon[29221]: pgmap v75: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:32:18.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:17 smithi117 ceph-mon[35309]: pgmap v75: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:32:19.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:19 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:32:19.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:19 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:32:20.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:20 smithi117 ceph-mon[35309]: pgmap v76: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:32:20.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:20 smithi022 ceph-mon[29221]: pgmap v76: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:32:22.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:22 smithi117 ceph-mon[35309]: pgmap v77: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:32:22.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:22 smithi022 ceph-mon[29221]: pgmap v77: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:32:23.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:23.323+0000 7fd084940700 1 -- 172.21.15.22:0/3605002064 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0800ff130 msgr2=0x7fd0800ff540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:23.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:23.323+0000 7fd084940700 1 --2- 172.21.15.22:0/3605002064 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0800ff130 0x7fd0800ff540 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd070009910 tx=0x7fd070009c20 comp rx=0 tx=0).stop 2024-02-16T19:32:23.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:23.323+0000 7fd084940700 1 -- 172.21.15.22:0/3605002064 shutdown_connections 2024-02-16T19:32:23.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:23.323+0000 7fd084940700 1 --2- 172.21.15.22:0/3605002064 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0801003d0 0x7fd080100840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:23.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:23.323+0000 7fd084940700 1 --2- 172.21.15.22:0/3605002064 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0800ff130 0x7fd0800ff540 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:23.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:23.323+0000 7fd084940700 1 -- 172.21.15.22:0/3605002064 >> 172.21.15.22:0/3605002064 conn(0x7fd0800fa780 msgr2=0x7fd0800fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:32:23.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:23.323+0000 7fd084940700 1 -- 172.21.15.22:0/3605002064 shutdown_connections 2024-02-16T19:32:23.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:23.323+0000 7fd084940700 1 -- 172.21.15.22:0/3605002064 wait complete. 2024-02-16T19:32:23.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:23.324+0000 7fd084940700 1 Processor -- start 2024-02-16T19:32:23.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:23.324+0000 7fd084940700 1 -- start start 2024-02-16T19:32:23.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:23.324+0000 7fd084940700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0800ff130 0x7fd080103160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:32:23.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:23.324+0000 7fd084940700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0801003d0 0x7fd0801017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:32:23.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:23.324+0000 7fd084940700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd080101cf0 con 0x7fd0801003d0 2024-02-16T19:32:23.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:23.324+0000 7fd084940700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd080101e30 con 0x7fd0800ff130 2024-02-16T19:32:23.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:23.324+0000 7fd07dd9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0801003d0 0x7fd0801017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:32:23.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:23.324+0000 7fd07dd9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0801003d0 0x7fd0801017b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41208/0 (socket says 172.21.15.22:41208) 2024-02-16T19:32:23.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:23.324+0000 7fd07dd9b700 1 -- 172.21.15.22:0/3526186292 learned_addr learned my addr 172.21.15.22:0/3526186292 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:32:23.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:23.324+0000 7fd07e59c700 1 --2- 172.21.15.22:0/3526186292 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0800ff130 0x7fd080103160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:32:23.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:23.325+0000 7fd07dd9b700 1 -- 172.21.15.22:0/3526186292 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0800ff130 msgr2=0x7fd080103160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:23.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:23.325+0000 7fd07dd9b700 1 --2- 172.21.15.22:0/3526186292 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0800ff130 0x7fd080103160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:23.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:23.325+0000 7fd07dd9b700 1 -- 172.21.15.22:0/3526186292 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0700095f0 con 0x7fd0801003d0 2024-02-16T19:32:23.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:23.325+0000 7fd07dd9b700 1 --2- 172.21.15.22:0/3526186292 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0801003d0 0x7fd0801017b0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd07400d8d0 tx=0x7fd07400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:32:23.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:23.325+0000 7fd06f7fe700 1 -- 172.21.15.22:0/3526186292 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd0740098a0 con 0x7fd0801003d0 2024-02-16T19:32:23.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:23.325+0000 7fd06f7fe700 1 -- 172.21.15.22:0/3526186292 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd074010460 con 0x7fd0801003d0 2024-02-16T19:32:23.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:23.325+0000 7fd084940700 1 -- 172.21.15.22:0/3526186292 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd080102110 con 0x7fd0801003d0 2024-02-16T19:32:23.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:23.325+0000 7fd06f7fe700 1 -- 172.21.15.22:0/3526186292 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd07400f5f0 con 0x7fd0801003d0 2024-02-16T19:32:23.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:23.325+0000 7fd084940700 1 -- 172.21.15.22:0/3526186292 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd08019db00 con 0x7fd0801003d0 2024-02-16T19:32:23.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:23.326+0000 7fd084940700 1 -- 172.21.15.22:0/3526186292 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd080107c10 con 0x7fd0801003d0 2024-02-16T19:32:23.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:23.327+0000 7fd06f7fe700 1 -- 172.21.15.22:0/3526186292 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 33) v1 ==== 91346+0+0 (secure 0 0 0) 0x7fd074009a00 con 0x7fd0801003d0 2024-02-16T19:32:23.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:23.327+0000 7fd06f7fe700 1 --2- 172.21.15.22:0/3526186292 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7fd06806d520 0x7fd06806f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:32:23.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:23.327+0000 7fd06f7fe700 1 -- 172.21.15.22:0/3526186292 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fd07408cd80 con 0x7fd0801003d0 2024-02-16T19:32:23.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:23.327+0000 7fd07e59c700 1 --2- 172.21.15.22:0/3526186292 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7fd06806d520 0x7fd06806f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:32:23.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:23.328+0000 7fd07e59c700 1 --2- 172.21.15.22:0/3526186292 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7fd06806d520 0x7fd06806f9d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd070006010 tx=0x7fd07000b540 comp rx=0 tx=0).ready entity=mgr.24407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:32:23.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:23.332+0000 7fd06f7fe700 1 -- 172.21.15.22:0/3526186292 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7fd07405a610 con 0x7fd0801003d0 2024-02-16T19:32:23.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:23.560+0000 7fd084940700 1 -- 172.21.15.22:0/3526186292 --> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd080068c50 con 0x7fd06806d520 2024-02-16T19:32:23.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:23.561+0000 7fd06f7fe700 1 -- 172.21.15.22:0/3526186292 <== mgr.24407 v2:172.21.15.22:6800/3703806543 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7fd080068c50 con 0x7fd06806d520 2024-02-16T19:32:23.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:23.563+0000 7fd084940700 1 -- 172.21.15.22:0/3526186292 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7fd06806d520 msgr2=0x7fd06806f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:23.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:23.563+0000 7fd084940700 1 --2- 172.21.15.22:0/3526186292 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7fd06806d520 0x7fd06806f9d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd070006010 tx=0x7fd07000b540 comp rx=0 tx=0).stop 2024-02-16T19:32:23.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:23.563+0000 7fd084940700 1 -- 172.21.15.22:0/3526186292 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0801003d0 msgr2=0x7fd0801017b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:23.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:23.563+0000 7fd084940700 1 --2- 172.21.15.22:0/3526186292 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0801003d0 0x7fd0801017b0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd07400d8d0 tx=0x7fd07400dbe0 comp rx=0 tx=0).stop 2024-02-16T19:32:23.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:23.563+0000 7fd084940700 1 -- 172.21.15.22:0/3526186292 shutdown_connections 2024-02-16T19:32:23.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:23.563+0000 7fd084940700 1 --2- 172.21.15.22:0/3526186292 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7fd06806d520 0x7fd06806f9d0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:23.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:23.563+0000 7fd084940700 1 --2- 172.21.15.22:0/3526186292 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0800ff130 0x7fd080103160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:23.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:23.563+0000 7fd084940700 1 --2- 172.21.15.22:0/3526186292 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0801003d0 0x7fd0801017b0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:23.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:23.563+0000 7fd084940700 1 -- 172.21.15.22:0/3526186292 >> 172.21.15.22:0/3526186292 conn(0x7fd0800fa780 msgr2=0x7fd080068540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:32:23.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:23.563+0000 7fd084940700 1 -- 172.21.15.22:0/3526186292 shutdown_connections 2024-02-16T19:32:23.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:23.563+0000 7fd084940700 1 -- 172.21.15.22:0/3526186292 wait complete. 2024-02-16T19:32:23.576 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:23 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:32:23.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:23 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:32:24.217 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph versions | jq -e '"'"'.mgr | length == 1'"'"'' 2024-02-16T19:32:24.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:24 smithi022 ceph-mon[29221]: pgmap v78: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:32:24.475 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi022/config 2024-02-16T19:32:24.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:24 smithi117 ceph-mon[35309]: pgmap v78: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:32:25.537 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:25 smithi022 ceph-mon[29221]: from='client.14780 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:32:25.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:25 smithi117 ceph-mon[35309]: from='client.14780 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:32:26.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:26.072+0000 7f541bc6f700 1 -- 172.21.15.22:0/2338377247 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5414073a00 msgr2=0x7f541410c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:26.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:26.072+0000 7f541bc6f700 1 --2- 172.21.15.22:0/2338377247 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5414073a00 0x7f541410c580 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f5408009910 tx=0x7f5408009c20 comp rx=0 tx=0).stop 2024-02-16T19:32:26.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:26.072+0000 7f541bc6f700 1 -- 172.21.15.22:0/2338377247 shutdown_connections 2024-02-16T19:32:26.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:26.072+0000 7f541bc6f700 1 --2- 172.21.15.22:0/2338377247 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5414073a00 0x7f541410c580 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:26.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:26.072+0000 7f541bc6f700 1 --2- 172.21.15.22:0/2338377247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54140730f0 0x7f54140734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:26.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:26.072+0000 7f541bc6f700 1 -- 172.21.15.22:0/2338377247 >> 172.21.15.22:0/2338377247 conn(0x7f54140fbda0 msgr2=0x7f54140fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:32:26.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:26.072+0000 7f541bc6f700 1 -- 172.21.15.22:0/2338377247 shutdown_connections 2024-02-16T19:32:26.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:26.072+0000 7f541bc6f700 1 -- 172.21.15.22:0/2338377247 wait complete. 2024-02-16T19:32:26.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:26.073+0000 7f541bc6f700 1 Processor -- start 2024-02-16T19:32:26.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:26.073+0000 7f541bc6f700 1 -- start start 2024-02-16T19:32:26.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:26.073+0000 7f541bc6f700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54140730f0 0x7f5414198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:32:26.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:26.073+0000 7f541bc6f700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5414073a00 0x7f5414198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:32:26.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:26.073+0000 7f541bc6f700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f541419cbc0 con 0x7f5414073a00 2024-02-16T19:32:26.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:26.073+0000 7f541bc6f700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f541419cd30 con 0x7f54140730f0 2024-02-16T19:32:26.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:26.073+0000 7f541920a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5414073a00 0x7f5414198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:32:26.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:26.074+0000 7f541920a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5414073a00 0x7f5414198880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34280/0 (socket says 172.21.15.22:34280) 2024-02-16T19:32:26.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:26.074+0000 7f541920a700 1 -- 172.21.15.22:0/1860051187 learned_addr learned my addr 172.21.15.22:0/1860051187 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:32:26.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:26.074+0000 7f5419a0b700 1 --2- 172.21.15.22:0/1860051187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54140730f0 0x7f5414198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:32:26.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:26.074+0000 7f541920a700 1 -- 172.21.15.22:0/1860051187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54140730f0 msgr2=0x7f5414198210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:26.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:26.074+0000 7f541920a700 1 --2- 172.21.15.22:0/1860051187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54140730f0 0x7f5414198210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:26.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:26.074+0000 7f541920a700 1 -- 172.21.15.22:0/1860051187 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54080095f0 con 0x7f5414073a00 2024-02-16T19:32:26.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:26.074+0000 7f5419a0b700 1 --2- 172.21.15.22:0/1860051187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54140730f0 0x7f5414198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T19:32:26.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:26.074+0000 7f541920a700 1 --2- 172.21.15.22:0/1860051187 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5414073a00 0x7f5414198880 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f5408000c00 tx=0x7f540800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:32:26.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:26.074+0000 7f5412ffd700 1 -- 172.21.15.22:0/1860051187 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f540801d070 con 0x7f5414073a00 2024-02-16T19:32:26.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:26.074+0000 7f5412ffd700 1 -- 172.21.15.22:0/1860051187 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f540800f460 con 0x7f5414073a00 2024-02-16T19:32:26.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:26.074+0000 7f541bc6f700 1 -- 172.21.15.22:0/1860051187 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f541419d010 con 0x7f5414073a00 2024-02-16T19:32:26.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:26.074+0000 7f5412ffd700 1 -- 172.21.15.22:0/1860051187 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5408017630 con 0x7f5414073a00 2024-02-16T19:32:26.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:26.074+0000 7f541bc6f700 1 -- 172.21.15.22:0/1860051187 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f541419d560 con 0x7f5414073a00 2024-02-16T19:32:26.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:26.075+0000 7f541bc6f700 1 -- 172.21.15.22:0/1860051187 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5414109c80 con 0x7f5414073a00 2024-02-16T19:32:26.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:26.076+0000 7f5412ffd700 1 -- 172.21.15.22:0/1860051187 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 33) v1 ==== 91346+0+0 (secure 0 0 0) 0x7f540800f5d0 con 0x7f5414073a00 2024-02-16T19:32:26.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:26.076+0000 7f5412ffd700 1 --2- 172.21.15.22:0/1860051187 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f53fc06d520 0x7f53fc06f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:32:26.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:26.076+0000 7f5412ffd700 1 -- 172.21.15.22:0/1860051187 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f540808e560 con 0x7f5414073a00 2024-02-16T19:32:26.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:26.076+0000 7f5419a0b700 1 --2- 172.21.15.22:0/1860051187 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f53fc06d520 0x7f53fc06f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:32:26.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:26.077+0000 7f5419a0b700 1 --2- 172.21.15.22:0/1860051187 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f53fc06d520 0x7f53fc06f9d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5404009fa0 tx=0x7f5404009570 comp rx=0 tx=0).ready entity=mgr.24407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:32:26.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:26.081+0000 7f5412ffd700 1 -- 172.21.15.22:0/1860051187 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f5408058240 con 0x7f5414073a00 2024-02-16T19:32:26.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:26.375+0000 7f541bc6f700 1 -- 172.21.15.22:0/1860051187 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f5414199570 con 0x7f5414073a00 2024-02-16T19:32:26.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:26.376+0000 7f5412ffd700 1 -- 172.21.15.22:0/1860051187 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7f540805b900 con 0x7f5414073a00 2024-02-16T19:32:26.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:26.378+0000 7f541bc6f700 1 -- 172.21.15.22:0/1860051187 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f53fc06d520 msgr2=0x7f53fc06f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:26.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:26.378+0000 7f541bc6f700 1 --2- 172.21.15.22:0/1860051187 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f53fc06d520 0x7f53fc06f9d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5404009fa0 tx=0x7f5404009570 comp rx=0 tx=0).stop 2024-02-16T19:32:26.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:26.378+0000 7f541bc6f700 1 -- 172.21.15.22:0/1860051187 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5414073a00 msgr2=0x7f5414198880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:26.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:26.378+0000 7f541bc6f700 1 --2- 172.21.15.22:0/1860051187 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5414073a00 0x7f5414198880 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f5408000c00 tx=0x7f540800be20 comp rx=0 tx=0).stop 2024-02-16T19:32:26.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:26.378+0000 7f541bc6f700 1 -- 172.21.15.22:0/1860051187 shutdown_connections 2024-02-16T19:32:26.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:26.378+0000 7f541bc6f700 1 --2- 172.21.15.22:0/1860051187 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f53fc06d520 0x7f53fc06f9d0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:26.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:26.378+0000 7f541bc6f700 1 --2- 172.21.15.22:0/1860051187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54140730f0 0x7f5414198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:26.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:26.378+0000 7f541bc6f700 1 --2- 172.21.15.22:0/1860051187 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5414073a00 0x7f5414198880 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:26.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:26.378+0000 7f541bc6f700 1 -- 172.21.15.22:0/1860051187 >> 172.21.15.22:0/1860051187 conn(0x7f54140fbda0 msgr2=0x7f5414106dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:32:26.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:26.378+0000 7f541bc6f700 1 -- 172.21.15.22:0/1860051187 shutdown_connections 2024-02-16T19:32:26.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:26.378+0000 7f541bc6f700 1 -- 172.21.15.22:0/1860051187 wait complete. 2024-02-16T19:32:26.390 INFO:teuthology.orchestra.run.smithi022.stdout:true 2024-02-16T19:32:26.617 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:26 smithi117 ceph-mon[35309]: pgmap v79: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:32:26.678 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:26 smithi022 ceph-mon[29221]: pgmap v79: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:32:26.968 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph versions | jq -e '"'"'.mgr | keys'"'"' | grep $sha1' 2024-02-16T19:32:27.183 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi022/config 2024-02-16T19:32:27.556 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:27 smithi022 ceph-mon[29221]: from='client.? 172.21.15.22:0/1860051187' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:32:27.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:27 smithi117 ceph-mon[35309]: from='client.? 172.21.15.22:0/1860051187' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:32:28.441 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:28 smithi022 ceph-mon[29221]: pgmap v80: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:32:28.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:28 smithi117 ceph-mon[35309]: pgmap v80: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:32:28.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:28.663+0000 7fd6dc507700 1 -- 172.21.15.22:0/1380243398 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd6d41004e0 msgr2=0x7fd6d41008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:28.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:28.663+0000 7fd6dc507700 1 --2- 172.21.15.22:0/1380243398 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd6d41004e0 0x7fd6d41008b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fd6c400b0a0 tx=0x7fd6c400b3b0 comp rx=0 tx=0).stop 2024-02-16T19:32:28.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:28.664+0000 7fd6dc507700 1 -- 172.21.15.22:0/1380243398 shutdown_connections 2024-02-16T19:32:28.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:28.664+0000 7fd6dc507700 1 --2- 172.21.15.22:0/1380243398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6d4100e80 0x7fd6d4104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:28.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:28.664+0000 7fd6dc507700 1 --2- 172.21.15.22:0/1380243398 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd6d41004e0 0x7fd6d41008b0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:28.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:28.664+0000 7fd6dc507700 1 -- 172.21.15.22:0/1380243398 >> 172.21.15.22:0/1380243398 conn(0x7fd6d40fbda0 msgr2=0x7fd6d40fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:32:28.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:28.664+0000 7fd6dc507700 1 -- 172.21.15.22:0/1380243398 shutdown_connections 2024-02-16T19:32:28.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:28.664+0000 7fd6dc507700 1 -- 172.21.15.22:0/1380243398 wait complete. 2024-02-16T19:32:28.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:28.664+0000 7fd6dc507700 1 Processor -- start 2024-02-16T19:32:28.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:28.664+0000 7fd6dc507700 1 -- start start 2024-02-16T19:32:28.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:28.664+0000 7fd6dc507700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd6d41004e0 0x7fd6d4073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:32:28.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:28.664+0000 7fd6dc507700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6d4100e80 0x7fd6d4073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:32:28.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:28.664+0000 7fd6dc507700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d4073ad0 con 0x7fd6d41004e0 2024-02-16T19:32:28.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:28.664+0000 7fd6dc507700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d4073c10 con 0x7fd6d4100e80 2024-02-16T19:32:28.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:28.664+0000 7fd6da2a3700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd6d41004e0 0x7fd6d4073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:32:28.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:28.665+0000 7fd6da2a3700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd6d41004e0 0x7fd6d4073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34292/0 (socket says 172.21.15.22:34292) 2024-02-16T19:32:28.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:28.665+0000 7fd6da2a3700 1 -- 172.21.15.22:0/352445505 learned_addr learned my addr 172.21.15.22:0/352445505 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:32:28.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:28.665+0000 7fd6d9aa2700 1 --2- 172.21.15.22:0/352445505 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6d4100e80 0x7fd6d4073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:32:28.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:28.665+0000 7fd6da2a3700 1 -- 172.21.15.22:0/352445505 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6d4100e80 msgr2=0x7fd6d4073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:28.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:28.665+0000 7fd6da2a3700 1 --2- 172.21.15.22:0/352445505 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6d4100e80 0x7fd6d4073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:28.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:28.665+0000 7fd6da2a3700 1 -- 172.21.15.22:0/352445505 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6c4009d00 con 0x7fd6d41004e0 2024-02-16T19:32:28.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:28.665+0000 7fd6da2a3700 1 --2- 172.21.15.22:0/352445505 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd6d41004e0 0x7fd6d4073050 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fd6c4000f80 tx=0x7fd6c400cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:32:28.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:28.665+0000 7fd6d37fe700 1 -- 172.21.15.22:0/352445505 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6c400e040 con 0x7fd6d41004e0 2024-02-16T19:32:28.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:28.665+0000 7fd6d37fe700 1 -- 172.21.15.22:0/352445505 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd6c4003c10 con 0x7fd6d41004e0 2024-02-16T19:32:28.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:28.665+0000 7fd6dc507700 1 -- 172.21.15.22:0/352445505 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6d4073e90 con 0x7fd6d41004e0 2024-02-16T19:32:28.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:28.665+0000 7fd6d37fe700 1 -- 172.21.15.22:0/352445505 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6c401d6b0 con 0x7fd6d41004e0 2024-02-16T19:32:28.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:28.665+0000 7fd6dc507700 1 -- 172.21.15.22:0/352445505 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6d41a23f0 con 0x7fd6d41004e0 2024-02-16T19:32:28.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:28.666+0000 7fd6dc507700 1 -- 172.21.15.22:0/352445505 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6d41087f0 con 0x7fd6d41004e0 2024-02-16T19:32:28.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:28.667+0000 7fd6d37fe700 1 -- 172.21.15.22:0/352445505 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 33) v1 ==== 91346+0+0 (secure 0 0 0) 0x7fd6c4004040 con 0x7fd6d41004e0 2024-02-16T19:32:28.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:28.667+0000 7fd6d37fe700 1 --2- 172.21.15.22:0/352445505 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7fd6bc06d520 0x7fd6bc06f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:32:28.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:28.667+0000 7fd6d37fe700 1 -- 172.21.15.22:0/352445505 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fd6c408e670 con 0x7fd6d41004e0 2024-02-16T19:32:28.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:28.667+0000 7fd6d9aa2700 1 --2- 172.21.15.22:0/352445505 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7fd6bc06d520 0x7fd6bc06f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:32:28.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:28.668+0000 7fd6d9aa2700 1 --2- 172.21.15.22:0/352445505 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7fd6bc06d520 0x7fd6bc06f9d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4074840 tx=0x7fd6c800b410 comp rx=0 tx=0).ready entity=mgr.24407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:32:28.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:28.672+0000 7fd6d37fe700 1 -- 172.21.15.22:0/352445505 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7fd6c40583d0 con 0x7fd6d41004e0 2024-02-16T19:32:28.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:28.969+0000 7fd6dc507700 1 -- 172.21.15.22:0/352445505 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd6d404f2a0 con 0x7fd6d41004e0 2024-02-16T19:32:28.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:28.970+0000 7fd6d37fe700 1 -- 172.21.15.22:0/352445505 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7fd6c405ba90 con 0x7fd6d41004e0 2024-02-16T19:32:28.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:28.972+0000 7fd6dc507700 1 -- 172.21.15.22:0/352445505 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7fd6bc06d520 msgr2=0x7fd6bc06f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:28.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:28.972+0000 7fd6dc507700 1 --2- 172.21.15.22:0/352445505 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7fd6bc06d520 0x7fd6bc06f9d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4074840 tx=0x7fd6c800b410 comp rx=0 tx=0).stop 2024-02-16T19:32:28.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:28.972+0000 7fd6dc507700 1 -- 172.21.15.22:0/352445505 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd6d41004e0 msgr2=0x7fd6d4073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:28.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:28.972+0000 7fd6dc507700 1 --2- 172.21.15.22:0/352445505 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd6d41004e0 0x7fd6d4073050 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fd6c4000f80 tx=0x7fd6c400cd20 comp rx=0 tx=0).stop 2024-02-16T19:32:28.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:28.972+0000 7fd6dc507700 1 -- 172.21.15.22:0/352445505 shutdown_connections 2024-02-16T19:32:28.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:28.972+0000 7fd6dc507700 1 --2- 172.21.15.22:0/352445505 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7fd6bc06d520 0x7fd6bc06f9d0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:28.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:28.972+0000 7fd6dc507700 1 --2- 172.21.15.22:0/352445505 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd6d41004e0 0x7fd6d4073050 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:28.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:28.972+0000 7fd6dc507700 1 --2- 172.21.15.22:0/352445505 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6d4100e80 0x7fd6d4073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:28.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:28.972+0000 7fd6dc507700 1 -- 172.21.15.22:0/352445505 >> 172.21.15.22:0/352445505 conn(0x7fd6d40fbda0 msgr2=0x7fd6d40fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:32:28.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:28.972+0000 7fd6dc507700 1 -- 172.21.15.22:0/352445505 shutdown_connections 2024-02-16T19:32:28.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:28.972+0000 7fd6dc507700 1 -- 172.21.15.22:0/352445505 wait complete. 2024-02-16T19:32:28.984 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)" 2024-02-16T19:32:29.571 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 2'"'"'' 2024-02-16T19:32:29.581 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:29 smithi022 ceph-mon[29221]: from='client.? 172.21.15.22:0/352445505' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:32:29.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:29 smithi117 ceph-mon[35309]: from='client.? 172.21.15.22:0/352445505' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:32:29.791 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi022/config 2024-02-16T19:32:30.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:30 smithi117 ceph-mon[35309]: pgmap v81: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:32:30.714 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:30 smithi022 ceph-mon[29221]: pgmap v81: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:32:32.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:32 smithi117 ceph-mon[35309]: pgmap v82: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:32:32.659 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:32 smithi022 ceph-mon[29221]: pgmap v82: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:32:34.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:34 smithi117 ceph-mon[35309]: pgmap v83: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:32:34.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:34 smithi022 ceph-mon[29221]: pgmap v83: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:32:36.609 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:36 smithi022 ceph-mon[29221]: pgmap v84: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:32:36.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:36 smithi117 ceph-mon[35309]: pgmap v84: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:32:36.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:36.639+0000 7efe292d0700 1 -- 172.21.15.22:0/3217910873 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efe24069140 msgr2=0x7efe24104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:36.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:36.639+0000 7efe292d0700 1 --2- 172.21.15.22:0/3217910873 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efe24069140 0x7efe24104eb0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7efe0c009910 tx=0x7efe0c009c20 comp rx=0 tx=0).stop 2024-02-16T19:32:36.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:36.639+0000 7efe292d0700 1 -- 172.21.15.22:0/3217910873 shutdown_connections 2024-02-16T19:32:36.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:36.639+0000 7efe292d0700 1 --2- 172.21.15.22:0/3217910873 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efe24069140 0x7efe24104eb0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:36.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:36.639+0000 7efe292d0700 1 --2- 172.21.15.22:0/3217910873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe240687a0 0x7efe24068b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:36.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:36.639+0000 7efe292d0700 1 -- 172.21.15.22:0/3217910873 >> 172.21.15.22:0/3217910873 conn(0x7efe24075130 msgr2=0x7efe24075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:32:36.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:36.639+0000 7efe292d0700 1 -- 172.21.15.22:0/3217910873 shutdown_connections 2024-02-16T19:32:36.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:36.639+0000 7efe292d0700 1 -- 172.21.15.22:0/3217910873 wait complete. 2024-02-16T19:32:36.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:36.639+0000 7efe292d0700 1 Processor -- start 2024-02-16T19:32:36.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:36.640+0000 7efe292d0700 1 -- start start 2024-02-16T19:32:36.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:36.640+0000 7efe292d0700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe240687a0 0x7efe24195e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:32:36.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:36.640+0000 7efe292d0700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efe24069140 0x7efe241963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:32:36.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:36.640+0000 7efe292d0700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe24196a80 con 0x7efe24069140 2024-02-16T19:32:36.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:36.640+0000 7efe292d0700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe2419a810 con 0x7efe240687a0 2024-02-16T19:32:36.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:36.640+0000 7efe227fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efe24069140 0x7efe241963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:32:36.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:36.640+0000 7efe227fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efe24069140 0x7efe241963a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43318/0 (socket says 172.21.15.22:43318) 2024-02-16T19:32:36.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:36.640+0000 7efe227fc700 1 -- 172.21.15.22:0/1024038333 learned_addr learned my addr 172.21.15.22:0/1024038333 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:32:36.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:36.640+0000 7efe22ffd700 1 --2- 172.21.15.22:0/1024038333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe240687a0 0x7efe24195e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:32:36.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:36.640+0000 7efe227fc700 1 -- 172.21.15.22:0/1024038333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe240687a0 msgr2=0x7efe24195e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:36.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:36.640+0000 7efe227fc700 1 --2- 172.21.15.22:0/1024038333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe240687a0 0x7efe24195e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:36.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:36.640+0000 7efe227fc700 1 -- 172.21.15.22:0/1024038333 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe0c0095f0 con 0x7efe24069140 2024-02-16T19:32:36.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:36.640+0000 7efe227fc700 1 --2- 172.21.15.22:0/1024038333 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efe24069140 0x7efe241963a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7efe0c00bda0 tx=0x7efe0c00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:32:36.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:36.640+0000 7efe1bfff700 1 -- 172.21.15.22:0/1024038333 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe0c01d070 con 0x7efe24069140 2024-02-16T19:32:36.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:36.641+0000 7efe292d0700 1 -- 172.21.15.22:0/1024038333 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe2419aa90 con 0x7efe24069140 2024-02-16T19:32:36.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:36.641+0000 7efe1bfff700 1 -- 172.21.15.22:0/1024038333 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efe0c004fb0 con 0x7efe24069140 2024-02-16T19:32:36.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:36.641+0000 7efe1bfff700 1 -- 172.21.15.22:0/1024038333 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe0c00f480 con 0x7efe24069140 2024-02-16T19:32:36.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:36.641+0000 7efe292d0700 1 -- 172.21.15.22:0/1024038333 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe2419af80 con 0x7efe24069140 2024-02-16T19:32:36.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:36.642+0000 7efe292d0700 1 -- 172.21.15.22:0/1024038333 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe24108750 con 0x7efe24069140 2024-02-16T19:32:36.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:36.642+0000 7efe1bfff700 1 -- 172.21.15.22:0/1024038333 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 33) v1 ==== 91346+0+0 (secure 0 0 0) 0x7efe0c00f6b0 con 0x7efe24069140 2024-02-16T19:32:36.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:36.642+0000 7efe1bfff700 1 --2- 172.21.15.22:0/1024038333 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7efe1006d520 0x7efe1006f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:32:36.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:36.642+0000 7efe1bfff700 1 -- 172.21.15.22:0/1024038333 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7efe0c08e6d0 con 0x7efe24069140 2024-02-16T19:32:36.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:36.643+0000 7efe22ffd700 1 --2- 172.21.15.22:0/1024038333 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7efe1006d520 0x7efe1006f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:32:36.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:36.643+0000 7efe22ffd700 1 --2- 172.21.15.22:0/1024038333 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7efe1006d520 0x7efe1006f9d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7efe1c005fd0 tx=0x7efe1c00a560 comp rx=0 tx=0).ready entity=mgr.24407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:32:36.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:36.647+0000 7efe1bfff700 1 -- 172.21.15.22:0/1024038333 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7efe0c058580 con 0x7efe24069140 2024-02-16T19:32:36.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:36.941+0000 7efe292d0700 1 -- 172.21.15.22:0/1024038333 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7efe24197260 con 0x7efe24069140 2024-02-16T19:32:36.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:36.942+0000 7efe1bfff700 1 -- 172.21.15.22:0/1024038333 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7efe0c08ce60 con 0x7efe24069140 2024-02-16T19:32:36.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:36.944+0000 7efe292d0700 1 -- 172.21.15.22:0/1024038333 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7efe1006d520 msgr2=0x7efe1006f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:36.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:36.944+0000 7efe292d0700 1 --2- 172.21.15.22:0/1024038333 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7efe1006d520 0x7efe1006f9d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7efe1c005fd0 tx=0x7efe1c00a560 comp rx=0 tx=0).stop 2024-02-16T19:32:36.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:36.944+0000 7efe292d0700 1 -- 172.21.15.22:0/1024038333 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efe24069140 msgr2=0x7efe241963a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:36.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:36.944+0000 7efe292d0700 1 --2- 172.21.15.22:0/1024038333 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efe24069140 0x7efe241963a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7efe0c00bda0 tx=0x7efe0c00be80 comp rx=0 tx=0).stop 2024-02-16T19:32:36.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:36.944+0000 7efe292d0700 1 -- 172.21.15.22:0/1024038333 shutdown_connections 2024-02-16T19:32:36.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:36.944+0000 7efe292d0700 1 --2- 172.21.15.22:0/1024038333 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7efe1006d520 0x7efe1006f9d0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:36.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:36.944+0000 7efe292d0700 1 --2- 172.21.15.22:0/1024038333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe240687a0 0x7efe24195e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:36.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:36.944+0000 7efe292d0700 1 --2- 172.21.15.22:0/1024038333 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efe24069140 0x7efe241963a0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:36.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:36.944+0000 7efe292d0700 1 -- 172.21.15.22:0/1024038333 >> 172.21.15.22:0/1024038333 conn(0x7efe24075130 msgr2=0x7efe24104720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:32:36.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:36.944+0000 7efe292d0700 1 -- 172.21.15.22:0/1024038333 shutdown_connections 2024-02-16T19:32:36.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:36.944+0000 7efe292d0700 1 -- 172.21.15.22:0/1024038333 wait complete. 2024-02-16T19:32:36.956 INFO:teuthology.orchestra.run.smithi022.stdout:true 2024-02-16T19:32:37.526 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '"'"'.up_to_date | length == 2'"'"'' 2024-02-16T19:32:37.781 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi022/config 2024-02-16T19:32:38.158 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:37 smithi022 ceph-mon[29221]: from='client.? 172.21.15.22:0/1024038333' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:32:38.158 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:37 smithi022 ceph-mon[29221]: pgmap v85: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:32:38.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:37 smithi117 ceph-mon[35309]: from='client.? 172.21.15.22:0/1024038333' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:32:38.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:37 smithi117 ceph-mon[35309]: pgmap v85: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:32:39.133 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:38 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:32:39.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:38 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:32:39.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:39.390+0000 7f5e1af3d700 1 -- 172.21.15.22:0/3320432915 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5e140ff3c0 msgr2=0x7f5e140ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:39.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:39.390+0000 7f5e1af3d700 1 --2- 172.21.15.22:0/3320432915 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5e140ff3c0 0x7f5e140ff790 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c00b0a0 tx=0x7f5e0c00b3b0 comp rx=0 tx=0).stop 2024-02-16T19:32:39.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:39.390+0000 7f5e1af3d700 1 -- 172.21.15.22:0/3320432915 shutdown_connections 2024-02-16T19:32:39.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:39.390+0000 7f5e1af3d700 1 --2- 172.21.15.22:0/3320432915 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e140ffd60 0x7f5e14103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:39.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:39.390+0000 7f5e1af3d700 1 --2- 172.21.15.22:0/3320432915 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5e140ff3c0 0x7f5e140ff790 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:39.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:39.390+0000 7f5e1af3d700 1 -- 172.21.15.22:0/3320432915 >> 172.21.15.22:0/3320432915 conn(0x7f5e140fac80 msgr2=0x7f5e140fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:32:39.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:39.390+0000 7f5e1af3d700 1 -- 172.21.15.22:0/3320432915 shutdown_connections 2024-02-16T19:32:39.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:39.390+0000 7f5e1af3d700 1 -- 172.21.15.22:0/3320432915 wait complete. 2024-02-16T19:32:39.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:39.391+0000 7f5e1af3d700 1 Processor -- start 2024-02-16T19:32:39.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:39.391+0000 7f5e1af3d700 1 -- start start 2024-02-16T19:32:39.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:39.391+0000 7f5e1af3d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5e140ff3c0 0x7f5e14100dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:32:39.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:39.391+0000 7f5e1af3d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e140ffd60 0x7f5e14101310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:32:39.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:39.391+0000 7f5e1af3d700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e14101850 con 0x7f5e140ff3c0 2024-02-16T19:32:39.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:39.391+0000 7f5e1af3d700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e14101990 con 0x7f5e140ffd60 2024-02-16T19:32:39.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:39.391+0000 7f5e18cd9700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5e140ff3c0 0x7f5e14100dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:32:39.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:39.391+0000 7f5e18cd9700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5e140ff3c0 0x7f5e14100dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43336/0 (socket says 172.21.15.22:43336) 2024-02-16T19:32:39.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:39.392+0000 7f5e18cd9700 1 -- 172.21.15.22:0/2167917301 learned_addr learned my addr 172.21.15.22:0/2167917301 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:32:39.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:39.392+0000 7f5e13fff700 1 --2- 172.21.15.22:0/2167917301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e140ffd60 0x7f5e14101310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:32:39.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:39.392+0000 7f5e18cd9700 1 -- 172.21.15.22:0/2167917301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e140ffd60 msgr2=0x7f5e14101310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:39.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:39.392+0000 7f5e18cd9700 1 --2- 172.21.15.22:0/2167917301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e140ffd60 0x7f5e14101310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:39.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:39.392+0000 7f5e18cd9700 1 -- 172.21.15.22:0/2167917301 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e0c009d00 con 0x7f5e140ff3c0 2024-02-16T19:32:39.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:39.392+0000 7f5e18cd9700 1 --2- 172.21.15.22:0/2167917301 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5e140ff3c0 0x7f5e14100dd0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c000f80 tx=0x7f5e0c00cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:32:39.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:39.392+0000 7f5e11ffb700 1 -- 172.21.15.22:0/2167917301 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e0c00e040 con 0x7f5e140ff3c0 2024-02-16T19:32:39.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:39.392+0000 7f5e11ffb700 1 -- 172.21.15.22:0/2167917301 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5e0c003c10 con 0x7f5e140ff3c0 2024-02-16T19:32:39.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:39.392+0000 7f5e1af3d700 1 -- 172.21.15.22:0/2167917301 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e14101c10 con 0x7f5e140ff3c0 2024-02-16T19:32:39.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:39.392+0000 7f5e11ffb700 1 -- 172.21.15.22:0/2167917301 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e0c01d6b0 con 0x7f5e140ff3c0 2024-02-16T19:32:39.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:39.392+0000 7f5e1af3d700 1 -- 172.21.15.22:0/2167917301 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e141a23b0 con 0x7f5e140ff3c0 2024-02-16T19:32:39.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:39.393+0000 7f5e1af3d700 1 -- 172.21.15.22:0/2167917301 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e1404f2a0 con 0x7f5e140ff3c0 2024-02-16T19:32:39.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:39.394+0000 7f5e11ffb700 1 -- 172.21.15.22:0/2167917301 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 33) v1 ==== 91346+0+0 (secure 0 0 0) 0x7f5e0c004040 con 0x7f5e140ff3c0 2024-02-16T19:32:39.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:39.394+0000 7f5e11ffb700 1 --2- 172.21.15.22:0/2167917301 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f5e0006d520 0x7f5e0006f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:32:39.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:39.394+0000 7f5e11ffb700 1 -- 172.21.15.22:0/2167917301 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f5e0c08e6b0 con 0x7f5e140ff3c0 2024-02-16T19:32:39.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:39.394+0000 7f5e13fff700 1 --2- 172.21.15.22:0/2167917301 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f5e0006d520 0x7f5e0006f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:32:39.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:39.395+0000 7f5e13fff700 1 --2- 172.21.15.22:0/2167917301 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f5e0006d520 0x7f5e0006f9d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5e14102550 tx=0x7f5dfc00b410 comp rx=0 tx=0).ready entity=mgr.24407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:32:39.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:39.399+0000 7f5e11ffb700 1 -- 172.21.15.22:0/2167917301 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f5e0c058370 con 0x7f5e140ff3c0 2024-02-16T19:32:39.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:39.636+0000 7f5e1af3d700 1 -- 172.21.15.22:0/2167917301 --> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] -- mgr_command(tid 0: {"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:f78a58c0ffd401d1493058a1022c35f011d65275", "target": ["mon-mgr", ""]}) v1 -- 0x7f5e14068ef0 con 0x7f5e0006d520 2024-02-16T19:32:40.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:39 smithi022 ceph-mon[29221]: pgmap v86: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:32:40.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:39 smithi117 ceph-mon[35309]: pgmap v86: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:32:41.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:40 smithi022 ceph-mon[29221]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:f78a58c0ffd401d1493058a1022c35f011d65275", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:32:41.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:40 smithi117 ceph-mon[35309]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:f78a58c0ffd401d1493058a1022c35f011d65275", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:32:41.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:41.656+0000 7f5e11ffb700 1 -- 172.21.15.22:0/2167917301 <== mgr.24407 v2:172.21.15.22:6800/3703806543 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+5391 (secure 0 0 0) 0x7f5e14068ef0 con 0x7f5e0006d520 2024-02-16T19:32:41.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:41.659+0000 7f5e1af3d700 1 -- 172.21.15.22:0/2167917301 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f5e0006d520 msgr2=0x7f5e0006f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:41.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:41.659+0000 7f5e1af3d700 1 --2- 172.21.15.22:0/2167917301 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f5e0006d520 0x7f5e0006f9d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5e14102550 tx=0x7f5dfc00b410 comp rx=0 tx=0).stop 2024-02-16T19:32:41.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:41.659+0000 7f5e1af3d700 1 -- 172.21.15.22:0/2167917301 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5e140ff3c0 msgr2=0x7f5e14100dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:41.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:41.659+0000 7f5e1af3d700 1 --2- 172.21.15.22:0/2167917301 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5e140ff3c0 0x7f5e14100dd0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c000f80 tx=0x7f5e0c00cd20 comp rx=0 tx=0).stop 2024-02-16T19:32:41.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:41.659+0000 7f5e1af3d700 1 -- 172.21.15.22:0/2167917301 shutdown_connections 2024-02-16T19:32:41.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:41.659+0000 7f5e1af3d700 1 --2- 172.21.15.22:0/2167917301 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f5e0006d520 0x7f5e0006f9d0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:41.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:41.659+0000 7f5e1af3d700 1 --2- 172.21.15.22:0/2167917301 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5e140ff3c0 0x7f5e14100dd0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:41.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:41.659+0000 7f5e1af3d700 1 --2- 172.21.15.22:0/2167917301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e140ffd60 0x7f5e14101310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:41.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:41.659+0000 7f5e1af3d700 1 -- 172.21.15.22:0/2167917301 >> 172.21.15.22:0/2167917301 conn(0x7f5e140fac80 msgr2=0x7f5e14103480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:32:41.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:41.659+0000 7f5e1af3d700 1 -- 172.21.15.22:0/2167917301 shutdown_connections 2024-02-16T19:32:41.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:41.659+0000 7f5e1af3d700 1 -- 172.21.15.22:0/2167917301 wait complete. 2024-02-16T19:32:41.671 INFO:teuthology.orchestra.run.smithi022.stdout:true 2024-02-16T19:32:42.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:41 smithi117 ceph-mon[35309]: pgmap v87: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:32:42.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:41 smithi022 ceph-mon[29221]: pgmap v87: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:32:42.337 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph orch ps' 2024-02-16T19:32:42.554 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi022/config 2024-02-16T19:32:44.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:44.108+0000 7f7d88eef700 1 -- 172.21.15.22:0/268260153 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d840ff350 msgr2=0x7f7d840ff7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:44.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:44.108+0000 7f7d88eef700 1 --2- 172.21.15.22:0/268260153 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d840ff350 0x7f7d840ff7c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7d78009910 tx=0x7f7d78009c20 comp rx=0 tx=0).stop 2024-02-16T19:32:44.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:44.108+0000 7f7d88eef700 1 -- 172.21.15.22:0/268260153 shutdown_connections 2024-02-16T19:32:44.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:44.108+0000 7f7d88eef700 1 --2- 172.21.15.22:0/268260153 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d840ff350 0x7f7d840ff7c0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:44.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:44.108+0000 7f7d88eef700 1 --2- 172.21.15.22:0/268260153 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d841038b0 0x7f7d84103c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:44.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:44.108+0000 7f7d88eef700 1 -- 172.21.15.22:0/268260153 >> 172.21.15.22:0/268260153 conn(0x7f7d840748b0 msgr2=0x7f7d84074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:32:44.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:44.108+0000 7f7d88eef700 1 -- 172.21.15.22:0/268260153 shutdown_connections 2024-02-16T19:32:44.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:44.109+0000 7f7d88eef700 1 -- 172.21.15.22:0/268260153 wait complete. 2024-02-16T19:32:44.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:44.109+0000 7f7d88eef700 1 Processor -- start 2024-02-16T19:32:44.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:44.109+0000 7f7d88eef700 1 -- start start 2024-02-16T19:32:44.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:44.109+0000 7f7d88eef700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d840ff350 0x7f7d84195f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:32:44.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:44.109+0000 7f7d88eef700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d841038b0 0x7f7d841964a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:32:44.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:44.109+0000 7f7d88eef700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d84196b80 con 0x7f7d840ff350 2024-02-16T19:32:44.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:44.109+0000 7f7d88eef700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d8419a910 con 0x7f7d841038b0 2024-02-16T19:32:44.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:44.110+0000 7f7d8259c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d840ff350 0x7f7d84195f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:32:44.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:44.110+0000 7f7d8259c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d840ff350 0x7f7d84195f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43354/0 (socket says 172.21.15.22:43354) 2024-02-16T19:32:44.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:44.110+0000 7f7d8259c700 1 -- 172.21.15.22:0/3353569090 learned_addr learned my addr 172.21.15.22:0/3353569090 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:32:44.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:44.110+0000 7f7d81d9b700 1 --2- 172.21.15.22:0/3353569090 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d841038b0 0x7f7d841964a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:32:44.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:44.110+0000 7f7d8259c700 1 -- 172.21.15.22:0/3353569090 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d841038b0 msgr2=0x7f7d841964a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:44.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:44.110+0000 7f7d8259c700 1 --2- 172.21.15.22:0/3353569090 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d841038b0 0x7f7d841964a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:44.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:44.110+0000 7f7d8259c700 1 -- 172.21.15.22:0/3353569090 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d780095f0 con 0x7f7d840ff350 2024-02-16T19:32:44.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:44.110+0000 7f7d8259c700 1 --2- 172.21.15.22:0/3353569090 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d840ff350 0x7f7d84195f60 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7d7400b4b0 tx=0x7f7d7400b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:32:44.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:44.110+0000 7f7d737fe700 1 -- 172.21.15.22:0/3353569090 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d74011800 con 0x7f7d840ff350 2024-02-16T19:32:44.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:44.110+0000 7f7d737fe700 1 -- 172.21.15.22:0/3353569090 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7d74011e20 con 0x7f7d840ff350 2024-02-16T19:32:44.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:44.110+0000 7f7d88eef700 1 -- 172.21.15.22:0/3353569090 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d8419abf0 con 0x7f7d840ff350 2024-02-16T19:32:44.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:44.110+0000 7f7d737fe700 1 -- 172.21.15.22:0/3353569090 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d7400f2e0 con 0x7f7d840ff350 2024-02-16T19:32:44.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:44.110+0000 7f7d88eef700 1 -- 172.21.15.22:0/3353569090 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d8419b110 con 0x7f7d840ff350 2024-02-16T19:32:44.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:44.111+0000 7f7d88eef700 1 -- 172.21.15.22:0/3353569090 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d8404f2a0 con 0x7f7d840ff350 2024-02-16T19:32:44.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:44.112+0000 7f7d737fe700 1 -- 172.21.15.22:0/3353569090 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 33) v1 ==== 91346+0+0 (secure 0 0 0) 0x7f7d74010380 con 0x7f7d840ff350 2024-02-16T19:32:44.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:44.112+0000 7f7d737fe700 1 --2- 172.21.15.22:0/3353569090 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f7d6c06d520 0x7f7d6c06f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:32:44.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:44.112+0000 7f7d737fe700 1 -- 172.21.15.22:0/3353569090 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f7d7408c9e0 con 0x7f7d840ff350 2024-02-16T19:32:44.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:44.112+0000 7f7d81d9b700 1 --2- 172.21.15.22:0/3353569090 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f7d6c06d520 0x7f7d6c06f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:32:44.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:44.113+0000 7f7d81d9b700 1 --2- 172.21.15.22:0/3353569090 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f7d6c06d520 0x7f7d6c06f9d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7d78006010 tx=0x7f7d7801a040 comp rx=0 tx=0).ready entity=mgr.24407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:32:44.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:44.117+0000 7f7d737fe700 1 -- 172.21.15.22:0/3353569090 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f7d74056740 con 0x7f7d840ff350 2024-02-16T19:32:44.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:44.334+0000 7f7d88eef700 1 -- 172.21.15.22:0/3353569090 --> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7d8419b4f0 con 0x7f7d6c06d520 2024-02-16T19:32:44.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:44.345+0000 7f7d737fe700 1 -- 172.21.15.22:0/3353569090 <== mgr.24407 v2:172.21.15.22:6800/3703806543 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f7d8419b4f0 con 0x7f7d6c06d520 2024-02-16T19:32:44.347 INFO:teuthology.orchestra.run.smithi022.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T19:32:44.347 INFO:teuthology.orchestra.run.smithi022.stdout:alertmanager.smithi022 smithi022 *:9093,9094 running (66s) 34s ago 11m 15.7M - 0.25.0 c8568f914cd2 c89bc76be374 2024-02-16T19:32:44.347 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-exporter.smithi022 smithi022 running (12m) 34s ago 12m 7910k - 18.2.0 dc2bc1663786 3f8bef4349ba 2024-02-16T19:32:44.347 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-exporter.smithi117 smithi117 running (11m) 97s ago 11m 7507k - 18.2.0 dc2bc1663786 59b65a7e309f 2024-02-16T19:32:44.347 INFO:teuthology.orchestra.run.smithi022.stdout:crash.smithi022 smithi022 running (12m) 34s ago 12m 6669k - 18.2.0 dc2bc1663786 efef476aa486 2024-02-16T19:32:44.347 INFO:teuthology.orchestra.run.smithi022.stdout:crash.smithi117 smithi117 running (11m) 97s ago 11m 6660k - 18.2.0 dc2bc1663786 5466453eb149 2024-02-16T19:32:44.347 INFO:teuthology.orchestra.run.smithi022.stdout:grafana.smithi022 smithi022 *:3000 running (40s) 34s ago 11m 52.6M - 9.4.12 a0ea5e3c46d9 fd27d11e9729 2024-02-16T19:32:44.347 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi022.btppwm smithi022 running (6m) 34s ago 6m 158M - 18.2.0 dc2bc1663786 22971fb3a907 2024-02-16T19:32:44.347 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi022.ocukvs smithi022 running (6m) 34s ago 6m 14.7M - 18.2.0 dc2bc1663786 81ee7bd5fd39 2024-02-16T19:32:44.347 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi117.ibmppn smithi117 running (6m) 97s ago 6m 16.1M - 18.2.0 dc2bc1663786 8c606cbeeb86 2024-02-16T19:32:44.347 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi117.orzors smithi117 running (6m) 97s ago 6m 15.1M - 18.2.0 dc2bc1663786 7d2bc4df3feb 2024-02-16T19:32:44.347 INFO:teuthology.orchestra.run.smithi022.stdout:mgr.smithi022.gmdcch smithi022 *:8443,9283,8765 running (3m) 34s ago 13m 499M - 19.0.0-1450-gf78a58c0 f081f847070a 7cb74cc9deea 2024-02-16T19:32:44.347 INFO:teuthology.orchestra.run.smithi022.stdout:mgr.smithi117.cohbtv smithi117 *:8443,9283,8765 running (2m) 97s ago 10m 450M - 19.0.0-1450-gf78a58c0 f081f847070a c825b1152189 2024-02-16T19:32:44.348 INFO:teuthology.orchestra.run.smithi022.stdout:mon.smithi022 smithi022 running (13m) 34s ago 13m 57.9M 2048M 18.2.0 dc2bc1663786 b4ffc3032e12 2024-02-16T19:32:44.348 INFO:teuthology.orchestra.run.smithi022.stdout:mon.smithi117 smithi117 running (10m) 97s ago 10m 44.6M 2048M 18.2.0 dc2bc1663786 aff3e21a73a0 2024-02-16T19:32:44.348 INFO:teuthology.orchestra.run.smithi022.stdout:node-exporter.smithi022 smithi022 *:9100 running (105s) 34s ago 12m 8887k - 1.5.0 0da6a335fe13 1f5b58292d27 2024-02-16T19:32:44.348 INFO:teuthology.orchestra.run.smithi022.stdout:node-exporter.smithi117 smithi117 *:9100 running (101s) 97s ago 10m 3275k - 1.5.0 0da6a335fe13 bb5e8491d5a6 2024-02-16T19:32:44.348 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0 smithi022 running (10m) 34s ago 10m 209M 1217M 18.2.0 dc2bc1663786 0027f86f16ce 2024-02-16T19:32:44.348 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1 smithi022 running (9m) 34s ago 9m 225M 1217M 18.2.0 dc2bc1663786 87300382b8a7 2024-02-16T19:32:44.348 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2 smithi022 running (9m) 34s ago 9m 185M 1217M 18.2.0 dc2bc1663786 be8a631b4f93 2024-02-16T19:32:44.348 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3 smithi117 running (8m) 97s ago 8m 242M 2240M 18.2.0 dc2bc1663786 9c0ce9a981cf 2024-02-16T19:32:44.348 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4 smithi117 running (8m) 97s ago 8m 208M 2240M 18.2.0 dc2bc1663786 1779c3496f4d 2024-02-16T19:32:44.348 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5 smithi117 running (7m) 97s ago 7m 194M 2240M 18.2.0 dc2bc1663786 fefaabca0638 2024-02-16T19:32:44.348 INFO:teuthology.orchestra.run.smithi022.stdout:prometheus.smithi022 smithi022 *:9095 running (83s) 34s ago 11m 51.1M - 2.43.0 a07b618ecd1d c3b8689d0c97 2024-02-16T19:32:44.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:44.347+0000 7f7d88eef700 1 -- 172.21.15.22:0/3353569090 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f7d6c06d520 msgr2=0x7f7d6c06f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:44.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:44.347+0000 7f7d88eef700 1 --2- 172.21.15.22:0/3353569090 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f7d6c06d520 0x7f7d6c06f9d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7d78006010 tx=0x7f7d7801a040 comp rx=0 tx=0).stop 2024-02-16T19:32:44.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:44.347+0000 7f7d88eef700 1 -- 172.21.15.22:0/3353569090 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d840ff350 msgr2=0x7f7d84195f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:44.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:44.347+0000 7f7d88eef700 1 --2- 172.21.15.22:0/3353569090 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d840ff350 0x7f7d84195f60 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7d7400b4b0 tx=0x7f7d7400b7c0 comp rx=0 tx=0).stop 2024-02-16T19:32:44.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:44.347+0000 7f7d88eef700 1 -- 172.21.15.22:0/3353569090 shutdown_connections 2024-02-16T19:32:44.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:44.347+0000 7f7d88eef700 1 --2- 172.21.15.22:0/3353569090 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f7d6c06d520 0x7f7d6c06f9d0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:44.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:44.347+0000 7f7d88eef700 1 --2- 172.21.15.22:0/3353569090 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d840ff350 0x7f7d84195f60 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:44.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:44.347+0000 7f7d88eef700 1 --2- 172.21.15.22:0/3353569090 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d841038b0 0x7f7d841964a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:44.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:44.347+0000 7f7d88eef700 1 -- 172.21.15.22:0/3353569090 >> 172.21.15.22:0/3353569090 conn(0x7f7d840748b0 msgr2=0x7f7d840feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:32:44.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:44.347+0000 7f7d88eef700 1 -- 172.21.15.22:0/3353569090 shutdown_connections 2024-02-16T19:32:44.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:44.347+0000 7f7d88eef700 1 -- 172.21.15.22:0/3353569090 wait complete. 2024-02-16T19:32:44.565 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:44 smithi022 ceph-mon[29221]: pgmap v88: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:32:44.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:44 smithi117 ceph-mon[35309]: pgmap v88: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:32:44.863 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-02-16T19:32:44.864 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi022.front.sepia.ceph.com 2024-02-16T19:32:44.864 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph config set mgr mgr/orchestrator/fail_fs true' 2024-02-16T19:32:45.074 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi022/config 2024-02-16T19:32:45.448 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:45 smithi022 ceph-mon[29221]: from='client.14800 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:32:45.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:45 smithi117 ceph-mon[35309]: from='client.14800 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:32:46.464 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:46 smithi022 ceph-mon[29221]: pgmap v89: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:32:46.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:46 smithi117 ceph-mon[35309]: pgmap v89: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:32:46.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:46.690+0000 7f2bd2be1700 1 -- 172.21.15.22:0/881173265 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2bcc100e60 msgr2=0x7f2bcc104e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:46.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:46.690+0000 7f2bd2be1700 1 --2- 172.21.15.22:0/881173265 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2bcc100e60 0x7f2bcc104e30 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f2bb4009910 tx=0x7f2bb4009c20 comp rx=0 tx=0).stop 2024-02-16T19:32:46.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:46.690+0000 7f2bd2be1700 1 -- 172.21.15.22:0/881173265 shutdown_connections 2024-02-16T19:32:46.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:46.690+0000 7f2bd2be1700 1 --2- 172.21.15.22:0/881173265 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2bcc100e60 0x7f2bcc104e30 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:46.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:46.690+0000 7f2bd2be1700 1 --2- 172.21.15.22:0/881173265 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bcc1004c0 0x7f2bcc100890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:46.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:46.690+0000 7f2bd2be1700 1 -- 172.21.15.22:0/881173265 >> 172.21.15.22:0/881173265 conn(0x7f2bcc0fbd80 msgr2=0x7f2bcc0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:32:46.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:46.690+0000 7f2bd2be1700 1 -- 172.21.15.22:0/881173265 shutdown_connections 2024-02-16T19:32:46.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:46.690+0000 7f2bd2be1700 1 -- 172.21.15.22:0/881173265 wait complete. 2024-02-16T19:32:46.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:46.691+0000 7f2bd2be1700 1 Processor -- start 2024-02-16T19:32:46.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:46.691+0000 7f2bd2be1700 1 -- start start 2024-02-16T19:32:46.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:46.691+0000 7f2bd2be1700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bcc1004c0 0x7f2bcc073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:32:46.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:46.691+0000 7f2bd2be1700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2bcc100e60 0x7f2bcc073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:32:46.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:46.691+0000 7f2bd2be1700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bcc073ad0 con 0x7f2bcc100e60 2024-02-16T19:32:46.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:46.691+0000 7f2bd2be1700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bcc073c10 con 0x7f2bcc1004c0 2024-02-16T19:32:46.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:46.692+0000 7f2bcbfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2bcc100e60 0x7f2bcc073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:32:46.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:46.692+0000 7f2bcbfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2bcc100e60 0x7f2bcc073590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46684/0 (socket says 172.21.15.22:46684) 2024-02-16T19:32:46.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:46.692+0000 7f2bcbfff700 1 -- 172.21.15.22:0/2644625870 learned_addr learned my addr 172.21.15.22:0/2644625870 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:32:46.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:46.692+0000 7f2bd097d700 1 --2- 172.21.15.22:0/2644625870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bcc1004c0 0x7f2bcc073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:32:46.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:46.692+0000 7f2bcbfff700 1 -- 172.21.15.22:0/2644625870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bcc1004c0 msgr2=0x7f2bcc073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:46.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:46.692+0000 7f2bcbfff700 1 --2- 172.21.15.22:0/2644625870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bcc1004c0 0x7f2bcc073050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:46.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:46.692+0000 7f2bcbfff700 1 -- 172.21.15.22:0/2644625870 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bb40095f0 con 0x7f2bcc100e60 2024-02-16T19:32:46.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:46.692+0000 7f2bcbfff700 1 --2- 172.21.15.22:0/2644625870 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2bcc100e60 0x7f2bcc073590 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f2bb40092e0 tx=0x7f2bb400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:32:46.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:46.692+0000 7f2bc9ffb700 1 -- 172.21.15.22:0/2644625870 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2bb401d070 con 0x7f2bcc100e60 2024-02-16T19:32:46.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:46.692+0000 7f2bd2be1700 1 -- 172.21.15.22:0/2644625870 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bcc073e90 con 0x7f2bcc100e60 2024-02-16T19:32:46.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:46.692+0000 7f2bc9ffb700 1 -- 172.21.15.22:0/2644625870 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2bb400f460 con 0x7f2bcc100e60 2024-02-16T19:32:46.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:46.692+0000 7f2bc9ffb700 1 -- 172.21.15.22:0/2644625870 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2bb40175d0 con 0x7f2bcc100e60 2024-02-16T19:32:46.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:46.693+0000 7f2bd2be1700 1 -- 172.21.15.22:0/2644625870 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bcc1a2480 con 0x7f2bcc100e60 2024-02-16T19:32:46.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:46.694+0000 7f2bd2be1700 1 -- 172.21.15.22:0/2644625870 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bcc108780 con 0x7f2bcc100e60 2024-02-16T19:32:46.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:46.694+0000 7f2bc9ffb700 1 -- 172.21.15.22:0/2644625870 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 33) v1 ==== 91346+0+0 (secure 0 0 0) 0x7f2bb400f5d0 con 0x7f2bcc100e60 2024-02-16T19:32:46.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:46.694+0000 7f2bc9ffb700 1 --2- 172.21.15.22:0/2644625870 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f2bb806d520 0x7f2bb806f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:32:46.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:46.695+0000 7f2bc9ffb700 1 -- 172.21.15.22:0/2644625870 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f2bb408e410 con 0x7f2bcc100e60 2024-02-16T19:32:46.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:46.695+0000 7f2bd097d700 1 --2- 172.21.15.22:0/2644625870 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f2bb806d520 0x7f2bb806f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:32:46.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:46.695+0000 7f2bd097d700 1 --2- 172.21.15.22:0/2644625870 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f2bb806d520 0x7f2bb806f9d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc00a400 tx=0x7f2bbc00d040 comp rx=0 tx=0).ready entity=mgr.24407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:32:46.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:46.699+0000 7f2bc9ffb700 1 -- 172.21.15.22:0/2644625870 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f2bb40580f0 con 0x7f2bcc100e60 2024-02-16T19:32:46.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:46.907+0000 7f2bd2be1700 1 -- 172.21.15.22:0/2644625870 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command([{prefix=config set, name=mgr/orchestrator/fail_fs}] v 0) v1 -- 0x7f2bcc074660 con 0x7f2bcc100e60 2024-02-16T19:32:46.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:46.911+0000 7f2bc9ffb700 1 -- 172.21.15.22:0/2644625870 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/orchestrator/fail_fs}]=0 v44)=0 v44) v1 ==== 125+0+0 (secure 0 0 0) 0x7f2bcc074660 con 0x7f2bcc100e60 2024-02-16T19:32:46.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:46.914+0000 7f2bd2be1700 1 -- 172.21.15.22:0/2644625870 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f2bb806d520 msgr2=0x7f2bb806f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:46.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:46.914+0000 7f2bd2be1700 1 --2- 172.21.15.22:0/2644625870 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f2bb806d520 0x7f2bb806f9d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc00a400 tx=0x7f2bbc00d040 comp rx=0 tx=0).stop 2024-02-16T19:32:46.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:46.914+0000 7f2bd2be1700 1 -- 172.21.15.22:0/2644625870 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2bcc100e60 msgr2=0x7f2bcc073590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:46.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:46.914+0000 7f2bd2be1700 1 --2- 172.21.15.22:0/2644625870 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2bcc100e60 0x7f2bcc073590 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f2bb40092e0 tx=0x7f2bb400bd70 comp rx=0 tx=0).stop 2024-02-16T19:32:46.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:46.914+0000 7f2bd2be1700 1 -- 172.21.15.22:0/2644625870 shutdown_connections 2024-02-16T19:32:46.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:46.914+0000 7f2bd2be1700 1 --2- 172.21.15.22:0/2644625870 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f2bb806d520 0x7f2bb806f9d0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:46.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:46.914+0000 7f2bd2be1700 1 --2- 172.21.15.22:0/2644625870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bcc1004c0 0x7f2bcc073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:46.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:46.914+0000 7f2bd2be1700 1 --2- 172.21.15.22:0/2644625870 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2bcc100e60 0x7f2bcc073590 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:46.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:46.914+0000 7f2bd2be1700 1 -- 172.21.15.22:0/2644625870 >> 172.21.15.22:0/2644625870 conn(0x7f2bcc0fbd80 msgr2=0x7f2bcc0fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:32:46.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:46.914+0000 7f2bd2be1700 1 -- 172.21.15.22:0/2644625870 shutdown_connections 2024-02-16T19:32:46.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:46.914+0000 7f2bd2be1700 1 -- 172.21.15.22:0/2644625870 wait complete. 2024-02-16T19:32:47.564 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-02-16T19:32:47.564 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi022.front.sepia.ceph.com 2024-02-16T19:32:47.565 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-02-16T19:32:47.779 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi022/config 2024-02-16T19:32:48.202 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:47 smithi022 ceph-mon[29221]: from='client.? 172.21.15.22:0/2644625870' entity='client.admin' 2024-02-16T19:32:48.202 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:47 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:32:48.202 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:47 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:32:48.202 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:47 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:32:48.202 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:47 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:32:48.202 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:47 smithi022 ceph-mon[29221]: pgmap v90: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:32:48.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:47 smithi117 ceph-mon[35309]: from='client.? 172.21.15.22:0/2644625870' entity='client.admin' 2024-02-16T19:32:48.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:47 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:32:48.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:47 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:32:48.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:47 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:32:48.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:47 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:32:48.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:47 smithi117 ceph-mon[35309]: pgmap v90: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:32:49.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:49.300+0000 7fb76333a700 1 -- 172.21.15.22:0/2128937619 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb75c0ff3c0 msgr2=0x7fb75c0ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:49.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:49.300+0000 7fb76333a700 1 --2- 172.21.15.22:0/2128937619 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb75c0ff3c0 0x7fb75c0ff790 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb758009910 tx=0x7fb758009c20 comp rx=0 tx=0).stop 2024-02-16T19:32:49.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:49.300+0000 7fb76333a700 1 -- 172.21.15.22:0/2128937619 shutdown_connections 2024-02-16T19:32:49.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:49.300+0000 7fb76333a700 1 --2- 172.21.15.22:0/2128937619 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb75c0ffd60 0x7fb75c103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:49.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:49.300+0000 7fb76333a700 1 --2- 172.21.15.22:0/2128937619 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb75c0ff3c0 0x7fb75c0ff790 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:49.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:49.300+0000 7fb76333a700 1 -- 172.21.15.22:0/2128937619 >> 172.21.15.22:0/2128937619 conn(0x7fb75c0fac80 msgr2=0x7fb75c0fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:32:49.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:49.301+0000 7fb76333a700 1 -- 172.21.15.22:0/2128937619 shutdown_connections 2024-02-16T19:32:49.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:49.301+0000 7fb76333a700 1 -- 172.21.15.22:0/2128937619 wait complete. 2024-02-16T19:32:49.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:49.301+0000 7fb76333a700 1 Processor -- start 2024-02-16T19:32:49.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:49.301+0000 7fb76333a700 1 -- start start 2024-02-16T19:32:49.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:49.301+0000 7fb76333a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb75c0ff3c0 0x7fb75c194d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:32:49.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:49.301+0000 7fb76333a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb75c0ffd60 0x7fb75c1952c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:32:49.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:49.301+0000 7fb76333a700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb75c1959a0 con 0x7fb75c0ffd60 2024-02-16T19:32:49.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:49.301+0000 7fb76333a700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb75c199730 con 0x7fb75c0ff3c0 2024-02-16T19:32:49.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:49.302+0000 7fb7608d5700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb75c0ffd60 0x7fb75c1952c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:32:49.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:49.302+0000 7fb7608d5700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb75c0ffd60 0x7fb75c1952c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46692/0 (socket says 172.21.15.22:46692) 2024-02-16T19:32:49.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:49.302+0000 7fb7608d5700 1 -- 172.21.15.22:0/2107902675 learned_addr learned my addr 172.21.15.22:0/2107902675 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:32:49.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:49.302+0000 7fb7610d6700 1 --2- 172.21.15.22:0/2107902675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb75c0ff3c0 0x7fb75c194d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:32:49.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:49.302+0000 7fb7608d5700 1 -- 172.21.15.22:0/2107902675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb75c0ff3c0 msgr2=0x7fb75c194d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:49.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:49.302+0000 7fb7608d5700 1 --2- 172.21.15.22:0/2107902675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb75c0ff3c0 0x7fb75c194d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:49.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:49.302+0000 7fb7608d5700 1 -- 172.21.15.22:0/2107902675 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7580095f0 con 0x7fb75c0ffd60 2024-02-16T19:32:49.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:49.302+0000 7fb7610d6700 1 --2- 172.21.15.22:0/2107902675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb75c0ff3c0 0x7fb75c194d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T19:32:49.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:49.302+0000 7fb7608d5700 1 --2- 172.21.15.22:0/2107902675 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb75c0ffd60 0x7fb75c1952c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb74c00d900 tx=0x7fb74c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:32:49.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:49.302+0000 7fb7527fc700 1 -- 172.21.15.22:0/2107902675 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb74c009900 con 0x7fb75c0ffd60 2024-02-16T19:32:49.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:49.302+0000 7fb7527fc700 1 -- 172.21.15.22:0/2107902675 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb74c010460 con 0x7fb75c0ffd60 2024-02-16T19:32:49.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:49.302+0000 7fb76333a700 1 -- 172.21.15.22:0/2107902675 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb75c199a10 con 0x7fb75c0ffd60 2024-02-16T19:32:49.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:49.302+0000 7fb7527fc700 1 -- 172.21.15.22:0/2107902675 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb74c00f5f0 con 0x7fb75c0ffd60 2024-02-16T19:32:49.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:49.302+0000 7fb76333a700 1 -- 172.21.15.22:0/2107902675 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb75c199f60 con 0x7fb75c0ffd60 2024-02-16T19:32:49.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:49.303+0000 7fb76333a700 1 -- 172.21.15.22:0/2107902675 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb75c1076d0 con 0x7fb75c0ffd60 2024-02-16T19:32:49.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:49.304+0000 7fb7527fc700 1 -- 172.21.15.22:0/2107902675 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 33) v1 ==== 91346+0+0 (secure 0 0 0) 0x7fb74c0105d0 con 0x7fb75c0ffd60 2024-02-16T19:32:49.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:49.304+0000 7fb7527fc700 1 --2- 172.21.15.22:0/2107902675 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7fb74806d520 0x7fb74806f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:32:49.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:49.304+0000 7fb7527fc700 1 -- 172.21.15.22:0/2107902675 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fb74c08cde0 con 0x7fb75c0ffd60 2024-02-16T19:32:49.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:49.304+0000 7fb7610d6700 1 --2- 172.21.15.22:0/2107902675 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7fb74806d520 0x7fb74806f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:32:49.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:49.305+0000 7fb7610d6700 1 --2- 172.21.15.22:0/2107902675 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7fb74806d520 0x7fb74806f9d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb75800b5c0 tx=0x7fb75801a040 comp rx=0 tx=0).ready entity=mgr.24407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:32:49.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:49.309+0000 7fb7527fc700 1 -- 172.21.15.22:0/2107902675 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7fb74c056ac0 con 0x7fb75c0ffd60 2024-02-16T19:32:49.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:49.512+0000 7fb76333a700 1 -- 172.21.15.22:0/2107902675 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7fb75c04f2a0 con 0x7fb75c0ffd60 2024-02-16T19:32:49.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:49.512+0000 7fb7527fc700 1 -- 172.21.15.22:0/2107902675 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v44)=0 v44) v1 ==== 155+0+0 (secure 0 0 0) 0x7fb74c05a180 con 0x7fb75c0ffd60 2024-02-16T19:32:49.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:49.514+0000 7fb76333a700 1 -- 172.21.15.22:0/2107902675 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7fb74806d520 msgr2=0x7fb74806f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:49.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:49.514+0000 7fb76333a700 1 --2- 172.21.15.22:0/2107902675 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7fb74806d520 0x7fb74806f9d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb75800b5c0 tx=0x7fb75801a040 comp rx=0 tx=0).stop 2024-02-16T19:32:49.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:49.514+0000 7fb76333a700 1 -- 172.21.15.22:0/2107902675 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb75c0ffd60 msgr2=0x7fb75c1952c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:49.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:49.514+0000 7fb76333a700 1 --2- 172.21.15.22:0/2107902675 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb75c0ffd60 0x7fb75c1952c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb74c00d900 tx=0x7fb74c00dcc0 comp rx=0 tx=0).stop 2024-02-16T19:32:49.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:49.514+0000 7fb76333a700 1 -- 172.21.15.22:0/2107902675 shutdown_connections 2024-02-16T19:32:49.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:49.514+0000 7fb76333a700 1 --2- 172.21.15.22:0/2107902675 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7fb74806d520 0x7fb74806f9d0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:49.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:49.514+0000 7fb76333a700 1 --2- 172.21.15.22:0/2107902675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb75c0ff3c0 0x7fb75c194d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:49.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:49.514+0000 7fb76333a700 1 --2- 172.21.15.22:0/2107902675 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb75c0ffd60 0x7fb75c1952c0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:49.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:49.514+0000 7fb76333a700 1 -- 172.21.15.22:0/2107902675 >> 172.21.15.22:0/2107902675 conn(0x7fb75c0fac80 msgr2=0x7fb75c0fc300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:32:49.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:49.514+0000 7fb76333a700 1 -- 172.21.15.22:0/2107902675 shutdown_connections 2024-02-16T19:32:49.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:49.514+0000 7fb76333a700 1 -- 172.21.15.22:0/2107902675 wait complete. 2024-02-16T19:32:50.203 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-02-16T19:32:50.416 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi022/config 2024-02-16T19:32:50.464 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:50 smithi022 ceph-mon[29221]: pgmap v91: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:32:50.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:50 smithi117 ceph-mon[35309]: pgmap v91: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:32:52.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:52.106+0000 7fccc5d3f700 1 -- 172.21.15.22:0/2059431500 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fccc01004e0 msgr2=0x7fccc01008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:52.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:52.106+0000 7fccc5d3f700 1 --2- 172.21.15.22:0/2059431500 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fccc01004e0 0x7fccc01008b0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fccb0009910 tx=0x7fccb0009c20 comp rx=0 tx=0).stop 2024-02-16T19:32:52.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:52.106+0000 7fccc5d3f700 1 -- 172.21.15.22:0/2059431500 shutdown_connections 2024-02-16T19:32:52.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:52.106+0000 7fccc5d3f700 1 --2- 172.21.15.22:0/2059431500 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccc0100e80 0x7fccc0104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:52.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:52.106+0000 7fccc5d3f700 1 --2- 172.21.15.22:0/2059431500 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fccc01004e0 0x7fccc01008b0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:52.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:52.106+0000 7fccc5d3f700 1 -- 172.21.15.22:0/2059431500 >> 172.21.15.22:0/2059431500 conn(0x7fccc00fbda0 msgr2=0x7fccc00fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:32:52.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:52.106+0000 7fccc5d3f700 1 -- 172.21.15.22:0/2059431500 shutdown_connections 2024-02-16T19:32:52.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:52.106+0000 7fccc5d3f700 1 -- 172.21.15.22:0/2059431500 wait complete. 2024-02-16T19:32:52.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:52.107+0000 7fccc5d3f700 1 Processor -- start 2024-02-16T19:32:52.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:52.107+0000 7fccc5d3f700 1 -- start start 2024-02-16T19:32:52.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:52.107+0000 7fccc5d3f700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fccc01004e0 0x7fccc0073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:32:52.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:52.107+0000 7fccc5d3f700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccc0100e80 0x7fccc0073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:32:52.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:52.107+0000 7fccc5d3f700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc0073ad0 con 0x7fccc01004e0 2024-02-16T19:32:52.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:52.107+0000 7fccc5d3f700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc0073c10 con 0x7fccc0100e80 2024-02-16T19:32:52.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:52.107+0000 7fccbf7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fccc01004e0 0x7fccc0073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:32:52.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:52.107+0000 7fccbf7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fccc01004e0 0x7fccc0073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46696/0 (socket says 172.21.15.22:46696) 2024-02-16T19:32:52.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:52.107+0000 7fccbf7fe700 1 -- 172.21.15.22:0/1207892209 learned_addr learned my addr 172.21.15.22:0/1207892209 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:32:52.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:52.107+0000 7fccbeffd700 1 --2- 172.21.15.22:0/1207892209 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccc0100e80 0x7fccc0073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:32:52.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:52.108+0000 7fccbf7fe700 1 -- 172.21.15.22:0/1207892209 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccc0100e80 msgr2=0x7fccc0073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:52.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:52.108+0000 7fccbf7fe700 1 --2- 172.21.15.22:0/1207892209 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccc0100e80 0x7fccc0073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:52.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:52.108+0000 7fccbf7fe700 1 -- 172.21.15.22:0/1207892209 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccb00095f0 con 0x7fccc01004e0 2024-02-16T19:32:52.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:52.108+0000 7fccbf7fe700 1 --2- 172.21.15.22:0/1207892209 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fccc01004e0 0x7fccc0073050 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fccb0006010 tx=0x7fccb000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:32:52.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:52.108+0000 7fccbcff9700 1 -- 172.21.15.22:0/1207892209 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fccb001d070 con 0x7fccc01004e0 2024-02-16T19:32:52.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:52.108+0000 7fccc5d3f700 1 -- 172.21.15.22:0/1207892209 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccc0073e90 con 0x7fccc01004e0 2024-02-16T19:32:52.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:52.108+0000 7fccbcff9700 1 -- 172.21.15.22:0/1207892209 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fccb0004fb0 con 0x7fccc01004e0 2024-02-16T19:32:52.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:52.108+0000 7fccbcff9700 1 -- 172.21.15.22:0/1207892209 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fccb0022ea0 con 0x7fccc01004e0 2024-02-16T19:32:52.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:52.108+0000 7fccc5d3f700 1 -- 172.21.15.22:0/1207892209 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccc01a2490 con 0x7fccc01004e0 2024-02-16T19:32:52.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:52.109+0000 7fccc5d3f700 1 -- 172.21.15.22:0/1207892209 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccc01087f0 con 0x7fccc01004e0 2024-02-16T19:32:52.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:52.109+0000 7fccbcff9700 1 -- 172.21.15.22:0/1207892209 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 33) v1 ==== 91346+0+0 (secure 0 0 0) 0x7fccb000f690 con 0x7fccc01004e0 2024-02-16T19:32:52.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:52.110+0000 7fccbcff9700 1 --2- 172.21.15.22:0/1207892209 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7fccac06d520 0x7fccac06f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:32:52.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:52.110+0000 7fccbcff9700 1 -- 172.21.15.22:0/1207892209 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fccb008e3d0 con 0x7fccc01004e0 2024-02-16T19:32:52.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:52.110+0000 7fccbeffd700 1 --2- 172.21.15.22:0/1207892209 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7fccac06d520 0x7fccac06f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:32:52.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:52.110+0000 7fccbeffd700 1 --2- 172.21.15.22:0/1207892209 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7fccac06d520 0x7fccac06f9d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcca8006fd0 tx=0x7fcca8008040 comp rx=0 tx=0).ready entity=mgr.24407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:32:52.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:52.115+0000 7fccbcff9700 1 -- 172.21.15.22:0/1207892209 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7fccb0058370 con 0x7fccc01004e0 2024-02-16T19:32:52.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:52.320+0000 7fccc5d3f700 1 -- 172.21.15.22:0/1207892209 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7fccc0074680 con 0x7fccc01004e0 2024-02-16T19:32:52.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:52.321+0000 7fccbcff9700 1 -- 172.21.15.22:0/1207892209 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v44)=0 v44) v1 ==== 163+0+0 (secure 0 0 0) 0x7fccb005ba30 con 0x7fccc01004e0 2024-02-16T19:32:52.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:52.323+0000 7fccc5d3f700 1 -- 172.21.15.22:0/1207892209 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7fccac06d520 msgr2=0x7fccac06f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:52.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:52.323+0000 7fccc5d3f700 1 --2- 172.21.15.22:0/1207892209 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7fccac06d520 0x7fccac06f9d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcca8006fd0 tx=0x7fcca8008040 comp rx=0 tx=0).stop 2024-02-16T19:32:52.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:52.323+0000 7fccc5d3f700 1 -- 172.21.15.22:0/1207892209 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fccc01004e0 msgr2=0x7fccc0073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:52.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:52.323+0000 7fccc5d3f700 1 --2- 172.21.15.22:0/1207892209 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fccc01004e0 0x7fccc0073050 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fccb0006010 tx=0x7fccb000be20 comp rx=0 tx=0).stop 2024-02-16T19:32:52.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:52.323+0000 7fccc5d3f700 1 -- 172.21.15.22:0/1207892209 shutdown_connections 2024-02-16T19:32:52.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:52.323+0000 7fccc5d3f700 1 --2- 172.21.15.22:0/1207892209 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7fccac06d520 0x7fccac06f9d0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:52.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:52.323+0000 7fccc5d3f700 1 --2- 172.21.15.22:0/1207892209 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fccc01004e0 0x7fccc0073050 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:52.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:52.323+0000 7fccc5d3f700 1 --2- 172.21.15.22:0/1207892209 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccc0100e80 0x7fccc0073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:52.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:52.323+0000 7fccc5d3f700 1 -- 172.21.15.22:0/1207892209 >> 172.21.15.22:0/1207892209 conn(0x7fccc00fbda0 msgr2=0x7fccc00fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:32:52.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:52.323+0000 7fccc5d3f700 1 -- 172.21.15.22:0/1207892209 shutdown_connections 2024-02-16T19:32:52.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:52.323+0000 7fccc5d3f700 1 -- 172.21.15.22:0/1207892209 wait complete. 2024-02-16T19:32:52.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:52 smithi022 ceph-mon[29221]: pgmap v92: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:32:52.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:52 smithi117 ceph-mon[35309]: pgmap v92: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:32:53.004 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph config set global log_to_journald false --force' 2024-02-16T19:32:53.224 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi022/config 2024-02-16T19:32:54.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:54 smithi022 ceph-mon[29221]: pgmap v93: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:32:54.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:54 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:32:54.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:54 smithi117 ceph-mon[35309]: pgmap v93: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:32:54.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:54 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:32:54.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:54.702+0000 7f8799635700 1 -- 172.21.15.22:0/1356480351 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87941004c0 msgr2=0x7f8794100890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:54.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:54.702+0000 7f8799635700 1 --2- 172.21.15.22:0/1356480351 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87941004c0 0x7f8794100890 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f8784009910 tx=0x7f8784009c20 comp rx=0 tx=0).stop 2024-02-16T19:32:54.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:54.702+0000 7f8799635700 1 -- 172.21.15.22:0/1356480351 shutdown_connections 2024-02-16T19:32:54.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:54.702+0000 7f8799635700 1 --2- 172.21.15.22:0/1356480351 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8794100e60 0x7f8794104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:54.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:54.702+0000 7f8799635700 1 --2- 172.21.15.22:0/1356480351 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87941004c0 0x7f8794100890 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:54.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:54.702+0000 7f8799635700 1 -- 172.21.15.22:0/1356480351 >> 172.21.15.22:0/1356480351 conn(0x7f87940fbd80 msgr2=0x7f87940fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:32:54.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:54.702+0000 7f8799635700 1 -- 172.21.15.22:0/1356480351 shutdown_connections 2024-02-16T19:32:54.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:54.702+0000 7f8799635700 1 -- 172.21.15.22:0/1356480351 wait complete. 2024-02-16T19:32:54.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:54.703+0000 7f8799635700 1 Processor -- start 2024-02-16T19:32:54.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:54.703+0000 7f8799635700 1 -- start start 2024-02-16T19:32:54.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:54.703+0000 7f8799635700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87941004c0 0x7f8794195f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:32:54.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:54.703+0000 7f8799635700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8794100e60 0x7f8794196470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:32:54.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:54.703+0000 7f8799635700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8794196b50 con 0x7f8794100e60 2024-02-16T19:32:54.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:54.703+0000 7f8799635700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f879419a8e0 con 0x7f87941004c0 2024-02-16T19:32:54.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:54.703+0000 7f87927fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8794100e60 0x7f8794196470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:32:54.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:54.703+0000 7f87927fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8794100e60 0x7f8794196470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49516/0 (socket says 172.21.15.22:49516) 2024-02-16T19:32:54.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:54.703+0000 7f87927fc700 1 -- 172.21.15.22:0/742428100 learned_addr learned my addr 172.21.15.22:0/742428100 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:32:54.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:54.703+0000 7f8792ffd700 1 --2- 172.21.15.22:0/742428100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87941004c0 0x7f8794195f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:32:54.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:54.704+0000 7f87927fc700 1 -- 172.21.15.22:0/742428100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87941004c0 msgr2=0x7f8794195f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:54.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:54.704+0000 7f87927fc700 1 --2- 172.21.15.22:0/742428100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87941004c0 0x7f8794195f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:54.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:54.704+0000 7f87927fc700 1 -- 172.21.15.22:0/742428100 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87840095f0 con 0x7f8794100e60 2024-02-16T19:32:54.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:54.704+0000 7f87927fc700 1 --2- 172.21.15.22:0/742428100 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8794100e60 0x7f8794196470 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f878800d8d0 tx=0x7f878800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:32:54.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:54.704+0000 7f877bfff700 1 -- 172.21.15.22:0/742428100 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87880041f0 con 0x7f8794100e60 2024-02-16T19:32:54.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:54.704+0000 7f877bfff700 1 -- 172.21.15.22:0/742428100 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8788007620 con 0x7f8794100e60 2024-02-16T19:32:54.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:54.704+0000 7f8799635700 1 -- 172.21.15.22:0/742428100 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f879419abc0 con 0x7f8794100e60 2024-02-16T19:32:54.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:54.704+0000 7f877bfff700 1 -- 172.21.15.22:0/742428100 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87880056b0 con 0x7f8794100e60 2024-02-16T19:32:54.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:54.704+0000 7f8799635700 1 -- 172.21.15.22:0/742428100 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f879419b110 con 0x7f8794100e60 2024-02-16T19:32:54.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:54.705+0000 7f8799635700 1 -- 172.21.15.22:0/742428100 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87941087d0 con 0x7f8794100e60 2024-02-16T19:32:54.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:54.706+0000 7f877bfff700 1 -- 172.21.15.22:0/742428100 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 33) v1 ==== 91346+0+0 (secure 0 0 0) 0x7f878800f770 con 0x7f8794100e60 2024-02-16T19:32:54.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:54.706+0000 7f877bfff700 1 --2- 172.21.15.22:0/742428100 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f877c06d520 0x7f877c06f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:32:54.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:54.706+0000 7f877bfff700 1 -- 172.21.15.22:0/742428100 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f878808ce80 con 0x7f8794100e60 2024-02-16T19:32:54.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:54.706+0000 7f8792ffd700 1 --2- 172.21.15.22:0/742428100 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f877c06d520 0x7f877c06f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:32:54.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:54.707+0000 7f8792ffd700 1 --2- 172.21.15.22:0/742428100 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f877c06d520 0x7f877c06f9d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8784006010 tx=0x7f878401a040 comp rx=0 tx=0).ready entity=mgr.24407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:32:54.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:54.711+0000 7f877bfff700 1 -- 172.21.15.22:0/742428100 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f8788056ac0 con 0x7f8794100e60 2024-02-16T19:32:54.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:54.918+0000 7f8799635700 1 -- 172.21.15.22:0/742428100 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7f879404f2a0 con 0x7f8794100e60 2024-02-16T19:32:54.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:54.919+0000 7f877bfff700 1 -- 172.21.15.22:0/742428100 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v44)=0 v44) v1 ==== 135+0+0 (secure 0 0 0) 0x7f878805a180 con 0x7f8794100e60 2024-02-16T19:32:54.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:54.921+0000 7f8799635700 1 -- 172.21.15.22:0/742428100 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f877c06d520 msgr2=0x7f877c06f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:54.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:54.921+0000 7f8799635700 1 --2- 172.21.15.22:0/742428100 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f877c06d520 0x7f877c06f9d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8784006010 tx=0x7f878401a040 comp rx=0 tx=0).stop 2024-02-16T19:32:54.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:54.921+0000 7f8799635700 1 -- 172.21.15.22:0/742428100 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8794100e60 msgr2=0x7f8794196470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:54.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:54.921+0000 7f8799635700 1 --2- 172.21.15.22:0/742428100 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8794100e60 0x7f8794196470 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f878800d8d0 tx=0x7f878800dbe0 comp rx=0 tx=0).stop 2024-02-16T19:32:54.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:54.921+0000 7f8799635700 1 -- 172.21.15.22:0/742428100 shutdown_connections 2024-02-16T19:32:54.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:54.921+0000 7f8799635700 1 --2- 172.21.15.22:0/742428100 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f877c06d520 0x7f877c06f9d0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:54.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:54.921+0000 7f8799635700 1 --2- 172.21.15.22:0/742428100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87941004c0 0x7f8794195f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:54.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:54.921+0000 7f8799635700 1 --2- 172.21.15.22:0/742428100 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8794100e60 0x7f8794196470 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:54.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:54.921+0000 7f8799635700 1 -- 172.21.15.22:0/742428100 >> 172.21.15.22:0/742428100 conn(0x7f87940fbd80 msgr2=0x7f87940fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:32:54.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:54.922+0000 7f8799635700 1 -- 172.21.15.22:0/742428100 shutdown_connections 2024-02-16T19:32:54.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:54.922+0000 7f8799635700 1 -- 172.21.15.22:0/742428100 wait complete. 2024-02-16T19:32:55.568 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1' 2024-02-16T19:32:55.789 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi022/config 2024-02-16T19:32:56.607 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:56 smithi022 ceph-mon[29221]: pgmap v94: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:32:56.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:56 smithi117 ceph-mon[35309]: pgmap v94: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:32:57.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:57.413+0000 7ff797b82700 1 -- 172.21.15.22:0/4156925201 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff7900ffc60 msgr2=0x7ff79010a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:57.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:57.413+0000 7ff797b82700 1 --2- 172.21.15.22:0/4156925201 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff7900ffc60 0x7ff79010a6d0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff780009910 tx=0x7ff780009c20 comp rx=0 tx=0).stop 2024-02-16T19:32:57.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:57.413+0000 7ff797b82700 1 -- 172.21.15.22:0/4156925201 shutdown_connections 2024-02-16T19:32:57.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:57.413+0000 7ff797b82700 1 --2- 172.21.15.22:0/4156925201 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff7900ffc60 0x7ff79010a6d0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:57.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:57.413+0000 7ff797b82700 1 --2- 172.21.15.22:0/4156925201 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7900ff350 0x7ff7900ff720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:57.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:57.413+0000 7ff797b82700 1 -- 172.21.15.22:0/4156925201 >> 172.21.15.22:0/4156925201 conn(0x7ff7900748b0 msgr2=0x7ff790074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:32:57.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:57.413+0000 7ff797b82700 1 -- 172.21.15.22:0/4156925201 shutdown_connections 2024-02-16T19:32:57.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:57.413+0000 7ff797b82700 1 -- 172.21.15.22:0/4156925201 wait complete. 2024-02-16T19:32:57.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:57.413+0000 7ff797b82700 1 Processor -- start 2024-02-16T19:32:57.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:57.414+0000 7ff797b82700 1 -- start start 2024-02-16T19:32:57.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:57.414+0000 7ff797b82700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7900ff350 0x7ff790195ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:32:57.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:57.414+0000 7ff797b82700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff7900ffc60 0x7ff790196420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:32:57.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:57.414+0000 7ff797b82700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff790196b00 con 0x7ff7900ffc60 2024-02-16T19:32:57.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:57.414+0000 7ff797b82700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff79019a890 con 0x7ff7900ff350 2024-02-16T19:32:57.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:57.414+0000 7ff79511d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff7900ffc60 0x7ff790196420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:32:57.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:57.414+0000 7ff79511d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff7900ffc60 0x7ff790196420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49538/0 (socket says 172.21.15.22:49538) 2024-02-16T19:32:57.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:57.414+0000 7ff79511d700 1 -- 172.21.15.22:0/2537231645 learned_addr learned my addr 172.21.15.22:0/2537231645 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:32:57.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:57.414+0000 7ff79591e700 1 --2- 172.21.15.22:0/2537231645 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7900ff350 0x7ff790195ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:32:57.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:57.414+0000 7ff79511d700 1 -- 172.21.15.22:0/2537231645 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7900ff350 msgr2=0x7ff790195ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:57.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:57.414+0000 7ff79511d700 1 --2- 172.21.15.22:0/2537231645 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7900ff350 0x7ff790195ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:57.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:57.414+0000 7ff79511d700 1 -- 172.21.15.22:0/2537231645 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7800095f0 con 0x7ff7900ffc60 2024-02-16T19:32:57.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:57.414+0000 7ff79511d700 1 --2- 172.21.15.22:0/2537231645 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff7900ffc60 0x7ff790196420 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff780000c00 tx=0x7ff78000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:32:57.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:57.415+0000 7ff786ffd700 1 -- 172.21.15.22:0/2537231645 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff78001d070 con 0x7ff7900ffc60 2024-02-16T19:32:57.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:57.415+0000 7ff797b82700 1 -- 172.21.15.22:0/2537231645 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff79019ab10 con 0x7ff7900ffc60 2024-02-16T19:32:57.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:57.415+0000 7ff786ffd700 1 -- 172.21.15.22:0/2537231645 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff780005dc0 con 0x7ff7900ffc60 2024-02-16T19:32:57.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:57.415+0000 7ff786ffd700 1 -- 172.21.15.22:0/2537231645 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff780017400 con 0x7ff7900ffc60 2024-02-16T19:32:57.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:57.415+0000 7ff797b82700 1 -- 172.21.15.22:0/2537231645 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff79019b000 con 0x7ff7900ffc60 2024-02-16T19:32:57.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:57.416+0000 7ff797b82700 1 -- 172.21.15.22:0/2537231645 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff790107dd0 con 0x7ff7900ffc60 2024-02-16T19:32:57.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:57.416+0000 7ff786ffd700 1 -- 172.21.15.22:0/2537231645 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 33) v1 ==== 91346+0+0 (secure 0 0 0) 0x7ff78000f460 con 0x7ff7900ffc60 2024-02-16T19:32:57.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:57.417+0000 7ff786ffd700 1 --2- 172.21.15.22:0/2537231645 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7ff77c06d520 0x7ff77c06f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:32:57.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:57.417+0000 7ff786ffd700 1 -- 172.21.15.22:0/2537231645 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7ff78008e400 con 0x7ff7900ffc60 2024-02-16T19:32:57.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:57.417+0000 7ff79591e700 1 --2- 172.21.15.22:0/2537231645 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7ff77c06d520 0x7ff77c06f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:32:57.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:57.417+0000 7ff79591e700 1 --2- 172.21.15.22:0/2537231645 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7ff77c06d520 0x7ff77c06f9d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff79006a640 tx=0x7ff78c00b410 comp rx=0 tx=0).ready entity=mgr.24407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:32:57.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:57.421+0000 7ff786ffd700 1 -- 172.21.15.22:0/2537231645 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7ff7800580e0 con 0x7ff7900ffc60 2024-02-16T19:32:57.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:57.648+0000 7ff797b82700 1 -- 172.21.15.22:0/2537231645 --> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:f78a58c0ffd401d1493058a1022c35f011d65275", "target": ["mon-mgr", ""]}) v1 -- 0x7ff7901972b0 con 0x7ff77c06d520 2024-02-16T19:32:57.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:57.653+0000 7ff786ffd700 1 -- 172.21.15.22:0/2537231645 <== mgr.24407 v2:172.21.15.22:6800/3703806543 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7ff7901972b0 con 0x7ff77c06d520 2024-02-16T19:32:57.655 INFO:teuthology.orchestra.run.smithi022.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:f78a58c0ffd401d1493058a1022c35f011d65275 2024-02-16T19:32:57.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:57.655+0000 7ff797b82700 1 -- 172.21.15.22:0/2537231645 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7ff77c06d520 msgr2=0x7ff77c06f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:57.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:57.656+0000 7ff797b82700 1 --2- 172.21.15.22:0/2537231645 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7ff77c06d520 0x7ff77c06f9d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff79006a640 tx=0x7ff78c00b410 comp rx=0 tx=0).stop 2024-02-16T19:32:57.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:57.656+0000 7ff797b82700 1 -- 172.21.15.22:0/2537231645 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff7900ffc60 msgr2=0x7ff790196420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:57.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:57.656+0000 7ff797b82700 1 --2- 172.21.15.22:0/2537231645 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff7900ffc60 0x7ff790196420 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff780000c00 tx=0x7ff78000bd70 comp rx=0 tx=0).stop 2024-02-16T19:32:57.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:57.656+0000 7ff797b82700 1 -- 172.21.15.22:0/2537231645 shutdown_connections 2024-02-16T19:32:57.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:57.656+0000 7ff797b82700 1 --2- 172.21.15.22:0/2537231645 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7ff77c06d520 0x7ff77c06f9d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:57.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:57.656+0000 7ff797b82700 1 --2- 172.21.15.22:0/2537231645 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7900ff350 0x7ff790195ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:57.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:57.656+0000 7ff797b82700 1 --2- 172.21.15.22:0/2537231645 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff7900ffc60 0x7ff790196420 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:57.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:57.656+0000 7ff797b82700 1 -- 172.21.15.22:0/2537231645 >> 172.21.15.22:0/2537231645 conn(0x7ff7900748b0 msgr2=0x7ff790104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:32:57.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:57.656+0000 7ff797b82700 1 -- 172.21.15.22:0/2537231645 shutdown_connections 2024-02-16T19:32:57.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:32:57.656+0000 7ff797b82700 1 -- 172.21.15.22:0/2537231645 wait complete. 2024-02-16T19:32:58.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:57 smithi022 ceph-mon[29221]: pgmap v95: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:32:58.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:57 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:32:58.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:57 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:32:58.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:57 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:32:58.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:57 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:32:58.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:57 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:32:58.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:57 smithi117 ceph-mon[35309]: pgmap v95: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:32:58.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:57 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:32:58.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:57 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:32:58.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:57 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:32:58.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:57 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:32:58.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:57 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:32:58.401 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-02-16T19:32:58.402 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi022.front.sepia.ceph.com 2024-02-16T19:32:58.402 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done' 2024-02-16T19:32:58.632 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi022/config 2024-02-16T19:32:59.171 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:58 smithi022 ceph-mon[29221]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:f78a58c0ffd401d1493058a1022c35f011d65275", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:32:59.171 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:58 smithi022 ceph-mon[29221]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:f78a58c0ffd401d1493058a1022c35f011d65275 2024-02-16T19:32:59.171 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:58 smithi022 ceph-mon[29221]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:f78a58c0ffd401d1493058a1022c35f011d65275 2024-02-16T19:32:59.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:58 smithi117 ceph-mon[35309]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:f78a58c0ffd401d1493058a1022c35f011d65275", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:32:59.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:58 smithi117 ceph-mon[35309]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:f78a58c0ffd401d1493058a1022c35f011d65275 2024-02-16T19:32:59.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:58 smithi117 ceph-mon[35309]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:f78a58c0ffd401d1493058a1022c35f011d65275 2024-02-16T19:32:59.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:32:59 smithi022 ceph-mon[29221]: pgmap v96: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:33:00.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:32:59 smithi117 ceph-mon[35309]: pgmap v96: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:33:00.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.593+0000 7fde9c6b6700 1 -- 172.21.15.22:0/1823598969 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fde940ffbd0 msgr2=0x7fde94100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:00.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.593+0000 7fde9c6b6700 1 --2- 172.21.15.22:0/1823598969 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fde940ffbd0 0x7fde94100040 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fde84009910 tx=0x7fde84009c20 comp rx=0 tx=0).stop 2024-02-16T19:33:00.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.593+0000 7fde9c6b6700 1 -- 172.21.15.22:0/1823598969 shutdown_connections 2024-02-16T19:33:00.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.593+0000 7fde9c6b6700 1 --2- 172.21.15.22:0/1823598969 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fde940ffbd0 0x7fde94100040 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:00.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.593+0000 7fde9c6b6700 1 --2- 172.21.15.22:0/1823598969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde94104130 0x7fde94104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:00.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.593+0000 7fde9c6b6700 1 -- 172.21.15.22:0/1823598969 >> 172.21.15.22:0/1823598969 conn(0x7fde94075130 msgr2=0x7fde94075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:33:00.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.593+0000 7fde9c6b6700 1 -- 172.21.15.22:0/1823598969 shutdown_connections 2024-02-16T19:33:00.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.593+0000 7fde9c6b6700 1 -- 172.21.15.22:0/1823598969 wait complete. 2024-02-16T19:33:00.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.594+0000 7fde9c6b6700 1 Processor -- start 2024-02-16T19:33:00.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.594+0000 7fde9c6b6700 1 -- start start 2024-02-16T19:33:00.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.594+0000 7fde9c6b6700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fde940ffbd0 0x7fde94194d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:00.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.594+0000 7fde9c6b6700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde94104130 0x7fde94195270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:00.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.594+0000 7fde9c6b6700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde94195950 con 0x7fde940ffbd0 2024-02-16T19:33:00.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.594+0000 7fde9c6b6700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde941996e0 con 0x7fde94104130 2024-02-16T19:33:00.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.594+0000 7fde9a452700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fde940ffbd0 0x7fde94194d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:00.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.594+0000 7fde9a452700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fde940ffbd0 0x7fde94194d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49564/0 (socket says 172.21.15.22:49564) 2024-02-16T19:33:00.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.594+0000 7fde9a452700 1 -- 172.21.15.22:0/507961832 learned_addr learned my addr 172.21.15.22:0/507961832 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:33:00.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.594+0000 7fde99c51700 1 --2- 172.21.15.22:0/507961832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde94104130 0x7fde94195270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:00.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.594+0000 7fde9a452700 1 -- 172.21.15.22:0/507961832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde94104130 msgr2=0x7fde94195270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:00.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.594+0000 7fde9a452700 1 --2- 172.21.15.22:0/507961832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde94104130 0x7fde94195270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:00.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.594+0000 7fde9a452700 1 -- 172.21.15.22:0/507961832 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde840095f0 con 0x7fde940ffbd0 2024-02-16T19:33:00.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.595+0000 7fde9a452700 1 --2- 172.21.15.22:0/507961832 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fde940ffbd0 0x7fde94194d30 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fde9000d900 tx=0x7fde9000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:33:00.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.595+0000 7fde8b7fe700 1 -- 172.21.15.22:0/507961832 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fde900041f0 con 0x7fde940ffbd0 2024-02-16T19:33:00.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.595+0000 7fde8b7fe700 1 -- 172.21.15.22:0/507961832 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fde90004cf0 con 0x7fde940ffbd0 2024-02-16T19:33:00.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.595+0000 7fde9c6b6700 1 -- 172.21.15.22:0/507961832 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde941999c0 con 0x7fde940ffbd0 2024-02-16T19:33:00.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.595+0000 7fde8b7fe700 1 -- 172.21.15.22:0/507961832 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fde9000b750 con 0x7fde940ffbd0 2024-02-16T19:33:00.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.595+0000 7fde9c6b6700 1 -- 172.21.15.22:0/507961832 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde94199f10 con 0x7fde940ffbd0 2024-02-16T19:33:00.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.596+0000 7fde9c6b6700 1 -- 172.21.15.22:0/507961832 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde9404f2a0 con 0x7fde940ffbd0 2024-02-16T19:33:00.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.597+0000 7fde8b7fe700 1 -- 172.21.15.22:0/507961832 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 33) v1 ==== 91346+0+0 (secure 0 0 0) 0x7fde90004750 con 0x7fde940ffbd0 2024-02-16T19:33:00.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.597+0000 7fde8b7fe700 1 --2- 172.21.15.22:0/507961832 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7fde8006d520 0x7fde8006f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:00.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.597+0000 7fde8b7fe700 1 -- 172.21.15.22:0/507961832 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fde9008ce40 con 0x7fde940ffbd0 2024-02-16T19:33:00.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.597+0000 7fde99c51700 1 --2- 172.21.15.22:0/507961832 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7fde8006d520 0x7fde8006f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:00.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.598+0000 7fde99c51700 1 --2- 172.21.15.22:0/507961832 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7fde8006d520 0x7fde8006f9d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fde94196350 tx=0x7fde8401a040 comp rx=0 tx=0).ready entity=mgr.24407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:33:00.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.602+0000 7fde8b7fe700 1 -- 172.21.15.22:0/507961832 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7fde90056b20 con 0x7fde940ffbd0 2024-02-16T19:33:00.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.832+0000 7fde9c6b6700 1 -- 172.21.15.22:0/507961832 --> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fde9419a2d0 con 0x7fde8006d520 2024-02-16T19:33:00.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.833+0000 7fde8b7fe700 1 -- 172.21.15.22:0/507961832 <== mgr.24407 v2:172.21.15.22:6800/3703806543 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7fde9419a2d0 con 0x7fde8006d520 2024-02-16T19:33:00.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.835+0000 7fde9c6b6700 1 -- 172.21.15.22:0/507961832 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7fde8006d520 msgr2=0x7fde8006f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:00.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.835+0000 7fde9c6b6700 1 --2- 172.21.15.22:0/507961832 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7fde8006d520 0x7fde8006f9d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fde94196350 tx=0x7fde8401a040 comp rx=0 tx=0).stop 2024-02-16T19:33:00.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.835+0000 7fde9c6b6700 1 -- 172.21.15.22:0/507961832 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fde940ffbd0 msgr2=0x7fde94194d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:00.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.835+0000 7fde9c6b6700 1 --2- 172.21.15.22:0/507961832 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fde940ffbd0 0x7fde94194d30 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fde9000d900 tx=0x7fde9000dc10 comp rx=0 tx=0).stop 2024-02-16T19:33:00.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.835+0000 7fde9c6b6700 1 -- 172.21.15.22:0/507961832 shutdown_connections 2024-02-16T19:33:00.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.835+0000 7fde9c6b6700 1 --2- 172.21.15.22:0/507961832 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7fde8006d520 0x7fde8006f9d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:00.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.835+0000 7fde9c6b6700 1 --2- 172.21.15.22:0/507961832 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fde940ffbd0 0x7fde94194d30 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:00.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.836+0000 7fde9c6b6700 1 --2- 172.21.15.22:0/507961832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde94104130 0x7fde94195270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:00.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.836+0000 7fde9c6b6700 1 -- 172.21.15.22:0/507961832 >> 172.21.15.22:0/507961832 conn(0x7fde94075130 msgr2=0x7fde940feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:33:00.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.836+0000 7fde9c6b6700 1 -- 172.21.15.22:0/507961832 shutdown_connections 2024-02-16T19:33:00.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.836+0000 7fde9c6b6700 1 -- 172.21.15.22:0/507961832 wait complete. 2024-02-16T19:33:00.847 INFO:teuthology.orchestra.run.smithi022.stdout:true 2024-02-16T19:33:00.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.943+0000 7fa00141e700 1 -- 172.21.15.22:0/2984691315 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ffc1004c0 msgr2=0x7f9ffc100890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:00.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.943+0000 7fa00141e700 1 --2- 172.21.15.22:0/2984691315 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ffc1004c0 0x7f9ffc100890 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f9fec009910 tx=0x7f9fec009c20 comp rx=0 tx=0).stop 2024-02-16T19:33:00.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.943+0000 7fa00141e700 1 -- 172.21.15.22:0/2984691315 shutdown_connections 2024-02-16T19:33:00.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.943+0000 7fa00141e700 1 --2- 172.21.15.22:0/2984691315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ffc100e60 0x7f9ffc104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:00.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.943+0000 7fa00141e700 1 --2- 172.21.15.22:0/2984691315 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ffc1004c0 0x7f9ffc100890 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:00.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.943+0000 7fa00141e700 1 -- 172.21.15.22:0/2984691315 >> 172.21.15.22:0/2984691315 conn(0x7f9ffc0fbd80 msgr2=0x7f9ffc0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:33:00.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.944+0000 7fa00141e700 1 -- 172.21.15.22:0/2984691315 shutdown_connections 2024-02-16T19:33:00.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.944+0000 7fa00141e700 1 -- 172.21.15.22:0/2984691315 wait complete. 2024-02-16T19:33:00.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.944+0000 7fa00141e700 1 Processor -- start 2024-02-16T19:33:00.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.944+0000 7fa00141e700 1 -- start start 2024-02-16T19:33:00.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.945+0000 7fa00141e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ffc1004c0 0x7f9ffc073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:00.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.945+0000 7fa00141e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ffc100e60 0x7f9ffc073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:00.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.945+0000 7fa00141e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ffc073ad0 con 0x7f9ffc100e60 2024-02-16T19:33:00.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.945+0000 7fa00141e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ffc073c10 con 0x7f9ffc1004c0 2024-02-16T19:33:00.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.945+0000 7f9ffa7fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ffc100e60 0x7f9ffc073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:00.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.945+0000 7f9ffa7fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ffc100e60 0x7f9ffc073590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49590/0 (socket says 172.21.15.22:49590) 2024-02-16T19:33:00.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.945+0000 7f9ffa7fc700 1 -- 172.21.15.22:0/455442137 learned_addr learned my addr 172.21.15.22:0/455442137 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:33:00.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.945+0000 7f9ffaffd700 1 --2- 172.21.15.22:0/455442137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ffc1004c0 0x7f9ffc073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:00.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.945+0000 7f9ffa7fc700 1 -- 172.21.15.22:0/455442137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ffc1004c0 msgr2=0x7f9ffc073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:00.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.945+0000 7f9ffa7fc700 1 --2- 172.21.15.22:0/455442137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ffc1004c0 0x7f9ffc073050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:00.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.945+0000 7f9ffa7fc700 1 -- 172.21.15.22:0/455442137 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fec0095f0 con 0x7f9ffc100e60 2024-02-16T19:33:00.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.945+0000 7f9ffaffd700 1 --2- 172.21.15.22:0/455442137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ffc1004c0 0x7f9ffc073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-16T19:33:00.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.945+0000 7f9ffa7fc700 1 --2- 172.21.15.22:0/455442137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ffc100e60 0x7f9ffc073590 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f9ff000e8f0 tx=0x7f9ff000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:33:00.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.945+0000 7f9fe3fff700 1 -- 172.21.15.22:0/455442137 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9ff0009790 con 0x7f9ffc100e60 2024-02-16T19:33:00.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.946+0000 7fa00141e700 1 -- 172.21.15.22:0/455442137 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ffc1a2100 con 0x7f9ffc100e60 2024-02-16T19:33:00.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.946+0000 7f9fe3fff700 1 -- 172.21.15.22:0/455442137 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9ff0004cf0 con 0x7f9ffc100e60 2024-02-16T19:33:00.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.946+0000 7f9fe3fff700 1 -- 172.21.15.22:0/455442137 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9ff0005690 con 0x7f9ffc100e60 2024-02-16T19:33:00.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.946+0000 7fa00141e700 1 -- 172.21.15.22:0/455442137 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ffc1a2510 con 0x7f9ffc100e60 2024-02-16T19:33:00.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.947+0000 7fa00141e700 1 -- 172.21.15.22:0/455442137 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ffc1087d0 con 0x7f9ffc100e60 2024-02-16T19:33:00.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.947+0000 7f9fe3fff700 1 -- 172.21.15.22:0/455442137 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 33) v1 ==== 91346+0+0 (secure 0 0 0) 0x7f9ff0004750 con 0x7f9ffc100e60 2024-02-16T19:33:00.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.948+0000 7f9fe3fff700 1 --2- 172.21.15.22:0/455442137 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f9fe406d520 0x7f9fe406f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:00.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.948+0000 7f9fe3fff700 1 -- 172.21.15.22:0/455442137 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f9ff0014070 con 0x7f9ffc100e60 2024-02-16T19:33:00.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.948+0000 7f9ffaffd700 1 --2- 172.21.15.22:0/455442137 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f9fe406d520 0x7f9fe406f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:00.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.948+0000 7f9ffaffd700 1 --2- 172.21.15.22:0/455442137 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f9fe406d520 0x7f9fe406f9d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9fec00b760 tx=0x7f9fec005fb0 comp rx=0 tx=0).ready entity=mgr.24407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:33:00.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:00.952+0000 7f9fe3fff700 1 -- 172.21.15.22:0/455442137 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f9ff0057050 con 0x7f9ffc100e60 2024-02-16T19:33:01.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.190+0000 7fa00141e700 1 -- 172.21.15.22:0/455442137 --> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9ffc061190 con 0x7f9fe406d520 2024-02-16T19:33:01.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.191+0000 7f9fe3fff700 1 -- 172.21.15.22:0/455442137 <== mgr.24407 v2:172.21.15.22:6800/3703806543 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7f9ffc061190 con 0x7f9fe406d520 2024-02-16T19:33:01.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.194+0000 7fa00141e700 1 -- 172.21.15.22:0/455442137 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f9fe406d520 msgr2=0x7f9fe406f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:01.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.194+0000 7fa00141e700 1 --2- 172.21.15.22:0/455442137 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f9fe406d520 0x7f9fe406f9d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9fec00b760 tx=0x7f9fec005fb0 comp rx=0 tx=0).stop 2024-02-16T19:33:01.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.194+0000 7fa00141e700 1 -- 172.21.15.22:0/455442137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ffc100e60 msgr2=0x7f9ffc073590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:01.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.194+0000 7fa00141e700 1 --2- 172.21.15.22:0/455442137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ffc100e60 0x7f9ffc073590 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f9ff000e8f0 tx=0x7f9ff000ecb0 comp rx=0 tx=0).stop 2024-02-16T19:33:01.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.194+0000 7fa00141e700 1 -- 172.21.15.22:0/455442137 shutdown_connections 2024-02-16T19:33:01.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.194+0000 7fa00141e700 1 --2- 172.21.15.22:0/455442137 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f9fe406d520 0x7f9fe406f9d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:01.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.194+0000 7fa00141e700 1 --2- 172.21.15.22:0/455442137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ffc1004c0 0x7f9ffc073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:01.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.194+0000 7fa00141e700 1 --2- 172.21.15.22:0/455442137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ffc100e60 0x7f9ffc073590 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:01.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.194+0000 7fa00141e700 1 -- 172.21.15.22:0/455442137 >> 172.21.15.22:0/455442137 conn(0x7f9ffc0fbd80 msgr2=0x7f9ffc0fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:33:01.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.194+0000 7fa00141e700 1 -- 172.21.15.22:0/455442137 shutdown_connections 2024-02-16T19:33:01.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.194+0000 7fa00141e700 1 -- 172.21.15.22:0/455442137 wait complete. 2024-02-16T19:33:01.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.301+0000 7f9094c27700 1 -- 172.21.15.22:0/1426045659 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f90900ffd60 msgr2=0x7f9090103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:01.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.301+0000 7f9094c27700 1 --2- 172.21.15.22:0/1426045659 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f90900ffd60 0x7f9090103d30 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f9084009910 tx=0x7f9084009c20 comp rx=0 tx=0).stop 2024-02-16T19:33:01.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.302+0000 7f9094c27700 1 -- 172.21.15.22:0/1426045659 shutdown_connections 2024-02-16T19:33:01.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.302+0000 7f9094c27700 1 --2- 172.21.15.22:0/1426045659 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f90900ffd60 0x7f9090103d30 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:01.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.302+0000 7f9094c27700 1 --2- 172.21.15.22:0/1426045659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90900ff3c0 0x7f90900ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:01.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.302+0000 7f9094c27700 1 -- 172.21.15.22:0/1426045659 >> 172.21.15.22:0/1426045659 conn(0x7f90900fac60 msgr2=0x7f90900fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:33:01.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.302+0000 7f9094c27700 1 -- 172.21.15.22:0/1426045659 shutdown_connections 2024-02-16T19:33:01.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.302+0000 7f9094c27700 1 -- 172.21.15.22:0/1426045659 wait complete. 2024-02-16T19:33:01.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.302+0000 7f9094c27700 1 Processor -- start 2024-02-16T19:33:01.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.302+0000 7f9094c27700 1 -- start start 2024-02-16T19:33:01.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.303+0000 7f9094c27700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f90900ff3c0 0x7f9090193d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:01.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.303+0000 7f9094c27700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90900ffd60 0x7f9090194250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:01.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.303+0000 7f9094c27700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9090194930 con 0x7f90900ff3c0 2024-02-16T19:33:01.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.303+0000 7f9094c27700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90901986c0 con 0x7f90900ffd60 2024-02-16T19:33:01.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.303+0000 7f908e59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f90900ff3c0 0x7f9090193d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:01.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.303+0000 7f908e59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f90900ff3c0 0x7f9090193d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49610/0 (socket says 172.21.15.22:49610) 2024-02-16T19:33:01.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.303+0000 7f908e59c700 1 -- 172.21.15.22:0/262664827 learned_addr learned my addr 172.21.15.22:0/262664827 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:33:01.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.303+0000 7f908dd9b700 1 --2- 172.21.15.22:0/262664827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90900ffd60 0x7f9090194250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:01.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.303+0000 7f908e59c700 1 -- 172.21.15.22:0/262664827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90900ffd60 msgr2=0x7f9090194250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:01.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.303+0000 7f908e59c700 1 --2- 172.21.15.22:0/262664827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90900ffd60 0x7f9090194250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:01.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.303+0000 7f908e59c700 1 -- 172.21.15.22:0/262664827 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90840095f0 con 0x7f90900ff3c0 2024-02-16T19:33:01.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.303+0000 7f908e59c700 1 --2- 172.21.15.22:0/262664827 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f90900ff3c0 0x7f9090193d10 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f90800099f0 tx=0x7f9080009d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:33:01.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.304+0000 7f907f7fe700 1 -- 172.21.15.22:0/262664827 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f908000d7a0 con 0x7f90900ff3c0 2024-02-16T19:33:01.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.304+0000 7f9094c27700 1 -- 172.21.15.22:0/262664827 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90901989a0 con 0x7f90900ff3c0 2024-02-16T19:33:01.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.304+0000 7f907f7fe700 1 -- 172.21.15.22:0/262664827 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9080013040 con 0x7f90900ff3c0 2024-02-16T19:33:01.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.304+0000 7f907f7fe700 1 -- 172.21.15.22:0/262664827 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9080016350 con 0x7f90900ff3c0 2024-02-16T19:33:01.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.304+0000 7f9094c27700 1 -- 172.21.15.22:0/262664827 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9090198ec0 con 0x7f90900ff3c0 2024-02-16T19:33:01.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.305+0000 7f9094c27700 1 -- 172.21.15.22:0/262664827 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9090107730 con 0x7f90900ff3c0 2024-02-16T19:33:01.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.305+0000 7f907f7fe700 1 -- 172.21.15.22:0/262664827 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 33) v1 ==== 91346+0+0 (secure 0 0 0) 0x7f90800165c0 con 0x7f90900ff3c0 2024-02-16T19:33:01.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.306+0000 7f907f7fe700 1 --2- 172.21.15.22:0/262664827 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f9078071930 0x7f9078073de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:01.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.306+0000 7f907f7fe700 1 -- 172.21.15.22:0/262664827 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f908008ccd0 con 0x7f90900ff3c0 2024-02-16T19:33:01.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.306+0000 7f908dd9b700 1 --2- 172.21.15.22:0/262664827 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f9078071930 0x7f9078073de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:01.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.307+0000 7f908dd9b700 1 --2- 172.21.15.22:0/262664827 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f9078071930 0x7f9078073de0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f90840059d0 tx=0x7f9084005940 comp rx=0 tx=0).ready entity=mgr.24407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:33:01.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.311+0000 7f907f7fe700 1 -- 172.21.15.22:0/262664827 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f90800569b0 con 0x7f90900ff3c0 2024-02-16T19:33:01.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.528+0000 7f9094c27700 1 -- 172.21.15.22:0/262664827 --> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9090195180 con 0x7f9078071930 2024-02-16T19:33:01.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.540+0000 7f907f7fe700 1 -- 172.21.15.22:0/262664827 <== mgr.24407 v2:172.21.15.22:6800/3703806543 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f9090195180 con 0x7f9078071930 2024-02-16T19:33:01.541 INFO:teuthology.orchestra.run.smithi022.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T19:33:01.541 INFO:teuthology.orchestra.run.smithi022.stdout:alertmanager.smithi022 smithi022 *:9093,9094 running (83s) 51s ago 12m 15.7M - 0.25.0 c8568f914cd2 c89bc76be374 2024-02-16T19:33:01.541 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-exporter.smithi022 smithi022 running (12m) 51s ago 12m 7910k - 18.2.0 dc2bc1663786 3f8bef4349ba 2024-02-16T19:33:01.541 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-exporter.smithi117 smithi117 running (11m) 114s ago 11m 7507k - 18.2.0 dc2bc1663786 59b65a7e309f 2024-02-16T19:33:01.541 INFO:teuthology.orchestra.run.smithi022.stdout:crash.smithi022 smithi022 running (12m) 51s ago 12m 6669k - 18.2.0 dc2bc1663786 efef476aa486 2024-02-16T19:33:01.541 INFO:teuthology.orchestra.run.smithi022.stdout:crash.smithi117 smithi117 running (11m) 114s ago 11m 6660k - 18.2.0 dc2bc1663786 5466453eb149 2024-02-16T19:33:01.541 INFO:teuthology.orchestra.run.smithi022.stdout:grafana.smithi022 smithi022 *:3000 running (57s) 51s ago 12m 52.6M - 9.4.12 a0ea5e3c46d9 fd27d11e9729 2024-02-16T19:33:01.542 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi022.btppwm smithi022 running (6m) 51s ago 6m 158M - 18.2.0 dc2bc1663786 22971fb3a907 2024-02-16T19:33:01.542 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi022.ocukvs smithi022 running (6m) 51s ago 6m 14.7M - 18.2.0 dc2bc1663786 81ee7bd5fd39 2024-02-16T19:33:01.542 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi117.ibmppn smithi117 running (6m) 114s ago 6m 16.1M - 18.2.0 dc2bc1663786 8c606cbeeb86 2024-02-16T19:33:01.542 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi117.orzors smithi117 running (6m) 114s ago 6m 15.1M - 18.2.0 dc2bc1663786 7d2bc4df3feb 2024-02-16T19:33:01.542 INFO:teuthology.orchestra.run.smithi022.stdout:mgr.smithi022.gmdcch smithi022 *:8443,9283,8765 running (3m) 51s ago 13m 499M - 19.0.0-1450-gf78a58c0 f081f847070a 7cb74cc9deea 2024-02-16T19:33:01.542 INFO:teuthology.orchestra.run.smithi022.stdout:mgr.smithi117.cohbtv smithi117 *:8443,9283,8765 running (2m) 114s ago 11m 450M - 19.0.0-1450-gf78a58c0 f081f847070a c825b1152189 2024-02-16T19:33:01.542 INFO:teuthology.orchestra.run.smithi022.stdout:mon.smithi022 smithi022 running (13m) 51s ago 13m 57.9M 2048M 18.2.0 dc2bc1663786 b4ffc3032e12 2024-02-16T19:33:01.542 INFO:teuthology.orchestra.run.smithi022.stdout:mon.smithi117 smithi117 running (11m) 114s ago 11m 44.6M 2048M 18.2.0 dc2bc1663786 aff3e21a73a0 2024-02-16T19:33:01.542 INFO:teuthology.orchestra.run.smithi022.stdout:node-exporter.smithi022 smithi022 *:9100 running (2m) 51s ago 12m 8887k - 1.5.0 0da6a335fe13 1f5b58292d27 2024-02-16T19:33:01.542 INFO:teuthology.orchestra.run.smithi022.stdout:node-exporter.smithi117 smithi117 *:9100 running (118s) 114s ago 11m 3275k - 1.5.0 0da6a335fe13 bb5e8491d5a6 2024-02-16T19:33:01.542 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0 smithi022 running (10m) 51s ago 10m 209M 1217M 18.2.0 dc2bc1663786 0027f86f16ce 2024-02-16T19:33:01.542 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1 smithi022 running (9m) 51s ago 9m 225M 1217M 18.2.0 dc2bc1663786 87300382b8a7 2024-02-16T19:33:01.542 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2 smithi022 running (9m) 51s ago 9m 185M 1217M 18.2.0 dc2bc1663786 be8a631b4f93 2024-02-16T19:33:01.542 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3 smithi117 running (8m) 114s ago 8m 242M 2240M 18.2.0 dc2bc1663786 9c0ce9a981cf 2024-02-16T19:33:01.542 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4 smithi117 running (8m) 114s ago 8m 208M 2240M 18.2.0 dc2bc1663786 1779c3496f4d 2024-02-16T19:33:01.543 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5 smithi117 running (8m) 114s ago 8m 194M 2240M 18.2.0 dc2bc1663786 fefaabca0638 2024-02-16T19:33:01.543 INFO:teuthology.orchestra.run.smithi022.stdout:prometheus.smithi022 smithi022 *:9095 running (100s) 51s ago 11m 51.1M - 2.43.0 a07b618ecd1d c3b8689d0c97 2024-02-16T19:33:01.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.542+0000 7f9094c27700 1 -- 172.21.15.22:0/262664827 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f9078071930 msgr2=0x7f9078073de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:01.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.542+0000 7f9094c27700 1 --2- 172.21.15.22:0/262664827 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f9078071930 0x7f9078073de0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f90840059d0 tx=0x7f9084005940 comp rx=0 tx=0).stop 2024-02-16T19:33:01.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.542+0000 7f9094c27700 1 -- 172.21.15.22:0/262664827 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f90900ff3c0 msgr2=0x7f9090193d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:01.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.542+0000 7f9094c27700 1 --2- 172.21.15.22:0/262664827 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f90900ff3c0 0x7f9090193d10 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f90800099f0 tx=0x7f9080009d00 comp rx=0 tx=0).stop 2024-02-16T19:33:01.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.542+0000 7f9094c27700 1 -- 172.21.15.22:0/262664827 shutdown_connections 2024-02-16T19:33:01.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.542+0000 7f9094c27700 1 --2- 172.21.15.22:0/262664827 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f9078071930 0x7f9078073de0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:01.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.542+0000 7f9094c27700 1 --2- 172.21.15.22:0/262664827 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f90900ff3c0 0x7f9090193d10 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:01.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.542+0000 7f9094c27700 1 --2- 172.21.15.22:0/262664827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90900ffd60 0x7f9090194250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:01.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.542+0000 7f9094c27700 1 -- 172.21.15.22:0/262664827 >> 172.21.15.22:0/262664827 conn(0x7f90900fac60 msgr2=0x7f90900fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:33:01.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.542+0000 7f9094c27700 1 -- 172.21.15.22:0/262664827 shutdown_connections 2024-02-16T19:33:01.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.542+0000 7f9094c27700 1 -- 172.21.15.22:0/262664827 wait complete. 2024-02-16T19:33:01.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.647+0000 7f25c93d2700 1 -- 172.21.15.22:0/916588407 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25c40ffbd0 msgr2=0x7f25c4100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:01.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.647+0000 7f25c93d2700 1 --2- 172.21.15.22:0/916588407 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25c40ffbd0 0x7f25c4100040 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f25b8009910 tx=0x7f25b8009c20 comp rx=0 tx=0).stop 2024-02-16T19:33:01.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.647+0000 7f25c93d2700 1 -- 172.21.15.22:0/916588407 shutdown_connections 2024-02-16T19:33:01.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.647+0000 7f25c93d2700 1 --2- 172.21.15.22:0/916588407 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25c40ffbd0 0x7f25c4100040 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:01.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.647+0000 7f25c93d2700 1 --2- 172.21.15.22:0/916588407 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25c4104130 0x7f25c4104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:01.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.647+0000 7f25c93d2700 1 -- 172.21.15.22:0/916588407 >> 172.21.15.22:0/916588407 conn(0x7f25c4075130 msgr2=0x7f25c4075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:33:01.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.647+0000 7f25c93d2700 1 -- 172.21.15.22:0/916588407 shutdown_connections 2024-02-16T19:33:01.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.647+0000 7f25c93d2700 1 -- 172.21.15.22:0/916588407 wait complete. 2024-02-16T19:33:01.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.648+0000 7f25c93d2700 1 Processor -- start 2024-02-16T19:33:01.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.648+0000 7f25c93d2700 1 -- start start 2024-02-16T19:33:01.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.648+0000 7f25c93d2700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25c40ffbd0 0x7f25c4194e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:01.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.648+0000 7f25c93d2700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25c4104130 0x7f25c4195390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:01.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.648+0000 7f25c93d2700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c4195a70 con 0x7f25c40ffbd0 2024-02-16T19:33:01.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.648+0000 7f25c93d2700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c4199800 con 0x7f25c4104130 2024-02-16T19:33:01.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.648+0000 7f25c2ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25c40ffbd0 0x7f25c4194e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:01.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.648+0000 7f25c2ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25c40ffbd0 0x7f25c4194e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49620/0 (socket says 172.21.15.22:49620) 2024-02-16T19:33:01.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.648+0000 7f25c2ffd700 1 -- 172.21.15.22:0/1247626542 learned_addr learned my addr 172.21.15.22:0/1247626542 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:33:01.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.648+0000 7f25c27fc700 1 --2- 172.21.15.22:0/1247626542 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25c4104130 0x7f25c4195390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:01.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.649+0000 7f25c2ffd700 1 -- 172.21.15.22:0/1247626542 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25c4104130 msgr2=0x7f25c4195390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:01.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.649+0000 7f25c2ffd700 1 --2- 172.21.15.22:0/1247626542 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25c4104130 0x7f25c4195390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:01.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.649+0000 7f25c2ffd700 1 -- 172.21.15.22:0/1247626542 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25b80095f0 con 0x7f25c40ffbd0 2024-02-16T19:33:01.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.649+0000 7f25c2ffd700 1 --2- 172.21.15.22:0/1247626542 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25c40ffbd0 0x7f25c4194e50 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f25b400b4b0 tx=0x7f25b400b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:33:01.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.649+0000 7f25abfff700 1 -- 172.21.15.22:0/1247626542 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25b4011800 con 0x7f25c40ffbd0 2024-02-16T19:33:01.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.649+0000 7f25abfff700 1 -- 172.21.15.22:0/1247626542 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f25b4011e20 con 0x7f25c40ffbd0 2024-02-16T19:33:01.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.649+0000 7f25c93d2700 1 -- 172.21.15.22:0/1247626542 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25c4199ae0 con 0x7f25c40ffbd0 2024-02-16T19:33:01.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.649+0000 7f25abfff700 1 -- 172.21.15.22:0/1247626542 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25b400f2e0 con 0x7f25c40ffbd0 2024-02-16T19:33:01.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.649+0000 7f25c93d2700 1 -- 172.21.15.22:0/1247626542 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25c419a030 con 0x7f25c40ffbd0 2024-02-16T19:33:01.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.650+0000 7f25c93d2700 1 -- 172.21.15.22:0/1247626542 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25c404f2a0 con 0x7f25c40ffbd0 2024-02-16T19:33:01.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.650+0000 7f25abfff700 1 -- 172.21.15.22:0/1247626542 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 33) v1 ==== 91346+0+0 (secure 0 0 0) 0x7f25b4010380 con 0x7f25c40ffbd0 2024-02-16T19:33:01.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.651+0000 7f25abfff700 1 --2- 172.21.15.22:0/1247626542 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f25ac06d520 0x7f25ac06f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:01.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.651+0000 7f25abfff700 1 -- 172.21.15.22:0/1247626542 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f25b408cad0 con 0x7f25c40ffbd0 2024-02-16T19:33:01.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.651+0000 7f25c27fc700 1 --2- 172.21.15.22:0/1247626542 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f25ac06d520 0x7f25ac06f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:01.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.651+0000 7f25c27fc700 1 --2- 172.21.15.22:0/1247626542 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f25ac06d520 0x7f25ac06f9d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f25c4196470 tx=0x7f25b801a040 comp rx=0 tx=0).ready entity=mgr.24407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:33:01.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.655+0000 7f25abfff700 1 -- 172.21.15.22:0/1247626542 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f25b40567b0 con 0x7f25c40ffbd0 2024-02-16T19:33:01.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.953+0000 7f25c93d2700 1 -- 172.21.15.22:0/1247626542 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f25c419a2e0 con 0x7f25c40ffbd0 2024-02-16T19:33:01.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.953+0000 7f25abfff700 1 -- 172.21.15.22:0/1247626542 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7f25b4059e70 con 0x7f25c40ffbd0 2024-02-16T19:33:01.955 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-02-16T19:33:01.955 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": { 2024-02-16T19:33:01.955 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-02-16T19:33:01.955 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:33:01.955 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": { 2024-02-16T19:33:01.955 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T19:33:01.955 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:33:01.955 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": { 2024-02-16T19:33:01.955 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-02-16T19:33:01.955 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:33:01.955 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": { 2024-02-16T19:33:01.955 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-02-16T19:33:01.956 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:33:01.956 INFO:teuthology.orchestra.run.smithi022.stdout: "overall": { 2024-02-16T19:33:01.956 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 12, 2024-02-16T19:33:01.956 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T19:33:01.956 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-02-16T19:33:01.956 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-02-16T19:33:01.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.956+0000 7f25c93d2700 1 -- 172.21.15.22:0/1247626542 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f25ac06d520 msgr2=0x7f25ac06f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:01.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.956+0000 7f25c93d2700 1 --2- 172.21.15.22:0/1247626542 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f25ac06d520 0x7f25ac06f9d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f25c4196470 tx=0x7f25b801a040 comp rx=0 tx=0).stop 2024-02-16T19:33:01.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.956+0000 7f25c93d2700 1 -- 172.21.15.22:0/1247626542 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25c40ffbd0 msgr2=0x7f25c4194e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:01.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.956+0000 7f25c93d2700 1 --2- 172.21.15.22:0/1247626542 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25c40ffbd0 0x7f25c4194e50 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f25b400b4b0 tx=0x7f25b400b7c0 comp rx=0 tx=0).stop 2024-02-16T19:33:01.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.956+0000 7f25c93d2700 1 -- 172.21.15.22:0/1247626542 shutdown_connections 2024-02-16T19:33:01.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.956+0000 7f25c93d2700 1 --2- 172.21.15.22:0/1247626542 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f25ac06d520 0x7f25ac06f9d0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:01.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.956+0000 7f25c93d2700 1 --2- 172.21.15.22:0/1247626542 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25c40ffbd0 0x7f25c4194e50 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:01.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.956+0000 7f25c93d2700 1 --2- 172.21.15.22:0/1247626542 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25c4104130 0x7f25c4195390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:01.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.956+0000 7f25c93d2700 1 -- 172.21.15.22:0/1247626542 >> 172.21.15.22:0/1247626542 conn(0x7f25c4075130 msgr2=0x7f25c40fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:33:01.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.956+0000 7f25c93d2700 1 -- 172.21.15.22:0/1247626542 shutdown_connections 2024-02-16T19:33:01.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:01.956+0000 7f25c93d2700 1 -- 172.21.15.22:0/1247626542 wait complete. 2024-02-16T19:33:02.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.065+0000 7f522945b700 1 -- 172.21.15.22:0/1357990134 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5224100e60 msgr2=0x7f5224104e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:02.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.065+0000 7f522945b700 1 --2- 172.21.15.22:0/1357990134 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5224100e60 0x7f5224104e30 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f5214009910 tx=0x7f5214009c20 comp rx=0 tx=0).stop 2024-02-16T19:33:02.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.065+0000 7f522945b700 1 -- 172.21.15.22:0/1357990134 shutdown_connections 2024-02-16T19:33:02.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.065+0000 7f522945b700 1 --2- 172.21.15.22:0/1357990134 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5224100e60 0x7f5224104e30 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:02.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.065+0000 7f522945b700 1 --2- 172.21.15.22:0/1357990134 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52241004c0 0x7f5224100890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:02.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.065+0000 7f522945b700 1 -- 172.21.15.22:0/1357990134 >> 172.21.15.22:0/1357990134 conn(0x7f52240fbd80 msgr2=0x7f52240fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:33:02.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.065+0000 7f522945b700 1 -- 172.21.15.22:0/1357990134 shutdown_connections 2024-02-16T19:33:02.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.065+0000 7f522945b700 1 -- 172.21.15.22:0/1357990134 wait complete. 2024-02-16T19:33:02.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.066+0000 7f522945b700 1 Processor -- start 2024-02-16T19:33:02.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.066+0000 7f522945b700 1 -- start start 2024-02-16T19:33:02.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.066+0000 7f522945b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52241004c0 0x7f5224195ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:02.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.066+0000 7f522945b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5224100e60 0x7f5224196430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:02.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.066+0000 7f522945b700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5224196a80 con 0x7f52241004c0 2024-02-16T19:33:02.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.066+0000 7f522945b700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5224196bc0 con 0x7f5224100e60 2024-02-16T19:33:02.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.066+0000 7f5222ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52241004c0 0x7f5224195ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:02.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.066+0000 7f5222ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52241004c0 0x7f5224195ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49638/0 (socket says 172.21.15.22:49638) 2024-02-16T19:33:02.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.066+0000 7f5222ffd700 1 -- 172.21.15.22:0/3064647726 learned_addr learned my addr 172.21.15.22:0/3064647726 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:33:02.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.066+0000 7f52227fc700 1 --2- 172.21.15.22:0/3064647726 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5224100e60 0x7f5224196430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:02.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.067+0000 7f5222ffd700 1 -- 172.21.15.22:0/3064647726 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5224100e60 msgr2=0x7f5224196430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:02.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.067+0000 7f5222ffd700 1 --2- 172.21.15.22:0/3064647726 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5224100e60 0x7f5224196430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:02.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.067+0000 7f5222ffd700 1 -- 172.21.15.22:0/3064647726 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52140095f0 con 0x7f52241004c0 2024-02-16T19:33:02.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.067+0000 7f5222ffd700 1 --2- 172.21.15.22:0/3064647726 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52241004c0 0x7f5224195ef0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f521800c930 tx=0x7f521800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:33:02.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.067+0000 7f520bfff700 1 -- 172.21.15.22:0/3064647726 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5218007ad0 con 0x7f52241004c0 2024-02-16T19:33:02.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.067+0000 7f520bfff700 1 -- 172.21.15.22:0/3064647726 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f521800ce80 con 0x7f52241004c0 2024-02-16T19:33:02.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.067+0000 7f522945b700 1 -- 172.21.15.22:0/3064647726 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f522419aa10 con 0x7f52241004c0 2024-02-16T19:33:02.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.067+0000 7f520bfff700 1 -- 172.21.15.22:0/3064647726 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5218018630 con 0x7f52241004c0 2024-02-16T19:33:02.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.067+0000 7f522945b700 1 -- 172.21.15.22:0/3064647726 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f522419af60 con 0x7f52241004c0 2024-02-16T19:33:02.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.068+0000 7f522945b700 1 -- 172.21.15.22:0/3064647726 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5224108830 con 0x7f52241004c0 2024-02-16T19:33:02.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.069+0000 7f520bfff700 1 -- 172.21.15.22:0/3064647726 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 33) v1 ==== 91346+0+0 (secure 0 0 0) 0x7f5218007c30 con 0x7f52241004c0 2024-02-16T19:33:02.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.069+0000 7f520bfff700 1 --2- 172.21.15.22:0/3064647726 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f520c06d520 0x7f520c06f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:02.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.070+0000 7f520bfff700 1 -- 172.21.15.22:0/3064647726 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f521808cfb0 con 0x7f52241004c0 2024-02-16T19:33:02.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.070+0000 7f52227fc700 1 --2- 172.21.15.22:0/3064647726 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f520c06d520 0x7f520c06f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:02.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.070+0000 7f52227fc700 1 --2- 172.21.15.22:0/3064647726 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f520c06d520 0x7f520c06f9d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5224197510 tx=0x7f521401a040 comp rx=0 tx=0).ready entity=mgr.24407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:33:02.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.074+0000 7f520bfff700 1 -- 172.21.15.22:0/3064647726 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f52180c0a50 con 0x7f52241004c0 2024-02-16T19:33:02.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.355+0000 7f522945b700 1 -- 172.21.15.22:0/3064647726 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f522419aba0 con 0x7f52241004c0 2024-02-16T19:33:02.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.355+0000 7f520bfff700 1 -- 172.21.15.22:0/3064647726 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1824 (secure 0 0 0) 0x7f522419aba0 con 0x7f52241004c0 2024-02-16T19:33:02.357 INFO:teuthology.orchestra.run.smithi022.stdout:e15 2024-02-16T19:33:02.357 INFO:teuthology.orchestra.run.smithi022.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T19:33:02.357 INFO:teuthology.orchestra.run.smithi022.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:33:02.357 INFO:teuthology.orchestra.run.smithi022.stdout:legacy client fscid: 1 2024-02-16T19:33:02.357 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:33:02.357 INFO:teuthology.orchestra.run.smithi022.stdout:Filesystem 'cephfs' (1) 2024-02-16T19:33:02.357 INFO:teuthology.orchestra.run.smithi022.stdout:fs_name cephfs 2024-02-16T19:33:02.357 INFO:teuthology.orchestra.run.smithi022.stdout:epoch 15 2024-02-16T19:33:02.357 INFO:teuthology.orchestra.run.smithi022.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T19:33:02.357 INFO:teuthology.orchestra.run.smithi022.stdout:created 2024-02-16T19:26:17.873644+0000 2024-02-16T19:33:02.357 INFO:teuthology.orchestra.run.smithi022.stdout:modified 2024-02-16T19:26:35.943353+0000 2024-02-16T19:33:02.358 INFO:teuthology.orchestra.run.smithi022.stdout:tableserver 0 2024-02-16T19:33:02.358 INFO:teuthology.orchestra.run.smithi022.stdout:root 0 2024-02-16T19:33:02.358 INFO:teuthology.orchestra.run.smithi022.stdout:session_timeout 60 2024-02-16T19:33:02.358 INFO:teuthology.orchestra.run.smithi022.stdout:session_autoclose 300 2024-02-16T19:33:02.358 INFO:teuthology.orchestra.run.smithi022.stdout:max_file_size 1099511627776 2024-02-16T19:33:02.358 INFO:teuthology.orchestra.run.smithi022.stdout:required_client_features {} 2024-02-16T19:33:02.358 INFO:teuthology.orchestra.run.smithi022.stdout:last_failure 0 2024-02-16T19:33:02.358 INFO:teuthology.orchestra.run.smithi022.stdout:last_failure_osd_epoch 47 2024-02-16T19:33:02.358 INFO:teuthology.orchestra.run.smithi022.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:33:02.358 INFO:teuthology.orchestra.run.smithi022.stdout:max_mds 1 2024-02-16T19:33:02.358 INFO:teuthology.orchestra.run.smithi022.stdout:in 0 2024-02-16T19:33:02.358 INFO:teuthology.orchestra.run.smithi022.stdout:up {0=14490} 2024-02-16T19:33:02.358 INFO:teuthology.orchestra.run.smithi022.stdout:failed 2024-02-16T19:33:02.358 INFO:teuthology.orchestra.run.smithi022.stdout:damaged 2024-02-16T19:33:02.358 INFO:teuthology.orchestra.run.smithi022.stdout:stopped 2024-02-16T19:33:02.359 INFO:teuthology.orchestra.run.smithi022.stdout:data_pools [3] 2024-02-16T19:33:02.359 INFO:teuthology.orchestra.run.smithi022.stdout:metadata_pool 2 2024-02-16T19:33:02.359 INFO:teuthology.orchestra.run.smithi022.stdout:inline_data enabled 2024-02-16T19:33:02.359 INFO:teuthology.orchestra.run.smithi022.stdout:balancer 2024-02-16T19:33:02.359 INFO:teuthology.orchestra.run.smithi022.stdout:bal_rank_mask -1 2024-02-16T19:33:02.359 INFO:teuthology.orchestra.run.smithi022.stdout:standby_count_wanted 1 2024-02-16T19:33:02.359 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.cephfs.smithi022.btppwm{0:14490} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.22:6828/3597685751,v1:172.21.15.22:6829/3597685751] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:33:02.359 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:33:02.359 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:33:02.359 INFO:teuthology.orchestra.run.smithi022.stdout:Standby daemons: 2024-02-16T19:33:02.359 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:33:02.359 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.cephfs.smithi022.ocukvs{-1:14500} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.22:6826/242865843,v1:172.21.15.22:6827/242865843] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:33:02.359 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.cephfs.smithi117.ibmppn{-1:24259} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.117:6824/565408102,v1:172.21.15.117:6825/565408102] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:33:02.359 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.cephfs.smithi117.orzors{-1:24265} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.117:6826/3122342028,v1:172.21.15.117:6827/3122342028] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:33:02.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.357+0000 7f522945b700 1 -- 172.21.15.22:0/3064647726 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f520c06d520 msgr2=0x7f520c06f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:02.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.357+0000 7f522945b700 1 --2- 172.21.15.22:0/3064647726 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f520c06d520 0x7f520c06f9d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5224197510 tx=0x7f521401a040 comp rx=0 tx=0).stop 2024-02-16T19:33:02.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.357+0000 7f522945b700 1 -- 172.21.15.22:0/3064647726 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52241004c0 msgr2=0x7f5224195ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:02.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.357+0000 7f522945b700 1 --2- 172.21.15.22:0/3064647726 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52241004c0 0x7f5224195ef0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f521800c930 tx=0x7f521800cc40 comp rx=0 tx=0).stop 2024-02-16T19:33:02.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.357+0000 7f522945b700 1 -- 172.21.15.22:0/3064647726 shutdown_connections 2024-02-16T19:33:02.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.357+0000 7f522945b700 1 --2- 172.21.15.22:0/3064647726 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f520c06d520 0x7f520c06f9d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:02.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.357+0000 7f522945b700 1 --2- 172.21.15.22:0/3064647726 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52241004c0 0x7f5224195ef0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:02.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.357+0000 7f522945b700 1 --2- 172.21.15.22:0/3064647726 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5224100e60 0x7f5224196430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:02.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.357+0000 7f522945b700 1 -- 172.21.15.22:0/3064647726 >> 172.21.15.22:0/3064647726 conn(0x7f52240fbd80 msgr2=0x7f52240fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:33:02.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.358+0000 7f522945b700 1 -- 172.21.15.22:0/3064647726 shutdown_connections 2024-02-16T19:33:02.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.358+0000 7f522945b700 1 -- 172.21.15.22:0/3064647726 wait complete. 2024-02-16T19:33:02.361 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 15 2024-02-16T19:33:02.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.476+0000 7f9acabc5700 1 -- 172.21.15.22:0/3582094958 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ac41004c0 msgr2=0x7f9ac4100890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:02.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.476+0000 7f9acabc5700 1 --2- 172.21.15.22:0/3582094958 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ac41004c0 0x7f9ac4100890 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f9ab8009910 tx=0x7f9ab8009c20 comp rx=0 tx=0).stop 2024-02-16T19:33:02.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.477+0000 7f9acabc5700 1 -- 172.21.15.22:0/3582094958 shutdown_connections 2024-02-16T19:33:02.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.477+0000 7f9acabc5700 1 --2- 172.21.15.22:0/3582094958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ac4100e60 0x7f9ac4104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:02.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.477+0000 7f9acabc5700 1 --2- 172.21.15.22:0/3582094958 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ac41004c0 0x7f9ac4100890 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:02.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.477+0000 7f9acabc5700 1 -- 172.21.15.22:0/3582094958 >> 172.21.15.22:0/3582094958 conn(0x7f9ac40fbd80 msgr2=0x7f9ac40fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:33:02.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.477+0000 7f9acabc5700 1 -- 172.21.15.22:0/3582094958 shutdown_connections 2024-02-16T19:33:02.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.477+0000 7f9acabc5700 1 -- 172.21.15.22:0/3582094958 wait complete. 2024-02-16T19:33:02.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.477+0000 7f9acabc5700 1 Processor -- start 2024-02-16T19:33:02.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.477+0000 7f9acabc5700 1 -- start start 2024-02-16T19:33:02.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.477+0000 7f9acabc5700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ac41004c0 0x7f9ac4195f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:02.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.477+0000 7f9acabc5700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ac4100e60 0x7f9ac41964d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:02.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.477+0000 7f9acabc5700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac4196bb0 con 0x7f9ac4100e60 2024-02-16T19:33:02.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.477+0000 7f9acabc5700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac419a940 con 0x7f9ac41004c0 2024-02-16T19:33:02.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.478+0000 7f9ac3fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ac4100e60 0x7f9ac41964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:02.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.478+0000 7f9ac3fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ac4100e60 0x7f9ac41964d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49662/0 (socket says 172.21.15.22:49662) 2024-02-16T19:33:02.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.478+0000 7f9ac3fff700 1 -- 172.21.15.22:0/3829197125 learned_addr learned my addr 172.21.15.22:0/3829197125 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:33:02.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.478+0000 7f9ac8961700 1 --2- 172.21.15.22:0/3829197125 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ac41004c0 0x7f9ac4195f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:02.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.478+0000 7f9ac3fff700 1 -- 172.21.15.22:0/3829197125 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ac41004c0 msgr2=0x7f9ac4195f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:02.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.478+0000 7f9ac3fff700 1 --2- 172.21.15.22:0/3829197125 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ac41004c0 0x7f9ac4195f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:02.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.478+0000 7f9ac3fff700 1 -- 172.21.15.22:0/3829197125 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ab80095f0 con 0x7f9ac4100e60 2024-02-16T19:33:02.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.478+0000 7f9ac3fff700 1 --2- 172.21.15.22:0/3829197125 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ac4100e60 0x7f9ac41964d0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f9ab000b480 tx=0x7f9ab000b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:33:02.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.478+0000 7f9ac1ffb700 1 -- 172.21.15.22:0/3829197125 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9ab0011800 con 0x7f9ac4100e60 2024-02-16T19:33:02.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.478+0000 7f9acabc5700 1 -- 172.21.15.22:0/3829197125 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ac419ac20 con 0x7f9ac4100e60 2024-02-16T19:33:02.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.478+0000 7f9ac1ffb700 1 -- 172.21.15.22:0/3829197125 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9ab0011e20 con 0x7f9ac4100e60 2024-02-16T19:33:02.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.479+0000 7f9ac1ffb700 1 -- 172.21.15.22:0/3829197125 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9ab0010ea0 con 0x7f9ac4100e60 2024-02-16T19:33:02.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.479+0000 7f9acabc5700 1 -- 172.21.15.22:0/3829197125 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ac419b170 con 0x7f9ac4100e60 2024-02-16T19:33:02.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.479+0000 7f9acabc5700 1 -- 172.21.15.22:0/3829197125 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ac41087d0 con 0x7f9ac4100e60 2024-02-16T19:33:02.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.480+0000 7f9ac1ffb700 1 -- 172.21.15.22:0/3829197125 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 33) v1 ==== 91346+0+0 (secure 0 0 0) 0x7f9ab0010380 con 0x7f9ac4100e60 2024-02-16T19:33:02.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.480+0000 7f9ac1ffb700 1 --2- 172.21.15.22:0/3829197125 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f9ab406d520 0x7f9ab406f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:02.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.480+0000 7f9ac1ffb700 1 -- 172.21.15.22:0/3829197125 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f9ab008ca40 con 0x7f9ac4100e60 2024-02-16T19:33:02.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.480+0000 7f9ac8961700 1 --2- 172.21.15.22:0/3829197125 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f9ab406d520 0x7f9ab406f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:02.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.481+0000 7f9ac8961700 1 --2- 172.21.15.22:0/3829197125 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f9ab406d520 0x7f9ab406f9d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9ab8006010 tx=0x7f9ab801a040 comp rx=0 tx=0).ready entity=mgr.24407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:33:02.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.485+0000 7f9ac1ffb700 1 -- 172.21.15.22:0/3829197125 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f9ab0056720 con 0x7f9ac4100e60 2024-02-16T19:33:02.574 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:02 smithi022 ceph-mon[29221]: from='client.14824 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:33:02.574 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:02 smithi022 ceph-mon[29221]: from='client.14828 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:33:02.574 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:02 smithi022 ceph-mon[29221]: pgmap v97: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:33:02.574 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:02 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:02.574 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:02 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:33:02.574 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:02 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:02.575 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:02 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:02.575 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:02 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-02-16T19:33:02.575 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:02 smithi022 ceph-mon[29221]: from='client.? 172.21.15.22:0/1247626542' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:02.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:02 smithi117 ceph-mon[35309]: from='client.14824 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:33:02.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:02 smithi117 ceph-mon[35309]: from='client.14828 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:33:02.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:02 smithi117 ceph-mon[35309]: pgmap v97: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:33:02.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:02 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:02.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:02 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:33:02.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:02 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:02.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:02 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:02.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:02 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-02-16T19:33:02.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:02 smithi117 ceph-mon[35309]: from='client.? 172.21.15.22:0/1247626542' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:02.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.717+0000 7f9acabc5700 1 -- 172.21.15.22:0/3829197125 --> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9ac419b460 con 0x7f9ab406d520 2024-02-16T19:33:02.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.718+0000 7f9ac1ffb700 1 -- 172.21.15.22:0/3829197125 <== mgr.24407 v2:172.21.15.22:6800/3703806543 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f9ac419b460 con 0x7f9ab406d520 2024-02-16T19:33:02.719 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-02-16T19:33:02.720 INFO:teuthology.orchestra.run.smithi022.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T19:33:02.720 INFO:teuthology.orchestra.run.smithi022.stdout: "in_progress": true, 2024-02-16T19:33:02.720 INFO:teuthology.orchestra.run.smithi022.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T19:33:02.720 INFO:teuthology.orchestra.run.smithi022.stdout: "services_complete": [ 2024-02-16T19:33:02.720 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr" 2024-02-16T19:33:02.720 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-02-16T19:33:02.720 INFO:teuthology.orchestra.run.smithi022.stdout: "progress": "2/23 daemons upgraded", 2024-02-16T19:33:02.720 INFO:teuthology.orchestra.run.smithi022.stdout: "message": "Currently upgrading mon daemons", 2024-02-16T19:33:02.720 INFO:teuthology.orchestra.run.smithi022.stdout: "is_paused": false 2024-02-16T19:33:02.720 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-02-16T19:33:02.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.720+0000 7f9acabc5700 1 -- 172.21.15.22:0/3829197125 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f9ab406d520 msgr2=0x7f9ab406f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:02.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.720+0000 7f9acabc5700 1 --2- 172.21.15.22:0/3829197125 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f9ab406d520 0x7f9ab406f9d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9ab8006010 tx=0x7f9ab801a040 comp rx=0 tx=0).stop 2024-02-16T19:33:02.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.720+0000 7f9acabc5700 1 -- 172.21.15.22:0/3829197125 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ac4100e60 msgr2=0x7f9ac41964d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:02.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.720+0000 7f9acabc5700 1 --2- 172.21.15.22:0/3829197125 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ac4100e60 0x7f9ac41964d0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f9ab000b480 tx=0x7f9ab000b790 comp rx=0 tx=0).stop 2024-02-16T19:33:02.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.720+0000 7f9acabc5700 1 -- 172.21.15.22:0/3829197125 shutdown_connections 2024-02-16T19:33:02.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.720+0000 7f9acabc5700 1 --2- 172.21.15.22:0/3829197125 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f9ab406d520 0x7f9ab406f9d0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:02.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.720+0000 7f9acabc5700 1 --2- 172.21.15.22:0/3829197125 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ac41004c0 0x7f9ac4195f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:02.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.720+0000 7f9acabc5700 1 --2- 172.21.15.22:0/3829197125 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ac4100e60 0x7f9ac41964d0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:02.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.720+0000 7f9acabc5700 1 -- 172.21.15.22:0/3829197125 >> 172.21.15.22:0/3829197125 conn(0x7f9ac40fbd80 msgr2=0x7f9ac40fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:33:02.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.720+0000 7f9acabc5700 1 -- 172.21.15.22:0/3829197125 shutdown_connections 2024-02-16T19:33:02.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.720+0000 7f9acabc5700 1 -- 172.21.15.22:0/3829197125 wait complete. 2024-02-16T19:33:02.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.826+0000 7f6f6ec81700 1 -- 172.21.15.22:0/624664864 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f680ff350 msgr2=0x7f6f680ff7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:02.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.826+0000 7f6f6ec81700 1 --2- 172.21.15.22:0/624664864 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f680ff350 0x7f6f680ff7c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f6f54009910 tx=0x7f6f54009c20 comp rx=0 tx=0).stop 2024-02-16T19:33:02.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.826+0000 7f6f6ec81700 1 -- 172.21.15.22:0/624664864 shutdown_connections 2024-02-16T19:33:02.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.826+0000 7f6f6ec81700 1 --2- 172.21.15.22:0/624664864 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f680ff350 0x7f6f680ff7c0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:02.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.826+0000 7f6f6ec81700 1 --2- 172.21.15.22:0/624664864 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f681038b0 0x7f6f68103c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:02.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.826+0000 7f6f6ec81700 1 -- 172.21.15.22:0/624664864 >> 172.21.15.22:0/624664864 conn(0x7f6f680748b0 msgr2=0x7f6f68074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:33:02.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.826+0000 7f6f6ec81700 1 -- 172.21.15.22:0/624664864 shutdown_connections 2024-02-16T19:33:02.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.826+0000 7f6f6ec81700 1 -- 172.21.15.22:0/624664864 wait complete. 2024-02-16T19:33:02.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.827+0000 7f6f6ec81700 1 Processor -- start 2024-02-16T19:33:02.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.827+0000 7f6f6ec81700 1 -- start start 2024-02-16T19:33:02.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.827+0000 7f6f6ec81700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f680ff350 0x7f6f681005b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:02.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.827+0000 7f6f6ec81700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f681038b0 0x7f6f68100af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:02.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.827+0000 7f6f6ec81700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f68101030 con 0x7f6f680ff350 2024-02-16T19:33:02.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.827+0000 7f6f6ec81700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f68101170 con 0x7f6f681038b0 2024-02-16T19:33:02.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.827+0000 7f6f6ca1d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f680ff350 0x7f6f681005b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:02.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.827+0000 7f6f6ca1d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f680ff350 0x7f6f681005b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49686/0 (socket says 172.21.15.22:49686) 2024-02-16T19:33:02.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.827+0000 7f6f6ca1d700 1 -- 172.21.15.22:0/2728557056 learned_addr learned my addr 172.21.15.22:0/2728557056 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:33:02.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.827+0000 7f6f67fff700 1 --2- 172.21.15.22:0/2728557056 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f681038b0 0x7f6f68100af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:02.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.827+0000 7f6f6ca1d700 1 -- 172.21.15.22:0/2728557056 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f681038b0 msgr2=0x7f6f68100af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:02.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.827+0000 7f6f6ca1d700 1 --2- 172.21.15.22:0/2728557056 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f681038b0 0x7f6f68100af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:02.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.828+0000 7f6f6ca1d700 1 -- 172.21.15.22:0/2728557056 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f540095f0 con 0x7f6f680ff350 2024-02-16T19:33:02.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.828+0000 7f6f6ca1d700 1 --2- 172.21.15.22:0/2728557056 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f680ff350 0x7f6f681005b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c00b4b0 tx=0x7f6f5c00b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:33:02.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.828+0000 7f6f65ffb700 1 -- 172.21.15.22:0/2728557056 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6f5c011800 con 0x7f6f680ff350 2024-02-16T19:33:02.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.828+0000 7f6f65ffb700 1 -- 172.21.15.22:0/2728557056 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6f5c011e20 con 0x7f6f680ff350 2024-02-16T19:33:02.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.828+0000 7f6f6ec81700 1 -- 172.21.15.22:0/2728557056 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f681a2290 con 0x7f6f680ff350 2024-02-16T19:33:02.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.828+0000 7f6f65ffb700 1 -- 172.21.15.22:0/2728557056 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6f5c00f2e0 con 0x7f6f680ff350 2024-02-16T19:33:02.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.828+0000 7f6f6ec81700 1 -- 172.21.15.22:0/2728557056 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f681a26a0 con 0x7f6f680ff350 2024-02-16T19:33:02.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.829+0000 7f6f6ec81700 1 -- 172.21.15.22:0/2728557056 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f6804f2a0 con 0x7f6f680ff350 2024-02-16T19:33:02.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.829+0000 7f6f65ffb700 1 -- 172.21.15.22:0/2728557056 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 33) v1 ==== 91346+0+0 (secure 0 0 0) 0x7f6f5c010380 con 0x7f6f680ff350 2024-02-16T19:33:02.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.830+0000 7f6f65ffb700 1 --2- 172.21.15.22:0/2728557056 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f6f5806d520 0x7f6f5806f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:02.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.830+0000 7f6f65ffb700 1 -- 172.21.15.22:0/2728557056 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f6f5c08c9e0 con 0x7f6f680ff350 2024-02-16T19:33:02.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.830+0000 7f6f67fff700 1 --2- 172.21.15.22:0/2728557056 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f6f5806d520 0x7f6f5806f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:02.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.830+0000 7f6f67fff700 1 --2- 172.21.15.22:0/2728557056 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f6f5806d520 0x7f6f5806f9d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6f68101da0 tx=0x7f6f5401a040 comp rx=0 tx=0).ready entity=mgr.24407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:33:02.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:02.835+0000 7f6f65ffb700 1 -- 172.21.15.22:0/2728557056 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f6f5c056740 con 0x7f6f680ff350 2024-02-16T19:33:03.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:03.125+0000 7f6f6ec81700 1 -- 172.21.15.22:0/2728557056 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f6f681a2940 con 0x7f6f680ff350 2024-02-16T19:33:03.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:03.125+0000 7f6f65ffb700 1 -- 172.21.15.22:0/2728557056 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f6f5c0150e0 con 0x7f6f680ff350 2024-02-16T19:33:03.127 INFO:teuthology.orchestra.run.smithi022.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-02-16T19:33:03.127 INFO:teuthology.orchestra.run.smithi022.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-02-16T19:33:03.127 INFO:teuthology.orchestra.run.smithi022.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-02-16T19:33:03.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:03.127+0000 7f6f6ec81700 1 -- 172.21.15.22:0/2728557056 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f6f5806d520 msgr2=0x7f6f5806f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:03.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:03.127+0000 7f6f6ec81700 1 --2- 172.21.15.22:0/2728557056 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f6f5806d520 0x7f6f5806f9d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6f68101da0 tx=0x7f6f5401a040 comp rx=0 tx=0).stop 2024-02-16T19:33:03.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:03.127+0000 7f6f6ec81700 1 -- 172.21.15.22:0/2728557056 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f680ff350 msgr2=0x7f6f681005b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:03.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:03.127+0000 7f6f6ec81700 1 --2- 172.21.15.22:0/2728557056 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f680ff350 0x7f6f681005b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c00b4b0 tx=0x7f6f5c00b7c0 comp rx=0 tx=0).stop 2024-02-16T19:33:03.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:03.127+0000 7f6f6ec81700 1 -- 172.21.15.22:0/2728557056 shutdown_connections 2024-02-16T19:33:03.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:03.127+0000 7f6f6ec81700 1 --2- 172.21.15.22:0/2728557056 >> [v2:172.21.15.22:6800/3703806543,v1:172.21.15.22:6801/3703806543] conn(0x7f6f5806d520 0x7f6f5806f9d0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:03.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:03.127+0000 7f6f6ec81700 1 --2- 172.21.15.22:0/2728557056 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f680ff350 0x7f6f681005b0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:03.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:03.127+0000 7f6f6ec81700 1 --2- 172.21.15.22:0/2728557056 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f681038b0 0x7f6f68100af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:03.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:03.127+0000 7f6f6ec81700 1 -- 172.21.15.22:0/2728557056 >> 172.21.15.22:0/2728557056 conn(0x7f6f680748b0 msgr2=0x7f6f680fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:33:03.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:03.128+0000 7f6f6ec81700 1 -- 172.21.15.22:0/2728557056 shutdown_connections 2024-02-16T19:33:03.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:03.128+0000 7f6f6ec81700 1 -- 172.21.15.22:0/2728557056 wait complete. 2024-02-16T19:33:03.404 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:03 smithi022 ceph-mon[29221]: from='client.14832 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:33:03.404 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:03 smithi022 ceph-mon[29221]: Upgrade: Target is version 19.0.0-1450-gf78a58c0 (squid) 2024-02-16T19:33:03.404 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:03 smithi022 ceph-mon[29221]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, digests ['quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526'] 2024-02-16T19:33:03.404 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:03 smithi022 ceph-mon[29221]: Upgrade: Setting container_image for all mgr 2024-02-16T19:33:03.404 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:03 smithi022 ceph-mon[29221]: from='client.? 172.21.15.22:0/3064647726' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:33:03.404 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:03 smithi022 ceph-mon[29221]: from='client.? 172.21.15.22:0/2728557056' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T19:33:03.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:03 smithi117 ceph-mon[35309]: from='client.14832 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:33:03.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:03 smithi117 ceph-mon[35309]: Upgrade: Target is version 19.0.0-1450-gf78a58c0 (squid) 2024-02-16T19:33:03.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:03 smithi117 ceph-mon[35309]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, digests ['quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526'] 2024-02-16T19:33:03.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:03 smithi117 ceph-mon[35309]: Upgrade: Setting container_image for all mgr 2024-02-16T19:33:03.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:03 smithi117 ceph-mon[35309]: from='client.? 172.21.15.22:0/3064647726' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:33:03.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:03 smithi117 ceph-mon[35309]: from='client.? 172.21.15.22:0/2728557056' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T19:33:04.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:04 smithi022 ceph-mon[29221]: from='client.14844 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:33:04.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:04 smithi022 ceph-mon[29221]: pgmap v98: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:33:04.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:04 smithi022 ceph-mon[29221]: Upgrade: Updating mon.smithi022 2024-02-16T19:33:04.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:04 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:04.716 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:04 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-16T19:33:04.716 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:04 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-16T19:33:04.716 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:04 smithi022 ceph-mon[29221]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:33:04.716 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:04 smithi022 ceph-mon[29221]: Deploying daemon mon.smithi022 on smithi022 2024-02-16T19:33:04.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:04 smithi117 ceph-mon[35309]: from='client.14844 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:33:04.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:04 smithi117 ceph-mon[35309]: pgmap v98: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:33:04.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:04 smithi117 ceph-mon[35309]: Upgrade: Updating mon.smithi022 2024-02-16T19:33:04.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:04 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:04.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:04 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-16T19:33:04.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:04 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-16T19:33:04.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:04 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:33:04.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:04 smithi117 ceph-mon[35309]: Deploying daemon mon.smithi022 on smithi022 2024-02-16T19:33:05.608 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:05 smithi022 systemd[1]: Stopping Ceph mon.smithi022 for 2a1346e6-cd00-11ee-95ba-87774f69a715... 2024-02-16T19:33:05.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:05 smithi022 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-mon-smithi022[29217]: 2024-02-16T19:33:05.606+0000 7f4405d37700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi022 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-16T19:33:05.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:05 smithi022 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-mon-smithi022[29217]: 2024-02-16T19:33:05.606+0000 7f4405d37700 -1 mon.smithi022@0(leader) e2 *** Got Signal Terminated *** 2024-02-16T19:33:05.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:05 smithi022 podman[88621]: 2024-02-16 19:33:05.719477153 +0000 UTC m=+0.226345819 container died b4ffc3032e12371bfc1764880a8e6b8944c21341bd978ae792b2fbbd42152ee8 (image=quay.io/ceph/ceph:v18.2.0, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-mon-smithi022, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20231212, ceph=True, io.buildah.version=1.29.1, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.schema-version=1.0, RELEASE=HEAD) 2024-02-16T19:33:06.358 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:06 smithi022 podman[88621]: 2024-02-16 19:33:06.123634591 +0000 UTC m=+0.630503269 container cleanup b4ffc3032e12371bfc1764880a8e6b8944c21341bd978ae792b2fbbd42152ee8 (image=quay.io/ceph/ceph:v18.2.0, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-mon-smithi022, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=https://github.com/ceph/ceph-container.git, CEPH_POINT_RELEASE=-18.2.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20231212, org.label-schema.vendor=CentOS, io.buildah.version=1.29.1, RELEASE=HEAD) 2024-02-16T19:33:06.358 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:06 smithi022 bash[88621]: ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-mon-smithi022 2024-02-16T19:33:06.714 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:06 smithi022 podman[88647]: 2024-02-16 19:33:06.358169517 +0000 UTC m=+0.636692314 container remove b4ffc3032e12371bfc1764880a8e6b8944c21341bd978ae792b2fbbd42152ee8 (image=quay.io/ceph/ceph:v18.2.0, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-mon-smithi022, RELEASE=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20231212, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.29.1, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=-18.2.0) 2024-02-16T19:33:06.714 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:06 smithi022 systemd[1]: ceph-2a1346e6-cd00-11ee-95ba-87774f69a715@mon.smithi022.service: Deactivated successfully. 2024-02-16T19:33:06.714 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:06 smithi022 systemd[1]: Stopped Ceph mon.smithi022 for 2a1346e6-cd00-11ee-95ba-87774f69a715. 2024-02-16T19:33:06.714 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:06 smithi022 systemd[1]: ceph-2a1346e6-cd00-11ee-95ba-87774f69a715@mon.smithi022.service: Consumed 14.103s CPU time. 2024-02-16T19:33:06.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:06 smithi022 systemd[1]: Starting Ceph mon.smithi022 for 2a1346e6-cd00-11ee-95ba-87774f69a715... 2024-02-16T19:33:06.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:06 smithi022 podman[88779]: 2024-02-16 19:33:06.830775436 +0000 UTC m=+0.018389408 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T19:33:06.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:06 smithi022 podman[88779]: 2024-02-16 19:33:06.959532378 +0000 UTC m=+0.147146349 container create 9c84755ae67dec075119dc6bbcd217bfd244588ae7131bc1b01843432cd24743 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-mon-smithi022, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-02-16T19:33:07.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 podman[88779]: 2024-02-16 19:33:07.195545541 +0000 UTC m=+0.383159507 container init 9c84755ae67dec075119dc6bbcd217bfd244588ae7131bc1b01843432cd24743 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-mon-smithi022, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240215.160715-f78a58c) 2024-02-16T19:33:07.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 podman[88779]: 2024-02-16 19:33:07.199236381 +0000 UTC m=+0.386850356 container start 9c84755ae67dec075119dc6bbcd217bfd244588ae7131bc1b01843432cd24743 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-mon-smithi022, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-16T19:33:07.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: set uid:gid to 167:167 (ceph:ceph) 2024-02-16T19:33:07.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev), process ceph-mon, pid 2 2024-02-16T19:33:07.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: pidfile_write: ignore empty --pid-file 2024-02-16T19:33:07.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: load: jerasure load: lrc 2024-02-16T19:33:07.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: RocksDB version: 7.9.2 2024-02-16T19:33:07.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Git sha 0 2024-02-16T19:33:07.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Compile date 2024-02-15 18:05:23 2024-02-16T19:33:07.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: DB SUMMARY 2024-02-16T19:33:07.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: DB Session ID: F442OMCYQZ9UDSYGBNEZ 2024-02-16T19:33:07.467 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: CURRENT file: CURRENT 2024-02-16T19:33:07.467 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: IDENTITY file: IDENTITY 2024-02-16T19:33:07.467 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: MANIFEST file: MANIFEST-000015 size: 1509 Bytes 2024-02-16T19:33:07.467 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi022/store.db dir, Total Num: 1, files: 000032.sst 2024-02-16T19:33:07.467 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi022/store.db: 000030.log size: 3656724 ; 2024-02-16T19:33:07.467 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.error_if_exists: 0 2024-02-16T19:33:07.467 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.create_if_missing: 0 2024-02-16T19:33:07.467 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.paranoid_checks: 1 2024-02-16T19:33:07.467 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.flush_verify_memtable_count: 1 2024-02-16T19:33:07.467 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-16T19:33:07.467 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-02-16T19:33:07.468 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.env: 0x5567137d7ae0 2024-02-16T19:33:07.468 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.fs: PosixFileSystem 2024-02-16T19:33:07.468 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.info_log: 0x5567148a3760 2024-02-16T19:33:07.468 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.max_file_opening_threads: 16 2024-02-16T19:33:07.468 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.statistics: (nil) 2024-02-16T19:33:07.468 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.use_fsync: 0 2024-02-16T19:33:07.468 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.max_log_file_size: 0 2024-02-16T19:33:07.468 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-16T19:33:07.468 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-16T19:33:07.468 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.keep_log_file_num: 1000 2024-02-16T19:33:07.468 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.recycle_log_file_num: 0 2024-02-16T19:33:07.468 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.allow_fallocate: 1 2024-02-16T19:33:07.469 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.allow_mmap_reads: 0 2024-02-16T19:33:07.469 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.allow_mmap_writes: 0 2024-02-16T19:33:07.469 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.use_direct_reads: 0 2024-02-16T19:33:07.469 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-16T19:33:07.469 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.create_missing_column_families: 0 2024-02-16T19:33:07.469 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.db_log_dir: 2024-02-16T19:33:07.469 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.wal_dir: 2024-02-16T19:33:07.469 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-16T19:33:07.469 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-16T19:33:07.469 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-16T19:33:07.469 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-16T19:33:07.469 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-16T19:33:07.469 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-16T19:33:07.470 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.advise_random_on_open: 1 2024-02-16T19:33:07.470 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.db_write_buffer_size: 0 2024-02-16T19:33:07.470 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.write_buffer_manager: 0x556714877400 2024-02-16T19:33:07.470 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-16T19:33:07.470 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-16T19:33:07.470 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-16T19:33:07.470 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.rate_limiter: (nil) 2024-02-16T19:33:07.470 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-16T19:33:07.470 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.wal_recovery_mode: 2 2024-02-16T19:33:07.470 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.enable_thread_tracking: 0 2024-02-16T19:33:07.470 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.enable_pipelined_write: 0 2024-02-16T19:33:07.470 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.unordered_write: 0 2024-02-16T19:33:07.471 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-16T19:33:07.471 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-16T19:33:07.471 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-16T19:33:07.471 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-16T19:33:07.471 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.row_cache: None 2024-02-16T19:33:07.471 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.wal_filter: None 2024-02-16T19:33:07.472 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-16T19:33:07.473 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.allow_ingest_behind: 0 2024-02-16T19:33:07.473 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.two_write_queues: 0 2024-02-16T19:33:07.473 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.manual_wal_flush: 0 2024-02-16T19:33:07.473 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.wal_compression: 0 2024-02-16T19:33:07.473 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.atomic_flush: 0 2024-02-16T19:33:07.473 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-16T19:33:07.473 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-16T19:33:07.473 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-16T19:33:07.473 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.log_readahead_size: 0 2024-02-16T19:33:07.473 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-16T19:33:07.473 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.best_efforts_recovery: 0 2024-02-16T19:33:07.473 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-16T19:33:07.473 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-16T19:33:07.473 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.allow_data_in_errors: 0 2024-02-16T19:33:07.474 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.db_host_id: __hostname__ 2024-02-16T19:33:07.474 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.enforce_single_del_contracts: true 2024-02-16T19:33:07.474 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.max_background_jobs: 2 2024-02-16T19:33:07.474 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.max_background_compactions: -1 2024-02-16T19:33:07.474 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.max_subcompactions: 1 2024-02-16T19:33:07.474 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-16T19:33:07.474 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-16T19:33:07.474 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-16T19:33:07.474 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.max_total_wal_size: 0 2024-02-16T19:33:07.474 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-16T19:33:07.474 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-16T19:33:07.474 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-16T19:33:07.474 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-16T19:33:07.474 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.max_open_files: -1 2024-02-16T19:33:07.474 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.bytes_per_sync: 0 2024-02-16T19:33:07.475 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-16T19:33:07.475 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-16T19:33:07.475 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.compaction_readahead_size: 0 2024-02-16T19:33:07.475 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.max_background_flushes: -1 2024-02-16T19:33:07.475 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Compression algorithms supported: 2024-02-16T19:33:07.475 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-16T19:33:07.475 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: kZSTD supported: 0 2024-02-16T19:33:07.475 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: kXpressCompression supported: 0 2024-02-16T19:33:07.475 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: kLZ4HCCompression supported: 1 2024-02-16T19:33:07.475 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: kZlibCompression supported: 1 2024-02-16T19:33:07.475 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: kSnappyCompression supported: 1 2024-02-16T19:33:07.475 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: kLZ4Compression supported: 1 2024-02-16T19:33:07.475 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: kBZip2Compression supported: 0 2024-02-16T19:33:07.476 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-16T19:33:07.476 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: DMutex implementation: pthread_mutex_t 2024-02-16T19:33:07.476 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi022/store.db/MANIFEST-000015 2024-02-16T19:33:07.476 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-02-16T19:33:07.476 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-16T19:33:07.476 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.merge_operator: 2024-02-16T19:33:07.476 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.compaction_filter: None 2024-02-16T19:33:07.476 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.compaction_filter_factory: None 2024-02-16T19:33:07.476 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.sst_partitioner_factory: None 2024-02-16T19:33:07.476 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-16T19:33:07.476 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-16T19:33:07.476 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5567148a3880) 2024-02-16T19:33:07.477 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: cache_index_and_filter_blocks: 1 2024-02-16T19:33:07.477 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-16T19:33:07.477 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-16T19:33:07.477 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: pin_top_level_index_and_filter: 1 2024-02-16T19:33:07.477 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: index_type: 0 2024-02-16T19:33:07.477 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: data_block_index_type: 0 2024-02-16T19:33:07.477 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: index_shortening: 1 2024-02-16T19:33:07.477 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-16T19:33:07.477 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: checksum: 4 2024-02-16T19:33:07.477 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: no_block_cache: 0 2024-02-16T19:33:07.477 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: block_cache: 0x5567148a7090 2024-02-16T19:33:07.477 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: block_cache_name: BinnedLRUCache 2024-02-16T19:33:07.477 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: block_cache_options: 2024-02-16T19:33:07.478 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: capacity : 536870912 2024-02-16T19:33:07.478 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: num_shard_bits : 4 2024-02-16T19:33:07.478 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: strict_capacity_limit : 0 2024-02-16T19:33:07.478 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: high_pri_pool_ratio: 0.000 2024-02-16T19:33:07.478 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: block_cache_compressed: (nil) 2024-02-16T19:33:07.478 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: persistent_cache: (nil) 2024-02-16T19:33:07.478 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: block_size: 4096 2024-02-16T19:33:07.478 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: block_size_deviation: 10 2024-02-16T19:33:07.478 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: block_restart_interval: 16 2024-02-16T19:33:07.478 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: index_block_restart_interval: 1 2024-02-16T19:33:07.478 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: metadata_block_size: 4096 2024-02-16T19:33:07.478 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: partition_filters: 0 2024-02-16T19:33:07.478 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: use_delta_encoding: 1 2024-02-16T19:33:07.478 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: filter_policy: bloomfilter 2024-02-16T19:33:07.478 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: whole_key_filtering: 1 2024-02-16T19:33:07.479 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: verify_compression: 0 2024-02-16T19:33:07.479 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: read_amp_bytes_per_bit: 0 2024-02-16T19:33:07.479 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: format_version: 5 2024-02-16T19:33:07.479 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: enable_index_compression: 1 2024-02-16T19:33:07.479 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: block_align: 0 2024-02-16T19:33:07.479 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: max_auto_readahead_size: 262144 2024-02-16T19:33:07.479 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: prepopulate_block_cache: 0 2024-02-16T19:33:07.479 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: initial_auto_readahead_size: 8192 2024-02-16T19:33:07.479 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: num_file_reads_for_auto_readahead: 2 2024-02-16T19:33:07.479 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.write_buffer_size: 33554432 2024-02-16T19:33:07.479 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.max_write_buffer_number: 2 2024-02-16T19:33:07.479 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.compression: NoCompression 2024-02-16T19:33:07.479 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.bottommost_compression: Disabled 2024-02-16T19:33:07.479 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.prefix_extractor: nullptr 2024-02-16T19:33:07.479 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-16T19:33:07.480 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.num_levels: 7 2024-02-16T19:33:07.480 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-16T19:33:07.480 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-16T19:33:07.480 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-16T19:33:07.480 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-16T19:33:07.480 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-16T19:33:07.480 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-16T19:33:07.480 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-16T19:33:07.480 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-16T19:33:07.480 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-16T19:33:07.480 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-16T19:33:07.480 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-02-16T19:33:07.480 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-02-16T19:33:07.481 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-16T19:33:07.481 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.compression_opts.level: 32767 2024-02-16T19:33:07.481 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.compression_opts.strategy: 0 2024-02-16T19:33:07.481 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-16T19:33:07.481 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-16T19:33:07.481 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-02-16T19:33:07.481 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-16T19:33:07.481 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.compression_opts.enabled: false 2024-02-16T19:33:07.481 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-02-16T19:33:07.481 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-16T19:33:07.481 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-16T19:33:07.481 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-16T19:33:07.481 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.target_file_size_base: 67108864 2024-02-16T19:33:07.481 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-16T19:33:07.481 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-16T19:33:07.482 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-16T19:33:07.482 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-16T19:33:07.482 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-16T19:33:07.482 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-16T19:33:07.482 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-16T19:33:07.482 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-16T19:33:07.482 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-16T19:33:07.482 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-16T19:33:07.482 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-16T19:33:07.482 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-16T19:33:07.482 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-16T19:33:07.482 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-02-16T19:33:07.482 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.arena_block_size: 1048576 2024-02-16T19:33:07.482 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-16T19:33:07.482 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-16T19:33:07.482 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.disable_auto_compactions: 0 2024-02-16T19:33:07.482 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-16T19:33:07.483 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-16T19:33:07.483 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-16T19:33:07.483 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-16T19:33:07.483 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-16T19:33:07.483 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-16T19:33:07.483 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-16T19:33:07.483 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-16T19:33:07.483 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-16T19:33:07.483 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-16T19:33:07.483 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.table_properties_collectors: 2024-02-16T19:33:07.483 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.inplace_update_support: 0 2024-02-16T19:33:07.483 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-16T19:33:07.483 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-16T19:33:07.483 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-16T19:33:07.483 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-16T19:33:07.484 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.bloom_locality: 0 2024-02-16T19:33:07.484 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.max_successive_merges: 0 2024-02-16T19:33:07.484 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-16T19:33:07.484 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.paranoid_file_checks: 0 2024-02-16T19:33:07.484 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.force_consistency_checks: 1 2024-02-16T19:33:07.484 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.report_bg_io_stats: 0 2024-02-16T19:33:07.484 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.ttl: 2592000 2024-02-16T19:33:07.484 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-16T19:33:07.484 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-02-16T19:33:07.484 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-02-16T19:33:07.484 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.enable_blob_files: false 2024-02-16T19:33:07.484 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.min_blob_size: 0 2024-02-16T19:33:07.484 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.blob_file_size: 268435456 2024-02-16T19:33:07.484 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-16T19:33:07.484 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-16T19:33:07.485 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-16T19:33:07.485 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-02-16T19:33:07.485 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-02-16T19:33:07.485 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.blob_file_starting_level: 0 2024-02-16T19:33:07.485 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-02-16T19:33:07.485 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-02-16T19:33:07.485 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi022/store.db/MANIFEST-000015 succeeded,manifest_file_number is 15, next_file_number is 34, last_sequence is 12098, log_number is 30,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 30 2024-02-16T19:33:07.485 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 30 2024-02-16T19:33:07.485 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 47a8333a-8246-4f53-a0b9-cb19c23501f3 2024-02-16T19:33:07.485 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708111987228172, "job": 1, "event": "recovery_started", "wal_files": [30]} 2024-02-16T19:33:07.485 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #30 mode 2 2024-02-16T19:33:07.485 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708111987239131, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 35, "file_size": 3162444, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 12099, "largest_seqno": 15166, "table_properties": {"data_size": 3150700, "index_size": 7267, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 3461, "raw_key_size": 30225, "raw_average_key_size": 21, "raw_value_size": 3123840, "raw_average_value_size": 2268, "num_data_blocks": 330, "num_entries": 1377, "num_filter_entries": 1377, "num_deletions": 259, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1708111987, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "47a8333a-8246-4f53-a0b9-cb19c23501f3", "db_session_id": "F442OMCYQZ9UDSYGBNEZ", "orig_file_number": 35, "seqno_to_time_mapping": "N/A"}} 2024-02-16T19:33:07.485 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708111987239261, "job": 1, "event": "recovery_finished"} 2024-02-16T19:33:07.485 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: [db/version_set.cc:5047] Creating manifest 37 2024-02-16T19:33:07.485 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-02-16T19:33:07.485 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi022/store.db/000030.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-16T19:33:07.486 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x556714996000 2024-02-16T19:33:07.486 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: DB pointer 0x556714982000 2024-02-16T19:33:07.486 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-02-16T19:33:07.486 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-02-16T19:33:07.486 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: ** DB Stats ** 2024-02-16T19:33:07.486 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-16T19:33:07.486 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-16T19:33:07.486 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-16T19:33:07.486 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-16T19:33:07.486 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-16T19:33:07.486 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-16T19:33:07.486 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-16T19:33:07.486 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: 2024-02-16T19:33:07.486 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: ** Compaction Stats [default] ** 2024-02-16T19:33:07.486 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-16T19:33:07.487 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-02-16T19:33:07.487 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: L0 1/0 3.02 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 587.6 0.01 0.00 1 0.005 0 0 0.0 0.0 2024-02-16T19:33:07.487 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: L6 1/0 5.81 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-02-16T19:33:07.487 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: Sum 2/0 8.82 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 587.6 0.01 0.00 1 0.005 0 0 0.0 0.0 2024-02-16T19:33:07.487 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 587.6 0.01 0.00 1 0.005 0 0 0.0 0.0 2024-02-16T19:33:07.487 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: 2024-02-16T19:33:07.487 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: ** Compaction Stats [default] ** 2024-02-16T19:33:07.487 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-16T19:33:07.487 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-16T19:33:07.487 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 587.6 0.01 0.00 1 0.005 0 0 0.0 0.0 2024-02-16T19:33:07.487 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: 2024-02-16T19:33:07.487 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-02-16T19:33:07.488 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: 2024-02-16T19:33:07.488 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-16T19:33:07.488 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: Flush(GB): cumulative 0.003, interval 0.003 2024-02-16T19:33:07.488 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-16T19:33:07.488 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-16T19:33:07.488 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-16T19:33:07.488 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-16T19:33:07.488 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: Cumulative compaction: 0.00 GB write, 224.70 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-16T19:33:07.488 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: Interval compaction: 0.00 GB write, 224.70 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-16T19:33:07.488 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-16T19:33:07.488 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: Block cache BinnedLRUCache@0x5567148a7090#2 capacity: 512.00 MB usage: 12.11 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-02-16T19:33:07.489 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,4.03 KB,0.0007689%) IndexBlock(1,8.08 KB,0.00154078%) Misc(1,0.00 KB,0%) 2024-02-16T19:33:07.489 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: 2024-02-16T19:33:07.489 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-16T19:33:07.489 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: starting mon.smithi022 rank 0 at public addrs [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] at bind addrs [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi022 fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 2024-02-16T19:33:07.489 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: mon.smithi022@-1(???) e2 preinit fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 2024-02-16T19:33:07.489 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: mon.smithi022@-1(???).mds e15 new map 2024-02-16T19:33:07.489 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: mon.smithi022@-1(???).mds e15 print_map 2024-02-16T19:33:07.489 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: e15 2024-02-16T19:33:07.489 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T19:33:07.489 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:33:07.489 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: legacy client fscid: 1 2024-02-16T19:33:07.489 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: 2024-02-16T19:33:07.489 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: Filesystem 'cephfs' (1) 2024-02-16T19:33:07.490 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: fs_name cephfs 2024-02-16T19:33:07.490 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: epoch 15 2024-02-16T19:33:07.490 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T19:33:07.490 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: created 2024-02-16T19:26:17.873644+0000 2024-02-16T19:33:07.490 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: modified 2024-02-16T19:26:35.943353+0000 2024-02-16T19:33:07.490 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: tableserver 0 2024-02-16T19:33:07.490 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: root 0 2024-02-16T19:33:07.490 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: session_timeout 60 2024-02-16T19:33:07.490 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: session_autoclose 300 2024-02-16T19:33:07.490 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: max_file_size 1099511627776 2024-02-16T19:33:07.490 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: max_xattr_size 65536 2024-02-16T19:33:07.490 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: required_client_features {} 2024-02-16T19:33:07.490 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: last_failure 0 2024-02-16T19:33:07.490 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: last_failure_osd_epoch 47 2024-02-16T19:33:07.491 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:33:07.491 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: max_mds 1 2024-02-16T19:33:07.491 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: in 0 2024-02-16T19:33:07.491 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: up {0=14490} 2024-02-16T19:33:07.491 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: failed 2024-02-16T19:33:07.491 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: damaged 2024-02-16T19:33:07.491 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: stopped 2024-02-16T19:33:07.491 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: data_pools [3] 2024-02-16T19:33:07.491 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: metadata_pool 2 2024-02-16T19:33:07.491 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: inline_data enabled 2024-02-16T19:33:07.491 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: balancer 2024-02-16T19:33:07.491 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: bal_rank_mask -1 2024-02-16T19:33:07.491 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: standby_count_wanted 1 2024-02-16T19:33:07.491 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: [mds.cephfs.smithi022.btppwm{0:14490} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.22:6828/3597685751,v1:172.21.15.22:6829/3597685751] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:33:07.492 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: 2024-02-16T19:33:07.492 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: 2024-02-16T19:33:07.492 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: Standby daemons: 2024-02-16T19:33:07.492 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: 2024-02-16T19:33:07.492 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: [mds.cephfs.smithi022.ocukvs{-1:14500} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.22:6826/242865843,v1:172.21.15.22:6827/242865843] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:33:07.492 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: [mds.cephfs.smithi117.ibmppn{-1:24259} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.117:6824/565408102,v1:172.21.15.117:6825/565408102] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:33:07.492 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout: [mds.cephfs.smithi117.orzors{-1:24265} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.117:6826/3122342028,v1:172.21.15.117:6827/3122342028] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:33:07.492 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: mon.smithi022@-1(???).osd e49 crush map has features 3314933000852226048, adjusting msgr requires 2024-02-16T19:33:07.492 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: mon.smithi022@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-02-16T19:33:07.492 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: mon.smithi022@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-02-16T19:33:07.492 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: mon.smithi022@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-02-16T19:33:07.492 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 ceph-mon[88798]: mon.smithi022@-1(???).paxosservice(auth 1..23) refresh upgraded, format 0 -> 3 2024-02-16T19:33:07.492 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 bash[88779]: 9c84755ae67dec075119dc6bbcd217bfd244588ae7131bc1b01843432cd24743 2024-02-16T19:33:07.492 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:07 smithi022 systemd[1]: Started Ceph mon.smithi022 for 2a1346e6-cd00-11ee-95ba-87774f69a715. 2024-02-16T19:33:09.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:08 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mon metadata", "id": "smithi022"}]: dispatch 2024-02-16T19:33:09.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:08 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mon metadata", "id": "smithi117"}]: dispatch 2024-02-16T19:33:09.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:08 smithi117 ceph-mon[35309]: pgmap v100: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:33:09.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:08 smithi117 ceph-mon[35309]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:33:09.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:08 smithi117 ceph-mon[35309]: mon.smithi022 calling monitor election 2024-02-16T19:33:09.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:08 smithi117 ceph-mon[35309]: mon.smithi022 is new leader, mons smithi022,smithi117 in quorum (ranks 0,1) 2024-02-16T19:33:09.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:08 smithi117 ceph-mon[35309]: monmap e2: 2 mons at {smithi022=[v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0],smithi117=[v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-16T19:33:09.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:08 smithi117 ceph-mon[35309]: fsmap cephfs:1 {0=cephfs.smithi022.btppwm=up:active} 3 up:standby 2024-02-16T19:33:09.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:08 smithi117 ceph-mon[35309]: osdmap e49: 6 total, 6 up, 6 in 2024-02-16T19:33:09.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:08 smithi117 ceph-mon[35309]: mgrmap e33: smithi022.gmdcch(active, since 3m), standbys: smithi117.cohbtv 2024-02-16T19:33:09.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:08 smithi117 ceph-mon[35309]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-02-16T19:33:09.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:08 smithi117 ceph-mon[35309]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-02-16T19:33:09.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:08 smithi117 ceph-mon[35309]: fs cephfs has deprecated feature inline_data enabled. 2024-02-16T19:33:09.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:08 smithi117 ceph-mon[35309]: from='mgr.24407 ' entity='' 2024-02-16T19:33:09.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:08 smithi117 ceph-mon[35309]: mgrmap e34: smithi022.gmdcch(active, since 3m), standbys: smithi117.cohbtv 2024-02-16T19:33:09.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:08 smithi022 ceph-mon[88798]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mon metadata", "id": "smithi022"}]: dispatch 2024-02-16T19:33:09.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:08 smithi022 ceph-mon[88798]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mon metadata", "id": "smithi117"}]: dispatch 2024-02-16T19:33:09.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:08 smithi022 ceph-mon[88798]: pgmap v100: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:33:09.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:08 smithi022 ceph-mon[88798]: from='mgr.24407 172.21.15.22:0/734716407' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:33:09.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:08 smithi022 ceph-mon[88798]: mon.smithi022 calling monitor election 2024-02-16T19:33:09.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:08 smithi022 ceph-mon[88798]: mon.smithi022 is new leader, mons smithi022,smithi117 in quorum (ranks 0,1) 2024-02-16T19:33:09.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:08 smithi022 ceph-mon[88798]: monmap e2: 2 mons at {smithi022=[v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0],smithi117=[v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-16T19:33:09.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:08 smithi022 ceph-mon[88798]: fsmap cephfs:1 {0=cephfs.smithi022.btppwm=up:active} 3 up:standby 2024-02-16T19:33:09.216 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:08 smithi022 ceph-mon[88798]: osdmap e49: 6 total, 6 up, 6 in 2024-02-16T19:33:09.216 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:08 smithi022 ceph-mon[88798]: mgrmap e33: smithi022.gmdcch(active, since 3m), standbys: smithi117.cohbtv 2024-02-16T19:33:09.216 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:08 smithi022 ceph-mon[88798]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-02-16T19:33:09.216 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:08 smithi022 ceph-mon[88798]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-02-16T19:33:09.216 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:08 smithi022 ceph-mon[88798]: fs cephfs has deprecated feature inline_data enabled. 2024-02-16T19:33:09.216 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:08 smithi022 ceph-mon[88798]: from='mgr.24407 ' entity='' 2024-02-16T19:33:09.216 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:08 smithi022 ceph-mon[88798]: mgrmap e34: smithi022.gmdcch(active, since 3m), standbys: smithi117.cohbtv 2024-02-16T19:33:15.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:15 smithi022 ceph-mon[88798]: Standby manager daemon smithi117.cohbtv restarted 2024-02-16T19:33:15.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:15 smithi022 ceph-mon[88798]: Standby manager daemon smithi117.cohbtv started 2024-02-16T19:33:15.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:15 smithi022 ceph-mon[88798]: from='mgr.? 172.21.15.117:0/1499812472' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi117.cohbtv/crt"}]: dispatch 2024-02-16T19:33:15.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:15 smithi022 ceph-mon[88798]: from='mgr.? 172.21.15.117:0/1499812472' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-16T19:33:15.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:15 smithi022 ceph-mon[88798]: from='mgr.? 172.21.15.117:0/1499812472' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi117.cohbtv/key"}]: dispatch 2024-02-16T19:33:15.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:15 smithi022 ceph-mon[88798]: from='mgr.? 172.21.15.117:0/1499812472' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-16T19:33:15.886 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:15 smithi117 ceph-mon[35309]: Standby manager daemon smithi117.cohbtv restarted 2024-02-16T19:33:15.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:15 smithi117 ceph-mon[35309]: Standby manager daemon smithi117.cohbtv started 2024-02-16T19:33:15.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:15 smithi117 ceph-mon[35309]: from='mgr.? 172.21.15.117:0/1499812472' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi117.cohbtv/crt"}]: dispatch 2024-02-16T19:33:15.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:15 smithi117 ceph-mon[35309]: from='mgr.? 172.21.15.117:0/1499812472' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-16T19:33:15.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:15 smithi117 ceph-mon[35309]: from='mgr.? 172.21.15.117:0/1499812472' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi117.cohbtv/key"}]: dispatch 2024-02-16T19:33:15.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:15 smithi117 ceph-mon[35309]: from='mgr.? 172.21.15.117:0/1499812472' entity='mgr.smithi117.cohbtv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-16T19:33:16.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:16 smithi117 ceph-mon[35309]: mgrmap e35: smithi022.gmdcch(active, since 3m), standbys: smithi117.cohbtv 2024-02-16T19:33:16.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:16 smithi117 ceph-mon[35309]: Active manager daemon smithi022.gmdcch restarted 2024-02-16T19:33:16.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:16 smithi117 ceph-mon[35309]: Activating manager daemon smithi022.gmdcch 2024-02-16T19:33:16.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:16 smithi117 ceph-mon[35309]: osdmap e50: 6 total, 6 up, 6 in 2024-02-16T19:33:16.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:16 smithi117 ceph-mon[35309]: mgrmap e36: smithi022.gmdcch(active, starting, since 0.00567413s), standbys: smithi117.cohbtv 2024-02-16T19:33:16.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:16 smithi117 ceph-mon[35309]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mon metadata", "id": "smithi022"}]: dispatch 2024-02-16T19:33:16.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:16 smithi117 ceph-mon[35309]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mon metadata", "id": "smithi117"}]: dispatch 2024-02-16T19:33:16.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:16 smithi117 ceph-mon[35309]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi022.btppwm"}]: dispatch 2024-02-16T19:33:16.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:16 smithi117 ceph-mon[35309]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi022.ocukvs"}]: dispatch 2024-02-16T19:33:16.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:16 smithi117 ceph-mon[35309]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi117.ibmppn"}]: dispatch 2024-02-16T19:33:16.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:16 smithi117 ceph-mon[35309]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi117.orzors"}]: dispatch 2024-02-16T19:33:16.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:16 smithi117 ceph-mon[35309]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mgr metadata", "who": "smithi022.gmdcch", "id": "smithi022.gmdcch"}]: dispatch 2024-02-16T19:33:16.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:16 smithi117 ceph-mon[35309]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mgr metadata", "who": "smithi117.cohbtv", "id": "smithi117.cohbtv"}]: dispatch 2024-02-16T19:33:16.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:16 smithi117 ceph-mon[35309]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-16T19:33:16.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:16 smithi117 ceph-mon[35309]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-16T19:33:16.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:16 smithi117 ceph-mon[35309]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-16T19:33:16.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:16 smithi117 ceph-mon[35309]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-16T19:33:16.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:16 smithi117 ceph-mon[35309]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-16T19:33:16.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:16 smithi117 ceph-mon[35309]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-16T19:33:16.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:16 smithi117 ceph-mon[35309]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-16T19:33:16.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:16 smithi117 ceph-mon[35309]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-16T19:33:16.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:16 smithi117 ceph-mon[35309]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-16T19:33:16.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:16 smithi117 ceph-mon[35309]: Manager daemon smithi022.gmdcch is now available 2024-02-16T19:33:16.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:16 smithi117 ceph-mon[35309]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:33:16.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:16 smithi117 ceph-mon[35309]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:33:16.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:16 smithi117 ceph-mon[35309]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:33:16.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:16 smithi117 ceph-mon[35309]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi022.gmdcch/mirror_snapshot_schedule"}]: dispatch 2024-02-16T19:33:16.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:16 smithi117 ceph-mon[35309]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi022.gmdcch/trash_purge_schedule"}]: dispatch 2024-02-16T19:33:16.934 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:16 smithi022 ceph-mon[88798]: mgrmap e35: smithi022.gmdcch(active, since 3m), standbys: smithi117.cohbtv 2024-02-16T19:33:16.934 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:16 smithi022 ceph-mon[88798]: Active manager daemon smithi022.gmdcch restarted 2024-02-16T19:33:16.935 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:16 smithi022 ceph-mon[88798]: Activating manager daemon smithi022.gmdcch 2024-02-16T19:33:16.935 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:16 smithi022 ceph-mon[88798]: osdmap e50: 6 total, 6 up, 6 in 2024-02-16T19:33:16.935 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:16 smithi022 ceph-mon[88798]: mgrmap e36: smithi022.gmdcch(active, starting, since 0.00567413s), standbys: smithi117.cohbtv 2024-02-16T19:33:16.935 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:16 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mon metadata", "id": "smithi022"}]: dispatch 2024-02-16T19:33:16.935 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:16 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mon metadata", "id": "smithi117"}]: dispatch 2024-02-16T19:33:16.935 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:16 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi022.btppwm"}]: dispatch 2024-02-16T19:33:16.935 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:16 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi022.ocukvs"}]: dispatch 2024-02-16T19:33:16.935 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:16 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi117.ibmppn"}]: dispatch 2024-02-16T19:33:16.935 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:16 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi117.orzors"}]: dispatch 2024-02-16T19:33:16.935 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:16 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mgr metadata", "who": "smithi022.gmdcch", "id": "smithi022.gmdcch"}]: dispatch 2024-02-16T19:33:16.935 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:16 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mgr metadata", "who": "smithi117.cohbtv", "id": "smithi117.cohbtv"}]: dispatch 2024-02-16T19:33:16.935 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:16 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-16T19:33:16.936 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:16 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-16T19:33:16.936 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:16 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-16T19:33:16.936 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:16 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-16T19:33:16.936 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:16 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-16T19:33:16.936 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:16 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-16T19:33:16.936 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:16 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-16T19:33:16.936 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:16 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-16T19:33:16.936 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:16 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-16T19:33:16.936 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:16 smithi022 ceph-mon[88798]: Manager daemon smithi022.gmdcch is now available 2024-02-16T19:33:16.936 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:16 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:33:16.936 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:16 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:33:16.936 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:16 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:33:16.937 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:16 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi022.gmdcch/mirror_snapshot_schedule"}]: dispatch 2024-02-16T19:33:16.937 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:16 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi022.gmdcch/trash_purge_schedule"}]: dispatch 2024-02-16T19:33:17.806 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:17 smithi117 ceph-mon[35309]: [16/Feb/2024:19:33:16] ENGINE Bus STARTING 2024-02-16T19:33:17.806 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:17 smithi117 ceph-mon[35309]: [16/Feb/2024:19:33:16] ENGINE Serving on https://172.21.15.22:7150 2024-02-16T19:33:17.806 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:17 smithi117 ceph-mon[35309]: [16/Feb/2024:19:33:16] ENGINE Serving on http://172.21.15.22:8765 2024-02-16T19:33:17.806 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:17 smithi117 ceph-mon[35309]: [16/Feb/2024:19:33:16] ENGINE Bus STARTED 2024-02-16T19:33:17.806 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:17 smithi117 ceph-mon[35309]: mgrmap e37: smithi022.gmdcch(active, since 1.00977s), standbys: smithi117.cohbtv 2024-02-16T19:33:17.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:17 smithi022 ceph-mon[88798]: [16/Feb/2024:19:33:16] ENGINE Bus STARTING 2024-02-16T19:33:17.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:17 smithi022 ceph-mon[88798]: [16/Feb/2024:19:33:16] ENGINE Serving on https://172.21.15.22:7150 2024-02-16T19:33:17.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:17 smithi022 ceph-mon[88798]: [16/Feb/2024:19:33:16] ENGINE Serving on http://172.21.15.22:8765 2024-02-16T19:33:17.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:17 smithi022 ceph-mon[88798]: [16/Feb/2024:19:33:16] ENGINE Bus STARTED 2024-02-16T19:33:17.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:17 smithi022 ceph-mon[88798]: mgrmap e37: smithi022.gmdcch(active, since 1.00977s), standbys: smithi117.cohbtv 2024-02-16T19:33:18.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:18 smithi022 ceph-mon[88798]: pgmap v3: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:33:18.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:18 smithi117 ceph-mon[35309]: pgmap v3: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:33:19.857 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:19 smithi022 ceph-mon[88798]: pgmap v4: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:33:19.857 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:19 smithi022 ceph-mon[88798]: mgrmap e38: smithi022.gmdcch(active, since 3s), standbys: smithi117.cohbtv 2024-02-16T19:33:19.857 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:19 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:19.857 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:19 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:19.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:19 smithi117 ceph-mon[35309]: pgmap v4: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:33:19.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:19 smithi117 ceph-mon[35309]: mgrmap e38: smithi022.gmdcch(active, since 3s), standbys: smithi117.cohbtv 2024-02-16T19:33:19.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:19 smithi117 ceph-mon[35309]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:19.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:19 smithi117 ceph-mon[35309]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:21.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:21 smithi117 ceph-mon[35309]: pgmap v5: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:33:21.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:21 smithi117 ceph-mon[35309]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:21.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:21 smithi117 ceph-mon[35309]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:21.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:21 smithi117 ceph-mon[35309]: mgrmap e39: smithi022.gmdcch(active, since 4s), standbys: smithi117.cohbtv 2024-02-16T19:33:21.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:21 smithi022 ceph-mon[88798]: pgmap v5: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:33:21.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:21 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:21.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:21 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:21.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:21 smithi022 ceph-mon[88798]: mgrmap e39: smithi022.gmdcch(active, since 4s), standbys: smithi117.cohbtv 2024-02-16T19:33:22.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:22 smithi117 ceph-mon[35309]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:22.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:22 smithi117 ceph-mon[35309]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:22.464 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:22 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:22.464 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:22 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:23.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:23 smithi117 ceph-mon[35309]: pgmap v6: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:33:23.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:23 smithi117 ceph-mon[35309]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:23.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:23 smithi117 ceph-mon[35309]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:23.716 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:23 smithi022 ceph-mon[88798]: pgmap v6: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:33:23.716 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:23 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:23.716 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:23 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:25.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:25 smithi117 ceph-mon[35309]: pgmap v7: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-02-16T19:33:25.648 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:25 smithi022 ceph-mon[88798]: pgmap v7: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-02-16T19:33:26.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:26 smithi022 ceph-mon[88798]: Detected new or changed devices on smithi022 2024-02-16T19:33:26.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:26 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:26.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:26 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:26.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:26 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:33:26.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:26 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:33:26.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:26 smithi022 ceph-mon[88798]: Updating smithi022:/etc/ceph/ceph.conf 2024-02-16T19:33:26.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:26 smithi022 ceph-mon[88798]: Updating smithi117:/etc/ceph/ceph.conf 2024-02-16T19:33:27.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:26 smithi117 ceph-mon[35309]: Detected new or changed devices on smithi022 2024-02-16T19:33:27.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:26 smithi117 ceph-mon[35309]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:27.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:26 smithi117 ceph-mon[35309]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:27.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:26 smithi117 ceph-mon[35309]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:33:27.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:26 smithi117 ceph-mon[35309]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:33:27.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:26 smithi117 ceph-mon[35309]: Updating smithi022:/etc/ceph/ceph.conf 2024-02-16T19:33:27.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:26 smithi117 ceph-mon[35309]: Updating smithi117:/etc/ceph/ceph.conf 2024-02-16T19:33:28.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:27 smithi117 ceph-mon[35309]: pgmap v8: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-02-16T19:33:28.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:27 smithi117 ceph-mon[35309]: Updating smithi022:/var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T19:33:28.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:27 smithi117 ceph-mon[35309]: Updating smithi117:/var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T19:33:28.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:27 smithi117 ceph-mon[35309]: Updating smithi022:/etc/ceph/ceph.client.admin.keyring 2024-02-16T19:33:28.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:27 smithi117 ceph-mon[35309]: Updating smithi117:/etc/ceph/ceph.client.admin.keyring 2024-02-16T19:33:28.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:27 smithi117 ceph-mon[35309]: Updating smithi022:/var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-16T19:33:28.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:27 smithi117 ceph-mon[35309]: Updating smithi117:/var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-16T19:33:28.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:27 smithi117 ceph-mon[35309]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:28.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:27 smithi117 ceph-mon[35309]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:28.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:27 smithi117 ceph-mon[35309]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:28.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:27 smithi117 ceph-mon[35309]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:28.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:27 smithi117 ceph-mon[35309]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:28.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:27 smithi117 ceph-mon[35309]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:33:28.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:27 smithi117 ceph-mon[35309]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:28.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:27 smithi117 ceph-mon[35309]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-02-16T19:33:28.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:27 smithi022 ceph-mon[88798]: pgmap v8: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-02-16T19:33:28.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:27 smithi022 ceph-mon[88798]: Updating smithi022:/var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T19:33:28.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:27 smithi022 ceph-mon[88798]: Updating smithi117:/var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T19:33:28.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:27 smithi022 ceph-mon[88798]: Updating smithi022:/etc/ceph/ceph.client.admin.keyring 2024-02-16T19:33:28.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:27 smithi022 ceph-mon[88798]: Updating smithi117:/etc/ceph/ceph.client.admin.keyring 2024-02-16T19:33:28.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:27 smithi022 ceph-mon[88798]: Updating smithi022:/var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-16T19:33:28.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:27 smithi022 ceph-mon[88798]: Updating smithi117:/var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-16T19:33:28.216 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:27 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:28.216 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:27 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:28.216 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:27 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:28.216 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:27 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:28.216 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:27 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:28.216 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:27 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:33:28.216 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:27 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:28.216 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:27 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-02-16T19:33:29.792 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:29 smithi117 ceph-mon[35309]: pgmap v9: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-16T19:33:29.792 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:29 smithi117 ceph-mon[35309]: Upgrade: Updating mon.smithi117 2024-02-16T19:33:29.793 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:29 smithi117 ceph-mon[35309]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:29.793 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:29 smithi117 ceph-mon[35309]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-16T19:33:29.793 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:29 smithi117 ceph-mon[35309]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-16T19:33:29.793 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:29 smithi117 ceph-mon[35309]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:33:29.793 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:29 smithi117 ceph-mon[35309]: Deploying daemon mon.smithi117 on smithi117 2024-02-16T19:33:29.855 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:29 smithi022 ceph-mon[88798]: pgmap v9: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-16T19:33:29.855 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:29 smithi022 ceph-mon[88798]: Upgrade: Updating mon.smithi117 2024-02-16T19:33:29.855 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:29 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:29.856 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:29 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-16T19:33:29.856 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:29 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-16T19:33:29.856 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:29 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:33:29.856 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:29 smithi022 ceph-mon[88798]: Deploying daemon mon.smithi117 on smithi117 2024-02-16T19:33:30.631 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:30 smithi117 systemd[1]: Stopping Ceph mon.smithi117 for 2a1346e6-cd00-11ee-95ba-87774f69a715... 2024-02-16T19:33:30.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:30 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-mon-smithi117[35305]: 2024-02-16T19:33:30.628+0000 7f26ababe700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi117 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-16T19:33:30.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:30 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-mon-smithi117[35305]: 2024-02-16T19:33:30.628+0000 7f26ababe700 -1 mon.smithi117@1(peon) e2 *** Got Signal Terminated *** 2024-02-16T19:33:30.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:30 smithi117 podman[67706]: 2024-02-16 19:33:30.711452023 +0000 UTC m=+0.200327748 container died aff3e21a73a066313217f8602d22198de2ba1116dd786752ae3316405b220af0 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-mon-smithi117, org.label-schema.license=GPLv2, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, GIT_BRANCH=HEAD, org.label-schema.build-date=20231212, org.label-schema.vendor=CentOS, RELEASE=HEAD, CEPH_POINT_RELEASE=-18.2.0, GIT_CLEAN=True, io.buildah.version=1.29.1, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=https://github.com/ceph/ceph-container.git, ceph=True) 2024-02-16T19:33:31.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:31 smithi117 podman[67706]: 2024-02-16 19:33:31.271240715 +0000 UTC m=+0.760116436 container cleanup aff3e21a73a066313217f8602d22198de2ba1116dd786752ae3316405b220af0 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-mon-smithi117, RELEASE=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20231212, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.29.1, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=-18.2.0, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308) 2024-02-16T19:33:31.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:31 smithi117 bash[67706]: ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-mon-smithi117 2024-02-16T19:33:31.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:31 smithi117 podman[67720]: 2024-02-16 19:33:31.539755384 +0000 UTC m=+0.830606451 container remove aff3e21a73a066313217f8602d22198de2ba1116dd786752ae3316405b220af0 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-mon-smithi117, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.build-date=20231212, CEPH_POINT_RELEASE=-18.2.0, RELEASE=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.29.1) 2024-02-16T19:33:31.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:31 smithi117 systemd[1]: ceph-2a1346e6-cd00-11ee-95ba-87774f69a715@mon.smithi117.service: Deactivated successfully. 2024-02-16T19:33:31.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:31 smithi117 systemd[1]: Stopped Ceph mon.smithi117 for 2a1346e6-cd00-11ee-95ba-87774f69a715. 2024-02-16T19:33:31.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:31 smithi117 systemd[1]: ceph-2a1346e6-cd00-11ee-95ba-87774f69a715@mon.smithi117.service: Consumed 7.458s CPU time. 2024-02-16T19:33:32.212 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:31 smithi117 systemd[1]: Starting Ceph mon.smithi117 for 2a1346e6-cd00-11ee-95ba-87774f69a715... 2024-02-16T19:33:32.212 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 podman[67852]: 2024-02-16 19:33:32.01597238 +0000 UTC m=+0.017873730 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T19:33:32.212 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 podman[67852]: 2024-02-16 19:33:32.124388208 +0000 UTC m=+0.126289603 container create a319ac50f1a9c776e6b81aca2833d96d5fb6d8949afc2ad72abe37a644414d62 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-mon-smithi117, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_BRANCH=HEAD) 2024-02-16T19:33:32.395 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 podman[67852]: 2024-02-16 19:33:32.360210463 +0000 UTC m=+0.362111812 container init a319ac50f1a9c776e6b81aca2833d96d5fb6d8949afc2ad72abe37a644414d62 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-mon-smithi117, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-02-16T19:33:32.395 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 podman[67852]: 2024-02-16 19:33:32.363341187 +0000 UTC m=+0.365242536 container start a319ac50f1a9c776e6b81aca2833d96d5fb6d8949afc2ad72abe37a644414d62 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-mon-smithi117, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-02-16T19:33:32.395 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: set uid:gid to 167:167 (ceph:ceph) 2024-02-16T19:33:32.395 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev), process ceph-mon, pid 2 2024-02-16T19:33:32.396 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: pidfile_write: ignore empty --pid-file 2024-02-16T19:33:32.396 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: load: jerasure load: lrc 2024-02-16T19:33:32.396 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: RocksDB version: 7.9.2 2024-02-16T19:33:32.396 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Git sha 0 2024-02-16T19:33:32.396 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Compile date 2024-02-15 18:05:23 2024-02-16T19:33:32.396 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: DB SUMMARY 2024-02-16T19:33:32.396 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: DB Session ID: 6Y6JEW5CUECKVCV69PNY 2024-02-16T19:33:32.396 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: CURRENT file: CURRENT 2024-02-16T19:33:32.396 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: IDENTITY file: IDENTITY 2024-02-16T19:33:32.396 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: MANIFEST file: MANIFEST-000010 size: 1403 Bytes 2024-02-16T19:33:32.396 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi117/store.db dir, Total Num: 1, files: 000027.sst 2024-02-16T19:33:32.396 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi117/store.db: 000025.log size: 8377460 ; 2024-02-16T19:33:32.396 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.error_if_exists: 0 2024-02-16T19:33:32.396 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.create_if_missing: 0 2024-02-16T19:33:32.397 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.paranoid_checks: 1 2024-02-16T19:33:32.397 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.flush_verify_memtable_count: 1 2024-02-16T19:33:32.397 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-16T19:33:32.397 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-02-16T19:33:32.397 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.env: 0x557e3d9edae0 2024-02-16T19:33:32.397 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.fs: PosixFileSystem 2024-02-16T19:33:32.397 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.info_log: 0x557e3e2bb760 2024-02-16T19:33:32.397 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.max_file_opening_threads: 16 2024-02-16T19:33:32.398 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.statistics: (nil) 2024-02-16T19:33:32.398 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.use_fsync: 0 2024-02-16T19:33:32.398 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.max_log_file_size: 0 2024-02-16T19:33:32.398 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-16T19:33:32.399 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-16T19:33:32.399 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.keep_log_file_num: 1000 2024-02-16T19:33:32.399 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.recycle_log_file_num: 0 2024-02-16T19:33:32.399 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.allow_fallocate: 1 2024-02-16T19:33:32.399 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.allow_mmap_reads: 0 2024-02-16T19:33:32.399 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.allow_mmap_writes: 0 2024-02-16T19:33:32.399 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.use_direct_reads: 0 2024-02-16T19:33:32.399 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-16T19:33:32.399 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.create_missing_column_families: 0 2024-02-16T19:33:32.399 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.db_log_dir: 2024-02-16T19:33:32.399 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.wal_dir: 2024-02-16T19:33:32.399 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-16T19:33:32.399 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-16T19:33:32.399 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-16T19:33:32.400 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-16T19:33:32.400 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-16T19:33:32.400 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-16T19:33:32.400 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.advise_random_on_open: 1 2024-02-16T19:33:32.400 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.db_write_buffer_size: 0 2024-02-16T19:33:32.400 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.write_buffer_manager: 0x557e3e28f400 2024-02-16T19:33:32.400 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-16T19:33:32.400 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-16T19:33:32.400 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-16T19:33:32.400 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.rate_limiter: (nil) 2024-02-16T19:33:32.400 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-16T19:33:32.401 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.wal_recovery_mode: 2 2024-02-16T19:33:32.401 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.enable_thread_tracking: 0 2024-02-16T19:33:32.401 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.enable_pipelined_write: 0 2024-02-16T19:33:32.401 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.unordered_write: 0 2024-02-16T19:33:32.401 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-16T19:33:32.401 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-16T19:33:32.401 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-16T19:33:32.401 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-16T19:33:32.401 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.row_cache: None 2024-02-16T19:33:32.401 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.wal_filter: None 2024-02-16T19:33:32.401 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-16T19:33:32.401 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.allow_ingest_behind: 0 2024-02-16T19:33:32.401 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.two_write_queues: 0 2024-02-16T19:33:32.401 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.manual_wal_flush: 0 2024-02-16T19:33:32.402 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.wal_compression: 0 2024-02-16T19:33:32.402 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.atomic_flush: 0 2024-02-16T19:33:32.402 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-16T19:33:32.402 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-16T19:33:32.402 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-16T19:33:32.402 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.log_readahead_size: 0 2024-02-16T19:33:32.402 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-16T19:33:32.402 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.best_efforts_recovery: 0 2024-02-16T19:33:32.402 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-16T19:33:32.402 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-16T19:33:32.402 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.allow_data_in_errors: 0 2024-02-16T19:33:32.402 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.db_host_id: __hostname__ 2024-02-16T19:33:32.402 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.enforce_single_del_contracts: true 2024-02-16T19:33:32.402 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.max_background_jobs: 2 2024-02-16T19:33:32.402 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.max_background_compactions: -1 2024-02-16T19:33:32.403 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.max_subcompactions: 1 2024-02-16T19:33:32.403 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-16T19:33:32.403 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-16T19:33:32.403 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-16T19:33:32.403 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.max_total_wal_size: 0 2024-02-16T19:33:32.403 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-16T19:33:32.403 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-16T19:33:32.403 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-16T19:33:32.403 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-16T19:33:32.403 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.max_open_files: -1 2024-02-16T19:33:32.403 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.bytes_per_sync: 0 2024-02-16T19:33:32.403 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-16T19:33:32.403 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-16T19:33:32.403 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.compaction_readahead_size: 0 2024-02-16T19:33:32.404 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.max_background_flushes: -1 2024-02-16T19:33:32.404 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Compression algorithms supported: 2024-02-16T19:33:32.404 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-16T19:33:32.404 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: kZSTD supported: 0 2024-02-16T19:33:32.404 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: kXpressCompression supported: 0 2024-02-16T19:33:32.404 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: kLZ4HCCompression supported: 1 2024-02-16T19:33:32.404 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: kZlibCompression supported: 1 2024-02-16T19:33:32.404 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: kSnappyCompression supported: 1 2024-02-16T19:33:32.404 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: kLZ4Compression supported: 1 2024-02-16T19:33:32.404 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: kBZip2Compression supported: 0 2024-02-16T19:33:32.404 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-16T19:33:32.404 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: DMutex implementation: pthread_mutex_t 2024-02-16T19:33:32.404 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi117/store.db/MANIFEST-000010 2024-02-16T19:33:32.404 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-02-16T19:33:32.405 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-16T19:33:32.405 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.merge_operator: 2024-02-16T19:33:32.405 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.compaction_filter: None 2024-02-16T19:33:32.405 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.compaction_filter_factory: None 2024-02-16T19:33:32.405 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.sst_partitioner_factory: None 2024-02-16T19:33:32.405 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-16T19:33:32.405 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-16T19:33:32.405 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x557e3e2bb880) 2024-02-16T19:33:32.405 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: cache_index_and_filter_blocks: 1 2024-02-16T19:33:32.405 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-16T19:33:32.405 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-16T19:33:32.405 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: pin_top_level_index_and_filter: 1 2024-02-16T19:33:32.405 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: index_type: 0 2024-02-16T19:33:32.405 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: data_block_index_type: 0 2024-02-16T19:33:32.405 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: index_shortening: 1 2024-02-16T19:33:32.405 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-16T19:33:32.406 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: checksum: 4 2024-02-16T19:33:32.406 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: no_block_cache: 0 2024-02-16T19:33:32.406 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: block_cache: 0x557e3e2bf090 2024-02-16T19:33:32.406 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: block_cache_name: BinnedLRUCache 2024-02-16T19:33:32.406 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: block_cache_options: 2024-02-16T19:33:32.406 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: capacity : 536870912 2024-02-16T19:33:32.406 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: num_shard_bits : 4 2024-02-16T19:33:32.406 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: strict_capacity_limit : 0 2024-02-16T19:33:32.406 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: high_pri_pool_ratio: 0.000 2024-02-16T19:33:32.406 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: block_cache_compressed: (nil) 2024-02-16T19:33:32.406 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: persistent_cache: (nil) 2024-02-16T19:33:32.406 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: block_size: 4096 2024-02-16T19:33:32.406 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: block_size_deviation: 10 2024-02-16T19:33:32.406 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: block_restart_interval: 16 2024-02-16T19:33:32.406 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: index_block_restart_interval: 1 2024-02-16T19:33:32.407 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: metadata_block_size: 4096 2024-02-16T19:33:32.407 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: partition_filters: 0 2024-02-16T19:33:32.407 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: use_delta_encoding: 1 2024-02-16T19:33:32.407 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: filter_policy: bloomfilter 2024-02-16T19:33:32.407 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: whole_key_filtering: 1 2024-02-16T19:33:32.407 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: verify_compression: 0 2024-02-16T19:33:32.407 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: read_amp_bytes_per_bit: 0 2024-02-16T19:33:32.407 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: format_version: 5 2024-02-16T19:33:32.407 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: enable_index_compression: 1 2024-02-16T19:33:32.407 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: block_align: 0 2024-02-16T19:33:32.407 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: max_auto_readahead_size: 262144 2024-02-16T19:33:32.407 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: prepopulate_block_cache: 0 2024-02-16T19:33:32.407 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: initial_auto_readahead_size: 8192 2024-02-16T19:33:32.408 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: num_file_reads_for_auto_readahead: 2 2024-02-16T19:33:32.408 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.write_buffer_size: 33554432 2024-02-16T19:33:32.408 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.max_write_buffer_number: 2 2024-02-16T19:33:32.408 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.compression: NoCompression 2024-02-16T19:33:32.408 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.bottommost_compression: Disabled 2024-02-16T19:33:32.408 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.prefix_extractor: nullptr 2024-02-16T19:33:32.408 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-16T19:33:32.408 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.num_levels: 7 2024-02-16T19:33:32.408 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-16T19:33:32.408 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-16T19:33:32.408 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-16T19:33:32.408 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-16T19:33:32.408 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-16T19:33:32.408 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-16T19:33:32.409 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-16T19:33:32.409 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-16T19:33:32.409 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-16T19:33:32.409 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-16T19:33:32.409 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-02-16T19:33:32.409 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-02-16T19:33:32.409 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-16T19:33:32.409 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.compression_opts.level: 32767 2024-02-16T19:33:32.409 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.compression_opts.strategy: 0 2024-02-16T19:33:32.409 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-16T19:33:32.409 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-16T19:33:32.409 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-02-16T19:33:32.409 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-16T19:33:32.409 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.compression_opts.enabled: false 2024-02-16T19:33:32.410 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-02-16T19:33:32.410 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-16T19:33:32.410 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-16T19:33:32.410 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-16T19:33:32.410 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.target_file_size_base: 67108864 2024-02-16T19:33:32.410 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-16T19:33:32.410 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-16T19:33:32.410 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-16T19:33:32.410 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-16T19:33:32.410 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-16T19:33:32.410 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-16T19:33:32.410 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-16T19:33:32.410 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-16T19:33:32.411 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-16T19:33:32.411 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-16T19:33:32.411 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-16T19:33:32.411 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-16T19:33:32.411 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-16T19:33:32.411 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-02-16T19:33:32.411 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.arena_block_size: 1048576 2024-02-16T19:33:32.411 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-16T19:33:32.411 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-16T19:33:32.411 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.disable_auto_compactions: 0 2024-02-16T19:33:32.411 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-16T19:33:32.411 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-16T19:33:32.411 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-16T19:33:32.411 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-16T19:33:32.411 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-16T19:33:32.412 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-16T19:33:32.412 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-16T19:33:32.412 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-16T19:33:32.412 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-16T19:33:32.412 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-16T19:33:32.412 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.table_properties_collectors: 2024-02-16T19:33:32.412 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.inplace_update_support: 0 2024-02-16T19:33:32.412 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-16T19:33:32.412 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-16T19:33:32.412 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-16T19:33:32.412 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-16T19:33:32.412 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.bloom_locality: 0 2024-02-16T19:33:32.412 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.max_successive_merges: 0 2024-02-16T19:33:32.412 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-16T19:33:32.412 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.paranoid_file_checks: 0 2024-02-16T19:33:32.413 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.force_consistency_checks: 1 2024-02-16T19:33:32.413 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.report_bg_io_stats: 0 2024-02-16T19:33:32.413 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.ttl: 2592000 2024-02-16T19:33:32.413 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-16T19:33:32.413 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-02-16T19:33:32.413 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-02-16T19:33:32.413 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.enable_blob_files: false 2024-02-16T19:33:32.413 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.min_blob_size: 0 2024-02-16T19:33:32.413 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.blob_file_size: 268435456 2024-02-16T19:33:32.413 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-16T19:33:32.413 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-16T19:33:32.413 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-16T19:33:32.413 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-02-16T19:33:32.413 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-02-16T19:33:32.413 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.blob_file_starting_level: 0 2024-02-16T19:33:32.413 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-02-16T19:33:32.414 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-02-16T19:33:32.414 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi117/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 29, last_sequence is 12112, log_number is 25,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 25 2024-02-16T19:33:32.414 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 25 2024-02-16T19:33:32.414 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 056bb860-300c-4be7-a776-5fb6a0ad3677 2024-02-16T19:33:32.414 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708112012392267, "job": 1, "event": "recovery_started", "wal_files": [25]} 2024-02-16T19:33:32.414 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #25 mode 2 2024-02-16T19:33:32.477 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708112012412666, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 30, "file_size": 4979273, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 12117, "largest_seqno": 15997, "table_properties": {"data_size": 4965562, "index_size": 8722, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 3973, "raw_key_size": 34977, "raw_average_key_size": 22, "raw_value_size": 4934375, "raw_average_value_size": 3121, "num_data_blocks": 394, "num_entries": 1581, "num_filter_entries": 1581, "num_deletions": 259, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1708112012, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "056bb860-300c-4be7-a776-5fb6a0ad3677", "db_session_id": "6Y6JEW5CUECKVCV69PNY", "orig_file_number": 30, "seqno_to_time_mapping": "N/A"}} 2024-02-16T19:33:32.477 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708112012412778, "job": 1, "event": "recovery_finished"} 2024-02-16T19:33:32.478 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: [db/version_set.cc:5047] Creating manifest 32 2024-02-16T19:33:32.478 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-02-16T19:33:32.478 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi117/store.db/000025.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-16T19:33:32.478 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x557e3e3ae000 2024-02-16T19:33:32.478 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: DB pointer 0x557e3e39a000 2024-02-16T19:33:32.478 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-02-16T19:33:32.478 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-02-16T19:33:32.478 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: ** DB Stats ** 2024-02-16T19:33:32.478 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-16T19:33:32.478 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-16T19:33:32.478 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-16T19:33:32.478 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-16T19:33:32.478 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-16T19:33:32.478 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-16T19:33:32.479 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-16T19:33:32.479 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: 2024-02-16T19:33:32.479 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: ** Compaction Stats [default] ** 2024-02-16T19:33:32.479 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-16T19:33:32.479 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-02-16T19:33:32.479 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: L0 1/0 4.75 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 542.9 0.01 0.00 1 0.009 0 0 0.0 0.0 2024-02-16T19:33:32.479 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: L6 1/0 5.81 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-02-16T19:33:32.479 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Sum 2/0 10.56 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 542.9 0.01 0.00 1 0.009 0 0 0.0 0.0 2024-02-16T19:33:32.479 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 542.9 0.01 0.00 1 0.009 0 0 0.0 0.0 2024-02-16T19:33:32.479 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: 2024-02-16T19:33:32.479 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: ** Compaction Stats [default] ** 2024-02-16T19:33:32.479 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-16T19:33:32.479 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-16T19:33:32.479 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 542.9 0.01 0.00 1 0.009 0 0 0.0 0.0 2024-02-16T19:33:32.480 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: 2024-02-16T19:33:32.480 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-02-16T19:33:32.480 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: 2024-02-16T19:33:32.480 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-16T19:33:32.480 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Flush(GB): cumulative 0.005, interval 0.005 2024-02-16T19:33:32.480 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-16T19:33:32.480 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-16T19:33:32.480 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-16T19:33:32.480 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-16T19:33:32.480 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Cumulative compaction: 0.00 GB write, 206.13 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-16T19:33:32.480 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Interval compaction: 0.00 GB write, 206.13 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-16T19:33:32.480 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-16T19:33:32.480 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Block cache BinnedLRUCache@0x557e3e2bf090#2 capacity: 512.00 MB usage: 13.11 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.2e-05 secs_since: 0 2024-02-16T19:33:32.480 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,4.03 KB,0.0007689%) IndexBlock(1,9.08 KB,0.00173151%) Misc(1,0.00 KB,0%) 2024-02-16T19:33:32.481 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: 2024-02-16T19:33:32.481 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-16T19:33:32.481 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: starting mon.smithi117 rank 1 at public addrs [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] at bind addrs [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi117 fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 2024-02-16T19:33:32.481 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: mon.smithi117@-1(???) e2 preinit fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 2024-02-16T19:33:32.481 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: mon.smithi117@-1(???).mds e15 new map 2024-02-16T19:33:32.481 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: mon.smithi117@-1(???).mds e15 print_map 2024-02-16T19:33:32.481 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: e15 2024-02-16T19:33:32.481 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T19:33:32.481 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:33:32.481 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: legacy client fscid: 1 2024-02-16T19:33:32.481 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: 2024-02-16T19:33:32.481 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Filesystem 'cephfs' (1) 2024-02-16T19:33:32.481 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: fs_name cephfs 2024-02-16T19:33:32.481 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: epoch 15 2024-02-16T19:33:32.482 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T19:33:32.482 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: created 2024-02-16T19:26:17.873644+0000 2024-02-16T19:33:32.482 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: modified 2024-02-16T19:26:35.943353+0000 2024-02-16T19:33:32.482 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: tableserver 0 2024-02-16T19:33:32.482 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: root 0 2024-02-16T19:33:32.482 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: session_timeout 60 2024-02-16T19:33:32.482 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: session_autoclose 300 2024-02-16T19:33:32.482 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: max_file_size 1099511627776 2024-02-16T19:33:32.482 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: max_xattr_size 65536 2024-02-16T19:33:32.482 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: required_client_features {} 2024-02-16T19:33:32.482 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: last_failure 0 2024-02-16T19:33:32.482 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: last_failure_osd_epoch 47 2024-02-16T19:33:32.482 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:33:32.482 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: max_mds 1 2024-02-16T19:33:32.483 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: in 0 2024-02-16T19:33:32.483 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: up {0=14490} 2024-02-16T19:33:32.483 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: failed 2024-02-16T19:33:32.483 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: damaged 2024-02-16T19:33:32.483 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: stopped 2024-02-16T19:33:32.483 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: data_pools [3] 2024-02-16T19:33:32.483 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: metadata_pool 2 2024-02-16T19:33:32.483 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: inline_data enabled 2024-02-16T19:33:32.483 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: balancer 2024-02-16T19:33:32.483 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: bal_rank_mask -1 2024-02-16T19:33:32.483 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: standby_count_wanted 1 2024-02-16T19:33:32.483 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: [mds.cephfs.smithi022.btppwm{0:14490} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.22:6828/3597685751,v1:172.21.15.22:6829/3597685751] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:33:32.483 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: 2024-02-16T19:33:32.483 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: 2024-02-16T19:33:32.483 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Standby daemons: 2024-02-16T19:33:32.483 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: 2024-02-16T19:33:32.483 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: [mds.cephfs.smithi022.ocukvs{-1:14500} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.22:6826/242865843,v1:172.21.15.22:6827/242865843] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:33:32.484 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: [mds.cephfs.smithi117.ibmppn{-1:24259} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.117:6824/565408102,v1:172.21.15.117:6825/565408102] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:33:32.484 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: [mds.cephfs.smithi117.orzors{-1:24265} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.117:6826/3122342028,v1:172.21.15.117:6827/3122342028] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:33:32.484 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: mon.smithi117@-1(???).osd e50 crush map has features 3314933000852226048, adjusting msgr requires 2024-02-16T19:33:32.484 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: mon.smithi117@-1(???).osd e50 crush map has features 288514051259236352, adjusting msgr requires 2024-02-16T19:33:32.484 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: mon.smithi117@-1(???).osd e50 crush map has features 288514051259236352, adjusting msgr requires 2024-02-16T19:33:32.484 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: mon.smithi117@-1(???).osd e50 crush map has features 288514051259236352, adjusting msgr requires 2024-02-16T19:33:32.484 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 ceph-mon[67870]: mon.smithi117@-1(???).paxosservice(auth 1..24) refresh upgraded, format 0 -> 3 2024-02-16T19:33:32.484 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 bash[67852]: a319ac50f1a9c776e6b81aca2833d96d5fb6d8949afc2ad72abe37a644414d62 2024-02-16T19:33:32.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:32 smithi117 systemd[1]: Started Ceph mon.smithi117 for 2a1346e6-cd00-11ee-95ba-87774f69a715. 2024-02-16T19:33:33.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.250+0000 7fd19752c700 1 -- 172.21.15.22:0/1416967829 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd190104130 msgr2=0x7fd190104500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:33.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.250+0000 7fd19752c700 1 --2- 172.21.15.22:0/1416967829 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd190104130 0x7fd190104500 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd18000b0a0 tx=0x7fd18000b3b0 comp rx=0 tx=0).stop 2024-02-16T19:33:33.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.250+0000 7fd19752c700 1 -- 172.21.15.22:0/1416967829 shutdown_connections 2024-02-16T19:33:33.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.250+0000 7fd19752c700 1 --2- 172.21.15.22:0/1416967829 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1900ffbd0 0x7fd190100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:33.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.250+0000 7fd19752c700 1 --2- 172.21.15.22:0/1416967829 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd190104130 0x7fd190104500 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:33.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.250+0000 7fd19752c700 1 -- 172.21.15.22:0/1416967829 >> 172.21.15.22:0/1416967829 conn(0x7fd190075130 msgr2=0x7fd190075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:33:33.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.250+0000 7fd19752c700 1 -- 172.21.15.22:0/1416967829 shutdown_connections 2024-02-16T19:33:33.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.250+0000 7fd19752c700 1 -- 172.21.15.22:0/1416967829 wait complete. 2024-02-16T19:33:33.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.251+0000 7fd19752c700 1 Processor -- start 2024-02-16T19:33:33.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.251+0000 7fd19752c700 1 -- start start 2024-02-16T19:33:33.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.251+0000 7fd19752c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1900ffbd0 0x7fd190100e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:33.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.251+0000 7fd19752c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd190104130 0x7fd190101370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:33.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.251+0000 7fd19752c700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd190101940 con 0x7fd1900ffbd0 2024-02-16T19:33:33.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.251+0000 7fd19752c700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd190101ab0 con 0x7fd190104130 2024-02-16T19:33:33.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.251+0000 7fd1952c8700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1900ffbd0 0x7fd190100e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:33.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.251+0000 7fd1952c8700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1900ffbd0 0x7fd190100e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47734/0 (socket says 172.21.15.22:47734) 2024-02-16T19:33:33.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.251+0000 7fd1952c8700 1 -- 172.21.15.22:0/1513387098 learned_addr learned my addr 172.21.15.22:0/1513387098 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:33:33.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.251+0000 7fd194ac7700 1 --2- 172.21.15.22:0/1513387098 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd190104130 0x7fd190101370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:33.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.252+0000 7fd1952c8700 1 -- 172.21.15.22:0/1513387098 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd190104130 msgr2=0x7fd190101370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:33.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.252+0000 7fd1952c8700 1 --2- 172.21.15.22:0/1513387098 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd190104130 0x7fd190101370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:33.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.252+0000 7fd1952c8700 1 -- 172.21.15.22:0/1513387098 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd180009d00 con 0x7fd1900ffbd0 2024-02-16T19:33:33.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.252+0000 7fd1952c8700 1 --2- 172.21.15.22:0/1513387098 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1900ffbd0 0x7fd190100e30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd18000b070 tx=0x7fd18000cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:33:33.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.252+0000 7fd18e7fc700 1 -- 172.21.15.22:0/1513387098 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd18000e040 con 0x7fd1900ffbd0 2024-02-16T19:33:33.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.252+0000 7fd18e7fc700 1 -- 172.21.15.22:0/1513387098 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd180003e60 con 0x7fd1900ffbd0 2024-02-16T19:33:33.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.252+0000 7fd18e7fc700 1 -- 172.21.15.22:0/1513387098 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd18001d910 con 0x7fd1900ffbd0 2024-02-16T19:33:33.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.252+0000 7fd19752c700 1 -- 172.21.15.22:0/1513387098 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1901a2160 con 0x7fd1900ffbd0 2024-02-16T19:33:33.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.252+0000 7fd19752c700 1 -- 172.21.15.22:0/1513387098 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1901a2510 con 0x7fd1900ffbd0 2024-02-16T19:33:33.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.253+0000 7fd19752c700 1 -- 172.21.15.22:0/1513387098 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1901086a0 con 0x7fd1900ffbd0 2024-02-16T19:33:33.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.253+0000 7fd18e7fc700 1 -- 172.21.15.22:0/1513387098 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 39) v1 ==== 91595+0+0 (secure 0 0 0) 0x7fd180004820 con 0x7fd1900ffbd0 2024-02-16T19:33:33.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.254+0000 7fd18e7fc700 1 --2- 172.21.15.22:0/1513387098 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fd17806d9d0 0x7fd17806fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:33.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.254+0000 7fd18e7fc700 1 -- 172.21.15.22:0/1513387098 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fd18008ef90 con 0x7fd1900ffbd0 2024-02-16T19:33:33.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.254+0000 7fd194ac7700 1 --2- 172.21.15.22:0/1513387098 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fd17806d9d0 0x7fd17806fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:33.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.254+0000 7fd194ac7700 1 --2- 172.21.15.22:0/1513387098 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fd17806d9d0 0x7fd17806fe80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd184006fd0 tx=0x7fd184008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:33:33.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.259+0000 7fd18e7fc700 1 -- 172.21.15.22:0/1513387098 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd180058650 con 0x7fd1900ffbd0 2024-02-16T19:33:33.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.485+0000 7fd19752c700 1 -- 172.21.15.22:0/1513387098 --> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd190061190 con 0x7fd17806d9d0 2024-02-16T19:33:33.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.486+0000 7fd18e7fc700 1 -- 172.21.15.22:0/1513387098 <== mgr.34104 v2:172.21.15.22:6800/505266753 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7fd190061190 con 0x7fd17806d9d0 2024-02-16T19:33:33.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.488+0000 7fd19752c700 1 -- 172.21.15.22:0/1513387098 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fd17806d9d0 msgr2=0x7fd17806fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:33.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.488+0000 7fd19752c700 1 --2- 172.21.15.22:0/1513387098 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fd17806d9d0 0x7fd17806fe80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd184006fd0 tx=0x7fd184008040 comp rx=0 tx=0).stop 2024-02-16T19:33:33.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.488+0000 7fd19752c700 1 -- 172.21.15.22:0/1513387098 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1900ffbd0 msgr2=0x7fd190100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:33.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.488+0000 7fd19752c700 1 --2- 172.21.15.22:0/1513387098 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1900ffbd0 0x7fd190100e30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd18000b070 tx=0x7fd18000cdd0 comp rx=0 tx=0).stop 2024-02-16T19:33:33.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.488+0000 7fd19752c700 1 -- 172.21.15.22:0/1513387098 shutdown_connections 2024-02-16T19:33:33.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.488+0000 7fd19752c700 1 --2- 172.21.15.22:0/1513387098 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fd17806d9d0 0x7fd17806fe80 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:33.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.488+0000 7fd19752c700 1 --2- 172.21.15.22:0/1513387098 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1900ffbd0 0x7fd190100e30 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:33.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.488+0000 7fd19752c700 1 --2- 172.21.15.22:0/1513387098 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd190104130 0x7fd190101370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:33.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.488+0000 7fd19752c700 1 -- 172.21.15.22:0/1513387098 >> 172.21.15.22:0/1513387098 conn(0x7fd190075130 msgr2=0x7fd1900febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:33:33.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.488+0000 7fd19752c700 1 -- 172.21.15.22:0/1513387098 shutdown_connections 2024-02-16T19:33:33.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.489+0000 7fd19752c700 1 -- 172.21.15.22:0/1513387098 wait complete. 2024-02-16T19:33:33.499 INFO:teuthology.orchestra.run.smithi022.stdout:true 2024-02-16T19:33:33.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.595+0000 7f68a4c2d700 1 -- 172.21.15.22:0/252280772 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f68a0100e80 msgr2=0x7f68a0104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:33.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.595+0000 7f68a4c2d700 1 --2- 172.21.15.22:0/252280772 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f68a0100e80 0x7f68a0104e50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6888009910 tx=0x7f6888009c20 comp rx=0 tx=0).stop 2024-02-16T19:33:33.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.595+0000 7f68a4c2d700 1 -- 172.21.15.22:0/252280772 shutdown_connections 2024-02-16T19:33:33.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.595+0000 7f68a4c2d700 1 --2- 172.21.15.22:0/252280772 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f68a0100e80 0x7f68a0104e50 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:33.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.595+0000 7f68a4c2d700 1 --2- 172.21.15.22:0/252280772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68a01004e0 0x7f68a01008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:33.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.595+0000 7f68a4c2d700 1 -- 172.21.15.22:0/252280772 >> 172.21.15.22:0/252280772 conn(0x7f68a00fbda0 msgr2=0x7f68a00fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:33:33.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.595+0000 7f68a4c2d700 1 -- 172.21.15.22:0/252280772 shutdown_connections 2024-02-16T19:33:33.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.595+0000 7f68a4c2d700 1 -- 172.21.15.22:0/252280772 wait complete. 2024-02-16T19:33:33.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.596+0000 7f68a4c2d700 1 Processor -- start 2024-02-16T19:33:33.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.596+0000 7f68a4c2d700 1 -- start start 2024-02-16T19:33:33.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.597+0000 7f68a4c2d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68a01004e0 0x7f68a0073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:33.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.597+0000 7f68a4c2d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f68a0100e80 0x7f68a0073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:33.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.597+0000 7f68a4c2d700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68a0073ad0 con 0x7f68a0100e80 2024-02-16T19:33:33.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.597+0000 7f68a4c2d700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68a0073c10 con 0x7f68a01004e0 2024-02-16T19:33:33.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.597+0000 7f689dd9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f68a0100e80 0x7f68a0073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:33.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.597+0000 7f689dd9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f68a0100e80 0x7f68a0073590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47764/0 (socket says 172.21.15.22:47764) 2024-02-16T19:33:33.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.597+0000 7f689dd9b700 1 -- 172.21.15.22:0/403302865 learned_addr learned my addr 172.21.15.22:0/403302865 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:33:33.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.597+0000 7f689e59c700 1 --2- 172.21.15.22:0/403302865 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68a01004e0 0x7f68a0073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:33.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.597+0000 7f689e59c700 1 -- 172.21.15.22:0/403302865 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68a01004e0 msgr2=0x7f68a0073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-02-16T19:33:33.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.597+0000 7f689e59c700 1 -- 172.21.15.22:0/403302865 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68a01004e0 msgr2=0x7f68a0073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-02-16T19:33:33.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.597+0000 7f689e59c700 1 --2- 172.21.15.22:0/403302865 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68a01004e0 0x7f68a0073050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-02-16T19:33:33.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.597+0000 7f689e59c700 1 --2- 172.21.15.22:0/403302865 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68a01004e0 0x7f68a0073050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-16T19:33:33.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.597+0000 7f689dd9b700 1 -- 172.21.15.22:0/403302865 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68a01004e0 msgr2=0x7f68a0073050 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-16T19:33:33.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.597+0000 7f689dd9b700 1 --2- 172.21.15.22:0/403302865 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68a01004e0 0x7f68a0073050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:33.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.597+0000 7f689dd9b700 1 -- 172.21.15.22:0/403302865 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68880095f0 con 0x7f68a0100e80 2024-02-16T19:33:33.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.597+0000 7f689dd9b700 1 --2- 172.21.15.22:0/403302865 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f68a0100e80 0x7f68a0073590 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f68880092e0 tx=0x7f688800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:33:33.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.597+0000 7f68977fe700 1 -- 172.21.15.22:0/403302865 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f688801d070 con 0x7f68a0100e80 2024-02-16T19:33:33.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.597+0000 7f68a4c2d700 1 -- 172.21.15.22:0/403302865 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68a0073e90 con 0x7f68a0100e80 2024-02-16T19:33:33.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.597+0000 7f68977fe700 1 -- 172.21.15.22:0/403302865 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6888005dc0 con 0x7f68a0100e80 2024-02-16T19:33:33.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.597+0000 7f68977fe700 1 -- 172.21.15.22:0/403302865 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6888017400 con 0x7f68a0100e80 2024-02-16T19:33:33.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.598+0000 7f68a4c2d700 1 -- 172.21.15.22:0/403302865 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68a01a2410 con 0x7f68a0100e80 2024-02-16T19:33:33.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.599+0000 7f68a4c2d700 1 -- 172.21.15.22:0/403302865 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6880005320 con 0x7f68a0100e80 2024-02-16T19:33:33.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.599+0000 7f68977fe700 1 -- 172.21.15.22:0/403302865 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 39) v1 ==== 91595+0+0 (secure 0 0 0) 0x7f6888003a60 con 0x7f68a0100e80 2024-02-16T19:33:33.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.599+0000 7f68977fe700 1 --2- 172.21.15.22:0/403302865 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f688c06d9e0 0x7f688c06fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:33.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.600+0000 7f68977fe700 1 -- 172.21.15.22:0/403302865 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f688808eb10 con 0x7f68a0100e80 2024-02-16T19:33:33.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.600+0000 7f689e59c700 1 --2- 172.21.15.22:0/403302865 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f688c06d9e0 0x7f688c06fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:33.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.600+0000 7f689e59c700 1 --2- 172.21.15.22:0/403302865 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f688c06d9e0 0x7f688c06fe90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6898005920 tx=0x7f689800b2c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:33:33.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.604+0000 7f68977fe700 1 -- 172.21.15.22:0/403302865 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6888058150 con 0x7f68a0100e80 2024-02-16T19:33:33.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.706+0000 7f68977fe700 1 -- 172.21.15.22:0/403302865 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6888093020 con 0x7f68a0100e80 2024-02-16T19:33:33.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.834+0000 7f68a4c2d700 1 -- 172.21.15.22:0/403302865 --> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6880000bf0 con 0x7f688c06d9e0 2024-02-16T19:33:33.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.836+0000 7f68977fe700 1 -- 172.21.15.22:0/403302865 <== mgr.34104 v2:172.21.15.22:6800/505266753 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f6880000bf0 con 0x7f688c06d9e0 2024-02-16T19:33:33.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.839+0000 7f68a4c2d700 1 -- 172.21.15.22:0/403302865 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f688c06d9e0 msgr2=0x7f688c06fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:33.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.839+0000 7f68a4c2d700 1 --2- 172.21.15.22:0/403302865 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f688c06d9e0 0x7f688c06fe90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6898005920 tx=0x7f689800b2c0 comp rx=0 tx=0).stop 2024-02-16T19:33:33.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.839+0000 7f68a4c2d700 1 -- 172.21.15.22:0/403302865 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f68a0100e80 msgr2=0x7f68a0073590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:33.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.839+0000 7f68a4c2d700 1 --2- 172.21.15.22:0/403302865 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f68a0100e80 0x7f68a0073590 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f68880092e0 tx=0x7f688800be20 comp rx=0 tx=0).stop 2024-02-16T19:33:33.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.839+0000 7f68a4c2d700 1 -- 172.21.15.22:0/403302865 shutdown_connections 2024-02-16T19:33:33.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.839+0000 7f68a4c2d700 1 --2- 172.21.15.22:0/403302865 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f688c06d9e0 0x7f688c06fe90 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:33.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.839+0000 7f68a4c2d700 1 --2- 172.21.15.22:0/403302865 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68a01004e0 0x7f68a0073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:33.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.839+0000 7f68a4c2d700 1 --2- 172.21.15.22:0/403302865 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f68a0100e80 0x7f68a0073590 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:33.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.839+0000 7f68a4c2d700 1 -- 172.21.15.22:0/403302865 >> 172.21.15.22:0/403302865 conn(0x7f68a00fbda0 msgr2=0x7f68a00fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:33:33.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.839+0000 7f68a4c2d700 1 -- 172.21.15.22:0/403302865 shutdown_connections 2024-02-16T19:33:33.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.839+0000 7f68a4c2d700 1 -- 172.21.15.22:0/403302865 wait complete. 2024-02-16T19:33:33.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:33 smithi117 ceph-mon[67870]: pgmap v10: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-16T19:33:33.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:33 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:33:33.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.945+0000 7f27e3258700 1 -- 172.21.15.22:0/3111517212 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27dc0ff090 msgr2=0x7f27dc106080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:33.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.945+0000 7f27e3258700 1 --2- 172.21.15.22:0/3111517212 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27dc0ff090 0x7f27dc106080 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f27d0009910 tx=0x7f27d0009c20 comp rx=0 tx=0).stop 2024-02-16T19:33:33.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.945+0000 7f27e3258700 1 -- 172.21.15.22:0/3111517212 shutdown_connections 2024-02-16T19:33:33.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.945+0000 7f27e3258700 1 --2- 172.21.15.22:0/3111517212 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27dc0ff090 0x7f27dc106080 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:33.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.945+0000 7f27e3258700 1 --2- 172.21.15.22:0/3111517212 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27dc0fe6f0 0x7f27dc0feac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:33.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.945+0000 7f27e3258700 1 -- 172.21.15.22:0/3111517212 >> 172.21.15.22:0/3111517212 conn(0x7f27dc075440 msgr2=0x7f27dc075840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:33:33.946 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:33 smithi022 ceph-mon[88798]: pgmap v10: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-16T19:33:33.946 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:33 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:33:33.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.945+0000 7f27e3258700 1 -- 172.21.15.22:0/3111517212 shutdown_connections 2024-02-16T19:33:33.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.945+0000 7f27e3258700 1 -- 172.21.15.22:0/3111517212 wait complete. 2024-02-16T19:33:33.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.946+0000 7f27e3258700 1 Processor -- start 2024-02-16T19:33:33.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.946+0000 7f27e3258700 1 -- start start 2024-02-16T19:33:33.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.946+0000 7f27e3258700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27dc0fe6f0 0x7f27dc195f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:33.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.946+0000 7f27e3258700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27dc0ff090 0x7f27dc196440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:33.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.946+0000 7f27e3258700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27dc196b20 con 0x7f27dc0ff090 2024-02-16T19:33:33.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.946+0000 7f27e3258700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27dc19a8b0 con 0x7f27dc0fe6f0 2024-02-16T19:33:33.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.946+0000 7f27dbfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27dc0ff090 0x7f27dc196440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:33.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.946+0000 7f27dbfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27dc0ff090 0x7f27dc196440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47778/0 (socket says 172.21.15.22:47778) 2024-02-16T19:33:33.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.946+0000 7f27dbfff700 1 -- 172.21.15.22:0/2575722504 learned_addr learned my addr 172.21.15.22:0/2575722504 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:33:33.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.946+0000 7f27e0ff4700 1 --2- 172.21.15.22:0/2575722504 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27dc0fe6f0 0x7f27dc195f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:33.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.946+0000 7f27e0ff4700 1 -- 172.21.15.22:0/2575722504 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27dc0fe6f0 msgr2=0x7f27dc195f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-02-16T19:33:33.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.946+0000 7f27e0ff4700 1 -- 172.21.15.22:0/2575722504 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27dc0fe6f0 msgr2=0x7f27dc195f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-02-16T19:33:33.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.946+0000 7f27e0ff4700 1 --2- 172.21.15.22:0/2575722504 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27dc0fe6f0 0x7f27dc195f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-02-16T19:33:33.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.946+0000 7f27e0ff4700 1 --2- 172.21.15.22:0/2575722504 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27dc0fe6f0 0x7f27dc195f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-16T19:33:33.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.947+0000 7f27dbfff700 1 -- 172.21.15.22:0/2575722504 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27dc0fe6f0 msgr2=0x7f27dc195f00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-16T19:33:33.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.947+0000 7f27dbfff700 1 --2- 172.21.15.22:0/2575722504 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27dc0fe6f0 0x7f27dc195f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:33.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.947+0000 7f27dbfff700 1 -- 172.21.15.22:0/2575722504 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27d00095f0 con 0x7f27dc0ff090 2024-02-16T19:33:33.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.947+0000 7f27dbfff700 1 --2- 172.21.15.22:0/2575722504 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27dc0ff090 0x7f27dc196440 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f27d00092e0 tx=0x7f27d000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:33:33.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.947+0000 7f27d9ffb700 1 -- 172.21.15.22:0/2575722504 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f27d001d070 con 0x7f27dc0ff090 2024-02-16T19:33:33.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.947+0000 7f27e3258700 1 -- 172.21.15.22:0/2575722504 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27dc19ab30 con 0x7f27dc0ff090 2024-02-16T19:33:33.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.947+0000 7f27d9ffb700 1 -- 172.21.15.22:0/2575722504 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f27d0005dc0 con 0x7f27dc0ff090 2024-02-16T19:33:33.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.947+0000 7f27d9ffb700 1 -- 172.21.15.22:0/2575722504 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f27d0017400 con 0x7f27dc0ff090 2024-02-16T19:33:33.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.947+0000 7f27e3258700 1 -- 172.21.15.22:0/2575722504 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27dc19b020 con 0x7f27dc0ff090 2024-02-16T19:33:33.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.948+0000 7f27e3258700 1 -- 172.21.15.22:0/2575722504 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27dc1037e0 con 0x7f27dc0ff090 2024-02-16T19:33:33.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.949+0000 7f27d9ffb700 1 -- 172.21.15.22:0/2575722504 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 39) v1 ==== 91595+0+0 (secure 0 0 0) 0x7f27d0017640 con 0x7f27dc0ff090 2024-02-16T19:33:33.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.949+0000 7f27d9ffb700 1 --2- 172.21.15.22:0/2575722504 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f27c406d9e0 0x7f27c406fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:33.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.949+0000 7f27d9ffb700 1 -- 172.21.15.22:0/2575722504 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f27d008eb80 con 0x7f27dc0ff090 2024-02-16T19:33:33.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.949+0000 7f27e0ff4700 1 --2- 172.21.15.22:0/2575722504 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f27c406d9e0 0x7f27c406fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:33.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.950+0000 7f27e0ff4700 1 --2- 172.21.15.22:0/2575722504 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f27c406d9e0 0x7f27c406fe90 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f27cc005950 tx=0x7f27cc0058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:33:33.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:33.954+0000 7f27d9ffb700 1 -- 172.21.15.22:0/2575722504 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f27d0058240 con 0x7f27dc0ff090 2024-02-16T19:33:34.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.169+0000 7f27e3258700 1 -- 172.21.15.22:0/2575722504 --> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f27dc1972d0 con 0x7f27c406d9e0 2024-02-16T19:33:34.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.180+0000 7f27d9ffb700 1 -- 172.21.15.22:0/2575722504 <== mgr.34104 v2:172.21.15.22:6800/505266753 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f27dc1972d0 con 0x7f27c406d9e0 2024-02-16T19:33:34.180 INFO:teuthology.orchestra.run.smithi022.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T19:33:34.180 INFO:teuthology.orchestra.run.smithi022.stdout:alertmanager.smithi022 smithi022 *:9093,9094 running (116s) 13s ago 12m 15.9M - 0.25.0 c8568f914cd2 c89bc76be374 2024-02-16T19:33:34.180 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-exporter.smithi022 smithi022 running (13m) 13s ago 13m 8002k - 18.2.0 dc2bc1663786 3f8bef4349ba 2024-02-16T19:33:34.180 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-exporter.smithi117 smithi117 running (11m) 15s ago 11m 7680k - 18.2.0 dc2bc1663786 59b65a7e309f 2024-02-16T19:33:34.180 INFO:teuthology.orchestra.run.smithi022.stdout:crash.smithi022 smithi022 running (13m) 13s ago 13m 6669k - 18.2.0 dc2bc1663786 efef476aa486 2024-02-16T19:33:34.180 INFO:teuthology.orchestra.run.smithi022.stdout:crash.smithi117 smithi117 running (11m) 15s ago 11m 6665k - 18.2.0 dc2bc1663786 5466453eb149 2024-02-16T19:33:34.180 INFO:teuthology.orchestra.run.smithi022.stdout:grafana.smithi022 smithi022 *:3000 running (90s) 13s ago 12m 76.4M - 9.4.12 a0ea5e3c46d9 fd27d11e9729 2024-02-16T19:33:34.180 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi022.btppwm smithi022 running (7m) 13s ago 7m 158M - 18.2.0 dc2bc1663786 22971fb3a907 2024-02-16T19:33:34.181 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi022.ocukvs smithi022 running (7m) 13s ago 7m 15.7M - 18.2.0 dc2bc1663786 81ee7bd5fd39 2024-02-16T19:33:34.181 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi117.ibmppn smithi117 running (7m) 15s ago 7m 17.0M - 18.2.0 dc2bc1663786 8c606cbeeb86 2024-02-16T19:33:34.181 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi117.orzors smithi117 running (7m) 15s ago 7m 16.1M - 18.2.0 dc2bc1663786 7d2bc4df3feb 2024-02-16T19:33:34.181 INFO:teuthology.orchestra.run.smithi022.stdout:mgr.smithi022.gmdcch smithi022 *:8443,9283,8765 running (3m) 13s ago 14m 472M - 19.0.0-1450-gf78a58c0 f081f847070a 7cb74cc9deea 2024-02-16T19:33:34.181 INFO:teuthology.orchestra.run.smithi022.stdout:mgr.smithi117.cohbtv smithi117 *:8443,9283,8765 running (2m) 15s ago 11m 448M - 19.0.0-1450-gf78a58c0 f081f847070a c825b1152189 2024-02-16T19:33:34.181 INFO:teuthology.orchestra.run.smithi022.stdout:mon.smithi022 smithi022 running (27s) 13s ago 14m 29.3M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 9c84755ae67d 2024-02-16T19:33:34.181 INFO:teuthology.orchestra.run.smithi022.stdout:mon.smithi117 smithi117 starting - - - 2048M 2024-02-16T19:33:34.181 INFO:teuthology.orchestra.run.smithi022.stdout:node-exporter.smithi022 smithi022 *:9100 running (2m) 13s ago 12m 16.9M - 1.5.0 0da6a335fe13 1f5b58292d27 2024-02-16T19:33:34.181 INFO:teuthology.orchestra.run.smithi022.stdout:node-exporter.smithi117 smithi117 *:9100 running (2m) 15s ago 11m 15.3M - 1.5.0 0da6a335fe13 bb5e8491d5a6 2024-02-16T19:33:34.181 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0 smithi022 running (10m) 13s ago 10m 210M 1217M 18.2.0 dc2bc1663786 0027f86f16ce 2024-02-16T19:33:34.181 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1 smithi022 running (10m) 13s ago 10m 227M 1217M 18.2.0 dc2bc1663786 87300382b8a7 2024-02-16T19:33:34.181 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2 smithi022 running (9m) 13s ago 9m 187M 1217M 18.2.0 dc2bc1663786 be8a631b4f93 2024-02-16T19:33:34.181 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3 smithi117 running (9m) 15s ago 9m 244M 2240M 18.2.0 dc2bc1663786 9c0ce9a981cf 2024-02-16T19:33:34.181 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4 smithi117 running (9m) 15s ago 9m 209M 2240M 18.2.0 dc2bc1663786 1779c3496f4d 2024-02-16T19:33:34.182 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5 smithi117 running (8m) 15s ago 8m 197M 2240M 18.2.0 dc2bc1663786 fefaabca0638 2024-02-16T19:33:34.182 INFO:teuthology.orchestra.run.smithi022.stdout:prometheus.smithi022 smithi022 *:9095 running (2m) 13s ago 12m 56.3M - 2.43.0 a07b618ecd1d c3b8689d0c97 2024-02-16T19:33:34.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.182+0000 7f27e3258700 1 -- 172.21.15.22:0/2575722504 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f27c406d9e0 msgr2=0x7f27c406fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:34.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.182+0000 7f27e3258700 1 --2- 172.21.15.22:0/2575722504 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f27c406d9e0 0x7f27c406fe90 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f27cc005950 tx=0x7f27cc0058e0 comp rx=0 tx=0).stop 2024-02-16T19:33:34.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.182+0000 7f27e3258700 1 -- 172.21.15.22:0/2575722504 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27dc0ff090 msgr2=0x7f27dc196440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:34.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.182+0000 7f27e3258700 1 --2- 172.21.15.22:0/2575722504 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27dc0ff090 0x7f27dc196440 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f27d00092e0 tx=0x7f27d000be20 comp rx=0 tx=0).stop 2024-02-16T19:33:34.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.182+0000 7f27e3258700 1 -- 172.21.15.22:0/2575722504 shutdown_connections 2024-02-16T19:33:34.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.182+0000 7f27e3258700 1 --2- 172.21.15.22:0/2575722504 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f27c406d9e0 0x7f27c406fe90 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:34.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.182+0000 7f27e3258700 1 --2- 172.21.15.22:0/2575722504 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27dc0fe6f0 0x7f27dc195f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:34.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.182+0000 7f27e3258700 1 --2- 172.21.15.22:0/2575722504 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27dc0ff090 0x7f27dc196440 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:34.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.182+0000 7f27e3258700 1 -- 172.21.15.22:0/2575722504 >> 172.21.15.22:0/2575722504 conn(0x7f27dc075440 msgr2=0x7f27dc109930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:33:34.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.182+0000 7f27e3258700 1 -- 172.21.15.22:0/2575722504 shutdown_connections 2024-02-16T19:33:34.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.182+0000 7f27e3258700 1 -- 172.21.15.22:0/2575722504 wait complete. 2024-02-16T19:33:34.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.286+0000 7f94b879f700 1 -- 172.21.15.22:0/1450129684 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94b01012f0 msgr2=0x7f94b01016c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:34.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.286+0000 7f94b879f700 1 --2- 172.21.15.22:0/1450129684 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94b01012f0 0x7f94b01016c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f94a0009910 tx=0x7f94a0009c20 comp rx=0 tx=0).stop 2024-02-16T19:33:34.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.286+0000 7f94b879f700 1 -- 172.21.15.22:0/1450129684 shutdown_connections 2024-02-16T19:33:34.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.286+0000 7f94b879f700 1 --2- 172.21.15.22:0/1450129684 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94b0101c90 0x7f94b010a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:34.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.286+0000 7f94b879f700 1 --2- 172.21.15.22:0/1450129684 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94b01012f0 0x7f94b01016c0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:34.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.286+0000 7f94b879f700 1 -- 172.21.15.22:0/1450129684 >> 172.21.15.22:0/1450129684 conn(0x7f94b00fac80 msgr2=0x7f94b00fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:33:34.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.286+0000 7f94b879f700 1 -- 172.21.15.22:0/1450129684 shutdown_connections 2024-02-16T19:33:34.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.286+0000 7f94b879f700 1 -- 172.21.15.22:0/1450129684 wait complete. 2024-02-16T19:33:34.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.287+0000 7f94b879f700 1 Processor -- start 2024-02-16T19:33:34.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.287+0000 7f94b879f700 1 -- start start 2024-02-16T19:33:34.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.287+0000 7f94b879f700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94b01012f0 0x7f94b0195eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:34.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.287+0000 7f94b879f700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94b0101c90 0x7f94b01963f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:34.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.287+0000 7f94b879f700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b0196ad0 con 0x7f94b01012f0 2024-02-16T19:33:34.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.287+0000 7f94b879f700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b019a810 con 0x7f94b0101c90 2024-02-16T19:33:34.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.287+0000 7f94b653b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94b01012f0 0x7f94b0195eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:34.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.287+0000 7f94b653b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94b01012f0 0x7f94b0195eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47796/0 (socket says 172.21.15.22:47796) 2024-02-16T19:33:34.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.287+0000 7f94b653b700 1 -- 172.21.15.22:0/1127272608 learned_addr learned my addr 172.21.15.22:0/1127272608 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:33:34.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.287+0000 7f94b5d3a700 1 --2- 172.21.15.22:0/1127272608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94b0101c90 0x7f94b01963f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:34.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.287+0000 7f94b5d3a700 1 -- 172.21.15.22:0/1127272608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94b0101c90 msgr2=0x7f94b01963f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-02-16T19:33:34.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.287+0000 7f94b5d3a700 1 -- 172.21.15.22:0/1127272608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94b0101c90 msgr2=0x7f94b01963f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-02-16T19:33:34.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.287+0000 7f94b5d3a700 1 --2- 172.21.15.22:0/1127272608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94b0101c90 0x7f94b01963f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-02-16T19:33:34.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.287+0000 7f94b653b700 1 -- 172.21.15.22:0/1127272608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94b0101c90 msgr2=0x7f94b01963f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:34.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.287+0000 7f94b5d3a700 1 --2- 172.21.15.22:0/1127272608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94b0101c90 0x7f94b01963f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-16T19:33:34.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.287+0000 7f94b653b700 1 --2- 172.21.15.22:0/1127272608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94b0101c90 0x7f94b01963f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:34.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.287+0000 7f94b653b700 1 -- 172.21.15.22:0/1127272608 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94a00095f0 con 0x7f94b01012f0 2024-02-16T19:33:34.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.288+0000 7f94b653b700 1 --2- 172.21.15.22:0/1127272608 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94b01012f0 0x7f94b0195eb0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f94a0000c00 tx=0x7f94a00056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:33:34.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.288+0000 7f94a77fe700 1 -- 172.21.15.22:0/1127272608 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94a001d070 con 0x7f94b01012f0 2024-02-16T19:33:34.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.288+0000 7f94a77fe700 1 -- 172.21.15.22:0/1127272608 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f94a000f460 con 0x7f94b01012f0 2024-02-16T19:33:34.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.288+0000 7f94b879f700 1 -- 172.21.15.22:0/1127272608 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94b019aa90 con 0x7f94b01012f0 2024-02-16T19:33:34.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.288+0000 7f94a77fe700 1 -- 172.21.15.22:0/1127272608 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94a0021640 con 0x7f94b01012f0 2024-02-16T19:33:34.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.288+0000 7f94b879f700 1 -- 172.21.15.22:0/1127272608 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94b019afe0 con 0x7f94b01012f0 2024-02-16T19:33:34.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.289+0000 7f94b879f700 1 -- 172.21.15.22:0/1127272608 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94b01079d0 con 0x7f94b01012f0 2024-02-16T19:33:34.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.289+0000 7f94a77fe700 1 -- 172.21.15.22:0/1127272608 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 39) v1 ==== 91595+0+0 (secure 0 0 0) 0x7f94a000fa60 con 0x7f94b01012f0 2024-02-16T19:33:34.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.290+0000 7f94a77fe700 1 --2- 172.21.15.22:0/1127272608 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f949c06d9e0 0x7f949c06fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:34.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.290+0000 7f94a77fe700 1 -- 172.21.15.22:0/1127272608 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f94a008e8a0 con 0x7f94b01012f0 2024-02-16T19:33:34.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.290+0000 7f94b5d3a700 1 --2- 172.21.15.22:0/1127272608 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f949c06d9e0 0x7f949c06fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:34.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.290+0000 7f94b5d3a700 1 --2- 172.21.15.22:0/1127272608 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f949c06d9e0 0x7f949c06fe90 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f94ac005950 tx=0x7f94ac0058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:33:34.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.294+0000 7f94a77fe700 1 -- 172.21.15.22:0/1127272608 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f94a0057f60 con 0x7f94b01012f0 2024-02-16T19:33:34.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.586+0000 7f94b879f700 1 -- 172.21.15.22:0/1127272608 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f94b004f2a0 con 0x7f94b01012f0 2024-02-16T19:33:34.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.586+0000 7f94a77fe700 1 -- 172.21.15.22:0/1127272608 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+688 (secure 0 0 0) 0x7f94a005b620 con 0x7f94b01012f0 2024-02-16T19:33:34.587 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-02-16T19:33:34.587 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": { 2024-02-16T19:33:34.587 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T19:33:34.587 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:33:34.587 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": { 2024-02-16T19:33:34.587 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T19:33:34.587 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:33:34.587 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": { 2024-02-16T19:33:34.587 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-02-16T19:33:34.587 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:33:34.587 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": { 2024-02-16T19:33:34.588 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-02-16T19:33:34.588 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:33:34.588 INFO:teuthology.orchestra.run.smithi022.stdout: "overall": { 2024-02-16T19:33:34.588 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 10, 2024-02-16T19:33:34.588 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 4 2024-02-16T19:33:34.588 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-02-16T19:33:34.588 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-02-16T19:33:34.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.588+0000 7f94b879f700 1 -- 172.21.15.22:0/1127272608 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f949c06d9e0 msgr2=0x7f949c06fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:34.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.588+0000 7f94b879f700 1 --2- 172.21.15.22:0/1127272608 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f949c06d9e0 0x7f949c06fe90 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f94ac005950 tx=0x7f94ac0058e0 comp rx=0 tx=0).stop 2024-02-16T19:33:34.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.588+0000 7f94b879f700 1 -- 172.21.15.22:0/1127272608 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94b01012f0 msgr2=0x7f94b0195eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:34.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.588+0000 7f94b879f700 1 --2- 172.21.15.22:0/1127272608 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94b01012f0 0x7f94b0195eb0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f94a0000c00 tx=0x7f94a00056a0 comp rx=0 tx=0).stop 2024-02-16T19:33:34.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.588+0000 7f94b879f700 1 -- 172.21.15.22:0/1127272608 shutdown_connections 2024-02-16T19:33:34.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.588+0000 7f94b879f700 1 --2- 172.21.15.22:0/1127272608 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f949c06d9e0 0x7f949c06fe90 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:34.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.588+0000 7f94b879f700 1 --2- 172.21.15.22:0/1127272608 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94b01012f0 0x7f94b0195eb0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:34.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.588+0000 7f94b879f700 1 --2- 172.21.15.22:0/1127272608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94b0101c90 0x7f94b01963f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:34.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.588+0000 7f94b879f700 1 -- 172.21.15.22:0/1127272608 >> 172.21.15.22:0/1127272608 conn(0x7f94b00fac80 msgr2=0x7f94b0104b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:33:34.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.589+0000 7f94b879f700 1 -- 172.21.15.22:0/1127272608 shutdown_connections 2024-02-16T19:33:34.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.589+0000 7f94b879f700 1 -- 172.21.15.22:0/1127272608 wait complete. 2024-02-16T19:33:34.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.694+0000 7f04a321e700 1 -- 172.21.15.22:0/1828336834 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f049c1038b0 msgr2=0x7f049c103c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:34.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.694+0000 7f04a321e700 1 --2- 172.21.15.22:0/1828336834 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f049c1038b0 0x7f049c103c80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0494009910 tx=0x7f0494009c20 comp rx=0 tx=0).stop 2024-02-16T19:33:34.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.694+0000 7f04a321e700 1 -- 172.21.15.22:0/1828336834 shutdown_connections 2024-02-16T19:33:34.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.694+0000 7f04a321e700 1 --2- 172.21.15.22:0/1828336834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f049c0ff350 0x7f049c0ff7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:34.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.694+0000 7f04a321e700 1 --2- 172.21.15.22:0/1828336834 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f049c1038b0 0x7f049c103c80 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:34.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.694+0000 7f04a321e700 1 -- 172.21.15.22:0/1828336834 >> 172.21.15.22:0/1828336834 conn(0x7f049c0748b0 msgr2=0x7f049c074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:33:34.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.694+0000 7f04a321e700 1 -- 172.21.15.22:0/1828336834 shutdown_connections 2024-02-16T19:33:34.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.694+0000 7f04a321e700 1 -- 172.21.15.22:0/1828336834 wait complete. 2024-02-16T19:33:34.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.694+0000 7f04a321e700 1 Processor -- start 2024-02-16T19:33:34.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.694+0000 7f04a321e700 1 -- start start 2024-02-16T19:33:34.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.695+0000 7f04a321e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f049c0ff350 0x7f049c194d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:34.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.695+0000 7f04a321e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f049c1038b0 0x7f049c195260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:34.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.695+0000 7f04a321e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f049c1958b0 con 0x7f049c1038b0 2024-02-16T19:33:34.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.695+0000 7f04a321e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f049c1959f0 con 0x7f049c0ff350 2024-02-16T19:33:34.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.695+0000 7f049bfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f049c1038b0 0x7f049c195260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:34.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.695+0000 7f04a0fba700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f049c0ff350 0x7f049c194d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:34.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.695+0000 7f049bfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f049c1038b0 0x7f049c195260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52110/0 (socket says 172.21.15.22:52110) 2024-02-16T19:33:34.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.695+0000 7f049bfff700 1 -- 172.21.15.22:0/2199895870 learned_addr learned my addr 172.21.15.22:0/2199895870 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:33:34.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.695+0000 7f04a0fba700 1 -- 172.21.15.22:0/2199895870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f049c0ff350 msgr2=0x7f049c194d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-02-16T19:33:34.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.695+0000 7f04a0fba700 1 -- 172.21.15.22:0/2199895870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f049c0ff350 msgr2=0x7f049c194d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-02-16T19:33:34.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.695+0000 7f04a0fba700 1 --2- 172.21.15.22:0/2199895870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f049c0ff350 0x7f049c194d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-02-16T19:33:34.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.695+0000 7f04a0fba700 1 --2- 172.21.15.22:0/2199895870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f049c0ff350 0x7f049c194d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-16T19:33:34.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.695+0000 7f049bfff700 1 -- 172.21.15.22:0/2199895870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f049c0ff350 msgr2=0x7f049c194d20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-16T19:33:34.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.695+0000 7f049bfff700 1 --2- 172.21.15.22:0/2199895870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f049c0ff350 0x7f049c194d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:34.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.695+0000 7f049bfff700 1 -- 172.21.15.22:0/2199895870 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04940095f0 con 0x7f049c1038b0 2024-02-16T19:33:34.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.695+0000 7f049bfff700 1 --2- 172.21.15.22:0/2199895870 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f049c1038b0 0x7f049c195260 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f048c00cfd0 tx=0x7f048c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:33:34.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.695+0000 7f0499ffb700 1 -- 172.21.15.22:0/2199895870 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f048c00ad70 con 0x7f049c1038b0 2024-02-16T19:33:34.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.695+0000 7f0499ffb700 1 -- 172.21.15.22:0/2199895870 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f048c0128e0 con 0x7f049c1038b0 2024-02-16T19:33:34.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.695+0000 7f04a321e700 1 -- 172.21.15.22:0/2199895870 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f049c199840 con 0x7f049c1038b0 2024-02-16T19:33:34.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.695+0000 7f0499ffb700 1 -- 172.21.15.22:0/2199895870 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f048c011880 con 0x7f049c1038b0 2024-02-16T19:33:34.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.696+0000 7f04a321e700 1 -- 172.21.15.22:0/2199895870 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f049c199d90 con 0x7f049c1038b0 2024-02-16T19:33:34.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.696+0000 7f04a321e700 1 -- 172.21.15.22:0/2199895870 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f049c107530 con 0x7f049c1038b0 2024-02-16T19:33:34.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.697+0000 7f0499ffb700 1 -- 172.21.15.22:0/2199895870 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 39) v1 ==== 91595+0+0 (secure 0 0 0) 0x7f048c012420 con 0x7f049c1038b0 2024-02-16T19:33:34.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.697+0000 7f0499ffb700 1 --2- 172.21.15.22:0/2199895870 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f048806d9e0 0x7f048806fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:34.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.697+0000 7f0499ffb700 1 -- 172.21.15.22:0/2199895870 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f048c08d690 con 0x7f049c1038b0 2024-02-16T19:33:34.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.697+0000 7f04a0fba700 1 --2- 172.21.15.22:0/2199895870 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f048806d9e0 0x7f048806fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:34.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.698+0000 7f04a0fba700 1 --2- 172.21.15.22:0/2199895870 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f048806d9e0 0x7f048806fe90 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0494005f50 tx=0x7f049401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:33:34.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.702+0000 7f0499ffb700 1 -- 172.21.15.22:0/2199895870 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f048c056e60 con 0x7f049c1038b0 2024-02-16T19:33:34.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:34 smithi117 ceph-mon[67870]: from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:33:34.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:34 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mon metadata", "id": "smithi022"}]: dispatch 2024-02-16T19:33:34.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:34 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mon metadata", "id": "smithi117"}]: dispatch 2024-02-16T19:33:34.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:34 smithi117 ceph-mon[67870]: mon.smithi117 calling monitor election 2024-02-16T19:33:34.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:34 smithi117 ceph-mon[67870]: mon.smithi022 calling monitor election 2024-02-16T19:33:34.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:34 smithi117 ceph-mon[67870]: mon.smithi022 is new leader, mons smithi022,smithi117 in quorum (ranks 0,1) 2024-02-16T19:33:34.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:34 smithi117 ceph-mon[67870]: monmap e3: 2 mons at {smithi022=[v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0],smithi117=[v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-16T19:33:34.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:34 smithi117 ceph-mon[67870]: fsmap cephfs:1 {0=cephfs.smithi022.btppwm=up:active} 3 up:standby 2024-02-16T19:33:34.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:34 smithi117 ceph-mon[67870]: osdmap e50: 6 total, 6 up, 6 in 2024-02-16T19:33:34.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:34 smithi117 ceph-mon[67870]: mgrmap e39: smithi022.gmdcch(active, since 17s), standbys: smithi117.cohbtv 2024-02-16T19:33:34.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:34 smithi117 ceph-mon[67870]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-02-16T19:33:34.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:34 smithi117 ceph-mon[67870]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-02-16T19:33:34.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:34 smithi117 ceph-mon[67870]: fs cephfs has deprecated feature inline_data enabled. 2024-02-16T19:33:34.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:34 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:34.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:34 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:34.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:34 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:33:34.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:34 smithi117 ceph-mon[67870]: from='client.? 172.21.15.22:0/1127272608' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:34.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.956+0000 7f04a321e700 1 -- 172.21.15.22:0/2199895870 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f049c02cc30 con 0x7f049c1038b0 2024-02-16T19:33:34.956 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:34 smithi022 ceph-mon[88798]: from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:33:34.956 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:34 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mon metadata", "id": "smithi022"}]: dispatch 2024-02-16T19:33:34.957 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:34 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mon metadata", "id": "smithi117"}]: dispatch 2024-02-16T19:33:34.957 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:34 smithi022 ceph-mon[88798]: mon.smithi117 calling monitor election 2024-02-16T19:33:34.957 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:34 smithi022 ceph-mon[88798]: mon.smithi022 calling monitor election 2024-02-16T19:33:34.957 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:34 smithi022 ceph-mon[88798]: mon.smithi022 is new leader, mons smithi022,smithi117 in quorum (ranks 0,1) 2024-02-16T19:33:34.957 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:34 smithi022 ceph-mon[88798]: monmap e3: 2 mons at {smithi022=[v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0],smithi117=[v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-16T19:33:34.957 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:34 smithi022 ceph-mon[88798]: fsmap cephfs:1 {0=cephfs.smithi022.btppwm=up:active} 3 up:standby 2024-02-16T19:33:34.957 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:34 smithi022 ceph-mon[88798]: osdmap e50: 6 total, 6 up, 6 in 2024-02-16T19:33:34.957 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:34 smithi022 ceph-mon[88798]: mgrmap e39: smithi022.gmdcch(active, since 17s), standbys: smithi117.cohbtv 2024-02-16T19:33:34.957 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:34 smithi022 ceph-mon[88798]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-02-16T19:33:34.957 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:34 smithi022 ceph-mon[88798]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-02-16T19:33:34.957 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:34 smithi022 ceph-mon[88798]: fs cephfs has deprecated feature inline_data enabled. 2024-02-16T19:33:34.957 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:34 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:34.957 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:34 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:34.957 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:34 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:33:34.957 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:34 smithi022 ceph-mon[88798]: from='client.? 172.21.15.22:0/1127272608' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:34.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.956+0000 7f0499ffb700 1 -- 172.21.15.22:0/2199895870 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1845 (secure 0 0 0) 0x7f048c05a520 con 0x7f049c1038b0 2024-02-16T19:33:34.958 INFO:teuthology.orchestra.run.smithi022.stdout:e15 2024-02-16T19:33:34.958 INFO:teuthology.orchestra.run.smithi022.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T19:33:34.958 INFO:teuthology.orchestra.run.smithi022.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:33:34.958 INFO:teuthology.orchestra.run.smithi022.stdout:legacy client fscid: 1 2024-02-16T19:33:34.958 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:33:34.958 INFO:teuthology.orchestra.run.smithi022.stdout:Filesystem 'cephfs' (1) 2024-02-16T19:33:34.958 INFO:teuthology.orchestra.run.smithi022.stdout:fs_name cephfs 2024-02-16T19:33:34.958 INFO:teuthology.orchestra.run.smithi022.stdout:epoch 15 2024-02-16T19:33:34.958 INFO:teuthology.orchestra.run.smithi022.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T19:33:34.958 INFO:teuthology.orchestra.run.smithi022.stdout:created 2024-02-16T19:26:17.873644+0000 2024-02-16T19:33:34.959 INFO:teuthology.orchestra.run.smithi022.stdout:modified 2024-02-16T19:26:35.943353+0000 2024-02-16T19:33:34.959 INFO:teuthology.orchestra.run.smithi022.stdout:tableserver 0 2024-02-16T19:33:34.959 INFO:teuthology.orchestra.run.smithi022.stdout:root 0 2024-02-16T19:33:34.959 INFO:teuthology.orchestra.run.smithi022.stdout:session_timeout 60 2024-02-16T19:33:34.959 INFO:teuthology.orchestra.run.smithi022.stdout:session_autoclose 300 2024-02-16T19:33:34.959 INFO:teuthology.orchestra.run.smithi022.stdout:max_file_size 1099511627776 2024-02-16T19:33:34.959 INFO:teuthology.orchestra.run.smithi022.stdout:max_xattr_size 65536 2024-02-16T19:33:34.959 INFO:teuthology.orchestra.run.smithi022.stdout:required_client_features {} 2024-02-16T19:33:34.959 INFO:teuthology.orchestra.run.smithi022.stdout:last_failure 0 2024-02-16T19:33:34.959 INFO:teuthology.orchestra.run.smithi022.stdout:last_failure_osd_epoch 47 2024-02-16T19:33:34.959 INFO:teuthology.orchestra.run.smithi022.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:33:34.959 INFO:teuthology.orchestra.run.smithi022.stdout:max_mds 1 2024-02-16T19:33:34.959 INFO:teuthology.orchestra.run.smithi022.stdout:in 0 2024-02-16T19:33:34.959 INFO:teuthology.orchestra.run.smithi022.stdout:up {0=14490} 2024-02-16T19:33:34.960 INFO:teuthology.orchestra.run.smithi022.stdout:failed 2024-02-16T19:33:34.960 INFO:teuthology.orchestra.run.smithi022.stdout:damaged 2024-02-16T19:33:34.960 INFO:teuthology.orchestra.run.smithi022.stdout:stopped 2024-02-16T19:33:34.960 INFO:teuthology.orchestra.run.smithi022.stdout:data_pools [3] 2024-02-16T19:33:34.960 INFO:teuthology.orchestra.run.smithi022.stdout:metadata_pool 2 2024-02-16T19:33:34.960 INFO:teuthology.orchestra.run.smithi022.stdout:inline_data enabled 2024-02-16T19:33:34.960 INFO:teuthology.orchestra.run.smithi022.stdout:balancer 2024-02-16T19:33:34.960 INFO:teuthology.orchestra.run.smithi022.stdout:bal_rank_mask -1 2024-02-16T19:33:34.960 INFO:teuthology.orchestra.run.smithi022.stdout:standby_count_wanted 1 2024-02-16T19:33:34.960 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.cephfs.smithi022.btppwm{0:14490} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.22:6828/3597685751,v1:172.21.15.22:6829/3597685751] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:33:34.960 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:33:34.960 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:33:34.960 INFO:teuthology.orchestra.run.smithi022.stdout:Standby daemons: 2024-02-16T19:33:34.960 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:33:34.961 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.cephfs.smithi022.ocukvs{-1:14500} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.22:6826/242865843,v1:172.21.15.22:6827/242865843] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:33:34.961 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.cephfs.smithi117.ibmppn{-1:24259} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.117:6824/565408102,v1:172.21.15.117:6825/565408102] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:33:34.961 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.cephfs.smithi117.orzors{-1:24265} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.117:6826/3122342028,v1:172.21.15.117:6827/3122342028] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:33:34.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.959+0000 7f04a321e700 1 -- 172.21.15.22:0/2199895870 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f048806d9e0 msgr2=0x7f048806fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:34.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.959+0000 7f04a321e700 1 --2- 172.21.15.22:0/2199895870 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f048806d9e0 0x7f048806fe90 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0494005f50 tx=0x7f049401a040 comp rx=0 tx=0).stop 2024-02-16T19:33:34.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.959+0000 7f04a321e700 1 -- 172.21.15.22:0/2199895870 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f049c1038b0 msgr2=0x7f049c195260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:34.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.959+0000 7f04a321e700 1 --2- 172.21.15.22:0/2199895870 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f049c1038b0 0x7f049c195260 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f048c00cfd0 tx=0x7f048c00a340 comp rx=0 tx=0).stop 2024-02-16T19:33:34.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.959+0000 7f04a321e700 1 -- 172.21.15.22:0/2199895870 shutdown_connections 2024-02-16T19:33:34.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.959+0000 7f04a321e700 1 --2- 172.21.15.22:0/2199895870 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f048806d9e0 0x7f048806fe90 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:34.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.959+0000 7f04a321e700 1 --2- 172.21.15.22:0/2199895870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f049c0ff350 0x7f049c194d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:34.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.959+0000 7f04a321e700 1 --2- 172.21.15.22:0/2199895870 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f049c1038b0 0x7f049c195260 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:34.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.959+0000 7f04a321e700 1 -- 172.21.15.22:0/2199895870 >> 172.21.15.22:0/2199895870 conn(0x7f049c0748b0 msgr2=0x7f049c104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:33:34.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.959+0000 7f04a321e700 1 -- 172.21.15.22:0/2199895870 shutdown_connections 2024-02-16T19:33:34.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:34.959+0000 7f04a321e700 1 -- 172.21.15.22:0/2199895870 wait complete. 2024-02-16T19:33:34.962 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 15 2024-02-16T19:33:35.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.071+0000 7fc424aae700 1 -- 172.21.15.22:0/650855800 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4200ffc60 msgr2=0x7fc42010a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:35.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.071+0000 7fc424aae700 1 --2- 172.21.15.22:0/650855800 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4200ffc60 0x7fc42010a6d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc408009910 tx=0x7fc408009c20 comp rx=0 tx=0).stop 2024-02-16T19:33:35.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.071+0000 7fc424aae700 1 -- 172.21.15.22:0/650855800 shutdown_connections 2024-02-16T19:33:35.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.071+0000 7fc424aae700 1 --2- 172.21.15.22:0/650855800 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4200ffc60 0x7fc42010a6d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:35.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.071+0000 7fc424aae700 1 --2- 172.21.15.22:0/650855800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4200ff350 0x7fc4200ff720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:35.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.071+0000 7fc424aae700 1 -- 172.21.15.22:0/650855800 >> 172.21.15.22:0/650855800 conn(0x7fc4200748b0 msgr2=0x7fc420074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:33:35.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.071+0000 7fc424aae700 1 -- 172.21.15.22:0/650855800 shutdown_connections 2024-02-16T19:33:35.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.071+0000 7fc424aae700 1 -- 172.21.15.22:0/650855800 wait complete. 2024-02-16T19:33:35.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.072+0000 7fc424aae700 1 Processor -- start 2024-02-16T19:33:35.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.072+0000 7fc424aae700 1 -- start start 2024-02-16T19:33:35.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.072+0000 7fc424aae700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4200ff350 0x7fc420195ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:35.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.072+0000 7fc424aae700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4200ffc60 0x7fc420196400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:35.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.072+0000 7fc424aae700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc420196ae0 con 0x7fc4200ffc60 2024-02-16T19:33:35.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.072+0000 7fc424aae700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc42019a870 con 0x7fc4200ff350 2024-02-16T19:33:35.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.072+0000 7fc415d9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4200ffc60 0x7fc420196400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:35.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.073+0000 7fc415d9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4200ffc60 0x7fc420196400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52118/0 (socket says 172.21.15.22:52118) 2024-02-16T19:33:35.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.073+0000 7fc415d9b700 1 -- 172.21.15.22:0/130880556 learned_addr learned my addr 172.21.15.22:0/130880556 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:33:35.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.073+0000 7fc41e59c700 1 --2- 172.21.15.22:0/130880556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4200ff350 0x7fc420195ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:35.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.073+0000 7fc41e59c700 1 -- 172.21.15.22:0/130880556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4200ff350 msgr2=0x7fc420195ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-02-16T19:33:35.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.073+0000 7fc41e59c700 1 -- 172.21.15.22:0/130880556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4200ff350 msgr2=0x7fc420195ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-02-16T19:33:35.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.073+0000 7fc41e59c700 1 --2- 172.21.15.22:0/130880556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4200ff350 0x7fc420195ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-02-16T19:33:35.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.073+0000 7fc41e59c700 1 --2- 172.21.15.22:0/130880556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4200ff350 0x7fc420195ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-16T19:33:35.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.073+0000 7fc415d9b700 1 -- 172.21.15.22:0/130880556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4200ff350 msgr2=0x7fc420195ec0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-16T19:33:35.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.073+0000 7fc415d9b700 1 --2- 172.21.15.22:0/130880556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4200ff350 0x7fc420195ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:35.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.073+0000 7fc415d9b700 1 -- 172.21.15.22:0/130880556 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4080095f0 con 0x7fc4200ffc60 2024-02-16T19:33:35.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.073+0000 7fc415d9b700 1 --2- 172.21.15.22:0/130880556 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4200ffc60 0x7fc420196400 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc4080092e0 tx=0x7fc4080056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:33:35.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.073+0000 7fc417fff700 1 -- 172.21.15.22:0/130880556 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc40801d070 con 0x7fc4200ffc60 2024-02-16T19:33:35.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.073+0000 7fc424aae700 1 -- 172.21.15.22:0/130880556 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc42019ab50 con 0x7fc4200ffc60 2024-02-16T19:33:35.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.073+0000 7fc417fff700 1 -- 172.21.15.22:0/130880556 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc40800f460 con 0x7fc4200ffc60 2024-02-16T19:33:35.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.073+0000 7fc417fff700 1 -- 172.21.15.22:0/130880556 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc408021640 con 0x7fc4200ffc60 2024-02-16T19:33:35.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.073+0000 7fc424aae700 1 -- 172.21.15.22:0/130880556 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc42019b0a0 con 0x7fc4200ffc60 2024-02-16T19:33:35.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.074+0000 7fc424aae700 1 -- 172.21.15.22:0/130880556 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc420107dd0 con 0x7fc4200ffc60 2024-02-16T19:33:35.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.075+0000 7fc417fff700 1 -- 172.21.15.22:0/130880556 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 39) v1 ==== 91595+0+0 (secure 0 0 0) 0x7fc40800f5d0 con 0x7fc4200ffc60 2024-02-16T19:33:35.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.075+0000 7fc417fff700 1 --2- 172.21.15.22:0/130880556 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fc40c071de0 0x7fc40c074290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:35.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.075+0000 7fc417fff700 1 -- 172.21.15.22:0/130880556 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fc40808e9c0 con 0x7fc4200ffc60 2024-02-16T19:33:35.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.075+0000 7fc41e59c700 1 --2- 172.21.15.22:0/130880556 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fc40c071de0 0x7fc40c074290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:35.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.076+0000 7fc41e59c700 1 --2- 172.21.15.22:0/130880556 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fc40c071de0 0x7fc40c074290 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc410005950 tx=0x7fc4100058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:33:35.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.080+0000 7fc417fff700 1 -- 172.21.15.22:0/130880556 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc408058000 con 0x7fc4200ffc60 2024-02-16T19:33:35.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.309+0000 7fc424aae700 1 -- 172.21.15.22:0/130880556 --> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc420197290 con 0x7fc40c071de0 2024-02-16T19:33:35.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.311+0000 7fc417fff700 1 -- 172.21.15.22:0/130880556 <== mgr.34104 v2:172.21.15.22:6800/505266753 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7fc420197290 con 0x7fc40c071de0 2024-02-16T19:33:35.311 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-02-16T19:33:35.311 INFO:teuthology.orchestra.run.smithi022.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T19:33:35.311 INFO:teuthology.orchestra.run.smithi022.stdout: "in_progress": true, 2024-02-16T19:33:35.311 INFO:teuthology.orchestra.run.smithi022.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T19:33:35.311 INFO:teuthology.orchestra.run.smithi022.stdout: "services_complete": [ 2024-02-16T19:33:35.311 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr" 2024-02-16T19:33:35.312 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-02-16T19:33:35.312 INFO:teuthology.orchestra.run.smithi022.stdout: "progress": "3/23 daemons upgraded", 2024-02-16T19:33:35.312 INFO:teuthology.orchestra.run.smithi022.stdout: "message": "Currently upgrading mon daemons", 2024-02-16T19:33:35.312 INFO:teuthology.orchestra.run.smithi022.stdout: "is_paused": false 2024-02-16T19:33:35.312 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-02-16T19:33:35.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.313+0000 7fc424aae700 1 -- 172.21.15.22:0/130880556 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fc40c071de0 msgr2=0x7fc40c074290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:35.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.313+0000 7fc424aae700 1 --2- 172.21.15.22:0/130880556 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fc40c071de0 0x7fc40c074290 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc410005950 tx=0x7fc4100058e0 comp rx=0 tx=0).stop 2024-02-16T19:33:35.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.313+0000 7fc424aae700 1 -- 172.21.15.22:0/130880556 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4200ffc60 msgr2=0x7fc420196400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:35.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.313+0000 7fc424aae700 1 --2- 172.21.15.22:0/130880556 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4200ffc60 0x7fc420196400 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc4080092e0 tx=0x7fc4080056a0 comp rx=0 tx=0).stop 2024-02-16T19:33:35.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.313+0000 7fc424aae700 1 -- 172.21.15.22:0/130880556 shutdown_connections 2024-02-16T19:33:35.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.313+0000 7fc424aae700 1 --2- 172.21.15.22:0/130880556 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fc40c071de0 0x7fc40c074290 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:35.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.313+0000 7fc424aae700 1 --2- 172.21.15.22:0/130880556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4200ff350 0x7fc420195ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:35.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.313+0000 7fc424aae700 1 --2- 172.21.15.22:0/130880556 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4200ffc60 0x7fc420196400 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:35.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.313+0000 7fc424aae700 1 -- 172.21.15.22:0/130880556 >> 172.21.15.22:0/130880556 conn(0x7fc4200748b0 msgr2=0x7fc420104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:33:35.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.313+0000 7fc424aae700 1 -- 172.21.15.22:0/130880556 shutdown_connections 2024-02-16T19:33:35.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.313+0000 7fc424aae700 1 -- 172.21.15.22:0/130880556 wait complete. 2024-02-16T19:33:35.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.419+0000 7ff2b3081700 1 -- 172.21.15.22:0/2113885747 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2ac100460 msgr2=0x7ff2ac100830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:35.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.419+0000 7ff2b3081700 1 --2- 172.21.15.22:0/2113885747 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2ac100460 0x7ff2ac100830 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff2a0009940 tx=0x7ff2a0009c50 comp rx=0 tx=0).stop 2024-02-16T19:33:35.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.419+0000 7ff2b3081700 1 -- 172.21.15.22:0/2113885747 shutdown_connections 2024-02-16T19:33:35.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.419+0000 7ff2b3081700 1 --2- 172.21.15.22:0/2113885747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2ac100d70 0x7ff2ac1094d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:35.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.419+0000 7ff2b3081700 1 --2- 172.21.15.22:0/2113885747 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2ac100460 0x7ff2ac100830 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:35.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.419+0000 7ff2b3081700 1 -- 172.21.15.22:0/2113885747 >> 172.21.15.22:0/2113885747 conn(0x7ff2ac0748b0 msgr2=0x7ff2ac074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:33:35.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.419+0000 7ff2b3081700 1 -- 172.21.15.22:0/2113885747 shutdown_connections 2024-02-16T19:33:35.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.419+0000 7ff2b3081700 1 -- 172.21.15.22:0/2113885747 wait complete. 2024-02-16T19:33:35.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.419+0000 7ff2b3081700 1 Processor -- start 2024-02-16T19:33:35.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.419+0000 7ff2b3081700 1 -- start start 2024-02-16T19:33:35.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.420+0000 7ff2b3081700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2ac100460 0x7ff2ac19aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:35.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.420+0000 7ff2b3081700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2ac100d70 0x7ff2ac19af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:35.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.420+0000 7ff2b3081700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2ac19b600 con 0x7ff2ac100d70 2024-02-16T19:33:35.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.420+0000 7ff2b3081700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2ac194ab0 con 0x7ff2ac100460 2024-02-16T19:33:35.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.420+0000 7ff2abfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2ac100d70 0x7ff2ac19af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:35.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.420+0000 7ff2abfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2ac100d70 0x7ff2ac19af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52140/0 (socket says 172.21.15.22:52140) 2024-02-16T19:33:35.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.420+0000 7ff2abfff700 1 -- 172.21.15.22:0/4090481807 learned_addr learned my addr 172.21.15.22:0/4090481807 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:33:35.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.420+0000 7ff2b0e1d700 1 --2- 172.21.15.22:0/4090481807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2ac100460 0x7ff2ac19aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:35.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.420+0000 7ff2b0e1d700 1 -- 172.21.15.22:0/4090481807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2ac100460 msgr2=0x7ff2ac19aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-02-16T19:33:35.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.420+0000 7ff2b0e1d700 1 -- 172.21.15.22:0/4090481807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2ac100460 msgr2=0x7ff2ac19aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-02-16T19:33:35.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.420+0000 7ff2b0e1d700 1 --2- 172.21.15.22:0/4090481807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2ac100460 0x7ff2ac19aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-02-16T19:33:35.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.420+0000 7ff2b0e1d700 1 --2- 172.21.15.22:0/4090481807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2ac100460 0x7ff2ac19aa30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-16T19:33:35.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.420+0000 7ff2abfff700 1 -- 172.21.15.22:0/4090481807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2ac100460 msgr2=0x7ff2ac19aa30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-16T19:33:35.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.420+0000 7ff2abfff700 1 --2- 172.21.15.22:0/4090481807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2ac100460 0x7ff2ac19aa30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:35.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.420+0000 7ff2abfff700 1 -- 172.21.15.22:0/4090481807 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2a00095f0 con 0x7ff2ac100d70 2024-02-16T19:33:35.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.420+0000 7ff2abfff700 1 --2- 172.21.15.22:0/4090481807 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2ac100d70 0x7ff2ac19af70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff29800d8d0 tx=0x7ff29800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:33:35.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.420+0000 7ff2a9ffb700 1 -- 172.21.15.22:0/4090481807 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff298009940 con 0x7ff2ac100d70 2024-02-16T19:33:35.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.420+0000 7ff2a9ffb700 1 -- 172.21.15.22:0/4090481807 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff298010460 con 0x7ff2ac100d70 2024-02-16T19:33:35.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.420+0000 7ff2b3081700 1 -- 172.21.15.22:0/4090481807 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2ac194d90 con 0x7ff2ac100d70 2024-02-16T19:33:35.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.421+0000 7ff2a9ffb700 1 -- 172.21.15.22:0/4090481807 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff29800f5f0 con 0x7ff2ac100d70 2024-02-16T19:33:35.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.421+0000 7ff2b3081700 1 -- 172.21.15.22:0/4090481807 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2ac1952e0 con 0x7ff2ac100d70 2024-02-16T19:33:35.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.422+0000 7ff2b3081700 1 -- 172.21.15.22:0/4090481807 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2ac04f2a0 con 0x7ff2ac100d70 2024-02-16T19:33:35.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.422+0000 7ff2a9ffb700 1 -- 172.21.15.22:0/4090481807 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 39) v1 ==== 91595+0+0 (secure 0 0 0) 0x7ff2980105d0 con 0x7ff2ac100d70 2024-02-16T19:33:35.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.422+0000 7ff2a9ffb700 1 --2- 172.21.15.22:0/4090481807 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7ff29c06d9e0 0x7ff29c06fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:35.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.423+0000 7ff2a9ffb700 1 -- 172.21.15.22:0/4090481807 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7ff29808d710 con 0x7ff2ac100d70 2024-02-16T19:33:35.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.423+0000 7ff2b0e1d700 1 --2- 172.21.15.22:0/4090481807 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7ff29c06d9e0 0x7ff29c06fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:35.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.423+0000 7ff2b0e1d700 1 --2- 172.21.15.22:0/4090481807 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7ff29c06d9e0 0x7ff29c06fe90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff2a0005b40 tx=0x7ff2a0005a90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:33:35.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.427+0000 7ff2a9ffb700 1 -- 172.21.15.22:0/4090481807 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff29805a880 con 0x7ff2ac100d70 2024-02-16T19:33:35.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.720+0000 7ff2b3081700 1 -- 172.21.15.22:0/4090481807 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ff2ac195590 con 0x7ff2ac100d70 2024-02-16T19:33:35.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.721+0000 7ff2a9ffb700 1 -- 172.21.15.22:0/4090481807 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7ff29805a410 con 0x7ff2ac100d70 2024-02-16T19:33:35.721 INFO:teuthology.orchestra.run.smithi022.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-02-16T19:33:35.721 INFO:teuthology.orchestra.run.smithi022.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-02-16T19:33:35.721 INFO:teuthology.orchestra.run.smithi022.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-02-16T19:33:35.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.723+0000 7ff2b3081700 1 -- 172.21.15.22:0/4090481807 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7ff29c06d9e0 msgr2=0x7ff29c06fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:35.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.723+0000 7ff2b3081700 1 --2- 172.21.15.22:0/4090481807 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7ff29c06d9e0 0x7ff29c06fe90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff2a0005b40 tx=0x7ff2a0005a90 comp rx=0 tx=0).stop 2024-02-16T19:33:35.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.723+0000 7ff2b3081700 1 -- 172.21.15.22:0/4090481807 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2ac100d70 msgr2=0x7ff2ac19af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:35.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.723+0000 7ff2b3081700 1 --2- 172.21.15.22:0/4090481807 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2ac100d70 0x7ff2ac19af70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff29800d8d0 tx=0x7ff29800dc90 comp rx=0 tx=0).stop 2024-02-16T19:33:35.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.723+0000 7ff2b3081700 1 -- 172.21.15.22:0/4090481807 shutdown_connections 2024-02-16T19:33:35.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.723+0000 7ff2b3081700 1 --2- 172.21.15.22:0/4090481807 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7ff29c06d9e0 0x7ff29c06fe90 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:35.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.723+0000 7ff2b3081700 1 --2- 172.21.15.22:0/4090481807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2ac100460 0x7ff2ac19aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:35.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.723+0000 7ff2b3081700 1 --2- 172.21.15.22:0/4090481807 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2ac100d70 0x7ff2ac19af70 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:35.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.723+0000 7ff2b3081700 1 -- 172.21.15.22:0/4090481807 >> 172.21.15.22:0/4090481807 conn(0x7ff2ac0748b0 msgr2=0x7ff2ac103d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:33:35.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.723+0000 7ff2b3081700 1 -- 172.21.15.22:0/4090481807 shutdown_connections 2024-02-16T19:33:35.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:33:35.723+0000 7ff2b3081700 1 -- 172.21.15.22:0/4090481807 wait complete. 2024-02-16T19:33:35.985 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:35 smithi117 ceph-mon[67870]: pgmap v12: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-16T19:33:35.985 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:35 smithi117 ceph-mon[67870]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:33:35.985 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:35 smithi117 ceph-mon[67870]: from='client.34140 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:33:35.985 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:35 smithi117 ceph-mon[67870]: from='client.? 172.21.15.22:0/2199895870' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:33:35.986 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:35 smithi117 ceph-mon[67870]: from='client.? 172.21.15.22:0/4090481807' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T19:33:36.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:35 smithi022 ceph-mon[88798]: pgmap v12: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-16T19:33:36.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:35 smithi022 ceph-mon[88798]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:33:36.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:35 smithi022 ceph-mon[88798]: from='client.34140 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:33:36.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:35 smithi022 ceph-mon[88798]: from='client.? 172.21.15.22:0/2199895870' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:33:36.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:35 smithi022 ceph-mon[88798]: from='client.? 172.21.15.22:0/4090481807' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T19:33:37.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:36 smithi117 ceph-mon[67870]: from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:33:37.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:36 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:37.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:36 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:37.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:36 smithi022 ceph-mon[88798]: from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:33:37.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:36 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:37.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:36 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:38.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:37 smithi117 ceph-mon[67870]: pgmap v13: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:33:38.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:37 smithi022 ceph-mon[88798]: pgmap v13: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:33:39.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:38 smithi117 ceph-mon[67870]: pgmap v14: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:33:39.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:38 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:39.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:38 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:39.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:38 smithi022 ceph-mon[88798]: pgmap v14: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:33:39.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:38 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:39.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:38 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:40.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:40 smithi117 ceph-mon[67870]: pgmap v15: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:33:40.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:40 smithi022 ceph-mon[88798]: pgmap v15: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:33:42.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:42 smithi117 ceph-mon[67870]: Detected new or changed devices on smithi117 2024-02-16T19:33:42.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:42 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:42.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:42 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:42.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:42 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:33:42.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:42 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:33:42.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:42 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:42.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:42 smithi117 ceph-mon[67870]: Reconfiguring mon.smithi022 (monmap changed)... 2024-02-16T19:33:42.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:42 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-16T19:33:42.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:42 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-16T19:33:42.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:42 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:33:42.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:42 smithi117 ceph-mon[67870]: Reconfiguring daemon mon.smithi022 on smithi022 2024-02-16T19:33:42.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:42 smithi022 ceph-mon[88798]: Detected new or changed devices on smithi117 2024-02-16T19:33:42.716 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:42 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:42.716 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:42 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:42.716 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:42 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:33:42.716 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:42 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:33:42.716 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:42 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:42.716 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:42 smithi022 ceph-mon[88798]: Reconfiguring mon.smithi022 (monmap changed)... 2024-02-16T19:33:42.716 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:42 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-16T19:33:42.716 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:42 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-16T19:33:42.716 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:42 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:33:42.716 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:42 smithi022 ceph-mon[88798]: Reconfiguring daemon mon.smithi022 on smithi022 2024-02-16T19:33:43.575 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:43 smithi117 ceph-mon[67870]: pgmap v16: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:33:43.575 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:43 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:43.575 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:43 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:43.575 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:43 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi022.gmdcch", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-16T19:33:43.575 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:43 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-16T19:33:43.575 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:43 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:33:43.674 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:43 smithi022 ceph-mon[88798]: pgmap v16: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:33:43.674 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:43 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:43.674 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:43 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:43.674 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:43 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi022.gmdcch", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-16T19:33:43.674 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:43 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-16T19:33:43.674 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:43 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:33:44.563 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:44 smithi022 ceph-mon[88798]: Reconfiguring mgr.smithi022.gmdcch (monmap changed)... 2024-02-16T19:33:44.563 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:44 smithi022 ceph-mon[88798]: Reconfiguring daemon mgr.smithi022.gmdcch on smithi022 2024-02-16T19:33:44.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:44 smithi117 ceph-mon[67870]: Reconfiguring mgr.smithi022.gmdcch (monmap changed)... 2024-02-16T19:33:44.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:44 smithi117 ceph-mon[67870]: Reconfiguring daemon mgr.smithi022.gmdcch on smithi022 2024-02-16T19:33:45.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:45 smithi117 ceph-mon[67870]: pgmap v17: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:33:45.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:45 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:45.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:45 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:45.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:45 smithi117 ceph-mon[67870]: Reconfiguring ceph-exporter.smithi022 (monmap changed)... 2024-02-16T19:33:45.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:45 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi022", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-02-16T19:33:45.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:45 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi022", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-02-16T19:33:45.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:45 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi022", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-02-16T19:33:45.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:45 smithi117 ceph-mon[67870]: Unable to update caps for client.ceph-exporter.smithi022 2024-02-16T19:33:45.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:45 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi022"}]: dispatch 2024-02-16T19:33:45.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:45 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:33:45.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:45 smithi117 ceph-mon[67870]: Reconfiguring daemon ceph-exporter.smithi022 on smithi022 2024-02-16T19:33:45.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:45 smithi022 ceph-mon[88798]: pgmap v17: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:33:45.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:45 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:45.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:45 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:45.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:45 smithi022 ceph-mon[88798]: Reconfiguring ceph-exporter.smithi022 (monmap changed)... 2024-02-16T19:33:45.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:45 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi022", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-02-16T19:33:45.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:45 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi022", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-02-16T19:33:45.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:45 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi022", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-02-16T19:33:45.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:45 smithi022 ceph-mon[88798]: Unable to update caps for client.ceph-exporter.smithi022 2024-02-16T19:33:45.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:45 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi022"}]: dispatch 2024-02-16T19:33:45.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:45 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:33:45.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:45 smithi022 ceph-mon[88798]: Reconfiguring daemon ceph-exporter.smithi022 on smithi022 2024-02-16T19:33:46.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:46 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:33:46.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:46 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:46.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:46 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:46.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:46 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi022", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-16T19:33:46.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:46 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:33:46.948 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:46 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:33:46.948 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:46 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:46.948 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:46 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:46.948 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:46 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi022", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-16T19:33:46.949 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:46 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:33:47.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:47 smithi117 ceph-mon[67870]: pgmap v18: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:33:47.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:47 smithi117 ceph-mon[67870]: Reconfiguring crash.smithi022 (monmap changed)... 2024-02-16T19:33:47.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:47 smithi117 ceph-mon[67870]: Reconfiguring daemon crash.smithi022 on smithi022 2024-02-16T19:33:47.964 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:47 smithi022 ceph-mon[88798]: pgmap v18: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:33:47.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:47 smithi022 ceph-mon[88798]: Reconfiguring crash.smithi022 (monmap changed)... 2024-02-16T19:33:47.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:47 smithi022 ceph-mon[88798]: Reconfiguring daemon crash.smithi022 on smithi022 2024-02-16T19:33:48.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:48 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:48.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:48 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:48.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:48 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-16T19:33:48.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:48 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:33:49.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:48 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:49.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:48 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:49.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:48 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-16T19:33:49.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:48 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:33:50.002 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:49 smithi022 ceph-mon[88798]: Reconfiguring osd.0 (monmap changed)... 2024-02-16T19:33:50.002 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:49 smithi022 ceph-mon[88798]: Reconfiguring daemon osd.0 on smithi022 2024-02-16T19:33:50.003 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:49 smithi022 ceph-mon[88798]: pgmap v19: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:33:50.003 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:49 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:50.003 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:49 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:50.003 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:49 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-16T19:33:50.003 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:49 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:33:50.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:49 smithi117 ceph-mon[67870]: Reconfiguring osd.0 (monmap changed)... 2024-02-16T19:33:50.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:49 smithi117 ceph-mon[67870]: Reconfiguring daemon osd.0 on smithi022 2024-02-16T19:33:50.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:49 smithi117 ceph-mon[67870]: pgmap v19: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:33:50.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:49 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:50.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:49 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:50.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:49 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-16T19:33:50.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:49 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:33:50.949 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:50 smithi022 ceph-mon[88798]: Reconfiguring osd.1 (monmap changed)... 2024-02-16T19:33:50.949 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:50 smithi022 ceph-mon[88798]: Reconfiguring daemon osd.1 on smithi022 2024-02-16T19:33:51.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:50 smithi117 ceph-mon[67870]: Reconfiguring osd.1 (monmap changed)... 2024-02-16T19:33:51.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:50 smithi117 ceph-mon[67870]: Reconfiguring daemon osd.1 on smithi022 2024-02-16T19:33:51.922 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:51 smithi022 ceph-mon[88798]: pgmap v20: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:33:51.922 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:51 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:51.922 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:51 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:51.922 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:51 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-16T19:33:51.922 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:51 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:33:52.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:51 smithi117 ceph-mon[67870]: pgmap v20: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:33:52.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:51 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:52.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:51 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:52.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:51 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-16T19:33:52.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:51 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:33:52.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:52 smithi022 ceph-mon[88798]: Reconfiguring osd.2 (monmap changed)... 2024-02-16T19:33:52.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:52 smithi022 ceph-mon[88798]: Reconfiguring daemon osd.2 on smithi022 2024-02-16T19:33:53.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:52 smithi117 ceph-mon[67870]: Reconfiguring osd.2 (monmap changed)... 2024-02-16T19:33:53.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:52 smithi117 ceph-mon[67870]: Reconfiguring daemon osd.2 on smithi022 2024-02-16T19:33:54.063 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:53 smithi022 ceph-mon[88798]: pgmap v21: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:33:54.064 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:53 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:54.064 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:53 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:54.064 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:53 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi022.ocukvs", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T19:33:54.064 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:53 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:33:54.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:53 smithi117 ceph-mon[67870]: pgmap v21: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:33:54.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:53 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:54.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:53 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:54.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:53 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi022.ocukvs", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T19:33:54.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:53 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:33:55.012 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:54 smithi022 ceph-mon[88798]: Reconfiguring mds.cephfs.smithi022.ocukvs (monmap changed)... 2024-02-16T19:33:55.012 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:54 smithi022 ceph-mon[88798]: Reconfiguring daemon mds.cephfs.smithi022.ocukvs on smithi022 2024-02-16T19:33:55.012 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:54 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:55.012 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:54 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:55.013 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:54 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi022.btppwm", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T19:33:55.013 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:54 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:33:55.038 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:54 smithi117 ceph-mon[67870]: Reconfiguring mds.cephfs.smithi022.ocukvs (monmap changed)... 2024-02-16T19:33:55.038 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:54 smithi117 ceph-mon[67870]: Reconfiguring daemon mds.cephfs.smithi022.ocukvs on smithi022 2024-02-16T19:33:55.038 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:54 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:55.039 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:54 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:55.039 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:54 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi022.btppwm", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T19:33:55.039 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:54 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:33:56.017 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:55 smithi022 ceph-mon[88798]: pgmap v22: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:33:56.017 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:55 smithi022 ceph-mon[88798]: Reconfiguring mds.cephfs.smithi022.btppwm (monmap changed)... 2024-02-16T19:33:56.017 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:55 smithi022 ceph-mon[88798]: Reconfiguring daemon mds.cephfs.smithi022.btppwm on smithi022 2024-02-16T19:33:56.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:55 smithi117 ceph-mon[67870]: pgmap v22: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:33:56.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:55 smithi117 ceph-mon[67870]: Reconfiguring mds.cephfs.smithi022.btppwm (monmap changed)... 2024-02-16T19:33:56.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:55 smithi117 ceph-mon[67870]: Reconfiguring daemon mds.cephfs.smithi022.btppwm on smithi022 2024-02-16T19:33:57.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:57 smithi117 ceph-mon[67870]: pgmap v23: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:33:57.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:57 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:57.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:57 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:57.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:57 smithi117 ceph-mon[67870]: Reconfiguring ceph-exporter.smithi117 (monmap changed)... 2024-02-16T19:33:57.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:57 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi117", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-02-16T19:33:57.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:57 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi117", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-02-16T19:33:57.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:57 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi117", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-02-16T19:33:57.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:57 smithi117 ceph-mon[67870]: Unable to update caps for client.ceph-exporter.smithi117 2024-02-16T19:33:57.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:57 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi117"}]: dispatch 2024-02-16T19:33:57.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:57 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:33:57.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:57 smithi117 ceph-mon[67870]: Reconfiguring daemon ceph-exporter.smithi117 on smithi117 2024-02-16T19:33:57.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:57 smithi022 ceph-mon[88798]: pgmap v23: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:33:57.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:57 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:57.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:57 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:57.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:57 smithi022 ceph-mon[88798]: Reconfiguring ceph-exporter.smithi117 (monmap changed)... 2024-02-16T19:33:57.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:57 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi117", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-02-16T19:33:57.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:57 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi117", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-02-16T19:33:57.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:57 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi117", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-02-16T19:33:57.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:57 smithi022 ceph-mon[88798]: Unable to update caps for client.ceph-exporter.smithi117 2024-02-16T19:33:57.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:57 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi117"}]: dispatch 2024-02-16T19:33:57.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:57 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:33:57.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:57 smithi022 ceph-mon[88798]: Reconfiguring daemon ceph-exporter.smithi117 on smithi117 2024-02-16T19:33:58.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:58 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:58.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:58 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:58.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:58 smithi117 ceph-mon[67870]: Reconfiguring crash.smithi117 (monmap changed)... 2024-02-16T19:33:58.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:58 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi117", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-16T19:33:58.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:58 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:33:58.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:58 smithi117 ceph-mon[67870]: Reconfiguring daemon crash.smithi117 on smithi117 2024-02-16T19:33:58.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:58 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:58.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:58 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:58.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:58 smithi022 ceph-mon[88798]: Reconfiguring crash.smithi117 (monmap changed)... 2024-02-16T19:33:58.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:58 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi117", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-16T19:33:58.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:58 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:33:58.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:58 smithi022 ceph-mon[88798]: Reconfiguring daemon crash.smithi117 on smithi117 2024-02-16T19:33:59.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:59 smithi022 ceph-mon[88798]: pgmap v24: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:33:59.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:59 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:59.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:59 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:59.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:59 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi117.cohbtv", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-16T19:33:59.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:59 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-16T19:33:59.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:33:59 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:33:59.987 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:59 smithi117 ceph-mon[67870]: pgmap v24: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:33:59.987 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:59 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:59.987 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:59 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:33:59.987 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:59 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi117.cohbtv", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-16T19:33:59.987 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:59 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-16T19:33:59.987 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:33:59 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:34:00.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:00 smithi022 ceph-mon[88798]: Reconfiguring mgr.smithi117.cohbtv (monmap changed)... 2024-02-16T19:34:00.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:00 smithi022 ceph-mon[88798]: Reconfiguring daemon mgr.smithi117.cohbtv on smithi117 2024-02-16T19:34:01.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:00 smithi117 ceph-mon[67870]: Reconfiguring mgr.smithi117.cohbtv (monmap changed)... 2024-02-16T19:34:01.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:00 smithi117 ceph-mon[67870]: Reconfiguring daemon mgr.smithi117.cohbtv on smithi117 2024-02-16T19:34:01.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:01 smithi117 ceph-mon[67870]: pgmap v25: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:34:01.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:01 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:01.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:01 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:01.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:01 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-16T19:34:01.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:01 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-16T19:34:01.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:01 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:34:01.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:01 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:34:02.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:01 smithi022 ceph-mon[88798]: pgmap v25: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:34:02.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:01 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:02.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:01 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:02.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:01 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-16T19:34:02.216 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:01 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-16T19:34:02.216 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:01 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:34:02.216 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:01 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:34:03.043 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:02 smithi117 ceph-mon[67870]: Reconfiguring mon.smithi117 (monmap changed)... 2024-02-16T19:34:03.043 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:02 smithi117 ceph-mon[67870]: Reconfiguring daemon mon.smithi117 on smithi117 2024-02-16T19:34:03.043 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:02 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:03.043 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:02 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:03.043 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:02 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-16T19:34:03.043 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:02 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:34:03.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:02 smithi022 ceph-mon[88798]: Reconfiguring mon.smithi117 (monmap changed)... 2024-02-16T19:34:03.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:02 smithi022 ceph-mon[88798]: Reconfiguring daemon mon.smithi117 on smithi117 2024-02-16T19:34:03.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:02 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:03.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:02 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:03.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:02 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-16T19:34:03.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:02 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:34:04.050 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:03 smithi117 ceph-mon[67870]: pgmap v26: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:34:04.050 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:03 smithi117 ceph-mon[67870]: Reconfiguring osd.3 (monmap changed)... 2024-02-16T19:34:04.050 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:03 smithi117 ceph-mon[67870]: Reconfiguring daemon osd.3 on smithi117 2024-02-16T19:34:04.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:03 smithi022 ceph-mon[88798]: pgmap v26: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:34:04.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:03 smithi022 ceph-mon[88798]: Reconfiguring osd.3 (monmap changed)... 2024-02-16T19:34:04.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:03 smithi022 ceph-mon[88798]: Reconfiguring daemon osd.3 on smithi117 2024-02-16T19:34:05.293 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:05 smithi117 ceph-mon[67870]: pgmap v27: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:34:05.293 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:05 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:05.293 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:05 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:05.293 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:05 smithi117 ceph-mon[67870]: Reconfiguring osd.4 (monmap changed)... 2024-02-16T19:34:05.293 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:05 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-16T19:34:05.293 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:05 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:34:05.293 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:05 smithi117 ceph-mon[67870]: Reconfiguring daemon osd.4 on smithi117 2024-02-16T19:34:05.347 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:05 smithi022 ceph-mon[88798]: pgmap v27: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:34:05.348 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:05 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:05.348 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:05 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:05.348 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:05 smithi022 ceph-mon[88798]: Reconfiguring osd.4 (monmap changed)... 2024-02-16T19:34:05.348 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:05 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-16T19:34:05.348 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:05 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:34:05.348 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:05 smithi022 ceph-mon[88798]: Reconfiguring daemon osd.4 on smithi117 2024-02-16T19:34:05.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:05.842+0000 7f3f187c3700 1 -- 172.21.15.22:0/3573806661 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f10100e80 msgr2=0x7f3f10104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:05.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:05.842+0000 7f3f187c3700 1 --2- 172.21.15.22:0/3573806661 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f10100e80 0x7f3f10104e50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3f00009910 tx=0x7f3f00009c20 comp rx=0 tx=0).stop 2024-02-16T19:34:05.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:05.842+0000 7f3f187c3700 1 -- 172.21.15.22:0/3573806661 shutdown_connections 2024-02-16T19:34:05.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:05.842+0000 7f3f187c3700 1 --2- 172.21.15.22:0/3573806661 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f10100e80 0x7f3f10104e50 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:05.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:05.842+0000 7f3f187c3700 1 --2- 172.21.15.22:0/3573806661 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f101004e0 0x7f3f101008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:05.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:05.842+0000 7f3f187c3700 1 -- 172.21.15.22:0/3573806661 >> 172.21.15.22:0/3573806661 conn(0x7f3f100fbda0 msgr2=0x7f3f100fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:34:05.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:05.842+0000 7f3f187c3700 1 -- 172.21.15.22:0/3573806661 shutdown_connections 2024-02-16T19:34:05.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:05.842+0000 7f3f187c3700 1 -- 172.21.15.22:0/3573806661 wait complete. 2024-02-16T19:34:05.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:05.842+0000 7f3f187c3700 1 Processor -- start 2024-02-16T19:34:05.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:05.842+0000 7f3f187c3700 1 -- start start 2024-02-16T19:34:05.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:05.843+0000 7f3f187c3700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f101004e0 0x7f3f10195f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:34:05.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:05.843+0000 7f3f187c3700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f10100e80 0x7f3f10196450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:34:05.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:05.843+0000 7f3f187c3700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f10196b30 con 0x7f3f10100e80 2024-02-16T19:34:05.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:05.843+0000 7f3f187c3700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1019a8c0 con 0x7f3f101004e0 2024-02-16T19:34:05.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:05.843+0000 7f3f15d5e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f10100e80 0x7f3f10196450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:34:05.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:05.843+0000 7f3f15d5e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f10100e80 0x7f3f10196450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49918/0 (socket says 172.21.15.22:49918) 2024-02-16T19:34:05.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:05.843+0000 7f3f1655f700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f101004e0 0x7f3f10195f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:34:05.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:05.843+0000 7f3f15d5e700 1 -- 172.21.15.22:0/506811188 learned_addr learned my addr 172.21.15.22:0/506811188 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:34:05.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:05.843+0000 7f3f15d5e700 1 -- 172.21.15.22:0/506811188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f101004e0 msgr2=0x7f3f10195f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:05.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:05.843+0000 7f3f15d5e700 1 --2- 172.21.15.22:0/506811188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f101004e0 0x7f3f10195f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:05.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:05.843+0000 7f3f15d5e700 1 -- 172.21.15.22:0/506811188 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f000095f0 con 0x7f3f10100e80 2024-02-16T19:34:05.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:05.843+0000 7f3f15d5e700 1 --2- 172.21.15.22:0/506811188 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f10100e80 0x7f3f10196450 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3f000059d0 tx=0x7f3f0000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:34:05.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:05.843+0000 7f3f077fe700 1 -- 172.21.15.22:0/506811188 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f0001d070 con 0x7f3f10100e80 2024-02-16T19:34:05.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:05.844+0000 7f3f187c3700 1 -- 172.21.15.22:0/506811188 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f1019ab40 con 0x7f3f10100e80 2024-02-16T19:34:05.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:05.844+0000 7f3f077fe700 1 -- 172.21.15.22:0/506811188 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3f0000b610 con 0x7f3f10100e80 2024-02-16T19:34:05.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:05.844+0000 7f3f077fe700 1 -- 172.21.15.22:0/506811188 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f00022470 con 0x7f3f10100e80 2024-02-16T19:34:05.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:05.844+0000 7f3f187c3700 1 -- 172.21.15.22:0/506811188 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f1019b030 con 0x7f3f10100e80 2024-02-16T19:34:05.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:05.844+0000 7f3f187c3700 1 -- 172.21.15.22:0/506811188 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f10108850 con 0x7f3f10100e80 2024-02-16T19:34:05.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:05.845+0000 7f3f077fe700 1 -- 172.21.15.22:0/506811188 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 39) v1 ==== 91595+0+0 (secure 0 0 0) 0x7f3f00003710 con 0x7f3f10100e80 2024-02-16T19:34:05.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:05.846+0000 7f3f077fe700 1 --2- 172.21.15.22:0/506811188 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f3efc06d9e0 0x7f3efc06fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:34:05.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:05.846+0000 7f3f1655f700 1 --2- 172.21.15.22:0/506811188 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f3efc06d9e0 0x7f3efc06fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:34:05.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:05.846+0000 7f3f077fe700 1 -- 172.21.15.22:0/506811188 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f3f0008ed80 con 0x7f3f10100e80 2024-02-16T19:34:05.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:05.846+0000 7f3f1655f700 1 --2- 172.21.15.22:0/506811188 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f3efc06d9e0 0x7f3efc06fe90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c009c60 tx=0x7f3f0c009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:34:05.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:05.850+0000 7f3f077fe700 1 -- 172.21.15.22:0/506811188 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3f00058440 con 0x7f3f10100e80 2024-02-16T19:34:06.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.078+0000 7f3f187c3700 1 -- 172.21.15.22:0/506811188 --> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3f1019b310 con 0x7f3efc06d9e0 2024-02-16T19:34:06.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.079+0000 7f3f077fe700 1 -- 172.21.15.22:0/506811188 <== mgr.34104 v2:172.21.15.22:6800/505266753 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f3f1019b310 con 0x7f3efc06d9e0 2024-02-16T19:34:06.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.081+0000 7f3f187c3700 1 -- 172.21.15.22:0/506811188 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f3efc06d9e0 msgr2=0x7f3efc06fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:06.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.081+0000 7f3f187c3700 1 --2- 172.21.15.22:0/506811188 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f3efc06d9e0 0x7f3efc06fe90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c009c60 tx=0x7f3f0c009260 comp rx=0 tx=0).stop 2024-02-16T19:34:06.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.081+0000 7f3f187c3700 1 -- 172.21.15.22:0/506811188 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f10100e80 msgr2=0x7f3f10196450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:06.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.081+0000 7f3f187c3700 1 --2- 172.21.15.22:0/506811188 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f10100e80 0x7f3f10196450 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3f000059d0 tx=0x7f3f0000f690 comp rx=0 tx=0).stop 2024-02-16T19:34:06.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.081+0000 7f3f187c3700 1 -- 172.21.15.22:0/506811188 shutdown_connections 2024-02-16T19:34:06.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.081+0000 7f3f187c3700 1 --2- 172.21.15.22:0/506811188 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f3efc06d9e0 0x7f3efc06fe90 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:06.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.081+0000 7f3f187c3700 1 --2- 172.21.15.22:0/506811188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f101004e0 0x7f3f10195f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:06.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.081+0000 7f3f187c3700 1 --2- 172.21.15.22:0/506811188 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f10100e80 0x7f3f10196450 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:06.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.081+0000 7f3f187c3700 1 -- 172.21.15.22:0/506811188 >> 172.21.15.22:0/506811188 conn(0x7f3f100fbda0 msgr2=0x7f3f100fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:34:06.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.081+0000 7f3f187c3700 1 -- 172.21.15.22:0/506811188 shutdown_connections 2024-02-16T19:34:06.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.081+0000 7f3f187c3700 1 -- 172.21.15.22:0/506811188 wait complete. 2024-02-16T19:34:06.092 INFO:teuthology.orchestra.run.smithi022.stdout:true 2024-02-16T19:34:06.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.187+0000 7fed4f62e700 1 -- 172.21.15.22:0/3524410303 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed48100e40 msgr2=0x7fed48104e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:06.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.187+0000 7fed4f62e700 1 --2- 172.21.15.22:0/3524410303 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed48100e40 0x7fed48104e10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fed44009910 tx=0x7fed44009c20 comp rx=0 tx=0).stop 2024-02-16T19:34:06.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.187+0000 7fed4f62e700 1 -- 172.21.15.22:0/3524410303 shutdown_connections 2024-02-16T19:34:06.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.187+0000 7fed4f62e700 1 --2- 172.21.15.22:0/3524410303 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed48100e40 0x7fed48104e10 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:06.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.187+0000 7fed4f62e700 1 --2- 172.21.15.22:0/3524410303 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed481004a0 0x7fed48100870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:06.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.187+0000 7fed4f62e700 1 -- 172.21.15.22:0/3524410303 >> 172.21.15.22:0/3524410303 conn(0x7fed480fbd40 msgr2=0x7fed480fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:34:06.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.187+0000 7fed4f62e700 1 -- 172.21.15.22:0/3524410303 shutdown_connections 2024-02-16T19:34:06.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.187+0000 7fed4f62e700 1 -- 172.21.15.22:0/3524410303 wait complete. 2024-02-16T19:34:06.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.188+0000 7fed4f62e700 1 Processor -- start 2024-02-16T19:34:06.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.188+0000 7fed4f62e700 1 -- start start 2024-02-16T19:34:06.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.188+0000 7fed4f62e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed481004a0 0x7fed481980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:34:06.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.188+0000 7fed4f62e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed48100e40 0x7fed48198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:34:06.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.188+0000 7fed4f62e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed48198c60 con 0x7fed48100e40 2024-02-16T19:34:06.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.188+0000 7fed4f62e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed48198da0 con 0x7fed481004a0 2024-02-16T19:34:06.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.188+0000 7fed4cbc9700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed48100e40 0x7fed48198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:34:06.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.188+0000 7fed4d3ca700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed481004a0 0x7fed481980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:34:06.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.188+0000 7fed4cbc9700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed48100e40 0x7fed48198610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49942/0 (socket says 172.21.15.22:49942) 2024-02-16T19:34:06.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.188+0000 7fed4cbc9700 1 -- 172.21.15.22:0/4178749596 learned_addr learned my addr 172.21.15.22:0/4178749596 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:34:06.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.189+0000 7fed4cbc9700 1 -- 172.21.15.22:0/4178749596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed481004a0 msgr2=0x7fed481980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:06.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.189+0000 7fed4cbc9700 1 --2- 172.21.15.22:0/4178749596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed481004a0 0x7fed481980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:06.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.189+0000 7fed4cbc9700 1 -- 172.21.15.22:0/4178749596 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed440095f0 con 0x7fed48100e40 2024-02-16T19:34:06.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.189+0000 7fed4cbc9700 1 --2- 172.21.15.22:0/4178749596 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed48100e40 0x7fed48198610 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fed4400bda0 tx=0x7fed4400bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:34:06.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.189+0000 7fed3e7fc700 1 -- 172.21.15.22:0/4178749596 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed4401d070 con 0x7fed48100e40 2024-02-16T19:34:06.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.189+0000 7fed3e7fc700 1 -- 172.21.15.22:0/4178749596 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fed4400f460 con 0x7fed48100e40 2024-02-16T19:34:06.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.189+0000 7fed4f62e700 1 -- 172.21.15.22:0/4178749596 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed4819cb90 con 0x7fed48100e40 2024-02-16T19:34:06.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.189+0000 7fed3e7fc700 1 -- 172.21.15.22:0/4178749596 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed44017630 con 0x7fed48100e40 2024-02-16T19:34:06.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.189+0000 7fed4f62e700 1 -- 172.21.15.22:0/4178749596 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed4819d080 con 0x7fed48100e40 2024-02-16T19:34:06.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.190+0000 7fed4f62e700 1 -- 172.21.15.22:0/4178749596 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed48104590 con 0x7fed48100e40 2024-02-16T19:34:06.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.191+0000 7fed3e7fc700 1 -- 172.21.15.22:0/4178749596 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 39) v1 ==== 91595+0+0 (secure 0 0 0) 0x7fed4400fab0 con 0x7fed48100e40 2024-02-16T19:34:06.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.191+0000 7fed3e7fc700 1 --2- 172.21.15.22:0/4178749596 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fed3406d9e0 0x7fed3406fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:34:06.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.191+0000 7fed3e7fc700 1 -- 172.21.15.22:0/4178749596 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fed4408eae0 con 0x7fed48100e40 2024-02-16T19:34:06.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.191+0000 7fed4d3ca700 1 --2- 172.21.15.22:0/4178749596 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fed3406d9e0 0x7fed3406fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:34:06.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.192+0000 7fed4d3ca700 1 --2- 172.21.15.22:0/4178749596 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fed3406d9e0 0x7fed3406fe90 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fed3800ac50 tx=0x7fed3800a380 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:34:06.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.196+0000 7fed3e7fc700 1 -- 172.21.15.22:0/4178749596 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fed440581a0 con 0x7fed48100e40 2024-02-16T19:34:06.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.421+0000 7fed4f62e700 1 -- 172.21.15.22:0/4178749596 --> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fed4802cca0 con 0x7fed3406d9e0 2024-02-16T19:34:06.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.422+0000 7fed3e7fc700 1 -- 172.21.15.22:0/4178749596 <== mgr.34104 v2:172.21.15.22:6800/505266753 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7fed4802cca0 con 0x7fed3406d9e0 2024-02-16T19:34:06.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.424+0000 7fed4f62e700 1 -- 172.21.15.22:0/4178749596 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fed3406d9e0 msgr2=0x7fed3406fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:06.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.424+0000 7fed4f62e700 1 --2- 172.21.15.22:0/4178749596 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fed3406d9e0 0x7fed3406fe90 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fed3800ac50 tx=0x7fed3800a380 comp rx=0 tx=0).stop 2024-02-16T19:34:06.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.424+0000 7fed4f62e700 1 -- 172.21.15.22:0/4178749596 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed48100e40 msgr2=0x7fed48198610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:06.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.424+0000 7fed4f62e700 1 --2- 172.21.15.22:0/4178749596 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed48100e40 0x7fed48198610 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fed4400bda0 tx=0x7fed4400bdd0 comp rx=0 tx=0).stop 2024-02-16T19:34:06.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.425+0000 7fed4f62e700 1 -- 172.21.15.22:0/4178749596 shutdown_connections 2024-02-16T19:34:06.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.425+0000 7fed4f62e700 1 --2- 172.21.15.22:0/4178749596 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fed3406d9e0 0x7fed3406fe90 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:06.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.425+0000 7fed4f62e700 1 --2- 172.21.15.22:0/4178749596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed481004a0 0x7fed481980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:06.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.425+0000 7fed4f62e700 1 --2- 172.21.15.22:0/4178749596 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed48100e40 0x7fed48198610 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:06.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.425+0000 7fed4f62e700 1 -- 172.21.15.22:0/4178749596 >> 172.21.15.22:0/4178749596 conn(0x7fed480fbd40 msgr2=0x7fed480fd2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:34:06.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.425+0000 7fed4f62e700 1 -- 172.21.15.22:0/4178749596 shutdown_connections 2024-02-16T19:34:06.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.425+0000 7fed4f62e700 1 -- 172.21.15.22:0/4178749596 wait complete. 2024-02-16T19:34:06.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.531+0000 7fa118fdf700 1 -- 172.21.15.22:0/4147376018 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa114068540 msgr2=0x7fa1140689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:06.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.531+0000 7fa118fdf700 1 --2- 172.21.15.22:0/4147376018 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa114068540 0x7fa1140689b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc009910 tx=0x7fa0fc009c20 comp rx=0 tx=0).stop 2024-02-16T19:34:06.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.531+0000 7fa118fdf700 1 -- 172.21.15.22:0/4147376018 shutdown_connections 2024-02-16T19:34:06.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.531+0000 7fa118fdf700 1 --2- 172.21.15.22:0/4147376018 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa114068540 0x7fa1140689b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:06.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.531+0000 7fa118fdf700 1 --2- 172.21.15.22:0/4147376018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa114101000 0x7fa1141013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:06.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.531+0000 7fa118fdf700 1 -- 172.21.15.22:0/4147376018 >> 172.21.15.22:0/4147376018 conn(0x7fa114075130 msgr2=0x7fa114075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:34:06.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.531+0000 7fa118fdf700 1 -- 172.21.15.22:0/4147376018 shutdown_connections 2024-02-16T19:34:06.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.531+0000 7fa118fdf700 1 -- 172.21.15.22:0/4147376018 wait complete. 2024-02-16T19:34:06.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.532+0000 7fa118fdf700 1 Processor -- start 2024-02-16T19:34:06.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.532+0000 7fa118fdf700 1 -- start start 2024-02-16T19:34:06.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.532+0000 7fa118fdf700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa114068540 0x7fa114198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:34:06.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.532+0000 7fa118fdf700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa114101000 0x7fa1141985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:34:06.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.532+0000 7fa118fdf700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa114198c90 con 0x7fa114101000 2024-02-16T19:34:06.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.532+0000 7fa118fdf700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa11419ca20 con 0x7fa114068540 2024-02-16T19:34:06.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.532+0000 7fa111d9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa114101000 0x7fa1141985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:34:06.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.532+0000 7fa111d9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa114101000 0x7fa1141985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49972/0 (socket says 172.21.15.22:49972) 2024-02-16T19:34:06.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.532+0000 7fa111d9b700 1 -- 172.21.15.22:0/2773841571 learned_addr learned my addr 172.21.15.22:0/2773841571 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:34:06.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.532+0000 7fa11259c700 1 --2- 172.21.15.22:0/2773841571 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa114068540 0x7fa114198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:34:06.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.532+0000 7fa111d9b700 1 -- 172.21.15.22:0/2773841571 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa114068540 msgr2=0x7fa114198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:06.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.532+0000 7fa111d9b700 1 --2- 172.21.15.22:0/2773841571 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa114068540 0x7fa114198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:06.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.532+0000 7fa111d9b700 1 -- 172.21.15.22:0/2773841571 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0fc0095f0 con 0x7fa114101000 2024-02-16T19:34:06.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.533+0000 7fa111d9b700 1 --2- 172.21.15.22:0/2773841571 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa114101000 0x7fa1141985b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc00f690 tx=0x7fa0fc00f6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:34:06.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.533+0000 7fa10b7fe700 1 -- 172.21.15.22:0/2773841571 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa0fc01d070 con 0x7fa114101000 2024-02-16T19:34:06.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.533+0000 7fa10b7fe700 1 -- 172.21.15.22:0/2773841571 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa0fc0050d0 con 0x7fa114101000 2024-02-16T19:34:06.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.533+0000 7fa118fdf700 1 -- 172.21.15.22:0/2773841571 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa11419cca0 con 0x7fa114101000 2024-02-16T19:34:06.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.533+0000 7fa10b7fe700 1 -- 172.21.15.22:0/2773841571 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa0fc022470 con 0x7fa114101000 2024-02-16T19:34:06.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.533+0000 7fa118fdf700 1 -- 172.21.15.22:0/2773841571 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa11419d190 con 0x7fa114101000 2024-02-16T19:34:06.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.534+0000 7fa118fdf700 1 -- 172.21.15.22:0/2773841571 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa11404f2a0 con 0x7fa114101000 2024-02-16T19:34:06.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.534+0000 7fa10b7fe700 1 -- 172.21.15.22:0/2773841571 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 39) v1 ==== 91595+0+0 (secure 0 0 0) 0x7fa0fc003780 con 0x7fa114101000 2024-02-16T19:34:06.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.535+0000 7fa10b7fe700 1 --2- 172.21.15.22:0/2773841571 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fa10006d9d0 0x7fa10006fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:34:06.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.535+0000 7fa10b7fe700 1 -- 172.21.15.22:0/2773841571 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fa0fc08eb00 con 0x7fa114101000 2024-02-16T19:34:06.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.535+0000 7fa11259c700 1 --2- 172.21.15.22:0/2773841571 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fa10006d9d0 0x7fa10006fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:34:06.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.535+0000 7fa11259c700 1 --2- 172.21.15.22:0/2773841571 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fa10006d9d0 0x7fa10006fe80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa10c00f540 tx=0x7fa10c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:34:06.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.539+0000 7fa10b7fe700 1 -- 172.21.15.22:0/2773841571 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa0fc0581f0 con 0x7fa114101000 2024-02-16T19:34:06.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.759+0000 7fa118fdf700 1 -- 172.21.15.22:0/2773841571 --> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa11419d4c0 con 0x7fa10006d9d0 2024-02-16T19:34:06.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.770+0000 7fa10b7fe700 1 -- 172.21.15.22:0/2773841571 <== mgr.34104 v2:172.21.15.22:6800/505266753 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fa11419d4c0 con 0x7fa10006d9d0 2024-02-16T19:34:06.772 INFO:teuthology.orchestra.run.smithi022.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T19:34:06.772 INFO:teuthology.orchestra.run.smithi022.stdout:alertmanager.smithi022 smithi022 *:9093,9094 running (2m) 45s ago 13m 15.9M - 0.25.0 c8568f914cd2 c89bc76be374 2024-02-16T19:34:06.772 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-exporter.smithi022 smithi022 running (13m) 45s ago 13m 8002k - 18.2.0 dc2bc1663786 3f8bef4349ba 2024-02-16T19:34:06.772 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-exporter.smithi117 smithi117 running (12m) 30s ago 12m 7704k - 18.2.0 dc2bc1663786 59b65a7e309f 2024-02-16T19:34:06.772 INFO:teuthology.orchestra.run.smithi022.stdout:crash.smithi022 smithi022 running (13m) 45s ago 13m 6669k - 18.2.0 dc2bc1663786 efef476aa486 2024-02-16T19:34:06.772 INFO:teuthology.orchestra.run.smithi022.stdout:crash.smithi117 smithi117 running (12m) 30s ago 12m 6665k - 18.2.0 dc2bc1663786 5466453eb149 2024-02-16T19:34:06.772 INFO:teuthology.orchestra.run.smithi022.stdout:grafana.smithi022 smithi022 *:3000 running (2m) 45s ago 13m 76.4M - 9.4.12 a0ea5e3c46d9 fd27d11e9729 2024-02-16T19:34:06.772 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi022.btppwm smithi022 running (7m) 45s ago 7m 158M - 18.2.0 dc2bc1663786 22971fb3a907 2024-02-16T19:34:06.772 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi022.ocukvs smithi022 running (7m) 45s ago 7m 15.7M - 18.2.0 dc2bc1663786 81ee7bd5fd39 2024-02-16T19:34:06.772 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi117.ibmppn smithi117 running (7m) 30s ago 7m 17.2M - 18.2.0 dc2bc1663786 8c606cbeeb86 2024-02-16T19:34:06.772 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi117.orzors smithi117 running (7m) 30s ago 7m 16.3M - 18.2.0 dc2bc1663786 7d2bc4df3feb 2024-02-16T19:34:06.773 INFO:teuthology.orchestra.run.smithi022.stdout:mgr.smithi022.gmdcch smithi022 *:8443,9283,8765 running (4m) 45s ago 14m 472M - 19.0.0-1450-gf78a58c0 f081f847070a 7cb74cc9deea 2024-02-16T19:34:06.773 INFO:teuthology.orchestra.run.smithi022.stdout:mgr.smithi117.cohbtv smithi117 *:8443,9283,8765 running (3m) 30s ago 12m 449M - 19.0.0-1450-gf78a58c0 f081f847070a c825b1152189 2024-02-16T19:34:06.773 INFO:teuthology.orchestra.run.smithi022.stdout:mon.smithi022 smithi022 running (59s) 45s ago 14m 29.3M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 9c84755ae67d 2024-02-16T19:34:06.773 INFO:teuthology.orchestra.run.smithi022.stdout:mon.smithi117 smithi117 running (34s) 30s ago 12m 25.4M 2048M 19.0.0-1450-gf78a58c0 f081f847070a a319ac50f1a9 2024-02-16T19:34:06.773 INFO:teuthology.orchestra.run.smithi022.stdout:node-exporter.smithi022 smithi022 *:9100 running (3m) 45s ago 13m 16.9M - 1.5.0 0da6a335fe13 1f5b58292d27 2024-02-16T19:34:06.773 INFO:teuthology.orchestra.run.smithi022.stdout:node-exporter.smithi117 smithi117 *:9100 running (3m) 30s ago 12m 15.5M - 1.5.0 0da6a335fe13 bb5e8491d5a6 2024-02-16T19:34:06.773 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0 smithi022 running (11m) 45s ago 11m 210M 1217M 18.2.0 dc2bc1663786 0027f86f16ce 2024-02-16T19:34:06.773 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1 smithi022 running (10m) 45s ago 10m 227M 1217M 18.2.0 dc2bc1663786 87300382b8a7 2024-02-16T19:34:06.773 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2 smithi022 running (10m) 45s ago 10m 187M 1217M 18.2.0 dc2bc1663786 be8a631b4f93 2024-02-16T19:34:06.773 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3 smithi117 running (10m) 30s ago 10m 245M 2240M 18.2.0 dc2bc1663786 9c0ce9a981cf 2024-02-16T19:34:06.773 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4 smithi117 running (9m) 30s ago 9m 209M 2240M 18.2.0 dc2bc1663786 1779c3496f4d 2024-02-16T19:34:06.773 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5 smithi117 running (9m) 30s ago 9m 197M 2240M 18.2.0 dc2bc1663786 fefaabca0638 2024-02-16T19:34:06.774 INFO:teuthology.orchestra.run.smithi022.stdout:prometheus.smithi022 smithi022 *:9095 running (2m) 45s ago 12m 56.3M - 2.43.0 a07b618ecd1d c3b8689d0c97 2024-02-16T19:34:06.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.772+0000 7fa118fdf700 1 -- 172.21.15.22:0/2773841571 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fa10006d9d0 msgr2=0x7fa10006fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:06.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.772+0000 7fa118fdf700 1 --2- 172.21.15.22:0/2773841571 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fa10006d9d0 0x7fa10006fe80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa10c00f540 tx=0x7fa10c00b040 comp rx=0 tx=0).stop 2024-02-16T19:34:06.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.772+0000 7fa118fdf700 1 -- 172.21.15.22:0/2773841571 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa114101000 msgr2=0x7fa1141985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:06.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.772+0000 7fa118fdf700 1 --2- 172.21.15.22:0/2773841571 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa114101000 0x7fa1141985b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc00f690 tx=0x7fa0fc00f6c0 comp rx=0 tx=0).stop 2024-02-16T19:34:06.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.772+0000 7fa118fdf700 1 -- 172.21.15.22:0/2773841571 shutdown_connections 2024-02-16T19:34:06.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.773+0000 7fa118fdf700 1 --2- 172.21.15.22:0/2773841571 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fa10006d9d0 0x7fa10006fe80 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:06.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.773+0000 7fa118fdf700 1 --2- 172.21.15.22:0/2773841571 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa114068540 0x7fa114198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:06.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.773+0000 7fa118fdf700 1 --2- 172.21.15.22:0/2773841571 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa114101000 0x7fa1141985b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:06.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.773+0000 7fa118fdf700 1 -- 172.21.15.22:0/2773841571 >> 172.21.15.22:0/2773841571 conn(0x7fa114075130 msgr2=0x7fa1140fdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:34:06.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.773+0000 7fa118fdf700 1 -- 172.21.15.22:0/2773841571 shutdown_connections 2024-02-16T19:34:06.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.773+0000 7fa118fdf700 1 -- 172.21.15.22:0/2773841571 wait complete. 2024-02-16T19:34:06.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.876+0000 7fe11636e700 1 -- 172.21.15.22:0/2142430753 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe1101004e0 msgr2=0x7fe1101008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:06.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.876+0000 7fe11636e700 1 --2- 172.21.15.22:0/2142430753 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe1101004e0 0x7fe1101008b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe100009910 tx=0x7fe100009c20 comp rx=0 tx=0).stop 2024-02-16T19:34:06.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.876+0000 7fe11636e700 1 -- 172.21.15.22:0/2142430753 shutdown_connections 2024-02-16T19:34:06.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.876+0000 7fe11636e700 1 --2- 172.21.15.22:0/2142430753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe110100e80 0x7fe110104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:06.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.876+0000 7fe11636e700 1 --2- 172.21.15.22:0/2142430753 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe1101004e0 0x7fe1101008b0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:06.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.876+0000 7fe11636e700 1 -- 172.21.15.22:0/2142430753 >> 172.21.15.22:0/2142430753 conn(0x7fe1100fbda0 msgr2=0x7fe1100fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:34:06.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.876+0000 7fe11636e700 1 -- 172.21.15.22:0/2142430753 shutdown_connections 2024-02-16T19:34:06.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.877+0000 7fe11636e700 1 -- 172.21.15.22:0/2142430753 wait complete. 2024-02-16T19:34:06.879 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:06 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:06.879 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:06 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:06.880 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:06 smithi022 ceph-mon[88798]: Reconfiguring osd.5 (monmap changed)... 2024-02-16T19:34:06.880 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:06 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-16T19:34:06.880 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:06 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:34:06.880 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:06 smithi022 ceph-mon[88798]: Reconfiguring daemon osd.5 on smithi117 2024-02-16T19:34:06.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.877+0000 7fe11636e700 1 Processor -- start 2024-02-16T19:34:06.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.877+0000 7fe11636e700 1 -- start start 2024-02-16T19:34:06.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.878+0000 7fe11636e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe1101004e0 0x7fe110073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:34:06.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.878+0000 7fe11636e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe110100e80 0x7fe110073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:34:06.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.878+0000 7fe11636e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe110073ad0 con 0x7fe1101004e0 2024-02-16T19:34:06.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.878+0000 7fe11636e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe110073c10 con 0x7fe110100e80 2024-02-16T19:34:06.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.878+0000 7fe10ffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe1101004e0 0x7fe110073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:34:06.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.878+0000 7fe10ffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe1101004e0 0x7fe110073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49988/0 (socket says 172.21.15.22:49988) 2024-02-16T19:34:06.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.878+0000 7fe10ffff700 1 -- 172.21.15.22:0/143951883 learned_addr learned my addr 172.21.15.22:0/143951883 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:34:06.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.878+0000 7fe10f7fe700 1 --2- 172.21.15.22:0/143951883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe110100e80 0x7fe110073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:34:06.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.878+0000 7fe10ffff700 1 -- 172.21.15.22:0/143951883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe110100e80 msgr2=0x7fe110073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:06.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.878+0000 7fe10ffff700 1 --2- 172.21.15.22:0/143951883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe110100e80 0x7fe110073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:06.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.878+0000 7fe10ffff700 1 -- 172.21.15.22:0/143951883 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1000095f0 con 0x7fe1101004e0 2024-02-16T19:34:06.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.878+0000 7fe10ffff700 1 --2- 172.21.15.22:0/143951883 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe1101004e0 0x7fe110073050 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe100006010 tx=0x7fe10000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:34:06.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.879+0000 7fe10d7fa700 1 -- 172.21.15.22:0/143951883 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe10001d070 con 0x7fe1101004e0 2024-02-16T19:34:06.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.879+0000 7fe10d7fa700 1 -- 172.21.15.22:0/143951883 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe100005dc0 con 0x7fe1101004e0 2024-02-16T19:34:06.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.879+0000 7fe11636e700 1 -- 172.21.15.22:0/143951883 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe110073e90 con 0x7fe1101004e0 2024-02-16T19:34:06.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.879+0000 7fe11636e700 1 -- 172.21.15.22:0/143951883 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1101a23f0 con 0x7fe1101004e0 2024-02-16T19:34:06.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.879+0000 7fe10d7fa700 1 -- 172.21.15.22:0/143951883 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe100017400 con 0x7fe1101004e0 2024-02-16T19:34:06.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.880+0000 7fe11636e700 1 -- 172.21.15.22:0/143951883 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1101087f0 con 0x7fe1101004e0 2024-02-16T19:34:06.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.880+0000 7fe10d7fa700 1 -- 172.21.15.22:0/143951883 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 39) v1 ==== 91595+0+0 (secure 0 0 0) 0x7fe100017560 con 0x7fe1101004e0 2024-02-16T19:34:06.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.881+0000 7fe10d7fa700 1 --2- 172.21.15.22:0/143951883 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fe0f806d9e0 0x7fe0f806fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:34:06.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.881+0000 7fe10f7fe700 1 --2- 172.21.15.22:0/143951883 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fe0f806d9e0 0x7fe0f806fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:34:06.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.881+0000 7fe10d7fa700 1 -- 172.21.15.22:0/143951883 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fe10008eab0 con 0x7fe1101004e0 2024-02-16T19:34:06.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.881+0000 7fe10f7fe700 1 --2- 172.21.15.22:0/143951883 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fe0f806d9e0 0x7fe0f806fe90 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe110074840 tx=0x7fe10400b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:34:06.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:06.886+0000 7fe10d7fa700 1 -- 172.21.15.22:0/143951883 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe100058170 con 0x7fe1101004e0 2024-02-16T19:34:06.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:06 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:06.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:06 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:06.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:06 smithi117 ceph-mon[67870]: Reconfiguring osd.5 (monmap changed)... 2024-02-16T19:34:06.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:06 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-16T19:34:06.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:06 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:34:06.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:06 smithi117 ceph-mon[67870]: Reconfiguring daemon osd.5 on smithi117 2024-02-16T19:34:07.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.180+0000 7fe11636e700 1 -- 172.21.15.22:0/143951883 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe11004f2a0 con 0x7fe1101004e0 2024-02-16T19:34:07.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.181+0000 7fe10d7fa700 1 -- 172.21.15.22:0/143951883 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+688 (secure 0 0 0) 0x7fe10005b830 con 0x7fe1101004e0 2024-02-16T19:34:07.182 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-02-16T19:34:07.182 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": { 2024-02-16T19:34:07.182 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T19:34:07.183 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:34:07.183 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": { 2024-02-16T19:34:07.183 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T19:34:07.183 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:34:07.183 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": { 2024-02-16T19:34:07.183 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-02-16T19:34:07.183 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:34:07.183 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": { 2024-02-16T19:34:07.183 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-02-16T19:34:07.183 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:34:07.183 INFO:teuthology.orchestra.run.smithi022.stdout: "overall": { 2024-02-16T19:34:07.183 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 10, 2024-02-16T19:34:07.183 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 4 2024-02-16T19:34:07.183 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-02-16T19:34:07.183 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-02-16T19:34:07.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.183+0000 7fe11636e700 1 -- 172.21.15.22:0/143951883 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fe0f806d9e0 msgr2=0x7fe0f806fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:07.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.183+0000 7fe11636e700 1 --2- 172.21.15.22:0/143951883 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fe0f806d9e0 0x7fe0f806fe90 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe110074840 tx=0x7fe10400b410 comp rx=0 tx=0).stop 2024-02-16T19:34:07.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.183+0000 7fe11636e700 1 -- 172.21.15.22:0/143951883 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe1101004e0 msgr2=0x7fe110073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:07.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.183+0000 7fe11636e700 1 --2- 172.21.15.22:0/143951883 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe1101004e0 0x7fe110073050 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe100006010 tx=0x7fe10000bd70 comp rx=0 tx=0).stop 2024-02-16T19:34:07.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.183+0000 7fe11636e700 1 -- 172.21.15.22:0/143951883 shutdown_connections 2024-02-16T19:34:07.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.183+0000 7fe11636e700 1 --2- 172.21.15.22:0/143951883 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fe0f806d9e0 0x7fe0f806fe90 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:07.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.183+0000 7fe11636e700 1 --2- 172.21.15.22:0/143951883 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe1101004e0 0x7fe110073050 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:07.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.183+0000 7fe11636e700 1 --2- 172.21.15.22:0/143951883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe110100e80 0x7fe110073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:07.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.183+0000 7fe11636e700 1 -- 172.21.15.22:0/143951883 >> 172.21.15.22:0/143951883 conn(0x7fe1100fbda0 msgr2=0x7fe1100fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:34:07.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.183+0000 7fe11636e700 1 -- 172.21.15.22:0/143951883 shutdown_connections 2024-02-16T19:34:07.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.183+0000 7fe11636e700 1 -- 172.21.15.22:0/143951883 wait complete. 2024-02-16T19:34:07.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.288+0000 7f03d9ec0700 1 -- 172.21.15.22:0/2284030132 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03d41004e0 msgr2=0x7f03d41008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:07.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.288+0000 7f03d9ec0700 1 --2- 172.21.15.22:0/2284030132 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03d41004e0 0x7f03d41008b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f03c4009910 tx=0x7f03c4009c20 comp rx=0 tx=0).stop 2024-02-16T19:34:07.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.288+0000 7f03d9ec0700 1 -- 172.21.15.22:0/2284030132 shutdown_connections 2024-02-16T19:34:07.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.288+0000 7f03d9ec0700 1 --2- 172.21.15.22:0/2284030132 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03d4100e80 0x7f03d4104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:07.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.288+0000 7f03d9ec0700 1 --2- 172.21.15.22:0/2284030132 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03d41004e0 0x7f03d41008b0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:07.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.288+0000 7f03d9ec0700 1 -- 172.21.15.22:0/2284030132 >> 172.21.15.22:0/2284030132 conn(0x7f03d40fbda0 msgr2=0x7f03d40fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:34:07.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.288+0000 7f03d9ec0700 1 -- 172.21.15.22:0/2284030132 shutdown_connections 2024-02-16T19:34:07.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.289+0000 7f03d9ec0700 1 -- 172.21.15.22:0/2284030132 wait complete. 2024-02-16T19:34:07.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.289+0000 7f03d9ec0700 1 Processor -- start 2024-02-16T19:34:07.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.289+0000 7f03d9ec0700 1 -- start start 2024-02-16T19:34:07.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.289+0000 7f03d9ec0700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03d41004e0 0x7f03d4195ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:34:07.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.289+0000 7f03d9ec0700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03d4100e80 0x7f03d4196430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:34:07.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.289+0000 7f03d9ec0700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d4196b10 con 0x7f03d41004e0 2024-02-16T19:34:07.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.289+0000 7f03d9ec0700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d419a8a0 con 0x7f03d4100e80 2024-02-16T19:34:07.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.290+0000 7f03d37fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03d41004e0 0x7f03d4195ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:34:07.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.290+0000 7f03d37fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03d41004e0 0x7f03d4195ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50004/0 (socket says 172.21.15.22:50004) 2024-02-16T19:34:07.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.290+0000 7f03d37fe700 1 -- 172.21.15.22:0/3133059328 learned_addr learned my addr 172.21.15.22:0/3133059328 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:34:07.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.290+0000 7f03d2ffd700 1 --2- 172.21.15.22:0/3133059328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03d4100e80 0x7f03d4196430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:34:07.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.290+0000 7f03d37fe700 1 -- 172.21.15.22:0/3133059328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03d4100e80 msgr2=0x7f03d4196430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:07.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.290+0000 7f03d37fe700 1 --2- 172.21.15.22:0/3133059328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03d4100e80 0x7f03d4196430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:07.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.290+0000 7f03d37fe700 1 -- 172.21.15.22:0/3133059328 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03c40095f0 con 0x7f03d41004e0 2024-02-16T19:34:07.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.290+0000 7f03d37fe700 1 --2- 172.21.15.22:0/3133059328 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03d41004e0 0x7f03d4195ef0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f03c4006010 tx=0x7f03c400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:34:07.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.290+0000 7f03d0ff9700 1 -- 172.21.15.22:0/3133059328 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f03c401d070 con 0x7f03d41004e0 2024-02-16T19:34:07.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.290+0000 7f03d0ff9700 1 -- 172.21.15.22:0/3133059328 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f03c4005dc0 con 0x7f03d41004e0 2024-02-16T19:34:07.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.290+0000 7f03d9ec0700 1 -- 172.21.15.22:0/3133059328 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03d419ab20 con 0x7f03d41004e0 2024-02-16T19:34:07.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.290+0000 7f03d0ff9700 1 -- 172.21.15.22:0/3133059328 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f03c4017400 con 0x7f03d41004e0 2024-02-16T19:34:07.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.290+0000 7f03d9ec0700 1 -- 172.21.15.22:0/3133059328 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03d419afc0 con 0x7f03d41004e0 2024-02-16T19:34:07.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.291+0000 7f03d9ec0700 1 -- 172.21.15.22:0/3133059328 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03d41087f0 con 0x7f03d41004e0 2024-02-16T19:34:07.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.292+0000 7f03d0ff9700 1 -- 172.21.15.22:0/3133059328 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 39) v1 ==== 91595+0+0 (secure 0 0 0) 0x7f03c4003a60 con 0x7f03d41004e0 2024-02-16T19:34:07.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.292+0000 7f03d0ff9700 1 --2- 172.21.15.22:0/3133059328 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f03bc06d9e0 0x7f03bc06fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:34:07.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.292+0000 7f03d0ff9700 1 -- 172.21.15.22:0/3133059328 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f03c408eba0 con 0x7f03d41004e0 2024-02-16T19:34:07.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.292+0000 7f03d2ffd700 1 --2- 172.21.15.22:0/3133059328 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f03bc06d9e0 0x7f03bc06fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:34:07.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.293+0000 7f03d2ffd700 1 --2- 172.21.15.22:0/3133059328 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f03bc06d9e0 0x7f03bc06fe90 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f03d4197510 tx=0x7f03c800b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:34:07.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.297+0000 7f03d0ff9700 1 -- 172.21.15.22:0/3133059328 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f03c40581e0 con 0x7f03d41004e0 2024-02-16T19:34:07.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.554+0000 7f03d9ec0700 1 -- 172.21.15.22:0/3133059328 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f03d404f2a0 con 0x7f03d41004e0 2024-02-16T19:34:07.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.554+0000 7f03d0ff9700 1 -- 172.21.15.22:0/3133059328 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1845 (secure 0 0 0) 0x7f03c405b8a0 con 0x7f03d41004e0 2024-02-16T19:34:07.556 INFO:teuthology.orchestra.run.smithi022.stdout:e15 2024-02-16T19:34:07.556 INFO:teuthology.orchestra.run.smithi022.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T19:34:07.556 INFO:teuthology.orchestra.run.smithi022.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:34:07.556 INFO:teuthology.orchestra.run.smithi022.stdout:legacy client fscid: 1 2024-02-16T19:34:07.556 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:34:07.556 INFO:teuthology.orchestra.run.smithi022.stdout:Filesystem 'cephfs' (1) 2024-02-16T19:34:07.556 INFO:teuthology.orchestra.run.smithi022.stdout:fs_name cephfs 2024-02-16T19:34:07.556 INFO:teuthology.orchestra.run.smithi022.stdout:epoch 15 2024-02-16T19:34:07.556 INFO:teuthology.orchestra.run.smithi022.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T19:34:07.557 INFO:teuthology.orchestra.run.smithi022.stdout:created 2024-02-16T19:26:17.873644+0000 2024-02-16T19:34:07.557 INFO:teuthology.orchestra.run.smithi022.stdout:modified 2024-02-16T19:26:35.943353+0000 2024-02-16T19:34:07.557 INFO:teuthology.orchestra.run.smithi022.stdout:tableserver 0 2024-02-16T19:34:07.557 INFO:teuthology.orchestra.run.smithi022.stdout:root 0 2024-02-16T19:34:07.557 INFO:teuthology.orchestra.run.smithi022.stdout:session_timeout 60 2024-02-16T19:34:07.557 INFO:teuthology.orchestra.run.smithi022.stdout:session_autoclose 300 2024-02-16T19:34:07.557 INFO:teuthology.orchestra.run.smithi022.stdout:max_file_size 1099511627776 2024-02-16T19:34:07.557 INFO:teuthology.orchestra.run.smithi022.stdout:max_xattr_size 65536 2024-02-16T19:34:07.557 INFO:teuthology.orchestra.run.smithi022.stdout:required_client_features {} 2024-02-16T19:34:07.557 INFO:teuthology.orchestra.run.smithi022.stdout:last_failure 0 2024-02-16T19:34:07.557 INFO:teuthology.orchestra.run.smithi022.stdout:last_failure_osd_epoch 47 2024-02-16T19:34:07.557 INFO:teuthology.orchestra.run.smithi022.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:34:07.557 INFO:teuthology.orchestra.run.smithi022.stdout:max_mds 1 2024-02-16T19:34:07.557 INFO:teuthology.orchestra.run.smithi022.stdout:in 0 2024-02-16T19:34:07.558 INFO:teuthology.orchestra.run.smithi022.stdout:up {0=14490} 2024-02-16T19:34:07.558 INFO:teuthology.orchestra.run.smithi022.stdout:failed 2024-02-16T19:34:07.558 INFO:teuthology.orchestra.run.smithi022.stdout:damaged 2024-02-16T19:34:07.558 INFO:teuthology.orchestra.run.smithi022.stdout:stopped 2024-02-16T19:34:07.558 INFO:teuthology.orchestra.run.smithi022.stdout:data_pools [3] 2024-02-16T19:34:07.558 INFO:teuthology.orchestra.run.smithi022.stdout:metadata_pool 2 2024-02-16T19:34:07.558 INFO:teuthology.orchestra.run.smithi022.stdout:inline_data enabled 2024-02-16T19:34:07.558 INFO:teuthology.orchestra.run.smithi022.stdout:balancer 2024-02-16T19:34:07.558 INFO:teuthology.orchestra.run.smithi022.stdout:bal_rank_mask -1 2024-02-16T19:34:07.558 INFO:teuthology.orchestra.run.smithi022.stdout:standby_count_wanted 1 2024-02-16T19:34:07.558 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.cephfs.smithi022.btppwm{0:14490} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.22:6828/3597685751,v1:172.21.15.22:6829/3597685751] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:34:07.558 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:34:07.558 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:34:07.559 INFO:teuthology.orchestra.run.smithi022.stdout:Standby daemons: 2024-02-16T19:34:07.559 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:34:07.559 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.cephfs.smithi022.ocukvs{-1:14500} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.22:6826/242865843,v1:172.21.15.22:6827/242865843] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:34:07.559 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.cephfs.smithi117.ibmppn{-1:24259} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.117:6824/565408102,v1:172.21.15.117:6825/565408102] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:34:07.559 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.cephfs.smithi117.orzors{-1:24265} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.117:6826/3122342028,v1:172.21.15.117:6827/3122342028] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:34:07.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.556+0000 7f03d9ec0700 1 -- 172.21.15.22:0/3133059328 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f03bc06d9e0 msgr2=0x7f03bc06fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:07.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.556+0000 7f03d9ec0700 1 --2- 172.21.15.22:0/3133059328 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f03bc06d9e0 0x7f03bc06fe90 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f03d4197510 tx=0x7f03c800b410 comp rx=0 tx=0).stop 2024-02-16T19:34:07.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.556+0000 7f03d9ec0700 1 -- 172.21.15.22:0/3133059328 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03d41004e0 msgr2=0x7f03d4195ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:07.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.556+0000 7f03d9ec0700 1 --2- 172.21.15.22:0/3133059328 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03d41004e0 0x7f03d4195ef0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f03c4006010 tx=0x7f03c400bd70 comp rx=0 tx=0).stop 2024-02-16T19:34:07.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.556+0000 7f03d9ec0700 1 -- 172.21.15.22:0/3133059328 shutdown_connections 2024-02-16T19:34:07.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.556+0000 7f03d9ec0700 1 --2- 172.21.15.22:0/3133059328 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f03bc06d9e0 0x7f03bc06fe90 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:07.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.556+0000 7f03d9ec0700 1 --2- 172.21.15.22:0/3133059328 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03d41004e0 0x7f03d4195ef0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:07.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.556+0000 7f03d9ec0700 1 --2- 172.21.15.22:0/3133059328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03d4100e80 0x7f03d4196430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:07.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.556+0000 7f03d9ec0700 1 -- 172.21.15.22:0/3133059328 >> 172.21.15.22:0/3133059328 conn(0x7f03d40fbda0 msgr2=0x7f03d40fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:34:07.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.557+0000 7f03d9ec0700 1 -- 172.21.15.22:0/3133059328 shutdown_connections 2024-02-16T19:34:07.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.557+0000 7f03d9ec0700 1 -- 172.21.15.22:0/3133059328 wait complete. 2024-02-16T19:34:07.560 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 15 2024-02-16T19:34:07.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.663+0000 7f2c7bc0b700 1 -- 172.21.15.22:0/2846059757 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c740ffd60 msgr2=0x7f2c74103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:07.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.663+0000 7f2c7bc0b700 1 --2- 172.21.15.22:0/2846059757 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c740ffd60 0x7f2c74103d30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2c68009910 tx=0x7f2c68009c20 comp rx=0 tx=0).stop 2024-02-16T19:34:07.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.663+0000 7f2c7bc0b700 1 -- 172.21.15.22:0/2846059757 shutdown_connections 2024-02-16T19:34:07.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.663+0000 7f2c7bc0b700 1 --2- 172.21.15.22:0/2846059757 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c740ffd60 0x7f2c74103d30 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:07.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.663+0000 7f2c7bc0b700 1 --2- 172.21.15.22:0/2846059757 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c740ff3c0 0x7f2c740ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:07.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.663+0000 7f2c7bc0b700 1 -- 172.21.15.22:0/2846059757 >> 172.21.15.22:0/2846059757 conn(0x7f2c740fac60 msgr2=0x7f2c740fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:34:07.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.663+0000 7f2c7bc0b700 1 -- 172.21.15.22:0/2846059757 shutdown_connections 2024-02-16T19:34:07.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.663+0000 7f2c7bc0b700 1 -- 172.21.15.22:0/2846059757 wait complete. 2024-02-16T19:34:07.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.664+0000 7f2c7bc0b700 1 Processor -- start 2024-02-16T19:34:07.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.664+0000 7f2c7bc0b700 1 -- start start 2024-02-16T19:34:07.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.664+0000 7f2c7bc0b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c740ff3c0 0x7f2c74100dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:34:07.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.664+0000 7f2c7bc0b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c740ffd60 0x7f2c74101310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:34:07.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.664+0000 7f2c7bc0b700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c74101850 con 0x7f2c740ffd60 2024-02-16T19:34:07.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.664+0000 7f2c7bc0b700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c74101990 con 0x7f2c740ff3c0 2024-02-16T19:34:07.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.664+0000 7f2c791a6700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c740ffd60 0x7f2c74101310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:34:07.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.664+0000 7f2c799a7700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c740ff3c0 0x7f2c74100dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:34:07.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.664+0000 7f2c791a6700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c740ffd60 0x7f2c74101310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50024/0 (socket says 172.21.15.22:50024) 2024-02-16T19:34:07.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.664+0000 7f2c791a6700 1 -- 172.21.15.22:0/205790058 learned_addr learned my addr 172.21.15.22:0/205790058 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:34:07.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.664+0000 7f2c799a7700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c740ff3c0 0x7f2c74100dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.22:46514/0 (socket says 172.21.15.22:46514) 2024-02-16T19:34:07.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.665+0000 7f2c799a7700 1 -- 172.21.15.22:0/205790058 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c740ffd60 msgr2=0x7f2c74101310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:07.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.665+0000 7f2c799a7700 1 --2- 172.21.15.22:0/205790058 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c740ffd60 0x7f2c74101310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:07.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.665+0000 7f2c799a7700 1 -- 172.21.15.22:0/205790058 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c680095f0 con 0x7f2c740ff3c0 2024-02-16T19:34:07.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.665+0000 7f2c791a6700 1 --2- 172.21.15.22:0/205790058 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c740ffd60 0x7f2c74101310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T19:34:07.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.665+0000 7f2c799a7700 1 --2- 172.21.15.22:0/205790058 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c740ff3c0 0x7f2c74100dd0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2c6400ef10 tx=0x7f2c6400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:34:07.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.665+0000 7f2c72ffd700 1 -- 172.21.15.22:0/205790058 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c6400c9f0 con 0x7f2c740ff3c0 2024-02-16T19:34:07.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.665+0000 7f2c72ffd700 1 -- 172.21.15.22:0/205790058 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2c6400f040 con 0x7f2c740ff3c0 2024-02-16T19:34:07.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.665+0000 7f2c7bc0b700 1 -- 172.21.15.22:0/205790058 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c74101c10 con 0x7f2c740ff3c0 2024-02-16T19:34:07.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.665+0000 7f2c72ffd700 1 -- 172.21.15.22:0/205790058 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c640146b0 con 0x7f2c740ff3c0 2024-02-16T19:34:07.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.665+0000 7f2c7bc0b700 1 -- 172.21.15.22:0/205790058 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c741a2420 con 0x7f2c740ff3c0 2024-02-16T19:34:07.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.666+0000 7f2c7bc0b700 1 -- 172.21.15.22:0/205790058 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c74067980 con 0x7f2c740ff3c0 2024-02-16T19:34:07.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.666+0000 7f2c72ffd700 1 -- 172.21.15.22:0/205790058 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 39) v1 ==== 91595+0+0 (secure 0 0 0) 0x7f2c64007500 con 0x7f2c740ff3c0 2024-02-16T19:34:07.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.667+0000 7f2c72ffd700 1 --2- 172.21.15.22:0/205790058 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f2c5c06d9e0 0x7f2c5c06fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:34:07.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.667+0000 7f2c72ffd700 1 -- 172.21.15.22:0/205790058 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f2c6408dad0 con 0x7f2c740ff3c0 2024-02-16T19:34:07.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.667+0000 7f2c791a6700 1 --2- 172.21.15.22:0/205790058 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f2c5c06d9e0 0x7f2c5c06fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:34:07.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.667+0000 7f2c791a6700 1 --2- 172.21.15.22:0/205790058 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f2c5c06d9e0 0x7f2c5c06fe90 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f2c6800b760 tx=0x7f2c68005940 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:34:07.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.671+0000 7f2c72ffd700 1 -- 172.21.15.22:0/205790058 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2c64057110 con 0x7f2c740ff3c0 2024-02-16T19:34:07.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:07 smithi117 ceph-mon[67870]: pgmap v28: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:34:07.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:07 smithi117 ceph-mon[67870]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:34:07.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:07 smithi117 ceph-mon[67870]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:34:07.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:07 smithi117 ceph-mon[67870]: from='client.? 172.21.15.22:0/143951883' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:07.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:07 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:07.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:07 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:07.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:07 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi117.ibmppn", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T19:34:07.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:07 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:34:07.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:07 smithi117 ceph-mon[67870]: from='client.? 172.21.15.22:0/3133059328' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:34:07.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.901+0000 7f2c7bc0b700 1 -- 172.21.15.22:0/205790058 --> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2c74102ef0 con 0x7f2c5c06d9e0 2024-02-16T19:34:07.903 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:07 smithi022 ceph-mon[88798]: pgmap v28: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:34:07.903 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:07 smithi022 ceph-mon[88798]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:34:07.903 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:07 smithi022 ceph-mon[88798]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:34:07.903 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:07 smithi022 ceph-mon[88798]: from='client.? 172.21.15.22:0/143951883' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:07.903 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:07 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:07.903 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:07 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:07.903 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:07 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi117.ibmppn", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T19:34:07.903 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:07 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:34:07.904 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:07 smithi022 ceph-mon[88798]: from='client.? 172.21.15.22:0/3133059328' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:34:07.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.902+0000 7f2c72ffd700 1 -- 172.21.15.22:0/205790058 <== mgr.34104 v2:172.21.15.22:6800/505266753 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f2c74102ef0 con 0x7f2c5c06d9e0 2024-02-16T19:34:07.904 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-02-16T19:34:07.904 INFO:teuthology.orchestra.run.smithi022.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T19:34:07.904 INFO:teuthology.orchestra.run.smithi022.stdout: "in_progress": true, 2024-02-16T19:34:07.904 INFO:teuthology.orchestra.run.smithi022.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T19:34:07.904 INFO:teuthology.orchestra.run.smithi022.stdout: "services_complete": [ 2024-02-16T19:34:07.905 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr", 2024-02-16T19:34:07.905 INFO:teuthology.orchestra.run.smithi022.stdout: "mon" 2024-02-16T19:34:07.905 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-02-16T19:34:07.905 INFO:teuthology.orchestra.run.smithi022.stdout: "progress": "4/23 daemons upgraded", 2024-02-16T19:34:07.905 INFO:teuthology.orchestra.run.smithi022.stdout: "message": "Currently upgrading mon daemons", 2024-02-16T19:34:07.905 INFO:teuthology.orchestra.run.smithi022.stdout: "is_paused": false 2024-02-16T19:34:07.905 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-02-16T19:34:07.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.905+0000 7f2c7bc0b700 1 -- 172.21.15.22:0/205790058 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f2c5c06d9e0 msgr2=0x7f2c5c06fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:07.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.905+0000 7f2c7bc0b700 1 --2- 172.21.15.22:0/205790058 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f2c5c06d9e0 0x7f2c5c06fe90 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f2c6800b760 tx=0x7f2c68005940 comp rx=0 tx=0).stop 2024-02-16T19:34:07.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.905+0000 7f2c7bc0b700 1 -- 172.21.15.22:0/205790058 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c740ff3c0 msgr2=0x7f2c74100dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:07.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.905+0000 7f2c7bc0b700 1 --2- 172.21.15.22:0/205790058 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c740ff3c0 0x7f2c74100dd0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2c6400ef10 tx=0x7f2c6400c3a0 comp rx=0 tx=0).stop 2024-02-16T19:34:07.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.905+0000 7f2c7bc0b700 1 -- 172.21.15.22:0/205790058 shutdown_connections 2024-02-16T19:34:07.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.905+0000 7f2c7bc0b700 1 --2- 172.21.15.22:0/205790058 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f2c5c06d9e0 0x7f2c5c06fe90 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:07.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.905+0000 7f2c7bc0b700 1 --2- 172.21.15.22:0/205790058 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c740ff3c0 0x7f2c74100dd0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:07.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.905+0000 7f2c7bc0b700 1 --2- 172.21.15.22:0/205790058 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c740ffd60 0x7f2c74101310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:07.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.905+0000 7f2c7bc0b700 1 -- 172.21.15.22:0/205790058 >> 172.21.15.22:0/205790058 conn(0x7f2c740fac60 msgr2=0x7f2c740fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:34:07.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.905+0000 7f2c7bc0b700 1 -- 172.21.15.22:0/205790058 shutdown_connections 2024-02-16T19:34:07.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:07.905+0000 7f2c7bc0b700 1 -- 172.21.15.22:0/205790058 wait complete. 2024-02-16T19:34:08.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:08.011+0000 7f5240806700 1 -- 172.21.15.22:0/691768977 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f523c100d70 msgr2=0x7f523c1094d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:08.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:08.011+0000 7f5240806700 1 --2- 172.21.15.22:0/691768977 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f523c100d70 0x7f523c1094d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5224009940 tx=0x7f5224009c50 comp rx=0 tx=0).stop 2024-02-16T19:34:08.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:08.011+0000 7f5240806700 1 -- 172.21.15.22:0/691768977 shutdown_connections 2024-02-16T19:34:08.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:08.011+0000 7f5240806700 1 --2- 172.21.15.22:0/691768977 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f523c100d70 0x7f523c1094d0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:08.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:08.011+0000 7f5240806700 1 --2- 172.21.15.22:0/691768977 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f523c100460 0x7f523c100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:08.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:08.011+0000 7f5240806700 1 -- 172.21.15.22:0/691768977 >> 172.21.15.22:0/691768977 conn(0x7f523c0748b0 msgr2=0x7f523c074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:34:08.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:08.011+0000 7f5240806700 1 -- 172.21.15.22:0/691768977 shutdown_connections 2024-02-16T19:34:08.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:08.011+0000 7f5240806700 1 -- 172.21.15.22:0/691768977 wait complete. 2024-02-16T19:34:08.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:08.012+0000 7f5240806700 1 Processor -- start 2024-02-16T19:34:08.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:08.012+0000 7f5240806700 1 -- start start 2024-02-16T19:34:08.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:08.012+0000 7f5240806700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f523c100460 0x7f523c195d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:34:08.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:08.012+0000 7f5240806700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f523c100d70 0x7f523c196260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:34:08.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:08.012+0000 7f5240806700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f523c196940 con 0x7f523c100d70 2024-02-16T19:34:08.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:08.012+0000 7f5240806700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f523c19a6d0 con 0x7f523c100460 2024-02-16T19:34:08.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:08.012+0000 7f5239d9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f523c100d70 0x7f523c196260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:34:08.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:08.012+0000 7f5239d9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f523c100d70 0x7f523c196260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50046/0 (socket says 172.21.15.22:50046) 2024-02-16T19:34:08.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:08.012+0000 7f5239d9b700 1 -- 172.21.15.22:0/3186431010 learned_addr learned my addr 172.21.15.22:0/3186431010 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:34:08.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:08.012+0000 7f523a59c700 1 --2- 172.21.15.22:0/3186431010 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f523c100460 0x7f523c195d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:34:08.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:08.012+0000 7f5239d9b700 1 -- 172.21.15.22:0/3186431010 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f523c100460 msgr2=0x7f523c195d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:08.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:08.012+0000 7f5239d9b700 1 --2- 172.21.15.22:0/3186431010 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f523c100460 0x7f523c195d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:08.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:08.012+0000 7f5239d9b700 1 -- 172.21.15.22:0/3186431010 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52240095f0 con 0x7f523c100d70 2024-02-16T19:34:08.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:08.012+0000 7f523a59c700 1 --2- 172.21.15.22:0/3186431010 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f523c100460 0x7f523c195d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-16T19:34:08.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:08.013+0000 7f5239d9b700 1 --2- 172.21.15.22:0/3186431010 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f523c100d70 0x7f523c196260 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f52240055f0 tx=0x7f52240056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:34:08.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:08.013+0000 7f52337fe700 1 -- 172.21.15.22:0/3186431010 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f522401d070 con 0x7f523c100d70 2024-02-16T19:34:08.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:08.013+0000 7f5240806700 1 -- 172.21.15.22:0/3186431010 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f523c19a9b0 con 0x7f523c100d70 2024-02-16T19:34:08.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:08.013+0000 7f52337fe700 1 -- 172.21.15.22:0/3186431010 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f522400f460 con 0x7f523c100d70 2024-02-16T19:34:08.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:08.013+0000 7f52337fe700 1 -- 172.21.15.22:0/3186431010 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5224021640 con 0x7f523c100d70 2024-02-16T19:34:08.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:08.013+0000 7f5240806700 1 -- 172.21.15.22:0/3186431010 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f523c19af00 con 0x7f523c100d70 2024-02-16T19:34:08.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:08.014+0000 7f5240806700 1 -- 172.21.15.22:0/3186431010 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f523c04f2a0 con 0x7f523c100d70 2024-02-16T19:34:08.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:08.014+0000 7f52337fe700 1 -- 172.21.15.22:0/3186431010 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 39) v1 ==== 91595+0+0 (secure 0 0 0) 0x7f522400fab0 con 0x7f523c100d70 2024-02-16T19:34:08.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:08.015+0000 7f52337fe700 1 --2- 172.21.15.22:0/3186431010 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f522806d9e0 0x7f522806fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:34:08.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:08.015+0000 7f52337fe700 1 -- 172.21.15.22:0/3186431010 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f522408e990 con 0x7f523c100d70 2024-02-16T19:34:08.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:08.015+0000 7f523a59c700 1 --2- 172.21.15.22:0/3186431010 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f522806d9e0 0x7f522806fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:34:08.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:08.015+0000 7f523a59c700 1 --2- 172.21.15.22:0/3186431010 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f522806d9e0 0x7f522806fe90 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f522c009690 tx=0x7f522c009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:34:08.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:08.019+0000 7f52337fe700 1 -- 172.21.15.22:0/3186431010 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5224057fd0 con 0x7f523c100d70 2024-02-16T19:34:08.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:08.313+0000 7f5240806700 1 -- 172.21.15.22:0/3186431010 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f523c19b1c0 con 0x7f523c100d70 2024-02-16T19:34:08.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:08.314+0000 7f52337fe700 1 -- 172.21.15.22:0/3186431010 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f522405b690 con 0x7f523c100d70 2024-02-16T19:34:08.315 INFO:teuthology.orchestra.run.smithi022.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-02-16T19:34:08.315 INFO:teuthology.orchestra.run.smithi022.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-02-16T19:34:08.316 INFO:teuthology.orchestra.run.smithi022.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-02-16T19:34:08.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:08.316+0000 7f5240806700 1 -- 172.21.15.22:0/3186431010 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f522806d9e0 msgr2=0x7f522806fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:08.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:08.316+0000 7f5240806700 1 --2- 172.21.15.22:0/3186431010 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f522806d9e0 0x7f522806fe90 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f522c009690 tx=0x7f522c009040 comp rx=0 tx=0).stop 2024-02-16T19:34:08.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:08.316+0000 7f5240806700 1 -- 172.21.15.22:0/3186431010 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f523c100d70 msgr2=0x7f523c196260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:08.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:08.316+0000 7f5240806700 1 --2- 172.21.15.22:0/3186431010 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f523c100d70 0x7f523c196260 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f52240055f0 tx=0x7f52240056d0 comp rx=0 tx=0).stop 2024-02-16T19:34:08.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:08.316+0000 7f5240806700 1 -- 172.21.15.22:0/3186431010 shutdown_connections 2024-02-16T19:34:08.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:08.316+0000 7f5240806700 1 --2- 172.21.15.22:0/3186431010 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f522806d9e0 0x7f522806fe90 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:08.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:08.316+0000 7f5240806700 1 --2- 172.21.15.22:0/3186431010 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f523c100460 0x7f523c195d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:08.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:08.316+0000 7f5240806700 1 --2- 172.21.15.22:0/3186431010 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f523c100d70 0x7f523c196260 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:08.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:08.316+0000 7f5240806700 1 -- 172.21.15.22:0/3186431010 >> 172.21.15.22:0/3186431010 conn(0x7f523c0748b0 msgr2=0x7f523c0febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:34:08.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:08.316+0000 7f5240806700 1 -- 172.21.15.22:0/3186431010 shutdown_connections 2024-02-16T19:34:08.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:08.316+0000 7f5240806700 1 -- 172.21.15.22:0/3186431010 wait complete. 2024-02-16T19:34:08.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:08 smithi022 ceph-mon[88798]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:34:08.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:08 smithi022 ceph-mon[88798]: Reconfiguring mds.cephfs.smithi117.ibmppn (monmap changed)... 2024-02-16T19:34:08.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:08 smithi022 ceph-mon[88798]: Reconfiguring daemon mds.cephfs.smithi117.ibmppn on smithi117 2024-02-16T19:34:08.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:08 smithi022 ceph-mon[88798]: from='client.? 172.21.15.22:0/3186431010' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T19:34:09.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:08 smithi117 ceph-mon[67870]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:34:09.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:08 smithi117 ceph-mon[67870]: Reconfiguring mds.cephfs.smithi117.ibmppn (monmap changed)... 2024-02-16T19:34:09.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:08 smithi117 ceph-mon[67870]: Reconfiguring daemon mds.cephfs.smithi117.ibmppn on smithi117 2024-02-16T19:34:09.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:08 smithi117 ceph-mon[67870]: from='client.? 172.21.15.22:0/3186431010' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T19:34:09.843 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:09 smithi117 ceph-mon[67870]: pgmap v29: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:34:09.843 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:09 smithi117 ceph-mon[67870]: from='client.44119 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:34:09.843 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:09 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:09.843 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:09 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:09.843 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:09 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi117.orzors", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T19:34:09.843 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:09 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:34:09.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:09 smithi022 ceph-mon[88798]: pgmap v29: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:34:09.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:09 smithi022 ceph-mon[88798]: from='client.44119 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:34:09.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:09 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:09.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:09 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:09.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:09 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi117.orzors", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T19:34:09.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:09 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:34:10.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:10 smithi117 ceph-mon[67870]: Reconfiguring mds.cephfs.smithi117.orzors (monmap changed)... 2024-02-16T19:34:10.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:10 smithi117 ceph-mon[67870]: Reconfiguring daemon mds.cephfs.smithi117.orzors on smithi117 2024-02-16T19:34:10.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:10 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:10.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:10 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:10.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:10 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:34:10.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:10 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:10.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:10 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:10.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:10 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:10.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:10 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi022"}]: dispatch 2024-02-16T19:34:10.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:10 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi022"}]': finished 2024-02-16T19:34:10.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:10 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi117"}]: dispatch 2024-02-16T19:34:10.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:10 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi117"}]': finished 2024-02-16T19:34:10.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:10 smithi022 ceph-mon[88798]: Reconfiguring mds.cephfs.smithi117.orzors (monmap changed)... 2024-02-16T19:34:10.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:10 smithi022 ceph-mon[88798]: Reconfiguring daemon mds.cephfs.smithi117.orzors on smithi117 2024-02-16T19:34:10.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:10 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:10.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:10 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:10.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:10 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:34:10.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:10 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:10.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:10 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:10.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:10 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:10.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:10 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi022"}]: dispatch 2024-02-16T19:34:10.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:10 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi022"}]': finished 2024-02-16T19:34:10.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:10 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi117"}]: dispatch 2024-02-16T19:34:10.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:10 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi117"}]': finished 2024-02-16T19:34:11.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:11 smithi022 ceph-mon[88798]: pgmap v30: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:34:11.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:11 smithi022 ceph-mon[88798]: Upgrade: Setting container_image for all mon 2024-02-16T19:34:12.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:11 smithi117 ceph-mon[67870]: pgmap v30: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:34:12.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:11 smithi117 ceph-mon[67870]: Upgrade: Setting container_image for all mon 2024-02-16T19:34:13.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:13 smithi022 ceph-mon[88798]: pgmap v31: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:34:13.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:13 smithi022 ceph-mon[88798]: Upgrade: Updating crash.smithi022 (1/2) 2024-02-16T19:34:13.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:13 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:13.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:13 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi022", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-16T19:34:13.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:13 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:34:13.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:13 smithi022 ceph-mon[88798]: Deploying daemon crash.smithi022 on smithi022 2024-02-16T19:34:13.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:13 smithi117 ceph-mon[67870]: pgmap v31: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:34:13.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:13 smithi117 ceph-mon[67870]: Upgrade: Updating crash.smithi022 (1/2) 2024-02-16T19:34:13.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:13 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:13.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:13 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi022", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-16T19:34:13.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:13 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:34:13.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:13 smithi117 ceph-mon[67870]: Deploying daemon crash.smithi022 on smithi022 2024-02-16T19:34:15.448 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:15 smithi022 ceph-mon[88798]: pgmap v32: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:34:15.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:15 smithi117 ceph-mon[67870]: pgmap v32: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:34:16.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:16 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:34:16.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:16 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:34:17.580 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:17 smithi022 ceph-mon[88798]: pgmap v33: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:34:17.580 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:17 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:17.581 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:17 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:17.595 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:17 smithi117 ceph-mon[67870]: pgmap v33: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:34:17.595 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:17 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:17.595 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:17 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:19.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:18 smithi117 ceph-mon[67870]: pgmap v34: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:34:19.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:18 smithi117 ceph-mon[67870]: Upgrade: Updating crash.smithi117 (2/2) 2024-02-16T19:34:19.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:18 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:19.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:18 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi117", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-16T19:34:19.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:18 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:34:19.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:18 smithi117 ceph-mon[67870]: Deploying daemon crash.smithi117 on smithi117 2024-02-16T19:34:19.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:18 smithi022 ceph-mon[88798]: pgmap v34: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:34:19.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:18 smithi022 ceph-mon[88798]: Upgrade: Updating crash.smithi117 (2/2) 2024-02-16T19:34:19.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:18 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:19.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:18 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi117", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-16T19:34:19.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:18 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:34:19.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:18 smithi022 ceph-mon[88798]: Deploying daemon crash.smithi117 on smithi117 2024-02-16T19:34:21.250 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:20 smithi117 ceph-mon[67870]: pgmap v35: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:34:21.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:20 smithi022 ceph-mon[88798]: pgmap v35: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:34:22.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:22 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:22.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:22 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:22.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:22 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:34:22.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:22 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:22.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:22 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:22.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:22 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:34:23.877 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:23 smithi022 ceph-mon[88798]: pgmap v36: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:34:23.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:23 smithi117 ceph-mon[67870]: pgmap v36: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:34:25.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:25 smithi117 ceph-mon[67870]: pgmap v37: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:34:25.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:25 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:25.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:25 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:25.802 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:25 smithi022 ceph-mon[88798]: pgmap v37: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:34:25.802 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:25 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:25.802 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:25 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:26.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:26 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:26.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:26 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:26.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:26 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:26.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:26 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:26.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:26 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:26.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:26 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:26.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:26 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:26.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:26 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:27.698 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:27 smithi117 ceph-mon[67870]: pgmap v38: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:34:27.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:27 smithi022 ceph-mon[88798]: pgmap v38: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:34:28.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:28 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:28.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:28 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:28.964 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:28 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:28.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:28 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:29.856 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:29 smithi022 ceph-mon[88798]: pgmap v39: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:34:29.856 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:29 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:29.856 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:29 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:29.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:29 smithi117 ceph-mon[67870]: pgmap v39: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:34:29.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:29 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:29.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:29 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:31.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:31 smithi117 ceph-mon[67870]: pgmap v40: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:34:31.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:31 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:34:31.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:31 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:31.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:31 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:31.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:31 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:34:31.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:31 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:34:31.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:31 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:31.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:31 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:34:31.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:31 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:31.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:31 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:31.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:31 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:31.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:31 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:31.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:31 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi022"}]: dispatch 2024-02-16T19:34:31.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:31 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi022"}]': finished 2024-02-16T19:34:31.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:31 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi117"}]: dispatch 2024-02-16T19:34:31.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:31 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi117"}]': finished 2024-02-16T19:34:31.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:31 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-02-16T19:34:31.920 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:31 smithi022 ceph-mon[88798]: pgmap v40: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:34:31.920 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:31 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:34:31.920 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:31 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:31.920 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:31 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:31.920 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:31 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:34:31.920 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:31 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:34:31.920 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:31 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:31.920 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:31 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:34:31.921 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:31 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:31.921 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:31 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:31.921 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:31 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:31.921 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:31 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:31.921 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:31 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi022"}]: dispatch 2024-02-16T19:34:31.921 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:31 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi022"}]': finished 2024-02-16T19:34:31.921 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:31 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi117"}]: dispatch 2024-02-16T19:34:31.921 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:31 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi117"}]': finished 2024-02-16T19:34:31.921 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:31 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-02-16T19:34:32.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:32 smithi117 ceph-mon[67870]: Upgrade: Setting container_image for all crash 2024-02-16T19:34:32.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:32 smithi117 ceph-mon[67870]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-02-16T19:34:32.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:32 smithi117 ceph-mon[67870]: Upgrade: osd.0 is safe to restart 2024-02-16T19:34:32.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:32 smithi022 ceph-mon[88798]: Upgrade: Setting container_image for all crash 2024-02-16T19:34:32.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:32 smithi022 ceph-mon[88798]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-02-16T19:34:32.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:32 smithi022 ceph-mon[88798]: Upgrade: osd.0 is safe to restart 2024-02-16T19:34:33.905 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:33 smithi022 ceph-mon[88798]: pgmap v41: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:34:33.906 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:33 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:33.906 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:33 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-16T19:34:33.906 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:33 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:34:34.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:33 smithi117 ceph-mon[67870]: pgmap v41: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:34:34.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:33 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:34.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:33 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-16T19:34:34.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:33 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:34:34.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:34 smithi022 ceph-mon[88798]: Upgrade: Updating osd.0 2024-02-16T19:34:34.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:34 smithi022 ceph-mon[88798]: Deploying daemon osd.0 on smithi022 2024-02-16T19:34:35.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:34 smithi117 ceph-mon[67870]: Upgrade: Updating osd.0 2024-02-16T19:34:35.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:34 smithi117 ceph-mon[67870]: Deploying daemon osd.0 on smithi022 2024-02-16T19:34:35.867 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:35 smithi022 ceph-mon[88798]: pgmap v42: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:34:36.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:35 smithi117 ceph-mon[67870]: pgmap v42: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:34:36.823 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 16 19:34:36 smithi022 systemd[1]: Stopping Ceph osd.0 for 2a1346e6-cd00-11ee-95ba-87774f69a715... 2024-02-16T19:34:37.215 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 16 19:34:36 smithi022 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-0[44351]: 2024-02-16T19:34:36.821+0000 7f5d15c1e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-16T19:34:37.215 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 16 19:34:36 smithi022 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-0[44351]: 2024-02-16T19:34:36.821+0000 7f5d15c1e700 -1 osd.0 50 *** Got signal Terminated *** 2024-02-16T19:34:37.215 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 16 19:34:36 smithi022 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-0[44351]: 2024-02-16T19:34:36.821+0000 7f5d15c1e700 -1 osd.0 50 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-16T19:34:37.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:37 smithi022 ceph-mon[88798]: pgmap v43: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:34:37.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:37 smithi022 ceph-mon[88798]: osd.0 marked itself down and dead 2024-02-16T19:34:38.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:37 smithi117 ceph-mon[67870]: pgmap v43: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:34:38.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:37 smithi117 ceph-mon[67870]: osd.0 marked itself down and dead 2024-02-16T19:34:38.439 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 16 19:34:38 smithi022 podman[99920]: 2024-02-16 19:34:38.182387192 +0000 UTC m=+1.506463175 container died 0027f86f16ce5b3f461d53e4eb4ee4250f2c4fbbb9e2c53fae66bb56894eadb6 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-0, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, maintainer=Guillaume Abrioux , GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=-18.2.0, io.buildah.version=1.29.1, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20231212) 2024-02-16T19:34:38.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.437+0000 7f44563ee700 1 -- 172.21.15.22:0/1678056485 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f44501004e0 msgr2=0x7f44501008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:38.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.437+0000 7f44563ee700 1 --2- 172.21.15.22:0/1678056485 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f44501004e0 0x7f44501008b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4438009910 tx=0x7f4438009c20 comp rx=0 tx=0).stop 2024-02-16T19:34:38.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.437+0000 7f44563ee700 1 -- 172.21.15.22:0/1678056485 shutdown_connections 2024-02-16T19:34:38.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.437+0000 7f44563ee700 1 --2- 172.21.15.22:0/1678056485 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4450100e80 0x7f4450104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:38.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.437+0000 7f44563ee700 1 --2- 172.21.15.22:0/1678056485 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f44501004e0 0x7f44501008b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:38.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.437+0000 7f44563ee700 1 -- 172.21.15.22:0/1678056485 >> 172.21.15.22:0/1678056485 conn(0x7f44500fbda0 msgr2=0x7f44500fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:34:38.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.437+0000 7f44563ee700 1 -- 172.21.15.22:0/1678056485 shutdown_connections 2024-02-16T19:34:38.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.437+0000 7f44563ee700 1 -- 172.21.15.22:0/1678056485 wait complete. 2024-02-16T19:34:38.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.438+0000 7f44563ee700 1 Processor -- start 2024-02-16T19:34:38.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.438+0000 7f44563ee700 1 -- start start 2024-02-16T19:34:38.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.438+0000 7f44563ee700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44501004e0 0x7f44501980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:34:38.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.438+0000 7f44563ee700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4450100e80 0x7f44501985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:34:38.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.438+0000 7f44563ee700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4450198cc0 con 0x7f4450100e80 2024-02-16T19:34:38.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.438+0000 7f44563ee700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f445019ca50 con 0x7f44501004e0 2024-02-16T19:34:38.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.438+0000 7f444f7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4450100e80 0x7f44501985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:34:38.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.438+0000 7f444f7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4450100e80 0x7f44501985e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48046/0 (socket says 172.21.15.22:48046) 2024-02-16T19:34:38.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.438+0000 7f444f7fe700 1 -- 172.21.15.22:0/3122090593 learned_addr learned my addr 172.21.15.22:0/3122090593 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:34:38.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.439+0000 7f444ffff700 1 --2- 172.21.15.22:0/3122090593 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44501004e0 0x7f44501980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:34:38.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.439+0000 7f444f7fe700 1 -- 172.21.15.22:0/3122090593 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44501004e0 msgr2=0x7f44501980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:38.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.439+0000 7f444f7fe700 1 --2- 172.21.15.22:0/3122090593 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44501004e0 0x7f44501980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:38.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.439+0000 7f444f7fe700 1 -- 172.21.15.22:0/3122090593 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44380095f0 con 0x7f4450100e80 2024-02-16T19:34:38.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.439+0000 7f444ffff700 1 --2- 172.21.15.22:0/3122090593 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44501004e0 0x7f44501980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T19:34:38.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.439+0000 7f444f7fe700 1 --2- 172.21.15.22:0/3122090593 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4450100e80 0x7f44501985e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f444000d8d0 tx=0x7f444000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:34:38.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.439+0000 7f444d7fa700 1 -- 172.21.15.22:0/3122090593 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f44400041f0 con 0x7f4450100e80 2024-02-16T19:34:38.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.439+0000 7f444d7fa700 1 -- 172.21.15.22:0/3122090593 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4440007620 con 0x7f4450100e80 2024-02-16T19:34:38.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.439+0000 7f44563ee700 1 -- 172.21.15.22:0/3122090593 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f445019cd30 con 0x7f4450100e80 2024-02-16T19:34:38.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.439+0000 7f444d7fa700 1 -- 172.21.15.22:0/3122090593 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4440003c50 con 0x7f4450100e80 2024-02-16T19:34:38.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.439+0000 7f44563ee700 1 -- 172.21.15.22:0/3122090593 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f445019d280 con 0x7f4450100e80 2024-02-16T19:34:38.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.440+0000 7f44563ee700 1 -- 172.21.15.22:0/3122090593 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44501045d0 con 0x7f4450100e80 2024-02-16T19:34:38.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.441+0000 7f444d7fa700 1 -- 172.21.15.22:0/3122090593 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 39) v1 ==== 91595+0+0 (secure 0 0 0) 0x7f4440004350 con 0x7f4450100e80 2024-02-16T19:34:38.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.441+0000 7f444d7fa700 1 --2- 172.21.15.22:0/3122090593 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f443c06d9d0 0x7f443c06fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:34:38.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.441+0000 7f444d7fa700 1 -- 172.21.15.22:0/3122090593 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f444008d210 con 0x7f4450100e80 2024-02-16T19:34:38.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.441+0000 7f444ffff700 1 --2- 172.21.15.22:0/3122090593 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f443c06d9d0 0x7f443c06fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:34:38.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.442+0000 7f444ffff700 1 --2- 172.21.15.22:0/3122090593 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f443c06d9d0 0x7f443c06fe80 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f443800b5c0 tx=0x7f443801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:34:38.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.447+0000 7f444d7fa700 1 -- 172.21.15.22:0/3122090593 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f44400568d0 con 0x7f4450100e80 2024-02-16T19:34:38.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.693+0000 7f44563ee700 1 -- 172.21.15.22:0/3122090593 --> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f445019d560 con 0x7f443c06d9d0 2024-02-16T19:34:38.695 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:38 smithi022 ceph-mon[88798]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-16T19:34:38.695 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:38 smithi022 ceph-mon[88798]: osdmap e51: 6 total, 5 up, 6 in 2024-02-16T19:34:38.695 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:38 smithi022 ceph-mon[88798]: osdmap e52: 6 total, 5 up, 6 in 2024-02-16T19:34:38.696 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 16 19:34:38 smithi022 podman[99920]: 2024-02-16 19:34:38.630649739 +0000 UTC m=+1.954725723 container cleanup 0027f86f16ce5b3f461d53e4eb4ee4250f2c4fbbb9e2c53fae66bb56894eadb6 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-0, RELEASE=HEAD, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20231212, CEPH_POINT_RELEASE=-18.2.0, GIT_REPO=https://github.com/ceph/ceph-container.git, io.buildah.version=1.29.1, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308) 2024-02-16T19:34:38.696 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 16 19:34:38 smithi022 bash[99920]: ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-0 2024-02-16T19:34:38.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.695+0000 7f444d7fa700 1 -- 172.21.15.22:0/3122090593 <== mgr.34104 v2:172.21.15.22:6800/505266753 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f445019d560 con 0x7f443c06d9d0 2024-02-16T19:34:38.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.698+0000 7f44563ee700 1 -- 172.21.15.22:0/3122090593 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f443c06d9d0 msgr2=0x7f443c06fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:38.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.698+0000 7f44563ee700 1 --2- 172.21.15.22:0/3122090593 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f443c06d9d0 0x7f443c06fe80 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f443800b5c0 tx=0x7f443801a040 comp rx=0 tx=0).stop 2024-02-16T19:34:38.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.698+0000 7f44563ee700 1 -- 172.21.15.22:0/3122090593 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4450100e80 msgr2=0x7f44501985e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:38.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.698+0000 7f44563ee700 1 --2- 172.21.15.22:0/3122090593 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4450100e80 0x7f44501985e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f444000d8d0 tx=0x7f444000dc90 comp rx=0 tx=0).stop 2024-02-16T19:34:38.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.698+0000 7f44563ee700 1 -- 172.21.15.22:0/3122090593 shutdown_connections 2024-02-16T19:34:38.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.698+0000 7f44563ee700 1 --2- 172.21.15.22:0/3122090593 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f443c06d9d0 0x7f443c06fe80 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:38.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.698+0000 7f44563ee700 1 --2- 172.21.15.22:0/3122090593 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44501004e0 0x7f44501980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:38.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.698+0000 7f44563ee700 1 --2- 172.21.15.22:0/3122090593 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4450100e80 0x7f44501985e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:38.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.698+0000 7f44563ee700 1 -- 172.21.15.22:0/3122090593 >> 172.21.15.22:0/3122090593 conn(0x7f44500fbda0 msgr2=0x7f44500fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:34:38.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.698+0000 7f44563ee700 1 -- 172.21.15.22:0/3122090593 shutdown_connections 2024-02-16T19:34:38.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.698+0000 7f44563ee700 1 -- 172.21.15.22:0/3122090593 wait complete. 2024-02-16T19:34:38.711 INFO:teuthology.orchestra.run.smithi022.stdout:true 2024-02-16T19:34:38.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.808+0000 7f7d79f19700 1 -- 172.21.15.22:0/3055104490 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d740ffd60 msgr2=0x7f7d74103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:38.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.808+0000 7f7d79f19700 1 --2- 172.21.15.22:0/3055104490 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d740ffd60 0x7f7d74103d30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c009910 tx=0x7f7d5c009c20 comp rx=0 tx=0).stop 2024-02-16T19:34:38.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.808+0000 7f7d79f19700 1 -- 172.21.15.22:0/3055104490 shutdown_connections 2024-02-16T19:34:38.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.808+0000 7f7d79f19700 1 --2- 172.21.15.22:0/3055104490 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d740ffd60 0x7f7d74103d30 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:38.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.808+0000 7f7d79f19700 1 --2- 172.21.15.22:0/3055104490 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d740ff3c0 0x7f7d740ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:38.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.808+0000 7f7d79f19700 1 -- 172.21.15.22:0/3055104490 >> 172.21.15.22:0/3055104490 conn(0x7f7d740fac80 msgr2=0x7f7d740fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:34:38.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.808+0000 7f7d79f19700 1 -- 172.21.15.22:0/3055104490 shutdown_connections 2024-02-16T19:34:38.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.808+0000 7f7d79f19700 1 -- 172.21.15.22:0/3055104490 wait complete. 2024-02-16T19:34:38.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.809+0000 7f7d79f19700 1 Processor -- start 2024-02-16T19:34:38.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.809+0000 7f7d79f19700 1 -- start start 2024-02-16T19:34:38.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.809+0000 7f7d79f19700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d740ff3c0 0x7f7d74194ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:34:38.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.809+0000 7f7d79f19700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d740ffd60 0x7f7d74195420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:34:38.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.809+0000 7f7d79f19700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d74195b00 con 0x7f7d740ff3c0 2024-02-16T19:34:38.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.809+0000 7f7d79f19700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d74199890 con 0x7f7d740ffd60 2024-02-16T19:34:38.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.809+0000 7f7d737fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d740ff3c0 0x7f7d74194ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:34:38.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.809+0000 7f7d737fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d740ff3c0 0x7f7d74194ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48068/0 (socket says 172.21.15.22:48068) 2024-02-16T19:34:38.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.809+0000 7f7d737fe700 1 -- 172.21.15.22:0/2612854987 learned_addr learned my addr 172.21.15.22:0/2612854987 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:34:38.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.809+0000 7f7d6bfff700 1 --2- 172.21.15.22:0/2612854987 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d740ffd60 0x7f7d74195420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:34:38.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.809+0000 7f7d737fe700 1 -- 172.21.15.22:0/2612854987 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d740ffd60 msgr2=0x7f7d74195420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:38.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.809+0000 7f7d737fe700 1 --2- 172.21.15.22:0/2612854987 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d740ffd60 0x7f7d74195420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:38.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.809+0000 7f7d737fe700 1 -- 172.21.15.22:0/2612854987 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d5c0095f0 con 0x7f7d740ff3c0 2024-02-16T19:34:38.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.810+0000 7f7d737fe700 1 --2- 172.21.15.22:0/2612854987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d740ff3c0 0x7f7d74194ee0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7d6400b4b0 tx=0x7f7d6400b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:34:38.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.810+0000 7f7d717fa700 1 -- 172.21.15.22:0/2612854987 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d64011800 con 0x7f7d740ff3c0 2024-02-16T19:34:38.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.810+0000 7f7d717fa700 1 -- 172.21.15.22:0/2612854987 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7d64011e20 con 0x7f7d740ff3c0 2024-02-16T19:34:38.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.810+0000 7f7d79f19700 1 -- 172.21.15.22:0/2612854987 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d74199b70 con 0x7f7d740ff3c0 2024-02-16T19:34:38.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.810+0000 7f7d717fa700 1 -- 172.21.15.22:0/2612854987 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d6400f2e0 con 0x7f7d740ff3c0 2024-02-16T19:34:38.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.810+0000 7f7d79f19700 1 -- 172.21.15.22:0/2612854987 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d7419a090 con 0x7f7d740ff3c0 2024-02-16T19:34:38.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.811+0000 7f7d79f19700 1 -- 172.21.15.22:0/2612854987 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d74107660 con 0x7f7d740ff3c0 2024-02-16T19:34:38.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.811+0000 7f7d717fa700 1 -- 172.21.15.22:0/2612854987 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 39) v1 ==== 91595+0+0 (secure 0 0 0) 0x7f7d64010380 con 0x7f7d740ff3c0 2024-02-16T19:34:38.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.812+0000 7f7d717fa700 1 --2- 172.21.15.22:0/2612854987 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f7d6006d9d0 0x7f7d6006fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:34:38.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.812+0000 7f7d717fa700 1 -- 172.21.15.22:0/2612854987 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f7d6408d110 con 0x7f7d740ff3c0 2024-02-16T19:34:38.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.812+0000 7f7d6bfff700 1 --2- 172.21.15.22:0/2612854987 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f7d6006d9d0 0x7f7d6006fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:34:38.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.812+0000 7f7d6bfff700 1 --2- 172.21.15.22:0/2612854987 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f7d6006d9d0 0x7f7d6006fe80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c006010 tx=0x7f7d5c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:34:38.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:38.817+0000 7f7d717fa700 1 -- 172.21.15.22:0/2612854987 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7d64056860 con 0x7f7d740ff3c0 2024-02-16T19:34:38.965 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 16 19:34:38 smithi022 podman[99937]: 2024-02-16 19:34:38.924128854 +0000 UTC m=+0.739757218 container remove 0027f86f16ce5b3f461d53e4eb4ee4250f2c4fbbb9e2c53fae66bb56894eadb6 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-0, io.buildah.version=1.29.1, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=https://github.com/ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, RELEASE=HEAD, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.build-date=20231212, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-16T19:34:39.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.085+0000 7f7d79f19700 1 -- 172.21.15.22:0/2612854987 --> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7d7419a360 con 0x7f7d6006d9d0 2024-02-16T19:34:39.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.086+0000 7f7d717fa700 1 -- 172.21.15.22:0/2612854987 <== mgr.34104 v2:172.21.15.22:6800/505266753 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f7d7419a360 con 0x7f7d6006d9d0 2024-02-16T19:34:39.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.089+0000 7f7d79f19700 1 -- 172.21.15.22:0/2612854987 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f7d6006d9d0 msgr2=0x7f7d6006fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:39.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.089+0000 7f7d79f19700 1 --2- 172.21.15.22:0/2612854987 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f7d6006d9d0 0x7f7d6006fe80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c006010 tx=0x7f7d5c01a040 comp rx=0 tx=0).stop 2024-02-16T19:34:39.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.089+0000 7f7d79f19700 1 -- 172.21.15.22:0/2612854987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d740ff3c0 msgr2=0x7f7d74194ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:39.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.089+0000 7f7d79f19700 1 --2- 172.21.15.22:0/2612854987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d740ff3c0 0x7f7d74194ee0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7d6400b4b0 tx=0x7f7d6400b7c0 comp rx=0 tx=0).stop 2024-02-16T19:34:39.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.089+0000 7f7d79f19700 1 -- 172.21.15.22:0/2612854987 shutdown_connections 2024-02-16T19:34:39.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.089+0000 7f7d79f19700 1 --2- 172.21.15.22:0/2612854987 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f7d6006d9d0 0x7f7d6006fe80 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:39.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.089+0000 7f7d79f19700 1 --2- 172.21.15.22:0/2612854987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d740ff3c0 0x7f7d74194ee0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:39.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.089+0000 7f7d79f19700 1 --2- 172.21.15.22:0/2612854987 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d740ffd60 0x7f7d74195420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:39.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.089+0000 7f7d79f19700 1 -- 172.21.15.22:0/2612854987 >> 172.21.15.22:0/2612854987 conn(0x7f7d740fac80 msgr2=0x7f7d740fc440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:34:39.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.089+0000 7f7d79f19700 1 -- 172.21.15.22:0/2612854987 shutdown_connections 2024-02-16T19:34:39.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.089+0000 7f7d79f19700 1 -- 172.21.15.22:0/2612854987 wait complete. 2024-02-16T19:34:39.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:38 smithi117 ceph-mon[67870]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-16T19:34:39.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:38 smithi117 ceph-mon[67870]: osdmap e51: 6 total, 5 up, 6 in 2024-02-16T19:34:39.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:38 smithi117 ceph-mon[67870]: osdmap e52: 6 total, 5 up, 6 in 2024-02-16T19:34:39.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.212+0000 7f4bb77b0700 1 -- 172.21.15.22:0/1826577324 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4bb0100d70 msgr2=0x7f4bb01094d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:39.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.212+0000 7f4bb77b0700 1 --2- 172.21.15.22:0/1826577324 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4bb0100d70 0x7f4bb01094d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4bac009910 tx=0x7f4bac009c20 comp rx=0 tx=0).stop 2024-02-16T19:34:39.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.212+0000 7f4bb77b0700 1 -- 172.21.15.22:0/1826577324 shutdown_connections 2024-02-16T19:34:39.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.212+0000 7f4bb77b0700 1 --2- 172.21.15.22:0/1826577324 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4bb0100d70 0x7f4bb01094d0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:39.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.212+0000 7f4bb77b0700 1 --2- 172.21.15.22:0/1826577324 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4bb0100460 0x7f4bb0100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:39.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.212+0000 7f4bb77b0700 1 -- 172.21.15.22:0/1826577324 >> 172.21.15.22:0/1826577324 conn(0x7f4bb00748b0 msgr2=0x7f4bb0074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:34:39.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.212+0000 7f4bb77b0700 1 -- 172.21.15.22:0/1826577324 shutdown_connections 2024-02-16T19:34:39.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.212+0000 7f4bb77b0700 1 -- 172.21.15.22:0/1826577324 wait complete. 2024-02-16T19:34:39.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.213+0000 7f4bb77b0700 1 Processor -- start 2024-02-16T19:34:39.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.213+0000 7f4bb77b0700 1 -- start start 2024-02-16T19:34:39.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.213+0000 7f4bb77b0700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4bb0100460 0x7f4bb0101540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:34:39.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.213+0000 7f4bb77b0700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4bb0100d70 0x7f4bb0101a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:34:39.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.213+0000 7f4bb77b0700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb0101fc0 con 0x7f4bb0100460 2024-02-16T19:34:39.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.213+0000 7f4bb77b0700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb0102100 con 0x7f4bb0100d70 2024-02-16T19:34:39.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.213+0000 7f4bb554c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4bb0100460 0x7f4bb0101540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:34:39.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.213+0000 7f4bb554c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4bb0100460 0x7f4bb0101540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48094/0 (socket says 172.21.15.22:48094) 2024-02-16T19:34:39.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.213+0000 7f4bb554c700 1 -- 172.21.15.22:0/270420481 learned_addr learned my addr 172.21.15.22:0/270420481 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:34:39.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.213+0000 7f4bb4d4b700 1 --2- 172.21.15.22:0/270420481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4bb0100d70 0x7f4bb0101a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:34:39.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.214+0000 7f4bb554c700 1 -- 172.21.15.22:0/270420481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4bb0100d70 msgr2=0x7f4bb0101a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:39.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.214+0000 7f4bb554c700 1 --2- 172.21.15.22:0/270420481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4bb0100d70 0x7f4bb0101a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:39.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.214+0000 7f4bb554c700 1 -- 172.21.15.22:0/270420481 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bac0095f0 con 0x7f4bb0100460 2024-02-16T19:34:39.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.214+0000 7f4bb554c700 1 --2- 172.21.15.22:0/270420481 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4bb0100460 0x7f4bb0101540 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4ba000e920 tx=0x7f4ba000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:34:39.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.214+0000 7f4ba67fc700 1 -- 172.21.15.22:0/270420481 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ba000c9d0 con 0x7f4bb0100460 2024-02-16T19:34:39.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.214+0000 7f4ba67fc700 1 -- 172.21.15.22:0/270420481 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4ba0010430 con 0x7f4bb0100460 2024-02-16T19:34:39.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.214+0000 7f4bb77b0700 1 -- 172.21.15.22:0/270420481 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bb01a1fa0 con 0x7f4bb0100460 2024-02-16T19:34:39.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.214+0000 7f4ba67fc700 1 -- 172.21.15.22:0/270420481 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ba0018630 con 0x7f4bb0100460 2024-02-16T19:34:39.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.214+0000 7f4bb77b0700 1 -- 172.21.15.22:0/270420481 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bb01a23b0 con 0x7f4bb0100460 2024-02-16T19:34:39.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.215+0000 7f4bb77b0700 1 -- 172.21.15.22:0/270420481 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bb0107e40 con 0x7f4bb0100460 2024-02-16T19:34:39.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.216+0000 7f4ba67fc700 1 -- 172.21.15.22:0/270420481 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 39) v1 ==== 91595+0+0 (secure 0 0 0) 0x7f4ba00105a0 con 0x7f4bb0100460 2024-02-16T19:34:39.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.216+0000 7f4ba67fc700 1 --2- 172.21.15.22:0/270420481 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f4b9c06d9d0 0x7f4b9c06fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:34:39.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.216+0000 7f4ba67fc700 1 -- 172.21.15.22:0/270420481 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f4ba0014070 con 0x7f4bb0100460 2024-02-16T19:34:39.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.216+0000 7f4bb4d4b700 1 --2- 172.21.15.22:0/270420481 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f4b9c06d9d0 0x7f4b9c06fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:34:39.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.217+0000 7f4bb4d4b700 1 --2- 172.21.15.22:0/270420481 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f4b9c06d9d0 0x7f4b9c06fe80 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4bb0102d30 tx=0x7f4bac005a90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:34:39.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.220+0000 7f4ba67fc700 1 -- 172.21.15.22:0/270420481 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4ba0052d20 con 0x7f4bb0100460 2024-02-16T19:34:39.427 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 16 19:34:39 smithi022 podman[100081]: 2024-02-16 19:34:39.18232209 +0000 UTC m=+0.018600770 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T19:34:39.427 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 16 19:34:39 smithi022 podman[100081]: 2024-02-16 19:34:39.308635429 +0000 UTC m=+0.144914109 container create 6171d81ba5a881f87c9f417bbf902b1d9c0fc55e3172cf443cc64e4ca091894e (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-0-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, ceph=True, GIT_CLEAN=True) 2024-02-16T19:34:39.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.442+0000 7f4bb77b0700 1 -- 172.21.15.22:0/270420481 --> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f4bb0061190 con 0x7f4b9c06d9d0 2024-02-16T19:34:39.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.453+0000 7f4ba67fc700 1 -- 172.21.15.22:0/270420481 <== mgr.34104 v2:172.21.15.22:6800/505266753 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f4bb0061190 con 0x7f4b9c06d9d0 2024-02-16T19:34:39.455 INFO:teuthology.orchestra.run.smithi022.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T19:34:39.455 INFO:teuthology.orchestra.run.smithi022.stdout:alertmanager.smithi022 smithi022 *:9093,9094 running (3m) 12s ago 13m 27.7M - 0.25.0 c8568f914cd2 c89bc76be374 2024-02-16T19:34:39.455 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-exporter.smithi022 smithi022 running (14m) 12s ago 14m 8094k - 18.2.0 dc2bc1663786 3f8bef4349ba 2024-02-16T19:34:39.455 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-exporter.smithi117 smithi117 running (13m) 14s ago 13m 7768k - 18.2.0 dc2bc1663786 59b65a7e309f 2024-02-16T19:34:39.455 INFO:teuthology.orchestra.run.smithi022.stdout:crash.smithi022 smithi022 running (23s) 12s ago 14m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a e0f78bdf43e4 2024-02-16T19:34:39.455 INFO:teuthology.orchestra.run.smithi022.stdout:crash.smithi117 smithi117 running (18s) 14s ago 12m 6669k - 19.0.0-1450-gf78a58c0 f081f847070a bce7c0590813 2024-02-16T19:34:39.455 INFO:teuthology.orchestra.run.smithi022.stdout:grafana.smithi022 smithi022 *:3000 running (2m) 12s ago 13m 76.1M - 9.4.12 a0ea5e3c46d9 fd27d11e9729 2024-02-16T19:34:39.455 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi022.btppwm smithi022 running (8m) 12s ago 8m 156M - 18.2.0 dc2bc1663786 22971fb3a907 2024-02-16T19:34:39.455 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi022.ocukvs smithi022 running (8m) 12s ago 8m 16.4M - 18.2.0 dc2bc1663786 81ee7bd5fd39 2024-02-16T19:34:39.455 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi117.ibmppn smithi117 running (8m) 14s ago 8m 17.6M - 18.2.0 dc2bc1663786 8c606cbeeb86 2024-02-16T19:34:39.455 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi117.orzors smithi117 running (8m) 14s ago 8m 16.6M - 18.2.0 dc2bc1663786 7d2bc4df3feb 2024-02-16T19:34:39.455 INFO:teuthology.orchestra.run.smithi022.stdout:mgr.smithi022.gmdcch smithi022 *:8443,9283,8765 running (4m) 12s ago 15m 495M - 19.0.0-1450-gf78a58c0 f081f847070a 7cb74cc9deea 2024-02-16T19:34:39.456 INFO:teuthology.orchestra.run.smithi022.stdout:mgr.smithi117.cohbtv smithi117 *:8443,9283,8765 running (3m) 14s ago 12m 451M - 19.0.0-1450-gf78a58c0 f081f847070a c825b1152189 2024-02-16T19:34:39.456 INFO:teuthology.orchestra.run.smithi022.stdout:mon.smithi022 smithi022 running (92s) 12s ago 15m 47.4M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 9c84755ae67d 2024-02-16T19:34:39.456 INFO:teuthology.orchestra.run.smithi022.stdout:mon.smithi117 smithi117 running (67s) 14s ago 12m 38.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a a319ac50f1a9 2024-02-16T19:34:39.456 INFO:teuthology.orchestra.run.smithi022.stdout:node-exporter.smithi022 smithi022 *:9100 running (3m) 12s ago 14m 16.4M - 1.5.0 0da6a335fe13 1f5b58292d27 2024-02-16T19:34:39.456 INFO:teuthology.orchestra.run.smithi022.stdout:node-exporter.smithi117 smithi117 *:9100 running (3m) 14s ago 12m 15.4M - 1.5.0 0da6a335fe13 bb5e8491d5a6 2024-02-16T19:34:39.456 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0 smithi022 running (12m) 12s ago 11m 210M 1217M 18.2.0 dc2bc1663786 0027f86f16ce 2024-02-16T19:34:39.456 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1 smithi022 running (11m) 12s ago 11m 227M 1217M 18.2.0 dc2bc1663786 87300382b8a7 2024-02-16T19:34:39.456 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2 smithi022 running (11m) 12s ago 10m 187M 1217M 18.2.0 dc2bc1663786 be8a631b4f93 2024-02-16T19:34:39.456 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3 smithi117 running (10m) 14s ago 10m 244M 2240M 18.2.0 dc2bc1663786 9c0ce9a981cf 2024-02-16T19:34:39.456 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4 smithi117 running (10m) 14s ago 10m 209M 2240M 18.2.0 dc2bc1663786 1779c3496f4d 2024-02-16T19:34:39.456 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5 smithi117 running (9m) 14s ago 9m 198M 2240M 18.2.0 dc2bc1663786 fefaabca0638 2024-02-16T19:34:39.456 INFO:teuthology.orchestra.run.smithi022.stdout:prometheus.smithi022 smithi022 *:9095 running (3m) 12s ago 13m 66.9M - 2.43.0 a07b618ecd1d c3b8689d0c97 2024-02-16T19:34:39.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.455+0000 7f4bb77b0700 1 -- 172.21.15.22:0/270420481 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f4b9c06d9d0 msgr2=0x7f4b9c06fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:39.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.455+0000 7f4bb77b0700 1 --2- 172.21.15.22:0/270420481 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f4b9c06d9d0 0x7f4b9c06fe80 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4bb0102d30 tx=0x7f4bac005a90 comp rx=0 tx=0).stop 2024-02-16T19:34:39.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.455+0000 7f4bb77b0700 1 -- 172.21.15.22:0/270420481 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4bb0100460 msgr2=0x7f4bb0101540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:39.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.455+0000 7f4bb77b0700 1 --2- 172.21.15.22:0/270420481 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4bb0100460 0x7f4bb0101540 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4ba000e920 tx=0x7f4ba000ec30 comp rx=0 tx=0).stop 2024-02-16T19:34:39.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.455+0000 7f4bb77b0700 1 -- 172.21.15.22:0/270420481 shutdown_connections 2024-02-16T19:34:39.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.455+0000 7f4bb77b0700 1 --2- 172.21.15.22:0/270420481 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f4b9c06d9d0 0x7f4b9c06fe80 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:39.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.455+0000 7f4bb77b0700 1 --2- 172.21.15.22:0/270420481 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4bb0100460 0x7f4bb0101540 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:39.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.455+0000 7f4bb77b0700 1 --2- 172.21.15.22:0/270420481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4bb0100d70 0x7f4bb0101a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:39.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.456+0000 7f4bb77b0700 1 -- 172.21.15.22:0/270420481 >> 172.21.15.22:0/270420481 conn(0x7f4bb00748b0 msgr2=0x7f4bb00ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:34:39.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.456+0000 7f4bb77b0700 1 -- 172.21.15.22:0/270420481 shutdown_connections 2024-02-16T19:34:39.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.456+0000 7f4bb77b0700 1 -- 172.21.15.22:0/270420481 wait complete. 2024-02-16T19:34:39.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.564+0000 7fd33305a700 1 -- 172.21.15.22:0/3177407318 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd32c100d70 msgr2=0x7fd32c1094d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:39.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.564+0000 7fd33305a700 1 --2- 172.21.15.22:0/3177407318 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd32c100d70 0x7fd32c1094d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd318009940 tx=0x7fd318009c50 comp rx=0 tx=0).stop 2024-02-16T19:34:39.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.564+0000 7fd33305a700 1 -- 172.21.15.22:0/3177407318 shutdown_connections 2024-02-16T19:34:39.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.564+0000 7fd33305a700 1 --2- 172.21.15.22:0/3177407318 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd32c100d70 0x7fd32c1094d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:39.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.564+0000 7fd33305a700 1 --2- 172.21.15.22:0/3177407318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd32c100460 0x7fd32c100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:39.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.564+0000 7fd33305a700 1 -- 172.21.15.22:0/3177407318 >> 172.21.15.22:0/3177407318 conn(0x7fd32c0748b0 msgr2=0x7fd32c074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:34:39.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.564+0000 7fd33305a700 1 -- 172.21.15.22:0/3177407318 shutdown_connections 2024-02-16T19:34:39.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.564+0000 7fd33305a700 1 -- 172.21.15.22:0/3177407318 wait complete. 2024-02-16T19:34:39.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.565+0000 7fd33305a700 1 Processor -- start 2024-02-16T19:34:39.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.565+0000 7fd33305a700 1 -- start start 2024-02-16T19:34:39.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.565+0000 7fd33305a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd32c100460 0x7fd32c195f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:34:39.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.565+0000 7fd33305a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd32c100d70 0x7fd32c196450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:34:39.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.565+0000 7fd33305a700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd32c196aa0 con 0x7fd32c100460 2024-02-16T19:34:39.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.565+0000 7fd33305a700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd32c196be0 con 0x7fd32c100d70 2024-02-16T19:34:39.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.565+0000 7fd330df6700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd32c100460 0x7fd32c195f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:34:39.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.565+0000 7fd330df6700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd32c100460 0x7fd32c195f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48114/0 (socket says 172.21.15.22:48114) 2024-02-16T19:34:39.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.565+0000 7fd330df6700 1 -- 172.21.15.22:0/2137670590 learned_addr learned my addr 172.21.15.22:0/2137670590 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:34:39.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.565+0000 7fd32bfff700 1 --2- 172.21.15.22:0/2137670590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd32c100d70 0x7fd32c196450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:34:39.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.565+0000 7fd330df6700 1 -- 172.21.15.22:0/2137670590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd32c100d70 msgr2=0x7fd32c196450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:39.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.565+0000 7fd330df6700 1 --2- 172.21.15.22:0/2137670590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd32c100d70 0x7fd32c196450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:39.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.565+0000 7fd330df6700 1 -- 172.21.15.22:0/2137670590 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3180095f0 con 0x7fd32c100460 2024-02-16T19:34:39.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.566+0000 7fd330df6700 1 --2- 172.21.15.22:0/2137670590 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd32c100460 0x7fd32c195f10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd32000c930 tx=0x7fd32000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:34:39.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.566+0000 7fd329ffb700 1 -- 172.21.15.22:0/2137670590 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd320007ad0 con 0x7fd32c100460 2024-02-16T19:34:39.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.566+0000 7fd33305a700 1 -- 172.21.15.22:0/2137670590 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd32c19aa30 con 0x7fd32c100460 2024-02-16T19:34:39.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.566+0000 7fd329ffb700 1 -- 172.21.15.22:0/2137670590 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd32000ce80 con 0x7fd32c100460 2024-02-16T19:34:39.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.566+0000 7fd329ffb700 1 -- 172.21.15.22:0/2137670590 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd320018630 con 0x7fd32c100460 2024-02-16T19:34:39.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.566+0000 7fd33305a700 1 -- 172.21.15.22:0/2137670590 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd32c19af50 con 0x7fd32c100460 2024-02-16T19:34:39.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.567+0000 7fd33305a700 1 -- 172.21.15.22:0/2137670590 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd32c107e40 con 0x7fd32c100460 2024-02-16T19:34:39.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.568+0000 7fd329ffb700 1 -- 172.21.15.22:0/2137670590 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 39) v1 ==== 91595+0+0 (secure 0 0 0) 0x7fd320007c30 con 0x7fd32c100460 2024-02-16T19:34:39.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.568+0000 7fd329ffb700 1 --2- 172.21.15.22:0/2137670590 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fd31c06d9e0 0x7fd31c06fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:34:39.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.568+0000 7fd329ffb700 1 -- 172.21.15.22:0/2137670590 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fd32008d5f0 con 0x7fd32c100460 2024-02-16T19:34:39.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.568+0000 7fd32bfff700 1 --2- 172.21.15.22:0/2137670590 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fd31c06d9e0 0x7fd31c06fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:34:39.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.569+0000 7fd32bfff700 1 --2- 172.21.15.22:0/2137670590 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fd31c06d9e0 0x7fd31c06fe90 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd318006010 tx=0x7fd31800b970 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:34:39.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.573+0000 7fd329ffb700 1 -- 172.21.15.22:0/2137670590 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd320056cb0 con 0x7fd32c100460 2024-02-16T19:34:39.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:39 smithi022 ceph-mon[88798]: pgmap v45: 65 pgs: 9 stale+active+clean, 56 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:34:39.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:39 smithi022 ceph-mon[88798]: from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:34:39.715 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 16 19:34:39 smithi022 podman[100081]: 2024-02-16 19:34:39.545008049 +0000 UTC m=+0.381286740 container init 6171d81ba5a881f87c9f417bbf902b1d9c0fc55e3172cf443cc64e4ca091894e (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-0-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-02-16T19:34:39.715 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 16 19:34:39 smithi022 podman[100081]: 2024-02-16 19:34:39.54831134 +0000 UTC m=+0.384590029 container start 6171d81ba5a881f87c9f417bbf902b1d9c0fc55e3172cf443cc64e4ca091894e (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-02-16T19:34:39.715 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 16 19:34:39 smithi022 podman[100081]: 2024-02-16 19:34:39.644772914 +0000 UTC m=+0.481051604 container attach 6171d81ba5a881f87c9f417bbf902b1d9c0fc55e3172cf443cc64e4ca091894e (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-0-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=) 2024-02-16T19:34:39.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.874+0000 7fd33305a700 1 -- 172.21.15.22:0/2137670590 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd32c107820 con 0x7fd32c100460 2024-02-16T19:34:39.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.875+0000 7fd329ffb700 1 -- 172.21.15.22:0/2137670590 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+687 (secure 0 0 0) 0x7fd32005a370 con 0x7fd32c100460 2024-02-16T19:34:39.877 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-02-16T19:34:39.877 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": { 2024-02-16T19:34:39.877 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T19:34:39.877 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:34:39.877 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": { 2024-02-16T19:34:39.877 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T19:34:39.877 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:34:39.877 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": { 2024-02-16T19:34:39.878 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 5 2024-02-16T19:34:39.878 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:34:39.878 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": { 2024-02-16T19:34:39.878 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-02-16T19:34:39.878 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:34:39.878 INFO:teuthology.orchestra.run.smithi022.stdout: "overall": { 2024-02-16T19:34:39.878 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 9, 2024-02-16T19:34:39.878 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 4 2024-02-16T19:34:39.878 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-02-16T19:34:39.878 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-02-16T19:34:39.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.877+0000 7fd33305a700 1 -- 172.21.15.22:0/2137670590 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fd31c06d9e0 msgr2=0x7fd31c06fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:39.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.877+0000 7fd33305a700 1 --2- 172.21.15.22:0/2137670590 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fd31c06d9e0 0x7fd31c06fe90 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd318006010 tx=0x7fd31800b970 comp rx=0 tx=0).stop 2024-02-16T19:34:39.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.877+0000 7fd33305a700 1 -- 172.21.15.22:0/2137670590 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd32c100460 msgr2=0x7fd32c195f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:39.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.877+0000 7fd33305a700 1 --2- 172.21.15.22:0/2137670590 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd32c100460 0x7fd32c195f10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd32000c930 tx=0x7fd32000cc40 comp rx=0 tx=0).stop 2024-02-16T19:34:39.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.877+0000 7fd33305a700 1 -- 172.21.15.22:0/2137670590 shutdown_connections 2024-02-16T19:34:39.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.877+0000 7fd33305a700 1 --2- 172.21.15.22:0/2137670590 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fd31c06d9e0 0x7fd31c06fe90 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:39.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.877+0000 7fd33305a700 1 --2- 172.21.15.22:0/2137670590 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd32c100460 0x7fd32c195f10 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:39.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.877+0000 7fd33305a700 1 --2- 172.21.15.22:0/2137670590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd32c100d70 0x7fd32c196450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:39.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.877+0000 7fd33305a700 1 -- 172.21.15.22:0/2137670590 >> 172.21.15.22:0/2137670590 conn(0x7fd32c0748b0 msgr2=0x7fd32c0fee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:34:39.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.877+0000 7fd33305a700 1 -- 172.21.15.22:0/2137670590 shutdown_connections 2024-02-16T19:34:39.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.877+0000 7fd33305a700 1 -- 172.21.15.22:0/2137670590 wait complete. 2024-02-16T19:34:39.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.985+0000 7fcdeec9b700 1 -- 172.21.15.22:0/1027355144 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcde80ff350 msgr2=0x7fcde80ff720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:39.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.985+0000 7fcdeec9b700 1 --2- 172.21.15.22:0/1027355144 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcde80ff350 0x7fcde80ff720 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcde000b0a0 tx=0x7fcde000b3b0 comp rx=0 tx=0).stop 2024-02-16T19:34:39.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.985+0000 7fcdeec9b700 1 -- 172.21.15.22:0/1027355144 shutdown_connections 2024-02-16T19:34:39.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.985+0000 7fcdeec9b700 1 --2- 172.21.15.22:0/1027355144 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcde80ffc60 0x7fcde810a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:39.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.985+0000 7fcdeec9b700 1 --2- 172.21.15.22:0/1027355144 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcde80ff350 0x7fcde80ff720 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:39.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.985+0000 7fcdeec9b700 1 -- 172.21.15.22:0/1027355144 >> 172.21.15.22:0/1027355144 conn(0x7fcde80748b0 msgr2=0x7fcde8074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:34:39.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.985+0000 7fcdeec9b700 1 -- 172.21.15.22:0/1027355144 shutdown_connections 2024-02-16T19:34:39.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.985+0000 7fcdeec9b700 1 -- 172.21.15.22:0/1027355144 wait complete. 2024-02-16T19:34:39.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.986+0000 7fcdeec9b700 1 Processor -- start 2024-02-16T19:34:39.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.986+0000 7fcdeec9b700 1 -- start start 2024-02-16T19:34:39.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.986+0000 7fcdeec9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcde80ff350 0x7fcde8195ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:34:39.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.986+0000 7fcdeec9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcde80ffc60 0x7fcde81963e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:34:39.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.986+0000 7fcdeec9b700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde8196ac0 con 0x7fcde80ffc60 2024-02-16T19:34:39.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.986+0000 7fcdeec9b700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde819a850 con 0x7fcde80ff350 2024-02-16T19:34:39.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.986+0000 7fcde7fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcde80ffc60 0x7fcde81963e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:34:39.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.986+0000 7fcde7fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcde80ffc60 0x7fcde81963e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48132/0 (socket says 172.21.15.22:48132) 2024-02-16T19:34:39.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.986+0000 7fcde7fff700 1 -- 172.21.15.22:0/171012956 learned_addr learned my addr 172.21.15.22:0/171012956 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:34:39.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.986+0000 7fcdeca37700 1 --2- 172.21.15.22:0/171012956 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcde80ff350 0x7fcde8195ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:34:39.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.986+0000 7fcde7fff700 1 -- 172.21.15.22:0/171012956 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcde80ff350 msgr2=0x7fcde8195ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:39.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.986+0000 7fcde7fff700 1 --2- 172.21.15.22:0/171012956 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcde80ff350 0x7fcde8195ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:39.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.986+0000 7fcde7fff700 1 -- 172.21.15.22:0/171012956 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcde0009d00 con 0x7fcde80ffc60 2024-02-16T19:34:39.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.986+0000 7fcde7fff700 1 --2- 172.21.15.22:0/171012956 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcde80ffc60 0x7fcde81963e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fcdd800e920 tx=0x7fcdd800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:34:39.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.987+0000 7fcde5ffb700 1 -- 172.21.15.22:0/171012956 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcdd800c9d0 con 0x7fcde80ffc60 2024-02-16T19:34:39.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.987+0000 7fcde5ffb700 1 -- 172.21.15.22:0/171012956 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcdd8010430 con 0x7fcde80ffc60 2024-02-16T19:34:39.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.987+0000 7fcdeec9b700 1 -- 172.21.15.22:0/171012956 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcde819ab30 con 0x7fcde80ffc60 2024-02-16T19:34:39.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.987+0000 7fcde5ffb700 1 -- 172.21.15.22:0/171012956 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcdd8018630 con 0x7fcde80ffc60 2024-02-16T19:34:39.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.987+0000 7fcdeec9b700 1 -- 172.21.15.22:0/171012956 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcde819b080 con 0x7fcde80ffc60 2024-02-16T19:34:39.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.988+0000 7fcdeec9b700 1 -- 172.21.15.22:0/171012956 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcde8107dd0 con 0x7fcde80ffc60 2024-02-16T19:34:39.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.988+0000 7fcde5ffb700 1 -- 172.21.15.22:0/171012956 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 39) v1 ==== 91595+0+0 (secure 0 0 0) 0x7fcdd80105a0 con 0x7fcde80ffc60 2024-02-16T19:34:39.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.989+0000 7fcde5ffb700 1 --2- 172.21.15.22:0/171012956 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fcdd406d9e0 0x7fcdd406fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:34:39.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.989+0000 7fcde5ffb700 1 -- 172.21.15.22:0/171012956 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fcdd8014070 con 0x7fcde80ffc60 2024-02-16T19:34:39.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.989+0000 7fcdeca37700 1 --2- 172.21.15.22:0/171012956 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fcdd406d9e0 0x7fcdd406fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:34:39.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.989+0000 7fcdeca37700 1 --2- 172.21.15.22:0/171012956 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fcdd406d9e0 0x7fcdd406fe90 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fcde0009480 tx=0x7fcde000b700 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:34:39.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:39.993+0000 7fcde5ffb700 1 -- 172.21.15.22:0/171012956 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcdd8052d20 con 0x7fcde80ffc60 2024-02-16T19:34:40.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:39 smithi117 ceph-mon[67870]: pgmap v45: 65 pgs: 9 stale+active+clean, 56 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:34:40.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:39 smithi117 ceph-mon[67870]: from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:34:40.191 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 16 19:34:39 smithi022 podman[100081]: 2024-02-16 19:34:39.851738833 +0000 UTC m=+0.688017505 container died 6171d81ba5a881f87c9f417bbf902b1d9c0fc55e3172cf443cc64e4ca091894e (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-0-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240215.160715-f78a58c) 2024-02-16T19:34:40.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.253+0000 7fcdeec9b700 1 -- 172.21.15.22:0/171012956 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fcde819b2a0 con 0x7fcde80ffc60 2024-02-16T19:34:40.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.254+0000 7fcde5ffb700 1 -- 172.21.15.22:0/171012956 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1845 (secure 0 0 0) 0x7fcdd8057040 con 0x7fcde80ffc60 2024-02-16T19:34:40.255 INFO:teuthology.orchestra.run.smithi022.stdout:e15 2024-02-16T19:34:40.255 INFO:teuthology.orchestra.run.smithi022.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T19:34:40.255 INFO:teuthology.orchestra.run.smithi022.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:34:40.256 INFO:teuthology.orchestra.run.smithi022.stdout:legacy client fscid: 1 2024-02-16T19:34:40.256 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:34:40.256 INFO:teuthology.orchestra.run.smithi022.stdout:Filesystem 'cephfs' (1) 2024-02-16T19:34:40.256 INFO:teuthology.orchestra.run.smithi022.stdout:fs_name cephfs 2024-02-16T19:34:40.256 INFO:teuthology.orchestra.run.smithi022.stdout:epoch 15 2024-02-16T19:34:40.256 INFO:teuthology.orchestra.run.smithi022.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T19:34:40.256 INFO:teuthology.orchestra.run.smithi022.stdout:created 2024-02-16T19:26:17.873644+0000 2024-02-16T19:34:40.256 INFO:teuthology.orchestra.run.smithi022.stdout:modified 2024-02-16T19:26:35.943353+0000 2024-02-16T19:34:40.256 INFO:teuthology.orchestra.run.smithi022.stdout:tableserver 0 2024-02-16T19:34:40.256 INFO:teuthology.orchestra.run.smithi022.stdout:root 0 2024-02-16T19:34:40.256 INFO:teuthology.orchestra.run.smithi022.stdout:session_timeout 60 2024-02-16T19:34:40.256 INFO:teuthology.orchestra.run.smithi022.stdout:session_autoclose 300 2024-02-16T19:34:40.256 INFO:teuthology.orchestra.run.smithi022.stdout:max_file_size 1099511627776 2024-02-16T19:34:40.256 INFO:teuthology.orchestra.run.smithi022.stdout:max_xattr_size 65536 2024-02-16T19:34:40.256 INFO:teuthology.orchestra.run.smithi022.stdout:required_client_features {} 2024-02-16T19:34:40.256 INFO:teuthology.orchestra.run.smithi022.stdout:last_failure 0 2024-02-16T19:34:40.257 INFO:teuthology.orchestra.run.smithi022.stdout:last_failure_osd_epoch 47 2024-02-16T19:34:40.257 INFO:teuthology.orchestra.run.smithi022.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:34:40.257 INFO:teuthology.orchestra.run.smithi022.stdout:max_mds 1 2024-02-16T19:34:40.257 INFO:teuthology.orchestra.run.smithi022.stdout:in 0 2024-02-16T19:34:40.257 INFO:teuthology.orchestra.run.smithi022.stdout:up {0=14490} 2024-02-16T19:34:40.257 INFO:teuthology.orchestra.run.smithi022.stdout:failed 2024-02-16T19:34:40.257 INFO:teuthology.orchestra.run.smithi022.stdout:damaged 2024-02-16T19:34:40.257 INFO:teuthology.orchestra.run.smithi022.stdout:stopped 2024-02-16T19:34:40.257 INFO:teuthology.orchestra.run.smithi022.stdout:data_pools [3] 2024-02-16T19:34:40.257 INFO:teuthology.orchestra.run.smithi022.stdout:metadata_pool 2 2024-02-16T19:34:40.257 INFO:teuthology.orchestra.run.smithi022.stdout:inline_data enabled 2024-02-16T19:34:40.257 INFO:teuthology.orchestra.run.smithi022.stdout:balancer 2024-02-16T19:34:40.257 INFO:teuthology.orchestra.run.smithi022.stdout:bal_rank_mask -1 2024-02-16T19:34:40.257 INFO:teuthology.orchestra.run.smithi022.stdout:standby_count_wanted 1 2024-02-16T19:34:40.257 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.cephfs.smithi022.btppwm{0:14490} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.22:6828/3597685751,v1:172.21.15.22:6829/3597685751] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:34:40.257 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:34:40.258 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:34:40.258 INFO:teuthology.orchestra.run.smithi022.stdout:Standby daemons: 2024-02-16T19:34:40.258 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:34:40.258 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.cephfs.smithi022.ocukvs{-1:14500} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.22:6826/242865843,v1:172.21.15.22:6827/242865843] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:34:40.258 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.cephfs.smithi117.ibmppn{-1:24259} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.117:6824/565408102,v1:172.21.15.117:6825/565408102] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:34:40.258 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.cephfs.smithi117.orzors{-1:24265} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.117:6826/3122342028,v1:172.21.15.117:6827/3122342028] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:34:40.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.256+0000 7fcdeec9b700 1 -- 172.21.15.22:0/171012956 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fcdd406d9e0 msgr2=0x7fcdd406fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:40.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.256+0000 7fcdeec9b700 1 --2- 172.21.15.22:0/171012956 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fcdd406d9e0 0x7fcdd406fe90 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fcde0009480 tx=0x7fcde000b700 comp rx=0 tx=0).stop 2024-02-16T19:34:40.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.256+0000 7fcdeec9b700 1 -- 172.21.15.22:0/171012956 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcde80ffc60 msgr2=0x7fcde81963e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:40.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.256+0000 7fcdeec9b700 1 --2- 172.21.15.22:0/171012956 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcde80ffc60 0x7fcde81963e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fcdd800e920 tx=0x7fcdd800ec30 comp rx=0 tx=0).stop 2024-02-16T19:34:40.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.256+0000 7fcdeec9b700 1 -- 172.21.15.22:0/171012956 shutdown_connections 2024-02-16T19:34:40.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.256+0000 7fcdeec9b700 1 --2- 172.21.15.22:0/171012956 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fcdd406d9e0 0x7fcdd406fe90 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:40.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.256+0000 7fcdeec9b700 1 --2- 172.21.15.22:0/171012956 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcde80ff350 0x7fcde8195ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:40.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.256+0000 7fcdeec9b700 1 --2- 172.21.15.22:0/171012956 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcde80ffc60 0x7fcde81963e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:40.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.256+0000 7fcdeec9b700 1 -- 172.21.15.22:0/171012956 >> 172.21.15.22:0/171012956 conn(0x7fcde80748b0 msgr2=0x7fcde8104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:34:40.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.256+0000 7fcdeec9b700 1 -- 172.21.15.22:0/171012956 shutdown_connections 2024-02-16T19:34:40.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.256+0000 7fcdeec9b700 1 -- 172.21.15.22:0/171012956 wait complete. 2024-02-16T19:34:40.259 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 15 2024-02-16T19:34:40.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.364+0000 7f6ef82c0700 1 -- 172.21.15.22:0/1542586029 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ef0068540 msgr2=0x7f6ef00689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:40.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.364+0000 7f6ef82c0700 1 --2- 172.21.15.22:0/1542586029 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ef0068540 0x7f6ef00689b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6ee4009910 tx=0x7f6ee4009c20 comp rx=0 tx=0).stop 2024-02-16T19:34:40.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.364+0000 7f6ef82c0700 1 -- 172.21.15.22:0/1542586029 shutdown_connections 2024-02-16T19:34:40.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.364+0000 7f6ef82c0700 1 --2- 172.21.15.22:0/1542586029 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ef0068540 0x7f6ef00689b0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:40.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.364+0000 7f6ef82c0700 1 --2- 172.21.15.22:0/1542586029 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ef0105230 0x7f6ef0105600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:40.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.364+0000 7f6ef82c0700 1 -- 172.21.15.22:0/1542586029 >> 172.21.15.22:0/1542586029 conn(0x7f6ef0075130 msgr2=0x7f6ef0075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:34:40.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.364+0000 7f6ef82c0700 1 -- 172.21.15.22:0/1542586029 shutdown_connections 2024-02-16T19:34:40.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.364+0000 7f6ef82c0700 1 -- 172.21.15.22:0/1542586029 wait complete. 2024-02-16T19:34:40.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.365+0000 7f6ef82c0700 1 Processor -- start 2024-02-16T19:34:40.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.365+0000 7f6ef82c0700 1 -- start start 2024-02-16T19:34:40.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.365+0000 7f6ef82c0700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ef0068540 0x7f6ef0195dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:34:40.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.365+0000 7f6ef82c0700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ef0105230 0x7f6ef0196300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:34:40.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.365+0000 7f6ef82c0700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef01969e0 con 0x7f6ef0105230 2024-02-16T19:34:40.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.365+0000 7f6ef82c0700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef019a770 con 0x7f6ef0068540 2024-02-16T19:34:40.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.365+0000 7f6ef585b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ef0105230 0x7f6ef0196300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:34:40.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.365+0000 7f6ef585b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ef0105230 0x7f6ef0196300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48154/0 (socket says 172.21.15.22:48154) 2024-02-16T19:34:40.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.365+0000 7f6ef585b700 1 -- 172.21.15.22:0/4176622104 learned_addr learned my addr 172.21.15.22:0/4176622104 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:34:40.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.365+0000 7f6ef605c700 1 --2- 172.21.15.22:0/4176622104 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ef0068540 0x7f6ef0195dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:34:40.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.365+0000 7f6ef585b700 1 -- 172.21.15.22:0/4176622104 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ef0068540 msgr2=0x7f6ef0195dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:40.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.365+0000 7f6ef585b700 1 --2- 172.21.15.22:0/4176622104 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ef0068540 0x7f6ef0195dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:40.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.365+0000 7f6ef585b700 1 -- 172.21.15.22:0/4176622104 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ee40095f0 con 0x7f6ef0105230 2024-02-16T19:34:40.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.365+0000 7f6ef585b700 1 --2- 172.21.15.22:0/4176622104 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ef0105230 0x7f6ef0196300 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6ee4006010 tx=0x7f6ee400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:34:40.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.366+0000 7f6eef7fe700 1 -- 172.21.15.22:0/4176622104 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ee401d070 con 0x7f6ef0105230 2024-02-16T19:34:40.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.366+0000 7f6ef82c0700 1 -- 172.21.15.22:0/4176622104 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ef019a9f0 con 0x7f6ef0105230 2024-02-16T19:34:40.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.366+0000 7f6eef7fe700 1 -- 172.21.15.22:0/4176622104 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6ee400f460 con 0x7f6ef0105230 2024-02-16T19:34:40.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.366+0000 7f6eef7fe700 1 -- 172.21.15.22:0/4176622104 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ee4017630 con 0x7f6ef0105230 2024-02-16T19:34:40.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.366+0000 7f6ef82c0700 1 -- 172.21.15.22:0/4176622104 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ef019aee0 con 0x7f6ef0105230 2024-02-16T19:34:40.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.367+0000 7f6ef82c0700 1 -- 172.21.15.22:0/4176622104 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ef0108750 con 0x7f6ef0105230 2024-02-16T19:34:40.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.367+0000 7f6eef7fe700 1 -- 172.21.15.22:0/4176622104 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 39) v1 ==== 91595+0+0 (secure 0 0 0) 0x7f6ee400f5d0 con 0x7f6ef0105230 2024-02-16T19:34:40.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.367+0000 7f6eef7fe700 1 --2- 172.21.15.22:0/4176622104 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f6ed806d9e0 0x7f6ed806fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:34:40.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.368+0000 7f6eef7fe700 1 -- 172.21.15.22:0/4176622104 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f6ee408eb60 con 0x7f6ef0105230 2024-02-16T19:34:40.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.368+0000 7f6ef605c700 1 --2- 172.21.15.22:0/4176622104 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f6ed806d9e0 0x7f6ed806fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:34:40.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.368+0000 7f6ef605c700 1 --2- 172.21.15.22:0/4176622104 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f6ed806d9e0 0x7f6ed806fe90 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6ee8009c10 tx=0x7f6ee8009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:34:40.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.372+0000 7f6eef7fe700 1 -- 172.21.15.22:0/4176622104 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6ee4058220 con 0x7f6ef0105230 2024-02-16T19:34:40.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.598+0000 7f6ef82c0700 1 -- 172.21.15.22:0/4176622104 --> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6ef002ccf0 con 0x7f6ed806d9e0 2024-02-16T19:34:40.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.599+0000 7f6eef7fe700 1 -- 172.21.15.22:0/4176622104 <== mgr.34104 v2:172.21.15.22:6800/505266753 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f6ef002ccf0 con 0x7f6ed806d9e0 2024-02-16T19:34:40.600 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-02-16T19:34:40.601 INFO:teuthology.orchestra.run.smithi022.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T19:34:40.601 INFO:teuthology.orchestra.run.smithi022.stdout: "in_progress": true, 2024-02-16T19:34:40.601 INFO:teuthology.orchestra.run.smithi022.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T19:34:40.601 INFO:teuthology.orchestra.run.smithi022.stdout: "services_complete": [ 2024-02-16T19:34:40.601 INFO:teuthology.orchestra.run.smithi022.stdout: "crash", 2024-02-16T19:34:40.601 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr", 2024-02-16T19:34:40.601 INFO:teuthology.orchestra.run.smithi022.stdout: "mon" 2024-02-16T19:34:40.601 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-02-16T19:34:40.601 INFO:teuthology.orchestra.run.smithi022.stdout: "progress": "6/23 daemons upgraded", 2024-02-16T19:34:40.601 INFO:teuthology.orchestra.run.smithi022.stdout: "message": "Currently upgrading osd daemons", 2024-02-16T19:34:40.601 INFO:teuthology.orchestra.run.smithi022.stdout: "is_paused": false 2024-02-16T19:34:40.602 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-02-16T19:34:40.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.601+0000 7f6ef82c0700 1 -- 172.21.15.22:0/4176622104 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f6ed806d9e0 msgr2=0x7f6ed806fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:40.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.601+0000 7f6ef82c0700 1 --2- 172.21.15.22:0/4176622104 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f6ed806d9e0 0x7f6ed806fe90 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6ee8009c10 tx=0x7f6ee8009040 comp rx=0 tx=0).stop 2024-02-16T19:34:40.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.601+0000 7f6ef82c0700 1 -- 172.21.15.22:0/4176622104 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ef0105230 msgr2=0x7f6ef0196300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:40.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.601+0000 7f6ef82c0700 1 --2- 172.21.15.22:0/4176622104 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ef0105230 0x7f6ef0196300 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6ee4006010 tx=0x7f6ee400be20 comp rx=0 tx=0).stop 2024-02-16T19:34:40.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.601+0000 7f6ef82c0700 1 -- 172.21.15.22:0/4176622104 shutdown_connections 2024-02-16T19:34:40.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.601+0000 7f6ef82c0700 1 --2- 172.21.15.22:0/4176622104 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f6ed806d9e0 0x7f6ed806fe90 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:40.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.601+0000 7f6ef82c0700 1 --2- 172.21.15.22:0/4176622104 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ef0068540 0x7f6ef0195dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:40.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.601+0000 7f6ef82c0700 1 --2- 172.21.15.22:0/4176622104 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ef0105230 0x7f6ef0196300 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:40.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.601+0000 7f6ef82c0700 1 -- 172.21.15.22:0/4176622104 >> 172.21.15.22:0/4176622104 conn(0x7f6ef0075130 msgr2=0x7f6ef00feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:34:40.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.601+0000 7f6ef82c0700 1 -- 172.21.15.22:0/4176622104 shutdown_connections 2024-02-16T19:34:40.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.601+0000 7f6ef82c0700 1 -- 172.21.15.22:0/4176622104 wait complete. 2024-02-16T19:34:40.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.720+0000 7fe974e9b700 1 -- 172.21.15.22:0/226599586 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9700ffd60 msgr2=0x7fe970103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:40.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.720+0000 7fe974e9b700 1 --2- 172.21.15.22:0/226599586 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9700ffd60 0x7fe970103d30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe964009910 tx=0x7fe964009c20 comp rx=0 tx=0).stop 2024-02-16T19:34:40.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.720+0000 7fe974e9b700 1 -- 172.21.15.22:0/226599586 shutdown_connections 2024-02-16T19:34:40.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.720+0000 7fe974e9b700 1 --2- 172.21.15.22:0/226599586 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9700ffd60 0x7fe970103d30 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:40.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.720+0000 7fe974e9b700 1 --2- 172.21.15.22:0/226599586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9700ff3c0 0x7fe9700ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:40.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.720+0000 7fe974e9b700 1 -- 172.21.15.22:0/226599586 >> 172.21.15.22:0/226599586 conn(0x7fe9700fac60 msgr2=0x7fe9700fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:34:40.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.720+0000 7fe974e9b700 1 -- 172.21.15.22:0/226599586 shutdown_connections 2024-02-16T19:34:40.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.720+0000 7fe974e9b700 1 -- 172.21.15.22:0/226599586 wait complete. 2024-02-16T19:34:40.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.721+0000 7fe974e9b700 1 Processor -- start 2024-02-16T19:34:40.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.721+0000 7fe974e9b700 1 -- start start 2024-02-16T19:34:40.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.722+0000 7fe974e9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9700ff3c0 0x7fe970196d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:34:40.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.722+0000 7fe974e9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9700ffd60 0x7fe970197260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:34:40.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.722+0000 7fe974e9b700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe970197880 con 0x7fe9700ffd60 2024-02-16T19:34:40.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.722+0000 7fe974e9b700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9701979c0 con 0x7fe9700ff3c0 2024-02-16T19:34:40.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.722+0000 7fe96dd9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9700ffd60 0x7fe970197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:34:40.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.722+0000 7fe96dd9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9700ffd60 0x7fe970197260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48172/0 (socket says 172.21.15.22:48172) 2024-02-16T19:34:40.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.722+0000 7fe96dd9b700 1 -- 172.21.15.22:0/4078488114 learned_addr learned my addr 172.21.15.22:0/4078488114 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:34:40.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.722+0000 7fe96e59c700 1 --2- 172.21.15.22:0/4078488114 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9700ff3c0 0x7fe970196d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:34:40.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.722+0000 7fe96dd9b700 1 -- 172.21.15.22:0/4078488114 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9700ff3c0 msgr2=0x7fe970196d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:40.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.722+0000 7fe96dd9b700 1 --2- 172.21.15.22:0/4078488114 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9700ff3c0 0x7fe970196d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:40.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.722+0000 7fe96dd9b700 1 -- 172.21.15.22:0/4078488114 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9640095f0 con 0x7fe9700ffd60 2024-02-16T19:34:40.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.722+0000 7fe96dd9b700 1 --2- 172.21.15.22:0/4078488114 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9700ffd60 0x7fe970197260 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe9640059d0 tx=0x7fe96400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:34:40.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.723+0000 7fe95f7fe700 1 -- 172.21.15.22:0/4078488114 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe96401d070 con 0x7fe9700ffd60 2024-02-16T19:34:40.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.723+0000 7fe95f7fe700 1 -- 172.21.15.22:0/4078488114 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe96400b610 con 0x7fe9700ffd60 2024-02-16T19:34:40.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.723+0000 7fe974e9b700 1 -- 172.21.15.22:0/4078488114 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe97019fd90 con 0x7fe9700ffd60 2024-02-16T19:34:40.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.723+0000 7fe95f7fe700 1 -- 172.21.15.22:0/4078488114 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe964022470 con 0x7fe9700ffd60 2024-02-16T19:34:40.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.723+0000 7fe974e9b700 1 -- 172.21.15.22:0/4078488114 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9701a00f0 con 0x7fe9700ffd60 2024-02-16T19:34:40.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.724+0000 7fe974e9b700 1 -- 172.21.15.22:0/4078488114 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe970107730 con 0x7fe9700ffd60 2024-02-16T19:34:40.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.724+0000 7fe95f7fe700 1 -- 172.21.15.22:0/4078488114 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 39) v1 ==== 91595+0+0 (secure 0 0 0) 0x7fe96400fac0 con 0x7fe9700ffd60 2024-02-16T19:34:40.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.725+0000 7fe95f7fe700 1 --2- 172.21.15.22:0/4078488114 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fe95806d9e0 0x7fe95806fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:34:40.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.725+0000 7fe95f7fe700 1 -- 172.21.15.22:0/4078488114 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fe96408eaf0 con 0x7fe9700ffd60 2024-02-16T19:34:40.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.725+0000 7fe96e59c700 1 --2- 172.21.15.22:0/4078488114 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fe95806d9e0 0x7fe95806fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:34:40.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.725+0000 7fe96e59c700 1 --2- 172.21.15.22:0/4078488114 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fe95806d9e0 0x7fe95806fe90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe960009c60 tx=0x7fe960009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:34:40.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:40.730+0000 7fe95f7fe700 1 -- 172.21.15.22:0/4078488114 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe964058260 con 0x7fe9700ffd60 2024-02-16T19:34:40.870 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:40 smithi022 ceph-mon[88798]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:34:40.870 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:40 smithi022 ceph-mon[88798]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:34:40.871 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:40 smithi022 ceph-mon[88798]: from='client.? 172.21.15.22:0/2137670590' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:40.871 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:40 smithi022 ceph-mon[88798]: from='client.? 172.21.15.22:0/171012956' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:34:40.871 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 16 19:34:40 smithi022 podman[100150]: 2024-02-16 19:34:40.633349727 +0000 UTC m=+0.773058367 container remove 6171d81ba5a881f87c9f417bbf902b1d9c0fc55e3172cf443cc64e4ca091894e (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-0-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240215.160715-f78a58c) 2024-02-16T19:34:40.871 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 16 19:34:40 smithi022 systemd[1]: ceph-2a1346e6-cd00-11ee-95ba-87774f69a715@osd.0.service: Deactivated successfully. 2024-02-16T19:34:40.871 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 16 19:34:40 smithi022 systemd[1]: Stopped Ceph osd.0 for 2a1346e6-cd00-11ee-95ba-87774f69a715. 2024-02-16T19:34:40.871 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 16 19:34:40 smithi022 systemd[1]: ceph-2a1346e6-cd00-11ee-95ba-87774f69a715@osd.0.service: Consumed 52.767s CPU time. 2024-02-16T19:34:41.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:41.039+0000 7fe974e9b700 1 -- 172.21.15.22:0/4078488114 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fe97004f2a0 con 0x7fe9700ffd60 2024-02-16T19:34:41.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:41.040+0000 7fe95f7fe700 1 -- 172.21.15.22:0/4078488114 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+290 (secure 0 0 0) 0x7fe96405b920 con 0x7fe9700ffd60 2024-02-16T19:34:41.042 INFO:teuthology.orchestra.run.smithi022.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; 1 osds down 2024-02-16T19:34:41.042 INFO:teuthology.orchestra.run.smithi022.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-02-16T19:34:41.042 INFO:teuthology.orchestra.run.smithi022.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-02-16T19:34:41.042 INFO:teuthology.orchestra.run.smithi022.stdout:[WRN] OSD_DOWN: 1 osds down 2024-02-16T19:34:41.042 INFO:teuthology.orchestra.run.smithi022.stdout: osd.0 (root=default,host=smithi022) is down 2024-02-16T19:34:41.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:41.042+0000 7fe974e9b700 1 -- 172.21.15.22:0/4078488114 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fe95806d9e0 msgr2=0x7fe95806fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:41.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:41.042+0000 7fe974e9b700 1 --2- 172.21.15.22:0/4078488114 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fe95806d9e0 0x7fe95806fe90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe960009c60 tx=0x7fe960009260 comp rx=0 tx=0).stop 2024-02-16T19:34:41.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:41.042+0000 7fe974e9b700 1 -- 172.21.15.22:0/4078488114 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9700ffd60 msgr2=0x7fe970197260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:41.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:41.042+0000 7fe974e9b700 1 --2- 172.21.15.22:0/4078488114 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9700ffd60 0x7fe970197260 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe9640059d0 tx=0x7fe96400f690 comp rx=0 tx=0).stop 2024-02-16T19:34:41.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:41.042+0000 7fe974e9b700 1 -- 172.21.15.22:0/4078488114 shutdown_connections 2024-02-16T19:34:41.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:41.042+0000 7fe974e9b700 1 --2- 172.21.15.22:0/4078488114 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fe95806d9e0 0x7fe95806fe90 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:41.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:41.042+0000 7fe974e9b700 1 --2- 172.21.15.22:0/4078488114 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9700ff3c0 0x7fe970196d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:41.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:41.042+0000 7fe974e9b700 1 --2- 172.21.15.22:0/4078488114 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9700ffd60 0x7fe970197260 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:41.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:41.042+0000 7fe974e9b700 1 -- 172.21.15.22:0/4078488114 >> 172.21.15.22:0/4078488114 conn(0x7fe9700fac60 msgr2=0x7fe9700fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:34:41.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:41.042+0000 7fe974e9b700 1 -- 172.21.15.22:0/4078488114 shutdown_connections 2024-02-16T19:34:41.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:34:41.042+0000 7fe974e9b700 1 -- 172.21.15.22:0/4078488114 wait complete. 2024-02-16T19:34:41.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:40 smithi117 ceph-mon[67870]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:34:41.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:40 smithi117 ceph-mon[67870]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:34:41.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:40 smithi117 ceph-mon[67870]: from='client.? 172.21.15.22:0/2137670590' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:41.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:40 smithi117 ceph-mon[67870]: from='client.? 172.21.15.22:0/171012956' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:34:41.171 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 16 19:34:40 smithi022 systemd[1]: Starting Ceph osd.0 for 2a1346e6-cd00-11ee-95ba-87774f69a715... 2024-02-16T19:34:41.171 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 16 19:34:41 smithi022 podman[100324]: 2024-02-16 19:34:40.99781525 +0000 UTC m=+0.018359886 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T19:34:41.171 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 16 19:34:41 smithi022 podman[100324]: 2024-02-16 19:34:41.10998662 +0000 UTC m=+0.130531250 container create d1c87eda9b8b83850204b3610e8b2913eb437f0103b7ccec4cfe7ffb7cf76c6b (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-0-activate, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-16T19:34:41.465 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 16 19:34:41 smithi022 podman[100324]: 2024-02-16 19:34:41.300122672 +0000 UTC m=+0.320667302 container init d1c87eda9b8b83850204b3610e8b2913eb437f0103b7ccec4cfe7ffb7cf76c6b (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-0-activate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, maintainer=Guillaume Abrioux ) 2024-02-16T19:34:41.465 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 16 19:34:41 smithi022 podman[100324]: 2024-02-16 19:34:41.303164564 +0000 UTC m=+0.323709194 container start d1c87eda9b8b83850204b3610e8b2913eb437f0103b7ccec4cfe7ffb7cf76c6b (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-0-activate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-16T19:34:41.465 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 16 19:34:41 smithi022 podman[100324]: 2024-02-16 19:34:41.395966848 +0000 UTC m=+0.416511478 container attach d1c87eda9b8b83850204b3610e8b2913eb437f0103b7ccec4cfe7ffb7cf76c6b (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-0-activate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, ceph=True) 2024-02-16T19:34:41.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:41 smithi022 ceph-mon[88798]: pgmap v47: 65 pgs: 9 stale+active+clean, 56 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:34:41.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:41 smithi022 ceph-mon[88798]: from='client.34208 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:34:41.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:41 smithi022 ceph-mon[88798]: from='client.? 172.21.15.22:0/4078488114' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T19:34:41.965 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 16 19:34:41 smithi022 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-0-activate[100342]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-02-16T19:34:41.966 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 16 19:34:41 smithi022 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-0-activate[100342]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-02-16T19:34:41.966 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 16 19:34:41 smithi022 bash[100324]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-02-16T19:34:41.966 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 16 19:34:41 smithi022 bash[100324]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-02-16T19:34:41.966 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 16 19:34:41 smithi022 bash[100324]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-02-16T19:34:41.966 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 16 19:34:41 smithi022 bash[100324]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-02-16T19:34:41.966 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 16 19:34:41 smithi022 bash[100324]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-02-16T19:34:41.966 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 16 19:34:41 smithi022 bash[100324]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-02-16T19:34:41.966 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 16 19:34:41 smithi022 bash[100324]: --> ceph-volume raw activate successful for osd ID: 0 2024-02-16T19:34:41.966 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 16 19:34:41 smithi022 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-0-activate[100342]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-02-16T19:34:41.966 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 16 19:34:41 smithi022 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-0-activate[100342]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-02-16T19:34:41.966 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 16 19:34:41 smithi022 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-0-activate[100342]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-02-16T19:34:41.966 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 16 19:34:41 smithi022 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-0-activate[100342]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-02-16T19:34:41.966 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 16 19:34:41 smithi022 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-0-activate[100342]: --> ceph-volume raw activate successful for osd ID: 0 2024-02-16T19:34:41.966 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 16 19:34:41 smithi022 podman[100324]: 2024-02-16 19:34:41.788986446 +0000 UTC m=+0.809531080 container died d1c87eda9b8b83850204b3610e8b2913eb437f0103b7ccec4cfe7ffb7cf76c6b (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-0-activate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-16T19:34:42.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:41 smithi117 ceph-mon[67870]: pgmap v47: 65 pgs: 9 stale+active+clean, 56 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:34:42.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:41 smithi117 ceph-mon[67870]: from='client.34208 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:34:42.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:41 smithi117 ceph-mon[67870]: from='client.? 172.21.15.22:0/4078488114' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T19:34:42.715 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 16 19:34:42 smithi022 podman[100467]: 2024-02-16 19:34:42.383240827 +0000 UTC m=+0.589121824 container remove d1c87eda9b8b83850204b3610e8b2913eb437f0103b7ccec4cfe7ffb7cf76c6b (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-0-activate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-02-16T19:34:42.715 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 16 19:34:42 smithi022 podman[100530]: 2024-02-16 19:34:42.503822378 +0000 UTC m=+0.018777647 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T19:34:42.715 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 16 19:34:42 smithi022 podman[100530]: 2024-02-16 19:34:42.625899085 +0000 UTC m=+0.140854389 container create 002412b3525f0cf864b7f21cbc310f8c070b81f979d03759b1dd184a70337369 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240215.160715-f78a58c) 2024-02-16T19:34:43.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:42 smithi117 ceph-mon[67870]: Health check failed: Degraded data redundancy: 50/267 objects degraded (18.727%), 19 pgs degraded (PG_DEGRADED) 2024-02-16T19:34:43.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:42 smithi022 ceph-mon[88798]: Health check failed: Degraded data redundancy: 50/267 objects degraded (18.727%), 19 pgs degraded (PG_DEGRADED) 2024-02-16T19:34:43.215 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 16 19:34:42 smithi022 podman[100530]: 2024-02-16 19:34:42.844943068 +0000 UTC m=+0.359898331 container init 002412b3525f0cf864b7f21cbc310f8c070b81f979d03759b1dd184a70337369 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-0, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS) 2024-02-16T19:34:43.215 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 16 19:34:42 smithi022 podman[100530]: 2024-02-16 19:34:42.847895201 +0000 UTC m=+0.362850478 container start 002412b3525f0cf864b7f21cbc310f8c070b81f979d03759b1dd184a70337369 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-02-16T19:34:43.215 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 16 19:34:42 smithi022 ceph-osd[100549]: -- 172.21.15.22:0/2668279357 <== mon.0 v2:172.21.15.22:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a8ba6771e0 con 0x55a8ba648c00 2024-02-16T19:34:43.215 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 16 19:34:42 smithi022 bash[100530]: 002412b3525f0cf864b7f21cbc310f8c070b81f979d03759b1dd184a70337369 2024-02-16T19:34:43.216 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 16 19:34:42 smithi022 systemd[1]: Started Ceph osd.0 for 2a1346e6-cd00-11ee-95ba-87774f69a715. 2024-02-16T19:34:43.715 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 16 19:34:43 smithi022 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-0[100545]: 2024-02-16T19:34:43.388+0000 7fd6af6057c0 -1 Falling back to public interface 2024-02-16T19:34:44.057 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:43 smithi022 ceph-mon[88798]: pgmap v48: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 50/267 objects degraded (18.727%) 2024-02-16T19:34:44.057 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:43 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:44.057 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:43 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:44.057 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:43 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:34:44.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:43 smithi117 ceph-mon[67870]: pgmap v48: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 50/267 objects degraded (18.727%) 2024-02-16T19:34:44.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:43 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:44.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:43 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:44.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:43 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:34:45.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:45 smithi022 ceph-mon[88798]: pgmap v49: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 50/267 objects degraded (18.727%) 2024-02-16T19:34:46.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:45 smithi117 ceph-mon[67870]: pgmap v49: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 50/267 objects degraded (18.727%) 2024-02-16T19:34:46.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:46 smithi022 ceph-mon[88798]: pgmap v50: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 50/267 objects degraded (18.727%) 2024-02-16T19:34:46.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:46 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:46.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:46 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:34:47.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:46 smithi117 ceph-mon[67870]: pgmap v50: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 50/267 objects degraded (18.727%) 2024-02-16T19:34:47.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:46 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:47.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:46 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:34:47.725 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 16 19:34:47 smithi022 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-0[100545]: 2024-02-16T19:34:47.571+0000 7fd6af6057c0 -1 osd.0 0 read_superblock omap replica is missing. 2024-02-16T19:34:48.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:47 smithi117 ceph-mon[67870]: mgrmap e40: smithi022.gmdcch(active, since 91s), standbys: smithi117.cohbtv 2024-02-16T19:34:48.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:47 smithi022 ceph-mon[88798]: mgrmap e40: smithi022.gmdcch(active, since 91s), standbys: smithi117.cohbtv 2024-02-16T19:34:48.215 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 16 19:34:47 smithi022 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-0[100545]: 2024-02-16T19:34:47.890+0000 7fd6af6057c0 -1 osd.0 50 log_to_monitors true 2024-02-16T19:34:49.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:48 smithi117 ceph-mon[67870]: pgmap v51: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 307 B/s wr, 0 op/s; 50/267 objects degraded (18.727%) 2024-02-16T19:34:49.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:48 smithi117 ceph-mon[67870]: from='osd.0 [v2:172.21.15.22:6802/1744550481,v1:172.21.15.22:6803/1744550481]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-16T19:34:49.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:48 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:49.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:48 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:49.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:48 smithi022 ceph-mon[88798]: pgmap v51: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 307 B/s wr, 0 op/s; 50/267 objects degraded (18.727%) 2024-02-16T19:34:49.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:48 smithi022 ceph-mon[88798]: from='osd.0 [v2:172.21.15.22:6802/1744550481,v1:172.21.15.22:6803/1744550481]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-16T19:34:49.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:48 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:49.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:48 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:49.848 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 16 19:34:49 smithi022 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-0[100545]: 2024-02-16T19:34:49.567+0000 7fd69ca55700 -1 osd.0 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-16T19:34:50.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:49 smithi117 ceph-mon[67870]: from='osd.0 [v2:172.21.15.22:6802/1744550481,v1:172.21.15.22:6803/1744550481]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-16T19:34:50.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:49 smithi117 ceph-mon[67870]: osdmap e53: 6 total, 5 up, 6 in 2024-02-16T19:34:50.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:49 smithi117 ceph-mon[67870]: from='osd.0 [v2:172.21.15.22:6802/1744550481,v1:172.21.15.22:6803/1744550481]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-02-16T19:34:50.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:49 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:50.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:49 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:50.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:49 smithi117 ceph-mon[67870]: from='osd.0 [v2:172.21.15.22:6802/1744550481,v1:172.21.15.22:6803/1744550481]' entity='osd.0' 2024-02-16T19:34:50.206 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:49 smithi022 ceph-mon[88798]: from='osd.0 [v2:172.21.15.22:6802/1744550481,v1:172.21.15.22:6803/1744550481]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-16T19:34:50.206 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:49 smithi022 ceph-mon[88798]: osdmap e53: 6 total, 5 up, 6 in 2024-02-16T19:34:50.206 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:49 smithi022 ceph-mon[88798]: from='osd.0 [v2:172.21.15.22:6802/1744550481,v1:172.21.15.22:6803/1744550481]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-02-16T19:34:50.206 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:49 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:50.206 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:49 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:50.206 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:49 smithi022 ceph-mon[88798]: from='osd.0 [v2:172.21.15.22:6802/1744550481,v1:172.21.15.22:6803/1744550481]' entity='osd.0' 2024-02-16T19:34:51.090 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:50 smithi022 ceph-mon[88798]: pgmap v53: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 307 B/s wr, 0 op/s; 50/267 objects degraded (18.727%) 2024-02-16T19:34:51.090 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:50 smithi022 ceph-mon[88798]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-16T19:34:51.090 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:50 smithi022 ceph-mon[88798]: osd.0 [v2:172.21.15.22:6802/1744550481,v1:172.21.15.22:6803/1744550481] boot 2024-02-16T19:34:51.090 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:50 smithi022 ceph-mon[88798]: osdmap e54: 6 total, 6 up, 6 in 2024-02-16T19:34:51.090 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:50 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-16T19:34:51.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:50 smithi117 ceph-mon[67870]: pgmap v53: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 307 B/s wr, 0 op/s; 50/267 objects degraded (18.727%) 2024-02-16T19:34:51.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:50 smithi117 ceph-mon[67870]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-16T19:34:51.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:50 smithi117 ceph-mon[67870]: osd.0 [v2:172.21.15.22:6802/1744550481,v1:172.21.15.22:6803/1744550481] boot 2024-02-16T19:34:51.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:50 smithi117 ceph-mon[67870]: osdmap e54: 6 total, 6 up, 6 in 2024-02-16T19:34:51.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:50 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-16T19:34:52.640 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:52 smithi022 ceph-mon[88798]: osdmap e55: 6 total, 6 up, 6 in 2024-02-16T19:34:52.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:52 smithi117 ceph-mon[67870]: osdmap e55: 6 total, 6 up, 6 in 2024-02-16T19:34:53.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:53 smithi117 ceph-mon[67870]: pgmap v56: 65 pgs: 3 active+undersized+degraded, 6 active+undersized, 56 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s; 7/267 objects degraded (2.622%) 2024-02-16T19:34:53.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:53 smithi117 ceph-mon[67870]: Health check update: Degraded data redundancy: 7/267 objects degraded (2.622%), 3 pgs degraded (PG_DEGRADED) 2024-02-16T19:34:53.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:53 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:53.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:53 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:53.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:53 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:34:53.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:53 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:34:53.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:53 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:53.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:53 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:34:53.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:53 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:53.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:53 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:53.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:53 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:53.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:53 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-02-16T19:34:53.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:53 smithi022 ceph-mon[88798]: pgmap v56: 65 pgs: 3 active+undersized+degraded, 6 active+undersized, 56 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s; 7/267 objects degraded (2.622%) 2024-02-16T19:34:53.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:53 smithi022 ceph-mon[88798]: Health check update: Degraded data redundancy: 7/267 objects degraded (2.622%), 3 pgs degraded (PG_DEGRADED) 2024-02-16T19:34:53.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:53 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:53.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:53 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:53.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:53 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:34:53.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:53 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:34:53.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:53 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:34:53.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:53 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:34:53.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:53 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:53.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:53 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:53.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:53 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:53.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:53 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-02-16T19:34:54.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:54 smithi117 ceph-mon[67870]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-02-16T19:34:54.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:54 smithi117 ceph-mon[67870]: Upgrade: unsafe to stop osd(s) at this time (2 PGs are or would become offline) 2024-02-16T19:34:54.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:54 smithi022 ceph-mon[88798]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-02-16T19:34:54.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:54 smithi022 ceph-mon[88798]: Upgrade: unsafe to stop osd(s) at this time (2 PGs are or would become offline) 2024-02-16T19:34:55.839 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:55 smithi022 ceph-mon[88798]: pgmap v57: 65 pgs: 3 active+undersized+degraded, 6 active+undersized, 56 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 7/267 objects degraded (2.622%) 2024-02-16T19:34:55.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:55 smithi117 ceph-mon[67870]: pgmap v57: 65 pgs: 3 active+undersized+degraded, 6 active+undersized, 56 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 7/267 objects degraded (2.622%) 2024-02-16T19:34:56.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:56 smithi117 ceph-mon[67870]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 7/267 objects degraded (2.622%), 3 pgs degraded) 2024-02-16T19:34:56.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:56 smithi022 ceph-mon[88798]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 7/267 objects degraded (2.622%), 3 pgs degraded) 2024-02-16T19:34:57.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:57 smithi117 ceph-mon[67870]: pgmap v58: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 40 B/s, 0 objects/s recovering 2024-02-16T19:34:57.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:57 smithi022 ceph-mon[88798]: pgmap v58: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 40 B/s, 0 objects/s recovering 2024-02-16T19:34:59.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:34:59 smithi117 ceph-mon[67870]: pgmap v59: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 35 B/s, 0 objects/s recovering 2024-02-16T19:34:59.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:34:59 smithi022 ceph-mon[88798]: pgmap v59: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 35 B/s, 0 objects/s recovering 2024-02-16T19:35:01.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:01 smithi022 ceph-mon[88798]: pgmap v60: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 30 B/s, 0 objects/s recovering 2024-02-16T19:35:01.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:01 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:35:01.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:01 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:35:01.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:01 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:35:02.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:01 smithi117 ceph-mon[67870]: pgmap v60: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 30 B/s, 0 objects/s recovering 2024-02-16T19:35:02.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:01 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:35:02.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:01 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:35:02.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:01 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:35:03.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:03 smithi022 ceph-mon[88798]: pgmap v61: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 27 B/s, 0 objects/s recovering 2024-02-16T19:35:04.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:03 smithi117 ceph-mon[67870]: pgmap v61: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 27 B/s, 0 objects/s recovering 2024-02-16T19:35:05.944 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:05 smithi022 ceph-mon[88798]: pgmap v62: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 23 B/s, 0 objects/s recovering 2024-02-16T19:35:06.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:05 smithi117 ceph-mon[67870]: pgmap v62: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 23 B/s, 0 objects/s recovering 2024-02-16T19:35:07.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:07 smithi117 ceph-mon[67870]: pgmap v63: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 23 B/s, 0 objects/s recovering 2024-02-16T19:35:07.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:07 smithi022 ceph-mon[88798]: pgmap v63: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 23 B/s, 0 objects/s recovering 2024-02-16T19:35:08.960 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:08 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-02-16T19:35:09.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:08 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-02-16T19:35:09.757 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:09 smithi022 ceph-mon[88798]: pgmap v64: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:35:09.757 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:09 smithi022 ceph-mon[88798]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-02-16T19:35:09.757 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:09 smithi022 ceph-mon[88798]: Upgrade: osd.1 is safe to restart 2024-02-16T19:35:10.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:09 smithi117 ceph-mon[67870]: pgmap v64: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:35:10.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:09 smithi117 ceph-mon[67870]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-02-16T19:35:10.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:09 smithi117 ceph-mon[67870]: Upgrade: osd.1 is safe to restart 2024-02-16T19:35:10.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:10 smithi022 ceph-mon[88798]: pgmap v65: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:35:10.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:10 smithi022 ceph-mon[88798]: Upgrade: Updating osd.1 2024-02-16T19:35:10.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:10 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:35:10.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:10 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-16T19:35:10.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:10 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:35:10.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:10 smithi022 ceph-mon[88798]: Deploying daemon osd.1 on smithi022 2024-02-16T19:35:11.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:10 smithi117 ceph-mon[67870]: pgmap v65: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:35:11.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:10 smithi117 ceph-mon[67870]: Upgrade: Updating osd.1 2024-02-16T19:35:11.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:10 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:35:11.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:10 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-16T19:35:11.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:10 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:35:11.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:10 smithi117 ceph-mon[67870]: Deploying daemon osd.1 on smithi022 2024-02-16T19:35:11.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.152+0000 7f8a07830700 1 -- 172.21.15.22:0/2533631414 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a000ffc60 msgr2=0x7f8a0010a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:35:11.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.152+0000 7f8a07830700 1 --2- 172.21.15.22:0/2533631414 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a000ffc60 0x7f8a0010a6d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f89f4009910 tx=0x7f89f4009c20 comp rx=0 tx=0).stop 2024-02-16T19:35:11.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.152+0000 7f8a07830700 1 -- 172.21.15.22:0/2533631414 shutdown_connections 2024-02-16T19:35:11.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.152+0000 7f8a07830700 1 --2- 172.21.15.22:0/2533631414 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a000ffc60 0x7f8a0010a6d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:11.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.152+0000 7f8a07830700 1 --2- 172.21.15.22:0/2533631414 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a000ff350 0x7f8a000ff720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:11.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.152+0000 7f8a07830700 1 -- 172.21.15.22:0/2533631414 >> 172.21.15.22:0/2533631414 conn(0x7f8a000748b0 msgr2=0x7f8a00074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:35:11.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.152+0000 7f8a07830700 1 -- 172.21.15.22:0/2533631414 shutdown_connections 2024-02-16T19:35:11.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.152+0000 7f8a07830700 1 -- 172.21.15.22:0/2533631414 wait complete. 2024-02-16T19:35:11.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.153+0000 7f8a07830700 1 Processor -- start 2024-02-16T19:35:11.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.153+0000 7f8a07830700 1 -- start start 2024-02-16T19:35:11.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.153+0000 7f8a07830700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a000ff350 0x7f8a00195ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:35:11.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.153+0000 7f8a07830700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a000ffc60 0x7f8a00196420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:35:11.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.153+0000 7f8a07830700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a00196b00 con 0x7f8a000ff350 2024-02-16T19:35:11.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.153+0000 7f8a07830700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0019a890 con 0x7f8a000ffc60 2024-02-16T19:35:11.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.153+0000 7f8a055cc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a000ff350 0x7f8a00195ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:35:11.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.153+0000 7f8a055cc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a000ff350 0x7f8a00195ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57706/0 (socket says 172.21.15.22:57706) 2024-02-16T19:35:11.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.153+0000 7f8a055cc700 1 -- 172.21.15.22:0/1357449451 learned_addr learned my addr 172.21.15.22:0/1357449451 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:35:11.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.153+0000 7f8a04dcb700 1 --2- 172.21.15.22:0/1357449451 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a000ffc60 0x7f8a00196420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:35:11.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.153+0000 7f8a055cc700 1 -- 172.21.15.22:0/1357449451 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a000ffc60 msgr2=0x7f8a00196420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:35:11.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.153+0000 7f8a055cc700 1 --2- 172.21.15.22:0/1357449451 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a000ffc60 0x7f8a00196420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:11.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.153+0000 7f8a055cc700 1 -- 172.21.15.22:0/1357449451 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89f40095f0 con 0x7f8a000ff350 2024-02-16T19:35:11.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.154+0000 7f8a055cc700 1 --2- 172.21.15.22:0/1357449451 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a000ff350 0x7f8a00195ee0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f89f000ada0 tx=0x7f89f000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:35:11.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.154+0000 7f89fe7fc700 1 -- 172.21.15.22:0/1357449451 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f89f000f070 con 0x7f8a000ff350 2024-02-16T19:35:11.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.154+0000 7f89fe7fc700 1 -- 172.21.15.22:0/1357449451 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f89f0007be0 con 0x7f8a000ff350 2024-02-16T19:35:11.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.154+0000 7f8a07830700 1 -- 172.21.15.22:0/1357449451 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a0019ab70 con 0x7f8a000ff350 2024-02-16T19:35:11.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.154+0000 7f89fe7fc700 1 -- 172.21.15.22:0/1357449451 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f89f00083f0 con 0x7f8a000ff350 2024-02-16T19:35:11.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.154+0000 7f8a07830700 1 -- 172.21.15.22:0/1357449451 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a0019b0c0 con 0x7f8a000ff350 2024-02-16T19:35:11.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.155+0000 7f8a07830700 1 -- 172.21.15.22:0/1357449451 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a00107dd0 con 0x7f8a000ff350 2024-02-16T19:35:11.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.156+0000 7f89fe7fc700 1 -- 172.21.15.22:0/1357449451 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f89f000cad0 con 0x7f8a000ff350 2024-02-16T19:35:11.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.156+0000 7f89fe7fc700 1 --2- 172.21.15.22:0/1357449451 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f89e806d8a0 0x7f89e806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:35:11.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.157+0000 7f89fe7fc700 1 -- 172.21.15.22:0/1357449451 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f89f008cd60 con 0x7f8a000ff350 2024-02-16T19:35:11.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.157+0000 7f8a04dcb700 1 --2- 172.21.15.22:0/1357449451 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f89e806d8a0 0x7f89e806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:35:11.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.157+0000 7f8a04dcb700 1 --2- 172.21.15.22:0/1357449451 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f89e806d8a0 0x7f89e806fd50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8a00197500 tx=0x7f89f401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:35:11.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.161+0000 7f89fe7fc700 1 -- 172.21.15.22:0/1357449451 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f89f0056640 con 0x7f8a000ff350 2024-02-16T19:35:11.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.388+0000 7f8a07830700 1 -- 172.21.15.22:0/1357449451 --> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8a0019b2a0 con 0x7f89e806d8a0 2024-02-16T19:35:11.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.389+0000 7f89fe7fc700 1 -- 172.21.15.22:0/1357449451 <== mgr.34104 v2:172.21.15.22:6800/505266753 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f8a0019b2a0 con 0x7f89e806d8a0 2024-02-16T19:35:11.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.391+0000 7f8a07830700 1 -- 172.21.15.22:0/1357449451 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f89e806d8a0 msgr2=0x7f89e806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:35:11.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.391+0000 7f8a07830700 1 --2- 172.21.15.22:0/1357449451 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f89e806d8a0 0x7f89e806fd50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8a00197500 tx=0x7f89f401a040 comp rx=0 tx=0).stop 2024-02-16T19:35:11.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.391+0000 7f8a07830700 1 -- 172.21.15.22:0/1357449451 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a000ff350 msgr2=0x7f8a00195ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:35:11.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.391+0000 7f8a07830700 1 --2- 172.21.15.22:0/1357449451 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a000ff350 0x7f8a00195ee0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f89f000ada0 tx=0x7f89f000c330 comp rx=0 tx=0).stop 2024-02-16T19:35:11.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.391+0000 7f8a07830700 1 -- 172.21.15.22:0/1357449451 shutdown_connections 2024-02-16T19:35:11.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.391+0000 7f8a07830700 1 --2- 172.21.15.22:0/1357449451 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f89e806d8a0 0x7f89e806fd50 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:11.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.391+0000 7f8a07830700 1 --2- 172.21.15.22:0/1357449451 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a000ff350 0x7f8a00195ee0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:11.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.391+0000 7f8a07830700 1 --2- 172.21.15.22:0/1357449451 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a000ffc60 0x7f8a00196420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:11.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.391+0000 7f8a07830700 1 -- 172.21.15.22:0/1357449451 >> 172.21.15.22:0/1357449451 conn(0x7f8a000748b0 msgr2=0x7f8a00104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:35:11.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.391+0000 7f8a07830700 1 -- 172.21.15.22:0/1357449451 shutdown_connections 2024-02-16T19:35:11.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.391+0000 7f8a07830700 1 -- 172.21.15.22:0/1357449451 wait complete. 2024-02-16T19:35:11.403 INFO:teuthology.orchestra.run.smithi022.stdout:true 2024-02-16T19:35:11.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.499+0000 7f3e1ecb9700 1 -- 172.21.15.22:0/798662843 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e18101d30 msgr2=0x7f3e1810a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:35:11.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.499+0000 7f3e1ecb9700 1 --2- 172.21.15.22:0/798662843 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e18101d30 0x7f3e1810a230 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3e04009910 tx=0x7f3e04009c20 comp rx=0 tx=0).stop 2024-02-16T19:35:11.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.499+0000 7f3e1ecb9700 1 -- 172.21.15.22:0/798662843 shutdown_connections 2024-02-16T19:35:11.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.499+0000 7f3e1ecb9700 1 --2- 172.21.15.22:0/798662843 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e18101d30 0x7f3e1810a230 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:11.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.499+0000 7f3e1ecb9700 1 --2- 172.21.15.22:0/798662843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e18101420 0x7f3e181017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:11.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.499+0000 7f3e1ecb9700 1 -- 172.21.15.22:0/798662843 >> 172.21.15.22:0/798662843 conn(0x7f3e180fac80 msgr2=0x7f3e180fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:35:11.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.499+0000 7f3e1ecb9700 1 -- 172.21.15.22:0/798662843 shutdown_connections 2024-02-16T19:35:11.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.499+0000 7f3e1ecb9700 1 -- 172.21.15.22:0/798662843 wait complete. 2024-02-16T19:35:11.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.500+0000 7f3e1ecb9700 1 Processor -- start 2024-02-16T19:35:11.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.500+0000 7f3e1ecb9700 1 -- start start 2024-02-16T19:35:11.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.500+0000 7f3e1ecb9700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e18101420 0x7f3e180ff080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:35:11.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.500+0000 7f3e1ecb9700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e18101d30 0x7f3e180ff5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:35:11.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.500+0000 7f3e1ecb9700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e180ffb00 con 0x7f3e18101420 2024-02-16T19:35:11.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.500+0000 7f3e1ecb9700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e180ffc40 con 0x7f3e18101d30 2024-02-16T19:35:11.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.500+0000 7f3e1ca55700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e18101420 0x7f3e180ff080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:35:11.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.500+0000 7f3e1ca55700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e18101420 0x7f3e180ff080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57736/0 (socket says 172.21.15.22:57736) 2024-02-16T19:35:11.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.500+0000 7f3e1ca55700 1 -- 172.21.15.22:0/1070644496 learned_addr learned my addr 172.21.15.22:0/1070644496 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:35:11.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.500+0000 7f3e17fff700 1 --2- 172.21.15.22:0/1070644496 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e18101d30 0x7f3e180ff5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:35:11.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.500+0000 7f3e1ca55700 1 -- 172.21.15.22:0/1070644496 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e18101d30 msgr2=0x7f3e180ff5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:35:11.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.500+0000 7f3e1ca55700 1 --2- 172.21.15.22:0/1070644496 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e18101d30 0x7f3e180ff5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:11.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.500+0000 7f3e1ca55700 1 -- 172.21.15.22:0/1070644496 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e040095f0 con 0x7f3e18101420 2024-02-16T19:35:11.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.501+0000 7f3e1ca55700 1 --2- 172.21.15.22:0/1070644496 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e18101420 0x7f3e180ff080 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c00e920 tx=0x7f3e0c00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:35:11.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.501+0000 7f3e15ffb700 1 -- 172.21.15.22:0/1070644496 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e0c00c9d0 con 0x7f3e18101420 2024-02-16T19:35:11.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.501+0000 7f3e1ecb9700 1 -- 172.21.15.22:0/1070644496 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e181a2050 con 0x7f3e18101420 2024-02-16T19:35:11.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.501+0000 7f3e15ffb700 1 -- 172.21.15.22:0/1070644496 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3e0c010430 con 0x7f3e18101420 2024-02-16T19:35:11.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.501+0000 7f3e15ffb700 1 -- 172.21.15.22:0/1070644496 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e0c018630 con 0x7f3e18101420 2024-02-16T19:35:11.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.501+0000 7f3e1ecb9700 1 -- 172.21.15.22:0/1070644496 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e181a2460 con 0x7f3e18101420 2024-02-16T19:35:11.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.502+0000 7f3e1ecb9700 1 -- 172.21.15.22:0/1070644496 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e181078a0 con 0x7f3e18101420 2024-02-16T19:35:11.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.502+0000 7f3e15ffb700 1 -- 172.21.15.22:0/1070644496 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f3e0c0105a0 con 0x7f3e18101420 2024-02-16T19:35:11.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.503+0000 7f3e15ffb700 1 --2- 172.21.15.22:0/1070644496 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f3e0806d8a0 0x7f3e0806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:35:11.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.503+0000 7f3e15ffb700 1 -- 172.21.15.22:0/1070644496 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f3e0c014070 con 0x7f3e18101420 2024-02-16T19:35:11.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.503+0000 7f3e17fff700 1 --2- 172.21.15.22:0/1070644496 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f3e0806d8a0 0x7f3e0806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:35:11.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.503+0000 7f3e17fff700 1 --2- 172.21.15.22:0/1070644496 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f3e0806d8a0 0x7f3e0806fd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3e18100870 tx=0x7f3e04005940 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:35:11.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.508+0000 7f3e15ffb700 1 -- 172.21.15.22:0/1070644496 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3e0c052d20 con 0x7f3e18101420 2024-02-16T19:35:11.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.742+0000 7f3e1ecb9700 1 -- 172.21.15.22:0/1070644496 --> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3e18061190 con 0x7f3e0806d8a0 2024-02-16T19:35:11.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.743+0000 7f3e15ffb700 1 -- 172.21.15.22:0/1070644496 <== mgr.34104 v2:172.21.15.22:6800/505266753 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f3e18061190 con 0x7f3e0806d8a0 2024-02-16T19:35:11.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.745+0000 7f3e1ecb9700 1 -- 172.21.15.22:0/1070644496 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f3e0806d8a0 msgr2=0x7f3e0806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:35:11.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.745+0000 7f3e1ecb9700 1 --2- 172.21.15.22:0/1070644496 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f3e0806d8a0 0x7f3e0806fd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3e18100870 tx=0x7f3e04005940 comp rx=0 tx=0).stop 2024-02-16T19:35:11.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.745+0000 7f3e1ecb9700 1 -- 172.21.15.22:0/1070644496 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e18101420 msgr2=0x7f3e180ff080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:35:11.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.745+0000 7f3e1ecb9700 1 --2- 172.21.15.22:0/1070644496 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e18101420 0x7f3e180ff080 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c00e920 tx=0x7f3e0c00ec30 comp rx=0 tx=0).stop 2024-02-16T19:35:11.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.745+0000 7f3e1ecb9700 1 -- 172.21.15.22:0/1070644496 shutdown_connections 2024-02-16T19:35:11.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.745+0000 7f3e1ecb9700 1 --2- 172.21.15.22:0/1070644496 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f3e0806d8a0 0x7f3e0806fd50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:11.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.745+0000 7f3e1ecb9700 1 --2- 172.21.15.22:0/1070644496 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e18101420 0x7f3e180ff080 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:11.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.745+0000 7f3e1ecb9700 1 --2- 172.21.15.22:0/1070644496 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e18101d30 0x7f3e180ff5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:11.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.745+0000 7f3e1ecb9700 1 -- 172.21.15.22:0/1070644496 >> 172.21.15.22:0/1070644496 conn(0x7f3e180fac80 msgr2=0x7f3e181049e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:35:11.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.745+0000 7f3e1ecb9700 1 -- 172.21.15.22:0/1070644496 shutdown_connections 2024-02-16T19:35:11.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.745+0000 7f3e1ecb9700 1 -- 172.21.15.22:0/1070644496 wait complete. 2024-02-16T19:35:11.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.857+0000 7fc4b0907700 1 -- 172.21.15.22:0/1398495764 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4ac101c00 msgr2=0x7fc4ac10a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:35:11.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.857+0000 7fc4b0907700 1 --2- 172.21.15.22:0/1398495764 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4ac101c00 0x7fc4ac10a2d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0009910 tx=0x7fc4a0009c20 comp rx=0 tx=0).stop 2024-02-16T19:35:11.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.857+0000 7fc4b0907700 1 -- 172.21.15.22:0/1398495764 shutdown_connections 2024-02-16T19:35:11.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.857+0000 7fc4b0907700 1 --2- 172.21.15.22:0/1398495764 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4ac101c00 0x7fc4ac10a2d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:11.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.857+0000 7fc4b0907700 1 --2- 172.21.15.22:0/1398495764 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4ac1012f0 0x7fc4ac1016c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:11.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.857+0000 7fc4b0907700 1 -- 172.21.15.22:0/1398495764 >> 172.21.15.22:0/1398495764 conn(0x7fc4ac0fac80 msgr2=0x7fc4ac0fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:35:11.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.857+0000 7fc4b0907700 1 -- 172.21.15.22:0/1398495764 shutdown_connections 2024-02-16T19:35:11.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.857+0000 7fc4b0907700 1 -- 172.21.15.22:0/1398495764 wait complete. 2024-02-16T19:35:11.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.858+0000 7fc4b0907700 1 Processor -- start 2024-02-16T19:35:11.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.858+0000 7fc4b0907700 1 -- start start 2024-02-16T19:35:11.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.858+0000 7fc4b0907700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4ac1012f0 0x7fc4ac198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:35:11.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.858+0000 7fc4b0907700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4ac101c00 0x7fc4ac198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:35:11.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.858+0000 7fc4b0907700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4ac198e50 con 0x7fc4ac1012f0 2024-02-16T19:35:11.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.858+0000 7fc4b0907700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4ac19cb90 con 0x7fc4ac101c00 2024-02-16T19:35:11.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.858+0000 7fc4aa59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4ac1012f0 0x7fc4ac198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:35:11.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.858+0000 7fc4aa59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4ac1012f0 0x7fc4ac198230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57754/0 (socket says 172.21.15.22:57754) 2024-02-16T19:35:11.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.858+0000 7fc4aa59c700 1 -- 172.21.15.22:0/3843042099 learned_addr learned my addr 172.21.15.22:0/3843042099 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:35:11.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.858+0000 7fc4a9d9b700 1 --2- 172.21.15.22:0/3843042099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4ac101c00 0x7fc4ac198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:35:11.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.858+0000 7fc4aa59c700 1 -- 172.21.15.22:0/3843042099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4ac101c00 msgr2=0x7fc4ac198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:35:11.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.858+0000 7fc4aa59c700 1 --2- 172.21.15.22:0/3843042099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4ac101c00 0x7fc4ac198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:11.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.858+0000 7fc4aa59c700 1 -- 172.21.15.22:0/3843042099 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4a00095f0 con 0x7fc4ac1012f0 2024-02-16T19:35:11.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.859+0000 7fc4aa59c700 1 --2- 172.21.15.22:0/3843042099 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4ac1012f0 0x7fc4ac198230 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc49c00b4d0 tx=0x7fc49c00b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:35:11.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.859+0000 7fc49b7fe700 1 -- 172.21.15.22:0/3843042099 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc49c011800 con 0x7fc4ac1012f0 2024-02-16T19:35:11.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.859+0000 7fc49b7fe700 1 -- 172.21.15.22:0/3843042099 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc49c011e20 con 0x7fc4ac1012f0 2024-02-16T19:35:11.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.859+0000 7fc4b0907700 1 -- 172.21.15.22:0/3843042099 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4ac19ce70 con 0x7fc4ac1012f0 2024-02-16T19:35:11.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.859+0000 7fc49b7fe700 1 -- 172.21.15.22:0/3843042099 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc49c00f2e0 con 0x7fc4ac1012f0 2024-02-16T19:35:11.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.859+0000 7fc4b0907700 1 -- 172.21.15.22:0/3843042099 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4ac19d360 con 0x7fc4ac1012f0 2024-02-16T19:35:11.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.860+0000 7fc4b0907700 1 -- 172.21.15.22:0/3843042099 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4ac109dd0 con 0x7fc4ac1012f0 2024-02-16T19:35:11.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.860+0000 7fc49b7fe700 1 -- 172.21.15.22:0/3843042099 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fc49c010380 con 0x7fc4ac1012f0 2024-02-16T19:35:11.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.861+0000 7fc49b7fe700 1 --2- 172.21.15.22:0/3843042099 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fc49406d8a0 0x7fc49406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:35:11.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.861+0000 7fc49b7fe700 1 -- 172.21.15.22:0/3843042099 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fc49c08ce80 con 0x7fc4ac1012f0 2024-02-16T19:35:11.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.861+0000 7fc4a9d9b700 1 --2- 172.21.15.22:0/3843042099 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fc49406d8a0 0x7fc49406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:35:11.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.861+0000 7fc4a9d9b700 1 --2- 172.21.15.22:0/3843042099 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fc49406d8a0 0x7fc49406fd50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac199800 tx=0x7fc4a001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:35:11.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:11.866+0000 7fc49b7fe700 1 -- 172.21.15.22:0/3843042099 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc49c056810 con 0x7fc4ac1012f0 2024-02-16T19:35:11.978 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:11 smithi022 ceph-mon[88798]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:35:11.979 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:11 smithi022 ceph-mon[88798]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:35:12.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.109+0000 7fc4b0907700 1 -- 172.21.15.22:0/3843042099 --> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc4ac19d690 con 0x7fc49406d8a0 2024-02-16T19:35:12.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.130+0000 7fc49b7fe700 1 -- 172.21.15.22:0/3843042099 <== mgr.34104 v2:172.21.15.22:6800/505266753 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fc4ac19d690 con 0x7fc49406d8a0 2024-02-16T19:35:12.131 INFO:teuthology.orchestra.run.smithi022.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T19:35:12.131 INFO:teuthology.orchestra.run.smithi022.stdout:alertmanager.smithi022 smithi022 *:9093,9094 running (3m) 23s ago 14m 27.7M - 0.25.0 c8568f914cd2 c89bc76be374 2024-02-16T19:35:12.132 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-exporter.smithi022 smithi022 running (14m) 23s ago 14m 8120k - 18.2.0 dc2bc1663786 3f8bef4349ba 2024-02-16T19:35:12.132 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-exporter.smithi117 smithi117 running (13m) 47s ago 13m 7768k - 18.2.0 dc2bc1663786 59b65a7e309f 2024-02-16T19:35:12.132 INFO:teuthology.orchestra.run.smithi022.stdout:crash.smithi022 smithi022 running (56s) 23s ago 14m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a e0f78bdf43e4 2024-02-16T19:35:12.132 INFO:teuthology.orchestra.run.smithi022.stdout:crash.smithi117 smithi117 running (50s) 47s ago 13m 6669k - 19.0.0-1450-gf78a58c0 f081f847070a bce7c0590813 2024-02-16T19:35:12.132 INFO:teuthology.orchestra.run.smithi022.stdout:grafana.smithi022 smithi022 *:3000 running (3m) 23s ago 14m 76.5M - 9.4.12 a0ea5e3c46d9 fd27d11e9729 2024-02-16T19:35:12.132 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi022.btppwm smithi022 running (8m) 23s ago 8m 155M - 18.2.0 dc2bc1663786 22971fb3a907 2024-02-16T19:35:12.132 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi022.ocukvs smithi022 running (8m) 23s ago 8m 16.6M - 18.2.0 dc2bc1663786 81ee7bd5fd39 2024-02-16T19:35:12.132 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi117.ibmppn smithi117 running (8m) 47s ago 8m 17.6M - 18.2.0 dc2bc1663786 8c606cbeeb86 2024-02-16T19:35:12.132 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi117.orzors smithi117 running (8m) 47s ago 8m 16.6M - 18.2.0 dc2bc1663786 7d2bc4df3feb 2024-02-16T19:35:12.132 INFO:teuthology.orchestra.run.smithi022.stdout:mgr.smithi022.gmdcch smithi022 *:8443,9283,8765 running (5m) 23s ago 15m 497M - 19.0.0-1450-gf78a58c0 f081f847070a 7cb74cc9deea 2024-02-16T19:35:12.132 INFO:teuthology.orchestra.run.smithi022.stdout:mgr.smithi117.cohbtv smithi117 *:8443,9283,8765 running (4m) 47s ago 13m 451M - 19.0.0-1450-gf78a58c0 f081f847070a c825b1152189 2024-02-16T19:35:12.132 INFO:teuthology.orchestra.run.smithi022.stdout:mon.smithi022 smithi022 running (2m) 23s ago 16m 45.0M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 9c84755ae67d 2024-02-16T19:35:12.132 INFO:teuthology.orchestra.run.smithi022.stdout:mon.smithi117 smithi117 running (100s) 47s ago 13m 38.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a a319ac50f1a9 2024-02-16T19:35:12.132 INFO:teuthology.orchestra.run.smithi022.stdout:node-exporter.smithi022 smithi022 *:9100 running (4m) 23s ago 14m 16.6M - 1.5.0 0da6a335fe13 1f5b58292d27 2024-02-16T19:35:12.133 INFO:teuthology.orchestra.run.smithi022.stdout:node-exporter.smithi117 smithi117 *:9100 running (4m) 47s ago 13m 15.4M - 1.5.0 0da6a335fe13 bb5e8491d5a6 2024-02-16T19:35:12.133 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0 smithi022 running (29s) 23s ago 12m 12.3M 1217M 19.0.0-1450-gf78a58c0 f081f847070a 002412b3525f 2024-02-16T19:35:12.133 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1 smithi022 running (12m) 23s ago 12m 227M 1217M 18.2.0 dc2bc1663786 87300382b8a7 2024-02-16T19:35:12.133 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2 smithi022 running (11m) 23s ago 11m 188M 1217M 18.2.0 dc2bc1663786 be8a631b4f93 2024-02-16T19:35:12.133 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3 smithi117 running (11m) 47s ago 11m 244M 2240M 18.2.0 dc2bc1663786 9c0ce9a981cf 2024-02-16T19:35:12.133 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4 smithi117 running (10m) 47s ago 10m 209M 2240M 18.2.0 dc2bc1663786 1779c3496f4d 2024-02-16T19:35:12.133 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5 smithi117 running (10m) 47s ago 10m 198M 2240M 18.2.0 dc2bc1663786 fefaabca0638 2024-02-16T19:35:12.133 INFO:teuthology.orchestra.run.smithi022.stdout:prometheus.smithi022 smithi022 *:9095 running (3m) 23s ago 13m 66.9M - 2.43.0 a07b618ecd1d c3b8689d0c97 2024-02-16T19:35:12.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.132+0000 7fc4b0907700 1 -- 172.21.15.22:0/3843042099 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fc49406d8a0 msgr2=0x7fc49406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:35:12.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.132+0000 7fc4b0907700 1 --2- 172.21.15.22:0/3843042099 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fc49406d8a0 0x7fc49406fd50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac199800 tx=0x7fc4a001a040 comp rx=0 tx=0).stop 2024-02-16T19:35:12.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.132+0000 7fc4b0907700 1 -- 172.21.15.22:0/3843042099 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4ac1012f0 msgr2=0x7fc4ac198230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:35:12.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.132+0000 7fc4b0907700 1 --2- 172.21.15.22:0/3843042099 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4ac1012f0 0x7fc4ac198230 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc49c00b4d0 tx=0x7fc49c00b7e0 comp rx=0 tx=0).stop 2024-02-16T19:35:12.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.132+0000 7fc4b0907700 1 -- 172.21.15.22:0/3843042099 shutdown_connections 2024-02-16T19:35:12.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.132+0000 7fc4b0907700 1 --2- 172.21.15.22:0/3843042099 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fc49406d8a0 0x7fc49406fd50 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:12.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.132+0000 7fc4b0907700 1 --2- 172.21.15.22:0/3843042099 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4ac1012f0 0x7fc4ac198230 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:12.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.132+0000 7fc4b0907700 1 --2- 172.21.15.22:0/3843042099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4ac101c00 0x7fc4ac198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:12.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.132+0000 7fc4b0907700 1 -- 172.21.15.22:0/3843042099 >> 172.21.15.22:0/3843042099 conn(0x7fc4ac0fac80 msgr2=0x7fc4ac0ff8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:35:12.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.132+0000 7fc4b0907700 1 -- 172.21.15.22:0/3843042099 shutdown_connections 2024-02-16T19:35:12.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.132+0000 7fc4b0907700 1 -- 172.21.15.22:0/3843042099 wait complete. 2024-02-16T19:35:12.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:11 smithi117 ceph-mon[67870]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:35:12.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:11 smithi117 ceph-mon[67870]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:35:12.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.240+0000 7f47bdca8700 1 -- 172.21.15.22:0/1596249025 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f47b80ffbd0 msgr2=0x7f47b8100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:35:12.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.240+0000 7f47bdca8700 1 --2- 172.21.15.22:0/1596249025 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f47b80ffbd0 0x7f47b8100040 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f47ac009910 tx=0x7f47ac009c20 comp rx=0 tx=0).stop 2024-02-16T19:35:12.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.240+0000 7f47bdca8700 1 -- 172.21.15.22:0/1596249025 shutdown_connections 2024-02-16T19:35:12.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.240+0000 7f47bdca8700 1 --2- 172.21.15.22:0/1596249025 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f47b80ffbd0 0x7f47b8100040 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:12.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.240+0000 7f47bdca8700 1 --2- 172.21.15.22:0/1596249025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47b8104130 0x7f47b8104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:12.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.240+0000 7f47bdca8700 1 -- 172.21.15.22:0/1596249025 >> 172.21.15.22:0/1596249025 conn(0x7f47b8075130 msgr2=0x7f47b8075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:35:12.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.240+0000 7f47bdca8700 1 -- 172.21.15.22:0/1596249025 shutdown_connections 2024-02-16T19:35:12.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.240+0000 7f47bdca8700 1 -- 172.21.15.22:0/1596249025 wait complete. 2024-02-16T19:35:12.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.241+0000 7f47bdca8700 1 Processor -- start 2024-02-16T19:35:12.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.241+0000 7f47bdca8700 1 -- start start 2024-02-16T19:35:12.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.241+0000 7f47bdca8700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47b80ffbd0 0x7f47b8195e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:35:12.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.241+0000 7f47bdca8700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f47b8104130 0x7f47b8196390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:35:12.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.241+0000 7f47bdca8700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b81969e0 con 0x7f47b8104130 2024-02-16T19:35:12.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.241+0000 7f47bdca8700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b8196b20 con 0x7f47b80ffbd0 2024-02-16T19:35:12.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.241+0000 7f47b6ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f47b8104130 0x7f47b8196390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:35:12.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.241+0000 7f47b77fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47b80ffbd0 0x7f47b8195e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:35:12.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.241+0000 7f47b6ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f47b8104130 0x7f47b8196390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57772/0 (socket says 172.21.15.22:57772) 2024-02-16T19:35:12.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.241+0000 7f47b6ffd700 1 -- 172.21.15.22:0/2229484785 learned_addr learned my addr 172.21.15.22:0/2229484785 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:35:12.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.242+0000 7f47b6ffd700 1 -- 172.21.15.22:0/2229484785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47b80ffbd0 msgr2=0x7f47b8195e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:35:12.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.242+0000 7f47b6ffd700 1 --2- 172.21.15.22:0/2229484785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47b80ffbd0 0x7f47b8195e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:12.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.242+0000 7f47b6ffd700 1 -- 172.21.15.22:0/2229484785 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47ac0095f0 con 0x7f47b8104130 2024-02-16T19:35:12.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.242+0000 7f47b6ffd700 1 --2- 172.21.15.22:0/2229484785 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f47b8104130 0x7f47b8196390 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f47ac006010 tx=0x7f47ac00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:35:12.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.242+0000 7f47b4ff9700 1 -- 172.21.15.22:0/2229484785 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47ac01d070 con 0x7f47b8104130 2024-02-16T19:35:12.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.242+0000 7f47bdca8700 1 -- 172.21.15.22:0/2229484785 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47b819a910 con 0x7f47b8104130 2024-02-16T19:35:12.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.242+0000 7f47b4ff9700 1 -- 172.21.15.22:0/2229484785 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f47ac00f460 con 0x7f47b8104130 2024-02-16T19:35:12.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.242+0000 7f47bdca8700 1 -- 172.21.15.22:0/2229484785 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47b819ae00 con 0x7f47b8104130 2024-02-16T19:35:12.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.242+0000 7f47b4ff9700 1 -- 172.21.15.22:0/2229484785 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47ac0175f0 con 0x7f47b8104130 2024-02-16T19:35:12.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.243+0000 7f47bdca8700 1 -- 172.21.15.22:0/2229484785 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47b81086a0 con 0x7f47b8104130 2024-02-16T19:35:12.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.244+0000 7f47b4ff9700 1 -- 172.21.15.22:0/2229484785 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f47ac00f5d0 con 0x7f47b8104130 2024-02-16T19:35:12.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.244+0000 7f47b4ff9700 1 --2- 172.21.15.22:0/2229484785 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f47a006d8a0 0x7f47a006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:35:12.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.245+0000 7f47b4ff9700 1 -- 172.21.15.22:0/2229484785 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f47ac08e920 con 0x7f47b8104130 2024-02-16T19:35:12.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.245+0000 7f47b77fe700 1 --2- 172.21.15.22:0/2229484785 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f47a006d8a0 0x7f47a006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:35:12.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.245+0000 7f47b77fe700 1 --2- 172.21.15.22:0/2229484785 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f47a006d8a0 0x7f47a006fd50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f47a800ac50 tx=0x7f47a800a380 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:35:12.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.252+0000 7f47b4ff9700 1 -- 172.21.15.22:0/2229484785 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f47ac058200 con 0x7f47b8104130 2024-02-16T19:35:12.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.557+0000 7f47bdca8700 1 -- 172.21.15.22:0/2229484785 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f47b8197250 con 0x7f47b8104130 2024-02-16T19:35:12.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.558+0000 7f47b4ff9700 1 -- 172.21.15.22:0/2229484785 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+791 (secure 0 0 0) 0x7f47ac05b8c0 con 0x7f47b8104130 2024-02-16T19:35:12.559 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-02-16T19:35:12.559 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": { 2024-02-16T19:35:12.559 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T19:35:12.560 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:35:12.560 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": { 2024-02-16T19:35:12.560 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T19:35:12.560 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:35:12.560 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": { 2024-02-16T19:35:12.560 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 5, 2024-02-16T19:35:12.560 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 1 2024-02-16T19:35:12.560 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:35:12.560 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": { 2024-02-16T19:35:12.560 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-02-16T19:35:12.560 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:35:12.560 INFO:teuthology.orchestra.run.smithi022.stdout: "overall": { 2024-02-16T19:35:12.560 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 9, 2024-02-16T19:35:12.560 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 5 2024-02-16T19:35:12.561 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-02-16T19:35:12.561 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-02-16T19:35:12.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.560+0000 7f47bdca8700 1 -- 172.21.15.22:0/2229484785 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f47a006d8a0 msgr2=0x7f47a006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:35:12.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.560+0000 7f47bdca8700 1 --2- 172.21.15.22:0/2229484785 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f47a006d8a0 0x7f47a006fd50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f47a800ac50 tx=0x7f47a800a380 comp rx=0 tx=0).stop 2024-02-16T19:35:12.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.560+0000 7f47bdca8700 1 -- 172.21.15.22:0/2229484785 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f47b8104130 msgr2=0x7f47b8196390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:35:12.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.560+0000 7f47bdca8700 1 --2- 172.21.15.22:0/2229484785 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f47b8104130 0x7f47b8196390 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f47ac006010 tx=0x7f47ac00bd70 comp rx=0 tx=0).stop 2024-02-16T19:35:12.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.560+0000 7f47bdca8700 1 -- 172.21.15.22:0/2229484785 shutdown_connections 2024-02-16T19:35:12.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.560+0000 7f47bdca8700 1 --2- 172.21.15.22:0/2229484785 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f47a006d8a0 0x7f47a006fd50 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:12.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.560+0000 7f47bdca8700 1 --2- 172.21.15.22:0/2229484785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47b80ffbd0 0x7f47b8195e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:12.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.560+0000 7f47bdca8700 1 --2- 172.21.15.22:0/2229484785 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f47b8104130 0x7f47b8196390 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:12.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.560+0000 7f47bdca8700 1 -- 172.21.15.22:0/2229484785 >> 172.21.15.22:0/2229484785 conn(0x7f47b8075130 msgr2=0x7f47b80febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:35:12.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.560+0000 7f47bdca8700 1 -- 172.21.15.22:0/2229484785 shutdown_connections 2024-02-16T19:35:12.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.560+0000 7f47bdca8700 1 -- 172.21.15.22:0/2229484785 wait complete. 2024-02-16T19:35:12.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.667+0000 7ff5c2df8700 1 -- 172.21.15.22:0/2148157144 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5bc100e80 msgr2=0x7ff5bc104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:35:12.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.667+0000 7ff5c2df8700 1 --2- 172.21.15.22:0/2148157144 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5bc100e80 0x7ff5bc104e50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac00b0a0 tx=0x7ff5ac00b3b0 comp rx=0 tx=0).stop 2024-02-16T19:35:12.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.667+0000 7ff5c2df8700 1 -- 172.21.15.22:0/2148157144 shutdown_connections 2024-02-16T19:35:12.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.667+0000 7ff5c2df8700 1 --2- 172.21.15.22:0/2148157144 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5bc100e80 0x7ff5bc104e50 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:12.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.667+0000 7ff5c2df8700 1 --2- 172.21.15.22:0/2148157144 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff5bc1004e0 0x7ff5bc1008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:12.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.667+0000 7ff5c2df8700 1 -- 172.21.15.22:0/2148157144 >> 172.21.15.22:0/2148157144 conn(0x7ff5bc0fbda0 msgr2=0x7ff5bc0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:35:12.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.667+0000 7ff5c2df8700 1 -- 172.21.15.22:0/2148157144 shutdown_connections 2024-02-16T19:35:12.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.667+0000 7ff5c2df8700 1 -- 172.21.15.22:0/2148157144 wait complete. 2024-02-16T19:35:12.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.667+0000 7ff5c2df8700 1 Processor -- start 2024-02-16T19:35:12.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.667+0000 7ff5c2df8700 1 -- start start 2024-02-16T19:35:12.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.668+0000 7ff5c2df8700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff5bc1004e0 0x7ff5bc073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:35:12.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.668+0000 7ff5c2df8700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5bc100e80 0x7ff5bc073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:35:12.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.668+0000 7ff5c2df8700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5bc073ad0 con 0x7ff5bc100e80 2024-02-16T19:35:12.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.668+0000 7ff5c2df8700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5bc073c10 con 0x7ff5bc1004e0 2024-02-16T19:35:12.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.668+0000 7ff5bbfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5bc100e80 0x7ff5bc073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:35:12.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.668+0000 7ff5bbfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5bc100e80 0x7ff5bc073590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57796/0 (socket says 172.21.15.22:57796) 2024-02-16T19:35:12.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.668+0000 7ff5bbfff700 1 -- 172.21.15.22:0/872891424 learned_addr learned my addr 172.21.15.22:0/872891424 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:35:12.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.668+0000 7ff5c0b94700 1 --2- 172.21.15.22:0/872891424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff5bc1004e0 0x7ff5bc073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:35:12.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.668+0000 7ff5bbfff700 1 -- 172.21.15.22:0/872891424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff5bc1004e0 msgr2=0x7ff5bc073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:35:12.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.668+0000 7ff5bbfff700 1 --2- 172.21.15.22:0/872891424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff5bc1004e0 0x7ff5bc073050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:12.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.668+0000 7ff5bbfff700 1 -- 172.21.15.22:0/872891424 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5ac009d00 con 0x7ff5bc100e80 2024-02-16T19:35:12.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.668+0000 7ff5bbfff700 1 --2- 172.21.15.22:0/872891424 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5bc100e80 0x7ff5bc073590 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac006130 tx=0x7ff5ac00cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:35:12.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.668+0000 7ff5b9ffb700 1 -- 172.21.15.22:0/872891424 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff5ac00e040 con 0x7ff5bc100e80 2024-02-16T19:35:12.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.668+0000 7ff5b9ffb700 1 -- 172.21.15.22:0/872891424 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff5ac003c10 con 0x7ff5bc100e80 2024-02-16T19:35:12.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.668+0000 7ff5c2df8700 1 -- 172.21.15.22:0/872891424 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5bc073e90 con 0x7ff5bc100e80 2024-02-16T19:35:12.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.668+0000 7ff5b9ffb700 1 -- 172.21.15.22:0/872891424 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff5ac01d6b0 con 0x7ff5bc100e80 2024-02-16T19:35:12.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.669+0000 7ff5c2df8700 1 -- 172.21.15.22:0/872891424 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5bc1a2410 con 0x7ff5bc100e80 2024-02-16T19:35:12.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.670+0000 7ff5c2df8700 1 -- 172.21.15.22:0/872891424 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5bc108850 con 0x7ff5bc100e80 2024-02-16T19:35:12.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.670+0000 7ff5b9ffb700 1 -- 172.21.15.22:0/872891424 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7ff5ac004600 con 0x7ff5bc100e80 2024-02-16T19:35:12.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.670+0000 7ff5b9ffb700 1 --2- 172.21.15.22:0/872891424 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7ff5a806d8a0 0x7ff5a806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:35:12.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.671+0000 7ff5b9ffb700 1 -- 172.21.15.22:0/872891424 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6414+0+0 (secure 0 0 0) 0x7ff5ac08ed30 con 0x7ff5bc100e80 2024-02-16T19:35:12.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.671+0000 7ff5c0b94700 1 --2- 172.21.15.22:0/872891424 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7ff5a806d8a0 0x7ff5a806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:35:12.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.671+0000 7ff5c0b94700 1 --2- 172.21.15.22:0/872891424 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7ff5a806d8a0 0x7ff5a806fd50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff5b4005950 tx=0x7ff5b400b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:35:12.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.675+0000 7ff5b9ffb700 1 -- 172.21.15.22:0/872891424 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff5ac0584f0 con 0x7ff5bc100e80 2024-02-16T19:35:12.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.929+0000 7ff5c2df8700 1 -- 172.21.15.22:0/872891424 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff5bc074670 con 0x7ff5bc100e80 2024-02-16T19:35:12.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.929+0000 7ff5b9ffb700 1 -- 172.21.15.22:0/872891424 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1845 (secure 0 0 0) 0x7ff5ac05bbb0 con 0x7ff5bc100e80 2024-02-16T19:35:12.931 INFO:teuthology.orchestra.run.smithi022.stdout:e15 2024-02-16T19:35:12.931 INFO:teuthology.orchestra.run.smithi022.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T19:35:12.931 INFO:teuthology.orchestra.run.smithi022.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:35:12.931 INFO:teuthology.orchestra.run.smithi022.stdout:legacy client fscid: 1 2024-02-16T19:35:12.931 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:35:12.931 INFO:teuthology.orchestra.run.smithi022.stdout:Filesystem 'cephfs' (1) 2024-02-16T19:35:12.931 INFO:teuthology.orchestra.run.smithi022.stdout:fs_name cephfs 2024-02-16T19:35:12.931 INFO:teuthology.orchestra.run.smithi022.stdout:epoch 15 2024-02-16T19:35:12.931 INFO:teuthology.orchestra.run.smithi022.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T19:35:12.932 INFO:teuthology.orchestra.run.smithi022.stdout:created 2024-02-16T19:26:17.873644+0000 2024-02-16T19:35:12.932 INFO:teuthology.orchestra.run.smithi022.stdout:modified 2024-02-16T19:26:35.943353+0000 2024-02-16T19:35:12.932 INFO:teuthology.orchestra.run.smithi022.stdout:tableserver 0 2024-02-16T19:35:12.932 INFO:teuthology.orchestra.run.smithi022.stdout:root 0 2024-02-16T19:35:12.932 INFO:teuthology.orchestra.run.smithi022.stdout:session_timeout 60 2024-02-16T19:35:12.932 INFO:teuthology.orchestra.run.smithi022.stdout:session_autoclose 300 2024-02-16T19:35:12.932 INFO:teuthology.orchestra.run.smithi022.stdout:max_file_size 1099511627776 2024-02-16T19:35:12.932 INFO:teuthology.orchestra.run.smithi022.stdout:max_xattr_size 65536 2024-02-16T19:35:12.932 INFO:teuthology.orchestra.run.smithi022.stdout:required_client_features {} 2024-02-16T19:35:12.932 INFO:teuthology.orchestra.run.smithi022.stdout:last_failure 0 2024-02-16T19:35:12.932 INFO:teuthology.orchestra.run.smithi022.stdout:last_failure_osd_epoch 47 2024-02-16T19:35:12.932 INFO:teuthology.orchestra.run.smithi022.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:35:12.932 INFO:teuthology.orchestra.run.smithi022.stdout:max_mds 1 2024-02-16T19:35:12.932 INFO:teuthology.orchestra.run.smithi022.stdout:in 0 2024-02-16T19:35:12.933 INFO:teuthology.orchestra.run.smithi022.stdout:up {0=14490} 2024-02-16T19:35:12.933 INFO:teuthology.orchestra.run.smithi022.stdout:failed 2024-02-16T19:35:12.933 INFO:teuthology.orchestra.run.smithi022.stdout:damaged 2024-02-16T19:35:12.933 INFO:teuthology.orchestra.run.smithi022.stdout:stopped 2024-02-16T19:35:12.933 INFO:teuthology.orchestra.run.smithi022.stdout:data_pools [3] 2024-02-16T19:35:12.933 INFO:teuthology.orchestra.run.smithi022.stdout:metadata_pool 2 2024-02-16T19:35:12.933 INFO:teuthology.orchestra.run.smithi022.stdout:inline_data enabled 2024-02-16T19:35:12.933 INFO:teuthology.orchestra.run.smithi022.stdout:balancer 2024-02-16T19:35:12.933 INFO:teuthology.orchestra.run.smithi022.stdout:bal_rank_mask -1 2024-02-16T19:35:12.933 INFO:teuthology.orchestra.run.smithi022.stdout:standby_count_wanted 1 2024-02-16T19:35:12.933 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.cephfs.smithi022.btppwm{0:14490} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.22:6828/3597685751,v1:172.21.15.22:6829/3597685751] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:35:12.933 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:35:12.933 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:35:12.933 INFO:teuthology.orchestra.run.smithi022.stdout:Standby daemons: 2024-02-16T19:35:12.933 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:35:12.934 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.cephfs.smithi022.ocukvs{-1:14500} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.22:6826/242865843,v1:172.21.15.22:6827/242865843] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:35:12.934 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.cephfs.smithi117.ibmppn{-1:24259} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.117:6824/565408102,v1:172.21.15.117:6825/565408102] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:35:12.934 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.cephfs.smithi117.orzors{-1:24265} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.117:6826/3122342028,v1:172.21.15.117:6827/3122342028] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:35:12.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.931+0000 7ff5c2df8700 1 -- 172.21.15.22:0/872891424 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7ff5a806d8a0 msgr2=0x7ff5a806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:35:12.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.931+0000 7ff5c2df8700 1 --2- 172.21.15.22:0/872891424 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7ff5a806d8a0 0x7ff5a806fd50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff5b4005950 tx=0x7ff5b400b410 comp rx=0 tx=0).stop 2024-02-16T19:35:12.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.931+0000 7ff5c2df8700 1 -- 172.21.15.22:0/872891424 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5bc100e80 msgr2=0x7ff5bc073590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:35:12.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.931+0000 7ff5c2df8700 1 --2- 172.21.15.22:0/872891424 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5bc100e80 0x7ff5bc073590 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac006130 tx=0x7ff5ac00cd00 comp rx=0 tx=0).stop 2024-02-16T19:35:12.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.931+0000 7ff5c2df8700 1 -- 172.21.15.22:0/872891424 shutdown_connections 2024-02-16T19:35:12.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.931+0000 7ff5c2df8700 1 --2- 172.21.15.22:0/872891424 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7ff5a806d8a0 0x7ff5a806fd50 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:12.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.931+0000 7ff5c2df8700 1 --2- 172.21.15.22:0/872891424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff5bc1004e0 0x7ff5bc073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:12.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.931+0000 7ff5c2df8700 1 --2- 172.21.15.22:0/872891424 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5bc100e80 0x7ff5bc073590 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:12.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.931+0000 7ff5c2df8700 1 -- 172.21.15.22:0/872891424 >> 172.21.15.22:0/872891424 conn(0x7ff5bc0fbda0 msgr2=0x7ff5bc0fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:35:12.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.931+0000 7ff5c2df8700 1 -- 172.21.15.22:0/872891424 shutdown_connections 2024-02-16T19:35:12.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:12.931+0000 7ff5c2df8700 1 -- 172.21.15.22:0/872891424 wait complete. 2024-02-16T19:35:12.935 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 15 2024-02-16T19:35:13.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.040+0000 7f4f9fb60700 1 -- 172.21.15.22:0/4146778410 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f981004a0 msgr2=0x7f4f98100870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:35:13.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.040+0000 7f4f9fb60700 1 --2- 172.21.15.22:0/4146778410 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f981004a0 0x7f4f98100870 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4f8800b0a0 tx=0x7f4f8800b3b0 comp rx=0 tx=0).stop 2024-02-16T19:35:13.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.040+0000 7f4f9fb60700 1 -- 172.21.15.22:0/4146778410 shutdown_connections 2024-02-16T19:35:13.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.040+0000 7f4f9fb60700 1 --2- 172.21.15.22:0/4146778410 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f98100e40 0x7f4f98104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:13.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.040+0000 7f4f9fb60700 1 --2- 172.21.15.22:0/4146778410 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f981004a0 0x7f4f98100870 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:13.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.040+0000 7f4f9fb60700 1 -- 172.21.15.22:0/4146778410 >> 172.21.15.22:0/4146778410 conn(0x7f4f980fbd40 msgr2=0x7f4f980fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:35:13.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.040+0000 7f4f9fb60700 1 -- 172.21.15.22:0/4146778410 shutdown_connections 2024-02-16T19:35:13.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.040+0000 7f4f9fb60700 1 -- 172.21.15.22:0/4146778410 wait complete. 2024-02-16T19:35:13.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.041+0000 7f4f9fb60700 1 Processor -- start 2024-02-16T19:35:13.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.041+0000 7f4f9fb60700 1 -- start start 2024-02-16T19:35:13.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.041+0000 7f4f9fb60700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f981004a0 0x7f4f98195eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:35:13.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.041+0000 7f4f9fb60700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f98100e40 0x7f4f981963f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:35:13.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.041+0000 7f4f9fb60700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f98196ad0 con 0x7f4f981004a0 2024-02-16T19:35:13.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.041+0000 7f4f9fb60700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9819a860 con 0x7f4f98100e40 2024-02-16T19:35:13.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.041+0000 7f4f9d8fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f981004a0 0x7f4f98195eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:35:13.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.041+0000 7f4f9d8fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f981004a0 0x7f4f98195eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57810/0 (socket says 172.21.15.22:57810) 2024-02-16T19:35:13.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.041+0000 7f4f9d8fc700 1 -- 172.21.15.22:0/3252610990 learned_addr learned my addr 172.21.15.22:0/3252610990 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:35:13.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.041+0000 7f4f9d0fb700 1 --2- 172.21.15.22:0/3252610990 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f98100e40 0x7f4f981963f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:35:13.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.041+0000 7f4f9d8fc700 1 -- 172.21.15.22:0/3252610990 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f98100e40 msgr2=0x7f4f981963f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:35:13.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.041+0000 7f4f9d8fc700 1 --2- 172.21.15.22:0/3252610990 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f98100e40 0x7f4f981963f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:13.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.041+0000 7f4f9d8fc700 1 -- 172.21.15.22:0/3252610990 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f88009d00 con 0x7f4f981004a0 2024-02-16T19:35:13.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.042+0000 7f4f9d8fc700 1 --2- 172.21.15.22:0/3252610990 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f981004a0 0x7f4f98195eb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4f8800b070 tx=0x7f4f8800cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:35:13.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.042+0000 7f4f96ffd700 1 -- 172.21.15.22:0/3252610990 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f8800e040 con 0x7f4f981004a0 2024-02-16T19:35:13.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.042+0000 7f4f96ffd700 1 -- 172.21.15.22:0/3252610990 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4f88003de0 con 0x7f4f981004a0 2024-02-16T19:35:13.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.042+0000 7f4f9fb60700 1 -- 172.21.15.22:0/3252610990 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f9819aae0 con 0x7f4f981004a0 2024-02-16T19:35:13.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.042+0000 7f4f96ffd700 1 -- 172.21.15.22:0/3252610990 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f8801d850 con 0x7f4f981004a0 2024-02-16T19:35:13.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.042+0000 7f4f9fb60700 1 -- 172.21.15.22:0/3252610990 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f9819af80 con 0x7f4f981004a0 2024-02-16T19:35:13.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.043+0000 7f4f9fb60700 1 -- 172.21.15.22:0/3252610990 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f981087b0 con 0x7f4f981004a0 2024-02-16T19:35:13.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.043+0000 7f4f96ffd700 1 -- 172.21.15.22:0/3252610990 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f4f88004040 con 0x7f4f981004a0 2024-02-16T19:35:13.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.044+0000 7f4f96ffd700 1 --2- 172.21.15.22:0/3252610990 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f4f8006d8a0 0x7f4f8006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:35:13.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.044+0000 7f4f96ffd700 1 -- 172.21.15.22:0/3252610990 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f4f8808ed10 con 0x7f4f981004a0 2024-02-16T19:35:13.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.044+0000 7f4f9d0fb700 1 --2- 172.21.15.22:0/3252610990 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f4f8006d8a0 0x7f4f8006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:35:13.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.045+0000 7f4f9d0fb700 1 --2- 172.21.15.22:0/3252610990 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f4f8006d8a0 0x7f4f8006fd50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4f981974d0 tx=0x7f4f8c00a300 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:35:13.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.049+0000 7f4f96ffd700 1 -- 172.21.15.22:0/3252610990 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4f88058570 con 0x7f4f981004a0 2024-02-16T19:35:13.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.292+0000 7f4f9fb60700 1 -- 172.21.15.22:0/3252610990 --> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4f9819b1f0 con 0x7f4f8006d8a0 2024-02-16T19:35:13.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.293+0000 7f4f96ffd700 1 -- 172.21.15.22:0/3252610990 <== mgr.34104 v2:172.21.15.22:6800/505266753 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f4f9819b1f0 con 0x7f4f8006d8a0 2024-02-16T19:35:13.295 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-02-16T19:35:13.295 INFO:teuthology.orchestra.run.smithi022.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T19:35:13.295 INFO:teuthology.orchestra.run.smithi022.stdout: "in_progress": true, 2024-02-16T19:35:13.295 INFO:teuthology.orchestra.run.smithi022.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T19:35:13.295 INFO:teuthology.orchestra.run.smithi022.stdout: "services_complete": [ 2024-02-16T19:35:13.295 INFO:teuthology.orchestra.run.smithi022.stdout: "crash", 2024-02-16T19:35:13.295 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr", 2024-02-16T19:35:13.295 INFO:teuthology.orchestra.run.smithi022.stdout: "mon" 2024-02-16T19:35:13.295 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-02-16T19:35:13.295 INFO:teuthology.orchestra.run.smithi022.stdout: "progress": "7/23 daemons upgraded", 2024-02-16T19:35:13.295 INFO:teuthology.orchestra.run.smithi022.stdout: "message": "Currently upgrading osd daemons", 2024-02-16T19:35:13.295 INFO:teuthology.orchestra.run.smithi022.stdout: "is_paused": false 2024-02-16T19:35:13.296 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-02-16T19:35:13.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.295+0000 7f4f9fb60700 1 -- 172.21.15.22:0/3252610990 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f4f8006d8a0 msgr2=0x7f4f8006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:35:13.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.295+0000 7f4f9fb60700 1 --2- 172.21.15.22:0/3252610990 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f4f8006d8a0 0x7f4f8006fd50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4f981974d0 tx=0x7f4f8c00a300 comp rx=0 tx=0).stop 2024-02-16T19:35:13.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.295+0000 7f4f9fb60700 1 -- 172.21.15.22:0/3252610990 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f981004a0 msgr2=0x7f4f98195eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:35:13.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.295+0000 7f4f9fb60700 1 --2- 172.21.15.22:0/3252610990 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f981004a0 0x7f4f98195eb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4f8800b070 tx=0x7f4f8800cd20 comp rx=0 tx=0).stop 2024-02-16T19:35:13.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.296+0000 7f4f9fb60700 1 -- 172.21.15.22:0/3252610990 shutdown_connections 2024-02-16T19:35:13.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.296+0000 7f4f9fb60700 1 --2- 172.21.15.22:0/3252610990 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f4f8006d8a0 0x7f4f8006fd50 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:13.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.296+0000 7f4f9fb60700 1 --2- 172.21.15.22:0/3252610990 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f981004a0 0x7f4f98195eb0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:13.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.296+0000 7f4f9fb60700 1 --2- 172.21.15.22:0/3252610990 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f98100e40 0x7f4f981963f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:13.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.296+0000 7f4f9fb60700 1 -- 172.21.15.22:0/3252610990 >> 172.21.15.22:0/3252610990 conn(0x7f4f980fbd40 msgr2=0x7f4f980fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:35:13.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.296+0000 7f4f9fb60700 1 -- 172.21.15.22:0/3252610990 shutdown_connections 2024-02-16T19:35:13.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.296+0000 7f4f9fb60700 1 -- 172.21.15.22:0/3252610990 wait complete. 2024-02-16T19:35:13.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:13 smithi117 ceph-mon[67870]: pgmap v66: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:35:13.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:13 smithi117 ceph-mon[67870]: from='client.34228 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:35:13.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:13 smithi117 ceph-mon[67870]: from='client.? 172.21.15.22:0/2229484785' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:35:13.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:13 smithi117 ceph-mon[67870]: from='client.? 172.21.15.22:0/872891424' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:35:13.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.408+0000 7f7f0e53e700 1 -- 172.21.15.22:0/2060572307 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f080ff3c0 msgr2=0x7f7f080ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:35:13.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.408+0000 7f7f0e53e700 1 --2- 172.21.15.22:0/2060572307 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f080ff3c0 0x7f7f080ff790 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7ef8009910 tx=0x7f7ef8009c20 comp rx=0 tx=0).stop 2024-02-16T19:35:13.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.409+0000 7f7f0e53e700 1 -- 172.21.15.22:0/2060572307 shutdown_connections 2024-02-16T19:35:13.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.409+0000 7f7f0e53e700 1 --2- 172.21.15.22:0/2060572307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f080ffd60 0x7f7f08103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:13.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.409+0000 7f7f0e53e700 1 --2- 172.21.15.22:0/2060572307 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f080ff3c0 0x7f7f080ff790 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:13.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.409+0000 7f7f0e53e700 1 -- 172.21.15.22:0/2060572307 >> 172.21.15.22:0/2060572307 conn(0x7f7f080fac60 msgr2=0x7f7f080fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:35:13.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.409+0000 7f7f0e53e700 1 -- 172.21.15.22:0/2060572307 shutdown_connections 2024-02-16T19:35:13.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.409+0000 7f7f0e53e700 1 -- 172.21.15.22:0/2060572307 wait complete. 2024-02-16T19:35:13.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.409+0000 7f7f0e53e700 1 Processor -- start 2024-02-16T19:35:13.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.409+0000 7f7f0e53e700 1 -- start start 2024-02-16T19:35:13.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.410+0000 7f7f0e53e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f080ff3c0 0x7f7f08194e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:35:13.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.410+0000 7f7f0e53e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f080ffd60 0x7f7f08195340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:35:13.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.410+0000 7f7f0e53e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f08195a20 con 0x7f7f080ffd60 2024-02-16T19:35:13.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.410+0000 7f7f0e53e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f081997b0 con 0x7f7f080ff3c0 2024-02-16T19:35:13.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.410+0000 7f7f077fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f080ffd60 0x7f7f08195340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:35:13.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.410+0000 7f7f077fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f080ffd60 0x7f7f08195340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57820/0 (socket says 172.21.15.22:57820) 2024-02-16T19:35:13.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.410+0000 7f7f077fe700 1 -- 172.21.15.22:0/3055602455 learned_addr learned my addr 172.21.15.22:0/3055602455 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:35:13.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.410+0000 7f7f07fff700 1 --2- 172.21.15.22:0/3055602455 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f080ff3c0 0x7f7f08194e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:35:13.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.410+0000 7f7f077fe700 1 -- 172.21.15.22:0/3055602455 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f080ff3c0 msgr2=0x7f7f08194e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:35:13.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.410+0000 7f7f077fe700 1 --2- 172.21.15.22:0/3055602455 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f080ff3c0 0x7f7f08194e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:13.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.410+0000 7f7f077fe700 1 -- 172.21.15.22:0/3055602455 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ef80095f0 con 0x7f7f080ffd60 2024-02-16T19:35:13.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.411+0000 7f7f077fe700 1 --2- 172.21.15.22:0/3055602455 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f080ffd60 0x7f7f08195340 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7efc00e8f0 tx=0x7f7efc00ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:35:13.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.411+0000 7f7f057fa700 1 -- 172.21.15.22:0/3055602455 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7efc009790 con 0x7f7f080ffd60 2024-02-16T19:35:13.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.411+0000 7f7f0e53e700 1 -- 172.21.15.22:0/3055602455 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f081999b0 con 0x7f7f080ffd60 2024-02-16T19:35:13.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.411+0000 7f7f057fa700 1 -- 172.21.15.22:0/3055602455 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7efc00c810 con 0x7f7f080ffd60 2024-02-16T19:35:13.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.411+0000 7f7f0e53e700 1 -- 172.21.15.22:0/3055602455 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f08199f00 con 0x7f7f080ffd60 2024-02-16T19:35:13.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.411+0000 7f7f057fa700 1 -- 172.21.15.22:0/3055602455 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7efc005580 con 0x7f7f080ffd60 2024-02-16T19:35:13.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.412+0000 7f7f0e53e700 1 -- 172.21.15.22:0/3055602455 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f0804f2a0 con 0x7f7f080ffd60 2024-02-16T19:35:13.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.412+0000 7f7f057fa700 1 -- 172.21.15.22:0/3055602455 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f7efc00c980 con 0x7f7f080ffd60 2024-02-16T19:35:13.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.413+0000 7f7f057fa700 1 --2- 172.21.15.22:0/3055602455 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f7ef006d8a0 0x7f7ef006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:35:13.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.413+0000 7f7f057fa700 1 -- 172.21.15.22:0/3055602455 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f7efc014070 con 0x7f7f080ffd60 2024-02-16T19:35:13.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.413+0000 7f7f07fff700 1 --2- 172.21.15.22:0/3055602455 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f7ef006d8a0 0x7f7ef006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:35:13.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.413+0000 7f7f07fff700 1 --2- 172.21.15.22:0/3055602455 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f7ef006d8a0 0x7f7ef006fd50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7ef8005b40 tx=0x7f7ef8005a90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:35:13.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.419+0000 7f7f057fa700 1 -- 172.21.15.22:0/3055602455 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7efc056f90 con 0x7f7f080ffd60 2024-02-16T19:35:13.529 INFO:journalctl@ceph.osd.1.smithi022.stdout:Feb 16 19:35:13 smithi022 systemd[1]: Stopping Ceph osd.1 for 2a1346e6-cd00-11ee-95ba-87774f69a715... 2024-02-16T19:35:13.529 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:13 smithi022 ceph-mon[88798]: pgmap v66: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:35:13.529 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:13 smithi022 ceph-mon[88798]: from='client.34228 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:35:13.529 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:13 smithi022 ceph-mon[88798]: from='client.? 172.21.15.22:0/2229484785' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:35:13.529 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:13 smithi022 ceph-mon[88798]: from='client.? 172.21.15.22:0/872891424' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:35:13.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.715+0000 7f7f0e53e700 1 -- 172.21.15.22:0/3055602455 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f7f0819a230 con 0x7f7f080ffd60 2024-02-16T19:35:13.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.716+0000 7f7f057fa700 1 -- 172.21.15.22:0/3055602455 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f7efc05a650 con 0x7f7f080ffd60 2024-02-16T19:35:13.718 INFO:teuthology.orchestra.run.smithi022.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-02-16T19:35:13.718 INFO:teuthology.orchestra.run.smithi022.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-02-16T19:35:13.718 INFO:teuthology.orchestra.run.smithi022.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-02-16T19:35:13.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.718+0000 7f7f0e53e700 1 -- 172.21.15.22:0/3055602455 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f7ef006d8a0 msgr2=0x7f7ef006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:35:13.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.718+0000 7f7f0e53e700 1 --2- 172.21.15.22:0/3055602455 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f7ef006d8a0 0x7f7ef006fd50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7ef8005b40 tx=0x7f7ef8005a90 comp rx=0 tx=0).stop 2024-02-16T19:35:13.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.718+0000 7f7f0e53e700 1 -- 172.21.15.22:0/3055602455 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f080ffd60 msgr2=0x7f7f08195340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:35:13.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.718+0000 7f7f0e53e700 1 --2- 172.21.15.22:0/3055602455 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f080ffd60 0x7f7f08195340 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7efc00e8f0 tx=0x7f7efc00ec00 comp rx=0 tx=0).stop 2024-02-16T19:35:13.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.718+0000 7f7f0e53e700 1 -- 172.21.15.22:0/3055602455 shutdown_connections 2024-02-16T19:35:13.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.718+0000 7f7f0e53e700 1 --2- 172.21.15.22:0/3055602455 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f7ef006d8a0 0x7f7ef006fd50 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:13.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.718+0000 7f7f0e53e700 1 --2- 172.21.15.22:0/3055602455 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f080ff3c0 0x7f7f08194e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:13.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.718+0000 7f7f0e53e700 1 --2- 172.21.15.22:0/3055602455 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f080ffd60 0x7f7f08195340 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:13.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.718+0000 7f7f0e53e700 1 -- 172.21.15.22:0/3055602455 >> 172.21.15.22:0/3055602455 conn(0x7f7f080fac60 msgr2=0x7f7f080fc270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:35:13.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.718+0000 7f7f0e53e700 1 -- 172.21.15.22:0/3055602455 shutdown_connections 2024-02-16T19:35:13.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:13.718+0000 7f7f0e53e700 1 -- 172.21.15.22:0/3055602455 wait complete. 2024-02-16T19:35:13.965 INFO:journalctl@ceph.osd.1.smithi022.stdout:Feb 16 19:35:13 smithi022 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-1[49665]: 2024-02-16T19:35:13.527+0000 7f309d578700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-16T19:35:13.965 INFO:journalctl@ceph.osd.1.smithi022.stdout:Feb 16 19:35:13 smithi022 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-1[49665]: 2024-02-16T19:35:13.527+0000 7f309d578700 -1 osd.1 55 *** Got signal Terminated *** 2024-02-16T19:35:13.965 INFO:journalctl@ceph.osd.1.smithi022.stdout:Feb 16 19:35:13 smithi022 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-1[49665]: 2024-02-16T19:35:13.527+0000 7f309d578700 -1 osd.1 55 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-16T19:35:14.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:14 smithi117 ceph-mon[67870]: from='client.34240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:35:14.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:14 smithi117 ceph-mon[67870]: osd.1 marked itself down and dead 2024-02-16T19:35:14.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:14 smithi117 ceph-mon[67870]: from='client.? 172.21.15.22:0/3055602455' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T19:35:14.664 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:14 smithi022 ceph-mon[88798]: from='client.34240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:35:14.664 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:14 smithi022 ceph-mon[88798]: osd.1 marked itself down and dead 2024-02-16T19:35:14.664 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:14 smithi022 ceph-mon[88798]: from='client.? 172.21.15.22:0/3055602455' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T19:35:14.965 INFO:journalctl@ceph.osd.1.smithi022.stdout:Feb 16 19:35:14 smithi022 podman[105623]: 2024-02-16 19:35:14.664298895 +0000 UTC m=+1.269860715 container died 87300382b8a73d1493e23f2d84de562c22121e3c8fe026633afdad89a93afe13 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-1, GIT_BRANCH=HEAD, io.buildah.version=1.29.1, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=https://github.com/ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.schema-version=1.0, RELEASE=HEAD, org.label-schema.build-date=20231212, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=-18.2.0) 2024-02-16T19:35:15.283 INFO:journalctl@ceph.osd.1.smithi022.stdout:Feb 16 19:35:15 smithi022 podman[105623]: 2024-02-16 19:35:15.086496235 +0000 UTC m=+1.692058054 container cleanup 87300382b8a73d1493e23f2d84de562c22121e3c8fe026633afdad89a93afe13 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-1, RELEASE=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20231212, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, ceph=True, io.buildah.version=1.29.1, org.label-schema.license=GPLv2) 2024-02-16T19:35:15.283 INFO:journalctl@ceph.osd.1.smithi022.stdout:Feb 16 19:35:15 smithi022 bash[105623]: ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-1 2024-02-16T19:35:15.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:15 smithi117 ceph-mon[67870]: pgmap v67: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:35:15.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:15 smithi117 ceph-mon[67870]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-16T19:35:15.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:15 smithi117 ceph-mon[67870]: osdmap e56: 6 total, 5 up, 6 in 2024-02-16T19:35:15.715 INFO:journalctl@ceph.osd.1.smithi022.stdout:Feb 16 19:35:15 smithi022 podman[105662]: 2024-02-16 19:35:15.321028687 +0000 UTC m=+0.659260638 container remove 87300382b8a73d1493e23f2d84de562c22121e3c8fe026633afdad89a93afe13 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-1, maintainer=Guillaume Abrioux , io.buildah.version=1.29.1, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=HEAD, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20231212) 2024-02-16T19:35:15.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:15 smithi022 ceph-mon[88798]: pgmap v67: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:35:15.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:15 smithi022 ceph-mon[88798]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-16T19:35:15.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:15 smithi022 ceph-mon[88798]: osdmap e56: 6 total, 5 up, 6 in 2024-02-16T19:35:16.047 INFO:journalctl@ceph.osd.1.smithi022.stdout:Feb 16 19:35:15 smithi022 podman[105755]: 2024-02-16 19:35:15.665109053 +0000 UTC m=+0.018319679 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T19:35:16.047 INFO:journalctl@ceph.osd.1.smithi022.stdout:Feb 16 19:35:15 smithi022 podman[105755]: 2024-02-16 19:35:15.788729211 +0000 UTC m=+0.141939893 container create 80f4f40d7833faf488f34812a9d4404ab8043b3a9b8141bd696725017d96cb12 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux ) 2024-02-16T19:35:16.048 INFO:journalctl@ceph.osd.1.smithi022.stdout:Feb 16 19:35:15 smithi022 podman[105755]: 2024-02-16 19:35:15.983018544 +0000 UTC m=+0.336229162 container init 80f4f40d7833faf488f34812a9d4404ab8043b3a9b8141bd696725017d96cb12 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-1-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, ceph=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=) 2024-02-16T19:35:16.048 INFO:journalctl@ceph.osd.1.smithi022.stdout:Feb 16 19:35:15 smithi022 podman[105755]: 2024-02-16 19:35:15.986037884 +0000 UTC m=+0.339248511 container start 80f4f40d7833faf488f34812a9d4404ab8043b3a9b8141bd696725017d96cb12 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True) 2024-02-16T19:35:16.465 INFO:journalctl@ceph.osd.1.smithi022.stdout:Feb 16 19:35:16 smithi022 podman[105755]: 2024-02-16 19:35:16.083171357 +0000 UTC m=+0.436381984 container attach 80f4f40d7833faf488f34812a9d4404ab8043b3a9b8141bd696725017d96cb12 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-1-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True) 2024-02-16T19:35:16.465 INFO:journalctl@ceph.osd.1.smithi022.stdout:Feb 16 19:35:16 smithi022 podman[105755]: 2024-02-16 19:35:16.286634823 +0000 UTC m=+0.639845445 container died 80f4f40d7833faf488f34812a9d4404ab8043b3a9b8141bd696725017d96cb12 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-1-deactivate, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-16T19:35:16.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:16 smithi022 ceph-mon[88798]: osdmap e57: 6 total, 5 up, 6 in 2024-02-16T19:35:16.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:16 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [3, 2]}]: dispatch 2024-02-16T19:35:16.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:16 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [3, 1]}]: dispatch 2024-02-16T19:35:16.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:16 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [3, 2]}]: dispatch 2024-02-16T19:35:16.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:16 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [0, 2]}]: dispatch 2024-02-16T19:35:16.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:16 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [0, 5]}]: dispatch 2024-02-16T19:35:16.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:16 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1e", "id": [0, 1]}]: dispatch 2024-02-16T19:35:16.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:16 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 1]}]: dispatch 2024-02-16T19:35:16.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:16 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.f", "id": [3, 1]}]: dispatch 2024-02-16T19:35:16.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:16 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.16", "id": [3, 0]}]: dispatch 2024-02-16T19:35:16.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:16 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 1]}]: dispatch 2024-02-16T19:35:16.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:16 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:35:16.467 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:16 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:35:16.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:16 smithi117 ceph-mon[67870]: osdmap e57: 6 total, 5 up, 6 in 2024-02-16T19:35:16.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:16 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [3, 2]}]: dispatch 2024-02-16T19:35:16.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:16 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [3, 1]}]: dispatch 2024-02-16T19:35:16.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:16 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [3, 2]}]: dispatch 2024-02-16T19:35:16.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:16 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [0, 2]}]: dispatch 2024-02-16T19:35:16.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:16 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [0, 5]}]: dispatch 2024-02-16T19:35:16.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:16 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1e", "id": [0, 1]}]: dispatch 2024-02-16T19:35:16.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:16 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 1]}]: dispatch 2024-02-16T19:35:16.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:16 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.f", "id": [3, 1]}]: dispatch 2024-02-16T19:35:16.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:16 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.16", "id": [3, 0]}]: dispatch 2024-02-16T19:35:16.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:16 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 1]}]: dispatch 2024-02-16T19:35:16.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:16 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:35:16.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:16 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:35:17.215 INFO:journalctl@ceph.osd.1.smithi022.stdout:Feb 16 19:35:16 smithi022 podman[105781]: 2024-02-16 19:35:16.903902545 +0000 UTC m=+0.611421659 container remove 80f4f40d7833faf488f34812a9d4404ab8043b3a9b8141bd696725017d96cb12 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.vendor=CentOS) 2024-02-16T19:35:17.215 INFO:journalctl@ceph.osd.1.smithi022.stdout:Feb 16 19:35:16 smithi022 systemd[1]: ceph-2a1346e6-cd00-11ee-95ba-87774f69a715@osd.1.service: Deactivated successfully. 2024-02-16T19:35:17.215 INFO:journalctl@ceph.osd.1.smithi022.stdout:Feb 16 19:35:16 smithi022 systemd[1]: Stopped Ceph osd.1 for 2a1346e6-cd00-11ee-95ba-87774f69a715. 2024-02-16T19:35:17.215 INFO:journalctl@ceph.osd.1.smithi022.stdout:Feb 16 19:35:16 smithi022 systemd[1]: ceph-2a1346e6-cd00-11ee-95ba-87774f69a715@osd.1.service: Consumed 57.781s CPU time. 2024-02-16T19:35:17.216 INFO:journalctl@ceph.osd.1.smithi022.stdout:Feb 16 19:35:17 smithi022 systemd[1]: Starting Ceph osd.1 for 2a1346e6-cd00-11ee-95ba-87774f69a715... 2024-02-16T19:35:17.550 INFO:journalctl@ceph.osd.1.smithi022.stdout:Feb 16 19:35:17 smithi022 podman[105888]: 2024-02-16 19:35:17.347196623 +0000 UTC m=+0.113901533 container create fd44ced8eb3918bc4d4be940aec1e9bea0112e6748ace367301b85a5d91429c0 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-1-activate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-16T19:35:17.550 INFO:journalctl@ceph.osd.1.smithi022.stdout:Feb 16 19:35:17 smithi022 podman[105888]: 2024-02-16 19:35:17.251247856 +0000 UTC m=+0.017952764 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T19:35:17.550 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:17 smithi022 ceph-mon[88798]: pgmap v70: 65 pgs: 12 stale+active+clean, 53 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:35:17.550 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:17 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [3, 2]}]': finished 2024-02-16T19:35:17.550 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:17 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [3, 1]}]': finished 2024-02-16T19:35:17.550 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:17 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [3, 2]}]': finished 2024-02-16T19:35:17.550 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:17 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [0, 2]}]': finished 2024-02-16T19:35:17.550 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:17 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [0, 5]}]': finished 2024-02-16T19:35:17.551 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:17 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1e", "id": [0, 1]}]': finished 2024-02-16T19:35:17.551 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:17 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 1]}]': finished 2024-02-16T19:35:17.551 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:17 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.f", "id": [3, 1]}]': finished 2024-02-16T19:35:17.551 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:17 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.16", "id": [3, 0]}]': finished 2024-02-16T19:35:17.551 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:17 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 1]}]': finished 2024-02-16T19:35:17.551 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:17 smithi022 ceph-mon[88798]: osdmap e58: 6 total, 5 up, 6 in 2024-02-16T19:35:17.551 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:17 smithi022 ceph-mon[88798]: osdmap e59: 6 total, 5 up, 6 in 2024-02-16T19:35:17.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:17 smithi117 ceph-mon[67870]: pgmap v70: 65 pgs: 12 stale+active+clean, 53 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:35:17.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:17 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [3, 2]}]': finished 2024-02-16T19:35:17.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:17 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [3, 1]}]': finished 2024-02-16T19:35:17.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:17 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [3, 2]}]': finished 2024-02-16T19:35:17.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:17 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [0, 2]}]': finished 2024-02-16T19:35:17.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:17 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [0, 5]}]': finished 2024-02-16T19:35:17.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:17 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1e", "id": [0, 1]}]': finished 2024-02-16T19:35:17.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:17 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 1]}]': finished 2024-02-16T19:35:17.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:17 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.f", "id": [3, 1]}]': finished 2024-02-16T19:35:17.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:17 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.16", "id": [3, 0]}]': finished 2024-02-16T19:35:17.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:17 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 1]}]': finished 2024-02-16T19:35:17.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:17 smithi117 ceph-mon[67870]: osdmap e58: 6 total, 5 up, 6 in 2024-02-16T19:35:17.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:17 smithi117 ceph-mon[67870]: osdmap e59: 6 total, 5 up, 6 in 2024-02-16T19:35:17.965 INFO:journalctl@ceph.osd.1.smithi022.stdout:Feb 16 19:35:17 smithi022 podman[105888]: 2024-02-16 19:35:17.549692926 +0000 UTC m=+0.316397828 container init fd44ced8eb3918bc4d4be940aec1e9bea0112e6748ace367301b85a5d91429c0 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-1-activate, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-batrick-testing-20240215.160715-f78a58c) 2024-02-16T19:35:17.965 INFO:journalctl@ceph.osd.1.smithi022.stdout:Feb 16 19:35:17 smithi022 podman[105888]: 2024-02-16 19:35:17.552959533 +0000 UTC m=+0.319664433 container start fd44ced8eb3918bc4d4be940aec1e9bea0112e6748ace367301b85a5d91429c0 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-1-activate, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240214) 2024-02-16T19:35:17.965 INFO:journalctl@ceph.osd.1.smithi022.stdout:Feb 16 19:35:17 smithi022 podman[105888]: 2024-02-16 19:35:17.632907047 +0000 UTC m=+0.399611964 container attach fd44ced8eb3918bc4d4be940aec1e9bea0112e6748ace367301b85a5d91429c0 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-1-activate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.schema-version=1.0) 2024-02-16T19:35:18.338 INFO:journalctl@ceph.osd.1.smithi022.stdout:Feb 16 19:35:18 smithi022 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-1-activate[105902]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-02-16T19:35:18.338 INFO:journalctl@ceph.osd.1.smithi022.stdout:Feb 16 19:35:18 smithi022 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-1-activate[105902]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-02-16T19:35:18.338 INFO:journalctl@ceph.osd.1.smithi022.stdout:Feb 16 19:35:18 smithi022 bash[105888]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-02-16T19:35:18.338 INFO:journalctl@ceph.osd.1.smithi022.stdout:Feb 16 19:35:18 smithi022 bash[105888]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-02-16T19:35:18.338 INFO:journalctl@ceph.osd.1.smithi022.stdout:Feb 16 19:35:18 smithi022 bash[105888]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-02-16T19:35:18.338 INFO:journalctl@ceph.osd.1.smithi022.stdout:Feb 16 19:35:18 smithi022 bash[105888]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-02-16T19:35:18.338 INFO:journalctl@ceph.osd.1.smithi022.stdout:Feb 16 19:35:18 smithi022 bash[105888]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-02-16T19:35:18.338 INFO:journalctl@ceph.osd.1.smithi022.stdout:Feb 16 19:35:18 smithi022 bash[105888]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-02-16T19:35:18.338 INFO:journalctl@ceph.osd.1.smithi022.stdout:Feb 16 19:35:18 smithi022 bash[105888]: --> ceph-volume raw activate successful for osd ID: 1 2024-02-16T19:35:18.338 INFO:journalctl@ceph.osd.1.smithi022.stdout:Feb 16 19:35:18 smithi022 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-1-activate[105902]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-02-16T19:35:18.338 INFO:journalctl@ceph.osd.1.smithi022.stdout:Feb 16 19:35:18 smithi022 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-1-activate[105902]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-02-16T19:35:18.338 INFO:journalctl@ceph.osd.1.smithi022.stdout:Feb 16 19:35:18 smithi022 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-1-activate[105902]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-02-16T19:35:18.339 INFO:journalctl@ceph.osd.1.smithi022.stdout:Feb 16 19:35:18 smithi022 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-1-activate[105902]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-02-16T19:35:18.339 INFO:journalctl@ceph.osd.1.smithi022.stdout:Feb 16 19:35:18 smithi022 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-1-activate[105902]: --> ceph-volume raw activate successful for osd ID: 1 2024-02-16T19:35:18.339 INFO:journalctl@ceph.osd.1.smithi022.stdout:Feb 16 19:35:18 smithi022 podman[105888]: 2024-02-16 19:35:18.026030873 +0000 UTC m=+0.792735789 container died fd44ced8eb3918bc4d4be940aec1e9bea0112e6748ace367301b85a5d91429c0 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-1-activate, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240215.160715-f78a58c) 2024-02-16T19:35:18.965 INFO:journalctl@ceph.osd.1.smithi022.stdout:Feb 16 19:35:18 smithi022 podman[106042]: 2024-02-16 19:35:18.650902372 +0000 UTC m=+0.619869354 container remove fd44ced8eb3918bc4d4be940aec1e9bea0112e6748ace367301b85a5d91429c0 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-1-activate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-02-16T19:35:18.965 INFO:journalctl@ceph.osd.1.smithi022.stdout:Feb 16 19:35:18 smithi022 podman[106105]: 2024-02-16 19:35:18.769208689 +0000 UTC m=+0.017744867 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T19:35:18.965 INFO:journalctl@ceph.osd.1.smithi022.stdout:Feb 16 19:35:18 smithi022 podman[106105]: 2024-02-16 19:35:18.871863335 +0000 UTC m=+0.120399575 container create c3488ce7b2a09e1cd142baaa0c58e48dd65a040fac998c39af24cd5d21f4fc1c (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-1, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-16T19:35:19.273 INFO:journalctl@ceph.osd.1.smithi022.stdout:Feb 16 19:35:19 smithi022 podman[106105]: 2024-02-16 19:35:19.099763032 +0000 UTC m=+0.348299220 container init c3488ce7b2a09e1cd142baaa0c58e48dd65a040fac998c39af24cd5d21f4fc1c (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-1, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-16T19:35:19.274 INFO:journalctl@ceph.osd.1.smithi022.stdout:Feb 16 19:35:19 smithi022 podman[106105]: 2024-02-16 19:35:19.102923464 +0000 UTC m=+0.351459643 container start c3488ce7b2a09e1cd142baaa0c58e48dd65a040fac998c39af24cd5d21f4fc1c (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-1, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-16T19:35:19.274 INFO:journalctl@ceph.osd.1.smithi022.stdout:Feb 16 19:35:19 smithi022 ceph-osd[106125]: -- 172.21.15.22:0/586444001 <== mon.0 v2:172.21.15.22:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c4b6f791e0 con 0x55c4b6f4ac00 2024-02-16T19:35:19.274 INFO:journalctl@ceph.osd.1.smithi022.stdout:Feb 16 19:35:19 smithi022 bash[106105]: c3488ce7b2a09e1cd142baaa0c58e48dd65a040fac998c39af24cd5d21f4fc1c 2024-02-16T19:35:19.274 INFO:journalctl@ceph.osd.1.smithi022.stdout:Feb 16 19:35:19 smithi022 systemd[1]: Started Ceph osd.1 for 2a1346e6-cd00-11ee-95ba-87774f69a715. 2024-02-16T19:35:19.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:19 smithi117 ceph-mon[67870]: pgmap v73: 65 pgs: 3 unknown, 4 peering, 7 active+undersized+degraded, 3 stale+active+clean, 17 active+undersized, 31 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 22/261 objects degraded (8.429%) 2024-02-16T19:35:19.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:19 smithi117 ceph-mon[67870]: osdmap e60: 6 total, 5 up, 6 in 2024-02-16T19:35:19.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:19 smithi117 ceph-mon[67870]: Health check failed: Degraded data redundancy: 22/261 objects degraded (8.429%), 7 pgs degraded (PG_DEGRADED) 2024-02-16T19:35:19.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:19 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:35:19.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:19 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:35:19.647 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:19 smithi022 ceph-mon[88798]: pgmap v73: 65 pgs: 3 unknown, 4 peering, 7 active+undersized+degraded, 3 stale+active+clean, 17 active+undersized, 31 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 22/261 objects degraded (8.429%) 2024-02-16T19:35:19.647 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:19 smithi022 ceph-mon[88798]: osdmap e60: 6 total, 5 up, 6 in 2024-02-16T19:35:19.647 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:19 smithi022 ceph-mon[88798]: Health check failed: Degraded data redundancy: 22/261 objects degraded (8.429%), 7 pgs degraded (PG_DEGRADED) 2024-02-16T19:35:19.647 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:19 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:35:19.648 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:19 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:35:19.965 INFO:journalctl@ceph.osd.1.smithi022.stdout:Feb 16 19:35:19 smithi022 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-1[106121]: 2024-02-16T19:35:19.646+0000 7fddb72d57c0 -1 Falling back to public interface 2024-02-16T19:35:20.386 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:20 smithi022 ceph-mon[88798]: osdmap e61: 6 total, 5 up, 6 in 2024-02-16T19:35:20.386 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:20 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:35:20.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:20 smithi117 ceph-mon[67870]: osdmap e61: 6 total, 5 up, 6 in 2024-02-16T19:35:20.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:20 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:35:21.634 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:21 smithi022 ceph-mon[88798]: pgmap v76: 65 pgs: 3 active+clean+remapped, 26 active+clean, 20 active+undersized, 1 active+undersized+remapped, 9 active+undersized+degraded, 4 peering, 2 unknown; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 24/264 objects degraded (9.091%); 5/264 objects misplaced (1.894%); 1024 KiB/s, 1 objects/s recovering 2024-02-16T19:35:21.634 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:21 smithi022 ceph-mon[88798]: osdmap e62: 6 total, 5 up, 6 in 2024-02-16T19:35:21.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:21 smithi117 ceph-mon[67870]: pgmap v76: 65 pgs: 3 active+clean+remapped, 26 active+clean, 20 active+undersized, 1 active+undersized+remapped, 9 active+undersized+degraded, 4 peering, 2 unknown; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 24/264 objects degraded (9.091%); 5/264 objects misplaced (1.894%); 1024 KiB/s, 1 objects/s recovering 2024-02-16T19:35:21.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:21 smithi117 ceph-mon[67870]: osdmap e62: 6 total, 5 up, 6 in 2024-02-16T19:35:23.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:23 smithi022 ceph-mon[88798]: pgmap v78: 65 pgs: 10 active+clean+remapped, 26 active+clean, 20 active+undersized, 9 active+undersized+degraded; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 453 B/s rd, 3.2 MiB/s wr, 164 op/s; 24/267 objects degraded (8.989%); 17/267 objects misplaced (6.367%); 4.4 MiB/s, 62 objects/s recovering 2024-02-16T19:35:23.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:23 smithi117 ceph-mon[67870]: pgmap v78: 65 pgs: 10 active+clean+remapped, 26 active+clean, 20 active+undersized, 9 active+undersized+degraded; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 453 B/s rd, 3.2 MiB/s wr, 164 op/s; 24/267 objects degraded (8.989%); 17/267 objects misplaced (6.367%); 4.4 MiB/s, 62 objects/s recovering 2024-02-16T19:35:24.465 INFO:journalctl@ceph.osd.1.smithi022.stdout:Feb 16 19:35:24 smithi022 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-1[106121]: 2024-02-16T19:35:24.145+0000 7fddb72d57c0 -1 osd.1 0 read_superblock omap replica is missing. 2024-02-16T19:35:24.965 INFO:journalctl@ceph.osd.1.smithi022.stdout:Feb 16 19:35:24 smithi022 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-1[106121]: 2024-02-16T19:35:24.486+0000 7fddb72d57c0 -1 osd.1 55 log_to_monitors true 2024-02-16T19:35:25.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:25 smithi117 ceph-mon[67870]: pgmap v79: 65 pgs: 10 active+clean+remapped, 26 active+clean, 20 active+undersized, 9 active+undersized+degraded; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 341 B/s rd, 2.4 MiB/s wr, 123 op/s; 24/267 objects degraded (8.989%); 17/267 objects misplaced (6.367%); 3.3 MiB/s, 46 objects/s recovering 2024-02-16T19:35:25.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:25 smithi117 ceph-mon[67870]: from='osd.1 [v2:172.21.15.22:6810/3321991378,v1:172.21.15.22:6811/3321991378]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-16T19:35:25.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:25 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:35:25.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:25 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:35:25.715 INFO:journalctl@ceph.osd.1.smithi022.stdout:Feb 16 19:35:25 smithi022 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-1[106121]: 2024-02-16T19:35:25.322+0000 7fdda872d700 -1 osd.1 55 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-16T19:35:25.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:25 smithi022 ceph-mon[88798]: pgmap v79: 65 pgs: 10 active+clean+remapped, 26 active+clean, 20 active+undersized, 9 active+undersized+degraded; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 341 B/s rd, 2.4 MiB/s wr, 123 op/s; 24/267 objects degraded (8.989%); 17/267 objects misplaced (6.367%); 3.3 MiB/s, 46 objects/s recovering 2024-02-16T19:35:25.716 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:25 smithi022 ceph-mon[88798]: from='osd.1 [v2:172.21.15.22:6810/3321991378,v1:172.21.15.22:6811/3321991378]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-16T19:35:25.716 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:25 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:35:25.716 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:25 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:35:26.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:26 smithi022 ceph-mon[88798]: from='osd.1 [v2:172.21.15.22:6810/3321991378,v1:172.21.15.22:6811/3321991378]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-16T19:35:26.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:26 smithi022 ceph-mon[88798]: osdmap e63: 6 total, 5 up, 6 in 2024-02-16T19:35:26.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:26 smithi022 ceph-mon[88798]: from='osd.1 [v2:172.21.15.22:6810/3321991378,v1:172.21.15.22:6811/3321991378]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-02-16T19:35:26.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:26 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:35:26.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:26 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:35:26.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:26 smithi117 ceph-mon[67870]: from='osd.1 [v2:172.21.15.22:6810/3321991378,v1:172.21.15.22:6811/3321991378]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-16T19:35:26.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:26 smithi117 ceph-mon[67870]: osdmap e63: 6 total, 5 up, 6 in 2024-02-16T19:35:26.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:26 smithi117 ceph-mon[67870]: from='osd.1 [v2:172.21.15.22:6810/3321991378,v1:172.21.15.22:6811/3321991378]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-02-16T19:35:26.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:26 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:35:26.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:26 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:35:27.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:27 smithi117 ceph-mon[67870]: pgmap v81: 65 pgs: 10 active+clean+remapped, 26 active+clean, 20 active+undersized, 9 active+undersized+degraded; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 315 B/s rd, 318 KiB/s wr, 24 op/s; 24/267 objects degraded (8.989%); 17/267 objects misplaced (6.367%); 2.5 MiB/s, 42 objects/s recovering 2024-02-16T19:35:27.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:27 smithi117 ceph-mon[67870]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-16T19:35:27.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:27 smithi117 ceph-mon[67870]: osd.1 [v2:172.21.15.22:6810/3321991378,v1:172.21.15.22:6811/3321991378] boot 2024-02-16T19:35:27.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:27 smithi117 ceph-mon[67870]: osdmap e64: 6 total, 6 up, 6 in 2024-02-16T19:35:27.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:27 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-16T19:35:27.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:27 smithi117 ceph-mon[67870]: Health check update: Degraded data redundancy: 24/267 objects degraded (8.989%), 9 pgs degraded (PG_DEGRADED) 2024-02-16T19:35:27.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:27 smithi117 ceph-mon[67870]: osdmap e65: 6 total, 6 up, 6 in 2024-02-16T19:35:27.664 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:27 smithi022 ceph-mon[88798]: pgmap v81: 65 pgs: 10 active+clean+remapped, 26 active+clean, 20 active+undersized, 9 active+undersized+degraded; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 315 B/s rd, 318 KiB/s wr, 24 op/s; 24/267 objects degraded (8.989%); 17/267 objects misplaced (6.367%); 2.5 MiB/s, 42 objects/s recovering 2024-02-16T19:35:27.664 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:27 smithi022 ceph-mon[88798]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-16T19:35:27.664 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:27 smithi022 ceph-mon[88798]: osd.1 [v2:172.21.15.22:6810/3321991378,v1:172.21.15.22:6811/3321991378] boot 2024-02-16T19:35:27.664 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:27 smithi022 ceph-mon[88798]: osdmap e64: 6 total, 6 up, 6 in 2024-02-16T19:35:27.664 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:27 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-16T19:35:27.664 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:27 smithi022 ceph-mon[88798]: Health check update: Degraded data redundancy: 24/267 objects degraded (8.989%), 9 pgs degraded (PG_DEGRADED) 2024-02-16T19:35:27.664 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:27 smithi022 ceph-mon[88798]: osdmap e65: 6 total, 6 up, 6 in 2024-02-16T19:35:29.484 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:29 smithi022 ceph-mon[88798]: pgmap v84: 65 pgs: 7 remapped+peering, 17 peering, 3 active+clean+remapped, 26 active+clean, 7 active+undersized, 5 active+undersized+degraded; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 15/267 objects degraded (5.618%); 4/267 objects misplaced (1.498%) 2024-02-16T19:35:29.484 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:29 smithi022 ceph-mon[88798]: osdmap e66: 6 total, 6 up, 6 in 2024-02-16T19:35:29.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:29 smithi117 ceph-mon[67870]: pgmap v84: 65 pgs: 7 remapped+peering, 17 peering, 3 active+clean+remapped, 26 active+clean, 7 active+undersized, 5 active+undersized+degraded; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 15/267 objects degraded (5.618%); 4/267 objects misplaced (1.498%) 2024-02-16T19:35:29.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:29 smithi117 ceph-mon[67870]: osdmap e66: 6 total, 6 up, 6 in 2024-02-16T19:35:30.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:30 smithi117 ceph-mon[67870]: osdmap e67: 6 total, 6 up, 6 in 2024-02-16T19:35:30.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:30 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:35:30.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:30 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:35:30.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:30 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:35:30.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:30 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:35:30.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:30 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:35:30.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:30 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:35:30.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:30 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:35:30.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:30 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:35:30.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:30 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:35:30.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:30 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-02-16T19:35:30.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:30 smithi022 ceph-mon[88798]: osdmap e67: 6 total, 6 up, 6 in 2024-02-16T19:35:30.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:30 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:35:30.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:30 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:35:30.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:30 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:35:30.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:30 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:35:30.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:30 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:35:30.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:30 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:35:30.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:30 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:35:30.716 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:30 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:35:30.716 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:30 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:35:30.716 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:30 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-02-16T19:35:31.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:31 smithi022 ceph-mon[88798]: pgmap v87: 65 pgs: 7 remapped+peering, 17 peering, 28 active+clean, 1 active+recovering+undersized+remapped, 5 active+undersized, 1 active+recovery_wait+undersized+degraded+remapped, 1 active+undersized+remapped, 5 active+undersized+degraded; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 252/267 objects degraded (94.382%); 4.0 MiB/s, 22 objects/s recovering 2024-02-16T19:35:31.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:31 smithi022 ceph-mon[88798]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-02-16T19:35:31.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:31 smithi022 ceph-mon[88798]: Upgrade: unsafe to stop osd(s) at this time (16 PGs are or would become offline) 2024-02-16T19:35:31.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:31 smithi022 ceph-mon[88798]: osdmap e68: 6 total, 6 up, 6 in 2024-02-16T19:35:31.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:31 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:35:31.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:31 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:35:31.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:31 smithi117 ceph-mon[67870]: pgmap v87: 65 pgs: 7 remapped+peering, 17 peering, 28 active+clean, 1 active+recovering+undersized+remapped, 5 active+undersized, 1 active+recovery_wait+undersized+degraded+remapped, 1 active+undersized+remapped, 5 active+undersized+degraded; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 252/267 objects degraded (94.382%); 4.0 MiB/s, 22 objects/s recovering 2024-02-16T19:35:31.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:31 smithi117 ceph-mon[67870]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-02-16T19:35:31.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:31 smithi117 ceph-mon[67870]: Upgrade: unsafe to stop osd(s) at this time (16 PGs are or would become offline) 2024-02-16T19:35:31.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:31 smithi117 ceph-mon[67870]: osdmap e68: 6 total, 6 up, 6 in 2024-02-16T19:35:31.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:31 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:35:31.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:31 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:35:32.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:32 smithi117 ceph-mon[67870]: osdmap e69: 6 total, 6 up, 6 in 2024-02-16T19:35:32.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:32 smithi117 ceph-mon[67870]: Health check update: Degraded data redundancy: 252/267 objects degraded (94.382%), 6 pgs degraded (PG_DEGRADED) 2024-02-16T19:35:32.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:32 smithi022 ceph-mon[88798]: osdmap e69: 6 total, 6 up, 6 in 2024-02-16T19:35:32.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:32 smithi022 ceph-mon[88798]: Health check update: Degraded data redundancy: 252/267 objects degraded (94.382%), 6 pgs degraded (PG_DEGRADED) 2024-02-16T19:35:33.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:33 smithi117 ceph-mon[67870]: pgmap v90: 65 pgs: 1 peering, 62 active+clean, 1 active+recovering+undersized+remapped, 1 active+recovery_wait+undersized+degraded+remapped; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 237/267 objects degraded (88.764%); 5.0 MiB/s, 80 objects/s recovering 2024-02-16T19:35:33.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:33 smithi022 ceph-mon[88798]: pgmap v90: 65 pgs: 1 peering, 62 active+clean, 1 active+recovering+undersized+remapped, 1 active+recovery_wait+undersized+degraded+remapped; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 237/267 objects degraded (88.764%); 5.0 MiB/s, 80 objects/s recovering 2024-02-16T19:35:35.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:35 smithi117 ceph-mon[67870]: pgmap v91: 65 pgs: 1 peering, 62 active+clean, 1 active+recovering+undersized+remapped, 1 active+recovery_wait+undersized+degraded+remapped; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 237/267 objects degraded (88.764%); 3.6 MiB/s, 58 objects/s recovering 2024-02-16T19:35:35.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:35 smithi022 ceph-mon[88798]: pgmap v91: 65 pgs: 1 peering, 62 active+clean, 1 active+recovering+undersized+remapped, 1 active+recovery_wait+undersized+degraded+remapped; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 237/267 objects degraded (88.764%); 3.6 MiB/s, 58 objects/s recovering 2024-02-16T19:35:36.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:36 smithi022 ceph-mon[88798]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 237/267 objects degraded (88.764%), 1 pg degraded) 2024-02-16T19:35:36.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:36 smithi117 ceph-mon[67870]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 237/267 objects degraded (88.764%), 1 pg degraded) 2024-02-16T19:35:37.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:37 smithi022 ceph-mon[88798]: pgmap v92: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 634 KiB/s, 97 objects/s recovering 2024-02-16T19:35:37.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:37 smithi117 ceph-mon[67870]: pgmap v92: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 634 KiB/s, 97 objects/s recovering 2024-02-16T19:35:39.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:39 smithi022 ceph-mon[88798]: pgmap v93: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 512 KiB/s, 78 objects/s recovering 2024-02-16T19:35:39.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:39 smithi117 ceph-mon[67870]: pgmap v93: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 512 KiB/s, 78 objects/s recovering 2024-02-16T19:35:41.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:41 smithi022 ceph-mon[88798]: pgmap v94: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s, 67 objects/s recovering 2024-02-16T19:35:41.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:41 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:35:41.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:41 smithi117 ceph-mon[67870]: pgmap v94: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s, 67 objects/s recovering 2024-02-16T19:35:41.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:41 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:35:43.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:43 smithi117 ceph-mon[67870]: pgmap v95: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s, 61 objects/s recovering 2024-02-16T19:35:43.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:43 smithi022 ceph-mon[88798]: pgmap v95: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s, 61 objects/s recovering 2024-02-16T19:35:43.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:43.840+0000 7f7fe59cf700 1 -- 172.21.15.22:0/1956408408 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7fe0100e80 msgr2=0x7f7fe0104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:35:43.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:43.840+0000 7f7fe59cf700 1 --2- 172.21.15.22:0/1956408408 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7fe0100e80 0x7f7fe0104e50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7fd0009910 tx=0x7f7fd0009c20 comp rx=0 tx=0).stop 2024-02-16T19:35:43.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:43.840+0000 7f7fe59cf700 1 -- 172.21.15.22:0/1956408408 shutdown_connections 2024-02-16T19:35:43.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:43.840+0000 7f7fe59cf700 1 --2- 172.21.15.22:0/1956408408 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7fe0100e80 0x7f7fe0104e50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:43.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:43.840+0000 7f7fe59cf700 1 --2- 172.21.15.22:0/1956408408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fe01004e0 0x7f7fe01008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:43.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:43.840+0000 7f7fe59cf700 1 -- 172.21.15.22:0/1956408408 >> 172.21.15.22:0/1956408408 conn(0x7f7fe00fbda0 msgr2=0x7f7fe00fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:35:43.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:43.840+0000 7f7fe59cf700 1 -- 172.21.15.22:0/1956408408 shutdown_connections 2024-02-16T19:35:43.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:43.841+0000 7f7fe59cf700 1 -- 172.21.15.22:0/1956408408 wait complete. 2024-02-16T19:35:43.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:43.841+0000 7f7fe59cf700 1 Processor -- start 2024-02-16T19:35:43.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:43.841+0000 7f7fe59cf700 1 -- start start 2024-02-16T19:35:43.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:43.842+0000 7f7fe59cf700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7fe01004e0 0x7f7fe0195f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:35:43.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:43.842+0000 7f7fe59cf700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fe0100e80 0x7f7fe0196450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:35:43.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:43.842+0000 7f7fe59cf700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe0196b30 con 0x7f7fe01004e0 2024-02-16T19:35:43.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:43.842+0000 7f7fe59cf700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe019a850 con 0x7f7fe0100e80 2024-02-16T19:35:43.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:43.842+0000 7f7fdeffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7fe01004e0 0x7f7fe0195f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:35:43.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:43.842+0000 7f7fdeffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7fe01004e0 0x7f7fe0195f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40430/0 (socket says 172.21.15.22:40430) 2024-02-16T19:35:43.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:43.842+0000 7f7fdeffd700 1 -- 172.21.15.22:0/344141724 learned_addr learned my addr 172.21.15.22:0/344141724 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:35:43.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:43.842+0000 7f7fde7fc700 1 --2- 172.21.15.22:0/344141724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fe0100e80 0x7f7fe0196450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:35:43.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:43.842+0000 7f7fdeffd700 1 -- 172.21.15.22:0/344141724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fe0100e80 msgr2=0x7f7fe0196450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:35:43.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:43.842+0000 7f7fdeffd700 1 --2- 172.21.15.22:0/344141724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fe0100e80 0x7f7fe0196450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:43.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:43.842+0000 7f7fdeffd700 1 -- 172.21.15.22:0/344141724 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fd00095f0 con 0x7f7fe01004e0 2024-02-16T19:35:43.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:43.842+0000 7f7fdeffd700 1 --2- 172.21.15.22:0/344141724 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7fe01004e0 0x7f7fe0195f10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7fd400d8d0 tx=0x7f7fd400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:35:43.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:43.842+0000 7f7fe49cd700 1 -- 172.21.15.22:0/344141724 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7fd40098a0 con 0x7f7fe01004e0 2024-02-16T19:35:43.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:43.843+0000 7f7fe59cf700 1 -- 172.21.15.22:0/344141724 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fe019ab30 con 0x7f7fe01004e0 2024-02-16T19:35:43.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:43.843+0000 7f7fe49cd700 1 -- 172.21.15.22:0/344141724 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7fd4010460 con 0x7f7fe01004e0 2024-02-16T19:35:43.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:43.843+0000 7f7fe49cd700 1 -- 172.21.15.22:0/344141724 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7fd400f5f0 con 0x7f7fe01004e0 2024-02-16T19:35:43.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:43.843+0000 7f7fe59cf700 1 -- 172.21.15.22:0/344141724 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fe019b080 con 0x7f7fe01004e0 2024-02-16T19:35:43.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:43.844+0000 7f7fe59cf700 1 -- 172.21.15.22:0/344141724 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fe0108850 con 0x7f7fe01004e0 2024-02-16T19:35:43.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:43.844+0000 7f7fe49cd700 1 -- 172.21.15.22:0/344141724 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f7fd4009a00 con 0x7f7fe01004e0 2024-02-16T19:35:43.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:43.844+0000 7f7fe49cd700 1 --2- 172.21.15.22:0/344141724 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f7fc806d8a0 0x7f7fc806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:35:43.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:43.845+0000 7f7fe49cd700 1 -- 172.21.15.22:0/344141724 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f7fd408d470 con 0x7f7fe01004e0 2024-02-16T19:35:43.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:43.845+0000 7f7fde7fc700 1 --2- 172.21.15.22:0/344141724 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f7fc806d8a0 0x7f7fc806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:35:43.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:43.845+0000 7f7fde7fc700 1 --2- 172.21.15.22:0/344141724 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f7fc806d8a0 0x7f7fc806fd50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7fe01974c0 tx=0x7f7fd001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:35:43.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:43.849+0000 7f7fe49cd700 1 -- 172.21.15.22:0/344141724 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7fd4056bb0 con 0x7f7fe01004e0 2024-02-16T19:35:44.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.078+0000 7f7fe59cf700 1 -- 172.21.15.22:0/344141724 --> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7fe01972e0 con 0x7f7fc806d8a0 2024-02-16T19:35:44.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.079+0000 7f7fe49cd700 1 -- 172.21.15.22:0/344141724 <== mgr.34104 v2:172.21.15.22:6800/505266753 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f7fe01972e0 con 0x7f7fc806d8a0 2024-02-16T19:35:44.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.082+0000 7f7fe59cf700 1 -- 172.21.15.22:0/344141724 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f7fc806d8a0 msgr2=0x7f7fc806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:35:44.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.082+0000 7f7fe59cf700 1 --2- 172.21.15.22:0/344141724 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f7fc806d8a0 0x7f7fc806fd50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7fe01974c0 tx=0x7f7fd001a040 comp rx=0 tx=0).stop 2024-02-16T19:35:44.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.082+0000 7f7fe59cf700 1 -- 172.21.15.22:0/344141724 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7fe01004e0 msgr2=0x7f7fe0195f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:35:44.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.082+0000 7f7fe59cf700 1 --2- 172.21.15.22:0/344141724 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7fe01004e0 0x7f7fe0195f10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7fd400d8d0 tx=0x7f7fd400dbe0 comp rx=0 tx=0).stop 2024-02-16T19:35:44.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.082+0000 7f7fe59cf700 1 -- 172.21.15.22:0/344141724 shutdown_connections 2024-02-16T19:35:44.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.082+0000 7f7fe59cf700 1 --2- 172.21.15.22:0/344141724 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f7fc806d8a0 0x7f7fc806fd50 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:44.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.082+0000 7f7fe59cf700 1 --2- 172.21.15.22:0/344141724 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7fe01004e0 0x7f7fe0195f10 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:44.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.082+0000 7f7fe59cf700 1 --2- 172.21.15.22:0/344141724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fe0100e80 0x7f7fe0196450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:44.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.082+0000 7f7fe59cf700 1 -- 172.21.15.22:0/344141724 >> 172.21.15.22:0/344141724 conn(0x7f7fe00fbda0 msgr2=0x7f7fe00fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:35:44.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.082+0000 7f7fe59cf700 1 -- 172.21.15.22:0/344141724 shutdown_connections 2024-02-16T19:35:44.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.082+0000 7f7fe59cf700 1 -- 172.21.15.22:0/344141724 wait complete. 2024-02-16T19:35:44.095 INFO:teuthology.orchestra.run.smithi022.stdout:true 2024-02-16T19:35:44.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.189+0000 7f8861ad9700 1 -- 172.21.15.22:0/2840662639 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f885c0ff3c0 msgr2=0x7f885c0ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:35:44.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.189+0000 7f8861ad9700 1 --2- 172.21.15.22:0/2840662639 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f885c0ff3c0 0x7f885c0ff790 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f884c00b0a0 tx=0x7f884c00b3b0 comp rx=0 tx=0).stop 2024-02-16T19:35:44.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.189+0000 7f8861ad9700 1 -- 172.21.15.22:0/2840662639 shutdown_connections 2024-02-16T19:35:44.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.189+0000 7f8861ad9700 1 --2- 172.21.15.22:0/2840662639 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f885c0ffd60 0x7f885c103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:44.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.189+0000 7f8861ad9700 1 --2- 172.21.15.22:0/2840662639 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f885c0ff3c0 0x7f885c0ff790 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:44.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.189+0000 7f8861ad9700 1 -- 172.21.15.22:0/2840662639 >> 172.21.15.22:0/2840662639 conn(0x7f885c0fac60 msgr2=0x7f885c0fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:35:44.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.189+0000 7f8861ad9700 1 -- 172.21.15.22:0/2840662639 shutdown_connections 2024-02-16T19:35:44.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.189+0000 7f8861ad9700 1 -- 172.21.15.22:0/2840662639 wait complete. 2024-02-16T19:35:44.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.190+0000 7f8861ad9700 1 Processor -- start 2024-02-16T19:35:44.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.190+0000 7f8861ad9700 1 -- start start 2024-02-16T19:35:44.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.190+0000 7f8861ad9700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f885c0ff3c0 0x7f885c196dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:35:44.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.190+0000 7f8861ad9700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f885c0ffd60 0x7f885c197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:35:44.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.190+0000 7f8861ad9700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f885c197930 con 0x7f885c0ff3c0 2024-02-16T19:35:44.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.190+0000 7f8861ad9700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f885c197a70 con 0x7f885c0ffd60 2024-02-16T19:35:44.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.190+0000 7f885b7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f885c0ff3c0 0x7f885c196dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:35:44.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.190+0000 7f885b7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f885c0ff3c0 0x7f885c196dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40448/0 (socket says 172.21.15.22:40448) 2024-02-16T19:35:44.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.190+0000 7f885b7fe700 1 -- 172.21.15.22:0/1216693534 learned_addr learned my addr 172.21.15.22:0/1216693534 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:35:44.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.190+0000 7f885affd700 1 --2- 172.21.15.22:0/1216693534 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f885c0ffd60 0x7f885c197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:35:44.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.190+0000 7f885b7fe700 1 -- 172.21.15.22:0/1216693534 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f885c0ffd60 msgr2=0x7f885c197310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:35:44.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.190+0000 7f885b7fe700 1 --2- 172.21.15.22:0/1216693534 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f885c0ffd60 0x7f885c197310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:44.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.190+0000 7f885b7fe700 1 -- 172.21.15.22:0/1216693534 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f884c009d00 con 0x7f885c0ff3c0 2024-02-16T19:35:44.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.191+0000 7f885b7fe700 1 --2- 172.21.15.22:0/1216693534 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f885c0ff3c0 0x7f885c196dd0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f884c00b070 tx=0x7f884c00cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:35:44.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.191+0000 7f8858ff9700 1 -- 172.21.15.22:0/1216693534 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f884c00e040 con 0x7f885c0ff3c0 2024-02-16T19:35:44.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.191+0000 7f8858ff9700 1 -- 172.21.15.22:0/1216693534 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f884c003de0 con 0x7f885c0ff3c0 2024-02-16T19:35:44.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.191+0000 7f8861ad9700 1 -- 172.21.15.22:0/1216693534 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f885c19fe40 con 0x7f885c0ff3c0 2024-02-16T19:35:44.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.191+0000 7f8858ff9700 1 -- 172.21.15.22:0/1216693534 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f884c01d850 con 0x7f885c0ff3c0 2024-02-16T19:35:44.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.191+0000 7f8861ad9700 1 -- 172.21.15.22:0/1216693534 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f885c1a0250 con 0x7f885c0ff3c0 2024-02-16T19:35:44.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.192+0000 7f8861ad9700 1 -- 172.21.15.22:0/1216693534 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f885c1076d0 con 0x7f885c0ff3c0 2024-02-16T19:35:44.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.192+0000 7f8858ff9700 1 -- 172.21.15.22:0/1216693534 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f884c004040 con 0x7f885c0ff3c0 2024-02-16T19:35:44.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.193+0000 7f8858ff9700 1 --2- 172.21.15.22:0/1216693534 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f884806d8a0 0x7f884806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:35:44.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.193+0000 7f8858ff9700 1 -- 172.21.15.22:0/1216693534 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f884c08ee00 con 0x7f885c0ff3c0 2024-02-16T19:35:44.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.193+0000 7f885affd700 1 --2- 172.21.15.22:0/1216693534 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f884806d8a0 0x7f884806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:35:44.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.193+0000 7f885affd700 1 --2- 172.21.15.22:0/1216693534 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f884806d8a0 0x7f884806fd50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f885c069680 tx=0x7f884400a300 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:35:44.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.197+0000 7f8858ff9700 1 -- 172.21.15.22:0/1216693534 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f884c058540 con 0x7f885c0ff3c0 2024-02-16T19:35:44.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.424+0000 7f8861ad9700 1 -- 172.21.15.22:0/1216693534 --> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f885c061190 con 0x7f884806d8a0 2024-02-16T19:35:44.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.425+0000 7f8858ff9700 1 -- 172.21.15.22:0/1216693534 <== mgr.34104 v2:172.21.15.22:6800/505266753 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f885c061190 con 0x7f884806d8a0 2024-02-16T19:35:44.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.427+0000 7f8861ad9700 1 -- 172.21.15.22:0/1216693534 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f884806d8a0 msgr2=0x7f884806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:35:44.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.427+0000 7f8861ad9700 1 --2- 172.21.15.22:0/1216693534 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f884806d8a0 0x7f884806fd50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f885c069680 tx=0x7f884400a300 comp rx=0 tx=0).stop 2024-02-16T19:35:44.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.427+0000 7f8861ad9700 1 -- 172.21.15.22:0/1216693534 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f885c0ff3c0 msgr2=0x7f885c196dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:35:44.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.427+0000 7f8861ad9700 1 --2- 172.21.15.22:0/1216693534 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f885c0ff3c0 0x7f885c196dd0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f884c00b070 tx=0x7f884c00cd20 comp rx=0 tx=0).stop 2024-02-16T19:35:44.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.427+0000 7f8861ad9700 1 -- 172.21.15.22:0/1216693534 shutdown_connections 2024-02-16T19:35:44.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.427+0000 7f8861ad9700 1 --2- 172.21.15.22:0/1216693534 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f884806d8a0 0x7f884806fd50 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:44.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.427+0000 7f8861ad9700 1 --2- 172.21.15.22:0/1216693534 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f885c0ff3c0 0x7f885c196dd0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:44.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.427+0000 7f8861ad9700 1 --2- 172.21.15.22:0/1216693534 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f885c0ffd60 0x7f885c197310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:44.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.427+0000 7f8861ad9700 1 -- 172.21.15.22:0/1216693534 >> 172.21.15.22:0/1216693534 conn(0x7f885c0fac60 msgr2=0x7f885c0fc290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:35:44.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.428+0000 7f8861ad9700 1 -- 172.21.15.22:0/1216693534 shutdown_connections 2024-02-16T19:35:44.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.428+0000 7f8861ad9700 1 -- 172.21.15.22:0/1216693534 wait complete. 2024-02-16T19:35:44.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.534+0000 7f4caf076700 1 -- 172.21.15.22:0/2127448884 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4ca8100e80 msgr2=0x7f4ca8104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:35:44.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.534+0000 7f4caf076700 1 --2- 172.21.15.22:0/2127448884 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4ca8100e80 0x7f4ca8104e50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c009910 tx=0x7f4c9c009c20 comp rx=0 tx=0).stop 2024-02-16T19:35:44.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.534+0000 7f4caf076700 1 -- 172.21.15.22:0/2127448884 shutdown_connections 2024-02-16T19:35:44.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.534+0000 7f4caf076700 1 --2- 172.21.15.22:0/2127448884 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4ca8100e80 0x7f4ca8104e50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:44.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.534+0000 7f4caf076700 1 --2- 172.21.15.22:0/2127448884 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ca81004e0 0x7f4ca81008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:44.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.534+0000 7f4caf076700 1 -- 172.21.15.22:0/2127448884 >> 172.21.15.22:0/2127448884 conn(0x7f4ca80fbda0 msgr2=0x7f4ca80fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:35:44.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.534+0000 7f4caf076700 1 -- 172.21.15.22:0/2127448884 shutdown_connections 2024-02-16T19:35:44.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.534+0000 7f4caf076700 1 -- 172.21.15.22:0/2127448884 wait complete. 2024-02-16T19:35:44.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.534+0000 7f4caf076700 1 Processor -- start 2024-02-16T19:35:44.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.534+0000 7f4caf076700 1 -- start start 2024-02-16T19:35:44.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.535+0000 7f4caf076700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ca81004e0 0x7f4ca8073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:35:44.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.535+0000 7f4caf076700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4ca8100e80 0x7f4ca8073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:35:44.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.535+0000 7f4caf076700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca8073ad0 con 0x7f4ca8100e80 2024-02-16T19:35:44.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.535+0000 7f4caf076700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca8073c10 con 0x7f4ca81004e0 2024-02-16T19:35:44.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.535+0000 7f4ca7fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4ca8100e80 0x7f4ca8073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:35:44.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.535+0000 7f4ca7fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4ca8100e80 0x7f4ca8073590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45670/0 (socket says 172.21.15.22:45670) 2024-02-16T19:35:44.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.535+0000 7f4ca7fff700 1 -- 172.21.15.22:0/389699361 learned_addr learned my addr 172.21.15.22:0/389699361 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:35:44.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.535+0000 7f4cace12700 1 --2- 172.21.15.22:0/389699361 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ca81004e0 0x7f4ca8073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:35:44.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.535+0000 7f4ca7fff700 1 -- 172.21.15.22:0/389699361 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ca81004e0 msgr2=0x7f4ca8073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:35:44.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.535+0000 7f4ca7fff700 1 --2- 172.21.15.22:0/389699361 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ca81004e0 0x7f4ca8073050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:44.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.535+0000 7f4ca7fff700 1 -- 172.21.15.22:0/389699361 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c9c0095f0 con 0x7f4ca8100e80 2024-02-16T19:35:44.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.535+0000 7f4ca7fff700 1 --2- 172.21.15.22:0/389699361 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4ca8100e80 0x7f4ca8073590 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c009fd0 tx=0x7f4c9c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:35:44.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.535+0000 7f4ca5ffb700 1 -- 172.21.15.22:0/389699361 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c9c01d070 con 0x7f4ca8100e80 2024-02-16T19:35:44.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.535+0000 7f4ca5ffb700 1 -- 172.21.15.22:0/389699361 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4c9c022470 con 0x7f4ca8100e80 2024-02-16T19:35:44.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.535+0000 7f4caf076700 1 -- 172.21.15.22:0/389699361 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ca8073e90 con 0x7f4ca8100e80 2024-02-16T19:35:44.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.536+0000 7f4ca5ffb700 1 -- 172.21.15.22:0/389699361 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c9c017650 con 0x7f4ca8100e80 2024-02-16T19:35:44.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.536+0000 7f4caf076700 1 -- 172.21.15.22:0/389699361 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ca81a2360 con 0x7f4ca8100e80 2024-02-16T19:35:44.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.537+0000 7f4caf076700 1 -- 172.21.15.22:0/389699361 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ca8108850 con 0x7f4ca8100e80 2024-02-16T19:35:44.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.537+0000 7f4ca5ffb700 1 -- 172.21.15.22:0/389699361 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f4c9c0225e0 con 0x7f4ca8100e80 2024-02-16T19:35:44.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.537+0000 7f4ca5ffb700 1 --2- 172.21.15.22:0/389699361 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f4c9006d8a0 0x7f4c9006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:35:44.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.538+0000 7f4ca5ffb700 1 -- 172.21.15.22:0/389699361 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f4c9c08e9e0 con 0x7f4ca8100e80 2024-02-16T19:35:44.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.538+0000 7f4cace12700 1 --2- 172.21.15.22:0/389699361 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f4c9006d8a0 0x7f4c9006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:35:44.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.538+0000 7f4cace12700 1 --2- 172.21.15.22:0/389699361 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f4c9006d8a0 0x7f4c9006fd50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4ca000f540 tx=0x7f4ca000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:35:44.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.542+0000 7f4ca5ffb700 1 -- 172.21.15.22:0/389699361 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4c9c0581a0 con 0x7f4ca8100e80 2024-02-16T19:35:44.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.760+0000 7f4caf076700 1 -- 172.21.15.22:0/389699361 --> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f4ca8061190 con 0x7f4c9006d8a0 2024-02-16T19:35:44.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.770+0000 7f4ca5ffb700 1 -- 172.21.15.22:0/389699361 <== mgr.34104 v2:172.21.15.22:6800/505266753 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f4ca8061190 con 0x7f4c9006d8a0 2024-02-16T19:35:44.772 INFO:teuthology.orchestra.run.smithi022.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T19:35:44.772 INFO:teuthology.orchestra.run.smithi022.stdout:alertmanager.smithi022 smithi022 *:9093,9094 running (4m) 19s ago 14m 27.7M - 0.25.0 c8568f914cd2 c89bc76be374 2024-02-16T19:35:44.772 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-exporter.smithi022 smithi022 running (15m) 19s ago 15m 8409k - 18.2.0 dc2bc1663786 3f8bef4349ba 2024-02-16T19:35:44.772 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-exporter.smithi117 smithi117 running (14m) 80s ago 14m 7768k - 18.2.0 dc2bc1663786 59b65a7e309f 2024-02-16T19:35:44.773 INFO:teuthology.orchestra.run.smithi022.stdout:crash.smithi022 smithi022 running (89s) 19s ago 15m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a e0f78bdf43e4 2024-02-16T19:35:44.773 INFO:teuthology.orchestra.run.smithi022.stdout:crash.smithi117 smithi117 running (83s) 80s ago 14m 6669k - 19.0.0-1450-gf78a58c0 f081f847070a bce7c0590813 2024-02-16T19:35:44.773 INFO:teuthology.orchestra.run.smithi022.stdout:grafana.smithi022 smithi022 *:3000 running (3m) 19s ago 14m 76.6M - 9.4.12 a0ea5e3c46d9 fd27d11e9729 2024-02-16T19:35:44.773 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi022.btppwm smithi022 running (9m) 19s ago 9m 155M - 18.2.0 dc2bc1663786 22971fb3a907 2024-02-16T19:35:44.773 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi022.ocukvs smithi022 running (9m) 19s ago 9m 16.8M - 18.2.0 dc2bc1663786 81ee7bd5fd39 2024-02-16T19:35:44.773 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi117.ibmppn smithi117 running (9m) 80s ago 9m 17.6M - 18.2.0 dc2bc1663786 8c606cbeeb86 2024-02-16T19:35:44.773 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi117.orzors smithi117 running (9m) 80s ago 9m 16.6M - 18.2.0 dc2bc1663786 7d2bc4df3feb 2024-02-16T19:35:44.773 INFO:teuthology.orchestra.run.smithi022.stdout:mgr.smithi022.gmdcch smithi022 *:8443,9283,8765 running (6m) 19s ago 16m 503M - 19.0.0-1450-gf78a58c0 f081f847070a 7cb74cc9deea 2024-02-16T19:35:44.773 INFO:teuthology.orchestra.run.smithi022.stdout:mgr.smithi117.cohbtv smithi117 *:8443,9283,8765 running (5m) 80s ago 13m 451M - 19.0.0-1450-gf78a58c0 f081f847070a c825b1152189 2024-02-16T19:35:44.773 INFO:teuthology.orchestra.run.smithi022.stdout:mon.smithi022 smithi022 running (2m) 19s ago 16m 46.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 9c84755ae67d 2024-02-16T19:35:44.773 INFO:teuthology.orchestra.run.smithi022.stdout:mon.smithi117 smithi117 running (2m) 80s ago 13m 38.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a a319ac50f1a9 2024-02-16T19:35:44.773 INFO:teuthology.orchestra.run.smithi022.stdout:node-exporter.smithi022 smithi022 *:9100 running (4m) 19s ago 15m 16.5M - 1.5.0 0da6a335fe13 1f5b58292d27 2024-02-16T19:35:44.773 INFO:teuthology.orchestra.run.smithi022.stdout:node-exporter.smithi117 smithi117 *:9100 running (4m) 80s ago 13m 15.4M - 1.5.0 0da6a335fe13 bb5e8491d5a6 2024-02-16T19:35:44.773 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0 smithi022 running (62s) 19s ago 13m 94.8M 1217M 19.0.0-1450-gf78a58c0 f081f847070a 002412b3525f 2024-02-16T19:35:44.773 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1 smithi022 running (25s) 19s ago 12m 12.6M 1217M 19.0.0-1450-gf78a58c0 f081f847070a c3488ce7b2a0 2024-02-16T19:35:44.774 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2 smithi022 running (12m) 19s ago 12m 192M 1217M 18.2.0 dc2bc1663786 be8a631b4f93 2024-02-16T19:35:44.774 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3 smithi117 running (11m) 80s ago 11m 244M 2240M 18.2.0 dc2bc1663786 9c0ce9a981cf 2024-02-16T19:35:44.774 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4 smithi117 running (11m) 80s ago 11m 209M 2240M 18.2.0 dc2bc1663786 1779c3496f4d 2024-02-16T19:35:44.774 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5 smithi117 running (10m) 80s ago 10m 198M 2240M 18.2.0 dc2bc1663786 fefaabca0638 2024-02-16T19:35:44.774 INFO:teuthology.orchestra.run.smithi022.stdout:prometheus.smithi022 smithi022 *:9095 running (4m) 19s ago 14m 66.9M - 2.43.0 a07b618ecd1d c3b8689d0c97 2024-02-16T19:35:44.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.773+0000 7f4caf076700 1 -- 172.21.15.22:0/389699361 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f4c9006d8a0 msgr2=0x7f4c9006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:35:44.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.773+0000 7f4caf076700 1 --2- 172.21.15.22:0/389699361 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f4c9006d8a0 0x7f4c9006fd50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4ca000f540 tx=0x7f4ca000b040 comp rx=0 tx=0).stop 2024-02-16T19:35:44.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.773+0000 7f4caf076700 1 -- 172.21.15.22:0/389699361 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4ca8100e80 msgr2=0x7f4ca8073590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:35:44.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.773+0000 7f4caf076700 1 --2- 172.21.15.22:0/389699361 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4ca8100e80 0x7f4ca8073590 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c009fd0 tx=0x7f4c9c00f690 comp rx=0 tx=0).stop 2024-02-16T19:35:44.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.773+0000 7f4caf076700 1 -- 172.21.15.22:0/389699361 shutdown_connections 2024-02-16T19:35:44.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.773+0000 7f4caf076700 1 --2- 172.21.15.22:0/389699361 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f4c9006d8a0 0x7f4c9006fd50 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:44.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.773+0000 7f4caf076700 1 --2- 172.21.15.22:0/389699361 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ca81004e0 0x7f4ca8073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:44.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.773+0000 7f4caf076700 1 --2- 172.21.15.22:0/389699361 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4ca8100e80 0x7f4ca8073590 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:44.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.773+0000 7f4caf076700 1 -- 172.21.15.22:0/389699361 >> 172.21.15.22:0/389699361 conn(0x7f4ca80fbda0 msgr2=0x7f4ca80fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:35:44.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.773+0000 7f4caf076700 1 -- 172.21.15.22:0/389699361 shutdown_connections 2024-02-16T19:35:44.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.773+0000 7f4caf076700 1 -- 172.21.15.22:0/389699361 wait complete. 2024-02-16T19:35:44.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.878+0000 7f7fdb371700 1 -- 172.21.15.22:0/2209553837 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7fd4100e80 msgr2=0x7f7fd4104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:35:44.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.878+0000 7f7fdb371700 1 --2- 172.21.15.22:0/2209553837 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7fd4100e80 0x7f7fd4104e50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8009910 tx=0x7f7fc8009c20 comp rx=0 tx=0).stop 2024-02-16T19:35:44.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.878+0000 7f7fdb371700 1 -- 172.21.15.22:0/2209553837 shutdown_connections 2024-02-16T19:35:44.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.878+0000 7f7fdb371700 1 --2- 172.21.15.22:0/2209553837 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7fd4100e80 0x7f7fd4104e50 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:44.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.878+0000 7f7fdb371700 1 --2- 172.21.15.22:0/2209553837 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fd41004e0 0x7f7fd41008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:44.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.878+0000 7f7fdb371700 1 -- 172.21.15.22:0/2209553837 >> 172.21.15.22:0/2209553837 conn(0x7f7fd40fbda0 msgr2=0x7f7fd40fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:35:44.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.878+0000 7f7fdb371700 1 -- 172.21.15.22:0/2209553837 shutdown_connections 2024-02-16T19:35:44.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.878+0000 7f7fdb371700 1 -- 172.21.15.22:0/2209553837 wait complete. 2024-02-16T19:35:44.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.879+0000 7f7fdb371700 1 Processor -- start 2024-02-16T19:35:44.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.879+0000 7f7fdb371700 1 -- start start 2024-02-16T19:35:44.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.879+0000 7f7fdb371700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7fd41004e0 0x7f7fd4073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:35:44.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.879+0000 7f7fdb371700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fd4100e80 0x7f7fd4073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:35:44.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.879+0000 7f7fdb371700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd4073ad0 con 0x7f7fd41004e0 2024-02-16T19:35:44.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.879+0000 7f7fdb371700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd4073c10 con 0x7f7fd4100e80 2024-02-16T19:35:44.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.879+0000 7f7fd910d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7fd41004e0 0x7f7fd4073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:35:44.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.879+0000 7f7fd910d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7fd41004e0 0x7f7fd4073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45684/0 (socket says 172.21.15.22:45684) 2024-02-16T19:35:44.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.879+0000 7f7fd910d700 1 -- 172.21.15.22:0/3392115988 learned_addr learned my addr 172.21.15.22:0/3392115988 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:35:44.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.879+0000 7f7fd890c700 1 --2- 172.21.15.22:0/3392115988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fd4100e80 0x7f7fd4073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:35:44.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.879+0000 7f7fd910d700 1 -- 172.21.15.22:0/3392115988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fd4100e80 msgr2=0x7f7fd4073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:35:44.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.879+0000 7f7fd910d700 1 --2- 172.21.15.22:0/3392115988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fd4100e80 0x7f7fd4073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:44.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.879+0000 7f7fd910d700 1 -- 172.21.15.22:0/3392115988 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fc80095f0 con 0x7f7fd41004e0 2024-02-16T19:35:44.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.879+0000 7f7fd890c700 1 --2- 172.21.15.22:0/3392115988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fd4100e80 0x7f7fd4073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T19:35:44.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.879+0000 7f7fd910d700 1 --2- 172.21.15.22:0/3392115988 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7fd41004e0 0x7f7fd4073050 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7fc400ada0 tx=0x7f7fc400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:35:44.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.880+0000 7f7fd27fc700 1 -- 172.21.15.22:0/3392115988 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7fc400f070 con 0x7f7fd41004e0 2024-02-16T19:35:44.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.880+0000 7f7fdb371700 1 -- 172.21.15.22:0/3392115988 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fd41a2100 con 0x7f7fd41004e0 2024-02-16T19:35:44.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.880+0000 7f7fd27fc700 1 -- 172.21.15.22:0/3392115988 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7fc4007bf0 con 0x7f7fd41004e0 2024-02-16T19:35:44.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.880+0000 7f7fd27fc700 1 -- 172.21.15.22:0/3392115988 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7fc4005690 con 0x7f7fd41004e0 2024-02-16T19:35:44.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.880+0000 7f7fdb371700 1 -- 172.21.15.22:0/3392115988 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fd41a2510 con 0x7f7fd41004e0 2024-02-16T19:35:44.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.881+0000 7f7fdb371700 1 -- 172.21.15.22:0/3392115988 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fd4108850 con 0x7f7fd41004e0 2024-02-16T19:35:44.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.881+0000 7f7fd27fc700 1 -- 172.21.15.22:0/3392115988 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f7fc4005920 con 0x7f7fd41004e0 2024-02-16T19:35:44.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.881+0000 7f7fd27fc700 1 --2- 172.21.15.22:0/3392115988 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f7fbc06d8a0 0x7f7fbc06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:35:44.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.882+0000 7f7fd27fc700 1 -- 172.21.15.22:0/3392115988 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f7fc408cfe0 con 0x7f7fd41004e0 2024-02-16T19:35:44.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.882+0000 7f7fd890c700 1 --2- 172.21.15.22:0/3392115988 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f7fbc06d8a0 0x7f7fbc06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:35:44.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.882+0000 7f7fd890c700 1 --2- 172.21.15.22:0/3392115988 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f7fbc06d8a0 0x7f7fbc06fd50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4074840 tx=0x7f7fc801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:35:44.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:44.886+0000 7f7fd27fc700 1 -- 172.21.15.22:0/3392115988 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7fc4056720 con 0x7f7fd41004e0 2024-02-16T19:35:45.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.191+0000 7f7fdb371700 1 -- 172.21.15.22:0/3392115988 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7fd4073dd0 con 0x7f7fd41004e0 2024-02-16T19:35:45.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.192+0000 7f7fd27fc700 1 -- 172.21.15.22:0/3392115988 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+791 (secure 0 0 0) 0x7f7fc4059de0 con 0x7f7fd41004e0 2024-02-16T19:35:45.194 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-02-16T19:35:45.194 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": { 2024-02-16T19:35:45.194 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T19:35:45.194 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:35:45.194 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": { 2024-02-16T19:35:45.194 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T19:35:45.194 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:35:45.194 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": { 2024-02-16T19:35:45.194 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4, 2024-02-16T19:35:45.194 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T19:35:45.194 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:35:45.195 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": { 2024-02-16T19:35:45.195 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-02-16T19:35:45.195 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:35:45.195 INFO:teuthology.orchestra.run.smithi022.stdout: "overall": { 2024-02-16T19:35:45.195 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 8, 2024-02-16T19:35:45.195 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-16T19:35:45.195 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-02-16T19:35:45.195 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-02-16T19:35:45.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.195+0000 7f7fdb371700 1 -- 172.21.15.22:0/3392115988 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f7fbc06d8a0 msgr2=0x7f7fbc06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:35:45.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.195+0000 7f7fdb371700 1 --2- 172.21.15.22:0/3392115988 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f7fbc06d8a0 0x7f7fbc06fd50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4074840 tx=0x7f7fc801a040 comp rx=0 tx=0).stop 2024-02-16T19:35:45.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.195+0000 7f7fdb371700 1 -- 172.21.15.22:0/3392115988 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7fd41004e0 msgr2=0x7f7fd4073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:35:45.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.195+0000 7f7fdb371700 1 --2- 172.21.15.22:0/3392115988 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7fd41004e0 0x7f7fd4073050 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7fc400ada0 tx=0x7f7fc400c330 comp rx=0 tx=0).stop 2024-02-16T19:35:45.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.195+0000 7f7fdb371700 1 -- 172.21.15.22:0/3392115988 shutdown_connections 2024-02-16T19:35:45.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.195+0000 7f7fdb371700 1 --2- 172.21.15.22:0/3392115988 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f7fbc06d8a0 0x7f7fbc06fd50 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:45.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.195+0000 7f7fdb371700 1 --2- 172.21.15.22:0/3392115988 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7fd41004e0 0x7f7fd4073050 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:45.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.195+0000 7f7fdb371700 1 --2- 172.21.15.22:0/3392115988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fd4100e80 0x7f7fd4073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:45.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.195+0000 7f7fdb371700 1 -- 172.21.15.22:0/3392115988 >> 172.21.15.22:0/3392115988 conn(0x7f7fd40fbda0 msgr2=0x7f7fd40fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:35:45.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.195+0000 7f7fdb371700 1 -- 172.21.15.22:0/3392115988 shutdown_connections 2024-02-16T19:35:45.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.195+0000 7f7fdb371700 1 -- 172.21.15.22:0/3392115988 wait complete. 2024-02-16T19:35:45.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.305+0000 7f0a94170700 1 -- 172.21.15.22:0/3414573555 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a8c100e80 msgr2=0x7f0a8c104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:35:45.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.305+0000 7f0a94170700 1 --2- 172.21.15.22:0/3414573555 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a8c100e80 0x7f0a8c104e50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c009910 tx=0x7f0a7c009c20 comp rx=0 tx=0).stop 2024-02-16T19:35:45.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.305+0000 7f0a94170700 1 -- 172.21.15.22:0/3414573555 shutdown_connections 2024-02-16T19:35:45.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.305+0000 7f0a94170700 1 --2- 172.21.15.22:0/3414573555 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a8c100e80 0x7f0a8c104e50 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:45.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.305+0000 7f0a94170700 1 --2- 172.21.15.22:0/3414573555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a8c1004e0 0x7f0a8c1008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:45.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.305+0000 7f0a94170700 1 -- 172.21.15.22:0/3414573555 >> 172.21.15.22:0/3414573555 conn(0x7f0a8c0fbda0 msgr2=0x7f0a8c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:35:45.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.306+0000 7f0a94170700 1 -- 172.21.15.22:0/3414573555 shutdown_connections 2024-02-16T19:35:45.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.306+0000 7f0a94170700 1 -- 172.21.15.22:0/3414573555 wait complete. 2024-02-16T19:35:45.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.306+0000 7f0a94170700 1 Processor -- start 2024-02-16T19:35:45.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.306+0000 7f0a94170700 1 -- start start 2024-02-16T19:35:45.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.306+0000 7f0a94170700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a8c1004e0 0x7f0a8c073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:35:45.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.306+0000 7f0a94170700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a8c100e80 0x7f0a8c073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:35:45.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.306+0000 7f0a94170700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8c073ad0 con 0x7f0a8c100e80 2024-02-16T19:35:45.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.306+0000 7f0a94170700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8c073c10 con 0x7f0a8c1004e0 2024-02-16T19:35:45.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.307+0000 7f0a9170b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a8c100e80 0x7f0a8c073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:35:45.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.307+0000 7f0a9170b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a8c100e80 0x7f0a8c073590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45700/0 (socket says 172.21.15.22:45700) 2024-02-16T19:35:45.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.307+0000 7f0a91f0c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a8c1004e0 0x7f0a8c073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:35:45.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.307+0000 7f0a9170b700 1 -- 172.21.15.22:0/251580174 learned_addr learned my addr 172.21.15.22:0/251580174 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:35:45.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.307+0000 7f0a9170b700 1 -- 172.21.15.22:0/251580174 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a8c1004e0 msgr2=0x7f0a8c073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:35:45.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.307+0000 7f0a9170b700 1 --2- 172.21.15.22:0/251580174 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a8c1004e0 0x7f0a8c073050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:45.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.307+0000 7f0a9170b700 1 -- 172.21.15.22:0/251580174 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a7c0095f0 con 0x7f0a8c100e80 2024-02-16T19:35:45.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.307+0000 7f0a9170b700 1 --2- 172.21.15.22:0/251580174 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a8c100e80 0x7f0a8c073590 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c0092e0 tx=0x7f0a7c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:35:45.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.307+0000 7f0a82ffd700 1 -- 172.21.15.22:0/251580174 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0a7c01d070 con 0x7f0a8c100e80 2024-02-16T19:35:45.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.307+0000 7f0a94170700 1 -- 172.21.15.22:0/251580174 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a8c073e90 con 0x7f0a8c100e80 2024-02-16T19:35:45.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.307+0000 7f0a82ffd700 1 -- 172.21.15.22:0/251580174 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0a7c005dc0 con 0x7f0a8c100e80 2024-02-16T19:35:45.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.307+0000 7f0a82ffd700 1 -- 172.21.15.22:0/251580174 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0a7c017400 con 0x7f0a8c100e80 2024-02-16T19:35:45.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.307+0000 7f0a94170700 1 -- 172.21.15.22:0/251580174 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a8c1a2430 con 0x7f0a8c100e80 2024-02-16T19:35:45.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.308+0000 7f0a94170700 1 -- 172.21.15.22:0/251580174 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a8c108850 con 0x7f0a8c100e80 2024-02-16T19:35:45.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.309+0000 7f0a82ffd700 1 -- 172.21.15.22:0/251580174 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f0a7c00f6f0 con 0x7f0a8c100e80 2024-02-16T19:35:45.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.310+0000 7f0a82ffd700 1 --2- 172.21.15.22:0/251580174 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f0a7806d8a0 0x7f0a7806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:35:45.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.310+0000 7f0a82ffd700 1 -- 172.21.15.22:0/251580174 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f0a7c08e980 con 0x7f0a8c100e80 2024-02-16T19:35:45.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.310+0000 7f0a91f0c700 1 --2- 172.21.15.22:0/251580174 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f0a7806d8a0 0x7f0a7806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:35:45.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.310+0000 7f0a91f0c700 1 --2- 172.21.15.22:0/251580174 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f0a7806d8a0 0x7f0a7806fd50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0a88007be0 tx=0x7f0a880073d0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:35:45.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.314+0000 7f0a82ffd700 1 -- 172.21.15.22:0/251580174 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0a7c0580c0 con 0x7f0a8c100e80 2024-02-16T19:35:45.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:45 smithi022 ceph-mon[88798]: pgmap v96: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s, 33 objects/s recovering 2024-02-16T19:35:45.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:45 smithi022 ceph-mon[88798]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:35:45.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:45 smithi022 ceph-mon[88798]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:35:45.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:45 smithi022 ceph-mon[88798]: from='client.34260 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:35:45.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:45 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-02-16T19:35:45.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:45 smithi022 ceph-mon[88798]: from='client.? 172.21.15.22:0/3392115988' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:35:45.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.578+0000 7f0a94170700 1 -- 172.21.15.22:0/251580174 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f0a8c04f2a0 con 0x7f0a8c100e80 2024-02-16T19:35:45.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.578+0000 7f0a82ffd700 1 -- 172.21.15.22:0/251580174 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1845 (secure 0 0 0) 0x7f0a7c05b780 con 0x7f0a8c100e80 2024-02-16T19:35:45.580 INFO:teuthology.orchestra.run.smithi022.stdout:e15 2024-02-16T19:35:45.580 INFO:teuthology.orchestra.run.smithi022.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T19:35:45.580 INFO:teuthology.orchestra.run.smithi022.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:35:45.580 INFO:teuthology.orchestra.run.smithi022.stdout:legacy client fscid: 1 2024-02-16T19:35:45.580 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:35:45.580 INFO:teuthology.orchestra.run.smithi022.stdout:Filesystem 'cephfs' (1) 2024-02-16T19:35:45.580 INFO:teuthology.orchestra.run.smithi022.stdout:fs_name cephfs 2024-02-16T19:35:45.581 INFO:teuthology.orchestra.run.smithi022.stdout:epoch 15 2024-02-16T19:35:45.581 INFO:teuthology.orchestra.run.smithi022.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T19:35:45.581 INFO:teuthology.orchestra.run.smithi022.stdout:created 2024-02-16T19:26:17.873644+0000 2024-02-16T19:35:45.581 INFO:teuthology.orchestra.run.smithi022.stdout:modified 2024-02-16T19:26:35.943353+0000 2024-02-16T19:35:45.581 INFO:teuthology.orchestra.run.smithi022.stdout:tableserver 0 2024-02-16T19:35:45.581 INFO:teuthology.orchestra.run.smithi022.stdout:root 0 2024-02-16T19:35:45.581 INFO:teuthology.orchestra.run.smithi022.stdout:session_timeout 60 2024-02-16T19:35:45.581 INFO:teuthology.orchestra.run.smithi022.stdout:session_autoclose 300 2024-02-16T19:35:45.581 INFO:teuthology.orchestra.run.smithi022.stdout:max_file_size 1099511627776 2024-02-16T19:35:45.581 INFO:teuthology.orchestra.run.smithi022.stdout:max_xattr_size 65536 2024-02-16T19:35:45.581 INFO:teuthology.orchestra.run.smithi022.stdout:required_client_features {} 2024-02-16T19:35:45.581 INFO:teuthology.orchestra.run.smithi022.stdout:last_failure 0 2024-02-16T19:35:45.581 INFO:teuthology.orchestra.run.smithi022.stdout:last_failure_osd_epoch 47 2024-02-16T19:35:45.581 INFO:teuthology.orchestra.run.smithi022.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:35:45.582 INFO:teuthology.orchestra.run.smithi022.stdout:max_mds 1 2024-02-16T19:35:45.582 INFO:teuthology.orchestra.run.smithi022.stdout:in 0 2024-02-16T19:35:45.582 INFO:teuthology.orchestra.run.smithi022.stdout:up {0=14490} 2024-02-16T19:35:45.582 INFO:teuthology.orchestra.run.smithi022.stdout:failed 2024-02-16T19:35:45.582 INFO:teuthology.orchestra.run.smithi022.stdout:damaged 2024-02-16T19:35:45.582 INFO:teuthology.orchestra.run.smithi022.stdout:stopped 2024-02-16T19:35:45.582 INFO:teuthology.orchestra.run.smithi022.stdout:data_pools [3] 2024-02-16T19:35:45.582 INFO:teuthology.orchestra.run.smithi022.stdout:metadata_pool 2 2024-02-16T19:35:45.582 INFO:teuthology.orchestra.run.smithi022.stdout:inline_data enabled 2024-02-16T19:35:45.582 INFO:teuthology.orchestra.run.smithi022.stdout:balancer 2024-02-16T19:35:45.582 INFO:teuthology.orchestra.run.smithi022.stdout:bal_rank_mask -1 2024-02-16T19:35:45.582 INFO:teuthology.orchestra.run.smithi022.stdout:standby_count_wanted 1 2024-02-16T19:35:45.582 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.cephfs.smithi022.btppwm{0:14490} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.22:6828/3597685751,v1:172.21.15.22:6829/3597685751] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:35:45.582 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:35:45.583 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:35:45.583 INFO:teuthology.orchestra.run.smithi022.stdout:Standby daemons: 2024-02-16T19:35:45.583 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:35:45.583 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.cephfs.smithi022.ocukvs{-1:14500} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.22:6826/242865843,v1:172.21.15.22:6827/242865843] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:35:45.583 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.cephfs.smithi117.ibmppn{-1:24259} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.117:6824/565408102,v1:172.21.15.117:6825/565408102] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:35:45.583 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.cephfs.smithi117.orzors{-1:24265} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.117:6826/3122342028,v1:172.21.15.117:6827/3122342028] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:35:45.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.581+0000 7f0a94170700 1 -- 172.21.15.22:0/251580174 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f0a7806d8a0 msgr2=0x7f0a7806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:35:45.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.581+0000 7f0a94170700 1 --2- 172.21.15.22:0/251580174 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f0a7806d8a0 0x7f0a7806fd50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0a88007be0 tx=0x7f0a880073d0 comp rx=0 tx=0).stop 2024-02-16T19:35:45.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.581+0000 7f0a94170700 1 -- 172.21.15.22:0/251580174 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a8c100e80 msgr2=0x7f0a8c073590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:35:45.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.581+0000 7f0a94170700 1 --2- 172.21.15.22:0/251580174 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a8c100e80 0x7f0a8c073590 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c0092e0 tx=0x7f0a7c00bd70 comp rx=0 tx=0).stop 2024-02-16T19:35:45.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.581+0000 7f0a94170700 1 -- 172.21.15.22:0/251580174 shutdown_connections 2024-02-16T19:35:45.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.581+0000 7f0a94170700 1 --2- 172.21.15.22:0/251580174 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f0a7806d8a0 0x7f0a7806fd50 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:45.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.581+0000 7f0a94170700 1 --2- 172.21.15.22:0/251580174 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a8c1004e0 0x7f0a8c073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:45.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.581+0000 7f0a94170700 1 --2- 172.21.15.22:0/251580174 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a8c100e80 0x7f0a8c073590 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:45.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.582+0000 7f0a94170700 1 -- 172.21.15.22:0/251580174 >> 172.21.15.22:0/251580174 conn(0x7f0a8c0fbda0 msgr2=0x7f0a8c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:35:45.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.582+0000 7f0a94170700 1 -- 172.21.15.22:0/251580174 shutdown_connections 2024-02-16T19:35:45.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.582+0000 7f0a94170700 1 -- 172.21.15.22:0/251580174 wait complete. 2024-02-16T19:35:45.584 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 15 2024-02-16T19:35:45.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.694+0000 7febe164a700 1 -- 172.21.15.22:0/255629820 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7febdc0ff3c0 msgr2=0x7febdc0ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:35:45.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.694+0000 7febe164a700 1 --2- 172.21.15.22:0/255629820 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7febdc0ff3c0 0x7febdc0ff790 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7febcc009910 tx=0x7febcc009c20 comp rx=0 tx=0).stop 2024-02-16T19:35:45.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.694+0000 7febe164a700 1 -- 172.21.15.22:0/255629820 shutdown_connections 2024-02-16T19:35:45.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.694+0000 7febe164a700 1 --2- 172.21.15.22:0/255629820 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febdc0ffd60 0x7febdc103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:45.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.694+0000 7febe164a700 1 --2- 172.21.15.22:0/255629820 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7febdc0ff3c0 0x7febdc0ff790 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:45.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.694+0000 7febe164a700 1 -- 172.21.15.22:0/255629820 >> 172.21.15.22:0/255629820 conn(0x7febdc0fac80 msgr2=0x7febdc0fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:35:45.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.694+0000 7febe164a700 1 -- 172.21.15.22:0/255629820 shutdown_connections 2024-02-16T19:35:45.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.694+0000 7febe164a700 1 -- 172.21.15.22:0/255629820 wait complete. 2024-02-16T19:35:45.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.694+0000 7febe164a700 1 Processor -- start 2024-02-16T19:35:45.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.694+0000 7febe164a700 1 -- start start 2024-02-16T19:35:45.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.695+0000 7febe164a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7febdc0ff3c0 0x7febdc1981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:35:45.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.695+0000 7febe164a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febdc0ffd60 0x7febdc198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:35:45.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.695+0000 7febe164a700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febdc198de0 con 0x7febdc0ff3c0 2024-02-16T19:35:45.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.695+0000 7febe164a700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febdc19cb70 con 0x7febdc0ffd60 2024-02-16T19:35:45.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.695+0000 7febdaffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7febdc0ff3c0 0x7febdc1981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:35:45.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.695+0000 7febdaffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7febdc0ff3c0 0x7febdc1981c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45714/0 (socket says 172.21.15.22:45714) 2024-02-16T19:35:45.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.695+0000 7febdaffd700 1 -- 172.21.15.22:0/4230814271 learned_addr learned my addr 172.21.15.22:0/4230814271 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:35:45.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.695+0000 7febda7fc700 1 --2- 172.21.15.22:0/4230814271 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febdc0ffd60 0x7febdc198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:35:45.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.695+0000 7febdaffd700 1 -- 172.21.15.22:0/4230814271 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febdc0ffd60 msgr2=0x7febdc198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:35:45.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.695+0000 7febdaffd700 1 --2- 172.21.15.22:0/4230814271 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febdc0ffd60 0x7febdc198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:45.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.695+0000 7febdaffd700 1 -- 172.21.15.22:0/4230814271 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febcc0095f0 con 0x7febdc0ff3c0 2024-02-16T19:35:45.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.695+0000 7febdaffd700 1 --2- 172.21.15.22:0/4230814271 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7febdc0ff3c0 0x7febdc1981c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7febcc006010 tx=0x7febcc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:35:45.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.696+0000 7febc3fff700 1 -- 172.21.15.22:0/4230814271 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7febcc01d070 con 0x7febdc0ff3c0 2024-02-16T19:35:45.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.696+0000 7febe164a700 1 -- 172.21.15.22:0/4230814271 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febdc19cdf0 con 0x7febdc0ff3c0 2024-02-16T19:35:45.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.696+0000 7febc3fff700 1 -- 172.21.15.22:0/4230814271 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7febcc004fb0 con 0x7febdc0ff3c0 2024-02-16T19:35:45.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.696+0000 7febc3fff700 1 -- 172.21.15.22:0/4230814271 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7febcc022ea0 con 0x7febdc0ff3c0 2024-02-16T19:35:45.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.696+0000 7febe164a700 1 -- 172.21.15.22:0/4230814271 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febdc19d2e0 con 0x7febdc0ff3c0 2024-02-16T19:35:45.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.697+0000 7febe164a700 1 -- 172.21.15.22:0/4230814271 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febdc1076d0 con 0x7febdc0ff3c0 2024-02-16T19:35:45.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.697+0000 7febc3fff700 1 -- 172.21.15.22:0/4230814271 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7febcc00f690 con 0x7febdc0ff3c0 2024-02-16T19:35:45.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.698+0000 7febc3fff700 1 --2- 172.21.15.22:0/4230814271 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7febc406d8a0 0x7febc406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:35:45.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.698+0000 7febc3fff700 1 -- 172.21.15.22:0/4230814271 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6704+0+0 (secure 0 0 0) 0x7febcc08eb00 con 0x7febdc0ff3c0 2024-02-16T19:35:45.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.698+0000 7febda7fc700 1 --2- 172.21.15.22:0/4230814271 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7febc406d8a0 0x7febc406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:35:45.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.698+0000 7febda7fc700 1 --2- 172.21.15.22:0/4230814271 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7febc406d8a0 0x7febc406fd50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7febdc1997e0 tx=0x7febd0005eb0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:35:45.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.703+0000 7febc3fff700 1 -- 172.21.15.22:0/4230814271 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7febcc058330 con 0x7febdc0ff3c0 2024-02-16T19:35:45.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:45 smithi117 ceph-mon[67870]: pgmap v96: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s, 33 objects/s recovering 2024-02-16T19:35:45.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:45 smithi117 ceph-mon[67870]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:35:45.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:45 smithi117 ceph-mon[67870]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:35:45.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:45 smithi117 ceph-mon[67870]: from='client.34260 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:35:45.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:45 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-02-16T19:35:45.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:45 smithi117 ceph-mon[67870]: from='client.? 172.21.15.22:0/3392115988' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:35:45.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.937+0000 7febe164a700 1 -- 172.21.15.22:0/4230814271 --> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7febdc199600 con 0x7febc406d8a0 2024-02-16T19:35:45.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.939+0000 7febc3fff700 1 -- 172.21.15.22:0/4230814271 <== mgr.34104 v2:172.21.15.22:6800/505266753 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7febdc199600 con 0x7febc406d8a0 2024-02-16T19:35:45.940 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-02-16T19:35:45.940 INFO:teuthology.orchestra.run.smithi022.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T19:35:45.940 INFO:teuthology.orchestra.run.smithi022.stdout: "in_progress": true, 2024-02-16T19:35:45.940 INFO:teuthology.orchestra.run.smithi022.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T19:35:45.940 INFO:teuthology.orchestra.run.smithi022.stdout: "services_complete": [ 2024-02-16T19:35:45.940 INFO:teuthology.orchestra.run.smithi022.stdout: "crash", 2024-02-16T19:35:45.941 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr", 2024-02-16T19:35:45.941 INFO:teuthology.orchestra.run.smithi022.stdout: "mon" 2024-02-16T19:35:45.941 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-02-16T19:35:45.941 INFO:teuthology.orchestra.run.smithi022.stdout: "progress": "8/23 daemons upgraded", 2024-02-16T19:35:45.941 INFO:teuthology.orchestra.run.smithi022.stdout: "message": "Currently upgrading osd daemons", 2024-02-16T19:35:45.941 INFO:teuthology.orchestra.run.smithi022.stdout: "is_paused": false 2024-02-16T19:35:45.941 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-02-16T19:35:45.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.941+0000 7febe164a700 1 -- 172.21.15.22:0/4230814271 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7febc406d8a0 msgr2=0x7febc406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:35:45.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.941+0000 7febe164a700 1 --2- 172.21.15.22:0/4230814271 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7febc406d8a0 0x7febc406fd50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7febdc1997e0 tx=0x7febd0005eb0 comp rx=0 tx=0).stop 2024-02-16T19:35:45.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.941+0000 7febe164a700 1 -- 172.21.15.22:0/4230814271 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7febdc0ff3c0 msgr2=0x7febdc1981c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:35:45.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.941+0000 7febe164a700 1 --2- 172.21.15.22:0/4230814271 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7febdc0ff3c0 0x7febdc1981c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7febcc006010 tx=0x7febcc00be20 comp rx=0 tx=0).stop 2024-02-16T19:35:45.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.941+0000 7febe164a700 1 -- 172.21.15.22:0/4230814271 shutdown_connections 2024-02-16T19:35:45.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.941+0000 7febe164a700 1 --2- 172.21.15.22:0/4230814271 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7febc406d8a0 0x7febc406fd50 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:45.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.941+0000 7febe164a700 1 --2- 172.21.15.22:0/4230814271 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7febdc0ff3c0 0x7febdc1981c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:45.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.941+0000 7febe164a700 1 --2- 172.21.15.22:0/4230814271 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febdc0ffd60 0x7febdc198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:45.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.941+0000 7febe164a700 1 -- 172.21.15.22:0/4230814271 >> 172.21.15.22:0/4230814271 conn(0x7febdc0fac80 msgr2=0x7febdc0fc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:35:45.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.941+0000 7febe164a700 1 -- 172.21.15.22:0/4230814271 shutdown_connections 2024-02-16T19:35:45.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:45.941+0000 7febe164a700 1 -- 172.21.15.22:0/4230814271 wait complete. 2024-02-16T19:35:46.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:46.048+0000 7f2e6f380700 1 -- 172.21.15.22:0/3982459150 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2e68100e80 msgr2=0x7f2e68104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:35:46.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:46.048+0000 7f2e6f380700 1 --2- 172.21.15.22:0/3982459150 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2e68100e80 0x7f2e68104e50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c009910 tx=0x7f2e5c009c20 comp rx=0 tx=0).stop 2024-02-16T19:35:46.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:46.048+0000 7f2e6f380700 1 -- 172.21.15.22:0/3982459150 shutdown_connections 2024-02-16T19:35:46.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:46.048+0000 7f2e6f380700 1 --2- 172.21.15.22:0/3982459150 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2e68100e80 0x7f2e68104e50 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:46.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:46.048+0000 7f2e6f380700 1 --2- 172.21.15.22:0/3982459150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e681004e0 0x7f2e681008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:46.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:46.048+0000 7f2e6f380700 1 -- 172.21.15.22:0/3982459150 >> 172.21.15.22:0/3982459150 conn(0x7f2e680fbda0 msgr2=0x7f2e680fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:35:46.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:46.048+0000 7f2e6f380700 1 -- 172.21.15.22:0/3982459150 shutdown_connections 2024-02-16T19:35:46.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:46.048+0000 7f2e6f380700 1 -- 172.21.15.22:0/3982459150 wait complete. 2024-02-16T19:35:46.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:46.048+0000 7f2e6f380700 1 Processor -- start 2024-02-16T19:35:46.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:46.049+0000 7f2e6f380700 1 -- start start 2024-02-16T19:35:46.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:46.049+0000 7f2e6f380700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e681004e0 0x7f2e68195f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:35:46.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:46.049+0000 7f2e6f380700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2e68100e80 0x7f2e68196450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:35:46.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:46.049+0000 7f2e6f380700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e68196b30 con 0x7f2e68100e80 2024-02-16T19:35:46.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:46.049+0000 7f2e6f380700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e6819a8c0 con 0x7f2e681004e0 2024-02-16T19:35:46.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:46.049+0000 7f2e6c91b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2e68100e80 0x7f2e68196450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:35:46.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:46.049+0000 7f2e6c91b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2e68100e80 0x7f2e68196450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45728/0 (socket says 172.21.15.22:45728) 2024-02-16T19:35:46.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:46.049+0000 7f2e6c91b700 1 -- 172.21.15.22:0/3405551831 learned_addr learned my addr 172.21.15.22:0/3405551831 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:35:46.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:46.049+0000 7f2e6d11c700 1 --2- 172.21.15.22:0/3405551831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e681004e0 0x7f2e68195f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:35:46.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:46.049+0000 7f2e6c91b700 1 -- 172.21.15.22:0/3405551831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e681004e0 msgr2=0x7f2e68195f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:35:46.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:46.049+0000 7f2e6c91b700 1 --2- 172.21.15.22:0/3405551831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e681004e0 0x7f2e68195f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:46.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:46.049+0000 7f2e6c91b700 1 -- 172.21.15.22:0/3405551831 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e5c0095f0 con 0x7f2e68100e80 2024-02-16T19:35:46.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:46.049+0000 7f2e6c91b700 1 --2- 172.21.15.22:0/3405551831 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2e68100e80 0x7f2e68196450 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c0092e0 tx=0x7f2e5c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:35:46.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:46.050+0000 7f2e667fc700 1 -- 172.21.15.22:0/3405551831 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e5c01d070 con 0x7f2e68100e80 2024-02-16T19:35:46.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:46.050+0000 7f2e667fc700 1 -- 172.21.15.22:0/3405551831 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2e5c005dc0 con 0x7f2e68100e80 2024-02-16T19:35:46.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:46.050+0000 7f2e667fc700 1 -- 172.21.15.22:0/3405551831 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e5c017400 con 0x7f2e68100e80 2024-02-16T19:35:46.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:46.050+0000 7f2e6f380700 1 -- 172.21.15.22:0/3405551831 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e6819ab40 con 0x7f2e68100e80 2024-02-16T19:35:46.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:46.050+0000 7f2e6f380700 1 -- 172.21.15.22:0/3405551831 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e6819afe0 con 0x7f2e68100e80 2024-02-16T19:35:46.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:46.051+0000 7f2e6f380700 1 -- 172.21.15.22:0/3405551831 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e68108850 con 0x7f2e68100e80 2024-02-16T19:35:46.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:46.051+0000 7f2e667fc700 1 -- 172.21.15.22:0/3405551831 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f2e5c00f6f0 con 0x7f2e68100e80 2024-02-16T19:35:46.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:46.052+0000 7f2e667fc700 1 --2- 172.21.15.22:0/3405551831 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f2e5006d8a0 0x7f2e5006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:35:46.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:46.052+0000 7f2e667fc700 1 -- 172.21.15.22:0/3405551831 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f2e5c08e980 con 0x7f2e68100e80 2024-02-16T19:35:46.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:46.052+0000 7f2e6d11c700 1 --2- 172.21.15.22:0/3405551831 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f2e5006d8a0 0x7f2e5006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:35:46.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:46.052+0000 7f2e6d11c700 1 --2- 172.21.15.22:0/3405551831 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f2e5006d8a0 0x7f2e5006fd50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2e60005950 tx=0x7f2e6000b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:35:46.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:46.056+0000 7f2e667fc700 1 -- 172.21.15.22:0/3405551831 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2e5c0580c0 con 0x7f2e68100e80 2024-02-16T19:35:46.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:46.355+0000 7f2e6f380700 1 -- 172.21.15.22:0/3405551831 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f2e68197270 con 0x7f2e68100e80 2024-02-16T19:35:46.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:46.355+0000 7f2e667fc700 1 -- 172.21.15.22:0/3405551831 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f2e5c05b780 con 0x7f2e68100e80 2024-02-16T19:35:46.357 INFO:teuthology.orchestra.run.smithi022.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-02-16T19:35:46.357 INFO:teuthology.orchestra.run.smithi022.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-02-16T19:35:46.357 INFO:teuthology.orchestra.run.smithi022.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-02-16T19:35:46.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:46.357+0000 7f2e6f380700 1 -- 172.21.15.22:0/3405551831 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f2e5006d8a0 msgr2=0x7f2e5006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:35:46.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:46.357+0000 7f2e6f380700 1 --2- 172.21.15.22:0/3405551831 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f2e5006d8a0 0x7f2e5006fd50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2e60005950 tx=0x7f2e6000b410 comp rx=0 tx=0).stop 2024-02-16T19:35:46.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:46.357+0000 7f2e6f380700 1 -- 172.21.15.22:0/3405551831 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2e68100e80 msgr2=0x7f2e68196450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:35:46.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:46.357+0000 7f2e6f380700 1 --2- 172.21.15.22:0/3405551831 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2e68100e80 0x7f2e68196450 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c0092e0 tx=0x7f2e5c00bd70 comp rx=0 tx=0).stop 2024-02-16T19:35:46.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:46.357+0000 7f2e6f380700 1 -- 172.21.15.22:0/3405551831 shutdown_connections 2024-02-16T19:35:46.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:46.357+0000 7f2e6f380700 1 --2- 172.21.15.22:0/3405551831 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f2e5006d8a0 0x7f2e5006fd50 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:46.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:46.357+0000 7f2e6f380700 1 --2- 172.21.15.22:0/3405551831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e681004e0 0x7f2e68195f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:46.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:46.357+0000 7f2e6f380700 1 --2- 172.21.15.22:0/3405551831 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2e68100e80 0x7f2e68196450 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:46.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:46.357+0000 7f2e6f380700 1 -- 172.21.15.22:0/3405551831 >> 172.21.15.22:0/3405551831 conn(0x7f2e680fbda0 msgr2=0x7f2e680fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:35:46.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:46.358+0000 7f2e6f380700 1 -- 172.21.15.22:0/3405551831 shutdown_connections 2024-02-16T19:35:46.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:35:46.358+0000 7f2e6f380700 1 -- 172.21.15.22:0/3405551831 wait complete. 2024-02-16T19:35:46.608 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:46 smithi022 ceph-mon[88798]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-02-16T19:35:46.608 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:46 smithi022 ceph-mon[88798]: Upgrade: osd.2 is safe to restart 2024-02-16T19:35:46.608 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:46 smithi022 ceph-mon[88798]: from='client.? 172.21.15.22:0/251580174' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:35:46.608 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:46 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:35:46.608 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:46 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:35:46.608 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:46 smithi022 ceph-mon[88798]: from='client.? 172.21.15.22:0/3405551831' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T19:35:46.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:46 smithi117 ceph-mon[67870]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-02-16T19:35:46.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:46 smithi117 ceph-mon[67870]: Upgrade: osd.2 is safe to restart 2024-02-16T19:35:46.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:46 smithi117 ceph-mon[67870]: from='client.? 172.21.15.22:0/251580174' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:35:46.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:46 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:35:46.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:46 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:35:46.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:46 smithi117 ceph-mon[67870]: from='client.? 172.21.15.22:0/3405551831' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T19:35:47.871 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:47 smithi022 ceph-mon[88798]: pgmap v97: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s, 33 objects/s recovering 2024-02-16T19:35:47.872 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:47 smithi022 ceph-mon[88798]: from='client.34272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:35:47.872 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:47 smithi022 ceph-mon[88798]: Upgrade: Updating osd.2 2024-02-16T19:35:47.872 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:47 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:35:47.872 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:47 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-16T19:35:47.872 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:47 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:35:47.872 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:47 smithi022 ceph-mon[88798]: Deploying daemon osd.2 on smithi022 2024-02-16T19:35:48.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:47 smithi117 ceph-mon[67870]: pgmap v97: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s, 33 objects/s recovering 2024-02-16T19:35:48.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:47 smithi117 ceph-mon[67870]: from='client.34272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:35:48.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:47 smithi117 ceph-mon[67870]: Upgrade: Updating osd.2 2024-02-16T19:35:48.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:47 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:35:48.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:47 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-16T19:35:48.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:47 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:35:48.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:47 smithi117 ceph-mon[67870]: Deploying daemon osd.2 on smithi022 2024-02-16T19:35:49.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:49 smithi022 ceph-mon[88798]: pgmap v98: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:35:49.965 INFO:journalctl@ceph.osd.2.smithi022.stdout:Feb 16 19:35:49 smithi022 systemd[1]: Stopping Ceph osd.2 for 2a1346e6-cd00-11ee-95ba-87774f69a715... 2024-02-16T19:35:50.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:49 smithi117 ceph-mon[67870]: pgmap v98: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:35:50.465 INFO:journalctl@ceph.osd.2.smithi022.stdout:Feb 16 19:35:50 smithi022 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-2[55140]: 2024-02-16T19:35:50.115+0000 7ff8ed026700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-16T19:35:50.465 INFO:journalctl@ceph.osd.2.smithi022.stdout:Feb 16 19:35:50 smithi022 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-2[55140]: 2024-02-16T19:35:50.115+0000 7ff8ed026700 -1 osd.2 69 *** Got signal Terminated *** 2024-02-16T19:35:50.465 INFO:journalctl@ceph.osd.2.smithi022.stdout:Feb 16 19:35:50 smithi022 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-2[55140]: 2024-02-16T19:35:50.115+0000 7ff8ed026700 -1 osd.2 69 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-16T19:35:50.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:50 smithi022 ceph-mon[88798]: osd.2 marked itself down and dead 2024-02-16T19:35:51.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:50 smithi117 ceph-mon[67870]: osd.2 marked itself down and dead 2024-02-16T19:35:51.465 INFO:journalctl@ceph.osd.2.smithi022.stdout:Feb 16 19:35:51 smithi022 podman[111526]: 2024-02-16 19:35:51.209417846 +0000 UTC m=+1.202035725 container died be8a631b4f93738b08a102458672fd8563099eaaaffcf7ccec1c314f32cc8db2 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, RELEASE=HEAD, ceph=True, org.label-schema.build-date=20231212, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.29.1, GIT_CLEAN=True, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=-18.2.0) 2024-02-16T19:35:51.851 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:51 smithi022 ceph-mon[88798]: pgmap v99: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:35:51.851 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:51 smithi022 ceph-mon[88798]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-16T19:35:51.851 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:51 smithi022 ceph-mon[88798]: osdmap e70: 6 total, 5 up, 6 in 2024-02-16T19:35:51.851 INFO:journalctl@ceph.osd.2.smithi022.stdout:Feb 16 19:35:51 smithi022 podman[111526]: 2024-02-16 19:35:51.615873745 +0000 UTC m=+1.608491617 container cleanup be8a631b4f93738b08a102458672fd8563099eaaaffcf7ccec1c314f32cc8db2 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, GIT_CLEAN=True, io.buildah.version=1.29.1, org.label-schema.schema-version=1.0, RELEASE=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.build-date=20231212, maintainer=Guillaume Abrioux , GIT_REPO=https://github.com/ceph/ceph-container.git) 2024-02-16T19:35:51.851 INFO:journalctl@ceph.osd.2.smithi022.stdout:Feb 16 19:35:51 smithi022 bash[111526]: ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-2 2024-02-16T19:35:52.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:51 smithi117 ceph-mon[67870]: pgmap v99: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:35:52.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:51 smithi117 ceph-mon[67870]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-16T19:35:52.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:51 smithi117 ceph-mon[67870]: osdmap e70: 6 total, 5 up, 6 in 2024-02-16T19:35:52.162 INFO:journalctl@ceph.osd.2.smithi022.stdout:Feb 16 19:35:51 smithi022 podman[111547]: 2024-02-16 19:35:51.85074642 +0000 UTC m=+0.636211066 container remove be8a631b4f93738b08a102458672fd8563099eaaaffcf7ccec1c314f32cc8db2 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-2, io.buildah.version=1.29.1, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20231212, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, GIT_REPO=https://github.com/ceph/ceph-container.git, RELEASE=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-16T19:35:52.463 INFO:journalctl@ceph.osd.2.smithi022.stdout:Feb 16 19:35:52 smithi022 podman[111641]: 2024-02-16 19:35:52.065899042 +0000 UTC m=+0.018108743 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T19:35:52.463 INFO:journalctl@ceph.osd.2.smithi022.stdout:Feb 16 19:35:52 smithi022 podman[111641]: 2024-02-16 19:35:52.185165117 +0000 UTC m=+0.137374817 container create 5c36e7c2ca10d62de32c77ea41d4716a571982a11798619622860f8746844143 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-2-deactivate, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-02-16T19:35:52.463 INFO:journalctl@ceph.osd.2.smithi022.stdout:Feb 16 19:35:52 smithi022 podman[111641]: 2024-02-16 19:35:52.379276057 +0000 UTC m=+0.331485760 container init 5c36e7c2ca10d62de32c77ea41d4716a571982a11798619622860f8746844143 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-2-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True) 2024-02-16T19:35:52.463 INFO:journalctl@ceph.osd.2.smithi022.stdout:Feb 16 19:35:52 smithi022 podman[111641]: 2024-02-16 19:35:52.382440401 +0000 UTC m=+0.334650114 container start 5c36e7c2ca10d62de32c77ea41d4716a571982a11798619622860f8746844143 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-2-deactivate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, ceph=True, org.label-schema.schema-version=1.0) 2024-02-16T19:35:52.463 INFO:journalctl@ceph.osd.2.smithi022.stdout:Feb 16 19:35:52 smithi022 podman[111641]: 2024-02-16 19:35:52.462533895 +0000 UTC m=+0.414743596 container attach 5c36e7c2ca10d62de32c77ea41d4716a571982a11798619622860f8746844143 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-2-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-02-16T19:35:52.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:52 smithi022 ceph-mon[88798]: osdmap e71: 6 total, 5 up, 6 in 2024-02-16T19:35:52.715 INFO:journalctl@ceph.osd.2.smithi022.stdout:Feb 16 19:35:52 smithi022 podman[111641]: 2024-02-16 19:35:52.703478821 +0000 UTC m=+0.655688516 container died 5c36e7c2ca10d62de32c77ea41d4716a571982a11798619622860f8746844143 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-2-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-02-16T19:35:53.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:52 smithi117 ceph-mon[67870]: osdmap e71: 6 total, 5 up, 6 in 2024-02-16T19:35:53.660 INFO:journalctl@ceph.osd.2.smithi022.stdout:Feb 16 19:35:53 smithi022 podman[111668]: 2024-02-16 19:35:53.375178731 +0000 UTC m=+0.666251915 container remove 5c36e7c2ca10d62de32c77ea41d4716a571982a11798619622860f8746844143 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-2-deactivate, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, GIT_CLEAN=True) 2024-02-16T19:35:53.661 INFO:journalctl@ceph.osd.2.smithi022.stdout:Feb 16 19:35:53 smithi022 systemd[1]: ceph-2a1346e6-cd00-11ee-95ba-87774f69a715@osd.2.service: Deactivated successfully. 2024-02-16T19:35:53.661 INFO:journalctl@ceph.osd.2.smithi022.stdout:Feb 16 19:35:53 smithi022 systemd[1]: Stopped Ceph osd.2 for 2a1346e6-cd00-11ee-95ba-87774f69a715. 2024-02-16T19:35:53.661 INFO:journalctl@ceph.osd.2.smithi022.stdout:Feb 16 19:35:53 smithi022 systemd[1]: ceph-2a1346e6-cd00-11ee-95ba-87774f69a715@osd.2.service: Consumed 48.531s CPU time. 2024-02-16T19:35:53.661 INFO:journalctl@ceph.osd.2.smithi022.stdout:Feb 16 19:35:53 smithi022 systemd[1]: Starting Ceph osd.2 for 2a1346e6-cd00-11ee-95ba-87774f69a715... 2024-02-16T19:35:53.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:53 smithi117 ceph-mon[67870]: pgmap v102: 65 pgs: 7 active+undersized, 5 active+undersized+degraded, 8 stale+active+clean, 45 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 10/267 objects degraded (3.745%) 2024-02-16T19:35:53.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:53 smithi117 ceph-mon[67870]: Health check failed: Degraded data redundancy: 10/267 objects degraded (3.745%), 5 pgs degraded (PG_DEGRADED) 2024-02-16T19:35:53.918 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:53 smithi022 ceph-mon[88798]: pgmap v102: 65 pgs: 7 active+undersized, 5 active+undersized+degraded, 8 stale+active+clean, 45 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 10/267 objects degraded (3.745%) 2024-02-16T19:35:53.918 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:53 smithi022 ceph-mon[88798]: Health check failed: Degraded data redundancy: 10/267 objects degraded (3.745%), 5 pgs degraded (PG_DEGRADED) 2024-02-16T19:35:53.918 INFO:journalctl@ceph.osd.2.smithi022.stdout:Feb 16 19:35:53 smithi022 podman[111767]: 2024-02-16 19:35:53.809333636 +0000 UTC m=+0.087551470 container create c2c4fdd6a1a5308066cfa60111111078eaab9f1c8940a571543b7ff6986db443 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-2-activate, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-16T19:35:53.918 INFO:journalctl@ceph.osd.2.smithi022.stdout:Feb 16 19:35:53 smithi022 podman[111767]: 2024-02-16 19:35:53.739776356 +0000 UTC m=+0.017994188 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T19:35:54.215 INFO:journalctl@ceph.osd.2.smithi022.stdout:Feb 16 19:35:54 smithi022 podman[111767]: 2024-02-16 19:35:54.028850501 +0000 UTC m=+0.307068333 container init c2c4fdd6a1a5308066cfa60111111078eaab9f1c8940a571543b7ff6986db443 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-2-activate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-16T19:35:54.215 INFO:journalctl@ceph.osd.2.smithi022.stdout:Feb 16 19:35:54 smithi022 podman[111767]: 2024-02-16 19:35:54.032347323 +0000 UTC m=+0.310565164 container start c2c4fdd6a1a5308066cfa60111111078eaab9f1c8940a571543b7ff6986db443 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-2-activate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-02-16T19:35:54.215 INFO:journalctl@ceph.osd.2.smithi022.stdout:Feb 16 19:35:54 smithi022 podman[111767]: 2024-02-16 19:35:54.112107245 +0000 UTC m=+0.390325084 container attach c2c4fdd6a1a5308066cfa60111111078eaab9f1c8940a571543b7ff6986db443 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-2-activate, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5) 2024-02-16T19:35:54.817 INFO:journalctl@ceph.osd.2.smithi022.stdout:Feb 16 19:35:54 smithi022 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-2-activate[111782]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-02-16T19:35:54.817 INFO:journalctl@ceph.osd.2.smithi022.stdout:Feb 16 19:35:54 smithi022 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-2-activate[111782]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-02-16T19:35:54.818 INFO:journalctl@ceph.osd.2.smithi022.stdout:Feb 16 19:35:54 smithi022 bash[111767]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-02-16T19:35:54.818 INFO:journalctl@ceph.osd.2.smithi022.stdout:Feb 16 19:35:54 smithi022 bash[111767]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-02-16T19:35:54.818 INFO:journalctl@ceph.osd.2.smithi022.stdout:Feb 16 19:35:54 smithi022 bash[111767]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-02-16T19:35:54.818 INFO:journalctl@ceph.osd.2.smithi022.stdout:Feb 16 19:35:54 smithi022 bash[111767]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-02-16T19:35:54.818 INFO:journalctl@ceph.osd.2.smithi022.stdout:Feb 16 19:35:54 smithi022 bash[111767]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-02-16T19:35:54.818 INFO:journalctl@ceph.osd.2.smithi022.stdout:Feb 16 19:35:54 smithi022 bash[111767]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-02-16T19:35:54.818 INFO:journalctl@ceph.osd.2.smithi022.stdout:Feb 16 19:35:54 smithi022 bash[111767]: --> ceph-volume raw activate successful for osd ID: 2 2024-02-16T19:35:54.818 INFO:journalctl@ceph.osd.2.smithi022.stdout:Feb 16 19:35:54 smithi022 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-2-activate[111782]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-02-16T19:35:54.818 INFO:journalctl@ceph.osd.2.smithi022.stdout:Feb 16 19:35:54 smithi022 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-2-activate[111782]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-02-16T19:35:54.818 INFO:journalctl@ceph.osd.2.smithi022.stdout:Feb 16 19:35:54 smithi022 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-2-activate[111782]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-02-16T19:35:54.818 INFO:journalctl@ceph.osd.2.smithi022.stdout:Feb 16 19:35:54 smithi022 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-2-activate[111782]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-02-16T19:35:54.818 INFO:journalctl@ceph.osd.2.smithi022.stdout:Feb 16 19:35:54 smithi022 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-2-activate[111782]: --> ceph-volume raw activate successful for osd ID: 2 2024-02-16T19:35:54.818 INFO:journalctl@ceph.osd.2.smithi022.stdout:Feb 16 19:35:54 smithi022 podman[111767]: 2024-02-16 19:35:54.508368393 +0000 UTC m=+0.786586225 container died c2c4fdd6a1a5308066cfa60111111078eaab9f1c8940a571543b7ff6986db443 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-2-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-16T19:35:55.577 INFO:journalctl@ceph.osd.2.smithi022.stdout:Feb 16 19:35:55 smithi022 podman[111922]: 2024-02-16 19:35:55.259902618 +0000 UTC m=+0.746592134 container remove c2c4fdd6a1a5308066cfa60111111078eaab9f1c8940a571543b7ff6986db443 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-2-activate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=) 2024-02-16T19:35:55.577 INFO:journalctl@ceph.osd.2.smithi022.stdout:Feb 16 19:35:55 smithi022 podman[111987]: 2024-02-16 19:35:55.469166181 +0000 UTC m=+0.103228088 container create 46138c50f79d2f56a71cf356b2d6cc7c6ffa725baea25f001c38b6e86ee8cde7 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2) 2024-02-16T19:35:55.577 INFO:journalctl@ceph.osd.2.smithi022.stdout:Feb 16 19:35:55 smithi022 podman[111987]: 2024-02-16 19:35:55.384338041 +0000 UTC m=+0.018399907 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T19:35:55.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:55 smithi022 ceph-mon[88798]: pgmap v103: 65 pgs: 7 active+undersized, 5 active+undersized+degraded, 8 stale+active+clean, 45 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 10/267 objects degraded (3.745%) 2024-02-16T19:35:55.965 INFO:journalctl@ceph.osd.2.smithi022.stdout:Feb 16 19:35:55 smithi022 podman[111987]: 2024-02-16 19:35:55.679875893 +0000 UTC m=+0.313937755 container init 46138c50f79d2f56a71cf356b2d6cc7c6ffa725baea25f001c38b6e86ee8cde7 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-2, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-16T19:35:55.965 INFO:journalctl@ceph.osd.2.smithi022.stdout:Feb 16 19:35:55 smithi022 podman[111987]: 2024-02-16 19:35:55.682902879 +0000 UTC m=+0.316964741 container start 46138c50f79d2f56a71cf356b2d6cc7c6ffa725baea25f001c38b6e86ee8cde7 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True) 2024-02-16T19:35:55.965 INFO:journalctl@ceph.osd.2.smithi022.stdout:Feb 16 19:35:55 smithi022 bash[111987]: 46138c50f79d2f56a71cf356b2d6cc7c6ffa725baea25f001c38b6e86ee8cde7 2024-02-16T19:35:55.965 INFO:journalctl@ceph.osd.2.smithi022.stdout:Feb 16 19:35:55 smithi022 systemd[1]: Started Ceph osd.2 for 2a1346e6-cd00-11ee-95ba-87774f69a715. 2024-02-16T19:35:56.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:55 smithi117 ceph-mon[67870]: pgmap v103: 65 pgs: 7 active+undersized, 5 active+undersized+degraded, 8 stale+active+clean, 45 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 10/267 objects degraded (3.745%) 2024-02-16T19:35:56.685 INFO:journalctl@ceph.osd.2.smithi022.stdout:Feb 16 19:35:56 smithi022 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-2[112002]: 2024-02-16T19:35:56.237+0000 7fd3abf0d7c0 -1 Falling back to public interface 2024-02-16T19:35:56.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:56 smithi022 ceph-mon[88798]: pgmap v104: 65 pgs: 15 active+undersized, 15 active+undersized+degraded, 35 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 38/267 objects degraded (14.232%) 2024-02-16T19:35:56.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:56 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:35:56.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:56 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:35:56.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:56 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:35:57.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:56 smithi117 ceph-mon[67870]: pgmap v104: 65 pgs: 15 active+undersized, 15 active+undersized+degraded, 35 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 38/267 objects degraded (14.232%) 2024-02-16T19:35:57.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:56 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:35:57.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:56 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:35:57.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:56 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:35:59.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:35:58 smithi117 ceph-mon[67870]: pgmap v105: 65 pgs: 15 active+undersized, 15 active+undersized+degraded, 35 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 38/267 objects degraded (14.232%) 2024-02-16T19:35:59.152 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:35:58 smithi022 ceph-mon[88798]: pgmap v105: 65 pgs: 15 active+undersized, 15 active+undersized+degraded, 35 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 38/267 objects degraded (14.232%) 2024-02-16T19:36:00.439 INFO:journalctl@ceph.osd.2.smithi022.stdout:Feb 16 19:36:00 smithi022 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-2[112002]: 2024-02-16T19:36:00.125+0000 7fd3abf0d7c0 -1 osd.2 0 read_superblock omap replica is missing. 2024-02-16T19:36:00.439 INFO:journalctl@ceph.osd.2.smithi022.stdout:Feb 16 19:36:00 smithi022 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-2[112002]: 2024-02-16T19:36:00.436+0000 7fd3abf0d7c0 -1 osd.2 69 log_to_monitors true 2024-02-16T19:36:00.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:00 smithi022 ceph-mon[88798]: pgmap v106: 65 pgs: 15 active+undersized, 15 active+undersized+degraded, 35 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 38/267 objects degraded (14.232%) 2024-02-16T19:36:00.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:00 smithi022 ceph-mon[88798]: from='osd.2 [v2:172.21.15.22:6818/2098766749,v1:172.21.15.22:6819/2098766749]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-16T19:36:01.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:00 smithi117 ceph-mon[67870]: pgmap v106: 65 pgs: 15 active+undersized, 15 active+undersized+degraded, 35 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 38/267 objects degraded (14.232%) 2024-02-16T19:36:01.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:00 smithi117 ceph-mon[67870]: from='osd.2 [v2:172.21.15.22:6818/2098766749,v1:172.21.15.22:6819/2098766749]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-16T19:36:01.836 INFO:journalctl@ceph.osd.2.smithi022.stdout:Feb 16 19:36:01 smithi022 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-2[112002]: 2024-02-16T19:36:01.574+0000 7fd39935d700 -1 osd.2 69 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-16T19:36:02.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:01 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:36:02.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:01 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:36:02.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:01 smithi117 ceph-mon[67870]: from='osd.2 [v2:172.21.15.22:6818/2098766749,v1:172.21.15.22:6819/2098766749]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-16T19:36:02.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:01 smithi117 ceph-mon[67870]: osdmap e72: 6 total, 5 up, 6 in 2024-02-16T19:36:02.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:01 smithi117 ceph-mon[67870]: from='osd.2 [v2:172.21.15.22:6818/2098766749,v1:172.21.15.22:6819/2098766749]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-02-16T19:36:02.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:01 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:36:02.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:01 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:36:02.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:01 smithi117 ceph-mon[67870]: from='osd.2 [v2:172.21.15.22:6818/2098766749,v1:172.21.15.22:6819/2098766749]' entity='osd.2' 2024-02-16T19:36:02.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:01 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:36:02.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:01 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:36:02.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:01 smithi022 ceph-mon[88798]: from='osd.2 [v2:172.21.15.22:6818/2098766749,v1:172.21.15.22:6819/2098766749]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-16T19:36:02.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:01 smithi022 ceph-mon[88798]: osdmap e72: 6 total, 5 up, 6 in 2024-02-16T19:36:02.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:01 smithi022 ceph-mon[88798]: from='osd.2 [v2:172.21.15.22:6818/2098766749,v1:172.21.15.22:6819/2098766749]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-02-16T19:36:02.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:01 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:36:02.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:01 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:36:02.216 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:01 smithi022 ceph-mon[88798]: from='osd.2 [v2:172.21.15.22:6818/2098766749,v1:172.21.15.22:6819/2098766749]' entity='osd.2' 2024-02-16T19:36:03.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:02 smithi117 ceph-mon[67870]: pgmap v108: 65 pgs: 15 active+undersized, 15 active+undersized+degraded, 35 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 38/267 objects degraded (14.232%) 2024-02-16T19:36:03.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:02 smithi117 ceph-mon[67870]: Health check update: Degraded data redundancy: 38/267 objects degraded (14.232%), 15 pgs degraded (PG_DEGRADED) 2024-02-16T19:36:03.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:02 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:36:03.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:02 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:36:03.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:02 smithi117 ceph-mon[67870]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-16T19:36:03.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:02 smithi117 ceph-mon[67870]: osd.2 [v2:172.21.15.22:6818/2098766749,v1:172.21.15.22:6819/2098766749] boot 2024-02-16T19:36:03.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:02 smithi117 ceph-mon[67870]: osdmap e73: 6 total, 6 up, 6 in 2024-02-16T19:36:03.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:02 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-16T19:36:03.153 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:02 smithi022 ceph-mon[88798]: pgmap v108: 65 pgs: 15 active+undersized, 15 active+undersized+degraded, 35 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 38/267 objects degraded (14.232%) 2024-02-16T19:36:03.153 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:02 smithi022 ceph-mon[88798]: Health check update: Degraded data redundancy: 38/267 objects degraded (14.232%), 15 pgs degraded (PG_DEGRADED) 2024-02-16T19:36:03.153 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:02 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:36:03.153 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:02 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:36:03.153 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:02 smithi022 ceph-mon[88798]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-16T19:36:03.153 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:02 smithi022 ceph-mon[88798]: osd.2 [v2:172.21.15.22:6818/2098766749,v1:172.21.15.22:6819/2098766749] boot 2024-02-16T19:36:03.153 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:02 smithi022 ceph-mon[88798]: osdmap e73: 6 total, 6 up, 6 in 2024-02-16T19:36:03.153 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:02 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-16T19:36:04.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:04 smithi117 ceph-mon[67870]: osdmap e74: 6 total, 6 up, 6 in 2024-02-16T19:36:04.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:04 smithi022 ceph-mon[88798]: osdmap e74: 6 total, 6 up, 6 in 2024-02-16T19:36:05.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:05 smithi117 ceph-mon[67870]: pgmap v111: 65 pgs: 15 active+undersized, 15 active+undersized+degraded, 35 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 38/267 objects degraded (14.232%) 2024-02-16T19:36:05.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:05 smithi022 ceph-mon[88798]: pgmap v111: 65 pgs: 15 active+undersized, 15 active+undersized+degraded, 35 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 38/267 objects degraded (14.232%) 2024-02-16T19:36:07.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:07 smithi117 ceph-mon[67870]: pgmap v112: 65 pgs: 10 active+undersized, 10 active+undersized+degraded, 45 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 27/267 objects degraded (10.112%) 2024-02-16T19:36:07.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:07 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:36:07.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:07 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:36:07.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:07 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:36:07.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:07 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:36:07.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:07 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:36:07.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:07 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:36:07.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:07 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:36:07.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:07 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:36:07.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:07 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:36:07.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:07 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-02-16T19:36:07.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:07 smithi117 ceph-mon[67870]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-02-16T19:36:07.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:07 smithi117 ceph-mon[67870]: Upgrade: unsafe to stop osd(s) at this time (8 PGs are or would become offline) 2024-02-16T19:36:07.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:07 smithi022 ceph-mon[88798]: pgmap v112: 65 pgs: 10 active+undersized, 10 active+undersized+degraded, 45 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 27/267 objects degraded (10.112%) 2024-02-16T19:36:07.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:07 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:36:07.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:07 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:36:07.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:07 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:36:07.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:07 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:36:07.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:07 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:36:07.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:07 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:36:07.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:07 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:36:07.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:07 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:36:07.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:07 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:36:07.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:07 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-02-16T19:36:07.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:07 smithi022 ceph-mon[88798]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-02-16T19:36:07.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:07 smithi022 ceph-mon[88798]: Upgrade: unsafe to stop osd(s) at this time (8 PGs are or would become offline) 2024-02-16T19:36:08.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:08 smithi117 ceph-mon[67870]: Health check update: Degraded data redundancy: 27/267 objects degraded (10.112%), 10 pgs degraded (PG_DEGRADED) 2024-02-16T19:36:08.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:08 smithi022 ceph-mon[88798]: Health check update: Degraded data redundancy: 27/267 objects degraded (10.112%), 10 pgs degraded (PG_DEGRADED) 2024-02-16T19:36:09.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:09 smithi117 ceph-mon[67870]: pgmap v113: 65 pgs: 65 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 589 KiB/s, 0 objects/s recovering 2024-02-16T19:36:09.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:09 smithi117 ceph-mon[67870]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 27/267 objects degraded (10.112%), 10 pgs degraded) 2024-02-16T19:36:09.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:09 smithi022 ceph-mon[88798]: pgmap v113: 65 pgs: 65 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 589 KiB/s, 0 objects/s recovering 2024-02-16T19:36:09.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:09 smithi022 ceph-mon[88798]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 27/267 objects degraded (10.112%), 10 pgs degraded) 2024-02-16T19:36:11.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:11 smithi117 ceph-mon[67870]: pgmap v114: 65 pgs: 65 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 512 KiB/s, 0 objects/s recovering 2024-02-16T19:36:11.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:11 smithi022 ceph-mon[88798]: pgmap v114: 65 pgs: 65 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 512 KiB/s, 0 objects/s recovering 2024-02-16T19:36:13.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:13 smithi117 ceph-mon[67870]: pgmap v115: 65 pgs: 65 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 444 KiB/s, 0 objects/s recovering 2024-02-16T19:36:13.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:13 smithi022 ceph-mon[88798]: pgmap v115: 65 pgs: 65 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 444 KiB/s, 0 objects/s recovering 2024-02-16T19:36:15.370 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:15 smithi022 ceph-mon[88798]: pgmap v116: 65 pgs: 65 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 400 KiB/s, 0 objects/s recovering 2024-02-16T19:36:15.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:15 smithi117 ceph-mon[67870]: pgmap v116: 65 pgs: 65 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 400 KiB/s, 0 objects/s recovering 2024-02-16T19:36:16.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.481+0000 7f8185bdf700 1 -- 172.21.15.22:0/1172352082 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81801004e0 msgr2=0x7f81801008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:16.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.481+0000 7f8185bdf700 1 --2- 172.21.15.22:0/1172352082 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81801004e0 0x7f81801008b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8170009910 tx=0x7f8170009c20 comp rx=0 tx=0).stop 2024-02-16T19:36:16.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.481+0000 7f8185bdf700 1 -- 172.21.15.22:0/1172352082 shutdown_connections 2024-02-16T19:36:16.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.481+0000 7f8185bdf700 1 --2- 172.21.15.22:0/1172352082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8180100e80 0x7f8180104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:16.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.481+0000 7f8185bdf700 1 --2- 172.21.15.22:0/1172352082 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81801004e0 0x7f81801008b0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:16.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.481+0000 7f8185bdf700 1 -- 172.21.15.22:0/1172352082 >> 172.21.15.22:0/1172352082 conn(0x7f81800fbda0 msgr2=0x7f81800fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:16.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.481+0000 7f8185bdf700 1 -- 172.21.15.22:0/1172352082 shutdown_connections 2024-02-16T19:36:16.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.481+0000 7f8185bdf700 1 -- 172.21.15.22:0/1172352082 wait complete. 2024-02-16T19:36:16.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.482+0000 7f8185bdf700 1 Processor -- start 2024-02-16T19:36:16.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.482+0000 7f8185bdf700 1 -- start start 2024-02-16T19:36:16.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.482+0000 7f8185bdf700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81801004e0 0x7f8180073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:16.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.482+0000 7f8185bdf700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8180100e80 0x7f8180073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:16.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.482+0000 7f8185bdf700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8180073ad0 con 0x7f8180100e80 2024-02-16T19:36:16.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.482+0000 7f8185bdf700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8180073c10 con 0x7f81801004e0 2024-02-16T19:36:16.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.482+0000 7f817effd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8180100e80 0x7f8180073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:16.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.482+0000 7f817effd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8180100e80 0x7f8180073590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42512/0 (socket says 172.21.15.22:42512) 2024-02-16T19:36:16.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.482+0000 7f817effd700 1 -- 172.21.15.22:0/725223902 learned_addr learned my addr 172.21.15.22:0/725223902 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:36:16.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.482+0000 7f817f7fe700 1 --2- 172.21.15.22:0/725223902 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81801004e0 0x7f8180073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:16.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.483+0000 7f817effd700 1 -- 172.21.15.22:0/725223902 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81801004e0 msgr2=0x7f8180073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:16.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.483+0000 7f817effd700 1 --2- 172.21.15.22:0/725223902 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81801004e0 0x7f8180073050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:16.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.483+0000 7f817effd700 1 -- 172.21.15.22:0/725223902 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81700095f0 con 0x7f8180100e80 2024-02-16T19:36:16.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.483+0000 7f817effd700 1 --2- 172.21.15.22:0/725223902 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8180100e80 0x7f8180073590 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f817400c8f0 tx=0x7f817400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:16.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.483+0000 7f817cff9700 1 -- 172.21.15.22:0/725223902 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f81740078a0 con 0x7f8180100e80 2024-02-16T19:36:16.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.483+0000 7f817cff9700 1 -- 172.21.15.22:0/725223902 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f817400f450 con 0x7f8180100e80 2024-02-16T19:36:16.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.483+0000 7f817cff9700 1 -- 172.21.15.22:0/725223902 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8174018610 con 0x7f8180100e80 2024-02-16T19:36:16.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.483+0000 7f8185bdf700 1 -- 172.21.15.22:0/725223902 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81801a20c0 con 0x7f8180100e80 2024-02-16T19:36:16.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.483+0000 7f8185bdf700 1 -- 172.21.15.22:0/725223902 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81801a24d0 con 0x7f8180100e80 2024-02-16T19:36:16.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.484+0000 7f8185bdf700 1 -- 172.21.15.22:0/725223902 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81801087f0 con 0x7f8180100e80 2024-02-16T19:36:16.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.485+0000 7f817cff9700 1 -- 172.21.15.22:0/725223902 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f817400f5c0 con 0x7f8180100e80 2024-02-16T19:36:16.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.485+0000 7f817cff9700 1 --2- 172.21.15.22:0/725223902 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f816806d8a0 0x7f816806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:16.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.485+0000 7f817cff9700 1 -- 172.21.15.22:0/725223902 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f817408d470 con 0x7f8180100e80 2024-02-16T19:36:16.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.485+0000 7f817f7fe700 1 --2- 172.21.15.22:0/725223902 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f816806d8a0 0x7f816806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:16.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.486+0000 7f817f7fe700 1 --2- 172.21.15.22:0/725223902 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f816806d8a0 0x7f816806fd50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8170006010 tx=0x7f817001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:16.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.490+0000 7f817cff9700 1 -- 172.21.15.22:0/725223902 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8174056c30 con 0x7f8180100e80 2024-02-16T19:36:16.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.732+0000 7f8185bdf700 1 -- 172.21.15.22:0/725223902 --> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8180074660 con 0x7f816806d8a0 2024-02-16T19:36:16.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.733+0000 7f817cff9700 1 -- 172.21.15.22:0/725223902 <== mgr.34104 v2:172.21.15.22:6800/505266753 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f8180074660 con 0x7f816806d8a0 2024-02-16T19:36:16.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.735+0000 7f8185bdf700 1 -- 172.21.15.22:0/725223902 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f816806d8a0 msgr2=0x7f816806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:16.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.735+0000 7f8185bdf700 1 --2- 172.21.15.22:0/725223902 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f816806d8a0 0x7f816806fd50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8170006010 tx=0x7f817001a040 comp rx=0 tx=0).stop 2024-02-16T19:36:16.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.735+0000 7f8185bdf700 1 -- 172.21.15.22:0/725223902 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8180100e80 msgr2=0x7f8180073590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:16.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.735+0000 7f8185bdf700 1 --2- 172.21.15.22:0/725223902 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8180100e80 0x7f8180073590 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f817400c8f0 tx=0x7f817400cc00 comp rx=0 tx=0).stop 2024-02-16T19:36:16.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.736+0000 7f8185bdf700 1 -- 172.21.15.22:0/725223902 shutdown_connections 2024-02-16T19:36:16.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.736+0000 7f8185bdf700 1 --2- 172.21.15.22:0/725223902 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f816806d8a0 0x7f816806fd50 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:16.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.736+0000 7f8185bdf700 1 --2- 172.21.15.22:0/725223902 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81801004e0 0x7f8180073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:16.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.736+0000 7f8185bdf700 1 --2- 172.21.15.22:0/725223902 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8180100e80 0x7f8180073590 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:16.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.736+0000 7f8185bdf700 1 -- 172.21.15.22:0/725223902 >> 172.21.15.22:0/725223902 conn(0x7f81800fbda0 msgr2=0x7f81800fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:16.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.736+0000 7f8185bdf700 1 -- 172.21.15.22:0/725223902 shutdown_connections 2024-02-16T19:36:16.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.736+0000 7f8185bdf700 1 -- 172.21.15.22:0/725223902 wait complete. 2024-02-16T19:36:16.747 INFO:teuthology.orchestra.run.smithi022.stdout:true 2024-02-16T19:36:16.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.844+0000 7f9480f41700 1 -- 172.21.15.22:0/3212241476 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f947c1004e0 msgr2=0x7f947c1008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:16.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.844+0000 7f9480f41700 1 --2- 172.21.15.22:0/3212241476 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f947c1004e0 0x7f947c1008b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f946c009910 tx=0x7f946c009c20 comp rx=0 tx=0).stop 2024-02-16T19:36:16.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.844+0000 7f9480f41700 1 -- 172.21.15.22:0/3212241476 shutdown_connections 2024-02-16T19:36:16.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.844+0000 7f9480f41700 1 --2- 172.21.15.22:0/3212241476 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f947c100e80 0x7f947c104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:16.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.844+0000 7f9480f41700 1 --2- 172.21.15.22:0/3212241476 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f947c1004e0 0x7f947c1008b0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:16.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.844+0000 7f9480f41700 1 -- 172.21.15.22:0/3212241476 >> 172.21.15.22:0/3212241476 conn(0x7f947c0fbda0 msgr2=0x7f947c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:16.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.844+0000 7f9480f41700 1 -- 172.21.15.22:0/3212241476 shutdown_connections 2024-02-16T19:36:16.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.844+0000 7f9480f41700 1 -- 172.21.15.22:0/3212241476 wait complete. 2024-02-16T19:36:16.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.845+0000 7f9480f41700 1 Processor -- start 2024-02-16T19:36:16.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.845+0000 7f9480f41700 1 -- start start 2024-02-16T19:36:16.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.845+0000 7f9480f41700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f947c1004e0 0x7f947c195ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:16.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.845+0000 7f9480f41700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f947c100e80 0x7f947c196430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:16.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.845+0000 7f9480f41700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f947c196b10 con 0x7f947c100e80 2024-02-16T19:36:16.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.845+0000 7f9480f41700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f947c19a8a0 con 0x7f947c1004e0 2024-02-16T19:36:16.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.845+0000 7f9479d9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f947c100e80 0x7f947c196430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:16.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.845+0000 7f9479d9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f947c100e80 0x7f947c196430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42532/0 (socket says 172.21.15.22:42532) 2024-02-16T19:36:16.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.845+0000 7f9479d9b700 1 -- 172.21.15.22:0/4096533722 learned_addr learned my addr 172.21.15.22:0/4096533722 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:36:16.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.845+0000 7f947a59c700 1 --2- 172.21.15.22:0/4096533722 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f947c1004e0 0x7f947c195ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:16.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.846+0000 7f9479d9b700 1 -- 172.21.15.22:0/4096533722 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f947c1004e0 msgr2=0x7f947c195ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:16.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.846+0000 7f9479d9b700 1 --2- 172.21.15.22:0/4096533722 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f947c1004e0 0x7f947c195ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:16.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.846+0000 7f9479d9b700 1 -- 172.21.15.22:0/4096533722 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f946c0095f0 con 0x7f947c100e80 2024-02-16T19:36:16.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.846+0000 7f947a59c700 1 --2- 172.21.15.22:0/4096533722 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f947c1004e0 0x7f947c195ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T19:36:16.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.846+0000 7f9479d9b700 1 --2- 172.21.15.22:0/4096533722 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f947c100e80 0x7f947c196430 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f947000e8f0 tx=0x7f947000ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:16.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.846+0000 7f946b7fe700 1 -- 172.21.15.22:0/4096533722 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9470009790 con 0x7f947c100e80 2024-02-16T19:36:16.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.846+0000 7f946b7fe700 1 -- 172.21.15.22:0/4096533722 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f947000c810 con 0x7f947c100e80 2024-02-16T19:36:16.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.846+0000 7f9480f41700 1 -- 172.21.15.22:0/4096533722 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f947c19ab80 con 0x7f947c100e80 2024-02-16T19:36:16.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.846+0000 7f946b7fe700 1 -- 172.21.15.22:0/4096533722 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9470005580 con 0x7f947c100e80 2024-02-16T19:36:16.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.846+0000 7f9480f41700 1 -- 172.21.15.22:0/4096533722 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f947c19b0d0 con 0x7f947c100e80 2024-02-16T19:36:16.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.847+0000 7f9480f41700 1 -- 172.21.15.22:0/4096533722 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f947c1087f0 con 0x7f947c100e80 2024-02-16T19:36:16.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.848+0000 7f946b7fe700 1 -- 172.21.15.22:0/4096533722 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f947000c980 con 0x7f947c100e80 2024-02-16T19:36:16.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.848+0000 7f946b7fe700 1 --2- 172.21.15.22:0/4096533722 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f946406d8a0 0x7f946406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:16.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.848+0000 7f947a59c700 1 --2- 172.21.15.22:0/4096533722 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f946406d8a0 0x7f946406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:16.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.848+0000 7f946b7fe700 1 -- 172.21.15.22:0/4096533722 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f9470014070 con 0x7f947c100e80 2024-02-16T19:36:16.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.849+0000 7f947a59c700 1 --2- 172.21.15.22:0/4096533722 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f946406d8a0 0x7f946406fd50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f946c00b760 tx=0x7f946c005940 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:16.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:16.853+0000 7f946b7fe700 1 -- 172.21.15.22:0/4096533722 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9470056f90 con 0x7f947c100e80 2024-02-16T19:36:17.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.080+0000 7f9480f41700 1 -- 172.21.15.22:0/4096533722 --> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f947c19b3b0 con 0x7f946406d8a0 2024-02-16T19:36:17.081 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:16 smithi022 ceph-mon[88798]: pgmap v117: 65 pgs: 65 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 341 KiB/s, 0 objects/s recovering 2024-02-16T19:36:17.082 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:16 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:36:17.082 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:16 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:36:17.082 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:16 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:36:17.082 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:16 smithi022 ceph-mon[88798]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:36:17.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.082+0000 7f946b7fe700 1 -- 172.21.15.22:0/4096533722 <== mgr.34104 v2:172.21.15.22:6800/505266753 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f947c19b3b0 con 0x7f946406d8a0 2024-02-16T19:36:17.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.084+0000 7f9480f41700 1 -- 172.21.15.22:0/4096533722 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f946406d8a0 msgr2=0x7f946406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:17.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.084+0000 7f9480f41700 1 --2- 172.21.15.22:0/4096533722 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f946406d8a0 0x7f946406fd50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f946c00b760 tx=0x7f946c005940 comp rx=0 tx=0).stop 2024-02-16T19:36:17.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.084+0000 7f9480f41700 1 -- 172.21.15.22:0/4096533722 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f947c100e80 msgr2=0x7f947c196430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:17.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.084+0000 7f9480f41700 1 --2- 172.21.15.22:0/4096533722 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f947c100e80 0x7f947c196430 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f947000e8f0 tx=0x7f947000ec00 comp rx=0 tx=0).stop 2024-02-16T19:36:17.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.084+0000 7f9480f41700 1 -- 172.21.15.22:0/4096533722 shutdown_connections 2024-02-16T19:36:17.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.084+0000 7f9480f41700 1 --2- 172.21.15.22:0/4096533722 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f946406d8a0 0x7f946406fd50 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:17.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.084+0000 7f9480f41700 1 --2- 172.21.15.22:0/4096533722 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f947c1004e0 0x7f947c195ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:17.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.084+0000 7f9480f41700 1 --2- 172.21.15.22:0/4096533722 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f947c100e80 0x7f947c196430 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:17.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.084+0000 7f9480f41700 1 -- 172.21.15.22:0/4096533722 >> 172.21.15.22:0/4096533722 conn(0x7f947c0fbda0 msgr2=0x7f947c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:17.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.084+0000 7f9480f41700 1 -- 172.21.15.22:0/4096533722 shutdown_connections 2024-02-16T19:36:17.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.084+0000 7f9480f41700 1 -- 172.21.15.22:0/4096533722 wait complete. 2024-02-16T19:36:17.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:16 smithi117 ceph-mon[67870]: pgmap v117: 65 pgs: 65 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 341 KiB/s, 0 objects/s recovering 2024-02-16T19:36:17.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:16 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:36:17.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:16 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:36:17.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:16 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:36:17.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:16 smithi117 ceph-mon[67870]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:36:17.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.190+0000 7fb014ba3700 1 -- 172.21.15.22:0/3920804566 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb0100ffd60 msgr2=0x7fb010103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:17.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.190+0000 7fb014ba3700 1 --2- 172.21.15.22:0/3920804566 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb0100ffd60 0x7fb010103d30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb000009910 tx=0x7fb000009c20 comp rx=0 tx=0).stop 2024-02-16T19:36:17.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.190+0000 7fb014ba3700 1 -- 172.21.15.22:0/3920804566 shutdown_connections 2024-02-16T19:36:17.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.190+0000 7fb014ba3700 1 --2- 172.21.15.22:0/3920804566 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb0100ffd60 0x7fb010103d30 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:17.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.190+0000 7fb014ba3700 1 --2- 172.21.15.22:0/3920804566 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0100ff3c0 0x7fb0100ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:17.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.190+0000 7fb014ba3700 1 -- 172.21.15.22:0/3920804566 >> 172.21.15.22:0/3920804566 conn(0x7fb0100fac80 msgr2=0x7fb0100fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:17.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.191+0000 7fb014ba3700 1 -- 172.21.15.22:0/3920804566 shutdown_connections 2024-02-16T19:36:17.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.191+0000 7fb014ba3700 1 -- 172.21.15.22:0/3920804566 wait complete. 2024-02-16T19:36:17.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.191+0000 7fb014ba3700 1 Processor -- start 2024-02-16T19:36:17.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.191+0000 7fb014ba3700 1 -- start start 2024-02-16T19:36:17.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.191+0000 7fb014ba3700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0100ff3c0 0x7fb010194e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:17.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.191+0000 7fb014ba3700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb0100ffd60 0x7fb010195350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:17.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.191+0000 7fb014ba3700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0101959a0 con 0x7fb0100ffd60 2024-02-16T19:36:17.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.192+0000 7fb014ba3700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb010195ae0 con 0x7fb0100ff3c0 2024-02-16T19:36:17.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.192+0000 7fb00dd9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb0100ffd60 0x7fb010195350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:17.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.192+0000 7fb00dd9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb0100ffd60 0x7fb010195350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42562/0 (socket says 172.21.15.22:42562) 2024-02-16T19:36:17.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.192+0000 7fb00dd9b700 1 -- 172.21.15.22:0/2194959335 learned_addr learned my addr 172.21.15.22:0/2194959335 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:36:17.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.192+0000 7fb00e59c700 1 --2- 172.21.15.22:0/2194959335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0100ff3c0 0x7fb010194e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:17.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.192+0000 7fb00dd9b700 1 -- 172.21.15.22:0/2194959335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0100ff3c0 msgr2=0x7fb010194e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:17.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.192+0000 7fb00dd9b700 1 --2- 172.21.15.22:0/2194959335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0100ff3c0 0x7fb010194e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:17.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.192+0000 7fb00dd9b700 1 -- 172.21.15.22:0/2194959335 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0000095f0 con 0x7fb0100ffd60 2024-02-16T19:36:17.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.192+0000 7fb00e59c700 1 --2- 172.21.15.22:0/2194959335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0100ff3c0 0x7fb010194e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T19:36:17.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.192+0000 7fb00dd9b700 1 --2- 172.21.15.22:0/2194959335 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb0100ffd60 0x7fb010195350 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb00000bda0 tx=0x7fb00000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:17.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.192+0000 7fafff7fe700 1 -- 172.21.15.22:0/2194959335 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb00001d070 con 0x7fb0100ffd60 2024-02-16T19:36:17.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.192+0000 7fafff7fe700 1 -- 172.21.15.22:0/2194959335 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb000004fb0 con 0x7fb0100ffd60 2024-02-16T19:36:17.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.192+0000 7fafff7fe700 1 -- 172.21.15.22:0/2194959335 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb00000f460 con 0x7fb0100ffd60 2024-02-16T19:36:17.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.192+0000 7fb014ba3700 1 -- 172.21.15.22:0/2194959335 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0101998d0 con 0x7fb0100ffd60 2024-02-16T19:36:17.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.192+0000 7fb014ba3700 1 -- 172.21.15.22:0/2194959335 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb010199dc0 con 0x7fb0100ffd60 2024-02-16T19:36:17.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.194+0000 7fb014ba3700 1 -- 172.21.15.22:0/2194959335 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb010107730 con 0x7fb0100ffd60 2024-02-16T19:36:17.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.194+0000 7fafff7fe700 1 -- 172.21.15.22:0/2194959335 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fb000022800 con 0x7fb0100ffd60 2024-02-16T19:36:17.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.194+0000 7fafff7fe700 1 --2- 172.21.15.22:0/2194959335 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7faff806d8a0 0x7faff806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:17.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.194+0000 7fafff7fe700 1 -- 172.21.15.22:0/2194959335 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6704+0+0 (secure 0 0 0) 0x7fb00008ea10 con 0x7fb0100ffd60 2024-02-16T19:36:17.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.195+0000 7fb00e59c700 1 --2- 172.21.15.22:0/2194959335 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7faff806d8a0 0x7faff806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:17.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.195+0000 7fb00e59c700 1 --2- 172.21.15.22:0/2194959335 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7faff806d8a0 0x7faff806fd50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb004005fd0 tx=0x7fb004005e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:17.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.199+0000 7fafff7fe700 1 -- 172.21.15.22:0/2194959335 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb0000583a0 con 0x7fb0100ffd60 2024-02-16T19:36:17.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.417+0000 7fb014ba3700 1 -- 172.21.15.22:0/2194959335 --> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb01019a030 con 0x7faff806d8a0 2024-02-16T19:36:17.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.428+0000 7fafff7fe700 1 -- 172.21.15.22:0/2194959335 <== mgr.34104 v2:172.21.15.22:6800/505266753 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fb01019a030 con 0x7faff806d8a0 2024-02-16T19:36:17.428 INFO:teuthology.orchestra.run.smithi022.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T19:36:17.429 INFO:teuthology.orchestra.run.smithi022.stdout:alertmanager.smithi022 smithi022 *:9093,9094 running (4m) 16s ago 15m 26.8M - 0.25.0 c8568f914cd2 c89bc76be374 2024-02-16T19:36:17.429 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-exporter.smithi022 smithi022 running (15m) 16s ago 15m 8606k - 18.2.0 dc2bc1663786 3f8bef4349ba 2024-02-16T19:36:17.429 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-exporter.smithi117 smithi117 running (14m) 112s ago 14m 7768k - 18.2.0 dc2bc1663786 59b65a7e309f 2024-02-16T19:36:17.429 INFO:teuthology.orchestra.run.smithi022.stdout:crash.smithi022 smithi022 running (2m) 16s ago 15m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a e0f78bdf43e4 2024-02-16T19:36:17.429 INFO:teuthology.orchestra.run.smithi022.stdout:crash.smithi117 smithi117 running (116s) 112s ago 14m 6669k - 19.0.0-1450-gf78a58c0 f081f847070a bce7c0590813 2024-02-16T19:36:17.429 INFO:teuthology.orchestra.run.smithi022.stdout:grafana.smithi022 smithi022 *:3000 running (4m) 16s ago 15m 81.0M - 9.4.12 a0ea5e3c46d9 fd27d11e9729 2024-02-16T19:36:17.429 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi022.btppwm smithi022 running (9m) 16s ago 9m 155M - 18.2.0 dc2bc1663786 22971fb3a907 2024-02-16T19:36:17.429 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi022.ocukvs smithi022 running (9m) 16s ago 9m 17.1M - 18.2.0 dc2bc1663786 81ee7bd5fd39 2024-02-16T19:36:17.429 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi117.ibmppn smithi117 running (9m) 112s ago 9m 17.6M - 18.2.0 dc2bc1663786 8c606cbeeb86 2024-02-16T19:36:17.429 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi117.orzors smithi117 running (9m) 112s ago 9m 16.6M - 18.2.0 dc2bc1663786 7d2bc4df3feb 2024-02-16T19:36:17.429 INFO:teuthology.orchestra.run.smithi022.stdout:mgr.smithi022.gmdcch smithi022 *:8443,9283,8765 running (6m) 16s ago 16m 504M - 19.0.0-1450-gf78a58c0 f081f847070a 7cb74cc9deea 2024-02-16T19:36:17.429 INFO:teuthology.orchestra.run.smithi022.stdout:mgr.smithi117.cohbtv smithi117 *:8443,9283,8765 running (5m) 112s ago 14m 451M - 19.0.0-1450-gf78a58c0 f081f847070a c825b1152189 2024-02-16T19:36:17.429 INFO:teuthology.orchestra.run.smithi022.stdout:mon.smithi022 smithi022 running (3m) 16s ago 17m 48.4M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 9c84755ae67d 2024-02-16T19:36:17.429 INFO:teuthology.orchestra.run.smithi022.stdout:mon.smithi117 smithi117 running (2m) 112s ago 14m 38.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a a319ac50f1a9 2024-02-16T19:36:17.430 INFO:teuthology.orchestra.run.smithi022.stdout:node-exporter.smithi022 smithi022 *:9100 running (5m) 16s ago 15m 16.6M - 1.5.0 0da6a335fe13 1f5b58292d27 2024-02-16T19:36:17.430 INFO:teuthology.orchestra.run.smithi022.stdout:node-exporter.smithi117 smithi117 *:9100 running (5m) 112s ago 14m 15.4M - 1.5.0 0da6a335fe13 bb5e8491d5a6 2024-02-16T19:36:17.430 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0 smithi022 running (94s) 16s ago 13m 107M 1217M 19.0.0-1450-gf78a58c0 f081f847070a 002412b3525f 2024-02-16T19:36:17.430 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1 smithi022 running (58s) 16s ago 13m 128M 1217M 19.0.0-1450-gf78a58c0 f081f847070a c3488ce7b2a0 2024-02-16T19:36:17.430 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2 smithi022 running (22s) 16s ago 12m 12.6M 1217M 19.0.0-1450-gf78a58c0 f081f847070a 46138c50f79d 2024-02-16T19:36:17.430 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3 smithi117 running (12m) 112s ago 12m 244M 2240M 18.2.0 dc2bc1663786 9c0ce9a981cf 2024-02-16T19:36:17.430 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4 smithi117 running (11m) 112s ago 11m 209M 2240M 18.2.0 dc2bc1663786 1779c3496f4d 2024-02-16T19:36:17.430 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5 smithi117 running (11m) 112s ago 11m 198M 2240M 18.2.0 dc2bc1663786 fefaabca0638 2024-02-16T19:36:17.430 INFO:teuthology.orchestra.run.smithi022.stdout:prometheus.smithi022 smithi022 *:9095 running (4m) 16s ago 14m 66.9M - 2.43.0 a07b618ecd1d c3b8689d0c97 2024-02-16T19:36:17.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.430+0000 7fb014ba3700 1 -- 172.21.15.22:0/2194959335 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7faff806d8a0 msgr2=0x7faff806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:17.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.430+0000 7fb014ba3700 1 --2- 172.21.15.22:0/2194959335 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7faff806d8a0 0x7faff806fd50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb004005fd0 tx=0x7fb004005e30 comp rx=0 tx=0).stop 2024-02-16T19:36:17.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.430+0000 7fb014ba3700 1 -- 172.21.15.22:0/2194959335 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb0100ffd60 msgr2=0x7fb010195350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:17.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.430+0000 7fb014ba3700 1 --2- 172.21.15.22:0/2194959335 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb0100ffd60 0x7fb010195350 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb00000bda0 tx=0x7fb00000be80 comp rx=0 tx=0).stop 2024-02-16T19:36:17.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.430+0000 7fb014ba3700 1 -- 172.21.15.22:0/2194959335 shutdown_connections 2024-02-16T19:36:17.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.430+0000 7fb014ba3700 1 --2- 172.21.15.22:0/2194959335 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7faff806d8a0 0x7faff806fd50 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:17.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.430+0000 7fb014ba3700 1 --2- 172.21.15.22:0/2194959335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0100ff3c0 0x7fb010194e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:17.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.430+0000 7fb014ba3700 1 --2- 172.21.15.22:0/2194959335 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb0100ffd60 0x7fb010195350 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:17.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.430+0000 7fb014ba3700 1 -- 172.21.15.22:0/2194959335 >> 172.21.15.22:0/2194959335 conn(0x7fb0100fac80 msgr2=0x7fb0100fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:17.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.430+0000 7fb014ba3700 1 -- 172.21.15.22:0/2194959335 shutdown_connections 2024-02-16T19:36:17.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.430+0000 7fb014ba3700 1 -- 172.21.15.22:0/2194959335 wait complete. 2024-02-16T19:36:17.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.534+0000 7f4fd8df7700 1 -- 172.21.15.22:0/4191894421 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4fd4100e80 msgr2=0x7f4fd4104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:17.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.534+0000 7f4fd8df7700 1 --2- 172.21.15.22:0/4191894421 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4fd4100e80 0x7f4fd4104e50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc009910 tx=0x7f4fbc009c20 comp rx=0 tx=0).stop 2024-02-16T19:36:17.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.534+0000 7f4fd8df7700 1 -- 172.21.15.22:0/4191894421 shutdown_connections 2024-02-16T19:36:17.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.534+0000 7f4fd8df7700 1 --2- 172.21.15.22:0/4191894421 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4fd4100e80 0x7f4fd4104e50 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:17.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.534+0000 7f4fd8df7700 1 --2- 172.21.15.22:0/4191894421 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fd41004e0 0x7f4fd41008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:17.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.534+0000 7f4fd8df7700 1 -- 172.21.15.22:0/4191894421 >> 172.21.15.22:0/4191894421 conn(0x7f4fd40fbda0 msgr2=0x7f4fd40fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:17.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.534+0000 7f4fd8df7700 1 -- 172.21.15.22:0/4191894421 shutdown_connections 2024-02-16T19:36:17.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.534+0000 7f4fd8df7700 1 -- 172.21.15.22:0/4191894421 wait complete. 2024-02-16T19:36:17.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.535+0000 7f4fd8df7700 1 Processor -- start 2024-02-16T19:36:17.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.535+0000 7f4fd8df7700 1 -- start start 2024-02-16T19:36:17.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.535+0000 7f4fd8df7700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4fd41004e0 0x7f4fd4195f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:17.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.535+0000 7f4fd8df7700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fd4100e80 0x7f4fd4196450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:17.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.535+0000 7f4fd8df7700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd4196b30 con 0x7f4fd41004e0 2024-02-16T19:36:17.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.535+0000 7f4fd8df7700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd419a8c0 con 0x7f4fd4100e80 2024-02-16T19:36:17.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.535+0000 7f4fd259c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4fd41004e0 0x7f4fd4195f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:17.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.536+0000 7f4fd259c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4fd41004e0 0x7f4fd4195f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42578/0 (socket says 172.21.15.22:42578) 2024-02-16T19:36:17.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.536+0000 7f4fd259c700 1 -- 172.21.15.22:0/1212997489 learned_addr learned my addr 172.21.15.22:0/1212997489 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:36:17.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.536+0000 7f4fd1d9b700 1 --2- 172.21.15.22:0/1212997489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fd4100e80 0x7f4fd4196450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:17.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.536+0000 7f4fd259c700 1 -- 172.21.15.22:0/1212997489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fd4100e80 msgr2=0x7f4fd4196450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:17.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.536+0000 7f4fd259c700 1 --2- 172.21.15.22:0/1212997489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fd4100e80 0x7f4fd4196450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:17.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.536+0000 7f4fd259c700 1 -- 172.21.15.22:0/1212997489 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fbc0095f0 con 0x7f4fd41004e0 2024-02-16T19:36:17.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.536+0000 7f4fd259c700 1 --2- 172.21.15.22:0/1212997489 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4fd41004e0 0x7f4fd4195f10 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc00d8d0 tx=0x7f4fcc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:17.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.536+0000 7f4fcb7fe700 1 -- 172.21.15.22:0/1212997489 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4fcc009940 con 0x7f4fd41004e0 2024-02-16T19:36:17.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.536+0000 7f4fd8df7700 1 -- 172.21.15.22:0/1212997489 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fd419aba0 con 0x7f4fd41004e0 2024-02-16T19:36:17.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.536+0000 7f4fcb7fe700 1 -- 172.21.15.22:0/1212997489 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4fcc010460 con 0x7f4fd41004e0 2024-02-16T19:36:17.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.536+0000 7f4fcb7fe700 1 -- 172.21.15.22:0/1212997489 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4fcc00f5f0 con 0x7f4fd41004e0 2024-02-16T19:36:17.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.536+0000 7f4fd8df7700 1 -- 172.21.15.22:0/1212997489 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fd419b0f0 con 0x7f4fd41004e0 2024-02-16T19:36:17.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.537+0000 7f4fd8df7700 1 -- 172.21.15.22:0/1212997489 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fd4108850 con 0x7f4fd41004e0 2024-02-16T19:36:17.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.538+0000 7f4fcb7fe700 1 -- 172.21.15.22:0/1212997489 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f4fcc0105d0 con 0x7f4fd41004e0 2024-02-16T19:36:17.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.538+0000 7f4fcb7fe700 1 --2- 172.21.15.22:0/1212997489 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f4fc006d8a0 0x7f4fc006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:17.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.538+0000 7f4fcb7fe700 1 -- 172.21.15.22:0/1212997489 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f4fcc08d5b0 con 0x7f4fd41004e0 2024-02-16T19:36:17.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.538+0000 7f4fd1d9b700 1 --2- 172.21.15.22:0/1212997489 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f4fc006d8a0 0x7f4fc006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:17.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.539+0000 7f4fd1d9b700 1 --2- 172.21.15.22:0/1212997489 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f4fc006d8a0 0x7f4fc006fd50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4197530 tx=0x7f4fbc01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:17.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.543+0000 7f4fcb7fe700 1 -- 172.21.15.22:0/1212997489 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4fcc056cf0 con 0x7f4fd41004e0 2024-02-16T19:36:17.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.843+0000 7f4fd8df7700 1 -- 172.21.15.22:0/1212997489 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f4fd419b310 con 0x7f4fd41004e0 2024-02-16T19:36:17.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.844+0000 7f4fcb7fe700 1 -- 172.21.15.22:0/1212997489 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+791 (secure 0 0 0) 0x7f4fcc05a3b0 con 0x7f4fd41004e0 2024-02-16T19:36:17.845 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-02-16T19:36:17.846 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": { 2024-02-16T19:36:17.846 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T19:36:17.846 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:36:17.846 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": { 2024-02-16T19:36:17.846 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T19:36:17.846 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:36:17.846 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": { 2024-02-16T19:36:17.846 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 3, 2024-02-16T19:36:17.846 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 3 2024-02-16T19:36:17.846 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:36:17.846 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": { 2024-02-16T19:36:17.846 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-02-16T19:36:17.846 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:36:17.846 INFO:teuthology.orchestra.run.smithi022.stdout: "overall": { 2024-02-16T19:36:17.846 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 7, 2024-02-16T19:36:17.846 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 7 2024-02-16T19:36:17.847 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-02-16T19:36:17.847 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-02-16T19:36:17.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.847+0000 7f4fd8df7700 1 -- 172.21.15.22:0/1212997489 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f4fc006d8a0 msgr2=0x7f4fc006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:17.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.847+0000 7f4fd8df7700 1 --2- 172.21.15.22:0/1212997489 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f4fc006d8a0 0x7f4fc006fd50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4197530 tx=0x7f4fbc01a040 comp rx=0 tx=0).stop 2024-02-16T19:36:17.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.847+0000 7f4fd8df7700 1 -- 172.21.15.22:0/1212997489 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4fd41004e0 msgr2=0x7f4fd4195f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:17.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.847+0000 7f4fd8df7700 1 --2- 172.21.15.22:0/1212997489 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4fd41004e0 0x7f4fd4195f10 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc00d8d0 tx=0x7f4fcc00dc90 comp rx=0 tx=0).stop 2024-02-16T19:36:17.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.847+0000 7f4fd8df7700 1 -- 172.21.15.22:0/1212997489 shutdown_connections 2024-02-16T19:36:17.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.847+0000 7f4fd8df7700 1 --2- 172.21.15.22:0/1212997489 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f4fc006d8a0 0x7f4fc006fd50 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:17.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.847+0000 7f4fd8df7700 1 --2- 172.21.15.22:0/1212997489 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4fd41004e0 0x7f4fd4195f10 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:17.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.847+0000 7f4fd8df7700 1 --2- 172.21.15.22:0/1212997489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fd4100e80 0x7f4fd4196450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:17.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.847+0000 7f4fd8df7700 1 -- 172.21.15.22:0/1212997489 >> 172.21.15.22:0/1212997489 conn(0x7f4fd40fbda0 msgr2=0x7f4fd40fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:17.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.847+0000 7f4fd8df7700 1 -- 172.21.15.22:0/1212997489 shutdown_connections 2024-02-16T19:36:17.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.847+0000 7f4fd8df7700 1 -- 172.21.15.22:0/1212997489 wait complete. 2024-02-16T19:36:17.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.953+0000 7f5d678db700 1 -- 172.21.15.22:0/409681296 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d60100ed0 msgr2=0x7f5d60104ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:17.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.953+0000 7f5d678db700 1 --2- 172.21.15.22:0/409681296 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d60100ed0 0x7f5d60104ea0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c009910 tx=0x7f5d5c009c20 comp rx=0 tx=0).stop 2024-02-16T19:36:17.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.953+0000 7f5d678db700 1 -- 172.21.15.22:0/409681296 shutdown_connections 2024-02-16T19:36:17.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.953+0000 7f5d678db700 1 --2- 172.21.15.22:0/409681296 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d60100ed0 0x7f5d60104ea0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:17.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.953+0000 7f5d678db700 1 --2- 172.21.15.22:0/409681296 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d60100530 0x7f5d60100900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:17.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.953+0000 7f5d678db700 1 -- 172.21.15.22:0/409681296 >> 172.21.15.22:0/409681296 conn(0x7f5d600fbe10 msgr2=0x7f5d600fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:17.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.953+0000 7f5d678db700 1 -- 172.21.15.22:0/409681296 shutdown_connections 2024-02-16T19:36:17.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.953+0000 7f5d678db700 1 -- 172.21.15.22:0/409681296 wait complete. 2024-02-16T19:36:17.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.954+0000 7f5d678db700 1 Processor -- start 2024-02-16T19:36:17.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.954+0000 7f5d678db700 1 -- start start 2024-02-16T19:36:17.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.954+0000 7f5d678db700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d60100530 0x7f5d60073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:17.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.954+0000 7f5d678db700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d60100ed0 0x7f5d60073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:17.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.954+0000 7f5d678db700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d60073ad0 con 0x7f5d60100ed0 2024-02-16T19:36:17.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.954+0000 7f5d678db700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d60073c10 con 0x7f5d60100530 2024-02-16T19:36:17.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.954+0000 7f5d64e76700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d60100ed0 0x7f5d60073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:17.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.954+0000 7f5d64e76700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d60100ed0 0x7f5d60073590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42602/0 (socket says 172.21.15.22:42602) 2024-02-16T19:36:17.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.954+0000 7f5d64e76700 1 -- 172.21.15.22:0/4044425044 learned_addr learned my addr 172.21.15.22:0/4044425044 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:36:17.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.954+0000 7f5d65677700 1 --2- 172.21.15.22:0/4044425044 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d60100530 0x7f5d60073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:17.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.955+0000 7f5d64e76700 1 -- 172.21.15.22:0/4044425044 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d60100530 msgr2=0x7f5d60073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:17.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.955+0000 7f5d64e76700 1 --2- 172.21.15.22:0/4044425044 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d60100530 0x7f5d60073050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:17.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.955+0000 7f5d64e76700 1 -- 172.21.15.22:0/4044425044 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d5c0095f0 con 0x7f5d60100ed0 2024-02-16T19:36:17.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.955+0000 7f5d64e76700 1 --2- 172.21.15.22:0/4044425044 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d60100ed0 0x7f5d60073590 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c0092e0 tx=0x7f5d5c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:17.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.955+0000 7f5d567fc700 1 -- 172.21.15.22:0/4044425044 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d5c01d070 con 0x7f5d60100ed0 2024-02-16T19:36:17.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.955+0000 7f5d567fc700 1 -- 172.21.15.22:0/4044425044 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5d5c005dc0 con 0x7f5d60100ed0 2024-02-16T19:36:17.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.955+0000 7f5d678db700 1 -- 172.21.15.22:0/4044425044 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d60073e90 con 0x7f5d60100ed0 2024-02-16T19:36:17.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.955+0000 7f5d567fc700 1 -- 172.21.15.22:0/4044425044 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d5c017400 con 0x7f5d60100ed0 2024-02-16T19:36:17.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.955+0000 7f5d678db700 1 -- 172.21.15.22:0/4044425044 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d601a23f0 con 0x7f5d60100ed0 2024-02-16T19:36:17.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.956+0000 7f5d678db700 1 -- 172.21.15.22:0/4044425044 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d601088a0 con 0x7f5d60100ed0 2024-02-16T19:36:17.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.956+0000 7f5d567fc700 1 -- 172.21.15.22:0/4044425044 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f5d5c00f6f0 con 0x7f5d60100ed0 2024-02-16T19:36:17.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.957+0000 7f5d567fc700 1 --2- 172.21.15.22:0/4044425044 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f5d4c06d8a0 0x7f5d4c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:17.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.957+0000 7f5d567fc700 1 -- 172.21.15.22:0/4044425044 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f5d5c08e890 con 0x7f5d60100ed0 2024-02-16T19:36:17.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.957+0000 7f5d65677700 1 --2- 172.21.15.22:0/4044425044 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f5d4c06d8a0 0x7f5d4c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:17.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.957+0000 7f5d65677700 1 --2- 172.21.15.22:0/4044425044 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f5d4c06d8a0 0x7f5d4c06fd50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5d50005950 tx=0x7f5d5000b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:17.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:17.961+0000 7f5d567fc700 1 -- 172.21.15.22:0/4044425044 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5d5c058050 con 0x7f5d60100ed0 2024-02-16T19:36:18.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:17 smithi117 ceph-mon[67870]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:36:18.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:17 smithi117 ceph-mon[67870]: from='client.34292 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:36:18.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.210+0000 7f5d678db700 1 -- 172.21.15.22:0/4044425044 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f5d6004f2a0 con 0x7f5d60100ed0 2024-02-16T19:36:18.212 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:17 smithi022 ceph-mon[88798]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:36:18.212 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:17 smithi022 ceph-mon[88798]: from='client.34292 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:36:18.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.211+0000 7f5d567fc700 1 -- 172.21.15.22:0/4044425044 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1845 (secure 0 0 0) 0x7f5d5c05b710 con 0x7f5d60100ed0 2024-02-16T19:36:18.213 INFO:teuthology.orchestra.run.smithi022.stdout:e15 2024-02-16T19:36:18.213 INFO:teuthology.orchestra.run.smithi022.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T19:36:18.213 INFO:teuthology.orchestra.run.smithi022.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:36:18.213 INFO:teuthology.orchestra.run.smithi022.stdout:legacy client fscid: 1 2024-02-16T19:36:18.213 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:36:18.213 INFO:teuthology.orchestra.run.smithi022.stdout:Filesystem 'cephfs' (1) 2024-02-16T19:36:18.213 INFO:teuthology.orchestra.run.smithi022.stdout:fs_name cephfs 2024-02-16T19:36:18.213 INFO:teuthology.orchestra.run.smithi022.stdout:epoch 15 2024-02-16T19:36:18.213 INFO:teuthology.orchestra.run.smithi022.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T19:36:18.213 INFO:teuthology.orchestra.run.smithi022.stdout:created 2024-02-16T19:26:17.873644+0000 2024-02-16T19:36:18.213 INFO:teuthology.orchestra.run.smithi022.stdout:modified 2024-02-16T19:26:35.943353+0000 2024-02-16T19:36:18.213 INFO:teuthology.orchestra.run.smithi022.stdout:tableserver 0 2024-02-16T19:36:18.214 INFO:teuthology.orchestra.run.smithi022.stdout:root 0 2024-02-16T19:36:18.214 INFO:teuthology.orchestra.run.smithi022.stdout:session_timeout 60 2024-02-16T19:36:18.214 INFO:teuthology.orchestra.run.smithi022.stdout:session_autoclose 300 2024-02-16T19:36:18.214 INFO:teuthology.orchestra.run.smithi022.stdout:max_file_size 1099511627776 2024-02-16T19:36:18.214 INFO:teuthology.orchestra.run.smithi022.stdout:max_xattr_size 65536 2024-02-16T19:36:18.214 INFO:teuthology.orchestra.run.smithi022.stdout:required_client_features {} 2024-02-16T19:36:18.214 INFO:teuthology.orchestra.run.smithi022.stdout:last_failure 0 2024-02-16T19:36:18.214 INFO:teuthology.orchestra.run.smithi022.stdout:last_failure_osd_epoch 47 2024-02-16T19:36:18.214 INFO:teuthology.orchestra.run.smithi022.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:36:18.214 INFO:teuthology.orchestra.run.smithi022.stdout:max_mds 1 2024-02-16T19:36:18.214 INFO:teuthology.orchestra.run.smithi022.stdout:in 0 2024-02-16T19:36:18.214 INFO:teuthology.orchestra.run.smithi022.stdout:up {0=14490} 2024-02-16T19:36:18.214 INFO:teuthology.orchestra.run.smithi022.stdout:failed 2024-02-16T19:36:18.214 INFO:teuthology.orchestra.run.smithi022.stdout:damaged 2024-02-16T19:36:18.214 INFO:teuthology.orchestra.run.smithi022.stdout:stopped 2024-02-16T19:36:18.214 INFO:teuthology.orchestra.run.smithi022.stdout:data_pools [3] 2024-02-16T19:36:18.215 INFO:teuthology.orchestra.run.smithi022.stdout:metadata_pool 2 2024-02-16T19:36:18.215 INFO:teuthology.orchestra.run.smithi022.stdout:inline_data enabled 2024-02-16T19:36:18.215 INFO:teuthology.orchestra.run.smithi022.stdout:balancer 2024-02-16T19:36:18.215 INFO:teuthology.orchestra.run.smithi022.stdout:bal_rank_mask -1 2024-02-16T19:36:18.215 INFO:teuthology.orchestra.run.smithi022.stdout:standby_count_wanted 1 2024-02-16T19:36:18.215 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.cephfs.smithi022.btppwm{0:14490} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.22:6828/3597685751,v1:172.21.15.22:6829/3597685751] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:36:18.215 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:36:18.215 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:36:18.215 INFO:teuthology.orchestra.run.smithi022.stdout:Standby daemons: 2024-02-16T19:36:18.215 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:36:18.215 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.cephfs.smithi022.ocukvs{-1:14500} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.22:6826/242865843,v1:172.21.15.22:6827/242865843] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:36:18.215 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.cephfs.smithi117.ibmppn{-1:24259} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.117:6824/565408102,v1:172.21.15.117:6825/565408102] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:36:18.215 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.cephfs.smithi117.orzors{-1:24265} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.117:6826/3122342028,v1:172.21.15.117:6827/3122342028] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:36:18.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.214+0000 7f5d678db700 1 -- 172.21.15.22:0/4044425044 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f5d4c06d8a0 msgr2=0x7f5d4c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:18.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.214+0000 7f5d678db700 1 --2- 172.21.15.22:0/4044425044 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f5d4c06d8a0 0x7f5d4c06fd50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5d50005950 tx=0x7f5d5000b410 comp rx=0 tx=0).stop 2024-02-16T19:36:18.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.214+0000 7f5d678db700 1 -- 172.21.15.22:0/4044425044 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d60100ed0 msgr2=0x7f5d60073590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:18.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.214+0000 7f5d678db700 1 --2- 172.21.15.22:0/4044425044 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d60100ed0 0x7f5d60073590 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c0092e0 tx=0x7f5d5c00bd70 comp rx=0 tx=0).stop 2024-02-16T19:36:18.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.214+0000 7f5d678db700 1 -- 172.21.15.22:0/4044425044 shutdown_connections 2024-02-16T19:36:18.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.214+0000 7f5d678db700 1 --2- 172.21.15.22:0/4044425044 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f5d4c06d8a0 0x7f5d4c06fd50 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:18.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.214+0000 7f5d678db700 1 --2- 172.21.15.22:0/4044425044 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d60100530 0x7f5d60073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:18.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.214+0000 7f5d678db700 1 --2- 172.21.15.22:0/4044425044 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d60100ed0 0x7f5d60073590 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:18.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.214+0000 7f5d678db700 1 -- 172.21.15.22:0/4044425044 >> 172.21.15.22:0/4044425044 conn(0x7f5d600fbe10 msgr2=0x7f5d600fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:18.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.214+0000 7f5d678db700 1 -- 172.21.15.22:0/4044425044 shutdown_connections 2024-02-16T19:36:18.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.214+0000 7f5d678db700 1 -- 172.21.15.22:0/4044425044 wait complete. 2024-02-16T19:36:18.217 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 15 2024-02-16T19:36:18.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.321+0000 7f5d0a9c3700 1 -- 172.21.15.22:0/286472894 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d04100e80 msgr2=0x7f5d04104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:18.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.321+0000 7f5d0a9c3700 1 --2- 172.21.15.22:0/286472894 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d04100e80 0x7f5d04104e50 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8009910 tx=0x7f5cf8009c20 comp rx=0 tx=0).stop 2024-02-16T19:36:18.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.321+0000 7f5d0a9c3700 1 -- 172.21.15.22:0/286472894 shutdown_connections 2024-02-16T19:36:18.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.321+0000 7f5d0a9c3700 1 --2- 172.21.15.22:0/286472894 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d04100e80 0x7f5d04104e50 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:18.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.321+0000 7f5d0a9c3700 1 --2- 172.21.15.22:0/286472894 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d041004e0 0x7f5d041008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:18.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.321+0000 7f5d0a9c3700 1 -- 172.21.15.22:0/286472894 >> 172.21.15.22:0/286472894 conn(0x7f5d040fbda0 msgr2=0x7f5d040fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:18.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.322+0000 7f5d0a9c3700 1 -- 172.21.15.22:0/286472894 shutdown_connections 2024-02-16T19:36:18.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.322+0000 7f5d0a9c3700 1 -- 172.21.15.22:0/286472894 wait complete. 2024-02-16T19:36:18.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.322+0000 7f5d0a9c3700 1 Processor -- start 2024-02-16T19:36:18.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.322+0000 7f5d0a9c3700 1 -- start start 2024-02-16T19:36:18.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.322+0000 7f5d0a9c3700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d041004e0 0x7f5d04073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:18.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.322+0000 7f5d0a9c3700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d04100e80 0x7f5d04073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:18.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.322+0000 7f5d0a9c3700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d04073ad0 con 0x7f5d04100e80 2024-02-16T19:36:18.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.322+0000 7f5d0a9c3700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d04073c10 con 0x7f5d041004e0 2024-02-16T19:36:18.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.323+0000 7f5d037fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d04100e80 0x7f5d04073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:18.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.323+0000 7f5d037fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d04100e80 0x7f5d04073590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42612/0 (socket says 172.21.15.22:42612) 2024-02-16T19:36:18.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.323+0000 7f5d037fe700 1 -- 172.21.15.22:0/3945747096 learned_addr learned my addr 172.21.15.22:0/3945747096 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:36:18.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.323+0000 7f5d03fff700 1 --2- 172.21.15.22:0/3945747096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d041004e0 0x7f5d04073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:18.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.323+0000 7f5d037fe700 1 -- 172.21.15.22:0/3945747096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d041004e0 msgr2=0x7f5d04073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:18.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.323+0000 7f5d037fe700 1 --2- 172.21.15.22:0/3945747096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d041004e0 0x7f5d04073050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:18.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.323+0000 7f5d037fe700 1 -- 172.21.15.22:0/3945747096 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cf80095f0 con 0x7f5d04100e80 2024-02-16T19:36:18.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.323+0000 7f5d03fff700 1 --2- 172.21.15.22:0/3945747096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d041004e0 0x7f5d04073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-16T19:36:18.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.323+0000 7f5d037fe700 1 --2- 172.21.15.22:0/3945747096 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d04100e80 0x7f5d04073590 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5cf80092e0 tx=0x7f5cf800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:18.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.323+0000 7f5d017fa700 1 -- 172.21.15.22:0/3945747096 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5cf801d070 con 0x7f5d04100e80 2024-02-16T19:36:18.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.323+0000 7f5d0a9c3700 1 -- 172.21.15.22:0/3945747096 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d04073e90 con 0x7f5d04100e80 2024-02-16T19:36:18.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.323+0000 7f5d017fa700 1 -- 172.21.15.22:0/3945747096 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5cf800f460 con 0x7f5d04100e80 2024-02-16T19:36:18.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.323+0000 7f5d017fa700 1 -- 172.21.15.22:0/3945747096 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5cf80175f0 con 0x7f5d04100e80 2024-02-16T19:36:18.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.323+0000 7f5d0a9c3700 1 -- 172.21.15.22:0/3945747096 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d041a2500 con 0x7f5d04100e80 2024-02-16T19:36:18.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.324+0000 7f5d0a9c3700 1 -- 172.21.15.22:0/3945747096 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d041087a0 con 0x7f5d04100e80 2024-02-16T19:36:18.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.325+0000 7f5d017fa700 1 -- 172.21.15.22:0/3945747096 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f5cf800f5d0 con 0x7f5d04100e80 2024-02-16T19:36:18.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.325+0000 7f5d017fa700 1 --2- 172.21.15.22:0/3945747096 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f5cec06d8a0 0x7f5cec06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:18.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.325+0000 7f5d017fa700 1 -- 172.21.15.22:0/3945747096 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f5cf808eac0 con 0x7f5d04100e80 2024-02-16T19:36:18.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.325+0000 7f5d03fff700 1 --2- 172.21.15.22:0/3945747096 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f5cec06d8a0 0x7f5cec06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:18.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.326+0000 7f5d03fff700 1 --2- 172.21.15.22:0/3945747096 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f5cec06d8a0 0x7f5cec06fd50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5cf4007900 tx=0x7f5cf4008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:18.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.330+0000 7f5d017fa700 1 -- 172.21.15.22:0/3945747096 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5cf8058200 con 0x7f5d04100e80 2024-02-16T19:36:18.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.557+0000 7f5d0a9c3700 1 -- 172.21.15.22:0/3945747096 --> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5d04074690 con 0x7f5cec06d8a0 2024-02-16T19:36:18.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.558+0000 7f5d017fa700 1 -- 172.21.15.22:0/3945747096 <== mgr.34104 v2:172.21.15.22:6800/505266753 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f5d04074690 con 0x7f5cec06d8a0 2024-02-16T19:36:18.559 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-02-16T19:36:18.559 INFO:teuthology.orchestra.run.smithi022.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T19:36:18.560 INFO:teuthology.orchestra.run.smithi022.stdout: "in_progress": true, 2024-02-16T19:36:18.560 INFO:teuthology.orchestra.run.smithi022.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T19:36:18.560 INFO:teuthology.orchestra.run.smithi022.stdout: "services_complete": [ 2024-02-16T19:36:18.560 INFO:teuthology.orchestra.run.smithi022.stdout: "crash", 2024-02-16T19:36:18.560 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr", 2024-02-16T19:36:18.560 INFO:teuthology.orchestra.run.smithi022.stdout: "mon" 2024-02-16T19:36:18.560 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-02-16T19:36:18.560 INFO:teuthology.orchestra.run.smithi022.stdout: "progress": "9/23 daemons upgraded", 2024-02-16T19:36:18.560 INFO:teuthology.orchestra.run.smithi022.stdout: "message": "Currently upgrading osd daemons", 2024-02-16T19:36:18.560 INFO:teuthology.orchestra.run.smithi022.stdout: "is_paused": false 2024-02-16T19:36:18.560 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-02-16T19:36:18.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.560+0000 7f5d0a9c3700 1 -- 172.21.15.22:0/3945747096 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f5cec06d8a0 msgr2=0x7f5cec06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:18.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.560+0000 7f5d0a9c3700 1 --2- 172.21.15.22:0/3945747096 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f5cec06d8a0 0x7f5cec06fd50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5cf4007900 tx=0x7f5cf4008040 comp rx=0 tx=0).stop 2024-02-16T19:36:18.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.560+0000 7f5d0a9c3700 1 -- 172.21.15.22:0/3945747096 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d04100e80 msgr2=0x7f5d04073590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:18.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.560+0000 7f5d0a9c3700 1 --2- 172.21.15.22:0/3945747096 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d04100e80 0x7f5d04073590 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5cf80092e0 tx=0x7f5cf800be20 comp rx=0 tx=0).stop 2024-02-16T19:36:18.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.560+0000 7f5d0a9c3700 1 -- 172.21.15.22:0/3945747096 shutdown_connections 2024-02-16T19:36:18.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.560+0000 7f5d0a9c3700 1 --2- 172.21.15.22:0/3945747096 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f5cec06d8a0 0x7f5cec06fd50 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:18.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.560+0000 7f5d0a9c3700 1 --2- 172.21.15.22:0/3945747096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d041004e0 0x7f5d04073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:18.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.560+0000 7f5d0a9c3700 1 --2- 172.21.15.22:0/3945747096 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d04100e80 0x7f5d04073590 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:18.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.560+0000 7f5d0a9c3700 1 -- 172.21.15.22:0/3945747096 >> 172.21.15.22:0/3945747096 conn(0x7f5d040fbda0 msgr2=0x7f5d040fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:18.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.560+0000 7f5d0a9c3700 1 -- 172.21.15.22:0/3945747096 shutdown_connections 2024-02-16T19:36:18.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.560+0000 7f5d0a9c3700 1 -- 172.21.15.22:0/3945747096 wait complete. 2024-02-16T19:36:18.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.665+0000 7f233d11d700 1 -- 172.21.15.22:0/1434922679 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2338100e80 msgr2=0x7f2338104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:18.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.665+0000 7f233d11d700 1 --2- 172.21.15.22:0/1434922679 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2338100e80 0x7f2338104e50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2320009910 tx=0x7f2320009c20 comp rx=0 tx=0).stop 2024-02-16T19:36:18.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.665+0000 7f233d11d700 1 -- 172.21.15.22:0/1434922679 shutdown_connections 2024-02-16T19:36:18.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.665+0000 7f233d11d700 1 --2- 172.21.15.22:0/1434922679 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2338100e80 0x7f2338104e50 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:18.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.665+0000 7f233d11d700 1 --2- 172.21.15.22:0/1434922679 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23381004e0 0x7f23381008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:18.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.665+0000 7f233d11d700 1 -- 172.21.15.22:0/1434922679 >> 172.21.15.22:0/1434922679 conn(0x7f23380fbda0 msgr2=0x7f23380fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:18.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.665+0000 7f233d11d700 1 -- 172.21.15.22:0/1434922679 shutdown_connections 2024-02-16T19:36:18.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.665+0000 7f233d11d700 1 -- 172.21.15.22:0/1434922679 wait complete. 2024-02-16T19:36:18.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.666+0000 7f233d11d700 1 Processor -- start 2024-02-16T19:36:18.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.666+0000 7f233d11d700 1 -- start start 2024-02-16T19:36:18.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.666+0000 7f233d11d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f23381004e0 0x7f2338195f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:18.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.666+0000 7f233d11d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2338100e80 0x7f2338196450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:18.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.666+0000 7f233d11d700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2338196b30 con 0x7f23381004e0 2024-02-16T19:36:18.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.666+0000 7f233d11d700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f233819a8c0 con 0x7f2338100e80 2024-02-16T19:36:18.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.667+0000 7f2336d9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f23381004e0 0x7f2338195f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:18.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.667+0000 7f2336d9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f23381004e0 0x7f2338195f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42644/0 (socket says 172.21.15.22:42644) 2024-02-16T19:36:18.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.667+0000 7f2336d9d700 1 -- 172.21.15.22:0/3716132995 learned_addr learned my addr 172.21.15.22:0/3716132995 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:36:18.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.667+0000 7f233659c700 1 --2- 172.21.15.22:0/3716132995 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2338100e80 0x7f2338196450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:18.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.667+0000 7f2336d9d700 1 -- 172.21.15.22:0/3716132995 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2338100e80 msgr2=0x7f2338196450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:18.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.667+0000 7f2336d9d700 1 --2- 172.21.15.22:0/3716132995 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2338100e80 0x7f2338196450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:18.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.667+0000 7f2336d9d700 1 -- 172.21.15.22:0/3716132995 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23200095f0 con 0x7f23381004e0 2024-02-16T19:36:18.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.667+0000 7f2336d9d700 1 --2- 172.21.15.22:0/3716132995 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f23381004e0 0x7f2338195f10 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f233000d8d0 tx=0x7f233000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:18.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.667+0000 7f232ffff700 1 -- 172.21.15.22:0/3716132995 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23300098a0 con 0x7f23381004e0 2024-02-16T19:36:18.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.667+0000 7f232ffff700 1 -- 172.21.15.22:0/3716132995 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2330010460 con 0x7f23381004e0 2024-02-16T19:36:18.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.667+0000 7f233d11d700 1 -- 172.21.15.22:0/3716132995 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f233819aba0 con 0x7f23381004e0 2024-02-16T19:36:18.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.667+0000 7f232ffff700 1 -- 172.21.15.22:0/3716132995 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f233000f5f0 con 0x7f23381004e0 2024-02-16T19:36:18.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.667+0000 7f233d11d700 1 -- 172.21.15.22:0/3716132995 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f233819b0f0 con 0x7f23381004e0 2024-02-16T19:36:18.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.668+0000 7f233d11d700 1 -- 172.21.15.22:0/3716132995 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2338108850 con 0x7f23381004e0 2024-02-16T19:36:18.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.669+0000 7f232ffff700 1 -- 172.21.15.22:0/3716132995 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f2330009a00 con 0x7f23381004e0 2024-02-16T19:36:18.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.669+0000 7f232ffff700 1 --2- 172.21.15.22:0/3716132995 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f232406d8a0 0x7f232406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:18.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.669+0000 7f233659c700 1 --2- 172.21.15.22:0/3716132995 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f232406d8a0 0x7f232406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:18.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.669+0000 7f232ffff700 1 -- 172.21.15.22:0/3716132995 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f233008d470 con 0x7f23381004e0 2024-02-16T19:36:18.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.670+0000 7f233659c700 1 --2- 172.21.15.22:0/3716132995 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f232406d8a0 0x7f232406fd50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2338197530 tx=0x7f232001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:18.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.674+0000 7f232ffff700 1 -- 172.21.15.22:0/3716132995 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2330056bb0 con 0x7f23381004e0 2024-02-16T19:36:18.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.965+0000 7f233d11d700 1 -- 172.21.15.22:0/3716132995 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f233804f2a0 con 0x7f23381004e0 2024-02-16T19:36:18.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.966+0000 7f232ffff700 1 -- 172.21.15.22:0/3716132995 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f233005a270 con 0x7f23381004e0 2024-02-16T19:36:18.968 INFO:teuthology.orchestra.run.smithi022.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-02-16T19:36:18.968 INFO:teuthology.orchestra.run.smithi022.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-02-16T19:36:18.968 INFO:teuthology.orchestra.run.smithi022.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-02-16T19:36:18.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.968+0000 7f233d11d700 1 -- 172.21.15.22:0/3716132995 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f232406d8a0 msgr2=0x7f232406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:18.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.968+0000 7f233d11d700 1 --2- 172.21.15.22:0/3716132995 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f232406d8a0 0x7f232406fd50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2338197530 tx=0x7f232001a040 comp rx=0 tx=0).stop 2024-02-16T19:36:18.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.968+0000 7f233d11d700 1 -- 172.21.15.22:0/3716132995 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f23381004e0 msgr2=0x7f2338195f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:18.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.968+0000 7f233d11d700 1 --2- 172.21.15.22:0/3716132995 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f23381004e0 0x7f2338195f10 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f233000d8d0 tx=0x7f233000dbe0 comp rx=0 tx=0).stop 2024-02-16T19:36:18.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.968+0000 7f233d11d700 1 -- 172.21.15.22:0/3716132995 shutdown_connections 2024-02-16T19:36:18.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.968+0000 7f233d11d700 1 --2- 172.21.15.22:0/3716132995 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f232406d8a0 0x7f232406fd50 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:18.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.968+0000 7f233d11d700 1 --2- 172.21.15.22:0/3716132995 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f23381004e0 0x7f2338195f10 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:18.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.968+0000 7f233d11d700 1 --2- 172.21.15.22:0/3716132995 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2338100e80 0x7f2338196450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:18.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.968+0000 7f233d11d700 1 -- 172.21.15.22:0/3716132995 >> 172.21.15.22:0/3716132995 conn(0x7f23380fbda0 msgr2=0x7f23380fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:18.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.968+0000 7f233d11d700 1 -- 172.21.15.22:0/3716132995 shutdown_connections 2024-02-16T19:36:18.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:18.968+0000 7f233d11d700 1 -- 172.21.15.22:0/3716132995 wait complete. 2024-02-16T19:36:19.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:18 smithi117 ceph-mon[67870]: pgmap v118: 65 pgs: 65 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 341 KiB/s, 0 objects/s recovering 2024-02-16T19:36:19.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:18 smithi117 ceph-mon[67870]: from='client.? 172.21.15.22:0/1212997489' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:36:19.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:18 smithi117 ceph-mon[67870]: from='client.? 172.21.15.22:0/4044425044' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:36:19.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:18 smithi117 ceph-mon[67870]: from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:36:19.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:18 smithi022 ceph-mon[88798]: pgmap v118: 65 pgs: 65 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 341 KiB/s, 0 objects/s recovering 2024-02-16T19:36:19.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:18 smithi022 ceph-mon[88798]: from='client.? 172.21.15.22:0/1212997489' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:36:19.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:18 smithi022 ceph-mon[88798]: from='client.? 172.21.15.22:0/4044425044' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:36:19.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:18 smithi022 ceph-mon[88798]: from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:36:20.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:19 smithi117 ceph-mon[67870]: from='client.? 172.21.15.22:0/3716132995' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T19:36:20.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:19 smithi022 ceph-mon[88798]: from='client.? 172.21.15.22:0/3716132995' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T19:36:21.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:20 smithi117 ceph-mon[67870]: pgmap v119: 65 pgs: 65 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:36:21.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:20 smithi022 ceph-mon[88798]: pgmap v119: 65 pgs: 65 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:36:22.045 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:21 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-02-16T19:36:22.046 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:21 smithi117 ceph-mon[67870]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-02-16T19:36:22.046 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:21 smithi117 ceph-mon[67870]: Upgrade: osd.3 is safe to restart 2024-02-16T19:36:22.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:21 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-02-16T19:36:22.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:21 smithi022 ceph-mon[88798]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-02-16T19:36:22.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:21 smithi022 ceph-mon[88798]: Upgrade: osd.3 is safe to restart 2024-02-16T19:36:23.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:22 smithi117 ceph-mon[67870]: pgmap v120: 65 pgs: 65 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:36:23.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:22 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:36:23.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:22 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-16T19:36:23.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:22 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:36:23.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:22 smithi022 ceph-mon[88798]: pgmap v120: 65 pgs: 65 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:36:23.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:22 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:36:23.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:22 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-16T19:36:23.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:22 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:36:24.053 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:23 smithi117 ceph-mon[67870]: Upgrade: Updating osd.3 2024-02-16T19:36:24.053 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:23 smithi117 ceph-mon[67870]: Deploying daemon osd.3 on smithi117 2024-02-16T19:36:24.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:23 smithi022 ceph-mon[88798]: Upgrade: Updating osd.3 2024-02-16T19:36:24.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:23 smithi022 ceph-mon[88798]: Deploying daemon osd.3 on smithi117 2024-02-16T19:36:25.016 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:24 smithi117 ceph-mon[67870]: pgmap v121: 65 pgs: 65 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:36:25.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:24 smithi022 ceph-mon[88798]: pgmap v121: 65 pgs: 65 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:36:26.311 INFO:journalctl@ceph.osd.3.smithi117.stdout:Feb 16 19:36:26 smithi117 systemd[1]: Stopping Ceph osd.3 for 2a1346e6-cd00-11ee-95ba-87774f69a715... 2024-02-16T19:36:26.637 INFO:journalctl@ceph.osd.3.smithi117.stdout:Feb 16 19:36:26 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-3[41233]: 2024-02-16T19:36:26.309+0000 7fc74a621700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-16T19:36:26.637 INFO:journalctl@ceph.osd.3.smithi117.stdout:Feb 16 19:36:26 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-3[41233]: 2024-02-16T19:36:26.309+0000 7fc74a621700 -1 osd.3 74 *** Got signal Terminated *** 2024-02-16T19:36:26.637 INFO:journalctl@ceph.osd.3.smithi117.stdout:Feb 16 19:36:26 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-3[41233]: 2024-02-16T19:36:26.309+0000 7fc74a621700 -1 osd.3 74 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-16T19:36:27.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:26 smithi117 ceph-mon[67870]: pgmap v122: 65 pgs: 65 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:36:27.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:26 smithi117 ceph-mon[67870]: osd.3 marked itself down and dead 2024-02-16T19:36:27.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:26 smithi022 ceph-mon[88798]: pgmap v122: 65 pgs: 65 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:36:27.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:26 smithi022 ceph-mon[88798]: osd.3 marked itself down and dead 2024-02-16T19:36:27.637 INFO:journalctl@ceph.osd.3.smithi117.stdout:Feb 16 19:36:27 smithi117 podman[77151]: 2024-02-16 19:36:27.383862616 +0000 UTC m=+1.190387849 container died 9c0ce9a981cfad2666d9db0dd26ed88ca5610432bcad992efe9ea0043b5270c8 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-3, org.label-schema.vendor=CentOS, GIT_REPO=https://github.com/ceph/ceph-container.git, RELEASE=HEAD, ceph=True, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.29.1, org.label-schema.build-date=20231212, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=-18.2.0, GIT_BRANCH=HEAD) 2024-02-16T19:36:28.141 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:27 smithi117 ceph-mon[67870]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-16T19:36:28.141 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:27 smithi117 ceph-mon[67870]: osdmap e75: 6 total, 5 up, 6 in 2024-02-16T19:36:28.141 INFO:journalctl@ceph.osd.3.smithi117.stdout:Feb 16 19:36:27 smithi117 podman[77151]: 2024-02-16 19:36:27.738277517 +0000 UTC m=+1.544802748 container cleanup 9c0ce9a981cfad2666d9db0dd26ed88ca5610432bcad992efe9ea0043b5270c8 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-3, RELEASE=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20231212, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, io.buildah.version=1.29.1, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.schema-version=1.0, ceph=True) 2024-02-16T19:36:28.141 INFO:journalctl@ceph.osd.3.smithi117.stdout:Feb 16 19:36:27 smithi117 bash[77151]: ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-3 2024-02-16T19:36:28.141 INFO:journalctl@ceph.osd.3.smithi117.stdout:Feb 16 19:36:27 smithi117 podman[77171]: 2024-02-16 19:36:27.906288017 +0000 UTC m=+0.522427456 container remove 9c0ce9a981cfad2666d9db0dd26ed88ca5610432bcad992efe9ea0043b5270c8 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-3, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.29.1, RELEASE=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.build-date=20231212, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-02-16T19:36:28.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:27 smithi022 ceph-mon[88798]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-16T19:36:28.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:27 smithi022 ceph-mon[88798]: osdmap e75: 6 total, 5 up, 6 in 2024-02-16T19:36:28.611 INFO:journalctl@ceph.osd.3.smithi117.stdout:Feb 16 19:36:28 smithi117 podman[77262]: 2024-02-16 19:36:28.340815432 +0000 UTC m=+0.108058080 container create 67e3e3b4128c1a3c790df3a1ce32b1486c11aa359fa15b7aabcdd5533bf0b69e (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-3-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, io.buildah.version=1.33.5) 2024-02-16T19:36:28.612 INFO:journalctl@ceph.osd.3.smithi117.stdout:Feb 16 19:36:28 smithi117 podman[77262]: 2024-02-16 19:36:28.25060343 +0000 UTC m=+0.017846049 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T19:36:28.612 INFO:journalctl@ceph.osd.3.smithi117.stdout:Feb 16 19:36:28 smithi117 podman[77262]: 2024-02-16 19:36:28.611246496 +0000 UTC m=+0.378489113 container init 67e3e3b4128c1a3c790df3a1ce32b1486c11aa359fa15b7aabcdd5533bf0b69e (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-3-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-16T19:36:28.871 INFO:journalctl@ceph.osd.3.smithi117.stdout:Feb 16 19:36:28 smithi117 podman[77262]: 2024-02-16 19:36:28.614969545 +0000 UTC m=+0.382212157 container start 67e3e3b4128c1a3c790df3a1ce32b1486c11aa359fa15b7aabcdd5533bf0b69e (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-3-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-16T19:36:28.871 INFO:journalctl@ceph.osd.3.smithi117.stdout:Feb 16 19:36:28 smithi117 podman[77262]: 2024-02-16 19:36:28.701340896 +0000 UTC m=+0.468583532 container attach 67e3e3b4128c1a3c790df3a1ce32b1486c11aa359fa15b7aabcdd5533bf0b69e (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-3-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240214) 2024-02-16T19:36:29.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:28 smithi117 ceph-mon[67870]: pgmap v124: 65 pgs: 14 stale+active+clean, 51 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:36:29.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:28 smithi117 ceph-mon[67870]: osdmap e76: 6 total, 5 up, 6 in 2024-02-16T19:36:29.137 INFO:journalctl@ceph.osd.3.smithi117.stdout:Feb 16 19:36:28 smithi117 podman[77262]: 2024-02-16 19:36:28.911022597 +0000 UTC m=+0.678265207 container died 67e3e3b4128c1a3c790df3a1ce32b1486c11aa359fa15b7aabcdd5533bf0b69e (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-3-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-02-16T19:36:29.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:28 smithi022 ceph-mon[88798]: pgmap v124: 65 pgs: 14 stale+active+clean, 51 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:36:29.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:28 smithi022 ceph-mon[88798]: osdmap e76: 6 total, 5 up, 6 in 2024-02-16T19:36:29.887 INFO:journalctl@ceph.osd.3.smithi117.stdout:Feb 16 19:36:29 smithi117 podman[77289]: 2024-02-16 19:36:29.496544137 +0000 UTC m=+0.580033863 container remove 67e3e3b4128c1a3c790df3a1ce32b1486c11aa359fa15b7aabcdd5533bf0b69e (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-3-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2) 2024-02-16T19:36:29.888 INFO:journalctl@ceph.osd.3.smithi117.stdout:Feb 16 19:36:29 smithi117 systemd[1]: ceph-2a1346e6-cd00-11ee-95ba-87774f69a715@osd.3.service: Deactivated successfully. 2024-02-16T19:36:29.888 INFO:journalctl@ceph.osd.3.smithi117.stdout:Feb 16 19:36:29 smithi117 systemd[1]: Stopped Ceph osd.3 for 2a1346e6-cd00-11ee-95ba-87774f69a715. 2024-02-16T19:36:29.888 INFO:journalctl@ceph.osd.3.smithi117.stdout:Feb 16 19:36:29 smithi117 systemd[1]: ceph-2a1346e6-cd00-11ee-95ba-87774f69a715@osd.3.service: Consumed 1min 796ms CPU time. 2024-02-16T19:36:29.888 INFO:journalctl@ceph.osd.3.smithi117.stdout:Feb 16 19:36:29 smithi117 systemd[1]: Starting Ceph osd.3 for 2a1346e6-cd00-11ee-95ba-87774f69a715... 2024-02-16T19:36:30.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:29 smithi022 ceph-mon[88798]: Health check failed: Degraded data redundancy: 5/267 objects degraded (1.873%), 2 pgs degraded (PG_DEGRADED) 2024-02-16T19:36:30.242 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:29 smithi117 ceph-mon[67870]: Health check failed: Degraded data redundancy: 5/267 objects degraded (1.873%), 2 pgs degraded (PG_DEGRADED) 2024-02-16T19:36:30.242 INFO:journalctl@ceph.osd.3.smithi117.stdout:Feb 16 19:36:29 smithi117 podman[77395]: 2024-02-16 19:36:29.835659146 +0000 UTC m=+0.019123450 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T19:36:30.242 INFO:journalctl@ceph.osd.3.smithi117.stdout:Feb 16 19:36:29 smithi117 podman[77395]: 2024-02-16 19:36:29.947898468 +0000 UTC m=+0.131362769 container create 49a8a13cbf91f07c026a5822ea6c06aea40d2c4863d3b3d2dc20ae2d0760033a (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-3-activate, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-02-16T19:36:30.242 INFO:journalctl@ceph.osd.3.smithi117.stdout:Feb 16 19:36:30 smithi117 podman[77395]: 2024-02-16 19:36:30.167227192 +0000 UTC m=+0.350691495 container init 49a8a13cbf91f07c026a5822ea6c06aea40d2c4863d3b3d2dc20ae2d0760033a (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-3-activate, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.schema-version=1.0) 2024-02-16T19:36:30.243 INFO:journalctl@ceph.osd.3.smithi117.stdout:Feb 16 19:36:30 smithi117 podman[77395]: 2024-02-16 19:36:30.170245242 +0000 UTC m=+0.353709546 container start 49a8a13cbf91f07c026a5822ea6c06aea40d2c4863d3b3d2dc20ae2d0760033a (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-3-activate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-16T19:36:30.243 INFO:journalctl@ceph.osd.3.smithi117.stdout:Feb 16 19:36:30 smithi117 podman[77395]: 2024-02-16 19:36:30.242018706 +0000 UTC m=+0.425483017 container attach 49a8a13cbf91f07c026a5822ea6c06aea40d2c4863d3b3d2dc20ae2d0760033a (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-3-activate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_CLEAN=True) 2024-02-16T19:36:30.887 INFO:journalctl@ceph.osd.3.smithi117.stdout:Feb 16 19:36:30 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-3-activate[77410]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-02-16T19:36:30.887 INFO:journalctl@ceph.osd.3.smithi117.stdout:Feb 16 19:36:30 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-3-activate[77410]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-02-16T19:36:30.888 INFO:journalctl@ceph.osd.3.smithi117.stdout:Feb 16 19:36:30 smithi117 bash[77395]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-02-16T19:36:30.888 INFO:journalctl@ceph.osd.3.smithi117.stdout:Feb 16 19:36:30 smithi117 bash[77395]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-02-16T19:36:30.888 INFO:journalctl@ceph.osd.3.smithi117.stdout:Feb 16 19:36:30 smithi117 bash[77395]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-02-16T19:36:30.888 INFO:journalctl@ceph.osd.3.smithi117.stdout:Feb 16 19:36:30 smithi117 bash[77395]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-02-16T19:36:30.888 INFO:journalctl@ceph.osd.3.smithi117.stdout:Feb 16 19:36:30 smithi117 bash[77395]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-02-16T19:36:30.888 INFO:journalctl@ceph.osd.3.smithi117.stdout:Feb 16 19:36:30 smithi117 bash[77395]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-02-16T19:36:30.888 INFO:journalctl@ceph.osd.3.smithi117.stdout:Feb 16 19:36:30 smithi117 bash[77395]: --> ceph-volume raw activate successful for osd ID: 3 2024-02-16T19:36:30.888 INFO:journalctl@ceph.osd.3.smithi117.stdout:Feb 16 19:36:30 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-3-activate[77410]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-02-16T19:36:30.888 INFO:journalctl@ceph.osd.3.smithi117.stdout:Feb 16 19:36:30 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-3-activate[77410]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-02-16T19:36:30.888 INFO:journalctl@ceph.osd.3.smithi117.stdout:Feb 16 19:36:30 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-3-activate[77410]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-02-16T19:36:30.888 INFO:journalctl@ceph.osd.3.smithi117.stdout:Feb 16 19:36:30 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-3-activate[77410]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-02-16T19:36:30.889 INFO:journalctl@ceph.osd.3.smithi117.stdout:Feb 16 19:36:30 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-3-activate[77410]: --> ceph-volume raw activate successful for osd ID: 3 2024-02-16T19:36:30.889 INFO:journalctl@ceph.osd.3.smithi117.stdout:Feb 16 19:36:30 smithi117 podman[77395]: 2024-02-16 19:36:30.636628604 +0000 UTC m=+0.820092904 container died 49a8a13cbf91f07c026a5822ea6c06aea40d2c4863d3b3d2dc20ae2d0760033a (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-3-activate, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-02-16T19:36:31.204 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:30 smithi117 ceph-mon[67870]: pgmap v126: 65 pgs: 4 active+undersized, 2 active+undersized+degraded, 12 stale+active+clean, 47 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 5/267 objects degraded (1.873%) 2024-02-16T19:36:31.204 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:30 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:36:31.204 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:30 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:36:31.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:30 smithi022 ceph-mon[88798]: pgmap v126: 65 pgs: 4 active+undersized, 2 active+undersized+degraded, 12 stale+active+clean, 47 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 5/267 objects degraded (1.873%) 2024-02-16T19:36:31.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:30 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:36:31.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:30 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:36:31.637 INFO:journalctl@ceph.osd.3.smithi117.stdout:Feb 16 19:36:31 smithi117 podman[77531]: 2024-02-16 19:36:31.203927688 +0000 UTC m=+0.561635020 container remove 49a8a13cbf91f07c026a5822ea6c06aea40d2c4863d3b3d2dc20ae2d0760033a (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-3-activate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-02-16T19:36:31.638 INFO:journalctl@ceph.osd.3.smithi117.stdout:Feb 16 19:36:31 smithi117 podman[77595]: 2024-02-16 19:36:31.413177404 +0000 UTC m=+0.111767435 container create 28d7d8519e64e1b196771c34bcfae200203c0798fcff2b240b0884403916aaf6 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-3, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-02-16T19:36:31.638 INFO:journalctl@ceph.osd.3.smithi117.stdout:Feb 16 19:36:31 smithi117 podman[77595]: 2024-02-16 19:36:31.319190288 +0000 UTC m=+0.017780282 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T19:36:32.137 INFO:journalctl@ceph.osd.3.smithi117.stdout:Feb 16 19:36:31 smithi117 podman[77595]: 2024-02-16 19:36:31.882900914 +0000 UTC m=+0.581490912 container init 28d7d8519e64e1b196771c34bcfae200203c0798fcff2b240b0884403916aaf6 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-3, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, ceph=True, GIT_CLEAN=True) 2024-02-16T19:36:32.137 INFO:journalctl@ceph.osd.3.smithi117.stdout:Feb 16 19:36:31 smithi117 podman[77595]: 2024-02-16 19:36:31.887079247 +0000 UTC m=+0.585669237 container start 28d7d8519e64e1b196771c34bcfae200203c0798fcff2b240b0884403916aaf6 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-3, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, RELEASE=wip-batrick-testing-20240215.160715-f78a58c) 2024-02-16T19:36:32.137 INFO:journalctl@ceph.osd.3.smithi117.stdout:Feb 16 19:36:31 smithi117 ceph-osd[77613]: -- 172.21.15.117:0/2739927202 <== mon.1 v2:172.21.15.117:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5588837931e0 con 0x558883765000 2024-02-16T19:36:32.137 INFO:journalctl@ceph.osd.3.smithi117.stdout:Feb 16 19:36:31 smithi117 bash[77595]: 28d7d8519e64e1b196771c34bcfae200203c0798fcff2b240b0884403916aaf6 2024-02-16T19:36:32.137 INFO:journalctl@ceph.osd.3.smithi117.stdout:Feb 16 19:36:31 smithi117 systemd[1]: Started Ceph osd.3 for 2a1346e6-cd00-11ee-95ba-87774f69a715. 2024-02-16T19:36:32.871 INFO:journalctl@ceph.osd.3.smithi117.stdout:Feb 16 19:36:32 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-3[77609]: 2024-02-16T19:36:32.430+0000 7f8ce50a97c0 -1 Falling back to public interface 2024-02-16T19:36:33.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:33 smithi117 ceph-mon[67870]: pgmap v127: 65 pgs: 18 active+undersized, 15 active+undersized+degraded, 32 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 54/267 objects degraded (20.225%) 2024-02-16T19:36:33.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:33 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:36:33.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:33 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:36:33.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:33 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:36:33.467 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:33 smithi022 ceph-mon[88798]: pgmap v127: 65 pgs: 18 active+undersized, 15 active+undersized+degraded, 32 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 54/267 objects degraded (20.225%) 2024-02-16T19:36:33.467 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:33 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:36:33.467 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:33 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:36:33.468 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:33 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:36:35.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:35 smithi117 ceph-mon[67870]: pgmap v128: 65 pgs: 18 active+undersized, 15 active+undersized+degraded, 32 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 54/267 objects degraded (20.225%) 2024-02-16T19:36:35.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:35 smithi022 ceph-mon[88798]: pgmap v128: 65 pgs: 18 active+undersized, 15 active+undersized+degraded, 32 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 54/267 objects degraded (20.225%) 2024-02-16T19:36:36.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:36 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:36:36.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:36 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:36:36.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:36 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:36:36.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:36 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:36:37.141 INFO:journalctl@ceph.osd.3.smithi117.stdout:Feb 16 19:36:36 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-3[77609]: 2024-02-16T19:36:36.986+0000 7f8ce50a97c0 -1 osd.3 0 read_superblock omap replica is missing. 2024-02-16T19:36:37.637 INFO:journalctl@ceph.osd.3.smithi117.stdout:Feb 16 19:36:37 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-3[77609]: 2024-02-16T19:36:37.321+0000 7f8ce50a97c0 -1 osd.3 74 log_to_monitors true 2024-02-16T19:36:37.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:37 smithi117 ceph-mon[67870]: pgmap v129: 65 pgs: 18 active+undersized, 15 active+undersized+degraded, 32 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 54/267 objects degraded (20.225%) 2024-02-16T19:36:37.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:37 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:36:37.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:37 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:36:37.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:37 smithi022 ceph-mon[88798]: pgmap v129: 65 pgs: 18 active+undersized, 15 active+undersized+degraded, 32 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 54/267 objects degraded (20.225%) 2024-02-16T19:36:37.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:37 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:36:37.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:37 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:36:38.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:38 smithi117 ceph-mon[67870]: Health check update: Degraded data redundancy: 54/267 objects degraded (20.225%), 15 pgs degraded (PG_DEGRADED) 2024-02-16T19:36:38.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:38 smithi117 ceph-mon[67870]: from='osd.3 [v2:172.21.15.117:6800/3703365528,v1:172.21.15.117:6801/3703365528]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-16T19:36:38.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:38 smithi117 ceph-mon[67870]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-16T19:36:38.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:38 smithi022 ceph-mon[88798]: Health check update: Degraded data redundancy: 54/267 objects degraded (20.225%), 15 pgs degraded (PG_DEGRADED) 2024-02-16T19:36:38.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:38 smithi022 ceph-mon[88798]: from='osd.3 [v2:172.21.15.117:6800/3703365528,v1:172.21.15.117:6801/3703365528]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-16T19:36:38.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:38 smithi022 ceph-mon[88798]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-16T19:36:39.524 INFO:journalctl@ceph.osd.3.smithi117.stdout:Feb 16 19:36:39 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-3[77609]: 2024-02-16T19:36:39.159+0000 7f8cd24f9700 -1 osd.3 74 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-16T19:36:39.524 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:39 smithi117 ceph-mon[67870]: pgmap v130: 65 pgs: 18 active+undersized, 15 active+undersized+degraded, 32 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 54/267 objects degraded (20.225%) 2024-02-16T19:36:39.524 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:39 smithi117 ceph-mon[67870]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-16T19:36:39.524 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:39 smithi117 ceph-mon[67870]: from='osd.3 [v2:172.21.15.117:6800/3703365528,v1:172.21.15.117:6801/3703365528]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-02-16T19:36:39.525 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:39 smithi117 ceph-mon[67870]: osdmap e77: 6 total, 5 up, 6 in 2024-02-16T19:36:39.525 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:39 smithi117 ceph-mon[67870]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-02-16T19:36:39.525 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:39 smithi117 ceph-mon[67870]: from='osd.3 ' entity='osd.3' 2024-02-16T19:36:39.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:39 smithi022 ceph-mon[88798]: pgmap v130: 65 pgs: 18 active+undersized, 15 active+undersized+degraded, 32 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 54/267 objects degraded (20.225%) 2024-02-16T19:36:39.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:39 smithi022 ceph-mon[88798]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-16T19:36:39.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:39 smithi022 ceph-mon[88798]: from='osd.3 [v2:172.21.15.117:6800/3703365528,v1:172.21.15.117:6801/3703365528]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-02-16T19:36:39.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:39 smithi022 ceph-mon[88798]: osdmap e77: 6 total, 5 up, 6 in 2024-02-16T19:36:39.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:39 smithi022 ceph-mon[88798]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-02-16T19:36:39.716 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:39 smithi022 ceph-mon[88798]: from='osd.3 ' entity='osd.3' 2024-02-16T19:36:41.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:40 smithi117 ceph-mon[67870]: pgmap v132: 65 pgs: 18 active+undersized, 15 active+undersized+degraded, 32 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 54/267 objects degraded (20.225%) 2024-02-16T19:36:41.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:40 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:36:41.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:40 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:36:41.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:40 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:36:41.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:40 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:36:41.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:40 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:36:41.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:40 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:36:41.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:40 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:36:41.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:40 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:36:41.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:40 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:36:41.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:40 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-02-16T19:36:41.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:40 smithi117 ceph-mon[67870]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-02-16T19:36:41.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:40 smithi117 ceph-mon[67870]: Upgrade: unsafe to stop osd(s) at this time (14 PGs are or would become offline) 2024-02-16T19:36:41.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:40 smithi117 ceph-mon[67870]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-16T19:36:41.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:40 smithi117 ceph-mon[67870]: osd.3 [v2:172.21.15.117:6800/3703365528,v1:172.21.15.117:6801/3703365528] boot 2024-02-16T19:36:41.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:40 smithi117 ceph-mon[67870]: osdmap e78: 6 total, 6 up, 6 in 2024-02-16T19:36:41.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:40 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-16T19:36:41.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:40 smithi022 ceph-mon[88798]: pgmap v132: 65 pgs: 18 active+undersized, 15 active+undersized+degraded, 32 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 54/267 objects degraded (20.225%) 2024-02-16T19:36:41.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:40 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:36:41.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:40 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:36:41.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:40 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:36:41.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:40 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:36:41.216 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:40 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:36:41.216 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:40 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:36:41.216 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:40 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:36:41.216 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:40 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:36:41.216 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:40 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:36:41.216 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:40 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-02-16T19:36:41.216 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:40 smithi022 ceph-mon[88798]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-02-16T19:36:41.216 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:40 smithi022 ceph-mon[88798]: Upgrade: unsafe to stop osd(s) at this time (14 PGs are or would become offline) 2024-02-16T19:36:41.216 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:40 smithi022 ceph-mon[88798]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-16T19:36:41.216 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:40 smithi022 ceph-mon[88798]: osd.3 [v2:172.21.15.117:6800/3703365528,v1:172.21.15.117:6801/3703365528] boot 2024-02-16T19:36:41.216 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:40 smithi022 ceph-mon[88798]: osdmap e78: 6 total, 6 up, 6 in 2024-02-16T19:36:41.216 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:40 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-16T19:36:42.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:42 smithi022 ceph-mon[88798]: osdmap e79: 6 total, 6 up, 6 in 2024-02-16T19:36:42.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:42 smithi117 ceph-mon[67870]: osdmap e79: 6 total, 6 up, 6 in 2024-02-16T19:36:43.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:43 smithi022 ceph-mon[88798]: pgmap v135: 65 pgs: 10 peering, 10 active+undersized, 8 active+undersized+degraded, 37 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 27/267 objects degraded (10.112%) 2024-02-16T19:36:43.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:43 smithi022 ceph-mon[88798]: Health check update: Degraded data redundancy: 27/267 objects degraded (10.112%), 8 pgs degraded (PG_DEGRADED) 2024-02-16T19:36:43.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:43 smithi117 ceph-mon[67870]: pgmap v135: 65 pgs: 10 peering, 10 active+undersized, 8 active+undersized+degraded, 37 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 27/267 objects degraded (10.112%) 2024-02-16T19:36:43.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:43 smithi117 ceph-mon[67870]: Health check update: Degraded data redundancy: 27/267 objects degraded (10.112%), 8 pgs degraded (PG_DEGRADED) 2024-02-16T19:36:45.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:45 smithi022 ceph-mon[88798]: pgmap v136: 65 pgs: 10 peering, 3 active+undersized, 1 active+undersized+degraded, 51 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 3/267 objects degraded (1.124%) 2024-02-16T19:36:45.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:45 smithi117 ceph-mon[67870]: pgmap v136: 65 pgs: 10 peering, 3 active+undersized, 1 active+undersized+degraded, 51 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 3/267 objects degraded (1.124%) 2024-02-16T19:36:47.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:46 smithi117 ceph-mon[67870]: pgmap v137: 65 pgs: 3 peering, 62 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:36:47.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:46 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:36:47.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:46 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:36:47.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:46 smithi117 ceph-mon[67870]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 3/267 objects degraded (1.124%), 1 pg degraded) 2024-02-16T19:36:47.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:46 smithi022 ceph-mon[88798]: pgmap v137: 65 pgs: 3 peering, 62 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:36:47.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:46 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:36:47.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:46 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:36:47.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:46 smithi022 ceph-mon[88798]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 3/267 objects degraded (1.124%), 1 pg degraded) 2024-02-16T19:36:49.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.089+0000 7f7119a37700 1 -- 172.21.15.22:0/3398839961 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71140687a0 msgr2=0x7f7114068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:49.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.089+0000 7f7119a37700 1 --2- 172.21.15.22:0/3398839961 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71140687a0 0x7f7114068b70 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7104009910 tx=0x7f7104009c20 comp rx=0 tx=0).stop 2024-02-16T19:36:49.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.089+0000 7f7119a37700 1 -- 172.21.15.22:0/3398839961 shutdown_connections 2024-02-16T19:36:49.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.089+0000 7f7119a37700 1 --2- 172.21.15.22:0/3398839961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7114069140 0x7f7114104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:49.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.089+0000 7f7119a37700 1 --2- 172.21.15.22:0/3398839961 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71140687a0 0x7f7114068b70 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:49.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.089+0000 7f7119a37700 1 -- 172.21.15.22:0/3398839961 >> 172.21.15.22:0/3398839961 conn(0x7f7114075130 msgr2=0x7f7114075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:49.091 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:48 smithi022 ceph-mon[88798]: pgmap v138: 65 pgs: 65 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:36:49.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.089+0000 7f7119a37700 1 -- 172.21.15.22:0/3398839961 shutdown_connections 2024-02-16T19:36:49.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.089+0000 7f7119a37700 1 -- 172.21.15.22:0/3398839961 wait complete. 2024-02-16T19:36:49.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.090+0000 7f7119a37700 1 Processor -- start 2024-02-16T19:36:49.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.090+0000 7f7119a37700 1 -- start start 2024-02-16T19:36:49.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.090+0000 7f7119a37700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71140687a0 0x7f71140ffca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:49.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.090+0000 7f7119a37700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7114069140 0x7f71141001e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:49.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.090+0000 7f7119a37700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7114100720 con 0x7f7114069140 2024-02-16T19:36:49.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.090+0000 7f7119a37700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7114100860 con 0x7f71140687a0 2024-02-16T19:36:49.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.090+0000 7f71127fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7114069140 0x7f71141001e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:49.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.090+0000 7f71127fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7114069140 0x7f71141001e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38172/0 (socket says 172.21.15.22:38172) 2024-02-16T19:36:49.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.090+0000 7f71127fc700 1 -- 172.21.15.22:0/1569969491 learned_addr learned my addr 172.21.15.22:0/1569969491 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:36:49.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.090+0000 7f7112ffd700 1 --2- 172.21.15.22:0/1569969491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71140687a0 0x7f71140ffca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:49.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.091+0000 7f71127fc700 1 -- 172.21.15.22:0/1569969491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71140687a0 msgr2=0x7f71140ffca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:49.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.091+0000 7f71127fc700 1 --2- 172.21.15.22:0/1569969491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71140687a0 0x7f71140ffca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:49.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.091+0000 7f71127fc700 1 -- 172.21.15.22:0/1569969491 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71040095f0 con 0x7f7114069140 2024-02-16T19:36:49.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.091+0000 7f71127fc700 1 --2- 172.21.15.22:0/1569969491 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7114069140 0x7f71141001e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f710800c960 tx=0x7f710800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:49.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.091+0000 7f7118a35700 1 -- 172.21.15.22:0/1569969491 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7108007a70 con 0x7f7114069140 2024-02-16T19:36:49.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.091+0000 7f7119a37700 1 -- 172.21.15.22:0/1569969491 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71141a20b0 con 0x7f7114069140 2024-02-16T19:36:49.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.091+0000 7f7118a35700 1 -- 172.21.15.22:0/1569969491 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7108007bd0 con 0x7f7114069140 2024-02-16T19:36:49.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.091+0000 7f7118a35700 1 -- 172.21.15.22:0/1569969491 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7108018630 con 0x7f7114069140 2024-02-16T19:36:49.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.091+0000 7f7119a37700 1 -- 172.21.15.22:0/1569969491 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71141a2410 con 0x7f7114069140 2024-02-16T19:36:49.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.092+0000 7f7119a37700 1 -- 172.21.15.22:0/1569969491 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7114108770 con 0x7f7114069140 2024-02-16T19:36:49.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.093+0000 7f7118a35700 1 -- 172.21.15.22:0/1569969491 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f710800f450 con 0x7f7114069140 2024-02-16T19:36:49.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.093+0000 7f7118a35700 1 --2- 172.21.15.22:0/1569969491 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f70fc06d8a0 0x7f70fc06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:49.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.093+0000 7f7118a35700 1 -- 172.21.15.22:0/1569969491 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f710808d5a0 con 0x7f7114069140 2024-02-16T19:36:49.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.093+0000 7f7112ffd700 1 --2- 172.21.15.22:0/1569969491 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f70fc06d8a0 0x7f70fc06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:49.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.094+0000 7f7112ffd700 1 --2- 172.21.15.22:0/1569969491 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f70fc06d8a0 0x7f70fc06fd50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7104006010 tx=0x7f710401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:49.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.098+0000 7f7118a35700 1 -- 172.21.15.22:0/1569969491 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7108056ce0 con 0x7f7114069140 2024-02-16T19:36:49.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:48 smithi117 ceph-mon[67870]: pgmap v138: 65 pgs: 65 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:36:49.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.327+0000 7f7119a37700 1 -- 172.21.15.22:0/1569969491 --> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f71141012e0 con 0x7f70fc06d8a0 2024-02-16T19:36:49.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.328+0000 7f7118a35700 1 -- 172.21.15.22:0/1569969491 <== mgr.34104 v2:172.21.15.22:6800/505266753 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f71141012e0 con 0x7f70fc06d8a0 2024-02-16T19:36:49.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.330+0000 7f7119a37700 1 -- 172.21.15.22:0/1569969491 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f70fc06d8a0 msgr2=0x7f70fc06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:49.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.330+0000 7f7119a37700 1 --2- 172.21.15.22:0/1569969491 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f70fc06d8a0 0x7f70fc06fd50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7104006010 tx=0x7f710401a040 comp rx=0 tx=0).stop 2024-02-16T19:36:49.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.330+0000 7f7119a37700 1 -- 172.21.15.22:0/1569969491 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7114069140 msgr2=0x7f71141001e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:49.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.330+0000 7f7119a37700 1 --2- 172.21.15.22:0/1569969491 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7114069140 0x7f71141001e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f710800c960 tx=0x7f710800cc70 comp rx=0 tx=0).stop 2024-02-16T19:36:49.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.330+0000 7f7119a37700 1 -- 172.21.15.22:0/1569969491 shutdown_connections 2024-02-16T19:36:49.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.330+0000 7f7119a37700 1 --2- 172.21.15.22:0/1569969491 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f70fc06d8a0 0x7f70fc06fd50 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:49.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.330+0000 7f7119a37700 1 --2- 172.21.15.22:0/1569969491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71140687a0 0x7f71140ffca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:49.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.330+0000 7f7119a37700 1 --2- 172.21.15.22:0/1569969491 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7114069140 0x7f71141001e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:49.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.330+0000 7f7119a37700 1 -- 172.21.15.22:0/1569969491 >> 172.21.15.22:0/1569969491 conn(0x7f7114075130 msgr2=0x7f71140fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:49.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.331+0000 7f7119a37700 1 -- 172.21.15.22:0/1569969491 shutdown_connections 2024-02-16T19:36:49.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.331+0000 7f7119a37700 1 -- 172.21.15.22:0/1569969491 wait complete. 2024-02-16T19:36:49.342 INFO:teuthology.orchestra.run.smithi022.stdout:true 2024-02-16T19:36:49.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.437+0000 7f234f3f1700 1 -- 172.21.15.22:0/2050075220 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f23481004e0 msgr2=0x7f23481008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:49.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.437+0000 7f234f3f1700 1 --2- 172.21.15.22:0/2050075220 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f23481004e0 0x7f23481008b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2344009910 tx=0x7f2344009c20 comp rx=0 tx=0).stop 2024-02-16T19:36:49.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.438+0000 7f234f3f1700 1 -- 172.21.15.22:0/2050075220 shutdown_connections 2024-02-16T19:36:49.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.438+0000 7f234f3f1700 1 --2- 172.21.15.22:0/2050075220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2348100e80 0x7f2348104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:49.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.438+0000 7f234f3f1700 1 --2- 172.21.15.22:0/2050075220 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f23481004e0 0x7f23481008b0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:49.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.438+0000 7f234f3f1700 1 -- 172.21.15.22:0/2050075220 >> 172.21.15.22:0/2050075220 conn(0x7f23480fbda0 msgr2=0x7f23480fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:49.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.438+0000 7f234f3f1700 1 -- 172.21.15.22:0/2050075220 shutdown_connections 2024-02-16T19:36:49.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.438+0000 7f234f3f1700 1 -- 172.21.15.22:0/2050075220 wait complete. 2024-02-16T19:36:49.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.438+0000 7f234f3f1700 1 Processor -- start 2024-02-16T19:36:49.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.438+0000 7f234f3f1700 1 -- start start 2024-02-16T19:36:49.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.438+0000 7f234f3f1700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f23481004e0 0x7f2348195ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:49.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.438+0000 7f234f3f1700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2348100e80 0x7f2348196430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:49.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.438+0000 7f234f3f1700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2348196b10 con 0x7f23481004e0 2024-02-16T19:36:49.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.438+0000 7f234f3f1700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f234819a8a0 con 0x7f2348100e80 2024-02-16T19:36:49.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.439+0000 7f234d18d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f23481004e0 0x7f2348195ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:49.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.439+0000 7f234d18d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f23481004e0 0x7f2348195ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38194/0 (socket says 172.21.15.22:38194) 2024-02-16T19:36:49.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.439+0000 7f234d18d700 1 -- 172.21.15.22:0/3480948548 learned_addr learned my addr 172.21.15.22:0/3480948548 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:36:49.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.439+0000 7f234c98c700 1 --2- 172.21.15.22:0/3480948548 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2348100e80 0x7f2348196430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:49.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.439+0000 7f234d18d700 1 -- 172.21.15.22:0/3480948548 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2348100e80 msgr2=0x7f2348196430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:49.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.439+0000 7f234d18d700 1 --2- 172.21.15.22:0/3480948548 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2348100e80 0x7f2348196430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:49.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.439+0000 7f234d18d700 1 -- 172.21.15.22:0/3480948548 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23440095f0 con 0x7f23481004e0 2024-02-16T19:36:49.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.439+0000 7f234d18d700 1 --2- 172.21.15.22:0/3480948548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f23481004e0 0x7f2348195ef0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f23440092e0 tx=0x7f234400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:49.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.439+0000 7f233e7fc700 1 -- 172.21.15.22:0/3480948548 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f234401d070 con 0x7f23481004e0 2024-02-16T19:36:49.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.439+0000 7f233e7fc700 1 -- 172.21.15.22:0/3480948548 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2344005e00 con 0x7f23481004e0 2024-02-16T19:36:49.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.439+0000 7f234f3f1700 1 -- 172.21.15.22:0/3480948548 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f234819ab20 con 0x7f23481004e0 2024-02-16T19:36:49.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.439+0000 7f233e7fc700 1 -- 172.21.15.22:0/3480948548 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2344017400 con 0x7f23481004e0 2024-02-16T19:36:49.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.439+0000 7f234f3f1700 1 -- 172.21.15.22:0/3480948548 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f234819b010 con 0x7f23481004e0 2024-02-16T19:36:49.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.440+0000 7f234f3f1700 1 -- 172.21.15.22:0/3480948548 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23481087f0 con 0x7f23481004e0 2024-02-16T19:36:49.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.441+0000 7f233e7fc700 1 -- 172.21.15.22:0/3480948548 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f234400f6f0 con 0x7f23481004e0 2024-02-16T19:36:49.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.441+0000 7f233e7fc700 1 --2- 172.21.15.22:0/3480948548 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f233406d8a0 0x7f233406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:49.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.441+0000 7f233e7fc700 1 -- 172.21.15.22:0/3480948548 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f234408ea70 con 0x7f23481004e0 2024-02-16T19:36:49.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.441+0000 7f234c98c700 1 --2- 172.21.15.22:0/3480948548 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f233406d8a0 0x7f233406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:49.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.442+0000 7f234c98c700 1 --2- 172.21.15.22:0/3480948548 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f233406d8a0 0x7f233406fd50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2348197510 tx=0x7f233800b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:49.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.446+0000 7f233e7fc700 1 -- 172.21.15.22:0/3480948548 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f23440581b0 con 0x7f23481004e0 2024-02-16T19:36:49.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.672+0000 7f234f3f1700 1 -- 172.21.15.22:0/3480948548 --> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f234819b2f0 con 0x7f233406d8a0 2024-02-16T19:36:49.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.674+0000 7f233e7fc700 1 -- 172.21.15.22:0/3480948548 <== mgr.34104 v2:172.21.15.22:6800/505266753 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f234819b2f0 con 0x7f233406d8a0 2024-02-16T19:36:49.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.676+0000 7f234f3f1700 1 -- 172.21.15.22:0/3480948548 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f233406d8a0 msgr2=0x7f233406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:49.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.676+0000 7f234f3f1700 1 --2- 172.21.15.22:0/3480948548 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f233406d8a0 0x7f233406fd50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2348197510 tx=0x7f233800b410 comp rx=0 tx=0).stop 2024-02-16T19:36:49.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.676+0000 7f234f3f1700 1 -- 172.21.15.22:0/3480948548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f23481004e0 msgr2=0x7f2348195ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:49.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.676+0000 7f234f3f1700 1 --2- 172.21.15.22:0/3480948548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f23481004e0 0x7f2348195ef0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f23440092e0 tx=0x7f234400bd70 comp rx=0 tx=0).stop 2024-02-16T19:36:49.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.677+0000 7f234f3f1700 1 -- 172.21.15.22:0/3480948548 shutdown_connections 2024-02-16T19:36:49.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.677+0000 7f234f3f1700 1 --2- 172.21.15.22:0/3480948548 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f233406d8a0 0x7f233406fd50 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:49.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.677+0000 7f234f3f1700 1 --2- 172.21.15.22:0/3480948548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f23481004e0 0x7f2348195ef0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:49.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.677+0000 7f234f3f1700 1 --2- 172.21.15.22:0/3480948548 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2348100e80 0x7f2348196430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:49.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.677+0000 7f234f3f1700 1 -- 172.21.15.22:0/3480948548 >> 172.21.15.22:0/3480948548 conn(0x7f23480fbda0 msgr2=0x7f23480fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:49.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.677+0000 7f234f3f1700 1 -- 172.21.15.22:0/3480948548 shutdown_connections 2024-02-16T19:36:49.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.677+0000 7f234f3f1700 1 -- 172.21.15.22:0/3480948548 wait complete. 2024-02-16T19:36:49.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.783+0000 7fbde6123700 1 -- 172.21.15.22:0/3175082105 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbde00ff3c0 msgr2=0x7fbde00ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:49.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.783+0000 7fbde6123700 1 --2- 172.21.15.22:0/3175082105 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbde00ff3c0 0x7fbde00ff790 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fbdd0009910 tx=0x7fbdd0009c20 comp rx=0 tx=0).stop 2024-02-16T19:36:49.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.783+0000 7fbde6123700 1 -- 172.21.15.22:0/3175082105 shutdown_connections 2024-02-16T19:36:49.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.783+0000 7fbde6123700 1 --2- 172.21.15.22:0/3175082105 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbde00ffd60 0x7fbde0103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:49.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.783+0000 7fbde6123700 1 --2- 172.21.15.22:0/3175082105 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbde00ff3c0 0x7fbde00ff790 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:49.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.783+0000 7fbde6123700 1 -- 172.21.15.22:0/3175082105 >> 172.21.15.22:0/3175082105 conn(0x7fbde00fac80 msgr2=0x7fbde00fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:49.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.783+0000 7fbde6123700 1 -- 172.21.15.22:0/3175082105 shutdown_connections 2024-02-16T19:36:49.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.783+0000 7fbde6123700 1 -- 172.21.15.22:0/3175082105 wait complete. 2024-02-16T19:36:49.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.784+0000 7fbde6123700 1 Processor -- start 2024-02-16T19:36:49.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.784+0000 7fbde6123700 1 -- start start 2024-02-16T19:36:49.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.784+0000 7fbde6123700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbde00ff3c0 0x7fbde0100dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:49.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.784+0000 7fbde6123700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbde00ffd60 0x7fbde0101310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:49.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.784+0000 7fbde6123700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde0101850 con 0x7fbde00ff3c0 2024-02-16T19:36:49.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.784+0000 7fbde6123700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde0101990 con 0x7fbde00ffd60 2024-02-16T19:36:49.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.784+0000 7fbddf7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbde00ff3c0 0x7fbde0100dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:49.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.784+0000 7fbddf7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbde00ff3c0 0x7fbde0100dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38208/0 (socket says 172.21.15.22:38208) 2024-02-16T19:36:49.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.784+0000 7fbddf7fe700 1 -- 172.21.15.22:0/1310243102 learned_addr learned my addr 172.21.15.22:0/1310243102 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:36:49.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.784+0000 7fbddeffd700 1 --2- 172.21.15.22:0/1310243102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbde00ffd60 0x7fbde0101310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:49.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.785+0000 7fbddf7fe700 1 -- 172.21.15.22:0/1310243102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbde00ffd60 msgr2=0x7fbde0101310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:49.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.785+0000 7fbddf7fe700 1 --2- 172.21.15.22:0/1310243102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbde00ffd60 0x7fbde0101310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:49.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.785+0000 7fbddf7fe700 1 -- 172.21.15.22:0/1310243102 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdd00095f0 con 0x7fbde00ff3c0 2024-02-16T19:36:49.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.785+0000 7fbddeffd700 1 --2- 172.21.15.22:0/1310243102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbde00ffd60 0x7fbde0101310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T19:36:49.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.785+0000 7fbddf7fe700 1 --2- 172.21.15.22:0/1310243102 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbde00ff3c0 0x7fbde0100dd0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fbdd0000c00 tx=0x7fbdd000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:49.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.785+0000 7fbddcff9700 1 -- 172.21.15.22:0/1310243102 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbdd001d070 con 0x7fbde00ff3c0 2024-02-16T19:36:49.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.785+0000 7fbddcff9700 1 -- 172.21.15.22:0/1310243102 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbdd000f460 con 0x7fbde00ff3c0 2024-02-16T19:36:49.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.785+0000 7fbde6123700 1 -- 172.21.15.22:0/1310243102 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbde0101c10 con 0x7fbde00ff3c0 2024-02-16T19:36:49.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.785+0000 7fbddcff9700 1 -- 172.21.15.22:0/1310243102 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbdd0017630 con 0x7fbde00ff3c0 2024-02-16T19:36:49.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.785+0000 7fbde6123700 1 -- 172.21.15.22:0/1310243102 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbde01a2450 con 0x7fbde00ff3c0 2024-02-16T19:36:49.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.786+0000 7fbde6123700 1 -- 172.21.15.22:0/1310243102 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbde004f2a0 con 0x7fbde00ff3c0 2024-02-16T19:36:49.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.786+0000 7fbddcff9700 1 -- 172.21.15.22:0/1310243102 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fbdd000f5d0 con 0x7fbde00ff3c0 2024-02-16T19:36:49.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.787+0000 7fbddcff9700 1 --2- 172.21.15.22:0/1310243102 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fbdc806d8a0 0x7fbdc806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:49.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.787+0000 7fbddcff9700 1 -- 172.21.15.22:0/1310243102 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6704+0+0 (secure 0 0 0) 0x7fbdd008ebb0 con 0x7fbde00ff3c0 2024-02-16T19:36:49.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.787+0000 7fbddeffd700 1 --2- 172.21.15.22:0/1310243102 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fbdc806d8a0 0x7fbdc806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:49.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.787+0000 7fbddeffd700 1 --2- 172.21.15.22:0/1310243102 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fbdc806d8a0 0x7fbdc806fd50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbde01025c0 tx=0x7fbdd4008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:49.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:49.791+0000 7fbddcff9700 1 -- 172.21.15.22:0/1310243102 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbdd00582f0 con 0x7fbde00ff3c0 2024-02-16T19:36:49.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:49 smithi022 ceph-mon[88798]: from='client.34316 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:36:49.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:49 smithi022 ceph-mon[88798]: from='client.34320 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:36:50.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.012+0000 7fbde6123700 1 -- 172.21.15.22:0/1310243102 --> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fbde00fbac0 con 0x7fbdc806d8a0 2024-02-16T19:36:50.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.023+0000 7fbddcff9700 1 -- 172.21.15.22:0/1310243102 <== mgr.34104 v2:172.21.15.22:6800/505266753 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fbde00fbac0 con 0x7fbdc806d8a0 2024-02-16T19:36:50.025 INFO:teuthology.orchestra.run.smithi022.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T19:36:50.025 INFO:teuthology.orchestra.run.smithi022.stdout:alertmanager.smithi022 smithi022 *:9093,9094 running (5m) 48s ago 16m 26.8M - 0.25.0 c8568f914cd2 c89bc76be374 2024-02-16T19:36:50.025 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-exporter.smithi022 smithi022 running (16m) 48s ago 16m 8606k - 18.2.0 dc2bc1663786 3f8bef4349ba 2024-02-16T19:36:50.025 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-exporter.smithi117 smithi117 running (15m) 14s ago 15m 7952k - 18.2.0 dc2bc1663786 59b65a7e309f 2024-02-16T19:36:50.025 INFO:teuthology.orchestra.run.smithi022.stdout:crash.smithi022 smithi022 running (2m) 48s ago 16m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a e0f78bdf43e4 2024-02-16T19:36:50.025 INFO:teuthology.orchestra.run.smithi022.stdout:crash.smithi117 smithi117 running (2m) 14s ago 15m 6669k - 19.0.0-1450-gf78a58c0 f081f847070a bce7c0590813 2024-02-16T19:36:50.025 INFO:teuthology.orchestra.run.smithi022.stdout:grafana.smithi022 smithi022 *:3000 running (4m) 48s ago 15m 81.0M - 9.4.12 a0ea5e3c46d9 fd27d11e9729 2024-02-16T19:36:50.025 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi022.btppwm smithi022 running (10m) 48s ago 10m 155M - 18.2.0 dc2bc1663786 22971fb3a907 2024-02-16T19:36:50.025 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi022.ocukvs smithi022 running (10m) 48s ago 10m 17.1M - 18.2.0 dc2bc1663786 81ee7bd5fd39 2024-02-16T19:36:50.025 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi117.ibmppn smithi117 running (10m) 14s ago 10m 18.5M - 18.2.0 dc2bc1663786 8c606cbeeb86 2024-02-16T19:36:50.025 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi117.orzors smithi117 running (10m) 14s ago 10m 17.5M - 18.2.0 dc2bc1663786 7d2bc4df3feb 2024-02-16T19:36:50.025 INFO:teuthology.orchestra.run.smithi022.stdout:mgr.smithi022.gmdcch smithi022 *:8443,9283,8765 running (7m) 48s ago 17m 504M - 19.0.0-1450-gf78a58c0 f081f847070a 7cb74cc9deea 2024-02-16T19:36:50.026 INFO:teuthology.orchestra.run.smithi022.stdout:mgr.smithi117.cohbtv smithi117 *:8443,9283,8765 running (6m) 14s ago 15m 452M - 19.0.0-1450-gf78a58c0 f081f847070a c825b1152189 2024-02-16T19:36:50.026 INFO:teuthology.orchestra.run.smithi022.stdout:mon.smithi022 smithi022 running (3m) 48s ago 17m 48.4M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 9c84755ae67d 2024-02-16T19:36:50.026 INFO:teuthology.orchestra.run.smithi022.stdout:mon.smithi117 smithi117 running (3m) 14s ago 14m 39.1M 2048M 19.0.0-1450-gf78a58c0 f081f847070a a319ac50f1a9 2024-02-16T19:36:50.026 INFO:teuthology.orchestra.run.smithi022.stdout:node-exporter.smithi022 smithi022 *:9100 running (5m) 48s ago 16m 16.6M - 1.5.0 0da6a335fe13 1f5b58292d27 2024-02-16T19:36:50.026 INFO:teuthology.orchestra.run.smithi022.stdout:node-exporter.smithi117 smithi117 *:9100 running (5m) 14s ago 15m 15.6M - 1.5.0 0da6a335fe13 bb5e8491d5a6 2024-02-16T19:36:50.026 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0 smithi022 running (2m) 48s ago 14m 107M 1217M 19.0.0-1450-gf78a58c0 f081f847070a 002412b3525f 2024-02-16T19:36:50.026 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1 smithi022 running (91s) 48s ago 13m 128M 1217M 19.0.0-1450-gf78a58c0 f081f847070a c3488ce7b2a0 2024-02-16T19:36:50.026 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2 smithi022 running (54s) 48s ago 13m 12.6M 1217M 19.0.0-1450-gf78a58c0 f081f847070a 46138c50f79d 2024-02-16T19:36:50.026 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3 smithi117 running (18s) 14s ago 12m 12.7M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 28d7d8519e64 2024-02-16T19:36:50.026 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4 smithi117 running (12m) 14s ago 12m 213M 2240M 18.2.0 dc2bc1663786 1779c3496f4d 2024-02-16T19:36:50.026 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5 smithi117 running (11m) 14s ago 11m 208M 2240M 18.2.0 dc2bc1663786 fefaabca0638 2024-02-16T19:36:50.026 INFO:teuthology.orchestra.run.smithi022.stdout:prometheus.smithi022 smithi022 *:9095 running (5m) 48s ago 15m 66.9M - 2.43.0 a07b618ecd1d c3b8689d0c97 2024-02-16T19:36:50.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.025+0000 7fbde6123700 1 -- 172.21.15.22:0/1310243102 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fbdc806d8a0 msgr2=0x7fbdc806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:50.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.025+0000 7fbde6123700 1 --2- 172.21.15.22:0/1310243102 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fbdc806d8a0 0x7fbdc806fd50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbde01025c0 tx=0x7fbdd4008040 comp rx=0 tx=0).stop 2024-02-16T19:36:50.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.025+0000 7fbde6123700 1 -- 172.21.15.22:0/1310243102 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbde00ff3c0 msgr2=0x7fbde0100dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:50.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.025+0000 7fbde6123700 1 --2- 172.21.15.22:0/1310243102 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbde00ff3c0 0x7fbde0100dd0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fbdd0000c00 tx=0x7fbdd000be20 comp rx=0 tx=0).stop 2024-02-16T19:36:50.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.025+0000 7fbde6123700 1 -- 172.21.15.22:0/1310243102 shutdown_connections 2024-02-16T19:36:50.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.025+0000 7fbde6123700 1 --2- 172.21.15.22:0/1310243102 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fbdc806d8a0 0x7fbdc806fd50 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:50.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.025+0000 7fbde6123700 1 --2- 172.21.15.22:0/1310243102 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbde00ff3c0 0x7fbde0100dd0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:50.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.025+0000 7fbde6123700 1 --2- 172.21.15.22:0/1310243102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbde00ffd60 0x7fbde0101310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:50.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.025+0000 7fbde6123700 1 -- 172.21.15.22:0/1310243102 >> 172.21.15.22:0/1310243102 conn(0x7fbde00fac80 msgr2=0x7fbde01034e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:50.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.025+0000 7fbde6123700 1 -- 172.21.15.22:0/1310243102 shutdown_connections 2024-02-16T19:36:50.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.025+0000 7fbde6123700 1 -- 172.21.15.22:0/1310243102 wait complete. 2024-02-16T19:36:50.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.130+0000 7f99e5c86700 1 -- 172.21.15.22:0/3248495193 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99e00ffbd0 msgr2=0x7f99e0100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:50.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.130+0000 7f99e5c86700 1 --2- 172.21.15.22:0/3248495193 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99e00ffbd0 0x7f99e0100040 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f99d4009910 tx=0x7f99d4009c20 comp rx=0 tx=0).stop 2024-02-16T19:36:50.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.130+0000 7f99e5c86700 1 -- 172.21.15.22:0/3248495193 shutdown_connections 2024-02-16T19:36:50.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.130+0000 7f99e5c86700 1 --2- 172.21.15.22:0/3248495193 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99e00ffbd0 0x7f99e0100040 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:50.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.130+0000 7f99e5c86700 1 --2- 172.21.15.22:0/3248495193 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99e0104130 0x7f99e0104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:50.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.130+0000 7f99e5c86700 1 -- 172.21.15.22:0/3248495193 >> 172.21.15.22:0/3248495193 conn(0x7f99e0075130 msgr2=0x7f99e0075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:50.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.130+0000 7f99e5c86700 1 -- 172.21.15.22:0/3248495193 shutdown_connections 2024-02-16T19:36:50.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.130+0000 7f99e5c86700 1 -- 172.21.15.22:0/3248495193 wait complete. 2024-02-16T19:36:50.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.131+0000 7f99e5c86700 1 Processor -- start 2024-02-16T19:36:50.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.131+0000 7f99e5c86700 1 -- start start 2024-02-16T19:36:50.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.131+0000 7f99e5c86700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99e00ffbd0 0x7f99e0196c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:50.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.131+0000 7f99e5c86700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99e0104130 0x7f99e01971c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:50.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.131+0000 7f99e5c86700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99e01977e0 con 0x7f99e00ffbd0 2024-02-16T19:36:50.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.131+0000 7f99e5c86700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99e0197950 con 0x7f99e0104130 2024-02-16T19:36:50.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.131+0000 7f99df7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99e00ffbd0 0x7f99e0196c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:50.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.131+0000 7f99df7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99e00ffbd0 0x7f99e0196c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38226/0 (socket says 172.21.15.22:38226) 2024-02-16T19:36:50.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.131+0000 7f99df7fe700 1 -- 172.21.15.22:0/1383243840 learned_addr learned my addr 172.21.15.22:0/1383243840 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:36:50.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.131+0000 7f99deffd700 1 --2- 172.21.15.22:0/1383243840 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99e0104130 0x7f99e01971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:50.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.131+0000 7f99df7fe700 1 -- 172.21.15.22:0/1383243840 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99e0104130 msgr2=0x7f99e01971c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:50.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.131+0000 7f99df7fe700 1 --2- 172.21.15.22:0/1383243840 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99e0104130 0x7f99e01971c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:50.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.131+0000 7f99df7fe700 1 -- 172.21.15.22:0/1383243840 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99d40095f0 con 0x7f99e00ffbd0 2024-02-16T19:36:50.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.132+0000 7f99df7fe700 1 --2- 172.21.15.22:0/1383243840 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99e00ffbd0 0x7f99e0196c80 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f99d000e920 tx=0x7f99d000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:50.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.132+0000 7f99dcff9700 1 -- 172.21.15.22:0/1383243840 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f99d000c9d0 con 0x7f99e00ffbd0 2024-02-16T19:36:50.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.132+0000 7f99dcff9700 1 -- 172.21.15.22:0/1383243840 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f99d0010430 con 0x7f99e00ffbd0 2024-02-16T19:36:50.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.132+0000 7f99e5c86700 1 -- 172.21.15.22:0/1383243840 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99e019fcf0 con 0x7f99e00ffbd0 2024-02-16T19:36:50.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.132+0000 7f99dcff9700 1 -- 172.21.15.22:0/1383243840 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f99d0018630 con 0x7f99e00ffbd0 2024-02-16T19:36:50.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.132+0000 7f99e5c86700 1 -- 172.21.15.22:0/1383243840 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99e01a01b0 con 0x7f99e00ffbd0 2024-02-16T19:36:50.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.133+0000 7f99e5c86700 1 -- 172.21.15.22:0/1383243840 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99e004f2a0 con 0x7f99e00ffbd0 2024-02-16T19:36:50.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.133+0000 7f99dcff9700 1 -- 172.21.15.22:0/1383243840 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f99d00105a0 con 0x7f99e00ffbd0 2024-02-16T19:36:50.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.134+0000 7f99dcff9700 1 --2- 172.21.15.22:0/1383243840 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f99c806d8a0 0x7f99c806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:50.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.134+0000 7f99deffd700 1 --2- 172.21.15.22:0/1383243840 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f99c806d8a0 0x7f99c806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:50.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.134+0000 7f99dcff9700 1 -- 172.21.15.22:0/1383243840 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f99d0014070 con 0x7f99e00ffbd0 2024-02-16T19:36:50.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.134+0000 7f99deffd700 1 --2- 172.21.15.22:0/1383243840 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f99c806d8a0 0x7f99c806fd50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f99e0075ad0 tx=0x7f99d4005a90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:50.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:49 smithi117 ceph-mon[67870]: from='client.34316 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:36:50.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:49 smithi117 ceph-mon[67870]: from='client.34320 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:36:50.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.139+0000 7f99dcff9700 1 -- 172.21.15.22:0/1383243840 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f99d0052d20 con 0x7f99e00ffbd0 2024-02-16T19:36:50.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.430+0000 7f99e5c86700 1 -- 172.21.15.22:0/1383243840 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f99e01a0490 con 0x7f99e00ffbd0 2024-02-16T19:36:50.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.430+0000 7f99dcff9700 1 -- 172.21.15.22:0/1383243840 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+791 (secure 0 0 0) 0x7f99d0056fe0 con 0x7f99e00ffbd0 2024-02-16T19:36:50.432 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-02-16T19:36:50.432 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": { 2024-02-16T19:36:50.432 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T19:36:50.432 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:36:50.433 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": { 2024-02-16T19:36:50.433 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T19:36:50.433 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:36:50.433 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": { 2024-02-16T19:36:50.433 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2, 2024-02-16T19:36:50.433 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 4 2024-02-16T19:36:50.433 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:36:50.433 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": { 2024-02-16T19:36:50.433 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-02-16T19:36:50.433 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:36:50.433 INFO:teuthology.orchestra.run.smithi022.stdout: "overall": { 2024-02-16T19:36:50.433 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6, 2024-02-16T19:36:50.434 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 8 2024-02-16T19:36:50.434 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-02-16T19:36:50.434 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-02-16T19:36:50.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.433+0000 7f99e5c86700 1 -- 172.21.15.22:0/1383243840 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f99c806d8a0 msgr2=0x7f99c806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:50.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.433+0000 7f99e5c86700 1 --2- 172.21.15.22:0/1383243840 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f99c806d8a0 0x7f99c806fd50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f99e0075ad0 tx=0x7f99d4005a90 comp rx=0 tx=0).stop 2024-02-16T19:36:50.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.433+0000 7f99e5c86700 1 -- 172.21.15.22:0/1383243840 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99e00ffbd0 msgr2=0x7f99e0196c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:50.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.433+0000 7f99e5c86700 1 --2- 172.21.15.22:0/1383243840 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99e00ffbd0 0x7f99e0196c80 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f99d000e920 tx=0x7f99d000ec30 comp rx=0 tx=0).stop 2024-02-16T19:36:50.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.433+0000 7f99e5c86700 1 -- 172.21.15.22:0/1383243840 shutdown_connections 2024-02-16T19:36:50.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.433+0000 7f99e5c86700 1 --2- 172.21.15.22:0/1383243840 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f99c806d8a0 0x7f99c806fd50 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:50.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.433+0000 7f99e5c86700 1 --2- 172.21.15.22:0/1383243840 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99e00ffbd0 0x7f99e0196c80 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:50.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.433+0000 7f99e5c86700 1 --2- 172.21.15.22:0/1383243840 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99e0104130 0x7f99e01971c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:50.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.433+0000 7f99e5c86700 1 -- 172.21.15.22:0/1383243840 >> 172.21.15.22:0/1383243840 conn(0x7f99e0075130 msgr2=0x7f99e00feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:50.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.433+0000 7f99e5c86700 1 -- 172.21.15.22:0/1383243840 shutdown_connections 2024-02-16T19:36:50.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.433+0000 7f99e5c86700 1 -- 172.21.15.22:0/1383243840 wait complete. 2024-02-16T19:36:50.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.539+0000 7ff64a73a700 1 -- 172.21.15.22:0/486265070 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff6440ff3c0 msgr2=0x7ff6440ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:50.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.539+0000 7ff64a73a700 1 --2- 172.21.15.22:0/486265070 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff6440ff3c0 0x7ff6440ff790 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff634009910 tx=0x7ff634009c20 comp rx=0 tx=0).stop 2024-02-16T19:36:50.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.539+0000 7ff64a73a700 1 -- 172.21.15.22:0/486265070 shutdown_connections 2024-02-16T19:36:50.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.539+0000 7ff64a73a700 1 --2- 172.21.15.22:0/486265070 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6440ffd60 0x7ff644103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:50.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.539+0000 7ff64a73a700 1 --2- 172.21.15.22:0/486265070 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff6440ff3c0 0x7ff6440ff790 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:50.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.539+0000 7ff64a73a700 1 -- 172.21.15.22:0/486265070 >> 172.21.15.22:0/486265070 conn(0x7ff6440fac60 msgr2=0x7ff6440fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:50.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.539+0000 7ff64a73a700 1 -- 172.21.15.22:0/486265070 shutdown_connections 2024-02-16T19:36:50.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.539+0000 7ff64a73a700 1 -- 172.21.15.22:0/486265070 wait complete. 2024-02-16T19:36:50.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.539+0000 7ff64a73a700 1 Processor -- start 2024-02-16T19:36:50.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.540+0000 7ff64a73a700 1 -- start start 2024-02-16T19:36:50.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.540+0000 7ff64a73a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6440ff3c0 0x7ff644194e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:50.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.540+0000 7ff64a73a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff6440ffd60 0x7ff6441953d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:50.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.540+0000 7ff64a73a700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff644195ab0 con 0x7ff6440ffd60 2024-02-16T19:36:50.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.540+0000 7ff64a73a700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff644199840 con 0x7ff6440ff3c0 2024-02-16T19:36:50.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.540+0000 7ff6437fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff6440ffd60 0x7ff6441953d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:50.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.540+0000 7ff6437fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff6440ffd60 0x7ff6441953d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38246/0 (socket says 172.21.15.22:38246) 2024-02-16T19:36:50.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.540+0000 7ff6437fe700 1 -- 172.21.15.22:0/239474017 learned_addr learned my addr 172.21.15.22:0/239474017 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:36:50.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.540+0000 7ff643fff700 1 --2- 172.21.15.22:0/239474017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6440ff3c0 0x7ff644194e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:50.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.540+0000 7ff6437fe700 1 -- 172.21.15.22:0/239474017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6440ff3c0 msgr2=0x7ff644194e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:50.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.540+0000 7ff6437fe700 1 --2- 172.21.15.22:0/239474017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6440ff3c0 0x7ff644194e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:50.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.540+0000 7ff6437fe700 1 -- 172.21.15.22:0/239474017 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6340095f0 con 0x7ff6440ffd60 2024-02-16T19:36:50.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.541+0000 7ff6437fe700 1 --2- 172.21.15.22:0/239474017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff6440ffd60 0x7ff6441953d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff63800b480 tx=0x7ff63800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:50.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.541+0000 7ff6417fa700 1 -- 172.21.15.22:0/239474017 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff6380118c0 con 0x7ff6440ffd60 2024-02-16T19:36:50.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.541+0000 7ff64a73a700 1 -- 172.21.15.22:0/239474017 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff644199b20 con 0x7ff6440ffd60 2024-02-16T19:36:50.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.541+0000 7ff6417fa700 1 -- 172.21.15.22:0/239474017 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff638010380 con 0x7ff6440ffd60 2024-02-16T19:36:50.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.541+0000 7ff6417fa700 1 -- 172.21.15.22:0/239474017 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff63800f4f0 con 0x7ff6440ffd60 2024-02-16T19:36:50.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.541+0000 7ff64a73a700 1 -- 172.21.15.22:0/239474017 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff64419a070 con 0x7ff6440ffd60 2024-02-16T19:36:50.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.542+0000 7ff64a73a700 1 -- 172.21.15.22:0/239474017 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6441076d0 con 0x7ff6440ffd60 2024-02-16T19:36:50.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.542+0000 7ff6417fa700 1 -- 172.21.15.22:0/239474017 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7ff6380104f0 con 0x7ff6440ffd60 2024-02-16T19:36:50.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.543+0000 7ff6417fa700 1 --2- 172.21.15.22:0/239474017 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7ff62c06d8a0 0x7ff62c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:50.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.543+0000 7ff6417fa700 1 -- 172.21.15.22:0/239474017 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6704+0+0 (secure 0 0 0) 0x7ff63808d1c0 con 0x7ff6440ffd60 2024-02-16T19:36:50.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.543+0000 7ff643fff700 1 --2- 172.21.15.22:0/239474017 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7ff62c06d8a0 0x7ff62c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:50.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.543+0000 7ff643fff700 1 --2- 172.21.15.22:0/239474017 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7ff62c06d8a0 0x7ff62c06fd50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff63400b5c0 tx=0x7ff634005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:50.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.547+0000 7ff6417fa700 1 -- 172.21.15.22:0/239474017 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff638056900 con 0x7ff6440ffd60 2024-02-16T19:36:50.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.802+0000 7ff64a73a700 1 -- 172.21.15.22:0/239474017 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff64404f2a0 con 0x7ff6440ffd60 2024-02-16T19:36:50.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.803+0000 7ff6417fa700 1 -- 172.21.15.22:0/239474017 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1845 (secure 0 0 0) 0x7ff638059fc0 con 0x7ff6440ffd60 2024-02-16T19:36:50.805 INFO:teuthology.orchestra.run.smithi022.stdout:e15 2024-02-16T19:36:50.805 INFO:teuthology.orchestra.run.smithi022.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T19:36:50.805 INFO:teuthology.orchestra.run.smithi022.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:36:50.805 INFO:teuthology.orchestra.run.smithi022.stdout:legacy client fscid: 1 2024-02-16T19:36:50.805 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:36:50.805 INFO:teuthology.orchestra.run.smithi022.stdout:Filesystem 'cephfs' (1) 2024-02-16T19:36:50.805 INFO:teuthology.orchestra.run.smithi022.stdout:fs_name cephfs 2024-02-16T19:36:50.805 INFO:teuthology.orchestra.run.smithi022.stdout:epoch 15 2024-02-16T19:36:50.805 INFO:teuthology.orchestra.run.smithi022.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T19:36:50.806 INFO:teuthology.orchestra.run.smithi022.stdout:created 2024-02-16T19:26:17.873644+0000 2024-02-16T19:36:50.806 INFO:teuthology.orchestra.run.smithi022.stdout:modified 2024-02-16T19:26:35.943353+0000 2024-02-16T19:36:50.806 INFO:teuthology.orchestra.run.smithi022.stdout:tableserver 0 2024-02-16T19:36:50.806 INFO:teuthology.orchestra.run.smithi022.stdout:root 0 2024-02-16T19:36:50.806 INFO:teuthology.orchestra.run.smithi022.stdout:session_timeout 60 2024-02-16T19:36:50.806 INFO:teuthology.orchestra.run.smithi022.stdout:session_autoclose 300 2024-02-16T19:36:50.806 INFO:teuthology.orchestra.run.smithi022.stdout:max_file_size 1099511627776 2024-02-16T19:36:50.806 INFO:teuthology.orchestra.run.smithi022.stdout:max_xattr_size 65536 2024-02-16T19:36:50.806 INFO:teuthology.orchestra.run.smithi022.stdout:required_client_features {} 2024-02-16T19:36:50.806 INFO:teuthology.orchestra.run.smithi022.stdout:last_failure 0 2024-02-16T19:36:50.807 INFO:teuthology.orchestra.run.smithi022.stdout:last_failure_osd_epoch 47 2024-02-16T19:36:50.807 INFO:teuthology.orchestra.run.smithi022.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:36:50.807 INFO:teuthology.orchestra.run.smithi022.stdout:max_mds 1 2024-02-16T19:36:50.807 INFO:teuthology.orchestra.run.smithi022.stdout:in 0 2024-02-16T19:36:50.807 INFO:teuthology.orchestra.run.smithi022.stdout:up {0=14490} 2024-02-16T19:36:50.807 INFO:teuthology.orchestra.run.smithi022.stdout:failed 2024-02-16T19:36:50.807 INFO:teuthology.orchestra.run.smithi022.stdout:damaged 2024-02-16T19:36:50.807 INFO:teuthology.orchestra.run.smithi022.stdout:stopped 2024-02-16T19:36:50.807 INFO:teuthology.orchestra.run.smithi022.stdout:data_pools [3] 2024-02-16T19:36:50.807 INFO:teuthology.orchestra.run.smithi022.stdout:metadata_pool 2 2024-02-16T19:36:50.807 INFO:teuthology.orchestra.run.smithi022.stdout:inline_data enabled 2024-02-16T19:36:50.807 INFO:teuthology.orchestra.run.smithi022.stdout:balancer 2024-02-16T19:36:50.807 INFO:teuthology.orchestra.run.smithi022.stdout:bal_rank_mask -1 2024-02-16T19:36:50.807 INFO:teuthology.orchestra.run.smithi022.stdout:standby_count_wanted 1 2024-02-16T19:36:50.808 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.cephfs.smithi022.btppwm{0:14490} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.22:6828/3597685751,v1:172.21.15.22:6829/3597685751] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:36:50.808 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:36:50.808 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:36:50.808 INFO:teuthology.orchestra.run.smithi022.stdout:Standby daemons: 2024-02-16T19:36:50.808 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:36:50.808 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.cephfs.smithi022.ocukvs{-1:14500} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.22:6826/242865843,v1:172.21.15.22:6827/242865843] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:36:50.808 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.cephfs.smithi117.ibmppn{-1:24259} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.117:6824/565408102,v1:172.21.15.117:6825/565408102] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:36:50.808 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.cephfs.smithi117.orzors{-1:24265} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.117:6826/3122342028,v1:172.21.15.117:6827/3122342028] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:36:50.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.805+0000 7ff64a73a700 1 -- 172.21.15.22:0/239474017 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7ff62c06d8a0 msgr2=0x7ff62c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:50.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.805+0000 7ff64a73a700 1 --2- 172.21.15.22:0/239474017 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7ff62c06d8a0 0x7ff62c06fd50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff63400b5c0 tx=0x7ff634005960 comp rx=0 tx=0).stop 2024-02-16T19:36:50.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.805+0000 7ff64a73a700 1 -- 172.21.15.22:0/239474017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff6440ffd60 msgr2=0x7ff6441953d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:50.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.805+0000 7ff64a73a700 1 --2- 172.21.15.22:0/239474017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff6440ffd60 0x7ff6441953d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff63800b480 tx=0x7ff63800b840 comp rx=0 tx=0).stop 2024-02-16T19:36:50.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.805+0000 7ff64a73a700 1 -- 172.21.15.22:0/239474017 shutdown_connections 2024-02-16T19:36:50.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.805+0000 7ff64a73a700 1 --2- 172.21.15.22:0/239474017 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7ff62c06d8a0 0x7ff62c06fd50 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:50.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.805+0000 7ff64a73a700 1 --2- 172.21.15.22:0/239474017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6440ff3c0 0x7ff644194e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:50.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.805+0000 7ff64a73a700 1 --2- 172.21.15.22:0/239474017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff6440ffd60 0x7ff6441953d0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:50.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.805+0000 7ff64a73a700 1 -- 172.21.15.22:0/239474017 >> 172.21.15.22:0/239474017 conn(0x7ff6440fac60 msgr2=0x7ff6440fc350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:50.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.805+0000 7ff64a73a700 1 -- 172.21.15.22:0/239474017 shutdown_connections 2024-02-16T19:36:50.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.805+0000 7ff64a73a700 1 -- 172.21.15.22:0/239474017 wait complete. 2024-02-16T19:36:50.810 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 15 2024-02-16T19:36:50.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.913+0000 7fd60b895700 1 -- 172.21.15.22:0/1228215582 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd604100460 msgr2=0x7fd604100830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:50.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.913+0000 7fd60b895700 1 --2- 172.21.15.22:0/1228215582 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd604100460 0x7fd604100830 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd5f400b0d0 tx=0x7fd5f400b3e0 comp rx=0 tx=0).stop 2024-02-16T19:36:50.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.913+0000 7fd60b895700 1 -- 172.21.15.22:0/1228215582 shutdown_connections 2024-02-16T19:36:50.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.913+0000 7fd60b895700 1 --2- 172.21.15.22:0/1228215582 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd604100d70 0x7fd6041094d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:50.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.913+0000 7fd60b895700 1 --2- 172.21.15.22:0/1228215582 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd604100460 0x7fd604100830 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:50.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.913+0000 7fd60b895700 1 -- 172.21.15.22:0/1228215582 >> 172.21.15.22:0/1228215582 conn(0x7fd6040748b0 msgr2=0x7fd604074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:50.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.913+0000 7fd60b895700 1 -- 172.21.15.22:0/1228215582 shutdown_connections 2024-02-16T19:36:50.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.913+0000 7fd60b895700 1 -- 172.21.15.22:0/1228215582 wait complete. 2024-02-16T19:36:50.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.913+0000 7fd60b895700 1 Processor -- start 2024-02-16T19:36:50.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.913+0000 7fd60b895700 1 -- start start 2024-02-16T19:36:50.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.914+0000 7fd60b895700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd604100460 0x7fd604196d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:50.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.914+0000 7fd60b895700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd604100d70 0x7fd604197270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:50.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.914+0000 7fd60b895700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd604197950 con 0x7fd604100460 2024-02-16T19:36:50.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.914+0000 7fd60b895700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd604197ac0 con 0x7fd604100d70 2024-02-16T19:36:50.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.914+0000 7fd609631700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd604100460 0x7fd604196d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:50.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.914+0000 7fd609631700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd604100460 0x7fd604196d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38260/0 (socket says 172.21.15.22:38260) 2024-02-16T19:36:50.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.914+0000 7fd609631700 1 -- 172.21.15.22:0/2637089347 learned_addr learned my addr 172.21.15.22:0/2637089347 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:36:50.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.914+0000 7fd608e30700 1 --2- 172.21.15.22:0/2637089347 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd604100d70 0x7fd604197270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:50.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.914+0000 7fd609631700 1 -- 172.21.15.22:0/2637089347 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd604100d70 msgr2=0x7fd604197270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:50.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.914+0000 7fd609631700 1 --2- 172.21.15.22:0/2637089347 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd604100d70 0x7fd604197270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:50.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.914+0000 7fd609631700 1 -- 172.21.15.22:0/2637089347 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5f4009d00 con 0x7fd604100460 2024-02-16T19:36:50.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.914+0000 7fd609631700 1 --2- 172.21.15.22:0/2637089347 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd604100460 0x7fd604196d30 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd5f400b0a0 tx=0x7fd5f400cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:50.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.914+0000 7fd6027fc700 1 -- 172.21.15.22:0/2637089347 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd5f400e040 con 0x7fd604100460 2024-02-16T19:36:50.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.914+0000 7fd6027fc700 1 -- 172.21.15.22:0/2637089347 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd5f4003de0 con 0x7fd604100460 2024-02-16T19:36:50.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.914+0000 7fd6027fc700 1 -- 172.21.15.22:0/2637089347 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd5f401d910 con 0x7fd604100460 2024-02-16T19:36:50.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.914+0000 7fd60b895700 1 -- 172.21.15.22:0/2637089347 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd60419fda0 con 0x7fd604100460 2024-02-16T19:36:50.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.915+0000 7fd60b895700 1 -- 172.21.15.22:0/2637089347 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6041a01c0 con 0x7fd604100460 2024-02-16T19:36:50.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.915+0000 7fd60b895700 1 -- 172.21.15.22:0/2637089347 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd604106bd0 con 0x7fd604100460 2024-02-16T19:36:50.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.916+0000 7fd6027fc700 1 -- 172.21.15.22:0/2637089347 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fd5f4004820 con 0x7fd604100460 2024-02-16T19:36:50.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.916+0000 7fd6027fc700 1 --2- 172.21.15.22:0/2637089347 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fd5ec06d8a0 0x7fd5ec06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:50.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.917+0000 7fd608e30700 1 --2- 172.21.15.22:0/2637089347 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fd5ec06d8a0 0x7fd5ec06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:50.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.917+0000 7fd6027fc700 1 -- 172.21.15.22:0/2637089347 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6704+0+0 (secure 0 0 0) 0x7fd5f408eeb0 con 0x7fd604100460 2024-02-16T19:36:50.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.917+0000 7fd608e30700 1 --2- 172.21.15.22:0/2637089347 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fd5ec06d8a0 0x7fd5ec06fd50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd6040ffc00 tx=0x7fd5f800b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:50.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:50.921+0000 7fd6027fc700 1 -- 172.21.15.22:0/2637089347 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd5f40585f0 con 0x7fd604100460 2024-02-16T19:36:51.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:50 smithi117 ceph-mon[67870]: pgmap v139: 65 pgs: 65 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:36:51.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:50 smithi117 ceph-mon[67870]: from='client.34324 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:36:51.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:50 smithi117 ceph-mon[67870]: from='client.? 172.21.15.22:0/1383243840' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:36:51.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:50 smithi117 ceph-mon[67870]: from='client.? 172.21.15.22:0/239474017' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:36:51.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:51.151+0000 7fd60b895700 1 -- 172.21.15.22:0/2637089347 --> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd6040ffa50 con 0x7fd5ec06d8a0 2024-02-16T19:36:51.153 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:50 smithi022 ceph-mon[88798]: pgmap v139: 65 pgs: 65 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:36:51.153 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:50 smithi022 ceph-mon[88798]: from='client.34324 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:36:51.153 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:50 smithi022 ceph-mon[88798]: from='client.? 172.21.15.22:0/1383243840' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:36:51.153 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:50 smithi022 ceph-mon[88798]: from='client.? 172.21.15.22:0/239474017' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:36:51.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:51.152+0000 7fd6027fc700 1 -- 172.21.15.22:0/2637089347 <== mgr.34104 v2:172.21.15.22:6800/505266753 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fd6040ffa50 con 0x7fd5ec06d8a0 2024-02-16T19:36:51.154 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-02-16T19:36:51.154 INFO:teuthology.orchestra.run.smithi022.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T19:36:51.154 INFO:teuthology.orchestra.run.smithi022.stdout: "in_progress": true, 2024-02-16T19:36:51.154 INFO:teuthology.orchestra.run.smithi022.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T19:36:51.154 INFO:teuthology.orchestra.run.smithi022.stdout: "services_complete": [ 2024-02-16T19:36:51.154 INFO:teuthology.orchestra.run.smithi022.stdout: "crash", 2024-02-16T19:36:51.154 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr", 2024-02-16T19:36:51.154 INFO:teuthology.orchestra.run.smithi022.stdout: "mon" 2024-02-16T19:36:51.154 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-02-16T19:36:51.154 INFO:teuthology.orchestra.run.smithi022.stdout: "progress": "10/23 daemons upgraded", 2024-02-16T19:36:51.155 INFO:teuthology.orchestra.run.smithi022.stdout: "message": "Currently upgrading osd daemons", 2024-02-16T19:36:51.155 INFO:teuthology.orchestra.run.smithi022.stdout: "is_paused": false 2024-02-16T19:36:51.155 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-02-16T19:36:51.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:51.155+0000 7fd60b895700 1 -- 172.21.15.22:0/2637089347 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fd5ec06d8a0 msgr2=0x7fd5ec06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:51.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:51.155+0000 7fd60b895700 1 --2- 172.21.15.22:0/2637089347 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fd5ec06d8a0 0x7fd5ec06fd50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd6040ffc00 tx=0x7fd5f800b410 comp rx=0 tx=0).stop 2024-02-16T19:36:51.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:51.155+0000 7fd60b895700 1 -- 172.21.15.22:0/2637089347 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd604100460 msgr2=0x7fd604196d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:51.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:51.155+0000 7fd60b895700 1 --2- 172.21.15.22:0/2637089347 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd604100460 0x7fd604196d30 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd5f400b0a0 tx=0x7fd5f400cd10 comp rx=0 tx=0).stop 2024-02-16T19:36:51.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:51.155+0000 7fd60b895700 1 -- 172.21.15.22:0/2637089347 shutdown_connections 2024-02-16T19:36:51.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:51.155+0000 7fd60b895700 1 --2- 172.21.15.22:0/2637089347 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fd5ec06d8a0 0x7fd5ec06fd50 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:51.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:51.155+0000 7fd60b895700 1 --2- 172.21.15.22:0/2637089347 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd604100460 0x7fd604196d30 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:51.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:51.155+0000 7fd60b895700 1 --2- 172.21.15.22:0/2637089347 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd604100d70 0x7fd604197270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:51.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:51.155+0000 7fd60b895700 1 -- 172.21.15.22:0/2637089347 >> 172.21.15.22:0/2637089347 conn(0x7fd6040748b0 msgr2=0x7fd604103d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:51.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:51.155+0000 7fd60b895700 1 -- 172.21.15.22:0/2637089347 shutdown_connections 2024-02-16T19:36:51.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:51.155+0000 7fd60b895700 1 -- 172.21.15.22:0/2637089347 wait complete. 2024-02-16T19:36:51.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:51.260+0000 7fdb9d191700 1 -- 172.21.15.22:0/1150350039 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb980ffd60 msgr2=0x7fdb98103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:51.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:51.260+0000 7fdb9d191700 1 --2- 172.21.15.22:0/1150350039 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb980ffd60 0x7fdb98103d30 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c009910 tx=0x7fdb8c009c20 comp rx=0 tx=0).stop 2024-02-16T19:36:51.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:51.260+0000 7fdb9d191700 1 -- 172.21.15.22:0/1150350039 shutdown_connections 2024-02-16T19:36:51.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:51.260+0000 7fdb9d191700 1 --2- 172.21.15.22:0/1150350039 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb980ffd60 0x7fdb98103d30 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:51.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:51.260+0000 7fdb9d191700 1 --2- 172.21.15.22:0/1150350039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb980ff3c0 0x7fdb980ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:51.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:51.260+0000 7fdb9d191700 1 -- 172.21.15.22:0/1150350039 >> 172.21.15.22:0/1150350039 conn(0x7fdb980fac80 msgr2=0x7fdb980fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:51.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:51.260+0000 7fdb9d191700 1 -- 172.21.15.22:0/1150350039 shutdown_connections 2024-02-16T19:36:51.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:51.260+0000 7fdb9d191700 1 -- 172.21.15.22:0/1150350039 wait complete. 2024-02-16T19:36:51.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:51.261+0000 7fdb9d191700 1 Processor -- start 2024-02-16T19:36:51.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:51.261+0000 7fdb9d191700 1 -- start start 2024-02-16T19:36:51.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:51.261+0000 7fdb9d191700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb980ff3c0 0x7fdb98196e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:51.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:51.261+0000 7fdb9d191700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb980ffd60 0x7fdb98197370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:51.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:51.261+0000 7fdb9d191700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb98197990 con 0x7fdb980ffd60 2024-02-16T19:36:51.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:51.261+0000 7fdb9d191700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb98197b00 con 0x7fdb980ff3c0 2024-02-16T19:36:51.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:51.262+0000 7fdb9659c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb980ffd60 0x7fdb98197370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:51.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:51.262+0000 7fdb9659c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb980ffd60 0x7fdb98197370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38292/0 (socket says 172.21.15.22:38292) 2024-02-16T19:36:51.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:51.262+0000 7fdb9659c700 1 -- 172.21.15.22:0/2790892982 learned_addr learned my addr 172.21.15.22:0/2790892982 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:36:51.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:51.262+0000 7fdb96d9d700 1 --2- 172.21.15.22:0/2790892982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb980ff3c0 0x7fdb98196e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:51.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:51.262+0000 7fdb9659c700 1 -- 172.21.15.22:0/2790892982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb980ff3c0 msgr2=0x7fdb98196e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:51.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:51.262+0000 7fdb9659c700 1 --2- 172.21.15.22:0/2790892982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb980ff3c0 0x7fdb98196e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:51.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:51.262+0000 7fdb9659c700 1 -- 172.21.15.22:0/2790892982 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb8c0095f0 con 0x7fdb980ffd60 2024-02-16T19:36:51.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:51.262+0000 7fdb96d9d700 1 --2- 172.21.15.22:0/2790892982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb980ff3c0 0x7fdb98196e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T19:36:51.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:51.262+0000 7fdb9659c700 1 --2- 172.21.15.22:0/2790892982 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb980ffd60 0x7fdb98197370 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c00bda0 tx=0x7fdb8c00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:51.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:51.262+0000 7fdb7ffff700 1 -- 172.21.15.22:0/2790892982 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb8c01d070 con 0x7fdb980ffd60 2024-02-16T19:36:51.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:51.262+0000 7fdb9d191700 1 -- 172.21.15.22:0/2790892982 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb9819fea0 con 0x7fdb980ffd60 2024-02-16T19:36:51.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:51.262+0000 7fdb7ffff700 1 -- 172.21.15.22:0/2790892982 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdb8c004fb0 con 0x7fdb980ffd60 2024-02-16T19:36:51.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:51.262+0000 7fdb7ffff700 1 -- 172.21.15.22:0/2790892982 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb8c00f480 con 0x7fdb980ffd60 2024-02-16T19:36:51.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:51.262+0000 7fdb9d191700 1 -- 172.21.15.22:0/2790892982 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb981a0310 con 0x7fdb980ffd60 2024-02-16T19:36:51.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:51.263+0000 7fdb9d191700 1 -- 172.21.15.22:0/2790892982 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb98107680 con 0x7fdb980ffd60 2024-02-16T19:36:51.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:51.264+0000 7fdb7ffff700 1 -- 172.21.15.22:0/2790892982 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fdb8c022870 con 0x7fdb980ffd60 2024-02-16T19:36:51.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:51.264+0000 7fdb7ffff700 1 --2- 172.21.15.22:0/2790892982 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fdb8006d8a0 0x7fdb8006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:51.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:51.264+0000 7fdb7ffff700 1 -- 172.21.15.22:0/2790892982 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6704+0+0 (secure 0 0 0) 0x7fdb8c08ed40 con 0x7fdb980ffd60 2024-02-16T19:36:51.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:51.264+0000 7fdb96d9d700 1 --2- 172.21.15.22:0/2790892982 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fdb8006d8a0 0x7fdb8006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:51.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:51.265+0000 7fdb96d9d700 1 --2- 172.21.15.22:0/2790892982 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fdb8006d8a0 0x7fdb8006fd50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdb880099d0 tx=0x7fdb88008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:51.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:51.269+0000 7fdb7ffff700 1 -- 172.21.15.22:0/2790892982 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdb8c092030 con 0x7fdb980ffd60 2024-02-16T19:36:51.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:51.564+0000 7fdb9d191700 1 -- 172.21.15.22:0/2790892982 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fdb9804f2a0 con 0x7fdb980ffd60 2024-02-16T19:36:51.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:51.565+0000 7fdb7ffff700 1 -- 172.21.15.22:0/2790892982 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fdb8c0270e0 con 0x7fdb980ffd60 2024-02-16T19:36:51.566 INFO:teuthology.orchestra.run.smithi022.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-02-16T19:36:51.567 INFO:teuthology.orchestra.run.smithi022.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-02-16T19:36:51.567 INFO:teuthology.orchestra.run.smithi022.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-02-16T19:36:51.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:51.567+0000 7fdb9d191700 1 -- 172.21.15.22:0/2790892982 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fdb8006d8a0 msgr2=0x7fdb8006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:51.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:51.567+0000 7fdb9d191700 1 --2- 172.21.15.22:0/2790892982 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fdb8006d8a0 0x7fdb8006fd50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdb880099d0 tx=0x7fdb88008040 comp rx=0 tx=0).stop 2024-02-16T19:36:51.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:51.567+0000 7fdb9d191700 1 -- 172.21.15.22:0/2790892982 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb980ffd60 msgr2=0x7fdb98197370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:51.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:51.567+0000 7fdb9d191700 1 --2- 172.21.15.22:0/2790892982 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb980ffd60 0x7fdb98197370 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c00bda0 tx=0x7fdb8c00be80 comp rx=0 tx=0).stop 2024-02-16T19:36:51.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:51.567+0000 7fdb9d191700 1 -- 172.21.15.22:0/2790892982 shutdown_connections 2024-02-16T19:36:51.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:51.567+0000 7fdb9d191700 1 --2- 172.21.15.22:0/2790892982 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fdb8006d8a0 0x7fdb8006fd50 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:51.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:51.567+0000 7fdb9d191700 1 --2- 172.21.15.22:0/2790892982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb980ff3c0 0x7fdb98196e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:51.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:51.567+0000 7fdb9d191700 1 --2- 172.21.15.22:0/2790892982 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb980ffd60 0x7fdb98197370 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:51.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:51.567+0000 7fdb9d191700 1 -- 172.21.15.22:0/2790892982 >> 172.21.15.22:0/2790892982 conn(0x7fdb980fac80 msgr2=0x7fdb980fc440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:51.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:51.567+0000 7fdb9d191700 1 -- 172.21.15.22:0/2790892982 shutdown_connections 2024-02-16T19:36:51.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:36:51.567+0000 7fdb9d191700 1 -- 172.21.15.22:0/2790892982 wait complete. 2024-02-16T19:36:52.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:51 smithi117 ceph-mon[67870]: from='client.34336 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:36:52.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:51 smithi117 ceph-mon[67870]: from='client.? 172.21.15.22:0/2790892982' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T19:36:52.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:51 smithi022 ceph-mon[88798]: from='client.34336 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:36:52.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:51 smithi022 ceph-mon[88798]: from='client.? 172.21.15.22:0/2790892982' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T19:36:53.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:52 smithi117 ceph-mon[67870]: pgmap v140: 65 pgs: 65 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:36:53.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:52 smithi022 ceph-mon[88798]: pgmap v140: 65 pgs: 65 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:36:55.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:54 smithi117 ceph-mon[67870]: pgmap v141: 65 pgs: 65 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:36:55.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:54 smithi022 ceph-mon[88798]: pgmap v141: 65 pgs: 65 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:36:55.926 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:55 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-02-16T19:36:55.926 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:55 smithi117 ceph-mon[67870]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-02-16T19:36:55.926 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:55 smithi117 ceph-mon[67870]: Upgrade: osd.4 is safe to restart 2024-02-16T19:36:56.216 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:55 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-02-16T19:36:56.216 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:55 smithi022 ceph-mon[88798]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-02-16T19:36:56.216 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:55 smithi022 ceph-mon[88798]: Upgrade: osd.4 is safe to restart 2024-02-16T19:36:57.134 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:56 smithi022 ceph-mon[88798]: pgmap v142: 65 pgs: 65 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:36:57.135 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:56 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:36:57.135 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:56 smithi022 ceph-mon[88798]: Upgrade: Updating osd.4 2024-02-16T19:36:57.135 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:56 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:36:57.135 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:56 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-16T19:36:57.135 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:56 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:36:57.135 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:56 smithi022 ceph-mon[88798]: Deploying daemon osd.4 on smithi117 2024-02-16T19:36:57.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:56 smithi117 ceph-mon[67870]: pgmap v142: 65 pgs: 65 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:36:57.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:56 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:36:57.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:56 smithi117 ceph-mon[67870]: Upgrade: Updating osd.4 2024-02-16T19:36:57.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:56 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:36:57.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:56 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-16T19:36:57.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:56 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:36:57.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:56 smithi117 ceph-mon[67870]: Deploying daemon osd.4 on smithi117 2024-02-16T19:36:59.378 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:36:59 smithi117 ceph-mon[67870]: pgmap v143: 65 pgs: 65 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:36:59.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:36:59 smithi022 ceph-mon[88798]: pgmap v143: 65 pgs: 65 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:36:59.966 INFO:journalctl@ceph.osd.4.smithi117.stdout:Feb 16 19:36:59 smithi117 systemd[1]: Stopping Ceph osd.4 for 2a1346e6-cd00-11ee-95ba-87774f69a715... 2024-02-16T19:37:00.297 INFO:journalctl@ceph.osd.4.smithi117.stdout:Feb 16 19:36:59 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-4[45912]: 2024-02-16T19:36:59.964+0000 7f57ee312700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-16T19:37:00.297 INFO:journalctl@ceph.osd.4.smithi117.stdout:Feb 16 19:36:59 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-4[45912]: 2024-02-16T19:36:59.964+0000 7f57ee312700 -1 osd.4 79 *** Got signal Terminated *** 2024-02-16T19:37:00.297 INFO:journalctl@ceph.osd.4.smithi117.stdout:Feb 16 19:36:59 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-4[45912]: 2024-02-16T19:36:59.964+0000 7f57ee312700 -1 osd.4 79 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-16T19:37:00.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:00 smithi117 ceph-mon[67870]: osd.4 marked itself down and dead 2024-02-16T19:37:00.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:00 smithi022 ceph-mon[88798]: osd.4 marked itself down and dead 2024-02-16T19:37:01.008 INFO:journalctl@ceph.osd.4.smithi117.stdout:Feb 16 19:37:00 smithi117 podman[82490]: 2024-02-16 19:37:00.669087951 +0000 UTC m=+0.805287585 container died 1779c3496f4d9fb43c9ef3146ccaf9c903fbb263e1c4cf2a1589b9e388af991e (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-4, GIT_BRANCH=HEAD, io.buildah.version=1.29.1, GIT_CLEAN=True, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , RELEASE=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.build-date=20231212) 2024-02-16T19:37:01.305 INFO:journalctl@ceph.osd.4.smithi117.stdout:Feb 16 19:37:01 smithi117 podman[82490]: 2024-02-16 19:37:01.147802371 +0000 UTC m=+1.284002005 container cleanup 1779c3496f4d9fb43c9ef3146ccaf9c903fbb263e1c4cf2a1589b9e388af991e (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-4, io.buildah.version=1.29.1, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.build-date=20231212, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=https://github.com/ceph/ceph-container.git, RELEASE=HEAD, maintainer=Guillaume Abrioux ) 2024-02-16T19:37:01.305 INFO:journalctl@ceph.osd.4.smithi117.stdout:Feb 16 19:37:01 smithi117 bash[82490]: ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-4 2024-02-16T19:37:01.637 INFO:journalctl@ceph.osd.4.smithi117.stdout:Feb 16 19:37:01 smithi117 podman[82509]: 2024-02-16 19:37:01.307564589 +0000 UTC m=+0.636824082 container remove 1779c3496f4d9fb43c9ef3146ccaf9c903fbb263e1c4cf2a1589b9e388af991e (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-4, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.29.1, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.schema-version=1.0, org.label-schema.build-date=20231212, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True) 2024-02-16T19:37:01.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:01 smithi117 ceph-mon[67870]: pgmap v144: 65 pgs: 65 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:37:01.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:01 smithi117 ceph-mon[67870]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-16T19:37:01.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:01 smithi117 ceph-mon[67870]: osdmap e80: 6 total, 5 up, 6 in 2024-02-16T19:37:01.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:01 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:37:01.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:01 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:37:01.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:01 smithi022 ceph-mon[88798]: pgmap v144: 65 pgs: 65 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:37:01.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:01 smithi022 ceph-mon[88798]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-16T19:37:01.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:01 smithi022 ceph-mon[88798]: osdmap e80: 6 total, 5 up, 6 in 2024-02-16T19:37:01.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:01 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:37:01.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:01 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:37:02.011 INFO:journalctl@ceph.osd.4.smithi117.stdout:Feb 16 19:37:01 smithi117 podman[82602]: 2024-02-16 19:37:01.60844049 +0000 UTC m=+0.018557382 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T19:37:02.012 INFO:journalctl@ceph.osd.4.smithi117.stdout:Feb 16 19:37:01 smithi117 podman[82602]: 2024-02-16 19:37:01.716935581 +0000 UTC m=+0.127052460 container create d1e359847af782f462077e4d58b9ce507b54a36b40da045cbcc24a5c0a1bbebe (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-4-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-16T19:37:02.012 INFO:journalctl@ceph.osd.4.smithi117.stdout:Feb 16 19:37:01 smithi117 podman[82602]: 2024-02-16 19:37:01.928200716 +0000 UTC m=+0.338317612 container init d1e359847af782f462077e4d58b9ce507b54a36b40da045cbcc24a5c0a1bbebe (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-4-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, ceph=True, io.buildah.version=1.33.5) 2024-02-16T19:37:02.012 INFO:journalctl@ceph.osd.4.smithi117.stdout:Feb 16 19:37:01 smithi117 podman[82602]: 2024-02-16 19:37:01.931538467 +0000 UTC m=+0.341655355 container start d1e359847af782f462077e4d58b9ce507b54a36b40da045cbcc24a5c0a1bbebe (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-4-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-02-16T19:37:02.012 INFO:journalctl@ceph.osd.4.smithi117.stdout:Feb 16 19:37:02 smithi117 podman[82602]: 2024-02-16 19:37:02.011223753 +0000 UTC m=+0.421340639 container attach d1e359847af782f462077e4d58b9ce507b54a36b40da045cbcc24a5c0a1bbebe (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-4-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-16T19:37:02.305 INFO:journalctl@ceph.osd.4.smithi117.stdout:Feb 16 19:37:02 smithi117 podman[82602]: 2024-02-16 19:37:02.220940354 +0000 UTC m=+0.631057234 container died d1e359847af782f462077e4d58b9ce507b54a36b40da045cbcc24a5c0a1bbebe (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-4-deactivate, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-02-16T19:37:02.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:02 smithi022 ceph-mon[88798]: osdmap e81: 6 total, 5 up, 6 in 2024-02-16T19:37:02.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:02 smithi117 ceph-mon[67870]: osdmap e81: 6 total, 5 up, 6 in 2024-02-16T19:37:03.137 INFO:journalctl@ceph.osd.4.smithi117.stdout:Feb 16 19:37:02 smithi117 podman[82628]: 2024-02-16 19:37:02.831637086 +0000 UTC m=+0.600355575 container remove d1e359847af782f462077e4d58b9ce507b54a36b40da045cbcc24a5c0a1bbebe (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-4-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214) 2024-02-16T19:37:03.137 INFO:journalctl@ceph.osd.4.smithi117.stdout:Feb 16 19:37:02 smithi117 systemd[1]: ceph-2a1346e6-cd00-11ee-95ba-87774f69a715@osd.4.service: Deactivated successfully. 2024-02-16T19:37:03.138 INFO:journalctl@ceph.osd.4.smithi117.stdout:Feb 16 19:37:02 smithi117 systemd[1]: Stopped Ceph osd.4 for 2a1346e6-cd00-11ee-95ba-87774f69a715. 2024-02-16T19:37:03.138 INFO:journalctl@ceph.osd.4.smithi117.stdout:Feb 16 19:37:02 smithi117 systemd[1]: ceph-2a1346e6-cd00-11ee-95ba-87774f69a715@osd.4.service: Consumed 50.286s CPU time. 2024-02-16T19:37:03.138 INFO:journalctl@ceph.osd.4.smithi117.stdout:Feb 16 19:37:03 smithi117 systemd[1]: Starting Ceph osd.4 for 2a1346e6-cd00-11ee-95ba-87774f69a715... 2024-02-16T19:37:03.611 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:03 smithi117 ceph-mon[67870]: pgmap v147: 65 pgs: 3 active+undersized, 2 active+undersized+degraded, 11 peering, 2 stale+active+clean, 47 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 8/267 objects degraded (2.996%) 2024-02-16T19:37:03.611 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:03 smithi117 ceph-mon[67870]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-02-16T19:37:03.611 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:03 smithi117 ceph-mon[67870]: Health check failed: Degraded data redundancy: 8/267 objects degraded (2.996%), 2 pgs degraded (PG_DEGRADED) 2024-02-16T19:37:03.611 INFO:journalctl@ceph.osd.4.smithi117.stdout:Feb 16 19:37:03 smithi117 podman[82735]: 2024-02-16 19:37:03.178036682 +0000 UTC m=+0.019008963 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T19:37:03.611 INFO:journalctl@ceph.osd.4.smithi117.stdout:Feb 16 19:37:03 smithi117 podman[82735]: 2024-02-16 19:37:03.291132891 +0000 UTC m=+0.132105163 container create 6361dd43159b728aee01c6c0b5c44e0efd537012eb8f2dea1275931b2230cdb3 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-4-activate, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-02-16T19:37:03.611 INFO:journalctl@ceph.osd.4.smithi117.stdout:Feb 16 19:37:03 smithi117 podman[82735]: 2024-02-16 19:37:03.510763973 +0000 UTC m=+0.351736244 container init 6361dd43159b728aee01c6c0b5c44e0efd537012eb8f2dea1275931b2230cdb3 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-4-activate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.vendor=CentOS) 2024-02-16T19:37:03.611 INFO:journalctl@ceph.osd.4.smithi117.stdout:Feb 16 19:37:03 smithi117 podman[82735]: 2024-02-16 19:37:03.514046837 +0000 UTC m=+0.355019117 container start 6361dd43159b728aee01c6c0b5c44e0efd537012eb8f2dea1275931b2230cdb3 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-4-activate, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-02-16T19:37:03.611 INFO:journalctl@ceph.osd.4.smithi117.stdout:Feb 16 19:37:03 smithi117 podman[82735]: 2024-02-16 19:37:03.610518495 +0000 UTC m=+0.451490769 container attach 6361dd43159b728aee01c6c0b5c44e0efd537012eb8f2dea1275931b2230cdb3 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-4-activate, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_CLEAN=True, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-16T19:37:03.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:03 smithi022 ceph-mon[88798]: pgmap v147: 65 pgs: 3 active+undersized, 2 active+undersized+degraded, 11 peering, 2 stale+active+clean, 47 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 8/267 objects degraded (2.996%) 2024-02-16T19:37:03.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:03 smithi022 ceph-mon[88798]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-02-16T19:37:03.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:03 smithi022 ceph-mon[88798]: Health check failed: Degraded data redundancy: 8/267 objects degraded (2.996%), 2 pgs degraded (PG_DEGRADED) 2024-02-16T19:37:04.294 INFO:journalctl@ceph.osd.4.smithi117.stdout:Feb 16 19:37:03 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-4-activate[82751]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-02-16T19:37:04.294 INFO:journalctl@ceph.osd.4.smithi117.stdout:Feb 16 19:37:03 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-4-activate[82751]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-02-16T19:37:04.294 INFO:journalctl@ceph.osd.4.smithi117.stdout:Feb 16 19:37:03 smithi117 bash[82735]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-02-16T19:37:04.294 INFO:journalctl@ceph.osd.4.smithi117.stdout:Feb 16 19:37:03 smithi117 bash[82735]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-02-16T19:37:04.294 INFO:journalctl@ceph.osd.4.smithi117.stdout:Feb 16 19:37:03 smithi117 bash[82735]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-02-16T19:37:04.295 INFO:journalctl@ceph.osd.4.smithi117.stdout:Feb 16 19:37:03 smithi117 bash[82735]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-02-16T19:37:04.295 INFO:journalctl@ceph.osd.4.smithi117.stdout:Feb 16 19:37:03 smithi117 bash[82735]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-02-16T19:37:04.295 INFO:journalctl@ceph.osd.4.smithi117.stdout:Feb 16 19:37:03 smithi117 bash[82735]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-02-16T19:37:04.295 INFO:journalctl@ceph.osd.4.smithi117.stdout:Feb 16 19:37:03 smithi117 bash[82735]: --> ceph-volume raw activate successful for osd ID: 4 2024-02-16T19:37:04.295 INFO:journalctl@ceph.osd.4.smithi117.stdout:Feb 16 19:37:03 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-4-activate[82751]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-02-16T19:37:04.295 INFO:journalctl@ceph.osd.4.smithi117.stdout:Feb 16 19:37:03 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-4-activate[82751]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-02-16T19:37:04.295 INFO:journalctl@ceph.osd.4.smithi117.stdout:Feb 16 19:37:03 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-4-activate[82751]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-02-16T19:37:04.295 INFO:journalctl@ceph.osd.4.smithi117.stdout:Feb 16 19:37:03 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-4-activate[82751]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-02-16T19:37:04.295 INFO:journalctl@ceph.osd.4.smithi117.stdout:Feb 16 19:37:03 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-4-activate[82751]: --> ceph-volume raw activate successful for osd ID: 4 2024-02-16T19:37:04.295 INFO:journalctl@ceph.osd.4.smithi117.stdout:Feb 16 19:37:03 smithi117 podman[82735]: 2024-02-16 19:37:03.983359194 +0000 UTC m=+0.824331465 container died 6361dd43159b728aee01c6c0b5c44e0efd537012eb8f2dea1275931b2230cdb3 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-4-activate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-02-16T19:37:04.887 INFO:journalctl@ceph.osd.4.smithi117.stdout:Feb 16 19:37:04 smithi117 podman[82887]: 2024-02-16 19:37:04.589559287 +0000 UTC m=+0.600719408 container remove 6361dd43159b728aee01c6c0b5c44e0efd537012eb8f2dea1275931b2230cdb3 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-4-activate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-16T19:37:04.887 INFO:journalctl@ceph.osd.4.smithi117.stdout:Feb 16 19:37:04 smithi117 podman[82950]: 2024-02-16 19:37:04.798819131 +0000 UTC m=+0.112232012 container create 367ba2085f6f29a06e3ca522a7fdf947c147e1569de2b2902493b10c610aebe8 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-02-16T19:37:04.887 INFO:journalctl@ceph.osd.4.smithi117.stdout:Feb 16 19:37:04 smithi117 podman[82950]: 2024-02-16 19:37:04.704733352 +0000 UTC m=+0.018146196 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T19:37:05.308 INFO:journalctl@ceph.osd.4.smithi117.stdout:Feb 16 19:37:04 smithi117 podman[82950]: 2024-02-16 19:37:04.992952679 +0000 UTC m=+0.306365529 container init 367ba2085f6f29a06e3ca522a7fdf947c147e1569de2b2902493b10c610aebe8 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-4, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-02-16T19:37:05.308 INFO:journalctl@ceph.osd.4.smithi117.stdout:Feb 16 19:37:04 smithi117 podman[82950]: 2024-02-16 19:37:04.997103227 +0000 UTC m=+0.310516067 container start 367ba2085f6f29a06e3ca522a7fdf947c147e1569de2b2902493b10c610aebe8 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-4, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-16T19:37:05.308 INFO:journalctl@ceph.osd.4.smithi117.stdout:Feb 16 19:37:05 smithi117 ceph-osd[82969]: -- 172.21.15.117:0/4247087892 <== mon.1 v2:172.21.15.117:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5627e6be91e0 con 0x5627e6bbb000 2024-02-16T19:37:05.308 INFO:journalctl@ceph.osd.4.smithi117.stdout:Feb 16 19:37:05 smithi117 bash[82950]: 367ba2085f6f29a06e3ca522a7fdf947c147e1569de2b2902493b10c610aebe8 2024-02-16T19:37:05.308 INFO:journalctl@ceph.osd.4.smithi117.stdout:Feb 16 19:37:05 smithi117 systemd[1]: Started Ceph osd.4 for 2a1346e6-cd00-11ee-95ba-87774f69a715. 2024-02-16T19:37:05.637 INFO:journalctl@ceph.osd.4.smithi117.stdout:Feb 16 19:37:05 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-4[82965]: 2024-02-16T19:37:05.540+0000 7f5256d1d7c0 -1 Falling back to public interface 2024-02-16T19:37:05.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:05 smithi117 ceph-mon[67870]: pgmap v148: 65 pgs: 9 active+undersized, 6 active+undersized+degraded, 11 peering, 1 stale+active+clean, 38 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 26/267 objects degraded (9.738%) 2024-02-16T19:37:05.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:05 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:37:05.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:05 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:37:05.638 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:05 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:37:05.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:05 smithi022 ceph-mon[88798]: pgmap v148: 65 pgs: 9 active+undersized, 6 active+undersized+degraded, 11 peering, 1 stale+active+clean, 38 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 26/267 objects degraded (9.738%) 2024-02-16T19:37:05.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:05 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:37:05.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:05 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:37:05.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:05 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:37:07.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:07 smithi117 ceph-mon[67870]: pgmap v149: 65 pgs: 15 active+undersized, 15 active+undersized+degraded, 2 peering, 33 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 47/267 objects degraded (17.603%) 2024-02-16T19:37:07.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:07 smithi022 ceph-mon[88798]: pgmap v149: 65 pgs: 15 active+undersized, 15 active+undersized+degraded, 2 peering, 33 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 47/267 objects degraded (17.603%) 2024-02-16T19:37:08.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:08 smithi117 ceph-mon[67870]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-02-16T19:37:08.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:08 smithi022 ceph-mon[88798]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-02-16T19:37:09.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:09 smithi022 ceph-mon[88798]: pgmap v150: 65 pgs: 17 active+undersized, 15 active+undersized+degraded, 33 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 47/267 objects degraded (17.603%) 2024-02-16T19:37:09.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:09 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:37:09.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:09 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:37:09.739 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:09 smithi117 ceph-mon[67870]: pgmap v150: 65 pgs: 17 active+undersized, 15 active+undersized+degraded, 33 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 47/267 objects degraded (17.603%) 2024-02-16T19:37:09.739 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:09 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:37:09.739 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:09 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:37:10.316 INFO:journalctl@ceph.osd.4.smithi117.stdout:Feb 16 19:37:10 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-4[82965]: 2024-02-16T19:37:10.047+0000 7f5256d1d7c0 -1 osd.4 0 read_superblock omap replica is missing. 2024-02-16T19:37:10.590 INFO:journalctl@ceph.osd.4.smithi117.stdout:Feb 16 19:37:10 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-4[82965]: 2024-02-16T19:37:10.380+0000 7f5256d1d7c0 -1 osd.4 79 log_to_monitors true 2024-02-16T19:37:10.590 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:10 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:37:10.591 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:10 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:37:10.591 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:10 smithi117 ceph-mon[67870]: from='osd.4 [v2:172.21.15.117:6808/3679983685,v1:172.21.15.117:6809/3679983685]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-16T19:37:10.591 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:10 smithi117 ceph-mon[67870]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-16T19:37:10.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:10 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:37:10.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:10 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:37:10.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:10 smithi022 ceph-mon[88798]: from='osd.4 [v2:172.21.15.117:6808/3679983685,v1:172.21.15.117:6809/3679983685]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-16T19:37:10.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:10 smithi022 ceph-mon[88798]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-16T19:37:11.526 INFO:journalctl@ceph.osd.4.smithi117.stdout:Feb 16 19:37:11 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-4[82965]: 2024-02-16T19:37:11.387+0000 7f524416d700 -1 osd.4 79 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-16T19:37:11.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:11 smithi117 ceph-mon[67870]: pgmap v151: 65 pgs: 17 active+undersized, 15 active+undersized+degraded, 33 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 47/267 objects degraded (17.603%) 2024-02-16T19:37:11.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:11 smithi117 ceph-mon[67870]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-16T19:37:11.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:11 smithi117 ceph-mon[67870]: osdmap e82: 6 total, 5 up, 6 in 2024-02-16T19:37:11.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:11 smithi117 ceph-mon[67870]: from='osd.4 [v2:172.21.15.117:6808/3679983685,v1:172.21.15.117:6809/3679983685]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-02-16T19:37:11.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:11 smithi117 ceph-mon[67870]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-02-16T19:37:11.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:11 smithi117 ceph-mon[67870]: from='osd.4 ' entity='osd.4' 2024-02-16T19:37:11.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:11 smithi022 ceph-mon[88798]: pgmap v151: 65 pgs: 17 active+undersized, 15 active+undersized+degraded, 33 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 47/267 objects degraded (17.603%) 2024-02-16T19:37:11.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:11 smithi022 ceph-mon[88798]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-16T19:37:11.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:11 smithi022 ceph-mon[88798]: osdmap e82: 6 total, 5 up, 6 in 2024-02-16T19:37:11.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:11 smithi022 ceph-mon[88798]: from='osd.4 [v2:172.21.15.117:6808/3679983685,v1:172.21.15.117:6809/3679983685]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-02-16T19:37:11.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:11 smithi022 ceph-mon[88798]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-02-16T19:37:11.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:11 smithi022 ceph-mon[88798]: from='osd.4 ' entity='osd.4' 2024-02-16T19:37:12.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:12 smithi022 ceph-mon[88798]: Health check update: Degraded data redundancy: 47/267 objects degraded (17.603%), 15 pgs degraded (PG_DEGRADED) 2024-02-16T19:37:12.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:12 smithi022 ceph-mon[88798]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-16T19:37:12.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:12 smithi022 ceph-mon[88798]: osd.4 [v2:172.21.15.117:6808/3679983685,v1:172.21.15.117:6809/3679983685] boot 2024-02-16T19:37:12.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:12 smithi022 ceph-mon[88798]: osdmap e83: 6 total, 6 up, 6 in 2024-02-16T19:37:12.716 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:12 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-16T19:37:12.742 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:12 smithi117 ceph-mon[67870]: Health check update: Degraded data redundancy: 47/267 objects degraded (17.603%), 15 pgs degraded (PG_DEGRADED) 2024-02-16T19:37:12.742 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:12 smithi117 ceph-mon[67870]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-16T19:37:12.743 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:12 smithi117 ceph-mon[67870]: osd.4 [v2:172.21.15.117:6808/3679983685,v1:172.21.15.117:6809/3679983685] boot 2024-02-16T19:37:12.743 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:12 smithi117 ceph-mon[67870]: osdmap e83: 6 total, 6 up, 6 in 2024-02-16T19:37:12.743 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:12 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-16T19:37:13.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:13 smithi117 ceph-mon[67870]: pgmap v153: 65 pgs: 17 active+undersized, 15 active+undersized+degraded, 33 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 47/267 objects degraded (17.603%) 2024-02-16T19:37:13.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:13 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:37:13.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:13 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:37:13.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:13 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:37:13.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:13 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:37:13.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:13 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:37:13.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:13 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:37:13.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:13 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:37:13.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:13 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:37:13.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:13 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:37:13.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:13 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-02-16T19:37:13.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:13 smithi117 ceph-mon[67870]: osdmap e84: 6 total, 6 up, 6 in 2024-02-16T19:37:13.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:13 smithi022 ceph-mon[88798]: pgmap v153: 65 pgs: 17 active+undersized, 15 active+undersized+degraded, 33 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 47/267 objects degraded (17.603%) 2024-02-16T19:37:13.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:13 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:37:13.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:13 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:37:13.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:13 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:37:13.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:13 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:37:13.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:13 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:37:13.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:13 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:37:13.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:13 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:37:13.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:13 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:37:13.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:13 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:37:13.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:13 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-02-16T19:37:13.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:13 smithi022 ceph-mon[88798]: osdmap e84: 6 total, 6 up, 6 in 2024-02-16T19:37:14.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:14 smithi117 ceph-mon[67870]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-02-16T19:37:14.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:14 smithi117 ceph-mon[67870]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-02-16T19:37:14.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:14 smithi022 ceph-mon[88798]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-02-16T19:37:14.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:14 smithi022 ceph-mon[88798]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-02-16T19:37:15.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:15 smithi117 ceph-mon[67870]: pgmap v156: 65 pgs: 9 peering, 12 active+undersized, 11 active+undersized+degraded, 33 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 29/267 objects degraded (10.861%) 2024-02-16T19:37:15.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:15 smithi022 ceph-mon[88798]: pgmap v156: 65 pgs: 9 peering, 12 active+undersized, 11 active+undersized+degraded, 33 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 29/267 objects degraded (10.861%) 2024-02-16T19:37:17.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:16 smithi117 ceph-mon[67870]: pgmap v157: 65 pgs: 9 peering, 4 active+undersized, 1 active+undersized+degraded, 51 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 6/267 objects degraded (2.247%) 2024-02-16T19:37:17.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:16 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:37:17.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:16 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:37:17.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:16 smithi022 ceph-mon[88798]: pgmap v157: 65 pgs: 9 peering, 4 active+undersized, 1 active+undersized+degraded, 51 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 6/267 objects degraded (2.247%) 2024-02-16T19:37:17.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:16 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:37:17.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:16 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:37:18.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:17 smithi117 ceph-mon[67870]: Health check update: Degraded data redundancy: 6/267 objects degraded (2.247%), 1 pg degraded (PG_DEGRADED) 2024-02-16T19:37:18.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:17 smithi022 ceph-mon[88798]: Health check update: Degraded data redundancy: 6/267 objects degraded (2.247%), 1 pg degraded (PG_DEGRADED) 2024-02-16T19:37:19.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:18 smithi117 ceph-mon[67870]: pgmap v158: 65 pgs: 9 peering, 56 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:37:19.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:18 smithi117 ceph-mon[67870]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 6/267 objects degraded (2.247%), 1 pg degraded) 2024-02-16T19:37:19.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:18 smithi022 ceph-mon[88798]: pgmap v158: 65 pgs: 9 peering, 56 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:37:19.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:18 smithi022 ceph-mon[88798]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 6/267 objects degraded (2.247%), 1 pg degraded) 2024-02-16T19:37:21.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:20 smithi117 ceph-mon[67870]: pgmap v159: 65 pgs: 65 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:37:21.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:20 smithi022 ceph-mon[88798]: pgmap v159: 65 pgs: 65 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:37:21.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:21.688+0000 7fc9a02ba700 1 -- 172.21.15.22:0/1243734192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc998100e80 msgr2=0x7fc998104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:37:21.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:21.688+0000 7fc9a02ba700 1 --2- 172.21.15.22:0/1243734192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc998100e80 0x7fc998104e50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc98c009910 tx=0x7fc98c009c20 comp rx=0 tx=0).stop 2024-02-16T19:37:21.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:21.688+0000 7fc9a02ba700 1 -- 172.21.15.22:0/1243734192 shutdown_connections 2024-02-16T19:37:21.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:21.688+0000 7fc9a02ba700 1 --2- 172.21.15.22:0/1243734192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc998100e80 0x7fc998104e50 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:21.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:21.688+0000 7fc9a02ba700 1 --2- 172.21.15.22:0/1243734192 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc9981004e0 0x7fc9981008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:21.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:21.688+0000 7fc9a02ba700 1 -- 172.21.15.22:0/1243734192 >> 172.21.15.22:0/1243734192 conn(0x7fc9980fbda0 msgr2=0x7fc9980fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:37:21.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:21.688+0000 7fc9a02ba700 1 -- 172.21.15.22:0/1243734192 shutdown_connections 2024-02-16T19:37:21.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:21.688+0000 7fc9a02ba700 1 -- 172.21.15.22:0/1243734192 wait complete. 2024-02-16T19:37:21.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:21.689+0000 7fc9a02ba700 1 Processor -- start 2024-02-16T19:37:21.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:21.689+0000 7fc9a02ba700 1 -- start start 2024-02-16T19:37:21.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:21.689+0000 7fc9a02ba700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc9981004e0 0x7fc9981980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:37:21.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:21.689+0000 7fc9a02ba700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc998100e80 0x7fc998198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:37:21.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:21.689+0000 7fc9a02ba700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc998198ce0 con 0x7fc998100e80 2024-02-16T19:37:21.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:21.689+0000 7fc9a02ba700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc99819ca70 con 0x7fc9981004e0 2024-02-16T19:37:21.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:21.689+0000 7fc99d855700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc998100e80 0x7fc998198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:37:21.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:21.689+0000 7fc99d855700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc998100e80 0x7fc998198600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58850/0 (socket says 172.21.15.22:58850) 2024-02-16T19:37:21.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:21.689+0000 7fc99d855700 1 -- 172.21.15.22:0/752248563 learned_addr learned my addr 172.21.15.22:0/752248563 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:37:21.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:21.690+0000 7fc99e056700 1 --2- 172.21.15.22:0/752248563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc9981004e0 0x7fc9981980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:37:21.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:21.690+0000 7fc99d855700 1 -- 172.21.15.22:0/752248563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc9981004e0 msgr2=0x7fc9981980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:37:21.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:21.690+0000 7fc99d855700 1 --2- 172.21.15.22:0/752248563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc9981004e0 0x7fc9981980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:21.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:21.690+0000 7fc99d855700 1 -- 172.21.15.22:0/752248563 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc98c0095f0 con 0x7fc998100e80 2024-02-16T19:37:21.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:21.690+0000 7fc99e056700 1 --2- 172.21.15.22:0/752248563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc9981004e0 0x7fc9981980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T19:37:21.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:21.690+0000 7fc99d855700 1 --2- 172.21.15.22:0/752248563 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc998100e80 0x7fc998198600 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc98c00bda0 tx=0x7fc98c00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:37:21.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:21.690+0000 7fc9977fe700 1 -- 172.21.15.22:0/752248563 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc98c01d070 con 0x7fc998100e80 2024-02-16T19:37:21.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:21.690+0000 7fc9977fe700 1 -- 172.21.15.22:0/752248563 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc98c004fb0 con 0x7fc998100e80 2024-02-16T19:37:21.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:21.690+0000 7fc9a02ba700 1 -- 172.21.15.22:0/752248563 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc99819ccf0 con 0x7fc998100e80 2024-02-16T19:37:21.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:21.690+0000 7fc9977fe700 1 -- 172.21.15.22:0/752248563 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc98c00f480 con 0x7fc998100e80 2024-02-16T19:37:21.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:21.690+0000 7fc9a02ba700 1 -- 172.21.15.22:0/752248563 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc99819d1e0 con 0x7fc998100e80 2024-02-16T19:37:21.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:21.691+0000 7fc9a02ba700 1 -- 172.21.15.22:0/752248563 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc998104ce0 con 0x7fc998100e80 2024-02-16T19:37:21.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:21.692+0000 7fc9977fe700 1 -- 172.21.15.22:0/752248563 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fc98c022870 con 0x7fc998100e80 2024-02-16T19:37:21.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:21.692+0000 7fc9977fe700 1 --2- 172.21.15.22:0/752248563 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fc98006d8a0 0x7fc98006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:37:21.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:21.692+0000 7fc9977fe700 1 -- 172.21.15.22:0/752248563 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6704+0+0 (secure 0 0 0) 0x7fc98c08ed20 con 0x7fc998100e80 2024-02-16T19:37:21.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:21.692+0000 7fc99e056700 1 --2- 172.21.15.22:0/752248563 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fc98006d8a0 0x7fc98006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:37:21.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:21.693+0000 7fc99e056700 1 --2- 172.21.15.22:0/752248563 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fc98006d8a0 0x7fc98006fd50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc988007fd0 tx=0x7fc988007810 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:37:21.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:21.697+0000 7fc9977fe700 1 -- 172.21.15.22:0/752248563 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc98c058630 con 0x7fc998100e80 2024-02-16T19:37:21.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:21.924+0000 7fc9a02ba700 1 -- 172.21.15.22:0/752248563 --> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc99802cca0 con 0x7fc98006d8a0 2024-02-16T19:37:21.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:21.926+0000 7fc9977fe700 1 -- 172.21.15.22:0/752248563 <== mgr.34104 v2:172.21.15.22:6800/505266753 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fc99802cca0 con 0x7fc98006d8a0 2024-02-16T19:37:21.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:21.928+0000 7fc9a02ba700 1 -- 172.21.15.22:0/752248563 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fc98006d8a0 msgr2=0x7fc98006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:37:21.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:21.928+0000 7fc9a02ba700 1 --2- 172.21.15.22:0/752248563 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fc98006d8a0 0x7fc98006fd50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc988007fd0 tx=0x7fc988007810 comp rx=0 tx=0).stop 2024-02-16T19:37:21.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:21.928+0000 7fc9a02ba700 1 -- 172.21.15.22:0/752248563 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc998100e80 msgr2=0x7fc998198600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:37:21.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:21.928+0000 7fc9a02ba700 1 --2- 172.21.15.22:0/752248563 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc998100e80 0x7fc998198600 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc98c00bda0 tx=0x7fc98c00be80 comp rx=0 tx=0).stop 2024-02-16T19:37:21.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:21.928+0000 7fc9a02ba700 1 -- 172.21.15.22:0/752248563 shutdown_connections 2024-02-16T19:37:21.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:21.928+0000 7fc9a02ba700 1 --2- 172.21.15.22:0/752248563 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fc98006d8a0 0x7fc98006fd50 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:21.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:21.928+0000 7fc9a02ba700 1 --2- 172.21.15.22:0/752248563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc9981004e0 0x7fc9981980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:21.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:21.928+0000 7fc9a02ba700 1 --2- 172.21.15.22:0/752248563 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc998100e80 0x7fc998198600 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:21.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:21.928+0000 7fc9a02ba700 1 -- 172.21.15.22:0/752248563 >> 172.21.15.22:0/752248563 conn(0x7fc9980fbda0 msgr2=0x7fc998101e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:37:21.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:21.928+0000 7fc9a02ba700 1 -- 172.21.15.22:0/752248563 shutdown_connections 2024-02-16T19:37:21.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:21.928+0000 7fc9a02ba700 1 -- 172.21.15.22:0/752248563 wait complete. 2024-02-16T19:37:21.940 INFO:teuthology.orchestra.run.smithi022.stdout:true 2024-02-16T19:37:22.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.035+0000 7fa18a30d700 1 -- 172.21.15.22:0/1462181137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1840ff3c0 msgr2=0x7fa1840ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:37:22.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.035+0000 7fa18a30d700 1 --2- 172.21.15.22:0/1462181137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1840ff3c0 0x7fa1840ff790 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa174009910 tx=0x7fa174009c20 comp rx=0 tx=0).stop 2024-02-16T19:37:22.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.035+0000 7fa18a30d700 1 -- 172.21.15.22:0/1462181137 shutdown_connections 2024-02-16T19:37:22.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.035+0000 7fa18a30d700 1 --2- 172.21.15.22:0/1462181137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1840ffd60 0x7fa184103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:22.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.035+0000 7fa18a30d700 1 --2- 172.21.15.22:0/1462181137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1840ff3c0 0x7fa1840ff790 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:22.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.035+0000 7fa18a30d700 1 -- 172.21.15.22:0/1462181137 >> 172.21.15.22:0/1462181137 conn(0x7fa1840fac60 msgr2=0x7fa1840fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:37:22.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.035+0000 7fa18a30d700 1 -- 172.21.15.22:0/1462181137 shutdown_connections 2024-02-16T19:37:22.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.035+0000 7fa18a30d700 1 -- 172.21.15.22:0/1462181137 wait complete. 2024-02-16T19:37:22.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.035+0000 7fa18a30d700 1 Processor -- start 2024-02-16T19:37:22.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.035+0000 7fa18a30d700 1 -- start start 2024-02-16T19:37:22.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.036+0000 7fa18a30d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1840ff3c0 0x7fa184196020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:37:22.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.036+0000 7fa18a30d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1840ffd60 0x7fa184196560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:37:22.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.036+0000 7fa18a30d700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa184196c40 con 0x7fa1840ff3c0 2024-02-16T19:37:22.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.036+0000 7fa18a30d700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa18419a9d0 con 0x7fa1840ffd60 2024-02-16T19:37:22.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.036+0000 7fa183fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1840ff3c0 0x7fa184196020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:37:22.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.036+0000 7fa183fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1840ff3c0 0x7fa184196020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58870/0 (socket says 172.21.15.22:58870) 2024-02-16T19:37:22.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.036+0000 7fa183fff700 1 -- 172.21.15.22:0/4001984078 learned_addr learned my addr 172.21.15.22:0/4001984078 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:37:22.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.036+0000 7fa1837fe700 1 --2- 172.21.15.22:0/4001984078 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1840ffd60 0x7fa184196560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:37:22.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.036+0000 7fa183fff700 1 -- 172.21.15.22:0/4001984078 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1840ffd60 msgr2=0x7fa184196560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:37:22.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.036+0000 7fa183fff700 1 --2- 172.21.15.22:0/4001984078 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1840ffd60 0x7fa184196560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:22.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.036+0000 7fa183fff700 1 -- 172.21.15.22:0/4001984078 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1740095f0 con 0x7fa1840ff3c0 2024-02-16T19:37:22.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.036+0000 7fa1837fe700 1 --2- 172.21.15.22:0/4001984078 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1840ffd60 0x7fa184196560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T19:37:22.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.036+0000 7fa183fff700 1 --2- 172.21.15.22:0/4001984078 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1840ff3c0 0x7fa184196020 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa174006010 tx=0x7fa17400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:37:22.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.037+0000 7fa1817fa700 1 -- 172.21.15.22:0/4001984078 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa17401d070 con 0x7fa1840ff3c0 2024-02-16T19:37:22.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.037+0000 7fa18a30d700 1 -- 172.21.15.22:0/4001984078 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa18419ac50 con 0x7fa1840ff3c0 2024-02-16T19:37:22.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.037+0000 7fa1817fa700 1 -- 172.21.15.22:0/4001984078 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa174004fb0 con 0x7fa1840ff3c0 2024-02-16T19:37:22.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.037+0000 7fa1817fa700 1 -- 172.21.15.22:0/4001984078 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa17400f690 con 0x7fa1840ff3c0 2024-02-16T19:37:22.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.037+0000 7fa18a30d700 1 -- 172.21.15.22:0/4001984078 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa18419b140 con 0x7fa1840ff3c0 2024-02-16T19:37:22.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.038+0000 7fa18a30d700 1 -- 172.21.15.22:0/4001984078 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa18404f2a0 con 0x7fa1840ff3c0 2024-02-16T19:37:22.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.038+0000 7fa1817fa700 1 -- 172.21.15.22:0/4001984078 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fa1740228a0 con 0x7fa1840ff3c0 2024-02-16T19:37:22.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.038+0000 7fa1817fa700 1 --2- 172.21.15.22:0/4001984078 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fa16c06d8a0 0x7fa16c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:37:22.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.039+0000 7fa1817fa700 1 -- 172.21.15.22:0/4001984078 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6704+0+0 (secure 0 0 0) 0x7fa17408ee50 con 0x7fa1840ff3c0 2024-02-16T19:37:22.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.039+0000 7fa1837fe700 1 --2- 172.21.15.22:0/4001984078 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fa16c06d8a0 0x7fa16c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:37:22.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.039+0000 7fa1837fe700 1 --2- 172.21.15.22:0/4001984078 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fa16c06d8a0 0x7fa16c06fd50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa184197640 tx=0x7fa178008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:37:22.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.043+0000 7fa1817fa700 1 -- 172.21.15.22:0/4001984078 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa174058590 con 0x7fa1840ff3c0 2024-02-16T19:37:22.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.269+0000 7fa18a30d700 1 -- 172.21.15.22:0/4001984078 --> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa18419b3d0 con 0x7fa16c06d8a0 2024-02-16T19:37:22.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.270+0000 7fa1817fa700 1 -- 172.21.15.22:0/4001984078 <== mgr.34104 v2:172.21.15.22:6800/505266753 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fa18419b3d0 con 0x7fa16c06d8a0 2024-02-16T19:37:22.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.272+0000 7fa18a30d700 1 -- 172.21.15.22:0/4001984078 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fa16c06d8a0 msgr2=0x7fa16c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:37:22.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.272+0000 7fa18a30d700 1 --2- 172.21.15.22:0/4001984078 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fa16c06d8a0 0x7fa16c06fd50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa184197640 tx=0x7fa178008040 comp rx=0 tx=0).stop 2024-02-16T19:37:22.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.272+0000 7fa18a30d700 1 -- 172.21.15.22:0/4001984078 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1840ff3c0 msgr2=0x7fa184196020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:37:22.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.272+0000 7fa18a30d700 1 --2- 172.21.15.22:0/4001984078 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1840ff3c0 0x7fa184196020 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa174006010 tx=0x7fa17400be20 comp rx=0 tx=0).stop 2024-02-16T19:37:22.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.272+0000 7fa18a30d700 1 -- 172.21.15.22:0/4001984078 shutdown_connections 2024-02-16T19:37:22.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.272+0000 7fa18a30d700 1 --2- 172.21.15.22:0/4001984078 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fa16c06d8a0 0x7fa16c06fd50 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:22.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.272+0000 7fa18a30d700 1 --2- 172.21.15.22:0/4001984078 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1840ff3c0 0x7fa184196020 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:22.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.272+0000 7fa18a30d700 1 --2- 172.21.15.22:0/4001984078 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1840ffd60 0x7fa184196560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:22.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.272+0000 7fa18a30d700 1 -- 172.21.15.22:0/4001984078 >> 172.21.15.22:0/4001984078 conn(0x7fa1840fac60 msgr2=0x7fa184103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:37:22.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.272+0000 7fa18a30d700 1 -- 172.21.15.22:0/4001984078 shutdown_connections 2024-02-16T19:37:22.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.272+0000 7fa18a30d700 1 -- 172.21.15.22:0/4001984078 wait complete. 2024-02-16T19:37:22.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.379+0000 7f7c51725700 1 -- 172.21.15.22:0/3220454118 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c4c0ffc60 msgr2=0x7f7c4c10a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:37:22.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.379+0000 7f7c51725700 1 --2- 172.21.15.22:0/3220454118 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c4c0ffc60 0x7f7c4c10a6d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7c40009910 tx=0x7f7c40009c20 comp rx=0 tx=0).stop 2024-02-16T19:37:22.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.379+0000 7f7c51725700 1 -- 172.21.15.22:0/3220454118 shutdown_connections 2024-02-16T19:37:22.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.379+0000 7f7c51725700 1 --2- 172.21.15.22:0/3220454118 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c4c0ffc60 0x7f7c4c10a6d0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:22.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.379+0000 7f7c51725700 1 --2- 172.21.15.22:0/3220454118 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c4c0ff350 0x7f7c4c0ff720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:22.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.379+0000 7f7c51725700 1 -- 172.21.15.22:0/3220454118 >> 172.21.15.22:0/3220454118 conn(0x7f7c4c0748b0 msgr2=0x7f7c4c074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:37:22.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.379+0000 7f7c51725700 1 -- 172.21.15.22:0/3220454118 shutdown_connections 2024-02-16T19:37:22.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.379+0000 7f7c51725700 1 -- 172.21.15.22:0/3220454118 wait complete. 2024-02-16T19:37:22.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.379+0000 7f7c51725700 1 Processor -- start 2024-02-16T19:37:22.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.379+0000 7f7c51725700 1 -- start start 2024-02-16T19:37:22.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.380+0000 7f7c51725700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c4c0ff350 0x7f7c4c195ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:37:22.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.380+0000 7f7c51725700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c4c0ffc60 0x7f7c4c196410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:37:22.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.380+0000 7f7c51725700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c4c196af0 con 0x7f7c4c0ffc60 2024-02-16T19:37:22.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.380+0000 7f7c51725700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c4c19a880 con 0x7f7c4c0ff350 2024-02-16T19:37:22.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.380+0000 7f7c4a7fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c4c0ffc60 0x7f7c4c196410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:37:22.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.380+0000 7f7c4a7fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c4c0ffc60 0x7f7c4c196410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58882/0 (socket says 172.21.15.22:58882) 2024-02-16T19:37:22.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.380+0000 7f7c4a7fc700 1 -- 172.21.15.22:0/1862377375 learned_addr learned my addr 172.21.15.22:0/1862377375 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:37:22.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.380+0000 7f7c4affd700 1 --2- 172.21.15.22:0/1862377375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c4c0ff350 0x7f7c4c195ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:37:22.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.380+0000 7f7c4a7fc700 1 -- 172.21.15.22:0/1862377375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c4c0ff350 msgr2=0x7f7c4c195ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:37:22.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.380+0000 7f7c4a7fc700 1 --2- 172.21.15.22:0/1862377375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c4c0ff350 0x7f7c4c195ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:22.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.380+0000 7f7c4a7fc700 1 -- 172.21.15.22:0/1862377375 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c400095f0 con 0x7f7c4c0ffc60 2024-02-16T19:37:22.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.380+0000 7f7c4a7fc700 1 --2- 172.21.15.22:0/1862377375 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c4c0ffc60 0x7f7c4c196410 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7c40006010 tx=0x7f7c4000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:37:22.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.380+0000 7f7c33fff700 1 -- 172.21.15.22:0/1862377375 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c4001d070 con 0x7f7c4c0ffc60 2024-02-16T19:37:22.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.380+0000 7f7c33fff700 1 -- 172.21.15.22:0/1862377375 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7c4000f460 con 0x7f7c4c0ffc60 2024-02-16T19:37:22.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.380+0000 7f7c51725700 1 -- 172.21.15.22:0/1862377375 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c4c19ab00 con 0x7f7c4c0ffc60 2024-02-16T19:37:22.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.380+0000 7f7c33fff700 1 -- 172.21.15.22:0/1862377375 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c40017630 con 0x7f7c4c0ffc60 2024-02-16T19:37:22.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.380+0000 7f7c51725700 1 -- 172.21.15.22:0/1862377375 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c4c19aff0 con 0x7f7c4c0ffc60 2024-02-16T19:37:22.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.381+0000 7f7c51725700 1 -- 172.21.15.22:0/1862377375 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c4c107dd0 con 0x7f7c4c0ffc60 2024-02-16T19:37:22.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.382+0000 7f7c33fff700 1 -- 172.21.15.22:0/1862377375 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f7c4000f5d0 con 0x7f7c4c0ffc60 2024-02-16T19:37:22.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.382+0000 7f7c33fff700 1 --2- 172.21.15.22:0/1862377375 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f7c3406d8a0 0x7f7c3406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:37:22.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.382+0000 7f7c33fff700 1 -- 172.21.15.22:0/1862377375 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f7c4008ea70 con 0x7f7c4c0ffc60 2024-02-16T19:37:22.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.382+0000 7f7c4affd700 1 --2- 172.21.15.22:0/1862377375 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f7c3406d8a0 0x7f7c3406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:37:22.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.383+0000 7f7c4affd700 1 --2- 172.21.15.22:0/1862377375 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f7c3406d8a0 0x7f7c3406fd50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c005950 tx=0x7f7c3c0058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:37:22.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.388+0000 7f7c33fff700 1 -- 172.21.15.22:0/1862377375 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7c400581b0 con 0x7f7c4c0ffc60 2024-02-16T19:37:22.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.611+0000 7f7c51725700 1 -- 172.21.15.22:0/1862377375 --> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7c4c02cca0 con 0x7f7c3406d8a0 2024-02-16T19:37:22.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.624+0000 7f7c33fff700 1 -- 172.21.15.22:0/1862377375 <== mgr.34104 v2:172.21.15.22:6800/505266753 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f7c4c02cca0 con 0x7f7c3406d8a0 2024-02-16T19:37:22.625 INFO:teuthology.orchestra.run.smithi022.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T19:37:22.625 INFO:teuthology.orchestra.run.smithi022.stdout:alertmanager.smithi022 smithi022 *:9093,9094 running (5m) 81s ago 16m 26.8M - 0.25.0 c8568f914cd2 c89bc76be374 2024-02-16T19:37:22.625 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-exporter.smithi022 smithi022 running (16m) 81s ago 16m 8606k - 18.2.0 dc2bc1663786 3f8bef4349ba 2024-02-16T19:37:22.625 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-exporter.smithi117 smithi117 running (15m) 14s ago 15m 8196k - 18.2.0 dc2bc1663786 59b65a7e309f 2024-02-16T19:37:22.625 INFO:teuthology.orchestra.run.smithi022.stdout:crash.smithi022 smithi022 running (3m) 81s ago 16m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a e0f78bdf43e4 2024-02-16T19:37:22.625 INFO:teuthology.orchestra.run.smithi022.stdout:crash.smithi117 smithi117 running (3m) 14s ago 15m 6669k - 19.0.0-1450-gf78a58c0 f081f847070a bce7c0590813 2024-02-16T19:37:22.625 INFO:teuthology.orchestra.run.smithi022.stdout:grafana.smithi022 smithi022 *:3000 running (5m) 81s ago 16m 81.0M - 9.4.12 a0ea5e3c46d9 fd27d11e9729 2024-02-16T19:37:22.626 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi022.btppwm smithi022 running (10m) 81s ago 10m 155M - 18.2.0 dc2bc1663786 22971fb3a907 2024-02-16T19:37:22.626 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi022.ocukvs smithi022 running (11m) 81s ago 10m 17.1M - 18.2.0 dc2bc1663786 81ee7bd5fd39 2024-02-16T19:37:22.626 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi117.ibmppn smithi117 running (10m) 14s ago 10m 18.7M - 18.2.0 dc2bc1663786 8c606cbeeb86 2024-02-16T19:37:22.626 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi117.orzors smithi117 running (10m) 14s ago 10m 17.7M - 18.2.0 dc2bc1663786 7d2bc4df3feb 2024-02-16T19:37:22.626 INFO:teuthology.orchestra.run.smithi022.stdout:mgr.smithi022.gmdcch smithi022 *:8443,9283,8765 running (7m) 81s ago 18m 504M - 19.0.0-1450-gf78a58c0 f081f847070a 7cb74cc9deea 2024-02-16T19:37:22.626 INFO:teuthology.orchestra.run.smithi022.stdout:mgr.smithi117.cohbtv smithi117 *:8443,9283,8765 running (6m) 14s ago 15m 453M - 19.0.0-1450-gf78a58c0 f081f847070a c825b1152189 2024-02-16T19:37:22.626 INFO:teuthology.orchestra.run.smithi022.stdout:mon.smithi022 smithi022 running (4m) 81s ago 18m 48.4M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 9c84755ae67d 2024-02-16T19:37:22.626 INFO:teuthology.orchestra.run.smithi022.stdout:mon.smithi117 smithi117 running (3m) 14s ago 15m 39.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a a319ac50f1a9 2024-02-16T19:37:22.626 INFO:teuthology.orchestra.run.smithi022.stdout:node-exporter.smithi022 smithi022 *:9100 running (6m) 81s ago 16m 16.6M - 1.5.0 0da6a335fe13 1f5b58292d27 2024-02-16T19:37:22.626 INFO:teuthology.orchestra.run.smithi022.stdout:node-exporter.smithi117 smithi117 *:9100 running (6m) 14s ago 15m 15.4M - 1.5.0 0da6a335fe13 bb5e8491d5a6 2024-02-16T19:37:22.626 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0 smithi022 running (2m) 81s ago 14m 107M 1217M 19.0.0-1450-gf78a58c0 f081f847070a 002412b3525f 2024-02-16T19:37:22.626 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1 smithi022 running (2m) 81s ago 14m 128M 1217M 19.0.0-1450-gf78a58c0 f081f847070a c3488ce7b2a0 2024-02-16T19:37:22.626 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2 smithi022 running (87s) 81s ago 13m 12.6M 1217M 19.0.0-1450-gf78a58c0 f081f847070a 46138c50f79d 2024-02-16T19:37:22.626 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3 smithi117 running (51s) 14s ago 13m 90.3M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 28d7d8519e64 2024-02-16T19:37:22.626 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4 smithi117 running (17s) 14s ago 12m 14.9M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 367ba2085f6f 2024-02-16T19:37:22.627 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5 smithi117 running (12m) 14s ago 12m 209M 2240M 18.2.0 dc2bc1663786 fefaabca0638 2024-02-16T19:37:22.627 INFO:teuthology.orchestra.run.smithi022.stdout:prometheus.smithi022 smithi022 *:9095 running (6m) 81s ago 16m 66.9M - 2.43.0 a07b618ecd1d c3b8689d0c97 2024-02-16T19:37:22.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.626+0000 7f7c51725700 1 -- 172.21.15.22:0/1862377375 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f7c3406d8a0 msgr2=0x7f7c3406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:37:22.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.626+0000 7f7c51725700 1 --2- 172.21.15.22:0/1862377375 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f7c3406d8a0 0x7f7c3406fd50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c005950 tx=0x7f7c3c0058e0 comp rx=0 tx=0).stop 2024-02-16T19:37:22.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.626+0000 7f7c51725700 1 -- 172.21.15.22:0/1862377375 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c4c0ffc60 msgr2=0x7f7c4c196410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:37:22.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.626+0000 7f7c51725700 1 --2- 172.21.15.22:0/1862377375 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c4c0ffc60 0x7f7c4c196410 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7c40006010 tx=0x7f7c4000be20 comp rx=0 tx=0).stop 2024-02-16T19:37:22.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.626+0000 7f7c51725700 1 -- 172.21.15.22:0/1862377375 shutdown_connections 2024-02-16T19:37:22.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.626+0000 7f7c51725700 1 --2- 172.21.15.22:0/1862377375 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f7c3406d8a0 0x7f7c3406fd50 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:22.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.626+0000 7f7c51725700 1 --2- 172.21.15.22:0/1862377375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c4c0ff350 0x7f7c4c195ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:22.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.626+0000 7f7c51725700 1 --2- 172.21.15.22:0/1862377375 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c4c0ffc60 0x7f7c4c196410 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:22.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.626+0000 7f7c51725700 1 -- 172.21.15.22:0/1862377375 >> 172.21.15.22:0/1862377375 conn(0x7f7c4c0748b0 msgr2=0x7f7c4c104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:37:22.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.627+0000 7f7c51725700 1 -- 172.21.15.22:0/1862377375 shutdown_connections 2024-02-16T19:37:22.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.627+0000 7f7c51725700 1 -- 172.21.15.22:0/1862377375 wait complete. 2024-02-16T19:37:22.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.732+0000 7fb9e31b4700 1 -- 172.21.15.22:0/3274440423 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb9dc0ff3c0 msgr2=0x7fb9dc0ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:37:22.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.732+0000 7fb9e31b4700 1 --2- 172.21.15.22:0/3274440423 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb9dc0ff3c0 0x7fb9dc0ff790 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb9d400b0a0 tx=0x7fb9d400b3b0 comp rx=0 tx=0).stop 2024-02-16T19:37:22.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.732+0000 7fb9e31b4700 1 -- 172.21.15.22:0/3274440423 shutdown_connections 2024-02-16T19:37:22.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.732+0000 7fb9e31b4700 1 --2- 172.21.15.22:0/3274440423 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9dc0ffd60 0x7fb9dc103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:22.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.732+0000 7fb9e31b4700 1 --2- 172.21.15.22:0/3274440423 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb9dc0ff3c0 0x7fb9dc0ff790 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:22.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.732+0000 7fb9e31b4700 1 -- 172.21.15.22:0/3274440423 >> 172.21.15.22:0/3274440423 conn(0x7fb9dc0fac80 msgr2=0x7fb9dc0fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:37:22.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.732+0000 7fb9e31b4700 1 -- 172.21.15.22:0/3274440423 shutdown_connections 2024-02-16T19:37:22.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.732+0000 7fb9e31b4700 1 -- 172.21.15.22:0/3274440423 wait complete. 2024-02-16T19:37:22.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.732+0000 7fb9e31b4700 1 Processor -- start 2024-02-16T19:37:22.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.733+0000 7fb9e31b4700 1 -- start start 2024-02-16T19:37:22.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.733+0000 7fb9e31b4700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9dc0ff3c0 0x7fb9dc195f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:37:22.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.733+0000 7fb9e31b4700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb9dc0ffd60 0x7fb9dc196440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:37:22.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.733+0000 7fb9e31b4700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9dc196a90 con 0x7fb9dc0ffd60 2024-02-16T19:37:22.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.733+0000 7fb9e31b4700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9dc196bd0 con 0x7fb9dc0ff3c0 2024-02-16T19:37:22.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.733+0000 7fb9dbfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb9dc0ffd60 0x7fb9dc196440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:37:22.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.733+0000 7fb9dbfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb9dc0ffd60 0x7fb9dc196440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58898/0 (socket says 172.21.15.22:58898) 2024-02-16T19:37:22.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.733+0000 7fb9dbfff700 1 -- 172.21.15.22:0/3784063811 learned_addr learned my addr 172.21.15.22:0/3784063811 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:37:22.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.733+0000 7fb9e0f50700 1 --2- 172.21.15.22:0/3784063811 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9dc0ff3c0 0x7fb9dc195f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:37:22.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.733+0000 7fb9dbfff700 1 -- 172.21.15.22:0/3784063811 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9dc0ff3c0 msgr2=0x7fb9dc195f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:37:22.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.733+0000 7fb9dbfff700 1 --2- 172.21.15.22:0/3784063811 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9dc0ff3c0 0x7fb9dc195f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:22.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.733+0000 7fb9dbfff700 1 -- 172.21.15.22:0/3784063811 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9d4009d00 con 0x7fb9dc0ffd60 2024-02-16T19:37:22.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.733+0000 7fb9dbfff700 1 --2- 172.21.15.22:0/3784063811 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb9dc0ffd60 0x7fb9dc196440 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb9d000c960 tx=0x7fb9d000cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:37:22.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.733+0000 7fb9d9ffb700 1 -- 172.21.15.22:0/3784063811 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb9d0007a70 con 0x7fb9dc0ffd60 2024-02-16T19:37:22.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.734+0000 7fb9e31b4700 1 -- 172.21.15.22:0/3784063811 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9dc19aa20 con 0x7fb9dc0ffd60 2024-02-16T19:37:22.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.734+0000 7fb9d9ffb700 1 -- 172.21.15.22:0/3784063811 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb9d0007bd0 con 0x7fb9dc0ffd60 2024-02-16T19:37:22.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.734+0000 7fb9d9ffb700 1 -- 172.21.15.22:0/3784063811 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb9d00186c0 con 0x7fb9dc0ffd60 2024-02-16T19:37:22.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.734+0000 7fb9e31b4700 1 -- 172.21.15.22:0/3784063811 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9dc19af70 con 0x7fb9dc0ffd60 2024-02-16T19:37:22.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.735+0000 7fb9e31b4700 1 -- 172.21.15.22:0/3784063811 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9dc04f2a0 con 0x7fb9dc0ffd60 2024-02-16T19:37:22.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.735+0000 7fb9d9ffb700 1 -- 172.21.15.22:0/3784063811 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fb9d000faa0 con 0x7fb9dc0ffd60 2024-02-16T19:37:22.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.735+0000 7fb9d9ffb700 1 --2- 172.21.15.22:0/3784063811 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fb9c406d8a0 0x7fb9c406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:37:22.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.736+0000 7fb9d9ffb700 1 -- 172.21.15.22:0/3784063811 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6704+0+0 (secure 0 0 0) 0x7fb9d008d5d0 con 0x7fb9dc0ffd60 2024-02-16T19:37:22.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.736+0000 7fb9e0f50700 1 --2- 172.21.15.22:0/3784063811 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fb9c406d8a0 0x7fb9c406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:37:22.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.736+0000 7fb9e0f50700 1 --2- 172.21.15.22:0/3784063811 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fb9c406d8a0 0x7fb9c406fd50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb9d400b830 tx=0x7fb9d4006000 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:37:22.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:22.740+0000 7fb9d9ffb700 1 -- 172.21.15.22:0/3784063811 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb9d005a840 con 0x7fb9dc0ffd60 2024-02-16T19:37:22.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:22 smithi022 ceph-mon[88798]: pgmap v160: 65 pgs: 65 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:37:22.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:22 smithi022 ceph-mon[88798]: from='client.34346 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:37:22.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:22 smithi022 ceph-mon[88798]: from='client.34350 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:37:22.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:22 smithi022 ceph-mon[88798]: from='client.34354 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:37:23.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.038+0000 7fb9e31b4700 1 -- 172.21.15.22:0/3784063811 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb9dc197260 con 0x7fb9dc0ffd60 2024-02-16T19:37:23.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.038+0000 7fb9d9ffb700 1 -- 172.21.15.22:0/3784063811 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+791 (secure 0 0 0) 0x7fb9d005a3d0 con 0x7fb9dc0ffd60 2024-02-16T19:37:23.039 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-02-16T19:37:23.039 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": { 2024-02-16T19:37:23.039 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T19:37:23.039 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:37:23.039 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": { 2024-02-16T19:37:23.039 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T19:37:23.039 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:37:23.039 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": { 2024-02-16T19:37:23.039 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-16T19:37:23.040 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 5 2024-02-16T19:37:23.040 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:37:23.040 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": { 2024-02-16T19:37:23.040 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-02-16T19:37:23.040 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:37:23.040 INFO:teuthology.orchestra.run.smithi022.stdout: "overall": { 2024-02-16T19:37:23.040 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 5, 2024-02-16T19:37:23.040 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 9 2024-02-16T19:37:23.040 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-02-16T19:37:23.040 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-02-16T19:37:23.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.040+0000 7fb9e31b4700 1 -- 172.21.15.22:0/3784063811 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fb9c406d8a0 msgr2=0x7fb9c406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:37:23.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.040+0000 7fb9e31b4700 1 --2- 172.21.15.22:0/3784063811 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fb9c406d8a0 0x7fb9c406fd50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb9d400b830 tx=0x7fb9d4006000 comp rx=0 tx=0).stop 2024-02-16T19:37:23.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.040+0000 7fb9e31b4700 1 -- 172.21.15.22:0/3784063811 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb9dc0ffd60 msgr2=0x7fb9dc196440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:37:23.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.040+0000 7fb9e31b4700 1 --2- 172.21.15.22:0/3784063811 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb9dc0ffd60 0x7fb9dc196440 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb9d000c960 tx=0x7fb9d000cd20 comp rx=0 tx=0).stop 2024-02-16T19:37:23.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.040+0000 7fb9e31b4700 1 -- 172.21.15.22:0/3784063811 shutdown_connections 2024-02-16T19:37:23.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.040+0000 7fb9e31b4700 1 --2- 172.21.15.22:0/3784063811 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fb9c406d8a0 0x7fb9c406fd50 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:23.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.040+0000 7fb9e31b4700 1 --2- 172.21.15.22:0/3784063811 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9dc0ff3c0 0x7fb9dc195f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:23.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.040+0000 7fb9e31b4700 1 --2- 172.21.15.22:0/3784063811 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb9dc0ffd60 0x7fb9dc196440 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:23.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.040+0000 7fb9e31b4700 1 -- 172.21.15.22:0/3784063811 >> 172.21.15.22:0/3784063811 conn(0x7fb9dc0fac80 msgr2=0x7fb9dc1034f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:37:23.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.041+0000 7fb9e31b4700 1 -- 172.21.15.22:0/3784063811 shutdown_connections 2024-02-16T19:37:23.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.041+0000 7fb9e31b4700 1 -- 172.21.15.22:0/3784063811 wait complete. 2024-02-16T19:37:23.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:22 smithi117 ceph-mon[67870]: pgmap v160: 65 pgs: 65 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:37:23.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:22 smithi117 ceph-mon[67870]: from='client.34346 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:37:23.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:22 smithi117 ceph-mon[67870]: from='client.34350 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:37:23.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:22 smithi117 ceph-mon[67870]: from='client.34354 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:37:23.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.147+0000 7f931c7a7700 1 -- 172.21.15.22:0/2355070255 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9314104130 msgr2=0x7f9314104500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:37:23.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.147+0000 7f931c7a7700 1 --2- 172.21.15.22:0/2355070255 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9314104130 0x7f9314104500 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9308009910 tx=0x7f9308009c20 comp rx=0 tx=0).stop 2024-02-16T19:37:23.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.147+0000 7f931c7a7700 1 -- 172.21.15.22:0/2355070255 shutdown_connections 2024-02-16T19:37:23.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.147+0000 7f931c7a7700 1 --2- 172.21.15.22:0/2355070255 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93140ffbd0 0x7f9314100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:23.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.147+0000 7f931c7a7700 1 --2- 172.21.15.22:0/2355070255 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9314104130 0x7f9314104500 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:23.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.147+0000 7f931c7a7700 1 -- 172.21.15.22:0/2355070255 >> 172.21.15.22:0/2355070255 conn(0x7f9314075130 msgr2=0x7f9314075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:37:23.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.148+0000 7f931c7a7700 1 -- 172.21.15.22:0/2355070255 shutdown_connections 2024-02-16T19:37:23.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.148+0000 7f931c7a7700 1 -- 172.21.15.22:0/2355070255 wait complete. 2024-02-16T19:37:23.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.148+0000 7f931c7a7700 1 Processor -- start 2024-02-16T19:37:23.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.148+0000 7f931c7a7700 1 -- start start 2024-02-16T19:37:23.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.148+0000 7f931c7a7700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93140ffbd0 0x7f9314195e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:37:23.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.148+0000 7f931c7a7700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9314104130 0x7f9314196370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:37:23.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.148+0000 7f931c7a7700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9314196a50 con 0x7f9314104130 2024-02-16T19:37:23.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.148+0000 7f931c7a7700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f931419a7e0 con 0x7f93140ffbd0 2024-02-16T19:37:23.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.149+0000 7f9319d42700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9314104130 0x7f9314196370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:37:23.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.149+0000 7f9319d42700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9314104130 0x7f9314196370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58910/0 (socket says 172.21.15.22:58910) 2024-02-16T19:37:23.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.149+0000 7f9319d42700 1 -- 172.21.15.22:0/617577924 learned_addr learned my addr 172.21.15.22:0/617577924 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:37:23.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.149+0000 7f931a543700 1 --2- 172.21.15.22:0/617577924 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93140ffbd0 0x7f9314195e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:37:23.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.149+0000 7f9319d42700 1 -- 172.21.15.22:0/617577924 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93140ffbd0 msgr2=0x7f9314195e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:37:23.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.149+0000 7f9319d42700 1 --2- 172.21.15.22:0/617577924 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93140ffbd0 0x7f9314195e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:23.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.149+0000 7f9319d42700 1 -- 172.21.15.22:0/617577924 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93080095f0 con 0x7f9314104130 2024-02-16T19:37:23.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.149+0000 7f9319d42700 1 --2- 172.21.15.22:0/617577924 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9314104130 0x7f9314196370 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f931000e920 tx=0x7f931000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:37:23.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.149+0000 7f93077fe700 1 -- 172.21.15.22:0/617577924 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f931000c9d0 con 0x7f9314104130 2024-02-16T19:37:23.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.149+0000 7f93077fe700 1 -- 172.21.15.22:0/617577924 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9310010430 con 0x7f9314104130 2024-02-16T19:37:23.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.149+0000 7f931c7a7700 1 -- 172.21.15.22:0/617577924 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f931419aac0 con 0x7f9314104130 2024-02-16T19:37:23.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.149+0000 7f93077fe700 1 -- 172.21.15.22:0/617577924 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9310018630 con 0x7f9314104130 2024-02-16T19:37:23.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.149+0000 7f931c7a7700 1 -- 172.21.15.22:0/617577924 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f931419b010 con 0x7f9314104130 2024-02-16T19:37:23.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.150+0000 7f931c7a7700 1 -- 172.21.15.22:0/617577924 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f931404f2a0 con 0x7f9314104130 2024-02-16T19:37:23.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.151+0000 7f93077fe700 1 -- 172.21.15.22:0/617577924 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f93100105a0 con 0x7f9314104130 2024-02-16T19:37:23.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.151+0000 7f93077fe700 1 --2- 172.21.15.22:0/617577924 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f930006d8a0 0x7f930006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:37:23.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.151+0000 7f93077fe700 1 -- 172.21.15.22:0/617577924 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f9310014070 con 0x7f9314104130 2024-02-16T19:37:23.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.151+0000 7f931a543700 1 --2- 172.21.15.22:0/617577924 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f930006d8a0 0x7f930006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:37:23.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.152+0000 7f931a543700 1 --2- 172.21.15.22:0/617577924 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f930006d8a0 0x7f930006fd50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9308000c00 tx=0x7f9308009f70 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:37:23.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.156+0000 7f93077fe700 1 -- 172.21.15.22:0/617577924 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9310052d20 con 0x7f9314104130 2024-02-16T19:37:23.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.409+0000 7f931c7a7700 1 -- 172.21.15.22:0/617577924 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f931419ac50 con 0x7f9314104130 2024-02-16T19:37:23.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.409+0000 7f93077fe700 1 -- 172.21.15.22:0/617577924 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1845 (secure 0 0 0) 0x7f931419ac50 con 0x7f9314104130 2024-02-16T19:37:23.410 INFO:teuthology.orchestra.run.smithi022.stdout:e15 2024-02-16T19:37:23.410 INFO:teuthology.orchestra.run.smithi022.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T19:37:23.410 INFO:teuthology.orchestra.run.smithi022.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:37:23.410 INFO:teuthology.orchestra.run.smithi022.stdout:legacy client fscid: 1 2024-02-16T19:37:23.410 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:37:23.410 INFO:teuthology.orchestra.run.smithi022.stdout:Filesystem 'cephfs' (1) 2024-02-16T19:37:23.410 INFO:teuthology.orchestra.run.smithi022.stdout:fs_name cephfs 2024-02-16T19:37:23.410 INFO:teuthology.orchestra.run.smithi022.stdout:epoch 15 2024-02-16T19:37:23.410 INFO:teuthology.orchestra.run.smithi022.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T19:37:23.410 INFO:teuthology.orchestra.run.smithi022.stdout:created 2024-02-16T19:26:17.873644+0000 2024-02-16T19:37:23.410 INFO:teuthology.orchestra.run.smithi022.stdout:modified 2024-02-16T19:26:35.943353+0000 2024-02-16T19:37:23.411 INFO:teuthology.orchestra.run.smithi022.stdout:tableserver 0 2024-02-16T19:37:23.411 INFO:teuthology.orchestra.run.smithi022.stdout:root 0 2024-02-16T19:37:23.411 INFO:teuthology.orchestra.run.smithi022.stdout:session_timeout 60 2024-02-16T19:37:23.411 INFO:teuthology.orchestra.run.smithi022.stdout:session_autoclose 300 2024-02-16T19:37:23.411 INFO:teuthology.orchestra.run.smithi022.stdout:max_file_size 1099511627776 2024-02-16T19:37:23.411 INFO:teuthology.orchestra.run.smithi022.stdout:max_xattr_size 65536 2024-02-16T19:37:23.411 INFO:teuthology.orchestra.run.smithi022.stdout:required_client_features {} 2024-02-16T19:37:23.411 INFO:teuthology.orchestra.run.smithi022.stdout:last_failure 0 2024-02-16T19:37:23.411 INFO:teuthology.orchestra.run.smithi022.stdout:last_failure_osd_epoch 47 2024-02-16T19:37:23.411 INFO:teuthology.orchestra.run.smithi022.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:37:23.411 INFO:teuthology.orchestra.run.smithi022.stdout:max_mds 1 2024-02-16T19:37:23.411 INFO:teuthology.orchestra.run.smithi022.stdout:in 0 2024-02-16T19:37:23.411 INFO:teuthology.orchestra.run.smithi022.stdout:up {0=14490} 2024-02-16T19:37:23.411 INFO:teuthology.orchestra.run.smithi022.stdout:failed 2024-02-16T19:37:23.411 INFO:teuthology.orchestra.run.smithi022.stdout:damaged 2024-02-16T19:37:23.411 INFO:teuthology.orchestra.run.smithi022.stdout:stopped 2024-02-16T19:37:23.412 INFO:teuthology.orchestra.run.smithi022.stdout:data_pools [3] 2024-02-16T19:37:23.412 INFO:teuthology.orchestra.run.smithi022.stdout:metadata_pool 2 2024-02-16T19:37:23.412 INFO:teuthology.orchestra.run.smithi022.stdout:inline_data enabled 2024-02-16T19:37:23.412 INFO:teuthology.orchestra.run.smithi022.stdout:balancer 2024-02-16T19:37:23.412 INFO:teuthology.orchestra.run.smithi022.stdout:bal_rank_mask -1 2024-02-16T19:37:23.412 INFO:teuthology.orchestra.run.smithi022.stdout:standby_count_wanted 1 2024-02-16T19:37:23.412 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.cephfs.smithi022.btppwm{0:14490} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.22:6828/3597685751,v1:172.21.15.22:6829/3597685751] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:37:23.412 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:37:23.412 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:37:23.412 INFO:teuthology.orchestra.run.smithi022.stdout:Standby daemons: 2024-02-16T19:37:23.412 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:37:23.412 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.cephfs.smithi022.ocukvs{-1:14500} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.22:6826/242865843,v1:172.21.15.22:6827/242865843] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:37:23.412 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.cephfs.smithi117.ibmppn{-1:24259} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.117:6824/565408102,v1:172.21.15.117:6825/565408102] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:37:23.412 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.cephfs.smithi117.orzors{-1:24265} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.117:6826/3122342028,v1:172.21.15.117:6827/3122342028] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:37:23.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.412+0000 7f931c7a7700 1 -- 172.21.15.22:0/617577924 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f930006d8a0 msgr2=0x7f930006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:37:23.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.412+0000 7f931c7a7700 1 --2- 172.21.15.22:0/617577924 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f930006d8a0 0x7f930006fd50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9308000c00 tx=0x7f9308009f70 comp rx=0 tx=0).stop 2024-02-16T19:37:23.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.412+0000 7f931c7a7700 1 -- 172.21.15.22:0/617577924 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9314104130 msgr2=0x7f9314196370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:37:23.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.412+0000 7f931c7a7700 1 --2- 172.21.15.22:0/617577924 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9314104130 0x7f9314196370 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f931000e920 tx=0x7f931000ec30 comp rx=0 tx=0).stop 2024-02-16T19:37:23.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.412+0000 7f931c7a7700 1 -- 172.21.15.22:0/617577924 shutdown_connections 2024-02-16T19:37:23.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.412+0000 7f931c7a7700 1 --2- 172.21.15.22:0/617577924 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f930006d8a0 0x7f930006fd50 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:23.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.412+0000 7f931c7a7700 1 --2- 172.21.15.22:0/617577924 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93140ffbd0 0x7f9314195e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:23.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.412+0000 7f931c7a7700 1 --2- 172.21.15.22:0/617577924 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9314104130 0x7f9314196370 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:23.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.413+0000 7f931c7a7700 1 -- 172.21.15.22:0/617577924 >> 172.21.15.22:0/617577924 conn(0x7f9314075130 msgr2=0x7f93140685e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:37:23.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.413+0000 7f931c7a7700 1 -- 172.21.15.22:0/617577924 shutdown_connections 2024-02-16T19:37:23.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.413+0000 7f931c7a7700 1 -- 172.21.15.22:0/617577924 wait complete. 2024-02-16T19:37:23.414 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 15 2024-02-16T19:37:23.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.521+0000 7f7d806e1700 1 -- 172.21.15.22:0/3111433227 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d780ff3d0 msgr2=0x7f7d780ff7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:37:23.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.521+0000 7f7d806e1700 1 --2- 172.21.15.22:0/3111433227 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d780ff3d0 0x7f7d780ff7a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7d74009910 tx=0x7f7d74009c20 comp rx=0 tx=0).stop 2024-02-16T19:37:23.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.521+0000 7f7d806e1700 1 -- 172.21.15.22:0/3111433227 shutdown_connections 2024-02-16T19:37:23.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.521+0000 7f7d806e1700 1 --2- 172.21.15.22:0/3111433227 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d780ffce0 0x7f7d7810a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:23.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.522+0000 7f7d806e1700 1 --2- 172.21.15.22:0/3111433227 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d780ff3d0 0x7f7d780ff7a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:23.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.522+0000 7f7d806e1700 1 -- 172.21.15.22:0/3111433227 >> 172.21.15.22:0/3111433227 conn(0x7f7d78074930 msgr2=0x7f7d78074d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:37:23.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.522+0000 7f7d806e1700 1 -- 172.21.15.22:0/3111433227 shutdown_connections 2024-02-16T19:37:23.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.522+0000 7f7d806e1700 1 -- 172.21.15.22:0/3111433227 wait complete. 2024-02-16T19:37:23.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.522+0000 7f7d806e1700 1 Processor -- start 2024-02-16T19:37:23.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.522+0000 7f7d806e1700 1 -- start start 2024-02-16T19:37:23.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.522+0000 7f7d806e1700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d780ff3d0 0x7f7d78195e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:37:23.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.522+0000 7f7d806e1700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d780ffce0 0x7f7d781963b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:37:23.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.522+0000 7f7d806e1700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d78196a90 con 0x7f7d780ffce0 2024-02-16T19:37:23.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.522+0000 7f7d806e1700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7819a820 con 0x7f7d780ff3d0 2024-02-16T19:37:23.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.523+0000 7f7d7dc7c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d780ffce0 0x7f7d781963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:37:23.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.523+0000 7f7d7dc7c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d780ffce0 0x7f7d781963b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58934/0 (socket says 172.21.15.22:58934) 2024-02-16T19:37:23.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.523+0000 7f7d7dc7c700 1 -- 172.21.15.22:0/2047079492 learned_addr learned my addr 172.21.15.22:0/2047079492 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:37:23.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.523+0000 7f7d7e47d700 1 --2- 172.21.15.22:0/2047079492 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d780ff3d0 0x7f7d78195e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:37:23.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.523+0000 7f7d7dc7c700 1 -- 172.21.15.22:0/2047079492 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d780ff3d0 msgr2=0x7f7d78195e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:37:23.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.523+0000 7f7d7dc7c700 1 --2- 172.21.15.22:0/2047079492 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d780ff3d0 0x7f7d78195e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:23.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.523+0000 7f7d7dc7c700 1 -- 172.21.15.22:0/2047079492 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d740095f0 con 0x7f7d780ffce0 2024-02-16T19:37:23.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.523+0000 7f7d7dc7c700 1 --2- 172.21.15.22:0/2047079492 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d780ffce0 0x7f7d781963b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7d6800d8d0 tx=0x7f7d6800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:37:23.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.523+0000 7f7d6f7fe700 1 -- 172.21.15.22:0/2047079492 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d680098a0 con 0x7f7d780ffce0 2024-02-16T19:37:23.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.523+0000 7f7d806e1700 1 -- 172.21.15.22:0/2047079492 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d7819ab00 con 0x7f7d780ffce0 2024-02-16T19:37:23.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.523+0000 7f7d6f7fe700 1 -- 172.21.15.22:0/2047079492 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7d68010460 con 0x7f7d780ffce0 2024-02-16T19:37:23.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.523+0000 7f7d6f7fe700 1 -- 172.21.15.22:0/2047079492 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d6800f5f0 con 0x7f7d780ffce0 2024-02-16T19:37:23.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.523+0000 7f7d806e1700 1 -- 172.21.15.22:0/2047079492 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d7819b050 con 0x7f7d780ffce0 2024-02-16T19:37:23.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.524+0000 7f7d6d7fa700 1 -- 172.21.15.22:0/2047079492 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d600052f0 con 0x7f7d780ffce0 2024-02-16T19:37:23.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.525+0000 7f7d6f7fe700 1 -- 172.21.15.22:0/2047079492 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f7d68009a00 con 0x7f7d780ffce0 2024-02-16T19:37:23.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.525+0000 7f7d6f7fe700 1 --2- 172.21.15.22:0/2047079492 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f7d6406d8a0 0x7f7d6406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:37:23.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.525+0000 7f7d6f7fe700 1 -- 172.21.15.22:0/2047079492 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f7d6808d470 con 0x7f7d780ffce0 2024-02-16T19:37:23.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.525+0000 7f7d7e47d700 1 --2- 172.21.15.22:0/2047079492 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f7d6406d8a0 0x7f7d6406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:37:23.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.526+0000 7f7d7e47d700 1 --2- 172.21.15.22:0/2047079492 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f7d6406d8a0 0x7f7d6406fd50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7d74006010 tx=0x7f7d7400b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:37:23.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.530+0000 7f7d6f7fe700 1 -- 172.21.15.22:0/2047079492 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7d68056bb0 con 0x7f7d780ffce0 2024-02-16T19:37:23.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.761+0000 7f7d6d7fa700 1 -- 172.21.15.22:0/2047079492 --> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7d60000bc0 con 0x7f7d6406d8a0 2024-02-16T19:37:23.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.763+0000 7f7d6f7fe700 1 -- 172.21.15.22:0/2047079492 <== mgr.34104 v2:172.21.15.22:6800/505266753 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f7d60000bc0 con 0x7f7d6406d8a0 2024-02-16T19:37:23.763 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-02-16T19:37:23.763 INFO:teuthology.orchestra.run.smithi022.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T19:37:23.763 INFO:teuthology.orchestra.run.smithi022.stdout: "in_progress": true, 2024-02-16T19:37:23.763 INFO:teuthology.orchestra.run.smithi022.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T19:37:23.763 INFO:teuthology.orchestra.run.smithi022.stdout: "services_complete": [ 2024-02-16T19:37:23.763 INFO:teuthology.orchestra.run.smithi022.stdout: "crash", 2024-02-16T19:37:23.763 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr", 2024-02-16T19:37:23.764 INFO:teuthology.orchestra.run.smithi022.stdout: "mon" 2024-02-16T19:37:23.764 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-02-16T19:37:23.764 INFO:teuthology.orchestra.run.smithi022.stdout: "progress": "11/23 daemons upgraded", 2024-02-16T19:37:23.764 INFO:teuthology.orchestra.run.smithi022.stdout: "message": "Currently upgrading osd daemons", 2024-02-16T19:37:23.764 INFO:teuthology.orchestra.run.smithi022.stdout: "is_paused": false 2024-02-16T19:37:23.764 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-02-16T19:37:23.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.765+0000 7f7d6d7fa700 1 -- 172.21.15.22:0/2047079492 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f7d6406d8a0 msgr2=0x7f7d6406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:37:23.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.765+0000 7f7d6d7fa700 1 --2- 172.21.15.22:0/2047079492 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f7d6406d8a0 0x7f7d6406fd50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7d74006010 tx=0x7f7d7400b540 comp rx=0 tx=0).stop 2024-02-16T19:37:23.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.765+0000 7f7d6d7fa700 1 -- 172.21.15.22:0/2047079492 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d780ffce0 msgr2=0x7f7d781963b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:37:23.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.765+0000 7f7d6d7fa700 1 --2- 172.21.15.22:0/2047079492 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d780ffce0 0x7f7d781963b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7d6800d8d0 tx=0x7f7d6800dbe0 comp rx=0 tx=0).stop 2024-02-16T19:37:23.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.765+0000 7f7d6d7fa700 1 -- 172.21.15.22:0/2047079492 shutdown_connections 2024-02-16T19:37:23.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.765+0000 7f7d6d7fa700 1 --2- 172.21.15.22:0/2047079492 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f7d6406d8a0 0x7f7d6406fd50 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:23.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.765+0000 7f7d6d7fa700 1 --2- 172.21.15.22:0/2047079492 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d780ff3d0 0x7f7d78195e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:23.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.765+0000 7f7d6d7fa700 1 --2- 172.21.15.22:0/2047079492 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d780ffce0 0x7f7d781963b0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:23.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.765+0000 7f7d6d7fa700 1 -- 172.21.15.22:0/2047079492 >> 172.21.15.22:0/2047079492 conn(0x7f7d78074930 msgr2=0x7f7d78104f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:37:23.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.765+0000 7f7d6d7fa700 1 -- 172.21.15.22:0/2047079492 shutdown_connections 2024-02-16T19:37:23.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.765+0000 7f7d6d7fa700 1 -- 172.21.15.22:0/2047079492 wait complete. 2024-02-16T19:37:23.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.872+0000 7f8236070700 1 -- 172.21.15.22:0/311807250 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8230106320 msgr2=0x7f82301066f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:37:23.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.872+0000 7f8236070700 1 --2- 172.21.15.22:0/311807250 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8230106320 0x7f82301066f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8220009910 tx=0x7f8220009c20 comp rx=0 tx=0).stop 2024-02-16T19:37:23.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.872+0000 7f8236070700 1 -- 172.21.15.22:0/311807250 shutdown_connections 2024-02-16T19:37:23.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.872+0000 7f8236070700 1 --2- 172.21.15.22:0/311807250 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8230068540 0x7f82300689b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:23.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.872+0000 7f8236070700 1 --2- 172.21.15.22:0/311807250 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8230106320 0x7f82301066f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:23.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.872+0000 7f8236070700 1 -- 172.21.15.22:0/311807250 >> 172.21.15.22:0/311807250 conn(0x7f8230075130 msgr2=0x7f8230075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:37:23.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.872+0000 7f8236070700 1 -- 172.21.15.22:0/311807250 shutdown_connections 2024-02-16T19:37:23.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.872+0000 7f8236070700 1 -- 172.21.15.22:0/311807250 wait complete. 2024-02-16T19:37:23.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.872+0000 7f8236070700 1 Processor -- start 2024-02-16T19:37:23.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.873+0000 7f8236070700 1 -- start start 2024-02-16T19:37:23.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.873+0000 7f8236070700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8230068540 0x7f8230195e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:37:23.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.873+0000 7f8236070700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8230106320 0x7f82301963b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:37:23.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.873+0000 7f8236070700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8230196a90 con 0x7f8230106320 2024-02-16T19:37:23.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.873+0000 7f8236070700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f823019a820 con 0x7f8230068540 2024-02-16T19:37:23.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.873+0000 7f822effd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8230106320 0x7f82301963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:37:23.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.873+0000 7f822effd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8230106320 0x7f82301963b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58946/0 (socket says 172.21.15.22:58946) 2024-02-16T19:37:23.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.873+0000 7f822effd700 1 -- 172.21.15.22:0/188520358 learned_addr learned my addr 172.21.15.22:0/188520358 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:37:23.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.873+0000 7f822f7fe700 1 --2- 172.21.15.22:0/188520358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8230068540 0x7f8230195e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:37:23.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.873+0000 7f822effd700 1 -- 172.21.15.22:0/188520358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8230068540 msgr2=0x7f8230195e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:37:23.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.873+0000 7f822effd700 1 --2- 172.21.15.22:0/188520358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8230068540 0x7f8230195e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:23.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.873+0000 7f822effd700 1 -- 172.21.15.22:0/188520358 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82200095f0 con 0x7f8230106320 2024-02-16T19:37:23.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.873+0000 7f822f7fe700 1 --2- 172.21.15.22:0/188520358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8230068540 0x7f8230195e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T19:37:23.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.874+0000 7f822effd700 1 --2- 172.21.15.22:0/188520358 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8230106320 0x7f82301963b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f822400c8a0 tx=0x7f822400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:37:23.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.874+0000 7f822cff9700 1 -- 172.21.15.22:0/188520358 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f822400cea0 con 0x7f8230106320 2024-02-16T19:37:23.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.874+0000 7f8236070700 1 -- 172.21.15.22:0/188520358 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f823019ab00 con 0x7f8230106320 2024-02-16T19:37:23.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.874+0000 7f822cff9700 1 -- 172.21.15.22:0/188520358 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8224004830 con 0x7f8230106320 2024-02-16T19:37:23.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.874+0000 7f822cff9700 1 -- 172.21.15.22:0/188520358 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f82240055b0 con 0x7f8230106320 2024-02-16T19:37:23.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.874+0000 7f8236070700 1 -- 172.21.15.22:0/188520358 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f823019b050 con 0x7f8230106320 2024-02-16T19:37:23.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.875+0000 7f8236070700 1 -- 172.21.15.22:0/188520358 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8230108770 con 0x7f8230106320 2024-02-16T19:37:23.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.875+0000 7f822cff9700 1 -- 172.21.15.22:0/188520358 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f82240049a0 con 0x7f8230106320 2024-02-16T19:37:23.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.876+0000 7f822cff9700 1 --2- 172.21.15.22:0/188520358 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f821806d8a0 0x7f821806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:37:23.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.876+0000 7f822cff9700 1 -- 172.21.15.22:0/188520358 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f822408d260 con 0x7f8230106320 2024-02-16T19:37:23.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.876+0000 7f822f7fe700 1 --2- 172.21.15.22:0/188520358 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f821806d8a0 0x7f821806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:37:23.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.876+0000 7f822f7fe700 1 --2- 172.21.15.22:0/188520358 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f821806d8a0 0x7f821806fd50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f82200059d0 tx=0x7f822001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:37:23.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:23.880+0000 7f822cff9700 1 -- 172.21.15.22:0/188520358 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f82240569a0 con 0x7f8230106320 2024-02-16T19:37:24.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:23 smithi117 ceph-mon[67870]: from='client.? 172.21.15.22:0/3784063811' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:37:24.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:23 smithi117 ceph-mon[67870]: from='client.? 172.21.15.22:0/617577924' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:37:24.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:23 smithi117 ceph-mon[67870]: from='client.34366 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:37:24.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:24.172+0000 7f8236070700 1 -- 172.21.15.22:0/188520358 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f823004f2a0 con 0x7f8230106320 2024-02-16T19:37:24.173 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:23 smithi022 ceph-mon[88798]: from='client.? 172.21.15.22:0/3784063811' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:37:24.173 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:23 smithi022 ceph-mon[88798]: from='client.? 172.21.15.22:0/617577924' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:37:24.173 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:23 smithi022 ceph-mon[88798]: from='client.34366 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:37:24.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:24.172+0000 7f822cff9700 1 -- 172.21.15.22:0/188520358 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f8224004c50 con 0x7f8230106320 2024-02-16T19:37:24.173 INFO:teuthology.orchestra.run.smithi022.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-02-16T19:37:24.173 INFO:teuthology.orchestra.run.smithi022.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-02-16T19:37:24.174 INFO:teuthology.orchestra.run.smithi022.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-02-16T19:37:24.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:24.175+0000 7f8236070700 1 -- 172.21.15.22:0/188520358 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f821806d8a0 msgr2=0x7f821806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:37:24.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:24.175+0000 7f8236070700 1 --2- 172.21.15.22:0/188520358 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f821806d8a0 0x7f821806fd50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f82200059d0 tx=0x7f822001a040 comp rx=0 tx=0).stop 2024-02-16T19:37:24.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:24.175+0000 7f8236070700 1 -- 172.21.15.22:0/188520358 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8230106320 msgr2=0x7f82301963b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:37:24.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:24.175+0000 7f8236070700 1 --2- 172.21.15.22:0/188520358 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8230106320 0x7f82301963b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f822400c8a0 tx=0x7f822400cc60 comp rx=0 tx=0).stop 2024-02-16T19:37:24.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:24.175+0000 7f8236070700 1 -- 172.21.15.22:0/188520358 shutdown_connections 2024-02-16T19:37:24.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:24.175+0000 7f8236070700 1 --2- 172.21.15.22:0/188520358 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f821806d8a0 0x7f821806fd50 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:24.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:24.175+0000 7f8236070700 1 --2- 172.21.15.22:0/188520358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8230068540 0x7f8230195e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:24.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:24.175+0000 7f8236070700 1 --2- 172.21.15.22:0/188520358 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8230106320 0x7f82301963b0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:24.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:24.175+0000 7f8236070700 1 -- 172.21.15.22:0/188520358 >> 172.21.15.22:0/188520358 conn(0x7f8230075130 msgr2=0x7f82300fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:37:24.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:24.175+0000 7f8236070700 1 -- 172.21.15.22:0/188520358 shutdown_connections 2024-02-16T19:37:24.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:24.175+0000 7f8236070700 1 -- 172.21.15.22:0/188520358 wait complete. 2024-02-16T19:37:25.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:24 smithi117 ceph-mon[67870]: pgmap v161: 65 pgs: 65 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:37:25.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:24 smithi117 ceph-mon[67870]: from='client.? 172.21.15.22:0/188520358' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T19:37:25.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:24 smithi022 ceph-mon[88798]: pgmap v161: 65 pgs: 65 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:37:25.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:24 smithi022 ceph-mon[88798]: from='client.? 172.21.15.22:0/188520358' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T19:37:27.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:26 smithi117 ceph-mon[67870]: pgmap v162: 65 pgs: 65 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:37:27.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:26 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:37:27.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:26 smithi022 ceph-mon[88798]: pgmap v162: 65 pgs: 65 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:37:27.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:26 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:37:28.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:28 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-02-16T19:37:28.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:28 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-02-16T19:37:29.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:29 smithi117 ceph-mon[67870]: pgmap v163: 65 pgs: 65 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:37:29.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:29 smithi117 ceph-mon[67870]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-02-16T19:37:29.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:29 smithi117 ceph-mon[67870]: Upgrade: osd.5 is safe to restart 2024-02-16T19:37:29.616 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:29 smithi022 ceph-mon[88798]: pgmap v163: 65 pgs: 65 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:37:29.616 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:29 smithi022 ceph-mon[88798]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-02-16T19:37:29.616 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:29 smithi022 ceph-mon[88798]: Upgrade: osd.5 is safe to restart 2024-02-16T19:37:31.048 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:30 smithi117 ceph-mon[67870]: Upgrade: Updating osd.5 2024-02-16T19:37:31.048 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:30 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:37:31.048 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:30 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-16T19:37:31.048 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:30 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:37:31.048 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:30 smithi117 ceph-mon[67870]: Deploying daemon osd.5 on smithi117 2024-02-16T19:37:31.048 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:30 smithi117 ceph-mon[67870]: pgmap v164: 65 pgs: 65 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:37:31.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:30 smithi022 ceph-mon[88798]: Upgrade: Updating osd.5 2024-02-16T19:37:31.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:30 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:37:31.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:30 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-16T19:37:31.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:30 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:37:31.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:30 smithi022 ceph-mon[88798]: Deploying daemon osd.5 on smithi117 2024-02-16T19:37:31.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:30 smithi022 ceph-mon[88798]: pgmap v164: 65 pgs: 65 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:37:31.886 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:31 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:37:31.886 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:31 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:37:32.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:31 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:37:32.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:31 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:37:33.095 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:32 smithi117 ceph-mon[67870]: pgmap v165: 65 pgs: 65 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:37:33.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:32 smithi022 ceph-mon[88798]: pgmap v165: 65 pgs: 65 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:37:33.387 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:37:33 smithi117 systemd[1]: Stopping Ceph osd.5 for 2a1346e6-cd00-11ee-95ba-87774f69a715... 2024-02-16T19:37:33.387 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:37:33 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-5[50556]: 2024-02-16T19:37:33.334+0000 7f17357fe700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-16T19:37:33.387 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:37:33 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-5[50556]: 2024-02-16T19:37:33.334+0000 7f17357fe700 -1 osd.5 84 *** Got signal Terminated *** 2024-02-16T19:37:33.387 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:37:33 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-5[50556]: 2024-02-16T19:37:33.334+0000 7f17357fe700 -1 osd.5 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-16T19:37:34.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:33 smithi117 ceph-mon[67870]: osd.5 marked itself down and dead 2024-02-16T19:37:34.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:33 smithi022 ceph-mon[88798]: osd.5 marked itself down and dead 2024-02-16T19:37:34.637 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:37:34 smithi117 podman[87846]: 2024-02-16 19:37:34.409106529 +0000 UTC m=+1.174793883 container died fefaabca06385835e61e187739a1b1910f8cba2748e84b52288d2ca2e508e8e0 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-5, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, maintainer=Guillaume Abrioux , RELEASE=HEAD, io.buildah.version=1.29.1, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=https://github.com/ceph/ceph-container.git, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20231212) 2024-02-16T19:37:35.086 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:37:34 smithi117 podman[87846]: 2024-02-16 19:37:34.793478033 +0000 UTC m=+1.559165428 container cleanup fefaabca06385835e61e187739a1b1910f8cba2748e84b52288d2ca2e508e8e0 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-5, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=https://github.com/ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=-18.2.0, io.buildah.version=1.29.1, org.label-schema.license=GPLv2, org.label-schema.build-date=20231212, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, RELEASE=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-02-16T19:37:35.086 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:37:34 smithi117 bash[87846]: ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-5 2024-02-16T19:37:35.087 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:34 smithi117 ceph-mon[67870]: pgmap v166: 65 pgs: 65 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:37:35.087 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:34 smithi117 ceph-mon[67870]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-16T19:37:35.087 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:34 smithi117 ceph-mon[67870]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-02-16T19:37:35.087 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:34 smithi117 ceph-mon[67870]: osdmap e85: 6 total, 5 up, 6 in 2024-02-16T19:37:35.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:34 smithi022 ceph-mon[88798]: pgmap v166: 65 pgs: 65 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:37:35.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:34 smithi022 ceph-mon[88798]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-16T19:37:35.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:34 smithi022 ceph-mon[88798]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-02-16T19:37:35.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:34 smithi022 ceph-mon[88798]: osdmap e85: 6 total, 5 up, 6 in 2024-02-16T19:37:35.387 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:37:35 smithi117 podman[87863]: 2024-02-16 19:37:35.086360086 +0000 UTC m=+0.674438256 container remove fefaabca06385835e61e187739a1b1910f8cba2748e84b52288d2ca2e508e8e0 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-5, RELEASE=HEAD, io.buildah.version=1.29.1, org.label-schema.build-date=20231212, CEPH_POINT_RELEASE=-18.2.0, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-02-16T19:37:35.690 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:37:35 smithi117 podman[87955]: 2024-02-16 19:37:35.323149464 +0000 UTC m=+0.018708213 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T19:37:35.690 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:37:35 smithi117 podman[87955]: 2024-02-16 19:37:35.429056001 +0000 UTC m=+0.124614788 container create 0732acfc69d7f8d3523daa49dac3ce32495cdca862d2548cc2509c7c665d22e3 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-5-deactivate, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_CLEAN=True, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-02-16T19:37:35.690 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:37:35 smithi117 podman[87955]: 2024-02-16 19:37:35.689683718 +0000 UTC m=+0.385242463 container init 0732acfc69d7f8d3523daa49dac3ce32495cdca862d2548cc2509c7c665d22e3 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-5-deactivate, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214) 2024-02-16T19:37:35.982 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:35 smithi117 ceph-mon[67870]: osdmap e86: 6 total, 5 up, 6 in 2024-02-16T19:37:35.983 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:37:35 smithi117 podman[87955]: 2024-02-16 19:37:35.692926924 +0000 UTC m=+0.388485670 container start 0732acfc69d7f8d3523daa49dac3ce32495cdca862d2548cc2509c7c665d22e3 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.build-date=20240214) 2024-02-16T19:37:35.983 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:37:35 smithi117 podman[87955]: 2024-02-16 19:37:35.798110412 +0000 UTC m=+0.493669157 container attach 0732acfc69d7f8d3523daa49dac3ce32495cdca862d2548cc2509c7c665d22e3 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-5-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-02-16T19:37:36.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:35 smithi022 ceph-mon[88798]: osdmap e86: 6 total, 5 up, 6 in 2024-02-16T19:37:36.268 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:37:35 smithi117 podman[87955]: 2024-02-16 19:37:35.982415784 +0000 UTC m=+0.677974533 container died 0732acfc69d7f8d3523daa49dac3ce32495cdca862d2548cc2509c7c665d22e3 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-5-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True) 2024-02-16T19:37:36.857 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:37:36 smithi117 podman[87981]: 2024-02-16 19:37:36.518514256 +0000 UTC m=+0.526408645 container remove 0732acfc69d7f8d3523daa49dac3ce32495cdca862d2548cc2509c7c665d22e3 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2) 2024-02-16T19:37:36.857 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:37:36 smithi117 systemd[1]: ceph-2a1346e6-cd00-11ee-95ba-87774f69a715@osd.5.service: Deactivated successfully. 2024-02-16T19:37:36.857 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:37:36 smithi117 systemd[1]: Stopped Ceph osd.5 for 2a1346e6-cd00-11ee-95ba-87774f69a715. 2024-02-16T19:37:36.857 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:37:36 smithi117 systemd[1]: ceph-2a1346e6-cd00-11ee-95ba-87774f69a715@osd.5.service: Consumed 51.877s CPU time. 2024-02-16T19:37:36.857 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:37:36 smithi117 systemd[1]: Starting Ceph osd.5 for 2a1346e6-cd00-11ee-95ba-87774f69a715... 2024-02-16T19:37:37.120 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:36 smithi117 ceph-mon[67870]: pgmap v169: 65 pgs: 6 active+undersized, 5 active+undersized+degraded, 4 peering, 4 stale+active+clean, 46 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 14/267 objects degraded (5.243%) 2024-02-16T19:37:37.120 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:36 smithi117 ceph-mon[67870]: Health check failed: Reduced data availability: 1 pg inactive, 3 pgs peering (PG_AVAILABILITY) 2024-02-16T19:37:37.120 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:36 smithi117 ceph-mon[67870]: Health check failed: Degraded data redundancy: 14/267 objects degraded (5.243%), 5 pgs degraded (PG_DEGRADED) 2024-02-16T19:37:37.121 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:37:36 smithi117 podman[88085]: 2024-02-16 19:37:36.859673754 +0000 UTC m=+0.019381560 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T19:37:37.121 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:37:36 smithi117 podman[88085]: 2024-02-16 19:37:36.969808168 +0000 UTC m=+0.129516011 container create b89c6c582ef1a1c9a9bb4e92d2037dc794ec6549e1a49ff02086d72fcef4dc57 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-5-activate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-02-16T19:37:37.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:36 smithi022 ceph-mon[88798]: pgmap v169: 65 pgs: 6 active+undersized, 5 active+undersized+degraded, 4 peering, 4 stale+active+clean, 46 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 14/267 objects degraded (5.243%) 2024-02-16T19:37:37.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:36 smithi022 ceph-mon[88798]: Health check failed: Reduced data availability: 1 pg inactive, 3 pgs peering (PG_AVAILABILITY) 2024-02-16T19:37:37.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:36 smithi022 ceph-mon[88798]: Health check failed: Degraded data redundancy: 14/267 objects degraded (5.243%), 5 pgs degraded (PG_DEGRADED) 2024-02-16T19:37:37.637 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:37:37 smithi117 podman[88085]: 2024-02-16 19:37:37.255729023 +0000 UTC m=+0.415436823 container init b89c6c582ef1a1c9a9bb4e92d2037dc794ec6549e1a49ff02086d72fcef4dc57 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-5-activate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2) 2024-02-16T19:37:37.637 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:37:37 smithi117 podman[88085]: 2024-02-16 19:37:37.258923042 +0000 UTC m=+0.418630851 container start b89c6c582ef1a1c9a9bb4e92d2037dc794ec6549e1a49ff02086d72fcef4dc57 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-5-activate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-02-16T19:37:37.637 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:37:37 smithi117 podman[88085]: 2024-02-16 19:37:37.330555895 +0000 UTC m=+0.490263716 container attach b89c6c582ef1a1c9a9bb4e92d2037dc794ec6549e1a49ff02086d72fcef4dc57 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-5-activate, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, RELEASE=wip-batrick-testing-20240215.160715-f78a58c) 2024-02-16T19:37:38.068 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:37:37 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-5-activate[88100]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-02-16T19:37:38.068 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:37:37 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-5-activate[88100]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-02-16T19:37:38.068 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:37:37 smithi117 bash[88085]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-02-16T19:37:38.068 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:37:37 smithi117 bash[88085]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-02-16T19:37:38.068 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:37:37 smithi117 bash[88085]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-02-16T19:37:38.068 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:37:37 smithi117 bash[88085]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-02-16T19:37:38.068 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:37:37 smithi117 bash[88085]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-02-16T19:37:38.068 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:37:37 smithi117 bash[88085]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-02-16T19:37:38.069 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:37:37 smithi117 bash[88085]: --> ceph-volume raw activate successful for osd ID: 5 2024-02-16T19:37:38.069 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:37:37 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-5-activate[88100]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-02-16T19:37:38.069 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:37:37 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-5-activate[88100]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-02-16T19:37:38.069 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:37:37 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-5-activate[88100]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-02-16T19:37:38.069 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:37:37 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-5-activate[88100]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-02-16T19:37:38.069 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:37:37 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-5-activate[88100]: --> ceph-volume raw activate successful for osd ID: 5 2024-02-16T19:37:38.069 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:37:37 smithi117 podman[88085]: 2024-02-16 19:37:37.726517312 +0000 UTC m=+0.886225115 container died b89c6c582ef1a1c9a9bb4e92d2037dc794ec6549e1a49ff02086d72fcef4dc57 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-5-activate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-02-16T19:37:38.637 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:37:38 smithi117 podman[88221]: 2024-02-16 19:37:38.342933427 +0000 UTC m=+0.610875975 container remove b89c6c582ef1a1c9a9bb4e92d2037dc794ec6549e1a49ff02086d72fcef4dc57 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-5-activate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240215.160715-f78a58c) 2024-02-16T19:37:38.637 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:37:38 smithi117 podman[88287]: 2024-02-16 19:37:38.462745284 +0000 UTC m=+0.017508327 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T19:37:38.638 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:37:38 smithi117 podman[88287]: 2024-02-16 19:37:38.568717658 +0000 UTC m=+0.123480696 container create a1a7625b7541fed907d97359d1cbacf5d0d164248b6c183e88ba88d5a3ba49b4 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=) 2024-02-16T19:37:39.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:38 smithi117 ceph-mon[67870]: pgmap v170: 65 pgs: 9 active+undersized, 8 active+undersized+degraded, 4 peering, 1 stale+active+clean, 43 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 24/267 objects degraded (8.989%) 2024-02-16T19:37:39.137 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:37:38 smithi117 podman[88287]: 2024-02-16 19:37:38.855153845 +0000 UTC m=+0.409916889 container init a1a7625b7541fed907d97359d1cbacf5d0d164248b6c183e88ba88d5a3ba49b4 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-16T19:37:39.137 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:37:38 smithi117 podman[88287]: 2024-02-16 19:37:38.858458357 +0000 UTC m=+0.413221398 container start a1a7625b7541fed907d97359d1cbacf5d0d164248b6c183e88ba88d5a3ba49b4 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-5, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-02-16T19:37:39.138 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:37:38 smithi117 ceph-osd[88306]: -- 172.21.15.117:0/1950667035 <== mon.1 v2:172.21.15.117:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a37631b1e0 con 0x55a3762ecc00 2024-02-16T19:37:39.138 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:37:38 smithi117 bash[88287]: a1a7625b7541fed907d97359d1cbacf5d0d164248b6c183e88ba88d5a3ba49b4 2024-02-16T19:37:39.138 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:37:38 smithi117 systemd[1]: Started Ceph osd.5 for 2a1346e6-cd00-11ee-95ba-87774f69a715. 2024-02-16T19:37:39.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:38 smithi022 ceph-mon[88798]: pgmap v170: 65 pgs: 9 active+undersized, 8 active+undersized+degraded, 4 peering, 1 stale+active+clean, 43 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 24/267 objects degraded (8.989%) 2024-02-16T19:37:39.851 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:37:39 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-5[88302]: 2024-02-16T19:37:39.408+0000 7f45585747c0 -1 Falling back to public interface 2024-02-16T19:37:40.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:39 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:37:40.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:39 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:37:40.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:39 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:37:40.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:39 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:37:40.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:39 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:37:40.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:39 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:37:41.326 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:40 smithi117 ceph-mon[67870]: pgmap v171: 65 pgs: 15 active+undersized, 14 active+undersized+degraded, 36 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 48/267 objects degraded (17.978%) 2024-02-16T19:37:41.326 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:40 smithi117 ceph-mon[67870]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 3 pgs peering) 2024-02-16T19:37:41.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:40 smithi022 ceph-mon[88798]: pgmap v171: 65 pgs: 15 active+undersized, 14 active+undersized+degraded, 36 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 48/267 objects degraded (17.978%) 2024-02-16T19:37:41.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:40 smithi022 ceph-mon[88798]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 3 pgs peering) 2024-02-16T19:37:43.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:42 smithi117 ceph-mon[67870]: pgmap v172: 65 pgs: 15 active+undersized, 14 active+undersized+degraded, 36 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 48/267 objects degraded (17.978%) 2024-02-16T19:37:43.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:42 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:37:43.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:42 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:37:43.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:42 smithi117 ceph-mon[67870]: Health check update: Degraded data redundancy: 48/267 objects degraded (17.978%), 14 pgs degraded (PG_DEGRADED) 2024-02-16T19:37:43.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:42 smithi022 ceph-mon[88798]: pgmap v172: 65 pgs: 15 active+undersized, 14 active+undersized+degraded, 36 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 48/267 objects degraded (17.978%) 2024-02-16T19:37:43.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:42 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:37:43.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:42 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:37:43.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:42 smithi022 ceph-mon[88798]: Health check update: Degraded data redundancy: 48/267 objects degraded (17.978%), 14 pgs degraded (PG_DEGRADED) 2024-02-16T19:37:43.833 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:37:43 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-5[88302]: 2024-02-16T19:37:43.570+0000 7f45585747c0 -1 osd.5 0 read_superblock omap replica is missing. 2024-02-16T19:37:44.137 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:37:43 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-5[88302]: 2024-02-16T19:37:43.859+0000 7f45585747c0 -1 osd.5 84 log_to_monitors true 2024-02-16T19:37:44.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:44 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:37:44.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:44 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:37:44.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:44 smithi117 ceph-mon[67870]: from='osd.5 [v2:172.21.15.117:6816/1403457815,v1:172.21.15.117:6817/1403457815]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-16T19:37:44.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:44 smithi117 ceph-mon[67870]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-16T19:37:44.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:44 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:37:44.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:44 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:37:44.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:44 smithi022 ceph-mon[88798]: from='osd.5 [v2:172.21.15.117:6816/1403457815,v1:172.21.15.117:6817/1403457815]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-16T19:37:44.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:44 smithi022 ceph-mon[88798]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-16T19:37:44.982 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:37:44 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-5[88302]: 2024-02-16T19:37:44.916+0000 7f45459c4700 -1 osd.5 84 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-16T19:37:45.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:45 smithi117 ceph-mon[67870]: pgmap v173: 65 pgs: 15 active+undersized, 14 active+undersized+degraded, 36 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 48/267 objects degraded (17.978%) 2024-02-16T19:37:45.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:45 smithi117 ceph-mon[67870]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-16T19:37:45.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:45 smithi117 ceph-mon[67870]: osdmap e87: 6 total, 5 up, 6 in 2024-02-16T19:37:45.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:45 smithi117 ceph-mon[67870]: from='osd.5 [v2:172.21.15.117:6816/1403457815,v1:172.21.15.117:6817/1403457815]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-02-16T19:37:45.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:45 smithi117 ceph-mon[67870]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-02-16T19:37:45.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:45 smithi117 ceph-mon[67870]: from='osd.5 ' entity='osd.5' 2024-02-16T19:37:45.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:45 smithi022 ceph-mon[88798]: pgmap v173: 65 pgs: 15 active+undersized, 14 active+undersized+degraded, 36 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 48/267 objects degraded (17.978%) 2024-02-16T19:37:45.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:45 smithi022 ceph-mon[88798]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-16T19:37:45.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:45 smithi022 ceph-mon[88798]: osdmap e87: 6 total, 5 up, 6 in 2024-02-16T19:37:45.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:45 smithi022 ceph-mon[88798]: from='osd.5 [v2:172.21.15.117:6816/1403457815,v1:172.21.15.117:6817/1403457815]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-02-16T19:37:45.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:45 smithi022 ceph-mon[88798]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-02-16T19:37:45.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:45 smithi022 ceph-mon[88798]: from='osd.5 ' entity='osd.5' 2024-02-16T19:37:47.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:46 smithi117 ceph-mon[67870]: pgmap v175: 65 pgs: 15 active+undersized, 14 active+undersized+degraded, 36 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 48/267 objects degraded (17.978%) 2024-02-16T19:37:47.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:46 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:37:47.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:46 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:37:47.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:46 smithi117 ceph-mon[67870]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-16T19:37:47.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:46 smithi117 ceph-mon[67870]: osd.5 [v2:172.21.15.117:6816/1403457815,v1:172.21.15.117:6817/1403457815] boot 2024-02-16T19:37:47.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:46 smithi117 ceph-mon[67870]: osdmap e88: 6 total, 6 up, 6 in 2024-02-16T19:37:47.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:46 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-16T19:37:47.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:46 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:37:47.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:46 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:37:47.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:46 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:37:47.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:46 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:37:47.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:46 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:37:47.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:46 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:37:47.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:46 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:37:47.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:46 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:37:47.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:46 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:37:47.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:46 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:37:47.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:46 smithi117 ceph-mon[67870]: Upgrade: Setting container_image for all osd 2024-02-16T19:37:47.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:46 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:37:47.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:46 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-02-16T19:37:47.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:46 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-02-16T19:37:47.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:46 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-02-16T19:37:47.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:46 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-02-16T19:37:47.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:46 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-02-16T19:37:47.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:46 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-02-16T19:37:47.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:46 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-02-16T19:37:47.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:46 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-02-16T19:37:47.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:46 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-02-16T19:37:47.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:46 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-02-16T19:37:47.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:46 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-02-16T19:37:47.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:46 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-02-16T19:37:47.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:46 smithi117 ceph-mon[67870]: Upgrade: Setting require_osd_release to 19 squid 2024-02-16T19:37:47.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:46 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-02-16T19:37:47.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:46 smithi022 ceph-mon[88798]: pgmap v175: 65 pgs: 15 active+undersized, 14 active+undersized+degraded, 36 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 48/267 objects degraded (17.978%) 2024-02-16T19:37:47.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:46 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:37:47.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:46 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:37:47.216 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:46 smithi022 ceph-mon[88798]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-16T19:37:47.216 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:46 smithi022 ceph-mon[88798]: osd.5 [v2:172.21.15.117:6816/1403457815,v1:172.21.15.117:6817/1403457815] boot 2024-02-16T19:37:47.216 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:46 smithi022 ceph-mon[88798]: osdmap e88: 6 total, 6 up, 6 in 2024-02-16T19:37:47.216 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:46 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-16T19:37:47.216 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:46 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:37:47.216 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:46 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:37:47.216 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:46 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:37:47.216 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:46 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:37:47.216 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:46 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:37:47.216 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:46 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:37:47.216 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:46 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:37:47.216 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:46 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:37:47.216 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:46 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:37:47.217 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:46 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:37:47.217 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:46 smithi022 ceph-mon[88798]: Upgrade: Setting container_image for all osd 2024-02-16T19:37:47.217 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:46 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:37:47.217 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:46 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-02-16T19:37:47.217 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:46 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-02-16T19:37:47.217 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:46 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-02-16T19:37:47.217 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:46 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-02-16T19:37:47.217 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:46 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-02-16T19:37:47.217 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:46 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-02-16T19:37:47.217 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:46 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-02-16T19:37:47.217 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:46 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-02-16T19:37:47.217 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:46 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-02-16T19:37:47.217 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:46 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-02-16T19:37:47.217 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:46 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-02-16T19:37:47.218 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:46 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-02-16T19:37:47.218 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:46 smithi022 ceph-mon[88798]: Upgrade: Setting require_osd_release to 19 squid 2024-02-16T19:37:47.218 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:46 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-02-16T19:37:48.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:47 smithi117 ceph-mon[67870]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-02-16T19:37:48.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:47 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-02-16T19:37:48.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:47 smithi117 ceph-mon[67870]: osdmap e89: 6 total, 6 up, 6 in 2024-02-16T19:37:48.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:47 smithi022 ceph-mon[88798]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-02-16T19:37:48.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:47 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-02-16T19:37:48.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:47 smithi022 ceph-mon[88798]: osdmap e89: 6 total, 6 up, 6 in 2024-02-16T19:37:49.857 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:49 smithi022 ceph-mon[88798]: pgmap v178: 65 pgs: 15 active+undersized, 14 active+undersized+degraded, 36 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 48/267 objects degraded (17.978%) 2024-02-16T19:37:49.857 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:49 smithi022 ceph-mon[88798]: Upgrade: Updating mds.cephfs.smithi022.ocukvs 2024-02-16T19:37:49.857 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:49 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:37:49.857 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:49 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi022.ocukvs", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T19:37:49.857 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:49 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:37:49.857 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:49 smithi022 ceph-mon[88798]: Deploying daemon mds.cephfs.smithi022.ocukvs on smithi022 2024-02-16T19:37:49.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:49 smithi117 ceph-mon[67870]: pgmap v178: 65 pgs: 15 active+undersized, 14 active+undersized+degraded, 36 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 48/267 objects degraded (17.978%) 2024-02-16T19:37:49.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:49 smithi117 ceph-mon[67870]: Upgrade: Updating mds.cephfs.smithi022.ocukvs 2024-02-16T19:37:49.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:49 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:37:49.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:49 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi022.ocukvs", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T19:37:49.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:49 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:37:49.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:49 smithi117 ceph-mon[67870]: Deploying daemon mds.cephfs.smithi022.ocukvs on smithi022 2024-02-16T19:37:50.809 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:50 smithi022 ceph-mon[88798]: Health check update: Degraded data redundancy: 15/267 objects degraded (5.618%), 5 pgs degraded (PG_DEGRADED) 2024-02-16T19:37:51.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:50 smithi117 ceph-mon[67870]: Health check update: Degraded data redundancy: 15/267 objects degraded (5.618%), 5 pgs degraded (PG_DEGRADED) 2024-02-16T19:37:52.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:51 smithi117 ceph-mon[67870]: pgmap v179: 65 pgs: 3 active+undersized, 5 active+undersized+degraded, 57 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 15/267 objects degraded (5.618%) 2024-02-16T19:37:52.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:51 smithi022 ceph-mon[88798]: pgmap v179: 65 pgs: 3 active+undersized, 5 active+undersized+degraded, 57 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 15/267 objects degraded (5.618%) 2024-02-16T19:37:53.006 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:52 smithi022 ceph-mon[88798]: osdmap e90: 6 total, 6 up, 6 in 2024-02-16T19:37:53.006 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:52 smithi022 ceph-mon[88798]: fsmap cephfs:1 {0=cephfs.smithi022.btppwm=up:active} 2 up:standby 2024-02-16T19:37:53.006 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:52 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi022.ocukvs"}]: dispatch 2024-02-16T19:37:53.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:52 smithi117 ceph-mon[67870]: osdmap e90: 6 total, 6 up, 6 in 2024-02-16T19:37:53.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:52 smithi117 ceph-mon[67870]: fsmap cephfs:1 {0=cephfs.smithi022.btppwm=up:active} 2 up:standby 2024-02-16T19:37:53.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:52 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi022.ocukvs"}]: dispatch 2024-02-16T19:37:54.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:53 smithi117 ceph-mon[67870]: pgmap v181: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:37:54.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:53 smithi117 ceph-mon[67870]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 15/267 objects degraded (5.618%), 5 pgs degraded) 2024-02-16T19:37:54.164 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:53 smithi022 ceph-mon[88798]: pgmap v181: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:37:54.165 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:53 smithi022 ceph-mon[88798]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 15/267 objects degraded (5.618%), 5 pgs degraded) 2024-02-16T19:37:54.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.299+0000 7f0c1729f700 1 -- 172.21.15.22:0/1820629624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c10104130 msgr2=0x7f0c10104500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:37:54.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.299+0000 7f0c1729f700 1 --2- 172.21.15.22:0/1820629624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c10104130 0x7f0c10104500 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c009910 tx=0x7f0c0c009c20 comp rx=0 tx=0).stop 2024-02-16T19:37:54.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.299+0000 7f0c1729f700 1 -- 172.21.15.22:0/1820629624 shutdown_connections 2024-02-16T19:37:54.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.299+0000 7f0c1729f700 1 --2- 172.21.15.22:0/1820629624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c100ffbd0 0x7f0c10100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:54.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.299+0000 7f0c1729f700 1 --2- 172.21.15.22:0/1820629624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c10104130 0x7f0c10104500 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:54.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.299+0000 7f0c1729f700 1 -- 172.21.15.22:0/1820629624 >> 172.21.15.22:0/1820629624 conn(0x7f0c10075130 msgr2=0x7f0c10075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:37:54.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.299+0000 7f0c1729f700 1 -- 172.21.15.22:0/1820629624 shutdown_connections 2024-02-16T19:37:54.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.299+0000 7f0c1729f700 1 -- 172.21.15.22:0/1820629624 wait complete. 2024-02-16T19:37:54.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.299+0000 7f0c1729f700 1 Processor -- start 2024-02-16T19:37:54.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.300+0000 7f0c1729f700 1 -- start start 2024-02-16T19:37:54.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.300+0000 7f0c1729f700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c100ffbd0 0x7f0c10196db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:37:54.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.300+0000 7f0c1729f700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c10104130 0x7f0c101972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:37:54.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.300+0000 7f0c1729f700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c101979d0 con 0x7f0c100ffbd0 2024-02-16T19:37:54.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.300+0000 7f0c1729f700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c10197b40 con 0x7f0c10104130 2024-02-16T19:37:54.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.300+0000 7f0c1503b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c100ffbd0 0x7f0c10196db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:37:54.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.300+0000 7f0c1503b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c100ffbd0 0x7f0c10196db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39172/0 (socket says 172.21.15.22:39172) 2024-02-16T19:37:54.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.300+0000 7f0c1503b700 1 -- 172.21.15.22:0/2280343498 learned_addr learned my addr 172.21.15.22:0/2280343498 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:37:54.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.300+0000 7f0c1483a700 1 --2- 172.21.15.22:0/2280343498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c10104130 0x7f0c101972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:37:54.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.300+0000 7f0c1503b700 1 -- 172.21.15.22:0/2280343498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c10104130 msgr2=0x7f0c101972f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:37:54.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.300+0000 7f0c1503b700 1 --2- 172.21.15.22:0/2280343498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c10104130 0x7f0c101972f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:54.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.300+0000 7f0c1503b700 1 -- 172.21.15.22:0/2280343498 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c0c0095f0 con 0x7f0c100ffbd0 2024-02-16T19:37:54.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.300+0000 7f0c1503b700 1 --2- 172.21.15.22:0/2280343498 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c100ffbd0 0x7f0c10196db0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c00bd70 tx=0x7f0c0c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:37:54.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.301+0000 7f0c067fc700 1 -- 172.21.15.22:0/2280343498 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c0c01d070 con 0x7f0c100ffbd0 2024-02-16T19:37:54.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.301+0000 7f0c1729f700 1 -- 172.21.15.22:0/2280343498 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c1019fe20 con 0x7f0c100ffbd0 2024-02-16T19:37:54.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.301+0000 7f0c067fc700 1 -- 172.21.15.22:0/2280343498 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0c0c00f460 con 0x7f0c100ffbd0 2024-02-16T19:37:54.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.301+0000 7f0c067fc700 1 -- 172.21.15.22:0/2280343498 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c0c017630 con 0x7f0c100ffbd0 2024-02-16T19:37:54.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.301+0000 7f0c1729f700 1 -- 172.21.15.22:0/2280343498 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c101a0290 con 0x7f0c100ffbd0 2024-02-16T19:37:54.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.302+0000 7f0c1729f700 1 -- 172.21.15.22:0/2280343498 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c101076a0 con 0x7f0c100ffbd0 2024-02-16T19:37:54.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.302+0000 7f0c067fc700 1 -- 172.21.15.22:0/2280343498 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f0c0c003a60 con 0x7f0c100ffbd0 2024-02-16T19:37:54.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.302+0000 7f0c067fc700 1 --2- 172.21.15.22:0/2280343498 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f0bfc06d8a0 0x7f0bfc06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:37:54.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.303+0000 7f0c067fc700 1 -- 172.21.15.22:0/2280343498 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f0c0c08ebe0 con 0x7f0c100ffbd0 2024-02-16T19:37:54.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.303+0000 7f0c1483a700 1 --2- 172.21.15.22:0/2280343498 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f0bfc06d8a0 0x7f0bfc06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:37:54.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.303+0000 7f0c1483a700 1 --2- 172.21.15.22:0/2280343498 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f0bfc06d8a0 0x7f0bfc06fd50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0c00006fd0 tx=0x7f0c00008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:37:54.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.307+0000 7f0c067fc700 1 -- 172.21.15.22:0/2280343498 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0c0c058320 con 0x7f0c100ffbd0 2024-02-16T19:37:54.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.535+0000 7f0c1729f700 1 -- 172.21.15.22:0/2280343498 --> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0c10061190 con 0x7f0bfc06d8a0 2024-02-16T19:37:54.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.536+0000 7f0c067fc700 1 -- 172.21.15.22:0/2280343498 <== mgr.34104 v2:172.21.15.22:6800/505266753 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f0c10061190 con 0x7f0bfc06d8a0 2024-02-16T19:37:54.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.538+0000 7f0c1729f700 1 -- 172.21.15.22:0/2280343498 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f0bfc06d8a0 msgr2=0x7f0bfc06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:37:54.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.538+0000 7f0c1729f700 1 --2- 172.21.15.22:0/2280343498 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f0bfc06d8a0 0x7f0bfc06fd50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0c00006fd0 tx=0x7f0c00008040 comp rx=0 tx=0).stop 2024-02-16T19:37:54.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.538+0000 7f0c1729f700 1 -- 172.21.15.22:0/2280343498 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c100ffbd0 msgr2=0x7f0c10196db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:37:54.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.538+0000 7f0c1729f700 1 --2- 172.21.15.22:0/2280343498 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c100ffbd0 0x7f0c10196db0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c00bd70 tx=0x7f0c0c00be50 comp rx=0 tx=0).stop 2024-02-16T19:37:54.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.538+0000 7f0c1729f700 1 -- 172.21.15.22:0/2280343498 shutdown_connections 2024-02-16T19:37:54.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.538+0000 7f0c1729f700 1 --2- 172.21.15.22:0/2280343498 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f0bfc06d8a0 0x7f0bfc06fd50 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:54.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.538+0000 7f0c1729f700 1 --2- 172.21.15.22:0/2280343498 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c100ffbd0 0x7f0c10196db0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:54.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.538+0000 7f0c1729f700 1 --2- 172.21.15.22:0/2280343498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c10104130 0x7f0c101972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:54.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.538+0000 7f0c1729f700 1 -- 172.21.15.22:0/2280343498 >> 172.21.15.22:0/2280343498 conn(0x7f0c10075130 msgr2=0x7f0c100fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:37:54.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.539+0000 7f0c1729f700 1 -- 172.21.15.22:0/2280343498 shutdown_connections 2024-02-16T19:37:54.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.539+0000 7f0c1729f700 1 -- 172.21.15.22:0/2280343498 wait complete. 2024-02-16T19:37:54.550 INFO:teuthology.orchestra.run.smithi022.stdout:true 2024-02-16T19:37:54.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.649+0000 7f24d360b700 1 -- 172.21.15.22:0/2313474513 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24cc101420 msgr2=0x7f24cc1017f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:37:54.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.649+0000 7f24d360b700 1 --2- 172.21.15.22:0/2313474513 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24cc101420 0x7f24cc1017f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f24c8009910 tx=0x7f24c8009c20 comp rx=0 tx=0).stop 2024-02-16T19:37:54.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.649+0000 7f24d360b700 1 -- 172.21.15.22:0/2313474513 shutdown_connections 2024-02-16T19:37:54.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.649+0000 7f24d360b700 1 --2- 172.21.15.22:0/2313474513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24cc101d30 0x7f24cc10a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:54.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.649+0000 7f24d360b700 1 --2- 172.21.15.22:0/2313474513 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24cc101420 0x7f24cc1017f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:54.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.649+0000 7f24d360b700 1 -- 172.21.15.22:0/2313474513 >> 172.21.15.22:0/2313474513 conn(0x7f24cc0fac80 msgr2=0x7f24cc0fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:37:54.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.650+0000 7f24d360b700 1 -- 172.21.15.22:0/2313474513 shutdown_connections 2024-02-16T19:37:54.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.650+0000 7f24d360b700 1 -- 172.21.15.22:0/2313474513 wait complete. 2024-02-16T19:37:54.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.650+0000 7f24d360b700 1 Processor -- start 2024-02-16T19:37:54.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.650+0000 7f24d360b700 1 -- start start 2024-02-16T19:37:54.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.651+0000 7f24d360b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24cc101420 0x7f24cc195fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:37:54.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.651+0000 7f24d360b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24cc101d30 0x7f24cc196510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:37:54.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.651+0000 7f24d360b700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24cc196bf0 con 0x7f24cc101420 2024-02-16T19:37:54.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.651+0000 7f24d360b700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24cc19a980 con 0x7f24cc101d30 2024-02-16T19:37:54.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.651+0000 7f24d13a7700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24cc101420 0x7f24cc195fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:37:54.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.651+0000 7f24d13a7700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24cc101420 0x7f24cc195fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50986/0 (socket says 172.21.15.22:50986) 2024-02-16T19:37:54.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.651+0000 7f24d13a7700 1 -- 172.21.15.22:0/3641439557 learned_addr learned my addr 172.21.15.22:0/3641439557 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:37:54.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.651+0000 7f24d0ba6700 1 --2- 172.21.15.22:0/3641439557 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24cc101d30 0x7f24cc196510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:37:54.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.651+0000 7f24d13a7700 1 -- 172.21.15.22:0/3641439557 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24cc101d30 msgr2=0x7f24cc196510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:37:54.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.651+0000 7f24d13a7700 1 --2- 172.21.15.22:0/3641439557 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24cc101d30 0x7f24cc196510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:54.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.651+0000 7f24d13a7700 1 -- 172.21.15.22:0/3641439557 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24c80095f0 con 0x7f24cc101420 2024-02-16T19:37:54.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.651+0000 7f24d0ba6700 1 --2- 172.21.15.22:0/3641439557 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24cc101d30 0x7f24cc196510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T19:37:54.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.651+0000 7f24d13a7700 1 --2- 172.21.15.22:0/3641439557 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24cc101420 0x7f24cc195fd0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f24c8006010 tx=0x7f24c800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:37:54.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.651+0000 7f24c27fc700 1 -- 172.21.15.22:0/3641439557 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f24c801d070 con 0x7f24cc101420 2024-02-16T19:37:54.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.652+0000 7f24d360b700 1 -- 172.21.15.22:0/3641439557 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24cc19ac00 con 0x7f24cc101420 2024-02-16T19:37:54.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.652+0000 7f24c27fc700 1 -- 172.21.15.22:0/3641439557 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f24c8004fb0 con 0x7f24cc101420 2024-02-16T19:37:54.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.652+0000 7f24c27fc700 1 -- 172.21.15.22:0/3641439557 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f24c8022ea0 con 0x7f24cc101420 2024-02-16T19:37:54.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.652+0000 7f24d360b700 1 -- 172.21.15.22:0/3641439557 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24cc19b0f0 con 0x7f24cc101420 2024-02-16T19:37:54.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.653+0000 7f24d360b700 1 -- 172.21.15.22:0/3641439557 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24cc1078e0 con 0x7f24cc101420 2024-02-16T19:37:54.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.653+0000 7f24c27fc700 1 -- 172.21.15.22:0/3641439557 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f24c8022470 con 0x7f24cc101420 2024-02-16T19:37:54.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.654+0000 7f24c27fc700 1 --2- 172.21.15.22:0/3641439557 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f24b806d8a0 0x7f24b806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:37:54.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.654+0000 7f24c27fc700 1 -- 172.21.15.22:0/3641439557 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f24c808ea30 con 0x7f24cc101420 2024-02-16T19:37:54.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.654+0000 7f24d0ba6700 1 --2- 172.21.15.22:0/3641439557 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f24b806d8a0 0x7f24b806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:37:54.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.654+0000 7f24d0ba6700 1 --2- 172.21.15.22:0/3641439557 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f24b806d8a0 0x7f24b806fd50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f24cc1975f0 tx=0x7f24bc008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:37:54.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.659+0000 7f24c27fc700 1 -- 172.21.15.22:0/3641439557 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f24c8090e60 con 0x7f24cc101420 2024-02-16T19:37:54.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.891+0000 7f24d360b700 1 -- 172.21.15.22:0/3641439557 --> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f24cc19b420 con 0x7f24b806d8a0 2024-02-16T19:37:54.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.892+0000 7f24c27fc700 1 -- 172.21.15.22:0/3641439557 <== mgr.34104 v2:172.21.15.22:6800/505266753 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f24cc19b420 con 0x7f24b806d8a0 2024-02-16T19:37:54.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.894+0000 7f24d360b700 1 -- 172.21.15.22:0/3641439557 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f24b806d8a0 msgr2=0x7f24b806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:37:54.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.894+0000 7f24d360b700 1 --2- 172.21.15.22:0/3641439557 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f24b806d8a0 0x7f24b806fd50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f24cc1975f0 tx=0x7f24bc008040 comp rx=0 tx=0).stop 2024-02-16T19:37:54.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.894+0000 7f24d360b700 1 -- 172.21.15.22:0/3641439557 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24cc101420 msgr2=0x7f24cc195fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:37:54.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.894+0000 7f24d360b700 1 --2- 172.21.15.22:0/3641439557 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24cc101420 0x7f24cc195fd0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f24c8006010 tx=0x7f24c800be20 comp rx=0 tx=0).stop 2024-02-16T19:37:54.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.894+0000 7f24d360b700 1 -- 172.21.15.22:0/3641439557 shutdown_connections 2024-02-16T19:37:54.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.894+0000 7f24d360b700 1 --2- 172.21.15.22:0/3641439557 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f24b806d8a0 0x7f24b806fd50 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:54.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.894+0000 7f24d360b700 1 --2- 172.21.15.22:0/3641439557 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24cc101420 0x7f24cc195fd0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:54.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.894+0000 7f24d360b700 1 --2- 172.21.15.22:0/3641439557 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24cc101d30 0x7f24cc196510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:54.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.894+0000 7f24d360b700 1 -- 172.21.15.22:0/3641439557 >> 172.21.15.22:0/3641439557 conn(0x7f24cc0fac80 msgr2=0x7f24cc104a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:37:54.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.895+0000 7f24d360b700 1 -- 172.21.15.22:0/3641439557 shutdown_connections 2024-02-16T19:37:54.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:54.895+0000 7f24d360b700 1 -- 172.21.15.22:0/3641439557 wait complete. 2024-02-16T19:37:55.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.006+0000 7efe25d79700 1 -- 172.21.15.22:0/1532869103 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efe200ffbd0 msgr2=0x7efe20100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:37:55.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.006+0000 7efe25d79700 1 --2- 172.21.15.22:0/1532869103 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efe200ffbd0 0x7efe20100040 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7efe10009910 tx=0x7efe10009c20 comp rx=0 tx=0).stop 2024-02-16T19:37:55.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.006+0000 7efe25d79700 1 -- 172.21.15.22:0/1532869103 shutdown_connections 2024-02-16T19:37:55.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.006+0000 7efe25d79700 1 --2- 172.21.15.22:0/1532869103 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efe200ffbd0 0x7efe20100040 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:55.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.006+0000 7efe25d79700 1 --2- 172.21.15.22:0/1532869103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe20104130 0x7efe20104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:55.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.006+0000 7efe25d79700 1 -- 172.21.15.22:0/1532869103 >> 172.21.15.22:0/1532869103 conn(0x7efe20075130 msgr2=0x7efe20075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:37:55.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.006+0000 7efe25d79700 1 -- 172.21.15.22:0/1532869103 shutdown_connections 2024-02-16T19:37:55.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.006+0000 7efe25d79700 1 -- 172.21.15.22:0/1532869103 wait complete. 2024-02-16T19:37:55.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.007+0000 7efe25d79700 1 Processor -- start 2024-02-16T19:37:55.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.007+0000 7efe25d79700 1 -- start start 2024-02-16T19:37:55.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.007+0000 7efe25d79700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efe200ffbd0 0x7efe20196c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:37:55.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.007+0000 7efe25d79700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe20104130 0x7efe201971c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:37:55.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.007+0000 7efe25d79700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe20197750 con 0x7efe200ffbd0 2024-02-16T19:37:55.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.007+0000 7efe25d79700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe201978c0 con 0x7efe20104130 2024-02-16T19:37:55.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.007+0000 7efe1f7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efe200ffbd0 0x7efe20196c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:37:55.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.007+0000 7efe1f7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efe200ffbd0 0x7efe20196c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51024/0 (socket says 172.21.15.22:51024) 2024-02-16T19:37:55.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.007+0000 7efe1f7fe700 1 -- 172.21.15.22:0/404393951 learned_addr learned my addr 172.21.15.22:0/404393951 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:37:55.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.007+0000 7efe1effd700 1 --2- 172.21.15.22:0/404393951 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe20104130 0x7efe201971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:37:55.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.007+0000 7efe1f7fe700 1 -- 172.21.15.22:0/404393951 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe20104130 msgr2=0x7efe201971c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:37:55.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.007+0000 7efe1f7fe700 1 --2- 172.21.15.22:0/404393951 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe20104130 0x7efe201971c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:55.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.007+0000 7efe1f7fe700 1 -- 172.21.15.22:0/404393951 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe100095f0 con 0x7efe200ffbd0 2024-02-16T19:37:55.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.008+0000 7efe1f7fe700 1 --2- 172.21.15.22:0/404393951 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efe200ffbd0 0x7efe20196c80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7efe0800c930 tx=0x7efe0800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:37:55.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.008+0000 7efe1cff9700 1 -- 172.21.15.22:0/404393951 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe08007ad0 con 0x7efe200ffbd0 2024-02-16T19:37:55.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.008+0000 7efe1cff9700 1 -- 172.21.15.22:0/404393951 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efe08007c30 con 0x7efe200ffbd0 2024-02-16T19:37:55.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.008+0000 7efe25d79700 1 -- 172.21.15.22:0/404393951 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe2019fcf0 con 0x7efe200ffbd0 2024-02-16T19:37:55.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.008+0000 7efe1cff9700 1 -- 172.21.15.22:0/404393951 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe08018610 con 0x7efe200ffbd0 2024-02-16T19:37:55.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.008+0000 7efe25d79700 1 -- 172.21.15.22:0/404393951 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe201a0160 con 0x7efe200ffbd0 2024-02-16T19:37:55.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.009+0000 7efe25d79700 1 -- 172.21.15.22:0/404393951 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe2004f2a0 con 0x7efe200ffbd0 2024-02-16T19:37:55.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.009+0000 7efe1cff9700 1 -- 172.21.15.22:0/404393951 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7efe0800f450 con 0x7efe200ffbd0 2024-02-16T19:37:55.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.010+0000 7efe1cff9700 1 --2- 172.21.15.22:0/404393951 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7efe0c06d8a0 0x7efe0c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:37:55.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.010+0000 7efe1cff9700 1 -- 172.21.15.22:0/404393951 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6704+0+0 (secure 0 0 0) 0x7efe0808d700 con 0x7efe200ffbd0 2024-02-16T19:37:55.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.010+0000 7efe1effd700 1 --2- 172.21.15.22:0/404393951 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7efe0c06d8a0 0x7efe0c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:37:55.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.010+0000 7efe1effd700 1 --2- 172.21.15.22:0/404393951 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7efe0c06d8a0 0x7efe0c06fd50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7efe20068e10 tx=0x7efe1001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:37:55.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.014+0000 7efe1cff9700 1 -- 172.21.15.22:0/404393951 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7efe08056ec0 con 0x7efe200ffbd0 2024-02-16T19:37:55.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.236+0000 7efe25d79700 1 -- 172.21.15.22:0/404393951 --> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7efe200ff240 con 0x7efe0c06d8a0 2024-02-16T19:37:55.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.248+0000 7efe1cff9700 1 -- 172.21.15.22:0/404393951 <== mgr.34104 v2:172.21.15.22:6800/505266753 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7efe200ff240 con 0x7efe0c06d8a0 2024-02-16T19:37:55.249 INFO:teuthology.orchestra.run.smithi022.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T19:37:55.249 INFO:teuthology.orchestra.run.smithi022.stdout:alertmanager.smithi022 smithi022 *:9093,9094 running (6m) 113s ago 17m 26.8M - 0.25.0 c8568f914cd2 c89bc76be374 2024-02-16T19:37:55.249 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-exporter.smithi022 smithi022 running (17m) 113s ago 17m 8606k - 18.2.0 dc2bc1663786 3f8bef4349ba 2024-02-16T19:37:55.249 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-exporter.smithi117 smithi117 running (16m) 13s ago 16m 8476k - 18.2.0 dc2bc1663786 59b65a7e309f 2024-02-16T19:37:55.250 INFO:teuthology.orchestra.run.smithi022.stdout:crash.smithi022 smithi022 running (3m) 113s ago 17m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a e0f78bdf43e4 2024-02-16T19:37:55.250 INFO:teuthology.orchestra.run.smithi022.stdout:crash.smithi117 smithi117 running (3m) 13s ago 16m 6669k - 19.0.0-1450-gf78a58c0 f081f847070a bce7c0590813 2024-02-16T19:37:55.250 INFO:teuthology.orchestra.run.smithi022.stdout:grafana.smithi022 smithi022 *:3000 running (5m) 113s ago 16m 81.0M - 9.4.12 a0ea5e3c46d9 fd27d11e9729 2024-02-16T19:37:55.250 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi022.btppwm smithi022 running (11m) 113s ago 11m 155M - 18.2.0 dc2bc1663786 22971fb3a907 2024-02-16T19:37:55.250 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi022.ocukvs smithi022 starting - - - - 2024-02-16T19:37:55.250 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi117.ibmppn smithi117 running (11m) 13s ago 11m 18.9M - 18.2.0 dc2bc1663786 8c606cbeeb86 2024-02-16T19:37:55.250 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi117.orzors smithi117 running (11m) 13s ago 11m 18.0M - 18.2.0 dc2bc1663786 7d2bc4df3feb 2024-02-16T19:37:55.250 INFO:teuthology.orchestra.run.smithi022.stdout:mgr.smithi022.gmdcch smithi022 *:8443,9283,8765 running (8m) 113s ago 18m 504M - 19.0.0-1450-gf78a58c0 f081f847070a 7cb74cc9deea 2024-02-16T19:37:55.250 INFO:teuthology.orchestra.run.smithi022.stdout:mgr.smithi117.cohbtv smithi117 *:8443,9283,8765 running (7m) 13s ago 16m 453M - 19.0.0-1450-gf78a58c0 f081f847070a c825b1152189 2024-02-16T19:37:55.250 INFO:teuthology.orchestra.run.smithi022.stdout:mon.smithi022 smithi022 running (4m) 113s ago 18m 48.4M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 9c84755ae67d 2024-02-16T19:37:55.250 INFO:teuthology.orchestra.run.smithi022.stdout:mon.smithi117 smithi117 running (4m) 13s ago 16m 40.7M 2048M 19.0.0-1450-gf78a58c0 f081f847070a a319ac50f1a9 2024-02-16T19:37:55.250 INFO:teuthology.orchestra.run.smithi022.stdout:node-exporter.smithi022 smithi022 *:9100 running (6m) 113s ago 17m 16.6M - 1.5.0 0da6a335fe13 1f5b58292d27 2024-02-16T19:37:55.250 INFO:teuthology.orchestra.run.smithi022.stdout:node-exporter.smithi117 smithi117 *:9100 running (6m) 13s ago 16m 15.2M - 1.5.0 0da6a335fe13 bb5e8491d5a6 2024-02-16T19:37:55.251 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0 smithi022 running (3m) 113s ago 15m 107M 1217M 19.0.0-1450-gf78a58c0 f081f847070a 002412b3525f 2024-02-16T19:37:55.251 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1 smithi022 running (2m) 113s ago 14m 128M 1217M 19.0.0-1450-gf78a58c0 f081f847070a c3488ce7b2a0 2024-02-16T19:37:55.251 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2 smithi022 running (119s) 113s ago 14m 12.6M 1217M 19.0.0-1450-gf78a58c0 f081f847070a 46138c50f79d 2024-02-16T19:37:55.251 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3 smithi117 running (83s) 13s ago 13m 92.3M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 28d7d8519e64 2024-02-16T19:37:55.251 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4 smithi117 running (50s) 13s ago 13m 90.2M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 367ba2085f6f 2024-02-16T19:37:55.251 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5 smithi117 running (16s) 13s ago 13m 15.1M 2240M 19.0.0-1450-gf78a58c0 f081f847070a a1a7625b7541 2024-02-16T19:37:55.251 INFO:teuthology.orchestra.run.smithi022.stdout:prometheus.smithi022 smithi022 *:9095 running (6m) 113s ago 16m 66.9M - 2.43.0 a07b618ecd1d c3b8689d0c97 2024-02-16T19:37:55.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.250+0000 7efe25d79700 1 -- 172.21.15.22:0/404393951 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7efe0c06d8a0 msgr2=0x7efe0c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:37:55.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.250+0000 7efe25d79700 1 --2- 172.21.15.22:0/404393951 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7efe0c06d8a0 0x7efe0c06fd50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7efe20068e10 tx=0x7efe1001a040 comp rx=0 tx=0).stop 2024-02-16T19:37:55.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.250+0000 7efe25d79700 1 -- 172.21.15.22:0/404393951 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efe200ffbd0 msgr2=0x7efe20196c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:37:55.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.250+0000 7efe25d79700 1 --2- 172.21.15.22:0/404393951 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efe200ffbd0 0x7efe20196c80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7efe0800c930 tx=0x7efe0800ccf0 comp rx=0 tx=0).stop 2024-02-16T19:37:55.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.250+0000 7efe25d79700 1 -- 172.21.15.22:0/404393951 shutdown_connections 2024-02-16T19:37:55.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.250+0000 7efe25d79700 1 --2- 172.21.15.22:0/404393951 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7efe0c06d8a0 0x7efe0c06fd50 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:55.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.250+0000 7efe25d79700 1 --2- 172.21.15.22:0/404393951 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efe200ffbd0 0x7efe20196c80 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:55.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.250+0000 7efe25d79700 1 --2- 172.21.15.22:0/404393951 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe20104130 0x7efe201971c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:55.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.250+0000 7efe25d79700 1 -- 172.21.15.22:0/404393951 >> 172.21.15.22:0/404393951 conn(0x7efe20075130 msgr2=0x7efe200feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:37:55.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.250+0000 7efe25d79700 1 -- 172.21.15.22:0/404393951 shutdown_connections 2024-02-16T19:37:55.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.250+0000 7efe25d79700 1 -- 172.21.15.22:0/404393951 wait complete. 2024-02-16T19:37:55.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.358+0000 7f3781d44700 1 -- 172.21.15.22:0/882607734 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f377c1004e0 msgr2=0x7f377c1008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:37:55.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.358+0000 7f3781d44700 1 --2- 172.21.15.22:0/882607734 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f377c1004e0 0x7f377c1008b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f376c009910 tx=0x7f376c009c20 comp rx=0 tx=0).stop 2024-02-16T19:37:55.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.358+0000 7f3781d44700 1 -- 172.21.15.22:0/882607734 shutdown_connections 2024-02-16T19:37:55.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.358+0000 7f3781d44700 1 --2- 172.21.15.22:0/882607734 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f377c100e80 0x7f377c104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:55.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.358+0000 7f3781d44700 1 --2- 172.21.15.22:0/882607734 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f377c1004e0 0x7f377c1008b0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:55.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.358+0000 7f3781d44700 1 -- 172.21.15.22:0/882607734 >> 172.21.15.22:0/882607734 conn(0x7f377c0fbda0 msgr2=0x7f377c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:37:55.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.358+0000 7f3781d44700 1 -- 172.21.15.22:0/882607734 shutdown_connections 2024-02-16T19:37:55.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.358+0000 7f3781d44700 1 -- 172.21.15.22:0/882607734 wait complete. 2024-02-16T19:37:55.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.359+0000 7f3781d44700 1 Processor -- start 2024-02-16T19:37:55.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.359+0000 7f3781d44700 1 -- start start 2024-02-16T19:37:55.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.360+0000 7f3781d44700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f377c1004e0 0x7f377c195ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:37:55.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.360+0000 7f3781d44700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f377c100e80 0x7f377c196430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:37:55.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.360+0000 7f3781d44700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f377c196b10 con 0x7f377c100e80 2024-02-16T19:37:55.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.360+0000 7f3781d44700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f377c19a8a0 con 0x7f377c1004e0 2024-02-16T19:37:55.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.360+0000 7f377affd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f377c100e80 0x7f377c196430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:37:55.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.360+0000 7f377affd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f377c100e80 0x7f377c196430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51042/0 (socket says 172.21.15.22:51042) 2024-02-16T19:37:55.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.360+0000 7f377affd700 1 -- 172.21.15.22:0/999208818 learned_addr learned my addr 172.21.15.22:0/999208818 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:37:55.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.360+0000 7f377b7fe700 1 --2- 172.21.15.22:0/999208818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f377c1004e0 0x7f377c195ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:37:55.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.360+0000 7f377affd700 1 -- 172.21.15.22:0/999208818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f377c1004e0 msgr2=0x7f377c195ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:37:55.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.360+0000 7f377affd700 1 --2- 172.21.15.22:0/999208818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f377c1004e0 0x7f377c195ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:55.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.360+0000 7f377affd700 1 -- 172.21.15.22:0/999208818 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f376c0095f0 con 0x7f377c100e80 2024-02-16T19:37:55.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.360+0000 7f377b7fe700 1 --2- 172.21.15.22:0/999208818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f377c1004e0 0x7f377c195ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-16T19:37:55.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.360+0000 7f377affd700 1 --2- 172.21.15.22:0/999208818 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f377c100e80 0x7f377c196430 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f376400d900 tx=0x7f376400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:37:55.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.360+0000 7f3778ff9700 1 -- 172.21.15.22:0/999208818 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3764009900 con 0x7f377c100e80 2024-02-16T19:37:55.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.360+0000 7f3778ff9700 1 -- 172.21.15.22:0/999208818 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3764010460 con 0x7f377c100e80 2024-02-16T19:37:55.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.360+0000 7f3778ff9700 1 -- 172.21.15.22:0/999208818 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f376400f5f0 con 0x7f377c100e80 2024-02-16T19:37:55.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.360+0000 7f3781d44700 1 -- 172.21.15.22:0/999208818 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f377c19ab80 con 0x7f377c100e80 2024-02-16T19:37:55.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.361+0000 7f3781d44700 1 -- 172.21.15.22:0/999208818 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f377c19b0d0 con 0x7f377c100e80 2024-02-16T19:37:55.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.362+0000 7f3781d44700 1 -- 172.21.15.22:0/999208818 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f377c1087f0 con 0x7f377c100e80 2024-02-16T19:37:55.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.363+0000 7f3778ff9700 1 -- 172.21.15.22:0/999208818 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f37640105d0 con 0x7f377c100e80 2024-02-16T19:37:55.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.363+0000 7f3778ff9700 1 --2- 172.21.15.22:0/999208818 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f376806d8a0 0x7f376806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:37:55.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.363+0000 7f3778ff9700 1 -- 172.21.15.22:0/999208818 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f376408d370 con 0x7f377c100e80 2024-02-16T19:37:55.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.363+0000 7f377b7fe700 1 --2- 172.21.15.22:0/999208818 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f376806d8a0 0x7f376806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:37:55.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.364+0000 7f377b7fe700 1 --2- 172.21.15.22:0/999208818 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f376806d8a0 0x7f376806fd50 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f376c00b5c0 tx=0x7f376c0053a0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:37:55.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.368+0000 7f3778ff9700 1 -- 172.21.15.22:0/999208818 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3764056b30 con 0x7f377c100e80 2024-02-16T19:37:55.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.674+0000 7f3781d44700 1 -- 172.21.15.22:0/999208818 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f377c04f2a0 con 0x7f377c100e80 2024-02-16T19:37:55.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.675+0000 7f3778ff9700 1 -- 172.21.15.22:0/999208818 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7f376405a1f0 con 0x7f377c100e80 2024-02-16T19:37:55.677 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-02-16T19:37:55.677 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": { 2024-02-16T19:37:55.677 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T19:37:55.677 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:37:55.677 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": { 2024-02-16T19:37:55.677 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T19:37:55.677 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:37:55.677 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": { 2024-02-16T19:37:55.677 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-16T19:37:55.677 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:37:55.677 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": { 2024-02-16T19:37:55.678 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 3, 2024-02-16T19:37:55.678 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 1 2024-02-16T19:37:55.678 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:37:55.678 INFO:teuthology.orchestra.run.smithi022.stdout: "overall": { 2024-02-16T19:37:55.678 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 3, 2024-02-16T19:37:55.678 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 11 2024-02-16T19:37:55.678 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-02-16T19:37:55.678 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-02-16T19:37:55.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.677+0000 7f3781d44700 1 -- 172.21.15.22:0/999208818 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f376806d8a0 msgr2=0x7f376806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:37:55.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.677+0000 7f3781d44700 1 --2- 172.21.15.22:0/999208818 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f376806d8a0 0x7f376806fd50 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f376c00b5c0 tx=0x7f376c0053a0 comp rx=0 tx=0).stop 2024-02-16T19:37:55.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.677+0000 7f3781d44700 1 -- 172.21.15.22:0/999208818 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f377c100e80 msgr2=0x7f377c196430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:37:55.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.677+0000 7f3781d44700 1 --2- 172.21.15.22:0/999208818 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f377c100e80 0x7f377c196430 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f376400d900 tx=0x7f376400dcc0 comp rx=0 tx=0).stop 2024-02-16T19:37:55.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.677+0000 7f3781d44700 1 -- 172.21.15.22:0/999208818 shutdown_connections 2024-02-16T19:37:55.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.677+0000 7f3781d44700 1 --2- 172.21.15.22:0/999208818 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f376806d8a0 0x7f376806fd50 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:55.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.677+0000 7f3781d44700 1 --2- 172.21.15.22:0/999208818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f377c1004e0 0x7f377c195ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:55.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.677+0000 7f3781d44700 1 --2- 172.21.15.22:0/999208818 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f377c100e80 0x7f377c196430 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:55.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.677+0000 7f3781d44700 1 -- 172.21.15.22:0/999208818 >> 172.21.15.22:0/999208818 conn(0x7f377c0fbda0 msgr2=0x7f377c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:37:55.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.677+0000 7f3781d44700 1 -- 172.21.15.22:0/999208818 shutdown_connections 2024-02-16T19:37:55.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.677+0000 7f3781d44700 1 -- 172.21.15.22:0/999208818 wait complete. 2024-02-16T19:37:55.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.787+0000 7f851a016700 1 -- 172.21.15.22:0/3674960493 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8514108470 msgr2=0x7f8514108840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:37:55.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.787+0000 7f851a016700 1 --2- 172.21.15.22:0/3674960493 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8514108470 0x7f8514108840 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8504009910 tx=0x7f8504009c20 comp rx=0 tx=0).stop 2024-02-16T19:37:55.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.787+0000 7f851a016700 1 -- 172.21.15.22:0/3674960493 shutdown_connections 2024-02-16T19:37:55.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.787+0000 7f851a016700 1 --2- 172.21.15.22:0/3674960493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8514102470 0x7f85141028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:55.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.787+0000 7f851a016700 1 --2- 172.21.15.22:0/3674960493 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8514108470 0x7f8514108840 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:55.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.787+0000 7f851a016700 1 -- 172.21.15.22:0/3674960493 >> 172.21.15.22:0/3674960493 conn(0x7f85140fdf90 msgr2=0x7f85141003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:37:55.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.788+0000 7f851a016700 1 -- 172.21.15.22:0/3674960493 shutdown_connections 2024-02-16T19:37:55.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.788+0000 7f851a016700 1 -- 172.21.15.22:0/3674960493 wait complete. 2024-02-16T19:37:55.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.788+0000 7f851a016700 1 Processor -- start 2024-02-16T19:37:55.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.788+0000 7f851a016700 1 -- start start 2024-02-16T19:37:55.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.789+0000 7f851a016700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8514102470 0x7f85141981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:37:55.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.789+0000 7f851a016700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8514108470 0x7f8514198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:37:55.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.789+0000 7f851a016700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8514198e10 con 0x7f8514102470 2024-02-16T19:37:55.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.789+0000 7f851a016700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f851419cba0 con 0x7f8514108470 2024-02-16T19:37:55.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.789+0000 7f85137fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8514102470 0x7f85141981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:37:55.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.789+0000 7f85137fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8514102470 0x7f85141981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51054/0 (socket says 172.21.15.22:51054) 2024-02-16T19:37:55.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.789+0000 7f85137fe700 1 -- 172.21.15.22:0/2743620819 learned_addr learned my addr 172.21.15.22:0/2743620819 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:37:55.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.789+0000 7f8512ffd700 1 --2- 172.21.15.22:0/2743620819 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8514108470 0x7f8514198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:37:55.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.789+0000 7f85137fe700 1 -- 172.21.15.22:0/2743620819 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8514108470 msgr2=0x7f8514198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:37:55.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.789+0000 7f85137fe700 1 --2- 172.21.15.22:0/2743620819 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8514108470 0x7f8514198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:55.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.789+0000 7f85137fe700 1 -- 172.21.15.22:0/2743620819 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85040095f0 con 0x7f8514102470 2024-02-16T19:37:55.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.789+0000 7f85137fe700 1 --2- 172.21.15.22:0/2743620819 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8514102470 0x7f85141981f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8504006010 tx=0x7f850400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:37:55.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.789+0000 7f8510ff9700 1 -- 172.21.15.22:0/2743620819 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f850401d070 con 0x7f8514102470 2024-02-16T19:37:55.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.789+0000 7f8510ff9700 1 -- 172.21.15.22:0/2743620819 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8504004fb0 con 0x7f8514102470 2024-02-16T19:37:55.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.789+0000 7f8510ff9700 1 -- 172.21.15.22:0/2743620819 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8504022ea0 con 0x7f8514102470 2024-02-16T19:37:55.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.789+0000 7f851a016700 1 -- 172.21.15.22:0/2743620819 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f851419ce20 con 0x7f8514102470 2024-02-16T19:37:55.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.789+0000 7f851a016700 1 -- 172.21.15.22:0/2743620819 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f851419d310 con 0x7f8514102470 2024-02-16T19:37:55.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.790+0000 7f851a016700 1 -- 172.21.15.22:0/2743620819 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f851410a9e0 con 0x7f8514102470 2024-02-16T19:37:55.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.791+0000 7f8510ff9700 1 -- 172.21.15.22:0/2743620819 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f8504022470 con 0x7f8514102470 2024-02-16T19:37:55.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.791+0000 7f8510ff9700 1 --2- 172.21.15.22:0/2743620819 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f84fc06d8a0 0x7f84fc06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:37:55.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.791+0000 7f8510ff9700 1 -- 172.21.15.22:0/2743620819 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f850408ec20 con 0x7f8514102470 2024-02-16T19:37:55.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.791+0000 7f8512ffd700 1 --2- 172.21.15.22:0/2743620819 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f84fc06d8a0 0x7f84fc06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:37:55.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.792+0000 7f8512ffd700 1 --2- 172.21.15.22:0/2743620819 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f84fc06d8a0 0x7f84fc06fd50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8508005fd0 tx=0x7f850800a560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:37:55.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:55.796+0000 7f8510ff9700 1 -- 172.21.15.22:0/2743620819 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8504058530 con 0x7f8514102470 2024-02-16T19:37:55.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:55 smithi022 ceph-mon[88798]: pgmap v182: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:37:55.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:55 smithi022 ceph-mon[88798]: from='client.34376 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:37:55.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:55 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:37:55.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:55 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:37:55.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:55 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:37:55.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:55 smithi022 ceph-mon[88798]: from='client.? 172.21.15.22:0/999208818' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:37:56.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.050+0000 7f851a016700 1 -- 172.21.15.22:0/2743620819 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f8514199610 con 0x7f8514102470 2024-02-16T19:37:56.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.051+0000 7f8510ff9700 1 -- 172.21.15.22:0/2743620819 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1847 (secure 0 0 0) 0x7f8504027070 con 0x7f8514102470 2024-02-16T19:37:56.052 INFO:teuthology.orchestra.run.smithi022.stdout:e17 2024-02-16T19:37:56.053 INFO:teuthology.orchestra.run.smithi022.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T19:37:56.053 INFO:teuthology.orchestra.run.smithi022.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:37:56.053 INFO:teuthology.orchestra.run.smithi022.stdout:legacy client fscid: 1 2024-02-16T19:37:56.053 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:37:56.053 INFO:teuthology.orchestra.run.smithi022.stdout:Filesystem 'cephfs' (1) 2024-02-16T19:37:56.053 INFO:teuthology.orchestra.run.smithi022.stdout:fs_name cephfs 2024-02-16T19:37:56.053 INFO:teuthology.orchestra.run.smithi022.stdout:epoch 15 2024-02-16T19:37:56.053 INFO:teuthology.orchestra.run.smithi022.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T19:37:56.053 INFO:teuthology.orchestra.run.smithi022.stdout:created 2024-02-16T19:26:17.873644+0000 2024-02-16T19:37:56.053 INFO:teuthology.orchestra.run.smithi022.stdout:modified 2024-02-16T19:26:35.943353+0000 2024-02-16T19:37:56.053 INFO:teuthology.orchestra.run.smithi022.stdout:tableserver 0 2024-02-16T19:37:56.053 INFO:teuthology.orchestra.run.smithi022.stdout:root 0 2024-02-16T19:37:56.053 INFO:teuthology.orchestra.run.smithi022.stdout:session_timeout 60 2024-02-16T19:37:56.053 INFO:teuthology.orchestra.run.smithi022.stdout:session_autoclose 300 2024-02-16T19:37:56.053 INFO:teuthology.orchestra.run.smithi022.stdout:max_file_size 1099511627776 2024-02-16T19:37:56.054 INFO:teuthology.orchestra.run.smithi022.stdout:max_xattr_size 65536 2024-02-16T19:37:56.054 INFO:teuthology.orchestra.run.smithi022.stdout:required_client_features {} 2024-02-16T19:37:56.054 INFO:teuthology.orchestra.run.smithi022.stdout:last_failure 0 2024-02-16T19:37:56.054 INFO:teuthology.orchestra.run.smithi022.stdout:last_failure_osd_epoch 47 2024-02-16T19:37:56.054 INFO:teuthology.orchestra.run.smithi022.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:37:56.054 INFO:teuthology.orchestra.run.smithi022.stdout:max_mds 1 2024-02-16T19:37:56.054 INFO:teuthology.orchestra.run.smithi022.stdout:in 0 2024-02-16T19:37:56.054 INFO:teuthology.orchestra.run.smithi022.stdout:up {0=14490} 2024-02-16T19:37:56.054 INFO:teuthology.orchestra.run.smithi022.stdout:failed 2024-02-16T19:37:56.054 INFO:teuthology.orchestra.run.smithi022.stdout:damaged 2024-02-16T19:37:56.054 INFO:teuthology.orchestra.run.smithi022.stdout:stopped 2024-02-16T19:37:56.054 INFO:teuthology.orchestra.run.smithi022.stdout:data_pools [3] 2024-02-16T19:37:56.054 INFO:teuthology.orchestra.run.smithi022.stdout:metadata_pool 2 2024-02-16T19:37:56.054 INFO:teuthology.orchestra.run.smithi022.stdout:inline_data enabled 2024-02-16T19:37:56.055 INFO:teuthology.orchestra.run.smithi022.stdout:balancer 2024-02-16T19:37:56.055 INFO:teuthology.orchestra.run.smithi022.stdout:bal_rank_mask -1 2024-02-16T19:37:56.055 INFO:teuthology.orchestra.run.smithi022.stdout:standby_count_wanted 1 2024-02-16T19:37:56.055 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.cephfs.smithi022.btppwm{0:14490} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.22:6828/3597685751,v1:172.21.15.22:6829/3597685751] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:37:56.055 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:37:56.055 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:37:56.055 INFO:teuthology.orchestra.run.smithi022.stdout:Standby daemons: 2024-02-16T19:37:56.055 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:37:56.055 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.cephfs.smithi117.ibmppn{-1:24259} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.117:6824/565408102,v1:172.21.15.117:6825/565408102] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:37:56.055 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.cephfs.smithi117.orzors{-1:24265} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.117:6826/3122342028,v1:172.21.15.117:6827/3122342028] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:37:56.055 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.cephfs.smithi022.ocukvs{-1:34384} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.22:6826/2535680483,v1:172.21.15.22:6827/2535680483] compat {c=[1],r=[1],i=[fff]}] 2024-02-16T19:37:56.056 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:55 smithi117 ceph-mon[67870]: pgmap v182: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:37:56.056 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:55 smithi117 ceph-mon[67870]: from='client.34376 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:37:56.056 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:55 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:37:56.056 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:55 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:37:56.056 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:55 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:37:56.056 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:55 smithi117 ceph-mon[67870]: from='client.? 172.21.15.22:0/999208818' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:37:56.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.053+0000 7f851a016700 1 -- 172.21.15.22:0/2743620819 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f84fc06d8a0 msgr2=0x7f84fc06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:37:56.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.053+0000 7f851a016700 1 --2- 172.21.15.22:0/2743620819 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f84fc06d8a0 0x7f84fc06fd50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8508005fd0 tx=0x7f850800a560 comp rx=0 tx=0).stop 2024-02-16T19:37:56.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.053+0000 7f851a016700 1 -- 172.21.15.22:0/2743620819 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8514102470 msgr2=0x7f85141981f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:37:56.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.053+0000 7f851a016700 1 --2- 172.21.15.22:0/2743620819 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8514102470 0x7f85141981f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8504006010 tx=0x7f850400be20 comp rx=0 tx=0).stop 2024-02-16T19:37:56.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.053+0000 7f851a016700 1 -- 172.21.15.22:0/2743620819 shutdown_connections 2024-02-16T19:37:56.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.053+0000 7f851a016700 1 --2- 172.21.15.22:0/2743620819 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f84fc06d8a0 0x7f84fc06fd50 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:56.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.053+0000 7f851a016700 1 --2- 172.21.15.22:0/2743620819 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8514102470 0x7f85141981f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:56.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.053+0000 7f851a016700 1 --2- 172.21.15.22:0/2743620819 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8514108470 0x7f8514198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:56.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.053+0000 7f851a016700 1 -- 172.21.15.22:0/2743620819 >> 172.21.15.22:0/2743620819 conn(0x7f85140fdf90 msgr2=0x7f85141002c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:37:56.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.053+0000 7f851a016700 1 -- 172.21.15.22:0/2743620819 shutdown_connections 2024-02-16T19:37:56.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.053+0000 7f851a016700 1 -- 172.21.15.22:0/2743620819 wait complete. 2024-02-16T19:37:56.058 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 17 2024-02-16T19:37:56.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.163+0000 7f9329bf3700 1 -- 172.21.15.22:0/1399426997 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93240ffd60 msgr2=0x7f9324103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:37:56.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.163+0000 7f9329bf3700 1 --2- 172.21.15.22:0/1399426997 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93240ffd60 0x7f9324103d30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f930c009910 tx=0x7f930c009c20 comp rx=0 tx=0).stop 2024-02-16T19:37:56.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.163+0000 7f9329bf3700 1 -- 172.21.15.22:0/1399426997 shutdown_connections 2024-02-16T19:37:56.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.163+0000 7f9329bf3700 1 --2- 172.21.15.22:0/1399426997 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93240ffd60 0x7f9324103d30 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:56.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.163+0000 7f9329bf3700 1 --2- 172.21.15.22:0/1399426997 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93240ff3c0 0x7f93240ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:56.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.163+0000 7f9329bf3700 1 -- 172.21.15.22:0/1399426997 >> 172.21.15.22:0/1399426997 conn(0x7f93240fac80 msgr2=0x7f93240fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:37:56.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.163+0000 7f9329bf3700 1 -- 172.21.15.22:0/1399426997 shutdown_connections 2024-02-16T19:37:56.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.163+0000 7f9329bf3700 1 -- 172.21.15.22:0/1399426997 wait complete. 2024-02-16T19:37:56.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.164+0000 7f9329bf3700 1 Processor -- start 2024-02-16T19:37:56.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.164+0000 7f9329bf3700 1 -- start start 2024-02-16T19:37:56.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.164+0000 7f9329bf3700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93240ff3c0 0x7f9324195e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:37:56.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.164+0000 7f9329bf3700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93240ffd60 0x7f93241963b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:37:56.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.164+0000 7f9329bf3700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9324196a90 con 0x7f93240ff3c0 2024-02-16T19:37:56.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.164+0000 7f9329bf3700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f932419a820 con 0x7f93240ffd60 2024-02-16T19:37:56.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.164+0000 7f93237fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93240ff3c0 0x7f9324195e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:37:56.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.164+0000 7f93237fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93240ff3c0 0x7f9324195e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51078/0 (socket says 172.21.15.22:51078) 2024-02-16T19:37:56.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.164+0000 7f93237fe700 1 -- 172.21.15.22:0/812045798 learned_addr learned my addr 172.21.15.22:0/812045798 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:37:56.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.164+0000 7f9322ffd700 1 --2- 172.21.15.22:0/812045798 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93240ffd60 0x7f93241963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:37:56.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.165+0000 7f93237fe700 1 -- 172.21.15.22:0/812045798 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93240ffd60 msgr2=0x7f93241963b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:37:56.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.165+0000 7f93237fe700 1 --2- 172.21.15.22:0/812045798 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93240ffd60 0x7f93241963b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:56.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.165+0000 7f93237fe700 1 -- 172.21.15.22:0/812045798 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f930c0095f0 con 0x7f93240ff3c0 2024-02-16T19:37:56.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.165+0000 7f9322ffd700 1 --2- 172.21.15.22:0/812045798 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93240ffd60 0x7f93241963b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T19:37:56.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.165+0000 7f93237fe700 1 --2- 172.21.15.22:0/812045798 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93240ff3c0 0x7f9324195e70 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f931400d900 tx=0x7f931400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:37:56.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.165+0000 7f9320ff9700 1 -- 172.21.15.22:0/812045798 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f93140041f0 con 0x7f93240ff3c0 2024-02-16T19:37:56.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.165+0000 7f9329bf3700 1 -- 172.21.15.22:0/812045798 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f932419ab00 con 0x7f93240ff3c0 2024-02-16T19:37:56.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.165+0000 7f9329bf3700 1 -- 172.21.15.22:0/812045798 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f932419b050 con 0x7f93240ff3c0 2024-02-16T19:37:56.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.165+0000 7f9320ff9700 1 -- 172.21.15.22:0/812045798 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9314004cf0 con 0x7f93240ff3c0 2024-02-16T19:37:56.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.165+0000 7f9320ff9700 1 -- 172.21.15.22:0/812045798 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f931400b750 con 0x7f93240ff3c0 2024-02-16T19:37:56.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.167+0000 7f9329bf3700 1 -- 172.21.15.22:0/812045798 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9304005320 con 0x7f93240ff3c0 2024-02-16T19:37:56.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.167+0000 7f9320ff9700 1 -- 172.21.15.22:0/812045798 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f9314004750 con 0x7f93240ff3c0 2024-02-16T19:37:56.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.167+0000 7f9320ff9700 1 --2- 172.21.15.22:0/812045798 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f931006d8a0 0x7f931006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:37:56.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.167+0000 7f9320ff9700 1 -- 172.21.15.22:0/812045798 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f931408d380 con 0x7f93240ff3c0 2024-02-16T19:37:56.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.168+0000 7f9322ffd700 1 --2- 172.21.15.22:0/812045798 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f931006d8a0 0x7f931006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:37:56.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.168+0000 7f9322ffd700 1 --2- 172.21.15.22:0/812045798 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f931006d8a0 0x7f931006fd50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9324197490 tx=0x7f930c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:37:56.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.172+0000 7f9320ff9700 1 -- 172.21.15.22:0/812045798 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9314056b40 con 0x7f93240ff3c0 2024-02-16T19:37:56.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.400+0000 7f9329bf3700 1 -- 172.21.15.22:0/812045798 --> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9304000bf0 con 0x7f931006d8a0 2024-02-16T19:37:56.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.401+0000 7f9320ff9700 1 -- 172.21.15.22:0/812045798 <== mgr.34104 v2:172.21.15.22:6800/505266753 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f9304000bf0 con 0x7f931006d8a0 2024-02-16T19:37:56.403 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-02-16T19:37:56.403 INFO:teuthology.orchestra.run.smithi022.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T19:37:56.403 INFO:teuthology.orchestra.run.smithi022.stdout: "in_progress": true, 2024-02-16T19:37:56.403 INFO:teuthology.orchestra.run.smithi022.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T19:37:56.403 INFO:teuthology.orchestra.run.smithi022.stdout: "services_complete": [ 2024-02-16T19:37:56.403 INFO:teuthology.orchestra.run.smithi022.stdout: "crash", 2024-02-16T19:37:56.403 INFO:teuthology.orchestra.run.smithi022.stdout: "osd", 2024-02-16T19:37:56.403 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr", 2024-02-16T19:37:56.403 INFO:teuthology.orchestra.run.smithi022.stdout: "mon" 2024-02-16T19:37:56.403 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-02-16T19:37:56.403 INFO:teuthology.orchestra.run.smithi022.stdout: "progress": "12/23 daemons upgraded", 2024-02-16T19:37:56.403 INFO:teuthology.orchestra.run.smithi022.stdout: "message": "Currently upgrading mds daemons", 2024-02-16T19:37:56.403 INFO:teuthology.orchestra.run.smithi022.stdout: "is_paused": false 2024-02-16T19:37:56.404 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-02-16T19:37:56.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.403+0000 7f9329bf3700 1 -- 172.21.15.22:0/812045798 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f931006d8a0 msgr2=0x7f931006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:37:56.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.403+0000 7f9329bf3700 1 --2- 172.21.15.22:0/812045798 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f931006d8a0 0x7f931006fd50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9324197490 tx=0x7f930c01a040 comp rx=0 tx=0).stop 2024-02-16T19:37:56.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.403+0000 7f9329bf3700 1 -- 172.21.15.22:0/812045798 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93240ff3c0 msgr2=0x7f9324195e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:37:56.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.403+0000 7f9329bf3700 1 --2- 172.21.15.22:0/812045798 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93240ff3c0 0x7f9324195e70 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f931400d900 tx=0x7f931400dc10 comp rx=0 tx=0).stop 2024-02-16T19:37:56.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.404+0000 7f9329bf3700 1 -- 172.21.15.22:0/812045798 shutdown_connections 2024-02-16T19:37:56.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.404+0000 7f9329bf3700 1 --2- 172.21.15.22:0/812045798 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f931006d8a0 0x7f931006fd50 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:56.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.404+0000 7f9329bf3700 1 --2- 172.21.15.22:0/812045798 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93240ff3c0 0x7f9324195e70 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:56.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.404+0000 7f9329bf3700 1 --2- 172.21.15.22:0/812045798 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93240ffd60 0x7f93241963b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:56.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.404+0000 7f9329bf3700 1 -- 172.21.15.22:0/812045798 >> 172.21.15.22:0/812045798 conn(0x7f93240fac80 msgr2=0x7f93240fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:37:56.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.404+0000 7f9329bf3700 1 -- 172.21.15.22:0/812045798 shutdown_connections 2024-02-16T19:37:56.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.404+0000 7f9329bf3700 1 -- 172.21.15.22:0/812045798 wait complete. 2024-02-16T19:37:56.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.523+0000 7f4d3d5e1700 1 -- 172.21.15.22:0/3818633987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d381004c0 msgr2=0x7f4d38100890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:37:56.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.523+0000 7f4d3d5e1700 1 --2- 172.21.15.22:0/3818633987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d381004c0 0x7f4d38100890 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4d28009910 tx=0x7f4d28009c20 comp rx=0 tx=0).stop 2024-02-16T19:37:56.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.523+0000 7f4d3d5e1700 1 -- 172.21.15.22:0/3818633987 shutdown_connections 2024-02-16T19:37:56.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.523+0000 7f4d3d5e1700 1 --2- 172.21.15.22:0/3818633987 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d38100e60 0x7f4d38104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:56.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.523+0000 7f4d3d5e1700 1 --2- 172.21.15.22:0/3818633987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d381004c0 0x7f4d38100890 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:56.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.523+0000 7f4d3d5e1700 1 -- 172.21.15.22:0/3818633987 >> 172.21.15.22:0/3818633987 conn(0x7f4d380fbd80 msgr2=0x7f4d380fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:37:56.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.523+0000 7f4d3d5e1700 1 -- 172.21.15.22:0/3818633987 shutdown_connections 2024-02-16T19:37:56.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.524+0000 7f4d3d5e1700 1 -- 172.21.15.22:0/3818633987 wait complete. 2024-02-16T19:37:56.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.524+0000 7f4d3d5e1700 1 Processor -- start 2024-02-16T19:37:56.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.524+0000 7f4d3d5e1700 1 -- start start 2024-02-16T19:37:56.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.525+0000 7f4d3d5e1700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d381004c0 0x7f4d38073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:37:56.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.525+0000 7f4d3d5e1700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d38100e60 0x7f4d38073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:37:56.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.525+0000 7f4d3d5e1700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d38073ad0 con 0x7f4d38100e60 2024-02-16T19:37:56.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.525+0000 7f4d3d5e1700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d38073c10 con 0x7f4d381004c0 2024-02-16T19:37:56.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.525+0000 7f4d367fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d38100e60 0x7f4d38073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:37:56.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.525+0000 7f4d367fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d38100e60 0x7f4d38073590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51098/0 (socket says 172.21.15.22:51098) 2024-02-16T19:37:56.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.525+0000 7f4d367fc700 1 -- 172.21.15.22:0/1724004321 learned_addr learned my addr 172.21.15.22:0/1724004321 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:37:56.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.525+0000 7f4d36ffd700 1 --2- 172.21.15.22:0/1724004321 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d381004c0 0x7f4d38073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:37:56.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.525+0000 7f4d36ffd700 1 -- 172.21.15.22:0/1724004321 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d38100e60 msgr2=0x7f4d38073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:37:56.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.525+0000 7f4d36ffd700 1 --2- 172.21.15.22:0/1724004321 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d38100e60 0x7f4d38073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:56.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.525+0000 7f4d36ffd700 1 -- 172.21.15.22:0/1724004321 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d280095f0 con 0x7f4d381004c0 2024-02-16T19:37:56.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.525+0000 7f4d367fc700 1 --2- 172.21.15.22:0/1724004321 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d38100e60 0x7f4d38073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T19:37:56.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.525+0000 7f4d36ffd700 1 --2- 172.21.15.22:0/1724004321 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d381004c0 0x7f4d38073050 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4d28009fd0 tx=0x7f4d2800f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:37:56.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.526+0000 7f4d1ffff700 1 -- 172.21.15.22:0/1724004321 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d2801d070 con 0x7f4d381004c0 2024-02-16T19:37:56.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.526+0000 7f4d1ffff700 1 -- 172.21.15.22:0/1724004321 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4d2800b610 con 0x7f4d381004c0 2024-02-16T19:37:56.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.526+0000 7f4d3d5e1700 1 -- 172.21.15.22:0/1724004321 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d38073e90 con 0x7f4d381004c0 2024-02-16T19:37:56.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.526+0000 7f4d1ffff700 1 -- 172.21.15.22:0/1724004321 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d28022470 con 0x7f4d381004c0 2024-02-16T19:37:56.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.526+0000 7f4d3d5e1700 1 -- 172.21.15.22:0/1724004321 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d381a23e0 con 0x7f4d381004c0 2024-02-16T19:37:56.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.527+0000 7f4d3d5e1700 1 -- 172.21.15.22:0/1724004321 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d381087d0 con 0x7f4d381004c0 2024-02-16T19:37:56.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.527+0000 7f4d1ffff700 1 -- 172.21.15.22:0/1724004321 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f4d2800fac0 con 0x7f4d381004c0 2024-02-16T19:37:56.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.528+0000 7f4d1ffff700 1 --2- 172.21.15.22:0/1724004321 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f4d2006d8a0 0x7f4d2006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:37:56.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.528+0000 7f4d1ffff700 1 -- 172.21.15.22:0/1724004321 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f4d2808eac0 con 0x7f4d381004c0 2024-02-16T19:37:56.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.528+0000 7f4d367fc700 1 --2- 172.21.15.22:0/1724004321 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f4d2006d8a0 0x7f4d2006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:37:56.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.529+0000 7f4d367fc700 1 --2- 172.21.15.22:0/1724004321 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f4d2006d8a0 0x7f4d2006fd50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4d38074840 tx=0x7f4d2c009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:37:56.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.533+0000 7f4d1ffff700 1 -- 172.21.15.22:0/1724004321 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4d28058180 con 0x7f4d381004c0 2024-02-16T19:37:56.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.842+0000 7f4d3d5e1700 1 -- 172.21.15.22:0/1724004321 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f4d38074680 con 0x7f4d381004c0 2024-02-16T19:37:56.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.843+0000 7f4d1ffff700 1 -- 172.21.15.22:0/1724004321 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f4d2805b840 con 0x7f4d381004c0 2024-02-16T19:37:56.845 INFO:teuthology.orchestra.run.smithi022.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-02-16T19:37:56.845 INFO:teuthology.orchestra.run.smithi022.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-02-16T19:37:56.845 INFO:teuthology.orchestra.run.smithi022.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-02-16T19:37:56.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.845+0000 7f4d3d5e1700 1 -- 172.21.15.22:0/1724004321 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f4d2006d8a0 msgr2=0x7f4d2006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:37:56.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.845+0000 7f4d3d5e1700 1 --2- 172.21.15.22:0/1724004321 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f4d2006d8a0 0x7f4d2006fd50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4d38074840 tx=0x7f4d2c009260 comp rx=0 tx=0).stop 2024-02-16T19:37:56.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.845+0000 7f4d3d5e1700 1 -- 172.21.15.22:0/1724004321 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d381004c0 msgr2=0x7f4d38073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:37:56.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.845+0000 7f4d3d5e1700 1 --2- 172.21.15.22:0/1724004321 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d381004c0 0x7f4d38073050 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4d28009fd0 tx=0x7f4d2800f690 comp rx=0 tx=0).stop 2024-02-16T19:37:56.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.846+0000 7f4d3d5e1700 1 -- 172.21.15.22:0/1724004321 shutdown_connections 2024-02-16T19:37:56.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.846+0000 7f4d3d5e1700 1 --2- 172.21.15.22:0/1724004321 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f4d2006d8a0 0x7f4d2006fd50 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:56.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.846+0000 7f4d3d5e1700 1 --2- 172.21.15.22:0/1724004321 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d381004c0 0x7f4d38073050 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:56.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.846+0000 7f4d3d5e1700 1 --2- 172.21.15.22:0/1724004321 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d38100e60 0x7f4d38073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:37:56.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.846+0000 7f4d3d5e1700 1 -- 172.21.15.22:0/1724004321 >> 172.21.15.22:0/1724004321 conn(0x7f4d380fbd80 msgr2=0x7f4d380fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:37:56.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.846+0000 7f4d3d5e1700 1 -- 172.21.15.22:0/1724004321 shutdown_connections 2024-02-16T19:37:56.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:37:56.846+0000 7f4d3d5e1700 1 -- 172.21.15.22:0/1724004321 wait complete. 2024-02-16T19:37:57.071 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:56 smithi022 ceph-mon[88798]: from='client.34380 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:37:57.072 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:56 smithi022 ceph-mon[88798]: from='client.34388 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:37:57.072 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:56 smithi022 ceph-mon[88798]: mds.? [v2:172.21.15.22:6826/2535680483,v1:172.21.15.22:6827/2535680483] up:boot 2024-02-16T19:37:57.072 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:56 smithi022 ceph-mon[88798]: fsmap cephfs:1 {0=cephfs.smithi022.btppwm=up:active} 3 up:standby 2024-02-16T19:37:57.072 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:56 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi022.ocukvs"}]: dispatch 2024-02-16T19:37:57.072 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:56 smithi022 ceph-mon[88798]: from='client.? 172.21.15.22:0/2743620819' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:37:57.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:56 smithi117 ceph-mon[67870]: from='client.34380 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:37:57.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:56 smithi117 ceph-mon[67870]: from='client.34388 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:37:57.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:56 smithi117 ceph-mon[67870]: mds.? [v2:172.21.15.22:6826/2535680483,v1:172.21.15.22:6827/2535680483] up:boot 2024-02-16T19:37:57.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:56 smithi117 ceph-mon[67870]: fsmap cephfs:1 {0=cephfs.smithi022.btppwm=up:active} 3 up:standby 2024-02-16T19:37:57.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:56 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi022.ocukvs"}]: dispatch 2024-02-16T19:37:57.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:56 smithi117 ceph-mon[67870]: from='client.? 172.21.15.22:0/2743620819' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:37:57.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:57 smithi022 ceph-mon[88798]: pgmap v183: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:37:57.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:57 smithi022 ceph-mon[88798]: from='client.34400 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:37:57.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:57 smithi022 ceph-mon[88798]: from='client.? 172.21.15.22:0/1724004321' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T19:37:58.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:57 smithi117 ceph-mon[67870]: pgmap v183: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:37:58.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:57 smithi117 ceph-mon[67870]: from='client.34400 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:37:58.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:57 smithi117 ceph-mon[67870]: from='client.? 172.21.15.22:0/1724004321' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T19:37:59.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:37:59 smithi117 ceph-mon[67870]: pgmap v184: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:38:00.063 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:37:59 smithi022 ceph-mon[88798]: pgmap v184: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:38:01.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:01 smithi117 ceph-mon[67870]: pgmap v185: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:38:01.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:01 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:38:01.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:01 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:38:01.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:01 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:38:01.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:01 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:38:01.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:01 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:38:01.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:01 smithi022 ceph-mon[88798]: pgmap v185: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:38:01.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:01 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:38:01.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:01 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:38:01.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:01 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:38:01.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:01 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:38:01.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:01 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:38:02.869 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:02 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:38:02.869 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:02 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:38:03.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:02 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:38:03.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:02 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:38:03.948 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:03 smithi022 ceph-mon[88798]: pgmap v186: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:38:04.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:03 smithi117 ceph-mon[67870]: pgmap v186: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:38:05.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:05 smithi022 ceph-mon[88798]: pgmap v187: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:38:05.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:05 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:38:05.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:05 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:38:05.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:05 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:38:05.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:05 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:38:05.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:05 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:38:05.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:05 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:38:05.967 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:05 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:38:05.967 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:05 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:38:05.967 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:05 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:38:05.967 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:05 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:38:06.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:05 smithi117 ceph-mon[67870]: pgmap v187: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:38:06.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:05 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:38:06.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:05 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:38:06.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:05 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:38:06.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:05 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:38:06.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:05 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:38:06.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:05 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:38:06.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:05 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:38:06.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:05 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:38:06.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:05 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:38:06.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:05 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:38:07.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:07 smithi022 ceph-mon[88798]: pgmap v188: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:38:07.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:07 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:38:07.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:07 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi022.btppwm", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T19:38:07.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:07 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:38:08.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:07 smithi117 ceph-mon[67870]: pgmap v188: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:38:08.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:07 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:38:08.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:07 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi022.btppwm", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T19:38:08.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:07 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:38:08.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:08 smithi022 ceph-mon[88798]: Upgrade: Updating mds.cephfs.smithi022.btppwm 2024-02-16T19:38:08.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:08 smithi022 ceph-mon[88798]: Deploying daemon mds.cephfs.smithi022.btppwm on smithi022 2024-02-16T19:38:09.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:08 smithi117 ceph-mon[67870]: Upgrade: Updating mds.cephfs.smithi022.btppwm 2024-02-16T19:38:09.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:08 smithi117 ceph-mon[67870]: Deploying daemon mds.cephfs.smithi022.btppwm on smithi022 2024-02-16T19:38:09.706 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:09 smithi022 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-mon-smithi022[88794]: 2024-02-16T19:38:09.296+0000 7fe662394700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-16T19:38:09.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:09 smithi022 ceph-mon[88798]: pgmap v189: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:38:09.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:09 smithi022 ceph-mon[88798]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-02-16T19:38:09.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:09 smithi022 ceph-mon[88798]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-16T19:38:09.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:09 smithi022 ceph-mon[88798]: osdmap e91: 6 total, 6 up, 6 in 2024-02-16T19:38:09.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:09 smithi022 ceph-mon[88798]: Standby daemon mds.cephfs.smithi117.ibmppn assigned to filesystem cephfs as rank 0 2024-02-16T19:38:09.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:09 smithi022 ceph-mon[88798]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-16T19:38:09.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:09 smithi022 ceph-mon[88798]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-02-16T19:38:09.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:09 smithi022 ceph-mon[88798]: fsmap cephfs:1/1 {0=cephfs.smithi117.ibmppn=up:replay} 2 up:standby 2024-02-16T19:38:10.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:09 smithi117 ceph-mon[67870]: pgmap v189: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:38:10.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:09 smithi117 ceph-mon[67870]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-02-16T19:38:10.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:09 smithi117 ceph-mon[67870]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-16T19:38:10.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:09 smithi117 ceph-mon[67870]: osdmap e91: 6 total, 6 up, 6 in 2024-02-16T19:38:10.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:09 smithi117 ceph-mon[67870]: Standby daemon mds.cephfs.smithi117.ibmppn assigned to filesystem cephfs as rank 0 2024-02-16T19:38:10.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:09 smithi117 ceph-mon[67870]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-16T19:38:10.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:09 smithi117 ceph-mon[67870]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-02-16T19:38:10.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:09 smithi117 ceph-mon[67870]: fsmap cephfs:1/1 {0=cephfs.smithi117.ibmppn=up:replay} 2 up:standby 2024-02-16T19:38:11.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:11 smithi022 ceph-mon[88798]: pgmap v191: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:38:12.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:11 smithi117 ceph-mon[67870]: pgmap v191: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:38:13.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:13 smithi022 ceph-mon[88798]: pgmap v192: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 3.2 MiB/s rd, 1 op/s 2024-02-16T19:38:14.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:13 smithi117 ceph-mon[67870]: pgmap v192: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 3.2 MiB/s rd, 1 op/s 2024-02-16T19:38:15.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:15 smithi022 ceph-mon[88798]: pgmap v193: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 7.6 MiB/s rd, 2 op/s 2024-02-16T19:38:16.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:15 smithi117 ceph-mon[67870]: pgmap v193: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 7.6 MiB/s rd, 2 op/s 2024-02-16T19:38:17.102 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:16 smithi022 ceph-mon[88798]: pgmap v194: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 18 MiB/s rd, 5 op/s 2024-02-16T19:38:17.102 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:16 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:38:17.102 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:16 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:38:17.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:16 smithi117 ceph-mon[67870]: pgmap v194: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 18 MiB/s rd, 5 op/s 2024-02-16T19:38:17.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:16 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:38:17.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:16 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:38:19.013 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:18 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:38:19.013 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:18 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:38:19.013 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:18 smithi022 ceph-mon[88798]: pgmap v195: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 6 op/s 2024-02-16T19:38:19.013 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:18 smithi022 ceph-mon[88798]: mds.? [v2:172.21.15.22:6828/3126701730,v1:172.21.15.22:6829/3126701730] up:boot 2024-02-16T19:38:19.013 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:18 smithi022 ceph-mon[88798]: fsmap cephfs:1/1 {0=cephfs.smithi117.ibmppn=up:replay} 3 up:standby 2024-02-16T19:38:19.013 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:18 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:38:19.013 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:18 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi022.btppwm"}]: dispatch 2024-02-16T19:38:19.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:18 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:38:19.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:18 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:38:19.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:18 smithi117 ceph-mon[67870]: pgmap v195: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 6 op/s 2024-02-16T19:38:19.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:18 smithi117 ceph-mon[67870]: mds.? [v2:172.21.15.22:6828/3126701730,v1:172.21.15.22:6829/3126701730] up:boot 2024-02-16T19:38:19.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:18 smithi117 ceph-mon[67870]: fsmap cephfs:1/1 {0=cephfs.smithi117.ibmppn=up:replay} 3 up:standby 2024-02-16T19:38:19.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:18 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:38:19.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:18 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi022.btppwm"}]: dispatch 2024-02-16T19:38:19.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:19 smithi022 ceph-mon[88798]: reconnect by client.24307 192.168.0.1:0/2099531446 after 0 2024-02-16T19:38:19.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:19 smithi022 ceph-mon[88798]: reconnect by client.14540 192.168.0.1:0/1593109629 after 0.000999979 2024-02-16T19:38:19.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:19 smithi022 ceph-mon[88798]: mds.? [v2:172.21.15.117:6824/565408102,v1:172.21.15.117:6825/565408102] up:reconnect 2024-02-16T19:38:19.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:19 smithi022 ceph-mon[88798]: fsmap cephfs:1/1 {0=cephfs.smithi117.ibmppn=up:reconnect} 3 up:standby 2024-02-16T19:38:20.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:19 smithi117 ceph-mon[67870]: reconnect by client.24307 192.168.0.1:0/2099531446 after 0 2024-02-16T19:38:20.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:19 smithi117 ceph-mon[67870]: reconnect by client.14540 192.168.0.1:0/1593109629 after 0.000999979 2024-02-16T19:38:20.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:19 smithi117 ceph-mon[67870]: mds.? [v2:172.21.15.117:6824/565408102,v1:172.21.15.117:6825/565408102] up:reconnect 2024-02-16T19:38:20.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:19 smithi117 ceph-mon[67870]: fsmap cephfs:1/1 {0=cephfs.smithi117.ibmppn=up:reconnect} 3 up:standby 2024-02-16T19:38:21.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:20 smithi117 ceph-mon[67870]: pgmap v196: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 22 MiB/s rd, 6 op/s 2024-02-16T19:38:21.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:20 smithi117 ceph-mon[67870]: mds.? [v2:172.21.15.117:6824/565408102,v1:172.21.15.117:6825/565408102] up:rejoin 2024-02-16T19:38:21.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:20 smithi117 ceph-mon[67870]: fsmap cephfs:1/1 {0=cephfs.smithi117.ibmppn=up:rejoin} 3 up:standby 2024-02-16T19:38:21.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:20 smithi117 ceph-mon[67870]: daemon mds.cephfs.smithi117.ibmppn is now active in filesystem cephfs as rank 0 2024-02-16T19:38:21.207 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:20 smithi022 ceph-mon[88798]: pgmap v196: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 22 MiB/s rd, 6 op/s 2024-02-16T19:38:21.207 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:20 smithi022 ceph-mon[88798]: mds.? [v2:172.21.15.117:6824/565408102,v1:172.21.15.117:6825/565408102] up:rejoin 2024-02-16T19:38:21.207 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:20 smithi022 ceph-mon[88798]: fsmap cephfs:1/1 {0=cephfs.smithi117.ibmppn=up:rejoin} 3 up:standby 2024-02-16T19:38:21.208 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:20 smithi022 ceph-mon[88798]: daemon mds.cephfs.smithi117.ibmppn is now active in filesystem cephfs as rank 0 2024-02-16T19:38:22.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:21 smithi117 ceph-mon[67870]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-02-16T19:38:22.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:21 smithi117 ceph-mon[67870]: mds.? [v2:172.21.15.117:6824/565408102,v1:172.21.15.117:6825/565408102] up:active 2024-02-16T19:38:22.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:21 smithi117 ceph-mon[67870]: fsmap cephfs:1 {0=cephfs.smithi117.ibmppn=up:active} 3 up:standby 2024-02-16T19:38:22.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:21 smithi022 ceph-mon[88798]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-02-16T19:38:22.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:21 smithi022 ceph-mon[88798]: mds.? [v2:172.21.15.117:6824/565408102,v1:172.21.15.117:6825/565408102] up:active 2024-02-16T19:38:22.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:21 smithi022 ceph-mon[88798]: fsmap cephfs:1 {0=cephfs.smithi117.ibmppn=up:active} 3 up:standby 2024-02-16T19:38:22.923 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:22 smithi022 ceph-mon[88798]: pgmap v197: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 255 B/s wr, 8 op/s 2024-02-16T19:38:23.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:22 smithi117 ceph-mon[67870]: pgmap v197: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 255 B/s wr, 8 op/s 2024-02-16T19:38:24.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:24 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:38:24.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:24 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:38:24.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:24 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:38:24.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:24 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:38:25.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:25 smithi022 ceph-mon[88798]: pgmap v198: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 19 MiB/s rd, 341 B/s wr, 7 op/s 2024-02-16T19:38:25.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:25 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:38:25.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:25 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:38:25.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:25 smithi117 ceph-mon[67870]: pgmap v198: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 19 MiB/s rd, 341 B/s wr, 7 op/s 2024-02-16T19:38:25.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:25 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:38:25.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:25 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:38:26.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:26.959+0000 7fae1201d700 1 -- 172.21.15.22:0/2052763422 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae0c0ffd60 msgr2=0x7fae0c103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:38:26.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:26.959+0000 7fae1201d700 1 --2- 172.21.15.22:0/2052763422 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae0c0ffd60 0x7fae0c103d30 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fae00009910 tx=0x7fae00009c20 comp rx=0 tx=0).stop 2024-02-16T19:38:26.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:26.959+0000 7fae1201d700 1 -- 172.21.15.22:0/2052763422 shutdown_connections 2024-02-16T19:38:26.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:26.959+0000 7fae1201d700 1 --2- 172.21.15.22:0/2052763422 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae0c0ffd60 0x7fae0c103d30 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:38:26.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:26.959+0000 7fae1201d700 1 --2- 172.21.15.22:0/2052763422 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae0c0ff3c0 0x7fae0c0ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:38:26.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:26.959+0000 7fae1201d700 1 -- 172.21.15.22:0/2052763422 >> 172.21.15.22:0/2052763422 conn(0x7fae0c0fac60 msgr2=0x7fae0c0fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:38:26.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:26.959+0000 7fae1201d700 1 -- 172.21.15.22:0/2052763422 shutdown_connections 2024-02-16T19:38:26.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:26.959+0000 7fae1201d700 1 -- 172.21.15.22:0/2052763422 wait complete. 2024-02-16T19:38:26.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:26.960+0000 7fae1201d700 1 Processor -- start 2024-02-16T19:38:26.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:26.960+0000 7fae1201d700 1 -- start start 2024-02-16T19:38:26.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:26.960+0000 7fae1201d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae0c0ff3c0 0x7fae0c196cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:38:26.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:26.960+0000 7fae1201d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae0c0ffd60 0x7fae0c197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:38:26.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:26.960+0000 7fae1201d700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0c197820 con 0x7fae0c0ffd60 2024-02-16T19:38:26.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:26.960+0000 7fae1201d700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0c197990 con 0x7fae0c0ff3c0 2024-02-16T19:38:26.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:26.961+0000 7fae0affd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae0c0ffd60 0x7fae0c197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:38:26.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:26.961+0000 7fae0affd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae0c0ffd60 0x7fae0c197200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33090/0 (socket says 172.21.15.22:33090) 2024-02-16T19:38:26.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:26.961+0000 7fae0affd700 1 -- 172.21.15.22:0/2822729261 learned_addr learned my addr 172.21.15.22:0/2822729261 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:38:26.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:26.961+0000 7fae0b7fe700 1 --2- 172.21.15.22:0/2822729261 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae0c0ff3c0 0x7fae0c196cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:38:26.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:26.961+0000 7fae0affd700 1 -- 172.21.15.22:0/2822729261 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae0c0ff3c0 msgr2=0x7fae0c196cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:38:26.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:26.961+0000 7fae0affd700 1 --2- 172.21.15.22:0/2822729261 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae0c0ff3c0 0x7fae0c196cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:38:26.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:26.961+0000 7fae0affd700 1 -- 172.21.15.22:0/2822729261 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae000095f0 con 0x7fae0c0ffd60 2024-02-16T19:38:26.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:26.961+0000 7fae0affd700 1 --2- 172.21.15.22:0/2822729261 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae0c0ffd60 0x7fae0c197200 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fae0000bda0 tx=0x7fae0000bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:38:26.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:26.961+0000 7fae08ff9700 1 -- 172.21.15.22:0/2822729261 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae0001d070 con 0x7fae0c0ffd60 2024-02-16T19:38:26.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:26.961+0000 7fae08ff9700 1 -- 172.21.15.22:0/2822729261 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fae00005dc0 con 0x7fae0c0ffd60 2024-02-16T19:38:26.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:26.961+0000 7fae1201d700 1 -- 172.21.15.22:0/2822729261 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae0c19fd30 con 0x7fae0c0ffd60 2024-02-16T19:38:26.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:26.961+0000 7fae08ff9700 1 -- 172.21.15.22:0/2822729261 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae00017400 con 0x7fae0c0ffd60 2024-02-16T19:38:26.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:26.962+0000 7fae1201d700 1 -- 172.21.15.22:0/2822729261 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae0c1a01a0 con 0x7fae0c0ffd60 2024-02-16T19:38:26.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:26.962+0000 7fae1201d700 1 -- 172.21.15.22:0/2822729261 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae0c107730 con 0x7fae0c0ffd60 2024-02-16T19:38:26.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:26.963+0000 7fae08ff9700 1 -- 172.21.15.22:0/2822729261 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fae0000f460 con 0x7fae0c0ffd60 2024-02-16T19:38:26.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:26.964+0000 7fae08ff9700 1 --2- 172.21.15.22:0/2822729261 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fadf406d8a0 0x7fadf406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:38:26.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:26.964+0000 7fae08ff9700 1 -- 172.21.15.22:0/2822729261 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6790+0+0 (secure 0 0 0) 0x7fae0008e9e0 con 0x7fae0c0ffd60 2024-02-16T19:38:26.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:26.964+0000 7fae0b7fe700 1 --2- 172.21.15.22:0/2822729261 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fadf406d8a0 0x7fadf406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:38:26.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:26.964+0000 7fae0b7fe700 1 --2- 172.21.15.22:0/2822729261 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fadf406d8a0 0x7fadf406fd50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fadfc007be0 tx=0x7fadfc0073d0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:38:26.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:26.969+0000 7fae08ff9700 1 -- 172.21.15.22:0/2822729261 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fae000580c0 con 0x7fae0c0ffd60 2024-02-16T19:38:27.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.200+0000 7fae1201d700 1 -- 172.21.15.22:0/2822729261 --> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fae0c061190 con 0x7fadf406d8a0 2024-02-16T19:38:27.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.202+0000 7fae08ff9700 1 -- 172.21.15.22:0/2822729261 <== mgr.34104 v2:172.21.15.22:6800/505266753 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fae0c061190 con 0x7fadf406d8a0 2024-02-16T19:38:27.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.204+0000 7fae1201d700 1 -- 172.21.15.22:0/2822729261 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fadf406d8a0 msgr2=0x7fadf406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:38:27.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.204+0000 7fae1201d700 1 --2- 172.21.15.22:0/2822729261 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fadf406d8a0 0x7fadf406fd50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fadfc007be0 tx=0x7fadfc0073d0 comp rx=0 tx=0).stop 2024-02-16T19:38:27.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.204+0000 7fae1201d700 1 -- 172.21.15.22:0/2822729261 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae0c0ffd60 msgr2=0x7fae0c197200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:38:27.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.204+0000 7fae1201d700 1 --2- 172.21.15.22:0/2822729261 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae0c0ffd60 0x7fae0c197200 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fae0000bda0 tx=0x7fae0000bdd0 comp rx=0 tx=0).stop 2024-02-16T19:38:27.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.204+0000 7fae1201d700 1 -- 172.21.15.22:0/2822729261 shutdown_connections 2024-02-16T19:38:27.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.204+0000 7fae1201d700 1 --2- 172.21.15.22:0/2822729261 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fadf406d8a0 0x7fadf406fd50 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:38:27.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.204+0000 7fae1201d700 1 --2- 172.21.15.22:0/2822729261 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae0c0ff3c0 0x7fae0c196cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:38:27.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.204+0000 7fae1201d700 1 --2- 172.21.15.22:0/2822729261 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae0c0ffd60 0x7fae0c197200 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:38:27.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.204+0000 7fae1201d700 1 -- 172.21.15.22:0/2822729261 >> 172.21.15.22:0/2822729261 conn(0x7fae0c0fac60 msgr2=0x7fae0c0fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:38:27.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.204+0000 7fae1201d700 1 -- 172.21.15.22:0/2822729261 shutdown_connections 2024-02-16T19:38:27.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.204+0000 7fae1201d700 1 -- 172.21.15.22:0/2822729261 wait complete. 2024-02-16T19:38:27.217 INFO:teuthology.orchestra.run.smithi022.stdout:true 2024-02-16T19:38:27.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.317+0000 7fd137ff1700 1 -- 172.21.15.22:0/2935077561 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1301038b0 msgr2=0x7fd130103c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:38:27.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.317+0000 7fd137ff1700 1 --2- 172.21.15.22:0/2935077561 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1301038b0 0x7fd130103c80 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd12c009910 tx=0x7fd12c009c20 comp rx=0 tx=0).stop 2024-02-16T19:38:27.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.317+0000 7fd137ff1700 1 -- 172.21.15.22:0/2935077561 shutdown_connections 2024-02-16T19:38:27.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.317+0000 7fd137ff1700 1 --2- 172.21.15.22:0/2935077561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1300ff350 0x7fd1300ff7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:38:27.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.317+0000 7fd137ff1700 1 --2- 172.21.15.22:0/2935077561 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1301038b0 0x7fd130103c80 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:38:27.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.317+0000 7fd137ff1700 1 -- 172.21.15.22:0/2935077561 >> 172.21.15.22:0/2935077561 conn(0x7fd1300748b0 msgr2=0x7fd130074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:38:27.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.317+0000 7fd137ff1700 1 -- 172.21.15.22:0/2935077561 shutdown_connections 2024-02-16T19:38:27.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.317+0000 7fd137ff1700 1 -- 172.21.15.22:0/2935077561 wait complete. 2024-02-16T19:38:27.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.317+0000 7fd137ff1700 1 Processor -- start 2024-02-16T19:38:27.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.318+0000 7fd137ff1700 1 -- start start 2024-02-16T19:38:27.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.318+0000 7fd137ff1700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1300ff350 0x7fd130196c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:38:27.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.318+0000 7fd137ff1700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1301038b0 0x7fd130197170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:38:27.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.318+0000 7fd137ff1700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd130197850 con 0x7fd1300ff350 2024-02-16T19:38:27.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.318+0000 7fd137ff1700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd130197990 con 0x7fd1301038b0 2024-02-16T19:38:27.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.318+0000 7fd135d8d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1300ff350 0x7fd130196c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:38:27.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.318+0000 7fd135d8d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1300ff350 0x7fd130196c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33102/0 (socket says 172.21.15.22:33102) 2024-02-16T19:38:27.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.318+0000 7fd135d8d700 1 -- 172.21.15.22:0/1841639115 learned_addr learned my addr 172.21.15.22:0/1841639115 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:38:27.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.318+0000 7fd13558c700 1 --2- 172.21.15.22:0/1841639115 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1301038b0 0x7fd130197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:38:27.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.318+0000 7fd135d8d700 1 -- 172.21.15.22:0/1841639115 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1301038b0 msgr2=0x7fd130197170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:38:27.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.318+0000 7fd135d8d700 1 --2- 172.21.15.22:0/1841639115 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1301038b0 0x7fd130197170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:38:27.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.318+0000 7fd135d8d700 1 -- 172.21.15.22:0/1841639115 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd12c0095f0 con 0x7fd1300ff350 2024-02-16T19:38:27.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.318+0000 7fd135d8d700 1 --2- 172.21.15.22:0/1841639115 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1300ff350 0x7fd130196c30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd12c000c00 tx=0x7fd12c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:38:27.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.319+0000 7fd126ffd700 1 -- 172.21.15.22:0/1841639115 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd12c01d070 con 0x7fd1300ff350 2024-02-16T19:38:27.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.319+0000 7fd137ff1700 1 -- 172.21.15.22:0/1841639115 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd13019fca0 con 0x7fd1300ff350 2024-02-16T19:38:27.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.319+0000 7fd126ffd700 1 -- 172.21.15.22:0/1841639115 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd12c00f460 con 0x7fd1300ff350 2024-02-16T19:38:27.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.319+0000 7fd126ffd700 1 -- 172.21.15.22:0/1841639115 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd12c005e20 con 0x7fd1300ff350 2024-02-16T19:38:27.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.319+0000 7fd137ff1700 1 -- 172.21.15.22:0/1841639115 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1301a0160 con 0x7fd1300ff350 2024-02-16T19:38:27.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.320+0000 7fd124ff9700 1 -- 172.21.15.22:0/1841639115 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd13004f2a0 con 0x7fd1300ff350 2024-02-16T19:38:27.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.321+0000 7fd126ffd700 1 -- 172.21.15.22:0/1841639115 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fd12c00f5d0 con 0x7fd1300ff350 2024-02-16T19:38:27.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.321+0000 7fd126ffd700 1 --2- 172.21.15.22:0/1841639115 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fd11c06d8a0 0x7fd11c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:38:27.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.321+0000 7fd126ffd700 1 -- 172.21.15.22:0/1841639115 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6790+0+0 (secure 0 0 0) 0x7fd12c08eb00 con 0x7fd1300ff350 2024-02-16T19:38:27.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.321+0000 7fd13558c700 1 --2- 172.21.15.22:0/1841639115 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fd11c06d8a0 0x7fd11c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:38:27.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.322+0000 7fd13558c700 1 --2- 172.21.15.22:0/1841639115 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fd11c06d8a0 0x7fd11c06fd50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd13010a370 tx=0x7fd12000b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:38:27.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.329+0000 7fd126ffd700 1 -- 172.21.15.22:0/1841639115 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd12c058320 con 0x7fd1300ff350 2024-02-16T19:38:27.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.560+0000 7fd124ff9700 1 -- 172.21.15.22:0/1841639115 --> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd130109d30 con 0x7fd11c06d8a0 2024-02-16T19:38:27.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.561+0000 7fd126ffd700 1 -- 172.21.15.22:0/1841639115 <== mgr.34104 v2:172.21.15.22:6800/505266753 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fd130109d30 con 0x7fd11c06d8a0 2024-02-16T19:38:27.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.563+0000 7fd124ff9700 1 -- 172.21.15.22:0/1841639115 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fd11c06d8a0 msgr2=0x7fd11c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:38:27.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.563+0000 7fd124ff9700 1 --2- 172.21.15.22:0/1841639115 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fd11c06d8a0 0x7fd11c06fd50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd13010a370 tx=0x7fd12000b410 comp rx=0 tx=0).stop 2024-02-16T19:38:27.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.563+0000 7fd124ff9700 1 -- 172.21.15.22:0/1841639115 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1300ff350 msgr2=0x7fd130196c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:38:27.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.563+0000 7fd124ff9700 1 --2- 172.21.15.22:0/1841639115 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1300ff350 0x7fd130196c30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd12c000c00 tx=0x7fd12c00bd70 comp rx=0 tx=0).stop 2024-02-16T19:38:27.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.564+0000 7fd124ff9700 1 -- 172.21.15.22:0/1841639115 shutdown_connections 2024-02-16T19:38:27.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.564+0000 7fd124ff9700 1 --2- 172.21.15.22:0/1841639115 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fd11c06d8a0 0x7fd11c06fd50 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:38:27.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.564+0000 7fd124ff9700 1 --2- 172.21.15.22:0/1841639115 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1300ff350 0x7fd130196c30 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:38:27.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.564+0000 7fd124ff9700 1 --2- 172.21.15.22:0/1841639115 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1301038b0 0x7fd130197170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:38:27.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.564+0000 7fd124ff9700 1 -- 172.21.15.22:0/1841639115 >> 172.21.15.22:0/1841639115 conn(0x7fd1300748b0 msgr2=0x7fd1300fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:38:27.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.564+0000 7fd124ff9700 1 -- 172.21.15.22:0/1841639115 shutdown_connections 2024-02-16T19:38:27.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.564+0000 7fd124ff9700 1 -- 172.21.15.22:0/1841639115 wait complete. 2024-02-16T19:38:27.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.676+0000 7f3220572700 1 -- 172.21.15.22:0/3419261406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3218100e60 msgr2=0x7f3218104e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:38:27.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.676+0000 7f3220572700 1 --2- 172.21.15.22:0/3419261406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3218100e60 0x7f3218104e30 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3208009910 tx=0x7f3208009c20 comp rx=0 tx=0).stop 2024-02-16T19:38:27.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.676+0000 7f3220572700 1 -- 172.21.15.22:0/3419261406 shutdown_connections 2024-02-16T19:38:27.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.676+0000 7f3220572700 1 --2- 172.21.15.22:0/3419261406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3218100e60 0x7f3218104e30 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:38:27.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.676+0000 7f3220572700 1 --2- 172.21.15.22:0/3419261406 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f32181004c0 0x7f3218100890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:38:27.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.676+0000 7f3220572700 1 -- 172.21.15.22:0/3419261406 >> 172.21.15.22:0/3419261406 conn(0x7f32180fbd80 msgr2=0x7f32180fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:38:27.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.676+0000 7f3220572700 1 -- 172.21.15.22:0/3419261406 shutdown_connections 2024-02-16T19:38:27.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.676+0000 7f3220572700 1 -- 172.21.15.22:0/3419261406 wait complete. 2024-02-16T19:38:27.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.676+0000 7f3220572700 1 Processor -- start 2024-02-16T19:38:27.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.676+0000 7f3220572700 1 -- start start 2024-02-16T19:38:27.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.677+0000 7f3220572700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32181004c0 0x7f3218198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:38:27.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.677+0000 7f3220572700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3218100e60 0x7f3218198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:38:27.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.677+0000 7f3220572700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3218198e50 con 0x7f32181004c0 2024-02-16T19:38:27.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.677+0000 7f3220572700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f321819cbe0 con 0x7f3218100e60 2024-02-16T19:38:27.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.677+0000 7f321e30e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32181004c0 0x7f3218198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:38:27.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.677+0000 7f321e30e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32181004c0 0x7f3218198230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33122/0 (socket says 172.21.15.22:33122) 2024-02-16T19:38:27.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.677+0000 7f321e30e700 1 -- 172.21.15.22:0/129783312 learned_addr learned my addr 172.21.15.22:0/129783312 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:38:27.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.677+0000 7f321db0d700 1 --2- 172.21.15.22:0/129783312 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3218100e60 0x7f3218198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:38:27.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.677+0000 7f321e30e700 1 -- 172.21.15.22:0/129783312 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3218100e60 msgr2=0x7f3218198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:38:27.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.677+0000 7f321e30e700 1 --2- 172.21.15.22:0/129783312 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3218100e60 0x7f3218198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:38:27.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.677+0000 7f321e30e700 1 -- 172.21.15.22:0/129783312 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32080095f0 con 0x7f32181004c0 2024-02-16T19:38:27.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.677+0000 7f321db0d700 1 --2- 172.21.15.22:0/129783312 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3218100e60 0x7f3218198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-16T19:38:27.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.677+0000 7f321e30e700 1 --2- 172.21.15.22:0/129783312 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32181004c0 0x7f3218198230 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f321400b4b0 tx=0x7f321400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:38:27.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.678+0000 7f320f7fe700 1 -- 172.21.15.22:0/129783312 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3214011800 con 0x7f32181004c0 2024-02-16T19:38:27.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.678+0000 7f320f7fe700 1 -- 172.21.15.22:0/129783312 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3214011e20 con 0x7f32181004c0 2024-02-16T19:38:27.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.678+0000 7f3220572700 1 -- 172.21.15.22:0/129783312 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f321819cec0 con 0x7f32181004c0 2024-02-16T19:38:27.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.678+0000 7f320f7fe700 1 -- 172.21.15.22:0/129783312 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f321400f2e0 con 0x7f32181004c0 2024-02-16T19:38:27.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.678+0000 7f3220572700 1 -- 172.21.15.22:0/129783312 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f321819d410 con 0x7f32181004c0 2024-02-16T19:38:27.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.679+0000 7f3220572700 1 -- 172.21.15.22:0/129783312 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32181045b0 con 0x7f32181004c0 2024-02-16T19:38:27.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.679+0000 7f320f7fe700 1 -- 172.21.15.22:0/129783312 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f3214010380 con 0x7f32181004c0 2024-02-16T19:38:27.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.680+0000 7f320f7fe700 1 --2- 172.21.15.22:0/129783312 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f320406d8a0 0x7f320406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:38:27.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.680+0000 7f320f7fe700 1 -- 172.21.15.22:0/129783312 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6790+0+0 (secure 0 0 0) 0x7f321408d110 con 0x7f32181004c0 2024-02-16T19:38:27.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.680+0000 7f321db0d700 1 --2- 172.21.15.22:0/129783312 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f320406d8a0 0x7f320406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:38:27.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.680+0000 7f321db0d700 1 --2- 172.21.15.22:0/129783312 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f320406d8a0 0x7f320406fd50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3208006010 tx=0x7f320800b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:38:27.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.684+0000 7f320f7fe700 1 -- 172.21.15.22:0/129783312 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3214056870 con 0x7f32181004c0 2024-02-16T19:38:27.824 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:27 smithi022 ceph-mon[88798]: pgmap v199: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 15 MiB/s rd, 682 B/s wr, 8 op/s 2024-02-16T19:38:27.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:27 smithi117 ceph-mon[67870]: pgmap v199: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 15 MiB/s rd, 682 B/s wr, 8 op/s 2024-02-16T19:38:27.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.914+0000 7f3220572700 1 -- 172.21.15.22:0/129783312 --> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f321819d6e0 con 0x7f320406d8a0 2024-02-16T19:38:27.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.926+0000 7f320f7fe700 1 -- 172.21.15.22:0/129783312 <== mgr.34104 v2:172.21.15.22:6800/505266753 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f321819d6e0 con 0x7f320406d8a0 2024-02-16T19:38:27.927 INFO:teuthology.orchestra.run.smithi022.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T19:38:27.927 INFO:teuthology.orchestra.run.smithi022.stdout:alertmanager.smithi022 smithi022 *:9093,9094 running (6m) 4s ago 17m 26.8M - 0.25.0 c8568f914cd2 c89bc76be374 2024-02-16T19:38:27.928 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-exporter.smithi022 smithi022 running (18m) 4s ago 17m 9198k - 18.2.0 dc2bc1663786 3f8bef4349ba 2024-02-16T19:38:27.928 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-exporter.smithi117 smithi117 running (16m) 45s ago 16m 8476k - 18.2.0 dc2bc1663786 59b65a7e309f 2024-02-16T19:38:27.928 INFO:teuthology.orchestra.run.smithi022.stdout:crash.smithi022 smithi022 running (4m) 4s ago 17m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a e0f78bdf43e4 2024-02-16T19:38:27.928 INFO:teuthology.orchestra.run.smithi022.stdout:crash.smithi117 smithi117 running (4m) 45s ago 16m 6669k - 19.0.0-1450-gf78a58c0 f081f847070a bce7c0590813 2024-02-16T19:38:27.928 INFO:teuthology.orchestra.run.smithi022.stdout:grafana.smithi022 smithi022 *:3000 running (6m) 4s ago 17m 84.3M - 9.4.12 a0ea5e3c46d9 fd27d11e9729 2024-02-16T19:38:27.928 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi022.btppwm smithi022 running (10s) 4s ago 11m 12.1M - 19.0.0-1450-gf78a58c0 f081f847070a 243e0becd1f8 2024-02-16T19:38:27.928 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi022.ocukvs smithi022 running (33s) 4s ago 12m 13.1M - 19.0.0-1450-gf78a58c0 f081f847070a e6ce6a81df63 2024-02-16T19:38:27.928 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi117.ibmppn smithi117 running (12m) 45s ago 12m 18.9M - 18.2.0 dc2bc1663786 8c606cbeeb86 2024-02-16T19:38:27.928 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi117.orzors smithi117 running (11m) 45s ago 11m 18.0M - 18.2.0 dc2bc1663786 7d2bc4df3feb 2024-02-16T19:38:27.928 INFO:teuthology.orchestra.run.smithi022.stdout:mgr.smithi022.gmdcch smithi022 *:8443,9283,8765 running (8m) 4s ago 19m 508M - 19.0.0-1450-gf78a58c0 f081f847070a 7cb74cc9deea 2024-02-16T19:38:27.928 INFO:teuthology.orchestra.run.smithi022.stdout:mgr.smithi117.cohbtv smithi117 *:8443,9283,8765 running (7m) 45s ago 16m 453M - 19.0.0-1450-gf78a58c0 f081f847070a c825b1152189 2024-02-16T19:38:27.928 INFO:teuthology.orchestra.run.smithi022.stdout:mon.smithi022 smithi022 running (5m) 4s ago 19m 54.1M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 9c84755ae67d 2024-02-16T19:38:27.928 INFO:teuthology.orchestra.run.smithi022.stdout:mon.smithi117 smithi117 running (4m) 45s ago 16m 40.7M 2048M 19.0.0-1450-gf78a58c0 f081f847070a a319ac50f1a9 2024-02-16T19:38:27.928 INFO:teuthology.orchestra.run.smithi022.stdout:node-exporter.smithi022 smithi022 *:9100 running (7m) 4s ago 17m 16.5M - 1.5.0 0da6a335fe13 1f5b58292d27 2024-02-16T19:38:27.929 INFO:teuthology.orchestra.run.smithi022.stdout:node-exporter.smithi117 smithi117 *:9100 running (7m) 45s ago 16m 15.2M - 1.5.0 0da6a335fe13 bb5e8491d5a6 2024-02-16T19:38:27.929 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0 smithi022 running (3m) 4s ago 15m 114M 1217M 19.0.0-1450-gf78a58c0 f081f847070a 002412b3525f 2024-02-16T19:38:27.929 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1 smithi022 running (3m) 4s ago 15m 118M 1217M 19.0.0-1450-gf78a58c0 f081f847070a c3488ce7b2a0 2024-02-16T19:38:27.929 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2 smithi022 running (2m) 4s ago 14m 95.2M 1217M 19.0.0-1450-gf78a58c0 f081f847070a 46138c50f79d 2024-02-16T19:38:27.929 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3 smithi117 running (116s) 45s ago 14m 92.3M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 28d7d8519e64 2024-02-16T19:38:27.929 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4 smithi117 running (83s) 45s ago 14m 90.2M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 367ba2085f6f 2024-02-16T19:38:27.929 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5 smithi117 running (49s) 45s ago 13m 15.1M 2240M 19.0.0-1450-gf78a58c0 f081f847070a a1a7625b7541 2024-02-16T19:38:27.929 INFO:teuthology.orchestra.run.smithi022.stdout:prometheus.smithi022 smithi022 *:9095 running (7m) 4s ago 17m 69.2M - 2.43.0 a07b618ecd1d c3b8689d0c97 2024-02-16T19:38:27.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.928+0000 7f3220572700 1 -- 172.21.15.22:0/129783312 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f320406d8a0 msgr2=0x7f320406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:38:27.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.928+0000 7f3220572700 1 --2- 172.21.15.22:0/129783312 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f320406d8a0 0x7f320406fd50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3208006010 tx=0x7f320800b540 comp rx=0 tx=0).stop 2024-02-16T19:38:27.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.928+0000 7f3220572700 1 -- 172.21.15.22:0/129783312 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32181004c0 msgr2=0x7f3218198230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:38:27.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.928+0000 7f3220572700 1 --2- 172.21.15.22:0/129783312 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32181004c0 0x7f3218198230 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f321400b4b0 tx=0x7f321400b870 comp rx=0 tx=0).stop 2024-02-16T19:38:27.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.928+0000 7f3220572700 1 -- 172.21.15.22:0/129783312 shutdown_connections 2024-02-16T19:38:27.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.928+0000 7f3220572700 1 --2- 172.21.15.22:0/129783312 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f320406d8a0 0x7f320406fd50 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:38:27.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.928+0000 7f3220572700 1 --2- 172.21.15.22:0/129783312 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32181004c0 0x7f3218198230 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:38:27.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.928+0000 7f3220572700 1 --2- 172.21.15.22:0/129783312 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3218100e60 0x7f3218198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:38:27.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.928+0000 7f3220572700 1 -- 172.21.15.22:0/129783312 >> 172.21.15.22:0/129783312 conn(0x7f32180fbd80 msgr2=0x7f32180fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:38:27.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.928+0000 7f3220572700 1 -- 172.21.15.22:0/129783312 shutdown_connections 2024-02-16T19:38:27.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:27.928+0000 7f3220572700 1 -- 172.21.15.22:0/129783312 wait complete. 2024-02-16T19:38:28.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.036+0000 7f58cb3c6700 1 -- 172.21.15.22:0/982216772 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58c4100e80 msgr2=0x7f58c4104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:38:28.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.036+0000 7f58cb3c6700 1 --2- 172.21.15.22:0/982216772 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58c4100e80 0x7f58c4104e50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f58b8009910 tx=0x7f58b8009c20 comp rx=0 tx=0).stop 2024-02-16T19:38:28.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.036+0000 7f58cb3c6700 1 -- 172.21.15.22:0/982216772 shutdown_connections 2024-02-16T19:38:28.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.036+0000 7f58cb3c6700 1 --2- 172.21.15.22:0/982216772 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58c4100e80 0x7f58c4104e50 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:38:28.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.036+0000 7f58cb3c6700 1 --2- 172.21.15.22:0/982216772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58c41004e0 0x7f58c41008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:38:28.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.036+0000 7f58cb3c6700 1 -- 172.21.15.22:0/982216772 >> 172.21.15.22:0/982216772 conn(0x7f58c40fbda0 msgr2=0x7f58c40fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:38:28.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.037+0000 7f58cb3c6700 1 -- 172.21.15.22:0/982216772 shutdown_connections 2024-02-16T19:38:28.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.037+0000 7f58cb3c6700 1 -- 172.21.15.22:0/982216772 wait complete. 2024-02-16T19:38:28.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.037+0000 7f58cb3c6700 1 Processor -- start 2024-02-16T19:38:28.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.037+0000 7f58cb3c6700 1 -- start start 2024-02-16T19:38:28.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.037+0000 7f58cb3c6700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58c41004e0 0x7f58c41980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:38:28.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.037+0000 7f58cb3c6700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58c4100e80 0x7f58c4198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:38:28.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.037+0000 7f58cb3c6700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c4198d00 con 0x7f58c4100e80 2024-02-16T19:38:28.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.037+0000 7f58cb3c6700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c419ca90 con 0x7f58c41004e0 2024-02-16T19:38:28.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.038+0000 7f58c8961700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58c4100e80 0x7f58c4198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:38:28.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.038+0000 7f58c8961700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58c4100e80 0x7f58c4198620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33138/0 (socket says 172.21.15.22:33138) 2024-02-16T19:38:28.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.038+0000 7f58c8961700 1 -- 172.21.15.22:0/4115378113 learned_addr learned my addr 172.21.15.22:0/4115378113 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:38:28.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.038+0000 7f58c9162700 1 --2- 172.21.15.22:0/4115378113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58c41004e0 0x7f58c41980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:38:28.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.038+0000 7f58c8961700 1 -- 172.21.15.22:0/4115378113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58c41004e0 msgr2=0x7f58c41980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:38:28.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.038+0000 7f58c8961700 1 --2- 172.21.15.22:0/4115378113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58c41004e0 0x7f58c41980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:38:28.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.038+0000 7f58c8961700 1 -- 172.21.15.22:0/4115378113 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58b80095f0 con 0x7f58c4100e80 2024-02-16T19:38:28.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.038+0000 7f58c8961700 1 --2- 172.21.15.22:0/4115378113 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58c4100e80 0x7f58c4198620 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f58b8005f50 tx=0x7f58b800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:38:28.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.038+0000 7f58b67fc700 1 -- 172.21.15.22:0/4115378113 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f58b801d070 con 0x7f58c4100e80 2024-02-16T19:38:28.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.038+0000 7f58cb3c6700 1 -- 172.21.15.22:0/4115378113 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58c419cd10 con 0x7f58c4100e80 2024-02-16T19:38:28.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.038+0000 7f58b67fc700 1 -- 172.21.15.22:0/4115378113 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f58b800f460 con 0x7f58c4100e80 2024-02-16T19:38:28.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.038+0000 7f58cb3c6700 1 -- 172.21.15.22:0/4115378113 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58c419d200 con 0x7f58c4100e80 2024-02-16T19:38:28.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.039+0000 7f58b67fc700 1 -- 172.21.15.22:0/4115378113 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f58b8017630 con 0x7f58c4100e80 2024-02-16T19:38:28.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.040+0000 7f58cb3c6700 1 -- 172.21.15.22:0/4115378113 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58a8005320 con 0x7f58c4100e80 2024-02-16T19:38:28.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.041+0000 7f58b67fc700 1 -- 172.21.15.22:0/4115378113 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f58b8003a60 con 0x7f58c4100e80 2024-02-16T19:38:28.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.041+0000 7f58b67fc700 1 --2- 172.21.15.22:0/4115378113 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f58b006d8a0 0x7f58b006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:38:28.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.041+0000 7f58b67fc700 1 -- 172.21.15.22:0/4115378113 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6790+0+0 (secure 0 0 0) 0x7f58b808e9e0 con 0x7f58c4100e80 2024-02-16T19:38:28.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.041+0000 7f58c9162700 1 --2- 172.21.15.22:0/4115378113 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f58b006d8a0 0x7f58b006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:38:28.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.042+0000 7f58c9162700 1 --2- 172.21.15.22:0/4115378113 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f58b006d8a0 0x7f58b006fd50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f58c0005950 tx=0x7f58c000a300 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:38:28.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.046+0000 7f58b67fc700 1 -- 172.21.15.22:0/4115378113 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f58b8058140 con 0x7f58c4100e80 2024-02-16T19:38:28.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.344+0000 7f58cb3c6700 1 -- 172.21.15.22:0/4115378113 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f58a8006200 con 0x7f58c4100e80 2024-02-16T19:38:28.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.345+0000 7f58b67fc700 1 -- 172.21.15.22:0/4115378113 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7f58b805b800 con 0x7f58c4100e80 2024-02-16T19:38:28.346 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-02-16T19:38:28.346 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": { 2024-02-16T19:38:28.346 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T19:38:28.346 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:38:28.346 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": { 2024-02-16T19:38:28.346 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T19:38:28.347 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:38:28.347 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": { 2024-02-16T19:38:28.347 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-16T19:38:28.347 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:38:28.347 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": { 2024-02-16T19:38:28.347 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2, 2024-02-16T19:38:28.347 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T19:38:28.347 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:38:28.347 INFO:teuthology.orchestra.run.smithi022.stdout: "overall": { 2024-02-16T19:38:28.347 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2, 2024-02-16T19:38:28.347 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 12 2024-02-16T19:38:28.347 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-02-16T19:38:28.347 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-02-16T19:38:28.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.347+0000 7f58cb3c6700 1 -- 172.21.15.22:0/4115378113 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f58b006d8a0 msgr2=0x7f58b006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:38:28.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.347+0000 7f58cb3c6700 1 --2- 172.21.15.22:0/4115378113 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f58b006d8a0 0x7f58b006fd50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f58c0005950 tx=0x7f58c000a300 comp rx=0 tx=0).stop 2024-02-16T19:38:28.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.347+0000 7f58cb3c6700 1 -- 172.21.15.22:0/4115378113 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58c4100e80 msgr2=0x7f58c4198620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:38:28.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.347+0000 7f58cb3c6700 1 --2- 172.21.15.22:0/4115378113 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58c4100e80 0x7f58c4198620 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f58b8005f50 tx=0x7f58b800bd70 comp rx=0 tx=0).stop 2024-02-16T19:38:28.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.347+0000 7f58cb3c6700 1 -- 172.21.15.22:0/4115378113 shutdown_connections 2024-02-16T19:38:28.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.347+0000 7f58cb3c6700 1 --2- 172.21.15.22:0/4115378113 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f58b006d8a0 0x7f58b006fd50 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:38:28.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.347+0000 7f58cb3c6700 1 --2- 172.21.15.22:0/4115378113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58c41004e0 0x7f58c41980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:38:28.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.347+0000 7f58cb3c6700 1 --2- 172.21.15.22:0/4115378113 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58c4100e80 0x7f58c4198620 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:38:28.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.347+0000 7f58cb3c6700 1 -- 172.21.15.22:0/4115378113 >> 172.21.15.22:0/4115378113 conn(0x7f58c40fbda0 msgr2=0x7f58c40fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:38:28.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.347+0000 7f58cb3c6700 1 -- 172.21.15.22:0/4115378113 shutdown_connections 2024-02-16T19:38:28.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.347+0000 7f58cb3c6700 1 -- 172.21.15.22:0/4115378113 wait complete. 2024-02-16T19:38:28.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.454+0000 7f22520b5700 1 -- 172.21.15.22:0/1523100637 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f224c104130 msgr2=0x7f224c104500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:38:28.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.454+0000 7f22520b5700 1 --2- 172.21.15.22:0/1523100637 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f224c104130 0x7f224c104500 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f223c009910 tx=0x7f223c009c20 comp rx=0 tx=0).stop 2024-02-16T19:38:28.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.454+0000 7f22520b5700 1 -- 172.21.15.22:0/1523100637 shutdown_connections 2024-02-16T19:38:28.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.454+0000 7f22520b5700 1 --2- 172.21.15.22:0/1523100637 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f224c0ffbd0 0x7f224c100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:38:28.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.454+0000 7f22520b5700 1 --2- 172.21.15.22:0/1523100637 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f224c104130 0x7f224c104500 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:38:28.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.454+0000 7f22520b5700 1 -- 172.21.15.22:0/1523100637 >> 172.21.15.22:0/1523100637 conn(0x7f224c075130 msgr2=0x7f224c075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:38:28.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.454+0000 7f22520b5700 1 -- 172.21.15.22:0/1523100637 shutdown_connections 2024-02-16T19:38:28.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.454+0000 7f22520b5700 1 -- 172.21.15.22:0/1523100637 wait complete. 2024-02-16T19:38:28.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.455+0000 7f22520b5700 1 Processor -- start 2024-02-16T19:38:28.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.455+0000 7f22520b5700 1 -- start start 2024-02-16T19:38:28.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.456+0000 7f22520b5700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f224c0ffbd0 0x7f224c194d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:38:28.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.456+0000 7f22520b5700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f224c104130 0x7f224c195280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:38:28.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.456+0000 7f22520b5700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f224c195960 con 0x7f224c0ffbd0 2024-02-16T19:38:28.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.456+0000 7f22520b5700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f224c1996f0 con 0x7f224c104130 2024-02-16T19:38:28.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.456+0000 7f224b7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f224c0ffbd0 0x7f224c194d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:38:28.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.456+0000 7f224b7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f224c0ffbd0 0x7f224c194d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33156/0 (socket says 172.21.15.22:33156) 2024-02-16T19:38:28.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.456+0000 7f224b7fe700 1 -- 172.21.15.22:0/4294520889 learned_addr learned my addr 172.21.15.22:0/4294520889 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:38:28.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.456+0000 7f224affd700 1 --2- 172.21.15.22:0/4294520889 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f224c104130 0x7f224c195280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:38:28.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.456+0000 7f224b7fe700 1 -- 172.21.15.22:0/4294520889 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f224c104130 msgr2=0x7f224c195280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:38:28.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.456+0000 7f224b7fe700 1 --2- 172.21.15.22:0/4294520889 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f224c104130 0x7f224c195280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:38:28.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.456+0000 7f224b7fe700 1 -- 172.21.15.22:0/4294520889 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f223c0095f0 con 0x7f224c0ffbd0 2024-02-16T19:38:28.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.456+0000 7f224affd700 1 --2- 172.21.15.22:0/4294520889 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f224c104130 0x7f224c195280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T19:38:28.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.456+0000 7f224b7fe700 1 --2- 172.21.15.22:0/4294520889 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f224c0ffbd0 0x7f224c194d40 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f223c00b7d0 tx=0x7f223c00bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:38:28.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.457+0000 7f2248ff9700 1 -- 172.21.15.22:0/4294520889 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f223c01d070 con 0x7f224c0ffbd0 2024-02-16T19:38:28.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.457+0000 7f22520b5700 1 -- 172.21.15.22:0/4294520889 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f224c199970 con 0x7f224c0ffbd0 2024-02-16T19:38:28.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.457+0000 7f22520b5700 1 -- 172.21.15.22:0/4294520889 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f224c199e60 con 0x7f224c0ffbd0 2024-02-16T19:38:28.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.457+0000 7f2248ff9700 1 -- 172.21.15.22:0/4294520889 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f223c0052a0 con 0x7f224c0ffbd0 2024-02-16T19:38:28.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.457+0000 7f2248ff9700 1 -- 172.21.15.22:0/4294520889 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f223c022470 con 0x7f224c0ffbd0 2024-02-16T19:38:28.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.458+0000 7f22520b5700 1 -- 172.21.15.22:0/4294520889 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f224c1076a0 con 0x7f224c0ffbd0 2024-02-16T19:38:28.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.459+0000 7f2248ff9700 1 -- 172.21.15.22:0/4294520889 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f223c00f850 con 0x7f224c0ffbd0 2024-02-16T19:38:28.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.459+0000 7f2248ff9700 1 --2- 172.21.15.22:0/4294520889 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f223406d8a0 0x7f223406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:38:28.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.459+0000 7f2248ff9700 1 -- 172.21.15.22:0/4294520889 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6790+0+0 (secure 0 0 0) 0x7f223c08ec10 con 0x7f224c0ffbd0 2024-02-16T19:38:28.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.460+0000 7f224affd700 1 --2- 172.21.15.22:0/4294520889 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f223406d8a0 0x7f223406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:38:28.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.460+0000 7f224affd700 1 --2- 172.21.15.22:0/4294520889 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f223406d8a0 0x7f223406fd50 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f224c196360 tx=0x7f2240009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:38:28.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.464+0000 7f2248ff9700 1 -- 172.21.15.22:0/4294520889 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f223c058410 con 0x7f224c0ffbd0 2024-02-16T19:38:28.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:28 smithi022 ceph-mon[88798]: from='client.34412 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:38:28.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:28 smithi022 ceph-mon[88798]: from='client.34416 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:38:28.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:28 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:38:28.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:28 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:38:28.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:28 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:38:28.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:28 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:38:28.716 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:28 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:38:28.716 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:28 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:38:28.716 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:28 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:38:28.716 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:28 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:38:28.716 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:28 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:38:28.716 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:28 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:38:28.716 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:28 smithi022 ceph-mon[88798]: from='client.? 172.21.15.22:0/4115378113' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:38:28.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.722+0000 7f22520b5700 1 -- 172.21.15.22:0/4294520889 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f224c19a190 con 0x7f224c0ffbd0 2024-02-16T19:38:28.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.723+0000 7f2248ff9700 1 -- 172.21.15.22:0/4294520889 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 23 v23) v1 ==== 76+0+1849 (secure 0 0 0) 0x7f223c05bad0 con 0x7f224c0ffbd0 2024-02-16T19:38:28.724 INFO:teuthology.orchestra.run.smithi022.stdout:e23 2024-02-16T19:38:28.724 INFO:teuthology.orchestra.run.smithi022.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T19:38:28.725 INFO:teuthology.orchestra.run.smithi022.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:38:28.725 INFO:teuthology.orchestra.run.smithi022.stdout:legacy client fscid: 1 2024-02-16T19:38:28.725 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:38:28.725 INFO:teuthology.orchestra.run.smithi022.stdout:Filesystem 'cephfs' (1) 2024-02-16T19:38:28.725 INFO:teuthology.orchestra.run.smithi022.stdout:fs_name cephfs 2024-02-16T19:38:28.725 INFO:teuthology.orchestra.run.smithi022.stdout:epoch 23 2024-02-16T19:38:28.725 INFO:teuthology.orchestra.run.smithi022.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T19:38:28.725 INFO:teuthology.orchestra.run.smithi022.stdout:created 2024-02-16T19:26:17.873644+0000 2024-02-16T19:38:28.725 INFO:teuthology.orchestra.run.smithi022.stdout:modified 2024-02-16T19:38:19.908559+0000 2024-02-16T19:38:28.725 INFO:teuthology.orchestra.run.smithi022.stdout:tableserver 0 2024-02-16T19:38:28.725 INFO:teuthology.orchestra.run.smithi022.stdout:root 0 2024-02-16T19:38:28.725 INFO:teuthology.orchestra.run.smithi022.stdout:session_timeout 60 2024-02-16T19:38:28.725 INFO:teuthology.orchestra.run.smithi022.stdout:session_autoclose 300 2024-02-16T19:38:28.725 INFO:teuthology.orchestra.run.smithi022.stdout:max_file_size 1099511627776 2024-02-16T19:38:28.725 INFO:teuthology.orchestra.run.smithi022.stdout:max_xattr_size 65536 2024-02-16T19:38:28.725 INFO:teuthology.orchestra.run.smithi022.stdout:required_client_features {} 2024-02-16T19:38:28.726 INFO:teuthology.orchestra.run.smithi022.stdout:last_failure 0 2024-02-16T19:38:28.726 INFO:teuthology.orchestra.run.smithi022.stdout:last_failure_osd_epoch 91 2024-02-16T19:38:28.726 INFO:teuthology.orchestra.run.smithi022.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:38:28.726 INFO:teuthology.orchestra.run.smithi022.stdout:max_mds 1 2024-02-16T19:38:28.726 INFO:teuthology.orchestra.run.smithi022.stdout:in 0 2024-02-16T19:38:28.726 INFO:teuthology.orchestra.run.smithi022.stdout:up {0=24259} 2024-02-16T19:38:28.726 INFO:teuthology.orchestra.run.smithi022.stdout:failed 2024-02-16T19:38:28.726 INFO:teuthology.orchestra.run.smithi022.stdout:damaged 2024-02-16T19:38:28.726 INFO:teuthology.orchestra.run.smithi022.stdout:stopped 2024-02-16T19:38:28.726 INFO:teuthology.orchestra.run.smithi022.stdout:data_pools [3] 2024-02-16T19:38:28.726 INFO:teuthology.orchestra.run.smithi022.stdout:metadata_pool 2 2024-02-16T19:38:28.726 INFO:teuthology.orchestra.run.smithi022.stdout:inline_data enabled 2024-02-16T19:38:28.726 INFO:teuthology.orchestra.run.smithi022.stdout:balancer 2024-02-16T19:38:28.726 INFO:teuthology.orchestra.run.smithi022.stdout:bal_rank_mask -1 2024-02-16T19:38:28.726 INFO:teuthology.orchestra.run.smithi022.stdout:standby_count_wanted 1 2024-02-16T19:38:28.727 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.cephfs.smithi117.ibmppn{0:24259} state up:active seq 182 join_fscid=1 addr [v2:172.21.15.117:6824/565408102,v1:172.21.15.117:6825/565408102] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:38:28.727 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:38:28.727 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:38:28.727 INFO:teuthology.orchestra.run.smithi022.stdout:Standby daemons: 2024-02-16T19:38:28.727 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:38:28.727 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.cephfs.smithi117.orzors{-1:24265} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.117:6826/3122342028,v1:172.21.15.117:6827/3122342028] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:38:28.727 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.cephfs.smithi022.ocukvs{-1:34384} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.22:6826/2535680483,v1:172.21.15.22:6827/2535680483] compat {c=[1],r=[1],i=[fff]}] 2024-02-16T19:38:28.727 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.cephfs.smithi022.btppwm{-1:34408} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.22:6828/3126701730,v1:172.21.15.22:6829/3126701730] compat {c=[1],r=[1],i=[fff]}] 2024-02-16T19:38:28.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.725+0000 7f22520b5700 1 -- 172.21.15.22:0/4294520889 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f223406d8a0 msgr2=0x7f223406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:38:28.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.725+0000 7f22520b5700 1 --2- 172.21.15.22:0/4294520889 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f223406d8a0 0x7f223406fd50 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f224c196360 tx=0x7f2240009260 comp rx=0 tx=0).stop 2024-02-16T19:38:28.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.725+0000 7f22520b5700 1 -- 172.21.15.22:0/4294520889 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f224c0ffbd0 msgr2=0x7f224c194d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:38:28.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.725+0000 7f22520b5700 1 --2- 172.21.15.22:0/4294520889 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f224c0ffbd0 0x7f224c194d40 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f223c00b7d0 tx=0x7f223c00bf90 comp rx=0 tx=0).stop 2024-02-16T19:38:28.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.725+0000 7f22520b5700 1 -- 172.21.15.22:0/4294520889 shutdown_connections 2024-02-16T19:38:28.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.725+0000 7f22520b5700 1 --2- 172.21.15.22:0/4294520889 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f223406d8a0 0x7f223406fd50 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:38:28.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.725+0000 7f22520b5700 1 --2- 172.21.15.22:0/4294520889 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f224c0ffbd0 0x7f224c194d40 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:38:28.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.725+0000 7f22520b5700 1 --2- 172.21.15.22:0/4294520889 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f224c104130 0x7f224c195280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:38:28.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.725+0000 7f22520b5700 1 -- 172.21.15.22:0/4294520889 >> 172.21.15.22:0/4294520889 conn(0x7f224c075130 msgr2=0x7f224c0fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:38:28.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.725+0000 7f22520b5700 1 -- 172.21.15.22:0/4294520889 shutdown_connections 2024-02-16T19:38:28.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.726+0000 7f22520b5700 1 -- 172.21.15.22:0/4294520889 wait complete. 2024-02-16T19:38:28.729 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 23 2024-02-16T19:38:28.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.833+0000 7f914f678700 1 -- 172.21.15.22:0/4091790572 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91480ffd50 msgr2=0x7f9148103d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:38:28.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.833+0000 7f914f678700 1 --2- 172.21.15.22:0/4091790572 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91480ffd50 0x7f9148103d20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9144009910 tx=0x7f9144009c20 comp rx=0 tx=0).stop 2024-02-16T19:38:28.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.833+0000 7f914f678700 1 -- 172.21.15.22:0/4091790572 shutdown_connections 2024-02-16T19:38:28.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.833+0000 7f914f678700 1 --2- 172.21.15.22:0/4091790572 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91480ffd50 0x7f9148103d20 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:38:28.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.833+0000 7f914f678700 1 --2- 172.21.15.22:0/4091790572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91480ff3b0 0x7f91480ff780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:38:28.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.833+0000 7f914f678700 1 -- 172.21.15.22:0/4091790572 >> 172.21.15.22:0/4091790572 conn(0x7f91480fac90 msgr2=0x7f91480fd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:38:28.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.833+0000 7f914f678700 1 -- 172.21.15.22:0/4091790572 shutdown_connections 2024-02-16T19:38:28.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.833+0000 7f914f678700 1 -- 172.21.15.22:0/4091790572 wait complete. 2024-02-16T19:38:28.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.834+0000 7f914f678700 1 Processor -- start 2024-02-16T19:38:28.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.834+0000 7f914f678700 1 -- start start 2024-02-16T19:38:28.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.834+0000 7f914f678700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91480ff3b0 0x7f9148195e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:38:28.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.834+0000 7f914f678700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91480ffd50 0x7f91481963c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:38:28.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.834+0000 7f914f678700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9148196aa0 con 0x7f91480ff3b0 2024-02-16T19:38:28.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.834+0000 7f914f678700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f914819a830 con 0x7f91480ffd50 2024-02-16T19:38:28.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.834+0000 7f914d414700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91480ff3b0 0x7f9148195e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:38:28.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.834+0000 7f914d414700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91480ff3b0 0x7f9148195e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33174/0 (socket says 172.21.15.22:33174) 2024-02-16T19:38:28.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.834+0000 7f914d414700 1 -- 172.21.15.22:0/3051518751 learned_addr learned my addr 172.21.15.22:0/3051518751 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:38:28.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.834+0000 7f914cc13700 1 --2- 172.21.15.22:0/3051518751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91480ffd50 0x7f91481963c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:38:28.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.834+0000 7f914d414700 1 -- 172.21.15.22:0/3051518751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91480ffd50 msgr2=0x7f91481963c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:38:28.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.834+0000 7f914d414700 1 --2- 172.21.15.22:0/3051518751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91480ffd50 0x7f91481963c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:38:28.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.834+0000 7f914d414700 1 -- 172.21.15.22:0/3051518751 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91440095f0 con 0x7f91480ff3b0 2024-02-16T19:38:28.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.835+0000 7f914d414700 1 --2- 172.21.15.22:0/3051518751 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91480ff3b0 0x7f9148195e80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f913800d8d0 tx=0x7f913800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:38:28.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.835+0000 7f913e7fc700 1 -- 172.21.15.22:0/3051518751 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9138009940 con 0x7f91480ff3b0 2024-02-16T19:38:28.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.835+0000 7f914f678700 1 -- 172.21.15.22:0/3051518751 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f914819ab10 con 0x7f91480ff3b0 2024-02-16T19:38:28.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.835+0000 7f913e7fc700 1 -- 172.21.15.22:0/3051518751 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9138010460 con 0x7f91480ff3b0 2024-02-16T19:38:28.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.835+0000 7f913e7fc700 1 -- 172.21.15.22:0/3051518751 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f913800f5f0 con 0x7f91480ff3b0 2024-02-16T19:38:28.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.835+0000 7f914f678700 1 -- 172.21.15.22:0/3051518751 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f914819b060 con 0x7f91480ff3b0 2024-02-16T19:38:28.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.836+0000 7f914f678700 1 -- 172.21.15.22:0/3051518751 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9148108730 con 0x7f91480ff3b0 2024-02-16T19:38:28.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.836+0000 7f913e7fc700 1 -- 172.21.15.22:0/3051518751 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f91380105d0 con 0x7f91480ff3b0 2024-02-16T19:38:28.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.837+0000 7f913e7fc700 1 --2- 172.21.15.22:0/3051518751 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f913406d8a0 0x7f913406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:38:28.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.837+0000 7f913e7fc700 1 -- 172.21.15.22:0/3051518751 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6790+0+0 (secure 0 0 0) 0x7f913808d6b0 con 0x7f91480ff3b0 2024-02-16T19:38:28.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.837+0000 7f914cc13700 1 --2- 172.21.15.22:0/3051518751 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f913406d8a0 0x7f913406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:38:28.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.837+0000 7f914cc13700 1 --2- 172.21.15.22:0/3051518751 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f913406d8a0 0x7f913406fd50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f91440059d0 tx=0x7f9144005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:38:28.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:28.842+0000 7f913e7fc700 1 -- 172.21.15.22:0/3051518751 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9138056cf0 con 0x7f91480ff3b0 2024-02-16T19:38:28.846 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:28 smithi117 ceph-mon[67870]: from='client.34412 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:38:28.846 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:28 smithi117 ceph-mon[67870]: from='client.34416 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:38:28.846 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:28 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:38:28.846 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:28 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:38:28.846 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:28 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:38:28.846 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:28 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:38:28.846 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:28 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:38:28.846 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:28 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:38:28.846 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:28 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:38:28.846 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:28 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:38:28.846 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:28 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:38:28.846 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:28 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:38:28.846 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:28 smithi117 ceph-mon[67870]: from='client.? 172.21.15.22:0/4115378113' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:38:29.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:29.068+0000 7f914f678700 1 -- 172.21.15.22:0/3051518751 --> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f914819b340 con 0x7f913406d8a0 2024-02-16T19:38:29.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:29.069+0000 7f913e7fc700 1 -- 172.21.15.22:0/3051518751 <== mgr.34104 v2:172.21.15.22:6800/505266753 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f914819b340 con 0x7f913406d8a0 2024-02-16T19:38:29.071 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-02-16T19:38:29.071 INFO:teuthology.orchestra.run.smithi022.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T19:38:29.071 INFO:teuthology.orchestra.run.smithi022.stdout: "in_progress": true, 2024-02-16T19:38:29.071 INFO:teuthology.orchestra.run.smithi022.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T19:38:29.071 INFO:teuthology.orchestra.run.smithi022.stdout: "services_complete": [ 2024-02-16T19:38:29.071 INFO:teuthology.orchestra.run.smithi022.stdout: "crash", 2024-02-16T19:38:29.071 INFO:teuthology.orchestra.run.smithi022.stdout: "osd", 2024-02-16T19:38:29.071 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr", 2024-02-16T19:38:29.072 INFO:teuthology.orchestra.run.smithi022.stdout: "mon" 2024-02-16T19:38:29.072 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-02-16T19:38:29.072 INFO:teuthology.orchestra.run.smithi022.stdout: "progress": "14/23 daemons upgraded", 2024-02-16T19:38:29.072 INFO:teuthology.orchestra.run.smithi022.stdout: "message": "Currently upgrading mds daemons", 2024-02-16T19:38:29.072 INFO:teuthology.orchestra.run.smithi022.stdout: "is_paused": false 2024-02-16T19:38:29.072 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-02-16T19:38:29.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:29.072+0000 7f914f678700 1 -- 172.21.15.22:0/3051518751 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f913406d8a0 msgr2=0x7f913406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:38:29.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:29.072+0000 7f914f678700 1 --2- 172.21.15.22:0/3051518751 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f913406d8a0 0x7f913406fd50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f91440059d0 tx=0x7f9144005960 comp rx=0 tx=0).stop 2024-02-16T19:38:29.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:29.072+0000 7f914f678700 1 -- 172.21.15.22:0/3051518751 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91480ff3b0 msgr2=0x7f9148195e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:38:29.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:29.072+0000 7f914f678700 1 --2- 172.21.15.22:0/3051518751 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91480ff3b0 0x7f9148195e80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f913800d8d0 tx=0x7f913800dc90 comp rx=0 tx=0).stop 2024-02-16T19:38:29.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:29.072+0000 7f914f678700 1 -- 172.21.15.22:0/3051518751 shutdown_connections 2024-02-16T19:38:29.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:29.072+0000 7f914f678700 1 --2- 172.21.15.22:0/3051518751 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f913406d8a0 0x7f913406fd50 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:38:29.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:29.072+0000 7f914f678700 1 --2- 172.21.15.22:0/3051518751 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91480ff3b0 0x7f9148195e80 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:38:29.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:29.072+0000 7f914f678700 1 --2- 172.21.15.22:0/3051518751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91480ffd50 0x7f91481963c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:38:29.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:29.072+0000 7f914f678700 1 -- 172.21.15.22:0/3051518751 >> 172.21.15.22:0/3051518751 conn(0x7f91480fac90 msgr2=0x7f9148069110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:38:29.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:29.072+0000 7f914f678700 1 -- 172.21.15.22:0/3051518751 shutdown_connections 2024-02-16T19:38:29.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:29.072+0000 7f914f678700 1 -- 172.21.15.22:0/3051518751 wait complete. 2024-02-16T19:38:29.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:29.178+0000 7f85a15b9700 1 -- 172.21.15.22:0/4241962115 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f859c1004c0 msgr2=0x7f859c100890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:38:29.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:29.178+0000 7f85a15b9700 1 --2- 172.21.15.22:0/4241962115 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f859c1004c0 0x7f859c100890 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f858c009910 tx=0x7f858c009c20 comp rx=0 tx=0).stop 2024-02-16T19:38:29.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:29.178+0000 7f85a15b9700 1 -- 172.21.15.22:0/4241962115 shutdown_connections 2024-02-16T19:38:29.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:29.178+0000 7f85a15b9700 1 --2- 172.21.15.22:0/4241962115 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f859c100e60 0x7f859c104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:38:29.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:29.178+0000 7f85a15b9700 1 --2- 172.21.15.22:0/4241962115 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f859c1004c0 0x7f859c100890 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:38:29.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:29.178+0000 7f85a15b9700 1 -- 172.21.15.22:0/4241962115 >> 172.21.15.22:0/4241962115 conn(0x7f859c0fbd80 msgr2=0x7f859c0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:38:29.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:29.178+0000 7f85a15b9700 1 -- 172.21.15.22:0/4241962115 shutdown_connections 2024-02-16T19:38:29.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:29.179+0000 7f85a15b9700 1 -- 172.21.15.22:0/4241962115 wait complete. 2024-02-16T19:38:29.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:29.179+0000 7f85a15b9700 1 Processor -- start 2024-02-16T19:38:29.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:29.179+0000 7f85a15b9700 1 -- start start 2024-02-16T19:38:29.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:29.179+0000 7f85a15b9700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f859c1004c0 0x7f859c195e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:38:29.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:29.179+0000 7f85a15b9700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f859c100e60 0x7f859c1963d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:38:29.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:29.179+0000 7f85a15b9700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f859c196ab0 con 0x7f859c100e60 2024-02-16T19:38:29.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:29.179+0000 7f85a15b9700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f859c19a840 con 0x7f859c1004c0 2024-02-16T19:38:29.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:29.180+0000 7f859a7fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f859c100e60 0x7f859c1963d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:38:29.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:29.180+0000 7f859a7fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f859c100e60 0x7f859c1963d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33190/0 (socket says 172.21.15.22:33190) 2024-02-16T19:38:29.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:29.180+0000 7f859a7fc700 1 -- 172.21.15.22:0/3481668652 learned_addr learned my addr 172.21.15.22:0/3481668652 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:38:29.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:29.180+0000 7f859affd700 1 --2- 172.21.15.22:0/3481668652 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f859c1004c0 0x7f859c195e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:38:29.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:29.180+0000 7f859a7fc700 1 -- 172.21.15.22:0/3481668652 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f859c1004c0 msgr2=0x7f859c195e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:38:29.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:29.180+0000 7f859a7fc700 1 --2- 172.21.15.22:0/3481668652 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f859c1004c0 0x7f859c195e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:38:29.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:29.180+0000 7f859a7fc700 1 -- 172.21.15.22:0/3481668652 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f858c0095f0 con 0x7f859c100e60 2024-02-16T19:38:29.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:29.180+0000 7f859affd700 1 --2- 172.21.15.22:0/3481668652 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f859c1004c0 0x7f859c195e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T19:38:29.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:29.180+0000 7f859a7fc700 1 --2- 172.21.15.22:0/3481668652 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f859c100e60 0x7f859c1963d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f859000c8f0 tx=0x7f859000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:38:29.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:29.180+0000 7f8583fff700 1 -- 172.21.15.22:0/3481668652 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8590007a50 con 0x7f859c100e60 2024-02-16T19:38:29.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:29.180+0000 7f85a15b9700 1 -- 172.21.15.22:0/3481668652 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f859c19ab20 con 0x7f859c100e60 2024-02-16T19:38:29.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:29.180+0000 7f8583fff700 1 -- 172.21.15.22:0/3481668652 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8590007bb0 con 0x7f859c100e60 2024-02-16T19:38:29.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:29.180+0000 7f8583fff700 1 -- 172.21.15.22:0/3481668652 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8590018630 con 0x7f859c100e60 2024-02-16T19:38:29.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:29.180+0000 7f85a15b9700 1 -- 172.21.15.22:0/3481668652 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f859c19b070 con 0x7f859c100e60 2024-02-16T19:38:29.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:29.181+0000 7f85a15b9700 1 -- 172.21.15.22:0/3481668652 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f859c1087d0 con 0x7f859c100e60 2024-02-16T19:38:29.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:29.182+0000 7f8583fff700 1 -- 172.21.15.22:0/3481668652 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f8590007d20 con 0x7f859c100e60 2024-02-16T19:38:29.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:29.182+0000 7f8583fff700 1 --2- 172.21.15.22:0/3481668652 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f858406d8a0 0x7f858406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:38:29.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:29.182+0000 7f8583fff700 1 -- 172.21.15.22:0/3481668652 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6790+0+0 (secure 0 0 0) 0x7f859008d680 con 0x7f859c100e60 2024-02-16T19:38:29.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:29.182+0000 7f859affd700 1 --2- 172.21.15.22:0/3481668652 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f858406d8a0 0x7f858406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:38:29.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:29.183+0000 7f859affd700 1 --2- 172.21.15.22:0/3481668652 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f858406d8a0 0x7f858406fd50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f858c005100 tx=0x7f858c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:38:29.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:29.187+0000 7f8583fff700 1 -- 172.21.15.22:0/3481668652 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8590056cc0 con 0x7f859c100e60 2024-02-16T19:38:29.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:29.483+0000 7f85a15b9700 1 -- 172.21.15.22:0/3481668652 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f859c19b2f0 con 0x7f859c100e60 2024-02-16T19:38:29.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:29.484+0000 7f8583fff700 1 -- 172.21.15.22:0/3481668652 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f859005a380 con 0x7f859c100e60 2024-02-16T19:38:29.485 INFO:teuthology.orchestra.run.smithi022.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-02-16T19:38:29.486 INFO:teuthology.orchestra.run.smithi022.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-02-16T19:38:29.486 INFO:teuthology.orchestra.run.smithi022.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-02-16T19:38:29.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:29.486+0000 7f85a15b9700 1 -- 172.21.15.22:0/3481668652 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f858406d8a0 msgr2=0x7f858406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:38:29.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:29.486+0000 7f85a15b9700 1 --2- 172.21.15.22:0/3481668652 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f858406d8a0 0x7f858406fd50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f858c005100 tx=0x7f858c01a040 comp rx=0 tx=0).stop 2024-02-16T19:38:29.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:29.486+0000 7f85a15b9700 1 -- 172.21.15.22:0/3481668652 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f859c100e60 msgr2=0x7f859c1963d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:38:29.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:29.486+0000 7f85a15b9700 1 --2- 172.21.15.22:0/3481668652 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f859c100e60 0x7f859c1963d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f859000c8f0 tx=0x7f859000ccb0 comp rx=0 tx=0).stop 2024-02-16T19:38:29.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:29.486+0000 7f85a15b9700 1 -- 172.21.15.22:0/3481668652 shutdown_connections 2024-02-16T19:38:29.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:29.486+0000 7f85a15b9700 1 --2- 172.21.15.22:0/3481668652 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f858406d8a0 0x7f858406fd50 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:38:29.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:29.486+0000 7f85a15b9700 1 --2- 172.21.15.22:0/3481668652 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f859c1004c0 0x7f859c195e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:38:29.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:29.486+0000 7f85a15b9700 1 --2- 172.21.15.22:0/3481668652 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f859c100e60 0x7f859c1963d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:38:29.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:29.486+0000 7f85a15b9700 1 -- 172.21.15.22:0/3481668652 >> 172.21.15.22:0/3481668652 conn(0x7f859c0fbd80 msgr2=0x7f859c0fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:38:29.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:29.487+0000 7f85a15b9700 1 -- 172.21.15.22:0/3481668652 shutdown_connections 2024-02-16T19:38:29.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:29.487+0000 7f85a15b9700 1 -- 172.21.15.22:0/3481668652 wait complete. 2024-02-16T19:38:29.856 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:29 smithi022 ceph-mon[88798]: pgmap v200: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 6.5 MiB/s rd, 682 B/s wr, 5 op/s 2024-02-16T19:38:29.856 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:29 smithi022 ceph-mon[88798]: from='client.34420 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:38:29.856 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:29 smithi022 ceph-mon[88798]: from='client.? 172.21.15.22:0/4294520889' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:38:29.856 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:29 smithi022 ceph-mon[88798]: from='client.? 172.21.15.22:0/3481668652' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T19:38:29.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:29 smithi117 ceph-mon[67870]: pgmap v200: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 6.5 MiB/s rd, 682 B/s wr, 5 op/s 2024-02-16T19:38:29.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:29 smithi117 ceph-mon[67870]: from='client.34420 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:38:29.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:29 smithi117 ceph-mon[67870]: from='client.? 172.21.15.22:0/4294520889' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:38:29.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:29 smithi117 ceph-mon[67870]: from='client.? 172.21.15.22:0/3481668652' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T19:38:30.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:30 smithi117 ceph-mon[67870]: from='client.34432 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:38:30.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:30 smithi117 ceph-mon[67870]: Upgrade: Updating mds.cephfs.smithi117.ibmppn 2024-02-16T19:38:30.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:30 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:38:30.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:30 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi117.ibmppn", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T19:38:30.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:30 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:38:30.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:30 smithi117 ceph-mon[67870]: Deploying daemon mds.cephfs.smithi117.ibmppn on smithi117 2024-02-16T19:38:30.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:30 smithi022 ceph-mon[88798]: from='client.34432 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:38:30.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:30 smithi022 ceph-mon[88798]: Upgrade: Updating mds.cephfs.smithi117.ibmppn 2024-02-16T19:38:30.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:30 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:38:30.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:30 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi117.ibmppn", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T19:38:30.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:30 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:38:30.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:30 smithi022 ceph-mon[88798]: Deploying daemon mds.cephfs.smithi117.ibmppn on smithi117 2024-02-16T19:38:31.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:31 smithi117 ceph-mon[67870]: pgmap v201: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 4.2 MiB/s rd, 5.2 KiB/s wr, 7 op/s 2024-02-16T19:38:31.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:31 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:38:31.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:31 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:38:31.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:31 smithi022 ceph-mon[88798]: pgmap v201: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 4.2 MiB/s rd, 5.2 KiB/s wr, 7 op/s 2024-02-16T19:38:31.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:31 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:38:31.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:31 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:38:31.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:31 smithi022 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-mon-smithi022[88794]: 2024-02-16T19:38:31.770+0000 7fe662394700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-16T19:38:32.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:32 smithi022 ceph-mon[88798]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-02-16T19:38:32.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:32 smithi022 ceph-mon[88798]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-16T19:38:32.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:32 smithi022 ceph-mon[88798]: osdmap e92: 6 total, 6 up, 6 in 2024-02-16T19:38:32.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:32 smithi022 ceph-mon[88798]: Standby daemon mds.cephfs.smithi117.orzors assigned to filesystem cephfs as rank 0 2024-02-16T19:38:32.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:32 smithi022 ceph-mon[88798]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-16T19:38:32.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:32 smithi022 ceph-mon[88798]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-02-16T19:38:32.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:32 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi117.ibmppn"}]: dispatch 2024-02-16T19:38:32.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:32 smithi022 ceph-mon[88798]: fsmap cephfs:1/1 {0=cephfs.smithi117.orzors=up:replay} 2 up:standby 2024-02-16T19:38:33.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:32 smithi117 ceph-mon[67870]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-02-16T19:38:33.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:32 smithi117 ceph-mon[67870]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-16T19:38:33.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:32 smithi117 ceph-mon[67870]: osdmap e92: 6 total, 6 up, 6 in 2024-02-16T19:38:33.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:32 smithi117 ceph-mon[67870]: Standby daemon mds.cephfs.smithi117.orzors assigned to filesystem cephfs as rank 0 2024-02-16T19:38:33.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:32 smithi117 ceph-mon[67870]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-16T19:38:33.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:32 smithi117 ceph-mon[67870]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-02-16T19:38:33.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:32 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi117.ibmppn"}]: dispatch 2024-02-16T19:38:33.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:32 smithi117 ceph-mon[67870]: fsmap cephfs:1/1 {0=cephfs.smithi117.orzors=up:replay} 2 up:standby 2024-02-16T19:38:34.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:33 smithi117 ceph-mon[67870]: pgmap v203: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.6 KiB/s rd, 6.0 KiB/s wr, 5 op/s 2024-02-16T19:38:34.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:33 smithi022 ceph-mon[88798]: pgmap v203: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.6 KiB/s rd, 6.0 KiB/s wr, 5 op/s 2024-02-16T19:38:36.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:35 smithi117 ceph-mon[67870]: pgmap v204: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 3.2 MiB/s rd, 5.9 KiB/s wr, 5 op/s 2024-02-16T19:38:36.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:35 smithi022 ceph-mon[88798]: pgmap v204: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 3.2 MiB/s rd, 5.9 KiB/s wr, 5 op/s 2024-02-16T19:38:38.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:37 smithi117 ceph-mon[67870]: pgmap v205: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 11 MiB/s rd, 5.5 KiB/s wr, 6 op/s 2024-02-16T19:38:38.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:37 smithi022 ceph-mon[88798]: pgmap v205: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 11 MiB/s rd, 5.5 KiB/s wr, 6 op/s 2024-02-16T19:38:40.097 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:39 smithi117 ceph-mon[67870]: pgmap v206: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 14 MiB/s rd, 5.5 KiB/s wr, 7 op/s 2024-02-16T19:38:40.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:39 smithi022 ceph-mon[88798]: pgmap v206: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 14 MiB/s rd, 5.5 KiB/s wr, 7 op/s 2024-02-16T19:38:42.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:41 smithi117 ceph-mon[67870]: pgmap v207: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 19 MiB/s rd, 5 op/s 2024-02-16T19:38:42.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:41 smithi022 ceph-mon[88798]: pgmap v207: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 19 MiB/s rd, 5 op/s 2024-02-16T19:38:43.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:42 smithi117 ceph-mon[67870]: reconnect by client.24307 192.168.0.1:0/2099531446 after 0.000999979 2024-02-16T19:38:43.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:42 smithi117 ceph-mon[67870]: reconnect by client.14540 192.168.0.1:0/1593109629 after 0.000999979 2024-02-16T19:38:43.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:42 smithi117 ceph-mon[67870]: mds.? [v2:172.21.15.117:6826/3122342028,v1:172.21.15.117:6827/3122342028] up:reconnect 2024-02-16T19:38:43.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:42 smithi117 ceph-mon[67870]: fsmap cephfs:1/1 {0=cephfs.smithi117.orzors=up:reconnect} 2 up:standby 2024-02-16T19:38:43.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:42 smithi117 ceph-mon[67870]: pgmap v208: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 25 MiB/s rd, 7 op/s 2024-02-16T19:38:43.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:42 smithi022 ceph-mon[88798]: reconnect by client.24307 192.168.0.1:0/2099531446 after 0.000999979 2024-02-16T19:38:43.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:42 smithi022 ceph-mon[88798]: reconnect by client.14540 192.168.0.1:0/1593109629 after 0.000999979 2024-02-16T19:38:43.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:42 smithi022 ceph-mon[88798]: mds.? [v2:172.21.15.117:6826/3122342028,v1:172.21.15.117:6827/3122342028] up:reconnect 2024-02-16T19:38:43.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:42 smithi022 ceph-mon[88798]: fsmap cephfs:1/1 {0=cephfs.smithi117.orzors=up:reconnect} 2 up:standby 2024-02-16T19:38:43.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:42 smithi022 ceph-mon[88798]: pgmap v208: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 25 MiB/s rd, 7 op/s 2024-02-16T19:38:44.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:43 smithi117 ceph-mon[67870]: mds.? [v2:172.21.15.117:6826/3122342028,v1:172.21.15.117:6827/3122342028] up:rejoin 2024-02-16T19:38:44.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:43 smithi117 ceph-mon[67870]: fsmap cephfs:1/1 {0=cephfs.smithi117.orzors=up:rejoin} 2 up:standby 2024-02-16T19:38:44.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:43 smithi117 ceph-mon[67870]: daemon mds.cephfs.smithi117.orzors is now active in filesystem cephfs as rank 0 2024-02-16T19:38:44.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:43 smithi022 ceph-mon[88798]: mds.? [v2:172.21.15.117:6826/3122342028,v1:172.21.15.117:6827/3122342028] up:rejoin 2024-02-16T19:38:44.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:43 smithi022 ceph-mon[88798]: fsmap cephfs:1/1 {0=cephfs.smithi117.orzors=up:rejoin} 2 up:standby 2024-02-16T19:38:44.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:43 smithi022 ceph-mon[88798]: daemon mds.cephfs.smithi117.orzors is now active in filesystem cephfs as rank 0 2024-02-16T19:38:45.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:44 smithi117 ceph-mon[67870]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-02-16T19:38:45.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:44 smithi117 ceph-mon[67870]: mds.? [v2:172.21.15.117:6826/3122342028,v1:172.21.15.117:6827/3122342028] up:active 2024-02-16T19:38:45.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:44 smithi117 ceph-mon[67870]: fsmap cephfs:1 {0=cephfs.smithi117.orzors=up:active} 2 up:standby 2024-02-16T19:38:45.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:44 smithi117 ceph-mon[67870]: pgmap v209: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 85 B/s wr, 6 op/s 2024-02-16T19:38:45.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:44 smithi022 ceph-mon[88798]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-02-16T19:38:45.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:44 smithi022 ceph-mon[88798]: mds.? [v2:172.21.15.117:6826/3122342028,v1:172.21.15.117:6827/3122342028] up:active 2024-02-16T19:38:45.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:44 smithi022 ceph-mon[88798]: fsmap cephfs:1 {0=cephfs.smithi117.orzors=up:active} 2 up:standby 2024-02-16T19:38:45.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:44 smithi022 ceph-mon[88798]: pgmap v209: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 85 B/s wr, 6 op/s 2024-02-16T19:38:46.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:45 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:38:46.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:45 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:38:47.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:47 smithi022 ceph-mon[88798]: pgmap v210: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 19 MiB/s rd, 170 B/s wr, 6 op/s 2024-02-16T19:38:47.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:47 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:38:47.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:47 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:38:47.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:47 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:38:47.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:47 smithi022 ceph-mon[88798]: mds.? [v2:172.21.15.117:6824/2389644474,v1:172.21.15.117:6825/2389644474] up:boot 2024-02-16T19:38:47.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:47 smithi022 ceph-mon[88798]: fsmap cephfs:1 {0=cephfs.smithi117.orzors=up:active} 3 up:standby 2024-02-16T19:38:47.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:47 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi117.ibmppn"}]: dispatch 2024-02-16T19:38:48.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:47 smithi117 ceph-mon[67870]: pgmap v210: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 19 MiB/s rd, 170 B/s wr, 6 op/s 2024-02-16T19:38:48.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:47 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:38:48.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:47 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:38:48.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:47 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:38:48.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:47 smithi117 ceph-mon[67870]: mds.? [v2:172.21.15.117:6824/2389644474,v1:172.21.15.117:6825/2389644474] up:boot 2024-02-16T19:38:48.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:47 smithi117 ceph-mon[67870]: fsmap cephfs:1 {0=cephfs.smithi117.orzors=up:active} 3 up:standby 2024-02-16T19:38:48.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:47 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi117.ibmppn"}]: dispatch 2024-02-16T19:38:49.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:49 smithi022 ceph-mon[88798]: pgmap v211: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 12 MiB/s rd, 170 B/s wr, 5 op/s 2024-02-16T19:38:49.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:49 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:38:49.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:49 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:38:50.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:49 smithi117 ceph-mon[67870]: pgmap v211: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 12 MiB/s rd, 170 B/s wr, 5 op/s 2024-02-16T19:38:50.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:49 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:38:50.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:49 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:38:51.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:51 smithi022 ceph-mon[88798]: pgmap v212: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 9.9 MiB/s rd, 4.3 KiB/s wr, 7 op/s 2024-02-16T19:38:51.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:51 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:38:51.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:51 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:38:52.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:51 smithi117 ceph-mon[67870]: pgmap v212: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 9.9 MiB/s rd, 4.3 KiB/s wr, 7 op/s 2024-02-16T19:38:52.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:51 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:38:52.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:51 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:38:53.977 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:53 smithi117 ceph-mon[67870]: pgmap v213: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 5.5 MiB/s rd, 4.3 KiB/s wr, 6 op/s 2024-02-16T19:38:54.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:53 smithi022 ceph-mon[88798]: pgmap v213: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 5.5 MiB/s rd, 4.3 KiB/s wr, 6 op/s 2024-02-16T19:38:55.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:55 smithi117 ceph-mon[67870]: pgmap v214: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 538 KiB/s rd, 4.3 KiB/s wr, 5 op/s 2024-02-16T19:38:55.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:55 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:38:55.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:55 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:38:55.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:55 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:38:55.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:55 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:38:55.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:55 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:38:55.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:55 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:38:55.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:55 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:38:55.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:55 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:38:55.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:55 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:38:55.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:55 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:38:55.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:55 smithi022 ceph-mon[88798]: pgmap v214: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 538 KiB/s rd, 4.3 KiB/s wr, 5 op/s 2024-02-16T19:38:55.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:55 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:38:55.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:55 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:38:55.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:55 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:38:55.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:55 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:38:55.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:55 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:38:55.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:55 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:38:55.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:55 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:38:55.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:55 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:38:55.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:55 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:38:55.466 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:55 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:38:56.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:56 smithi117 ceph-mon[67870]: Upgrade: Updating mds.cephfs.smithi117.orzors 2024-02-16T19:38:56.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:56 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:38:56.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:56 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi117.orzors", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T19:38:56.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:56 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:38:56.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:56 smithi117 ceph-mon[67870]: Deploying daemon mds.cephfs.smithi117.orzors on smithi117 2024-02-16T19:38:56.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:56 smithi022 ceph-mon[88798]: Upgrade: Updating mds.cephfs.smithi117.orzors 2024-02-16T19:38:56.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:56 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:38:56.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:56 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi117.orzors", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T19:38:56.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:56 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:38:56.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:56 smithi022 ceph-mon[88798]: Deploying daemon mds.cephfs.smithi117.orzors on smithi117 2024-02-16T19:38:57.804 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:57 smithi117 ceph-mon[67870]: pgmap v215: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 197 KiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-02-16T19:38:57.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:57 smithi022 ceph-mon[88798]: pgmap v215: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 197 KiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-02-16T19:38:57.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:57 smithi022 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-mon-smithi022[88794]: 2024-02-16T19:38:57.803+0000 7fe662394700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-16T19:38:58.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:58 smithi022 ceph-mon[88798]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-02-16T19:38:58.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:58 smithi022 ceph-mon[88798]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-16T19:38:58.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:58 smithi022 ceph-mon[88798]: osdmap e93: 6 total, 6 up, 6 in 2024-02-16T19:38:58.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:58 smithi022 ceph-mon[88798]: Standby daemon mds.cephfs.smithi022.ocukvs assigned to filesystem cephfs as rank 0 2024-02-16T19:38:58.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:58 smithi022 ceph-mon[88798]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-16T19:38:58.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:58 smithi022 ceph-mon[88798]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-02-16T19:38:58.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:58 smithi022 ceph-mon[88798]: fsmap cephfs:1/1 {0=cephfs.smithi022.ocukvs=up:replay} 2 up:standby 2024-02-16T19:38:59.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:58 smithi117 ceph-mon[67870]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-02-16T19:38:59.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:58 smithi117 ceph-mon[67870]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-16T19:38:59.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:58 smithi117 ceph-mon[67870]: osdmap e93: 6 total, 6 up, 6 in 2024-02-16T19:38:59.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:58 smithi117 ceph-mon[67870]: Standby daemon mds.cephfs.smithi022.ocukvs assigned to filesystem cephfs as rank 0 2024-02-16T19:38:59.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:58 smithi117 ceph-mon[67870]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-16T19:38:59.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:58 smithi117 ceph-mon[67870]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-02-16T19:38:59.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:58 smithi117 ceph-mon[67870]: fsmap cephfs:1/1 {0=cephfs.smithi022.ocukvs=up:replay} 2 up:standby 2024-02-16T19:38:59.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.597+0000 7f76fd4c4700 1 -- 172.21.15.22:0/1642055940 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76f8101d30 msgr2=0x7f76f810a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:38:59.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.597+0000 7f76fd4c4700 1 --2- 172.21.15.22:0/1642055940 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76f8101d30 0x7f76f810a230 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f76ec009910 tx=0x7f76ec009c20 comp rx=0 tx=0).stop 2024-02-16T19:38:59.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.597+0000 7f76fd4c4700 1 -- 172.21.15.22:0/1642055940 shutdown_connections 2024-02-16T19:38:59.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.597+0000 7f76fd4c4700 1 --2- 172.21.15.22:0/1642055940 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76f8101d30 0x7f76f810a230 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:38:59.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.597+0000 7f76fd4c4700 1 --2- 172.21.15.22:0/1642055940 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f76f8101420 0x7f76f81017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:38:59.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.597+0000 7f76fd4c4700 1 -- 172.21.15.22:0/1642055940 >> 172.21.15.22:0/1642055940 conn(0x7f76f80fac60 msgr2=0x7f76f80fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:38:59.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.597+0000 7f76fd4c4700 1 -- 172.21.15.22:0/1642055940 shutdown_connections 2024-02-16T19:38:59.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.597+0000 7f76fd4c4700 1 -- 172.21.15.22:0/1642055940 wait complete. 2024-02-16T19:38:59.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.598+0000 7f76fd4c4700 1 Processor -- start 2024-02-16T19:38:59.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.598+0000 7f76fd4c4700 1 -- start start 2024-02-16T19:38:59.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.598+0000 7f76fd4c4700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f76f8101420 0x7f76f80ff080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:38:59.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.598+0000 7f76fd4c4700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76f8101d30 0x7f76f80ff5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:38:59.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.599+0000 7f76fd4c4700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f8100e30 con 0x7f76f8101d30 2024-02-16T19:38:59.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.599+0000 7f76fd4c4700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f8100fa0 con 0x7f76f8101420 2024-02-16T19:38:59.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.603+0000 7f76f67fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76f8101d30 0x7f76f80ff5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:38:59.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.603+0000 7f76f67fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76f8101d30 0x7f76f80ff5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:56722/0 (socket says 172.21.15.22:56722) 2024-02-16T19:38:59.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.603+0000 7f76f67fc700 1 -- 172.21.15.22:0/1806876927 learned_addr learned my addr 172.21.15.22:0/1806876927 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:38:59.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.603+0000 7f76f6ffd700 1 --2- 172.21.15.22:0/1806876927 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f76f8101420 0x7f76f80ff080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:38:59.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.604+0000 7f76f67fc700 1 -- 172.21.15.22:0/1806876927 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f76f8101420 msgr2=0x7f76f80ff080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:38:59.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.604+0000 7f76f67fc700 1 --2- 172.21.15.22:0/1806876927 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f76f8101420 0x7f76f80ff080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:38:59.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.604+0000 7f76f67fc700 1 -- 172.21.15.22:0/1806876927 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76ec0095f0 con 0x7f76f8101d30 2024-02-16T19:38:59.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.605+0000 7f76f67fc700 1 --2- 172.21.15.22:0/1806876927 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76f8101d30 0x7f76f80ff5c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f76ec0092e0 tx=0x7f76ec00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:38:59.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.605+0000 7f76dffff700 1 -- 172.21.15.22:0/1806876927 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f76ec01d070 con 0x7f76f8101d30 2024-02-16T19:38:59.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.605+0000 7f76dffff700 1 -- 172.21.15.22:0/1806876927 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f76ec005e00 con 0x7f76f8101d30 2024-02-16T19:38:59.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.605+0000 7f76dffff700 1 -- 172.21.15.22:0/1806876927 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f76ec017400 con 0x7f76f8101d30 2024-02-16T19:38:59.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.605+0000 7f76fd4c4700 1 -- 172.21.15.22:0/1806876927 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76f80ffb60 con 0x7f76f8101d30 2024-02-16T19:38:59.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.605+0000 7f76fd4c4700 1 -- 172.21.15.22:0/1806876927 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76f8199fc0 con 0x7f76f8101d30 2024-02-16T19:38:59.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.606+0000 7f76fd4c4700 1 -- 172.21.15.22:0/1806876927 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76f81078a0 con 0x7f76f8101d30 2024-02-16T19:38:59.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.608+0000 7f76dffff700 1 -- 172.21.15.22:0/1806876927 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f76ec00f6f0 con 0x7f76f8101d30 2024-02-16T19:38:59.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.608+0000 7f76dffff700 1 --2- 172.21.15.22:0/1806876927 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f76e006d8a0 0x7f76e006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:38:59.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.608+0000 7f76dffff700 1 -- 172.21.15.22:0/1806876927 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f76ec08ec40 con 0x7f76f8101d30 2024-02-16T19:38:59.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.608+0000 7f76f6ffd700 1 --2- 172.21.15.22:0/1806876927 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f76e006d8a0 0x7f76e006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:38:59.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.609+0000 7f76f6ffd700 1 --2- 172.21.15.22:0/1806876927 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f76e006d8a0 0x7f76e006fd50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f76e8005950 tx=0x7f76e800b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:38:59.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.612+0000 7f76dffff700 1 -- 172.21.15.22:0/1806876927 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f76ec058390 con 0x7f76f8101d30 2024-02-16T19:38:59.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.841+0000 7f76fd4c4700 1 -- 172.21.15.22:0/1806876927 --> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f76f8061190 con 0x7f76e006d8a0 2024-02-16T19:38:59.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.843+0000 7f76dffff700 1 -- 172.21.15.22:0/1806876927 <== mgr.34104 v2:172.21.15.22:6800/505266753 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f76f8061190 con 0x7f76e006d8a0 2024-02-16T19:38:59.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.846+0000 7f76fd4c4700 1 -- 172.21.15.22:0/1806876927 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f76e006d8a0 msgr2=0x7f76e006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:38:59.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.846+0000 7f76fd4c4700 1 --2- 172.21.15.22:0/1806876927 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f76e006d8a0 0x7f76e006fd50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f76e8005950 tx=0x7f76e800b410 comp rx=0 tx=0).stop 2024-02-16T19:38:59.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.846+0000 7f76fd4c4700 1 -- 172.21.15.22:0/1806876927 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76f8101d30 msgr2=0x7f76f80ff5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:38:59.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.846+0000 7f76fd4c4700 1 --2- 172.21.15.22:0/1806876927 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76f8101d30 0x7f76f80ff5c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f76ec0092e0 tx=0x7f76ec00bd70 comp rx=0 tx=0).stop 2024-02-16T19:38:59.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.846+0000 7f76fd4c4700 1 -- 172.21.15.22:0/1806876927 shutdown_connections 2024-02-16T19:38:59.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.846+0000 7f76fd4c4700 1 --2- 172.21.15.22:0/1806876927 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f76e006d8a0 0x7f76e006fd50 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:38:59.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.846+0000 7f76fd4c4700 1 --2- 172.21.15.22:0/1806876927 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f76f8101420 0x7f76f80ff080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:38:59.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.846+0000 7f76fd4c4700 1 --2- 172.21.15.22:0/1806876927 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76f8101d30 0x7f76f80ff5c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:38:59.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.846+0000 7f76fd4c4700 1 -- 172.21.15.22:0/1806876927 >> 172.21.15.22:0/1806876927 conn(0x7f76f80fac60 msgr2=0x7f76f81049e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:38:59.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.846+0000 7f76fd4c4700 1 -- 172.21.15.22:0/1806876927 shutdown_connections 2024-02-16T19:38:59.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.846+0000 7f76fd4c4700 1 -- 172.21.15.22:0/1806876927 wait complete. 2024-02-16T19:38:59.855 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:38:59 smithi022 ceph-mon[88798]: pgmap v217: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 921 B/s rd, 5.0 KiB/s wr, 3 op/s 2024-02-16T19:38:59.862 INFO:teuthology.orchestra.run.smithi022.stdout:true 2024-02-16T19:38:59.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.958+0000 7f96550d2700 1 -- 172.21.15.22:0/391559653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9650106320 msgr2=0x7f96501066f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:38:59.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.958+0000 7f96550d2700 1 --2- 172.21.15.22:0/391559653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9650106320 0x7f96501066f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9640009910 tx=0x7f9640009c20 comp rx=0 tx=0).stop 2024-02-16T19:38:59.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.958+0000 7f96550d2700 1 -- 172.21.15.22:0/391559653 shutdown_connections 2024-02-16T19:38:59.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.958+0000 7f96550d2700 1 --2- 172.21.15.22:0/391559653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9650068540 0x7f96500689b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:38:59.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.958+0000 7f96550d2700 1 --2- 172.21.15.22:0/391559653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9650106320 0x7f96501066f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:38:59.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.958+0000 7f96550d2700 1 -- 172.21.15.22:0/391559653 >> 172.21.15.22:0/391559653 conn(0x7f9650075130 msgr2=0x7f9650075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:38:59.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.958+0000 7f96550d2700 1 -- 172.21.15.22:0/391559653 shutdown_connections 2024-02-16T19:38:59.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.958+0000 7f96550d2700 1 -- 172.21.15.22:0/391559653 wait complete. 2024-02-16T19:38:59.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.959+0000 7f96550d2700 1 Processor -- start 2024-02-16T19:38:59.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.959+0000 7f96550d2700 1 -- start start 2024-02-16T19:38:59.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.960+0000 7f96550d2700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9650068540 0x7f96500ffca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:38:59.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.960+0000 7f96550d2700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9650106320 0x7f96501001e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:38:59.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.960+0000 7f96550d2700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9650100720 con 0x7f9650106320 2024-02-16T19:38:59.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.960+0000 7f96550d2700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9650100860 con 0x7f9650068540 2024-02-16T19:38:59.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.960+0000 7f964e59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9650106320 0x7f96501001e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:38:59.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.960+0000 7f964e59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9650106320 0x7f96501001e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:56732/0 (socket says 172.21.15.22:56732) 2024-02-16T19:38:59.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.960+0000 7f964e59c700 1 -- 172.21.15.22:0/397640744 learned_addr learned my addr 172.21.15.22:0/397640744 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:38:59.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.960+0000 7f964ed9d700 1 --2- 172.21.15.22:0/397640744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9650068540 0x7f96500ffca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:38:59.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.960+0000 7f964e59c700 1 -- 172.21.15.22:0/397640744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9650068540 msgr2=0x7f96500ffca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:38:59.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.960+0000 7f964e59c700 1 --2- 172.21.15.22:0/397640744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9650068540 0x7f96500ffca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:38:59.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.960+0000 7f964e59c700 1 -- 172.21.15.22:0/397640744 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96400095f0 con 0x7f9650106320 2024-02-16T19:38:59.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.960+0000 7f964e59c700 1 --2- 172.21.15.22:0/397640744 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9650106320 0x7f96501001e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f964400b4b0 tx=0x7f964400b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:38:59.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.960+0000 7f9637fff700 1 -- 172.21.15.22:0/397640744 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9644011800 con 0x7f9650106320 2024-02-16T19:38:59.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.961+0000 7f96550d2700 1 -- 172.21.15.22:0/397640744 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96501a2160 con 0x7f9650106320 2024-02-16T19:38:59.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.961+0000 7f9637fff700 1 -- 172.21.15.22:0/397640744 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9644011e20 con 0x7f9650106320 2024-02-16T19:38:59.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.961+0000 7f96550d2700 1 -- 172.21.15.22:0/397640744 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96501a2570 con 0x7f9650106320 2024-02-16T19:38:59.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.962+0000 7f9637fff700 1 -- 172.21.15.22:0/397640744 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f964400f2e0 con 0x7f9650106320 2024-02-16T19:38:59.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.962+0000 7f96550d2700 1 -- 172.21.15.22:0/397640744 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f963c005320 con 0x7f9650106320 2024-02-16T19:38:59.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.963+0000 7f9637fff700 1 -- 172.21.15.22:0/397640744 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f9644010380 con 0x7f9650106320 2024-02-16T19:38:59.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.963+0000 7f9637fff700 1 --2- 172.21.15.22:0/397640744 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f963806d8a0 0x7f963806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:38:59.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.964+0000 7f9637fff700 1 -- 172.21.15.22:0/397640744 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f964408d0b0 con 0x7f9650106320 2024-02-16T19:38:59.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.964+0000 7f964ed9d700 1 --2- 172.21.15.22:0/397640744 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f963806d8a0 0x7f963806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:38:59.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.964+0000 7f964ed9d700 1 --2- 172.21.15.22:0/397640744 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f963806d8a0 0x7f963806fd50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9640006010 tx=0x7f964001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:38:59.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:38:59.968+0000 7f9637fff700 1 -- 172.21.15.22:0/397640744 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9644056760 con 0x7f9650106320 2024-02-16T19:39:00.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:38:59 smithi117 ceph-mon[67870]: pgmap v217: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 921 B/s rd, 5.0 KiB/s wr, 3 op/s 2024-02-16T19:39:00.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.240+0000 7f96550d2700 1 -- 172.21.15.22:0/397640744 --> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f963c000bf0 con 0x7f963806d8a0 2024-02-16T19:39:00.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.241+0000 7f9637fff700 1 -- 172.21.15.22:0/397640744 <== mgr.34104 v2:172.21.15.22:6800/505266753 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f963c000bf0 con 0x7f963806d8a0 2024-02-16T19:39:00.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.244+0000 7f96550d2700 1 -- 172.21.15.22:0/397640744 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f963806d8a0 msgr2=0x7f963806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:39:00.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.244+0000 7f96550d2700 1 --2- 172.21.15.22:0/397640744 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f963806d8a0 0x7f963806fd50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9640006010 tx=0x7f964001a040 comp rx=0 tx=0).stop 2024-02-16T19:39:00.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.244+0000 7f96550d2700 1 -- 172.21.15.22:0/397640744 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9650106320 msgr2=0x7f96501001e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:39:00.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.244+0000 7f96550d2700 1 --2- 172.21.15.22:0/397640744 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9650106320 0x7f96501001e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f964400b4b0 tx=0x7f964400b7c0 comp rx=0 tx=0).stop 2024-02-16T19:39:00.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.244+0000 7f96550d2700 1 -- 172.21.15.22:0/397640744 shutdown_connections 2024-02-16T19:39:00.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.244+0000 7f96550d2700 1 --2- 172.21.15.22:0/397640744 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f963806d8a0 0x7f963806fd50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:39:00.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.244+0000 7f96550d2700 1 --2- 172.21.15.22:0/397640744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9650068540 0x7f96500ffca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:39:00.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.244+0000 7f96550d2700 1 --2- 172.21.15.22:0/397640744 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9650106320 0x7f96501001e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:39:00.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.244+0000 7f96550d2700 1 -- 172.21.15.22:0/397640744 >> 172.21.15.22:0/397640744 conn(0x7f9650075130 msgr2=0x7f96500febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:39:00.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.244+0000 7f96550d2700 1 -- 172.21.15.22:0/397640744 shutdown_connections 2024-02-16T19:39:00.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.244+0000 7f96550d2700 1 -- 172.21.15.22:0/397640744 wait complete. 2024-02-16T19:39:00.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.352+0000 7f4678687700 1 -- 172.21.15.22:0/1825260670 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46700687a0 msgr2=0x7f4670068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:39:00.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.352+0000 7f4678687700 1 --2- 172.21.15.22:0/1825260670 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46700687a0 0x7f4670068b70 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f466c009910 tx=0x7f466c009c20 comp rx=0 tx=0).stop 2024-02-16T19:39:00.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.352+0000 7f4678687700 1 -- 172.21.15.22:0/1825260670 shutdown_connections 2024-02-16T19:39:00.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.352+0000 7f4678687700 1 --2- 172.21.15.22:0/1825260670 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4670069140 0x7f4670104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:39:00.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.352+0000 7f4678687700 1 --2- 172.21.15.22:0/1825260670 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46700687a0 0x7f4670068b70 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:39:00.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.352+0000 7f4678687700 1 -- 172.21.15.22:0/1825260670 >> 172.21.15.22:0/1825260670 conn(0x7f4670075130 msgr2=0x7f4670075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:39:00.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.352+0000 7f4678687700 1 -- 172.21.15.22:0/1825260670 shutdown_connections 2024-02-16T19:39:00.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.352+0000 7f4678687700 1 -- 172.21.15.22:0/1825260670 wait complete. 2024-02-16T19:39:00.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.352+0000 7f4678687700 1 Processor -- start 2024-02-16T19:39:00.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.353+0000 7f4678687700 1 -- start start 2024-02-16T19:39:00.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.353+0000 7f4678687700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46700687a0 0x7f4670195ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:39:00.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.353+0000 7f4678687700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4670069140 0x7f4670196420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:39:00.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.353+0000 7f4678687700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4670196b00 con 0x7f46700687a0 2024-02-16T19:39:00.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.353+0000 7f4678687700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f467019a890 con 0x7f4670069140 2024-02-16T19:39:00.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.353+0000 7f4676423700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46700687a0 0x7f4670195ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:39:00.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.353+0000 7f4675c22700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4670069140 0x7f4670196420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:39:00.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.353+0000 7f4675c22700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4670069140 0x7f4670196420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.22:40622/0 (socket says 172.21.15.22:40622) 2024-02-16T19:39:00.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.353+0000 7f4675c22700 1 -- 172.21.15.22:0/143447000 learned_addr learned my addr 172.21.15.22:0/143447000 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:39:00.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.354+0000 7f4675c22700 1 -- 172.21.15.22:0/143447000 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46700687a0 msgr2=0x7f4670195ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:39:00.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.354+0000 7f4675c22700 1 --2- 172.21.15.22:0/143447000 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46700687a0 0x7f4670195ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:39:00.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.354+0000 7f4675c22700 1 -- 172.21.15.22:0/143447000 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f466c0095f0 con 0x7f4670069140 2024-02-16T19:39:00.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.354+0000 7f4676423700 1 --2- 172.21.15.22:0/143447000 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46700687a0 0x7f4670195ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T19:39:00.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.354+0000 7f4675c22700 1 --2- 172.21.15.22:0/143447000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4670069140 0x7f4670196420 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f466000d8d0 tx=0x7f466000dbe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:39:00.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.354+0000 7f46677fe700 1 -- 172.21.15.22:0/143447000 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46600041f0 con 0x7f4670069140 2024-02-16T19:39:00.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.354+0000 7f46677fe700 1 -- 172.21.15.22:0/143447000 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4660007620 con 0x7f4670069140 2024-02-16T19:39:00.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.354+0000 7f4678687700 1 -- 172.21.15.22:0/143447000 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f467019ab70 con 0x7f4670069140 2024-02-16T19:39:00.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.355+0000 7f46677fe700 1 -- 172.21.15.22:0/143447000 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46600056b0 con 0x7f4670069140 2024-02-16T19:39:00.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.355+0000 7f4678687700 1 -- 172.21.15.22:0/143447000 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f467019b0c0 con 0x7f4670069140 2024-02-16T19:39:00.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.356+0000 7f46677fe700 1 -- 172.21.15.22:0/143447000 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f4660034730 con 0x7f4670069140 2024-02-16T19:39:00.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.356+0000 7f46677fe700 1 --2- 172.21.15.22:0/143447000 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f465c06d8a0 0x7f465c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:39:00.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.356+0000 7f4678687700 1 -- 172.21.15.22:0/143447000 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4670108790 con 0x7f4670069140 2024-02-16T19:39:00.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.357+0000 7f46677fe700 1 -- 172.21.15.22:0/143447000 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f466008ca80 con 0x7f4670069140 2024-02-16T19:39:00.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.357+0000 7f4676423700 1 --2- 172.21.15.22:0/143447000 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f465c06d8a0 0x7f465c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:39:00.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.357+0000 7f4676423700 1 --2- 172.21.15.22:0/143447000 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f465c06d8a0 0x7f465c06fd50 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f466c006010 tx=0x7f466c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:39:00.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.362+0000 7f46677fe700 1 -- 172.21.15.22:0/143447000 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4660056130 con 0x7f4670069140 2024-02-16T19:39:00.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.585+0000 7f4678687700 1 -- 172.21.15.22:0/143447000 --> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f4670197350 con 0x7f465c06d8a0 2024-02-16T19:39:00.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.597+0000 7f46677fe700 1 -- 172.21.15.22:0/143447000 <== mgr.34104 v2:172.21.15.22:6800/505266753 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f4670197350 con 0x7f465c06d8a0 2024-02-16T19:39:00.598 INFO:teuthology.orchestra.run.smithi022.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T19:39:00.599 INFO:teuthology.orchestra.run.smithi022.stdout:alertmanager.smithi022 smithi022 *:9093,9094 running (7m) 37s ago 18m 26.8M - 0.25.0 c8568f914cd2 c89bc76be374 2024-02-16T19:39:00.599 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-exporter.smithi022 smithi022 running (18m) 37s ago 18m 9198k - 18.2.0 dc2bc1663786 3f8bef4349ba 2024-02-16T19:39:00.599 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-exporter.smithi117 smithi117 running (17m) 10s ago 17m 9034k - 18.2.0 dc2bc1663786 59b65a7e309f 2024-02-16T19:39:00.599 INFO:teuthology.orchestra.run.smithi022.stdout:crash.smithi022 smithi022 running (4m) 37s ago 18m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a e0f78bdf43e4 2024-02-16T19:39:00.599 INFO:teuthology.orchestra.run.smithi022.stdout:crash.smithi117 smithi117 running (4m) 10s ago 17m 6669k - 19.0.0-1450-gf78a58c0 f081f847070a bce7c0590813 2024-02-16T19:39:00.599 INFO:teuthology.orchestra.run.smithi022.stdout:grafana.smithi022 smithi022 *:3000 running (6m) 37s ago 17m 84.3M - 9.4.12 a0ea5e3c46d9 fd27d11e9729 2024-02-16T19:39:00.599 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi022.btppwm smithi022 running (43s) 37s ago 12m 12.1M - 19.0.0-1450-gf78a58c0 f081f847070a 243e0becd1f8 2024-02-16T19:39:00.599 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi022.ocukvs smithi022 running (66s) 37s ago 12m 13.1M - 19.0.0-1450-gf78a58c0 f081f847070a e6ce6a81df63 2024-02-16T19:39:00.599 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi117.ibmppn smithi117 running (16s) 10s ago 12m 11.6M - 19.0.0-1450-gf78a58c0 f081f847070a 05b2e23f1aa1 2024-02-16T19:39:00.599 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi117.orzors smithi117 running (12m) 10s ago 12m 162M - 18.2.0 dc2bc1663786 7d2bc4df3feb 2024-02-16T19:39:00.599 INFO:teuthology.orchestra.run.smithi022.stdout:mgr.smithi022.gmdcch smithi022 *:8443,9283,8765 running (9m) 37s ago 19m 508M - 19.0.0-1450-gf78a58c0 f081f847070a 7cb74cc9deea 2024-02-16T19:39:00.599 INFO:teuthology.orchestra.run.smithi022.stdout:mgr.smithi117.cohbtv smithi117 *:8443,9283,8765 running (8m) 10s ago 17m 453M - 19.0.0-1450-gf78a58c0 f081f847070a c825b1152189 2024-02-16T19:39:00.599 INFO:teuthology.orchestra.run.smithi022.stdout:mon.smithi022 smithi022 running (5m) 37s ago 19m 54.1M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 9c84755ae67d 2024-02-16T19:39:00.599 INFO:teuthology.orchestra.run.smithi022.stdout:mon.smithi117 smithi117 running (5m) 10s ago 17m 45.4M 2048M 19.0.0-1450-gf78a58c0 f081f847070a a319ac50f1a9 2024-02-16T19:39:00.600 INFO:teuthology.orchestra.run.smithi022.stdout:node-exporter.smithi022 smithi022 *:9100 running (8m) 37s ago 18m 16.5M - 1.5.0 0da6a335fe13 1f5b58292d27 2024-02-16T19:39:00.600 INFO:teuthology.orchestra.run.smithi022.stdout:node-exporter.smithi117 smithi117 *:9100 running (7m) 10s ago 17m 15.8M - 1.5.0 0da6a335fe13 bb5e8491d5a6 2024-02-16T19:39:00.600 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0 smithi022 running (4m) 37s ago 16m 114M 1217M 19.0.0-1450-gf78a58c0 f081f847070a 002412b3525f 2024-02-16T19:39:00.600 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1 smithi022 running (3m) 37s ago 15m 118M 1217M 19.0.0-1450-gf78a58c0 f081f847070a c3488ce7b2a0 2024-02-16T19:39:00.600 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2 smithi022 running (3m) 37s ago 15m 95.2M 1217M 19.0.0-1450-gf78a58c0 f081f847070a 46138c50f79d 2024-02-16T19:39:00.600 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3 smithi117 running (2m) 10s ago 14m 101M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 28d7d8519e64 2024-02-16T19:39:00.600 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4 smithi117 running (115s) 10s ago 14m 94.9M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 367ba2085f6f 2024-02-16T19:39:00.600 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5 smithi117 running (82s) 10s ago 14m 95.1M 2240M 19.0.0-1450-gf78a58c0 f081f847070a a1a7625b7541 2024-02-16T19:39:00.600 INFO:teuthology.orchestra.run.smithi022.stdout:prometheus.smithi022 smithi022 *:9095 running (7m) 37s ago 17m 69.2M - 2.43.0 a07b618ecd1d c3b8689d0c97 2024-02-16T19:39:00.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.599+0000 7f4678687700 1 -- 172.21.15.22:0/143447000 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f465c06d8a0 msgr2=0x7f465c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:39:00.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.599+0000 7f4678687700 1 --2- 172.21.15.22:0/143447000 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f465c06d8a0 0x7f465c06fd50 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f466c006010 tx=0x7f466c01a040 comp rx=0 tx=0).stop 2024-02-16T19:39:00.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.599+0000 7f4678687700 1 -- 172.21.15.22:0/143447000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4670069140 msgr2=0x7f4670196420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:39:00.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.599+0000 7f4678687700 1 --2- 172.21.15.22:0/143447000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4670069140 0x7f4670196420 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f466000d8d0 tx=0x7f466000dbe0 comp rx=0 tx=0).stop 2024-02-16T19:39:00.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.599+0000 7f4678687700 1 -- 172.21.15.22:0/143447000 shutdown_connections 2024-02-16T19:39:00.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.599+0000 7f4678687700 1 --2- 172.21.15.22:0/143447000 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f465c06d8a0 0x7f465c06fd50 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:39:00.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.599+0000 7f4678687700 1 --2- 172.21.15.22:0/143447000 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46700687a0 0x7f4670195ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:39:00.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.599+0000 7f4678687700 1 --2- 172.21.15.22:0/143447000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4670069140 0x7f4670196420 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:39:00.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.599+0000 7f4678687700 1 -- 172.21.15.22:0/143447000 >> 172.21.15.22:0/143447000 conn(0x7f4670075130 msgr2=0x7f46700fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:39:00.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.599+0000 7f4678687700 1 -- 172.21.15.22:0/143447000 shutdown_connections 2024-02-16T19:39:00.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.599+0000 7f4678687700 1 -- 172.21.15.22:0/143447000 wait complete. 2024-02-16T19:39:00.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.706+0000 7f38a0ab3700 1 -- 172.21.15.22:0/2078630016 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f389c0ff350 msgr2=0x7f389c0ff720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:39:00.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.706+0000 7f38a0ab3700 1 --2- 172.21.15.22:0/2078630016 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f389c0ff350 0x7f389c0ff720 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f388c009910 tx=0x7f388c009c20 comp rx=0 tx=0).stop 2024-02-16T19:39:00.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.706+0000 7f38a0ab3700 1 -- 172.21.15.22:0/2078630016 shutdown_connections 2024-02-16T19:39:00.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.706+0000 7f38a0ab3700 1 --2- 172.21.15.22:0/2078630016 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f389c0ffc60 0x7f389c10a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:39:00.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.706+0000 7f38a0ab3700 1 --2- 172.21.15.22:0/2078630016 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f389c0ff350 0x7f389c0ff720 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:39:00.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.706+0000 7f38a0ab3700 1 -- 172.21.15.22:0/2078630016 >> 172.21.15.22:0/2078630016 conn(0x7f389c0748b0 msgr2=0x7f389c074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:39:00.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.706+0000 7f38a0ab3700 1 -- 172.21.15.22:0/2078630016 shutdown_connections 2024-02-16T19:39:00.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.706+0000 7f38a0ab3700 1 -- 172.21.15.22:0/2078630016 wait complete. 2024-02-16T19:39:00.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.707+0000 7f38a0ab3700 1 Processor -- start 2024-02-16T19:39:00.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.707+0000 7f38a0ab3700 1 -- start start 2024-02-16T19:39:00.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.707+0000 7f38a0ab3700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f389c0ff350 0x7f389c195f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:39:00.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.707+0000 7f38a0ab3700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f389c0ffc60 0x7f389c1964a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:39:00.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.707+0000 7f38a0ab3700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f389c196b80 con 0x7f389c0ffc60 2024-02-16T19:39:00.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.707+0000 7f38a0ab3700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f389c19a910 con 0x7f389c0ff350 2024-02-16T19:39:00.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.708+0000 7f3899d9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f389c0ffc60 0x7f389c1964a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:39:00.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.708+0000 7f3899d9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f389c0ffc60 0x7f389c1964a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:56762/0 (socket says 172.21.15.22:56762) 2024-02-16T19:39:00.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.708+0000 7f3899d9b700 1 -- 172.21.15.22:0/2211327666 learned_addr learned my addr 172.21.15.22:0/2211327666 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:39:00.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.708+0000 7f389a59c700 1 --2- 172.21.15.22:0/2211327666 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f389c0ff350 0x7f389c195f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:39:00.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.708+0000 7f3899d9b700 1 -- 172.21.15.22:0/2211327666 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f389c0ff350 msgr2=0x7f389c195f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:39:00.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.708+0000 7f3899d9b700 1 --2- 172.21.15.22:0/2211327666 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f389c0ff350 0x7f389c195f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:39:00.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.708+0000 7f3899d9b700 1 -- 172.21.15.22:0/2211327666 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f388c0095f0 con 0x7f389c0ffc60 2024-02-16T19:39:00.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.708+0000 7f3899d9b700 1 --2- 172.21.15.22:0/2211327666 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f389c0ffc60 0x7f389c1964a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f389000d8d0 tx=0x7f389000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:39:00.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.708+0000 7f388b7fe700 1 -- 172.21.15.22:0/2211327666 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f38900098a0 con 0x7f389c0ffc60 2024-02-16T19:39:00.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.708+0000 7f38a0ab3700 1 -- 172.21.15.22:0/2211327666 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f389c19abf0 con 0x7f389c0ffc60 2024-02-16T19:39:00.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.708+0000 7f388b7fe700 1 -- 172.21.15.22:0/2211327666 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3890010460 con 0x7f389c0ffc60 2024-02-16T19:39:00.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.709+0000 7f388b7fe700 1 -- 172.21.15.22:0/2211327666 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f389000f5f0 con 0x7f389c0ffc60 2024-02-16T19:39:00.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.709+0000 7f38a0ab3700 1 -- 172.21.15.22:0/2211327666 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f389c19b140 con 0x7f389c0ffc60 2024-02-16T19:39:00.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.710+0000 7f38a0ab3700 1 -- 172.21.15.22:0/2211327666 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f389c107dd0 con 0x7f389c0ffc60 2024-02-16T19:39:00.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.711+0000 7f388b7fe700 1 -- 172.21.15.22:0/2211327666 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f3890009a00 con 0x7f389c0ffc60 2024-02-16T19:39:00.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.711+0000 7f388b7fe700 1 --2- 172.21.15.22:0/2211327666 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f388406d8a0 0x7f388406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:39:00.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.712+0000 7f388b7fe700 1 -- 172.21.15.22:0/2211327666 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f389008d5a0 con 0x7f389c0ffc60 2024-02-16T19:39:00.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.712+0000 7f389a59c700 1 --2- 172.21.15.22:0/2211327666 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f388406d8a0 0x7f388406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:39:00.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.712+0000 7f389a59c700 1 --2- 172.21.15.22:0/2211327666 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f388406d8a0 0x7f388406fd50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f388c006010 tx=0x7f388c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:39:00.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:00.715+0000 7f388b7fe700 1 -- 172.21.15.22:0/2211327666 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3890056d00 con 0x7f389c0ffc60 2024-02-16T19:39:01.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.068+0000 7f38a0ab3700 1 -- 172.21.15.22:0/2211327666 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f389c04f2a0 con 0x7f389c0ffc60 2024-02-16T19:39:01.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.069+0000 7f388b7fe700 1 -- 172.21.15.22:0/2211327666 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f389005a3c0 con 0x7f389c0ffc60 2024-02-16T19:39:01.071 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-02-16T19:39:01.071 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": { 2024-02-16T19:39:01.071 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T19:39:01.071 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:39:01.071 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": { 2024-02-16T19:39:01.071 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T19:39:01.071 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:39:01.071 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": { 2024-02-16T19:39:01.071 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-16T19:39:01.071 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:39:01.071 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": { 2024-02-16T19:39:01.071 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 3 2024-02-16T19:39:01.071 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:39:01.072 INFO:teuthology.orchestra.run.smithi022.stdout: "overall": { 2024-02-16T19:39:01.072 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 13 2024-02-16T19:39:01.072 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-02-16T19:39:01.072 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-02-16T19:39:01.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.072+0000 7f38a0ab3700 1 -- 172.21.15.22:0/2211327666 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f388406d8a0 msgr2=0x7f388406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:39:01.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.072+0000 7f38a0ab3700 1 --2- 172.21.15.22:0/2211327666 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f388406d8a0 0x7f388406fd50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f388c006010 tx=0x7f388c01a040 comp rx=0 tx=0).stop 2024-02-16T19:39:01.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.072+0000 7f38a0ab3700 1 -- 172.21.15.22:0/2211327666 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f389c0ffc60 msgr2=0x7f389c1964a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:39:01.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.072+0000 7f38a0ab3700 1 --2- 172.21.15.22:0/2211327666 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f389c0ffc60 0x7f389c1964a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f389000d8d0 tx=0x7f389000dbe0 comp rx=0 tx=0).stop 2024-02-16T19:39:01.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.072+0000 7f38a0ab3700 1 -- 172.21.15.22:0/2211327666 shutdown_connections 2024-02-16T19:39:01.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.072+0000 7f38a0ab3700 1 --2- 172.21.15.22:0/2211327666 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f388406d8a0 0x7f388406fd50 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:39:01.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.072+0000 7f38a0ab3700 1 --2- 172.21.15.22:0/2211327666 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f389c0ff350 0x7f389c195f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:39:01.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.072+0000 7f38a0ab3700 1 --2- 172.21.15.22:0/2211327666 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f389c0ffc60 0x7f389c1964a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:39:01.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.072+0000 7f38a0ab3700 1 -- 172.21.15.22:0/2211327666 >> 172.21.15.22:0/2211327666 conn(0x7f389c0748b0 msgr2=0x7f389c104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:39:01.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.073+0000 7f38a0ab3700 1 -- 172.21.15.22:0/2211327666 shutdown_connections 2024-02-16T19:39:01.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.073+0000 7f38a0ab3700 1 -- 172.21.15.22:0/2211327666 wait complete. 2024-02-16T19:39:01.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.187+0000 7f5bd46a2700 1 -- 172.21.15.22:0/3250447215 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5bcc1004c0 msgr2=0x7f5bcc100890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:39:01.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.187+0000 7f5bd46a2700 1 --2- 172.21.15.22:0/3250447215 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5bcc1004c0 0x7f5bcc100890 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5bc8009910 tx=0x7f5bc8009c20 comp rx=0 tx=0).stop 2024-02-16T19:39:01.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.187+0000 7f5bd46a2700 1 -- 172.21.15.22:0/3250447215 shutdown_connections 2024-02-16T19:39:01.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.187+0000 7f5bd46a2700 1 --2- 172.21.15.22:0/3250447215 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bcc100e60 0x7f5bcc104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:39:01.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.187+0000 7f5bd46a2700 1 --2- 172.21.15.22:0/3250447215 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5bcc1004c0 0x7f5bcc100890 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:39:01.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.187+0000 7f5bd46a2700 1 -- 172.21.15.22:0/3250447215 >> 172.21.15.22:0/3250447215 conn(0x7f5bcc0fbd80 msgr2=0x7f5bcc0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:39:01.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.188+0000 7f5bd46a2700 1 -- 172.21.15.22:0/3250447215 shutdown_connections 2024-02-16T19:39:01.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.188+0000 7f5bd46a2700 1 -- 172.21.15.22:0/3250447215 wait complete. 2024-02-16T19:39:01.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.188+0000 7f5bd46a2700 1 Processor -- start 2024-02-16T19:39:01.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.188+0000 7f5bd46a2700 1 -- start start 2024-02-16T19:39:01.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.189+0000 7f5bd46a2700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5bcc1004c0 0x7f5bcc195f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:39:01.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.189+0000 7f5bd46a2700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bcc100e60 0x7f5bcc196470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:39:01.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.189+0000 7f5bd46a2700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bcc196b50 con 0x7f5bcc1004c0 2024-02-16T19:39:01.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.189+0000 7f5bd46a2700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bcc19a8e0 con 0x7f5bcc100e60 2024-02-16T19:39:01.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.189+0000 7f5bd243e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5bcc1004c0 0x7f5bcc195f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:39:01.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.189+0000 7f5bd243e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5bcc1004c0 0x7f5bcc195f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:56780/0 (socket says 172.21.15.22:56780) 2024-02-16T19:39:01.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.189+0000 7f5bd243e700 1 -- 172.21.15.22:0/1887178963 learned_addr learned my addr 172.21.15.22:0/1887178963 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:39:01.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.189+0000 7f5bd1c3d700 1 --2- 172.21.15.22:0/1887178963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bcc100e60 0x7f5bcc196470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:39:01.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.189+0000 7f5bd243e700 1 -- 172.21.15.22:0/1887178963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bcc100e60 msgr2=0x7f5bcc196470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:39:01.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.189+0000 7f5bd243e700 1 --2- 172.21.15.22:0/1887178963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bcc100e60 0x7f5bcc196470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:39:01.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.189+0000 7f5bd243e700 1 -- 172.21.15.22:0/1887178963 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bc80095f0 con 0x7f5bcc1004c0 2024-02-16T19:39:01.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.189+0000 7f5bd243e700 1 --2- 172.21.15.22:0/1887178963 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5bcc1004c0 0x7f5bcc195f30 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5bc8006010 tx=0x7f5bc800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:39:01.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.190+0000 7f5bc37fe700 1 -- 172.21.15.22:0/1887178963 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5bc801d070 con 0x7f5bcc1004c0 2024-02-16T19:39:01.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.190+0000 7f5bc37fe700 1 -- 172.21.15.22:0/1887178963 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5bc8004fb0 con 0x7f5bcc1004c0 2024-02-16T19:39:01.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.190+0000 7f5bc37fe700 1 -- 172.21.15.22:0/1887178963 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5bc8022ea0 con 0x7f5bcc1004c0 2024-02-16T19:39:01.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.190+0000 7f5bd46a2700 1 -- 172.21.15.22:0/1887178963 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bcc19ab60 con 0x7f5bcc1004c0 2024-02-16T19:39:01.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.190+0000 7f5bd46a2700 1 -- 172.21.15.22:0/1887178963 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bcc19b050 con 0x7f5bcc1004c0 2024-02-16T19:39:01.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.191+0000 7f5bd46a2700 1 -- 172.21.15.22:0/1887178963 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bcc1087d0 con 0x7f5bcc1004c0 2024-02-16T19:39:01.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.191+0000 7f5bc37fe700 1 -- 172.21.15.22:0/1887178963 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f5bc8022470 con 0x7f5bcc1004c0 2024-02-16T19:39:01.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.192+0000 7f5bc37fe700 1 --2- 172.21.15.22:0/1887178963 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f5bb806d8a0 0x7f5bb806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:39:01.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.192+0000 7f5bc37fe700 1 -- 172.21.15.22:0/1887178963 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f5bc808ed30 con 0x7f5bcc1004c0 2024-02-16T19:39:01.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.192+0000 7f5bd1c3d700 1 --2- 172.21.15.22:0/1887178963 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f5bb806d8a0 0x7f5bb806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:39:01.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.193+0000 7f5bd1c3d700 1 --2- 172.21.15.22:0/1887178963 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f5bb806d8a0 0x7f5bb806fd50 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc005fd0 tx=0x7f5bbc00a560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:39:01.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.201+0000 7f5bc37fe700 1 -- 172.21.15.22:0/1887178963 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5bc8058530 con 0x7f5bcc1004c0 2024-02-16T19:39:01.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.501+0000 7f5bd46a2700 1 -- 172.21.15.22:0/1887178963 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f5bcc04f2a0 con 0x7f5bcc1004c0 2024-02-16T19:39:01.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.501+0000 7f5bc37fe700 1 -- 172.21.15.22:0/1887178963 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1693 (secure 0 0 0) 0x7f5bc8027070 con 0x7f5bcc1004c0 2024-02-16T19:39:01.503 INFO:teuthology.orchestra.run.smithi022.stdout:e31 2024-02-16T19:39:01.503 INFO:teuthology.orchestra.run.smithi022.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T19:39:01.503 INFO:teuthology.orchestra.run.smithi022.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:39:01.503 INFO:teuthology.orchestra.run.smithi022.stdout:legacy client fscid: 1 2024-02-16T19:39:01.504 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:39:01.504 INFO:teuthology.orchestra.run.smithi022.stdout:Filesystem 'cephfs' (1) 2024-02-16T19:39:01.504 INFO:teuthology.orchestra.run.smithi022.stdout:fs_name cephfs 2024-02-16T19:39:01.504 INFO:teuthology.orchestra.run.smithi022.stdout:epoch 31 2024-02-16T19:39:01.504 INFO:teuthology.orchestra.run.smithi022.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T19:39:01.504 INFO:teuthology.orchestra.run.smithi022.stdout:created 2024-02-16T19:26:17.873644+0000 2024-02-16T19:39:01.504 INFO:teuthology.orchestra.run.smithi022.stdout:modified 2024-02-16T19:38:57.810288+0000 2024-02-16T19:39:01.504 INFO:teuthology.orchestra.run.smithi022.stdout:tableserver 0 2024-02-16T19:39:01.504 INFO:teuthology.orchestra.run.smithi022.stdout:root 0 2024-02-16T19:39:01.504 INFO:teuthology.orchestra.run.smithi022.stdout:session_timeout 60 2024-02-16T19:39:01.504 INFO:teuthology.orchestra.run.smithi022.stdout:session_autoclose 300 2024-02-16T19:39:01.504 INFO:teuthology.orchestra.run.smithi022.stdout:max_file_size 1099511627776 2024-02-16T19:39:01.504 INFO:teuthology.orchestra.run.smithi022.stdout:max_xattr_size 65536 2024-02-16T19:39:01.504 INFO:teuthology.orchestra.run.smithi022.stdout:required_client_features {} 2024-02-16T19:39:01.505 INFO:teuthology.orchestra.run.smithi022.stdout:last_failure 0 2024-02-16T19:39:01.505 INFO:teuthology.orchestra.run.smithi022.stdout:last_failure_osd_epoch 93 2024-02-16T19:39:01.505 INFO:teuthology.orchestra.run.smithi022.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-16T19:39:01.505 INFO:teuthology.orchestra.run.smithi022.stdout:max_mds 1 2024-02-16T19:39:01.505 INFO:teuthology.orchestra.run.smithi022.stdout:in 0 2024-02-16T19:39:01.505 INFO:teuthology.orchestra.run.smithi022.stdout:up {0=34384} 2024-02-16T19:39:01.505 INFO:teuthology.orchestra.run.smithi022.stdout:failed 2024-02-16T19:39:01.505 INFO:teuthology.orchestra.run.smithi022.stdout:damaged 2024-02-16T19:39:01.505 INFO:teuthology.orchestra.run.smithi022.stdout:stopped 2024-02-16T19:39:01.505 INFO:teuthology.orchestra.run.smithi022.stdout:data_pools [3] 2024-02-16T19:39:01.505 INFO:teuthology.orchestra.run.smithi022.stdout:metadata_pool 2 2024-02-16T19:39:01.505 INFO:teuthology.orchestra.run.smithi022.stdout:inline_data enabled 2024-02-16T19:39:01.506 INFO:teuthology.orchestra.run.smithi022.stdout:balancer 2024-02-16T19:39:01.506 INFO:teuthology.orchestra.run.smithi022.stdout:bal_rank_mask -1 2024-02-16T19:39:01.506 INFO:teuthology.orchestra.run.smithi022.stdout:standby_count_wanted 1 2024-02-16T19:39:01.506 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.cephfs.smithi022.ocukvs{0:34384} state up:replay seq 1 join_fscid=1 addr [v2:172.21.15.22:6826/2535680483,v1:172.21.15.22:6827/2535680483] compat {c=[1],r=[1],i=[fff]}] 2024-02-16T19:39:01.506 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:39:01.506 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:39:01.506 INFO:teuthology.orchestra.run.smithi022.stdout:Standby daemons: 2024-02-16T19:39:01.506 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:39:01.506 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.cephfs.smithi022.btppwm{-1:34408} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.22:6828/3126701730,v1:172.21.15.22:6829/3126701730] compat {c=[1],r=[1],i=[fff]}] 2024-02-16T19:39:01.506 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.cephfs.smithi117.ibmppn{-1:44299} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6824/2389644474,v1:172.21.15.117:6825/2389644474] compat {c=[1],r=[1],i=[fff]}] 2024-02-16T19:39:01.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.503+0000 7f5bd46a2700 1 -- 172.21.15.22:0/1887178963 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f5bb806d8a0 msgr2=0x7f5bb806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:39:01.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.503+0000 7f5bd46a2700 1 --2- 172.21.15.22:0/1887178963 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f5bb806d8a0 0x7f5bb806fd50 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc005fd0 tx=0x7f5bbc00a560 comp rx=0 tx=0).stop 2024-02-16T19:39:01.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.504+0000 7f5bd46a2700 1 -- 172.21.15.22:0/1887178963 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5bcc1004c0 msgr2=0x7f5bcc195f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:39:01.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.504+0000 7f5bd46a2700 1 --2- 172.21.15.22:0/1887178963 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5bcc1004c0 0x7f5bcc195f30 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5bc8006010 tx=0x7f5bc800be20 comp rx=0 tx=0).stop 2024-02-16T19:39:01.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.504+0000 7f5bd46a2700 1 -- 172.21.15.22:0/1887178963 shutdown_connections 2024-02-16T19:39:01.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.504+0000 7f5bd46a2700 1 --2- 172.21.15.22:0/1887178963 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f5bb806d8a0 0x7f5bb806fd50 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:39:01.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.504+0000 7f5bd46a2700 1 --2- 172.21.15.22:0/1887178963 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5bcc1004c0 0x7f5bcc195f30 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:39:01.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.504+0000 7f5bd46a2700 1 --2- 172.21.15.22:0/1887178963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bcc100e60 0x7f5bcc196470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:39:01.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.504+0000 7f5bd46a2700 1 -- 172.21.15.22:0/1887178963 >> 172.21.15.22:0/1887178963 conn(0x7f5bcc0fbd80 msgr2=0x7f5bcc0fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:39:01.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.504+0000 7f5bd46a2700 1 -- 172.21.15.22:0/1887178963 shutdown_connections 2024-02-16T19:39:01.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.504+0000 7f5bd46a2700 1 -- 172.21.15.22:0/1887178963 wait complete. 2024-02-16T19:39:01.508 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 31 2024-02-16T19:39:01.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.620+0000 7f867fc7a700 1 -- 172.21.15.22:0/1569859142 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8678101d30 msgr2=0x7f867810a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:39:01.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.620+0000 7f867fc7a700 1 --2- 172.21.15.22:0/1569859142 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8678101d30 0x7f867810a230 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8668009910 tx=0x7f8668009c20 comp rx=0 tx=0).stop 2024-02-16T19:39:01.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.620+0000 7f867fc7a700 1 -- 172.21.15.22:0/1569859142 shutdown_connections 2024-02-16T19:39:01.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.620+0000 7f867fc7a700 1 --2- 172.21.15.22:0/1569859142 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8678101d30 0x7f867810a230 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:39:01.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.620+0000 7f867fc7a700 1 --2- 172.21.15.22:0/1569859142 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8678101420 0x7f86781017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:39:01.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.620+0000 7f867fc7a700 1 -- 172.21.15.22:0/1569859142 >> 172.21.15.22:0/1569859142 conn(0x7f86780fac60 msgr2=0x7f86780fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:39:01.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.620+0000 7f867fc7a700 1 -- 172.21.15.22:0/1569859142 shutdown_connections 2024-02-16T19:39:01.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.620+0000 7f867fc7a700 1 -- 172.21.15.22:0/1569859142 wait complete. 2024-02-16T19:39:01.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.621+0000 7f867fc7a700 1 Processor -- start 2024-02-16T19:39:01.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.621+0000 7f867fc7a700 1 -- start start 2024-02-16T19:39:01.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.621+0000 7f867fc7a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8678101420 0x7f8678196000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:39:01.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.621+0000 7f867fc7a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8678101d30 0x7f8678196540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:39:01.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.621+0000 7f867fc7a700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8678196c20 con 0x7f8678101d30 2024-02-16T19:39:01.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.621+0000 7f867fc7a700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f867819a9b0 con 0x7f8678101420 2024-02-16T19:39:01.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.621+0000 7f867d215700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8678101d30 0x7f8678196540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:39:01.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.621+0000 7f867d215700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8678101d30 0x7f8678196540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:56796/0 (socket says 172.21.15.22:56796) 2024-02-16T19:39:01.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.621+0000 7f867d215700 1 -- 172.21.15.22:0/554257997 learned_addr learned my addr 172.21.15.22:0/554257997 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:39:01.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.621+0000 7f867da16700 1 --2- 172.21.15.22:0/554257997 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8678101420 0x7f8678196000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:39:01.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.622+0000 7f867d215700 1 -- 172.21.15.22:0/554257997 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8678101420 msgr2=0x7f8678196000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:39:01.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.622+0000 7f867d215700 1 --2- 172.21.15.22:0/554257997 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8678101420 0x7f8678196000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:39:01.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.622+0000 7f867d215700 1 -- 172.21.15.22:0/554257997 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86680095f0 con 0x7f8678101d30 2024-02-16T19:39:01.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.622+0000 7f867d215700 1 --2- 172.21.15.22:0/554257997 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8678101d30 0x7f8678196540 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8668006010 tx=0x7f866800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:39:01.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.622+0000 7f866effd700 1 -- 172.21.15.22:0/554257997 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f866801d070 con 0x7f8678101d30 2024-02-16T19:39:01.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.622+0000 7f866effd700 1 -- 172.21.15.22:0/554257997 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f866800f460 con 0x7f8678101d30 2024-02-16T19:39:01.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.622+0000 7f867fc7a700 1 -- 172.21.15.22:0/554257997 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f867819ac30 con 0x7f8678101d30 2024-02-16T19:39:01.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.623+0000 7f866effd700 1 -- 172.21.15.22:0/554257997 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8668017610 con 0x7f8678101d30 2024-02-16T19:39:01.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.623+0000 7f867fc7a700 1 -- 172.21.15.22:0/554257997 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f867819b120 con 0x7f8678101d30 2024-02-16T19:39:01.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.624+0000 7f867fc7a700 1 -- 172.21.15.22:0/554257997 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86781078a0 con 0x7f8678101d30 2024-02-16T19:39:01.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.624+0000 7f866effd700 1 -- 172.21.15.22:0/554257997 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f866800f5d0 con 0x7f8678101d30 2024-02-16T19:39:01.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.625+0000 7f866effd700 1 --2- 172.21.15.22:0/554257997 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f866406d8a0 0x7f866406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:39:01.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.625+0000 7f866effd700 1 -- 172.21.15.22:0/554257997 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f866808eb50 con 0x7f8678101d30 2024-02-16T19:39:01.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.625+0000 7f867da16700 1 --2- 172.21.15.22:0/554257997 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f866406d8a0 0x7f866406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:39:01.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.625+0000 7f867da16700 1 --2- 172.21.15.22:0/554257997 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f866406d8a0 0x7f866406fd50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8674009a10 tx=0x7f8674009190 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:39:01.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.629+0000 7f866effd700 1 -- 172.21.15.22:0/554257997 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8668058200 con 0x7f8678101d30 2024-02-16T19:39:01.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.912+0000 7f867fc7a700 1 -- 172.21.15.22:0/554257997 --> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8678197470 con 0x7f866406d8a0 2024-02-16T19:39:01.914 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:01 smithi022 ceph-mon[88798]: pgmap v218: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 4.0 MiB/s rd, 0 B/s wr, 1 op/s 2024-02-16T19:39:01.914 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:01 smithi022 ceph-mon[88798]: from='client.34442 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:39:01.914 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:01 smithi022 ceph-mon[88798]: from='client.34446 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:39:01.914 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:01 smithi022 ceph-mon[88798]: from='client.44309 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:39:01.914 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:01 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:01.915 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:01 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:39:01.915 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:01 smithi022 ceph-mon[88798]: from='client.? 172.21.15.22:0/2211327666' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:39:01.915 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:01 smithi022 ceph-mon[88798]: from='client.? 172.21.15.22:0/1887178963' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:39:01.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.914+0000 7f866effd700 1 -- 172.21.15.22:0/554257997 <== mgr.34104 v2:172.21.15.22:6800/505266753 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f8678197470 con 0x7f866406d8a0 2024-02-16T19:39:01.915 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-02-16T19:39:01.915 INFO:teuthology.orchestra.run.smithi022.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T19:39:01.915 INFO:teuthology.orchestra.run.smithi022.stdout: "in_progress": true, 2024-02-16T19:39:01.916 INFO:teuthology.orchestra.run.smithi022.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T19:39:01.916 INFO:teuthology.orchestra.run.smithi022.stdout: "services_complete": [ 2024-02-16T19:39:01.916 INFO:teuthology.orchestra.run.smithi022.stdout: "crash", 2024-02-16T19:39:01.916 INFO:teuthology.orchestra.run.smithi022.stdout: "osd", 2024-02-16T19:39:01.916 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr", 2024-02-16T19:39:01.916 INFO:teuthology.orchestra.run.smithi022.stdout: "mon" 2024-02-16T19:39:01.916 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-02-16T19:39:01.916 INFO:teuthology.orchestra.run.smithi022.stdout: "progress": "15/23 daemons upgraded", 2024-02-16T19:39:01.916 INFO:teuthology.orchestra.run.smithi022.stdout: "message": "Currently upgrading mds daemons", 2024-02-16T19:39:01.916 INFO:teuthology.orchestra.run.smithi022.stdout: "is_paused": false 2024-02-16T19:39:01.916 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-02-16T19:39:01.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.917+0000 7f867fc7a700 1 -- 172.21.15.22:0/554257997 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f866406d8a0 msgr2=0x7f866406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:39:01.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.917+0000 7f867fc7a700 1 --2- 172.21.15.22:0/554257997 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f866406d8a0 0x7f866406fd50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8674009a10 tx=0x7f8674009190 comp rx=0 tx=0).stop 2024-02-16T19:39:01.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.917+0000 7f867fc7a700 1 -- 172.21.15.22:0/554257997 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8678101d30 msgr2=0x7f8678196540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:39:01.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.917+0000 7f867fc7a700 1 --2- 172.21.15.22:0/554257997 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8678101d30 0x7f8678196540 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8668006010 tx=0x7f866800bd70 comp rx=0 tx=0).stop 2024-02-16T19:39:01.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.917+0000 7f867fc7a700 1 -- 172.21.15.22:0/554257997 shutdown_connections 2024-02-16T19:39:01.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.917+0000 7f867fc7a700 1 --2- 172.21.15.22:0/554257997 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f866406d8a0 0x7f866406fd50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:39:01.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.917+0000 7f867fc7a700 1 --2- 172.21.15.22:0/554257997 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8678101420 0x7f8678196000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:39:01.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.917+0000 7f867fc7a700 1 --2- 172.21.15.22:0/554257997 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8678101d30 0x7f8678196540 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:39:01.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.917+0000 7f867fc7a700 1 -- 172.21.15.22:0/554257997 >> 172.21.15.22:0/554257997 conn(0x7f86780fac60 msgr2=0x7f86781049e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:39:01.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.918+0000 7f867fc7a700 1 -- 172.21.15.22:0/554257997 shutdown_connections 2024-02-16T19:39:01.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:01.918+0000 7f867fc7a700 1 -- 172.21.15.22:0/554257997 wait complete. 2024-02-16T19:39:02.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:02.031+0000 7f54a35b2700 1 -- 172.21.15.22:0/3680408240 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f549c1038b0 msgr2=0x7f549c103c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:39:02.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:02.031+0000 7f54a35b2700 1 --2- 172.21.15.22:0/3680408240 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f549c1038b0 0x7f549c103c80 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5498009910 tx=0x7f5498009c20 comp rx=0 tx=0).stop 2024-02-16T19:39:02.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:02.031+0000 7f54a35b2700 1 -- 172.21.15.22:0/3680408240 shutdown_connections 2024-02-16T19:39:02.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:02.031+0000 7f54a35b2700 1 --2- 172.21.15.22:0/3680408240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f549c0ff350 0x7f549c0ff7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:39:02.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:02.031+0000 7f54a35b2700 1 --2- 172.21.15.22:0/3680408240 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f549c1038b0 0x7f549c103c80 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:39:02.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:02.031+0000 7f54a35b2700 1 -- 172.21.15.22:0/3680408240 >> 172.21.15.22:0/3680408240 conn(0x7f549c0748b0 msgr2=0x7f549c074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:39:02.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:02.031+0000 7f54a35b2700 1 -- 172.21.15.22:0/3680408240 shutdown_connections 2024-02-16T19:39:02.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:02.031+0000 7f54a35b2700 1 -- 172.21.15.22:0/3680408240 wait complete. 2024-02-16T19:39:02.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:02.032+0000 7f54a35b2700 1 Processor -- start 2024-02-16T19:39:02.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:02.032+0000 7f54a35b2700 1 -- start start 2024-02-16T19:39:02.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:02.032+0000 7f54a35b2700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f549c0ff350 0x7f549c1005b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:39:02.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:02.032+0000 7f54a35b2700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f549c1038b0 0x7f549c100af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:39:02.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:02.032+0000 7f54a35b2700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f549c101030 con 0x7f549c1038b0 2024-02-16T19:39:02.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:02.032+0000 7f54a35b2700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f549c101170 con 0x7f549c0ff350 2024-02-16T19:39:02.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:02.032+0000 7f54a0b4d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f549c1038b0 0x7f549c100af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:39:02.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:02.032+0000 7f54a0b4d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f549c1038b0 0x7f549c100af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:56804/0 (socket says 172.21.15.22:56804) 2024-02-16T19:39:02.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:02.032+0000 7f54a0b4d700 1 -- 172.21.15.22:0/3565628906 learned_addr learned my addr 172.21.15.22:0/3565628906 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:39:02.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:02.033+0000 7f54a134e700 1 --2- 172.21.15.22:0/3565628906 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f549c0ff350 0x7f549c1005b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:39:02.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:02.033+0000 7f54a0b4d700 1 -- 172.21.15.22:0/3565628906 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f549c0ff350 msgr2=0x7f549c1005b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:39:02.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:02.033+0000 7f54a0b4d700 1 --2- 172.21.15.22:0/3565628906 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f549c0ff350 0x7f549c1005b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:39:02.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:02.033+0000 7f54a0b4d700 1 -- 172.21.15.22:0/3565628906 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54980095f0 con 0x7f549c1038b0 2024-02-16T19:39:02.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:02.033+0000 7f54a0b4d700 1 --2- 172.21.15.22:0/3565628906 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f549c1038b0 0x7f549c100af0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f548c00ea30 tx=0x7f548c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:39:02.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:02.033+0000 7f54927fc700 1 -- 172.21.15.22:0/3565628906 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f548c00cd60 con 0x7f549c1038b0 2024-02-16T19:39:02.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:02.033+0000 7f54a35b2700 1 -- 172.21.15.22:0/3565628906 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f549c1a2010 con 0x7f549c1038b0 2024-02-16T19:39:02.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:02.033+0000 7f54927fc700 1 -- 172.21.15.22:0/3565628906 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f548c0168a0 con 0x7f549c1038b0 2024-02-16T19:39:02.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:02.033+0000 7f54927fc700 1 -- 172.21.15.22:0/3565628906 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f548c015860 con 0x7f549c1038b0 2024-02-16T19:39:02.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:02.033+0000 7f54a35b2700 1 -- 172.21.15.22:0/3565628906 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f549c1a2420 con 0x7f549c1038b0 2024-02-16T19:39:02.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:02.035+0000 7f54a35b2700 1 -- 172.21.15.22:0/3565628906 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f549c108e30 con 0x7f549c1038b0 2024-02-16T19:39:02.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:02.036+0000 7f54927fc700 1 -- 172.21.15.22:0/3565628906 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f548c016a10 con 0x7f549c1038b0 2024-02-16T19:39:02.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:02.036+0000 7f54927fc700 1 --2- 172.21.15.22:0/3565628906 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f548806d8a0 0x7f548806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:39:02.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:02.036+0000 7f54927fc700 1 -- 172.21.15.22:0/3565628906 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f548c08db80 con 0x7f549c1038b0 2024-02-16T19:39:02.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:02.036+0000 7f54a134e700 1 --2- 172.21.15.22:0/3565628906 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f548806d8a0 0x7f548806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:39:02.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:02.037+0000 7f54a134e700 1 --2- 172.21.15.22:0/3565628906 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f548806d8a0 0x7f548806fd50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5498005f50 tx=0x7f5498005e20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:39:02.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:02.045+0000 7f54927fc700 1 -- 172.21.15.22:0/3565628906 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f548c057230 con 0x7f549c1038b0 2024-02-16T19:39:02.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:01 smithi117 ceph-mon[67870]: pgmap v218: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 4.0 MiB/s rd, 0 B/s wr, 1 op/s 2024-02-16T19:39:02.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:01 smithi117 ceph-mon[67870]: from='client.34442 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:39:02.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:01 smithi117 ceph-mon[67870]: from='client.34446 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:39:02.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:01 smithi117 ceph-mon[67870]: from='client.44309 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:39:02.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:01 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:02.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:01 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:39:02.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:01 smithi117 ceph-mon[67870]: from='client.? 172.21.15.22:0/2211327666' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:39:02.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:01 smithi117 ceph-mon[67870]: from='client.? 172.21.15.22:0/1887178963' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:39:02.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:02.368+0000 7f54a35b2700 1 -- 172.21.15.22:0/3565628906 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f549c04f2a0 con 0x7f549c1038b0 2024-02-16T19:39:02.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:02.369+0000 7f54927fc700 1 -- 172.21.15.22:0/3565628906 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+297 (secure 0 0 0) 0x7f548c05a8f0 con 0x7f549c1038b0 2024-02-16T19:39:02.370 INFO:teuthology.orchestra.run.smithi022.stdout:HEALTH_WARN 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data 2024-02-16T19:39:02.370 INFO:teuthology.orchestra.run.smithi022.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-16T19:39:02.370 INFO:teuthology.orchestra.run.smithi022.stdout: fs cephfs is degraded 2024-02-16T19:39:02.371 INFO:teuthology.orchestra.run.smithi022.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-02-16T19:39:02.371 INFO:teuthology.orchestra.run.smithi022.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-02-16T19:39:02.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:02.371+0000 7f54a35b2700 1 -- 172.21.15.22:0/3565628906 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f548806d8a0 msgr2=0x7f548806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:39:02.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:02.371+0000 7f54a35b2700 1 --2- 172.21.15.22:0/3565628906 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f548806d8a0 0x7f548806fd50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5498005f50 tx=0x7f5498005e20 comp rx=0 tx=0).stop 2024-02-16T19:39:02.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:02.371+0000 7f54a35b2700 1 -- 172.21.15.22:0/3565628906 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f549c1038b0 msgr2=0x7f549c100af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:39:02.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:02.371+0000 7f54a35b2700 1 --2- 172.21.15.22:0/3565628906 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f549c1038b0 0x7f549c100af0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f548c00ea30 tx=0x7f548c00edf0 comp rx=0 tx=0).stop 2024-02-16T19:39:02.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:02.371+0000 7f54a35b2700 1 -- 172.21.15.22:0/3565628906 shutdown_connections 2024-02-16T19:39:02.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:02.371+0000 7f54a35b2700 1 --2- 172.21.15.22:0/3565628906 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f548806d8a0 0x7f548806fd50 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:39:02.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:02.371+0000 7f54a35b2700 1 --2- 172.21.15.22:0/3565628906 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f549c0ff350 0x7f549c1005b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:39:02.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:02.371+0000 7f54a35b2700 1 --2- 172.21.15.22:0/3565628906 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f549c1038b0 0x7f549c100af0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:39:02.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:02.371+0000 7f54a35b2700 1 -- 172.21.15.22:0/3565628906 >> 172.21.15.22:0/3565628906 conn(0x7f549c0748b0 msgr2=0x7f549c0fe470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:39:02.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:02.371+0000 7f54a35b2700 1 -- 172.21.15.22:0/3565628906 shutdown_connections 2024-02-16T19:39:02.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:02.371+0000 7f54a35b2700 1 -- 172.21.15.22:0/3565628906 wait complete. 2024-02-16T19:39:02.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:02 smithi022 ceph-mon[88798]: from='client.? 172.21.15.22:0/3565628906' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T19:39:03.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:02 smithi117 ceph-mon[67870]: from='client.? 172.21.15.22:0/3565628906' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T19:39:03.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:03 smithi022 ceph-mon[88798]: pgmap v219: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 10 MiB/s rd, 3 op/s 2024-02-16T19:39:03.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:03 smithi022 ceph-mon[88798]: from='client.34462 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:39:04.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:03 smithi117 ceph-mon[67870]: pgmap v219: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 10 MiB/s rd, 3 op/s 2024-02-16T19:39:04.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:03 smithi117 ceph-mon[67870]: from='client.34462 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:39:05.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:05 smithi117 ceph-mon[67870]: pgmap v220: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 14 MiB/s rd, 4 op/s 2024-02-16T19:39:05.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:05 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:05.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:05 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:05.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:05 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:39:05.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:05 smithi022 ceph-mon[88798]: pgmap v220: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 14 MiB/s rd, 4 op/s 2024-02-16T19:39:05.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:05 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:05.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:05 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:05.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:05 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:39:06.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:06 smithi022 ceph-mon[88798]: mds.? [v2:172.21.15.117:6826/166927597,v1:172.21.15.117:6827/166927597] up:boot 2024-02-16T19:39:06.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:06 smithi022 ceph-mon[88798]: fsmap cephfs:1/1 {0=cephfs.smithi022.ocukvs=up:replay} 3 up:standby 2024-02-16T19:39:06.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:06 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi117.orzors"}]: dispatch 2024-02-16T19:39:07.111 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:06 smithi117 ceph-mon[67870]: mds.? [v2:172.21.15.117:6826/166927597,v1:172.21.15.117:6827/166927597] up:boot 2024-02-16T19:39:07.111 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:06 smithi117 ceph-mon[67870]: fsmap cephfs:1/1 {0=cephfs.smithi022.ocukvs=up:replay} 3 up:standby 2024-02-16T19:39:07.111 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:06 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi117.orzors"}]: dispatch 2024-02-16T19:39:07.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:07 smithi022 ceph-mon[88798]: pgmap v221: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 22 MiB/s rd, 6 op/s 2024-02-16T19:39:08.002 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:07 smithi117 ceph-mon[67870]: pgmap v221: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 22 MiB/s rd, 6 op/s 2024-02-16T19:39:08.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:08 smithi022 ceph-mon[88798]: reconnect by client.14540 192.168.0.1:0/1593109629 after 0.000999988 2024-02-16T19:39:08.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:08 smithi022 ceph-mon[88798]: reconnect by client.24307 192.168.0.1:0/2099531446 after 0.000999988 2024-02-16T19:39:08.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:08 smithi022 ceph-mon[88798]: mds.? [v2:172.21.15.22:6826/2535680483,v1:172.21.15.22:6827/2535680483] up:reconnect 2024-02-16T19:39:08.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:08 smithi022 ceph-mon[88798]: fsmap cephfs:1/1 {0=cephfs.smithi022.ocukvs=up:reconnect} 3 up:standby 2024-02-16T19:39:08.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:08 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:08.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:08 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:09.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:08 smithi117 ceph-mon[67870]: reconnect by client.14540 192.168.0.1:0/1593109629 after 0.000999988 2024-02-16T19:39:09.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:08 smithi117 ceph-mon[67870]: reconnect by client.24307 192.168.0.1:0/2099531446 after 0.000999988 2024-02-16T19:39:09.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:08 smithi117 ceph-mon[67870]: mds.? [v2:172.21.15.22:6826/2535680483,v1:172.21.15.22:6827/2535680483] up:reconnect 2024-02-16T19:39:09.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:08 smithi117 ceph-mon[67870]: fsmap cephfs:1/1 {0=cephfs.smithi022.ocukvs=up:reconnect} 3 up:standby 2024-02-16T19:39:09.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:08 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:09.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:08 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:10.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:09 smithi117 ceph-mon[67870]: pgmap v222: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 24 MiB/s rd, 7 op/s 2024-02-16T19:39:10.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:09 smithi117 ceph-mon[67870]: mds.? [v2:172.21.15.22:6826/2535680483,v1:172.21.15.22:6827/2535680483] up:rejoin 2024-02-16T19:39:10.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:09 smithi117 ceph-mon[67870]: fsmap cephfs:1/1 {0=cephfs.smithi022.ocukvs=up:rejoin} 3 up:standby 2024-02-16T19:39:10.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:09 smithi117 ceph-mon[67870]: daemon mds.cephfs.smithi022.ocukvs is now active in filesystem cephfs as rank 0 2024-02-16T19:39:10.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:09 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:10.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:09 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:10.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:09 smithi022 ceph-mon[88798]: pgmap v222: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 24 MiB/s rd, 7 op/s 2024-02-16T19:39:10.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:09 smithi022 ceph-mon[88798]: mds.? [v2:172.21.15.22:6826/2535680483,v1:172.21.15.22:6827/2535680483] up:rejoin 2024-02-16T19:39:10.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:09 smithi022 ceph-mon[88798]: fsmap cephfs:1/1 {0=cephfs.smithi022.ocukvs=up:rejoin} 3 up:standby 2024-02-16T19:39:10.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:09 smithi022 ceph-mon[88798]: daemon mds.cephfs.smithi022.ocukvs is now active in filesystem cephfs as rank 0 2024-02-16T19:39:10.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:09 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:10.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:09 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:10.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:10 smithi117 ceph-mon[67870]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-02-16T19:39:10.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:10 smithi117 ceph-mon[67870]: mds.? [v2:172.21.15.22:6826/2535680483,v1:172.21.15.22:6827/2535680483] up:active 2024-02-16T19:39:10.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:10 smithi117 ceph-mon[67870]: fsmap cephfs:1 {0=cephfs.smithi022.ocukvs=up:active} 3 up:standby 2024-02-16T19:39:11.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:10 smithi022 ceph-mon[88798]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-02-16T19:39:11.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:10 smithi022 ceph-mon[88798]: mds.? [v2:172.21.15.22:6826/2535680483,v1:172.21.15.22:6827/2535680483] up:active 2024-02-16T19:39:11.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:10 smithi022 ceph-mon[88798]: fsmap cephfs:1 {0=cephfs.smithi022.ocukvs=up:active} 3 up:standby 2024-02-16T19:39:11.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:11 smithi022 ceph-mon[88798]: pgmap v223: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 85 B/s wr, 6 op/s 2024-02-16T19:39:12.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:11 smithi117 ceph-mon[67870]: pgmap v223: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 85 B/s wr, 6 op/s 2024-02-16T19:39:13.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:13 smithi022 ceph-mon[88798]: pgmap v224: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 18 MiB/s rd, 85 B/s wr, 6 op/s 2024-02-16T19:39:13.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:13 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:13.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:13 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:13.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:13 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:39:13.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:13 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:39:13.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:13 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:13.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:13 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:39:13.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:13 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:39:13.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:13 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:39:13.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:13 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:39:13.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:13 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:39:13.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:13 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:39:13.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:13 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:13.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:13 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi022.btppwm"}]: dispatch 2024-02-16T19:39:13.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:13 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi022.btppwm"}]': finished 2024-02-16T19:39:13.966 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:13 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi022.ocukvs"}]: dispatch 2024-02-16T19:39:13.967 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:13 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi022.ocukvs"}]': finished 2024-02-16T19:39:13.967 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:13 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi117.ibmppn"}]: dispatch 2024-02-16T19:39:13.967 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:13 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi117.ibmppn"}]': finished 2024-02-16T19:39:13.967 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:13 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi117.orzors"}]: dispatch 2024-02-16T19:39:13.967 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:13 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi117.orzors"}]': finished 2024-02-16T19:39:13.967 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:13 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-02-16T19:39:14.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:13 smithi117 ceph-mon[67870]: pgmap v224: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 18 MiB/s rd, 85 B/s wr, 6 op/s 2024-02-16T19:39:14.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:13 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:14.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:13 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:14.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:13 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:39:14.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:13 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:39:14.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:13 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:14.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:13 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:39:14.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:13 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:39:14.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:13 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:39:14.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:13 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:39:14.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:13 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:39:14.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:13 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:39:14.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:13 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:14.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:13 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi022.btppwm"}]: dispatch 2024-02-16T19:39:14.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:13 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi022.btppwm"}]': finished 2024-02-16T19:39:14.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:13 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi022.ocukvs"}]: dispatch 2024-02-16T19:39:14.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:13 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi022.ocukvs"}]': finished 2024-02-16T19:39:14.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:13 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi117.ibmppn"}]: dispatch 2024-02-16T19:39:14.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:13 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi117.ibmppn"}]': finished 2024-02-16T19:39:14.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:13 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi117.orzors"}]: dispatch 2024-02-16T19:39:14.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:13 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi117.orzors"}]': finished 2024-02-16T19:39:14.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:13 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-02-16T19:39:14.911 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:14 smithi022 ceph-mon[88798]: Upgrade: Setting container_image for all mds 2024-02-16T19:39:14.911 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:14 smithi022 ceph-mon[88798]: Upgrade: Setting filesystem cephfs Joinable 2024-02-16T19:39:14.912 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:14 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-02-16T19:39:14.912 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:14 smithi022 ceph-mon[88798]: fsmap cephfs:1 {0=cephfs.smithi022.ocukvs=up:active} 3 up:standby 2024-02-16T19:39:14.912 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:14 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:39:14.912 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:14 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:14.912 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:14 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:39:14.912 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:14 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:14.912 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:14 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:39:15.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:14 smithi117 ceph-mon[67870]: Upgrade: Setting container_image for all mds 2024-02-16T19:39:15.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:14 smithi117 ceph-mon[67870]: Upgrade: Setting filesystem cephfs Joinable 2024-02-16T19:39:15.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:14 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-02-16T19:39:15.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:14 smithi117 ceph-mon[67870]: fsmap cephfs:1 {0=cephfs.smithi022.ocukvs=up:active} 3 up:standby 2024-02-16T19:39:15.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:14 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:39:15.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:14 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:15.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:14 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:39:15.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:14 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:15.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:14 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:39:15.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:15 smithi022 ceph-mon[88798]: pgmap v225: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 13 MiB/s rd, 170 B/s wr, 5 op/s 2024-02-16T19:39:15.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:15 smithi022 ceph-mon[88798]: Upgrade: Setting container_image for all rgw 2024-02-16T19:39:15.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:15 smithi022 ceph-mon[88798]: Upgrade: Setting container_image for all rbd-mirror 2024-02-16T19:39:15.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:15 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:15.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:15 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi022", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-02-16T19:39:15.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:15 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:39:16.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:15 smithi117 ceph-mon[67870]: pgmap v225: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 13 MiB/s rd, 170 B/s wr, 5 op/s 2024-02-16T19:39:16.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:15 smithi117 ceph-mon[67870]: Upgrade: Setting container_image for all rgw 2024-02-16T19:39:16.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:15 smithi117 ceph-mon[67870]: Upgrade: Setting container_image for all rbd-mirror 2024-02-16T19:39:16.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:15 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:16.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:15 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi022", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-02-16T19:39:16.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:15 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:39:16.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:16 smithi022 ceph-mon[88798]: Upgrade: Updating ceph-exporter.smithi022 (1/2) 2024-02-16T19:39:16.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:16 smithi022 ceph-mon[88798]: Deploying daemon ceph-exporter.smithi022 on smithi022 2024-02-16T19:39:16.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:16 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:16.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:16 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:39:17.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:16 smithi117 ceph-mon[67870]: Upgrade: Updating ceph-exporter.smithi022 (1/2) 2024-02-16T19:39:17.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:16 smithi117 ceph-mon[67870]: Deploying daemon ceph-exporter.smithi022 on smithi022 2024-02-16T19:39:17.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:16 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:17.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:16 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:39:17.877 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:17 smithi022 ceph-mon[88798]: pgmap v226: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 9.9 MiB/s rd, 4.3 KiB/s wr, 7 op/s 2024-02-16T19:39:18.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:17 smithi117 ceph-mon[67870]: pgmap v226: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 9.9 MiB/s rd, 4.3 KiB/s wr, 7 op/s 2024-02-16T19:39:20.079 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:19 smithi117 ceph-mon[67870]: pgmap v227: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 3.2 MiB/s rd, 4.3 KiB/s wr, 5 op/s 2024-02-16T19:39:20.079 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:19 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:20.079 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:19 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:20.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:19 smithi022 ceph-mon[88798]: pgmap v227: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 3.2 MiB/s rd, 4.3 KiB/s wr, 5 op/s 2024-02-16T19:39:20.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:19 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:20.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:19 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:21.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:21 smithi117 ceph-mon[67870]: pgmap v228: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 4.3 KiB/s wr, 5 op/s 2024-02-16T19:39:21.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:21 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:21.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:21 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi117", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-02-16T19:39:21.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:21 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:39:22.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:21 smithi022 ceph-mon[88798]: pgmap v228: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 4.3 KiB/s wr, 5 op/s 2024-02-16T19:39:22.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:21 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:22.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:21 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi117", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-02-16T19:39:22.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:21 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:39:22.842 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:22 smithi117 ceph-mon[67870]: Upgrade: Updating ceph-exporter.smithi117 (2/2) 2024-02-16T19:39:22.842 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:22 smithi117 ceph-mon[67870]: Deploying daemon ceph-exporter.smithi117 on smithi117 2024-02-16T19:39:23.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:22 smithi022 ceph-mon[88798]: Upgrade: Updating ceph-exporter.smithi117 (2/2) 2024-02-16T19:39:23.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:22 smithi022 ceph-mon[88798]: Deploying daemon ceph-exporter.smithi117 on smithi117 2024-02-16T19:39:24.031 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:23 smithi117 ceph-mon[67870]: pgmap v229: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 3.4 KiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-02-16T19:39:24.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:23 smithi022 ceph-mon[88798]: pgmap v229: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 3.4 KiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-02-16T19:39:25.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:25 smithi022 ceph-mon[88798]: pgmap v230: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4.2 KiB/s wr, 3 op/s 2024-02-16T19:39:25.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:25 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:25.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:25 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:25.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:25 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:39:25.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:25 smithi117 ceph-mon[67870]: pgmap v230: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4.2 KiB/s wr, 3 op/s 2024-02-16T19:39:25.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:25 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:25.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:25 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:25.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:25 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:39:27.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:27 smithi117 ceph-mon[67870]: pgmap v231: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 938 B/s rd, 4.2 KiB/s wr, 2 op/s 2024-02-16T19:39:27.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:27 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:27.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:27 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:27.946 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:27 smithi022 ceph-mon[88798]: pgmap v231: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 938 B/s rd, 4.2 KiB/s wr, 2 op/s 2024-02-16T19:39:27.947 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:27 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:27.947 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:27 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:29.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:29 smithi117 ceph-mon[67870]: pgmap v232: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:39:29.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:29 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:29.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:29 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:29.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:29 smithi022 ceph-mon[88798]: pgmap v232: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:39:29.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:29 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:29.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:29 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:30.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:30 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:30.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:30 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:30.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:30 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:30.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:30 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:31.365 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:31 smithi117 ceph-mon[67870]: pgmap v233: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:39:31.366 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:31 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:31.366 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:31 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:31.366 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:31 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:39:31.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:31 smithi022 ceph-mon[88798]: pgmap v233: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:39:31.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:31 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:31.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:31 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:31.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:31 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:39:32.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.508+0000 7f3fd47f8700 1 -- 172.21.15.22:0/888022481 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3fcc100e80 msgr2=0x7f3fcc104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:39:32.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.508+0000 7f3fd47f8700 1 --2- 172.21.15.22:0/888022481 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3fcc100e80 0x7f3fcc104e50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3fc0009910 tx=0x7f3fc0009c20 comp rx=0 tx=0).stop 2024-02-16T19:39:32.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.509+0000 7f3fd47f8700 1 -- 172.21.15.22:0/888022481 shutdown_connections 2024-02-16T19:39:32.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.509+0000 7f3fd47f8700 1 --2- 172.21.15.22:0/888022481 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3fcc100e80 0x7f3fcc104e50 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:39:32.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.509+0000 7f3fd47f8700 1 --2- 172.21.15.22:0/888022481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fcc1004e0 0x7f3fcc1008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:39:32.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.509+0000 7f3fd47f8700 1 -- 172.21.15.22:0/888022481 >> 172.21.15.22:0/888022481 conn(0x7f3fcc0fbda0 msgr2=0x7f3fcc0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:39:32.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.509+0000 7f3fd47f8700 1 -- 172.21.15.22:0/888022481 shutdown_connections 2024-02-16T19:39:32.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.509+0000 7f3fd47f8700 1 -- 172.21.15.22:0/888022481 wait complete. 2024-02-16T19:39:32.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.509+0000 7f3fd47f8700 1 Processor -- start 2024-02-16T19:39:32.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.509+0000 7f3fd47f8700 1 -- start start 2024-02-16T19:39:32.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.509+0000 7f3fd47f8700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fcc1004e0 0x7f3fcc195f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:39:32.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.509+0000 7f3fd47f8700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3fcc100e80 0x7f3fcc196450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:39:32.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.509+0000 7f3fd47f8700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fcc196b30 con 0x7f3fcc100e80 2024-02-16T19:39:32.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.509+0000 7f3fd47f8700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fcc19a8c0 con 0x7f3fcc1004e0 2024-02-16T19:39:32.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.510+0000 7f3fd1d93700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3fcc100e80 0x7f3fcc196450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:39:32.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.510+0000 7f3fd1d93700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3fcc100e80 0x7f3fcc196450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50224/0 (socket says 172.21.15.22:50224) 2024-02-16T19:39:32.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.510+0000 7f3fd1d93700 1 -- 172.21.15.22:0/679430653 learned_addr learned my addr 172.21.15.22:0/679430653 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:39:32.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.510+0000 7f3fd2594700 1 --2- 172.21.15.22:0/679430653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fcc1004e0 0x7f3fcc195f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:39:32.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.510+0000 7f3fd1d93700 1 -- 172.21.15.22:0/679430653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fcc1004e0 msgr2=0x7f3fcc195f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:39:32.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.510+0000 7f3fd1d93700 1 --2- 172.21.15.22:0/679430653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fcc1004e0 0x7f3fcc195f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:39:32.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.510+0000 7f3fd1d93700 1 -- 172.21.15.22:0/679430653 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fc00095f0 con 0x7f3fcc100e80 2024-02-16T19:39:32.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.510+0000 7f3fd1d93700 1 --2- 172.21.15.22:0/679430653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3fcc100e80 0x7f3fcc196450 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3fc00092e0 tx=0x7f3fc000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:39:32.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.510+0000 7f3fcb7fe700 1 -- 172.21.15.22:0/679430653 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3fc001d070 con 0x7f3fcc100e80 2024-02-16T19:39:32.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.510+0000 7f3fcb7fe700 1 -- 172.21.15.22:0/679430653 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3fc0005dc0 con 0x7f3fcc100e80 2024-02-16T19:39:32.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.510+0000 7f3fcb7fe700 1 -- 172.21.15.22:0/679430653 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3fc0017400 con 0x7f3fcc100e80 2024-02-16T19:39:32.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.510+0000 7f3fd47f8700 1 -- 172.21.15.22:0/679430653 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fcc19ab40 con 0x7f3fcc100e80 2024-02-16T19:39:32.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.510+0000 7f3fd47f8700 1 -- 172.21.15.22:0/679430653 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fcc19b030 con 0x7f3fcc100e80 2024-02-16T19:39:32.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.511+0000 7f3fd47f8700 1 -- 172.21.15.22:0/679430653 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fcc108850 con 0x7f3fcc100e80 2024-02-16T19:39:32.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.512+0000 7f3fcb7fe700 1 -- 172.21.15.22:0/679430653 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f3fc000f6f0 con 0x7f3fcc100e80 2024-02-16T19:39:32.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.512+0000 7f3fcb7fe700 1 --2- 172.21.15.22:0/679430653 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f3fb406d8a0 0x7f3fb406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:39:32.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.512+0000 7f3fcb7fe700 1 -- 172.21.15.22:0/679430653 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f3fc008e9c0 con 0x7f3fcc100e80 2024-02-16T19:39:32.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.512+0000 7f3fd2594700 1 --2- 172.21.15.22:0/679430653 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f3fb406d8a0 0x7f3fb406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:39:32.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.513+0000 7f3fd2594700 1 --2- 172.21.15.22:0/679430653 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f3fb406d8a0 0x7f3fb406fd50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc009550 tx=0x7f3fbc0091b0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:39:32.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.517+0000 7f3fcb7fe700 1 -- 172.21.15.22:0/679430653 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3fc0058070 con 0x7f3fcc100e80 2024-02-16T19:39:32.618 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:32 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:32.618 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:32 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:32.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.745+0000 7f3fd47f8700 1 -- 172.21.15.22:0/679430653 --> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3fcc1972e0 con 0x7f3fb406d8a0 2024-02-16T19:39:32.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.746+0000 7f3fcb7fe700 1 -- 172.21.15.22:0/679430653 <== mgr.34104 v2:172.21.15.22:6800/505266753 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+466 (secure 0 0 0) 0x7f3fcc1972e0 con 0x7f3fb406d8a0 2024-02-16T19:39:32.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.749+0000 7f3fd47f8700 1 -- 172.21.15.22:0/679430653 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f3fb406d8a0 msgr2=0x7f3fb406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:39:32.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.749+0000 7f3fd47f8700 1 --2- 172.21.15.22:0/679430653 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f3fb406d8a0 0x7f3fb406fd50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc009550 tx=0x7f3fbc0091b0 comp rx=0 tx=0).stop 2024-02-16T19:39:32.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.749+0000 7f3fd47f8700 1 -- 172.21.15.22:0/679430653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3fcc100e80 msgr2=0x7f3fcc196450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:39:32.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.749+0000 7f3fd47f8700 1 --2- 172.21.15.22:0/679430653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3fcc100e80 0x7f3fcc196450 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3fc00092e0 tx=0x7f3fc000bd70 comp rx=0 tx=0).stop 2024-02-16T19:39:32.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.749+0000 7f3fd47f8700 1 -- 172.21.15.22:0/679430653 shutdown_connections 2024-02-16T19:39:32.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.749+0000 7f3fd47f8700 1 --2- 172.21.15.22:0/679430653 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f3fb406d8a0 0x7f3fb406fd50 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:39:32.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.749+0000 7f3fd47f8700 1 --2- 172.21.15.22:0/679430653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fcc1004e0 0x7f3fcc195f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:39:32.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.749+0000 7f3fd47f8700 1 --2- 172.21.15.22:0/679430653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3fcc100e80 0x7f3fcc196450 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:39:32.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.749+0000 7f3fd47f8700 1 -- 172.21.15.22:0/679430653 >> 172.21.15.22:0/679430653 conn(0x7f3fcc0fbda0 msgr2=0x7f3fcc0fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:39:32.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.749+0000 7f3fd47f8700 1 -- 172.21.15.22:0/679430653 shutdown_connections 2024-02-16T19:39:32.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.749+0000 7f3fd47f8700 1 -- 172.21.15.22:0/679430653 wait complete. 2024-02-16T19:39:32.760 INFO:teuthology.orchestra.run.smithi022.stdout:true 2024-02-16T19:39:32.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.857+0000 7fb2ef36e700 1 -- 172.21.15.22:0/3020002665 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2e81004a0 msgr2=0x7fb2e8100870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:39:32.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.857+0000 7fb2ef36e700 1 --2- 172.21.15.22:0/3020002665 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2e81004a0 0x7fb2e8100870 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb2d8009910 tx=0x7fb2d8009c20 comp rx=0 tx=0).stop 2024-02-16T19:39:32.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.857+0000 7fb2ef36e700 1 -- 172.21.15.22:0/3020002665 shutdown_connections 2024-02-16T19:39:32.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.857+0000 7fb2ef36e700 1 --2- 172.21.15.22:0/3020002665 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2e8100e40 0x7fb2e8104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:39:32.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.857+0000 7fb2ef36e700 1 --2- 172.21.15.22:0/3020002665 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2e81004a0 0x7fb2e8100870 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:39:32.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.857+0000 7fb2ef36e700 1 -- 172.21.15.22:0/3020002665 >> 172.21.15.22:0/3020002665 conn(0x7fb2e80fbd40 msgr2=0x7fb2e80fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:39:32.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.857+0000 7fb2ef36e700 1 -- 172.21.15.22:0/3020002665 shutdown_connections 2024-02-16T19:39:32.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.857+0000 7fb2ef36e700 1 -- 172.21.15.22:0/3020002665 wait complete. 2024-02-16T19:39:32.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.857+0000 7fb2ef36e700 1 Processor -- start 2024-02-16T19:39:32.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.858+0000 7fb2ef36e700 1 -- start start 2024-02-16T19:39:32.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.858+0000 7fb2ef36e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2e81004a0 0x7fb2e8073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:39:32.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.858+0000 7fb2ef36e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2e8100e40 0x7fb2e8073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:39:32.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.858+0000 7fb2ef36e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e8073ad0 con 0x7fb2e8100e40 2024-02-16T19:39:32.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.858+0000 7fb2ef36e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e8073c10 con 0x7fb2e81004a0 2024-02-16T19:39:32.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.858+0000 7fb2ec909700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2e8100e40 0x7fb2e8073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:39:32.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.858+0000 7fb2ec909700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2e8100e40 0x7fb2e8073590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50240/0 (socket says 172.21.15.22:50240) 2024-02-16T19:39:32.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.858+0000 7fb2ec909700 1 -- 172.21.15.22:0/442324414 learned_addr learned my addr 172.21.15.22:0/442324414 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:39:32.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.858+0000 7fb2ed10a700 1 --2- 172.21.15.22:0/442324414 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2e81004a0 0x7fb2e8073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:39:32.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.858+0000 7fb2ec909700 1 -- 172.21.15.22:0/442324414 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2e81004a0 msgr2=0x7fb2e8073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:39:32.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.858+0000 7fb2ec909700 1 --2- 172.21.15.22:0/442324414 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2e81004a0 0x7fb2e8073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:39:32.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.858+0000 7fb2ec909700 1 -- 172.21.15.22:0/442324414 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2d80095f0 con 0x7fb2e8100e40 2024-02-16T19:39:32.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.858+0000 7fb2ec909700 1 --2- 172.21.15.22:0/442324414 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2e8100e40 0x7fb2e8073590 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb2e400e8f0 tx=0x7fb2e400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:39:32.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.858+0000 7fb2de7fc700 1 -- 172.21.15.22:0/442324414 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb2e4009790 con 0x7fb2e8100e40 2024-02-16T19:39:32.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.859+0000 7fb2ef36e700 1 -- 172.21.15.22:0/442324414 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2e81a20a0 con 0x7fb2e8100e40 2024-02-16T19:39:32.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.859+0000 7fb2de7fc700 1 -- 172.21.15.22:0/442324414 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb2e4004cf0 con 0x7fb2e8100e40 2024-02-16T19:39:32.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.859+0000 7fb2de7fc700 1 -- 172.21.15.22:0/442324414 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb2e4005690 con 0x7fb2e8100e40 2024-02-16T19:39:32.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.859+0000 7fb2ef36e700 1 -- 172.21.15.22:0/442324414 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2e81a24b0 con 0x7fb2e8100e40 2024-02-16T19:39:32.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.860+0000 7fb2ef36e700 1 -- 172.21.15.22:0/442324414 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2e81087b0 con 0x7fb2e8100e40 2024-02-16T19:39:32.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.860+0000 7fb2de7fc700 1 -- 172.21.15.22:0/442324414 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fb2e4004830 con 0x7fb2e8100e40 2024-02-16T19:39:32.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.861+0000 7fb2de7fc700 1 --2- 172.21.15.22:0/442324414 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fb2d406d8a0 0x7fb2d406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:39:32.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.861+0000 7fb2de7fc700 1 -- 172.21.15.22:0/442324414 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6962+0+0 (secure 0 0 0) 0x7fb2e4014070 con 0x7fb2e8100e40 2024-02-16T19:39:32.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.861+0000 7fb2ed10a700 1 --2- 172.21.15.22:0/442324414 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fb2d406d8a0 0x7fb2d406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:39:32.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.861+0000 7fb2ed10a700 1 --2- 172.21.15.22:0/442324414 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fb2d406d8a0 0x7fb2d406fd50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb2d8009910 tx=0x7fb2d8009f70 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:39:32.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:32.867+0000 7fb2de7fc700 1 -- 172.21.15.22:0/442324414 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb2e40570d0 con 0x7fb2e8100e40 2024-02-16T19:39:32.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:32 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:32.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:32 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:33.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.095+0000 7fb2ef36e700 1 -- 172.21.15.22:0/442324414 --> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb2e8061190 con 0x7fb2d406d8a0 2024-02-16T19:39:33.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.097+0000 7fb2de7fc700 1 -- 172.21.15.22:0/442324414 <== mgr.34104 v2:172.21.15.22:6800/505266753 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+466 (secure 0 0 0) 0x7fb2e8061190 con 0x7fb2d406d8a0 2024-02-16T19:39:33.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.099+0000 7fb2ef36e700 1 -- 172.21.15.22:0/442324414 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fb2d406d8a0 msgr2=0x7fb2d406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:39:33.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.099+0000 7fb2ef36e700 1 --2- 172.21.15.22:0/442324414 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fb2d406d8a0 0x7fb2d406fd50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb2d8009910 tx=0x7fb2d8009f70 comp rx=0 tx=0).stop 2024-02-16T19:39:33.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.099+0000 7fb2ef36e700 1 -- 172.21.15.22:0/442324414 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2e8100e40 msgr2=0x7fb2e8073590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:39:33.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.099+0000 7fb2ef36e700 1 --2- 172.21.15.22:0/442324414 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2e8100e40 0x7fb2e8073590 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb2e400e8f0 tx=0x7fb2e400ecb0 comp rx=0 tx=0).stop 2024-02-16T19:39:33.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.099+0000 7fb2ef36e700 1 -- 172.21.15.22:0/442324414 shutdown_connections 2024-02-16T19:39:33.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.099+0000 7fb2ef36e700 1 --2- 172.21.15.22:0/442324414 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fb2d406d8a0 0x7fb2d406fd50 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:39:33.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.099+0000 7fb2ef36e700 1 --2- 172.21.15.22:0/442324414 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2e81004a0 0x7fb2e8073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:39:33.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.099+0000 7fb2ef36e700 1 --2- 172.21.15.22:0/442324414 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2e8100e40 0x7fb2e8073590 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:39:33.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.099+0000 7fb2ef36e700 1 -- 172.21.15.22:0/442324414 >> 172.21.15.22:0/442324414 conn(0x7fb2e80fbd40 msgr2=0x7fb2e80fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:39:33.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.099+0000 7fb2ef36e700 1 -- 172.21.15.22:0/442324414 shutdown_connections 2024-02-16T19:39:33.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.099+0000 7fb2ef36e700 1 -- 172.21.15.22:0/442324414 wait complete. 2024-02-16T19:39:33.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.208+0000 7f28eb886700 1 -- 172.21.15.22:0/2571492864 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28e41004e0 msgr2=0x7f28e41008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:39:33.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.208+0000 7f28eb886700 1 --2- 172.21.15.22:0/2571492864 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28e41004e0 0x7f28e41008b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f28e0009910 tx=0x7f28e0009c20 comp rx=0 tx=0).stop 2024-02-16T19:39:33.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.208+0000 7f28eb886700 1 -- 172.21.15.22:0/2571492864 shutdown_connections 2024-02-16T19:39:33.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.208+0000 7f28eb886700 1 --2- 172.21.15.22:0/2571492864 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28e4100e80 0x7f28e4104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:39:33.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.208+0000 7f28eb886700 1 --2- 172.21.15.22:0/2571492864 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28e41004e0 0x7f28e41008b0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:39:33.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.208+0000 7f28eb886700 1 -- 172.21.15.22:0/2571492864 >> 172.21.15.22:0/2571492864 conn(0x7f28e40fbda0 msgr2=0x7f28e40fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:39:33.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.208+0000 7f28eb886700 1 -- 172.21.15.22:0/2571492864 shutdown_connections 2024-02-16T19:39:33.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.208+0000 7f28eb886700 1 -- 172.21.15.22:0/2571492864 wait complete. 2024-02-16T19:39:33.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.209+0000 7f28eb886700 1 Processor -- start 2024-02-16T19:39:33.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.209+0000 7f28eb886700 1 -- start start 2024-02-16T19:39:33.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.209+0000 7f28eb886700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28e41004e0 0x7f28e4195ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:39:33.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.209+0000 7f28eb886700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28e4100e80 0x7f28e4196430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:39:33.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.209+0000 7f28eb886700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e4196b10 con 0x7f28e4100e80 2024-02-16T19:39:33.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.209+0000 7f28eb886700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e419a8a0 con 0x7f28e41004e0 2024-02-16T19:39:33.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.210+0000 7f28e8e21700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28e4100e80 0x7f28e4196430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:39:33.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.210+0000 7f28e8e21700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28e4100e80 0x7f28e4196430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50252/0 (socket says 172.21.15.22:50252) 2024-02-16T19:39:33.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.210+0000 7f28e8e21700 1 -- 172.21.15.22:0/3679262737 learned_addr learned my addr 172.21.15.22:0/3679262737 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:39:33.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.210+0000 7f28e9622700 1 --2- 172.21.15.22:0/3679262737 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28e41004e0 0x7f28e4195ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:39:33.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.210+0000 7f28e8e21700 1 -- 172.21.15.22:0/3679262737 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28e41004e0 msgr2=0x7f28e4195ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:39:33.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.210+0000 7f28e8e21700 1 --2- 172.21.15.22:0/3679262737 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28e41004e0 0x7f28e4195ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:39:33.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.210+0000 7f28e8e21700 1 -- 172.21.15.22:0/3679262737 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28e00095f0 con 0x7f28e4100e80 2024-02-16T19:39:33.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.210+0000 7f28e8e21700 1 --2- 172.21.15.22:0/3679262737 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28e4100e80 0x7f28e4196430 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f28d400d900 tx=0x7f28d400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:39:33.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.210+0000 7f28da7fc700 1 -- 172.21.15.22:0/3679262737 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28d4009900 con 0x7f28e4100e80 2024-02-16T19:39:33.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.210+0000 7f28da7fc700 1 -- 172.21.15.22:0/3679262737 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f28d4010460 con 0x7f28e4100e80 2024-02-16T19:39:33.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.210+0000 7f28da7fc700 1 -- 172.21.15.22:0/3679262737 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28d400f5f0 con 0x7f28e4100e80 2024-02-16T19:39:33.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.210+0000 7f28eb886700 1 -- 172.21.15.22:0/3679262737 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28e419ab80 con 0x7f28e4100e80 2024-02-16T19:39:33.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.211+0000 7f28eb886700 1 -- 172.21.15.22:0/3679262737 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28e419b0d0 con 0x7f28e4100e80 2024-02-16T19:39:33.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.212+0000 7f28eb886700 1 -- 172.21.15.22:0/3679262737 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28e41087f0 con 0x7f28e4100e80 2024-02-16T19:39:33.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.212+0000 7f28da7fc700 1 -- 172.21.15.22:0/3679262737 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f28d40105d0 con 0x7f28e4100e80 2024-02-16T19:39:33.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.213+0000 7f28da7fc700 1 --2- 172.21.15.22:0/3679262737 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f28d006d8a0 0x7f28d006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:39:33.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.213+0000 7f28da7fc700 1 -- 172.21.15.22:0/3679262737 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f28d408d150 con 0x7f28e4100e80 2024-02-16T19:39:33.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.213+0000 7f28e9622700 1 --2- 172.21.15.22:0/3679262737 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f28d006d8a0 0x7f28d006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:39:33.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.213+0000 7f28e9622700 1 --2- 172.21.15.22:0/3679262737 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f28d006d8a0 0x7f28d006fd50 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f28e0006010 tx=0x7f28e001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:39:33.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.217+0000 7f28da7fc700 1 -- 172.21.15.22:0/3679262737 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f28d4056800 con 0x7f28e4100e80 2024-02-16T19:39:33.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.438+0000 7f28eb886700 1 -- 172.21.15.22:0/3679262737 --> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f28e419b400 con 0x7f28d006d8a0 2024-02-16T19:39:33.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.450+0000 7f28da7fc700 1 -- 172.21.15.22:0/3679262737 <== mgr.34104 v2:172.21.15.22:6800/505266753 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f28e419b400 con 0x7f28d006d8a0 2024-02-16T19:39:33.451 INFO:teuthology.orchestra.run.smithi022.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T19:39:33.451 INFO:teuthology.orchestra.run.smithi022.stdout:alertmanager.smithi022 smithi022 *:9093,9094 running (7m) 4s ago 18m 26.8M - 0.25.0 c8568f914cd2 c89bc76be374 2024-02-16T19:39:33.451 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-exporter.smithi022 smithi022 running (14s) 4s ago 19m 7260k - 19.0.0-1450-gf78a58c0 f081f847070a e12b859aed67 2024-02-16T19:39:33.451 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-exporter.smithi117 smithi117 running (9s) 6s ago 17m 6924k - 19.0.0-1450-gf78a58c0 f081f847070a b275193d62db 2024-02-16T19:39:33.451 INFO:teuthology.orchestra.run.smithi022.stdout:crash.smithi022 smithi022 running (5m) 4s ago 19m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a e0f78bdf43e4 2024-02-16T19:39:33.451 INFO:teuthology.orchestra.run.smithi022.stdout:crash.smithi117 smithi117 running (5m) 6s ago 17m 6669k - 19.0.0-1450-gf78a58c0 f081f847070a bce7c0590813 2024-02-16T19:39:33.452 INFO:teuthology.orchestra.run.smithi022.stdout:grafana.smithi022 smithi022 *:3000 running (7m) 4s ago 18m 82.2M - 9.4.12 a0ea5e3c46d9 fd27d11e9729 2024-02-16T19:39:33.452 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi022.btppwm smithi022 running (76s) 4s ago 13m 12.9M - 19.0.0-1450-gf78a58c0 f081f847070a 243e0becd1f8 2024-02-16T19:39:33.452 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi022.ocukvs smithi022 running (99s) 4s ago 13m 87.3M - 19.0.0-1450-gf78a58c0 f081f847070a e6ce6a81df63 2024-02-16T19:39:33.452 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi117.ibmppn smithi117 running (49s) 6s ago 13m 12.8M - 19.0.0-1450-gf78a58c0 f081f847070a 05b2e23f1aa1 2024-02-16T19:39:33.452 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi117.orzors smithi117 running (28s) 6s ago 13m 16.0M - 19.0.0-1450-gf78a58c0 f081f847070a 599a29785d0b 2024-02-16T19:39:33.452 INFO:teuthology.orchestra.run.smithi022.stdout:mgr.smithi022.gmdcch smithi022 *:8443,9283,8765 running (9m) 4s ago 20m 510M - 19.0.0-1450-gf78a58c0 f081f847070a 7cb74cc9deea 2024-02-16T19:39:33.452 INFO:teuthology.orchestra.run.smithi022.stdout:mgr.smithi117.cohbtv smithi117 *:8443,9283,8765 running (8m) 6s ago 17m 454M - 19.0.0-1450-gf78a58c0 f081f847070a c825b1152189 2024-02-16T19:39:33.452 INFO:teuthology.orchestra.run.smithi022.stdout:mon.smithi022 smithi022 running (6m) 4s ago 20m 57.5M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 9c84755ae67d 2024-02-16T19:39:33.452 INFO:teuthology.orchestra.run.smithi022.stdout:mon.smithi117 smithi117 running (6m) 6s ago 17m 46.4M 2048M 19.0.0-1450-gf78a58c0 f081f847070a a319ac50f1a9 2024-02-16T19:39:33.452 INFO:teuthology.orchestra.run.smithi022.stdout:node-exporter.smithi022 smithi022 *:9100 running (8m) 4s ago 18m 16.4M - 1.5.0 0da6a335fe13 1f5b58292d27 2024-02-16T19:39:33.452 INFO:teuthology.orchestra.run.smithi022.stdout:node-exporter.smithi117 smithi117 *:9100 running (8m) 6s ago 17m 15.5M - 1.5.0 0da6a335fe13 bb5e8491d5a6 2024-02-16T19:39:33.452 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0 smithi022 running (4m) 4s ago 16m 117M 1217M 19.0.0-1450-gf78a58c0 f081f847070a 002412b3525f 2024-02-16T19:39:33.452 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1 smithi022 running (4m) 4s ago 16m 117M 1217M 19.0.0-1450-gf78a58c0 f081f847070a c3488ce7b2a0 2024-02-16T19:39:33.452 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2 smithi022 running (3m) 4s ago 15m 95.8M 1217M 19.0.0-1450-gf78a58c0 f081f847070a 46138c50f79d 2024-02-16T19:39:33.453 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3 smithi117 running (3m) 6s ago 15m 103M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 28d7d8519e64 2024-02-16T19:39:33.453 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4 smithi117 running (2m) 6s ago 15m 96.2M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 367ba2085f6f 2024-02-16T19:39:33.453 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5 smithi117 running (114s) 6s ago 14m 96.0M 2240M 19.0.0-1450-gf78a58c0 f081f847070a a1a7625b7541 2024-02-16T19:39:33.453 INFO:teuthology.orchestra.run.smithi022.stdout:prometheus.smithi022 smithi022 *:9095 running (8m) 4s ago 18m 69.2M - 2.43.0 a07b618ecd1d c3b8689d0c97 2024-02-16T19:39:33.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.452+0000 7f28eb886700 1 -- 172.21.15.22:0/3679262737 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f28d006d8a0 msgr2=0x7f28d006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:39:33.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.452+0000 7f28eb886700 1 --2- 172.21.15.22:0/3679262737 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f28d006d8a0 0x7f28d006fd50 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f28e0006010 tx=0x7f28e001a040 comp rx=0 tx=0).stop 2024-02-16T19:39:33.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.452+0000 7f28eb886700 1 -- 172.21.15.22:0/3679262737 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28e4100e80 msgr2=0x7f28e4196430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:39:33.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.452+0000 7f28eb886700 1 --2- 172.21.15.22:0/3679262737 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28e4100e80 0x7f28e4196430 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f28d400d900 tx=0x7f28d400dc10 comp rx=0 tx=0).stop 2024-02-16T19:39:33.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.452+0000 7f28eb886700 1 -- 172.21.15.22:0/3679262737 shutdown_connections 2024-02-16T19:39:33.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.452+0000 7f28eb886700 1 --2- 172.21.15.22:0/3679262737 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f28d006d8a0 0x7f28d006fd50 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:39:33.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.452+0000 7f28eb886700 1 --2- 172.21.15.22:0/3679262737 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28e41004e0 0x7f28e4195ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:39:33.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.452+0000 7f28eb886700 1 --2- 172.21.15.22:0/3679262737 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28e4100e80 0x7f28e4196430 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:39:33.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.452+0000 7f28eb886700 1 -- 172.21.15.22:0/3679262737 >> 172.21.15.22:0/3679262737 conn(0x7f28e40fbda0 msgr2=0x7f28e40fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:39:33.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.452+0000 7f28eb886700 1 -- 172.21.15.22:0/3679262737 shutdown_connections 2024-02-16T19:39:33.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.452+0000 7f28eb886700 1 -- 172.21.15.22:0/3679262737 wait complete. 2024-02-16T19:39:33.464 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:33 smithi022 ceph-mon[88798]: pgmap v234: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:39:33.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:33 smithi022 ceph-mon[88798]: from='client.34476 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:39:33.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.559+0000 7f6df3c82700 1 -- 172.21.15.22:0/1527945700 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6dec0ff3c0 msgr2=0x7f6dec0ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:39:33.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.559+0000 7f6df3c82700 1 --2- 172.21.15.22:0/1527945700 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6dec0ff3c0 0x7f6dec0ff790 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6de0009910 tx=0x7f6de0009c20 comp rx=0 tx=0).stop 2024-02-16T19:39:33.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.559+0000 7f6df3c82700 1 -- 172.21.15.22:0/1527945700 shutdown_connections 2024-02-16T19:39:33.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.559+0000 7f6df3c82700 1 --2- 172.21.15.22:0/1527945700 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6dec0ffd60 0x7f6dec103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:39:33.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.559+0000 7f6df3c82700 1 --2- 172.21.15.22:0/1527945700 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6dec0ff3c0 0x7f6dec0ff790 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:39:33.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.559+0000 7f6df3c82700 1 -- 172.21.15.22:0/1527945700 >> 172.21.15.22:0/1527945700 conn(0x7f6dec0fac80 msgr2=0x7f6dec0fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:39:33.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.559+0000 7f6df3c82700 1 -- 172.21.15.22:0/1527945700 shutdown_connections 2024-02-16T19:39:33.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.559+0000 7f6df3c82700 1 -- 172.21.15.22:0/1527945700 wait complete. 2024-02-16T19:39:33.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.560+0000 7f6df3c82700 1 Processor -- start 2024-02-16T19:39:33.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.560+0000 7f6df3c82700 1 -- start start 2024-02-16T19:39:33.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.560+0000 7f6df3c82700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6dec0ff3c0 0x7f6dec195e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:39:33.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.560+0000 7f6df3c82700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6dec0ffd60 0x7f6dec1963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:39:33.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.560+0000 7f6df3c82700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dec196a80 con 0x7f6dec0ffd60 2024-02-16T19:39:33.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.560+0000 7f6df3c82700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dec19a810 con 0x7f6dec0ff3c0 2024-02-16T19:39:33.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.560+0000 7f6df121d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6dec0ffd60 0x7f6dec1963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:39:33.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.560+0000 7f6df121d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6dec0ffd60 0x7f6dec1963a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50268/0 (socket says 172.21.15.22:50268) 2024-02-16T19:39:33.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.560+0000 7f6df121d700 1 -- 172.21.15.22:0/3030923313 learned_addr learned my addr 172.21.15.22:0/3030923313 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:39:33.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.560+0000 7f6df1a1e700 1 --2- 172.21.15.22:0/3030923313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6dec0ff3c0 0x7f6dec195e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:39:33.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.560+0000 7f6df121d700 1 -- 172.21.15.22:0/3030923313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6dec0ff3c0 msgr2=0x7f6dec195e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:39:33.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.560+0000 7f6df121d700 1 --2- 172.21.15.22:0/3030923313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6dec0ff3c0 0x7f6dec195e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:39:33.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.560+0000 7f6df121d700 1 -- 172.21.15.22:0/3030923313 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6de00095f0 con 0x7f6dec0ffd60 2024-02-16T19:39:33.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.561+0000 7f6df121d700 1 --2- 172.21.15.22:0/3030923313 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6dec0ffd60 0x7f6dec1963a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6de800c8f0 tx=0x7f6de800cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:39:33.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.561+0000 7f6ddeffd700 1 -- 172.21.15.22:0/3030923313 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6de80078a0 con 0x7f6dec0ffd60 2024-02-16T19:39:33.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.561+0000 7f6ddeffd700 1 -- 172.21.15.22:0/3030923313 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6de800f450 con 0x7f6dec0ffd60 2024-02-16T19:39:33.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.561+0000 7f6df3c82700 1 -- 172.21.15.22:0/3030923313 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dec19aaf0 con 0x7f6dec0ffd60 2024-02-16T19:39:33.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.561+0000 7f6ddeffd700 1 -- 172.21.15.22:0/3030923313 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6de8018610 con 0x7f6dec0ffd60 2024-02-16T19:39:33.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.561+0000 7f6df3c82700 1 -- 172.21.15.22:0/3030923313 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dec19b040 con 0x7f6dec0ffd60 2024-02-16T19:39:33.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.562+0000 7f6df3c82700 1 -- 172.21.15.22:0/3030923313 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dec04f2a0 con 0x7f6dec0ffd60 2024-02-16T19:39:33.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.562+0000 7f6ddeffd700 1 -- 172.21.15.22:0/3030923313 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f6de800f5c0 con 0x7f6dec0ffd60 2024-02-16T19:39:33.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.563+0000 7f6ddeffd700 1 --2- 172.21.15.22:0/3030923313 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f6dd806d8a0 0x7f6dd806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:39:33.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.563+0000 7f6ddeffd700 1 -- 172.21.15.22:0/3030923313 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f6de808d610 con 0x7f6dec0ffd60 2024-02-16T19:39:33.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.563+0000 7f6df1a1e700 1 --2- 172.21.15.22:0/3030923313 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f6dd806d8a0 0x7f6dd806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:39:33.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.563+0000 7f6df1a1e700 1 --2- 172.21.15.22:0/3030923313 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f6dd806d8a0 0x7f6dd806fd50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6de0006010 tx=0x7f6de001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:39:33.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.567+0000 7f6ddeffd700 1 -- 172.21.15.22:0/3030923313 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6de8056c40 con 0x7f6dec0ffd60 2024-02-16T19:39:33.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.862+0000 7f6df3c82700 1 -- 172.21.15.22:0/3030923313 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6dec19b320 con 0x7f6dec0ffd60 2024-02-16T19:39:33.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.863+0000 7f6ddeffd700 1 -- 172.21.15.22:0/3030923313 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f6de805a300 con 0x7f6dec0ffd60 2024-02-16T19:39:33.865 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-02-16T19:39:33.865 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": { 2024-02-16T19:39:33.865 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T19:39:33.865 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:39:33.865 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": { 2024-02-16T19:39:33.865 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T19:39:33.865 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:39:33.865 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": { 2024-02-16T19:39:33.865 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-16T19:39:33.865 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:39:33.865 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": { 2024-02-16T19:39:33.865 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 4 2024-02-16T19:39:33.865 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:39:33.866 INFO:teuthology.orchestra.run.smithi022.stdout: "overall": { 2024-02-16T19:39:33.866 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 14 2024-02-16T19:39:33.866 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-02-16T19:39:33.866 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-02-16T19:39:33.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.865+0000 7f6df3c82700 1 -- 172.21.15.22:0/3030923313 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f6dd806d8a0 msgr2=0x7f6dd806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:39:33.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.865+0000 7f6df3c82700 1 --2- 172.21.15.22:0/3030923313 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f6dd806d8a0 0x7f6dd806fd50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6de0006010 tx=0x7f6de001a040 comp rx=0 tx=0).stop 2024-02-16T19:39:33.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.865+0000 7f6df3c82700 1 -- 172.21.15.22:0/3030923313 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6dec0ffd60 msgr2=0x7f6dec1963a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:39:33.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.865+0000 7f6df3c82700 1 --2- 172.21.15.22:0/3030923313 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6dec0ffd60 0x7f6dec1963a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6de800c8f0 tx=0x7f6de800cc00 comp rx=0 tx=0).stop 2024-02-16T19:39:33.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.865+0000 7f6df3c82700 1 -- 172.21.15.22:0/3030923313 shutdown_connections 2024-02-16T19:39:33.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.865+0000 7f6df3c82700 1 --2- 172.21.15.22:0/3030923313 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f6dd806d8a0 0x7f6dd806fd50 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:39:33.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.865+0000 7f6df3c82700 1 --2- 172.21.15.22:0/3030923313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6dec0ff3c0 0x7f6dec195e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:39:33.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.865+0000 7f6df3c82700 1 --2- 172.21.15.22:0/3030923313 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6dec0ffd60 0x7f6dec1963a0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:39:33.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.865+0000 7f6df3c82700 1 -- 172.21.15.22:0/3030923313 >> 172.21.15.22:0/3030923313 conn(0x7f6dec0fac80 msgr2=0x7f6dec0fc300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:39:33.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.865+0000 7f6df3c82700 1 -- 172.21.15.22:0/3030923313 shutdown_connections 2024-02-16T19:39:33.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.865+0000 7f6df3c82700 1 -- 172.21.15.22:0/3030923313 wait complete. 2024-02-16T19:39:33.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:33 smithi117 ceph-mon[67870]: pgmap v234: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:39:33.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:33 smithi117 ceph-mon[67870]: from='client.34476 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:39:33.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.971+0000 7f55c2a1e700 1 -- 172.21.15.22:0/3119429509 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55bc0ff3c0 msgr2=0x7f55bc0ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:39:33.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.971+0000 7f55c2a1e700 1 --2- 172.21.15.22:0/3119429509 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55bc0ff3c0 0x7f55bc0ff790 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f55a4009910 tx=0x7f55a4009c20 comp rx=0 tx=0).stop 2024-02-16T19:39:33.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.971+0000 7f55c2a1e700 1 -- 172.21.15.22:0/3119429509 shutdown_connections 2024-02-16T19:39:33.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.971+0000 7f55c2a1e700 1 --2- 172.21.15.22:0/3119429509 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55bc0ffd60 0x7f55bc103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:39:33.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.971+0000 7f55c2a1e700 1 --2- 172.21.15.22:0/3119429509 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55bc0ff3c0 0x7f55bc0ff790 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:39:33.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.971+0000 7f55c2a1e700 1 -- 172.21.15.22:0/3119429509 >> 172.21.15.22:0/3119429509 conn(0x7f55bc0fac80 msgr2=0x7f55bc0fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:39:33.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.971+0000 7f55c2a1e700 1 -- 172.21.15.22:0/3119429509 shutdown_connections 2024-02-16T19:39:33.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.971+0000 7f55c2a1e700 1 -- 172.21.15.22:0/3119429509 wait complete. 2024-02-16T19:39:33.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.972+0000 7f55c2a1e700 1 Processor -- start 2024-02-16T19:39:33.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.972+0000 7f55c2a1e700 1 -- start start 2024-02-16T19:39:33.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.972+0000 7f55c2a1e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55bc0ff3c0 0x7f55bc195f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:39:33.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.972+0000 7f55c2a1e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55bc0ffd60 0x7f55bc1964c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:39:33.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.972+0000 7f55c2a1e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55bc196ba0 con 0x7f55bc0ffd60 2024-02-16T19:39:33.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.972+0000 7f55c2a1e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55bc19a930 con 0x7f55bc0ff3c0 2024-02-16T19:39:33.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.972+0000 7f55bb7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55bc0ffd60 0x7f55bc1964c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:39:33.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.972+0000 7f55bb7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55bc0ffd60 0x7f55bc1964c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50276/0 (socket says 172.21.15.22:50276) 2024-02-16T19:39:33.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.972+0000 7f55bb7fe700 1 -- 172.21.15.22:0/2507824375 learned_addr learned my addr 172.21.15.22:0/2507824375 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:39:33.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.972+0000 7f55bbfff700 1 --2- 172.21.15.22:0/2507824375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55bc0ff3c0 0x7f55bc195f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:39:33.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.973+0000 7f55bb7fe700 1 -- 172.21.15.22:0/2507824375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55bc0ff3c0 msgr2=0x7f55bc195f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:39:33.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.973+0000 7f55bb7fe700 1 --2- 172.21.15.22:0/2507824375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55bc0ff3c0 0x7f55bc195f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:39:33.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.973+0000 7f55bb7fe700 1 -- 172.21.15.22:0/2507824375 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55a40095f0 con 0x7f55bc0ffd60 2024-02-16T19:39:33.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.973+0000 7f55bbfff700 1 --2- 172.21.15.22:0/2507824375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55bc0ff3c0 0x7f55bc195f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T19:39:33.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.973+0000 7f55bb7fe700 1 --2- 172.21.15.22:0/2507824375 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55bc0ffd60 0x7f55bc1964c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f55ac00b480 tx=0x7f55ac00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:39:33.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.973+0000 7f55b97fa700 1 -- 172.21.15.22:0/2507824375 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f55ac0118c0 con 0x7f55bc0ffd60 2024-02-16T19:39:33.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.973+0000 7f55b97fa700 1 -- 172.21.15.22:0/2507824375 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f55ac010380 con 0x7f55bc0ffd60 2024-02-16T19:39:33.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.973+0000 7f55c2a1e700 1 -- 172.21.15.22:0/2507824375 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55bc19ac10 con 0x7f55bc0ffd60 2024-02-16T19:39:33.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.973+0000 7f55b97fa700 1 -- 172.21.15.22:0/2507824375 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f55ac00f4f0 con 0x7f55bc0ffd60 2024-02-16T19:39:33.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.973+0000 7f55c2a1e700 1 -- 172.21.15.22:0/2507824375 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55bc19b160 con 0x7f55bc0ffd60 2024-02-16T19:39:33.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.974+0000 7f55c2a1e700 1 -- 172.21.15.22:0/2507824375 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55bc04f2a0 con 0x7f55bc0ffd60 2024-02-16T19:39:33.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.975+0000 7f55b97fa700 1 -- 172.21.15.22:0/2507824375 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f55ac0104f0 con 0x7f55bc0ffd60 2024-02-16T19:39:33.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.975+0000 7f55b97fa700 1 --2- 172.21.15.22:0/2507824375 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f55a806d8a0 0x7f55a806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:39:33.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.975+0000 7f55b97fa700 1 -- 172.21.15.22:0/2507824375 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f55ac08d220 con 0x7f55bc0ffd60 2024-02-16T19:39:33.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.975+0000 7f55bbfff700 1 --2- 172.21.15.22:0/2507824375 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f55a806d8a0 0x7f55a806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:39:33.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.975+0000 7f55bbfff700 1 --2- 172.21.15.22:0/2507824375 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f55a806d8a0 0x7f55a806fd50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f55a4005100 tx=0x7f55a401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:39:33.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:33.980+0000 7f55b97fa700 1 -- 172.21.15.22:0/2507824375 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f55ac0568d0 con 0x7f55bc0ffd60 2024-02-16T19:39:34.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.249+0000 7f55c2a1e700 1 -- 172.21.15.22:0/2507824375 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f55bc19b430 con 0x7f55bc0ffd60 2024-02-16T19:39:34.251 INFO:teuthology.orchestra.run.smithi022.stdout:e36 2024-02-16T19:39:34.251 INFO:teuthology.orchestra.run.smithi022.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T19:39:34.251 INFO:teuthology.orchestra.run.smithi022.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:39:34.251 INFO:teuthology.orchestra.run.smithi022.stdout:legacy client fscid: 1 2024-02-16T19:39:34.252 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:39:34.252 INFO:teuthology.orchestra.run.smithi022.stdout:Filesystem 'cephfs' (1) 2024-02-16T19:39:34.252 INFO:teuthology.orchestra.run.smithi022.stdout:fs_name cephfs 2024-02-16T19:39:34.252 INFO:teuthology.orchestra.run.smithi022.stdout:epoch 36 2024-02-16T19:39:34.252 INFO:teuthology.orchestra.run.smithi022.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T19:39:34.252 INFO:teuthology.orchestra.run.smithi022.stdout:created 2024-02-16T19:26:17.873644+0000 2024-02-16T19:39:34.252 INFO:teuthology.orchestra.run.smithi022.stdout:modified 2024-02-16T19:39:13.044372+0000 2024-02-16T19:39:34.252 INFO:teuthology.orchestra.run.smithi022.stdout:tableserver 0 2024-02-16T19:39:34.252 INFO:teuthology.orchestra.run.smithi022.stdout:root 0 2024-02-16T19:39:34.252 INFO:teuthology.orchestra.run.smithi022.stdout:session_timeout 60 2024-02-16T19:39:34.252 INFO:teuthology.orchestra.run.smithi022.stdout:session_autoclose 300 2024-02-16T19:39:34.252 INFO:teuthology.orchestra.run.smithi022.stdout:max_file_size 1099511627776 2024-02-16T19:39:34.252 INFO:teuthology.orchestra.run.smithi022.stdout:max_xattr_size 65536 2024-02-16T19:39:34.253 INFO:teuthology.orchestra.run.smithi022.stdout:required_client_features {} 2024-02-16T19:39:34.253 INFO:teuthology.orchestra.run.smithi022.stdout:last_failure 0 2024-02-16T19:39:34.253 INFO:teuthology.orchestra.run.smithi022.stdout:last_failure_osd_epoch 93 2024-02-16T19:39:34.253 INFO:teuthology.orchestra.run.smithi022.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-16T19:39:34.253 INFO:teuthology.orchestra.run.smithi022.stdout:max_mds 1 2024-02-16T19:39:34.253 INFO:teuthology.orchestra.run.smithi022.stdout:in 0 2024-02-16T19:39:34.253 INFO:teuthology.orchestra.run.smithi022.stdout:up {0=34384} 2024-02-16T19:39:34.253 INFO:teuthology.orchestra.run.smithi022.stdout:failed 2024-02-16T19:39:34.253 INFO:teuthology.orchestra.run.smithi022.stdout:damaged 2024-02-16T19:39:34.253 INFO:teuthology.orchestra.run.smithi022.stdout:stopped 2024-02-16T19:39:34.253 INFO:teuthology.orchestra.run.smithi022.stdout:data_pools [3] 2024-02-16T19:39:34.253 INFO:teuthology.orchestra.run.smithi022.stdout:metadata_pool 2 2024-02-16T19:39:34.253 INFO:teuthology.orchestra.run.smithi022.stdout:inline_data enabled 2024-02-16T19:39:34.253 INFO:teuthology.orchestra.run.smithi022.stdout:balancer 2024-02-16T19:39:34.254 INFO:teuthology.orchestra.run.smithi022.stdout:bal_rank_mask -1 2024-02-16T19:39:34.254 INFO:teuthology.orchestra.run.smithi022.stdout:standby_count_wanted 1 2024-02-16T19:39:34.254 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.cephfs.smithi022.ocukvs{0:34384} state up:active seq 22 join_fscid=1 addr [v2:172.21.15.22:6826/2535680483,v1:172.21.15.22:6827/2535680483] compat {c=[1],r=[1],i=[fff]}] 2024-02-16T19:39:34.254 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:39:34.254 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:39:34.254 INFO:teuthology.orchestra.run.smithi022.stdout:Standby daemons: 2024-02-16T19:39:34.254 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:39:34.254 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.cephfs.smithi022.btppwm{-1:34408} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.22:6828/3126701730,v1:172.21.15.22:6829/3126701730] compat {c=[1],r=[1],i=[fff]}] 2024-02-16T19:39:34.254 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.cephfs.smithi117.ibmppn{-1:44299} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6824/2389644474,v1:172.21.15.117:6825/2389644474] compat {c=[1],r=[1],i=[fff]}] 2024-02-16T19:39:34.254 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.cephfs.smithi117.orzors{-1:44325} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6826/166927597,v1:172.21.15.117:6827/166927597] compat {c=[1],r=[1],i=[fff]}] 2024-02-16T19:39:34.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.249+0000 7f55b97fa700 1 -- 172.21.15.22:0/2507824375 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 36 v36) v1 ==== 76+0+1870 (secure 0 0 0) 0x7f55ac059f90 con 0x7f55bc0ffd60 2024-02-16T19:39:34.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.251+0000 7f55c2a1e700 1 -- 172.21.15.22:0/2507824375 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f55a806d8a0 msgr2=0x7f55a806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:39:34.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.251+0000 7f55c2a1e700 1 --2- 172.21.15.22:0/2507824375 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f55a806d8a0 0x7f55a806fd50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f55a4005100 tx=0x7f55a401a040 comp rx=0 tx=0).stop 2024-02-16T19:39:34.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.251+0000 7f55c2a1e700 1 -- 172.21.15.22:0/2507824375 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55bc0ffd60 msgr2=0x7f55bc1964c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:39:34.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.251+0000 7f55c2a1e700 1 --2- 172.21.15.22:0/2507824375 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55bc0ffd60 0x7f55bc1964c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f55ac00b480 tx=0x7f55ac00b840 comp rx=0 tx=0).stop 2024-02-16T19:39:34.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.251+0000 7f55c2a1e700 1 -- 172.21.15.22:0/2507824375 shutdown_connections 2024-02-16T19:39:34.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.251+0000 7f55c2a1e700 1 --2- 172.21.15.22:0/2507824375 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f55a806d8a0 0x7f55a806fd50 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:39:34.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.251+0000 7f55c2a1e700 1 --2- 172.21.15.22:0/2507824375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55bc0ff3c0 0x7f55bc195f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:39:34.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.251+0000 7f55c2a1e700 1 --2- 172.21.15.22:0/2507824375 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55bc0ffd60 0x7f55bc1964c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:39:34.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.252+0000 7f55c2a1e700 1 -- 172.21.15.22:0/2507824375 >> 172.21.15.22:0/2507824375 conn(0x7f55bc0fac80 msgr2=0x7f55bc0fc370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:39:34.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.252+0000 7f55c2a1e700 1 -- 172.21.15.22:0/2507824375 shutdown_connections 2024-02-16T19:39:34.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.252+0000 7f55c2a1e700 1 -- 172.21.15.22:0/2507824375 wait complete. 2024-02-16T19:39:34.256 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 36 2024-02-16T19:39:34.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.358+0000 7fa1bd872700 1 -- 172.21.15.22:0/2591206228 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1b8100e80 msgr2=0x7fa1b8104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:39:34.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.358+0000 7fa1bd872700 1 --2- 172.21.15.22:0/2591206228 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1b8100e80 0x7fa1b8104e50 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa1a0009910 tx=0x7fa1a0009c20 comp rx=0 tx=0).stop 2024-02-16T19:39:34.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.358+0000 7fa1bd872700 1 -- 172.21.15.22:0/2591206228 shutdown_connections 2024-02-16T19:39:34.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.358+0000 7fa1bd872700 1 --2- 172.21.15.22:0/2591206228 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1b8100e80 0x7fa1b8104e50 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:39:34.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.358+0000 7fa1bd872700 1 --2- 172.21.15.22:0/2591206228 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1b81004e0 0x7fa1b81008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:39:34.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.358+0000 7fa1bd872700 1 -- 172.21.15.22:0/2591206228 >> 172.21.15.22:0/2591206228 conn(0x7fa1b80fbda0 msgr2=0x7fa1b80fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:39:34.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.358+0000 7fa1bd872700 1 -- 172.21.15.22:0/2591206228 shutdown_connections 2024-02-16T19:39:34.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.358+0000 7fa1bd872700 1 -- 172.21.15.22:0/2591206228 wait complete. 2024-02-16T19:39:34.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.359+0000 7fa1bd872700 1 Processor -- start 2024-02-16T19:39:34.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.359+0000 7fa1bd872700 1 -- start start 2024-02-16T19:39:34.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.359+0000 7fa1bd872700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1b81004e0 0x7fa1b8196000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:39:34.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.359+0000 7fa1bd872700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1b8100e80 0x7fa1b8196540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:39:34.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.359+0000 7fa1bd872700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b8196c20 con 0x7fa1b8100e80 2024-02-16T19:39:34.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.359+0000 7fa1bd872700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b819a9b0 con 0x7fa1b81004e0 2024-02-16T19:39:34.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.359+0000 7fa1b67fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1b8100e80 0x7fa1b8196540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:39:34.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.359+0000 7fa1b67fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1b8100e80 0x7fa1b8196540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52948/0 (socket says 172.21.15.22:52948) 2024-02-16T19:39:34.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.359+0000 7fa1b67fc700 1 -- 172.21.15.22:0/2680241696 learned_addr learned my addr 172.21.15.22:0/2680241696 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:39:34.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.359+0000 7fa1b6ffd700 1 --2- 172.21.15.22:0/2680241696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1b81004e0 0x7fa1b8196000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:39:34.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.360+0000 7fa1b67fc700 1 -- 172.21.15.22:0/2680241696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1b81004e0 msgr2=0x7fa1b8196000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:39:34.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.360+0000 7fa1b67fc700 1 --2- 172.21.15.22:0/2680241696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1b81004e0 0x7fa1b8196000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:39:34.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.360+0000 7fa1b67fc700 1 -- 172.21.15.22:0/2680241696 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1a00095f0 con 0x7fa1b8100e80 2024-02-16T19:39:34.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.360+0000 7fa1b67fc700 1 --2- 172.21.15.22:0/2680241696 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1b8100e80 0x7fa1b8196540 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa1a00092e0 tx=0x7fa1a000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:39:34.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.360+0000 7fa1bc870700 1 -- 172.21.15.22:0/2680241696 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa1a001d070 con 0x7fa1b8100e80 2024-02-16T19:39:34.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.360+0000 7fa1bc870700 1 -- 172.21.15.22:0/2680241696 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa1a000f460 con 0x7fa1b8100e80 2024-02-16T19:39:34.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.360+0000 7fa1bd872700 1 -- 172.21.15.22:0/2680241696 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1b819ac30 con 0x7fa1b8100e80 2024-02-16T19:39:34.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.360+0000 7fa1bc870700 1 -- 172.21.15.22:0/2680241696 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa1a00175d0 con 0x7fa1b8100e80 2024-02-16T19:39:34.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.360+0000 7fa1bd872700 1 -- 172.21.15.22:0/2680241696 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1b819b120 con 0x7fa1b8100e80 2024-02-16T19:39:34.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.361+0000 7fa1bd872700 1 -- 172.21.15.22:0/2680241696 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1b81087a0 con 0x7fa1b8100e80 2024-02-16T19:39:34.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.361+0000 7fa1bc870700 1 -- 172.21.15.22:0/2680241696 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fa1a0003a60 con 0x7fa1b8100e80 2024-02-16T19:39:34.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.362+0000 7fa1bc870700 1 --2- 172.21.15.22:0/2680241696 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fa1a406d8a0 0x7fa1a406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:39:34.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.362+0000 7fa1bc870700 1 -- 172.21.15.22:0/2680241696 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6962+0+0 (secure 0 0 0) 0x7fa1a008eb50 con 0x7fa1b8100e80 2024-02-16T19:39:34.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.362+0000 7fa1b6ffd700 1 --2- 172.21.15.22:0/2680241696 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fa1a406d8a0 0x7fa1a406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:39:34.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.362+0000 7fa1b6ffd700 1 --2- 172.21.15.22:0/2680241696 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fa1a406d8a0 0x7fa1a406fd50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8009a10 tx=0x7fa1a8009190 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:39:34.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.366+0000 7fa1bc870700 1 -- 172.21.15.22:0/2680241696 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa1a0058180 con 0x7fa1b8100e80 2024-02-16T19:39:34.566 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:34 smithi022 ceph-mon[88798]: from='client.34480 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:39:34.567 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:34 smithi022 ceph-mon[88798]: from='client.34484 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:39:34.567 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:34 smithi022 ceph-mon[88798]: from='client.? 172.21.15.22:0/3030923313' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:39:34.567 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:34 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:34.567 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:34 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:34.567 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:34 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:39:34.567 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:34 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:39:34.567 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:34 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:34.567 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:34 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:39:34.567 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:34 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:39:34.567 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:34 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:39:34.567 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:34 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:39:34.568 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:34 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:39:34.568 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:34 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:39:34.568 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:34 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-02-16T19:39:34.568 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:34 smithi022 ceph-mon[88798]: from='client.? 172.21.15.22:0/2507824375' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:39:34.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.597+0000 7fa1bd872700 1 -- 172.21.15.22:0/2680241696 --> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa1b8197470 con 0x7fa1a406d8a0 2024-02-16T19:39:34.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.598+0000 7fa1bc870700 1 -- 172.21.15.22:0/2680241696 <== mgr.34104 v2:172.21.15.22:6800/505266753 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+466 (secure 0 0 0) 0x7fa1b8197470 con 0x7fa1a406d8a0 2024-02-16T19:39:34.600 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-02-16T19:39:34.600 INFO:teuthology.orchestra.run.smithi022.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T19:39:34.600 INFO:teuthology.orchestra.run.smithi022.stdout: "in_progress": true, 2024-02-16T19:39:34.600 INFO:teuthology.orchestra.run.smithi022.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T19:39:34.600 INFO:teuthology.orchestra.run.smithi022.stdout: "services_complete": [ 2024-02-16T19:39:34.600 INFO:teuthology.orchestra.run.smithi022.stdout: "osd", 2024-02-16T19:39:34.600 INFO:teuthology.orchestra.run.smithi022.stdout: "crash", 2024-02-16T19:39:34.600 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph-exporter", 2024-02-16T19:39:34.600 INFO:teuthology.orchestra.run.smithi022.stdout: "mds", 2024-02-16T19:39:34.600 INFO:teuthology.orchestra.run.smithi022.stdout: "mon", 2024-02-16T19:39:34.600 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr" 2024-02-16T19:39:34.600 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-02-16T19:39:34.600 INFO:teuthology.orchestra.run.smithi022.stdout: "progress": "18/23 daemons upgraded", 2024-02-16T19:39:34.601 INFO:teuthology.orchestra.run.smithi022.stdout: "message": "Currently upgrading ceph-exporter daemons", 2024-02-16T19:39:34.601 INFO:teuthology.orchestra.run.smithi022.stdout: "is_paused": false 2024-02-16T19:39:34.601 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-02-16T19:39:34.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.600+0000 7fa1bd872700 1 -- 172.21.15.22:0/2680241696 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fa1a406d8a0 msgr2=0x7fa1a406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:39:34.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.601+0000 7fa1bd872700 1 --2- 172.21.15.22:0/2680241696 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fa1a406d8a0 0x7fa1a406fd50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8009a10 tx=0x7fa1a8009190 comp rx=0 tx=0).stop 2024-02-16T19:39:34.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.601+0000 7fa1bd872700 1 -- 172.21.15.22:0/2680241696 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1b8100e80 msgr2=0x7fa1b8196540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:39:34.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.601+0000 7fa1bd872700 1 --2- 172.21.15.22:0/2680241696 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1b8100e80 0x7fa1b8196540 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa1a00092e0 tx=0x7fa1a000bd70 comp rx=0 tx=0).stop 2024-02-16T19:39:34.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.601+0000 7fa1bd872700 1 -- 172.21.15.22:0/2680241696 shutdown_connections 2024-02-16T19:39:34.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.601+0000 7fa1bd872700 1 --2- 172.21.15.22:0/2680241696 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fa1a406d8a0 0x7fa1a406fd50 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:39:34.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.601+0000 7fa1bd872700 1 --2- 172.21.15.22:0/2680241696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1b81004e0 0x7fa1b8196000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:39:34.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.601+0000 7fa1bd872700 1 --2- 172.21.15.22:0/2680241696 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1b8100e80 0x7fa1b8196540 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:39:34.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.601+0000 7fa1bd872700 1 -- 172.21.15.22:0/2680241696 >> 172.21.15.22:0/2680241696 conn(0x7fa1b80fbda0 msgr2=0x7fa1b80fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:39:34.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.601+0000 7fa1bd872700 1 -- 172.21.15.22:0/2680241696 shutdown_connections 2024-02-16T19:39:34.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.601+0000 7fa1bd872700 1 -- 172.21.15.22:0/2680241696 wait complete. 2024-02-16T19:39:34.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.706+0000 7f8f64d8f700 1 -- 172.21.15.22:0/868784149 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f600ffd50 msgr2=0x7f8f60103d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:39:34.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.706+0000 7f8f64d8f700 1 --2- 172.21.15.22:0/868784149 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f600ffd50 0x7f8f60103d20 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8f54009910 tx=0x7f8f54009c20 comp rx=0 tx=0).stop 2024-02-16T19:39:34.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.706+0000 7f8f64d8f700 1 -- 172.21.15.22:0/868784149 shutdown_connections 2024-02-16T19:39:34.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.706+0000 7f8f64d8f700 1 --2- 172.21.15.22:0/868784149 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f600ffd50 0x7f8f60103d20 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:39:34.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.706+0000 7f8f64d8f700 1 --2- 172.21.15.22:0/868784149 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f600ff3b0 0x7f8f600ff780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:39:34.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.706+0000 7f8f64d8f700 1 -- 172.21.15.22:0/868784149 >> 172.21.15.22:0/868784149 conn(0x7f8f600fac90 msgr2=0x7f8f600fd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:39:34.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.706+0000 7f8f64d8f700 1 -- 172.21.15.22:0/868784149 shutdown_connections 2024-02-16T19:39:34.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.706+0000 7f8f64d8f700 1 -- 172.21.15.22:0/868784149 wait complete. 2024-02-16T19:39:34.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.707+0000 7f8f64d8f700 1 Processor -- start 2024-02-16T19:39:34.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.707+0000 7f8f64d8f700 1 -- start start 2024-02-16T19:39:34.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.707+0000 7f8f64d8f700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f600ff3b0 0x7f8f60195e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:39:34.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.707+0000 7f8f64d8f700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f600ffd50 0x7f8f601963c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:39:34.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.707+0000 7f8f64d8f700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f60196aa0 con 0x7f8f600ffd50 2024-02-16T19:39:34.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.707+0000 7f8f64d8f700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6019a830 con 0x7f8f600ff3b0 2024-02-16T19:39:34.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.708+0000 7f8f5dd9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f600ffd50 0x7f8f601963c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:39:34.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.708+0000 7f8f5dd9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f600ffd50 0x7f8f601963c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52970/0 (socket says 172.21.15.22:52970) 2024-02-16T19:39:34.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.708+0000 7f8f5dd9b700 1 -- 172.21.15.22:0/573441555 learned_addr learned my addr 172.21.15.22:0/573441555 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:39:34.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.708+0000 7f8f5e59c700 1 --2- 172.21.15.22:0/573441555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f600ff3b0 0x7f8f60195e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:39:34.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.708+0000 7f8f5dd9b700 1 -- 172.21.15.22:0/573441555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f600ff3b0 msgr2=0x7f8f60195e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:39:34.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.708+0000 7f8f5dd9b700 1 --2- 172.21.15.22:0/573441555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f600ff3b0 0x7f8f60195e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:39:34.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.708+0000 7f8f5dd9b700 1 -- 172.21.15.22:0/573441555 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f540095f0 con 0x7f8f600ffd50 2024-02-16T19:39:34.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.708+0000 7f8f5e59c700 1 --2- 172.21.15.22:0/573441555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f600ff3b0 0x7f8f60195e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T19:39:34.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.708+0000 7f8f5dd9b700 1 --2- 172.21.15.22:0/573441555 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f600ffd50 0x7f8f601963c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8f5400bd70 tx=0x7f8f5400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:39:34.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.708+0000 7f8f4f7fe700 1 -- 172.21.15.22:0/573441555 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f5401d070 con 0x7f8f600ffd50 2024-02-16T19:39:34.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.708+0000 7f8f4f7fe700 1 -- 172.21.15.22:0/573441555 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8f54005dc0 con 0x7f8f600ffd50 2024-02-16T19:39:34.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.708+0000 7f8f64d8f700 1 -- 172.21.15.22:0/573441555 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f6019aab0 con 0x7f8f600ffd50 2024-02-16T19:39:34.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.708+0000 7f8f4f7fe700 1 -- 172.21.15.22:0/573441555 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f54017400 con 0x7f8f600ffd50 2024-02-16T19:39:34.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.708+0000 7f8f64d8f700 1 -- 172.21.15.22:0/573441555 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f6019afa0 con 0x7f8f600ffd50 2024-02-16T19:39:34.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.709+0000 7f8f64d8f700 1 -- 172.21.15.22:0/573441555 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f60108730 con 0x7f8f600ffd50 2024-02-16T19:39:34.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.710+0000 7f8f4f7fe700 1 -- 172.21.15.22:0/573441555 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f8f5400f7f0 con 0x7f8f600ffd50 2024-02-16T19:39:34.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.710+0000 7f8f4f7fe700 1 --2- 172.21.15.22:0/573441555 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f8f4806d8a0 0x7f8f4806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:39:34.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.710+0000 7f8f4f7fe700 1 -- 172.21.15.22:0/573441555 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f8f5408ebe0 con 0x7f8f600ffd50 2024-02-16T19:39:34.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.710+0000 7f8f5e59c700 1 --2- 172.21.15.22:0/573441555 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f8f4806d8a0 0x7f8f4806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:39:34.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.711+0000 7f8f5e59c700 1 --2- 172.21.15.22:0/573441555 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f8f4806d8a0 0x7f8f4806fd50 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8f50005950 tx=0x7f8f500058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:39:34.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:34.715+0000 7f8f4f7fe700 1 -- 172.21.15.22:0/573441555 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8f54058210 con 0x7f8f600ffd50 2024-02-16T19:39:34.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:34 smithi117 ceph-mon[67870]: from='client.34480 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:39:34.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:34 smithi117 ceph-mon[67870]: from='client.34484 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:39:34.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:34 smithi117 ceph-mon[67870]: from='client.? 172.21.15.22:0/3030923313' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:39:34.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:34 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:34.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:34 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:34.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:34 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:39:34.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:34 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:39:34.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:34 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:34.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:34 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:39:34.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:34 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:39:34.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:34 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:39:34.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:34 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:39:34.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:34 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:39:34.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:34 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:39:34.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:34 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-02-16T19:39:34.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:34 smithi117 ceph-mon[67870]: from='client.? 172.21.15.22:0/2507824375' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:39:35.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:35.009+0000 7f8f64d8f700 1 -- 172.21.15.22:0/573441555 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f8f60197280 con 0x7f8f600ffd50 2024-02-16T19:39:35.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:35.009+0000 7f8f4f7fe700 1 -- 172.21.15.22:0/573441555 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f8f5405b8d0 con 0x7f8f600ffd50 2024-02-16T19:39:35.011 INFO:teuthology.orchestra.run.smithi022.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-02-16T19:39:35.011 INFO:teuthology.orchestra.run.smithi022.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-02-16T19:39:35.011 INFO:teuthology.orchestra.run.smithi022.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-02-16T19:39:35.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:35.012+0000 7f8f64d8f700 1 -- 172.21.15.22:0/573441555 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f8f4806d8a0 msgr2=0x7f8f4806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:39:35.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:35.012+0000 7f8f64d8f700 1 --2- 172.21.15.22:0/573441555 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f8f4806d8a0 0x7f8f4806fd50 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8f50005950 tx=0x7f8f500058e0 comp rx=0 tx=0).stop 2024-02-16T19:39:35.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:35.012+0000 7f8f64d8f700 1 -- 172.21.15.22:0/573441555 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f600ffd50 msgr2=0x7f8f601963c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:39:35.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:35.012+0000 7f8f64d8f700 1 --2- 172.21.15.22:0/573441555 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f600ffd50 0x7f8f601963c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8f5400bd70 tx=0x7f8f5400be50 comp rx=0 tx=0).stop 2024-02-16T19:39:35.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:35.012+0000 7f8f64d8f700 1 -- 172.21.15.22:0/573441555 shutdown_connections 2024-02-16T19:39:35.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:35.012+0000 7f8f64d8f700 1 --2- 172.21.15.22:0/573441555 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f8f4806d8a0 0x7f8f4806fd50 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:39:35.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:35.012+0000 7f8f64d8f700 1 --2- 172.21.15.22:0/573441555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f600ff3b0 0x7f8f60195e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:39:35.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:35.012+0000 7f8f64d8f700 1 --2- 172.21.15.22:0/573441555 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f600ffd50 0x7f8f601963c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:39:35.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:35.012+0000 7f8f64d8f700 1 -- 172.21.15.22:0/573441555 >> 172.21.15.22:0/573441555 conn(0x7f8f600fac90 msgr2=0x7f8f60069110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:39:35.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:35.012+0000 7f8f64d8f700 1 -- 172.21.15.22:0/573441555 shutdown_connections 2024-02-16T19:39:35.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:39:35.012+0000 7f8f64d8f700 1 -- 172.21.15.22:0/573441555 wait complete. 2024-02-16T19:39:35.716 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:35 smithi022 ceph-mon[88798]: pgmap v235: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:39:35.716 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:35 smithi022 ceph-mon[88798]: Upgrade: Setting filesystem cephfs Joinable 2024-02-16T19:39:35.716 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:35 smithi022 ceph-mon[88798]: from='client.34496 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:39:35.719 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:35 smithi022 ceph-mon[88798]: from='client.? 172.21.15.22:0/573441555' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T19:39:35.719 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:35 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-02-16T19:39:35.719 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:35 smithi022 ceph-mon[88798]: fsmap cephfs:1 {0=cephfs.smithi022.ocukvs=up:active} 3 up:standby 2024-02-16T19:39:35.719 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:35 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:39:35.719 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:35 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:39:35.719 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:35 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:39:35.720 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:35 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:39:35.720 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:35 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:35.720 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:35 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi022"}]: dispatch 2024-02-16T19:39:35.720 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:35 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi022"}]': finished 2024-02-16T19:39:35.720 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:35 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi117"}]: dispatch 2024-02-16T19:39:35.720 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:35 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi117"}]': finished 2024-02-16T19:39:35.720 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:35 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:39:35.720 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:35 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:35.720 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:35 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:39:35.720 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:35 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:35.720 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:35 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:39:35.720 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:35 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:35.720 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:35 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:39:35.721 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:35 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:39:35.721 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:35 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:39:35.721 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:35 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:39:35.721 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:35 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:39:35.721 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:35 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:39:35.721 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:35 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-02-16T19:39:35.721 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:35 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-02-16T19:39:35.721 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:35 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T19:39:35.721 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:35 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-02-16T19:39:35.721 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:35 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-02-16T19:39:35.721 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:35 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-02-16T19:39:35.721 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:35 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-02-16T19:39:35.721 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:35 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-02-16T19:39:35.721 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:35 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-02-16T19:39:35.722 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:35 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-02-16T19:39:35.722 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:35 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-02-16T19:39:35.722 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:35 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-02-16T19:39:35.722 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:35 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-02-16T19:39:35.722 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:35 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-02-16T19:39:35.722 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:35 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T19:39:35.722 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:35 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-02-16T19:39:35.722 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:35 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-02-16T19:39:35.722 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:35 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-02-16T19:39:35.722 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:35 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-02-16T19:39:35.722 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:35 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-02-16T19:39:35.722 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:35 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-02-16T19:39:35.722 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:35 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-02-16T19:39:35.723 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:35 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-02-16T19:39:35.723 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:35 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T19:39:35.723 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:35 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T19:39:35.723 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:35 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T19:39:35.723 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:35 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T19:39:35.723 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:35 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T19:39:35.723 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:35 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T19:39:35.723 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:35 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-02-16T19:39:35.723 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:35 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-02-16T19:39:35.723 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:35 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:39:35.723 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:35 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:39:35.723 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:35 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:39:35.723 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:35 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:35.723 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:35 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:39:35.723 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:35 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:39:35.724 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:35 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:39:35.724 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:35 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:35.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:35 smithi117 ceph-mon[67870]: pgmap v235: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:39:35.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:35 smithi117 ceph-mon[67870]: Upgrade: Setting filesystem cephfs Joinable 2024-02-16T19:39:35.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:35 smithi117 ceph-mon[67870]: from='client.34496 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:39:35.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:35 smithi117 ceph-mon[67870]: from='client.? 172.21.15.22:0/573441555' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T19:39:35.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:35 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-02-16T19:39:35.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:35 smithi117 ceph-mon[67870]: fsmap cephfs:1 {0=cephfs.smithi022.ocukvs=up:active} 3 up:standby 2024-02-16T19:39:35.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:35 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:39:35.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:35 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:39:35.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:35 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:39:35.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:35 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:39:35.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:35 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:35.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:35 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi022"}]: dispatch 2024-02-16T19:39:35.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:35 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi022"}]': finished 2024-02-16T19:39:35.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:35 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi117"}]: dispatch 2024-02-16T19:39:35.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:35 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi117"}]': finished 2024-02-16T19:39:35.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:35 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:39:35.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:35 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:35.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:35 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:39:35.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:35 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:35.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:35 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:39:35.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:35 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:35.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:35 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:39:35.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:35 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:39:35.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:35 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:39:35.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:35 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:39:35.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:35 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:39:35.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:35 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:39:35.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:35 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-02-16T19:39:35.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:35 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-02-16T19:39:35.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:35 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T19:39:35.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:35 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-02-16T19:39:35.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:35 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-02-16T19:39:35.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:35 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-02-16T19:39:35.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:35 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-02-16T19:39:35.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:35 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-02-16T19:39:35.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:35 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-02-16T19:39:35.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:35 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-02-16T19:39:35.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:35 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-02-16T19:39:35.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:35 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-02-16T19:39:35.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:35 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-02-16T19:39:35.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:35 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-02-16T19:39:35.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:35 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T19:39:35.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:35 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-02-16T19:39:35.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:35 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-02-16T19:39:35.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:35 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-02-16T19:39:35.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:35 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-02-16T19:39:35.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:35 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-02-16T19:39:35.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:35 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-02-16T19:39:35.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:35 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-02-16T19:39:35.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:35 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-02-16T19:39:35.892 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:35 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T19:39:35.892 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:35 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T19:39:35.892 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:35 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T19:39:35.892 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:35 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T19:39:35.892 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:35 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T19:39:35.892 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:35 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T19:39:35.892 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:35 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-02-16T19:39:35.892 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:35 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-02-16T19:39:35.892 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:35 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:39:35.892 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:35 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:39:35.892 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:35 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:39:35.892 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:35 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:35.892 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:35 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:39:35.892 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:35 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:39:35.893 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:35 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:39:35.893 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:35 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:36.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:36 smithi022 ceph-mon[88798]: Upgrade: Setting container_image for all ceph-exporter 2024-02-16T19:39:36.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:36 smithi022 ceph-mon[88798]: Upgrade: Setting container_image for all iscsi 2024-02-16T19:39:36.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:36 smithi022 ceph-mon[88798]: Upgrade: Setting container_image for all nfs 2024-02-16T19:39:36.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:36 smithi022 ceph-mon[88798]: Upgrade: Setting container_image for all nvmeof 2024-02-16T19:39:36.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:36 smithi022 ceph-mon[88798]: Upgrade: Finalizing container_image settings 2024-02-16T19:39:36.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:36 smithi022 ceph-mon[88798]: Upgrade: Complete! 2024-02-16T19:39:36.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:36 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:36.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:36 smithi117 ceph-mon[67870]: Upgrade: Setting container_image for all ceph-exporter 2024-02-16T19:39:36.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:36 smithi117 ceph-mon[67870]: Upgrade: Setting container_image for all iscsi 2024-02-16T19:39:36.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:36 smithi117 ceph-mon[67870]: Upgrade: Setting container_image for all nfs 2024-02-16T19:39:36.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:36 smithi117 ceph-mon[67870]: Upgrade: Setting container_image for all nvmeof 2024-02-16T19:39:36.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:36 smithi117 ceph-mon[67870]: Upgrade: Finalizing container_image settings 2024-02-16T19:39:36.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:36 smithi117 ceph-mon[67870]: Upgrade: Complete! 2024-02-16T19:39:36.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:36 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:39:37.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:37 smithi022 ceph-mon[88798]: pgmap v236: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:39:37.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:37 smithi117 ceph-mon[67870]: pgmap v236: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:39:39.690 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:39 smithi117 ceph-mon[67870]: pgmap v237: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:39:39.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:39 smithi022 ceph-mon[88798]: pgmap v237: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:39:41.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:41 smithi022 ceph-mon[88798]: pgmap v238: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:39:41.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:41 smithi117 ceph-mon[67870]: pgmap v238: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:39:43.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:43 smithi022 ceph-mon[88798]: pgmap v239: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:39:43.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:43 smithi117 ceph-mon[67870]: pgmap v239: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:39:45.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:45 smithi022 ceph-mon[88798]: pgmap v240: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:39:45.888 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:45 smithi117 ceph-mon[67870]: pgmap v240: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:39:46.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:46 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:39:46.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:46 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:39:47.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:47 smithi022 ceph-mon[88798]: pgmap v241: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:39:47.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:47 smithi117 ceph-mon[67870]: pgmap v241: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:39:49.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:49 smithi022 ceph-mon[88798]: pgmap v242: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:39:49.787 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:49 smithi117 ceph-mon[67870]: pgmap v242: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:39:51.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:51 smithi022 ceph-mon[88798]: pgmap v243: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:39:51.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:51 smithi117 ceph-mon[67870]: pgmap v243: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:39:53.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:53 smithi022 ceph-mon[88798]: pgmap v244: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:39:53.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:53 smithi117 ceph-mon[67870]: pgmap v244: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:39:55.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:55 smithi022 ceph-mon[88798]: pgmap v245: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:39:55.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:55 smithi117 ceph-mon[67870]: pgmap v245: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:39:57.714 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:57 smithi022 ceph-mon[88798]: pgmap v246: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:39:57.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:57 smithi117 ceph-mon[67870]: pgmap v246: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:39:59.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:39:59 smithi022 ceph-mon[88798]: pgmap v247: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:39:59.885 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:39:59 smithi117 ceph-mon[67870]: pgmap v247: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:40:00.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:40:00 smithi022 ceph-mon[88798]: overall HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-02-16T19:40:00.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:40:00 smithi117 ceph-mon[67870]: overall HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-02-16T19:40:01.767 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:40:01 smithi022 ceph-mon[88798]: pgmap v248: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:40:01.768 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:40:01 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:40:01.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:40:01 smithi117 ceph-mon[67870]: pgmap v248: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:40:01.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:40:01 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:40:03.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:40:03 smithi117 ceph-mon[67870]: pgmap v249: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:40:03.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:40:03 smithi022 ceph-mon[88798]: pgmap v249: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:40:05.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:05.131+0000 7fa67681e700 1 -- 172.21.15.22:0/1087559475 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6700ff3c0 msgr2=0x7fa6700ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:40:05.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:05.131+0000 7fa67681e700 1 --2- 172.21.15.22:0/1087559475 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6700ff3c0 0x7fa6700ff790 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa660009910 tx=0x7fa660009c20 comp rx=0 tx=0).stop 2024-02-16T19:40:05.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:05.131+0000 7fa67681e700 1 -- 172.21.15.22:0/1087559475 shutdown_connections 2024-02-16T19:40:05.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:05.131+0000 7fa67681e700 1 --2- 172.21.15.22:0/1087559475 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6700ffd60 0x7fa670103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:40:05.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:05.131+0000 7fa67681e700 1 --2- 172.21.15.22:0/1087559475 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6700ff3c0 0x7fa6700ff790 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:40:05.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:05.131+0000 7fa67681e700 1 -- 172.21.15.22:0/1087559475 >> 172.21.15.22:0/1087559475 conn(0x7fa6700fac80 msgr2=0x7fa6700fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:40:05.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:05.132+0000 7fa67681e700 1 -- 172.21.15.22:0/1087559475 shutdown_connections 2024-02-16T19:40:05.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:05.132+0000 7fa67681e700 1 -- 172.21.15.22:0/1087559475 wait complete. 2024-02-16T19:40:05.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:05.132+0000 7fa67681e700 1 Processor -- start 2024-02-16T19:40:05.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:05.132+0000 7fa67681e700 1 -- start start 2024-02-16T19:40:05.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:05.133+0000 7fa67681e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6700ff3c0 0x7fa670195ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:40:05.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:05.133+0000 7fa67681e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6700ffd60 0x7fa670196430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:40:05.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:05.133+0000 7fa67681e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa670196b10 con 0x7fa6700ff3c0 2024-02-16T19:40:05.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:05.133+0000 7fa67681e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa67019a8a0 con 0x7fa6700ffd60 2024-02-16T19:40:05.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:05.133+0000 7fa66ffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6700ff3c0 0x7fa670195ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:40:05.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:05.133+0000 7fa66ffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6700ff3c0 0x7fa670195ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36714/0 (socket says 172.21.15.22:36714) 2024-02-16T19:40:05.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:05.133+0000 7fa66ffff700 1 -- 172.21.15.22:0/4252026758 learned_addr learned my addr 172.21.15.22:0/4252026758 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:40:05.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:05.133+0000 7fa66f7fe700 1 --2- 172.21.15.22:0/4252026758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6700ffd60 0x7fa670196430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:40:05.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:05.133+0000 7fa66ffff700 1 -- 172.21.15.22:0/4252026758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6700ffd60 msgr2=0x7fa670196430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:40:05.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:05.133+0000 7fa66ffff700 1 --2- 172.21.15.22:0/4252026758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6700ffd60 0x7fa670196430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:40:05.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:05.133+0000 7fa66ffff700 1 -- 172.21.15.22:0/4252026758 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6600095f0 con 0x7fa6700ff3c0 2024-02-16T19:40:05.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:05.133+0000 7fa66ffff700 1 --2- 172.21.15.22:0/4252026758 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6700ff3c0 0x7fa670195ef0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa660006010 tx=0x7fa66000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:40:05.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:05.133+0000 7fa66d7fa700 1 -- 172.21.15.22:0/4252026758 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa66001d070 con 0x7fa6700ff3c0 2024-02-16T19:40:05.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:05.134+0000 7fa66d7fa700 1 -- 172.21.15.22:0/4252026758 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa660004fb0 con 0x7fa6700ff3c0 2024-02-16T19:40:05.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:05.134+0000 7fa67681e700 1 -- 172.21.15.22:0/4252026758 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa67019ab20 con 0x7fa6700ff3c0 2024-02-16T19:40:05.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:05.134+0000 7fa66d7fa700 1 -- 172.21.15.22:0/4252026758 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa660022ea0 con 0x7fa6700ff3c0 2024-02-16T19:40:05.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:05.134+0000 7fa67681e700 1 -- 172.21.15.22:0/4252026758 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa67019b010 con 0x7fa6700ff3c0 2024-02-16T19:40:05.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:05.135+0000 7fa67681e700 1 -- 172.21.15.22:0/4252026758 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa67004f2a0 con 0x7fa6700ff3c0 2024-02-16T19:40:05.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:05.135+0000 7fa66d7fa700 1 -- 172.21.15.22:0/4252026758 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fa660004af0 con 0x7fa6700ff3c0 2024-02-16T19:40:05.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:05.135+0000 7fa66d7fa700 1 --2- 172.21.15.22:0/4252026758 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fa65806d8a0 0x7fa65806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:40:05.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:05.136+0000 7fa66d7fa700 1 -- 172.21.15.22:0/4252026758 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6962+0+0 (secure 0 0 0) 0x7fa66008ee60 con 0x7fa6700ff3c0 2024-02-16T19:40:05.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:05.136+0000 7fa66f7fe700 1 --2- 172.21.15.22:0/4252026758 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fa65806d8a0 0x7fa65806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:40:05.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:05.136+0000 7fa66f7fe700 1 --2- 172.21.15.22:0/4252026758 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fa65806d8a0 0x7fa65806fd50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa670197510 tx=0x7fa664005cf0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:40:05.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:05.140+0000 7fa66d7fa700 1 -- 172.21.15.22:0/4252026758 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa660058660 con 0x7fa6700ff3c0 2024-02-16T19:40:05.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:05.368+0000 7fa67681e700 1 -- 172.21.15.22:0/4252026758 --> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa67019b2d0 con 0x7fa65806d8a0 2024-02-16T19:40:05.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:05.369+0000 7fa66d7fa700 1 -- 172.21.15.22:0/4252026758 <== mgr.34104 v2:172.21.15.22:6800/505266753 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7fa67019b2d0 con 0x7fa65806d8a0 2024-02-16T19:40:05.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:05.371+0000 7fa67681e700 1 -- 172.21.15.22:0/4252026758 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fa65806d8a0 msgr2=0x7fa65806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:40:05.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:05.371+0000 7fa67681e700 1 --2- 172.21.15.22:0/4252026758 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fa65806d8a0 0x7fa65806fd50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa670197510 tx=0x7fa664005cf0 comp rx=0 tx=0).stop 2024-02-16T19:40:05.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:05.371+0000 7fa67681e700 1 -- 172.21.15.22:0/4252026758 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6700ff3c0 msgr2=0x7fa670195ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:40:05.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:05.371+0000 7fa67681e700 1 --2- 172.21.15.22:0/4252026758 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6700ff3c0 0x7fa670195ef0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa660006010 tx=0x7fa66000be20 comp rx=0 tx=0).stop 2024-02-16T19:40:05.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:05.372+0000 7fa67681e700 1 -- 172.21.15.22:0/4252026758 shutdown_connections 2024-02-16T19:40:05.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:05.372+0000 7fa67681e700 1 --2- 172.21.15.22:0/4252026758 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fa65806d8a0 0x7fa65806fd50 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:40:05.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:05.372+0000 7fa67681e700 1 --2- 172.21.15.22:0/4252026758 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6700ff3c0 0x7fa670195ef0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:40:05.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:05.372+0000 7fa67681e700 1 --2- 172.21.15.22:0/4252026758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6700ffd60 0x7fa670196430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:40:05.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:05.372+0000 7fa67681e700 1 -- 172.21.15.22:0/4252026758 >> 172.21.15.22:0/4252026758 conn(0x7fa6700fac80 msgr2=0x7fa6701034b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:40:05.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:05.372+0000 7fa67681e700 1 -- 172.21.15.22:0/4252026758 shutdown_connections 2024-02-16T19:40:05.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:05.372+0000 7fa67681e700 1 -- 172.21.15.22:0/4252026758 wait complete. 2024-02-16T19:40:05.709 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:40:05 smithi022 ceph-mon[88798]: pgmap v250: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:40:05.835 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:40:05 smithi117 ceph-mon[67870]: pgmap v250: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:40:06.077 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph orch ps' 2024-02-16T19:40:06.296 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi022/config 2024-02-16T19:40:06.698 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:40:06 smithi022 ceph-mon[88798]: from='client.34504 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:40:06.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:40:06 smithi117 ceph-mon[67870]: from='client.34504 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:40:07.686 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:40:07 smithi022 ceph-mon[88798]: pgmap v251: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:40:07.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:40:07 smithi117 ceph-mon[67870]: pgmap v251: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:40:07.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:07.927+0000 7fb02af67700 1 -- 172.21.15.22:0/3868741563 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb024101420 msgr2=0x7fb0241017f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:40:07.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:07.927+0000 7fb02af67700 1 --2- 172.21.15.22:0/3868741563 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb024101420 0x7fb0241017f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb01c00b0a0 tx=0x7fb01c00b3b0 comp rx=0 tx=0).stop 2024-02-16T19:40:07.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:07.927+0000 7fb02af67700 1 -- 172.21.15.22:0/3868741563 shutdown_connections 2024-02-16T19:40:07.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:07.927+0000 7fb02af67700 1 --2- 172.21.15.22:0/3868741563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb024101d30 0x7fb02410a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:40:07.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:07.927+0000 7fb02af67700 1 --2- 172.21.15.22:0/3868741563 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb024101420 0x7fb0241017f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:40:07.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:07.927+0000 7fb02af67700 1 -- 172.21.15.22:0/3868741563 >> 172.21.15.22:0/3868741563 conn(0x7fb0240fac80 msgr2=0x7fb0240fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:40:07.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:07.927+0000 7fb02af67700 1 -- 172.21.15.22:0/3868741563 shutdown_connections 2024-02-16T19:40:07.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:07.927+0000 7fb02af67700 1 -- 172.21.15.22:0/3868741563 wait complete. 2024-02-16T19:40:07.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:07.928+0000 7fb02af67700 1 Processor -- start 2024-02-16T19:40:07.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:07.928+0000 7fb02af67700 1 -- start start 2024-02-16T19:40:07.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:07.928+0000 7fb02af67700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb024101420 0x7fb024195e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:40:07.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:07.928+0000 7fb02af67700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb024101d30 0x7fb0241963b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:40:07.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:07.928+0000 7fb02af67700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb024196a90 con 0x7fb024101d30 2024-02-16T19:40:07.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:07.928+0000 7fb02af67700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb02419a820 con 0x7fb024101420 2024-02-16T19:40:07.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:07.928+0000 7fb023fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb024101d30 0x7fb0241963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:40:07.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:07.928+0000 7fb023fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb024101d30 0x7fb0241963b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36736/0 (socket says 172.21.15.22:36736) 2024-02-16T19:40:07.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:07.928+0000 7fb023fff700 1 -- 172.21.15.22:0/4006885818 learned_addr learned my addr 172.21.15.22:0/4006885818 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:40:07.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:07.928+0000 7fb028d03700 1 --2- 172.21.15.22:0/4006885818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb024101420 0x7fb024195e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:40:07.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:07.928+0000 7fb023fff700 1 -- 172.21.15.22:0/4006885818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb024101420 msgr2=0x7fb024195e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:40:07.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:07.928+0000 7fb023fff700 1 --2- 172.21.15.22:0/4006885818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb024101420 0x7fb024195e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:40:07.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:07.928+0000 7fb023fff700 1 -- 172.21.15.22:0/4006885818 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb01c009d00 con 0x7fb024101d30 2024-02-16T19:40:07.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:07.928+0000 7fb023fff700 1 --2- 172.21.15.22:0/4006885818 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb024101d30 0x7fb0241963b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb01800d8d0 tx=0x7fb01800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:40:07.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:07.929+0000 7fb021ffb700 1 -- 172.21.15.22:0/4006885818 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb018009940 con 0x7fb024101d30 2024-02-16T19:40:07.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:07.929+0000 7fb02af67700 1 -- 172.21.15.22:0/4006885818 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb02419ab00 con 0x7fb024101d30 2024-02-16T19:40:07.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:07.929+0000 7fb021ffb700 1 -- 172.21.15.22:0/4006885818 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb018010460 con 0x7fb024101d30 2024-02-16T19:40:07.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:07.929+0000 7fb021ffb700 1 -- 172.21.15.22:0/4006885818 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb01800f5f0 con 0x7fb024101d30 2024-02-16T19:40:07.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:07.929+0000 7fb02af67700 1 -- 172.21.15.22:0/4006885818 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb02419b050 con 0x7fb024101d30 2024-02-16T19:40:07.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:07.930+0000 7fb02af67700 1 -- 172.21.15.22:0/4006885818 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb024107900 con 0x7fb024101d30 2024-02-16T19:40:07.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:07.930+0000 7fb021ffb700 1 -- 172.21.15.22:0/4006885818 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fb0180105d0 con 0x7fb024101d30 2024-02-16T19:40:07.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:07.930+0000 7fb021ffb700 1 --2- 172.21.15.22:0/4006885818 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fb00c06d8a0 0x7fb00c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:40:07.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:07.931+0000 7fb021ffb700 1 -- 172.21.15.22:0/4006885818 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6962+0+0 (secure 0 0 0) 0x7fb01808d6c0 con 0x7fb024101d30 2024-02-16T19:40:07.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:07.931+0000 7fb028d03700 1 --2- 172.21.15.22:0/4006885818 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fb00c06d8a0 0x7fb00c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:40:07.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:07.931+0000 7fb028d03700 1 --2- 172.21.15.22:0/4006885818 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fb00c06d8a0 0x7fb00c06fd50 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb01c00b830 tx=0x7fb01c0061f0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:40:07.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:07.935+0000 7fb021ffb700 1 -- 172.21.15.22:0/4006885818 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb018056cf0 con 0x7fb024101d30 2024-02-16T19:40:08.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:08.153+0000 7fb02af67700 1 -- 172.21.15.22:0/4006885818 --> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb02419b2e0 con 0x7fb00c06d8a0 2024-02-16T19:40:08.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:08.164+0000 7fb021ffb700 1 -- 172.21.15.22:0/4006885818 <== mgr.34104 v2:172.21.15.22:6800/505266753 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fb02419b2e0 con 0x7fb00c06d8a0 2024-02-16T19:40:08.164 INFO:teuthology.orchestra.run.smithi022.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T19:40:08.164 INFO:teuthology.orchestra.run.smithi022.stdout:alertmanager.smithi022 smithi022 *:9093,9094 running (8m) 38s ago 19m 26.8M - 0.25.0 c8568f914cd2 c89bc76be374 2024-02-16T19:40:08.164 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-exporter.smithi022 smithi022 running (49s) 38s ago 19m 7260k - 19.0.0-1450-gf78a58c0 f081f847070a e12b859aed67 2024-02-16T19:40:08.164 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-exporter.smithi117 smithi117 running (44s) 41s ago 18m 6924k - 19.0.0-1450-gf78a58c0 f081f847070a b275193d62db 2024-02-16T19:40:08.165 INFO:teuthology.orchestra.run.smithi022.stdout:crash.smithi022 smithi022 running (5m) 38s ago 19m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a e0f78bdf43e4 2024-02-16T19:40:08.165 INFO:teuthology.orchestra.run.smithi022.stdout:crash.smithi117 smithi117 running (5m) 41s ago 18m 6669k - 19.0.0-1450-gf78a58c0 f081f847070a bce7c0590813 2024-02-16T19:40:08.165 INFO:teuthology.orchestra.run.smithi022.stdout:grafana.smithi022 smithi022 *:3000 running (8m) 38s ago 19m 82.2M - 9.4.12 a0ea5e3c46d9 fd27d11e9729 2024-02-16T19:40:08.165 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi022.btppwm smithi022 running (110s) 38s ago 13m 12.9M - 19.0.0-1450-gf78a58c0 f081f847070a 243e0becd1f8 2024-02-16T19:40:08.165 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi022.ocukvs smithi022 running (2m) 38s ago 13m 87.3M - 19.0.0-1450-gf78a58c0 f081f847070a e6ce6a81df63 2024-02-16T19:40:08.165 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi117.ibmppn smithi117 running (84s) 41s ago 13m 12.8M - 19.0.0-1450-gf78a58c0 f081f847070a 05b2e23f1aa1 2024-02-16T19:40:08.165 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi117.orzors smithi117 running (63s) 41s ago 13m 16.0M - 19.0.0-1450-gf78a58c0 f081f847070a 599a29785d0b 2024-02-16T19:40:08.165 INFO:teuthology.orchestra.run.smithi022.stdout:mgr.smithi022.gmdcch smithi022 *:8443,9283,8765 running (10m) 38s ago 20m 510M - 19.0.0-1450-gf78a58c0 f081f847070a 7cb74cc9deea 2024-02-16T19:40:08.165 INFO:teuthology.orchestra.run.smithi022.stdout:mgr.smithi117.cohbtv smithi117 *:8443,9283,8765 running (9m) 41s ago 18m 454M - 19.0.0-1450-gf78a58c0 f081f847070a c825b1152189 2024-02-16T19:40:08.165 INFO:teuthology.orchestra.run.smithi022.stdout:mon.smithi022 smithi022 running (7m) 38s ago 20m 57.5M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 9c84755ae67d 2024-02-16T19:40:08.165 INFO:teuthology.orchestra.run.smithi022.stdout:mon.smithi117 smithi117 running (6m) 41s ago 18m 46.4M 2048M 19.0.0-1450-gf78a58c0 f081f847070a a319ac50f1a9 2024-02-16T19:40:08.165 INFO:teuthology.orchestra.run.smithi022.stdout:node-exporter.smithi022 smithi022 *:9100 running (9m) 38s ago 19m 16.4M - 1.5.0 0da6a335fe13 1f5b58292d27 2024-02-16T19:40:08.165 INFO:teuthology.orchestra.run.smithi022.stdout:node-exporter.smithi117 smithi117 *:9100 running (9m) 41s ago 18m 15.5M - 1.5.0 0da6a335fe13 bb5e8491d5a6 2024-02-16T19:40:08.165 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0 smithi022 running (5m) 38s ago 17m 117M 1217M 19.0.0-1450-gf78a58c0 f081f847070a 002412b3525f 2024-02-16T19:40:08.166 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1 smithi022 running (4m) 38s ago 16m 117M 1217M 19.0.0-1450-gf78a58c0 f081f847070a c3488ce7b2a0 2024-02-16T19:40:08.166 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2 smithi022 running (4m) 38s ago 16m 95.8M 1217M 19.0.0-1450-gf78a58c0 f081f847070a 46138c50f79d 2024-02-16T19:40:08.166 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3 smithi117 running (3m) 41s ago 16m 103M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 28d7d8519e64 2024-02-16T19:40:08.166 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4 smithi117 running (3m) 41s ago 15m 96.2M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 367ba2085f6f 2024-02-16T19:40:08.166 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5 smithi117 running (2m) 41s ago 15m 96.0M 2240M 19.0.0-1450-gf78a58c0 f081f847070a a1a7625b7541 2024-02-16T19:40:08.166 INFO:teuthology.orchestra.run.smithi022.stdout:prometheus.smithi022 smithi022 *:9095 running (8m) 38s ago 18m 69.2M - 2.43.0 a07b618ecd1d c3b8689d0c97 2024-02-16T19:40:08.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:08.166+0000 7fb02af67700 1 -- 172.21.15.22:0/4006885818 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fb00c06d8a0 msgr2=0x7fb00c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:40:08.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:08.166+0000 7fb02af67700 1 --2- 172.21.15.22:0/4006885818 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fb00c06d8a0 0x7fb00c06fd50 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb01c00b830 tx=0x7fb01c0061f0 comp rx=0 tx=0).stop 2024-02-16T19:40:08.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:08.166+0000 7fb02af67700 1 -- 172.21.15.22:0/4006885818 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb024101d30 msgr2=0x7fb0241963b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:40:08.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:08.166+0000 7fb02af67700 1 --2- 172.21.15.22:0/4006885818 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb024101d30 0x7fb0241963b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb01800d8d0 tx=0x7fb01800dc90 comp rx=0 tx=0).stop 2024-02-16T19:40:08.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:08.166+0000 7fb02af67700 1 -- 172.21.15.22:0/4006885818 shutdown_connections 2024-02-16T19:40:08.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:08.166+0000 7fb02af67700 1 --2- 172.21.15.22:0/4006885818 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fb00c06d8a0 0x7fb00c06fd50 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:40:08.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:08.166+0000 7fb02af67700 1 --2- 172.21.15.22:0/4006885818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb024101420 0x7fb024195e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:40:08.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:08.166+0000 7fb02af67700 1 --2- 172.21.15.22:0/4006885818 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb024101d30 0x7fb0241963b0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:40:08.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:08.166+0000 7fb02af67700 1 -- 172.21.15.22:0/4006885818 >> 172.21.15.22:0/4006885818 conn(0x7fb0240fac80 msgr2=0x7fb024104a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:40:08.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:08.166+0000 7fb02af67700 1 -- 172.21.15.22:0/4006885818 shutdown_connections 2024-02-16T19:40:08.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:08.166+0000 7fb02af67700 1 -- 172.21.15.22:0/4006885818 wait complete. 2024-02-16T19:40:08.789 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph orch upgrade status' 2024-02-16T19:40:09.000 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi022/config 2024-02-16T19:40:09.855 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:40:09 smithi022 ceph-mon[88798]: pgmap v252: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:40:09.855 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:40:09 smithi022 ceph-mon[88798]: from='client.34508 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:40:09.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:40:09 smithi117 ceph-mon[67870]: pgmap v252: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:40:09.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:40:09 smithi117 ceph-mon[67870]: from='client.34508 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:40:10.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:10.706+0000 7ff4667ac700 1 -- 172.21.15.22:0/3664130621 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4600ffc60 msgr2=0x7ff46010a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:40:10.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:10.706+0000 7ff4667ac700 1 --2- 172.21.15.22:0/3664130621 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4600ffc60 0x7ff46010a6d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff454009910 tx=0x7ff454009c20 comp rx=0 tx=0).stop 2024-02-16T19:40:10.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:10.706+0000 7ff4667ac700 1 -- 172.21.15.22:0/3664130621 shutdown_connections 2024-02-16T19:40:10.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:10.706+0000 7ff4667ac700 1 --2- 172.21.15.22:0/3664130621 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4600ffc60 0x7ff46010a6d0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:40:10.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:10.706+0000 7ff4667ac700 1 --2- 172.21.15.22:0/3664130621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4600ff350 0x7ff4600ff720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:40:10.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:10.706+0000 7ff4667ac700 1 -- 172.21.15.22:0/3664130621 >> 172.21.15.22:0/3664130621 conn(0x7ff4600748b0 msgr2=0x7ff460074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:40:10.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:10.706+0000 7ff4667ac700 1 -- 172.21.15.22:0/3664130621 shutdown_connections 2024-02-16T19:40:10.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:10.706+0000 7ff4667ac700 1 -- 172.21.15.22:0/3664130621 wait complete. 2024-02-16T19:40:10.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:10.707+0000 7ff4667ac700 1 Processor -- start 2024-02-16T19:40:10.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:10.707+0000 7ff4667ac700 1 -- start start 2024-02-16T19:40:10.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:10.707+0000 7ff4667ac700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4600ff350 0x7ff460100530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:40:10.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:10.707+0000 7ff4667ac700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4600ffc60 0x7ff460100a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:40:10.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:10.707+0000 7ff4667ac700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff460100fb0 con 0x7ff4600ffc60 2024-02-16T19:40:10.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:10.707+0000 7ff4667ac700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4601010f0 con 0x7ff4600ff350 2024-02-16T19:40:10.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:10.707+0000 7ff45f7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4600ffc60 0x7ff460100a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:40:10.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:10.707+0000 7ff45f7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4600ffc60 0x7ff460100a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36756/0 (socket says 172.21.15.22:36756) 2024-02-16T19:40:10.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:10.707+0000 7ff45f7fe700 1 -- 172.21.15.22:0/383489516 learned_addr learned my addr 172.21.15.22:0/383489516 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:40:10.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:10.708+0000 7ff45ffff700 1 --2- 172.21.15.22:0/383489516 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4600ff350 0x7ff460100530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:40:10.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:10.708+0000 7ff45f7fe700 1 -- 172.21.15.22:0/383489516 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4600ff350 msgr2=0x7ff460100530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:40:10.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:10.708+0000 7ff45f7fe700 1 --2- 172.21.15.22:0/383489516 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4600ff350 0x7ff460100530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:40:10.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:10.708+0000 7ff45f7fe700 1 -- 172.21.15.22:0/383489516 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4540095f0 con 0x7ff4600ffc60 2024-02-16T19:40:10.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:10.708+0000 7ff45ffff700 1 --2- 172.21.15.22:0/383489516 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4600ff350 0x7ff460100530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T19:40:10.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:10.708+0000 7ff45f7fe700 1 --2- 172.21.15.22:0/383489516 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4600ffc60 0x7ff460100a70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff454005f50 tx=0x7ff45400f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:40:10.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:10.708+0000 7ff45d7fa700 1 -- 172.21.15.22:0/383489516 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff45401d070 con 0x7ff4600ffc60 2024-02-16T19:40:10.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:10.708+0000 7ff45d7fa700 1 -- 172.21.15.22:0/383489516 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff45400b610 con 0x7ff4600ffc60 2024-02-16T19:40:10.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:10.708+0000 7ff4667ac700 1 -- 172.21.15.22:0/383489516 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff460101370 con 0x7ff4600ffc60 2024-02-16T19:40:10.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:10.708+0000 7ff45d7fa700 1 -- 172.21.15.22:0/383489516 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff454022470 con 0x7ff4600ffc60 2024-02-16T19:40:10.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:10.708+0000 7ff4667ac700 1 -- 172.21.15.22:0/383489516 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4601a23b0 con 0x7ff4600ffc60 2024-02-16T19:40:10.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:10.709+0000 7ff4667ac700 1 -- 172.21.15.22:0/383489516 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff460107dd0 con 0x7ff4600ffc60 2024-02-16T19:40:10.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:10.710+0000 7ff45d7fa700 1 -- 172.21.15.22:0/383489516 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7ff45400fca0 con 0x7ff4600ffc60 2024-02-16T19:40:10.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:10.710+0000 7ff45d7fa700 1 --2- 172.21.15.22:0/383489516 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7ff44806d8a0 0x7ff44806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:40:10.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:10.710+0000 7ff45d7fa700 1 -- 172.21.15.22:0/383489516 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6962+0+0 (secure 0 0 0) 0x7ff45408ea80 con 0x7ff4600ffc60 2024-02-16T19:40:10.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:10.710+0000 7ff45ffff700 1 --2- 172.21.15.22:0/383489516 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7ff44806d8a0 0x7ff44806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:40:10.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:10.710+0000 7ff45ffff700 1 --2- 172.21.15.22:0/383489516 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7ff44806d8a0 0x7ff44806fd50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff46006a640 tx=0x7ff450009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:40:10.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:10.715+0000 7ff45d7fa700 1 -- 172.21.15.22:0/383489516 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff454058130 con 0x7ff4600ffc60 2024-02-16T19:40:10.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:10.944+0000 7ff4667ac700 1 -- 172.21.15.22:0/383489516 --> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff460061190 con 0x7ff44806d8a0 2024-02-16T19:40:10.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:10.945+0000 7ff45d7fa700 1 -- 172.21.15.22:0/383489516 <== mgr.34104 v2:172.21.15.22:6800/505266753 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7ff460061190 con 0x7ff44806d8a0 2024-02-16T19:40:10.946 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-02-16T19:40:10.946 INFO:teuthology.orchestra.run.smithi022.stdout: "target_image": null, 2024-02-16T19:40:10.946 INFO:teuthology.orchestra.run.smithi022.stdout: "in_progress": false, 2024-02-16T19:40:10.946 INFO:teuthology.orchestra.run.smithi022.stdout: "which": "", 2024-02-16T19:40:10.946 INFO:teuthology.orchestra.run.smithi022.stdout: "services_complete": [], 2024-02-16T19:40:10.946 INFO:teuthology.orchestra.run.smithi022.stdout: "progress": null, 2024-02-16T19:40:10.946 INFO:teuthology.orchestra.run.smithi022.stdout: "message": "", 2024-02-16T19:40:10.946 INFO:teuthology.orchestra.run.smithi022.stdout: "is_paused": false 2024-02-16T19:40:10.946 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-02-16T19:40:10.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:10.947+0000 7ff4667ac700 1 -- 172.21.15.22:0/383489516 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7ff44806d8a0 msgr2=0x7ff44806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:40:10.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:10.947+0000 7ff4667ac700 1 --2- 172.21.15.22:0/383489516 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7ff44806d8a0 0x7ff44806fd50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff46006a640 tx=0x7ff450009260 comp rx=0 tx=0).stop 2024-02-16T19:40:10.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:10.947+0000 7ff4667ac700 1 -- 172.21.15.22:0/383489516 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4600ffc60 msgr2=0x7ff460100a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:40:10.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:10.947+0000 7ff4667ac700 1 --2- 172.21.15.22:0/383489516 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4600ffc60 0x7ff460100a70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff454005f50 tx=0x7ff45400f740 comp rx=0 tx=0).stop 2024-02-16T19:40:10.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:10.948+0000 7ff4667ac700 1 -- 172.21.15.22:0/383489516 shutdown_connections 2024-02-16T19:40:10.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:10.948+0000 7ff4667ac700 1 --2- 172.21.15.22:0/383489516 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7ff44806d8a0 0x7ff44806fd50 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:40:10.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:10.948+0000 7ff4667ac700 1 --2- 172.21.15.22:0/383489516 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4600ff350 0x7ff460100530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:40:10.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:10.948+0000 7ff4667ac700 1 --2- 172.21.15.22:0/383489516 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4600ffc60 0x7ff460100a70 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:40:10.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:10.948+0000 7ff4667ac700 1 -- 172.21.15.22:0/383489516 >> 172.21.15.22:0/383489516 conn(0x7ff4600748b0 msgr2=0x7ff460104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:40:10.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:10.948+0000 7ff4667ac700 1 -- 172.21.15.22:0/383489516 shutdown_connections 2024-02-16T19:40:10.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:10.948+0000 7ff4667ac700 1 -- 172.21.15.22:0/383489516 wait complete. 2024-02-16T19:40:11.648 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:40:11 smithi022 ceph-mon[88798]: pgmap v253: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:40:11.674 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph health detail' 2024-02-16T19:40:11.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:40:11 smithi117 ceph-mon[67870]: pgmap v253: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:40:11.894 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi022/config 2024-02-16T19:40:12.705 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:40:12 smithi022 ceph-mon[88798]: from='client.34512 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:40:12.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:40:12 smithi117 ceph-mon[67870]: from='client.34512 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:40:13.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:13.588+0000 7fec0a0c8700 1 -- 172.21.15.22:0/939507657 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec040ff3c0 msgr2=0x7fec040ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:40:13.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:13.588+0000 7fec0a0c8700 1 --2- 172.21.15.22:0/939507657 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec040ff3c0 0x7fec040ff790 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7febf4009910 tx=0x7febf4009c20 comp rx=0 tx=0).stop 2024-02-16T19:40:13.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:13.588+0000 7fec0a0c8700 1 -- 172.21.15.22:0/939507657 shutdown_connections 2024-02-16T19:40:13.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:13.588+0000 7fec0a0c8700 1 --2- 172.21.15.22:0/939507657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec040ffd60 0x7fec04103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:40:13.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:13.588+0000 7fec0a0c8700 1 --2- 172.21.15.22:0/939507657 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec040ff3c0 0x7fec040ff790 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:40:13.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:13.588+0000 7fec0a0c8700 1 -- 172.21.15.22:0/939507657 >> 172.21.15.22:0/939507657 conn(0x7fec040fac60 msgr2=0x7fec040fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:40:13.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:13.588+0000 7fec0a0c8700 1 -- 172.21.15.22:0/939507657 shutdown_connections 2024-02-16T19:40:13.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:13.588+0000 7fec0a0c8700 1 -- 172.21.15.22:0/939507657 wait complete. 2024-02-16T19:40:13.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:13.588+0000 7fec0a0c8700 1 Processor -- start 2024-02-16T19:40:13.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:13.589+0000 7fec0a0c8700 1 -- start start 2024-02-16T19:40:13.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:13.589+0000 7fec0a0c8700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec040ff3c0 0x7fec04196da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:40:13.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:13.589+0000 7fec0a0c8700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec040ffd60 0x7fec041972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:40:13.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:13.589+0000 7fec0a0c8700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec041979c0 con 0x7fec040ff3c0 2024-02-16T19:40:13.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:13.589+0000 7fec0a0c8700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec04197b00 con 0x7fec040ffd60 2024-02-16T19:40:13.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:13.589+0000 7fec037fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec040ff3c0 0x7fec04196da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:40:13.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:13.589+0000 7fec037fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec040ff3c0 0x7fec04196da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36770/0 (socket says 172.21.15.22:36770) 2024-02-16T19:40:13.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:13.589+0000 7fec037fe700 1 -- 172.21.15.22:0/154055897 learned_addr learned my addr 172.21.15.22:0/154055897 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:40:13.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:13.589+0000 7fec02ffd700 1 --2- 172.21.15.22:0/154055897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec040ffd60 0x7fec041972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:40:13.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:13.589+0000 7fec037fe700 1 -- 172.21.15.22:0/154055897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec040ffd60 msgr2=0x7fec041972e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:40:13.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:13.589+0000 7fec037fe700 1 --2- 172.21.15.22:0/154055897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec040ffd60 0x7fec041972e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:40:13.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:13.589+0000 7fec037fe700 1 -- 172.21.15.22:0/154055897 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febf40095f0 con 0x7fec040ff3c0 2024-02-16T19:40:13.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:13.589+0000 7fec02ffd700 1 --2- 172.21.15.22:0/154055897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec040ffd60 0x7fec041972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-16T19:40:13.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:13.589+0000 7fec037fe700 1 --2- 172.21.15.22:0/154055897 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec040ff3c0 0x7fec04196da0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7febf4006010 tx=0x7febf400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:40:13.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:13.590+0000 7fec00ff9700 1 -- 172.21.15.22:0/154055897 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7febf401d070 con 0x7fec040ff3c0 2024-02-16T19:40:13.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:13.590+0000 7fec0a0c8700 1 -- 172.21.15.22:0/154055897 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec0419fe10 con 0x7fec040ff3c0 2024-02-16T19:40:13.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:13.590+0000 7fec00ff9700 1 -- 172.21.15.22:0/154055897 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7febf4004fb0 con 0x7fec040ff3c0 2024-02-16T19:40:13.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:13.590+0000 7fec00ff9700 1 -- 172.21.15.22:0/154055897 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7febf400f690 con 0x7fec040ff3c0 2024-02-16T19:40:13.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:13.590+0000 7fec0a0c8700 1 -- 172.21.15.22:0/154055897 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec041a0280 con 0x7fec040ff3c0 2024-02-16T19:40:13.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:13.591+0000 7fec0a0c8700 1 -- 172.21.15.22:0/154055897 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec041076d0 con 0x7fec040ff3c0 2024-02-16T19:40:13.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:13.591+0000 7fec00ff9700 1 -- 172.21.15.22:0/154055897 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7febf40228a0 con 0x7fec040ff3c0 2024-02-16T19:40:13.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:13.592+0000 7fec00ff9700 1 --2- 172.21.15.22:0/154055897 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7febec06d8a0 0x7febec06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:40:13.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:13.592+0000 7fec00ff9700 1 -- 172.21.15.22:0/154055897 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6962+0+0 (secure 0 0 0) 0x7febf408ef50 con 0x7fec040ff3c0 2024-02-16T19:40:13.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:13.592+0000 7fec02ffd700 1 --2- 172.21.15.22:0/154055897 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7febec06d8a0 0x7febec06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:40:13.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:13.592+0000 7fec02ffd700 1 --2- 172.21.15.22:0/154055897 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7febec06d8a0 0x7febec06fd50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fec04069eb0 tx=0x7febf8008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:40:13.594 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:40:13 smithi022 ceph-mon[88798]: pgmap v254: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:40:13.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:13.597+0000 7fec00ff9700 1 -- 172.21.15.22:0/154055897 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7febf40c2d20 con 0x7fec040ff3c0 2024-02-16T19:40:13.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:40:13 smithi117 ceph-mon[67870]: pgmap v254: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:40:13.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:13.887+0000 7fec0a0c8700 1 -- 172.21.15.22:0/154055897 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fec041070b0 con 0x7fec040ff3c0 2024-02-16T19:40:13.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:13.888+0000 7fec00ff9700 1 -- 172.21.15.22:0/154055897 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7febf4027090 con 0x7fec040ff3c0 2024-02-16T19:40:13.888 INFO:teuthology.orchestra.run.smithi022.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-02-16T19:40:13.888 INFO:teuthology.orchestra.run.smithi022.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-02-16T19:40:13.888 INFO:teuthology.orchestra.run.smithi022.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-02-16T19:40:13.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:13.890+0000 7fec0a0c8700 1 -- 172.21.15.22:0/154055897 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7febec06d8a0 msgr2=0x7febec06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:40:13.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:13.890+0000 7fec0a0c8700 1 --2- 172.21.15.22:0/154055897 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7febec06d8a0 0x7febec06fd50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fec04069eb0 tx=0x7febf8008040 comp rx=0 tx=0).stop 2024-02-16T19:40:13.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:13.890+0000 7fec0a0c8700 1 -- 172.21.15.22:0/154055897 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec040ff3c0 msgr2=0x7fec04196da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:40:13.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:13.890+0000 7fec0a0c8700 1 --2- 172.21.15.22:0/154055897 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec040ff3c0 0x7fec04196da0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7febf4006010 tx=0x7febf400be20 comp rx=0 tx=0).stop 2024-02-16T19:40:13.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:13.890+0000 7fec0a0c8700 1 -- 172.21.15.22:0/154055897 shutdown_connections 2024-02-16T19:40:13.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:13.890+0000 7fec0a0c8700 1 --2- 172.21.15.22:0/154055897 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7febec06d8a0 0x7febec06fd50 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:40:13.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:13.890+0000 7fec0a0c8700 1 --2- 172.21.15.22:0/154055897 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec040ff3c0 0x7fec04196da0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:40:13.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:13.890+0000 7fec0a0c8700 1 --2- 172.21.15.22:0/154055897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec040ffd60 0x7fec041972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:40:13.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:13.890+0000 7fec0a0c8700 1 -- 172.21.15.22:0/154055897 >> 172.21.15.22:0/154055897 conn(0x7fec040fac60 msgr2=0x7fec040fc350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:40:13.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:13.890+0000 7fec0a0c8700 1 -- 172.21.15.22:0/154055897 shutdown_connections 2024-02-16T19:40:13.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:13.890+0000 7fec0a0c8700 1 -- 172.21.15.22:0/154055897 wait complete. 2024-02-16T19:40:14.639 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph versions' 2024-02-16T19:40:14.858 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi022/config 2024-02-16T19:40:14.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:40:14 smithi117 ceph-mon[67870]: from='client.? 172.21.15.22:0/154055897' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T19:40:14.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:40:14 smithi022 ceph-mon[88798]: from='client.? 172.21.15.22:0/154055897' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T19:40:15.578 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:40:15 smithi022 ceph-mon[88798]: pgmap v255: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:40:15.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:40:15 smithi117 ceph-mon[67870]: pgmap v255: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:40:16.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:16.392+0000 7f94671ce700 1 -- 172.21.15.22:0/572391678 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9460068540 msgr2=0x7f94600689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:40:16.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:16.392+0000 7f94671ce700 1 --2- 172.21.15.22:0/572391678 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9460068540 0x7f94600689b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9454009910 tx=0x7f9454009c20 comp rx=0 tx=0).stop 2024-02-16T19:40:16.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:16.392+0000 7f94671ce700 1 -- 172.21.15.22:0/572391678 shutdown_connections 2024-02-16T19:40:16.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:16.392+0000 7f94671ce700 1 --2- 172.21.15.22:0/572391678 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9460068540 0x7f94600689b0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:40:16.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:16.392+0000 7f94671ce700 1 --2- 172.21.15.22:0/572391678 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9460105230 0x7f9460105600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:40:16.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:16.392+0000 7f94671ce700 1 -- 172.21.15.22:0/572391678 >> 172.21.15.22:0/572391678 conn(0x7f9460075130 msgr2=0x7f9460075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:40:16.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:16.392+0000 7f94671ce700 1 -- 172.21.15.22:0/572391678 shutdown_connections 2024-02-16T19:40:16.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:16.392+0000 7f94671ce700 1 -- 172.21.15.22:0/572391678 wait complete. 2024-02-16T19:40:16.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:16.393+0000 7f94671ce700 1 Processor -- start 2024-02-16T19:40:16.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:16.393+0000 7f94671ce700 1 -- start start 2024-02-16T19:40:16.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:16.393+0000 7f94671ce700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9460068540 0x7f94600ffca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:40:16.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:16.393+0000 7f94671ce700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9460105230 0x7f94601001e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:40:16.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:16.393+0000 7f94671ce700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9460100720 con 0x7f9460105230 2024-02-16T19:40:16.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:16.393+0000 7f94671ce700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9460100860 con 0x7f9460068540 2024-02-16T19:40:16.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:16.393+0000 7f945ffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9460105230 0x7f94601001e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:40:16.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:16.393+0000 7f945ffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9460105230 0x7f94601001e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33320/0 (socket says 172.21.15.22:33320) 2024-02-16T19:40:16.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:16.393+0000 7f945ffff700 1 -- 172.21.15.22:0/1499955486 learned_addr learned my addr 172.21.15.22:0/1499955486 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:40:16.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:16.393+0000 7f9464f6a700 1 --2- 172.21.15.22:0/1499955486 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9460068540 0x7f94600ffca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:40:16.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:16.393+0000 7f945ffff700 1 -- 172.21.15.22:0/1499955486 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9460068540 msgr2=0x7f94600ffca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:40:16.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:16.393+0000 7f945ffff700 1 --2- 172.21.15.22:0/1499955486 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9460068540 0x7f94600ffca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:40:16.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:16.393+0000 7f945ffff700 1 -- 172.21.15.22:0/1499955486 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94540095f0 con 0x7f9460105230 2024-02-16T19:40:16.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:16.393+0000 7f9464f6a700 1 --2- 172.21.15.22:0/1499955486 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9460068540 0x7f94600ffca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-16T19:40:16.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:16.393+0000 7f945ffff700 1 --2- 172.21.15.22:0/1499955486 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9460105230 0x7f94601001e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f94540092e0 tx=0x7f945400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:40:16.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:16.394+0000 7f945dffb700 1 -- 172.21.15.22:0/1499955486 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f945401d070 con 0x7f9460105230 2024-02-16T19:40:16.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:16.394+0000 7f945dffb700 1 -- 172.21.15.22:0/1499955486 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9454005e00 con 0x7f9460105230 2024-02-16T19:40:16.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:16.394+0000 7f94671ce700 1 -- 172.21.15.22:0/1499955486 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9460100ae0 con 0x7f9460105230 2024-02-16T19:40:16.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:16.394+0000 7f945dffb700 1 -- 172.21.15.22:0/1499955486 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9454017400 con 0x7f9460105230 2024-02-16T19:40:16.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:16.394+0000 7f94671ce700 1 -- 172.21.15.22:0/1499955486 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94601a22c0 con 0x7f9460105230 2024-02-16T19:40:16.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:16.395+0000 7f94671ce700 1 -- 172.21.15.22:0/1499955486 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9460108750 con 0x7f9460105230 2024-02-16T19:40:16.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:16.395+0000 7f945dffb700 1 -- 172.21.15.22:0/1499955486 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f945400f460 con 0x7f9460105230 2024-02-16T19:40:16.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:16.396+0000 7f945dffb700 1 --2- 172.21.15.22:0/1499955486 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f944806d8a0 0x7f944806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:40:16.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:16.396+0000 7f945dffb700 1 -- 172.21.15.22:0/1499955486 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f945408ebc0 con 0x7f9460105230 2024-02-16T19:40:16.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:16.396+0000 7f9464f6a700 1 --2- 172.21.15.22:0/1499955486 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f944806d8a0 0x7f944806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:40:16.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:16.396+0000 7f9464f6a700 1 --2- 172.21.15.22:0/1499955486 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f944806d8a0 0x7f944806fd50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f94580094f0 tx=0x7f9458009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:40:16.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:16.400+0000 7f945dffb700 1 -- 172.21.15.22:0/1499955486 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f94540581f0 con 0x7f9460105230 2024-02-16T19:40:16.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:16.694+0000 7f94671ce700 1 -- 172.21.15.22:0/1499955486 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f946004f2a0 con 0x7f9460105230 2024-02-16T19:40:16.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:16.695+0000 7f945dffb700 1 -- 172.21.15.22:0/1499955486 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f945405b8b0 con 0x7f9460105230 2024-02-16T19:40:16.695 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-02-16T19:40:16.695 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": { 2024-02-16T19:40:16.695 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T19:40:16.695 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:40:16.695 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": { 2024-02-16T19:40:16.696 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T19:40:16.696 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:40:16.696 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": { 2024-02-16T19:40:16.696 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-16T19:40:16.696 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:40:16.696 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": { 2024-02-16T19:40:16.696 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 4 2024-02-16T19:40:16.696 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:40:16.696 INFO:teuthology.orchestra.run.smithi022.stdout: "overall": { 2024-02-16T19:40:16.696 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 14 2024-02-16T19:40:16.696 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-02-16T19:40:16.696 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-02-16T19:40:16.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:16.697+0000 7f94671ce700 1 -- 172.21.15.22:0/1499955486 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f944806d8a0 msgr2=0x7f944806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:40:16.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:16.697+0000 7f94671ce700 1 --2- 172.21.15.22:0/1499955486 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f944806d8a0 0x7f944806fd50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f94580094f0 tx=0x7f9458009040 comp rx=0 tx=0).stop 2024-02-16T19:40:16.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:16.697+0000 7f94671ce700 1 -- 172.21.15.22:0/1499955486 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9460105230 msgr2=0x7f94601001e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:40:16.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:16.697+0000 7f94671ce700 1 --2- 172.21.15.22:0/1499955486 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9460105230 0x7f94601001e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f94540092e0 tx=0x7f945400be20 comp rx=0 tx=0).stop 2024-02-16T19:40:16.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:16.697+0000 7f94671ce700 1 -- 172.21.15.22:0/1499955486 shutdown_connections 2024-02-16T19:40:16.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:16.697+0000 7f94671ce700 1 --2- 172.21.15.22:0/1499955486 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f944806d8a0 0x7f944806fd50 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:40:16.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:16.697+0000 7f94671ce700 1 --2- 172.21.15.22:0/1499955486 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9460068540 0x7f94600ffca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:40:16.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:16.697+0000 7f94671ce700 1 --2- 172.21.15.22:0/1499955486 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9460105230 0x7f94601001e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:40:16.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:16.697+0000 7f94671ce700 1 -- 172.21.15.22:0/1499955486 >> 172.21.15.22:0/1499955486 conn(0x7f9460075130 msgr2=0x7f94600feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:40:16.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:16.697+0000 7f94671ce700 1 -- 172.21.15.22:0/1499955486 shutdown_connections 2024-02-16T19:40:16.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:40:16.697+0000 7f94671ce700 1 -- 172.21.15.22:0/1499955486 wait complete. 2024-02-16T19:40:16.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:40:16 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:40:16.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:40:16 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:40:17.460 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'echo "wait for servicemap items w/ changing names to refresh"' 2024-02-16T19:40:17.673 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi022/config 2024-02-16T19:40:17.714 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:40:17 smithi022 ceph-mon[88798]: pgmap v256: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:40:17.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:40:17 smithi022 ceph-mon[88798]: from='client.? 172.21.15.22:0/1499955486' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:40:17.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:40:17 smithi117 ceph-mon[67870]: pgmap v256: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:40:17.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:40:17 smithi117 ceph-mon[67870]: from='client.? 172.21.15.22:0/1499955486' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:40:19.480 INFO:teuthology.orchestra.run.smithi022.stdout:wait for servicemap items w/ changing names to refresh 2024-02-16T19:40:19.856 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:40:19 smithi022 ceph-mon[88798]: pgmap v257: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:40:19.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:40:19 smithi117 ceph-mon[67870]: pgmap v257: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:40:20.055 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'sleep 60' 2024-02-16T19:40:20.275 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi022/config 2024-02-16T19:40:21.668 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:40:21 smithi022 ceph-mon[88798]: pgmap v258: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:40:21.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:40:21 smithi117 ceph-mon[67870]: pgmap v258: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:40:23.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:40:23 smithi117 ceph-mon[67870]: pgmap v259: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:40:23.967 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:40:23 smithi022 ceph-mon[88798]: pgmap v259: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:40:25.658 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:40:25 smithi117 ceph-mon[67870]: pgmap v260: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:40:25.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:40:25 smithi022 ceph-mon[88798]: pgmap v260: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:40:27.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:40:27 smithi117 ceph-mon[67870]: pgmap v261: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:40:27.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:40:27 smithi022 ceph-mon[88798]: pgmap v261: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:40:29.768 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:40:29 smithi022 ceph-mon[88798]: pgmap v262: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:40:29.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:40:29 smithi117 ceph-mon[67870]: pgmap v262: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:40:31.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:40:31 smithi117 ceph-mon[67870]: pgmap v263: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:40:31.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:40:31 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:40:31.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:40:31 smithi022 ceph-mon[88798]: pgmap v263: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:40:31.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:40:31 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:40:33.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:40:33 smithi117 ceph-mon[67870]: pgmap v264: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:40:33.967 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:40:33 smithi022 ceph-mon[88798]: pgmap v264: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:40:35.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:40:35 smithi117 ceph-mon[67870]: pgmap v265: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:40:35.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:40:35 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:40:35.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:40:35 smithi022 ceph-mon[88798]: pgmap v265: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:40:35.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:40:35 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:40:37.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:40:36 smithi022 ceph-mon[88798]: pgmap v266: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:40:37.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:40:36 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:40:37.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:40:36 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:40:37.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:40:36 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:40:37.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:40:36 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:40:37.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:40:36 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:40:37.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:40:36 smithi117 ceph-mon[67870]: pgmap v266: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:40:37.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:40:36 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:40:37.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:40:36 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:40:37.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:40:36 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:40:37.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:40:36 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:40:37.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:40:36 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:40:39.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:40:38 smithi022 ceph-mon[88798]: pgmap v267: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:40:39.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:40:38 smithi117 ceph-mon[67870]: pgmap v267: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:40:41.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:40:40 smithi022 ceph-mon[88798]: pgmap v268: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:40:41.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:40:40 smithi117 ceph-mon[67870]: pgmap v268: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:40:43.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:40:42 smithi022 ceph-mon[88798]: pgmap v269: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:40:43.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:40:42 smithi117 ceph-mon[67870]: pgmap v269: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:40:45.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:40:44 smithi022 ceph-mon[88798]: pgmap v270: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:40:45.322 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:40:44 smithi117 ceph-mon[67870]: pgmap v270: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:40:46.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:40:45 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:40:46.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:40:45 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:40:47.215 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:40:46 smithi022 ceph-mon[88798]: pgmap v271: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:40:47.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:40:46 smithi117 ceph-mon[67870]: pgmap v271: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:40:49.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:40:48 smithi117 ceph-mon[67870]: pgmap v272: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:40:49.464 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:40:48 smithi022 ceph-mon[88798]: pgmap v272: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:40:51.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:40:50 smithi117 ceph-mon[67870]: pgmap v273: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:40:51.464 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:40:50 smithi022 ceph-mon[88798]: pgmap v273: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:40:53.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:40:52 smithi117 ceph-mon[67870]: pgmap v274: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:40:53.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:40:52 smithi022 ceph-mon[88798]: pgmap v274: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:40:55.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:40:54 smithi117 ceph-mon[67870]: pgmap v275: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:40:55.403 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:40:54 smithi022 ceph-mon[88798]: pgmap v275: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:40:57.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:40:56 smithi117 ceph-mon[67870]: pgmap v276: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:40:57.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:40:56 smithi022 ceph-mon[88798]: pgmap v276: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:40:59.267 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:40:58 smithi022 ceph-mon[88798]: pgmap v277: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:40:59.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:40:58 smithi117 ceph-mon[67870]: pgmap v277: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:41:01.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:41:00 smithi117 ceph-mon[67870]: pgmap v278: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:41:01.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:41:00 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:41:01.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:41:00 smithi022 ceph-mon[88798]: pgmap v278: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:41:01.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:41:00 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:41:03.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:41:02 smithi117 ceph-mon[67870]: pgmap v279: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:41:03.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:41:02 smithi022 ceph-mon[88798]: pgmap v279: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:41:05.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:41:04 smithi117 ceph-mon[67870]: pgmap v280: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:41:05.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:41:04 smithi022 ceph-mon[88798]: pgmap v280: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:41:07.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:41:06 smithi117 ceph-mon[67870]: pgmap v281: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:41:07.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:41:06 smithi022 ceph-mon[88798]: pgmap v281: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:41:09.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:41:08 smithi117 ceph-mon[67870]: pgmap v282: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:41:09.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:41:08 smithi022 ceph-mon[88798]: pgmap v282: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:41:11.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:41:11 smithi117 ceph-mon[67870]: pgmap v283: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:41:11.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:41:11 smithi022 ceph-mon[88798]: pgmap v283: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:41:13.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:41:13 smithi117 ceph-mon[67870]: pgmap v284: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:41:13.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:41:13 smithi022 ceph-mon[88798]: pgmap v284: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:41:15.613 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:41:15 smithi117 ceph-mon[67870]: pgmap v285: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:41:15.614 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:41:15 smithi022 ceph-mon[88798]: pgmap v285: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:41:16.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:41:16 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:41:16.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:41:16 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:41:17.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:41:17 smithi117 ceph-mon[67870]: pgmap v286: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:41:17.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:41:17 smithi022 ceph-mon[88798]: pgmap v286: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:41:19.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:41:19 smithi117 ceph-mon[67870]: pgmap v287: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:41:19.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:41:19 smithi022 ceph-mon[88798]: pgmap v287: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:41:21.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:41:21 smithi117 ceph-mon[67870]: pgmap v288: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:41:21.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:41:21 smithi022 ceph-mon[88798]: pgmap v288: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:41:22.665 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph orch ps' 2024-02-16T19:41:22.888 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi022/config 2024-02-16T19:41:23.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:41:23 smithi117 ceph-mon[67870]: pgmap v289: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:41:23.710 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:41:23 smithi022 ceph-mon[88798]: pgmap v289: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:41:24.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:24.662+0000 7f6802495700 1 -- 172.21.15.22:0/2379538917 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67fc1012f0 msgr2=0x7f67fc1016c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:24.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:24.662+0000 7f6802495700 1 --2- 172.21.15.22:0/2379538917 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67fc1012f0 0x7f67fc1016c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f67ec009910 tx=0x7f67ec009c20 comp rx=0 tx=0).stop 2024-02-16T19:41:24.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:24.662+0000 7f6802495700 1 -- 172.21.15.22:0/2379538917 shutdown_connections 2024-02-16T19:41:24.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:24.662+0000 7f6802495700 1 --2- 172.21.15.22:0/2379538917 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67fc101c90 0x7f67fc10a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:24.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:24.662+0000 7f6802495700 1 --2- 172.21.15.22:0/2379538917 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67fc1012f0 0x7f67fc1016c0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:24.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:24.662+0000 7f6802495700 1 -- 172.21.15.22:0/2379538917 >> 172.21.15.22:0/2379538917 conn(0x7f67fc0fac80 msgr2=0x7f67fc0fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:24.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:24.662+0000 7f6802495700 1 -- 172.21.15.22:0/2379538917 shutdown_connections 2024-02-16T19:41:24.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:24.662+0000 7f6802495700 1 -- 172.21.15.22:0/2379538917 wait complete. 2024-02-16T19:41:24.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:24.662+0000 7f6802495700 1 Processor -- start 2024-02-16T19:41:24.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:24.662+0000 7f6802495700 1 -- start start 2024-02-16T19:41:24.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:24.663+0000 7f6802495700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67fc1012f0 0x7f67fc198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:24.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:24.663+0000 7f6802495700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67fc101c90 0x7f67fc1985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:24.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:24.663+0000 7f6802495700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67fc198c80 con 0x7f67fc1012f0 2024-02-16T19:41:24.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:24.663+0000 7f6802495700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67fc19ca10 con 0x7f67fc101c90 2024-02-16T19:41:24.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:24.663+0000 7f67fbfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67fc1012f0 0x7f67fc198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:24.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:24.663+0000 7f67fbfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67fc1012f0 0x7f67fc198060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57682/0 (socket says 172.21.15.22:57682) 2024-02-16T19:41:24.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:24.663+0000 7f67fbfff700 1 -- 172.21.15.22:0/1024201360 learned_addr learned my addr 172.21.15.22:0/1024201360 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:41:24.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:24.663+0000 7f67fbfff700 1 -- 172.21.15.22:0/1024201360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67fc101c90 msgr2=0x7f67fc1985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:24.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:24.663+0000 7f67fbfff700 1 --2- 172.21.15.22:0/1024201360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67fc101c90 0x7f67fc1985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:24.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:24.663+0000 7f67fbfff700 1 -- 172.21.15.22:0/1024201360 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67ec0095f0 con 0x7f67fc1012f0 2024-02-16T19:41:24.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:24.663+0000 7f67fbfff700 1 --2- 172.21.15.22:0/1024201360 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67fc1012f0 0x7f67fc198060 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f67ec000c00 tx=0x7f67ec0056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:24.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:24.664+0000 7f67f97fa700 1 -- 172.21.15.22:0/1024201360 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f67ec01d070 con 0x7f67fc1012f0 2024-02-16T19:41:24.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:24.664+0000 7f6802495700 1 -- 172.21.15.22:0/1024201360 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67fc19cc90 con 0x7f67fc1012f0 2024-02-16T19:41:24.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:24.664+0000 7f67f97fa700 1 -- 172.21.15.22:0/1024201360 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f67ec00f460 con 0x7f67fc1012f0 2024-02-16T19:41:24.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:24.664+0000 7f67f97fa700 1 -- 172.21.15.22:0/1024201360 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f67ec021640 con 0x7f67fc1012f0 2024-02-16T19:41:24.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:24.664+0000 7f6802495700 1 -- 172.21.15.22:0/1024201360 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67fc19d1e0 con 0x7f67fc1012f0 2024-02-16T19:41:24.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:24.665+0000 7f6802495700 1 -- 172.21.15.22:0/1024201360 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67fc109e30 con 0x7f67fc1012f0 2024-02-16T19:41:24.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:24.665+0000 7f67f97fa700 1 -- 172.21.15.22:0/1024201360 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f67ec003a60 con 0x7f67fc1012f0 2024-02-16T19:41:24.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:24.666+0000 7f67f97fa700 1 --2- 172.21.15.22:0/1024201360 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f67e406d8a0 0x7f67e406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:24.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:24.666+0000 7f67f97fa700 1 -- 172.21.15.22:0/1024201360 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f67ec08e910 con 0x7f67fc1012f0 2024-02-16T19:41:24.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:24.666+0000 7f67fb7fe700 1 --2- 172.21.15.22:0/1024201360 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f67e406d8a0 0x7f67e406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:24.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:24.666+0000 7f67fb7fe700 1 --2- 172.21.15.22:0/1024201360 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f67e406d8a0 0x7f67e406fd50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f67f0005950 tx=0x7f67f00058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:24.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:24.670+0000 7f67f97fa700 1 -- 172.21.15.22:0/1024201360 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f67ec057f40 con 0x7f67fc1012f0 2024-02-16T19:41:24.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:24.888+0000 7f6802495700 1 -- 172.21.15.22:0/1024201360 --> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f67fc02cc80 con 0x7f67e406d8a0 2024-02-16T19:41:24.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:24.899+0000 7f67f97fa700 1 -- 172.21.15.22:0/1024201360 <== mgr.34104 v2:172.21.15.22:6800/505266753 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f67fc02cc80 con 0x7f67e406d8a0 2024-02-16T19:41:24.899 INFO:teuthology.orchestra.run.smithi022.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T19:41:24.900 INFO:teuthology.orchestra.run.smithi022.stdout:alertmanager.smithi022 smithi022 *:9093,9094 running (9m) 115s ago 20m 26.8M - 0.25.0 c8568f914cd2 c89bc76be374 2024-02-16T19:41:24.900 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-exporter.smithi022 smithi022 running (2m) 115s ago 20m 7260k - 19.0.0-1450-gf78a58c0 f081f847070a e12b859aed67 2024-02-16T19:41:24.900 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-exporter.smithi117 smithi117 running (2m) 117s ago 19m 6924k - 19.0.0-1450-gf78a58c0 f081f847070a b275193d62db 2024-02-16T19:41:24.900 INFO:teuthology.orchestra.run.smithi022.stdout:crash.smithi022 smithi022 running (7m) 115s ago 20m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a e0f78bdf43e4 2024-02-16T19:41:24.900 INFO:teuthology.orchestra.run.smithi022.stdout:crash.smithi117 smithi117 running (7m) 117s ago 19m 6669k - 19.0.0-1450-gf78a58c0 f081f847070a bce7c0590813 2024-02-16T19:41:24.900 INFO:teuthology.orchestra.run.smithi022.stdout:grafana.smithi022 smithi022 *:3000 running (9m) 115s ago 20m 82.2M - 9.4.12 a0ea5e3c46d9 fd27d11e9729 2024-02-16T19:41:24.900 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi022.btppwm smithi022 running (3m) 115s ago 14m 12.9M - 19.0.0-1450-gf78a58c0 f081f847070a 243e0becd1f8 2024-02-16T19:41:24.900 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi022.ocukvs smithi022 running (3m) 115s ago 15m 87.3M - 19.0.0-1450-gf78a58c0 f081f847070a e6ce6a81df63 2024-02-16T19:41:24.900 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi117.ibmppn smithi117 running (2m) 117s ago 14m 12.8M - 19.0.0-1450-gf78a58c0 f081f847070a 05b2e23f1aa1 2024-02-16T19:41:24.900 INFO:teuthology.orchestra.run.smithi022.stdout:mds.cephfs.smithi117.orzors smithi117 running (2m) 117s ago 14m 16.0M - 19.0.0-1450-gf78a58c0 f081f847070a 599a29785d0b 2024-02-16T19:41:24.900 INFO:teuthology.orchestra.run.smithi022.stdout:mgr.smithi022.gmdcch smithi022 *:8443,9283,8765 running (11m) 115s ago 22m 510M - 19.0.0-1450-gf78a58c0 f081f847070a 7cb74cc9deea 2024-02-16T19:41:24.900 INFO:teuthology.orchestra.run.smithi022.stdout:mgr.smithi117.cohbtv smithi117 *:8443,9283,8765 running (10m) 117s ago 19m 454M - 19.0.0-1450-gf78a58c0 f081f847070a c825b1152189 2024-02-16T19:41:24.900 INFO:teuthology.orchestra.run.smithi022.stdout:mon.smithi022 smithi022 running (8m) 115s ago 22m 57.5M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 9c84755ae67d 2024-02-16T19:41:24.901 INFO:teuthology.orchestra.run.smithi022.stdout:mon.smithi117 smithi117 running (7m) 117s ago 19m 46.4M 2048M 19.0.0-1450-gf78a58c0 f081f847070a a319ac50f1a9 2024-02-16T19:41:24.901 INFO:teuthology.orchestra.run.smithi022.stdout:node-exporter.smithi022 smithi022 *:9100 running (10m) 115s ago 20m 16.4M - 1.5.0 0da6a335fe13 1f5b58292d27 2024-02-16T19:41:24.901 INFO:teuthology.orchestra.run.smithi022.stdout:node-exporter.smithi117 smithi117 *:9100 running (10m) 117s ago 19m 15.5M - 1.5.0 0da6a335fe13 bb5e8491d5a6 2024-02-16T19:41:24.901 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0 smithi022 running (6m) 115s ago 18m 117M 1217M 19.0.0-1450-gf78a58c0 f081f847070a 002412b3525f 2024-02-16T19:41:24.901 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1 smithi022 running (6m) 115s ago 18m 117M 1217M 19.0.0-1450-gf78a58c0 f081f847070a c3488ce7b2a0 2024-02-16T19:41:24.901 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2 smithi022 running (5m) 115s ago 17m 95.8M 1217M 19.0.0-1450-gf78a58c0 f081f847070a 46138c50f79d 2024-02-16T19:41:24.901 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3 smithi117 running (4m) 117s ago 17m 103M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 28d7d8519e64 2024-02-16T19:41:24.901 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4 smithi117 running (4m) 117s ago 16m 96.2M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 367ba2085f6f 2024-02-16T19:41:24.901 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5 smithi117 running (3m) 117s ago 16m 96.0M 2240M 19.0.0-1450-gf78a58c0 f081f847070a a1a7625b7541 2024-02-16T19:41:24.901 INFO:teuthology.orchestra.run.smithi022.stdout:prometheus.smithi022 smithi022 *:9095 running (10m) 115s ago 20m 69.2M - 2.43.0 a07b618ecd1d c3b8689d0c97 2024-02-16T19:41:24.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:24.902+0000 7f6802495700 1 -- 172.21.15.22:0/1024201360 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f67e406d8a0 msgr2=0x7f67e406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:24.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:24.902+0000 7f6802495700 1 --2- 172.21.15.22:0/1024201360 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f67e406d8a0 0x7f67e406fd50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f67f0005950 tx=0x7f67f00058e0 comp rx=0 tx=0).stop 2024-02-16T19:41:24.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:24.902+0000 7f6802495700 1 -- 172.21.15.22:0/1024201360 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67fc1012f0 msgr2=0x7f67fc198060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:24.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:24.902+0000 7f6802495700 1 --2- 172.21.15.22:0/1024201360 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67fc1012f0 0x7f67fc198060 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f67ec000c00 tx=0x7f67ec0056a0 comp rx=0 tx=0).stop 2024-02-16T19:41:24.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:24.902+0000 7f6802495700 1 -- 172.21.15.22:0/1024201360 shutdown_connections 2024-02-16T19:41:24.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:24.902+0000 7f6802495700 1 --2- 172.21.15.22:0/1024201360 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f67e406d8a0 0x7f67e406fd50 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:24.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:24.902+0000 7f6802495700 1 --2- 172.21.15.22:0/1024201360 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67fc1012f0 0x7f67fc198060 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:24.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:24.902+0000 7f6802495700 1 --2- 172.21.15.22:0/1024201360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67fc101c90 0x7f67fc1985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:24.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:24.902+0000 7f6802495700 1 -- 172.21.15.22:0/1024201360 >> 172.21.15.22:0/1024201360 conn(0x7f67fc0fac80 msgr2=0x7f67fc0ff8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:24.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:24.902+0000 7f6802495700 1 -- 172.21.15.22:0/1024201360 shutdown_connections 2024-02-16T19:41:24.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:24.902+0000 7f6802495700 1 -- 172.21.15.22:0/1024201360 wait complete. 2024-02-16T19:41:25.572 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:41:25 smithi022 ceph-mon[88798]: pgmap v290: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:41:25.573 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph versions' 2024-02-16T19:41:25.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:41:25 smithi117 ceph-mon[67870]: pgmap v290: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:41:25.797 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi022/config 2024-02-16T19:41:26.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:41:26 smithi022 ceph-mon[88798]: from='client.34524 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:41:26.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:41:26 smithi117 ceph-mon[67870]: from='client.34524 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:41:27.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:27.387+0000 7feb59cca700 1 -- 172.21.15.22:0/3339074756 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb54069140 msgr2=0x7feb54104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:27.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:27.387+0000 7feb59cca700 1 --2- 172.21.15.22:0/3339074756 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb54069140 0x7feb54104eb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7feb48009910 tx=0x7feb48009c20 comp rx=0 tx=0).stop 2024-02-16T19:41:27.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:27.387+0000 7feb59cca700 1 -- 172.21.15.22:0/3339074756 shutdown_connections 2024-02-16T19:41:27.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:27.387+0000 7feb59cca700 1 --2- 172.21.15.22:0/3339074756 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb54069140 0x7feb54104eb0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:27.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:27.387+0000 7feb59cca700 1 --2- 172.21.15.22:0/3339074756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb540687a0 0x7feb54068b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:27.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:27.387+0000 7feb59cca700 1 -- 172.21.15.22:0/3339074756 >> 172.21.15.22:0/3339074756 conn(0x7feb54075130 msgr2=0x7feb54075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:27.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:27.387+0000 7feb59cca700 1 -- 172.21.15.22:0/3339074756 shutdown_connections 2024-02-16T19:41:27.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:27.387+0000 7feb59cca700 1 -- 172.21.15.22:0/3339074756 wait complete. 2024-02-16T19:41:27.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:27.388+0000 7feb59cca700 1 Processor -- start 2024-02-16T19:41:27.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:27.388+0000 7feb59cca700 1 -- start start 2024-02-16T19:41:27.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:27.388+0000 7feb59cca700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb540687a0 0x7feb54195e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:27.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:27.388+0000 7feb59cca700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb54069140 0x7feb541963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:27.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:27.388+0000 7feb59cca700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb54196a80 con 0x7feb540687a0 2024-02-16T19:41:27.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:27.388+0000 7feb59cca700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb5419a810 con 0x7feb54069140 2024-02-16T19:41:27.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:27.388+0000 7feb537fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb540687a0 0x7feb54195e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:27.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:27.388+0000 7feb537fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb540687a0 0x7feb54195e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57700/0 (socket says 172.21.15.22:57700) 2024-02-16T19:41:27.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:27.388+0000 7feb537fe700 1 -- 172.21.15.22:0/2806868560 learned_addr learned my addr 172.21.15.22:0/2806868560 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:41:27.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:27.388+0000 7feb537fe700 1 -- 172.21.15.22:0/2806868560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb54069140 msgr2=0x7feb541963a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:27.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:27.388+0000 7feb52ffd700 1 --2- 172.21.15.22:0/2806868560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb54069140 0x7feb541963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:27.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:27.388+0000 7feb537fe700 1 --2- 172.21.15.22:0/2806868560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb54069140 0x7feb541963a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:27.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:27.389+0000 7feb537fe700 1 -- 172.21.15.22:0/2806868560 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb480095f0 con 0x7feb540687a0 2024-02-16T19:41:27.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:27.389+0000 7feb537fe700 1 --2- 172.21.15.22:0/2806868560 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb540687a0 0x7feb54195e60 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7feb4400d900 tx=0x7feb4400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:27.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:27.389+0000 7feb50ff9700 1 -- 172.21.15.22:0/2806868560 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb44004a00 con 0x7feb540687a0 2024-02-16T19:41:27.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:27.389+0000 7feb50ff9700 1 -- 172.21.15.22:0/2806868560 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feb440054e0 con 0x7feb540687a0 2024-02-16T19:41:27.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:27.389+0000 7feb59cca700 1 -- 172.21.15.22:0/2806868560 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb5419aaf0 con 0x7feb540687a0 2024-02-16T19:41:27.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:27.389+0000 7feb50ff9700 1 -- 172.21.15.22:0/2806868560 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb44009d60 con 0x7feb540687a0 2024-02-16T19:41:27.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:27.389+0000 7feb59cca700 1 -- 172.21.15.22:0/2806868560 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb5419b040 con 0x7feb540687a0 2024-02-16T19:41:27.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:27.390+0000 7feb59cca700 1 -- 172.21.15.22:0/2806868560 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb54108750 con 0x7feb540687a0 2024-02-16T19:41:27.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:27.390+0000 7feb50ff9700 1 -- 172.21.15.22:0/2806868560 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7feb44005020 con 0x7feb540687a0 2024-02-16T19:41:27.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:27.391+0000 7feb50ff9700 1 --2- 172.21.15.22:0/2806868560 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7feb3c06d8a0 0x7feb3c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:27.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:27.391+0000 7feb50ff9700 1 -- 172.21.15.22:0/2806868560 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6962+0+0 (secure 0 0 0) 0x7feb4408d8d0 con 0x7feb540687a0 2024-02-16T19:41:27.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:27.391+0000 7feb52ffd700 1 --2- 172.21.15.22:0/2806868560 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7feb3c06d8a0 0x7feb3c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:27.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:27.391+0000 7feb52ffd700 1 --2- 172.21.15.22:0/2806868560 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7feb3c06d8a0 0x7feb3c06fd50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7feb48006010 tx=0x7feb4800b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:27.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:27.395+0000 7feb50ff9700 1 -- 172.21.15.22:0/2806868560 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7feb44056f00 con 0x7feb540687a0 2024-02-16T19:41:27.464 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:41:27 smithi022 ceph-mon[88798]: pgmap v291: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:41:27.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:41:27 smithi117 ceph-mon[67870]: pgmap v291: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:41:27.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:27.687+0000 7feb59cca700 1 -- 172.21.15.22:0/2806868560 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7feb5419b260 con 0x7feb540687a0 2024-02-16T19:41:27.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:27.688+0000 7feb50ff9700 1 -- 172.21.15.22:0/2806868560 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7feb4405a5c0 con 0x7feb540687a0 2024-02-16T19:41:27.688 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-02-16T19:41:27.688 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": { 2024-02-16T19:41:27.689 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T19:41:27.689 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:41:27.689 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": { 2024-02-16T19:41:27.689 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T19:41:27.689 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:41:27.689 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": { 2024-02-16T19:41:27.689 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-16T19:41:27.689 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:41:27.689 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": { 2024-02-16T19:41:27.689 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 4 2024-02-16T19:41:27.689 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-02-16T19:41:27.690 INFO:teuthology.orchestra.run.smithi022.stdout: "overall": { 2024-02-16T19:41:27.690 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 14 2024-02-16T19:41:27.690 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-02-16T19:41:27.690 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-02-16T19:41:27.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:27.690+0000 7feb59cca700 1 -- 172.21.15.22:0/2806868560 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7feb3c06d8a0 msgr2=0x7feb3c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:27.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:27.690+0000 7feb59cca700 1 --2- 172.21.15.22:0/2806868560 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7feb3c06d8a0 0x7feb3c06fd50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7feb48006010 tx=0x7feb4800b540 comp rx=0 tx=0).stop 2024-02-16T19:41:27.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:27.690+0000 7feb59cca700 1 -- 172.21.15.22:0/2806868560 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb540687a0 msgr2=0x7feb54195e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:27.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:27.690+0000 7feb59cca700 1 --2- 172.21.15.22:0/2806868560 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb540687a0 0x7feb54195e60 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7feb4400d900 tx=0x7feb4400dcc0 comp rx=0 tx=0).stop 2024-02-16T19:41:27.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:27.690+0000 7feb59cca700 1 -- 172.21.15.22:0/2806868560 shutdown_connections 2024-02-16T19:41:27.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:27.690+0000 7feb59cca700 1 --2- 172.21.15.22:0/2806868560 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7feb3c06d8a0 0x7feb3c06fd50 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:27.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:27.690+0000 7feb59cca700 1 --2- 172.21.15.22:0/2806868560 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb540687a0 0x7feb54195e60 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:27.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:27.690+0000 7feb59cca700 1 --2- 172.21.15.22:0/2806868560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb54069140 0x7feb541963a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:27.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:27.690+0000 7feb59cca700 1 -- 172.21.15.22:0/2806868560 >> 172.21.15.22:0/2806868560 conn(0x7feb54075130 msgr2=0x7feb54104720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:27.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:27.690+0000 7feb59cca700 1 -- 172.21.15.22:0/2806868560 shutdown_connections 2024-02-16T19:41:27.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:27.690+0000 7feb59cca700 1 -- 172.21.15.22:0/2806868560 wait complete. 2024-02-16T19:41:28.244 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 1'"'"'' 2024-02-16T19:41:28.458 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi022/config 2024-02-16T19:41:28.580 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:41:28 smithi022 ceph-mon[88798]: from='client.? 172.21.15.22:0/2806868560' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:41:28.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:41:28 smithi117 ceph-mon[67870]: from='client.? 172.21.15.22:0/2806868560' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:41:29.620 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:41:29 smithi022 ceph-mon[88798]: pgmap v292: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:41:29.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:41:29 smithi117 ceph-mon[67870]: pgmap v292: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:41:30.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:30.184+0000 7f8b4c3ab700 1 -- 172.21.15.22:0/929252742 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b44101e30 msgr2=0x7f8b4410a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:30.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:30.184+0000 7f8b4c3ab700 1 --2- 172.21.15.22:0/929252742 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b44101e30 0x7f8b4410a330 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8b34009910 tx=0x7f8b34009c20 comp rx=0 tx=0).stop 2024-02-16T19:41:30.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:30.184+0000 7f8b4c3ab700 1 -- 172.21.15.22:0/929252742 shutdown_connections 2024-02-16T19:41:30.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:30.184+0000 7f8b4c3ab700 1 --2- 172.21.15.22:0/929252742 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b44101e30 0x7f8b4410a330 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:30.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:30.184+0000 7f8b4c3ab700 1 --2- 172.21.15.22:0/929252742 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b44101520 0x7f8b441018f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:30.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:30.184+0000 7f8b4c3ab700 1 -- 172.21.15.22:0/929252742 >> 172.21.15.22:0/929252742 conn(0x7f8b440face0 msgr2=0x7f8b440fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:30.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:30.184+0000 7f8b4c3ab700 1 -- 172.21.15.22:0/929252742 shutdown_connections 2024-02-16T19:41:30.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:30.184+0000 7f8b4c3ab700 1 -- 172.21.15.22:0/929252742 wait complete. 2024-02-16T19:41:30.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:30.184+0000 7f8b4c3ab700 1 Processor -- start 2024-02-16T19:41:30.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:30.184+0000 7f8b4c3ab700 1 -- start start 2024-02-16T19:41:30.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:30.185+0000 7f8b4c3ab700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b44101520 0x7f8b440ff180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:30.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:30.185+0000 7f8b4c3ab700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b44101e30 0x7f8b440ff6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:30.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:30.185+0000 7f8b4c3ab700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b440ffc00 con 0x7f8b44101520 2024-02-16T19:41:30.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:30.185+0000 7f8b4c3ab700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b440ffd40 con 0x7f8b44101e30 2024-02-16T19:41:30.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:30.185+0000 7f8b4a147700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b44101520 0x7f8b440ff180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:30.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:30.185+0000 7f8b4a147700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b44101520 0x7f8b440ff180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57724/0 (socket says 172.21.15.22:57724) 2024-02-16T19:41:30.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:30.185+0000 7f8b4a147700 1 -- 172.21.15.22:0/2593880089 learned_addr learned my addr 172.21.15.22:0/2593880089 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:41:30.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:30.185+0000 7f8b4a147700 1 -- 172.21.15.22:0/2593880089 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b44101e30 msgr2=0x7f8b440ff6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:30.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:30.185+0000 7f8b4a147700 1 --2- 172.21.15.22:0/2593880089 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b44101e30 0x7f8b440ff6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:30.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:30.185+0000 7f8b4a147700 1 -- 172.21.15.22:0/2593880089 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b340095f0 con 0x7f8b44101520 2024-02-16T19:41:30.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:30.185+0000 7f8b4a147700 1 --2- 172.21.15.22:0/2593880089 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b44101520 0x7f8b440ff180 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8b4000b4b0 tx=0x7f8b4000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:30.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:30.185+0000 7f8b3b7fe700 1 -- 172.21.15.22:0/2593880089 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b40011800 con 0x7f8b44101520 2024-02-16T19:41:30.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:30.185+0000 7f8b3b7fe700 1 -- 172.21.15.22:0/2593880089 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8b40011e20 con 0x7f8b44101520 2024-02-16T19:41:30.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:30.185+0000 7f8b3b7fe700 1 -- 172.21.15.22:0/2593880089 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b4000f2e0 con 0x7f8b44101520 2024-02-16T19:41:30.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:30.185+0000 7f8b4c3ab700 1 -- 172.21.15.22:0/2593880089 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b441a2180 con 0x7f8b44101520 2024-02-16T19:41:30.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:30.186+0000 7f8b4c3ab700 1 -- 172.21.15.22:0/2593880089 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b441a2530 con 0x7f8b44101520 2024-02-16T19:41:30.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:30.186+0000 7f8b4c3ab700 1 -- 172.21.15.22:0/2593880089 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b441079a0 con 0x7f8b44101520 2024-02-16T19:41:30.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:30.187+0000 7f8b3b7fe700 1 -- 172.21.15.22:0/2593880089 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f8b40010380 con 0x7f8b44101520 2024-02-16T19:41:30.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:30.187+0000 7f8b3b7fe700 1 --2- 172.21.15.22:0/2593880089 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f8b3006d8a0 0x7f8b3006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:30.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:30.187+0000 7f8b3b7fe700 1 -- 172.21.15.22:0/2593880089 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f8b4008d210 con 0x7f8b44101520 2024-02-16T19:41:30.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:30.188+0000 7f8b49946700 1 --2- 172.21.15.22:0/2593880089 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f8b3006d8a0 0x7f8b3006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:30.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:30.188+0000 7f8b49946700 1 --2- 172.21.15.22:0/2593880089 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f8b3006d8a0 0x7f8b3006fd50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8b3400b5c0 tx=0x7f8b34005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:30.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:30.192+0000 7f8b3b7fe700 1 -- 172.21.15.22:0/2593880089 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8b400568d0 con 0x7f8b44101520 2024-02-16T19:41:30.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:30.495+0000 7f8b4c3ab700 1 -- 172.21.15.22:0/2593880089 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f8b4404f2a0 con 0x7f8b44101520 2024-02-16T19:41:30.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:30.495+0000 7f8b3b7fe700 1 -- 172.21.15.22:0/2593880089 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f8b40059f90 con 0x7f8b44101520 2024-02-16T19:41:30.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:30.497+0000 7f8b4c3ab700 1 -- 172.21.15.22:0/2593880089 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f8b3006d8a0 msgr2=0x7f8b3006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:30.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:30.497+0000 7f8b4c3ab700 1 --2- 172.21.15.22:0/2593880089 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f8b3006d8a0 0x7f8b3006fd50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8b3400b5c0 tx=0x7f8b34005960 comp rx=0 tx=0).stop 2024-02-16T19:41:30.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:30.497+0000 7f8b4c3ab700 1 -- 172.21.15.22:0/2593880089 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b44101520 msgr2=0x7f8b440ff180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:30.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:30.497+0000 7f8b4c3ab700 1 --2- 172.21.15.22:0/2593880089 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b44101520 0x7f8b440ff180 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8b4000b4b0 tx=0x7f8b4000b870 comp rx=0 tx=0).stop 2024-02-16T19:41:30.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:30.497+0000 7f8b4c3ab700 1 -- 172.21.15.22:0/2593880089 shutdown_connections 2024-02-16T19:41:30.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:30.497+0000 7f8b4c3ab700 1 --2- 172.21.15.22:0/2593880089 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f8b3006d8a0 0x7f8b3006fd50 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:30.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:30.497+0000 7f8b4c3ab700 1 --2- 172.21.15.22:0/2593880089 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b44101520 0x7f8b440ff180 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:30.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:30.497+0000 7f8b4c3ab700 1 --2- 172.21.15.22:0/2593880089 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b44101e30 0x7f8b440ff6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:30.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:30.497+0000 7f8b4c3ab700 1 -- 172.21.15.22:0/2593880089 >> 172.21.15.22:0/2593880089 conn(0x7f8b440face0 msgr2=0x7f8b44104ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:30.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:30.498+0000 7f8b4c3ab700 1 -- 172.21.15.22:0/2593880089 shutdown_connections 2024-02-16T19:41:30.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:30.498+0000 7f8b4c3ab700 1 -- 172.21.15.22:0/2593880089 wait complete. 2024-02-16T19:41:30.508 INFO:teuthology.orchestra.run.smithi022.stdout:true 2024-02-16T19:41:31.183 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph versions | jq -e '"'"'.overall | keys'"'"' | grep $sha1' 2024-02-16T19:41:31.391 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi022/config 2024-02-16T19:41:31.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:41:31 smithi022 ceph-mon[88798]: pgmap v293: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:41:31.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:41:31 smithi022 ceph-mon[88798]: from='client.? 172.21.15.22:0/2593880089' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:41:31.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:41:31 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:41:31.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:41:31 smithi117 ceph-mon[67870]: pgmap v293: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:41:31.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:41:31 smithi117 ceph-mon[67870]: from='client.? 172.21.15.22:0/2593880089' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:41:31.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:41:31 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:41:33.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:33.158+0000 7f81e0a50700 1 -- 172.21.15.22:0/3809796422 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81dc0ffd60 msgr2=0x7f81dc103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:33.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:33.158+0000 7f81e0a50700 1 --2- 172.21.15.22:0/3809796422 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81dc0ffd60 0x7f81dc103d30 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f81d0009910 tx=0x7f81d0009c20 comp rx=0 tx=0).stop 2024-02-16T19:41:33.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:33.158+0000 7f81e0a50700 1 -- 172.21.15.22:0/3809796422 shutdown_connections 2024-02-16T19:41:33.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:33.158+0000 7f81e0a50700 1 --2- 172.21.15.22:0/3809796422 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81dc0ffd60 0x7f81dc103d30 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:33.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:33.158+0000 7f81e0a50700 1 --2- 172.21.15.22:0/3809796422 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81dc0ff3c0 0x7f81dc0ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:33.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:33.158+0000 7f81e0a50700 1 -- 172.21.15.22:0/3809796422 >> 172.21.15.22:0/3809796422 conn(0x7f81dc0fac80 msgr2=0x7f81dc0fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:33.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:33.158+0000 7f81e0a50700 1 -- 172.21.15.22:0/3809796422 shutdown_connections 2024-02-16T19:41:33.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:33.158+0000 7f81e0a50700 1 -- 172.21.15.22:0/3809796422 wait complete. 2024-02-16T19:41:33.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:33.159+0000 7f81e0a50700 1 Processor -- start 2024-02-16T19:41:33.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:33.159+0000 7f81e0a50700 1 -- start start 2024-02-16T19:41:33.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:33.159+0000 7f81e0a50700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81dc0ff3c0 0x7f81dc195ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:33.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:33.159+0000 7f81e0a50700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81dc0ffd60 0x7f81dc1963e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:33.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:33.159+0000 7f81e0a50700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81dc196a30 con 0x7f81dc0ff3c0 2024-02-16T19:41:33.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:33.159+0000 7f81e0a50700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81dc196b70 con 0x7f81dc0ffd60 2024-02-16T19:41:33.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:33.159+0000 7f81da59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81dc0ff3c0 0x7f81dc195ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:33.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:33.159+0000 7f81da59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81dc0ff3c0 0x7f81dc195ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57750/0 (socket says 172.21.15.22:57750) 2024-02-16T19:41:33.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:33.159+0000 7f81da59c700 1 -- 172.21.15.22:0/262714942 learned_addr learned my addr 172.21.15.22:0/262714942 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:41:33.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:33.159+0000 7f81d9d9b700 1 --2- 172.21.15.22:0/262714942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81dc0ffd60 0x7f81dc1963e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:33.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:33.159+0000 7f81da59c700 1 -- 172.21.15.22:0/262714942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81dc0ffd60 msgr2=0x7f81dc1963e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:33.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:33.159+0000 7f81da59c700 1 --2- 172.21.15.22:0/262714942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81dc0ffd60 0x7f81dc1963e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:33.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:33.159+0000 7f81da59c700 1 -- 172.21.15.22:0/262714942 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81d00095f0 con 0x7f81dc0ff3c0 2024-02-16T19:41:33.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:33.160+0000 7f81da59c700 1 --2- 172.21.15.22:0/262714942 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81dc0ff3c0 0x7f81dc195ea0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f81cc00c930 tx=0x7f81cc00ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:33.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:33.160+0000 7f81cb7fe700 1 -- 172.21.15.22:0/262714942 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f81cc007ad0 con 0x7f81dc0ff3c0 2024-02-16T19:41:33.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:33.160+0000 7f81e0a50700 1 -- 172.21.15.22:0/262714942 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81dc19a9c0 con 0x7f81dc0ff3c0 2024-02-16T19:41:33.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:33.160+0000 7f81cb7fe700 1 -- 172.21.15.22:0/262714942 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f81cc007c30 con 0x7f81dc0ff3c0 2024-02-16T19:41:33.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:33.160+0000 7f81cb7fe700 1 -- 172.21.15.22:0/262714942 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f81cc018610 con 0x7f81dc0ff3c0 2024-02-16T19:41:33.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:33.160+0000 7f81e0a50700 1 -- 172.21.15.22:0/262714942 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81dc19af10 con 0x7f81dc0ff3c0 2024-02-16T19:41:33.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:33.161+0000 7f81e0a50700 1 -- 172.21.15.22:0/262714942 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81dc04f2a0 con 0x7f81dc0ff3c0 2024-02-16T19:41:33.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:33.161+0000 7f81cb7fe700 1 -- 172.21.15.22:0/262714942 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f81cc00f450 con 0x7f81dc0ff3c0 2024-02-16T19:41:33.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:33.162+0000 7f81cb7fe700 1 --2- 172.21.15.22:0/262714942 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f81c406d8a0 0x7f81c406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:33.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:33.162+0000 7f81cb7fe700 1 -- 172.21.15.22:0/262714942 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f81cc08d8b0 con 0x7f81dc0ff3c0 2024-02-16T19:41:33.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:33.162+0000 7f81d9d9b700 1 --2- 172.21.15.22:0/262714942 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f81c406d8a0 0x7f81c406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:33.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:33.162+0000 7f81d9d9b700 1 --2- 172.21.15.22:0/262714942 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f81c406d8a0 0x7f81c406fd50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f81dc1974c0 tx=0x7f81d001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:33.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:33.167+0000 7f81cb7fe700 1 -- 172.21.15.22:0/262714942 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f81cc056ee0 con 0x7f81dc0ff3c0 2024-02-16T19:41:33.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:33.458+0000 7f81e0a50700 1 -- 172.21.15.22:0/262714942 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f81dc19b1f0 con 0x7f81dc0ff3c0 2024-02-16T19:41:33.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:33.458+0000 7f81cb7fe700 1 -- 172.21.15.22:0/262714942 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f81cc05a5a0 con 0x7f81dc0ff3c0 2024-02-16T19:41:33.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:33.460+0000 7f81e0a50700 1 -- 172.21.15.22:0/262714942 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f81c406d8a0 msgr2=0x7f81c406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:33.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:33.460+0000 7f81e0a50700 1 --2- 172.21.15.22:0/262714942 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f81c406d8a0 0x7f81c406fd50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f81dc1974c0 tx=0x7f81d001a040 comp rx=0 tx=0).stop 2024-02-16T19:41:33.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:33.460+0000 7f81e0a50700 1 -- 172.21.15.22:0/262714942 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81dc0ff3c0 msgr2=0x7f81dc195ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:33.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:33.460+0000 7f81e0a50700 1 --2- 172.21.15.22:0/262714942 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81dc0ff3c0 0x7f81dc195ea0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f81cc00c930 tx=0x7f81cc00ccf0 comp rx=0 tx=0).stop 2024-02-16T19:41:33.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:33.460+0000 7f81e0a50700 1 -- 172.21.15.22:0/262714942 shutdown_connections 2024-02-16T19:41:33.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:33.460+0000 7f81e0a50700 1 --2- 172.21.15.22:0/262714942 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f81c406d8a0 0x7f81c406fd50 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:33.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:33.460+0000 7f81e0a50700 1 --2- 172.21.15.22:0/262714942 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81dc0ff3c0 0x7f81dc195ea0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:33.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:33.460+0000 7f81e0a50700 1 --2- 172.21.15.22:0/262714942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81dc0ffd60 0x7f81dc1963e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:33.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:33.461+0000 7f81e0a50700 1 -- 172.21.15.22:0/262714942 >> 172.21.15.22:0/262714942 conn(0x7f81dc0fac80 msgr2=0x7f81dc068e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:33.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:33.461+0000 7f81e0a50700 1 -- 172.21.15.22:0/262714942 shutdown_connections 2024-02-16T19:41:33.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:33.461+0000 7f81e0a50700 1 -- 172.21.15.22:0/262714942 wait complete. 2024-02-16T19:41:33.471 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)" 2024-02-16T19:41:33.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:41:33 smithi117 ceph-mon[67870]: pgmap v294: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:41:33.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:41:33 smithi022 ceph-mon[88798]: pgmap v294: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:41:34.134 DEBUG:teuthology.parallel:result is None 2024-02-16T19:41:34.134 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-16T19:41:34.147 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi022.front.sepia.ceph.com 2024-02-16T19:41:34.148 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 -- bash -c 'ceph fs dump' 2024-02-16T19:41:34.364 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/mon.smithi022/config 2024-02-16T19:41:34.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:41:34 smithi117 ceph-mon[67870]: from='client.? 172.21.15.22:0/262714942' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:41:34.711 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:41:34 smithi022 ceph-mon[88798]: from='client.? 172.21.15.22:0/262714942' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:41:35.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:41:35 smithi117 ceph-mon[67870]: pgmap v295: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:41:35.638 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:41:35 smithi022 ceph-mon[88798]: pgmap v295: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:41:36.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:35.989+0000 7fab0b4db700 1 -- 172.21.15.22:0/167680536 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab040ffc60 msgr2=0x7fab0410a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:36.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:35.989+0000 7fab0b4db700 1 --2- 172.21.15.22:0/167680536 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab040ffc60 0x7fab0410a6d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7faaf4009910 tx=0x7faaf4009c20 comp rx=0 tx=0).stop 2024-02-16T19:41:36.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:35.989+0000 7fab0b4db700 1 -- 172.21.15.22:0/167680536 shutdown_connections 2024-02-16T19:41:36.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:35.989+0000 7fab0b4db700 1 --2- 172.21.15.22:0/167680536 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab040ffc60 0x7fab0410a6d0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:36.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:35.989+0000 7fab0b4db700 1 --2- 172.21.15.22:0/167680536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab040ff350 0x7fab040ff720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:36.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:35.989+0000 7fab0b4db700 1 -- 172.21.15.22:0/167680536 >> 172.21.15.22:0/167680536 conn(0x7fab040748b0 msgr2=0x7fab04074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:36.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:35.989+0000 7fab0b4db700 1 -- 172.21.15.22:0/167680536 shutdown_connections 2024-02-16T19:41:36.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:35.990+0000 7fab0b4db700 1 -- 172.21.15.22:0/167680536 wait complete. 2024-02-16T19:41:36.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:35.990+0000 7fab0b4db700 1 Processor -- start 2024-02-16T19:41:36.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:35.990+0000 7fab0b4db700 1 -- start start 2024-02-16T19:41:36.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:35.990+0000 7fab0b4db700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab040ff350 0x7fab04195e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:36.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:35.990+0000 7fab0b4db700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab040ffc60 0x7fab04196380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:36.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:35.990+0000 7fab0b4db700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab04196a60 con 0x7fab040ff350 2024-02-16T19:41:36.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:35.990+0000 7fab0b4db700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab0419a7f0 con 0x7fab040ffc60 2024-02-16T19:41:36.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:35.990+0000 7fab09277700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab040ff350 0x7fab04195e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:36.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:35.990+0000 7fab09277700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab040ff350 0x7fab04195e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45394/0 (socket says 172.21.15.22:45394) 2024-02-16T19:41:36.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:35.990+0000 7fab09277700 1 -- 172.21.15.22:0/160983643 learned_addr learned my addr 172.21.15.22:0/160983643 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:41:36.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:35.991+0000 7fab08a76700 1 --2- 172.21.15.22:0/160983643 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab040ffc60 0x7fab04196380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:36.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:35.991+0000 7fab09277700 1 -- 172.21.15.22:0/160983643 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab040ffc60 msgr2=0x7fab04196380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:36.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:35.991+0000 7fab09277700 1 --2- 172.21.15.22:0/160983643 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab040ffc60 0x7fab04196380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:36.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:35.991+0000 7fab09277700 1 -- 172.21.15.22:0/160983643 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaf40095f0 con 0x7fab040ff350 2024-02-16T19:41:36.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:35.991+0000 7fab08a76700 1 --2- 172.21.15.22:0/160983643 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab040ffc60 0x7fab04196380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T19:41:36.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:35.991+0000 7fab09277700 1 --2- 172.21.15.22:0/160983643 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab040ff350 0x7fab04195e40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fab0000c8a0 tx=0x7fab0000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:36.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:35.991+0000 7faafa7fc700 1 -- 172.21.15.22:0/160983643 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab0000cea0 con 0x7fab040ff350 2024-02-16T19:41:36.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:35.991+0000 7faafa7fc700 1 -- 172.21.15.22:0/160983643 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fab00004830 con 0x7fab040ff350 2024-02-16T19:41:36.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:35.991+0000 7fab0b4db700 1 -- 172.21.15.22:0/160983643 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab0419aad0 con 0x7fab040ff350 2024-02-16T19:41:36.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:35.991+0000 7faafa7fc700 1 -- 172.21.15.22:0/160983643 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab000055b0 con 0x7fab040ff350 2024-02-16T19:41:36.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:35.991+0000 7fab0b4db700 1 -- 172.21.15.22:0/160983643 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab0419b020 con 0x7fab040ff350 2024-02-16T19:41:36.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:35.992+0000 7fab0b4db700 1 -- 172.21.15.22:0/160983643 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab04107dd0 con 0x7fab040ff350 2024-02-16T19:41:36.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:35.992+0000 7faafa7fc700 1 -- 172.21.15.22:0/160983643 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fab000049a0 con 0x7fab040ff350 2024-02-16T19:41:36.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:35.993+0000 7faafa7fc700 1 --2- 172.21.15.22:0/160983643 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7faaf006d8a0 0x7faaf006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:36.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:35.993+0000 7faafa7fc700 1 -- 172.21.15.22:0/160983643 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6962+0+0 (secure 0 0 0) 0x7fab0008d280 con 0x7fab040ff350 2024-02-16T19:41:36.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:35.993+0000 7fab08a76700 1 --2- 172.21.15.22:0/160983643 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7faaf006d8a0 0x7faaf006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:36.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:35.993+0000 7fab08a76700 1 --2- 172.21.15.22:0/160983643 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7faaf006d8a0 0x7faaf006fd50 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7faaf400b5c0 tx=0x7faaf4005e80 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:36.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:35.998+0000 7faafa7fc700 1 -- 172.21.15.22:0/160983643 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fab00056930 con 0x7fab040ff350 2024-02-16T19:41:36.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:36.254+0000 7fab0b4db700 1 -- 172.21.15.22:0/160983643 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fab0419b2f0 con 0x7fab040ff350 2024-02-16T19:41:36.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:36.255+0000 7faafa7fc700 1 -- 172.21.15.22:0/160983643 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 37 v37) v1 ==== 76+0+1870 (secure 0 0 0) 0x7fab0001d760 con 0x7fab040ff350 2024-02-16T19:41:36.256 INFO:teuthology.orchestra.run.smithi022.stdout:e37 2024-02-16T19:41:36.257 INFO:teuthology.orchestra.run.smithi022.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T19:41:36.257 INFO:teuthology.orchestra.run.smithi022.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:41:36.257 INFO:teuthology.orchestra.run.smithi022.stdout:legacy client fscid: 1 2024-02-16T19:41:36.257 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:41:36.257 INFO:teuthology.orchestra.run.smithi022.stdout:Filesystem 'cephfs' (1) 2024-02-16T19:41:36.257 INFO:teuthology.orchestra.run.smithi022.stdout:fs_name cephfs 2024-02-16T19:41:36.257 INFO:teuthology.orchestra.run.smithi022.stdout:epoch 37 2024-02-16T19:41:36.257 INFO:teuthology.orchestra.run.smithi022.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T19:41:36.258 INFO:teuthology.orchestra.run.smithi022.stdout:created 2024-02-16T19:26:17.873644+0000 2024-02-16T19:41:36.258 INFO:teuthology.orchestra.run.smithi022.stdout:modified 2024-02-16T19:39:34.241312+0000 2024-02-16T19:41:36.258 INFO:teuthology.orchestra.run.smithi022.stdout:tableserver 0 2024-02-16T19:41:36.258 INFO:teuthology.orchestra.run.smithi022.stdout:root 0 2024-02-16T19:41:36.258 INFO:teuthology.orchestra.run.smithi022.stdout:session_timeout 60 2024-02-16T19:41:36.258 INFO:teuthology.orchestra.run.smithi022.stdout:session_autoclose 300 2024-02-16T19:41:36.258 INFO:teuthology.orchestra.run.smithi022.stdout:max_file_size 1099511627776 2024-02-16T19:41:36.258 INFO:teuthology.orchestra.run.smithi022.stdout:max_xattr_size 65536 2024-02-16T19:41:36.259 INFO:teuthology.orchestra.run.smithi022.stdout:required_client_features {} 2024-02-16T19:41:36.259 INFO:teuthology.orchestra.run.smithi022.stdout:last_failure 0 2024-02-16T19:41:36.259 INFO:teuthology.orchestra.run.smithi022.stdout:last_failure_osd_epoch 93 2024-02-16T19:41:36.259 INFO:teuthology.orchestra.run.smithi022.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-16T19:41:36.259 INFO:teuthology.orchestra.run.smithi022.stdout:max_mds 1 2024-02-16T19:41:36.259 INFO:teuthology.orchestra.run.smithi022.stdout:in 0 2024-02-16T19:41:36.259 INFO:teuthology.orchestra.run.smithi022.stdout:up {0=34384} 2024-02-16T19:41:36.259 INFO:teuthology.orchestra.run.smithi022.stdout:failed 2024-02-16T19:41:36.259 INFO:teuthology.orchestra.run.smithi022.stdout:damaged 2024-02-16T19:41:36.260 INFO:teuthology.orchestra.run.smithi022.stdout:stopped 2024-02-16T19:41:36.260 INFO:teuthology.orchestra.run.smithi022.stdout:data_pools [3] 2024-02-16T19:41:36.260 INFO:teuthology.orchestra.run.smithi022.stdout:metadata_pool 2 2024-02-16T19:41:36.260 INFO:teuthology.orchestra.run.smithi022.stdout:inline_data enabled 2024-02-16T19:41:36.260 INFO:teuthology.orchestra.run.smithi022.stdout:balancer 2024-02-16T19:41:36.260 INFO:teuthology.orchestra.run.smithi022.stdout:bal_rank_mask -1 2024-02-16T19:41:36.260 INFO:teuthology.orchestra.run.smithi022.stdout:standby_count_wanted 1 2024-02-16T19:41:36.260 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.cephfs.smithi022.ocukvs{0:34384} state up:active seq 22 join_fscid=1 addr [v2:172.21.15.22:6826/2535680483,v1:172.21.15.22:6827/2535680483] compat {c=[1],r=[1],i=[fff]}] 2024-02-16T19:41:36.260 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:41:36.260 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:41:36.260 INFO:teuthology.orchestra.run.smithi022.stdout:Standby daemons: 2024-02-16T19:41:36.260 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:41:36.260 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.cephfs.smithi022.btppwm{-1:34408} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.22:6828/3126701730,v1:172.21.15.22:6829/3126701730] compat {c=[1],r=[1],i=[fff]}] 2024-02-16T19:41:36.260 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.cephfs.smithi117.ibmppn{-1:44299} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6824/2389644474,v1:172.21.15.117:6825/2389644474] compat {c=[1],r=[1],i=[fff]}] 2024-02-16T19:41:36.261 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.cephfs.smithi117.orzors{-1:44325} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6826/166927597,v1:172.21.15.117:6827/166927597] compat {c=[1],r=[1],i=[fff]}] 2024-02-16T19:41:36.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:36.257+0000 7fab0b4db700 1 -- 172.21.15.22:0/160983643 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7faaf006d8a0 msgr2=0x7faaf006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:36.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:36.257+0000 7fab0b4db700 1 --2- 172.21.15.22:0/160983643 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7faaf006d8a0 0x7faaf006fd50 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7faaf400b5c0 tx=0x7faaf4005e80 comp rx=0 tx=0).stop 2024-02-16T19:41:36.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:36.258+0000 7fab0b4db700 1 -- 172.21.15.22:0/160983643 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab040ff350 msgr2=0x7fab04195e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:36.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:36.258+0000 7fab0b4db700 1 --2- 172.21.15.22:0/160983643 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab040ff350 0x7fab04195e40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fab0000c8a0 tx=0x7fab0000cc60 comp rx=0 tx=0).stop 2024-02-16T19:41:36.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:36.258+0000 7fab0b4db700 1 -- 172.21.15.22:0/160983643 shutdown_connections 2024-02-16T19:41:36.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:36.258+0000 7fab0b4db700 1 --2- 172.21.15.22:0/160983643 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7faaf006d8a0 0x7faaf006fd50 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:36.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:36.258+0000 7fab0b4db700 1 --2- 172.21.15.22:0/160983643 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab040ff350 0x7fab04195e40 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:36.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:36.258+0000 7fab0b4db700 1 --2- 172.21.15.22:0/160983643 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab040ffc60 0x7fab04196380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:36.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:36.258+0000 7fab0b4db700 1 -- 172.21.15.22:0/160983643 >> 172.21.15.22:0/160983643 conn(0x7fab040748b0 msgr2=0x7fab04104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:36.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:36.258+0000 7fab0b4db700 1 -- 172.21.15.22:0/160983643 shutdown_connections 2024-02-16T19:41:36.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:36.258+0000 7fab0b4db700 1 -- 172.21.15.22:0/160983643 wait complete. 2024-02-16T19:41:36.262 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 37 2024-02-16T19:41:36.615 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:41:36 smithi022 ceph-mon[88798]: from='client.? 172.21.15.22:0/160983643' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:41:36.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:41:36 smithi117 ceph-mon[67870]: from='client.? 172.21.15.22:0/160983643' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:41:36.953 INFO:teuthology.run_tasks:Running task fs.post_upgrade_checks... 2024-02-16T19:41:36.967 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-16T19:41:37.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.071+0000 7f229504a640 1 -- 172.21.15.22:0/499477886 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f22901011e0 msgr2=0x7f22901096d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:37.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.071+0000 7f229504a640 1 --2- 172.21.15.22:0/499477886 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f22901011e0 0x7f22901096d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2280009880 tx=0x7f228002bd70 comp rx=0 tx=0).stop 2024-02-16T19:41:37.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.071+0000 7f229504a640 1 -- 172.21.15.22:0/499477886 shutdown_connections 2024-02-16T19:41:37.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.071+0000 7f229504a640 1 --2- 172.21.15.22:0/499477886 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f22901011e0 0x7f22901096d0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:37.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.071+0000 7f229504a640 1 --2- 172.21.15.22:0/499477886 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22901008d0 0x7f2290100ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:37.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.071+0000 7f229504a640 1 -- 172.21.15.22:0/499477886 >> 172.21.15.22:0/499477886 conn(0x7f2290075270 msgr2=0x7f2290075670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:37.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.071+0000 7f229504a640 1 -- 172.21.15.22:0/499477886 shutdown_connections 2024-02-16T19:41:37.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.071+0000 7f229504a640 1 -- 172.21.15.22:0/499477886 wait complete. 2024-02-16T19:41:37.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.071+0000 7f229504a640 1 Processor -- start 2024-02-16T19:41:37.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.072+0000 7f229504a640 1 -- start start 2024-02-16T19:41:37.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.072+0000 7f229504a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f22901008d0 0x7f2290195260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:37.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.072+0000 7f229504a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22901011e0 0x7f22901957a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:37.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.072+0000 7f229504a640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2290195e30 con 0x7f22901008d0 2024-02-16T19:41:37.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.072+0000 7f229504a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2290199c00 con 0x7f22901011e0 2024-02-16T19:41:37.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.072+0000 7f228ed9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f22901008d0 0x7f2290195260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:37.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.072+0000 7f228ed9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f22901008d0 0x7f2290195260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45404/0 (socket says 172.21.15.22:45404) 2024-02-16T19:41:37.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.072+0000 7f228ed9d640 1 -- 172.21.15.22:0/386281153 learned_addr learned my addr 172.21.15.22:0/386281153 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:41:37.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.072+0000 7f228e59c640 1 --2- 172.21.15.22:0/386281153 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22901011e0 0x7f22901957a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:37.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.072+0000 7f228ed9d640 1 -- 172.21.15.22:0/386281153 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22901011e0 msgr2=0x7f22901957a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:37.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.072+0000 7f228ed9d640 1 --2- 172.21.15.22:0/386281153 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22901011e0 0x7f22901957a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:37.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.072+0000 7f228ed9d640 1 -- 172.21.15.22:0/386281153 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2280009560 con 0x7f22901008d0 2024-02-16T19:41:37.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.073+0000 7f228ed9d640 1 --2- 172.21.15.22:0/386281153 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f22901008d0 0x7f2290195260 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f228400e970 tx=0x7f228400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:37.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.073+0000 7f2277fff640 1 -- 172.21.15.22:0/386281153 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f228400cc90 con 0x7f22901008d0 2024-02-16T19:41:37.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.073+0000 7f229504a640 1 -- 172.21.15.22:0/386281153 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2290199ee0 con 0x7f22901008d0 2024-02-16T19:41:37.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.073+0000 7f2277fff640 1 -- 172.21.15.22:0/386281153 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f228400cdf0 con 0x7f22901008d0 2024-02-16T19:41:37.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.073+0000 7f2277fff640 1 -- 172.21.15.22:0/386281153 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2284010430 con 0x7f22901008d0 2024-02-16T19:41:37.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.073+0000 7f229504a640 1 -- 172.21.15.22:0/386281153 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f229019a410 con 0x7f22901008d0 2024-02-16T19:41:37.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.074+0000 7f229504a640 1 -- 172.21.15.22:0/386281153 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f229004e000 con 0x7f22901008d0 2024-02-16T19:41:37.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.075+0000 7f2277fff640 1 -- 172.21.15.22:0/386281153 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f22840046c0 con 0x7f22901008d0 2024-02-16T19:41:37.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.075+0000 7f2277fff640 1 --2- 172.21.15.22:0/386281153 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f227806d8d0 0x7f227806fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:37.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.075+0000 7f2277fff640 1 -- 172.21.15.22:0/386281153 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f2284014070 con 0x7f22901008d0 2024-02-16T19:41:37.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.075+0000 7f228e59c640 1 --2- 172.21.15.22:0/386281153 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f227806d8d0 0x7f227806fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:37.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.076+0000 7f228e59c640 1 --2- 172.21.15.22:0/386281153 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f227806d8d0 0x7f227806fd80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2280006010 tx=0x7f2280005dd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:37.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.080+0000 7f2277fff640 1 -- 172.21.15.22:0/386281153 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2284057230 con 0x7f22901008d0 2024-02-16T19:41:37.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.276+0000 7f229504a640 1 -- 172.21.15.22:0/386281153 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f229010c880 con 0x7f22901008d0 2024-02-16T19:41:37.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.277+0000 7f2277fff640 1 -- 172.21.15.22:0/386281153 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+17322 (secure 0 0 0) 0x7f228405a8f0 con 0x7f22901008d0 2024-02-16T19:41:37.279 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:41:37.279 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":93,"fsid":"2a1346e6-cd00-11ee-95ba-87774f69a715","created":"2024-02-16T19:19:11.528280+0000","modified":"2024-02-16T19:38:57.804537+0000","last_up_change":"2024-02-16T19:37:45.909206+0000","last_in_change":"2024-02-16T19:24:40.575373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-16T19:23:47.101426+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-02-16T19:26:16.533404+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-02-16T19:26:17.113076+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d16e47f2-7209-45f4-ad50-1e0f6789ffad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":50,"up_from":54,"up_thru":88,"down_at":51,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":1744550481},{"type":"v1","addr":"172.21.15.22:6803","nonce":1744550481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":1744550481},{"type":"v1","addr":"172.21.15.22:6805","nonce":1744550481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":1744550481},{"type":"v1","addr":"172.21.15.22:6809","nonce":1744550481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":1744550481},{"type":"v1","addr":"172.21.15.22:6807","nonce":1744550481}]},"public_addr":"172.21.15.22:6803/1744550481","cluster_addr":"172.21.15.22:6805/1744550481","heartbeat_back_addr":"172.21.15.22:6809/1744550481","heartbeat_front_addr":"172.21.15.22:6807/1744550481","state":["exists","up"]},{"osd":1,"uuid":"59c3d085-3422-4e92-9048-46d7a39e31a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":55,"up_from":64,"up_thru":88,"down_at":56,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":3321991378},{"type":"v1","addr":"172.21.15.22:6811","nonce":3321991378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":3321991378},{"type":"v1","addr":"172.21.15.22:6813","nonce":3321991378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":3321991378},{"type":"v1","addr":"172.21.15.22:6817","nonce":3321991378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":3321991378},{"type":"v1","addr":"172.21.15.22:6815","nonce":3321991378}]},"public_addr":"172.21.15.22:6811/3321991378","cluster_addr":"172.21.15.22:6813/3321991378","heartbeat_back_addr":"172.21.15.22:6817/3321991378","heartbeat_front_addr":"172.21.15.22:6815/3321991378","state":["exists","up"]},{"osd":2,"uuid":"5952e710-f0b6-406c-855a-00f1cb8059c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":69,"up_from":73,"up_thru":88,"down_at":70,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":2098766749},{"type":"v1","addr":"172.21.15.22:6819","nonce":2098766749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":2098766749},{"type":"v1","addr":"172.21.15.22:6821","nonce":2098766749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":2098766749},{"type":"v1","addr":"172.21.15.22:6825","nonce":2098766749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":2098766749},{"type":"v1","addr":"172.21.15.22:6823","nonce":2098766749}]},"public_addr":"172.21.15.22:6819/2098766749","cluster_addr":"172.21.15.22:6821/2098766749","heartbeat_back_addr":"172.21.15.22:6825/2098766749","heartbeat_front_addr":"172.21.15.22:6823/2098766749","state":["exists","up"]},{"osd":3,"uuid":"5dc87494-e400-4111-91a7-306d9905febb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":27,"last_clean_end":74,"up_from":78,"up_thru":88,"down_at":75,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3703365528},{"type":"v1","addr":"172.21.15.117:6801","nonce":3703365528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":3703365528},{"type":"v1","addr":"172.21.15.117:6803","nonce":3703365528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":3703365528},{"type":"v1","addr":"172.21.15.117:6807","nonce":3703365528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":3703365528},{"type":"v1","addr":"172.21.15.117:6805","nonce":3703365528}]},"public_addr":"172.21.15.117:6801/3703365528","cluster_addr":"172.21.15.117:6803/3703365528","heartbeat_back_addr":"172.21.15.117:6807/3703365528","heartbeat_front_addr":"172.21.15.117:6805/3703365528","state":["exists","up"]},{"osd":4,"uuid":"d27bde8d-1a77-4f95-acb6-7d05becd9480","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":35,"last_clean_end":79,"up_from":83,"up_thru":88,"down_at":80,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":3679983685},{"type":"v1","addr":"172.21.15.117:6809","nonce":3679983685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":3679983685},{"type":"v1","addr":"172.21.15.117:6811","nonce":3679983685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":3679983685},{"type":"v1","addr":"172.21.15.117:6815","nonce":3679983685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":3679983685},{"type":"v1","addr":"172.21.15.117:6813","nonce":3679983685}]},"public_addr":"172.21.15.117:6809/3679983685","cluster_addr":"172.21.15.117:6811/3679983685","heartbeat_back_addr":"172.21.15.117:6815/3679983685","heartbeat_front_addr":"172.21.15.117:6813/3679983685","state":["exists","up"]},{"osd":5,"uuid":"5873798f-3a2c-4a29-becd-6280b8866b6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":40,"last_clean_end":84,"up_from":88,"up_thru":88,"down_at":85,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1403457815},{"type":"v1","addr":"172.21.15.117:6817","nonce":1403457815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1403457815},{"type":"v1","addr":"172.21.15.117:6819","nonce":1403457815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1403457815},{"type":"v1","addr":"172.21.15.117:6823","nonce":1403457815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1403457815},{"type":"v1","addr":"172.21.15.117:6821","nonce":1403457815}]},"public_addr":"172.21.15.117:6817/1403457815","cluster_addr":"172.21.15.117:6819/1403457815","heartbeat_back_addr":"172.21.15.117:6823/1403457815","heartbeat_front_addr":"172.21.15.117:6821/1403457815","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:22:44.777427+0000","dead_epoch":50},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:23:14.935167+0000","dead_epoch":55},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:23:44.496184+0000","dead_epoch":69},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:24:07.447481+0000","dead_epoch":74},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:24:32.205064+0000","dead_epoch":79},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:24:57.982560+0000","dead_epoch":84}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":3,"to":2}]},{"pgid":"2.5","mappings":[{"from":3,"to":1}]},{"pgid":"2.6","mappings":[{"from":3,"to":2}]},{"pgid":"2.9","mappings":[{"from":0,"to":2}]},{"pgid":"2.10","mappings":[{"from":0,"to":5}]},{"pgid":"2.1e","mappings":[{"from":0,"to":1}]},{"pgid":"3.3","mappings":[{"from":4,"to":1}]},{"pgid":"3.f","mappings":[{"from":3,"to":1}]},{"pgid":"3.16","mappings":[{"from":3,"to":0}]},{"pgid":"3.1b","mappings":[{"from":4,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.117:6827/3122342028":"2024-02-17T19:38:57.804505+0000","172.21.15.117:6826/3122342028":"2024-02-17T19:38:57.804505+0000","172.21.15.22:6829/3597685751":"2024-02-17T19:38:09.297300+0000","172.21.15.22:6828/3597685751":"2024-02-17T19:38:09.297300+0000","172.21.15.22:0/4238358751":"2024-02-17T19:33:15.748281+0000","172.21.15.22:6801/2755289428":"2024-02-17T19:20:11.619146+0000","172.21.15.22:6800/2253867279":"2024-02-17T19:28:59.405724+0000","172.21.15.22:0/2301835367":"2024-02-17T19:28:59.405724+0000","172.21.15.22:6801/3109015836":"2024-02-17T19:19:38.544291+0000","172.21.15.117:6825/565408102":"2024-02-17T19:38:31.770686+0000","172.21.15.22:0/206933569":"2024-02-17T19:28:59.405724+0000","172.21.15.22:0/3012142271":"2024-02-17T19:33:15.748281+0000","172.21.15.117:0/2453907739":"2024-02-17T19:29:53.194933+0000","172.21.15.22:6800/3109015836":"2024-02-17T19:19:38.544291+0000","172.21.15.117:0/3042073611":"2024-02-17T19:29:53.194933+0000","172.21.15.117:0/4194262516":"2024-02-17T19:29:53.194933+0000","172.21.15.22:6800/2945983569":"2024-02-17T19:21:26.760165+0000","172.21.15.22:0/1603216752":"2024-02-17T19:19:38.544291+0000","172.21.15.22:6827/149959963":"2024-02-17T19:26:32.932573+0000","172.21.15.22:6801/2945983569":"2024-02-17T19:21:26.760165+0000","172.21.15.22:0/668279955":"2024-02-17T19:21:26.760165+0000","172.21.15.22:0/2705605267":"2024-02-17T19:28:59.405724+0000","172.21.15.117:6824/565408102":"2024-02-17T19:38:31.770686+0000","172.21.15.22:6800/2755289428":"2024-02-17T19:20:11.619146+0000","172.21.15.22:0/2099601555":"2024-02-17T19:20:11.619146+0000","172.21.15.22:6801/2253867279":"2024-02-17T19:28:59.405724+0000","172.21.15.117:0/333707173":"2024-02-17T19:29:53.194933+0000","172.21.15.117:6828/10500008":"2024-02-17T19:29:53.194933+0000","172.21.15.117:0/199482916":"2024-02-17T19:29:53.194933+0000","172.21.15.117:6829/10500008":"2024-02-17T19:29:53.194933+0000","172.21.15.22:0/2017591724":"2024-02-17T19:20:11.619146+0000","172.21.15.22:0/2371245756":"2024-02-17T19:19:38.544291+0000","172.21.15.22:6801/3703806543":"2024-02-17T19:33:15.748281+0000","172.21.15.22:0/2384083375":"2024-02-17T19:21:26.760165+0000","172.21.15.22:0/402850490":"2024-02-17T19:28:59.405724+0000","172.21.15.22:0/1229330195":"2024-02-17T19:20:11.619146+0000","172.21.15.22:0/8466130":"2024-02-17T19:19:38.544291+0000","172.21.15.22:0/3200629226":"2024-02-17T19:21:26.760165+0000","172.21.15.117:0/2264991341":"2024-02-17T19:29:53.194933+0000","172.21.15.22:0/2853892105":"2024-02-17T19:33:15.748281+0000","172.21.15.22:6826/149959963":"2024-02-17T19:26:32.932573+0000","172.21.15.22:6800/3703806543":"2024-02-17T19:33:15.748281+0000","172.21.15.22:0/2353857756":"2024-02-17T19:33:15.748281+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-16T19:41:37.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.279+0000 7f229504a640 1 -- 172.21.15.22:0/386281153 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f227806d8d0 msgr2=0x7f227806fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:37.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.279+0000 7f229504a640 1 --2- 172.21.15.22:0/386281153 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f227806d8d0 0x7f227806fd80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2280006010 tx=0x7f2280005dd0 comp rx=0 tx=0).stop 2024-02-16T19:41:37.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.279+0000 7f229504a640 1 -- 172.21.15.22:0/386281153 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f22901008d0 msgr2=0x7f2290195260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:37.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.279+0000 7f229504a640 1 --2- 172.21.15.22:0/386281153 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f22901008d0 0x7f2290195260 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f228400e970 tx=0x7f228400ed70 comp rx=0 tx=0).stop 2024-02-16T19:41:37.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.279+0000 7f229504a640 1 -- 172.21.15.22:0/386281153 shutdown_connections 2024-02-16T19:41:37.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.279+0000 7f229504a640 1 --2- 172.21.15.22:0/386281153 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f227806d8d0 0x7f227806fd80 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:37.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.279+0000 7f229504a640 1 --2- 172.21.15.22:0/386281153 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22901011e0 0x7f22901957a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:37.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.279+0000 7f229504a640 1 --2- 172.21.15.22:0/386281153 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f22901008d0 0x7f2290195260 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:37.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.279+0000 7f229504a640 1 -- 172.21.15.22:0/386281153 >> 172.21.15.22:0/386281153 conn(0x7f2290075270 msgr2=0x7f2290103e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:37.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.280+0000 7f229504a640 1 -- 172.21.15.22:0/386281153 shutdown_connections 2024-02-16T19:41:37.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.280+0000 7f229504a640 1 -- 172.21.15.22:0/386281153 wait complete. 2024-02-16T19:41:37.292 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-16T19:23:47.101426+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-02-16T19:26:16.533404+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-02-16T19:26:17.113076+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-16T19:41:37.293 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-16T19:41:37.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.399+0000 7f00dd4e0640 1 -- 172.21.15.22:0/601343232 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00d80fe7f0 msgr2=0x7f00d80fec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:37.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.399+0000 7f00dd4e0640 1 --2- 172.21.15.22:0/601343232 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00d80fe7f0 0x7f00d80fec60 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f00cc009880 tx=0x7f00cc02bce0 comp rx=0 tx=0).stop 2024-02-16T19:41:37.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.399+0000 7f00dd4e0640 1 -- 172.21.15.22:0/601343232 shutdown_connections 2024-02-16T19:41:37.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.399+0000 7f00dd4e0640 1 --2- 172.21.15.22:0/601343232 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00d80fe7f0 0x7f00d80fec60 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:37.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.399+0000 7f00dd4e0640 1 --2- 172.21.15.22:0/601343232 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00d8104800 0x7f00d8104bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:37.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.399+0000 7f00dd4e0640 1 -- 172.21.15.22:0/601343232 >> 172.21.15.22:0/601343232 conn(0x7f00d80fa2f0 msgr2=0x7f00d80fc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:37.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.399+0000 7f00dd4e0640 1 -- 172.21.15.22:0/601343232 shutdown_connections 2024-02-16T19:41:37.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.399+0000 7f00dd4e0640 1 -- 172.21.15.22:0/601343232 wait complete. 2024-02-16T19:41:37.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.399+0000 7f00dd4e0640 1 Processor -- start 2024-02-16T19:41:37.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.400+0000 7f00dd4e0640 1 -- start start 2024-02-16T19:41:37.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.400+0000 7f00dd4e0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00d80fe7f0 0x7f00d81954f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:37.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.400+0000 7f00dd4e0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00d8104800 0x7f00d8195a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:37.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.400+0000 7f00dd4e0640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d81960c0 con 0x7f00d8104800 2024-02-16T19:41:37.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.400+0000 7f00dd4e0640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d8199e90 con 0x7f00d80fe7f0 2024-02-16T19:41:37.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.400+0000 7f00d67fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00d8104800 0x7f00d8195a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:37.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.400+0000 7f00d67fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00d8104800 0x7f00d8195a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45428/0 (socket says 172.21.15.22:45428) 2024-02-16T19:41:37.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.400+0000 7f00d67fc640 1 -- 172.21.15.22:0/544395856 learned_addr learned my addr 172.21.15.22:0/544395856 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:41:37.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.400+0000 7f00d6ffd640 1 --2- 172.21.15.22:0/544395856 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00d80fe7f0 0x7f00d81954f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:37.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.400+0000 7f00d67fc640 1 -- 172.21.15.22:0/544395856 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00d80fe7f0 msgr2=0x7f00d81954f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:37.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.400+0000 7f00d67fc640 1 --2- 172.21.15.22:0/544395856 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00d80fe7f0 0x7f00d81954f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:37.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.400+0000 7f00d67fc640 1 -- 172.21.15.22:0/544395856 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00cc009560 con 0x7f00d8104800 2024-02-16T19:41:37.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.400+0000 7f00d67fc640 1 --2- 172.21.15.22:0/544395856 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00d8104800 0x7f00d8195a30 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f00cc009230 tx=0x7f00cc005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:37.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.401+0000 7f00bffff640 1 -- 172.21.15.22:0/544395856 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f00cc0044a0 con 0x7f00d8104800 2024-02-16T19:41:37.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.401+0000 7f00bffff640 1 -- 172.21.15.22:0/544395856 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f00cc003ab0 con 0x7f00d8104800 2024-02-16T19:41:37.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.401+0000 7f00bffff640 1 -- 172.21.15.22:0/544395856 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f00cc0055f0 con 0x7f00d8104800 2024-02-16T19:41:37.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.401+0000 7f00dd4e0640 1 -- 172.21.15.22:0/544395856 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00d819a110 con 0x7f00d8104800 2024-02-16T19:41:37.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.401+0000 7f00dd4e0640 1 -- 172.21.15.22:0/544395856 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00d819a690 con 0x7f00d8104800 2024-02-16T19:41:37.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.402+0000 7f00dd4e0640 1 -- 172.21.15.22:0/544395856 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00d804e000 con 0x7f00d8104800 2024-02-16T19:41:37.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.404+0000 7f00bffff640 1 -- 172.21.15.22:0/544395856 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f00cc004e60 con 0x7f00d8104800 2024-02-16T19:41:37.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.404+0000 7f00bffff640 1 --2- 172.21.15.22:0/544395856 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f00c006d860 0x7f00c006fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:37.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.404+0000 7f00bffff640 1 -- 172.21.15.22:0/544395856 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f00cc0af400 con 0x7f00d8104800 2024-02-16T19:41:37.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.404+0000 7f00d6ffd640 1 --2- 172.21.15.22:0/544395856 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f00c006d860 0x7f00c006fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:37.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.405+0000 7f00d6ffd640 1 --2- 172.21.15.22:0/544395856 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f00c006d860 0x7f00c006fd10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f00d80ff930 tx=0x7f00c8005f90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:37.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.409+0000 7f00bffff640 1 -- 172.21.15.22:0/544395856 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f00cc078ab0 con 0x7f00d8104800 2024-02-16T19:41:37.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.589+0000 7f00dd4e0640 1 -- 172.21.15.22:0/544395856 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f00d810b9e0 con 0x7f00d8104800 2024-02-16T19:41:37.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.590+0000 7f00bffff640 1 -- 172.21.15.22:0/544395856 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7f00cc07c170 con 0x7f00d8104800 2024-02-16T19:41:37.591 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-02-16T19:41:37.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.592+0000 7f00dd4e0640 1 -- 172.21.15.22:0/544395856 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f00c006d860 msgr2=0x7f00c006fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:37.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.592+0000 7f00dd4e0640 1 --2- 172.21.15.22:0/544395856 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f00c006d860 0x7f00c006fd10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f00d80ff930 tx=0x7f00c8005f90 comp rx=0 tx=0).stop 2024-02-16T19:41:37.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.592+0000 7f00dd4e0640 1 -- 172.21.15.22:0/544395856 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00d8104800 msgr2=0x7f00d8195a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:37.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.592+0000 7f00dd4e0640 1 --2- 172.21.15.22:0/544395856 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00d8104800 0x7f00d8195a30 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f00cc009230 tx=0x7f00cc005e50 comp rx=0 tx=0).stop 2024-02-16T19:41:37.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.592+0000 7f00dd4e0640 1 -- 172.21.15.22:0/544395856 shutdown_connections 2024-02-16T19:41:37.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.592+0000 7f00dd4e0640 1 --2- 172.21.15.22:0/544395856 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f00c006d860 0x7f00c006fd10 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:37.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.592+0000 7f00dd4e0640 1 --2- 172.21.15.22:0/544395856 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00d8104800 0x7f00d8195a30 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:37.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.592+0000 7f00dd4e0640 1 --2- 172.21.15.22:0/544395856 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00d80fe7f0 0x7f00d81954f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:37.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.592+0000 7f00dd4e0640 1 -- 172.21.15.22:0/544395856 >> 172.21.15.22:0/544395856 conn(0x7f00d80fa2f0 msgr2=0x7f00d81029e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:37.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.592+0000 7f00dd4e0640 1 -- 172.21.15.22:0/544395856 shutdown_connections 2024-02-16T19:41:37.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.592+0000 7f00dd4e0640 1 -- 172.21.15.22:0/544395856 wait complete. 2024-02-16T19:41:37.603 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:41:37 smithi022 ceph-mon[88798]: pgmap v296: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:41:37.603 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:41:37 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:41:37.603 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:41:37 smithi022 ceph-mon[88798]: from='client.? 172.21.15.22:0/386281153' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-16T19:41:37.603 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:41:37 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:41:37.603 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:41:37 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:41:37.603 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:41:37 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:41:37.604 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-02-16T19:41:37.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.708+0000 7feb0496e640 1 -- 172.21.15.22:0/1464193095 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb000fe7f0 msgr2=0x7feb000fec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:37.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.708+0000 7feb0496e640 1 --2- 172.21.15.22:0/1464193095 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb000fe7f0 0x7feb000fec60 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7feae8009880 tx=0x7feae802bce0 comp rx=0 tx=0).stop 2024-02-16T19:41:37.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.708+0000 7feb0496e640 1 -- 172.21.15.22:0/1464193095 shutdown_connections 2024-02-16T19:41:37.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.708+0000 7feb0496e640 1 --2- 172.21.15.22:0/1464193095 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb000fe7f0 0x7feb000fec60 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:37.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.708+0000 7feb0496e640 1 --2- 172.21.15.22:0/1464193095 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb00104800 0x7feb00104bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:37.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.708+0000 7feb0496e640 1 -- 172.21.15.22:0/1464193095 >> 172.21.15.22:0/1464193095 conn(0x7feb000fa2f0 msgr2=0x7feb000fc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:37.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.708+0000 7feb0496e640 1 -- 172.21.15.22:0/1464193095 shutdown_connections 2024-02-16T19:41:37.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.708+0000 7feb0496e640 1 -- 172.21.15.22:0/1464193095 wait complete. 2024-02-16T19:41:37.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.709+0000 7feb0496e640 1 Processor -- start 2024-02-16T19:41:37.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.709+0000 7feb0496e640 1 -- start start 2024-02-16T19:41:37.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.709+0000 7feb0496e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb000fe7f0 0x7feb00195540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:37.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.709+0000 7feb0496e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb00104800 0x7feb00195a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:37.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.709+0000 7feb0496e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb00196110 con 0x7feb00104800 2024-02-16T19:41:37.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.709+0000 7feb0496e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb00199ee0 con 0x7feb000fe7f0 2024-02-16T19:41:37.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.709+0000 7feafdd9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb00104800 0x7feb00195a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:37.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.709+0000 7feafdd9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb00104800 0x7feb00195a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45456/0 (socket says 172.21.15.22:45456) 2024-02-16T19:41:37.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.709+0000 7feafdd9b640 1 -- 172.21.15.22:0/1700822627 learned_addr learned my addr 172.21.15.22:0/1700822627 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:41:37.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.709+0000 7feafe59c640 1 --2- 172.21.15.22:0/1700822627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb000fe7f0 0x7feb00195540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:37.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.709+0000 7feafdd9b640 1 -- 172.21.15.22:0/1700822627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb000fe7f0 msgr2=0x7feb00195540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:37.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.709+0000 7feafdd9b640 1 --2- 172.21.15.22:0/1700822627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb000fe7f0 0x7feb00195540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:37.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.709+0000 7feafdd9b640 1 -- 172.21.15.22:0/1700822627 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feae8009560 con 0x7feb00104800 2024-02-16T19:41:37.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.709+0000 7feafdd9b640 1 --2- 172.21.15.22:0/1700822627 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb00104800 0x7feb00195a80 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7feae8009230 tx=0x7feae8004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:37.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.710+0000 7feaf77fe640 1 -- 172.21.15.22:0/1700822627 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feae80035f0 con 0x7feb00104800 2024-02-16T19:41:37.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.710+0000 7feaf77fe640 1 -- 172.21.15.22:0/1700822627 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feae8003c10 con 0x7feb00104800 2024-02-16T19:41:37.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.710+0000 7feb0496e640 1 -- 172.21.15.22:0/1700822627 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb0019a160 con 0x7feb00104800 2024-02-16T19:41:37.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.710+0000 7feaf77fe640 1 -- 172.21.15.22:0/1700822627 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feae8005670 con 0x7feb00104800 2024-02-16T19:41:37.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.710+0000 7feb0496e640 1 -- 172.21.15.22:0/1700822627 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb0019a660 con 0x7feb00104800 2024-02-16T19:41:37.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.711+0000 7feb0496e640 1 -- 172.21.15.22:0/1700822627 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb0004e000 con 0x7feb00104800 2024-02-16T19:41:37.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.711+0000 7feaf77fe640 1 -- 172.21.15.22:0/1700822627 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7feae8004dd0 con 0x7feb00104800 2024-02-16T19:41:37.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.712+0000 7feaf77fe640 1 --2- 172.21.15.22:0/1700822627 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7feaec06d7c0 0x7feaec06fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:37.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.712+0000 7feafe59c640 1 --2- 172.21.15.22:0/1700822627 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7feaec06d7c0 0x7feaec06fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:37.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.712+0000 7feaf77fe640 1 -- 172.21.15.22:0/1700822627 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6962+0+0 (secure 0 0 0) 0x7feae80af9e0 con 0x7feb00104800 2024-02-16T19:41:37.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.712+0000 7feafe59c640 1 --2- 172.21.15.22:0/1700822627 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7feaec06d7c0 0x7feaec06fc70 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7feb000ff930 tx=0x7feaf8009720 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:37.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.716+0000 7feaf77fe640 1 -- 172.21.15.22:0/1700822627 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7feae80790c0 con 0x7feb00104800 2024-02-16T19:41:37.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:41:37 smithi117 ceph-mon[67870]: pgmap v296: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:41:37.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:41:37 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:41:37.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:41:37 smithi117 ceph-mon[67870]: from='client.? 172.21.15.22:0/386281153' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-16T19:41:37.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:41:37 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:41:37.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:41:37 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:41:37.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:41:37 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:41:37.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.898+0000 7feb0496e640 1 -- 172.21.15.22:0/1700822627 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7feb00066860 con 0x7feb00104800 2024-02-16T19:41:37.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.898+0000 7feaf77fe640 1 -- 172.21.15.22:0/1700822627 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v93) v1 ==== 107+0+11 (secure 0 0 0) 0x7feae807c780 con 0x7feb00104800 2024-02-16T19:41:37.900 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-02-16T19:41:37.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.900+0000 7feb0496e640 1 -- 172.21.15.22:0/1700822627 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7feaec06d7c0 msgr2=0x7feaec06fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:37.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.900+0000 7feb0496e640 1 --2- 172.21.15.22:0/1700822627 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7feaec06d7c0 0x7feaec06fc70 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7feb000ff930 tx=0x7feaf8009720 comp rx=0 tx=0).stop 2024-02-16T19:41:37.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.900+0000 7feb0496e640 1 -- 172.21.15.22:0/1700822627 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb00104800 msgr2=0x7feb00195a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:37.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.900+0000 7feb0496e640 1 --2- 172.21.15.22:0/1700822627 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb00104800 0x7feb00195a80 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7feae8009230 tx=0x7feae8004040 comp rx=0 tx=0).stop 2024-02-16T19:41:37.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.901+0000 7feb0496e640 1 -- 172.21.15.22:0/1700822627 shutdown_connections 2024-02-16T19:41:37.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.901+0000 7feb0496e640 1 --2- 172.21.15.22:0/1700822627 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7feaec06d7c0 0x7feaec06fc70 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:37.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.901+0000 7feb0496e640 1 --2- 172.21.15.22:0/1700822627 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb00104800 0x7feb00195a80 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:37.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.901+0000 7feb0496e640 1 --2- 172.21.15.22:0/1700822627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb000fe7f0 0x7feb00195540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:37.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.901+0000 7feb0496e640 1 -- 172.21.15.22:0/1700822627 >> 172.21.15.22:0/1700822627 conn(0x7feb000fa2f0 msgr2=0x7feb000683e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:37.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.901+0000 7feb0496e640 1 -- 172.21.15.22:0/1700822627 shutdown_connections 2024-02-16T19:41:37.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:37.901+0000 7feb0496e640 1 -- 172.21.15.22:0/1700822627 wait complete. 2024-02-16T19:41:37.913 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-02-16T19:41:38.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.016+0000 7fa84e5fb640 1 -- 172.21.15.22:0/2537598043 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa848100a30 msgr2=0x7fa848100e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:38.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.016+0000 7fa84e5fb640 1 --2- 172.21.15.22:0/2537598043 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa848100a30 0x7fa848100e00 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa83c0098b0 tx=0x7fa83c02bcd0 comp rx=0 tx=0).stop 2024-02-16T19:41:38.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.016+0000 7fa84e5fb640 1 -- 172.21.15.22:0/2537598043 shutdown_connections 2024-02-16T19:41:38.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.016+0000 7fa84e5fb640 1 --2- 172.21.15.22:0/2537598043 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa848101340 0x7fa848109830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:38.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.016+0000 7fa84e5fb640 1 --2- 172.21.15.22:0/2537598043 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa848100a30 0x7fa848100e00 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:38.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.016+0000 7fa84e5fb640 1 -- 172.21.15.22:0/2537598043 >> 172.21.15.22:0/2537598043 conn(0x7fa8480fa260 msgr2=0x7fa8480fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:38.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.016+0000 7fa84e5fb640 1 -- 172.21.15.22:0/2537598043 shutdown_connections 2024-02-16T19:41:38.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.016+0000 7fa84e5fb640 1 -- 172.21.15.22:0/2537598043 wait complete. 2024-02-16T19:41:38.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.017+0000 7fa84e5fb640 1 Processor -- start 2024-02-16T19:41:38.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.017+0000 7fa84e5fb640 1 -- start start 2024-02-16T19:41:38.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.017+0000 7fa84e5fb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa848100a30 0x7fa848195590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:38.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.017+0000 7fa84e5fb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa848101340 0x7fa848195ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:38.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.017+0000 7fa84e5fb640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa848196160 con 0x7fa848100a30 2024-02-16T19:41:38.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.017+0000 7fa84e5fb640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa848199f30 con 0x7fa848101340 2024-02-16T19:41:38.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.017+0000 7fa847fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa848100a30 0x7fa848195590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:38.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.017+0000 7fa847fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa848100a30 0x7fa848195590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45480/0 (socket says 172.21.15.22:45480) 2024-02-16T19:41:38.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.017+0000 7fa847fff640 1 -- 172.21.15.22:0/1274023722 learned_addr learned my addr 172.21.15.22:0/1274023722 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:41:38.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.017+0000 7fa8477fe640 1 --2- 172.21.15.22:0/1274023722 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa848101340 0x7fa848195ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:38.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.018+0000 7fa847fff640 1 -- 172.21.15.22:0/1274023722 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa848101340 msgr2=0x7fa848195ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:38.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.018+0000 7fa847fff640 1 --2- 172.21.15.22:0/1274023722 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa848101340 0x7fa848195ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:38.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.018+0000 7fa847fff640 1 -- 172.21.15.22:0/1274023722 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa83c009560 con 0x7fa848100a30 2024-02-16T19:41:38.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.018+0000 7fa847fff640 1 --2- 172.21.15.22:0/1274023722 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa848100a30 0x7fa848195590 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa83c009880 tx=0x7fa83c02c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:38.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.018+0000 7fa8457fa640 1 -- 172.21.15.22:0/1274023722 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa83c0044e0 con 0x7fa848100a30 2024-02-16T19:41:38.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.018+0000 7fa84e5fb640 1 -- 172.21.15.22:0/1274023722 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa84819a1b0 con 0x7fa848100a30 2024-02-16T19:41:38.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.018+0000 7fa8457fa640 1 -- 172.21.15.22:0/1274023722 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa83c003ab0 con 0x7fa848100a30 2024-02-16T19:41:38.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.018+0000 7fa8457fa640 1 -- 172.21.15.22:0/1274023722 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa83c005560 con 0x7fa848100a30 2024-02-16T19:41:38.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.018+0000 7fa84e5fb640 1 -- 172.21.15.22:0/1274023722 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa84819a710 con 0x7fa848100a30 2024-02-16T19:41:38.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.019+0000 7fa84e5fb640 1 -- 172.21.15.22:0/1274023722 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa84804e000 con 0x7fa848100a30 2024-02-16T19:41:38.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.019+0000 7fa8457fa640 1 -- 172.21.15.22:0/1274023722 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fa83c004dd0 con 0x7fa848100a30 2024-02-16T19:41:38.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.020+0000 7fa8457fa640 1 --2- 172.21.15.22:0/1274023722 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fa83006d810 0x7fa83006fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:38.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.020+0000 7fa8457fa640 1 -- 172.21.15.22:0/1274023722 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6962+0+0 (secure 0 0 0) 0x7fa83c0b0110 con 0x7fa848100a30 2024-02-16T19:41:38.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.020+0000 7fa8477fe640 1 --2- 172.21.15.22:0/1274023722 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fa83006d810 0x7fa83006fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:38.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.020+0000 7fa8477fe640 1 --2- 172.21.15.22:0/1274023722 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fa83006d810 0x7fa83006fcc0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa848196ba0 tx=0x7fa838005f90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:38.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.025+0000 7fa8457fa640 1 -- 172.21.15.22:0/1274023722 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa83c079860 con 0x7fa848100a30 2024-02-16T19:41:38.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.208+0000 7fa84e5fb640 1 -- 172.21.15.22:0/1274023722 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7fa848066860 con 0x7fa848100a30 2024-02-16T19:41:38.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.208+0000 7fa8457fa640 1 -- 172.21.15.22:0/1274023722 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v93) v1 ==== 107+0+11 (secure 0 0 0) 0x7fa83c07cf20 con 0x7fa848100a30 2024-02-16T19:41:38.210 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-02-16T19:41:38.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.211+0000 7fa84e5fb640 1 -- 172.21.15.22:0/1274023722 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fa83006d810 msgr2=0x7fa83006fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:38.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.211+0000 7fa84e5fb640 1 --2- 172.21.15.22:0/1274023722 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fa83006d810 0x7fa83006fcc0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa848196ba0 tx=0x7fa838005f90 comp rx=0 tx=0).stop 2024-02-16T19:41:38.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.211+0000 7fa84e5fb640 1 -- 172.21.15.22:0/1274023722 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa848100a30 msgr2=0x7fa848195590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:38.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.211+0000 7fa84e5fb640 1 --2- 172.21.15.22:0/1274023722 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa848100a30 0x7fa848195590 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa83c009880 tx=0x7fa83c02c6e0 comp rx=0 tx=0).stop 2024-02-16T19:41:38.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.211+0000 7fa84e5fb640 1 -- 172.21.15.22:0/1274023722 shutdown_connections 2024-02-16T19:41:38.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.211+0000 7fa84e5fb640 1 --2- 172.21.15.22:0/1274023722 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fa83006d810 0x7fa83006fcc0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:38.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.211+0000 7fa84e5fb640 1 --2- 172.21.15.22:0/1274023722 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa848101340 0x7fa848195ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:38.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.211+0000 7fa84e5fb640 1 --2- 172.21.15.22:0/1274023722 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa848100a30 0x7fa848195590 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:38.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.211+0000 7fa84e5fb640 1 -- 172.21.15.22:0/1274023722 >> 172.21.15.22:0/1274023722 conn(0x7fa8480fa260 msgr2=0x7fa848103fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:38.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.211+0000 7fa84e5fb640 1 -- 172.21.15.22:0/1274023722 shutdown_connections 2024-02-16T19:41:38.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.211+0000 7fa84e5fb640 1 -- 172.21.15.22:0/1274023722 wait complete. 2024-02-16T19:41:38.223 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-16T19:41:38.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.324+0000 7f5b75ac0640 1 -- 172.21.15.22:0/1378364109 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b70101360 msgr2=0x7f5b70109850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:38.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.324+0000 7f5b75ac0640 1 --2- 172.21.15.22:0/1378364109 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b70101360 0x7f5b70109850 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5b640098b0 tx=0x7f5b6402bcd0 comp rx=0 tx=0).stop 2024-02-16T19:41:38.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.324+0000 7f5b75ac0640 1 -- 172.21.15.22:0/1378364109 shutdown_connections 2024-02-16T19:41:38.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.324+0000 7f5b75ac0640 1 --2- 172.21.15.22:0/1378364109 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b70101360 0x7f5b70109850 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:38.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.324+0000 7f5b75ac0640 1 --2- 172.21.15.22:0/1378364109 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b70100a50 0x7f5b70100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:38.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.324+0000 7f5b75ac0640 1 -- 172.21.15.22:0/1378364109 >> 172.21.15.22:0/1378364109 conn(0x7f5b700fa280 msgr2=0x7f5b700fc690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:38.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.324+0000 7f5b75ac0640 1 -- 172.21.15.22:0/1378364109 shutdown_connections 2024-02-16T19:41:38.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.324+0000 7f5b75ac0640 1 -- 172.21.15.22:0/1378364109 wait complete. 2024-02-16T19:41:38.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.325+0000 7f5b75ac0640 1 Processor -- start 2024-02-16T19:41:38.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.325+0000 7f5b75ac0640 1 -- start start 2024-02-16T19:41:38.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.325+0000 7f5b75ac0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b70100a50 0x7f5b700fe6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:38.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.325+0000 7f5b75ac0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b70101360 0x7f5b700febe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:38.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.325+0000 7f5b75ac0640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b700ff120 con 0x7f5b70101360 2024-02-16T19:41:38.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.325+0000 7f5b75ac0640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b700ff290 con 0x7f5b70100a50 2024-02-16T19:41:38.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.325+0000 7f5b6effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b70101360 0x7f5b700febe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:38.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.325+0000 7f5b6effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b70101360 0x7f5b700febe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45496/0 (socket says 172.21.15.22:45496) 2024-02-16T19:41:38.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.325+0000 7f5b6effd640 1 -- 172.21.15.22:0/3878479771 learned_addr learned my addr 172.21.15.22:0/3878479771 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:41:38.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.325+0000 7f5b6f7fe640 1 --2- 172.21.15.22:0/3878479771 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b70100a50 0x7f5b700fe6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:38.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.325+0000 7f5b6effd640 1 -- 172.21.15.22:0/3878479771 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b70100a50 msgr2=0x7f5b700fe6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:38.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.325+0000 7f5b6effd640 1 --2- 172.21.15.22:0/3878479771 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b70100a50 0x7f5b700fe6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:38.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.325+0000 7f5b6effd640 1 -- 172.21.15.22:0/3878479771 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b64009560 con 0x7f5b70101360 2024-02-16T19:41:38.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.326+0000 7f5b6effd640 1 --2- 172.21.15.22:0/3878479771 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b70101360 0x7f5b700febe0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5b64009880 tx=0x7f5b6402c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:38.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.326+0000 7f5b6cff9640 1 -- 172.21.15.22:0/3878479771 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b640035f0 con 0x7f5b70101360 2024-02-16T19:41:38.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.326+0000 7f5b6cff9640 1 -- 172.21.15.22:0/3878479771 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5b64003c10 con 0x7f5b70101360 2024-02-16T19:41:38.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.326+0000 7f5b75ac0640 1 -- 172.21.15.22:0/3878479771 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b700ff4b0 con 0x7f5b70101360 2024-02-16T19:41:38.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.326+0000 7f5b6cff9640 1 -- 172.21.15.22:0/3878479771 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b640055a0 con 0x7f5b70101360 2024-02-16T19:41:38.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.326+0000 7f5b75ac0640 1 -- 172.21.15.22:0/3878479771 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b701a1970 con 0x7f5b70101360 2024-02-16T19:41:38.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.327+0000 7f5b75ac0640 1 -- 172.21.15.22:0/3878479771 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b7004e000 con 0x7f5b70101360 2024-02-16T19:41:38.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.327+0000 7f5b6cff9640 1 -- 172.21.15.22:0/3878479771 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f5b64004dd0 con 0x7f5b70101360 2024-02-16T19:41:38.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.328+0000 7f5b6cff9640 1 --2- 172.21.15.22:0/3878479771 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f5b5806d8d0 0x7f5b5806fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:38.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.328+0000 7f5b6cff9640 1 -- 172.21.15.22:0/3878479771 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f5b640afac0 con 0x7f5b70101360 2024-02-16T19:41:38.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.328+0000 7f5b6f7fe640 1 --2- 172.21.15.22:0/3878479771 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f5b5806d8d0 0x7f5b5806fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:38.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.328+0000 7f5b6f7fe640 1 --2- 172.21.15.22:0/3878479771 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f5b5806d8d0 0x7f5b5806fd80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5b60009b60 tx=0x7f5b60005f90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:38.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.332+0000 7f5b6cff9640 1 -- 172.21.15.22:0/3878479771 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5b640790f0 con 0x7f5b70101360 2024-02-16T19:41:38.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.551+0000 7f5b75ac0640 1 -- 172.21.15.22:0/3878479771 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5b700ffc70 con 0x7f5b70101360 2024-02-16T19:41:38.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.552+0000 7f5b6cff9640 1 -- 172.21.15.22:0/3878479771 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+4999 (secure 0 0 0) 0x7f5b6407c7b0 con 0x7f5b70101360 2024-02-16T19:41:38.553 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:41:38.554 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34408,"name":"cephfs.smithi022.btppwm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6829/3126701730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3126701730},{"type":"v1","addr":"172.21.15.22:6829","nonce":3126701730}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":44299,"name":"cephfs.smithi117.ibmppn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6825/2389644474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2389644474},{"type":"v1","addr":"172.21.15.117:6825","nonce":2389644474}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":44325,"name":"cephfs.smithi117.orzors","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6827/166927597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":166927597},{"type":"v1","addr":"172.21.15.117:6827","nonce":166927597}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T19:26:17.873644+0000","modified":"2024-02-16T19:39:34.241312+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34384},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34384":{"gid":34384,"name":"cephfs.smithi022.ocukvs","rank":0,"incarnation":31,"state":"up:active","state_seq":22,"addr":"172.21.15.22:6827/2535680483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":2535680483},{"type":"v1","addr":"172.21.15.22:6827","nonce":2535680483}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T19:41:38.554 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 37 2024-02-16T19:41:38.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.554+0000 7f5b75ac0640 1 -- 172.21.15.22:0/3878479771 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f5b5806d8d0 msgr2=0x7f5b5806fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:38.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.554+0000 7f5b75ac0640 1 --2- 172.21.15.22:0/3878479771 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f5b5806d8d0 0x7f5b5806fd80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5b60009b60 tx=0x7f5b60005f90 comp rx=0 tx=0).stop 2024-02-16T19:41:38.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.554+0000 7f5b75ac0640 1 -- 172.21.15.22:0/3878479771 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b70101360 msgr2=0x7f5b700febe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:38.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.554+0000 7f5b75ac0640 1 --2- 172.21.15.22:0/3878479771 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b70101360 0x7f5b700febe0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5b64009880 tx=0x7f5b6402c740 comp rx=0 tx=0).stop 2024-02-16T19:41:38.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.554+0000 7f5b75ac0640 1 -- 172.21.15.22:0/3878479771 shutdown_connections 2024-02-16T19:41:38.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.554+0000 7f5b75ac0640 1 --2- 172.21.15.22:0/3878479771 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f5b5806d8d0 0x7f5b5806fd80 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:38.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.554+0000 7f5b75ac0640 1 --2- 172.21.15.22:0/3878479771 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b70101360 0x7f5b700febe0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:38.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.554+0000 7f5b75ac0640 1 --2- 172.21.15.22:0/3878479771 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b70100a50 0x7f5b700fe6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:38.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.554+0000 7f5b75ac0640 1 -- 172.21.15.22:0/3878479771 >> 172.21.15.22:0/3878479771 conn(0x7f5b700fa280 msgr2=0x7f5b70104090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:38.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.554+0000 7f5b75ac0640 1 -- 172.21.15.22:0/3878479771 shutdown_connections 2024-02-16T19:41:38.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.554+0000 7f5b75ac0640 1 -- 172.21.15.22:0/3878479771 wait complete. 2024-02-16T19:41:38.565 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:41:38 smithi022 ceph-mon[88798]: from='client.? 172.21.15.22:0/544395856' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-16T19:41:38.565 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:41:38 smithi022 ceph-mon[88798]: from='client.? 172.21.15.22:0/1700822627' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-02-16T19:41:38.565 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:41:38 smithi022 ceph-mon[88798]: from='client.? 172.21.15.22:0/1274023722' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-02-16T19:41:38.566 DEBUG:tasks.fs:checking fs fscid=1,name=cephfs state = {'epoch': 15, 'max_mds': 1, 'flags': 18} 2024-02-16T19:41:38.567 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 16 2024-02-16T19:41:38.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.668+0000 7f9329511640 1 -- 172.21.15.22:0/1892682561 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9324104750 msgr2=0x7f9324104b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:38.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.668+0000 7f9329511640 1 --2- 172.21.15.22:0/1892682561 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9324104750 0x7f9324104b20 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f931c009880 tx=0x7f931c02bce0 comp rx=0 tx=0).stop 2024-02-16T19:41:38.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.669+0000 7f9329511640 1 -- 172.21.15.22:0/1892682561 shutdown_connections 2024-02-16T19:41:38.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.669+0000 7f9329511640 1 --2- 172.21.15.22:0/1892682561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93240fe740 0x7f93240febb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:38.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.669+0000 7f9329511640 1 --2- 172.21.15.22:0/1892682561 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9324104750 0x7f9324104b20 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:38.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.669+0000 7f9329511640 1 -- 172.21.15.22:0/1892682561 >> 172.21.15.22:0/1892682561 conn(0x7f93240fa220 msgr2=0x7f93240fc630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:38.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.669+0000 7f9329511640 1 -- 172.21.15.22:0/1892682561 shutdown_connections 2024-02-16T19:41:38.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.669+0000 7f9329511640 1 -- 172.21.15.22:0/1892682561 wait complete. 2024-02-16T19:41:38.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.669+0000 7f9329511640 1 Processor -- start 2024-02-16T19:41:38.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.669+0000 7f9329511640 1 -- start start 2024-02-16T19:41:38.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.669+0000 7f9329511640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93240fe740 0x7f9324196350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:38.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.670+0000 7f9329511640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9324104750 0x7f9324196890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:38.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.670+0000 7f9329511640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9324196eb0 con 0x7f93240fe740 2024-02-16T19:41:38.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.670+0000 7f9329511640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9324196ff0 con 0x7f9324104750 2024-02-16T19:41:38.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.670+0000 7f9322ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93240fe740 0x7f9324196350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:38.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.670+0000 7f9322ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93240fe740 0x7f9324196350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45518/0 (socket says 172.21.15.22:45518) 2024-02-16T19:41:38.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.670+0000 7f9322ffd640 1 -- 172.21.15.22:0/3782303710 learned_addr learned my addr 172.21.15.22:0/3782303710 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:41:38.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.670+0000 7f9322ffd640 1 -- 172.21.15.22:0/3782303710 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9324104750 msgr2=0x7f9324196890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:38.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.670+0000 7f9322ffd640 1 --2- 172.21.15.22:0/3782303710 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9324104750 0x7f9324196890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:38.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.670+0000 7f9322ffd640 1 -- 172.21.15.22:0/3782303710 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f931c009560 con 0x7f93240fe740 2024-02-16T19:41:38.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.670+0000 7f9322ffd640 1 --2- 172.21.15.22:0/3782303710 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93240fe740 0x7f9324196350 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f931c009230 tx=0x7f931c02c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:38.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.670+0000 7f931bfff640 1 -- 172.21.15.22:0/3782303710 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f931c0035f0 con 0x7f93240fe740 2024-02-16T19:41:38.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.670+0000 7f931bfff640 1 -- 172.21.15.22:0/3782303710 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f931c003c10 con 0x7f93240fe740 2024-02-16T19:41:38.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.670+0000 7f9329511640 1 -- 172.21.15.22:0/3782303710 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f932419f3c0 con 0x7f93240fe740 2024-02-16T19:41:38.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.670+0000 7f931bfff640 1 -- 172.21.15.22:0/3782303710 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f931c005690 con 0x7f93240fe740 2024-02-16T19:41:38.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.670+0000 7f9329511640 1 -- 172.21.15.22:0/3782303710 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f932419f860 con 0x7f93240fe740 2024-02-16T19:41:38.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.671+0000 7f9329511640 1 -- 172.21.15.22:0/3782303710 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f932404e000 con 0x7f93240fe740 2024-02-16T19:41:38.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.672+0000 7f931bfff640 1 -- 172.21.15.22:0/3782303710 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f931c004ed0 con 0x7f93240fe740 2024-02-16T19:41:38.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.672+0000 7f931bfff640 1 --2- 172.21.15.22:0/3782303710 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f931006d810 0x7f931006fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:38.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.672+0000 7f931bfff640 1 -- 172.21.15.22:0/3782303710 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f931c0afa90 con 0x7f93240fe740 2024-02-16T19:41:38.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.672+0000 7f93227fc640 1 --2- 172.21.15.22:0/3782303710 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f931006d810 0x7f931006fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:38.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.673+0000 7f93227fc640 1 --2- 172.21.15.22:0/3782303710 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f931006d810 0x7f931006fcc0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f930c005fd0 tx=0x7f930c005850 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:38.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.677+0000 7f931bfff640 1 -- 172.21.15.22:0/3782303710 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f931c079140 con 0x7f93240fe740 2024-02-16T19:41:38.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:41:38 smithi117 ceph-mon[67870]: from='client.? 172.21.15.22:0/544395856' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-16T19:41:38.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:41:38 smithi117 ceph-mon[67870]: from='client.? 172.21.15.22:0/1700822627' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-02-16T19:41:38.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:41:38 smithi117 ceph-mon[67870]: from='client.? 172.21.15.22:0/1274023722' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-02-16T19:41:38.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.895+0000 7f9329511640 1 -- 172.21.15.22:0/3782303710 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 16, "format": "json"} v 0) v1 -- 0x7f93241971b0 con 0x7f93240fe740 2024-02-16T19:41:38.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.895+0000 7f931bfff640 1 -- 172.21.15.22:0/3782303710 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 16, "format": "json"}]=0 dumped fsmap epoch 16 v37) v1 ==== 107+0+4049 (secure 0 0 0) 0x7f931c07c800 con 0x7f93240fe740 2024-02-16T19:41:38.897 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:41:38.897 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24259,"name":"cephfs.smithi117.ibmppn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.117:6825/565408102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":565408102},{"type":"v1","addr":"172.21.15.117:6825","nonce":565408102}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":24265,"name":"cephfs.smithi117.orzors","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.117:6827/3122342028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":3122342028},{"type":"v1","addr":"172.21.15.117:6827","nonce":3122342028}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T19:26:17.873644+0000","modified":"2024-02-16T19:26:35.943353+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14490":{"gid":14490,"name":"cephfs.smithi022.btppwm","rank":0,"incarnation":12,"state":"up:active","state_seq":5,"addr":"172.21.15.22:6829/3597685751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3597685751},{"type":"v1","addr":"172.21.15.22:6829","nonce":3597685751}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T19:41:38.897 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 16 2024-02-16T19:41:38.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.897+0000 7f9329511640 1 -- 172.21.15.22:0/3782303710 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f931006d810 msgr2=0x7f931006fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:38.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.897+0000 7f9329511640 1 --2- 172.21.15.22:0/3782303710 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f931006d810 0x7f931006fcc0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f930c005fd0 tx=0x7f930c005850 comp rx=0 tx=0).stop 2024-02-16T19:41:38.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.897+0000 7f9329511640 1 -- 172.21.15.22:0/3782303710 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93240fe740 msgr2=0x7f9324196350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:38.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.897+0000 7f9329511640 1 --2- 172.21.15.22:0/3782303710 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93240fe740 0x7f9324196350 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f931c009230 tx=0x7f931c02c850 comp rx=0 tx=0).stop 2024-02-16T19:41:38.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.897+0000 7f9329511640 1 -- 172.21.15.22:0/3782303710 shutdown_connections 2024-02-16T19:41:38.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.897+0000 7f9329511640 1 --2- 172.21.15.22:0/3782303710 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f931006d810 0x7f931006fcc0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:38.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.897+0000 7f9329511640 1 --2- 172.21.15.22:0/3782303710 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9324104750 0x7f9324196890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:38.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.897+0000 7f9329511640 1 --2- 172.21.15.22:0/3782303710 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93240fe740 0x7f9324196350 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:38.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.897+0000 7f9329511640 1 -- 172.21.15.22:0/3782303710 >> 172.21.15.22:0/3782303710 conn(0x7f93240fa220 msgr2=0x7f93240fbb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:38.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.897+0000 7f9329511640 1 -- 172.21.15.22:0/3782303710 shutdown_connections 2024-02-16T19:41:38.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:38.897+0000 7f9329511640 1 -- 172.21.15.22:0/3782303710 wait complete. 2024-02-16T19:41:38.910 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 17 2024-02-16T19:41:39.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.012+0000 7f85e71b1640 1 -- 172.21.15.22:0/1943952607 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85e0104590 msgr2=0x7f85e0104960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:39.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.012+0000 7f85e71b1640 1 --2- 172.21.15.22:0/1943952607 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85e0104590 0x7f85e0104960 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f85d4009880 tx=0x7f85d402bce0 comp rx=0 tx=0).stop 2024-02-16T19:41:39.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.012+0000 7f85e71b1640 1 -- 172.21.15.22:0/1943952607 shutdown_connections 2024-02-16T19:41:39.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.012+0000 7f85e71b1640 1 --2- 172.21.15.22:0/1943952607 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85e00fe5b0 0x7f85e00fea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:39.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.012+0000 7f85e71b1640 1 --2- 172.21.15.22:0/1943952607 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85e0104590 0x7f85e0104960 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:39.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.012+0000 7f85e71b1640 1 -- 172.21.15.22:0/1943952607 >> 172.21.15.22:0/1943952607 conn(0x7f85e00743d0 msgr2=0x7f85e00747d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:39.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.012+0000 7f85e71b1640 1 -- 172.21.15.22:0/1943952607 shutdown_connections 2024-02-16T19:41:39.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.012+0000 7f85e71b1640 1 -- 172.21.15.22:0/1943952607 wait complete. 2024-02-16T19:41:39.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.012+0000 7f85e71b1640 1 Processor -- start 2024-02-16T19:41:39.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.013+0000 7f85e71b1640 1 -- start start 2024-02-16T19:41:39.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.013+0000 7f85e71b1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85e00fe5b0 0x7f85e0195270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:39.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.013+0000 7f85e71b1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85e0104590 0x7f85e01957b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:39.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.013+0000 7f85e71b1640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e0195db0 con 0x7f85e0104590 2024-02-16T19:41:39.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.013+0000 7f85e71b1640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e0195f20 con 0x7f85e00fe5b0 2024-02-16T19:41:39.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.013+0000 7f85dffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85e0104590 0x7f85e01957b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:39.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.013+0000 7f85dffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85e0104590 0x7f85e01957b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45526/0 (socket says 172.21.15.22:45526) 2024-02-16T19:41:39.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.013+0000 7f85dffff640 1 -- 172.21.15.22:0/496357957 learned_addr learned my addr 172.21.15.22:0/496357957 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:41:39.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.013+0000 7f85e4f4d640 1 --2- 172.21.15.22:0/496357957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85e00fe5b0 0x7f85e0195270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:39.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.013+0000 7f85dffff640 1 -- 172.21.15.22:0/496357957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85e00fe5b0 msgr2=0x7f85e0195270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:39.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.013+0000 7f85dffff640 1 --2- 172.21.15.22:0/496357957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85e00fe5b0 0x7f85e0195270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:39.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.013+0000 7f85dffff640 1 -- 172.21.15.22:0/496357957 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85d4009560 con 0x7f85e0104590 2024-02-16T19:41:39.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.013+0000 7f85e4f4d640 1 --2- 172.21.15.22:0/496357957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85e00fe5b0 0x7f85e0195270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-16T19:41:39.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.013+0000 7f85dffff640 1 --2- 172.21.15.22:0/496357957 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85e0104590 0x7f85e01957b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f85cc00c8a0 tx=0x7f85cc00cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:39.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.013+0000 7f85ddffb640 1 -- 172.21.15.22:0/496357957 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f85cc007b80 con 0x7f85e0104590 2024-02-16T19:41:39.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.013+0000 7f85e71b1640 1 -- 172.21.15.22:0/496357957 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85e0199db0 con 0x7f85e0104590 2024-02-16T19:41:39.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.013+0000 7f85ddffb640 1 -- 172.21.15.22:0/496357957 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f85cc007ce0 con 0x7f85e0104590 2024-02-16T19:41:39.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.014+0000 7f85ddffb640 1 -- 172.21.15.22:0/496357957 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f85cc018810 con 0x7f85e0104590 2024-02-16T19:41:39.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.014+0000 7f85e71b1640 1 -- 172.21.15.22:0/496357957 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85e019a390 con 0x7f85e0104590 2024-02-16T19:41:39.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.015+0000 7f85e71b1640 1 -- 172.21.15.22:0/496357957 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85e004e000 con 0x7f85e0104590 2024-02-16T19:41:39.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.015+0000 7f85ddffb640 1 -- 172.21.15.22:0/496357957 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f85cc018a80 con 0x7f85e0104590 2024-02-16T19:41:39.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.015+0000 7f85ddffb640 1 --2- 172.21.15.22:0/496357957 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f85d006d810 0x7f85d006fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:39.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.016+0000 7f85ddffb640 1 -- 172.21.15.22:0/496357957 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f85cc08d920 con 0x7f85e0104590 2024-02-16T19:41:39.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.016+0000 7f85e4f4d640 1 --2- 172.21.15.22:0/496357957 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f85d006d810 0x7f85d006fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:39.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.016+0000 7f85e4f4d640 1 --2- 172.21.15.22:0/496357957 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f85d006d810 0x7f85d006fcc0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f85d4005ec0 tx=0x7f85d4005fd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:39.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.020+0000 7f85ddffb640 1 -- 172.21.15.22:0/496357957 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f85cc056eb0 con 0x7f85e0104590 2024-02-16T19:41:39.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.240+0000 7f85e71b1640 1 -- 172.21.15.22:0/496357957 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 17, "format": "json"} v 0) v1 -- 0x7f85e010b930 con 0x7f85e0104590 2024-02-16T19:41:39.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.241+0000 7f85ddffb640 1 -- 172.21.15.22:0/496357957 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 17, "format": "json"}]=0 dumped fsmap epoch 17 v37) v1 ==== 107+0+4862 (secure 0 0 0) 0x7f85cc05a570 con 0x7f85e0104590 2024-02-16T19:41:39.243 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:41:39.243 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24259,"name":"cephfs.smithi117.ibmppn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.117:6825/565408102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":565408102},{"type":"v1","addr":"172.21.15.117:6825","nonce":565408102}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":24265,"name":"cephfs.smithi117.orzors","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.117:6827/3122342028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":3122342028},{"type":"v1","addr":"172.21.15.117:6827","nonce":3122342028}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":34384,"name":"cephfs.smithi022.ocukvs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6827/2535680483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":2535680483},{"type":"v1","addr":"172.21.15.22:6827","nonce":2535680483}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T19:26:17.873644+0000","modified":"2024-02-16T19:26:35.943353+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14490":{"gid":14490,"name":"cephfs.smithi022.btppwm","rank":0,"incarnation":12,"state":"up:active","state_seq":5,"addr":"172.21.15.22:6829/3597685751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3597685751},{"type":"v1","addr":"172.21.15.22:6829","nonce":3597685751}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T19:41:39.243 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 17 2024-02-16T19:41:39.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.243+0000 7f85e71b1640 1 -- 172.21.15.22:0/496357957 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f85d006d810 msgr2=0x7f85d006fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:39.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.243+0000 7f85e71b1640 1 --2- 172.21.15.22:0/496357957 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f85d006d810 0x7f85d006fcc0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f85d4005ec0 tx=0x7f85d4005fd0 comp rx=0 tx=0).stop 2024-02-16T19:41:39.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.243+0000 7f85e71b1640 1 -- 172.21.15.22:0/496357957 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85e0104590 msgr2=0x7f85e01957b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:39.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.243+0000 7f85e71b1640 1 --2- 172.21.15.22:0/496357957 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85e0104590 0x7f85e01957b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f85cc00c8a0 tx=0x7f85cc00cd60 comp rx=0 tx=0).stop 2024-02-16T19:41:39.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.243+0000 7f85e71b1640 1 -- 172.21.15.22:0/496357957 shutdown_connections 2024-02-16T19:41:39.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.243+0000 7f85e71b1640 1 --2- 172.21.15.22:0/496357957 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f85d006d810 0x7f85d006fcc0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:39.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.243+0000 7f85e71b1640 1 --2- 172.21.15.22:0/496357957 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85e0104590 0x7f85e01957b0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:39.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.243+0000 7f85e71b1640 1 --2- 172.21.15.22:0/496357957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85e00fe5b0 0x7f85e0195270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:39.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.243+0000 7f85e71b1640 1 -- 172.21.15.22:0/496357957 >> 172.21.15.22:0/496357957 conn(0x7f85e00743d0 msgr2=0x7f85e0102690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:39.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.243+0000 7f85e71b1640 1 -- 172.21.15.22:0/496357957 shutdown_connections 2024-02-16T19:41:39.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.243+0000 7f85e71b1640 1 -- 172.21.15.22:0/496357957 wait complete. 2024-02-16T19:41:39.256 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 18 2024-02-16T19:41:39.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.361+0000 7f7a7c7d0640 1 -- 172.21.15.22:0/129770575 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7a740fe730 msgr2=0x7f7a740feba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:39.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.361+0000 7f7a7c7d0640 1 --2- 172.21.15.22:0/129770575 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7a740fe730 0x7f7a740feba0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7a6400a990 tx=0x7f7a6402d0a0 comp rx=0 tx=0).stop 2024-02-16T19:41:39.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.361+0000 7f7a7c7d0640 1 -- 172.21.15.22:0/129770575 shutdown_connections 2024-02-16T19:41:39.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.361+0000 7f7a7c7d0640 1 --2- 172.21.15.22:0/129770575 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7a740fe730 0x7f7a740feba0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:39.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.361+0000 7f7a7c7d0640 1 --2- 172.21.15.22:0/129770575 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a74102d60 0x7f7a74103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:39.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.361+0000 7f7a7c7d0640 1 -- 172.21.15.22:0/129770575 >> 172.21.15.22:0/129770575 conn(0x7f7a74073d20 msgr2=0x7f7a74074120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:39.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.361+0000 7f7a7c7d0640 1 -- 172.21.15.22:0/129770575 shutdown_connections 2024-02-16T19:41:39.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.361+0000 7f7a7c7d0640 1 -- 172.21.15.22:0/129770575 wait complete. 2024-02-16T19:41:39.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.361+0000 7f7a7c7d0640 1 Processor -- start 2024-02-16T19:41:39.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.361+0000 7f7a7c7d0640 1 -- start start 2024-02-16T19:41:39.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.362+0000 7f7a7c7d0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7a740fe730 0x7f7a74195310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:39.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.362+0000 7f7a7c7d0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a74102d60 0x7f7a74195850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:39.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.362+0000 7f7a7c7d0640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a74195ee0 con 0x7f7a740fe730 2024-02-16T19:41:39.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.362+0000 7f7a7c7d0640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a74199cb0 con 0x7f7a74102d60 2024-02-16T19:41:39.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.362+0000 7f7a7a56c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7a740fe730 0x7f7a74195310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:39.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.362+0000 7f7a7a56c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7a740fe730 0x7f7a74195310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45546/0 (socket says 172.21.15.22:45546) 2024-02-16T19:41:39.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.362+0000 7f7a7a56c640 1 -- 172.21.15.22:0/668813462 learned_addr learned my addr 172.21.15.22:0/668813462 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:41:39.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.362+0000 7f7a79d6b640 1 --2- 172.21.15.22:0/668813462 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a74102d60 0x7f7a74195850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:39.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.362+0000 7f7a7a56c640 1 -- 172.21.15.22:0/668813462 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a74102d60 msgr2=0x7f7a74195850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:39.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.362+0000 7f7a7a56c640 1 --2- 172.21.15.22:0/668813462 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a74102d60 0x7f7a74195850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:39.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.362+0000 7f7a7a56c640 1 -- 172.21.15.22:0/668813462 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a6400a670 con 0x7f7a740fe730 2024-02-16T19:41:39.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.362+0000 7f7a7a56c640 1 --2- 172.21.15.22:0/668813462 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7a740fe730 0x7f7a74195310 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c00c810 tx=0x7f7a6c00ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:39.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.362+0000 7f7a6b7fe640 1 -- 172.21.15.22:0/668813462 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a6c0041f0 con 0x7f7a740fe730 2024-02-16T19:41:39.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.362+0000 7f7a6b7fe640 1 -- 172.21.15.22:0/668813462 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7a6c007e70 con 0x7f7a740fe730 2024-02-16T19:41:39.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.362+0000 7f7a7c7d0640 1 -- 172.21.15.22:0/668813462 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a74199f90 con 0x7f7a740fe730 2024-02-16T19:41:39.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.362+0000 7f7a6b7fe640 1 -- 172.21.15.22:0/668813462 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a6c0049d0 con 0x7f7a740fe730 2024-02-16T19:41:39.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.362+0000 7f7a7c7d0640 1 -- 172.21.15.22:0/668813462 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a7419a4f0 con 0x7f7a740fe730 2024-02-16T19:41:39.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.364+0000 7f7a7c7d0640 1 -- 172.21.15.22:0/668813462 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a7404e000 con 0x7f7a740fe730 2024-02-16T19:41:39.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.364+0000 7f7a6b7fe640 1 -- 172.21.15.22:0/668813462 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f7a6c0079b0 con 0x7f7a740fe730 2024-02-16T19:41:39.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.364+0000 7f7a6b7fe640 1 --2- 172.21.15.22:0/668813462 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f7a6006d810 0x7f7a6006fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:39.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.365+0000 7f7a79d6b640 1 --2- 172.21.15.22:0/668813462 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f7a6006d810 0x7f7a6006fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:39.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.365+0000 7f7a6b7fe640 1 -- 172.21.15.22:0/668813462 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f7a6c08d5d0 con 0x7f7a740fe730 2024-02-16T19:41:39.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.365+0000 7f7a79d6b640 1 --2- 172.21.15.22:0/668813462 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f7a6006d810 0x7f7a6006fcc0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7a64005ab0 tx=0x7f7a64005a20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:39.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.369+0000 7f7a6b7fe640 1 -- 172.21.15.22:0/668813462 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7a6c056b60 con 0x7f7a740fe730 2024-02-16T19:41:39.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.588+0000 7f7a7c7d0640 1 -- 172.21.15.22:0/668813462 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 18, "format": "json"} v 0) v1 -- 0x7f7a741080a0 con 0x7f7a740fe730 2024-02-16T19:41:39.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.589+0000 7f7a6b7fe640 1 -- 172.21.15.22:0/668813462 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 18, "format": "json"}]=0 dumped fsmap epoch 18 v37) v1 ==== 107+0+4072 (secure 0 0 0) 0x7f7a6c05a220 con 0x7f7a740fe730 2024-02-16T19:41:39.591 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:41:39.591 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24259,"name":"cephfs.smithi117.ibmppn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.117:6825/565408102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":565408102},{"type":"v1","addr":"172.21.15.117:6825","nonce":565408102}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":24265,"name":"cephfs.smithi117.orzors","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.117:6827/3122342028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":3122342028},{"type":"v1","addr":"172.21.15.117:6827","nonce":3122342028}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":34384,"name":"cephfs.smithi022.ocukvs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6827/2535680483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":2535680483},{"type":"v1","addr":"172.21.15.22:6827","nonce":2535680483}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T19:26:17.873644+0000","modified":"2024-02-16T19:38:09.297314+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":91,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T19:41:39.591 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 18 2024-02-16T19:41:39.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.591+0000 7f7a7c7d0640 1 -- 172.21.15.22:0/668813462 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f7a6006d810 msgr2=0x7f7a6006fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:39.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.591+0000 7f7a7c7d0640 1 --2- 172.21.15.22:0/668813462 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f7a6006d810 0x7f7a6006fcc0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7a64005ab0 tx=0x7f7a64005a20 comp rx=0 tx=0).stop 2024-02-16T19:41:39.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.591+0000 7f7a7c7d0640 1 -- 172.21.15.22:0/668813462 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7a740fe730 msgr2=0x7f7a74195310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:39.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.591+0000 7f7a7c7d0640 1 --2- 172.21.15.22:0/668813462 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7a740fe730 0x7f7a74195310 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c00c810 tx=0x7f7a6c00ccd0 comp rx=0 tx=0).stop 2024-02-16T19:41:39.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.591+0000 7f7a7c7d0640 1 -- 172.21.15.22:0/668813462 shutdown_connections 2024-02-16T19:41:39.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.591+0000 7f7a7c7d0640 1 --2- 172.21.15.22:0/668813462 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f7a6006d810 0x7f7a6006fcc0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:39.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.591+0000 7f7a7c7d0640 1 --2- 172.21.15.22:0/668813462 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a74102d60 0x7f7a74195850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:39.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.591+0000 7f7a7c7d0640 1 --2- 172.21.15.22:0/668813462 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7a740fe730 0x7f7a74195310 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:39.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.591+0000 7f7a7c7d0640 1 -- 172.21.15.22:0/668813462 >> 172.21.15.22:0/668813462 conn(0x7f7a74073d20 msgr2=0x7f7a740fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:39.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.591+0000 7f7a7c7d0640 1 -- 172.21.15.22:0/668813462 shutdown_connections 2024-02-16T19:41:39.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.591+0000 7f7a7c7d0640 1 -- 172.21.15.22:0/668813462 wait complete. 2024-02-16T19:41:39.602 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:41:39 smithi022 ceph-mon[88798]: pgmap v297: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:41:39.602 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:41:39 smithi022 ceph-mon[88798]: from='client.? 172.21.15.22:0/3878479771' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-16T19:41:39.602 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:41:39 smithi022 ceph-mon[88798]: from='client.? 172.21.15.22:0/3782303710' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 16, "format": "json"}]: dispatch 2024-02-16T19:41:39.602 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:41:39 smithi022 ceph-mon[88798]: from='client.? 172.21.15.22:0/496357957' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 17, "format": "json"}]: dispatch 2024-02-16T19:41:39.604 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 19 2024-02-16T19:41:39.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.705+0000 7fafb3111640 1 -- 172.21.15.22:0/1598431478 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fafac104590 msgr2=0x7fafac104960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:39.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.705+0000 7fafb3111640 1 --2- 172.21.15.22:0/1598431478 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fafac104590 0x7fafac104960 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fafa4009c20 tx=0x7fafa402c1b0 comp rx=0 tx=0).stop 2024-02-16T19:41:39.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.705+0000 7fafb3111640 1 -- 172.21.15.22:0/1598431478 shutdown_connections 2024-02-16T19:41:39.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.705+0000 7fafb3111640 1 --2- 172.21.15.22:0/1598431478 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fafac0fe5b0 0x7fafac0fea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:39.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.705+0000 7fafb3111640 1 --2- 172.21.15.22:0/1598431478 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fafac104590 0x7fafac104960 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:39.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.705+0000 7fafb3111640 1 -- 172.21.15.22:0/1598431478 >> 172.21.15.22:0/1598431478 conn(0x7fafac0743d0 msgr2=0x7fafac0747d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:39.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.705+0000 7fafb3111640 1 -- 172.21.15.22:0/1598431478 shutdown_connections 2024-02-16T19:41:39.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.705+0000 7fafb3111640 1 -- 172.21.15.22:0/1598431478 wait complete. 2024-02-16T19:41:39.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.706+0000 7fafb3111640 1 Processor -- start 2024-02-16T19:41:39.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.706+0000 7fafb3111640 1 -- start start 2024-02-16T19:41:39.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.706+0000 7fafb3111640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fafac0fe5b0 0x7fafac195330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:39.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.706+0000 7fafb3111640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fafac104590 0x7fafac195870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:39.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.706+0000 7fafb3111640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafac195f00 con 0x7fafac104590 2024-02-16T19:41:39.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.706+0000 7fafb3111640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafac199cd0 con 0x7fafac0fe5b0 2024-02-16T19:41:39.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.706+0000 7fafabfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fafac104590 0x7fafac195870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:39.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.706+0000 7fafabfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fafac104590 0x7fafac195870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45560/0 (socket says 172.21.15.22:45560) 2024-02-16T19:41:39.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.706+0000 7fafabfff640 1 -- 172.21.15.22:0/3567348446 learned_addr learned my addr 172.21.15.22:0/3567348446 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:41:39.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.707+0000 7fafb0ead640 1 --2- 172.21.15.22:0/3567348446 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fafac0fe5b0 0x7fafac195330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:39.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.707+0000 7fafabfff640 1 -- 172.21.15.22:0/3567348446 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fafac0fe5b0 msgr2=0x7fafac195330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:39.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.707+0000 7fafabfff640 1 --2- 172.21.15.22:0/3567348446 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fafac0fe5b0 0x7fafac195330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:39.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.707+0000 7fafabfff640 1 -- 172.21.15.22:0/3567348446 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafa4009900 con 0x7fafac104590 2024-02-16T19:41:39.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.707+0000 7fafabfff640 1 --2- 172.21.15.22:0/3567348446 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fafac104590 0x7fafac195870 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7faf9400c8f0 tx=0x7faf9400cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:39.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.707+0000 7fafa9ffb640 1 -- 172.21.15.22:0/3567348446 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf94007b80 con 0x7fafac104590 2024-02-16T19:41:39.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.707+0000 7fafa9ffb640 1 -- 172.21.15.22:0/3567348446 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faf940054e0 con 0x7fafac104590 2024-02-16T19:41:39.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.707+0000 7fafb3111640 1 -- 172.21.15.22:0/3567348446 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafac199fb0 con 0x7fafac104590 2024-02-16T19:41:39.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.707+0000 7fafa9ffb640 1 -- 172.21.15.22:0/3567348446 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf9400f450 con 0x7fafac104590 2024-02-16T19:41:39.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.707+0000 7fafb3111640 1 -- 172.21.15.22:0/3567348446 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafac19a510 con 0x7fafac104590 2024-02-16T19:41:39.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.708+0000 7fafb3111640 1 -- 172.21.15.22:0/3567348446 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafac04e000 con 0x7fafac104590 2024-02-16T19:41:39.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.708+0000 7fafa9ffb640 1 -- 172.21.15.22:0/3567348446 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7faf94004f60 con 0x7fafac104590 2024-02-16T19:41:39.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.709+0000 7fafa9ffb640 1 --2- 172.21.15.22:0/3567348446 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7faf9806d810 0x7faf9806fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:39.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.709+0000 7fafa9ffb640 1 -- 172.21.15.22:0/3567348446 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6962+0+0 (secure 0 0 0) 0x7faf9408d4d0 con 0x7fafac104590 2024-02-16T19:41:39.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.709+0000 7fafb0ead640 1 --2- 172.21.15.22:0/3567348446 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7faf9806d810 0x7faf9806fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:39.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.709+0000 7fafb0ead640 1 --2- 172.21.15.22:0/3567348446 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7faf9806d810 0x7faf9806fcc0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fafa4033000 tx=0x7fafa4005f10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:39.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.713+0000 7fafa9ffb640 1 -- 172.21.15.22:0/3567348446 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7faf94056b00 con 0x7fafac104590 2024-02-16T19:41:39.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:41:39 smithi117 ceph-mon[67870]: pgmap v297: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:41:39.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:41:39 smithi117 ceph-mon[67870]: from='client.? 172.21.15.22:0/3878479771' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-16T19:41:39.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:41:39 smithi117 ceph-mon[67870]: from='client.? 172.21.15.22:0/3782303710' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 16, "format": "json"}]: dispatch 2024-02-16T19:41:39.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:41:39 smithi117 ceph-mon[67870]: from='client.? 172.21.15.22:0/496357957' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 17, "format": "json"}]: dispatch 2024-02-16T19:41:39.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.934+0000 7fafb3111640 1 -- 172.21.15.22:0/3567348446 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 19, "format": "json"} v 0) v1 -- 0x7fafac196680 con 0x7fafac104590 2024-02-16T19:41:39.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.934+0000 7fafa9ffb640 1 -- 172.21.15.22:0/3567348446 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 19, "format": "json"}]=0 dumped fsmap epoch 19 v37) v1 ==== 107+0+4083 (secure 0 0 0) 0x7faf9405a1c0 con 0x7fafac104590 2024-02-16T19:41:39.936 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:41:39.936 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24265,"name":"cephfs.smithi117.orzors","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.117:6827/3122342028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":3122342028},{"type":"v1","addr":"172.21.15.117:6827","nonce":3122342028}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":34384,"name":"cephfs.smithi022.ocukvs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6827/2535680483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":2535680483},{"type":"v1","addr":"172.21.15.22:6827","nonce":2535680483}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T19:26:17.873644+0000","modified":"2024-02-16T19:38:09.302089+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":91,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24259},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24259":{"gid":24259,"name":"cephfs.smithi117.ibmppn","rank":0,"incarnation":19,"state":"up:replay","state_seq":3,"addr":"172.21.15.117:6825/565408102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":565408102},{"type":"v1","addr":"172.21.15.117:6825","nonce":565408102}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T19:41:39.936 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 19 2024-02-16T19:41:39.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.936+0000 7fafb3111640 1 -- 172.21.15.22:0/3567348446 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7faf9806d810 msgr2=0x7faf9806fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:39.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.936+0000 7fafb3111640 1 --2- 172.21.15.22:0/3567348446 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7faf9806d810 0x7faf9806fcc0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fafa4033000 tx=0x7fafa4005f10 comp rx=0 tx=0).stop 2024-02-16T19:41:39.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.936+0000 7fafb3111640 1 -- 172.21.15.22:0/3567348446 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fafac104590 msgr2=0x7fafac195870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:39.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.936+0000 7fafb3111640 1 --2- 172.21.15.22:0/3567348446 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fafac104590 0x7fafac195870 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7faf9400c8f0 tx=0x7faf9400cdb0 comp rx=0 tx=0).stop 2024-02-16T19:41:39.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.936+0000 7fafb3111640 1 -- 172.21.15.22:0/3567348446 shutdown_connections 2024-02-16T19:41:39.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.936+0000 7fafb3111640 1 --2- 172.21.15.22:0/3567348446 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7faf9806d810 0x7faf9806fcc0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:39.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.936+0000 7fafb3111640 1 --2- 172.21.15.22:0/3567348446 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fafac104590 0x7fafac195870 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:39.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.936+0000 7fafb3111640 1 --2- 172.21.15.22:0/3567348446 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fafac0fe5b0 0x7fafac195330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:39.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.936+0000 7fafb3111640 1 -- 172.21.15.22:0/3567348446 >> 172.21.15.22:0/3567348446 conn(0x7fafac0743d0 msgr2=0x7fafac067390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:39.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.936+0000 7fafb3111640 1 -- 172.21.15.22:0/3567348446 shutdown_connections 2024-02-16T19:41:39.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:39.936+0000 7fafb3111640 1 -- 172.21.15.22:0/3567348446 wait complete. 2024-02-16T19:41:39.949 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 20 2024-02-16T19:41:40.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.050+0000 7fa884767640 1 -- 172.21.15.22:0/2700528888 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa87c067350 msgr2=0x7fa87c0677c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:40.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.050+0000 7fa884767640 1 --2- 172.21.15.22:0/2700528888 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa87c067350 0x7fa87c0677c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa870009880 tx=0x7fa87002bce0 comp rx=0 tx=0).stop 2024-02-16T19:41:40.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.050+0000 7fa884767640 1 -- 172.21.15.22:0/2700528888 shutdown_connections 2024-02-16T19:41:40.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.050+0000 7fa884767640 1 --2- 172.21.15.22:0/2700528888 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa87c067350 0x7fa87c0677c0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:40.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.050+0000 7fa884767640 1 --2- 172.21.15.22:0/2700528888 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa87c105690 0x7fa87c105a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:40.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.050+0000 7fa884767640 1 -- 172.21.15.22:0/2700528888 >> 172.21.15.22:0/2700528888 conn(0x7fa87c0743d0 msgr2=0x7fa87c0747d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:40.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.050+0000 7fa884767640 1 -- 172.21.15.22:0/2700528888 shutdown_connections 2024-02-16T19:41:40.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.050+0000 7fa884767640 1 -- 172.21.15.22:0/2700528888 wait complete. 2024-02-16T19:41:40.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.051+0000 7fa884767640 1 Processor -- start 2024-02-16T19:41:40.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.051+0000 7fa884767640 1 -- start start 2024-02-16T19:41:40.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.051+0000 7fa884767640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa87c067350 0x7fa87c197480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:40.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.051+0000 7fa884767640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa87c105690 0x7fa87c1979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:40.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.051+0000 7fa884767640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa87c197fc0 con 0x7fa87c067350 2024-02-16T19:41:40.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.051+0000 7fa884767640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa87c198130 con 0x7fa87c105690 2024-02-16T19:41:40.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.051+0000 7fa882503640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa87c067350 0x7fa87c197480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:40.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.051+0000 7fa882503640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa87c067350 0x7fa87c197480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45586/0 (socket says 172.21.15.22:45586) 2024-02-16T19:41:40.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.051+0000 7fa882503640 1 -- 172.21.15.22:0/4157013569 learned_addr learned my addr 172.21.15.22:0/4157013569 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:41:40.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.051+0000 7fa881d02640 1 --2- 172.21.15.22:0/4157013569 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa87c105690 0x7fa87c1979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:40.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.051+0000 7fa882503640 1 -- 172.21.15.22:0/4157013569 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa87c105690 msgr2=0x7fa87c1979c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:40.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.051+0000 7fa882503640 1 --2- 172.21.15.22:0/4157013569 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa87c105690 0x7fa87c1979c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:40.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.051+0000 7fa882503640 1 -- 172.21.15.22:0/4157013569 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa870009560 con 0x7fa87c067350 2024-02-16T19:41:40.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.052+0000 7fa882503640 1 --2- 172.21.15.22:0/4157013569 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa87c067350 0x7fa87c197480 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa87400c8a0 tx=0x7fa87400cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:40.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.052+0000 7fa87b7fe640 1 -- 172.21.15.22:0/4157013569 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa874007b80 con 0x7fa87c067350 2024-02-16T19:41:40.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.052+0000 7fa87b7fe640 1 -- 172.21.15.22:0/4157013569 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa874007ce0 con 0x7fa87c067350 2024-02-16T19:41:40.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.052+0000 7fa884767640 1 -- 172.21.15.22:0/4157013569 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa87c19bfc0 con 0x7fa87c067350 2024-02-16T19:41:40.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.052+0000 7fa87b7fe640 1 -- 172.21.15.22:0/4157013569 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa874018810 con 0x7fa87c067350 2024-02-16T19:41:40.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.052+0000 7fa884767640 1 -- 172.21.15.22:0/4157013569 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa87c19c5a0 con 0x7fa87c067350 2024-02-16T19:41:40.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.053+0000 7fa884767640 1 -- 172.21.15.22:0/4157013569 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa87c04e000 con 0x7fa87c067350 2024-02-16T19:41:40.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.053+0000 7fa87b7fe640 1 -- 172.21.15.22:0/4157013569 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fa874018a80 con 0x7fa87c067350 2024-02-16T19:41:40.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.054+0000 7fa87b7fe640 1 --2- 172.21.15.22:0/4157013569 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fa86406d810 0x7fa86406fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:40.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.054+0000 7fa87b7fe640 1 -- 172.21.15.22:0/4157013569 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6962+0+0 (secure 0 0 0) 0x7fa87408d880 con 0x7fa87c067350 2024-02-16T19:41:40.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.054+0000 7fa881d02640 1 --2- 172.21.15.22:0/4157013569 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fa86406d810 0x7fa86406fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:40.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.054+0000 7fa881d02640 1 --2- 172.21.15.22:0/4157013569 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fa86406d810 0x7fa86406fcc0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa870036040 tx=0x7fa870005fd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:40.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.058+0000 7fa87b7fe640 1 -- 172.21.15.22:0/4157013569 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa874056eb0 con 0x7fa87c067350 2024-02-16T19:41:40.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.278+0000 7fa884767640 1 -- 172.21.15.22:0/4157013569 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 20, "format": "json"} v 0) v1 -- 0x7fa87c1987d0 con 0x7fa87c067350 2024-02-16T19:41:40.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.279+0000 7fa87b7fe640 1 -- 172.21.15.22:0/4157013569 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 20, "format": "json"}]=0 dumped fsmap epoch 20 v37) v1 ==== 107+0+4896 (secure 0 0 0) 0x7fa87405a570 con 0x7fa87c067350 2024-02-16T19:41:40.280 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:41:40.280 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24265,"name":"cephfs.smithi117.orzors","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.117:6827/3122342028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":3122342028},{"type":"v1","addr":"172.21.15.117:6827","nonce":3122342028}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":34384,"name":"cephfs.smithi022.ocukvs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6827/2535680483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":2535680483},{"type":"v1","addr":"172.21.15.22:6827","nonce":2535680483}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17},{"gid":34408,"name":"cephfs.smithi022.btppwm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6829/3126701730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3126701730},{"type":"v1","addr":"172.21.15.22:6829","nonce":3126701730}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T19:26:17.873644+0000","modified":"2024-02-16T19:38:09.302089+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":91,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24259},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24259":{"gid":24259,"name":"cephfs.smithi117.ibmppn","rank":0,"incarnation":19,"state":"up:replay","state_seq":3,"addr":"172.21.15.117:6825/565408102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":565408102},{"type":"v1","addr":"172.21.15.117:6825","nonce":565408102}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T19:41:40.281 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 20 2024-02-16T19:41:40.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.281+0000 7fa884767640 1 -- 172.21.15.22:0/4157013569 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fa86406d810 msgr2=0x7fa86406fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:40.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.281+0000 7fa884767640 1 --2- 172.21.15.22:0/4157013569 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fa86406d810 0x7fa86406fcc0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa870036040 tx=0x7fa870005fd0 comp rx=0 tx=0).stop 2024-02-16T19:41:40.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.281+0000 7fa884767640 1 -- 172.21.15.22:0/4157013569 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa87c067350 msgr2=0x7fa87c197480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:40.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.281+0000 7fa884767640 1 --2- 172.21.15.22:0/4157013569 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa87c067350 0x7fa87c197480 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa87400c8a0 tx=0x7fa87400cd60 comp rx=0 tx=0).stop 2024-02-16T19:41:40.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.281+0000 7fa884767640 1 -- 172.21.15.22:0/4157013569 shutdown_connections 2024-02-16T19:41:40.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.281+0000 7fa884767640 1 --2- 172.21.15.22:0/4157013569 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fa86406d810 0x7fa86406fcc0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:40.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.281+0000 7fa884767640 1 --2- 172.21.15.22:0/4157013569 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa87c105690 0x7fa87c1979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:40.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.281+0000 7fa884767640 1 --2- 172.21.15.22:0/4157013569 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa87c067350 0x7fa87c197480 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:40.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.281+0000 7fa884767640 1 -- 172.21.15.22:0/4157013569 >> 172.21.15.22:0/4157013569 conn(0x7fa87c0743d0 msgr2=0x7fa87c0fcca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:40.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.281+0000 7fa884767640 1 -- 172.21.15.22:0/4157013569 shutdown_connections 2024-02-16T19:41:40.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.281+0000 7fa884767640 1 -- 172.21.15.22:0/4157013569 wait complete. 2024-02-16T19:41:40.293 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 21 2024-02-16T19:41:40.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.396+0000 7f99b07c4640 1 -- 172.21.15.22:0/2928035976 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99a8104590 msgr2=0x7f99a8104960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:40.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.396+0000 7f99b07c4640 1 --2- 172.21.15.22:0/2928035976 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99a8104590 0x7f99a8104960 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f99a0009c20 tx=0x7f99a002c1b0 comp rx=0 tx=0).stop 2024-02-16T19:41:40.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.396+0000 7f99b07c4640 1 -- 172.21.15.22:0/2928035976 shutdown_connections 2024-02-16T19:41:40.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.396+0000 7f99b07c4640 1 --2- 172.21.15.22:0/2928035976 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99a80fe5b0 0x7f99a80fea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:40.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.396+0000 7f99b07c4640 1 --2- 172.21.15.22:0/2928035976 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99a8104590 0x7f99a8104960 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:40.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.396+0000 7f99b07c4640 1 -- 172.21.15.22:0/2928035976 >> 172.21.15.22:0/2928035976 conn(0x7f99a80743d0 msgr2=0x7f99a80747d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:40.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.396+0000 7f99b07c4640 1 -- 172.21.15.22:0/2928035976 shutdown_connections 2024-02-16T19:41:40.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.396+0000 7f99b07c4640 1 -- 172.21.15.22:0/2928035976 wait complete. 2024-02-16T19:41:40.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.397+0000 7f99b07c4640 1 Processor -- start 2024-02-16T19:41:40.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.397+0000 7f99b07c4640 1 -- start start 2024-02-16T19:41:40.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.397+0000 7f99b07c4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99a80fe5b0 0x7f99a81941a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:40.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.397+0000 7f99b07c4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99a8104590 0x7f99a81946e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:40.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.397+0000 7f99b07c4640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a8194d70 con 0x7f99a80fe5b0 2024-02-16T19:41:40.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.397+0000 7f99b07c4640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a8198b40 con 0x7f99a8104590 2024-02-16T19:41:40.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.397+0000 7f99ae560640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99a80fe5b0 0x7f99a81941a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:40.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.397+0000 7f99ae560640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99a80fe5b0 0x7f99a81941a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45592/0 (socket says 172.21.15.22:45592) 2024-02-16T19:41:40.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.397+0000 7f99ae560640 1 -- 172.21.15.22:0/250235556 learned_addr learned my addr 172.21.15.22:0/250235556 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:41:40.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.397+0000 7f99add5f640 1 --2- 172.21.15.22:0/250235556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99a8104590 0x7f99a81946e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:40.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.397+0000 7f99ae560640 1 -- 172.21.15.22:0/250235556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99a8104590 msgr2=0x7f99a81946e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:40.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.397+0000 7f99ae560640 1 --2- 172.21.15.22:0/250235556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99a8104590 0x7f99a81946e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:40.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.397+0000 7f99ae560640 1 -- 172.21.15.22:0/250235556 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99a0009900 con 0x7f99a80fe5b0 2024-02-16T19:41:40.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.397+0000 7f99ae560640 1 --2- 172.21.15.22:0/250235556 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99a80fe5b0 0x7f99a81941a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f99a002efd0 tx=0x7f99a0032b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:40.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.398+0000 7f999f7fe640 1 -- 172.21.15.22:0/250235556 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f99a0034650 con 0x7f99a80fe5b0 2024-02-16T19:41:40.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.398+0000 7f999f7fe640 1 -- 172.21.15.22:0/250235556 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f99a0003e30 con 0x7f99a80fe5b0 2024-02-16T19:41:40.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.398+0000 7f99b07c4640 1 -- 172.21.15.22:0/250235556 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99a8198dc0 con 0x7f99a80fe5b0 2024-02-16T19:41:40.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.398+0000 7f999f7fe640 1 -- 172.21.15.22:0/250235556 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f99a0004800 con 0x7f99a80fe5b0 2024-02-16T19:41:40.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.398+0000 7f99b07c4640 1 -- 172.21.15.22:0/250235556 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99a8199340 con 0x7f99a80fe5b0 2024-02-16T19:41:40.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.399+0000 7f99b07c4640 1 -- 172.21.15.22:0/250235556 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99a804e000 con 0x7f99a80fe5b0 2024-02-16T19:41:40.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.400+0000 7f999f7fe640 1 -- 172.21.15.22:0/250235556 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f99a003b050 con 0x7f99a80fe5b0 2024-02-16T19:41:40.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.400+0000 7f999f7fe640 1 --2- 172.21.15.22:0/250235556 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f999406d810 0x7f999406fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:40.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.400+0000 7f999f7fe640 1 -- 172.21.15.22:0/250235556 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f99a00b1e50 con 0x7f99a80fe5b0 2024-02-16T19:41:40.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.400+0000 7f99add5f640 1 --2- 172.21.15.22:0/250235556 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f999406d810 0x7f999406fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:40.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.401+0000 7f99add5f640 1 --2- 172.21.15.22:0/250235556 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f999406d810 0x7f999406fcc0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f99a81957b0 tx=0x7f9998005f90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:40.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.405+0000 7f999f7fe640 1 -- 172.21.15.22:0/250235556 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f99a007b5b0 con 0x7f99a80fe5b0 2024-02-16T19:41:40.563 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:41:40 smithi022 ceph-mon[88798]: from='client.? 172.21.15.22:0/668813462' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-02-16T19:41:40.564 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:41:40 smithi022 ceph-mon[88798]: from='client.? 172.21.15.22:0/3567348446' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-02-16T19:41:40.564 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:41:40 smithi022 ceph-mon[88798]: from='client.? 172.21.15.22:0/4157013569' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-02-16T19:41:40.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.625+0000 7f99b07c4640 1 -- 172.21.15.22:0/250235556 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 21, "format": "json"} v 0) v1 -- 0x7f99a81954f0 con 0x7f99a80fe5b0 2024-02-16T19:41:40.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.626+0000 7f999f7fe640 1 -- 172.21.15.22:0/250235556 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 21, "format": "json"}]=0 dumped fsmap epoch 21 v37) v1 ==== 107+0+4901 (secure 0 0 0) 0x7f99a007ec70 con 0x7f99a80fe5b0 2024-02-16T19:41:40.627 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:41:40.627 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24265,"name":"cephfs.smithi117.orzors","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.117:6827/3122342028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":3122342028},{"type":"v1","addr":"172.21.15.117:6827","nonce":3122342028}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":34384,"name":"cephfs.smithi022.ocukvs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6827/2535680483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":2535680483},{"type":"v1","addr":"172.21.15.22:6827","nonce":2535680483}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17},{"gid":34408,"name":"cephfs.smithi022.btppwm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6829/3126701730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3126701730},{"type":"v1","addr":"172.21.15.22:6829","nonce":3126701730}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T19:26:17.873644+0000","modified":"2024-02-16T19:38:18.584549+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":91,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24259},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24259":{"gid":24259,"name":"cephfs.smithi117.ibmppn","rank":0,"incarnation":19,"state":"up:reconnect","state_seq":180,"addr":"172.21.15.117:6825/565408102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":565408102},{"type":"v1","addr":"172.21.15.117:6825","nonce":565408102}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T19:41:40.627 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 21 2024-02-16T19:41:40.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.627+0000 7f99b07c4640 1 -- 172.21.15.22:0/250235556 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f999406d810 msgr2=0x7f999406fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:40.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.627+0000 7f99b07c4640 1 --2- 172.21.15.22:0/250235556 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f999406d810 0x7f999406fcc0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f99a81957b0 tx=0x7f9998005f90 comp rx=0 tx=0).stop 2024-02-16T19:41:40.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.628+0000 7f99b07c4640 1 -- 172.21.15.22:0/250235556 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99a80fe5b0 msgr2=0x7f99a81941a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:40.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.628+0000 7f99b07c4640 1 --2- 172.21.15.22:0/250235556 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99a80fe5b0 0x7f99a81941a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f99a002efd0 tx=0x7f99a0032b60 comp rx=0 tx=0).stop 2024-02-16T19:41:40.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.628+0000 7f99b07c4640 1 -- 172.21.15.22:0/250235556 shutdown_connections 2024-02-16T19:41:40.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.628+0000 7f99b07c4640 1 --2- 172.21.15.22:0/250235556 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f999406d810 0x7f999406fcc0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:40.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.628+0000 7f99b07c4640 1 --2- 172.21.15.22:0/250235556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99a8104590 0x7f99a81946e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:40.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.628+0000 7f99b07c4640 1 --2- 172.21.15.22:0/250235556 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99a80fe5b0 0x7f99a81941a0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:40.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.628+0000 7f99b07c4640 1 -- 172.21.15.22:0/250235556 >> 172.21.15.22:0/250235556 conn(0x7f99a80743d0 msgr2=0x7f99a80fce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:40.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.628+0000 7f99b07c4640 1 -- 172.21.15.22:0/250235556 shutdown_connections 2024-02-16T19:41:40.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.628+0000 7f99b07c4640 1 -- 172.21.15.22:0/250235556 wait complete. 2024-02-16T19:41:40.640 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 22 2024-02-16T19:41:40.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.743+0000 7f7bec7d0640 1 -- 172.21.15.22:0/3430907298 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7be4105880 msgr2=0x7f7be4105c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:40.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.743+0000 7f7bec7d0640 1 --2- 172.21.15.22:0/3430907298 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7be4105880 0x7f7be4105c50 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7bd4009c20 tx=0x7f7bd402c1b0 comp rx=0 tx=0).stop 2024-02-16T19:41:40.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.743+0000 7f7bec7d0640 1 -- 172.21.15.22:0/3430907298 shutdown_connections 2024-02-16T19:41:40.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.743+0000 7f7bec7d0640 1 --2- 172.21.15.22:0/3430907298 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7be40ff870 0x7f7be40ffce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:40.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.743+0000 7f7bec7d0640 1 --2- 172.21.15.22:0/3430907298 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7be4105880 0x7f7be4105c50 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:40.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.743+0000 7f7bec7d0640 1 -- 172.21.15.22:0/3430907298 >> 172.21.15.22:0/3430907298 conn(0x7f7be40fb370 msgr2=0x7f7be40fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:40.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.743+0000 7f7bec7d0640 1 -- 172.21.15.22:0/3430907298 shutdown_connections 2024-02-16T19:41:40.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.743+0000 7f7bec7d0640 1 -- 172.21.15.22:0/3430907298 wait complete. 2024-02-16T19:41:40.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.743+0000 7f7bec7d0640 1 Processor -- start 2024-02-16T19:41:40.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.743+0000 7f7bec7d0640 1 -- start start 2024-02-16T19:41:40.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.743+0000 7f7bec7d0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7be40ff870 0x7f7be4195540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:40.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.743+0000 7f7bec7d0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7be4105880 0x7f7be4195a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:40.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.743+0000 7f7bec7d0640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7be4196110 con 0x7f7be40ff870 2024-02-16T19:41:40.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.743+0000 7f7bec7d0640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7be4199ee0 con 0x7f7be4105880 2024-02-16T19:41:40.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.744+0000 7f7bea56c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7be40ff870 0x7f7be4195540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:40.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.744+0000 7f7bea56c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7be40ff870 0x7f7be4195540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45602/0 (socket says 172.21.15.22:45602) 2024-02-16T19:41:40.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.744+0000 7f7bea56c640 1 -- 172.21.15.22:0/605028403 learned_addr learned my addr 172.21.15.22:0/605028403 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:41:40.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.744+0000 7f7bea56c640 1 -- 172.21.15.22:0/605028403 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7be4105880 msgr2=0x7f7be4195a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:40.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.744+0000 7f7bea56c640 1 --2- 172.21.15.22:0/605028403 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7be4105880 0x7f7be4195a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:40.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.744+0000 7f7bea56c640 1 -- 172.21.15.22:0/605028403 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bd4009900 con 0x7f7be40ff870 2024-02-16T19:41:40.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.744+0000 7f7bea56c640 1 --2- 172.21.15.22:0/605028403 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7be40ff870 0x7f7be4195540 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7bd4009430 tx=0x7f7bd4032b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:40.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.744+0000 7f7be37fe640 1 -- 172.21.15.22:0/605028403 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7bd4032ed0 con 0x7f7be40ff870 2024-02-16T19:41:40.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.744+0000 7f7be37fe640 1 -- 172.21.15.22:0/605028403 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7bd4003e30 con 0x7f7be40ff870 2024-02-16T19:41:40.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.744+0000 7f7bec7d0640 1 -- 172.21.15.22:0/605028403 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7be419a160 con 0x7f7be40ff870 2024-02-16T19:41:40.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.744+0000 7f7be37fe640 1 -- 172.21.15.22:0/605028403 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7bd403f440 con 0x7f7be40ff870 2024-02-16T19:41:40.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.744+0000 7f7bec7d0640 1 -- 172.21.15.22:0/605028403 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7be419a6e0 con 0x7f7be40ff870 2024-02-16T19:41:40.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.745+0000 7f7bec7d0640 1 -- 172.21.15.22:0/605028403 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7be404e000 con 0x7f7be40ff870 2024-02-16T19:41:40.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.746+0000 7f7be37fe640 1 -- 172.21.15.22:0/605028403 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f7bd403b040 con 0x7f7be40ff870 2024-02-16T19:41:40.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.746+0000 7f7be37fe640 1 --2- 172.21.15.22:0/605028403 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f7bcc06d810 0x7f7bcc06fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:40.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.746+0000 7f7be37fe640 1 -- 172.21.15.22:0/605028403 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f7bd40b02d0 con 0x7f7be40ff870 2024-02-16T19:41:40.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.746+0000 7f7be9d6b640 1 --2- 172.21.15.22:0/605028403 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f7bcc06d810 0x7f7bcc06fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:40.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.747+0000 7f7be9d6b640 1 --2- 172.21.15.22:0/605028403 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f7bcc06d810 0x7f7bcc06fcc0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7bd8005fd0 tx=0x7f7bd8005850 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:40.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.751+0000 7f7be37fe640 1 -- 172.21.15.22:0/605028403 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7bd4079a40 con 0x7f7be40ff870 2024-02-16T19:41:40.856 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:41:40 smithi117 ceph-mon[67870]: from='client.? 172.21.15.22:0/668813462' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-02-16T19:41:40.856 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:41:40 smithi117 ceph-mon[67870]: from='client.? 172.21.15.22:0/3567348446' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-02-16T19:41:40.856 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:41:40 smithi117 ceph-mon[67870]: from='client.? 172.21.15.22:0/4157013569' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-02-16T19:41:40.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.972+0000 7f7bec7d0640 1 -- 172.21.15.22:0/605028403 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 22, "format": "json"} v 0) v1 -- 0x7f7be4066860 con 0x7f7be40ff870 2024-02-16T19:41:40.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.972+0000 7f7be37fe640 1 -- 172.21.15.22:0/605028403 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 22, "format": "json"}]=0 dumped fsmap epoch 22 v37) v1 ==== 107+0+4898 (secure 0 0 0) 0x7f7bd407d100 con 0x7f7be40ff870 2024-02-16T19:41:40.973 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:41:40.974 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24265,"name":"cephfs.smithi117.orzors","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.117:6827/3122342028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":3122342028},{"type":"v1","addr":"172.21.15.117:6827","nonce":3122342028}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":34384,"name":"cephfs.smithi022.ocukvs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6827/2535680483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":2535680483},{"type":"v1","addr":"172.21.15.22:6827","nonce":2535680483}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17},{"gid":34408,"name":"cephfs.smithi022.btppwm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6829/3126701730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3126701730},{"type":"v1","addr":"172.21.15.22:6829","nonce":3126701730}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T19:26:17.873644+0000","modified":"2024-02-16T19:38:18.840999+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":91,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24259},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24259":{"gid":24259,"name":"cephfs.smithi117.ibmppn","rank":0,"incarnation":19,"state":"up:rejoin","state_seq":181,"addr":"172.21.15.117:6825/565408102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":565408102},{"type":"v1","addr":"172.21.15.117:6825","nonce":565408102}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T19:41:40.974 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 22 2024-02-16T19:41:40.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.974+0000 7f7bec7d0640 1 -- 172.21.15.22:0/605028403 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f7bcc06d810 msgr2=0x7f7bcc06fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:40.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.974+0000 7f7bec7d0640 1 --2- 172.21.15.22:0/605028403 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f7bcc06d810 0x7f7bcc06fcc0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7bd8005fd0 tx=0x7f7bd8005850 comp rx=0 tx=0).stop 2024-02-16T19:41:40.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.974+0000 7f7bec7d0640 1 -- 172.21.15.22:0/605028403 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7be40ff870 msgr2=0x7f7be4195540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:40.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.974+0000 7f7bec7d0640 1 --2- 172.21.15.22:0/605028403 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7be40ff870 0x7f7be4195540 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7bd4009430 tx=0x7f7bd4032b60 comp rx=0 tx=0).stop 2024-02-16T19:41:40.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.974+0000 7f7bec7d0640 1 -- 172.21.15.22:0/605028403 shutdown_connections 2024-02-16T19:41:40.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.974+0000 7f7bec7d0640 1 --2- 172.21.15.22:0/605028403 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f7bcc06d810 0x7f7bcc06fcc0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:40.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.974+0000 7f7bec7d0640 1 --2- 172.21.15.22:0/605028403 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7be4105880 0x7f7be4195a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:40.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.974+0000 7f7bec7d0640 1 --2- 172.21.15.22:0/605028403 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7be40ff870 0x7f7be4195540 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:40.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.974+0000 7f7bec7d0640 1 -- 172.21.15.22:0/605028403 >> 172.21.15.22:0/605028403 conn(0x7f7be40fb370 msgr2=0x7f7be40fccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:40.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.974+0000 7f7bec7d0640 1 -- 172.21.15.22:0/605028403 shutdown_connections 2024-02-16T19:41:40.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:40.974+0000 7f7bec7d0640 1 -- 172.21.15.22:0/605028403 wait complete. 2024-02-16T19:41:40.987 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 23 2024-02-16T19:41:41.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.088+0000 7f566daea640 1 -- 172.21.15.22:0/313502400 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5668104780 msgr2=0x7f5668104b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:41.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.088+0000 7f566daea640 1 --2- 172.21.15.22:0/313502400 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5668104780 0x7f5668104b50 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5658009880 tx=0x7f565802bce0 comp rx=0 tx=0).stop 2024-02-16T19:41:41.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.088+0000 7f566daea640 1 -- 172.21.15.22:0/313502400 shutdown_connections 2024-02-16T19:41:41.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.088+0000 7f566daea640 1 --2- 172.21.15.22:0/313502400 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56680fe770 0x7f56680febe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:41.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.088+0000 7f566daea640 1 --2- 172.21.15.22:0/313502400 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5668104780 0x7f5668104b50 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:41.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.088+0000 7f566daea640 1 -- 172.21.15.22:0/313502400 >> 172.21.15.22:0/313502400 conn(0x7f56680fa270 msgr2=0x7f56680fc680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:41.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.088+0000 7f566daea640 1 -- 172.21.15.22:0/313502400 shutdown_connections 2024-02-16T19:41:41.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.088+0000 7f566daea640 1 -- 172.21.15.22:0/313502400 wait complete. 2024-02-16T19:41:41.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.088+0000 7f566daea640 1 Processor -- start 2024-02-16T19:41:41.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.088+0000 7f566daea640 1 -- start start 2024-02-16T19:41:41.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.089+0000 7f566daea640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f56680fe770 0x7f5668100380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:41.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.089+0000 7f566daea640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5668104780 0x7f56681008c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:41.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.089+0000 7f566daea640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5668100e90 con 0x7f56680fe770 2024-02-16T19:41:41.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.089+0000 7f566daea640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5668100fd0 con 0x7f5668104780 2024-02-16T19:41:41.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.089+0000 7f56677fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f56680fe770 0x7f5668100380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:41.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.089+0000 7f56677fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f56680fe770 0x7f5668100380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45622/0 (socket says 172.21.15.22:45622) 2024-02-16T19:41:41.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.089+0000 7f56677fe640 1 -- 172.21.15.22:0/440901727 learned_addr learned my addr 172.21.15.22:0/440901727 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:41:41.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.089+0000 7f5666ffd640 1 --2- 172.21.15.22:0/440901727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5668104780 0x7f56681008c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:41.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.089+0000 7f56677fe640 1 -- 172.21.15.22:0/440901727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5668104780 msgr2=0x7f56681008c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:41.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.089+0000 7f56677fe640 1 --2- 172.21.15.22:0/440901727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5668104780 0x7f56681008c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:41.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.089+0000 7f56677fe640 1 -- 172.21.15.22:0/440901727 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5658009560 con 0x7f56680fe770 2024-02-16T19:41:41.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.089+0000 7f5666ffd640 1 --2- 172.21.15.22:0/440901727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5668104780 0x7f56681008c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-16T19:41:41.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.089+0000 7f56677fe640 1 --2- 172.21.15.22:0/440901727 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f56680fe770 0x7f5668100380 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5658009230 tx=0x7f5658004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:41.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.089+0000 7f5664ff9640 1 -- 172.21.15.22:0/440901727 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f56580037c0 con 0x7f56680fe770 2024-02-16T19:41:41.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.089+0000 7f5664ff9640 1 -- 172.21.15.22:0/440901727 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5658003de0 con 0x7f56680fe770 2024-02-16T19:41:41.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.089+0000 7f5664ff9640 1 -- 172.21.15.22:0/440901727 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5658034440 con 0x7f56680fe770 2024-02-16T19:41:41.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.089+0000 7f566daea640 1 -- 172.21.15.22:0/440901727 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5668101200 con 0x7f56680fe770 2024-02-16T19:41:41.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.089+0000 7f566daea640 1 -- 172.21.15.22:0/440901727 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56681a1a50 con 0x7f56680fe770 2024-02-16T19:41:41.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.090+0000 7f566daea640 1 -- 172.21.15.22:0/440901727 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f566804e000 con 0x7f56680fe770 2024-02-16T19:41:41.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.091+0000 7f5664ff9640 1 -- 172.21.15.22:0/440901727 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f5658004fb0 con 0x7f56680fe770 2024-02-16T19:41:41.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.091+0000 7f5664ff9640 1 --2- 172.21.15.22:0/440901727 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f565006d810 0x7f565006fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:41.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.091+0000 7f5664ff9640 1 -- 172.21.15.22:0/440901727 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f56580afc70 con 0x7f56680fe770 2024-02-16T19:41:41.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.091+0000 7f5666ffd640 1 --2- 172.21.15.22:0/440901727 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f565006d810 0x7f565006fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:41.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.092+0000 7f5666ffd640 1 --2- 172.21.15.22:0/440901727 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f565006d810 0x7f565006fcc0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5668101ef0 tx=0x7f565c005d30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:41.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.096+0000 7f5664ff9640 1 -- 172.21.15.22:0/440901727 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f56580b6080 con 0x7f56680fe770 2024-02-16T19:41:41.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.314+0000 7f566daea640 1 -- 172.21.15.22:0/440901727 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 23, "format": "json"} v 0) v1 -- 0x7f5668066860 con 0x7f56680fe770 2024-02-16T19:41:41.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.314+0000 7f5664ff9640 1 -- 172.21.15.22:0/440901727 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 23, "format": "json"}]=0 dumped fsmap epoch 23 v37) v1 ==== 107+0+4898 (secure 0 0 0) 0x7f5658079200 con 0x7f56680fe770 2024-02-16T19:41:41.316 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:41:41.316 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24265,"name":"cephfs.smithi117.orzors","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.117:6827/3122342028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":3122342028},{"type":"v1","addr":"172.21.15.117:6827","nonce":3122342028}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":34384,"name":"cephfs.smithi022.ocukvs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6827/2535680483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":2535680483},{"type":"v1","addr":"172.21.15.22:6827","nonce":2535680483}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17},{"gid":34408,"name":"cephfs.smithi022.btppwm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6829/3126701730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3126701730},{"type":"v1","addr":"172.21.15.22:6829","nonce":3126701730}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T19:26:17.873644+0000","modified":"2024-02-16T19:38:19.908559+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":91,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24259},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24259":{"gid":24259,"name":"cephfs.smithi117.ibmppn","rank":0,"incarnation":19,"state":"up:active","state_seq":182,"addr":"172.21.15.117:6825/565408102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":565408102},{"type":"v1","addr":"172.21.15.117:6825","nonce":565408102}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T19:41:41.316 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 23 2024-02-16T19:41:41.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.316+0000 7f566daea640 1 -- 172.21.15.22:0/440901727 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f565006d810 msgr2=0x7f565006fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:41.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.316+0000 7f566daea640 1 --2- 172.21.15.22:0/440901727 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f565006d810 0x7f565006fcc0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5668101ef0 tx=0x7f565c005d30 comp rx=0 tx=0).stop 2024-02-16T19:41:41.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.316+0000 7f566daea640 1 -- 172.21.15.22:0/440901727 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f56680fe770 msgr2=0x7f5668100380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:41.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.316+0000 7f566daea640 1 --2- 172.21.15.22:0/440901727 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f56680fe770 0x7f5668100380 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5658009230 tx=0x7f5658004270 comp rx=0 tx=0).stop 2024-02-16T19:41:41.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.316+0000 7f566daea640 1 -- 172.21.15.22:0/440901727 shutdown_connections 2024-02-16T19:41:41.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.316+0000 7f566daea640 1 --2- 172.21.15.22:0/440901727 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f565006d810 0x7f565006fcc0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:41.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.316+0000 7f566daea640 1 --2- 172.21.15.22:0/440901727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5668104780 0x7f56681008c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:41.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.316+0000 7f566daea640 1 --2- 172.21.15.22:0/440901727 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f56680fe770 0x7f5668100380 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:41.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.316+0000 7f566daea640 1 -- 172.21.15.22:0/440901727 >> 172.21.15.22:0/440901727 conn(0x7f56680fa270 msgr2=0x7f56680683e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:41.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.317+0000 7f566daea640 1 -- 172.21.15.22:0/440901727 shutdown_connections 2024-02-16T19:41:41.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.317+0000 7f566daea640 1 -- 172.21.15.22:0/440901727 wait complete. 2024-02-16T19:41:41.329 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 24 2024-02-16T19:41:41.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.431+0000 7f68777e4640 1 -- 172.21.15.22:0/2976960202 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6870100830 msgr2=0x7f6870100c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:41.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.431+0000 7f68777e4640 1 --2- 172.21.15.22:0/2976960202 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6870100830 0x7f6870100c00 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f68680098b0 tx=0x7f686802bd60 comp rx=0 tx=0).stop 2024-02-16T19:41:41.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.432+0000 7f68777e4640 1 -- 172.21.15.22:0/2976960202 shutdown_connections 2024-02-16T19:41:41.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.432+0000 7f68777e4640 1 --2- 172.21.15.22:0/2976960202 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6870101140 0x7f6870109630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:41.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.432+0000 7f68777e4640 1 --2- 172.21.15.22:0/2976960202 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6870100830 0x7f6870100c00 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:41.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.432+0000 7f68777e4640 1 -- 172.21.15.22:0/2976960202 >> 172.21.15.22:0/2976960202 conn(0x7f68700fa060 msgr2=0x7f68700fc470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:41.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.432+0000 7f68777e4640 1 -- 172.21.15.22:0/2976960202 shutdown_connections 2024-02-16T19:41:41.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.432+0000 7f68777e4640 1 -- 172.21.15.22:0/2976960202 wait complete. 2024-02-16T19:41:41.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.432+0000 7f68777e4640 1 Processor -- start 2024-02-16T19:41:41.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.432+0000 7f68777e4640 1 -- start start 2024-02-16T19:41:41.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.432+0000 7f68777e4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6870100830 0x7f6870195360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:41.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.432+0000 7f68777e4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6870101140 0x7f68701958a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:41.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.432+0000 7f68777e4640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6870195ea0 con 0x7f6870100830 2024-02-16T19:41:41.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.432+0000 7f68777e4640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6870196010 con 0x7f6870101140 2024-02-16T19:41:41.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.433+0000 7f6875580640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6870100830 0x7f6870195360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:41.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.433+0000 7f6875580640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6870100830 0x7f6870195360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45640/0 (socket says 172.21.15.22:45640) 2024-02-16T19:41:41.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.433+0000 7f6875580640 1 -- 172.21.15.22:0/1227458521 learned_addr learned my addr 172.21.15.22:0/1227458521 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:41:41.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.433+0000 7f6874d7f640 1 --2- 172.21.15.22:0/1227458521 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6870101140 0x7f68701958a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:41.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.433+0000 7f6875580640 1 -- 172.21.15.22:0/1227458521 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6870101140 msgr2=0x7f68701958a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:41.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.433+0000 7f6875580640 1 --2- 172.21.15.22:0/1227458521 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6870101140 0x7f68701958a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:41.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.433+0000 7f6875580640 1 -- 172.21.15.22:0/1227458521 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6868009560 con 0x7f6870100830 2024-02-16T19:41:41.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.433+0000 7f6875580640 1 --2- 172.21.15.22:0/1227458521 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6870100830 0x7f6870195360 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6868009880 tx=0x7f6868004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:41.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.433+0000 7f686e7fc640 1 -- 172.21.15.22:0/1227458521 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f68680037c0 con 0x7f6870100830 2024-02-16T19:41:41.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.433+0000 7f686e7fc640 1 -- 172.21.15.22:0/1227458521 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6868003de0 con 0x7f6870100830 2024-02-16T19:41:41.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.433+0000 7f68777e4640 1 -- 172.21.15.22:0/1227458521 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6870199e40 con 0x7f6870100830 2024-02-16T19:41:41.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.433+0000 7f686e7fc640 1 -- 172.21.15.22:0/1227458521 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f686803e450 con 0x7f6870100830 2024-02-16T19:41:41.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.433+0000 7f68777e4640 1 -- 172.21.15.22:0/1227458521 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f687019a3a0 con 0x7f6870100830 2024-02-16T19:41:41.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.434+0000 7f68777e4640 1 -- 172.21.15.22:0/1227458521 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f687004e000 con 0x7f6870100830 2024-02-16T19:41:41.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.435+0000 7f686e7fc640 1 -- 172.21.15.22:0/1227458521 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f6868004fd0 con 0x7f6870100830 2024-02-16T19:41:41.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.435+0000 7f686e7fc640 1 --2- 172.21.15.22:0/1227458521 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f685806d810 0x7f685806fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:41.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.435+0000 7f686e7fc640 1 -- 172.21.15.22:0/1227458521 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f68680b0470 con 0x7f6870100830 2024-02-16T19:41:41.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.435+0000 7f6874d7f640 1 --2- 172.21.15.22:0/1227458521 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f685806d810 0x7f685806fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:41.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.436+0000 7f6874d7f640 1 --2- 172.21.15.22:0/1227458521 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f685806d810 0x7f685806fcc0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6864005e10 tx=0x7f6864005da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:41.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.440+0000 7f686e7fc640 1 -- 172.21.15.22:0/1227458521 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6868079bc0 con 0x7f6870100830 2024-02-16T19:41:41.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.657+0000 7f68777e4640 1 -- 172.21.15.22:0/1227458521 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 24, "format": "json"} v 0) v1 -- 0x7f687010c9f0 con 0x7f6870100830 2024-02-16T19:41:41.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.658+0000 7f686e7fc640 1 -- 172.21.15.22:0/1227458521 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 24, "format": "json"}]=0 dumped fsmap epoch 24 v37) v1 ==== 107+0+4106 (secure 0 0 0) 0x7f686807d280 con 0x7f6870100830 2024-02-16T19:41:41.659 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:41:41.660 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24265,"name":"cephfs.smithi117.orzors","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.117:6827/3122342028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":3122342028},{"type":"v1","addr":"172.21.15.117:6827","nonce":3122342028}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":34384,"name":"cephfs.smithi022.ocukvs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6827/2535680483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":2535680483},{"type":"v1","addr":"172.21.15.22:6827","nonce":2535680483}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17},{"gid":34408,"name":"cephfs.smithi022.btppwm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6829/3126701730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3126701730},{"type":"v1","addr":"172.21.15.22:6829","nonce":3126701730}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T19:26:17.873644+0000","modified":"2024-02-16T19:38:31.770707+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T19:41:41.660 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 24 2024-02-16T19:41:41.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.660+0000 7f68777e4640 1 -- 172.21.15.22:0/1227458521 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f685806d810 msgr2=0x7f685806fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:41.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.660+0000 7f68777e4640 1 --2- 172.21.15.22:0/1227458521 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f685806d810 0x7f685806fcc0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6864005e10 tx=0x7f6864005da0 comp rx=0 tx=0).stop 2024-02-16T19:41:41.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.660+0000 7f68777e4640 1 -- 172.21.15.22:0/1227458521 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6870100830 msgr2=0x7f6870195360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:41.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.660+0000 7f68777e4640 1 --2- 172.21.15.22:0/1227458521 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6870100830 0x7f6870195360 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6868009880 tx=0x7f6868004270 comp rx=0 tx=0).stop 2024-02-16T19:41:41.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.660+0000 7f68777e4640 1 -- 172.21.15.22:0/1227458521 shutdown_connections 2024-02-16T19:41:41.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.660+0000 7f68777e4640 1 --2- 172.21.15.22:0/1227458521 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f685806d810 0x7f685806fcc0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:41.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.660+0000 7f68777e4640 1 --2- 172.21.15.22:0/1227458521 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6870101140 0x7f68701958a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:41.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.660+0000 7f68777e4640 1 --2- 172.21.15.22:0/1227458521 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6870100830 0x7f6870195360 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:41.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.660+0000 7f68777e4640 1 -- 172.21.15.22:0/1227458521 >> 172.21.15.22:0/1227458521 conn(0x7f68700fa060 msgr2=0x7f6870103e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:41.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.660+0000 7f68777e4640 1 -- 172.21.15.22:0/1227458521 shutdown_connections 2024-02-16T19:41:41.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.660+0000 7f68777e4640 1 -- 172.21.15.22:0/1227458521 wait complete. 2024-02-16T19:41:41.671 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:41:41 smithi022 ceph-mon[88798]: pgmap v298: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:41:41.671 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:41:41 smithi022 ceph-mon[88798]: from='client.? 172.21.15.22:0/250235556' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-02-16T19:41:41.671 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:41:41 smithi022 ceph-mon[88798]: from='client.? 172.21.15.22:0/605028403' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-02-16T19:41:41.671 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:41:41 smithi022 ceph-mon[88798]: from='client.? 172.21.15.22:0/440901727' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-02-16T19:41:41.672 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 25 2024-02-16T19:41:41.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.774+0000 7f28e4a41640 1 -- 172.21.15.22:0/3696375546 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28e00ff870 msgr2=0x7f28e00ffce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:41.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.774+0000 7f28e4a41640 1 --2- 172.21.15.22:0/3696375546 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28e00ff870 0x7f28e00ffce0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f28c8009880 tx=0x7f28c802bce0 comp rx=0 tx=0).stop 2024-02-16T19:41:41.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.774+0000 7f28e4a41640 1 -- 172.21.15.22:0/3696375546 shutdown_connections 2024-02-16T19:41:41.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.774+0000 7f28e4a41640 1 --2- 172.21.15.22:0/3696375546 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28e00ff870 0x7f28e00ffce0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:41.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.774+0000 7f28e4a41640 1 --2- 172.21.15.22:0/3696375546 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28e0105880 0x7f28e0105c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:41.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.774+0000 7f28e4a41640 1 -- 172.21.15.22:0/3696375546 >> 172.21.15.22:0/3696375546 conn(0x7f28e00fb370 msgr2=0x7f28e00fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:41.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.774+0000 7f28e4a41640 1 -- 172.21.15.22:0/3696375546 shutdown_connections 2024-02-16T19:41:41.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.774+0000 7f28e4a41640 1 -- 172.21.15.22:0/3696375546 wait complete. 2024-02-16T19:41:41.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.775+0000 7f28e4a41640 1 Processor -- start 2024-02-16T19:41:41.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.775+0000 7f28e4a41640 1 -- start start 2024-02-16T19:41:41.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.775+0000 7f28e4a41640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28e00ff870 0x7f28e0195580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:41.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.775+0000 7f28e4a41640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28e0105880 0x7f28e0195ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:41.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.775+0000 7f28e4a41640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e0196150 con 0x7f28e0105880 2024-02-16T19:41:41.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.775+0000 7f28e4a41640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e0199f20 con 0x7f28e00ff870 2024-02-16T19:41:41.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.775+0000 7f28d5d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28e0105880 0x7f28e0195ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:41.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.775+0000 7f28d5d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28e0105880 0x7f28e0195ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45658/0 (socket says 172.21.15.22:45658) 2024-02-16T19:41:41.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.775+0000 7f28d5d9b640 1 -- 172.21.15.22:0/2886925166 learned_addr learned my addr 172.21.15.22:0/2886925166 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:41:41.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.776+0000 7f28de59c640 1 --2- 172.21.15.22:0/2886925166 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28e00ff870 0x7f28e0195580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:41.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.776+0000 7f28d5d9b640 1 -- 172.21.15.22:0/2886925166 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28e00ff870 msgr2=0x7f28e0195580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:41.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.776+0000 7f28d5d9b640 1 --2- 172.21.15.22:0/2886925166 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28e00ff870 0x7f28e0195580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:41.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.776+0000 7f28d5d9b640 1 -- 172.21.15.22:0/2886925166 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28c8009560 con 0x7f28e0105880 2024-02-16T19:41:41.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.776+0000 7f28d5d9b640 1 --2- 172.21.15.22:0/2886925166 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28e0105880 0x7f28e0195ac0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f28c8009230 tx=0x7f28c8004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:41.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.776+0000 7f28d7fff640 1 -- 172.21.15.22:0/2886925166 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28c80035f0 con 0x7f28e0105880 2024-02-16T19:41:41.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.776+0000 7f28e4a41640 1 -- 172.21.15.22:0/2886925166 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28e019a1a0 con 0x7f28e0105880 2024-02-16T19:41:41.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.776+0000 7f28d7fff640 1 -- 172.21.15.22:0/2886925166 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f28c8003c10 con 0x7f28e0105880 2024-02-16T19:41:41.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.776+0000 7f28d7fff640 1 -- 172.21.15.22:0/2886925166 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28c8034440 con 0x7f28e0105880 2024-02-16T19:41:41.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.776+0000 7f28e4a41640 1 -- 172.21.15.22:0/2886925166 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28e019a6a0 con 0x7f28e0105880 2024-02-16T19:41:41.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.777+0000 7f28e4a41640 1 -- 172.21.15.22:0/2886925166 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28e004e000 con 0x7f28e0105880 2024-02-16T19:41:41.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.778+0000 7f28d7fff640 1 -- 172.21.15.22:0/2886925166 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f28c8004fd0 con 0x7f28e0105880 2024-02-16T19:41:41.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.778+0000 7f28d7fff640 1 --2- 172.21.15.22:0/2886925166 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f28cc06d810 0x7f28cc06fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:41.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.778+0000 7f28d7fff640 1 -- 172.21.15.22:0/2886925166 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f28c80afa90 con 0x7f28e0105880 2024-02-16T19:41:41.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.778+0000 7f28de59c640 1 --2- 172.21.15.22:0/2886925166 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f28cc06d810 0x7f28cc06fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:41.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.778+0000 7f28de59c640 1 --2- 172.21.15.22:0/2886925166 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f28cc06d810 0x7f28cc06fcc0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f28d00058c0 tx=0x7f28d000a2d0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:41.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:41.782+0000 7f28d7fff640 1 -- 172.21.15.22:0/2886925166 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f28c8079170 con 0x7f28e0105880 2024-02-16T19:41:41.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:41:41 smithi117 ceph-mon[67870]: pgmap v298: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:41:41.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:41:41 smithi117 ceph-mon[67870]: from='client.? 172.21.15.22:0/250235556' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-02-16T19:41:41.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:41:41 smithi117 ceph-mon[67870]: from='client.? 172.21.15.22:0/605028403' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-02-16T19:41:41.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:41:41 smithi117 ceph-mon[67870]: from='client.? 172.21.15.22:0/440901727' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-02-16T19:41:42.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.001+0000 7f28e4a41640 1 -- 172.21.15.22:0/2886925166 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 25, "format": "json"} v 0) v1 -- 0x7f28e0066860 con 0x7f28e0105880 2024-02-16T19:41:42.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.002+0000 7f28d7fff640 1 -- 172.21.15.22:0/2886925166 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 25, "format": "json"}]=0 dumped fsmap epoch 25 v37) v1 ==== 107+0+4117 (secure 0 0 0) 0x7f28c807c830 con 0x7f28e0105880 2024-02-16T19:41:42.003 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:41:42.003 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34384,"name":"cephfs.smithi022.ocukvs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6827/2535680483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":2535680483},{"type":"v1","addr":"172.21.15.22:6827","nonce":2535680483}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17},{"gid":34408,"name":"cephfs.smithi022.btppwm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6829/3126701730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3126701730},{"type":"v1","addr":"172.21.15.22:6829","nonce":3126701730}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T19:26:17.873644+0000","modified":"2024-02-16T19:38:31.776110+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24265},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24265":{"gid":24265,"name":"cephfs.smithi117.orzors","rank":0,"incarnation":25,"state":"up:replay","state_seq":2,"addr":"172.21.15.117:6827/3122342028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":3122342028},{"type":"v1","addr":"172.21.15.117:6827","nonce":3122342028}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T19:41:42.004 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 25 2024-02-16T19:41:42.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.004+0000 7f28e4a41640 1 -- 172.21.15.22:0/2886925166 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f28cc06d810 msgr2=0x7f28cc06fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:42.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.004+0000 7f28e4a41640 1 --2- 172.21.15.22:0/2886925166 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f28cc06d810 0x7f28cc06fcc0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f28d00058c0 tx=0x7f28d000a2d0 comp rx=0 tx=0).stop 2024-02-16T19:41:42.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.004+0000 7f28e4a41640 1 -- 172.21.15.22:0/2886925166 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28e0105880 msgr2=0x7f28e0195ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:42.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.004+0000 7f28e4a41640 1 --2- 172.21.15.22:0/2886925166 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28e0105880 0x7f28e0195ac0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f28c8009230 tx=0x7f28c8004040 comp rx=0 tx=0).stop 2024-02-16T19:41:42.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.004+0000 7f28e4a41640 1 -- 172.21.15.22:0/2886925166 shutdown_connections 2024-02-16T19:41:42.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.004+0000 7f28e4a41640 1 --2- 172.21.15.22:0/2886925166 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f28cc06d810 0x7f28cc06fcc0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:42.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.004+0000 7f28e4a41640 1 --2- 172.21.15.22:0/2886925166 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28e0105880 0x7f28e0195ac0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:42.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.004+0000 7f28e4a41640 1 --2- 172.21.15.22:0/2886925166 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28e00ff870 0x7f28e0195580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:42.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.004+0000 7f28e4a41640 1 -- 172.21.15.22:0/2886925166 >> 172.21.15.22:0/2886925166 conn(0x7f28e00fb370 msgr2=0x7f28e00fcd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:42.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.004+0000 7f28e4a41640 1 -- 172.21.15.22:0/2886925166 shutdown_connections 2024-02-16T19:41:42.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.004+0000 7f28e4a41640 1 -- 172.21.15.22:0/2886925166 wait complete. 2024-02-16T19:41:42.016 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 26 2024-02-16T19:41:42.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.118+0000 7fcf3404a640 1 -- 172.21.15.22:0/863415568 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcf2c104800 msgr2=0x7fcf2c104bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:42.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.118+0000 7fcf3404a640 1 --2- 172.21.15.22:0/863415568 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcf2c104800 0x7fcf2c104bd0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fcf28009880 tx=0x7fcf2802bce0 comp rx=0 tx=0).stop 2024-02-16T19:41:42.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.118+0000 7fcf3404a640 1 -- 172.21.15.22:0/863415568 shutdown_connections 2024-02-16T19:41:42.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.118+0000 7fcf3404a640 1 --2- 172.21.15.22:0/863415568 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf2c0fe7f0 0x7fcf2c0fec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:42.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.118+0000 7fcf3404a640 1 --2- 172.21.15.22:0/863415568 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcf2c104800 0x7fcf2c104bd0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:42.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.118+0000 7fcf3404a640 1 -- 172.21.15.22:0/863415568 >> 172.21.15.22:0/863415568 conn(0x7fcf2c0fa2f0 msgr2=0x7fcf2c0fc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:42.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.118+0000 7fcf3404a640 1 -- 172.21.15.22:0/863415568 shutdown_connections 2024-02-16T19:41:42.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.118+0000 7fcf3404a640 1 -- 172.21.15.22:0/863415568 wait complete. 2024-02-16T19:41:42.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.118+0000 7fcf3404a640 1 Processor -- start 2024-02-16T19:41:42.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.119+0000 7fcf3404a640 1 -- start start 2024-02-16T19:41:42.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.119+0000 7fcf3404a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf2c0fe7f0 0x7fcf2c195530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:42.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.119+0000 7fcf3404a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcf2c104800 0x7fcf2c195a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:42.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.119+0000 7fcf3404a640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2c196100 con 0x7fcf2c104800 2024-02-16T19:41:42.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.119+0000 7fcf3404a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2c199ed0 con 0x7fcf2c0fe7f0 2024-02-16T19:41:42.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.119+0000 7fcf315e5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcf2c104800 0x7fcf2c195a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:42.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.119+0000 7fcf315e5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcf2c104800 0x7fcf2c195a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45672/0 (socket says 172.21.15.22:45672) 2024-02-16T19:41:42.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.119+0000 7fcf315e5640 1 -- 172.21.15.22:0/3631915193 learned_addr learned my addr 172.21.15.22:0/3631915193 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:41:42.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.119+0000 7fcf31de6640 1 --2- 172.21.15.22:0/3631915193 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf2c0fe7f0 0x7fcf2c195530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:42.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.119+0000 7fcf315e5640 1 -- 172.21.15.22:0/3631915193 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf2c0fe7f0 msgr2=0x7fcf2c195530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:42.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.119+0000 7fcf315e5640 1 --2- 172.21.15.22:0/3631915193 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf2c0fe7f0 0x7fcf2c195530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:42.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.119+0000 7fcf315e5640 1 -- 172.21.15.22:0/3631915193 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf28009560 con 0x7fcf2c104800 2024-02-16T19:41:42.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.119+0000 7fcf315e5640 1 --2- 172.21.15.22:0/3631915193 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcf2c104800 0x7fcf2c195a70 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c00d6f0 tx=0x7fcf1c00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:42.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.119+0000 7fcf22ffd640 1 -- 172.21.15.22:0/3631915193 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf1c009a30 con 0x7fcf2c104800 2024-02-16T19:41:42.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.119+0000 7fcf3404a640 1 -- 172.21.15.22:0/3631915193 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf2c19a1b0 con 0x7fcf2c104800 2024-02-16T19:41:42.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.119+0000 7fcf22ffd640 1 -- 172.21.15.22:0/3631915193 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcf1c0054e0 con 0x7fcf2c104800 2024-02-16T19:41:42.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.119+0000 7fcf22ffd640 1 -- 172.21.15.22:0/3631915193 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf1c010460 con 0x7fcf2c104800 2024-02-16T19:41:42.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.119+0000 7fcf3404a640 1 -- 172.21.15.22:0/3631915193 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf2c19a790 con 0x7fcf2c104800 2024-02-16T19:41:42.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.121+0000 7fcf3404a640 1 -- 172.21.15.22:0/3631915193 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf2c04e000 con 0x7fcf2c104800 2024-02-16T19:41:42.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.121+0000 7fcf22ffd640 1 -- 172.21.15.22:0/3631915193 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fcf1c009b90 con 0x7fcf2c104800 2024-02-16T19:41:42.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.121+0000 7fcf22ffd640 1 --2- 172.21.15.22:0/3631915193 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fcf1806d810 0x7fcf1806fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:42.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.121+0000 7fcf22ffd640 1 -- 172.21.15.22:0/3631915193 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6962+0+0 (secure 0 0 0) 0x7fcf1c08d190 con 0x7fcf2c104800 2024-02-16T19:41:42.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.122+0000 7fcf31de6640 1 --2- 172.21.15.22:0/3631915193 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fcf1806d810 0x7fcf1806fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:42.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.122+0000 7fcf31de6640 1 --2- 172.21.15.22:0/3631915193 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fcf1806d810 0x7fcf1806fcc0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fcf28000c00 tx=0x7fcf28005fd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:42.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.126+0000 7fcf22ffd640 1 -- 172.21.15.22:0/3631915193 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcf1c056870 con 0x7fcf2c104800 2024-02-16T19:41:42.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.346+0000 7fcf3404a640 1 -- 172.21.15.22:0/3631915193 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 26, "format": "json"} v 0) v1 -- 0x7fcf2c066860 con 0x7fcf2c104800 2024-02-16T19:41:42.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.346+0000 7fcf22ffd640 1 -- 172.21.15.22:0/3631915193 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 26, "format": "json"}]=0 dumped fsmap epoch 26 v37) v1 ==== 107+0+4122 (secure 0 0 0) 0x7fcf1c059f30 con 0x7fcf2c104800 2024-02-16T19:41:42.348 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:41:42.348 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34384,"name":"cephfs.smithi022.ocukvs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6827/2535680483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":2535680483},{"type":"v1","addr":"172.21.15.22:6827","nonce":2535680483}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17},{"gid":34408,"name":"cephfs.smithi022.btppwm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6829/3126701730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3126701730},{"type":"v1","addr":"172.21.15.22:6829","nonce":3126701730}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T19:26:17.873644+0000","modified":"2024-02-16T19:38:41.235715+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24265},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24265":{"gid":24265,"name":"cephfs.smithi117.orzors","rank":0,"incarnation":25,"state":"up:reconnect","state_seq":184,"addr":"172.21.15.117:6827/3122342028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":3122342028},{"type":"v1","addr":"172.21.15.117:6827","nonce":3122342028}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T19:41:42.353 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 26 2024-02-16T19:41:42.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.349+0000 7fcf3404a640 1 -- 172.21.15.22:0/3631915193 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fcf1806d810 msgr2=0x7fcf1806fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:42.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.349+0000 7fcf3404a640 1 --2- 172.21.15.22:0/3631915193 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fcf1806d810 0x7fcf1806fcc0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fcf28000c00 tx=0x7fcf28005fd0 comp rx=0 tx=0).stop 2024-02-16T19:41:42.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.349+0000 7fcf3404a640 1 -- 172.21.15.22:0/3631915193 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcf2c104800 msgr2=0x7fcf2c195a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:42.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.349+0000 7fcf3404a640 1 --2- 172.21.15.22:0/3631915193 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcf2c104800 0x7fcf2c195a70 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c00d6f0 tx=0x7fcf1c00dbb0 comp rx=0 tx=0).stop 2024-02-16T19:41:42.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.349+0000 7fcf3404a640 1 -- 172.21.15.22:0/3631915193 shutdown_connections 2024-02-16T19:41:42.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.349+0000 7fcf3404a640 1 --2- 172.21.15.22:0/3631915193 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fcf1806d810 0x7fcf1806fcc0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:42.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.349+0000 7fcf3404a640 1 --2- 172.21.15.22:0/3631915193 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcf2c104800 0x7fcf2c195a70 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:42.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.349+0000 7fcf3404a640 1 --2- 172.21.15.22:0/3631915193 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf2c0fe7f0 0x7fcf2c195530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:42.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.349+0000 7fcf3404a640 1 -- 172.21.15.22:0/3631915193 >> 172.21.15.22:0/3631915193 conn(0x7fcf2c0fa2f0 msgr2=0x7fcf2c0684a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:42.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.349+0000 7fcf3404a640 1 -- 172.21.15.22:0/3631915193 shutdown_connections 2024-02-16T19:41:42.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.349+0000 7fcf3404a640 1 -- 172.21.15.22:0/3631915193 wait complete. 2024-02-16T19:41:42.361 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 27 2024-02-16T19:41:42.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.463+0000 7f5d0c30a640 1 -- 172.21.15.22:0/4191717112 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d04104590 msgr2=0x7f5d04104960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:42.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.463+0000 7f5d0c30a640 1 --2- 172.21.15.22:0/4191717112 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d04104590 0x7f5d04104960 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc009880 tx=0x7f5cfc02bce0 comp rx=0 tx=0).stop 2024-02-16T19:41:42.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.464+0000 7f5d0c30a640 1 -- 172.21.15.22:0/4191717112 shutdown_connections 2024-02-16T19:41:42.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.464+0000 7f5d0c30a640 1 --2- 172.21.15.22:0/4191717112 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d040fe5b0 0x7f5d040fea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:42.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.464+0000 7f5d0c30a640 1 --2- 172.21.15.22:0/4191717112 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d04104590 0x7f5d04104960 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:42.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.464+0000 7f5d0c30a640 1 -- 172.21.15.22:0/4191717112 >> 172.21.15.22:0/4191717112 conn(0x7f5d040743d0 msgr2=0x7f5d040747d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:42.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.464+0000 7f5d0c30a640 1 -- 172.21.15.22:0/4191717112 shutdown_connections 2024-02-16T19:41:42.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.464+0000 7f5d0c30a640 1 -- 172.21.15.22:0/4191717112 wait complete. 2024-02-16T19:41:42.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.464+0000 7f5d0c30a640 1 Processor -- start 2024-02-16T19:41:42.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.464+0000 7f5d0c30a640 1 -- start start 2024-02-16T19:41:42.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.464+0000 7f5d0c30a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d040fe5b0 0x7f5d04195290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:42.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.464+0000 7f5d0c30a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d04104590 0x7f5d041957d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:42.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.464+0000 7f5d0c30a640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d04195e60 con 0x7f5d04104590 2024-02-16T19:41:42.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.464+0000 7f5d0c30a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d04199c30 con 0x7f5d040fe5b0 2024-02-16T19:41:42.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.464+0000 7f5d098a5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d04104590 0x7f5d041957d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:42.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.465+0000 7f5d098a5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d04104590 0x7f5d041957d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45696/0 (socket says 172.21.15.22:45696) 2024-02-16T19:41:42.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.465+0000 7f5d098a5640 1 -- 172.21.15.22:0/1495460831 learned_addr learned my addr 172.21.15.22:0/1495460831 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:41:42.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.465+0000 7f5d0a0a6640 1 --2- 172.21.15.22:0/1495460831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d040fe5b0 0x7f5d04195290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:42.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.465+0000 7f5d098a5640 1 -- 172.21.15.22:0/1495460831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d040fe5b0 msgr2=0x7f5d04195290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:42.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.465+0000 7f5d098a5640 1 --2- 172.21.15.22:0/1495460831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d040fe5b0 0x7f5d04195290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:42.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.465+0000 7f5d098a5640 1 -- 172.21.15.22:0/1495460831 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cfc009560 con 0x7f5d04104590 2024-02-16T19:41:42.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.465+0000 7f5d098a5640 1 --2- 172.21.15.22:0/1495460831 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d04104590 0x7f5d041957d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f5cf400d6f0 tx=0x7f5cf400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:42.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.465+0000 7f5cfb7fe640 1 -- 172.21.15.22:0/1495460831 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5cf4009a30 con 0x7f5d04104590 2024-02-16T19:41:42.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.465+0000 7f5d0c30a640 1 -- 172.21.15.22:0/1495460831 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d04199f10 con 0x7f5d04104590 2024-02-16T19:41:42.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.465+0000 7f5cfb7fe640 1 -- 172.21.15.22:0/1495460831 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5cf40054e0 con 0x7f5d04104590 2024-02-16T19:41:42.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.465+0000 7f5cfb7fe640 1 -- 172.21.15.22:0/1495460831 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5cf4010460 con 0x7f5d04104590 2024-02-16T19:41:42.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.465+0000 7f5d0c30a640 1 -- 172.21.15.22:0/1495460831 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d0419a4f0 con 0x7f5d04104590 2024-02-16T19:41:42.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.466+0000 7f5d0c30a640 1 -- 172.21.15.22:0/1495460831 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d0404e000 con 0x7f5d04104590 2024-02-16T19:41:42.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.466+0000 7f5cfb7fe640 1 -- 172.21.15.22:0/1495460831 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f5cf4009b90 con 0x7f5d04104590 2024-02-16T19:41:42.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.467+0000 7f5cfb7fe640 1 --2- 172.21.15.22:0/1495460831 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f5cf006d810 0x7f5cf006fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:42.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.467+0000 7f5cfb7fe640 1 -- 172.21.15.22:0/1495460831 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f5cf408d190 con 0x7f5d04104590 2024-02-16T19:41:42.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.467+0000 7f5d0a0a6640 1 --2- 172.21.15.22:0/1495460831 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f5cf006d810 0x7f5cf006fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:42.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.467+0000 7f5d0a0a6640 1 --2- 172.21.15.22:0/1495460831 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f5cf006d810 0x7f5cf006fcc0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc006010 tx=0x7f5cfc005dd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:42.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.472+0000 7f5cfb7fe640 1 -- 172.21.15.22:0/1495460831 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5cf4056870 con 0x7f5d04104590 2024-02-16T19:41:42.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.690+0000 7f5d0c30a640 1 -- 172.21.15.22:0/1495460831 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 27, "format": "json"} v 0) v1 -- 0x7f5d041965e0 con 0x7f5d04104590 2024-02-16T19:41:42.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.691+0000 7f5cfb7fe640 1 -- 172.21.15.22:0/1495460831 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 27, "format": "json"}]=0 dumped fsmap epoch 27 v37) v1 ==== 107+0+4119 (secure 0 0 0) 0x7f5cf4059f30 con 0x7f5d04104590 2024-02-16T19:41:42.692 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:41:42.692 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34384,"name":"cephfs.smithi022.ocukvs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6827/2535680483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":2535680483},{"type":"v1","addr":"172.21.15.22:6827","nonce":2535680483}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17},{"gid":34408,"name":"cephfs.smithi022.btppwm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6829/3126701730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3126701730},{"type":"v1","addr":"172.21.15.22:6829","nonce":3126701730}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T19:26:17.873644+0000","modified":"2024-02-16T19:38:41.797012+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24265},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24265":{"gid":24265,"name":"cephfs.smithi117.orzors","rank":0,"incarnation":25,"state":"up:rejoin","state_seq":185,"addr":"172.21.15.117:6827/3122342028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":3122342028},{"type":"v1","addr":"172.21.15.117:6827","nonce":3122342028}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T19:41:42.693 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 27 2024-02-16T19:41:42.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.693+0000 7f5d0c30a640 1 -- 172.21.15.22:0/1495460831 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f5cf006d810 msgr2=0x7f5cf006fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:42.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.693+0000 7f5d0c30a640 1 --2- 172.21.15.22:0/1495460831 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f5cf006d810 0x7f5cf006fcc0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc006010 tx=0x7f5cfc005dd0 comp rx=0 tx=0).stop 2024-02-16T19:41:42.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.693+0000 7f5d0c30a640 1 -- 172.21.15.22:0/1495460831 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d04104590 msgr2=0x7f5d041957d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:42.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.693+0000 7f5d0c30a640 1 --2- 172.21.15.22:0/1495460831 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d04104590 0x7f5d041957d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f5cf400d6f0 tx=0x7f5cf400dbb0 comp rx=0 tx=0).stop 2024-02-16T19:41:42.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.693+0000 7f5d0c30a640 1 -- 172.21.15.22:0/1495460831 shutdown_connections 2024-02-16T19:41:42.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.693+0000 7f5d0c30a640 1 --2- 172.21.15.22:0/1495460831 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f5cf006d810 0x7f5cf006fcc0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:42.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.693+0000 7f5d0c30a640 1 --2- 172.21.15.22:0/1495460831 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d04104590 0x7f5d041957d0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:42.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.693+0000 7f5d0c30a640 1 --2- 172.21.15.22:0/1495460831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d040fe5b0 0x7f5d04195290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:42.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.693+0000 7f5d0c30a640 1 -- 172.21.15.22:0/1495460831 >> 172.21.15.22:0/1495460831 conn(0x7f5d040743d0 msgr2=0x7f5d04102690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:42.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.693+0000 7f5d0c30a640 1 -- 172.21.15.22:0/1495460831 shutdown_connections 2024-02-16T19:41:42.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.693+0000 7f5d0c30a640 1 -- 172.21.15.22:0/1495460831 wait complete. 2024-02-16T19:41:42.704 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:41:42 smithi022 ceph-mon[88798]: from='client.? 172.21.15.22:0/1227458521' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-02-16T19:41:42.704 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:41:42 smithi022 ceph-mon[88798]: from='client.? 172.21.15.22:0/2886925166' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-02-16T19:41:42.704 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:41:42 smithi022 ceph-mon[88798]: from='client.? 172.21.15.22:0/3631915193' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-02-16T19:41:42.705 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 28 2024-02-16T19:41:42.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.808+0000 7faf78f50640 1 -- 172.21.15.22:0/2696079482 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf74105880 msgr2=0x7faf74105c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:42.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.808+0000 7faf78f50640 1 --2- 172.21.15.22:0/2696079482 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf74105880 0x7faf74105c50 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7faf64009880 tx=0x7faf6402bce0 comp rx=0 tx=0).stop 2024-02-16T19:41:42.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.808+0000 7faf78f50640 1 -- 172.21.15.22:0/2696079482 shutdown_connections 2024-02-16T19:41:42.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.808+0000 7faf78f50640 1 --2- 172.21.15.22:0/2696079482 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf740ff870 0x7faf740ffce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:42.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.808+0000 7faf78f50640 1 --2- 172.21.15.22:0/2696079482 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf74105880 0x7faf74105c50 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:42.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.808+0000 7faf78f50640 1 -- 172.21.15.22:0/2696079482 >> 172.21.15.22:0/2696079482 conn(0x7faf740fb370 msgr2=0x7faf740fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:42.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.808+0000 7faf78f50640 1 -- 172.21.15.22:0/2696079482 shutdown_connections 2024-02-16T19:41:42.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.808+0000 7faf78f50640 1 -- 172.21.15.22:0/2696079482 wait complete. 2024-02-16T19:41:42.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.809+0000 7faf78f50640 1 Processor -- start 2024-02-16T19:41:42.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.809+0000 7faf78f50640 1 -- start start 2024-02-16T19:41:42.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.809+0000 7faf78f50640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf740ff870 0x7faf74195540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:42.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.809+0000 7faf78f50640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf74105880 0x7faf74195a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:42.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.809+0000 7faf78f50640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf74196110 con 0x7faf74105880 2024-02-16T19:41:42.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.809+0000 7faf78f50640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf74199ee0 con 0x7faf740ff870 2024-02-16T19:41:42.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.809+0000 7faf71d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf74105880 0x7faf74195a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:42.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.809+0000 7faf71d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf74105880 0x7faf74195a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45716/0 (socket says 172.21.15.22:45716) 2024-02-16T19:41:42.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.809+0000 7faf71d9b640 1 -- 172.21.15.22:0/796267688 learned_addr learned my addr 172.21.15.22:0/796267688 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:41:42.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.809+0000 7faf7259c640 1 --2- 172.21.15.22:0/796267688 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf740ff870 0x7faf74195540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:42.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.809+0000 7faf71d9b640 1 -- 172.21.15.22:0/796267688 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf740ff870 msgr2=0x7faf74195540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:42.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.809+0000 7faf71d9b640 1 --2- 172.21.15.22:0/796267688 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf740ff870 0x7faf74195540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:42.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.809+0000 7faf71d9b640 1 -- 172.21.15.22:0/796267688 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf64009560 con 0x7faf74105880 2024-02-16T19:41:42.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.809+0000 7faf71d9b640 1 --2- 172.21.15.22:0/796267688 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf74105880 0x7faf74195a80 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7faf6800c8f0 tx=0x7faf6800cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:42.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.810+0000 7faf637fe640 1 -- 172.21.15.22:0/796267688 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf68007b80 con 0x7faf74105880 2024-02-16T19:41:42.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.810+0000 7faf637fe640 1 -- 172.21.15.22:0/796267688 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faf680054e0 con 0x7faf74105880 2024-02-16T19:41:42.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.810+0000 7faf637fe640 1 -- 172.21.15.22:0/796267688 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf6800f450 con 0x7faf74105880 2024-02-16T19:41:42.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.810+0000 7faf78f50640 1 -- 172.21.15.22:0/796267688 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf7419a1c0 con 0x7faf74105880 2024-02-16T19:41:42.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.810+0000 7faf78f50640 1 -- 172.21.15.22:0/796267688 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf7419a7a0 con 0x7faf74105880 2024-02-16T19:41:42.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.811+0000 7faf78f50640 1 -- 172.21.15.22:0/796267688 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf7404e000 con 0x7faf74105880 2024-02-16T19:41:42.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.811+0000 7faf637fe640 1 -- 172.21.15.22:0/796267688 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7faf68004f60 con 0x7faf74105880 2024-02-16T19:41:42.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.811+0000 7faf637fe640 1 --2- 172.21.15.22:0/796267688 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7faf5c06d810 0x7faf5c06fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:42.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.812+0000 7faf637fe640 1 -- 172.21.15.22:0/796267688 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6962+0+0 (secure 0 0 0) 0x7faf6808d4d0 con 0x7faf74105880 2024-02-16T19:41:42.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.812+0000 7faf7259c640 1 --2- 172.21.15.22:0/796267688 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7faf5c06d810 0x7faf5c06fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:42.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.812+0000 7faf7259c640 1 --2- 172.21.15.22:0/796267688 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7faf5c06d810 0x7faf5c06fcc0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7faf64036040 tx=0x7faf6402cf90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:42.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:42.816+0000 7faf637fe640 1 -- 172.21.15.22:0/796267688 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7faf68056b00 con 0x7faf74105880 2024-02-16T19:41:42.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:41:42 smithi117 ceph-mon[67870]: from='client.? 172.21.15.22:0/1227458521' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-02-16T19:41:42.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:41:42 smithi117 ceph-mon[67870]: from='client.? 172.21.15.22:0/2886925166' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-02-16T19:41:42.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:41:42 smithi117 ceph-mon[67870]: from='client.? 172.21.15.22:0/3631915193' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-02-16T19:41:43.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.034+0000 7faf78f50640 1 -- 172.21.15.22:0/796267688 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 28, "format": "json"} v 0) v1 -- 0x7faf74066860 con 0x7faf74105880 2024-02-16T19:41:43.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.034+0000 7faf637fe640 1 -- 172.21.15.22:0/796267688 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 28, "format": "json"}]=0 dumped fsmap epoch 28 v37) v1 ==== 107+0+4119 (secure 0 0 0) 0x7faf6805a1c0 con 0x7faf74105880 2024-02-16T19:41:43.036 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:41:43.036 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34384,"name":"cephfs.smithi022.ocukvs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6827/2535680483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":2535680483},{"type":"v1","addr":"172.21.15.22:6827","nonce":2535680483}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17},{"gid":34408,"name":"cephfs.smithi022.btppwm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6829/3126701730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3126701730},{"type":"v1","addr":"172.21.15.22:6829","nonce":3126701730}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T19:26:17.873644+0000","modified":"2024-02-16T19:38:42.898962+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24265},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24265":{"gid":24265,"name":"cephfs.smithi117.orzors","rank":0,"incarnation":25,"state":"up:active","state_seq":186,"addr":"172.21.15.117:6827/3122342028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":3122342028},{"type":"v1","addr":"172.21.15.117:6827","nonce":3122342028}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T19:41:43.036 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 28 2024-02-16T19:41:43.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.036+0000 7faf78f50640 1 -- 172.21.15.22:0/796267688 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7faf5c06d810 msgr2=0x7faf5c06fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:43.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.036+0000 7faf78f50640 1 --2- 172.21.15.22:0/796267688 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7faf5c06d810 0x7faf5c06fcc0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7faf64036040 tx=0x7faf6402cf90 comp rx=0 tx=0).stop 2024-02-16T19:41:43.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.036+0000 7faf78f50640 1 -- 172.21.15.22:0/796267688 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf74105880 msgr2=0x7faf74195a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:43.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.036+0000 7faf78f50640 1 --2- 172.21.15.22:0/796267688 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf74105880 0x7faf74195a80 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7faf6800c8f0 tx=0x7faf6800cdb0 comp rx=0 tx=0).stop 2024-02-16T19:41:43.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.036+0000 7faf78f50640 1 -- 172.21.15.22:0/796267688 shutdown_connections 2024-02-16T19:41:43.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.036+0000 7faf78f50640 1 --2- 172.21.15.22:0/796267688 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7faf5c06d810 0x7faf5c06fcc0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:43.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.036+0000 7faf78f50640 1 --2- 172.21.15.22:0/796267688 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf74105880 0x7faf74195a80 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:43.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.036+0000 7faf78f50640 1 --2- 172.21.15.22:0/796267688 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf740ff870 0x7faf74195540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:43.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.036+0000 7faf78f50640 1 -- 172.21.15.22:0/796267688 >> 172.21.15.22:0/796267688 conn(0x7faf740fb370 msgr2=0x7faf740fccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:43.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.037+0000 7faf78f50640 1 -- 172.21.15.22:0/796267688 shutdown_connections 2024-02-16T19:41:43.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.037+0000 7faf78f50640 1 -- 172.21.15.22:0/796267688 wait complete. 2024-02-16T19:41:43.049 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 29 2024-02-16T19:41:43.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.151+0000 7fc7cd6a4640 1 -- 172.21.15.22:0/1048883717 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc7c8104590 msgr2=0x7fc7c8104960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:43.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.151+0000 7fc7cd6a4640 1 --2- 172.21.15.22:0/1048883717 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc7c8104590 0x7fc7c8104960 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc7b8009880 tx=0x7fc7b802bce0 comp rx=0 tx=0).stop 2024-02-16T19:41:43.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.151+0000 7fc7cd6a4640 1 -- 172.21.15.22:0/1048883717 shutdown_connections 2024-02-16T19:41:43.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.151+0000 7fc7cd6a4640 1 --2- 172.21.15.22:0/1048883717 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7c80fe5b0 0x7fc7c80fea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:43.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.151+0000 7fc7cd6a4640 1 --2- 172.21.15.22:0/1048883717 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc7c8104590 0x7fc7c8104960 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:43.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.151+0000 7fc7cd6a4640 1 -- 172.21.15.22:0/1048883717 >> 172.21.15.22:0/1048883717 conn(0x7fc7c80743d0 msgr2=0x7fc7c80747d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:43.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.151+0000 7fc7cd6a4640 1 -- 172.21.15.22:0/1048883717 shutdown_connections 2024-02-16T19:41:43.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.151+0000 7fc7cd6a4640 1 -- 172.21.15.22:0/1048883717 wait complete. 2024-02-16T19:41:43.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.152+0000 7fc7cd6a4640 1 Processor -- start 2024-02-16T19:41:43.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.152+0000 7fc7cd6a4640 1 -- start start 2024-02-16T19:41:43.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.152+0000 7fc7cd6a4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7c80fe5b0 0x7fc7c8195330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:43.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.152+0000 7fc7cd6a4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc7c8104590 0x7fc7c8195870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:43.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.152+0000 7fc7cd6a4640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c8195f00 con 0x7fc7c8104590 2024-02-16T19:41:43.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.152+0000 7fc7cd6a4640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c8199cd0 con 0x7fc7c80fe5b0 2024-02-16T19:41:43.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.152+0000 7fc7c67fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc7c8104590 0x7fc7c8195870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:43.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.152+0000 7fc7c67fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc7c8104590 0x7fc7c8195870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45730/0 (socket says 172.21.15.22:45730) 2024-02-16T19:41:43.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.152+0000 7fc7c67fc640 1 -- 172.21.15.22:0/2423351878 learned_addr learned my addr 172.21.15.22:0/2423351878 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:41:43.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.153+0000 7fc7c6ffd640 1 --2- 172.21.15.22:0/2423351878 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7c80fe5b0 0x7fc7c8195330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:43.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.153+0000 7fc7c67fc640 1 -- 172.21.15.22:0/2423351878 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7c80fe5b0 msgr2=0x7fc7c8195330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:43.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.153+0000 7fc7c67fc640 1 --2- 172.21.15.22:0/2423351878 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7c80fe5b0 0x7fc7c8195330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:43.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.153+0000 7fc7c67fc640 1 -- 172.21.15.22:0/2423351878 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7b8009560 con 0x7fc7c8104590 2024-02-16T19:41:43.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.153+0000 7fc7c67fc640 1 --2- 172.21.15.22:0/2423351878 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc7c8104590 0x7fc7c8195870 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc7b000e970 tx=0x7fc7b000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:43.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.153+0000 7fc7bffff640 1 -- 172.21.15.22:0/2423351878 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc7b000cc90 con 0x7fc7c8104590 2024-02-16T19:41:43.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.153+0000 7fc7cd6a4640 1 -- 172.21.15.22:0/2423351878 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7c8199fb0 con 0x7fc7c8104590 2024-02-16T19:41:43.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.153+0000 7fc7bffff640 1 -- 172.21.15.22:0/2423351878 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc7b000cdf0 con 0x7fc7c8104590 2024-02-16T19:41:43.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.153+0000 7fc7bffff640 1 -- 172.21.15.22:0/2423351878 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc7b0010430 con 0x7fc7c8104590 2024-02-16T19:41:43.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.153+0000 7fc7cd6a4640 1 -- 172.21.15.22:0/2423351878 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7c819a510 con 0x7fc7c8104590 2024-02-16T19:41:43.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.154+0000 7fc7cd6a4640 1 -- 172.21.15.22:0/2423351878 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7c804e000 con 0x7fc7c8104590 2024-02-16T19:41:43.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.154+0000 7fc7bffff640 1 -- 172.21.15.22:0/2423351878 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fc7b00046c0 con 0x7fc7c8104590 2024-02-16T19:41:43.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.155+0000 7fc7bffff640 1 --2- 172.21.15.22:0/2423351878 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fc7b406d810 0x7fc7b406fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:43.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.155+0000 7fc7bffff640 1 -- 172.21.15.22:0/2423351878 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6962+0+0 (secure 0 0 0) 0x7fc7b0014070 con 0x7fc7c8104590 2024-02-16T19:41:43.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.155+0000 7fc7c6ffd640 1 --2- 172.21.15.22:0/2423351878 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fc7b406d810 0x7fc7b406fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:43.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.155+0000 7fc7c6ffd640 1 --2- 172.21.15.22:0/2423351878 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fc7b406d810 0x7fc7b406fcc0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc7b8006010 tx=0x7fc7b8005dd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:43.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.159+0000 7fc7bffff640 1 -- 172.21.15.22:0/2423351878 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc7b0057230 con 0x7fc7c8104590 2024-02-16T19:41:43.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.379+0000 7fc7cd6a4640 1 -- 172.21.15.22:0/2423351878 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 29, "format": "json"} v 0) v1 -- 0x7fc7c8196680 con 0x7fc7c8104590 2024-02-16T19:41:43.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.380+0000 7fc7bffff640 1 -- 172.21.15.22:0/2423351878 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 29, "format": "json"}]=0 dumped fsmap epoch 29 v37) v1 ==== 107+0+4935 (secure 0 0 0) 0x7fc7b005a8f0 con 0x7fc7c8104590 2024-02-16T19:41:43.381 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:41:43.382 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34384,"name":"cephfs.smithi022.ocukvs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6827/2535680483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":2535680483},{"type":"v1","addr":"172.21.15.22:6827","nonce":2535680483}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17},{"gid":34408,"name":"cephfs.smithi022.btppwm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6829/3126701730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3126701730},{"type":"v1","addr":"172.21.15.22:6829","nonce":3126701730}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":44299,"name":"cephfs.smithi117.ibmppn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6825/2389644474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2389644474},{"type":"v1","addr":"172.21.15.117:6825","nonce":2389644474}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T19:26:17.873644+0000","modified":"2024-02-16T19:38:42.898962+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24265},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24265":{"gid":24265,"name":"cephfs.smithi117.orzors","rank":0,"incarnation":25,"state":"up:active","state_seq":186,"addr":"172.21.15.117:6827/3122342028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":3122342028},{"type":"v1","addr":"172.21.15.117:6827","nonce":3122342028}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T19:41:43.382 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 29 2024-02-16T19:41:43.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.382+0000 7fc7cd6a4640 1 -- 172.21.15.22:0/2423351878 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fc7b406d810 msgr2=0x7fc7b406fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:43.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.382+0000 7fc7cd6a4640 1 --2- 172.21.15.22:0/2423351878 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fc7b406d810 0x7fc7b406fcc0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc7b8006010 tx=0x7fc7b8005dd0 comp rx=0 tx=0).stop 2024-02-16T19:41:43.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.382+0000 7fc7cd6a4640 1 -- 172.21.15.22:0/2423351878 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc7c8104590 msgr2=0x7fc7c8195870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:43.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.382+0000 7fc7cd6a4640 1 --2- 172.21.15.22:0/2423351878 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc7c8104590 0x7fc7c8195870 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc7b000e970 tx=0x7fc7b000ed70 comp rx=0 tx=0).stop 2024-02-16T19:41:43.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.382+0000 7fc7cd6a4640 1 -- 172.21.15.22:0/2423351878 shutdown_connections 2024-02-16T19:41:43.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.382+0000 7fc7cd6a4640 1 --2- 172.21.15.22:0/2423351878 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fc7b406d810 0x7fc7b406fcc0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:43.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.382+0000 7fc7cd6a4640 1 --2- 172.21.15.22:0/2423351878 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc7c8104590 0x7fc7c8195870 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:43.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.382+0000 7fc7cd6a4640 1 --2- 172.21.15.22:0/2423351878 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7c80fe5b0 0x7fc7c8195330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:43.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.382+0000 7fc7cd6a4640 1 -- 172.21.15.22:0/2423351878 >> 172.21.15.22:0/2423351878 conn(0x7fc7c80743d0 msgr2=0x7fc7c8067390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:43.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.383+0000 7fc7cd6a4640 1 -- 172.21.15.22:0/2423351878 shutdown_connections 2024-02-16T19:41:43.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.383+0000 7fc7cd6a4640 1 -- 172.21.15.22:0/2423351878 wait complete. 2024-02-16T19:41:43.395 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 30 2024-02-16T19:41:43.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.496+0000 7fe1eab7e640 1 -- 172.21.15.22:0/227357853 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe1e40fe760 msgr2=0x7fe1e40febd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:43.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.496+0000 7fe1eab7e640 1 --2- 172.21.15.22:0/227357853 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe1e40fe760 0x7fe1e40febd0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8009880 tx=0x7fe1d802bce0 comp rx=0 tx=0).stop 2024-02-16T19:41:43.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.497+0000 7fe1eab7e640 1 -- 172.21.15.22:0/227357853 shutdown_connections 2024-02-16T19:41:43.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.497+0000 7fe1eab7e640 1 --2- 172.21.15.22:0/227357853 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe1e40fe760 0x7fe1e40febd0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:43.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.497+0000 7fe1eab7e640 1 --2- 172.21.15.22:0/227357853 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1e4104770 0x7fe1e4104b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:43.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.497+0000 7fe1eab7e640 1 -- 172.21.15.22:0/227357853 >> 172.21.15.22:0/227357853 conn(0x7fe1e40fa260 msgr2=0x7fe1e40fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:43.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.497+0000 7fe1eab7e640 1 -- 172.21.15.22:0/227357853 shutdown_connections 2024-02-16T19:41:43.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.497+0000 7fe1eab7e640 1 -- 172.21.15.22:0/227357853 wait complete. 2024-02-16T19:41:43.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.497+0000 7fe1eab7e640 1 Processor -- start 2024-02-16T19:41:43.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.497+0000 7fe1eab7e640 1 -- start start 2024-02-16T19:41:43.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.497+0000 7fe1eab7e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe1e40fe760 0x7fe1e4195610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:43.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.497+0000 7fe1eab7e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1e4104770 0x7fe1e4195b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:43.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.497+0000 7fe1eab7e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e41961e0 con 0x7fe1e40fe760 2024-02-16T19:41:43.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.497+0000 7fe1eab7e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e4199fb0 con 0x7fe1e4104770 2024-02-16T19:41:43.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.497+0000 7fe1e891a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe1e40fe760 0x7fe1e4195610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:43.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.497+0000 7fe1e891a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe1e40fe760 0x7fe1e4195610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45746/0 (socket says 172.21.15.22:45746) 2024-02-16T19:41:43.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.497+0000 7fe1e891a640 1 -- 172.21.15.22:0/688908434 learned_addr learned my addr 172.21.15.22:0/688908434 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:41:43.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.498+0000 7fe1e3fff640 1 --2- 172.21.15.22:0/688908434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1e4104770 0x7fe1e4195b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:43.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.498+0000 7fe1e891a640 1 -- 172.21.15.22:0/688908434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1e4104770 msgr2=0x7fe1e4195b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:43.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.498+0000 7fe1e891a640 1 --2- 172.21.15.22:0/688908434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1e4104770 0x7fe1e4195b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:43.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.498+0000 7fe1e891a640 1 -- 172.21.15.22:0/688908434 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1d8009560 con 0x7fe1e40fe760 2024-02-16T19:41:43.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.498+0000 7fe1e3fff640 1 --2- 172.21.15.22:0/688908434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1e4104770 0x7fe1e4195b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-16T19:41:43.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.498+0000 7fe1e891a640 1 --2- 172.21.15.22:0/688908434 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe1e40fe760 0x7fe1e4195610 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe1d400b3d0 tx=0x7fe1d400b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:43.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.498+0000 7fe1e1ffb640 1 -- 172.21.15.22:0/688908434 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1d4011800 con 0x7fe1e40fe760 2024-02-16T19:41:43.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.498+0000 7fe1e1ffb640 1 -- 172.21.15.22:0/688908434 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe1d4011e20 con 0x7fe1e40fe760 2024-02-16T19:41:43.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.498+0000 7fe1eab7e640 1 -- 172.21.15.22:0/688908434 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1e419a290 con 0x7fe1e40fe760 2024-02-16T19:41:43.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.498+0000 7fe1e1ffb640 1 -- 172.21.15.22:0/688908434 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1d400f510 con 0x7fe1e40fe760 2024-02-16T19:41:43.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.498+0000 7fe1eab7e640 1 -- 172.21.15.22:0/688908434 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1e419a870 con 0x7fe1e40fe760 2024-02-16T19:41:43.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.499+0000 7fe1eab7e640 1 -- 172.21.15.22:0/688908434 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1e404e000 con 0x7fe1e40fe760 2024-02-16T19:41:43.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.500+0000 7fe1e1ffb640 1 -- 172.21.15.22:0/688908434 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fe1d4010380 con 0x7fe1e40fe760 2024-02-16T19:41:43.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.500+0000 7fe1e1ffb640 1 --2- 172.21.15.22:0/688908434 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fe1cc06d810 0x7fe1cc06fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:43.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.500+0000 7fe1e1ffb640 1 -- 172.21.15.22:0/688908434 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6962+0+0 (secure 0 0 0) 0x7fe1d408d300 con 0x7fe1e40fe760 2024-02-16T19:41:43.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.500+0000 7fe1e3fff640 1 --2- 172.21.15.22:0/688908434 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fe1cc06d810 0x7fe1cc06fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:43.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.501+0000 7fe1e3fff640 1 --2- 172.21.15.22:0/688908434 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fe1cc06d810 0x7fe1cc06fcc0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8005ec0 tx=0x7fe1d8005fd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:43.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.506+0000 7fe1e1ffb640 1 -- 172.21.15.22:0/688908434 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe1d40569b0 con 0x7fe1e40fe760 2024-02-16T19:41:43.714 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:41:43 smithi022 ceph-mon[88798]: pgmap v299: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:41:43.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:41:43 smithi022 ceph-mon[88798]: from='client.? 172.21.15.22:0/1495460831' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-02-16T19:41:43.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:41:43 smithi022 ceph-mon[88798]: from='client.? 172.21.15.22:0/796267688' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-02-16T19:41:43.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:41:43 smithi022 ceph-mon[88798]: from='client.? 172.21.15.22:0/2423351878' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-02-16T19:41:43.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.725+0000 7fe1eab7e640 1 -- 172.21.15.22:0/688908434 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 30, "format": "json"} v 0) v1 -- 0x7fe1e4066860 con 0x7fe1e40fe760 2024-02-16T19:41:43.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.725+0000 7fe1e1ffb640 1 -- 172.21.15.22:0/688908434 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 30, "format": "json"}]=0 dumped fsmap epoch 30 v37) v1 ==== 107+0+4140 (secure 0 0 0) 0x7fe1d405a070 con 0x7fe1e40fe760 2024-02-16T19:41:43.727 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:41:43.727 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34384,"name":"cephfs.smithi022.ocukvs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6827/2535680483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":2535680483},{"type":"v1","addr":"172.21.15.22:6827","nonce":2535680483}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17},{"gid":34408,"name":"cephfs.smithi022.btppwm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6829/3126701730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3126701730},{"type":"v1","addr":"172.21.15.22:6829","nonce":3126701730}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":44299,"name":"cephfs.smithi117.ibmppn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6825/2389644474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2389644474},{"type":"v1","addr":"172.21.15.117:6825","nonce":2389644474}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T19:26:17.873644+0000","modified":"2024-02-16T19:38:57.804520+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T19:41:43.727 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 30 2024-02-16T19:41:43.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.728+0000 7fe1eab7e640 1 -- 172.21.15.22:0/688908434 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fe1cc06d810 msgr2=0x7fe1cc06fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:43.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.728+0000 7fe1eab7e640 1 --2- 172.21.15.22:0/688908434 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fe1cc06d810 0x7fe1cc06fcc0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8005ec0 tx=0x7fe1d8005fd0 comp rx=0 tx=0).stop 2024-02-16T19:41:43.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.728+0000 7fe1eab7e640 1 -- 172.21.15.22:0/688908434 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe1e40fe760 msgr2=0x7fe1e4195610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:43.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.728+0000 7fe1eab7e640 1 --2- 172.21.15.22:0/688908434 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe1e40fe760 0x7fe1e4195610 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe1d400b3d0 tx=0x7fe1d400b890 comp rx=0 tx=0).stop 2024-02-16T19:41:43.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.728+0000 7fe1eab7e640 1 -- 172.21.15.22:0/688908434 shutdown_connections 2024-02-16T19:41:43.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.728+0000 7fe1eab7e640 1 --2- 172.21.15.22:0/688908434 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fe1cc06d810 0x7fe1cc06fcc0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:43.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.728+0000 7fe1eab7e640 1 --2- 172.21.15.22:0/688908434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1e4104770 0x7fe1e4195b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:43.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.728+0000 7fe1eab7e640 1 --2- 172.21.15.22:0/688908434 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe1e40fe760 0x7fe1e4195610 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:43.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.728+0000 7fe1eab7e640 1 -- 172.21.15.22:0/688908434 >> 172.21.15.22:0/688908434 conn(0x7fe1e40fa260 msgr2=0x7fe1e4107c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:43.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.728+0000 7fe1eab7e640 1 -- 172.21.15.22:0/688908434 shutdown_connections 2024-02-16T19:41:43.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.728+0000 7fe1eab7e640 1 -- 172.21.15.22:0/688908434 wait complete. 2024-02-16T19:41:43.740 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 31 2024-02-16T19:41:43.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.842+0000 7f8ea6fae640 1 -- 172.21.15.22:0/2140407475 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8ea0104590 msgr2=0x7f8ea0104960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:43.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.842+0000 7f8ea6fae640 1 --2- 172.21.15.22:0/2140407475 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8ea0104590 0x7f8ea0104960 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8e94009880 tx=0x7f8e9402bce0 comp rx=0 tx=0).stop 2024-02-16T19:41:43.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.842+0000 7f8ea6fae640 1 -- 172.21.15.22:0/2140407475 shutdown_connections 2024-02-16T19:41:43.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.842+0000 7f8ea6fae640 1 --2- 172.21.15.22:0/2140407475 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ea00fe5b0 0x7f8ea00fea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:43.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.842+0000 7f8ea6fae640 1 --2- 172.21.15.22:0/2140407475 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8ea0104590 0x7f8ea0104960 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:43.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.842+0000 7f8ea6fae640 1 -- 172.21.15.22:0/2140407475 >> 172.21.15.22:0/2140407475 conn(0x7f8ea00743d0 msgr2=0x7f8ea00747d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:43.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.842+0000 7f8ea6fae640 1 -- 172.21.15.22:0/2140407475 shutdown_connections 2024-02-16T19:41:43.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.842+0000 7f8ea6fae640 1 -- 172.21.15.22:0/2140407475 wait complete. 2024-02-16T19:41:43.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.842+0000 7f8ea6fae640 1 Processor -- start 2024-02-16T19:41:43.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.842+0000 7f8ea6fae640 1 -- start start 2024-02-16T19:41:43.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.842+0000 7f8ea6fae640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8ea00fe5b0 0x7f8ea0195330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:43.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.842+0000 7f8ea6fae640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ea0104590 0x7f8ea0195870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:43.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.843+0000 7f8ea6fae640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea0195f00 con 0x7f8ea00fe5b0 2024-02-16T19:41:43.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.843+0000 7f8ea6fae640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea0199cd0 con 0x7f8ea0104590 2024-02-16T19:41:43.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.843+0000 7f8ea4d4a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8ea00fe5b0 0x7f8ea0195330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:43.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.843+0000 7f8ea4d4a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8ea00fe5b0 0x7f8ea0195330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45774/0 (socket says 172.21.15.22:45774) 2024-02-16T19:41:43.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.843+0000 7f8ea4d4a640 1 -- 172.21.15.22:0/860277867 learned_addr learned my addr 172.21.15.22:0/860277867 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:41:43.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.843+0000 7f8e9ffff640 1 --2- 172.21.15.22:0/860277867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ea0104590 0x7f8ea0195870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:43.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.843+0000 7f8ea4d4a640 1 -- 172.21.15.22:0/860277867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ea0104590 msgr2=0x7f8ea0195870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:43.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.843+0000 7f8ea4d4a640 1 --2- 172.21.15.22:0/860277867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ea0104590 0x7f8ea0195870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:43.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.843+0000 7f8ea4d4a640 1 -- 172.21.15.22:0/860277867 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e94009560 con 0x7f8ea00fe5b0 2024-02-16T19:41:43.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.843+0000 7f8ea4d4a640 1 --2- 172.21.15.22:0/860277867 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8ea00fe5b0 0x7f8ea0195330 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8e94009230 tx=0x7f8e94005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:43.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.843+0000 7f8e9dffb640 1 -- 172.21.15.22:0/860277867 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e940044a0 con 0x7f8ea00fe5b0 2024-02-16T19:41:43.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.843+0000 7f8ea6fae640 1 -- 172.21.15.22:0/860277867 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ea0199f50 con 0x7f8ea00fe5b0 2024-02-16T19:41:43.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.843+0000 7f8e9dffb640 1 -- 172.21.15.22:0/860277867 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8e94003ab0 con 0x7f8ea00fe5b0 2024-02-16T19:41:43.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.843+0000 7f8e9dffb640 1 -- 172.21.15.22:0/860277867 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e940055f0 con 0x7f8ea00fe5b0 2024-02-16T19:41:43.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.843+0000 7f8ea6fae640 1 -- 172.21.15.22:0/860277867 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ea019a450 con 0x7f8ea00fe5b0 2024-02-16T19:41:43.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.844+0000 7f8ea6fae640 1 -- 172.21.15.22:0/860277867 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ea004e000 con 0x7f8ea00fe5b0 2024-02-16T19:41:43.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.845+0000 7f8e9dffb640 1 -- 172.21.15.22:0/860277867 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f8e94004e60 con 0x7f8ea00fe5b0 2024-02-16T19:41:43.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.845+0000 7f8e9dffb640 1 --2- 172.21.15.22:0/860277867 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f8e9006d810 0x7f8e9006fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:43.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.845+0000 7f8e9dffb640 1 -- 172.21.15.22:0/860277867 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f8e940afae0 con 0x7f8ea00fe5b0 2024-02-16T19:41:43.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.845+0000 7f8e9ffff640 1 --2- 172.21.15.22:0/860277867 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f8e9006d810 0x7f8e9006fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:43.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.846+0000 7f8e9ffff640 1 --2- 172.21.15.22:0/860277867 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f8e9006d810 0x7f8e9006fcc0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8ea0196940 tx=0x7f8e8c005f90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:43.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:43.850+0000 7f8e9dffb640 1 -- 172.21.15.22:0/860277867 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8e94079110 con 0x7f8ea00fe5b0 2024-02-16T19:41:43.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:41:43 smithi117 ceph-mon[67870]: pgmap v299: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:41:43.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:41:43 smithi117 ceph-mon[67870]: from='client.? 172.21.15.22:0/1495460831' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-02-16T19:41:43.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:41:43 smithi117 ceph-mon[67870]: from='client.? 172.21.15.22:0/796267688' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-02-16T19:41:43.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:41:43 smithi117 ceph-mon[67870]: from='client.? 172.21.15.22:0/2423351878' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-02-16T19:41:44.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.070+0000 7f8ea6fae640 1 -- 172.21.15.22:0/860277867 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 31, "format": "json"} v 0) v1 -- 0x7f8ea0196680 con 0x7f8ea00fe5b0 2024-02-16T19:41:44.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.070+0000 7f8e9dffb640 1 -- 172.21.15.22:0/860277867 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 31, "format": "json"}]=0 dumped fsmap epoch 31 v37) v1 ==== 107+0+4185 (secure 0 0 0) 0x7f8e9407c7d0 con 0x7f8ea00fe5b0 2024-02-16T19:41:44.072 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:41:44.072 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34408,"name":"cephfs.smithi022.btppwm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6829/3126701730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3126701730},{"type":"v1","addr":"172.21.15.22:6829","nonce":3126701730}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":44299,"name":"cephfs.smithi117.ibmppn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6825/2389644474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2389644474},{"type":"v1","addr":"172.21.15.117:6825","nonce":2389644474}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T19:26:17.873644+0000","modified":"2024-02-16T19:38:57.810288+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34384},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34384":{"gid":34384,"name":"cephfs.smithi022.ocukvs","rank":0,"incarnation":31,"state":"up:replay","state_seq":1,"addr":"172.21.15.22:6827/2535680483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":2535680483},{"type":"v1","addr":"172.21.15.22:6827","nonce":2535680483}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T19:41:44.072 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 31 2024-02-16T19:41:44.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.072+0000 7f8ea6fae640 1 -- 172.21.15.22:0/860277867 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f8e9006d810 msgr2=0x7f8e9006fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:44.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.072+0000 7f8ea6fae640 1 --2- 172.21.15.22:0/860277867 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f8e9006d810 0x7f8e9006fcc0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8ea0196940 tx=0x7f8e8c005f90 comp rx=0 tx=0).stop 2024-02-16T19:41:44.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.072+0000 7f8ea6fae640 1 -- 172.21.15.22:0/860277867 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8ea00fe5b0 msgr2=0x7f8ea0195330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:44.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.072+0000 7f8ea6fae640 1 --2- 172.21.15.22:0/860277867 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8ea00fe5b0 0x7f8ea0195330 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8e94009230 tx=0x7f8e94005e50 comp rx=0 tx=0).stop 2024-02-16T19:41:44.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.072+0000 7f8ea6fae640 1 -- 172.21.15.22:0/860277867 shutdown_connections 2024-02-16T19:41:44.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.072+0000 7f8ea6fae640 1 --2- 172.21.15.22:0/860277867 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f8e9006d810 0x7f8e9006fcc0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:44.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.072+0000 7f8ea6fae640 1 --2- 172.21.15.22:0/860277867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ea0104590 0x7f8ea0195870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:44.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.072+0000 7f8ea6fae640 1 --2- 172.21.15.22:0/860277867 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8ea00fe5b0 0x7f8ea0195330 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:44.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.072+0000 7f8ea6fae640 1 -- 172.21.15.22:0/860277867 >> 172.21.15.22:0/860277867 conn(0x7f8ea00743d0 msgr2=0x7f8ea0067330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:44.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.073+0000 7f8ea6fae640 1 -- 172.21.15.22:0/860277867 shutdown_connections 2024-02-16T19:41:44.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.073+0000 7f8ea6fae640 1 -- 172.21.15.22:0/860277867 wait complete. 2024-02-16T19:41:44.086 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 32 2024-02-16T19:41:44.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.189+0000 7f7f4f48b640 1 -- 172.21.15.22:0/2043025022 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f480fe760 msgr2=0x7f7f480febd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:44.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.189+0000 7f7f4f48b640 1 --2- 172.21.15.22:0/2043025022 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f480fe760 0x7f7f480febd0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7f38009880 tx=0x7f7f3802bce0 comp rx=0 tx=0).stop 2024-02-16T19:41:44.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.189+0000 7f7f4f48b640 1 -- 172.21.15.22:0/2043025022 shutdown_connections 2024-02-16T19:41:44.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.189+0000 7f7f4f48b640 1 --2- 172.21.15.22:0/2043025022 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f480fe760 0x7f7f480febd0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:44.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.189+0000 7f7f4f48b640 1 --2- 172.21.15.22:0/2043025022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f48104770 0x7f7f48104b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:44.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.189+0000 7f7f4f48b640 1 -- 172.21.15.22:0/2043025022 >> 172.21.15.22:0/2043025022 conn(0x7f7f480fa260 msgr2=0x7f7f480fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:44.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.189+0000 7f7f4f48b640 1 -- 172.21.15.22:0/2043025022 shutdown_connections 2024-02-16T19:41:44.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.189+0000 7f7f4f48b640 1 -- 172.21.15.22:0/2043025022 wait complete. 2024-02-16T19:41:44.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.189+0000 7f7f4f48b640 1 Processor -- start 2024-02-16T19:41:44.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.189+0000 7f7f4f48b640 1 -- start start 2024-02-16T19:41:44.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.189+0000 7f7f4f48b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f480fe760 0x7f7f48196340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:44.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.189+0000 7f7f4f48b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f48104770 0x7f7f48196880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:44.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.190+0000 7f7f4f48b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f48196e10 con 0x7f7f48104770 2024-02-16T19:41:44.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.190+0000 7f7f4f48b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f48196f50 con 0x7f7f480fe760 2024-02-16T19:41:44.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.190+0000 7f7f4ca26640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f48104770 0x7f7f48196880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:44.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.190+0000 7f7f4ca26640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f48104770 0x7f7f48196880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45782/0 (socket says 172.21.15.22:45782) 2024-02-16T19:41:44.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.190+0000 7f7f4ca26640 1 -- 172.21.15.22:0/3451444046 learned_addr learned my addr 172.21.15.22:0/3451444046 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:41:44.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.190+0000 7f7f4ca26640 1 -- 172.21.15.22:0/3451444046 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f480fe760 msgr2=0x7f7f48196340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:44.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.190+0000 7f7f4d227640 1 --2- 172.21.15.22:0/3451444046 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f480fe760 0x7f7f48196340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:44.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.190+0000 7f7f4ca26640 1 --2- 172.21.15.22:0/3451444046 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f480fe760 0x7f7f48196340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:44.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.190+0000 7f7f4ca26640 1 -- 172.21.15.22:0/3451444046 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f38009560 con 0x7f7f48104770 2024-02-16T19:41:44.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.190+0000 7f7f4ca26640 1 --2- 172.21.15.22:0/3451444046 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f48104770 0x7f7f48196880 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7f38009230 tx=0x7f7f38004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:44.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.190+0000 7f7f3e7fc640 1 -- 172.21.15.22:0/3451444046 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f380035f0 con 0x7f7f48104770 2024-02-16T19:41:44.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.190+0000 7f7f4f48b640 1 -- 172.21.15.22:0/3451444046 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f4819f3b0 con 0x7f7f48104770 2024-02-16T19:41:44.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.190+0000 7f7f3e7fc640 1 -- 172.21.15.22:0/3451444046 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7f38003c10 con 0x7f7f48104770 2024-02-16T19:41:44.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.190+0000 7f7f3e7fc640 1 -- 172.21.15.22:0/3451444046 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f38034440 con 0x7f7f48104770 2024-02-16T19:41:44.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.190+0000 7f7f4f48b640 1 -- 172.21.15.22:0/3451444046 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f4819f7f0 con 0x7f7f48104770 2024-02-16T19:41:44.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.191+0000 7f7f4f48b640 1 -- 172.21.15.22:0/3451444046 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f4804e000 con 0x7f7f48104770 2024-02-16T19:41:44.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.192+0000 7f7f3e7fc640 1 -- 172.21.15.22:0/3451444046 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f7f38004fd0 con 0x7f7f48104770 2024-02-16T19:41:44.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.192+0000 7f7f3e7fc640 1 --2- 172.21.15.22:0/3451444046 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f7f3406d810 0x7f7f3406fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:44.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.192+0000 7f7f3e7fc640 1 -- 172.21.15.22:0/3451444046 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f7f380afb40 con 0x7f7f48104770 2024-02-16T19:41:44.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.192+0000 7f7f4d227640 1 --2- 172.21.15.22:0/3451444046 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f7f3406d810 0x7f7f3406fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:44.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.193+0000 7f7f4d227640 1 --2- 172.21.15.22:0/3451444046 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f7f3406d810 0x7f7f3406fcc0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7f44005fd0 tx=0x7f7f44005ce0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:44.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.197+0000 7f7f3e7fc640 1 -- 172.21.15.22:0/3451444046 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7f38079170 con 0x7f7f48104770 2024-02-16T19:41:44.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.417+0000 7f7f4f48b640 1 -- 172.21.15.22:0/3451444046 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 32, "format": "json"} v 0) v1 -- 0x7f7f4810b9d0 con 0x7f7f48104770 2024-02-16T19:41:44.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.417+0000 7f7f3e7fc640 1 -- 172.21.15.22:0/3451444046 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 32, "format": "json"}]=0 dumped fsmap epoch 32 v37) v1 ==== 107+0+4998 (secure 0 0 0) 0x7f7f3807c830 con 0x7f7f48104770 2024-02-16T19:41:44.420 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:41:44.420 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34408,"name":"cephfs.smithi022.btppwm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6829/3126701730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3126701730},{"type":"v1","addr":"172.21.15.22:6829","nonce":3126701730}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":44299,"name":"cephfs.smithi117.ibmppn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6825/2389644474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2389644474},{"type":"v1","addr":"172.21.15.117:6825","nonce":2389644474}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":44325,"name":"cephfs.smithi117.orzors","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6827/166927597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":166927597},{"type":"v1","addr":"172.21.15.117:6827","nonce":166927597}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T19:26:17.873644+0000","modified":"2024-02-16T19:38:57.810288+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34384},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34384":{"gid":34384,"name":"cephfs.smithi022.ocukvs","rank":0,"incarnation":31,"state":"up:replay","state_seq":1,"addr":"172.21.15.22:6827/2535680483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":2535680483},{"type":"v1","addr":"172.21.15.22:6827","nonce":2535680483}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T19:41:44.420 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 32 2024-02-16T19:41:44.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.420+0000 7f7f4f48b640 1 -- 172.21.15.22:0/3451444046 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f7f3406d810 msgr2=0x7f7f3406fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:44.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.420+0000 7f7f4f48b640 1 --2- 172.21.15.22:0/3451444046 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f7f3406d810 0x7f7f3406fcc0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7f44005fd0 tx=0x7f7f44005ce0 comp rx=0 tx=0).stop 2024-02-16T19:41:44.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.420+0000 7f7f4f48b640 1 -- 172.21.15.22:0/3451444046 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f48104770 msgr2=0x7f7f48196880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:44.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.420+0000 7f7f4f48b640 1 --2- 172.21.15.22:0/3451444046 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f48104770 0x7f7f48196880 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7f38009230 tx=0x7f7f38004040 comp rx=0 tx=0).stop 2024-02-16T19:41:44.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.420+0000 7f7f4f48b640 1 -- 172.21.15.22:0/3451444046 shutdown_connections 2024-02-16T19:41:44.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.420+0000 7f7f4f48b640 1 --2- 172.21.15.22:0/3451444046 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f7f3406d810 0x7f7f3406fcc0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:44.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.420+0000 7f7f4f48b640 1 --2- 172.21.15.22:0/3451444046 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f48104770 0x7f7f48196880 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:44.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.420+0000 7f7f4f48b640 1 --2- 172.21.15.22:0/3451444046 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f480fe760 0x7f7f48196340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:44.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.420+0000 7f7f4f48b640 1 -- 172.21.15.22:0/3451444046 >> 172.21.15.22:0/3451444046 conn(0x7f7f480fa260 msgr2=0x7f7f480fbc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:44.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.420+0000 7f7f4f48b640 1 -- 172.21.15.22:0/3451444046 shutdown_connections 2024-02-16T19:41:44.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.420+0000 7f7f4f48b640 1 -- 172.21.15.22:0/3451444046 wait complete. 2024-02-16T19:41:44.433 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 33 2024-02-16T19:41:44.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.535+0000 7f75cb1e1640 1 -- 172.21.15.22:0/4066260569 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75c4104770 msgr2=0x7f75c4104b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:44.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.535+0000 7f75cb1e1640 1 --2- 172.21.15.22:0/4066260569 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75c4104770 0x7f75c4104b40 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f75b8009880 tx=0x7f75b802bce0 comp rx=0 tx=0).stop 2024-02-16T19:41:44.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.535+0000 7f75cb1e1640 1 -- 172.21.15.22:0/4066260569 shutdown_connections 2024-02-16T19:41:44.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.535+0000 7f75cb1e1640 1 --2- 172.21.15.22:0/4066260569 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75c40fe760 0x7f75c40febd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:44.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.535+0000 7f75cb1e1640 1 --2- 172.21.15.22:0/4066260569 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75c4104770 0x7f75c4104b40 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:44.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.535+0000 7f75cb1e1640 1 -- 172.21.15.22:0/4066260569 >> 172.21.15.22:0/4066260569 conn(0x7f75c40fa260 msgr2=0x7f75c40fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:44.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.535+0000 7f75cb1e1640 1 -- 172.21.15.22:0/4066260569 shutdown_connections 2024-02-16T19:41:44.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.535+0000 7f75cb1e1640 1 -- 172.21.15.22:0/4066260569 wait complete. 2024-02-16T19:41:44.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.535+0000 7f75cb1e1640 1 Processor -- start 2024-02-16T19:41:44.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.535+0000 7f75cb1e1640 1 -- start start 2024-02-16T19:41:44.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.535+0000 7f75cb1e1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75c40fe760 0x7f75c41954f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:44.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.535+0000 7f75cb1e1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75c4104770 0x7f75c4195a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:44.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.535+0000 7f75cb1e1640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75c41960c0 con 0x7f75c40fe760 2024-02-16T19:41:44.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.535+0000 7f75cb1e1640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75c4199e90 con 0x7f75c4104770 2024-02-16T19:41:44.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.535+0000 7f75c8f7d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75c40fe760 0x7f75c41954f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:44.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.535+0000 7f75c8f7d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75c40fe760 0x7f75c41954f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59774/0 (socket says 172.21.15.22:59774) 2024-02-16T19:41:44.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.535+0000 7f75c8f7d640 1 -- 172.21.15.22:0/3893594772 learned_addr learned my addr 172.21.15.22:0/3893594772 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:41:44.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.536+0000 7f75c3fff640 1 --2- 172.21.15.22:0/3893594772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75c4104770 0x7f75c4195a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:44.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.536+0000 7f75c8f7d640 1 -- 172.21.15.22:0/3893594772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75c4104770 msgr2=0x7f75c4195a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:44.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.536+0000 7f75c8f7d640 1 --2- 172.21.15.22:0/3893594772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75c4104770 0x7f75c4195a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:44.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.536+0000 7f75c8f7d640 1 -- 172.21.15.22:0/3893594772 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75b8009560 con 0x7f75c40fe760 2024-02-16T19:41:44.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.536+0000 7f75c8f7d640 1 --2- 172.21.15.22:0/3893594772 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75c40fe760 0x7f75c41954f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f75b8009230 tx=0x7f75b802c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:44.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.536+0000 7f75c1ffb640 1 -- 172.21.15.22:0/3893594772 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f75b80035f0 con 0x7f75c40fe760 2024-02-16T19:41:44.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.536+0000 7f75c1ffb640 1 -- 172.21.15.22:0/3893594772 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f75b8003c10 con 0x7f75c40fe760 2024-02-16T19:41:44.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.536+0000 7f75cb1e1640 1 -- 172.21.15.22:0/3893594772 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75c419a110 con 0x7f75c40fe760 2024-02-16T19:41:44.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.536+0000 7f75c1ffb640 1 -- 172.21.15.22:0/3893594772 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f75b8005690 con 0x7f75c40fe760 2024-02-16T19:41:44.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.536+0000 7f75cb1e1640 1 -- 172.21.15.22:0/3893594772 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75c419a690 con 0x7f75c40fe760 2024-02-16T19:41:44.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.537+0000 7f75cb1e1640 1 -- 172.21.15.22:0/3893594772 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75c404e000 con 0x7f75c40fe760 2024-02-16T19:41:44.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.538+0000 7f75c1ffb640 1 -- 172.21.15.22:0/3893594772 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f75b8004ed0 con 0x7f75c40fe760 2024-02-16T19:41:44.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.538+0000 7f75c1ffb640 1 --2- 172.21.15.22:0/3893594772 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f75b406d810 0x7f75b406fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:44.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.538+0000 7f75c1ffb640 1 -- 172.21.15.22:0/3893594772 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f75b80afb10 con 0x7f75c40fe760 2024-02-16T19:41:44.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.538+0000 7f75c3fff640 1 --2- 172.21.15.22:0/3893594772 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f75b406d810 0x7f75b406fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:44.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.539+0000 7f75c3fff640 1 --2- 172.21.15.22:0/3893594772 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f75b406d810 0x7f75b406fcc0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f75b0005fd0 tx=0x7f75b0005850 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:44.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.543+0000 7f75c1ffb640 1 -- 172.21.15.22:0/3893594772 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f75b8079140 con 0x7f75c40fe760 2024-02-16T19:41:44.714 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:41:44 smithi022 ceph-mon[88798]: from='client.? 172.21.15.22:0/688908434' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-02-16T19:41:44.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:41:44 smithi022 ceph-mon[88798]: from='client.? 172.21.15.22:0/860277867' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-02-16T19:41:44.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.766+0000 7f75cb1e1640 1 -- 172.21.15.22:0/3893594772 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 33, "format": "json"} v 0) v1 -- 0x7f75c4066860 con 0x7f75c40fe760 2024-02-16T19:41:44.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.767+0000 7f75c1ffb640 1 -- 172.21.15.22:0/3893594772 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 33, "format": "json"}]=0 dumped fsmap epoch 33 v37) v1 ==== 107+0+5002 (secure 0 0 0) 0x7f75b807c800 con 0x7f75c40fe760 2024-02-16T19:41:44.768 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:41:44.768 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34408,"name":"cephfs.smithi022.btppwm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6829/3126701730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3126701730},{"type":"v1","addr":"172.21.15.22:6829","nonce":3126701730}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":44299,"name":"cephfs.smithi117.ibmppn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6825/2389644474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2389644474},{"type":"v1","addr":"172.21.15.117:6825","nonce":2389644474}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":44325,"name":"cephfs.smithi117.orzors","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6827/166927597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":166927597},{"type":"v1","addr":"172.21.15.117:6827","nonce":166927597}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T19:26:17.873644+0000","modified":"2024-02-16T19:39:07.197829+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34384},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34384":{"gid":34384,"name":"cephfs.smithi022.ocukvs","rank":0,"incarnation":31,"state":"up:reconnect","state_seq":20,"addr":"172.21.15.22:6827/2535680483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":2535680483},{"type":"v1","addr":"172.21.15.22:6827","nonce":2535680483}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T19:41:44.769 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 33 2024-02-16T19:41:44.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.769+0000 7f75cb1e1640 1 -- 172.21.15.22:0/3893594772 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f75b406d810 msgr2=0x7f75b406fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:44.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.769+0000 7f75cb1e1640 1 --2- 172.21.15.22:0/3893594772 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f75b406d810 0x7f75b406fcc0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f75b0005fd0 tx=0x7f75b0005850 comp rx=0 tx=0).stop 2024-02-16T19:41:44.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.769+0000 7f75cb1e1640 1 -- 172.21.15.22:0/3893594772 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75c40fe760 msgr2=0x7f75c41954f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:44.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.769+0000 7f75cb1e1640 1 --2- 172.21.15.22:0/3893594772 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75c40fe760 0x7f75c41954f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f75b8009230 tx=0x7f75b802c850 comp rx=0 tx=0).stop 2024-02-16T19:41:44.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.769+0000 7f75cb1e1640 1 -- 172.21.15.22:0/3893594772 shutdown_connections 2024-02-16T19:41:44.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.769+0000 7f75cb1e1640 1 --2- 172.21.15.22:0/3893594772 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f75b406d810 0x7f75b406fcc0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:44.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.769+0000 7f75cb1e1640 1 --2- 172.21.15.22:0/3893594772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75c4104770 0x7f75c4195a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:44.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.769+0000 7f75cb1e1640 1 --2- 172.21.15.22:0/3893594772 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75c40fe760 0x7f75c41954f0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:44.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.769+0000 7f75cb1e1640 1 -- 172.21.15.22:0/3893594772 >> 172.21.15.22:0/3893594772 conn(0x7f75c40fa260 msgr2=0x7f75c4068400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:44.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.769+0000 7f75cb1e1640 1 -- 172.21.15.22:0/3893594772 shutdown_connections 2024-02-16T19:41:44.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.769+0000 7f75cb1e1640 1 -- 172.21.15.22:0/3893594772 wait complete. 2024-02-16T19:41:44.782 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 34 2024-02-16T19:41:44.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.884+0000 7fbb15918640 1 -- 172.21.15.22:0/2440135543 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb100ff870 msgr2=0x7fbb100ffce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:44.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.884+0000 7fbb15918640 1 --2- 172.21.15.22:0/2440135543 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb100ff870 0x7fbb100ffce0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fbb04009880 tx=0x7fbb0402bce0 comp rx=0 tx=0).stop 2024-02-16T19:41:44.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.884+0000 7fbb15918640 1 -- 172.21.15.22:0/2440135543 shutdown_connections 2024-02-16T19:41:44.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.884+0000 7fbb15918640 1 --2- 172.21.15.22:0/2440135543 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb100ff870 0x7fbb100ffce0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:44.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.884+0000 7fbb15918640 1 --2- 172.21.15.22:0/2440135543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb10105880 0x7fbb10105c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:44.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.884+0000 7fbb15918640 1 -- 172.21.15.22:0/2440135543 >> 172.21.15.22:0/2440135543 conn(0x7fbb100fb370 msgr2=0x7fbb100fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:44.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.884+0000 7fbb15918640 1 -- 172.21.15.22:0/2440135543 shutdown_connections 2024-02-16T19:41:44.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.884+0000 7fbb15918640 1 -- 172.21.15.22:0/2440135543 wait complete. 2024-02-16T19:41:44.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.885+0000 7fbb15918640 1 Processor -- start 2024-02-16T19:41:44.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.885+0000 7fbb15918640 1 -- start start 2024-02-16T19:41:44.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:41:44 smithi117 ceph-mon[67870]: from='client.? 172.21.15.22:0/688908434' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-02-16T19:41:44.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:41:44 smithi117 ceph-mon[67870]: from='client.? 172.21.15.22:0/860277867' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-02-16T19:41:44.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.885+0000 7fbb15918640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb100ff870 0x7fbb10195690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:44.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.885+0000 7fbb15918640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb10105880 0x7fbb10195bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:44.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.885+0000 7fbb15918640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb101962b0 con 0x7fbb10105880 2024-02-16T19:41:44.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.885+0000 7fbb15918640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb10198f80 con 0x7fbb100ff870 2024-02-16T19:41:44.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.885+0000 7fbb0e7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb10105880 0x7fbb10195bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:44.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.885+0000 7fbb0e7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb10105880 0x7fbb10195bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59788/0 (socket says 172.21.15.22:59788) 2024-02-16T19:41:44.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.885+0000 7fbb0e7fc640 1 -- 172.21.15.22:0/3986607572 learned_addr learned my addr 172.21.15.22:0/3986607572 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:41:44.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.885+0000 7fbb0effd640 1 --2- 172.21.15.22:0/3986607572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb100ff870 0x7fbb10195690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:44.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.885+0000 7fbb0e7fc640 1 -- 172.21.15.22:0/3986607572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb100ff870 msgr2=0x7fbb10195690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:44.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.885+0000 7fbb0e7fc640 1 --2- 172.21.15.22:0/3986607572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb100ff870 0x7fbb10195690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:44.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.885+0000 7fbb0e7fc640 1 -- 172.21.15.22:0/3986607572 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb04009560 con 0x7fbb10105880 2024-02-16T19:41:44.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.885+0000 7fbb0effd640 1 --2- 172.21.15.22:0/3986607572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb100ff870 0x7fbb10195690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-16T19:41:44.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.886+0000 7fbb0e7fc640 1 --2- 172.21.15.22:0/3986607572 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb10105880 0x7fbb10195bd0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fbb04000c00 tx=0x7fbb04004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:44.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.886+0000 7fbb14916640 1 -- 172.21.15.22:0/3986607572 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb040037c0 con 0x7fbb10105880 2024-02-16T19:41:44.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.886+0000 7fbb15918640 1 -- 172.21.15.22:0/3986607572 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb10199200 con 0x7fbb10105880 2024-02-16T19:41:44.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.886+0000 7fbb14916640 1 -- 172.21.15.22:0/3986607572 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbb04003de0 con 0x7fbb10105880 2024-02-16T19:41:44.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.886+0000 7fbb14916640 1 -- 172.21.15.22:0/3986607572 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb04034460 con 0x7fbb10105880 2024-02-16T19:41:44.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.886+0000 7fbb15918640 1 -- 172.21.15.22:0/3986607572 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb10199700 con 0x7fbb10105880 2024-02-16T19:41:44.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.887+0000 7fbb15918640 1 -- 172.21.15.22:0/3986607572 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb1004e000 con 0x7fbb10105880 2024-02-16T19:41:44.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.887+0000 7fbb14916640 1 -- 172.21.15.22:0/3986607572 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fbb04003920 con 0x7fbb10105880 2024-02-16T19:41:44.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.888+0000 7fbb14916640 1 --2- 172.21.15.22:0/3986607572 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fbaf806d810 0x7fbaf806fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:44.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.888+0000 7fbb14916640 1 -- 172.21.15.22:0/3986607572 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6962+0+0 (secure 0 0 0) 0x7fbb040afbe0 con 0x7fbb10105880 2024-02-16T19:41:44.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.888+0000 7fbb0effd640 1 --2- 172.21.15.22:0/3986607572 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fbaf806d810 0x7fbaf806fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:44.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.888+0000 7fbb0effd640 1 --2- 172.21.15.22:0/3986607572 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fbaf806d810 0x7fbaf806fcc0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fbb00006fd0 tx=0x7fbb00008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:44.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:44.893+0000 7fbb14916640 1 -- 172.21.15.22:0/3986607572 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbb040792d0 con 0x7fbb10105880 2024-02-16T19:41:45.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.110+0000 7fbb15918640 1 -- 172.21.15.22:0/3986607572 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 34, "format": "json"} v 0) v1 -- 0x7fbb101969e0 con 0x7fbb10105880 2024-02-16T19:41:45.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.111+0000 7fbb14916640 1 -- 172.21.15.22:0/3986607572 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 34, "format": "json"}]=0 dumped fsmap epoch 34 v37) v1 ==== 107+0+4999 (secure 0 0 0) 0x7fbb0407c990 con 0x7fbb10105880 2024-02-16T19:41:45.112 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:41:45.112 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34408,"name":"cephfs.smithi022.btppwm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6829/3126701730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3126701730},{"type":"v1","addr":"172.21.15.22:6829","nonce":3126701730}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":44299,"name":"cephfs.smithi117.ibmppn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6825/2389644474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2389644474},{"type":"v1","addr":"172.21.15.117:6825","nonce":2389644474}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":44325,"name":"cephfs.smithi117.orzors","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6827/166927597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":166927597},{"type":"v1","addr":"172.21.15.117:6827","nonce":166927597}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T19:26:17.873644+0000","modified":"2024-02-16T19:39:07.713806+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34384},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34384":{"gid":34384,"name":"cephfs.smithi022.ocukvs","rank":0,"incarnation":31,"state":"up:rejoin","state_seq":21,"addr":"172.21.15.22:6827/2535680483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":2535680483},{"type":"v1","addr":"172.21.15.22:6827","nonce":2535680483}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T19:41:45.112 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 34 2024-02-16T19:41:45.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.113+0000 7fbb15918640 1 -- 172.21.15.22:0/3986607572 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fbaf806d810 msgr2=0x7fbaf806fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:45.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.113+0000 7fbb15918640 1 --2- 172.21.15.22:0/3986607572 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fbaf806d810 0x7fbaf806fcc0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fbb00006fd0 tx=0x7fbb00008040 comp rx=0 tx=0).stop 2024-02-16T19:41:45.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.113+0000 7fbb15918640 1 -- 172.21.15.22:0/3986607572 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb10105880 msgr2=0x7fbb10195bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:45.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.113+0000 7fbb15918640 1 --2- 172.21.15.22:0/3986607572 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb10105880 0x7fbb10195bd0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fbb04000c00 tx=0x7fbb04004270 comp rx=0 tx=0).stop 2024-02-16T19:41:45.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.113+0000 7fbb15918640 1 -- 172.21.15.22:0/3986607572 shutdown_connections 2024-02-16T19:41:45.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.113+0000 7fbb15918640 1 --2- 172.21.15.22:0/3986607572 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fbaf806d810 0x7fbaf806fcc0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:45.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.113+0000 7fbb15918640 1 --2- 172.21.15.22:0/3986607572 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb10105880 0x7fbb10195bd0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:45.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.113+0000 7fbb15918640 1 --2- 172.21.15.22:0/3986607572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb100ff870 0x7fbb10195690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:45.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.113+0000 7fbb15918640 1 -- 172.21.15.22:0/3986607572 >> 172.21.15.22:0/3986607572 conn(0x7fbb100fb370 msgr2=0x7fbb100fce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:45.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.113+0000 7fbb15918640 1 -- 172.21.15.22:0/3986607572 shutdown_connections 2024-02-16T19:41:45.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.113+0000 7fbb15918640 1 -- 172.21.15.22:0/3986607572 wait complete. 2024-02-16T19:41:45.125 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 35 2024-02-16T19:41:45.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.227+0000 7fb09ab4e640 1 -- 172.21.15.22:0/273519385 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb094104590 msgr2=0x7fb094104960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:45.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.227+0000 7fb09ab4e640 1 --2- 172.21.15.22:0/273519385 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb094104590 0x7fb094104960 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb08c009c20 tx=0x7fb08c02c1b0 comp rx=0 tx=0).stop 2024-02-16T19:41:45.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.227+0000 7fb09ab4e640 1 -- 172.21.15.22:0/273519385 shutdown_connections 2024-02-16T19:41:45.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.227+0000 7fb09ab4e640 1 --2- 172.21.15.22:0/273519385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0940fe5b0 0x7fb0940fea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:45.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.227+0000 7fb09ab4e640 1 --2- 172.21.15.22:0/273519385 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb094104590 0x7fb094104960 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:45.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.227+0000 7fb09ab4e640 1 -- 172.21.15.22:0/273519385 >> 172.21.15.22:0/273519385 conn(0x7fb0940743d0 msgr2=0x7fb0940747d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:45.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.228+0000 7fb09ab4e640 1 -- 172.21.15.22:0/273519385 shutdown_connections 2024-02-16T19:41:45.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.228+0000 7fb09ab4e640 1 -- 172.21.15.22:0/273519385 wait complete. 2024-02-16T19:41:45.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.228+0000 7fb09ab4e640 1 Processor -- start 2024-02-16T19:41:45.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.228+0000 7fb09ab4e640 1 -- start start 2024-02-16T19:41:45.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.228+0000 7fb09ab4e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0940fe5b0 0x7fb0941952e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:45.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.228+0000 7fb09ab4e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb094104590 0x7fb094195820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:45.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.228+0000 7fb09ab4e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb094195eb0 con 0x7fb094104590 2024-02-16T19:41:45.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.228+0000 7fb09ab4e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb094199c80 con 0x7fb0940fe5b0 2024-02-16T19:41:45.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.228+0000 7fb093fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb094104590 0x7fb094195820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:45.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.228+0000 7fb093fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb094104590 0x7fb094195820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59798/0 (socket says 172.21.15.22:59798) 2024-02-16T19:41:45.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.228+0000 7fb093fff640 1 -- 172.21.15.22:0/3260528065 learned_addr learned my addr 172.21.15.22:0/3260528065 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:41:45.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.229+0000 7fb0988ea640 1 --2- 172.21.15.22:0/3260528065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0940fe5b0 0x7fb0941952e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:45.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.229+0000 7fb093fff640 1 -- 172.21.15.22:0/3260528065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0940fe5b0 msgr2=0x7fb0941952e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:45.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.229+0000 7fb093fff640 1 --2- 172.21.15.22:0/3260528065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0940fe5b0 0x7fb0941952e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:45.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.229+0000 7fb093fff640 1 -- 172.21.15.22:0/3260528065 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb08c009900 con 0x7fb094104590 2024-02-16T19:41:45.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.229+0000 7fb0988ea640 1 --2- 172.21.15.22:0/3260528065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0940fe5b0 0x7fb0941952e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T19:41:45.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.229+0000 7fb093fff640 1 --2- 172.21.15.22:0/3260528065 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb094104590 0x7fb094195820 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb08800d6f0 tx=0x7fb08800dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:45.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.229+0000 7fb091ffb640 1 -- 172.21.15.22:0/3260528065 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb088009a30 con 0x7fb094104590 2024-02-16T19:41:45.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.229+0000 7fb091ffb640 1 -- 172.21.15.22:0/3260528065 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb0880054e0 con 0x7fb094104590 2024-02-16T19:41:45.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.229+0000 7fb09ab4e640 1 -- 172.21.15.22:0/3260528065 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb094199f60 con 0x7fb094104590 2024-02-16T19:41:45.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.229+0000 7fb091ffb640 1 -- 172.21.15.22:0/3260528065 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb088010460 con 0x7fb094104590 2024-02-16T19:41:45.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.229+0000 7fb09ab4e640 1 -- 172.21.15.22:0/3260528065 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb09419a540 con 0x7fb094104590 2024-02-16T19:41:45.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.230+0000 7fb09ab4e640 1 -- 172.21.15.22:0/3260528065 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb09404e000 con 0x7fb094104590 2024-02-16T19:41:45.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.231+0000 7fb091ffb640 1 -- 172.21.15.22:0/3260528065 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fb088009b90 con 0x7fb094104590 2024-02-16T19:41:45.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.231+0000 7fb091ffb640 1 --2- 172.21.15.22:0/3260528065 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fb07c06d810 0x7fb07c06fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:45.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.231+0000 7fb091ffb640 1 -- 172.21.15.22:0/3260528065 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6962+0+0 (secure 0 0 0) 0x7fb08808d240 con 0x7fb094104590 2024-02-16T19:41:45.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.231+0000 7fb0988ea640 1 --2- 172.21.15.22:0/3260528065 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fb07c06d810 0x7fb07c06fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:45.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.231+0000 7fb0988ea640 1 --2- 172.21.15.22:0/3260528065 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fb07c06d810 0x7fb07c06fcc0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb08c005e30 tx=0x7fb08c005da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:45.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.236+0000 7fb091ffb640 1 -- 172.21.15.22:0/3260528065 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb088056870 con 0x7fb094104590 2024-02-16T19:41:45.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.459+0000 7fb09ab4e640 1 -- 172.21.15.22:0/3260528065 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 35, "format": "json"} v 0) v1 -- 0x7fb094196630 con 0x7fb094104590 2024-02-16T19:41:45.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.459+0000 7fb091ffb640 1 -- 172.21.15.22:0/3260528065 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 35, "format": "json"}]=0 dumped fsmap epoch 35 v37) v1 ==== 107+0+4999 (secure 0 0 0) 0x7fb088059f30 con 0x7fb094104590 2024-02-16T19:41:45.461 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:41:45.461 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34408,"name":"cephfs.smithi022.btppwm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6829/3126701730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3126701730},{"type":"v1","addr":"172.21.15.22:6829","nonce":3126701730}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":44299,"name":"cephfs.smithi117.ibmppn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6825/2389644474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2389644474},{"type":"v1","addr":"172.21.15.117:6825","nonce":2389644474}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":44325,"name":"cephfs.smithi117.orzors","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6827/166927597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":166927597},{"type":"v1","addr":"172.21.15.117:6827","nonce":166927597}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T19:26:17.873644+0000","modified":"2024-02-16T19:39:08.778741+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34384},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34384":{"gid":34384,"name":"cephfs.smithi022.ocukvs","rank":0,"incarnation":31,"state":"up:active","state_seq":22,"addr":"172.21.15.22:6827/2535680483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":2535680483},{"type":"v1","addr":"172.21.15.22:6827","nonce":2535680483}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T19:41:45.461 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 35 2024-02-16T19:41:45.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.461+0000 7fb09ab4e640 1 -- 172.21.15.22:0/3260528065 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fb07c06d810 msgr2=0x7fb07c06fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:45.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.461+0000 7fb09ab4e640 1 --2- 172.21.15.22:0/3260528065 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fb07c06d810 0x7fb07c06fcc0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb08c005e30 tx=0x7fb08c005da0 comp rx=0 tx=0).stop 2024-02-16T19:41:45.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.461+0000 7fb09ab4e640 1 -- 172.21.15.22:0/3260528065 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb094104590 msgr2=0x7fb094195820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:45.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.461+0000 7fb09ab4e640 1 --2- 172.21.15.22:0/3260528065 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb094104590 0x7fb094195820 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb08800d6f0 tx=0x7fb08800dbb0 comp rx=0 tx=0).stop 2024-02-16T19:41:45.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.461+0000 7fb09ab4e640 1 -- 172.21.15.22:0/3260528065 shutdown_connections 2024-02-16T19:41:45.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.461+0000 7fb09ab4e640 1 --2- 172.21.15.22:0/3260528065 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fb07c06d810 0x7fb07c06fcc0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:45.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.461+0000 7fb09ab4e640 1 --2- 172.21.15.22:0/3260528065 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb094104590 0x7fb094195820 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:45.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.461+0000 7fb09ab4e640 1 --2- 172.21.15.22:0/3260528065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0940fe5b0 0x7fb0941952e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:45.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.461+0000 7fb09ab4e640 1 -- 172.21.15.22:0/3260528065 >> 172.21.15.22:0/3260528065 conn(0x7fb0940743d0 msgr2=0x7fb094067310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:45.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.461+0000 7fb09ab4e640 1 -- 172.21.15.22:0/3260528065 shutdown_connections 2024-02-16T19:41:45.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.461+0000 7fb09ab4e640 1 -- 172.21.15.22:0/3260528065 wait complete. 2024-02-16T19:41:45.474 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 36 2024-02-16T19:41:45.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.576+0000 7f5571741640 1 -- 172.21.15.22:0/1325730168 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f556c100150 msgr2=0x7f556c1088b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:45.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.576+0000 7f5571741640 1 --2- 172.21.15.22:0/1325730168 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f556c100150 0x7f556c1088b0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5560009880 tx=0x7f556002bce0 comp rx=0 tx=0).stop 2024-02-16T19:41:45.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.576+0000 7f5571741640 1 -- 172.21.15.22:0/1325730168 shutdown_connections 2024-02-16T19:41:45.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.576+0000 7f5571741640 1 --2- 172.21.15.22:0/1325730168 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f556c100150 0x7f556c1088b0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:45.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.576+0000 7f5571741640 1 --2- 172.21.15.22:0/1325730168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f556c0ff840 0x7f556c0ffc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:45.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.576+0000 7f5571741640 1 -- 172.21.15.22:0/1325730168 >> 172.21.15.22:0/1325730168 conn(0x7f556c073d20 msgr2=0x7f556c074120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:45.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.576+0000 7f5571741640 1 -- 172.21.15.22:0/1325730168 shutdown_connections 2024-02-16T19:41:45.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.576+0000 7f5571741640 1 -- 172.21.15.22:0/1325730168 wait complete. 2024-02-16T19:41:45.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.576+0000 7f5571741640 1 Processor -- start 2024-02-16T19:41:45.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.577+0000 7f5571741640 1 -- start start 2024-02-16T19:41:45.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.577+0000 7f5571741640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f556c0ff840 0x7f556c195380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:45.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.577+0000 7f5571741640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f556c100150 0x7f556c1958c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:45.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.577+0000 7f5571741640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f556c195f50 con 0x7f556c0ff840 2024-02-16T19:41:45.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.577+0000 7f5571741640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f556c199d20 con 0x7f556c100150 2024-02-16T19:41:45.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.577+0000 7f556affd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f556c0ff840 0x7f556c195380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:45.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.577+0000 7f556affd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f556c0ff840 0x7f556c195380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59804/0 (socket says 172.21.15.22:59804) 2024-02-16T19:41:45.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.577+0000 7f556affd640 1 -- 172.21.15.22:0/861647110 learned_addr learned my addr 172.21.15.22:0/861647110 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:41:45.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.577+0000 7f556a7fc640 1 --2- 172.21.15.22:0/861647110 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f556c100150 0x7f556c1958c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:45.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.577+0000 7f556affd640 1 -- 172.21.15.22:0/861647110 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f556c100150 msgr2=0x7f556c1958c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:45.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.577+0000 7f556affd640 1 --2- 172.21.15.22:0/861647110 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f556c100150 0x7f556c1958c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:45.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.577+0000 7f556affd640 1 -- 172.21.15.22:0/861647110 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5560009560 con 0x7f556c0ff840 2024-02-16T19:41:45.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.577+0000 7f556a7fc640 1 --2- 172.21.15.22:0/861647110 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f556c100150 0x7f556c1958c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T19:41:45.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.577+0000 7f556affd640 1 --2- 172.21.15.22:0/861647110 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f556c0ff840 0x7f556c195380 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f555c00d6f0 tx=0x7f555c00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:45.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.577+0000 7f5553fff640 1 -- 172.21.15.22:0/861647110 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f555c009a30 con 0x7f556c0ff840 2024-02-16T19:41:45.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.577+0000 7f5553fff640 1 -- 172.21.15.22:0/861647110 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f555c0054e0 con 0x7f556c0ff840 2024-02-16T19:41:45.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.577+0000 7f5571741640 1 -- 172.21.15.22:0/861647110 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f556c19a000 con 0x7f556c0ff840 2024-02-16T19:41:45.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.577+0000 7f5553fff640 1 -- 172.21.15.22:0/861647110 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f555c010460 con 0x7f556c0ff840 2024-02-16T19:41:45.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.578+0000 7f5571741640 1 -- 172.21.15.22:0/861647110 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f556c19a560 con 0x7f556c0ff840 2024-02-16T19:41:45.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.578+0000 7f5571741640 1 -- 172.21.15.22:0/861647110 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f556c04e000 con 0x7f556c0ff840 2024-02-16T19:41:45.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.579+0000 7f5553fff640 1 -- 172.21.15.22:0/861647110 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f555c009b90 con 0x7f556c0ff840 2024-02-16T19:41:45.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.579+0000 7f5553fff640 1 --2- 172.21.15.22:0/861647110 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f555406d810 0x7f555406fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:45.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.579+0000 7f5553fff640 1 -- 172.21.15.22:0/861647110 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f555c08d150 con 0x7f556c0ff840 2024-02-16T19:41:45.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.579+0000 7f556a7fc640 1 --2- 172.21.15.22:0/861647110 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f555406d810 0x7f555406fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:45.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.580+0000 7f556a7fc640 1 --2- 172.21.15.22:0/861647110 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f555406d810 0x7f555406fcc0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f556c196990 tx=0x7f556002cb40 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:45.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.584+0000 7f5553fff640 1 -- 172.21.15.22:0/861647110 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f555c056800 con 0x7f556c0ff840 2024-02-16T19:41:45.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:41:45 smithi022 ceph-mon[88798]: pgmap v300: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:41:45.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:41:45 smithi022 ceph-mon[88798]: from='client.? 172.21.15.22:0/3451444046' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-02-16T19:41:45.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:41:45 smithi022 ceph-mon[88798]: from='client.? 172.21.15.22:0/3893594772' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-02-16T19:41:45.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:41:45 smithi022 ceph-mon[88798]: from='client.? 172.21.15.22:0/3986607572' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-02-16T19:41:45.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.803+0000 7f5571741640 1 -- 172.21.15.22:0/861647110 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 36, "format": "json"} v 0) v1 -- 0x7f556c10b770 con 0x7f556c0ff840 2024-02-16T19:41:45.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.803+0000 7f5553fff640 1 -- 172.21.15.22:0/861647110 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 36, "format": "json"}]=0 dumped fsmap epoch 36 v37) v1 ==== 107+0+4999 (secure 0 0 0) 0x7f555c059ec0 con 0x7f556c0ff840 2024-02-16T19:41:45.805 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:41:45.805 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34408,"name":"cephfs.smithi022.btppwm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6829/3126701730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3126701730},{"type":"v1","addr":"172.21.15.22:6829","nonce":3126701730}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":44299,"name":"cephfs.smithi117.ibmppn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6825/2389644474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2389644474},{"type":"v1","addr":"172.21.15.117:6825","nonce":2389644474}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":44325,"name":"cephfs.smithi117.orzors","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6827/166927597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":166927597},{"type":"v1","addr":"172.21.15.117:6827","nonce":166927597}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T19:26:17.873644+0000","modified":"2024-02-16T19:39:13.044372+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34384},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34384":{"gid":34384,"name":"cephfs.smithi022.ocukvs","rank":0,"incarnation":31,"state":"up:active","state_seq":22,"addr":"172.21.15.22:6827/2535680483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":2535680483},{"type":"v1","addr":"172.21.15.22:6827","nonce":2535680483}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T19:41:45.805 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 36 2024-02-16T19:41:45.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.806+0000 7f5571741640 1 -- 172.21.15.22:0/861647110 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f555406d810 msgr2=0x7f555406fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:45.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.806+0000 7f5571741640 1 --2- 172.21.15.22:0/861647110 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f555406d810 0x7f555406fcc0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f556c196990 tx=0x7f556002cb40 comp rx=0 tx=0).stop 2024-02-16T19:41:45.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.806+0000 7f5571741640 1 -- 172.21.15.22:0/861647110 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f556c0ff840 msgr2=0x7f556c195380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:45.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.806+0000 7f5571741640 1 --2- 172.21.15.22:0/861647110 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f556c0ff840 0x7f556c195380 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f555c00d6f0 tx=0x7f555c00dbb0 comp rx=0 tx=0).stop 2024-02-16T19:41:45.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.806+0000 7f5571741640 1 -- 172.21.15.22:0/861647110 shutdown_connections 2024-02-16T19:41:45.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.806+0000 7f5571741640 1 --2- 172.21.15.22:0/861647110 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f555406d810 0x7f555406fcc0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:45.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.806+0000 7f5571741640 1 --2- 172.21.15.22:0/861647110 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f556c100150 0x7f556c1958c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:45.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.806+0000 7f5571741640 1 --2- 172.21.15.22:0/861647110 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f556c0ff840 0x7f556c195380 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:45.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.806+0000 7f5571741640 1 -- 172.21.15.22:0/861647110 >> 172.21.15.22:0/861647110 conn(0x7f556c073d20 msgr2=0x7f556c0ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:45.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.806+0000 7f5571741640 1 -- 172.21.15.22:0/861647110 shutdown_connections 2024-02-16T19:41:45.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:45.806+0000 7f5571741640 1 -- 172.21.15.22:0/861647110 wait complete. 2024-02-16T19:41:45.818 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-02-16T19:41:45.833 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-02-16T19:41:45.833 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-02-16T19:41:45.833 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-16T19:41:45.850 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-02-16T19:41:45.850 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-16T19:41:45.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:41:45 smithi117 ceph-mon[67870]: pgmap v300: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:41:45.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:41:45 smithi117 ceph-mon[67870]: from='client.? 172.21.15.22:0/3451444046' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-02-16T19:41:45.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:41:45 smithi117 ceph-mon[67870]: from='client.? 172.21.15.22:0/3893594772' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-02-16T19:41:45.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:41:45 smithi117 ceph-mon[67870]: from='client.? 172.21.15.22:0/3986607572' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-02-16T19:41:45.906 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-16T19:41:46.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.048+0000 7fb54cffb640 1 -- 172.21.15.22:0/4113429262 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb5481058b0 msgr2=0x7fb548105c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:46.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.048+0000 7fb54cffb640 1 --2- 172.21.15.22:0/4113429262 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb5481058b0 0x7fb548105c80 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb540009880 tx=0x7fb54002bce0 comp rx=0 tx=0).stop 2024-02-16T19:41:46.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.048+0000 7fb54cffb640 1 -- 172.21.15.22:0/4113429262 shutdown_connections 2024-02-16T19:41:46.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.048+0000 7fb54cffb640 1 --2- 172.21.15.22:0/4113429262 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5480ff8a0 0x7fb5480ffd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:46.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.048+0000 7fb54cffb640 1 --2- 172.21.15.22:0/4113429262 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb5481058b0 0x7fb548105c80 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:46.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.048+0000 7fb54cffb640 1 -- 172.21.15.22:0/4113429262 >> 172.21.15.22:0/4113429262 conn(0x7fb5480fb3a0 msgr2=0x7fb5480fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:46.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.048+0000 7fb54cffb640 1 -- 172.21.15.22:0/4113429262 shutdown_connections 2024-02-16T19:41:46.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.048+0000 7fb54cffb640 1 -- 172.21.15.22:0/4113429262 wait complete. 2024-02-16T19:41:46.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.049+0000 7fb54cffb640 1 Processor -- start 2024-02-16T19:41:46.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.049+0000 7fb54cffb640 1 -- start start 2024-02-16T19:41:46.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.049+0000 7fb54cffb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb5480ff8a0 0x7fb5481954e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:46.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.049+0000 7fb54cffb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5481058b0 0x7fb548195a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:46.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.049+0000 7fb54cffb640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5481960b0 con 0x7fb5480ff8a0 2024-02-16T19:41:46.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.049+0000 7fb54cffb640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb548199e80 con 0x7fb5481058b0 2024-02-16T19:41:46.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.049+0000 7fb54659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb5480ff8a0 0x7fb5481954e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:46.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.049+0000 7fb54659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb5480ff8a0 0x7fb5481954e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59826/0 (socket says 172.21.15.22:59826) 2024-02-16T19:41:46.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.049+0000 7fb54659c640 1 -- 172.21.15.22:0/3758746209 learned_addr learned my addr 172.21.15.22:0/3758746209 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:41:46.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.049+0000 7fb54659c640 1 -- 172.21.15.22:0/3758746209 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5481058b0 msgr2=0x7fb548195a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:46.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.049+0000 7fb545d9b640 1 --2- 172.21.15.22:0/3758746209 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5481058b0 0x7fb548195a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:46.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.050+0000 7fb54659c640 1 --2- 172.21.15.22:0/3758746209 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5481058b0 0x7fb548195a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:46.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.050+0000 7fb54659c640 1 -- 172.21.15.22:0/3758746209 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb540009560 con 0x7fb5480ff8a0 2024-02-16T19:41:46.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.050+0000 7fb545d9b640 1 --2- 172.21.15.22:0/3758746209 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5481058b0 0x7fb548195a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-16T19:41:46.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.050+0000 7fb54659c640 1 --2- 172.21.15.22:0/3758746209 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb5480ff8a0 0x7fb5481954e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb540009230 tx=0x7fb540004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:46.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.050+0000 7fb53f7fe640 1 -- 172.21.15.22:0/3758746209 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5400037c0 con 0x7fb5480ff8a0 2024-02-16T19:41:46.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.050+0000 7fb53f7fe640 1 -- 172.21.15.22:0/3758746209 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb540003de0 con 0x7fb5480ff8a0 2024-02-16T19:41:46.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.050+0000 7fb54cffb640 1 -- 172.21.15.22:0/3758746209 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb54819a100 con 0x7fb5480ff8a0 2024-02-16T19:41:46.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.050+0000 7fb53f7fe640 1 -- 172.21.15.22:0/3758746209 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb540034440 con 0x7fb5480ff8a0 2024-02-16T19:41:46.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.050+0000 7fb54cffb640 1 -- 172.21.15.22:0/3758746209 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb54819a680 con 0x7fb5480ff8a0 2024-02-16T19:41:46.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.051+0000 7fb54cffb640 1 -- 172.21.15.22:0/3758746209 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb54804e000 con 0x7fb5480ff8a0 2024-02-16T19:41:46.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.051+0000 7fb53f7fe640 1 -- 172.21.15.22:0/3758746209 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fb540004fb0 con 0x7fb5480ff8a0 2024-02-16T19:41:46.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.052+0000 7fb53f7fe640 1 --2- 172.21.15.22:0/3758746209 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fb53406d8d0 0x7fb53406fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:46.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.052+0000 7fb53f7fe640 1 -- 172.21.15.22:0/3758746209 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6962+0+0 (secure 0 0 0) 0x7fb5400afb90 con 0x7fb5480ff8a0 2024-02-16T19:41:46.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.052+0000 7fb545d9b640 1 --2- 172.21.15.22:0/3758746209 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fb53406d8d0 0x7fb53406fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:46.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.052+0000 7fb545d9b640 1 --2- 172.21.15.22:0/3758746209 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fb53406d8d0 0x7fb53406fd80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb530006fd0 tx=0x7fb530008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:46.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.057+0000 7fb53f7fe640 1 -- 172.21.15.22:0/3758746209 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb5400791c0 con 0x7fb5480ff8a0 2024-02-16T19:41:46.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.240+0000 7fb54cffb640 1 -- 172.21.15.22:0/3758746209 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb548066860 con 0x7fb5480ff8a0 2024-02-16T19:41:46.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.241+0000 7fb53f7fe640 1 -- 172.21.15.22:0/3758746209 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+17322 (secure 0 0 0) 0x7fb54007c880 con 0x7fb5480ff8a0 2024-02-16T19:41:46.242 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:41:46.243 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":93,"fsid":"2a1346e6-cd00-11ee-95ba-87774f69a715","created":"2024-02-16T19:19:11.528280+0000","modified":"2024-02-16T19:38:57.804537+0000","last_up_change":"2024-02-16T19:37:45.909206+0000","last_in_change":"2024-02-16T19:24:40.575373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-16T19:23:47.101426+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-02-16T19:26:16.533404+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-02-16T19:26:17.113076+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d16e47f2-7209-45f4-ad50-1e0f6789ffad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":50,"up_from":54,"up_thru":88,"down_at":51,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":1744550481},{"type":"v1","addr":"172.21.15.22:6803","nonce":1744550481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":1744550481},{"type":"v1","addr":"172.21.15.22:6805","nonce":1744550481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":1744550481},{"type":"v1","addr":"172.21.15.22:6809","nonce":1744550481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":1744550481},{"type":"v1","addr":"172.21.15.22:6807","nonce":1744550481}]},"public_addr":"172.21.15.22:6803/1744550481","cluster_addr":"172.21.15.22:6805/1744550481","heartbeat_back_addr":"172.21.15.22:6809/1744550481","heartbeat_front_addr":"172.21.15.22:6807/1744550481","state":["exists","up"]},{"osd":1,"uuid":"59c3d085-3422-4e92-9048-46d7a39e31a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":55,"up_from":64,"up_thru":88,"down_at":56,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":3321991378},{"type":"v1","addr":"172.21.15.22:6811","nonce":3321991378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":3321991378},{"type":"v1","addr":"172.21.15.22:6813","nonce":3321991378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":3321991378},{"type":"v1","addr":"172.21.15.22:6817","nonce":3321991378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":3321991378},{"type":"v1","addr":"172.21.15.22:6815","nonce":3321991378}]},"public_addr":"172.21.15.22:6811/3321991378","cluster_addr":"172.21.15.22:6813/3321991378","heartbeat_back_addr":"172.21.15.22:6817/3321991378","heartbeat_front_addr":"172.21.15.22:6815/3321991378","state":["exists","up"]},{"osd":2,"uuid":"5952e710-f0b6-406c-855a-00f1cb8059c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":69,"up_from":73,"up_thru":88,"down_at":70,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":2098766749},{"type":"v1","addr":"172.21.15.22:6819","nonce":2098766749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":2098766749},{"type":"v1","addr":"172.21.15.22:6821","nonce":2098766749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":2098766749},{"type":"v1","addr":"172.21.15.22:6825","nonce":2098766749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":2098766749},{"type":"v1","addr":"172.21.15.22:6823","nonce":2098766749}]},"public_addr":"172.21.15.22:6819/2098766749","cluster_addr":"172.21.15.22:6821/2098766749","heartbeat_back_addr":"172.21.15.22:6825/2098766749","heartbeat_front_addr":"172.21.15.22:6823/2098766749","state":["exists","up"]},{"osd":3,"uuid":"5dc87494-e400-4111-91a7-306d9905febb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":27,"last_clean_end":74,"up_from":78,"up_thru":88,"down_at":75,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3703365528},{"type":"v1","addr":"172.21.15.117:6801","nonce":3703365528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":3703365528},{"type":"v1","addr":"172.21.15.117:6803","nonce":3703365528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":3703365528},{"type":"v1","addr":"172.21.15.117:6807","nonce":3703365528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":3703365528},{"type":"v1","addr":"172.21.15.117:6805","nonce":3703365528}]},"public_addr":"172.21.15.117:6801/3703365528","cluster_addr":"172.21.15.117:6803/3703365528","heartbeat_back_addr":"172.21.15.117:6807/3703365528","heartbeat_front_addr":"172.21.15.117:6805/3703365528","state":["exists","up"]},{"osd":4,"uuid":"d27bde8d-1a77-4f95-acb6-7d05becd9480","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":35,"last_clean_end":79,"up_from":83,"up_thru":88,"down_at":80,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":3679983685},{"type":"v1","addr":"172.21.15.117:6809","nonce":3679983685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":3679983685},{"type":"v1","addr":"172.21.15.117:6811","nonce":3679983685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":3679983685},{"type":"v1","addr":"172.21.15.117:6815","nonce":3679983685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":3679983685},{"type":"v1","addr":"172.21.15.117:6813","nonce":3679983685}]},"public_addr":"172.21.15.117:6809/3679983685","cluster_addr":"172.21.15.117:6811/3679983685","heartbeat_back_addr":"172.21.15.117:6815/3679983685","heartbeat_front_addr":"172.21.15.117:6813/3679983685","state":["exists","up"]},{"osd":5,"uuid":"5873798f-3a2c-4a29-becd-6280b8866b6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":40,"last_clean_end":84,"up_from":88,"up_thru":88,"down_at":85,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1403457815},{"type":"v1","addr":"172.21.15.117:6817","nonce":1403457815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1403457815},{"type":"v1","addr":"172.21.15.117:6819","nonce":1403457815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1403457815},{"type":"v1","addr":"172.21.15.117:6823","nonce":1403457815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1403457815},{"type":"v1","addr":"172.21.15.117:6821","nonce":1403457815}]},"public_addr":"172.21.15.117:6817/1403457815","cluster_addr":"172.21.15.117:6819/1403457815","heartbeat_back_addr":"172.21.15.117:6823/1403457815","heartbeat_front_addr":"172.21.15.117:6821/1403457815","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:22:44.777427+0000","dead_epoch":50},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:23:14.935167+0000","dead_epoch":55},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:23:44.496184+0000","dead_epoch":69},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:24:07.447481+0000","dead_epoch":74},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:24:32.205064+0000","dead_epoch":79},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:24:57.982560+0000","dead_epoch":84}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":3,"to":2}]},{"pgid":"2.5","mappings":[{"from":3,"to":1}]},{"pgid":"2.6","mappings":[{"from":3,"to":2}]},{"pgid":"2.9","mappings":[{"from":0,"to":2}]},{"pgid":"2.10","mappings":[{"from":0,"to":5}]},{"pgid":"2.1e","mappings":[{"from":0,"to":1}]},{"pgid":"3.3","mappings":[{"from":4,"to":1}]},{"pgid":"3.f","mappings":[{"from":3,"to":1}]},{"pgid":"3.16","mappings":[{"from":3,"to":0}]},{"pgid":"3.1b","mappings":[{"from":4,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.117:6827/3122342028":"2024-02-17T19:38:57.804505+0000","172.21.15.117:6826/3122342028":"2024-02-17T19:38:57.804505+0000","172.21.15.22:6829/3597685751":"2024-02-17T19:38:09.297300+0000","172.21.15.22:6828/3597685751":"2024-02-17T19:38:09.297300+0000","172.21.15.22:0/4238358751":"2024-02-17T19:33:15.748281+0000","172.21.15.22:6801/2755289428":"2024-02-17T19:20:11.619146+0000","172.21.15.22:6800/2253867279":"2024-02-17T19:28:59.405724+0000","172.21.15.22:0/2301835367":"2024-02-17T19:28:59.405724+0000","172.21.15.22:6801/3109015836":"2024-02-17T19:19:38.544291+0000","172.21.15.117:6825/565408102":"2024-02-17T19:38:31.770686+0000","172.21.15.22:0/206933569":"2024-02-17T19:28:59.405724+0000","172.21.15.22:0/3012142271":"2024-02-17T19:33:15.748281+0000","172.21.15.117:0/2453907739":"2024-02-17T19:29:53.194933+0000","172.21.15.22:6800/3109015836":"2024-02-17T19:19:38.544291+0000","172.21.15.117:0/3042073611":"2024-02-17T19:29:53.194933+0000","172.21.15.117:0/4194262516":"2024-02-17T19:29:53.194933+0000","172.21.15.22:6800/2945983569":"2024-02-17T19:21:26.760165+0000","172.21.15.22:0/1603216752":"2024-02-17T19:19:38.544291+0000","172.21.15.22:6827/149959963":"2024-02-17T19:26:32.932573+0000","172.21.15.22:6801/2945983569":"2024-02-17T19:21:26.760165+0000","172.21.15.22:0/668279955":"2024-02-17T19:21:26.760165+0000","172.21.15.22:0/2705605267":"2024-02-17T19:28:59.405724+0000","172.21.15.117:6824/565408102":"2024-02-17T19:38:31.770686+0000","172.21.15.22:6800/2755289428":"2024-02-17T19:20:11.619146+0000","172.21.15.22:0/2099601555":"2024-02-17T19:20:11.619146+0000","172.21.15.22:6801/2253867279":"2024-02-17T19:28:59.405724+0000","172.21.15.117:0/333707173":"2024-02-17T19:29:53.194933+0000","172.21.15.117:6828/10500008":"2024-02-17T19:29:53.194933+0000","172.21.15.117:0/199482916":"2024-02-17T19:29:53.194933+0000","172.21.15.117:6829/10500008":"2024-02-17T19:29:53.194933+0000","172.21.15.22:0/2017591724":"2024-02-17T19:20:11.619146+0000","172.21.15.22:0/2371245756":"2024-02-17T19:19:38.544291+0000","172.21.15.22:6801/3703806543":"2024-02-17T19:33:15.748281+0000","172.21.15.22:0/2384083375":"2024-02-17T19:21:26.760165+0000","172.21.15.22:0/402850490":"2024-02-17T19:28:59.405724+0000","172.21.15.22:0/1229330195":"2024-02-17T19:20:11.619146+0000","172.21.15.22:0/8466130":"2024-02-17T19:19:38.544291+0000","172.21.15.22:0/3200629226":"2024-02-17T19:21:26.760165+0000","172.21.15.117:0/2264991341":"2024-02-17T19:29:53.194933+0000","172.21.15.22:0/2853892105":"2024-02-17T19:33:15.748281+0000","172.21.15.22:6826/149959963":"2024-02-17T19:26:32.932573+0000","172.21.15.22:6800/3703806543":"2024-02-17T19:33:15.748281+0000","172.21.15.22:0/2353857756":"2024-02-17T19:33:15.748281+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-16T19:41:46.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.243+0000 7fb54cffb640 1 -- 172.21.15.22:0/3758746209 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fb53406d8d0 msgr2=0x7fb53406fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:46.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.243+0000 7fb54cffb640 1 --2- 172.21.15.22:0/3758746209 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fb53406d8d0 0x7fb53406fd80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb530006fd0 tx=0x7fb530008040 comp rx=0 tx=0).stop 2024-02-16T19:41:46.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.243+0000 7fb54cffb640 1 -- 172.21.15.22:0/3758746209 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb5480ff8a0 msgr2=0x7fb5481954e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:46.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.243+0000 7fb54cffb640 1 --2- 172.21.15.22:0/3758746209 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb5480ff8a0 0x7fb5481954e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb540009230 tx=0x7fb540004270 comp rx=0 tx=0).stop 2024-02-16T19:41:46.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.243+0000 7fb54cffb640 1 -- 172.21.15.22:0/3758746209 shutdown_connections 2024-02-16T19:41:46.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.243+0000 7fb54cffb640 1 --2- 172.21.15.22:0/3758746209 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fb53406d8d0 0x7fb53406fd80 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:46.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.243+0000 7fb54cffb640 1 --2- 172.21.15.22:0/3758746209 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5481058b0 0x7fb548195a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:46.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.243+0000 7fb54cffb640 1 --2- 172.21.15.22:0/3758746209 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb5480ff8a0 0x7fb5481954e0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:46.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.243+0000 7fb54cffb640 1 -- 172.21.15.22:0/3758746209 >> 172.21.15.22:0/3758746209 conn(0x7fb5480fb3a0 msgr2=0x7fb5480fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:46.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.243+0000 7fb54cffb640 1 -- 172.21.15.22:0/3758746209 shutdown_connections 2024-02-16T19:41:46.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.243+0000 7fb54cffb640 1 -- 172.21.15.22:0/3758746209 wait complete. 2024-02-16T19:41:46.256 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-16T19:23:47.101426+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-02-16T19:26:16.533404+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-02-16T19:26:17.113076+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-16T19:41:46.256 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-16T19:41:46.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.360+0000 7f65c2118640 1 -- 172.21.15.22:0/3234647232 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f65bc0ff7f0 msgr2=0x7f65bc0ffbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:46.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.360+0000 7f65c2118640 1 --2- 172.21.15.22:0/3234647232 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f65bc0ff7f0 0x7f65bc0ffbc0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f65ac009880 tx=0x7f65ac02bce0 comp rx=0 tx=0).stop 2024-02-16T19:41:46.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.360+0000 7f65c2118640 1 -- 172.21.15.22:0/3234647232 shutdown_connections 2024-02-16T19:41:46.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.360+0000 7f65c2118640 1 --2- 172.21.15.22:0/3234647232 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65bc100100 0x7f65bc108860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:46.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.360+0000 7f65c2118640 1 --2- 172.21.15.22:0/3234647232 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f65bc0ff7f0 0x7f65bc0ffbc0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:46.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.360+0000 7f65c2118640 1 -- 172.21.15.22:0/3234647232 >> 172.21.15.22:0/3234647232 conn(0x7f65bc074cf0 msgr2=0x7f65bc0750f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:46.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.360+0000 7f65c2118640 1 -- 172.21.15.22:0/3234647232 shutdown_connections 2024-02-16T19:41:46.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.360+0000 7f65c2118640 1 -- 172.21.15.22:0/3234647232 wait complete. 2024-02-16T19:41:46.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.360+0000 7f65c2118640 1 Processor -- start 2024-02-16T19:41:46.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.361+0000 7f65c2118640 1 -- start start 2024-02-16T19:41:46.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.361+0000 7f65c2118640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65bc0ff7f0 0x7f65bc1954c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:46.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.361+0000 7f65c2118640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f65bc100100 0x7f65bc195a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:46.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.361+0000 7f65c2118640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65bc196090 con 0x7f65bc100100 2024-02-16T19:41:46.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.361+0000 7f65c2118640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65bc199e60 con 0x7f65bc0ff7f0 2024-02-16T19:41:46.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.361+0000 7f65baffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f65bc100100 0x7f65bc195a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:46.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.361+0000 7f65baffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f65bc100100 0x7f65bc195a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59842/0 (socket says 172.21.15.22:59842) 2024-02-16T19:41:46.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.361+0000 7f65baffd640 1 -- 172.21.15.22:0/4108180507 learned_addr learned my addr 172.21.15.22:0/4108180507 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:41:46.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.361+0000 7f65bb7fe640 1 --2- 172.21.15.22:0/4108180507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65bc0ff7f0 0x7f65bc1954c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:46.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.361+0000 7f65baffd640 1 -- 172.21.15.22:0/4108180507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65bc0ff7f0 msgr2=0x7f65bc1954c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:46.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.361+0000 7f65baffd640 1 --2- 172.21.15.22:0/4108180507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65bc0ff7f0 0x7f65bc1954c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:46.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.361+0000 7f65baffd640 1 -- 172.21.15.22:0/4108180507 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65ac009560 con 0x7f65bc100100 2024-02-16T19:41:46.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.361+0000 7f65baffd640 1 --2- 172.21.15.22:0/4108180507 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f65bc100100 0x7f65bc195a00 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f65b000b420 tx=0x7f65b000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:46.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.362+0000 7f65b8ff9640 1 -- 172.21.15.22:0/4108180507 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f65b0011800 con 0x7f65bc100100 2024-02-16T19:41:46.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.362+0000 7f65b8ff9640 1 -- 172.21.15.22:0/4108180507 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f65b0011e20 con 0x7f65bc100100 2024-02-16T19:41:46.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.362+0000 7f65c2118640 1 -- 172.21.15.22:0/4108180507 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65bc19a140 con 0x7f65bc100100 2024-02-16T19:41:46.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.362+0000 7f65b8ff9640 1 -- 172.21.15.22:0/4108180507 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f65b000f2e0 con 0x7f65bc100100 2024-02-16T19:41:46.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.362+0000 7f65c2118640 1 -- 172.21.15.22:0/4108180507 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65bc19a6a0 con 0x7f65bc100100 2024-02-16T19:41:46.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.363+0000 7f65c2118640 1 -- 172.21.15.22:0/4108180507 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65bc04e000 con 0x7f65bc100100 2024-02-16T19:41:46.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.363+0000 7f65b8ff9640 1 -- 172.21.15.22:0/4108180507 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f65b0010380 con 0x7f65bc100100 2024-02-16T19:41:46.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.364+0000 7f65b8ff9640 1 --2- 172.21.15.22:0/4108180507 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f65a406d810 0x7f65a406fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:46.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.364+0000 7f65b8ff9640 1 -- 172.21.15.22:0/4108180507 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f65b008d040 con 0x7f65bc100100 2024-02-16T19:41:46.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.364+0000 7f65bb7fe640 1 --2- 172.21.15.22:0/4108180507 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f65a406d810 0x7f65a406fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:46.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.364+0000 7f65bb7fe640 1 --2- 172.21.15.22:0/4108180507 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f65a406d810 0x7f65a406fcc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f65ac006010 tx=0x7f65ac000bc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:46.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.368+0000 7f65b8ff9640 1 -- 172.21.15.22:0/4108180507 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f65b0056670 con 0x7f65bc100100 2024-02-16T19:41:46.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.554+0000 7f65c2118640 1 -- 172.21.15.22:0/4108180507 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f65bc10b7c0 con 0x7f65bc100100 2024-02-16T19:41:46.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.554+0000 7f65b8ff9640 1 -- 172.21.15.22:0/4108180507 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7f65b0059d30 con 0x7f65bc100100 2024-02-16T19:41:46.556 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-02-16T19:41:46.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.557+0000 7f65c2118640 1 -- 172.21.15.22:0/4108180507 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f65a406d810 msgr2=0x7f65a406fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:46.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.557+0000 7f65c2118640 1 --2- 172.21.15.22:0/4108180507 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f65a406d810 0x7f65a406fcc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f65ac006010 tx=0x7f65ac000bc0 comp rx=0 tx=0).stop 2024-02-16T19:41:46.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.557+0000 7f65c2118640 1 -- 172.21.15.22:0/4108180507 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f65bc100100 msgr2=0x7f65bc195a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:46.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.557+0000 7f65c2118640 1 --2- 172.21.15.22:0/4108180507 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f65bc100100 0x7f65bc195a00 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f65b000b420 tx=0x7f65b000b8e0 comp rx=0 tx=0).stop 2024-02-16T19:41:46.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.557+0000 7f65c2118640 1 -- 172.21.15.22:0/4108180507 shutdown_connections 2024-02-16T19:41:46.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.557+0000 7f65c2118640 1 --2- 172.21.15.22:0/4108180507 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f65a406d810 0x7f65a406fcc0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:46.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.557+0000 7f65c2118640 1 --2- 172.21.15.22:0/4108180507 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f65bc100100 0x7f65bc195a00 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:46.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.557+0000 7f65c2118640 1 --2- 172.21.15.22:0/4108180507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65bc0ff7f0 0x7f65bc1954c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:46.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.557+0000 7f65c2118640 1 -- 172.21.15.22:0/4108180507 >> 172.21.15.22:0/4108180507 conn(0x7f65bc074cf0 msgr2=0x7f65bc0fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:46.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.557+0000 7f65c2118640 1 -- 172.21.15.22:0/4108180507 shutdown_connections 2024-02-16T19:41:46.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.557+0000 7f65c2118640 1 -- 172.21.15.22:0/4108180507 wait complete. 2024-02-16T19:41:46.568 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:41:46 smithi022 ceph-mon[88798]: from='client.? 172.21.15.22:0/3260528065' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-02-16T19:41:46.568 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:41:46 smithi022 ceph-mon[88798]: from='client.? 172.21.15.22:0/861647110' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 36, "format": "json"}]: dispatch 2024-02-16T19:41:46.568 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:41:46 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:41:46.568 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:41:46 smithi022 ceph-mon[88798]: from='client.? 172.21.15.22:0/3758746209' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-16T19:41:46.569 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-02-16T19:41:46.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.673+0000 7f224e837640 1 -- 172.21.15.22:0/1295895593 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f22480feff0 msgr2=0x7f2248109a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:46.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.673+0000 7f224e837640 1 --2- 172.21.15.22:0/1295895593 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f22480feff0 0x7f2248109a60 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f223c009880 tx=0x7f223c02bd30 comp rx=0 tx=0).stop 2024-02-16T19:41:46.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.673+0000 7f224e837640 1 -- 172.21.15.22:0/1295895593 shutdown_connections 2024-02-16T19:41:46.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.673+0000 7f224e837640 1 --2- 172.21.15.22:0/1295895593 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f22480feff0 0x7f2248109a60 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:46.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.673+0000 7f224e837640 1 --2- 172.21.15.22:0/1295895593 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22480fe6e0 0x7f22480feab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:46.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.673+0000 7f224e837640 1 -- 172.21.15.22:0/1295895593 >> 172.21.15.22:0/1295895593 conn(0x7f2248074cf0 msgr2=0x7f22480750f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:46.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.673+0000 7f224e837640 1 -- 172.21.15.22:0/1295895593 shutdown_connections 2024-02-16T19:41:46.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.673+0000 7f224e837640 1 -- 172.21.15.22:0/1295895593 wait complete. 2024-02-16T19:41:46.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.673+0000 7f224e837640 1 Processor -- start 2024-02-16T19:41:46.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.673+0000 7f224e837640 1 -- start start 2024-02-16T19:41:46.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.674+0000 7f224e837640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22480fe6e0 0x7f2248195250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:46.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.674+0000 7f224e837640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f22480feff0 0x7f2248195790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:46.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.674+0000 7f224e837640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2248195e20 con 0x7f22480feff0 2024-02-16T19:41:46.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.674+0000 7f224e837640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2248199bf0 con 0x7f22480fe6e0 2024-02-16T19:41:46.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.674+0000 7f22477fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f22480feff0 0x7f2248195790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:46.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.674+0000 7f22477fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f22480feff0 0x7f2248195790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59864/0 (socket says 172.21.15.22:59864) 2024-02-16T19:41:46.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.674+0000 7f22477fe640 1 -- 172.21.15.22:0/3797245235 learned_addr learned my addr 172.21.15.22:0/3797245235 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:41:46.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.674+0000 7f2247fff640 1 --2- 172.21.15.22:0/3797245235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22480fe6e0 0x7f2248195250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:46.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.674+0000 7f22477fe640 1 -- 172.21.15.22:0/3797245235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22480fe6e0 msgr2=0x7f2248195250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:46.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.674+0000 7f22477fe640 1 --2- 172.21.15.22:0/3797245235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22480fe6e0 0x7f2248195250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:46.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.674+0000 7f22477fe640 1 -- 172.21.15.22:0/3797245235 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f223c009560 con 0x7f22480feff0 2024-02-16T19:41:46.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.674+0000 7f2247fff640 1 --2- 172.21.15.22:0/3797245235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22480fe6e0 0x7f2248195250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T19:41:46.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.674+0000 7f22477fe640 1 --2- 172.21.15.22:0/3797245235 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f22480feff0 0x7f2248195790 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f223c009230 tx=0x7f223c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:46.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.675+0000 7f22457fa640 1 -- 172.21.15.22:0/3797245235 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f223c0037c0 con 0x7f22480feff0 2024-02-16T19:41:46.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.675+0000 7f22457fa640 1 -- 172.21.15.22:0/3797245235 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f223c005d50 con 0x7f22480feff0 2024-02-16T19:41:46.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.675+0000 7f224e837640 1 -- 172.21.15.22:0/3797245235 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2248199e70 con 0x7f22480feff0 2024-02-16T19:41:46.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.675+0000 7f22457fa640 1 -- 172.21.15.22:0/3797245235 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f223c034440 con 0x7f22480feff0 2024-02-16T19:41:46.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.675+0000 7f224e837640 1 -- 172.21.15.22:0/3797245235 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f224819a450 con 0x7f22480feff0 2024-02-16T19:41:46.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.676+0000 7f224e837640 1 -- 172.21.15.22:0/3797245235 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f224804e000 con 0x7f22480feff0 2024-02-16T19:41:46.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.676+0000 7f22457fa640 1 -- 172.21.15.22:0/3797245235 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f223c004f60 con 0x7f22480feff0 2024-02-16T19:41:46.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.676+0000 7f22457fa640 1 --2- 172.21.15.22:0/3797245235 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f223006d810 0x7f223006fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:46.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.676+0000 7f22457fa640 1 -- 172.21.15.22:0/3797245235 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f223c0afac0 con 0x7f22480feff0 2024-02-16T19:41:46.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.676+0000 7f2247fff640 1 --2- 172.21.15.22:0/3797245235 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f223006d810 0x7f223006fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:46.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.677+0000 7f2247fff640 1 --2- 172.21.15.22:0/3797245235 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f223006d810 0x7f223006fcc0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f22380058c0 tx=0x7f2238005850 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:46.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.681+0000 7f22457fa640 1 -- 172.21.15.22:0/3797245235 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f223c0790f0 con 0x7f22480feff0 2024-02-16T19:41:46.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.861+0000 7f224e837640 1 -- 172.21.15.22:0/3797245235 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f22481070d0 con 0x7f22480feff0 2024-02-16T19:41:46.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.861+0000 7f22457fa640 1 -- 172.21.15.22:0/3797245235 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v93) v1 ==== 107+0+11 (secure 0 0 0) 0x7f223c07c7b0 con 0x7f22480feff0 2024-02-16T19:41:46.863 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-02-16T19:41:46.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.863+0000 7f224e837640 1 -- 172.21.15.22:0/3797245235 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f223006d810 msgr2=0x7f223006fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:46.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.863+0000 7f224e837640 1 --2- 172.21.15.22:0/3797245235 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f223006d810 0x7f223006fcc0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f22380058c0 tx=0x7f2238005850 comp rx=0 tx=0).stop 2024-02-16T19:41:46.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.863+0000 7f224e837640 1 -- 172.21.15.22:0/3797245235 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f22480feff0 msgr2=0x7f2248195790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:46.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.863+0000 7f224e837640 1 --2- 172.21.15.22:0/3797245235 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f22480feff0 0x7f2248195790 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f223c009230 tx=0x7f223c004270 comp rx=0 tx=0).stop 2024-02-16T19:41:46.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.863+0000 7f224e837640 1 -- 172.21.15.22:0/3797245235 shutdown_connections 2024-02-16T19:41:46.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.863+0000 7f224e837640 1 --2- 172.21.15.22:0/3797245235 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f223006d810 0x7f223006fcc0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:46.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.863+0000 7f224e837640 1 --2- 172.21.15.22:0/3797245235 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f22480feff0 0x7f2248195790 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:46.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.863+0000 7f224e837640 1 --2- 172.21.15.22:0/3797245235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22480fe6e0 0x7f2248195250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:46.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.863+0000 7f224e837640 1 -- 172.21.15.22:0/3797245235 >> 172.21.15.22:0/3797245235 conn(0x7f2248074cf0 msgr2=0x7f22481042a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:46.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.863+0000 7f224e837640 1 -- 172.21.15.22:0/3797245235 shutdown_connections 2024-02-16T19:41:46.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.863+0000 7f224e837640 1 -- 172.21.15.22:0/3797245235 wait complete. 2024-02-16T19:41:46.876 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-02-16T19:41:46.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:41:46 smithi117 ceph-mon[67870]: from='client.? 172.21.15.22:0/3260528065' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-02-16T19:41:46.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:41:46 smithi117 ceph-mon[67870]: from='client.? 172.21.15.22:0/861647110' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 36, "format": "json"}]: dispatch 2024-02-16T19:41:46.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:41:46 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:41:46.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:41:46 smithi117 ceph-mon[67870]: from='client.? 172.21.15.22:0/3758746209' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-16T19:41:46.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.981+0000 7f5aebe41640 1 -- 172.21.15.22:0/4222505458 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5ae40fe5b0 msgr2=0x7f5ae40fea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:46.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.981+0000 7f5aebe41640 1 --2- 172.21.15.22:0/4222505458 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5ae40fe5b0 0x7f5ae40fea20 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f5ad8009880 tx=0x7f5ad802bce0 comp rx=0 tx=0).stop 2024-02-16T19:41:46.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.981+0000 7f5aebe41640 1 -- 172.21.15.22:0/4222505458 shutdown_connections 2024-02-16T19:41:46.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.981+0000 7f5aebe41640 1 --2- 172.21.15.22:0/4222505458 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5ae40fe5b0 0x7f5ae40fea20 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:46.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.981+0000 7f5aebe41640 1 --2- 172.21.15.22:0/4222505458 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ae4104590 0x7f5ae4104960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:46.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.981+0000 7f5aebe41640 1 -- 172.21.15.22:0/4222505458 >> 172.21.15.22:0/4222505458 conn(0x7f5ae40743d0 msgr2=0x7f5ae40747d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:46.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.981+0000 7f5aebe41640 1 -- 172.21.15.22:0/4222505458 shutdown_connections 2024-02-16T19:41:46.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.981+0000 7f5aebe41640 1 -- 172.21.15.22:0/4222505458 wait complete. 2024-02-16T19:41:46.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.981+0000 7f5aebe41640 1 Processor -- start 2024-02-16T19:41:46.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.981+0000 7f5aebe41640 1 -- start start 2024-02-16T19:41:46.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.981+0000 7f5aebe41640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ae40fe5b0 0x7f5ae4193ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:46.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.982+0000 7f5aebe41640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5ae4104590 0x7f5ae4194420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:46.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.982+0000 7f5aebe41640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae4194960 con 0x7f5ae4104590 2024-02-16T19:41:46.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.982+0000 7f5aebe41640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae4194ad0 con 0x7f5ae40fe5b0 2024-02-16T19:41:46.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.982+0000 7f5ae93dc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5ae4104590 0x7f5ae4194420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:46.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.982+0000 7f5ae93dc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5ae4104590 0x7f5ae4194420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59878/0 (socket says 172.21.15.22:59878) 2024-02-16T19:41:46.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.982+0000 7f5ae93dc640 1 -- 172.21.15.22:0/468504957 learned_addr learned my addr 172.21.15.22:0/468504957 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:41:46.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.982+0000 7f5ae93dc640 1 -- 172.21.15.22:0/468504957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ae40fe5b0 msgr2=0x7f5ae4193ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:46.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.982+0000 7f5ae9bdd640 1 --2- 172.21.15.22:0/468504957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ae40fe5b0 0x7f5ae4193ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:46.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.982+0000 7f5ae93dc640 1 --2- 172.21.15.22:0/468504957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ae40fe5b0 0x7f5ae4193ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:46.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.982+0000 7f5ae93dc640 1 -- 172.21.15.22:0/468504957 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ad8009560 con 0x7f5ae4104590 2024-02-16T19:41:46.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.982+0000 7f5ae9bdd640 1 --2- 172.21.15.22:0/468504957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ae40fe5b0 0x7f5ae4193ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-16T19:41:46.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.982+0000 7f5ae93dc640 1 --2- 172.21.15.22:0/468504957 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5ae4104590 0x7f5ae4194420 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5ad8009230 tx=0x7f5ad8004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:46.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.982+0000 7f5ae2ffd640 1 -- 172.21.15.22:0/468504957 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ad80035f0 con 0x7f5ae4104590 2024-02-16T19:41:46.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.982+0000 7f5ae2ffd640 1 -- 172.21.15.22:0/468504957 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5ad8003c10 con 0x7f5ae4104590 2024-02-16T19:41:46.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.982+0000 7f5aebe41640 1 -- 172.21.15.22:0/468504957 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ae4194d50 con 0x7f5ae4104590 2024-02-16T19:41:46.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.982+0000 7f5ae2ffd640 1 -- 172.21.15.22:0/468504957 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ad8034440 con 0x7f5ae4104590 2024-02-16T19:41:46.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.982+0000 7f5aebe41640 1 -- 172.21.15.22:0/468504957 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ae41a1860 con 0x7f5ae4104590 2024-02-16T19:41:46.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.983+0000 7f5aebe41640 1 -- 172.21.15.22:0/468504957 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ae404e000 con 0x7f5ae4104590 2024-02-16T19:41:46.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.984+0000 7f5ae2ffd640 1 -- 172.21.15.22:0/468504957 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f5ad8004fd0 con 0x7f5ae4104590 2024-02-16T19:41:46.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.984+0000 7f5ae2ffd640 1 --2- 172.21.15.22:0/468504957 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f5acc06d810 0x7f5acc06fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:46.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.984+0000 7f5ae2ffd640 1 -- 172.21.15.22:0/468504957 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f5ad80af9a0 con 0x7f5ae4104590 2024-02-16T19:41:46.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.984+0000 7f5ae9bdd640 1 --2- 172.21.15.22:0/468504957 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f5acc06d810 0x7f5acc06fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:46.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.985+0000 7f5ae9bdd640 1 --2- 172.21.15.22:0/468504957 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f5acc06d810 0x7f5acc06fcc0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5adc0058c0 tx=0x7f5adc005d30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:46.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:46.989+0000 7f5ae2ffd640 1 -- 172.21.15.22:0/468504957 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5ad8078fb0 con 0x7f5ae4104590 2024-02-16T19:41:47.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.167+0000 7f5aebe41640 1 -- 172.21.15.22:0/468504957 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f5ae410b9c0 con 0x7f5ae4104590 2024-02-16T19:41:47.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.168+0000 7f5ae2ffd640 1 -- 172.21.15.22:0/468504957 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v93) v1 ==== 107+0+11 (secure 0 0 0) 0x7f5ad807c670 con 0x7f5ae4104590 2024-02-16T19:41:47.169 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-02-16T19:41:47.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.170+0000 7f5aebe41640 1 -- 172.21.15.22:0/468504957 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f5acc06d810 msgr2=0x7f5acc06fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:47.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.170+0000 7f5aebe41640 1 --2- 172.21.15.22:0/468504957 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f5acc06d810 0x7f5acc06fcc0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5adc0058c0 tx=0x7f5adc005d30 comp rx=0 tx=0).stop 2024-02-16T19:41:47.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.170+0000 7f5aebe41640 1 -- 172.21.15.22:0/468504957 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5ae4104590 msgr2=0x7f5ae4194420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:47.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.170+0000 7f5aebe41640 1 --2- 172.21.15.22:0/468504957 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5ae4104590 0x7f5ae4194420 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5ad8009230 tx=0x7f5ad8004040 comp rx=0 tx=0).stop 2024-02-16T19:41:47.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.170+0000 7f5aebe41640 1 -- 172.21.15.22:0/468504957 shutdown_connections 2024-02-16T19:41:47.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.170+0000 7f5aebe41640 1 --2- 172.21.15.22:0/468504957 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f5acc06d810 0x7f5acc06fcc0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:47.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.170+0000 7f5aebe41640 1 --2- 172.21.15.22:0/468504957 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5ae4104590 0x7f5ae4194420 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:47.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.170+0000 7f5aebe41640 1 --2- 172.21.15.22:0/468504957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ae40fe5b0 0x7f5ae4193ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:47.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.170+0000 7f5aebe41640 1 -- 172.21.15.22:0/468504957 >> 172.21.15.22:0/468504957 conn(0x7f5ae40743d0 msgr2=0x7f5ae40fcdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:47.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.170+0000 7f5aebe41640 1 -- 172.21.15.22:0/468504957 shutdown_connections 2024-02-16T19:41:47.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.170+0000 7f5aebe41640 1 -- 172.21.15.22:0/468504957 wait complete. 2024-02-16T19:41:47.182 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-16T19:41:47.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.284+0000 7f0fc78c4640 1 -- 172.21.15.22:0/3025091886 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0fc00fe590 msgr2=0x7f0fc00fea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:47.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.284+0000 7f0fc78c4640 1 --2- 172.21.15.22:0/3025091886 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0fc00fe590 0x7f0fc00fea00 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0fb0009880 tx=0x7f0fb002bce0 comp rx=0 tx=0).stop 2024-02-16T19:41:47.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.284+0000 7f0fc78c4640 1 -- 172.21.15.22:0/3025091886 shutdown_connections 2024-02-16T19:41:47.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.284+0000 7f0fc78c4640 1 --2- 172.21.15.22:0/3025091886 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0fc00fe590 0x7f0fc00fea00 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:47.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.284+0000 7f0fc78c4640 1 --2- 172.21.15.22:0/3025091886 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fc01045c0 0x7f0fc0104990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:47.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.284+0000 7f0fc78c4640 1 -- 172.21.15.22:0/3025091886 >> 172.21.15.22:0/3025091886 conn(0x7f0fc00fa0b0 msgr2=0x7f0fc00fc4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:47.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.284+0000 7f0fc78c4640 1 -- 172.21.15.22:0/3025091886 shutdown_connections 2024-02-16T19:41:47.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.284+0000 7f0fc78c4640 1 -- 172.21.15.22:0/3025091886 wait complete. 2024-02-16T19:41:47.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.284+0000 7f0fc78c4640 1 Processor -- start 2024-02-16T19:41:47.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.284+0000 7f0fc78c4640 1 -- start start 2024-02-16T19:41:47.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.284+0000 7f0fc78c4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fc00fe590 0x7f0fc01940a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:47.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.284+0000 7f0fc78c4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0fc01045c0 0x7f0fc01945e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:47.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.284+0000 7f0fc78c4640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc0194c70 con 0x7f0fc01045c0 2024-02-16T19:41:47.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.284+0000 7f0fc78c4640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc0198a40 con 0x7f0fc00fe590 2024-02-16T19:41:47.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.285+0000 7f0fc4e5f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0fc01045c0 0x7f0fc01945e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:47.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.285+0000 7f0fc4e5f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0fc01045c0 0x7f0fc01945e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59900/0 (socket says 172.21.15.22:59900) 2024-02-16T19:41:47.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.285+0000 7f0fc4e5f640 1 -- 172.21.15.22:0/3606484216 learned_addr learned my addr 172.21.15.22:0/3606484216 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:41:47.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.285+0000 7f0fc5660640 1 --2- 172.21.15.22:0/3606484216 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fc00fe590 0x7f0fc01940a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:47.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.285+0000 7f0fc4e5f640 1 -- 172.21.15.22:0/3606484216 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fc00fe590 msgr2=0x7f0fc01940a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:47.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.285+0000 7f0fc4e5f640 1 --2- 172.21.15.22:0/3606484216 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fc00fe590 0x7f0fc01940a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:47.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.285+0000 7f0fc4e5f640 1 -- 172.21.15.22:0/3606484216 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fb0009560 con 0x7f0fc01045c0 2024-02-16T19:41:47.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.285+0000 7f0fc4e5f640 1 --2- 172.21.15.22:0/3606484216 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0fc01045c0 0x7f0fc01945e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0fb0009230 tx=0x7f0fb0005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:47.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.285+0000 7f0fb67fc640 1 -- 172.21.15.22:0/3606484216 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0fb00044a0 con 0x7f0fc01045c0 2024-02-16T19:41:47.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.285+0000 7f0fb67fc640 1 -- 172.21.15.22:0/3606484216 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0fb0003ab0 con 0x7f0fc01045c0 2024-02-16T19:41:47.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.285+0000 7f0fc78c4640 1 -- 172.21.15.22:0/3606484216 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fc0198cc0 con 0x7f0fc01045c0 2024-02-16T19:41:47.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.285+0000 7f0fb67fc640 1 -- 172.21.15.22:0/3606484216 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0fb00055f0 con 0x7f0fc01045c0 2024-02-16T19:41:47.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.285+0000 7f0fc78c4640 1 -- 172.21.15.22:0/3606484216 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fc0199240 con 0x7f0fc01045c0 2024-02-16T19:41:47.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.286+0000 7f0fc78c4640 1 -- 172.21.15.22:0/3606484216 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fc004e000 con 0x7f0fc01045c0 2024-02-16T19:41:47.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.287+0000 7f0fb67fc640 1 -- 172.21.15.22:0/3606484216 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f0fb0004e60 con 0x7f0fc01045c0 2024-02-16T19:41:47.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.287+0000 7f0fb67fc640 1 --2- 172.21.15.22:0/3606484216 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f0fac06d8d0 0x7f0fac06fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:47.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.287+0000 7f0fb67fc640 1 -- 172.21.15.22:0/3606484216 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f0fb00b18d0 con 0x7f0fc01045c0 2024-02-16T19:41:47.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.287+0000 7f0fc5660640 1 --2- 172.21.15.22:0/3606484216 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f0fac06d8d0 0x7f0fac06fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:47.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.288+0000 7f0fc5660640 1 --2- 172.21.15.22:0/3606484216 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f0fac06d8d0 0x7f0fac06fd80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc009b60 tx=0x7f0fbc005f90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:47.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.292+0000 7f0fb67fc640 1 -- 172.21.15.22:0/3606484216 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0fb007af00 con 0x7f0fc01045c0 2024-02-16T19:41:47.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:41:47 smithi022 ceph-mon[88798]: pgmap v301: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:41:47.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:41:47 smithi022 ceph-mon[88798]: from='client.? 172.21.15.22:0/4108180507' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-16T19:41:47.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:41:47 smithi022 ceph-mon[88798]: from='client.? 172.21.15.22:0/3797245235' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-02-16T19:41:47.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:41:47 smithi022 ceph-mon[88798]: from='client.? 172.21.15.22:0/468504957' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-02-16T19:41:47.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.486+0000 7f0fc78c4640 1 -- 172.21.15.22:0/3606484216 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f0fc01953f0 con 0x7f0fc01045c0 2024-02-16T19:41:47.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.486+0000 7f0fb67fc640 1 -- 172.21.15.22:0/3606484216 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 43 entries v93) v1 ==== 81+0+2550 (secure 0 0 0) 0x7f0fb007e5c0 con 0x7f0fc01045c0 2024-02-16T19:41:47.487 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.117:6827/3122342028 2024-02-17T19:38:57.804505+0000 2024-02-16T19:41:47.487 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.117:6826/3122342028 2024-02-17T19:38:57.804505+0000 2024-02-16T19:41:47.488 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:6829/3597685751 2024-02-17T19:38:09.297300+0000 2024-02-16T19:41:47.488 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:6828/3597685751 2024-02-17T19:38:09.297300+0000 2024-02-16T19:41:47.488 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/4238358751 2024-02-17T19:33:15.748281+0000 2024-02-16T19:41:47.488 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:6801/2755289428 2024-02-17T19:20:11.619146+0000 2024-02-16T19:41:47.488 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:6800/2253867279 2024-02-17T19:28:59.405724+0000 2024-02-16T19:41:47.488 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/2301835367 2024-02-17T19:28:59.405724+0000 2024-02-16T19:41:47.488 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:6801/3109015836 2024-02-17T19:19:38.544291+0000 2024-02-16T19:41:47.488 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.117:6825/565408102 2024-02-17T19:38:31.770686+0000 2024-02-16T19:41:47.488 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/206933569 2024-02-17T19:28:59.405724+0000 2024-02-16T19:41:47.488 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/3012142271 2024-02-17T19:33:15.748281+0000 2024-02-16T19:41:47.488 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.117:0/2453907739 2024-02-17T19:29:53.194933+0000 2024-02-16T19:41:47.488 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:6800/3109015836 2024-02-17T19:19:38.544291+0000 2024-02-16T19:41:47.488 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.117:0/3042073611 2024-02-17T19:29:53.194933+0000 2024-02-16T19:41:47.488 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.117:0/4194262516 2024-02-17T19:29:53.194933+0000 2024-02-16T19:41:47.489 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:6800/2945983569 2024-02-17T19:21:26.760165+0000 2024-02-16T19:41:47.489 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/1603216752 2024-02-17T19:19:38.544291+0000 2024-02-16T19:41:47.489 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:6827/149959963 2024-02-17T19:26:32.932573+0000 2024-02-16T19:41:47.489 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:6801/2945983569 2024-02-17T19:21:26.760165+0000 2024-02-16T19:41:47.489 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/668279955 2024-02-17T19:21:26.760165+0000 2024-02-16T19:41:47.489 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/2705605267 2024-02-17T19:28:59.405724+0000 2024-02-16T19:41:47.489 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.117:6824/565408102 2024-02-17T19:38:31.770686+0000 2024-02-16T19:41:47.489 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:6800/2755289428 2024-02-17T19:20:11.619146+0000 2024-02-16T19:41:47.489 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/2099601555 2024-02-17T19:20:11.619146+0000 2024-02-16T19:41:47.489 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:6801/2253867279 2024-02-17T19:28:59.405724+0000 2024-02-16T19:41:47.489 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.117:0/333707173 2024-02-17T19:29:53.194933+0000 2024-02-16T19:41:47.489 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.117:6828/10500008 2024-02-17T19:29:53.194933+0000 2024-02-16T19:41:47.489 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.117:0/199482916 2024-02-17T19:29:53.194933+0000 2024-02-16T19:41:47.489 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.117:6829/10500008 2024-02-17T19:29:53.194933+0000 2024-02-16T19:41:47.489 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/2017591724 2024-02-17T19:20:11.619146+0000 2024-02-16T19:41:47.489 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/2371245756 2024-02-17T19:19:38.544291+0000 2024-02-16T19:41:47.490 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:6801/3703806543 2024-02-17T19:33:15.748281+0000 2024-02-16T19:41:47.490 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/2384083375 2024-02-17T19:21:26.760165+0000 2024-02-16T19:41:47.490 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/402850490 2024-02-17T19:28:59.405724+0000 2024-02-16T19:41:47.490 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/1229330195 2024-02-17T19:20:11.619146+0000 2024-02-16T19:41:47.490 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/8466130 2024-02-17T19:19:38.544291+0000 2024-02-16T19:41:47.490 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/3200629226 2024-02-17T19:21:26.760165+0000 2024-02-16T19:41:47.490 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.117:0/2264991341 2024-02-17T19:29:53.194933+0000 2024-02-16T19:41:47.490 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/2853892105 2024-02-17T19:33:15.748281+0000 2024-02-16T19:41:47.490 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:6826/149959963 2024-02-17T19:26:32.932573+0000 2024-02-16T19:41:47.490 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:6800/3703806543 2024-02-17T19:33:15.748281+0000 2024-02-16T19:41:47.490 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/2353857756 2024-02-17T19:33:15.748281+0000 2024-02-16T19:41:47.491 INFO:teuthology.orchestra.run.smithi022.stderr:listed 43 entries 2024-02-16T19:41:47.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.488+0000 7f0fc78c4640 1 -- 172.21.15.22:0/3606484216 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f0fac06d8d0 msgr2=0x7f0fac06fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:47.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.488+0000 7f0fc78c4640 1 --2- 172.21.15.22:0/3606484216 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f0fac06d8d0 0x7f0fac06fd80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc009b60 tx=0x7f0fbc005f90 comp rx=0 tx=0).stop 2024-02-16T19:41:47.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.488+0000 7f0fc78c4640 1 -- 172.21.15.22:0/3606484216 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0fc01045c0 msgr2=0x7f0fc01945e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:47.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.488+0000 7f0fc78c4640 1 --2- 172.21.15.22:0/3606484216 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0fc01045c0 0x7f0fc01945e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0fb0009230 tx=0x7f0fb0005e50 comp rx=0 tx=0).stop 2024-02-16T19:41:47.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.488+0000 7f0fc78c4640 1 -- 172.21.15.22:0/3606484216 shutdown_connections 2024-02-16T19:41:47.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.488+0000 7f0fc78c4640 1 --2- 172.21.15.22:0/3606484216 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f0fac06d8d0 0x7f0fac06fd80 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:47.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.488+0000 7f0fc78c4640 1 --2- 172.21.15.22:0/3606484216 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0fc01045c0 0x7f0fc01945e0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:47.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.488+0000 7f0fc78c4640 1 --2- 172.21.15.22:0/3606484216 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fc00fe590 0x7f0fc01940a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:47.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.488+0000 7f0fc78c4640 1 -- 172.21.15.22:0/3606484216 >> 172.21.15.22:0/3606484216 conn(0x7f0fc00fa0b0 msgr2=0x7f0fc00fb870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:47.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.488+0000 7f0fc78c4640 1 -- 172.21.15.22:0/3606484216 shutdown_connections 2024-02-16T19:41:47.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.488+0000 7f0fc78c4640 1 -- 172.21.15.22:0/3606484216 wait complete. 2024-02-16T19:41:47.501 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-02-16T19:41:47.501 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-16T19:41:47.517 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-16T19:41:47.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.658+0000 7f897d6bd640 1 -- 172.21.15.22:0/415370983 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8978100850 msgr2=0x7f8978100c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:47.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.658+0000 7f897d6bd640 1 --2- 172.21.15.22:0/415370983 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8978100850 0x7f8978100c20 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f89700098b0 tx=0x7f897002bd60 comp rx=0 tx=0).stop 2024-02-16T19:41:47.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.658+0000 7f897d6bd640 1 -- 172.21.15.22:0/415370983 shutdown_connections 2024-02-16T19:41:47.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.658+0000 7f897d6bd640 1 --2- 172.21.15.22:0/415370983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8978101160 0x7f8978109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:47.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.658+0000 7f897d6bd640 1 --2- 172.21.15.22:0/415370983 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8978100850 0x7f8978100c20 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:47.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.658+0000 7f897d6bd640 1 -- 172.21.15.22:0/415370983 >> 172.21.15.22:0/415370983 conn(0x7f8978075270 msgr2=0x7f8978075670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:47.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.659+0000 7f897d6bd640 1 -- 172.21.15.22:0/415370983 shutdown_connections 2024-02-16T19:41:47.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.659+0000 7f897d6bd640 1 -- 172.21.15.22:0/415370983 wait complete. 2024-02-16T19:41:47.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.659+0000 7f897d6bd640 1 Processor -- start 2024-02-16T19:41:47.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.659+0000 7f897d6bd640 1 -- start start 2024-02-16T19:41:47.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.659+0000 7f897d6bd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8978100850 0x7f89781952b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:47.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.659+0000 7f897d6bd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8978101160 0x7f89781957f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:47.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.659+0000 7f897d6bd640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8978195e80 con 0x7f8978101160 2024-02-16T19:41:47.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.659+0000 7f897d6bd640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8978199c50 con 0x7f8978100850 2024-02-16T19:41:47.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.660+0000 7f89767fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8978101160 0x7f89781957f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:47.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.660+0000 7f89767fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8978101160 0x7f89781957f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59924/0 (socket says 172.21.15.22:59924) 2024-02-16T19:41:47.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.660+0000 7f89767fc640 1 -- 172.21.15.22:0/4142435882 learned_addr learned my addr 172.21.15.22:0/4142435882 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:41:47.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.660+0000 7f8976ffd640 1 --2- 172.21.15.22:0/4142435882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8978100850 0x7f89781952b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:47.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.660+0000 7f89767fc640 1 -- 172.21.15.22:0/4142435882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8978100850 msgr2=0x7f89781952b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:47.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.660+0000 7f89767fc640 1 --2- 172.21.15.22:0/4142435882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8978100850 0x7f89781952b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:47.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.660+0000 7f89767fc640 1 -- 172.21.15.22:0/4142435882 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8970009560 con 0x7f8978101160 2024-02-16T19:41:47.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.660+0000 7f89767fc640 1 --2- 172.21.15.22:0/4142435882 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8978101160 0x7f89781957f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f896000c8f0 tx=0x7f896000cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:47.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.660+0000 7f896ffff640 1 -- 172.21.15.22:0/4142435882 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8960007b80 con 0x7f8978101160 2024-02-16T19:41:47.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.660+0000 7f896ffff640 1 -- 172.21.15.22:0/4142435882 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f89600054e0 con 0x7f8978101160 2024-02-16T19:41:47.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.660+0000 7f897d6bd640 1 -- 172.21.15.22:0/4142435882 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8978199f30 con 0x7f8978101160 2024-02-16T19:41:47.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.660+0000 7f896ffff640 1 -- 172.21.15.22:0/4142435882 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f896000f450 con 0x7f8978101160 2024-02-16T19:41:47.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.660+0000 7f897d6bd640 1 -- 172.21.15.22:0/4142435882 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f897819a490 con 0x7f8978101160 2024-02-16T19:41:47.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.661+0000 7f897d6bd640 1 -- 172.21.15.22:0/4142435882 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f897804e000 con 0x7f8978101160 2024-02-16T19:41:47.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.662+0000 7f896ffff640 1 -- 172.21.15.22:0/4142435882 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f8960004f60 con 0x7f8978101160 2024-02-16T19:41:47.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.662+0000 7f896ffff640 1 --2- 172.21.15.22:0/4142435882 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f896406d8d0 0x7f896406fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:47.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.662+0000 7f896ffff640 1 -- 172.21.15.22:0/4142435882 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f896008d420 con 0x7f8978101160 2024-02-16T19:41:47.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.662+0000 7f8976ffd640 1 --2- 172.21.15.22:0/4142435882 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f896406d8d0 0x7f896406fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:47.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.663+0000 7f8976ffd640 1 --2- 172.21.15.22:0/4142435882 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f896406d8d0 0x7f896406fd80 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8970000c00 tx=0x7f8970005fd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:47.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.667+0000 7f896ffff640 1 -- 172.21.15.22:0/4142435882 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8960056b00 con 0x7f8978101160 2024-02-16T19:41:47.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.848+0000 7f897d6bd640 1 -- 172.21.15.22:0/4142435882 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f897810c880 con 0x7f8978101160 2024-02-16T19:41:47.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.849+0000 7f896ffff640 1 -- 172.21.15.22:0/4142435882 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+17322 (secure 0 0 0) 0x7f896005a1c0 con 0x7f8978101160 2024-02-16T19:41:47.851 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-16T19:41:47.851 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":93,"fsid":"2a1346e6-cd00-11ee-95ba-87774f69a715","created":"2024-02-16T19:19:11.528280+0000","modified":"2024-02-16T19:38:57.804537+0000","last_up_change":"2024-02-16T19:37:45.909206+0000","last_in_change":"2024-02-16T19:24:40.575373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-16T19:23:47.101426+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-02-16T19:26:16.533404+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-02-16T19:26:17.113076+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d16e47f2-7209-45f4-ad50-1e0f6789ffad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":50,"up_from":54,"up_thru":88,"down_at":51,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":1744550481},{"type":"v1","addr":"172.21.15.22:6803","nonce":1744550481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":1744550481},{"type":"v1","addr":"172.21.15.22:6805","nonce":1744550481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":1744550481},{"type":"v1","addr":"172.21.15.22:6809","nonce":1744550481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":1744550481},{"type":"v1","addr":"172.21.15.22:6807","nonce":1744550481}]},"public_addr":"172.21.15.22:6803/1744550481","cluster_addr":"172.21.15.22:6805/1744550481","heartbeat_back_addr":"172.21.15.22:6809/1744550481","heartbeat_front_addr":"172.21.15.22:6807/1744550481","state":["exists","up"]},{"osd":1,"uuid":"59c3d085-3422-4e92-9048-46d7a39e31a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":55,"up_from":64,"up_thru":88,"down_at":56,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":3321991378},{"type":"v1","addr":"172.21.15.22:6811","nonce":3321991378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":3321991378},{"type":"v1","addr":"172.21.15.22:6813","nonce":3321991378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":3321991378},{"type":"v1","addr":"172.21.15.22:6817","nonce":3321991378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":3321991378},{"type":"v1","addr":"172.21.15.22:6815","nonce":3321991378}]},"public_addr":"172.21.15.22:6811/3321991378","cluster_addr":"172.21.15.22:6813/3321991378","heartbeat_back_addr":"172.21.15.22:6817/3321991378","heartbeat_front_addr":"172.21.15.22:6815/3321991378","state":["exists","up"]},{"osd":2,"uuid":"5952e710-f0b6-406c-855a-00f1cb8059c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":69,"up_from":73,"up_thru":88,"down_at":70,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":2098766749},{"type":"v1","addr":"172.21.15.22:6819","nonce":2098766749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":2098766749},{"type":"v1","addr":"172.21.15.22:6821","nonce":2098766749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":2098766749},{"type":"v1","addr":"172.21.15.22:6825","nonce":2098766749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":2098766749},{"type":"v1","addr":"172.21.15.22:6823","nonce":2098766749}]},"public_addr":"172.21.15.22:6819/2098766749","cluster_addr":"172.21.15.22:6821/2098766749","heartbeat_back_addr":"172.21.15.22:6825/2098766749","heartbeat_front_addr":"172.21.15.22:6823/2098766749","state":["exists","up"]},{"osd":3,"uuid":"5dc87494-e400-4111-91a7-306d9905febb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":27,"last_clean_end":74,"up_from":78,"up_thru":88,"down_at":75,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3703365528},{"type":"v1","addr":"172.21.15.117:6801","nonce":3703365528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":3703365528},{"type":"v1","addr":"172.21.15.117:6803","nonce":3703365528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":3703365528},{"type":"v1","addr":"172.21.15.117:6807","nonce":3703365528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":3703365528},{"type":"v1","addr":"172.21.15.117:6805","nonce":3703365528}]},"public_addr":"172.21.15.117:6801/3703365528","cluster_addr":"172.21.15.117:6803/3703365528","heartbeat_back_addr":"172.21.15.117:6807/3703365528","heartbeat_front_addr":"172.21.15.117:6805/3703365528","state":["exists","up"]},{"osd":4,"uuid":"d27bde8d-1a77-4f95-acb6-7d05becd9480","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":35,"last_clean_end":79,"up_from":83,"up_thru":88,"down_at":80,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":3679983685},{"type":"v1","addr":"172.21.15.117:6809","nonce":3679983685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":3679983685},{"type":"v1","addr":"172.21.15.117:6811","nonce":3679983685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":3679983685},{"type":"v1","addr":"172.21.15.117:6815","nonce":3679983685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":3679983685},{"type":"v1","addr":"172.21.15.117:6813","nonce":3679983685}]},"public_addr":"172.21.15.117:6809/3679983685","cluster_addr":"172.21.15.117:6811/3679983685","heartbeat_back_addr":"172.21.15.117:6815/3679983685","heartbeat_front_addr":"172.21.15.117:6813/3679983685","state":["exists","up"]},{"osd":5,"uuid":"5873798f-3a2c-4a29-becd-6280b8866b6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":40,"last_clean_end":84,"up_from":88,"up_thru":88,"down_at":85,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1403457815},{"type":"v1","addr":"172.21.15.117:6817","nonce":1403457815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1403457815},{"type":"v1","addr":"172.21.15.117:6819","nonce":1403457815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1403457815},{"type":"v1","addr":"172.21.15.117:6823","nonce":1403457815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1403457815},{"type":"v1","addr":"172.21.15.117:6821","nonce":1403457815}]},"public_addr":"172.21.15.117:6817/1403457815","cluster_addr":"172.21.15.117:6819/1403457815","heartbeat_back_addr":"172.21.15.117:6823/1403457815","heartbeat_front_addr":"172.21.15.117:6821/1403457815","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:22:44.777427+0000","dead_epoch":50},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:23:14.935167+0000","dead_epoch":55},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:23:44.496184+0000","dead_epoch":69},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:24:07.447481+0000","dead_epoch":74},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:24:32.205064+0000","dead_epoch":79},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:24:57.982560+0000","dead_epoch":84}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":3,"to":2}]},{"pgid":"2.5","mappings":[{"from":3,"to":1}]},{"pgid":"2.6","mappings":[{"from":3,"to":2}]},{"pgid":"2.9","mappings":[{"from":0,"to":2}]},{"pgid":"2.10","mappings":[{"from":0,"to":5}]},{"pgid":"2.1e","mappings":[{"from":0,"to":1}]},{"pgid":"3.3","mappings":[{"from":4,"to":1}]},{"pgid":"3.f","mappings":[{"from":3,"to":1}]},{"pgid":"3.16","mappings":[{"from":3,"to":0}]},{"pgid":"3.1b","mappings":[{"from":4,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.117:6827/3122342028":"2024-02-17T19:38:57.804505+0000","172.21.15.117:6826/3122342028":"2024-02-17T19:38:57.804505+0000","172.21.15.22:6829/3597685751":"2024-02-17T19:38:09.297300+0000","172.21.15.22:6828/3597685751":"2024-02-17T19:38:09.297300+0000","172.21.15.22:0/4238358751":"2024-02-17T19:33:15.748281+0000","172.21.15.22:6801/2755289428":"2024-02-17T19:20:11.619146+0000","172.21.15.22:6800/2253867279":"2024-02-17T19:28:59.405724+0000","172.21.15.22:0/2301835367":"2024-02-17T19:28:59.405724+0000","172.21.15.22:6801/3109015836":"2024-02-17T19:19:38.544291+0000","172.21.15.117:6825/565408102":"2024-02-17T19:38:31.770686+0000","172.21.15.22:0/206933569":"2024-02-17T19:28:59.405724+0000","172.21.15.22:0/3012142271":"2024-02-17T19:33:15.748281+0000","172.21.15.117:0/2453907739":"2024-02-17T19:29:53.194933+0000","172.21.15.22:6800/3109015836":"2024-02-17T19:19:38.544291+0000","172.21.15.117:0/3042073611":"2024-02-17T19:29:53.194933+0000","172.21.15.117:0/4194262516":"2024-02-17T19:29:53.194933+0000","172.21.15.22:6800/2945983569":"2024-02-17T19:21:26.760165+0000","172.21.15.22:0/1603216752":"2024-02-17T19:19:38.544291+0000","172.21.15.22:6827/149959963":"2024-02-17T19:26:32.932573+0000","172.21.15.22:6801/2945983569":"2024-02-17T19:21:26.760165+0000","172.21.15.22:0/668279955":"2024-02-17T19:21:26.760165+0000","172.21.15.22:0/2705605267":"2024-02-17T19:28:59.405724+0000","172.21.15.117:6824/565408102":"2024-02-17T19:38:31.770686+0000","172.21.15.22:6800/2755289428":"2024-02-17T19:20:11.619146+0000","172.21.15.22:0/2099601555":"2024-02-17T19:20:11.619146+0000","172.21.15.22:6801/2253867279":"2024-02-17T19:28:59.405724+0000","172.21.15.117:0/333707173":"2024-02-17T19:29:53.194933+0000","172.21.15.117:6828/10500008":"2024-02-17T19:29:53.194933+0000","172.21.15.117:0/199482916":"2024-02-17T19:29:53.194933+0000","172.21.15.117:6829/10500008":"2024-02-17T19:29:53.194933+0000","172.21.15.22:0/2017591724":"2024-02-17T19:20:11.619146+0000","172.21.15.22:0/2371245756":"2024-02-17T19:19:38.544291+0000","172.21.15.22:6801/3703806543":"2024-02-17T19:33:15.748281+0000","172.21.15.22:0/2384083375":"2024-02-17T19:21:26.760165+0000","172.21.15.22:0/402850490":"2024-02-17T19:28:59.405724+0000","172.21.15.22:0/1229330195":"2024-02-17T19:20:11.619146+0000","172.21.15.22:0/8466130":"2024-02-17T19:19:38.544291+0000","172.21.15.22:0/3200629226":"2024-02-17T19:21:26.760165+0000","172.21.15.117:0/2264991341":"2024-02-17T19:29:53.194933+0000","172.21.15.22:0/2853892105":"2024-02-17T19:33:15.748281+0000","172.21.15.22:6826/149959963":"2024-02-17T19:26:32.932573+0000","172.21.15.22:6800/3703806543":"2024-02-17T19:33:15.748281+0000","172.21.15.22:0/2353857756":"2024-02-17T19:33:15.748281+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-16T19:41:47.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.852+0000 7f897d6bd640 1 -- 172.21.15.22:0/4142435882 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f896406d8d0 msgr2=0x7f896406fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:47.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.852+0000 7f897d6bd640 1 --2- 172.21.15.22:0/4142435882 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f896406d8d0 0x7f896406fd80 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8970000c00 tx=0x7f8970005fd0 comp rx=0 tx=0).stop 2024-02-16T19:41:47.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.852+0000 7f897d6bd640 1 -- 172.21.15.22:0/4142435882 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8978101160 msgr2=0x7f89781957f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:47.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.852+0000 7f897d6bd640 1 --2- 172.21.15.22:0/4142435882 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8978101160 0x7f89781957f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f896000c8f0 tx=0x7f896000cdb0 comp rx=0 tx=0).stop 2024-02-16T19:41:47.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.852+0000 7f897d6bd640 1 -- 172.21.15.22:0/4142435882 shutdown_connections 2024-02-16T19:41:47.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.852+0000 7f897d6bd640 1 --2- 172.21.15.22:0/4142435882 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f896406d8d0 0x7f896406fd80 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:47.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.852+0000 7f897d6bd640 1 --2- 172.21.15.22:0/4142435882 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8978101160 0x7f89781957f0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:47.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.852+0000 7f897d6bd640 1 --2- 172.21.15.22:0/4142435882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8978100850 0x7f89781952b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:47.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.852+0000 7f897d6bd640 1 -- 172.21.15.22:0/4142435882 >> 172.21.15.22:0/4142435882 conn(0x7f8978075270 msgr2=0x7f8978103e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:47.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.852+0000 7f897d6bd640 1 -- 172.21.15.22:0/4142435882 shutdown_connections 2024-02-16T19:41:47.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.852+0000 7f897d6bd640 1 -- 172.21.15.22:0/4142435882 wait complete. 2024-02-16T19:41:47.864 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-16T19:23:47.101426+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-02-16T19:26:16.533404+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-02-16T19:26:17.113076+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-16T19:41:47.865 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-16T19:41:47.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:41:47 smithi117 ceph-mon[67870]: pgmap v301: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:41:47.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:41:47 smithi117 ceph-mon[67870]: from='client.? 172.21.15.22:0/4108180507' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-16T19:41:47.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:41:47 smithi117 ceph-mon[67870]: from='client.? 172.21.15.22:0/3797245235' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-02-16T19:41:47.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:41:47 smithi117 ceph-mon[67870]: from='client.? 172.21.15.22:0/468504957' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-02-16T19:41:47.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.968+0000 7efe66392640 1 -- 172.21.15.22:0/2800923077 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efe600ff870 msgr2=0x7efe600ffce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:47.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.968+0000 7efe66392640 1 --2- 172.21.15.22:0/2800923077 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efe600ff870 0x7efe600ffce0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7efe54009880 tx=0x7efe5402bce0 comp rx=0 tx=0).stop 2024-02-16T19:41:47.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.968+0000 7efe66392640 1 -- 172.21.15.22:0/2800923077 shutdown_connections 2024-02-16T19:41:47.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.968+0000 7efe66392640 1 --2- 172.21.15.22:0/2800923077 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efe600ff870 0x7efe600ffce0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:47.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.968+0000 7efe66392640 1 --2- 172.21.15.22:0/2800923077 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe60105880 0x7efe60105c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:47.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.968+0000 7efe66392640 1 -- 172.21.15.22:0/2800923077 >> 172.21.15.22:0/2800923077 conn(0x7efe600fb370 msgr2=0x7efe600fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:47.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.968+0000 7efe66392640 1 -- 172.21.15.22:0/2800923077 shutdown_connections 2024-02-16T19:41:47.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.968+0000 7efe66392640 1 -- 172.21.15.22:0/2800923077 wait complete. 2024-02-16T19:41:47.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.969+0000 7efe66392640 1 Processor -- start 2024-02-16T19:41:47.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.969+0000 7efe66392640 1 -- start start 2024-02-16T19:41:47.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.969+0000 7efe66392640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efe600ff870 0x7efe60195580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:47.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.969+0000 7efe66392640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe60105880 0x7efe60195ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:47.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.969+0000 7efe66392640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe60196150 con 0x7efe600ff870 2024-02-16T19:41:47.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.969+0000 7efe66392640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe60199f20 con 0x7efe60105880 2024-02-16T19:41:47.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.969+0000 7efe5ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efe600ff870 0x7efe60195580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:47.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.969+0000 7efe5ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efe600ff870 0x7efe60195580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59940/0 (socket says 172.21.15.22:59940) 2024-02-16T19:41:47.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.969+0000 7efe5ffff640 1 -- 172.21.15.22:0/1564292204 learned_addr learned my addr 172.21.15.22:0/1564292204 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:41:47.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.969+0000 7efe5ffff640 1 -- 172.21.15.22:0/1564292204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe60105880 msgr2=0x7efe60195ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:47.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.969+0000 7efe5ffff640 1 --2- 172.21.15.22:0/1564292204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe60105880 0x7efe60195ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:47.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.969+0000 7efe5ffff640 1 -- 172.21.15.22:0/1564292204 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe54009560 con 0x7efe600ff870 2024-02-16T19:41:47.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.970+0000 7efe5ffff640 1 --2- 172.21.15.22:0/1564292204 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efe600ff870 0x7efe60195580 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7efe5000d8d0 tx=0x7efe5000dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:47.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.970+0000 7efe5d7fa640 1 -- 172.21.15.22:0/1564292204 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe50009a30 con 0x7efe600ff870 2024-02-16T19:41:47.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.970+0000 7efe5d7fa640 1 -- 172.21.15.22:0/1564292204 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efe50009b90 con 0x7efe600ff870 2024-02-16T19:41:47.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.970+0000 7efe5d7fa640 1 -- 172.21.15.22:0/1564292204 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe5000f680 con 0x7efe600ff870 2024-02-16T19:41:47.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.970+0000 7efe66392640 1 -- 172.21.15.22:0/1564292204 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe6019a200 con 0x7efe600ff870 2024-02-16T19:41:47.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.970+0000 7efe66392640 1 -- 172.21.15.22:0/1564292204 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe6019a710 con 0x7efe600ff870 2024-02-16T19:41:47.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.971+0000 7efe66392640 1 -- 172.21.15.22:0/1564292204 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe6004e000 con 0x7efe600ff870 2024-02-16T19:41:47.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.971+0000 7efe5d7fa640 1 -- 172.21.15.22:0/1564292204 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7efe50009d00 con 0x7efe600ff870 2024-02-16T19:41:47.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.972+0000 7efe5d7fa640 1 --2- 172.21.15.22:0/1564292204 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7efe4806d810 0x7efe4806fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:47.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.972+0000 7efe5d7fa640 1 -- 172.21.15.22:0/1564292204 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6962+0+0 (secure 0 0 0) 0x7efe5008d510 con 0x7efe600ff870 2024-02-16T19:41:47.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.972+0000 7efe5f7fe640 1 --2- 172.21.15.22:0/1564292204 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7efe4806d810 0x7efe4806fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:47.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.972+0000 7efe5f7fe640 1 --2- 172.21.15.22:0/1564292204 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7efe4806d810 0x7efe4806fcc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7efe54036040 tx=0x7efe54000bc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:47.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:47.977+0000 7efe5d7fa640 1 -- 172.21.15.22:0/1564292204 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7efe50056b40 con 0x7efe600ff870 2024-02-16T19:41:48.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.159+0000 7efe66392640 1 -- 172.21.15.22:0/1564292204 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efe601969c0 con 0x7efe600ff870 2024-02-16T19:41:48.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.160+0000 7efe5d7fa640 1 -- 172.21.15.22:0/1564292204 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7efe5005a200 con 0x7efe600ff870 2024-02-16T19:41:48.161 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-02-16T19:41:48.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.162+0000 7efe66392640 1 -- 172.21.15.22:0/1564292204 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7efe4806d810 msgr2=0x7efe4806fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:48.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.162+0000 7efe66392640 1 --2- 172.21.15.22:0/1564292204 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7efe4806d810 0x7efe4806fcc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7efe54036040 tx=0x7efe54000bc0 comp rx=0 tx=0).stop 2024-02-16T19:41:48.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.162+0000 7efe66392640 1 -- 172.21.15.22:0/1564292204 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efe600ff870 msgr2=0x7efe60195580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:48.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.162+0000 7efe66392640 1 --2- 172.21.15.22:0/1564292204 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efe600ff870 0x7efe60195580 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7efe5000d8d0 tx=0x7efe5000dd90 comp rx=0 tx=0).stop 2024-02-16T19:41:48.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.162+0000 7efe66392640 1 -- 172.21.15.22:0/1564292204 shutdown_connections 2024-02-16T19:41:48.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.162+0000 7efe66392640 1 --2- 172.21.15.22:0/1564292204 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7efe4806d810 0x7efe4806fcc0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:48.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.162+0000 7efe66392640 1 --2- 172.21.15.22:0/1564292204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe60105880 0x7efe60195ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:48.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.162+0000 7efe66392640 1 --2- 172.21.15.22:0/1564292204 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efe600ff870 0x7efe60195580 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:48.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.162+0000 7efe66392640 1 -- 172.21.15.22:0/1564292204 >> 172.21.15.22:0/1564292204 conn(0x7efe600fb370 msgr2=0x7efe600fcd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:48.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.162+0000 7efe66392640 1 -- 172.21.15.22:0/1564292204 shutdown_connections 2024-02-16T19:41:48.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.162+0000 7efe66392640 1 -- 172.21.15.22:0/1564292204 wait complete. 2024-02-16T19:41:48.174 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-02-16T19:41:48.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.277+0000 7f4da4015640 1 -- 172.21.15.22:0/377044317 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d9c0fe7f0 msgr2=0x7f4d9c0fec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:48.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.277+0000 7f4da4015640 1 --2- 172.21.15.22:0/377044317 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d9c0fe7f0 0x7f4d9c0fec60 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c009880 tx=0x7f4d8c02bce0 comp rx=0 tx=0).stop 2024-02-16T19:41:48.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.278+0000 7f4da4015640 1 -- 172.21.15.22:0/377044317 shutdown_connections 2024-02-16T19:41:48.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.278+0000 7f4da4015640 1 --2- 172.21.15.22:0/377044317 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d9c0fe7f0 0x7f4d9c0fec60 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:48.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.278+0000 7f4da4015640 1 --2- 172.21.15.22:0/377044317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d9c104800 0x7f4d9c104bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:48.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.278+0000 7f4da4015640 1 -- 172.21.15.22:0/377044317 >> 172.21.15.22:0/377044317 conn(0x7f4d9c0fa2f0 msgr2=0x7f4d9c0fc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:48.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.278+0000 7f4da4015640 1 -- 172.21.15.22:0/377044317 shutdown_connections 2024-02-16T19:41:48.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.278+0000 7f4da4015640 1 -- 172.21.15.22:0/377044317 wait complete. 2024-02-16T19:41:48.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.278+0000 7f4da4015640 1 Processor -- start 2024-02-16T19:41:48.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.278+0000 7f4da4015640 1 -- start start 2024-02-16T19:41:48.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.278+0000 7f4da4015640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d9c0fe7f0 0x7f4d9c194440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:48.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.278+0000 7f4da4015640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d9c104800 0x7f4d9c194980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:48.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.278+0000 7f4da4015640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9c195010 con 0x7f4d9c0fe7f0 2024-02-16T19:41:48.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.278+0000 7f4da4015640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9c198de0 con 0x7f4d9c104800 2024-02-16T19:41:48.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.279+0000 7f4da1db1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d9c0fe7f0 0x7f4d9c194440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:48.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.279+0000 7f4da1db1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d9c0fe7f0 0x7f4d9c194440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59960/0 (socket says 172.21.15.22:59960) 2024-02-16T19:41:48.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.279+0000 7f4da1db1640 1 -- 172.21.15.22:0/181491829 learned_addr learned my addr 172.21.15.22:0/181491829 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:41:48.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.279+0000 7f4da15b0640 1 --2- 172.21.15.22:0/181491829 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d9c104800 0x7f4d9c194980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:48.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.279+0000 7f4da1db1640 1 -- 172.21.15.22:0/181491829 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d9c104800 msgr2=0x7f4d9c194980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:48.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.279+0000 7f4da1db1640 1 --2- 172.21.15.22:0/181491829 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d9c104800 0x7f4d9c194980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:48.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.279+0000 7f4da1db1640 1 -- 172.21.15.22:0/181491829 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d8c009560 con 0x7f4d9c0fe7f0 2024-02-16T19:41:48.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.279+0000 7f4da1db1640 1 --2- 172.21.15.22:0/181491829 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d9c0fe7f0 0x7f4d9c194440 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f4d9800d8d0 tx=0x7f4d9800dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:48.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.279+0000 7f4d92ffd640 1 -- 172.21.15.22:0/181491829 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d98009a30 con 0x7f4d9c0fe7f0 2024-02-16T19:41:48.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.279+0000 7f4d92ffd640 1 -- 172.21.15.22:0/181491829 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4d98009b90 con 0x7f4d9c0fe7f0 2024-02-16T19:41:48.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.279+0000 7f4da4015640 1 -- 172.21.15.22:0/181491829 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d9c1990c0 con 0x7f4d9c0fe7f0 2024-02-16T19:41:48.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.279+0000 7f4d92ffd640 1 -- 172.21.15.22:0/181491829 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d9800f680 con 0x7f4d9c0fe7f0 2024-02-16T19:41:48.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.279+0000 7f4da4015640 1 -- 172.21.15.22:0/181491829 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d9c1996a0 con 0x7f4d9c0fe7f0 2024-02-16T19:41:48.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.280+0000 7f4da4015640 1 -- 172.21.15.22:0/181491829 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d9c04e000 con 0x7f4d9c0fe7f0 2024-02-16T19:41:48.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.281+0000 7f4d92ffd640 1 -- 172.21.15.22:0/181491829 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f4d98009d00 con 0x7f4d9c0fe7f0 2024-02-16T19:41:48.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.281+0000 7f4d92ffd640 1 --2- 172.21.15.22:0/181491829 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f4d8806d810 0x7f4d8806fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:48.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.281+0000 7f4d92ffd640 1 -- 172.21.15.22:0/181491829 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f4d9808d510 con 0x7f4d9c0fe7f0 2024-02-16T19:41:48.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.281+0000 7f4da15b0640 1 --2- 172.21.15.22:0/181491829 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f4d8806d810 0x7f4d8806fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:48.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.281+0000 7f4da15b0640 1 --2- 172.21.15.22:0/181491829 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f4d8806d810 0x7f4d8806fcc0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c195a50 tx=0x7f4d8c000bc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:48.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.286+0000 7f4d92ffd640 1 -- 172.21.15.22:0/181491829 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4d98056b40 con 0x7f4d9c0fe7f0 2024-02-16T19:41:48.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:41:48 smithi022 ceph-mon[88798]: from='client.? 172.21.15.22:0/3606484216' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-16T19:41:48.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:41:48 smithi022 ceph-mon[88798]: from='client.? 172.21.15.22:0/4142435882' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-16T19:41:48.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:41:48 smithi022 ceph-mon[88798]: from='client.? 172.21.15.22:0/1564292204' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-16T19:41:48.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.465+0000 7f4da4015640 1 -- 172.21.15.22:0/181491829 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f4d9c066860 con 0x7f4d9c0fe7f0 2024-02-16T19:41:48.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.466+0000 7f4d92ffd640 1 -- 172.21.15.22:0/181491829 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v93) v1 ==== 107+0+11 (secure 0 0 0) 0x7f4d9805a200 con 0x7f4d9c0fe7f0 2024-02-16T19:41:48.467 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-02-16T19:41:48.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.467+0000 7f4da4015640 1 -- 172.21.15.22:0/181491829 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f4d8806d810 msgr2=0x7f4d8806fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:48.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.467+0000 7f4da4015640 1 --2- 172.21.15.22:0/181491829 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f4d8806d810 0x7f4d8806fcc0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c195a50 tx=0x7f4d8c000bc0 comp rx=0 tx=0).stop 2024-02-16T19:41:48.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.467+0000 7f4da4015640 1 -- 172.21.15.22:0/181491829 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d9c0fe7f0 msgr2=0x7f4d9c194440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:48.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.467+0000 7f4da4015640 1 --2- 172.21.15.22:0/181491829 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d9c0fe7f0 0x7f4d9c194440 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f4d9800d8d0 tx=0x7f4d9800dd90 comp rx=0 tx=0).stop 2024-02-16T19:41:48.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.468+0000 7f4da4015640 1 -- 172.21.15.22:0/181491829 shutdown_connections 2024-02-16T19:41:48.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.468+0000 7f4da4015640 1 --2- 172.21.15.22:0/181491829 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f4d8806d810 0x7f4d8806fcc0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:48.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.468+0000 7f4da4015640 1 --2- 172.21.15.22:0/181491829 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d9c104800 0x7f4d9c194980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:48.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.468+0000 7f4da4015640 1 --2- 172.21.15.22:0/181491829 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d9c0fe7f0 0x7f4d9c194440 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:48.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.468+0000 7f4da4015640 1 -- 172.21.15.22:0/181491829 >> 172.21.15.22:0/181491829 conn(0x7f4d9c0fa2f0 msgr2=0x7f4d9c0fbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:48.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.468+0000 7f4da4015640 1 -- 172.21.15.22:0/181491829 shutdown_connections 2024-02-16T19:41:48.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.468+0000 7f4da4015640 1 -- 172.21.15.22:0/181491829 wait complete. 2024-02-16T19:41:48.480 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-02-16T19:41:48.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.584+0000 7f3457c2f640 1 -- 172.21.15.22:0/1257769803 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3450105880 msgr2=0x7f3450105c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:48.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.584+0000 7f3457c2f640 1 --2- 172.21.15.22:0/1257769803 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3450105880 0x7f3450105c50 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f3448009880 tx=0x7f344802bce0 comp rx=0 tx=0).stop 2024-02-16T19:41:48.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.584+0000 7f3457c2f640 1 -- 172.21.15.22:0/1257769803 shutdown_connections 2024-02-16T19:41:48.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.584+0000 7f3457c2f640 1 --2- 172.21.15.22:0/1257769803 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34500ff870 0x7f34500ffce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:48.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.584+0000 7f3457c2f640 1 --2- 172.21.15.22:0/1257769803 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3450105880 0x7f3450105c50 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:48.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.584+0000 7f3457c2f640 1 -- 172.21.15.22:0/1257769803 >> 172.21.15.22:0/1257769803 conn(0x7f34500fb370 msgr2=0x7f34500fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:48.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.584+0000 7f3457c2f640 1 -- 172.21.15.22:0/1257769803 shutdown_connections 2024-02-16T19:41:48.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.584+0000 7f3457c2f640 1 -- 172.21.15.22:0/1257769803 wait complete. 2024-02-16T19:41:48.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.584+0000 7f3457c2f640 1 Processor -- start 2024-02-16T19:41:48.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.584+0000 7f3457c2f640 1 -- start start 2024-02-16T19:41:48.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.584+0000 7f3457c2f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34500ff870 0x7f34501954e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:48.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.584+0000 7f3457c2f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3450105880 0x7f3450195a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:48.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.584+0000 7f3457c2f640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3450196020 con 0x7f3450105880 2024-02-16T19:41:48.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.584+0000 7f3457c2f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3450196190 con 0x7f34500ff870 2024-02-16T19:41:48.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.585+0000 7f34551ca640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3450105880 0x7f3450195a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:48.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.585+0000 7f34551ca640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3450105880 0x7f3450195a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59974/0 (socket says 172.21.15.22:59974) 2024-02-16T19:41:48.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.585+0000 7f34551ca640 1 -- 172.21.15.22:0/4099371564 learned_addr learned my addr 172.21.15.22:0/4099371564 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:41:48.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.585+0000 7f34559cb640 1 --2- 172.21.15.22:0/4099371564 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34500ff870 0x7f34501954e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:48.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.585+0000 7f34551ca640 1 -- 172.21.15.22:0/4099371564 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34500ff870 msgr2=0x7f34501954e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:48.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.585+0000 7f34551ca640 1 --2- 172.21.15.22:0/4099371564 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34500ff870 0x7f34501954e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:48.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.585+0000 7f34551ca640 1 -- 172.21.15.22:0/4099371564 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3448009560 con 0x7f3450105880 2024-02-16T19:41:48.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.585+0000 7f34551ca640 1 --2- 172.21.15.22:0/4099371564 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3450105880 0x7f3450195a20 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f344400c8a0 tx=0x7f344400cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:48.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.585+0000 7f344effd640 1 -- 172.21.15.22:0/4099371564 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3444007b80 con 0x7f3450105880 2024-02-16T19:41:48.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.585+0000 7f344effd640 1 -- 172.21.15.22:0/4099371564 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3444007ce0 con 0x7f3450105880 2024-02-16T19:41:48.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.585+0000 7f344effd640 1 -- 172.21.15.22:0/4099371564 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3444018810 con 0x7f3450105880 2024-02-16T19:41:48.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.585+0000 7f3457c2f640 1 -- 172.21.15.22:0/4099371564 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f345019a020 con 0x7f3450105880 2024-02-16T19:41:48.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.585+0000 7f3457c2f640 1 -- 172.21.15.22:0/4099371564 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f345019a600 con 0x7f3450105880 2024-02-16T19:41:48.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.586+0000 7f3457c2f640 1 -- 172.21.15.22:0/4099371564 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f345004e000 con 0x7f3450105880 2024-02-16T19:41:48.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.587+0000 7f344effd640 1 -- 172.21.15.22:0/4099371564 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f3444018970 con 0x7f3450105880 2024-02-16T19:41:48.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.587+0000 7f344effd640 1 --2- 172.21.15.22:0/4099371564 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f343806d810 0x7f343806fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:48.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.587+0000 7f344effd640 1 -- 172.21.15.22:0/4099371564 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f344408d760 con 0x7f3450105880 2024-02-16T19:41:48.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.587+0000 7f34559cb640 1 --2- 172.21.15.22:0/4099371564 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f343806d810 0x7f343806fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:48.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.587+0000 7f34559cb640 1 --2- 172.21.15.22:0/4099371564 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f343806d810 0x7f343806fcc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3448006010 tx=0x7f3448000bc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:48.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.592+0000 7f344effd640 1 -- 172.21.15.22:0/4099371564 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3444056e10 con 0x7f3450105880 2024-02-16T19:41:48.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.771+0000 7f3457c2f640 1 -- 172.21.15.22:0/4099371564 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f3450066860 con 0x7f3450105880 2024-02-16T19:41:48.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.772+0000 7f344effd640 1 -- 172.21.15.22:0/4099371564 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v93) v1 ==== 107+0+11 (secure 0 0 0) 0x7f344405a4d0 con 0x7f3450105880 2024-02-16T19:41:48.773 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-02-16T19:41:48.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.773+0000 7f3457c2f640 1 -- 172.21.15.22:0/4099371564 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f343806d810 msgr2=0x7f343806fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:48.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.773+0000 7f3457c2f640 1 --2- 172.21.15.22:0/4099371564 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f343806d810 0x7f343806fcc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3448006010 tx=0x7f3448000bc0 comp rx=0 tx=0).stop 2024-02-16T19:41:48.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.773+0000 7f3457c2f640 1 -- 172.21.15.22:0/4099371564 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3450105880 msgr2=0x7f3450195a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:48.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.774+0000 7f3457c2f640 1 --2- 172.21.15.22:0/4099371564 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3450105880 0x7f3450195a20 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f344400c8a0 tx=0x7f344400cd60 comp rx=0 tx=0).stop 2024-02-16T19:41:48.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.774+0000 7f3457c2f640 1 -- 172.21.15.22:0/4099371564 shutdown_connections 2024-02-16T19:41:48.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.774+0000 7f3457c2f640 1 --2- 172.21.15.22:0/4099371564 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7f343806d810 0x7f343806fcc0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:48.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.774+0000 7f3457c2f640 1 --2- 172.21.15.22:0/4099371564 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3450105880 0x7f3450195a20 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:48.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.774+0000 7f3457c2f640 1 --2- 172.21.15.22:0/4099371564 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34500ff870 0x7f34501954e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:48.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.774+0000 7f3457c2f640 1 -- 172.21.15.22:0/4099371564 >> 172.21.15.22:0/4099371564 conn(0x7f34500fb370 msgr2=0x7f34500fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:48.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.774+0000 7f3457c2f640 1 -- 172.21.15.22:0/4099371564 shutdown_connections 2024-02-16T19:41:48.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.774+0000 7f3457c2f640 1 -- 172.21.15.22:0/4099371564 wait complete. 2024-02-16T19:41:48.786 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-16T19:41:48.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:41:48 smithi117 ceph-mon[67870]: from='client.? 172.21.15.22:0/3606484216' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-16T19:41:48.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:41:48 smithi117 ceph-mon[67870]: from='client.? 172.21.15.22:0/4142435882' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-16T19:41:48.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:41:48 smithi117 ceph-mon[67870]: from='client.? 172.21.15.22:0/1564292204' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-16T19:41:48.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.887+0000 7fb95e4e2640 1 -- 172.21.15.22:0/2001748202 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb9580fe5a0 msgr2=0x7fb9580fea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:48.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.887+0000 7fb95e4e2640 1 --2- 172.21.15.22:0/2001748202 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb9580fe5a0 0x7fb9580fea10 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb94c009880 tx=0x7fb94c02bce0 comp rx=0 tx=0).stop 2024-02-16T19:41:48.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.888+0000 7fb95e4e2640 1 -- 172.21.15.22:0/2001748202 shutdown_connections 2024-02-16T19:41:48.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.888+0000 7fb95e4e2640 1 --2- 172.21.15.22:0/2001748202 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb9580fe5a0 0x7fb9580fea10 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:48.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.888+0000 7fb95e4e2640 1 --2- 172.21.15.22:0/2001748202 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9581045d0 0x7fb9581049a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:48.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.888+0000 7fb95e4e2640 1 -- 172.21.15.22:0/2001748202 >> 172.21.15.22:0/2001748202 conn(0x7fb958075270 msgr2=0x7fb958075670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:48.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.888+0000 7fb95e4e2640 1 -- 172.21.15.22:0/2001748202 shutdown_connections 2024-02-16T19:41:48.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.888+0000 7fb95e4e2640 1 -- 172.21.15.22:0/2001748202 wait complete. 2024-02-16T19:41:48.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.888+0000 7fb95e4e2640 1 Processor -- start 2024-02-16T19:41:48.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.888+0000 7fb95e4e2640 1 -- start start 2024-02-16T19:41:48.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.888+0000 7fb95e4e2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb9580fe5a0 0x7fb958194050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:48.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.888+0000 7fb95e4e2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9581045d0 0x7fb958194590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:48.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.888+0000 7fb95e4e2640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb958194c20 con 0x7fb9580fe5a0 2024-02-16T19:41:48.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.888+0000 7fb95e4e2640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9581989f0 con 0x7fb9581045d0 2024-02-16T19:41:48.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.889+0000 7fb957fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb9580fe5a0 0x7fb958194050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:48.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.889+0000 7fb957fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb9580fe5a0 0x7fb958194050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59994/0 (socket says 172.21.15.22:59994) 2024-02-16T19:41:48.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.889+0000 7fb957fff640 1 -- 172.21.15.22:0/3847128112 learned_addr learned my addr 172.21.15.22:0/3847128112 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-02-16T19:41:48.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.889+0000 7fb9577fe640 1 --2- 172.21.15.22:0/3847128112 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9581045d0 0x7fb958194590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:48.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.889+0000 7fb957fff640 1 -- 172.21.15.22:0/3847128112 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9581045d0 msgr2=0x7fb958194590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:48.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.889+0000 7fb957fff640 1 --2- 172.21.15.22:0/3847128112 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9581045d0 0x7fb958194590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:48.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.889+0000 7fb957fff640 1 -- 172.21.15.22:0/3847128112 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb94c009560 con 0x7fb9580fe5a0 2024-02-16T19:41:48.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.889+0000 7fb9577fe640 1 --2- 172.21.15.22:0/3847128112 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9581045d0 0x7fb958194590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T19:41:48.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.889+0000 7fb957fff640 1 --2- 172.21.15.22:0/3847128112 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb9580fe5a0 0x7fb958194050 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb94800e970 tx=0x7fb94800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:48.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.889+0000 7fb9557fa640 1 -- 172.21.15.22:0/3847128112 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb94800cc90 con 0x7fb9580fe5a0 2024-02-16T19:41:48.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.889+0000 7fb95e4e2640 1 -- 172.21.15.22:0/3847128112 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb958198cd0 con 0x7fb9580fe5a0 2024-02-16T19:41:48.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.889+0000 7fb9557fa640 1 -- 172.21.15.22:0/3847128112 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb94800cdf0 con 0x7fb9580fe5a0 2024-02-16T19:41:48.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.889+0000 7fb9557fa640 1 -- 172.21.15.22:0/3847128112 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb948010430 con 0x7fb9580fe5a0 2024-02-16T19:41:48.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.889+0000 7fb95e4e2640 1 -- 172.21.15.22:0/3847128112 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9581992b0 con 0x7fb9580fe5a0 2024-02-16T19:41:48.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.890+0000 7fb95e4e2640 1 -- 172.21.15.22:0/3847128112 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb95804e000 con 0x7fb9580fe5a0 2024-02-16T19:41:48.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.891+0000 7fb9557fa640 1 -- 172.21.15.22:0/3847128112 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 40) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fb9480046c0 con 0x7fb9580fe5a0 2024-02-16T19:41:48.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.891+0000 7fb9557fa640 1 --2- 172.21.15.22:0/3847128112 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fb94006d8d0 0x7fb94006fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:41:48.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.891+0000 7fb9557fa640 1 -- 172.21.15.22:0/3847128112 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6962+0+0 (secure 0 0 0) 0x7fb948014070 con 0x7fb9580fe5a0 2024-02-16T19:41:48.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.891+0000 7fb9577fe640 1 --2- 172.21.15.22:0/3847128112 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fb94006d8d0 0x7fb94006fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:41:48.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.892+0000 7fb9577fe640 1 --2- 172.21.15.22:0/3847128112 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fb94006d8d0 0x7fb94006fd80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb958195660 tx=0x7fb94c02cb40 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:41:48.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:48.896+0000 7fb9557fa640 1 -- 172.21.15.22:0/3847128112 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb9480572f0 con 0x7fb9580fe5a0 2024-02-16T19:41:49.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:49.089+0000 7fb95e4e2640 1 -- 172.21.15.22:0/3847128112 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb9581953a0 con 0x7fb9580fe5a0 2024-02-16T19:41:49.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:49.089+0000 7fb9557fa640 1 -- 172.21.15.22:0/3847128112 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 43 entries v93) v1 ==== 81+0+2550 (secure 0 0 0) 0x7fb9581953a0 con 0x7fb9580fe5a0 2024-02-16T19:41:49.091 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.117:6827/3122342028 2024-02-17T19:38:57.804505+0000 2024-02-16T19:41:49.091 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.117:6826/3122342028 2024-02-17T19:38:57.804505+0000 2024-02-16T19:41:49.091 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:6829/3597685751 2024-02-17T19:38:09.297300+0000 2024-02-16T19:41:49.091 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:6828/3597685751 2024-02-17T19:38:09.297300+0000 2024-02-16T19:41:49.091 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/4238358751 2024-02-17T19:33:15.748281+0000 2024-02-16T19:41:49.091 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:6801/2755289428 2024-02-17T19:20:11.619146+0000 2024-02-16T19:41:49.091 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:6800/2253867279 2024-02-17T19:28:59.405724+0000 2024-02-16T19:41:49.091 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/2301835367 2024-02-17T19:28:59.405724+0000 2024-02-16T19:41:49.091 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:6801/3109015836 2024-02-17T19:19:38.544291+0000 2024-02-16T19:41:49.091 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.117:6825/565408102 2024-02-17T19:38:31.770686+0000 2024-02-16T19:41:49.091 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/206933569 2024-02-17T19:28:59.405724+0000 2024-02-16T19:41:49.092 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/3012142271 2024-02-17T19:33:15.748281+0000 2024-02-16T19:41:49.092 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.117:0/2453907739 2024-02-17T19:29:53.194933+0000 2024-02-16T19:41:49.092 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:6800/3109015836 2024-02-17T19:19:38.544291+0000 2024-02-16T19:41:49.092 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.117:0/3042073611 2024-02-17T19:29:53.194933+0000 2024-02-16T19:41:49.092 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.117:0/4194262516 2024-02-17T19:29:53.194933+0000 2024-02-16T19:41:49.092 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:6800/2945983569 2024-02-17T19:21:26.760165+0000 2024-02-16T19:41:49.092 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/1603216752 2024-02-17T19:19:38.544291+0000 2024-02-16T19:41:49.092 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:6827/149959963 2024-02-17T19:26:32.932573+0000 2024-02-16T19:41:49.092 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:6801/2945983569 2024-02-17T19:21:26.760165+0000 2024-02-16T19:41:49.092 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/668279955 2024-02-17T19:21:26.760165+0000 2024-02-16T19:41:49.092 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/2705605267 2024-02-17T19:28:59.405724+0000 2024-02-16T19:41:49.092 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.117:6824/565408102 2024-02-17T19:38:31.770686+0000 2024-02-16T19:41:49.092 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:6800/2755289428 2024-02-17T19:20:11.619146+0000 2024-02-16T19:41:49.092 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/2099601555 2024-02-17T19:20:11.619146+0000 2024-02-16T19:41:49.093 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:6801/2253867279 2024-02-17T19:28:59.405724+0000 2024-02-16T19:41:49.093 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.117:0/333707173 2024-02-17T19:29:53.194933+0000 2024-02-16T19:41:49.093 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.117:6828/10500008 2024-02-17T19:29:53.194933+0000 2024-02-16T19:41:49.093 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.117:0/199482916 2024-02-17T19:29:53.194933+0000 2024-02-16T19:41:49.093 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.117:6829/10500008 2024-02-17T19:29:53.194933+0000 2024-02-16T19:41:49.093 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/2017591724 2024-02-17T19:20:11.619146+0000 2024-02-16T19:41:49.093 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/2371245756 2024-02-17T19:19:38.544291+0000 2024-02-16T19:41:49.093 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:6801/3703806543 2024-02-17T19:33:15.748281+0000 2024-02-16T19:41:49.093 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/2384083375 2024-02-17T19:21:26.760165+0000 2024-02-16T19:41:49.093 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/402850490 2024-02-17T19:28:59.405724+0000 2024-02-16T19:41:49.093 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/1229330195 2024-02-17T19:20:11.619146+0000 2024-02-16T19:41:49.093 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/8466130 2024-02-17T19:19:38.544291+0000 2024-02-16T19:41:49.093 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/3200629226 2024-02-17T19:21:26.760165+0000 2024-02-16T19:41:49.093 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.117:0/2264991341 2024-02-17T19:29:53.194933+0000 2024-02-16T19:41:49.093 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/2853892105 2024-02-17T19:33:15.748281+0000 2024-02-16T19:41:49.094 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:6826/149959963 2024-02-17T19:26:32.932573+0000 2024-02-16T19:41:49.094 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:6800/3703806543 2024-02-17T19:33:15.748281+0000 2024-02-16T19:41:49.094 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/2353857756 2024-02-17T19:33:15.748281+0000 2024-02-16T19:41:49.094 INFO:teuthology.orchestra.run.smithi022.stderr:listed 43 entries 2024-02-16T19:41:49.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:49.091+0000 7fb95e4e2640 1 -- 172.21.15.22:0/3847128112 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fb94006d8d0 msgr2=0x7fb94006fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:49.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:49.091+0000 7fb95e4e2640 1 --2- 172.21.15.22:0/3847128112 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fb94006d8d0 0x7fb94006fd80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb958195660 tx=0x7fb94c02cb40 comp rx=0 tx=0).stop 2024-02-16T19:41:49.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:49.091+0000 7fb95e4e2640 1 -- 172.21.15.22:0/3847128112 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb9580fe5a0 msgr2=0x7fb958194050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:41:49.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:49.091+0000 7fb95e4e2640 1 --2- 172.21.15.22:0/3847128112 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb9580fe5a0 0x7fb958194050 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb94800e970 tx=0x7fb94800ed70 comp rx=0 tx=0).stop 2024-02-16T19:41:49.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:49.091+0000 7fb95e4e2640 1 -- 172.21.15.22:0/3847128112 shutdown_connections 2024-02-16T19:41:49.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:49.091+0000 7fb95e4e2640 1 --2- 172.21.15.22:0/3847128112 >> [v2:172.21.15.22:6800/505266753,v1:172.21.15.22:6801/505266753] conn(0x7fb94006d8d0 0x7fb94006fd80 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:49.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:49.091+0000 7fb95e4e2640 1 --2- 172.21.15.22:0/3847128112 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9581045d0 0x7fb958194590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:49.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:49.091+0000 7fb95e4e2640 1 --2- 172.21.15.22:0/3847128112 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb9580fe5a0 0x7fb958194050 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:41:49.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:49.091+0000 7fb95e4e2640 1 -- 172.21.15.22:0/3847128112 >> 172.21.15.22:0/3847128112 conn(0x7fb958075270 msgr2=0x7fb9580fcbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:41:49.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:49.091+0000 7fb95e4e2640 1 -- 172.21.15.22:0/3847128112 shutdown_connections 2024-02-16T19:41:49.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-02-16T19:41:49.091+0000 7fb95e4e2640 1 -- 172.21.15.22:0/3847128112 wait complete. 2024-02-16T19:41:49.103 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi022.front.sepia.ceph.com... 2024-02-16T19:41:49.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:41:49.104 DEBUG:teuthology.orchestra.run.smithi022:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-02-16T19:41:49.130 INFO:teuthology.orchestra.run:waiting for 300 2024-02-16T19:41:49.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:41:49 smithi022 ceph-mon[88798]: pgmap v302: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:41:49.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:41:49 smithi022 ceph-mon[88798]: from='client.? 172.21.15.22:0/181491829' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-02-16T19:41:49.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:41:49 smithi022 ceph-mon[88798]: from='client.? 172.21.15.22:0/4099371564' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-02-16T19:41:49.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:41:49 smithi022 ceph-mon[88798]: from='client.? 172.21.15.22:0/3847128112' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-16T19:41:49.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:41:49 smithi117 ceph-mon[67870]: pgmap v302: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:41:49.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:41:49 smithi117 ceph-mon[67870]: from='client.? 172.21.15.22:0/181491829' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-02-16T19:41:49.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:41:49 smithi117 ceph-mon[67870]: from='client.? 172.21.15.22:0/4099371564' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-02-16T19:41:49.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:41:49 smithi117 ceph-mon[67870]: from='client.? 172.21.15.22:0/3847128112' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-16T19:41:51.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:41:51 smithi022 ceph-mon[88798]: pgmap v303: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:41:51.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:41:51 smithi117 ceph-mon[67870]: pgmap v303: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:41:53.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:41:53 smithi022 ceph-mon[88798]: pgmap v304: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:41:53.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:41:53 smithi117 ceph-mon[67870]: pgmap v304: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:41:55.689 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:41:55 smithi117 ceph-mon[67870]: pgmap v305: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:41:55.714 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:41:55 smithi022 ceph-mon[88798]: pgmap v305: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:41:57.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:41:57 smithi022 ceph-mon[88798]: pgmap v306: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:41:57.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:41:57 smithi117 ceph-mon[67870]: pgmap v306: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:41:59.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:41:59 smithi022 ceph-mon[88798]: pgmap v307: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:41:59.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:41:59 smithi117 ceph-mon[67870]: pgmap v307: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:42:01.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:42:01 smithi022 ceph-mon[88798]: pgmap v308: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:42:01.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:42:01 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:42:01.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:42:01 smithi117 ceph-mon[67870]: pgmap v308: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:42:01.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:42:01 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:42:03.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:42:03 smithi022 ceph-mon[88798]: pgmap v309: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:42:03.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:42:03 smithi117 ceph-mon[67870]: pgmap v309: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:42:05.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:42:05 smithi022 ceph-mon[88798]: pgmap v310: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:42:05.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:42:05 smithi117 ceph-mon[67870]: pgmap v310: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:42:07.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:42:07 smithi022 ceph-mon[88798]: pgmap v311: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:42:07.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:42:07 smithi117 ceph-mon[67870]: pgmap v311: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:42:09.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:42:09 smithi022 ceph-mon[88798]: pgmap v312: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:42:09.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:42:09 smithi117 ceph-mon[67870]: pgmap v312: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:42:11.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:42:11 smithi022 ceph-mon[88798]: pgmap v313: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:42:11.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:42:11 smithi117 ceph-mon[67870]: pgmap v313: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:42:13.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:42:13 smithi022 ceph-mon[88798]: pgmap v314: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:42:13.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:42:13 smithi117 ceph-mon[67870]: pgmap v314: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:42:15.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:42:15 smithi022 ceph-mon[88798]: pgmap v315: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:42:15.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:42:15 smithi117 ceph-mon[67870]: pgmap v315: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:42:16.714 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:42:16 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:42:16.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:42:16 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:42:17.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:42:17 smithi117 ceph-mon[67870]: pgmap v316: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:42:17.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:42:17 smithi022 ceph-mon[88798]: pgmap v316: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:42:19.855 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:42:19 smithi022 ceph-mon[88798]: pgmap v317: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:42:19.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:42:19 smithi117 ceph-mon[67870]: pgmap v317: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:42:21.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:42:21 smithi117 ceph-mon[67870]: pgmap v318: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:42:21.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:42:21 smithi022 ceph-mon[88798]: pgmap v318: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:42:23.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:42:23 smithi117 ceph-mon[67870]: pgmap v319: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:42:23.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:42:23 smithi022 ceph-mon[88798]: pgmap v319: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:42:25.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:42:25 smithi117 ceph-mon[67870]: pgmap v320: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:42:25.964 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:42:25 smithi022 ceph-mon[88798]: pgmap v320: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:42:27.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:42:27 smithi117 ceph-mon[67870]: pgmap v321: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:42:27.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:42:27 smithi022 ceph-mon[88798]: pgmap v321: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:42:29.767 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:42:29 smithi022 ceph-mon[88798]: pgmap v322: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:42:29.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:42:29 smithi117 ceph-mon[67870]: pgmap v322: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:42:31.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:42:31 smithi117 ceph-mon[67870]: pgmap v323: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:42:31.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:42:31 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:42:31.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:42:31 smithi022 ceph-mon[88798]: pgmap v323: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:42:31.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:42:31 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:42:33.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:42:33 smithi117 ceph-mon[67870]: pgmap v324: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:42:33.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:42:33 smithi022 ceph-mon[88798]: pgmap v324: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:42:35.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:42:35 smithi117 ceph-mon[67870]: pgmap v325: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:42:35.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:42:35 smithi022 ceph-mon[88798]: pgmap v325: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:42:37.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:42:37 smithi117 ceph-mon[67870]: pgmap v326: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:42:37.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:42:37 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:42:37.964 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:42:37 smithi022 ceph-mon[88798]: pgmap v326: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:42:37.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:42:37 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:42:39.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:42:38 smithi117 ceph-mon[67870]: pgmap v327: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:42:39.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:42:38 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:42:39.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:42:38 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:42:39.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:42:38 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:42:39.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:42:38 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:42:39.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:42:38 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:42:39.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:42:38 smithi022 ceph-mon[88798]: pgmap v327: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:42:39.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:42:38 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:42:39.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:42:38 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:42:39.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:42:38 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:42:39.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:42:38 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:42:39.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:42:38 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:42:41.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:42:40 smithi117 ceph-mon[67870]: pgmap v328: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:42:41.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:42:40 smithi022 ceph-mon[88798]: pgmap v328: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:42:43.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:42:42 smithi117 ceph-mon[67870]: pgmap v329: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:42:43.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:42:42 smithi022 ceph-mon[88798]: pgmap v329: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:42:45.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:42:44 smithi117 ceph-mon[67870]: pgmap v330: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:42:45.464 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:42:44 smithi022 ceph-mon[88798]: pgmap v330: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:42:46.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:42:45 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:42:46.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:42:45 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:42:47.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:42:46 smithi117 ceph-mon[67870]: pgmap v331: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:42:47.464 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:42:46 smithi022 ceph-mon[88798]: pgmap v331: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:42:49.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:42:48 smithi117 ceph-mon[67870]: pgmap v332: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:42:49.464 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:42:48 smithi022 ceph-mon[88798]: pgmap v332: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:42:51.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:42:51 smithi117 ceph-mon[67870]: pgmap v333: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:42:51.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:42:51 smithi022 ceph-mon[88798]: pgmap v333: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:42:53.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:42:53 smithi117 ceph-mon[67870]: pgmap v334: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:42:53.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:42:53 smithi022 ceph-mon[88798]: pgmap v334: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:42:55.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:42:55 smithi117 ceph-mon[67870]: pgmap v335: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:42:55.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:42:55 smithi022 ceph-mon[88798]: pgmap v335: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:42:57.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:42:57 smithi117 ceph-mon[67870]: pgmap v336: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:42:57.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:42:57 smithi022 ceph-mon[88798]: pgmap v336: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:42:59.267 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:42:59 smithi022 ceph-mon[88798]: pgmap v337: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:42:59.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:42:59 smithi117 ceph-mon[67870]: pgmap v337: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:43:01.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:43:01 smithi117 ceph-mon[67870]: pgmap v338: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:43:01.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:43:01 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:43:01.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:43:01 smithi022 ceph-mon[88798]: pgmap v338: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:43:01.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:43:01 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:43:03.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:43:03 smithi117 ceph-mon[67870]: pgmap v339: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:43:03.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:43:03 smithi022 ceph-mon[88798]: pgmap v339: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:43:05.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:43:05 smithi117 ceph-mon[67870]: pgmap v340: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:43:05.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:43:05 smithi022 ceph-mon[88798]: pgmap v340: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:43:07.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:43:07 smithi117 ceph-mon[67870]: pgmap v341: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:43:07.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:43:07 smithi022 ceph-mon[88798]: pgmap v341: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:43:09.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:43:09 smithi117 ceph-mon[67870]: pgmap v342: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:43:09.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:43:09 smithi022 ceph-mon[88798]: pgmap v342: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:43:11.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:43:11 smithi117 ceph-mon[67870]: pgmap v343: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:43:11.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:43:11 smithi022 ceph-mon[88798]: pgmap v343: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:43:13.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:43:13 smithi117 ceph-mon[67870]: pgmap v344: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:43:13.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:43:13 smithi022 ceph-mon[88798]: pgmap v344: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:43:15.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:43:15 smithi117 ceph-mon[67870]: pgmap v345: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:43:15.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:43:15 smithi022 ceph-mon[88798]: pgmap v345: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:43:16.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:43:16 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:43:16.714 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:43:16 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:43:17.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:43:17 smithi117 ceph-mon[67870]: pgmap v346: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:43:17.714 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:43:17 smithi022 ceph-mon[88798]: pgmap v346: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:43:19.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:43:19 smithi117 ceph-mon[67870]: pgmap v347: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:43:19.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:43:19 smithi022 ceph-mon[88798]: pgmap v347: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:43:21.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:43:21 smithi117 ceph-mon[67870]: pgmap v348: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:43:21.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:43:21 smithi022 ceph-mon[88798]: pgmap v348: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:43:23.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:43:23 smithi117 ceph-mon[67870]: pgmap v349: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:43:23.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:43:23 smithi022 ceph-mon[88798]: pgmap v349: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:43:25.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:43:25 smithi117 ceph-mon[67870]: pgmap v350: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:43:25.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:43:25 smithi022 ceph-mon[88798]: pgmap v350: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:43:27.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:43:27 smithi117 ceph-mon[67870]: pgmap v351: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:43:27.714 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:43:27 smithi022 ceph-mon[88798]: pgmap v351: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:43:29.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:43:29 smithi117 ceph-mon[67870]: pgmap v352: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:43:29.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:43:29 smithi022 ceph-mon[88798]: pgmap v352: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:43:31.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:43:31 smithi022 ceph-mon[88798]: pgmap v353: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:43:31.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:43:31 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:43:31.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:43:31 smithi117 ceph-mon[67870]: pgmap v353: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:43:31.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:43:31 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:43:33.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:43:33 smithi022 ceph-mon[88798]: pgmap v354: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:43:33.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:43:33 smithi117 ceph-mon[67870]: pgmap v354: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:43:35.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:43:35 smithi022 ceph-mon[88798]: pgmap v355: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:43:35.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:43:35 smithi117 ceph-mon[67870]: pgmap v355: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:43:37.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:43:37 smithi022 ceph-mon[88798]: pgmap v356: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:43:37.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:43:37 smithi117 ceph-mon[67870]: pgmap v356: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:43:39.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:43:39 smithi022 ceph-mon[88798]: pgmap v357: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:43:39.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:43:39 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:43:39.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:43:39 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:43:39.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:43:39 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:43:39.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:43:39 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:43:39.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:43:39 smithi117 ceph-mon[67870]: pgmap v357: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:43:39.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:43:39 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:43:39.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:43:39 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:43:39.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:43:39 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:43:39.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:43:39 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:43:41.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:43:41 smithi022 ceph-mon[88798]: pgmap v358: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:43:41.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:43:41 smithi117 ceph-mon[67870]: pgmap v358: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:43:43.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:43:43 smithi022 ceph-mon[88798]: pgmap v359: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:43:43.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:43:43 smithi117 ceph-mon[67870]: pgmap v359: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:43:45.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:43:45 smithi022 ceph-mon[88798]: pgmap v360: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:43:45.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:43:45 smithi117 ceph-mon[67870]: pgmap v360: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:43:46.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:43:46 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:43:46.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:43:46 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:43:47.464 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:43:47 smithi022 ceph-mon[88798]: pgmap v361: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:43:47.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:43:47 smithi117 ceph-mon[67870]: pgmap v361: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:43:49.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:43:49 smithi022 ceph-mon[88798]: pgmap v362: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:43:49.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:43:49 smithi117 ceph-mon[67870]: pgmap v362: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:43:51.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:43:51 smithi022 ceph-mon[88798]: pgmap v363: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:43:51.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:43:51 smithi117 ceph-mon[67870]: pgmap v363: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:43:53.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:43:53 smithi022 ceph-mon[88798]: pgmap v364: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:43:53.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:43:53 smithi117 ceph-mon[67870]: pgmap v364: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:43:55.689 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:43:55 smithi117 ceph-mon[67870]: pgmap v365: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:43:55.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:43:55 smithi022 ceph-mon[88798]: pgmap v365: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:43:57.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:43:57 smithi022 ceph-mon[88798]: pgmap v366: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:43:57.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:43:57 smithi117 ceph-mon[67870]: pgmap v366: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:43:59.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:43:59 smithi022 ceph-mon[88798]: pgmap v367: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:43:59.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:43:59 smithi117 ceph-mon[67870]: pgmap v367: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:44:01.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:44:01 smithi022 ceph-mon[88798]: pgmap v368: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:44:01.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:44:01 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:44:01.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:44:01 smithi117 ceph-mon[67870]: pgmap v368: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:44:01.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:44:01 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:44:03.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:44:03 smithi022 ceph-mon[88798]: pgmap v369: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:44:03.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:44:03 smithi117 ceph-mon[67870]: pgmap v369: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:44:05.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:44:05 smithi022 ceph-mon[88798]: pgmap v370: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:44:05.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:44:05 smithi117 ceph-mon[67870]: pgmap v370: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:44:07.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:44:07 smithi117 ceph-mon[67870]: pgmap v371: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:44:07.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:44:07 smithi022 ceph-mon[88798]: pgmap v371: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:44:09.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:44:09 smithi022 ceph-mon[88798]: pgmap v372: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:44:09.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:44:09 smithi117 ceph-mon[67870]: pgmap v372: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:44:11.714 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:44:11 smithi022 ceph-mon[88798]: pgmap v373: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:44:11.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:44:11 smithi117 ceph-mon[67870]: pgmap v373: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:44:13.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:44:13 smithi022 ceph-mon[88798]: pgmap v374: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:44:13.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:44:13 smithi117 ceph-mon[67870]: pgmap v374: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:44:15.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:44:15 smithi022 ceph-mon[88798]: pgmap v375: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:44:15.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:44:15 smithi117 ceph-mon[67870]: pgmap v375: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:44:16.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:44:16 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:44:16.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:44:16 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:44:17.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:44:17 smithi117 ceph-mon[67870]: pgmap v376: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:44:17.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:44:17 smithi022 ceph-mon[88798]: pgmap v376: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:44:19.855 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:44:19 smithi022 ceph-mon[88798]: pgmap v377: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:44:19.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:44:19 smithi117 ceph-mon[67870]: pgmap v377: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:44:21.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:44:21 smithi117 ceph-mon[67870]: pgmap v378: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:44:21.964 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:44:21 smithi022 ceph-mon[88798]: pgmap v378: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:44:23.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:44:23 smithi117 ceph-mon[67870]: pgmap v379: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:44:23.964 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:44:23 smithi022 ceph-mon[88798]: pgmap v379: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:44:25.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:44:25 smithi117 ceph-mon[67870]: pgmap v380: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:44:25.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:44:25 smithi022 ceph-mon[88798]: pgmap v380: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:44:27.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:44:27 smithi117 ceph-mon[67870]: pgmap v381: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:44:27.964 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:44:27 smithi022 ceph-mon[88798]: pgmap v381: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:44:29.766 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:44:29 smithi022 ceph-mon[88798]: pgmap v382: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:44:29.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:44:29 smithi117 ceph-mon[67870]: pgmap v382: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:44:31.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:44:31 smithi117 ceph-mon[67870]: pgmap v383: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:44:31.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:44:31 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:44:31.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:44:31 smithi022 ceph-mon[88798]: pgmap v383: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:44:31.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:44:31 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:44:33.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:44:33 smithi117 ceph-mon[67870]: pgmap v384: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:44:33.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:44:33 smithi022 ceph-mon[88798]: pgmap v384: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:44:35.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:44:35 smithi117 ceph-mon[67870]: pgmap v385: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:44:35.964 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:44:35 smithi022 ceph-mon[88798]: pgmap v385: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:44:37.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:44:37 smithi117 ceph-mon[67870]: pgmap v386: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:44:37.964 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:44:37 smithi022 ceph-mon[88798]: pgmap v386: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:44:39.855 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:44:39 smithi022 ceph-mon[88798]: pgmap v387: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:44:39.855 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:44:39 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:44:39.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:44:39 smithi117 ceph-mon[67870]: pgmap v387: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:44:39.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:44:39 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:44:40.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:44:40 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:44:40.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:44:40 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:44:40.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:44:40 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:44:40.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:44:40 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:44:40.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:44:40 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:44:40.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:44:40 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:44:41.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:44:41 smithi117 ceph-mon[67870]: pgmap v388: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:44:41.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:44:41 smithi022 ceph-mon[88798]: pgmap v388: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:44:43.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:44:43 smithi117 ceph-mon[67870]: pgmap v389: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:44:43.964 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:44:43 smithi022 ceph-mon[88798]: pgmap v389: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:44:45.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:44:45 smithi117 ceph-mon[67870]: pgmap v390: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:44:45.964 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:44:45 smithi022 ceph-mon[88798]: pgmap v390: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:44:46.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:44:46 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:44:46.964 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:44:46 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:44:47.826 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:44:47 smithi022 ceph-mon[88798]: pgmap v391: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:44:47.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:44:47 smithi117 ceph-mon[67870]: pgmap v391: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:44:49.855 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:44:49 smithi022 ceph-mon[88798]: pgmap v392: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:44:49.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:44:49 smithi117 ceph-mon[67870]: pgmap v392: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:44:51.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:44:51 smithi117 ceph-mon[67870]: pgmap v393: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:44:51.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:44:51 smithi022 ceph-mon[88798]: pgmap v393: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:44:53.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:44:53 smithi117 ceph-mon[67870]: pgmap v394: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:44:53.964 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:44:53 smithi022 ceph-mon[88798]: pgmap v394: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:44:55.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:44:55 smithi117 ceph-mon[67870]: pgmap v395: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:44:55.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:44:55 smithi022 ceph-mon[88798]: pgmap v395: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:44:57.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:44:57 smithi117 ceph-mon[67870]: pgmap v396: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:44:57.933 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:44:57 smithi022 ceph-mon[88798]: pgmap v396: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:44:59.855 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:44:59 smithi022 ceph-mon[88798]: pgmap v397: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:44:59.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:44:59 smithi117 ceph-mon[67870]: pgmap v397: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:45:01.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:45:01 smithi117 ceph-mon[67870]: pgmap v398: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:45:01.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:45:01 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:45:01.964 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:45:01 smithi022 ceph-mon[88798]: pgmap v398: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:45:01.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:45:01 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:45:03.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:45:03 smithi117 ceph-mon[67870]: pgmap v399: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:45:03.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:45:03 smithi022 ceph-mon[88798]: pgmap v399: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:45:05.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:45:05 smithi117 ceph-mon[67870]: pgmap v400: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:45:05.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:45:05 smithi022 ceph-mon[88798]: pgmap v400: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:45:07.854 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:45:07 smithi117 ceph-mon[67870]: pgmap v401: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:45:07.964 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:45:07 smithi022 ceph-mon[88798]: pgmap v401: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:45:09.854 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:45:09 smithi022 ceph-mon[88798]: pgmap v402: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:45:09.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:45:09 smithi117 ceph-mon[67870]: pgmap v402: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:45:11.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:45:11 smithi117 ceph-mon[67870]: pgmap v403: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:45:11.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:45:11 smithi022 ceph-mon[88798]: pgmap v403: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:45:13.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:45:13 smithi117 ceph-mon[67870]: pgmap v404: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:45:13.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:45:13 smithi022 ceph-mon[88798]: pgmap v404: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:45:15.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:45:15 smithi117 ceph-mon[67870]: pgmap v405: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:45:15.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:45:15 smithi022 ceph-mon[88798]: pgmap v405: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:45:16.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:45:16 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:45:16.964 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:45:16 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:45:17.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:45:17 smithi117 ceph-mon[67870]: pgmap v406: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:45:17.964 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:45:17 smithi022 ceph-mon[88798]: pgmap v406: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:45:19.854 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:45:19 smithi022 ceph-mon[88798]: pgmap v407: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:45:19.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:45:19 smithi117 ceph-mon[67870]: pgmap v407: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:45:21.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:45:21 smithi117 ceph-mon[67870]: pgmap v408: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:45:21.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:45:21 smithi022 ceph-mon[88798]: pgmap v408: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:45:23.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:45:23 smithi117 ceph-mon[67870]: pgmap v409: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:45:23.964 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:45:23 smithi022 ceph-mon[88798]: pgmap v409: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:45:25.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:45:25 smithi117 ceph-mon[67870]: pgmap v410: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:45:25.964 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:45:25 smithi022 ceph-mon[88798]: pgmap v410: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:45:27.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:45:27 smithi117 ceph-mon[67870]: pgmap v411: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:45:27.964 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:45:27 smithi022 ceph-mon[88798]: pgmap v411: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:45:29.855 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:45:29 smithi022 ceph-mon[88798]: pgmap v412: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:45:29.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:45:29 smithi117 ceph-mon[67870]: pgmap v412: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:45:31.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:45:31 smithi117 ceph-mon[67870]: pgmap v413: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:45:31.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:45:31 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:45:31.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:45:31 smithi022 ceph-mon[88798]: pgmap v413: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:45:31.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:45:31 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:45:33.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:45:33 smithi117 ceph-mon[67870]: pgmap v414: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:45:33.964 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:45:33 smithi022 ceph-mon[88798]: pgmap v414: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:45:35.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:45:35 smithi117 ceph-mon[67870]: pgmap v415: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:45:35.964 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:45:35 smithi022 ceph-mon[88798]: pgmap v415: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:45:37.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:45:37 smithi117 ceph-mon[67870]: pgmap v416: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:45:37.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:45:37 smithi022 ceph-mon[88798]: pgmap v416: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:45:39.855 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:45:39 smithi022 ceph-mon[88798]: pgmap v417: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:45:39.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:45:39 smithi117 ceph-mon[67870]: pgmap v417: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:45:40.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:45:40 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:45:40.964 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:45:40 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:45:41.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:45:41 smithi117 ceph-mon[67870]: pgmap v418: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:45:41.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:45:41 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:45:41.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:45:41 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:45:41.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:45:41 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:45:41.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:45:41 smithi022 ceph-mon[88798]: pgmap v418: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:45:41.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:45:41 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:45:41.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:45:41 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:45:41.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:45:41 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:45:43.715 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:45:43 smithi022 ceph-mon[88798]: pgmap v419: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:45:43.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:45:43 smithi117 ceph-mon[67870]: pgmap v419: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:45:45.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:45:45 smithi117 ceph-mon[67870]: pgmap v420: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:45:45.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:45:45 smithi022 ceph-mon[88798]: pgmap v420: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:45:46.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:45:46 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:45:46.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:45:46 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:45:47.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:45:47 smithi117 ceph-mon[67870]: pgmap v421: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:45:47.964 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:45:47 smithi022 ceph-mon[88798]: pgmap v421: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:45:49.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:45:49 smithi117 ceph-mon[67870]: pgmap v422: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:45:49.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:45:49 smithi022 ceph-mon[88798]: pgmap v422: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:45:51.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:45:51 smithi117 ceph-mon[67870]: pgmap v423: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:45:51.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:45:51 smithi022 ceph-mon[88798]: pgmap v423: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:45:53.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:45:53 smithi117 ceph-mon[67870]: pgmap v424: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:45:53.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:45:53 smithi022 ceph-mon[88798]: pgmap v424: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:45:55.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:45:55 smithi117 ceph-mon[67870]: pgmap v425: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:45:55.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:45:55 smithi022 ceph-mon[88798]: pgmap v425: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:45:57.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:45:57 smithi117 ceph-mon[67870]: pgmap v426: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:45:57.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:45:57 smithi022 ceph-mon[88798]: pgmap v426: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:45:59.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:45:59 smithi117 ceph-mon[67870]: pgmap v427: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:45:59.964 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:45:59 smithi022 ceph-mon[88798]: pgmap v427: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:46:01.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:46:01 smithi117 ceph-mon[67870]: pgmap v428: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:46:01.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:46:01 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:46:01.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:46:01 smithi022 ceph-mon[88798]: pgmap v428: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:46:01.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:46:01 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:46:03.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:46:03 smithi117 ceph-mon[67870]: pgmap v429: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:46:03.964 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:46:03 smithi022 ceph-mon[88798]: pgmap v429: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:46:05.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:46:05 smithi022 ceph-mon[88798]: pgmap v430: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:46:06.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:46:05 smithi117 ceph-mon[67870]: pgmap v430: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:46:07.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:46:07 smithi022 ceph-mon[88798]: pgmap v431: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:46:08.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:46:07 smithi117 ceph-mon[67870]: pgmap v431: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:46:09.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:46:09 smithi022 ceph-mon[88798]: pgmap v432: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:46:10.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:46:09 smithi117 ceph-mon[67870]: pgmap v432: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:46:11.964 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:46:11 smithi022 ceph-mon[88798]: pgmap v433: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:46:12.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:46:11 smithi117 ceph-mon[67870]: pgmap v433: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:46:13.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:46:13 smithi117 ceph-mon[67870]: pgmap v434: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:46:13.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:46:13 smithi022 ceph-mon[88798]: pgmap v434: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:46:15.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:46:15 smithi022 ceph-mon[88798]: pgmap v435: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:46:16.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:46:15 smithi117 ceph-mon[67870]: pgmap v435: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:46:16.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:46:16 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:46:17.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:46:16 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:46:17.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:46:17 smithi022 ceph-mon[88798]: pgmap v436: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:46:18.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:46:17 smithi117 ceph-mon[67870]: pgmap v436: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:46:19.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:46:19 smithi022 ceph-mon[88798]: pgmap v437: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:46:20.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:46:19 smithi117 ceph-mon[67870]: pgmap v437: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:46:21.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:46:21 smithi022 ceph-mon[88798]: pgmap v438: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:46:22.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:46:21 smithi117 ceph-mon[67870]: pgmap v438: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:46:23.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:46:23 smithi117 ceph-mon[67870]: pgmap v439: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:46:23.964 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:46:23 smithi022 ceph-mon[88798]: pgmap v439: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:46:25.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:46:25 smithi022 ceph-mon[88798]: pgmap v440: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:46:26.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:46:25 smithi117 ceph-mon[67870]: pgmap v440: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:46:27.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:46:27 smithi022 ceph-mon[88798]: pgmap v441: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:46:28.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:46:27 smithi117 ceph-mon[67870]: pgmap v441: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:46:29.855 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:46:29 smithi022 ceph-mon[88798]: pgmap v442: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:46:30.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:46:29 smithi117 ceph-mon[67870]: pgmap v442: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:46:31.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:46:31 smithi022 ceph-mon[88798]: pgmap v443: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:46:31.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:46:31 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:46:32.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:46:31 smithi117 ceph-mon[67870]: pgmap v443: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:46:32.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:46:31 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:46:33.938 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:46:33 smithi022 ceph-mon[88798]: pgmap v444: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:46:34.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:46:33 smithi117 ceph-mon[67870]: pgmap v444: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:46:35.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:46:35 smithi022 ceph-mon[88798]: pgmap v445: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:46:36.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:46:35 smithi117 ceph-mon[67870]: pgmap v445: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:46:37.964 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:46:37 smithi022 ceph-mon[88798]: pgmap v446: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:46:38.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:46:37 smithi117 ceph-mon[67870]: pgmap v446: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:46:39.945 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:46:39 smithi022 ceph-mon[88798]: pgmap v447: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:46:40.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:46:39 smithi117 ceph-mon[67870]: pgmap v447: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:46:41.964 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:46:41 smithi022 ceph-mon[88798]: pgmap v448: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:46:41.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:46:41 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:46:41.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:46:41 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:46:41.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:46:41 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:46:41.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:46:41 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:46:42.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:46:41 smithi117 ceph-mon[67870]: pgmap v448: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:46:42.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:46:41 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:46:42.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:46:41 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:46:42.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:46:41 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:46:42.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:46:41 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' 2024-02-16T19:46:43.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:46:43 smithi022 ceph-mon[88798]: pgmap v449: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:46:44.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:46:43 smithi117 ceph-mon[67870]: pgmap v449: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:46:45.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:46:45 smithi022 ceph-mon[88798]: pgmap v450: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:46:46.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:46:45 smithi117 ceph-mon[67870]: pgmap v450: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:46:46.964 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:46:46 smithi022 ceph-mon[88798]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:46:47.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:46:46 smithi117 ceph-mon[67870]: from='mgr.34104 172.21.15.22:0/3611919567' entity='mgr.smithi022.gmdcch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:46:47.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:46:47 smithi022 ceph-mon[88798]: pgmap v451: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:46:48.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:46:47 smithi117 ceph-mon[67870]: pgmap v451: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:46:48.199 ERROR:tasks.cephfs.fuse_mount:process failed to terminate after unmount. This probably indicates a bug within ceph-fuse. 2024-02-16T19:46:48.200 ERROR:teuthology.run_tasks:Manager failed: ceph-fuse Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_batrick_ceph_ed7ab725e1a43615009dfb74a42bf33d80081bc8/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/github.com_batrick_ceph_ed7ab725e1a43615009dfb74a42bf33d80081bc8/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-02-16T19:46:48.202 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-02-16T19:46:48.213 ERROR:teuthology.run_tasks:Manager failed: full_sequential_finally Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/task/full_sequential_finally.py", line 38, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_batrick_ceph_ed7ab725e1a43615009dfb74a42bf33d80081bc8/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/github.com_batrick_ceph_ed7ab725e1a43615009dfb74a42bf33d80081bc8/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/task/full_sequential_finally.py", line 43, in task ((taskname, confg),) = entry.items() ValueError: too many values to unpack (expected 1) 2024-02-16T19:46:48.213 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-02-16T19:46:48.225 INFO:tasks.cephadm:Teardown begin 2024-02-16T19:46:48.225 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/github.com_batrick_ceph_ed7ab725e1a43615009dfb74a42bf33d80081bc8/qa/tasks/cephadm.py", line 1807, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_batrick_ceph_ed7ab725e1a43615009dfb74a42bf33d80081bc8/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/github.com_batrick_ceph_ed7ab725e1a43615009dfb74a42bf33d80081bc8/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-02-16T19:46:48.226 DEBUG:teuthology.orchestra.run.smithi022:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-16T19:46:48.262 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-16T19:46:48.297 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-02-16T19:46:48.297 DEBUG:teuthology.orchestra.run.smithi022:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-16T19:46:48.319 DEBUG:teuthology.orchestra.run.smithi117:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-16T19:46:48.353 INFO:tasks.cephadm:Stopping all daemons... 2024-02-16T19:46:48.353 INFO:tasks.cephadm.mon.smithi022:Stopping mon.smithi022... 2024-02-16T19:46:48.354 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ceph-2a1346e6-cd00-11ee-95ba-87774f69a715@mon.smithi022 2024-02-16T19:46:48.678 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:46:48 smithi022 systemd[1]: Stopping Ceph mon.smithi022 for 2a1346e6-cd00-11ee-95ba-87774f69a715... 2024-02-16T19:46:48.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:46:48 smithi022 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-mon-smithi022[88794]: 2024-02-16T19:46:48.676+0000 7fe667b9f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi022 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-16T19:46:48.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:46:48 smithi022 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-mon-smithi022[88794]: 2024-02-16T19:46:48.676+0000 7fe667b9f700 -1 mon.smithi022@0(leader) e3 *** Got Signal Terminated *** 2024-02-16T19:46:48.965 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:46:48 smithi022 podman[134521]: 2024-02-16 19:46:48.718483768 +0000 UTC m=+0.162688495 container died 9c84755ae67dec075119dc6bbcd217bfd244588ae7131bc1b01843432cd24743 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-mon-smithi022, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0) 2024-02-16T19:46:49.464 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:46:49 smithi022 podman[134521]: 2024-02-16 19:46:49.302866059 +0000 UTC m=+0.747070785 container cleanup 9c84755ae67dec075119dc6bbcd217bfd244588ae7131bc1b01843432cd24743 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-mon-smithi022, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-16T19:46:49.465 INFO:journalctl@ceph.mon.smithi022.smithi022.stdout:Feb 16 19:46:49 smithi022 bash[134521]: ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-mon-smithi022 2024-02-16T19:46:49.727 DEBUG:teuthology.orchestra.run.smithi022:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a1346e6-cd00-11ee-95ba-87774f69a715@mon.smithi022.service' 2024-02-16T19:46:49.786 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-16T19:46:49.786 INFO:tasks.cephadm.mon.smithi022:Stopped mon.smithi022 2024-02-16T19:46:49.786 INFO:tasks.cephadm.mon.smithi117:Stopping mon.smithi117... 2024-02-16T19:46:49.786 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-2a1346e6-cd00-11ee-95ba-87774f69a715@mon.smithi117 2024-02-16T19:46:50.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:46:49 smithi117 systemd[1]: Stopping Ceph mon.smithi117 for 2a1346e6-cd00-11ee-95ba-87774f69a715... 2024-02-16T19:46:50.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:46:50 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-mon-smithi117[67866]: 2024-02-16T19:46:50.311+0000 7f48cddfc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi117 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-16T19:46:50.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:46:50 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-mon-smithi117[67866]: 2024-02-16T19:46:50.311+0000 7f48cddfc700 -1 mon.smithi117@1(peon) e3 *** Got Signal Terminated *** 2024-02-16T19:46:50.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:46:50 smithi117 podman[104903]: 2024-02-16 19:46:50.547116871 +0000 UTC m=+0.564885978 container died a319ac50f1a9c776e6b81aca2833d96d5fb6d8949afc2ad72abe37a644414d62 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-mon-smithi117, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-02-16T19:46:51.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:46:51 smithi117 podman[104903]: 2024-02-16 19:46:51.012391007 +0000 UTC m=+1.030160112 container cleanup a319ac50f1a9c776e6b81aca2833d96d5fb6d8949afc2ad72abe37a644414d62 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-mon-smithi117, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-16T19:46:51.137 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Feb 16 19:46:51 smithi117 bash[104903]: ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-mon-smithi117 2024-02-16T19:46:51.478 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a1346e6-cd00-11ee-95ba-87774f69a715@mon.smithi117.service' 2024-02-16T19:46:51.521 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-16T19:46:51.521 INFO:tasks.cephadm.mon.smithi117:Stopped mon.smithi117 2024-02-16T19:46:51.521 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-02-16T19:46:51.521 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ceph-2a1346e6-cd00-11ee-95ba-87774f69a715@osd.0 2024-02-16T19:46:51.824 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 16 19:46:51 smithi022 systemd[1]: Stopping Ceph osd.0 for 2a1346e6-cd00-11ee-95ba-87774f69a715... 2024-02-16T19:46:52.215 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 16 19:46:51 smithi022 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-0[100545]: 2024-02-16T19:46:51.823+0000 7fd6a546b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-16T19:46:52.215 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 16 19:46:51 smithi022 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-0[100545]: 2024-02-16T19:46:51.823+0000 7fd6a546b700 -1 osd.0 93 *** Got signal Terminated *** 2024-02-16T19:46:52.215 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 16 19:46:51 smithi022 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-0[100545]: 2024-02-16T19:46:51.823+0000 7fd6a546b700 -1 osd.0 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-16T19:46:57.634 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 16 19:46:57 smithi022 podman[134655]: 2024-02-16 19:46:57.243598024 +0000 UTC m=+5.525764299 container died 002412b3525f0cf864b7f21cbc310f8c070b81f979d03759b1dd184a70337369 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-02-16T19:46:57.965 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 16 19:46:57 smithi022 podman[134655]: 2024-02-16 19:46:57.723615248 +0000 UTC m=+6.005781521 container cleanup 002412b3525f0cf864b7f21cbc310f8c070b81f979d03759b1dd184a70337369 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2) 2024-02-16T19:46:57.965 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 16 19:46:57 smithi022 bash[134655]: ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-0 2024-02-16T19:46:58.349 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 16 19:46:57 smithi022 podman[134672]: 2024-02-16 19:46:57.992251911 +0000 UTC m=+0.747429096 container remove 002412b3525f0cf864b7f21cbc310f8c070b81f979d03759b1dd184a70337369 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-0, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-02-16T19:46:58.601 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 16 19:46:58 smithi022 podman[134765]: 2024-02-16 19:46:58.252533397 +0000 UTC m=+0.018401014 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T19:46:58.601 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 16 19:46:58 smithi022 podman[134765]: 2024-02-16 19:46:58.484833263 +0000 UTC m=+0.250700872 container create a97e168d0c3788b49da85014a7f2b420a48975f2dd29c930e38f40ef95476c3b (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-0-deactivate, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-02-16T19:46:58.965 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 16 19:46:58 smithi022 podman[134765]: 2024-02-16 19:46:58.729383746 +0000 UTC m=+0.495251371 container init a97e168d0c3788b49da85014a7f2b420a48975f2dd29c930e38f40ef95476c3b (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-0-deactivate, org.label-schema.build-date=20240214, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-02-16T19:46:58.965 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 16 19:46:58 smithi022 podman[134765]: 2024-02-16 19:46:58.732947189 +0000 UTC m=+0.498814808 container start a97e168d0c3788b49da85014a7f2b420a48975f2dd29c930e38f40ef95476c3b (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-0-deactivate, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-16T19:46:58.965 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 16 19:46:58 smithi022 podman[134765]: 2024-02-16 19:46:58.829083349 +0000 UTC m=+0.594950961 container attach a97e168d0c3788b49da85014a7f2b420a48975f2dd29c930e38f40ef95476c3b (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-0-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-02-16T19:46:59.322 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 16 19:46:59 smithi022 podman[134765]: 2024-02-16 19:46:59.037165266 +0000 UTC m=+0.803032876 container died a97e168d0c3788b49da85014a7f2b420a48975f2dd29c930e38f40ef95476c3b (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5) 2024-02-16T19:46:59.850 DEBUG:teuthology.orchestra.run.smithi022:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a1346e6-cd00-11ee-95ba-87774f69a715@osd.0.service' 2024-02-16T19:46:59.892 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-16T19:46:59.892 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-02-16T19:46:59.892 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-02-16T19:46:59.892 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ceph-2a1346e6-cd00-11ee-95ba-87774f69a715@osd.1 2024-02-16T19:47:00.215 INFO:journalctl@ceph.osd.1.smithi022.stdout:Feb 16 19:46:59 smithi022 systemd[1]: Stopping Ceph osd.1 for 2a1346e6-cd00-11ee-95ba-87774f69a715... 2024-02-16T19:47:00.715 INFO:journalctl@ceph.osd.1.smithi022.stdout:Feb 16 19:47:00 smithi022 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-1[106121]: 2024-02-16T19:47:00.247+0000 7fddad13b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-16T19:47:00.715 INFO:journalctl@ceph.osd.1.smithi022.stdout:Feb 16 19:47:00 smithi022 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-1[106121]: 2024-02-16T19:47:00.247+0000 7fddad13b700 -1 osd.1 93 *** Got signal Terminated *** 2024-02-16T19:47:00.715 INFO:journalctl@ceph.osd.1.smithi022.stdout:Feb 16 19:47:00 smithi022 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-1[106121]: 2024-02-16T19:47:00.247+0000 7fddad13b700 -1 osd.1 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-16T19:47:05.947 INFO:journalctl@ceph.osd.1.smithi022.stdout:Feb 16 19:47:05 smithi022 podman[134883]: 2024-02-16 19:47:05.667594281 +0000 UTC m=+5.540507002 container died c3488ce7b2a09e1cd142baaa0c58e48dd65a040fac998c39af24cd5d21f4fc1c (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-1, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-02-16T19:47:06.215 INFO:journalctl@ceph.osd.1.smithi022.stdout:Feb 16 19:47:06 smithi022 podman[134883]: 2024-02-16 19:47:06.036267498 +0000 UTC m=+5.909180222 container cleanup c3488ce7b2a09e1cd142baaa0c58e48dd65a040fac998c39af24cd5d21f4fc1c (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True) 2024-02-16T19:47:06.215 INFO:journalctl@ceph.osd.1.smithi022.stdout:Feb 16 19:47:06 smithi022 bash[134883]: ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-1 2024-02-16T19:47:06.605 INFO:journalctl@ceph.osd.1.smithi022.stdout:Feb 16 19:47:06 smithi022 podman[134901]: 2024-02-16 19:47:06.279026035 +0000 UTC m=+0.615012390 container remove c3488ce7b2a09e1cd142baaa0c58e48dd65a040fac998c39af24cd5d21f4fc1c (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-1, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-02-16T19:47:06.899 INFO:journalctl@ceph.osd.1.smithi022.stdout:Feb 16 19:47:06 smithi022 podman[134995]: 2024-02-16 19:47:06.50842433 +0000 UTC m=+0.018079987 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T19:47:06.899 INFO:journalctl@ceph.osd.1.smithi022.stdout:Feb 16 19:47:06 smithi022 podman[134995]: 2024-02-16 19:47:06.613333134 +0000 UTC m=+0.122988790 container create bdbaa9fe08181b99177556695360042a006c3f41515c3310c1543a11b2695aa9 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-1-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-02-16T19:47:06.900 INFO:journalctl@ceph.osd.1.smithi022.stdout:Feb 16 19:47:06 smithi022 podman[134995]: 2024-02-16 19:47:06.799386149 +0000 UTC m=+0.309041813 container init bdbaa9fe08181b99177556695360042a006c3f41515c3310c1543a11b2695aa9 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-1-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.build-date=20240214) 2024-02-16T19:47:06.900 INFO:journalctl@ceph.osd.1.smithi022.stdout:Feb 16 19:47:06 smithi022 podman[134995]: 2024-02-16 19:47:06.802448522 +0000 UTC m=+0.312104181 container start bdbaa9fe08181b99177556695360042a006c3f41515c3310c1543a11b2695aa9 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-1-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-16T19:47:06.900 INFO:journalctl@ceph.osd.1.smithi022.stdout:Feb 16 19:47:06 smithi022 podman[134995]: 2024-02-16 19:47:06.899297872 +0000 UTC m=+0.408953540 container attach bdbaa9fe08181b99177556695360042a006c3f41515c3310c1543a11b2695aa9 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-1-deactivate, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS) 2024-02-16T19:47:07.215 INFO:journalctl@ceph.osd.1.smithi022.stdout:Feb 16 19:47:07 smithi022 podman[134995]: 2024-02-16 19:47:07.108608201 +0000 UTC m=+0.618263869 container died bdbaa9fe08181b99177556695360042a006c3f41515c3310c1543a11b2695aa9 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-1-deactivate, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.build-date=20240214, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-16T19:47:07.772 DEBUG:teuthology.orchestra.run.smithi022:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a1346e6-cd00-11ee-95ba-87774f69a715@osd.1.service' 2024-02-16T19:47:07.816 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-16T19:47:07.816 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-02-16T19:47:07.817 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-02-16T19:47:07.817 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ceph-2a1346e6-cd00-11ee-95ba-87774f69a715@osd.2 2024-02-16T19:47:08.148 INFO:journalctl@ceph.osd.2.smithi022.stdout:Feb 16 19:47:07 smithi022 systemd[1]: Stopping Ceph osd.2 for 2a1346e6-cd00-11ee-95ba-87774f69a715... 2024-02-16T19:47:08.465 INFO:journalctl@ceph.osd.2.smithi022.stdout:Feb 16 19:47:08 smithi022 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-2[112002]: 2024-02-16T19:47:08.146+0000 7fd3a1d73700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-16T19:47:08.465 INFO:journalctl@ceph.osd.2.smithi022.stdout:Feb 16 19:47:08 smithi022 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-2[112002]: 2024-02-16T19:47:08.146+0000 7fd3a1d73700 -1 osd.2 93 *** Got signal Terminated *** 2024-02-16T19:47:08.465 INFO:journalctl@ceph.osd.2.smithi022.stdout:Feb 16 19:47:08 smithi022 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-2[112002]: 2024-02-16T19:47:08.146+0000 7fd3a1d73700 -1 osd.2 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-16T19:47:13.965 INFO:journalctl@ceph.osd.2.smithi022.stdout:Feb 16 19:47:13 smithi022 podman[135100]: 2024-02-16 19:47:13.606358963 +0000 UTC m=+5.562824680 container died 46138c50f79d2f56a71cf356b2d6cc7c6ffa725baea25f001c38b6e86ee8cde7 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-02-16T19:47:14.232 INFO:journalctl@ceph.osd.2.smithi022.stdout:Feb 16 19:47:13 smithi022 podman[135100]: 2024-02-16 19:47:13.968812548 +0000 UTC m=+5.925278259 container cleanup 46138c50f79d2f56a71cf356b2d6cc7c6ffa725baea25f001c38b6e86ee8cde7 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-2, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-16T19:47:14.232 INFO:journalctl@ceph.osd.2.smithi022.stdout:Feb 16 19:47:13 smithi022 bash[135100]: ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-2 2024-02-16T19:47:14.599 INFO:journalctl@ceph.osd.2.smithi022.stdout:Feb 16 19:47:14 smithi022 podman[135119]: 2024-02-16 19:47:14.231980707 +0000 UTC m=+0.623456997 container remove 46138c50f79d2f56a71cf356b2d6cc7c6ffa725baea25f001c38b6e86ee8cde7 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-2, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-02-16T19:47:14.919 INFO:journalctl@ceph.osd.2.smithi022.stdout:Feb 16 19:47:14 smithi022 podman[135211]: 2024-02-16 19:47:14.501944047 +0000 UTC m=+0.018607513 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T19:47:14.919 INFO:journalctl@ceph.osd.2.smithi022.stdout:Feb 16 19:47:14 smithi022 podman[135211]: 2024-02-16 19:47:14.607969153 +0000 UTC m=+0.124632656 container create 5522bfcaab9662a0c7ea96a2f78b5825c6dd488ac5d85946d6fa78b8f44613e8 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-2-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-02-16T19:47:14.919 INFO:journalctl@ceph.osd.2.smithi022.stdout:Feb 16 19:47:14 smithi022 podman[135211]: 2024-02-16 19:47:14.835242366 +0000 UTC m=+0.351905828 container init 5522bfcaab9662a0c7ea96a2f78b5825c6dd488ac5d85946d6fa78b8f44613e8 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-2-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-02-16T19:47:14.919 INFO:journalctl@ceph.osd.2.smithi022.stdout:Feb 16 19:47:14 smithi022 podman[135211]: 2024-02-16 19:47:14.838431856 +0000 UTC m=+0.355095321 container start 5522bfcaab9662a0c7ea96a2f78b5825c6dd488ac5d85946d6fa78b8f44613e8 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-2-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214) 2024-02-16T19:47:14.919 INFO:journalctl@ceph.osd.2.smithi022.stdout:Feb 16 19:47:14 smithi022 podman[135211]: 2024-02-16 19:47:14.918498869 +0000 UTC m=+0.435162331 container attach 5522bfcaab9662a0c7ea96a2f78b5825c6dd488ac5d85946d6fa78b8f44613e8 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-2-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240215.160715-f78a58c) 2024-02-16T19:47:15.215 INFO:journalctl@ceph.osd.2.smithi022.stdout:Feb 16 19:47:15 smithi022 podman[135211]: 2024-02-16 19:47:15.135887271 +0000 UTC m=+0.652550736 container died 5522bfcaab9662a0c7ea96a2f78b5825c6dd488ac5d85946d6fa78b8f44613e8 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-batrick-testing-20240215.160715-f78a58c) 2024-02-16T19:47:15.631 DEBUG:teuthology.orchestra.run.smithi022:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a1346e6-cd00-11ee-95ba-87774f69a715@osd.2.service' 2024-02-16T19:47:15.672 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-16T19:47:15.672 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-02-16T19:47:15.672 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-02-16T19:47:15.673 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-2a1346e6-cd00-11ee-95ba-87774f69a715@osd.3 2024-02-16T19:47:15.996 INFO:journalctl@ceph.osd.3.smithi117.stdout:Feb 16 19:47:15 smithi117 systemd[1]: Stopping Ceph osd.3 for 2a1346e6-cd00-11ee-95ba-87774f69a715... 2024-02-16T19:47:16.387 INFO:journalctl@ceph.osd.3.smithi117.stdout:Feb 16 19:47:15 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-3[77609]: 2024-02-16T19:47:15.995+0000 7f8cdaf0f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-16T19:47:16.387 INFO:journalctl@ceph.osd.3.smithi117.stdout:Feb 16 19:47:15 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-3[77609]: 2024-02-16T19:47:15.995+0000 7f8cdaf0f700 -1 osd.3 93 *** Got signal Terminated *** 2024-02-16T19:47:16.387 INFO:journalctl@ceph.osd.3.smithi117.stdout:Feb 16 19:47:15 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-3[77609]: 2024-02-16T19:47:15.995+0000 7f8cdaf0f700 -1 osd.3 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-16T19:47:18.637 INFO:journalctl@ceph.osd.3.smithi117.stdout:Feb 16 19:47:18 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-3[77609]: 2024-02-16T19:47:18.331+0000 7f8cd3cfc700 -1 osd.3 93 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-02-16T19:46:56.206414+0000 front 2024-02-16T19:46:56.206312+0000 (oldest deadline 2024-02-16T19:47:17.906110+0000) 2024-02-16T19:47:19.387 INFO:journalctl@ceph.osd.3.smithi117.stdout:Feb 16 19:47:19 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-3[77609]: 2024-02-16T19:47:19.352+0000 7f8cd3cfc700 -1 osd.3 93 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-02-16T19:46:56.206414+0000 front 2024-02-16T19:46:56.206312+0000 (oldest deadline 2024-02-16T19:47:17.906110+0000) 2024-02-16T19:47:19.387 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:47:19 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-5[88302]: 2024-02-16T19:47:19.207+0000 7f45471c7700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-02-16T19:46:53.228875+0000 front 2024-02-16T19:46:53.228863+0000 (oldest deadline 2024-02-16T19:47:19.128661+0000) 2024-02-16T19:47:20.637 INFO:journalctl@ceph.osd.3.smithi117.stdout:Feb 16 19:47:20 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-3[77609]: 2024-02-16T19:47:20.320+0000 7f8cd3cfc700 -1 osd.3 93 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-02-16T19:46:56.206414+0000 front 2024-02-16T19:46:56.206312+0000 (oldest deadline 2024-02-16T19:47:17.906110+0000) 2024-02-16T19:47:20.637 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:47:20 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-5[88302]: 2024-02-16T19:47:20.172+0000 7f45471c7700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-02-16T19:46:53.228875+0000 front 2024-02-16T19:46:53.228863+0000 (oldest deadline 2024-02-16T19:47:19.128661+0000) 2024-02-16T19:47:21.637 INFO:journalctl@ceph.osd.4.smithi117.stdout:Feb 16 19:47:21 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-4[82965]: 2024-02-16T19:47:21.191+0000 7f5245970700 -1 osd.4 93 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-02-16T19:46:54.856780+0000 front 2024-02-16T19:46:54.856963+0000 (oldest deadline 2024-02-16T19:47:20.756753+0000) 2024-02-16T19:47:21.638 INFO:journalctl@ceph.osd.3.smithi117.stdout:Feb 16 19:47:21 smithi117 podman[105041]: 2024-02-16 19:47:21.407044317 +0000 UTC m=+5.535361961 container died 28d7d8519e64e1b196771c34bcfae200203c0798fcff2b240b0884403916aaf6 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-3, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c) 2024-02-16T19:47:21.638 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:47:21 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-5[88302]: 2024-02-16T19:47:21.178+0000 7f45471c7700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-02-16T19:46:53.228875+0000 front 2024-02-16T19:46:53.228863+0000 (oldest deadline 2024-02-16T19:47:19.128661+0000) 2024-02-16T19:47:22.137 INFO:journalctl@ceph.osd.3.smithi117.stdout:Feb 16 19:47:21 smithi117 podman[105041]: 2024-02-16 19:47:21.730308756 +0000 UTC m=+5.858626407 container cleanup 28d7d8519e64e1b196771c34bcfae200203c0798fcff2b240b0884403916aaf6 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-3, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240215.160715-f78a58c) 2024-02-16T19:47:22.137 INFO:journalctl@ceph.osd.3.smithi117.stdout:Feb 16 19:47:21 smithi117 bash[105041]: ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-3 2024-02-16T19:47:22.137 INFO:journalctl@ceph.osd.3.smithi117.stdout:Feb 16 19:47:21 smithi117 podman[105059]: 2024-02-16 19:47:21.898530511 +0000 UTC m=+0.487160755 container remove 28d7d8519e64e1b196771c34bcfae200203c0798fcff2b240b0884403916aaf6 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-3, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-02-16T19:47:22.408 INFO:journalctl@ceph.osd.3.smithi117.stdout:Feb 16 19:47:22 smithi117 podman[105154]: 2024-02-16 19:47:22.193159325 +0000 UTC m=+0.018088118 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T19:47:22.408 INFO:journalctl@ceph.osd.4.smithi117.stdout:Feb 16 19:47:22 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-4[82965]: 2024-02-16T19:47:22.155+0000 7f5245970700 -1 osd.4 93 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-02-16T19:46:54.856780+0000 front 2024-02-16T19:46:54.856963+0000 (oldest deadline 2024-02-16T19:47:20.756753+0000) 2024-02-16T19:47:22.408 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:47:22 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-5[88302]: 2024-02-16T19:47:22.210+0000 7f45471c7700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-02-16T19:46:53.228875+0000 front 2024-02-16T19:46:53.228863+0000 (oldest deadline 2024-02-16T19:47:19.128661+0000) 2024-02-16T19:47:22.719 INFO:journalctl@ceph.osd.3.smithi117.stdout:Feb 16 19:47:22 smithi117 podman[105154]: 2024-02-16 19:47:22.407648924 +0000 UTC m=+0.232577716 container create 414e972946045af3986ddc1919b078d4d49a2a434adbdae6f44838c4bc814f6a (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , ceph=True) 2024-02-16T19:47:22.720 INFO:journalctl@ceph.osd.3.smithi117.stdout:Feb 16 19:47:22 smithi117 podman[105154]: 2024-02-16 19:47:22.618469498 +0000 UTC m=+0.443398291 container init 414e972946045af3986ddc1919b078d4d49a2a434adbdae6f44838c4bc814f6a (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-3-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240215.160715-f78a58c, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-02-16T19:47:22.720 INFO:journalctl@ceph.osd.3.smithi117.stdout:Feb 16 19:47:22 smithi117 podman[105154]: 2024-02-16 19:47:22.621496488 +0000 UTC m=+0.446425289 container start 414e972946045af3986ddc1919b078d4d49a2a434adbdae6f44838c4bc814f6a (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-3-deactivate, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-02-16T19:47:23.137 INFO:journalctl@ceph.osd.3.smithi117.stdout:Feb 16 19:47:22 smithi117 podman[105154]: 2024-02-16 19:47:22.718538892 +0000 UTC m=+0.543467699 container attach 414e972946045af3986ddc1919b078d4d49a2a434adbdae6f44838c4bc814f6a (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-3-deactivate, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2) 2024-02-16T19:47:23.137 INFO:journalctl@ceph.osd.3.smithi117.stdout:Feb 16 19:47:22 smithi117 podman[105154]: 2024-02-16 19:47:22.915053547 +0000 UTC m=+0.739982349 container died 414e972946045af3986ddc1919b078d4d49a2a434adbdae6f44838c4bc814f6a (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-3-deactivate, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240215.160715-f78a58c) 2024-02-16T19:47:23.522 INFO:journalctl@ceph.osd.4.smithi117.stdout:Feb 16 19:47:23 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-4[82965]: 2024-02-16T19:47:23.139+0000 7f5245970700 -1 osd.4 93 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-02-16T19:46:54.856780+0000 front 2024-02-16T19:46:54.856963+0000 (oldest deadline 2024-02-16T19:47:20.756753+0000) 2024-02-16T19:47:23.523 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:47:23 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-5[88302]: 2024-02-16T19:47:23.201+0000 7f45471c7700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-02-16T19:46:53.228875+0000 front 2024-02-16T19:46:53.228863+0000 (oldest deadline 2024-02-16T19:47:19.128661+0000) 2024-02-16T19:47:23.546 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a1346e6-cd00-11ee-95ba-87774f69a715@osd.3.service' 2024-02-16T19:47:23.588 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-16T19:47:23.588 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-02-16T19:47:23.589 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-02-16T19:47:23.589 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-2a1346e6-cd00-11ee-95ba-87774f69a715@osd.4 2024-02-16T19:47:23.830 INFO:journalctl@ceph.osd.4.smithi117.stdout:Feb 16 19:47:23 smithi117 systemd[1]: Stopping Ceph osd.4 for 2a1346e6-cd00-11ee-95ba-87774f69a715... 2024-02-16T19:47:24.137 INFO:journalctl@ceph.osd.4.smithi117.stdout:Feb 16 19:47:23 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-4[82965]: 2024-02-16T19:47:23.958+0000 7f524cb83700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-16T19:47:24.137 INFO:journalctl@ceph.osd.4.smithi117.stdout:Feb 16 19:47:23 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-4[82965]: 2024-02-16T19:47:23.958+0000 7f524cb83700 -1 osd.4 93 *** Got signal Terminated *** 2024-02-16T19:47:24.137 INFO:journalctl@ceph.osd.4.smithi117.stdout:Feb 16 19:47:23 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-4[82965]: 2024-02-16T19:47:23.958+0000 7f524cb83700 -1 osd.4 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-16T19:47:24.637 INFO:journalctl@ceph.osd.4.smithi117.stdout:Feb 16 19:47:24 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-4[82965]: 2024-02-16T19:47:24.163+0000 7f5245970700 -1 osd.4 93 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-02-16T19:46:54.856780+0000 front 2024-02-16T19:46:54.856963+0000 (oldest deadline 2024-02-16T19:47:20.756753+0000) 2024-02-16T19:47:24.637 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:47:24 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-5[88302]: 2024-02-16T19:47:24.178+0000 7f45471c7700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-02-16T19:46:53.228875+0000 front 2024-02-16T19:46:53.228863+0000 (oldest deadline 2024-02-16T19:47:19.128661+0000) 2024-02-16T19:47:25.387 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:47:25 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-5[88302]: 2024-02-16T19:47:25.215+0000 7f45471c7700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-02-16T19:46:53.228875+0000 front 2024-02-16T19:46:53.228863+0000 (oldest deadline 2024-02-16T19:47:19.128661+0000) 2024-02-16T19:47:25.387 INFO:journalctl@ceph.osd.4.smithi117.stdout:Feb 16 19:47:25 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-4[82965]: 2024-02-16T19:47:25.114+0000 7f5245970700 -1 osd.4 93 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-02-16T19:46:54.856780+0000 front 2024-02-16T19:46:54.856963+0000 (oldest deadline 2024-02-16T19:47:20.756753+0000) 2024-02-16T19:47:26.387 INFO:journalctl@ceph.osd.4.smithi117.stdout:Feb 16 19:47:26 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-4[82965]: 2024-02-16T19:47:26.112+0000 7f5245970700 -1 osd.4 93 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-02-16T19:46:54.856780+0000 front 2024-02-16T19:46:54.856963+0000 (oldest deadline 2024-02-16T19:47:20.756753+0000) 2024-02-16T19:47:26.387 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:47:26 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-5[88302]: 2024-02-16T19:47:26.177+0000 7f45471c7700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-02-16T19:46:53.228875+0000 front 2024-02-16T19:46:53.228863+0000 (oldest deadline 2024-02-16T19:47:19.128661+0000) 2024-02-16T19:47:27.387 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:47:27 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-5[88302]: 2024-02-16T19:47:27.210+0000 7f45471c7700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-02-16T19:46:53.228875+0000 front 2024-02-16T19:46:53.228863+0000 (oldest deadline 2024-02-16T19:47:19.128661+0000) 2024-02-16T19:47:27.387 INFO:journalctl@ceph.osd.4.smithi117.stdout:Feb 16 19:47:27 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-4[82965]: 2024-02-16T19:47:27.063+0000 7f5245970700 -1 osd.4 93 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-02-16T19:46:54.856780+0000 front 2024-02-16T19:46:54.856963+0000 (oldest deadline 2024-02-16T19:47:20.756753+0000) 2024-02-16T19:47:28.387 INFO:journalctl@ceph.osd.4.smithi117.stdout:Feb 16 19:47:28 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-4[82965]: 2024-02-16T19:47:28.084+0000 7f5245970700 -1 osd.4 93 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-02-16T19:46:54.856780+0000 front 2024-02-16T19:46:54.856963+0000 (oldest deadline 2024-02-16T19:47:20.756753+0000) 2024-02-16T19:47:28.387 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:47:28 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-5[88302]: 2024-02-16T19:47:28.224+0000 7f45471c7700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-02-16T19:46:53.228875+0000 front 2024-02-16T19:46:53.228863+0000 (oldest deadline 2024-02-16T19:47:19.128661+0000) 2024-02-16T19:47:28.387 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:47:28 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-5[88302]: 2024-02-16T19:47:28.224+0000 7f45471c7700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.22:6814 osd.1 since back 2024-02-16T19:47:04.929841+0000 front 2024-02-16T19:47:04.929885+0000 (oldest deadline 2024-02-16T19:47:27.229676+0000) 2024-02-16T19:47:29.523 INFO:journalctl@ceph.osd.4.smithi117.stdout:Feb 16 19:47:29 smithi117 podman[105258]: 2024-02-16 19:47:29.361305533 +0000 UTC m=+5.527774364 container died 367ba2085f6f29a06e3ca522a7fdf947c147e1569de2b2902493b10c610aebe8 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-4, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-02-16T19:47:29.523 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:47:29 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-5[88302]: 2024-02-16T19:47:29.186+0000 7f45471c7700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-02-16T19:46:53.228875+0000 front 2024-02-16T19:46:53.228863+0000 (oldest deadline 2024-02-16T19:47:19.128661+0000) 2024-02-16T19:47:29.523 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:47:29 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-5[88302]: 2024-02-16T19:47:29.186+0000 7f45471c7700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.22:6814 osd.1 since back 2024-02-16T19:47:04.929841+0000 front 2024-02-16T19:47:04.929885+0000 (oldest deadline 2024-02-16T19:47:27.229676+0000) 2024-02-16T19:47:29.887 INFO:journalctl@ceph.osd.4.smithi117.stdout:Feb 16 19:47:29 smithi117 podman[105258]: 2024-02-16 19:47:29.692116395 +0000 UTC m=+5.858585233 container cleanup 367ba2085f6f29a06e3ca522a7fdf947c147e1569de2b2902493b10c610aebe8 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-4, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS) 2024-02-16T19:47:29.887 INFO:journalctl@ceph.osd.4.smithi117.stdout:Feb 16 19:47:29 smithi117 bash[105258]: ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-4 2024-02-16T19:47:30.236 INFO:journalctl@ceph.osd.4.smithi117.stdout:Feb 16 19:47:29 smithi117 podman[105286]: 2024-02-16 19:47:29.893547742 +0000 UTC m=+0.529170721 container remove 367ba2085f6f29a06e3ca522a7fdf947c147e1569de2b2902493b10c610aebe8 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-4, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True) 2024-02-16T19:47:30.572 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:47:30 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-5[88302]: 2024-02-16T19:47:30.234+0000 7f45471c7700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-02-16T19:46:53.228875+0000 front 2024-02-16T19:46:53.228863+0000 (oldest deadline 2024-02-16T19:47:19.128661+0000) 2024-02-16T19:47:30.572 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:47:30 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-5[88302]: 2024-02-16T19:47:30.234+0000 7f45471c7700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.22:6814 osd.1 since back 2024-02-16T19:47:04.929841+0000 front 2024-02-16T19:47:04.929885+0000 (oldest deadline 2024-02-16T19:47:27.229676+0000) 2024-02-16T19:47:30.572 INFO:journalctl@ceph.osd.4.smithi117.stdout:Feb 16 19:47:30 smithi117 podman[105378]: 2024-02-16 19:47:30.213696508 +0000 UTC m=+0.017695346 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T19:47:30.572 INFO:journalctl@ceph.osd.4.smithi117.stdout:Feb 16 19:47:30 smithi117 podman[105378]: 2024-02-16 19:47:30.336083534 +0000 UTC m=+0.140082356 container create 91ef503223f205c78001485d8a966ba4e4b92cd83ed08a6f9b942ca5e4c323fe (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-4-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-02-16T19:47:30.573 INFO:journalctl@ceph.osd.4.smithi117.stdout:Feb 16 19:47:30 smithi117 podman[105378]: 2024-02-16 19:47:30.571812759 +0000 UTC m=+0.375811582 container init 91ef503223f205c78001485d8a966ba4e4b92cd83ed08a6f9b942ca5e4c323fe (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-16T19:47:30.863 INFO:journalctl@ceph.osd.4.smithi117.stdout:Feb 16 19:47:30 smithi117 podman[105378]: 2024-02-16 19:47:30.575227846 +0000 UTC m=+0.379226678 container start 91ef503223f205c78001485d8a966ba4e4b92cd83ed08a6f9b942ca5e4c323fe (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-4-deactivate, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-02-16T19:47:30.863 INFO:journalctl@ceph.osd.4.smithi117.stdout:Feb 16 19:47:30 smithi117 podman[105378]: 2024-02-16 19:47:30.663779289 +0000 UTC m=+0.467778127 container attach 91ef503223f205c78001485d8a966ba4e4b92cd83ed08a6f9b942ca5e4c323fe (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-4-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-02-16T19:47:31.137 INFO:journalctl@ceph.osd.4.smithi117.stdout:Feb 16 19:47:30 smithi117 podman[105378]: 2024-02-16 19:47:30.863512823 +0000 UTC m=+0.667511649 container died 91ef503223f205c78001485d8a966ba4e4b92cd83ed08a6f9b942ca5e4c323fe (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-4-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-16T19:47:31.451 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:47:31 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-5[88302]: 2024-02-16T19:47:31.243+0000 7f45471c7700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-02-16T19:46:53.228875+0000 front 2024-02-16T19:46:53.228863+0000 (oldest deadline 2024-02-16T19:47:19.128661+0000) 2024-02-16T19:47:31.451 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:47:31 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-5[88302]: 2024-02-16T19:47:31.243+0000 7f45471c7700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.22:6814 osd.1 since back 2024-02-16T19:47:04.929841+0000 front 2024-02-16T19:47:04.929885+0000 (oldest deadline 2024-02-16T19:47:27.229676+0000) 2024-02-16T19:47:31.469 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a1346e6-cd00-11ee-95ba-87774f69a715@osd.4.service' 2024-02-16T19:47:31.511 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-16T19:47:31.511 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-02-16T19:47:31.511 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-02-16T19:47:31.511 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-2a1346e6-cd00-11ee-95ba-87774f69a715@osd.5 2024-02-16T19:47:31.829 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:47:31 smithi117 systemd[1]: Stopping Ceph osd.5 for 2a1346e6-cd00-11ee-95ba-87774f69a715... 2024-02-16T19:47:32.137 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:47:31 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-5[88302]: 2024-02-16T19:47:31.828+0000 7f454e3da700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-16T19:47:32.137 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:47:31 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-5[88302]: 2024-02-16T19:47:31.828+0000 7f454e3da700 -1 osd.5 93 *** Got signal Terminated *** 2024-02-16T19:47:32.137 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:47:31 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-5[88302]: 2024-02-16T19:47:31.828+0000 7f454e3da700 -1 osd.5 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-16T19:47:32.637 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:47:32 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-5[88302]: 2024-02-16T19:47:32.235+0000 7f45471c7700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-02-16T19:46:53.228875+0000 front 2024-02-16T19:46:53.228863+0000 (oldest deadline 2024-02-16T19:47:19.128661+0000) 2024-02-16T19:47:32.637 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:47:32 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-5[88302]: 2024-02-16T19:47:32.235+0000 7f45471c7700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.22:6814 osd.1 since back 2024-02-16T19:47:04.929841+0000 front 2024-02-16T19:47:04.929885+0000 (oldest deadline 2024-02-16T19:47:27.229676+0000) 2024-02-16T19:47:33.637 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:47:33 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-5[88302]: 2024-02-16T19:47:33.219+0000 7f45471c7700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-02-16T19:46:53.228875+0000 front 2024-02-16T19:46:53.228863+0000 (oldest deadline 2024-02-16T19:47:19.128661+0000) 2024-02-16T19:47:33.637 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:47:33 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-5[88302]: 2024-02-16T19:47:33.219+0000 7f45471c7700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.22:6814 osd.1 since back 2024-02-16T19:47:04.929841+0000 front 2024-02-16T19:47:04.929885+0000 (oldest deadline 2024-02-16T19:47:27.229676+0000) 2024-02-16T19:47:34.637 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:47:34 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-5[88302]: 2024-02-16T19:47:34.204+0000 7f45471c7700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-02-16T19:46:53.228875+0000 front 2024-02-16T19:46:53.228863+0000 (oldest deadline 2024-02-16T19:47:19.128661+0000) 2024-02-16T19:47:34.637 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:47:34 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-5[88302]: 2024-02-16T19:47:34.204+0000 7f45471c7700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.22:6814 osd.1 since back 2024-02-16T19:47:04.929841+0000 front 2024-02-16T19:47:04.929885+0000 (oldest deadline 2024-02-16T19:47:27.229676+0000) 2024-02-16T19:47:35.637 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:47:35 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-5[88302]: 2024-02-16T19:47:35.227+0000 7f45471c7700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-02-16T19:46:53.228875+0000 front 2024-02-16T19:46:53.228863+0000 (oldest deadline 2024-02-16T19:47:19.128661+0000) 2024-02-16T19:47:35.637 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:47:35 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-5[88302]: 2024-02-16T19:47:35.227+0000 7f45471c7700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.22:6814 osd.1 since back 2024-02-16T19:47:04.929841+0000 front 2024-02-16T19:47:04.929885+0000 (oldest deadline 2024-02-16T19:47:27.229676+0000) 2024-02-16T19:47:36.637 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:47:36 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-5[88302]: 2024-02-16T19:47:36.198+0000 7f45471c7700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-02-16T19:46:53.228875+0000 front 2024-02-16T19:46:53.228863+0000 (oldest deadline 2024-02-16T19:47:19.128661+0000) 2024-02-16T19:47:36.637 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:47:36 smithi117 ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-5[88302]: 2024-02-16T19:47:36.198+0000 7f45471c7700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.22:6814 osd.1 since back 2024-02-16T19:47:04.929841+0000 front 2024-02-16T19:47:04.929885+0000 (oldest deadline 2024-02-16T19:47:27.229676+0000) 2024-02-16T19:47:37.570 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:47:37 smithi117 podman[105484]: 2024-02-16 19:47:37.297860376 +0000 UTC m=+5.563169803 container died a1a7625b7541fed907d97359d1cbacf5d0d164248b6c183e88ba88d5a3ba49b4 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240215.160715-f78a58c) 2024-02-16T19:47:37.887 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:47:37 smithi117 podman[105484]: 2024-02-16 19:47:37.669350695 +0000 UTC m=+5.934660121 container cleanup a1a7625b7541fed907d97359d1cbacf5d0d164248b6c183e88ba88d5a3ba49b4 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-5, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-02-16T19:47:37.887 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:47:37 smithi117 bash[105484]: ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-5 2024-02-16T19:47:38.281 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:47:37 smithi117 podman[105501]: 2024-02-16 19:47:37.920860812 +0000 UTC m=+0.618319156 container remove a1a7625b7541fed907d97359d1cbacf5d0d164248b6c183e88ba88d5a3ba49b4 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-5, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-02-16T19:47:38.582 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:47:38 smithi117 podman[105592]: 2024-02-16 19:47:38.184305313 +0000 UTC m=+0.018096681 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T19:47:38.582 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:47:38 smithi117 podman[105592]: 2024-02-16 19:47:38.28832511 +0000 UTC m=+0.122116506 container create 7fcfb3ee8fd09955865ca0b15b56b4a68280981d783569f4156cb0a377b2c24b (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-5-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-02-16T19:47:38.583 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:47:38 smithi117 podman[105592]: 2024-02-16 19:47:38.499097241 +0000 UTC m=+0.332888592 container init 7fcfb3ee8fd09955865ca0b15b56b4a68280981d783569f4156cb0a377b2c24b (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-5-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-02-16T19:47:38.583 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:47:38 smithi117 podman[105592]: 2024-02-16 19:47:38.502431053 +0000 UTC m=+0.336222407 container start 7fcfb3ee8fd09955865ca0b15b56b4a68280981d783569f4156cb0a377b2c24b (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-5-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-16T19:47:38.583 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:47:38 smithi117 podman[105592]: 2024-02-16 19:47:38.582262963 +0000 UTC m=+0.416054314 container attach 7fcfb3ee8fd09955865ca0b15b56b4a68280981d783569f4156cb0a377b2c24b (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-5-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS) 2024-02-16T19:47:38.887 INFO:journalctl@ceph.osd.5.smithi117.stdout:Feb 16 19:47:38 smithi117 podman[105592]: 2024-02-16 19:47:38.789235487 +0000 UTC m=+0.623026838 container died 7fcfb3ee8fd09955865ca0b15b56b4a68280981d783569f4156cb0a377b2c24b (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-2a1346e6-cd00-11ee-95ba-87774f69a715-osd-5-deactivate, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-16T19:47:39.372 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a1346e6-cd00-11ee-95ba-87774f69a715@osd.5.service' 2024-02-16T19:47:39.412 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-16T19:47:39.413 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-02-16T19:47:39.413 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 --force --keep-logs 2024-02-16T19:47:39.576 INFO:teuthology.orchestra.run.smithi022.stdout:Deleting cluster with fsid: 2a1346e6-cd00-11ee-95ba-87774f69a715 2024-02-16T19:47:44.091 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi022.stderr:ceph-fuse[67263]: fuse finished with error 0 and tester_r 0 2024-02-16T19:47:59.158 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 --force --keep-logs 2024-02-16T19:47:59.319 INFO:teuthology.orchestra.run.smithi117.stdout:Deleting cluster with fsid: 2a1346e6-cd00-11ee-95ba-87774f69a715 2024-02-16T19:48:10.977 DEBUG:teuthology.orchestra.run.smithi022:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-16T19:48:11.014 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-16T19:48:11.043 INFO:tasks.cephadm:Archiving crash dumps... 2024-02-16T19:48:11.045 DEBUG:teuthology.misc:Transferring archived files from smithi022:/var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/crash to /home/teuthworker/archive/pdonnell-2024-02-16_18:26:20-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi/7562690/remote/smithi022/crash 2024-02-16T19:48:11.045 DEBUG:teuthology.orchestra.run.smithi022:> sudo tar c -f - -C /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/crash -- . 2024-02-16T19:48:11.083 INFO:teuthology.orchestra.run.smithi022.stderr:tar: /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/crash: Cannot open: No such file or directory 2024-02-16T19:48:11.083 INFO:teuthology.orchestra.run.smithi022.stderr:tar: Error is not recoverable: exiting now 2024-02-16T19:48:11.085 DEBUG:teuthology.misc:Transferring archived files from smithi117:/var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/crash to /home/teuthworker/archive/pdonnell-2024-02-16_18:26:20-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi/7562690/remote/smithi117/crash 2024-02-16T19:48:11.086 DEBUG:teuthology.orchestra.run.smithi117:> sudo tar c -f - -C /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/crash -- . 2024-02-16T19:48:11.112 INFO:teuthology.orchestra.run.smithi117.stderr:tar: /var/lib/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/crash: Cannot open: No such file or directory 2024-02-16T19:48:11.112 INFO:teuthology.orchestra.run.smithi117.stderr:tar: Error is not recoverable: exiting now 2024-02-16T19:48:11.113 INFO:tasks.cephadm:Checking cluster log for badness... 2024-02-16T19:48:11.113 DEBUG:teuthology.orchestra.run.smithi022:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-02-16T19:48:11.160 INFO:tasks.cephadm:Compressing logs... 2024-02-16T19:48:11.160 DEBUG:teuthology.orchestra.run.smithi022:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-16T19:48:11.203 DEBUG:teuthology.orchestra.run.smithi117:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-16T19:48:11.230 INFO:teuthology.orchestra.run.smithi022.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-16T19:48:11.230 INFO:teuthology.orchestra.run.smithi022.stderr:: No such file or directory 2024-02-16T19:48:11.230 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-02-16T19:48:11.230 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/ceph.audit.log 2024-02-16T19:48:11.230 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.0.log: gzip -5 --verbose -- /var/log/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/ceph-osd.1.log 2024-02-16T19:48:11.234 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/ceph-mon.smithi022.log 2024-02-16T19:48:11.234 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/ceph-mds.cephfs.smithi022.ocukvs.log 2024-02-16T19:48:11.234 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/ceph-mon.smithi022.log: gzip -5 --verbose -- /var/log/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/ceph-mds.cephfs.smithi022.btppwm.log 2024-02-16T19:48:11.234 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/ceph-mds.cephfs.smithi022.ocukvs.log: gzip -5 --verbose -- /var/log/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/ceph-osd.0.log 2024-02-16T19:48:11.234 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/ceph-mds.cephfs.smithi022.btppwm.log: gzip -5 --verbose -- /var/log/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/ceph-mgr.smithi022.gmdcch.log 2024-02-16T19:48:11.238 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/ceph-osd.0.log: 91.4% -- replaced with /var/log/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/ceph.audit.log.gz 2024-02-16T19:48:11.238 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/ceph.cephadm.log 2024-02-16T19:48:11.239 INFO:teuthology.orchestra.run.smithi117.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-16T19:48:11.239 INFO:teuthology.orchestra.run.smithi117.stderr:: No such file or directory 2024-02-16T19:48:11.239 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/ceph-mds.cephfs.smithi117.ibmppn.log 2024-02-16T19:48:11.239 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/ceph.audit.log 2024-02-16T19:48:11.239 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/ceph-mds.cephfs.smithi117.ibmppn.log: gzip -5 --verbose -- /var/log/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/ceph-osd.4.log 2024-02-16T19:48:11.239 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/ceph-mgr.smithi117.cohbtv.log 2024-02-16T19:48:11.239 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/ceph-osd.5.log 2024-02-16T19:48:11.239 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/ceph-osd.4.log: /var/log/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/ceph-mgr.smithi117.cohbtv.log: gzip -5 --verbose -- /var/log/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/ceph-client.ceph-exporter.smithi117.log 2024-02-16T19:48:11.239 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/ceph-osd.3.log 2024-02-16T19:48:11.239 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/ceph-client.ceph-exporter.smithi117.log: gzip -5 --verbose -- /var/log/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/ceph-mon.smithi117.log 2024-02-16T19:48:11.239 INFO:teuthology.orchestra.run.smithi117.stderr: 92.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-16T19:48:11.240 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/ceph-mds.cephfs.smithi117.orzors.log 2024-02-16T19:48:11.240 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/ceph-mon.smithi117.log: 94.2% -- replaced with /var/log/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/ceph-client.ceph-exporter.smithi117.log.gz 2024-02-16T19:48:11.240 INFO:teuthology.orchestra.run.smithi117.stderr: 91.6% -- replaced with /var/log/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/ceph.audit.log.gz 2024-02-16T19:48:11.244 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/ceph-mgr.smithi022.gmdcch.log: 90.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-16T19:48:11.245 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/ceph.log 2024-02-16T19:48:11.247 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/ceph.cephadm.log: 86.2% -- replaced with /var/log/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/ceph.cephadm.log.gz 2024-02-16T19:48:11.247 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/ceph-osd.2.log 2024-02-16T19:48:11.249 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/ceph.cephadm.log 2024-02-16T19:48:11.249 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/ceph-mds.cephfs.smithi117.orzors.log: gzip -5 --verbose -- /var/log/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/ceph.log 2024-02-16T19:48:11.250 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/ceph.log: 88.4% -- replaced with /var/log/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/ceph.log.gz 2024-02-16T19:48:11.251 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/ceph.cephadm.log: 86.1% -- replaced with /var/log/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/ceph.cephadm.log.gz 2024-02-16T19:48:11.251 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/ceph-volume.log 2024-02-16T19:48:11.251 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/ceph-volume.log 2024-02-16T19:48:11.254 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/ceph.log: 88.4% -- replaced with /var/log/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/ceph.log.gz 2024-02-16T19:48:11.254 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.log 2024-02-16T19:48:11.261 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/ceph-client.ceph-exporter.smithi022.log 2024-02-16T19:48:11.270 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/ceph-volume.log: /var/log/ceph/ceph-client.1.log: 89.3% -- replaced with /var/log/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/ceph-mgr.smithi117.cohbtv.log.gz 2024-02-16T19:48:11.272 INFO:teuthology.orchestra.run.smithi117.stderr: 91.5% -- replaced with /var/log/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/ceph-volume.log.gz 2024-02-16T19:48:11.281 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/ceph-volume.log: /var/log/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/ceph-client.ceph-exporter.smithi022.log: 94.3% -- replaced with /var/log/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/ceph-client.ceph-exporter.smithi022.log.gz 2024-02-16T19:48:11.300 INFO:teuthology.orchestra.run.smithi022.stderr: 91.8% -- replaced with /var/log/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/ceph-volume.log.gz 2024-02-16T19:48:11.664 INFO:teuthology.orchestra.run.smithi022.stderr: 88.4% -- replaced with /var/log/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/ceph-mgr.smithi022.gmdcch.log.gz 2024-02-16T19:48:11.707 INFO:teuthology.orchestra.run.smithi117.stderr: 92.6% -- replaced with /var/log/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/ceph-mon.smithi117.log.gz 2024-02-16T19:48:12.562 INFO:teuthology.orchestra.run.smithi022.stderr: 90.5% -- replaced with /var/log/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/ceph-mon.smithi022.log.gz 2024-02-16T19:48:14.661 INFO:teuthology.orchestra.run.smithi117.stderr: 93.8% -- replaced with /var/log/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/ceph-osd.4.log.gz 2024-02-16T19:48:16.064 INFO:teuthology.orchestra.run.smithi022.stderr: 93.9% -- replaced with /var/log/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/ceph-osd.2.log.gz 2024-02-16T19:48:17.132 INFO:teuthology.orchestra.run.smithi022.stderr: 93.7% -- replaced with /var/log/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/ceph-osd.0.log.gz 2024-02-16T19:48:17.350 INFO:teuthology.orchestra.run.smithi022.stderr: 93.8% -- replaced with /var/log/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/ceph-osd.1.log.gz 2024-02-16T19:48:17.402 INFO:teuthology.orchestra.run.smithi117.stderr: 93.9% -- replaced with /var/log/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/ceph-osd.3.log.gz 2024-02-16T19:48:17.880 INFO:teuthology.orchestra.run.smithi117.stderr: 93.9% -- replaced with /var/log/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/ceph-osd.5.log.gz 2024-02-16T19:48:18.418 INFO:teuthology.orchestra.run.smithi117.stderr: 94.8% -- replaced with /var/log/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/ceph-mds.cephfs.smithi117.ibmppn.log.gz 2024-02-16T19:48:19.056 INFO:teuthology.orchestra.run.smithi022.stderr: 94.8% -- replaced with /var/log/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/ceph-mds.cephfs.smithi022.ocukvs.log.gz 2024-02-16T19:48:19.295 INFO:teuthology.orchestra.run.smithi117.stderr: 94.8% -- replaced with /var/log/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/ceph-mds.cephfs.smithi117.orzors.log.gz 2024-02-16T19:48:24.955 INFO:teuthology.orchestra.run.smithi117.stderr:gzip: /var/log/ceph/ceph-client.1.log: file size changed while zipping 2024-02-16T19:48:24.955 INFO:teuthology.orchestra.run.smithi117.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.1.log.gz 2024-02-16T19:48:24.957 INFO:teuthology.orchestra.run.smithi117.stderr: 2024-02-16T19:48:24.957 INFO:teuthology.orchestra.run.smithi117.stderr:real 0m13.739s 2024-02-16T19:48:24.957 INFO:teuthology.orchestra.run.smithi117.stderr:user 0m44.366s 2024-02-16T19:48:24.957 INFO:teuthology.orchestra.run.smithi117.stderr:sys 0m1.287s 2024-02-16T19:48:26.039 INFO:teuthology.orchestra.run.smithi022.stderr:gzip: /var/log/ceph/ceph-client.0.log: file size changed while zipping 2024-02-16T19:48:26.226 INFO:teuthology.orchestra.run.smithi022.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-02-16T19:49:44.011 INFO:teuthology.orchestra.run.smithi022.stderr: 92.7% -- replaced with /var/log/ceph/2a1346e6-cd00-11ee-95ba-87774f69a715/ceph-mds.cephfs.smithi022.btppwm.log.gz 2024-02-16T19:49:44.013 INFO:teuthology.orchestra.run.smithi022.stderr: 2024-02-16T19:49:44.013 INFO:teuthology.orchestra.run.smithi022.stderr:real 1m32.797s 2024-02-16T19:49:44.013 INFO:teuthology.orchestra.run.smithi022.stderr:user 2m10.234s 2024-02-16T19:49:44.013 INFO:teuthology.orchestra.run.smithi022.stderr:sys 0m3.766s 2024-02-16T19:49:44.013 INFO:tasks.cephadm:Archiving logs... 2024-02-16T19:49:44.014 DEBUG:teuthology.misc:Transferring archived files from smithi022:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-02-16_18:26:20-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi/7562690/remote/smithi022/log 2024-02-16T19:49:44.015 DEBUG:teuthology.orchestra.run.smithi022:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-16T19:50:02.203 DEBUG:teuthology.misc:Transferring archived files from smithi117:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-02-16_18:26:20-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi/7562690/remote/smithi117/log 2024-02-16T19:50:02.204 DEBUG:teuthology.orchestra.run.smithi117:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-16T19:50:07.810 INFO:tasks.cephadm:Removing cluster... 2024-02-16T19:50:07.811 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 --force 2024-02-16T19:50:07.972 INFO:teuthology.orchestra.run.smithi022.stdout:Deleting cluster with fsid: 2a1346e6-cd00-11ee-95ba-87774f69a715 2024-02-16T19:50:08.377 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2a1346e6-cd00-11ee-95ba-87774f69a715 --force 2024-02-16T19:50:08.534 INFO:teuthology.orchestra.run.smithi117.stdout:Deleting cluster with fsid: 2a1346e6-cd00-11ee-95ba-87774f69a715 2024-02-16T19:50:08.826 INFO:tasks.cephadm:Removing cephadm ... 2024-02-16T19:50:08.827 DEBUG:teuthology.orchestra.run.smithi022:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-16T19:50:08.842 DEBUG:teuthology.orchestra.run.smithi117:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-16T19:50:08.857 INFO:tasks.cephadm:Teardown complete 2024-02-16T19:50:08.857 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-02-16T19:50:08.869 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_batrick_ceph_ed7ab725e1a43615009dfb74a42bf33d80081bc8/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/github.com_batrick_ceph_ed7ab725e1a43615009dfb74a42bf33d80081bc8/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-02-16T19:50:08.870 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-02-16T19:50:08.870 DEBUG:teuthology.orchestra.run.smithi022:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-16T19:50:08.884 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-16T19:50:08.973 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-02-16T19:50:08.984 INFO:teuthology.task.clock:Checking final clock skew... 2024-02-16T19:50:08.985 DEBUG:teuthology.orchestra.run.smithi022:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-16T19:50:08.987 DEBUG:teuthology.orchestra.run.smithi117:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-16T19:50:08.998 INFO:teuthology.orchestra.run.smithi022.stderr:bash: line 1: ntpq: command not found 2024-02-16T19:50:09.001 INFO:teuthology.orchestra.run.smithi117.stderr:bash: line 1: ntpq: command not found 2024-02-16T19:50:09.002 INFO:teuthology.orchestra.run.smithi022.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-16T19:50:09.002 INFO:teuthology.orchestra.run.smithi022.stdout:=============================================================================== 2024-02-16T19:50:09.003 INFO:teuthology.orchestra.run.smithi022.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 21 +1533us[+1533us] +/- 85ms 2024-02-16T19:50:09.003 INFO:teuthology.orchestra.run.smithi022.stdout:^* hv02.front.sepia.ceph.com 3 6 377 23 -1884us[-1869us] +/- 58ms 2024-02-16T19:50:09.003 INFO:teuthology.orchestra.run.smithi022.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 20 +1152us[+1152us] +/- 71ms 2024-02-16T19:50:09.003 INFO:teuthology.orchestra.run.smithi022.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-02-16T19:50:09.005 INFO:teuthology.orchestra.run.smithi117.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-16T19:50:09.005 INFO:teuthology.orchestra.run.smithi117.stdout:=============================================================================== 2024-02-16T19:50:09.005 INFO:teuthology.orchestra.run.smithi117.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 20 +1533us[+1533us] +/- 85ms 2024-02-16T19:50:09.005 INFO:teuthology.orchestra.run.smithi117.stdout:^* hv02.front.sepia.ceph.com 3 6 377 21 -1899us[-1896us] +/- 58ms 2024-02-16T19:50:09.005 INFO:teuthology.orchestra.run.smithi117.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 18 +1133us[+1133us] +/- 71ms 2024-02-16T19:50:09.005 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-02-16T19:50:09.006 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-02-16T19:50:09.018 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-02-16T19:50:09.019 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-02-16T19:50:09.030 DEBUG:teuthology.orchestra.run.smithi022:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-16T19:50:09.132 DEBUG:teuthology.orchestra.run.smithi117:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-16T19:50:09.210 DEBUG:teuthology.orchestra.run.smithi022:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-16T19:50:09.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-16T19:50:09.241 DEBUG:teuthology.orchestra.run.smithi117:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-16T19:50:09.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-16T19:50:09.271 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-02-16T19:50:09.283 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-02-16T19:50:09.295 INFO:teuthology.task.internal:Duration was 2348.785493 seconds 2024-02-16T19:50:09.296 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-02-16T19:50:09.308 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-02-16T19:50:09.308 DEBUG:teuthology.orchestra.run.smithi022:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-16T19:50:09.311 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-16T19:50:09.352 INFO:teuthology.orchestra.run.smithi022.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-16T19:50:09.354 INFO:teuthology.orchestra.run.smithi117.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-16T19:50:09.756 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-02-16T19:50:09.756 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi022.front.sepia.ceph.com 2024-02-16T19:50:09.756 DEBUG:teuthology.orchestra.run.smithi022:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-16T19:50:09.815 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi117.front.sepia.ceph.com 2024-02-16T19:50:09.815 DEBUG:teuthology.orchestra.run.smithi117:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-16T19:50:09.839 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-02-16T19:50:09.840 DEBUG:teuthology.orchestra.run.smithi022:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-16T19:50:09.858 DEBUG:teuthology.orchestra.run.smithi117:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-16T19:50:09.908 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-02-16T19:50:09.908 DEBUG:teuthology.orchestra.run.smithi022:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-16T19:50:09.927 DEBUG:teuthology.orchestra.run.smithi117:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-16T19:50:10.576 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-02-16T19:50:10.589 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-02-16T19:50:10.589 DEBUG:teuthology.orchestra.run.smithi022:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-16T19:50:10.617 DEBUG:teuthology.orchestra.run.smithi117:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-16T19:50:10.645 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-02-16T19:50:10.657 DEBUG:teuthology.orchestra.run.smithi022:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-16T19:50:10.660 DEBUG:teuthology.orchestra.run.smithi117:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-16T19:50:10.683 INFO:teuthology.orchestra.run.smithi022.stdout:kernel.core_pattern = core 2024-02-16T19:50:10.710 INFO:teuthology.orchestra.run.smithi117.stdout:kernel.core_pattern = core 2024-02-16T19:50:10.740 DEBUG:teuthology.orchestra.run.smithi022:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-16T19:50:10.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-16T19:50:10.768 DEBUG:teuthology.orchestra.run.smithi117:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-16T19:50:10.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-16T19:50:10.794 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-02-16T19:50:10.807 INFO:teuthology.task.internal:Transferring archived files... 2024-02-16T19:50:10.807 DEBUG:teuthology.misc:Transferring archived files from smithi022:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-02-16_18:26:20-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi/7562690/remote/smithi022 2024-02-16T19:50:10.808 DEBUG:teuthology.orchestra.run.smithi022:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-16T19:50:10.861 DEBUG:teuthology.misc:Transferring archived files from smithi117:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-02-16_18:26:20-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi/7562690/remote/smithi117 2024-02-16T19:50:10.862 DEBUG:teuthology.orchestra.run.smithi117:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-16T19:50:10.902 INFO:teuthology.task.internal:Removing archive directory... 2024-02-16T19:50:10.902 DEBUG:teuthology.orchestra.run.smithi022:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-16T19:50:10.904 DEBUG:teuthology.orchestra.run.smithi117:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-16T19:50:10.950 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-02-16T19:50:10.965 INFO:teuthology.task.internal:Not uploading archives. 2024-02-16T19:50:10.965 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-02-16T19:50:10.978 INFO:teuthology.task.internal:Tidying up after the test... 2024-02-16T19:50:10.978 DEBUG:teuthology.orchestra.run.smithi022:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-16T19:50:10.980 DEBUG:teuthology.orchestra.run.smithi117:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-16T19:50:10.992 INFO:teuthology.orchestra.run.smithi022.stdout: 262166 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Feb 16 19:50 /home/ubuntu/cephtest 2024-02-16T19:50:10.992 INFO:teuthology.orchestra.run.smithi022.stdout: 655612 4 d--------- 2 ubuntu ubuntu 4096 Feb 16 19:26 /home/ubuntu/cephtest/mnt.0 2024-02-16T19:50:10.993 INFO:teuthology.orchestra.run.smithi022.stderr:find: ‘/home/ubuntu/cephtest/mnt.0’: Permission denied 2024-02-16T19:50:10.993 INFO:teuthology.orchestra.run.smithi022.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2024-02-16T19:50:10.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-16T19:50:10.994 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/task/internal/__init__.py", line 48, in base yield File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_batrick_ceph_ed7ab725e1a43615009dfb74a42bf33d80081bc8/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/github.com_batrick_ceph_ed7ab725e1a43615009dfb74a42bf33d80081bc8/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi022 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2024-02-16T19:50:10.995 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-02-16T19:50:11.008 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-02-16T19:50:11.046 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-02-16T19:50:11.046 INFO:teuthology.nuke:Checking targets against current locks 2024-02-16T19:50:11.072 DEBUG:teuthology.nuke:shortname: smithi022 2024-02-16T19:50:11.073 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-16T19:50:11.096 DEBUG:teuthology.nuke:shortname: smithi117 2024-02-16T19:50:11.096 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-16T19:50:11.099 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi022.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-02-16_18:26:20-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi/7562690', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-16 19:01:56.506535', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMIZ2DUw6BxcUnqEaKE+IKXnccw9fWwvrdV4p+4inJbsRKm506Qrl49uIpTK0S5Pjer+EmQo/vhpKVtqBwUsjBg='} 2024-02-16T19:50:11.123 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi117.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-02-16_18:26:20-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi/7562690', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-16 19:01:56.508386', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEJxXBiOuUWPDc39YvRbfeVRlYJD1jWheXbOUbxnt1vZaWfcFBGNqi0zuGez3+y8nMaxYncImhSdtedLvRNbCNI='} 2024-02-16T19:50:11.156 INFO:teuthology.orchestra.console.smithi022:Power off 2024-02-16T19:50:11.156 DEBUG:teuthology.orchestra.console.smithi022:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-16T19:50:11.181 DEBUG:teuthology.orchestra.console.smithi022:power off output: Chassis Power Control: Down/Off 2024-02-16T19:50:11.181 DEBUG:teuthology.orchestra.console.smithi022:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-16T19:50:11.195 INFO:teuthology.orchestra.console.smithi117:Power off 2024-02-16T19:50:11.196 DEBUG:teuthology.orchestra.console.smithi117:pexpect command: ipmitool -H smithi117.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-16T19:50:11.209 DEBUG:teuthology.orchestra.console.smithi022:check power output: Chassis Power is on 2024-02-16T19:50:11.219 DEBUG:teuthology.orchestra.console.smithi117:power off output: Chassis Power Control: Down/Off 2024-02-16T19:50:11.219 DEBUG:teuthology.orchestra.console.smithi117:pexpect command: ipmitool -H smithi117.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-16T19:50:11.244 DEBUG:teuthology.orchestra.console.smithi117:check power output: Chassis Power is on 2024-02-16T19:50:15.210 DEBUG:teuthology.orchestra.console.smithi022:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-16T19:50:15.245 DEBUG:teuthology.orchestra.console.smithi117:pexpect command: ipmitool -H smithi117.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-16T19:50:15.321 DEBUG:teuthology.orchestra.console.smithi022:check power output: Chassis Power is on 2024-02-16T19:50:15.356 DEBUG:teuthology.orchestra.console.smithi117:check power output: Chassis Power is on 2024-02-16T19:50:19.324 DEBUG:teuthology.orchestra.console.smithi022:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-16T19:50:19.356 DEBUG:teuthology.orchestra.console.smithi117:pexpect command: ipmitool -H smithi117.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-16T19:50:19.438 DEBUG:teuthology.orchestra.console.smithi022:check power output: Chassis Power is off 2024-02-16T19:50:19.466 DEBUG:teuthology.orchestra.console.smithi117:check power output: Chassis Power is on 2024-02-16T19:50:19.540 INFO:teuthology.orchestra.console.smithi022:Power off completed 2024-02-16T19:50:23.468 DEBUG:teuthology.orchestra.console.smithi117:pexpect command: ipmitool -H smithi117.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-16T19:50:23.578 DEBUG:teuthology.orchestra.console.smithi117:check power output: Chassis Power is off 2024-02-16T19:50:23.679 INFO:teuthology.orchestra.console.smithi117:Power off completed 2024-02-16T19:50:23.784 INFO:teuthology.run:Summary data: description: fs:upgrade:mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{v18.2.0} 0a 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} duration: 2348.785493373871 failure_reason: reached maximum tries (51) after waiting for 300 seconds flavor: default owner: scheduled_pdonnell@teuthology status: fail success: false 2024-02-16T19:50:23.784 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-16T19:50:23.860 INFO:teuthology.run:FAIL